Databend Weekly

Databend is an open source elastic and reliable Modern Cloud Data Warehouse, it offers blazing fast query and combines elasticity, simplicity, low cost of the cloud, built to make the Data Cloud easy.

Big changes

Below is a list of some major changes that we don't want you to miss.

Features

add call procedure for sync stage (#6344)
show settings support like (#6394)
support all JsonEachRowOutputFormat variants (#6434)
support any, all and some subquery in parser (#6438)
support geo_to_h3 function (#6389)

storage

add xz compression support (#6421)
introduce system.tables_with_history (#6435)

new planner

migrate call statement to new planner (#6361)
support IS [NOT] DISTINCT FROM in planner_v2 (#6170)
support qualified column name with database specified (#6444)
support mark join, (not)in/any subquery, make tpch16 and tpch18 happy (#6412)

RFC

add Presign statement (#6503)

Improvement

add span info for TableReference (#6370)
improve optimize table compact (#6373)

refactor

split formats (#6443)
intro common-http to reduce duplicate code (#6484)

Build/Testing/CI

logic test with clickhouse handler (#6329)
enable semantic PRs and fully migrate to mergify and gh cli (#6386, #6419 and more)

Bug fixes

fix hashmap memory leak (#6354)
fix array inner type with null (#6407)
fix lost event in resize processor (#6501)

cluster

show correctly progress in cluster mode (#6253)
fix cannot destroy thread in cluster mode (#6436)

format

add NestedCheckpointReader for input format parser (#6385)
fix tsv deserialization (#6453)

Tips

Let's learn a weekly tip from Databend.

Monitoring Databend with Sentry

Sentry is cross-platform application monitoring, with a focus on error reporting.

Databend supports error tracking and performance monitoring with Sentry.

Preparing

Use the SaaS service provided by sentry.io, or deploy it yourself by following Self-Hosted Sentry.
Create a project in Sentry and get a DSN. The DSN looks the same as http://[email protected]:9000/5.

Error Tracking

This will only use the sentry-log feature, which will help us with error tracking.

DATABEND_SENTRY_DSN="<your-sentry-dsn>" ./databend-query

sentry-error

Performance Monitoring

Setting SENTRY_TRACES_SAMPLE_RATE greater than 0.0 will allow sentry to perform trace sampling, which will help set up performance monitoring.

DATABEND_SENTRY_DSN="<your-sentry-dsn>" SENTRY_TRACES_SAMPLE_RATE=1.0 LOG_LEVEL=DEBUG ./databend-query

Note: Set SENTRY_TRACES_SAMPLE_RATE to a lower value in production.

sentry-performance

Learn more

Changelogs

You can check the changelogs of Databend nightly to learn about our latest developments.

Contributors

Thanks a lot to the contributors for their excellent work this week.


ariesdevil	b41sh	BohuTANG	ClSlaid	dantengsky	databend-bot


drmingdrmer	everpcpc	flaneur2020	junnplus	leiysky	lichuang


mergify[bot]	PragmaTwice	PsiACE	soyeric128	sundy-li	TCeason


Veeupup	Xuanwo	xudong963	youngsofun	ZeaLoVe	zhang2014


ZhiHanZ	zhyass

Meet Us

Please join the DatafuseLabs Community if you are interested in Databend.

We are looking forward to seeing you try our code. We have a strong team behind you to ensure a smooth experience in trying our code for your projects. If you are a hacker passionate about database internals, feel free to play with our code.

You can submit issues for any problems you find. We also highly appreciate any of your pull requests.

This week in Databend #49