Databend Weekly

Databend is an open source elastic and reliable Modern Cloud Data Warehouse, it offers blazing fast query and combines elasticity, simplicity, low cost of the cloud, built to make the Data Cloud easy.

Big changes

Below is a list of some major changes that we don't want you to miss.

Features

multi-catalog by @dantengsky, (#4947)
implement hash inner join by @leiysky, (#5175)
support timezone by @Veeupup, (#4878)
impl alter database rename by @TCeason, (#5286)
change all the KV from serde to protobuf by @lichuang, (#5289 & #5296)

Migrate to new planner

planning subqueries by @leiysky, (#5283)
having, scalar expression in group by and order by by @xudong963, (#5200 & #5253)

Work around Array

ArrayType support inner DataType by @b41sh, (#5049)
get generic Array elements via index by @b41sh, (#5244)
length function for Array & Array<T> by @fkuner, (#5274)

Improvement

databend-query: support two-level hashmap by @fkuner, (#5075)
databend-query: implement format trait, load CSV/Parquet will be a streaming read by @zhang2014, (#5167 & #5271)
databend-query: make expression serialized to raw sql by @sundy-li, (#5260)
databend-meta: store endpoints to metasrv and use balance endpoints grpc connection channel by @ariesdevil, (#4987)

Bug fixes

databend-query(pipeline): fix state machine of hash join by @leiysky, (#5242)
databend-query(parser): show alternative tokens even if the branch is optional by @andylokandy, (#5230)

Tips

Let's learn a weekly tip from Databend.

Start a Databend Cluster

Databend has been designed from day one to be a cloud-native and distributed data warehouse.

The new databend-query node only needs to register itself to the databend-meta with the same cluster_id, they will autodiscovery and formed into a cluster.

Cluster Arch

Read the following article to start your first Databend cluster:

Note

Databend Cluster mode only works on shared storage(AWS S3 or MinIO s3-like storage).
This cluster mainly used for testing purpose, it is not targeted for production use.

Changelogs

You can check the changelogs of Databend nightly to learn about our latest developments.

Meet Us

Please join the DatafuseLabs Community if you are interested in Databend.

We are looking forward to seeing you try our code. We have a strong team behind you to ensure a smooth experience in trying our code for your projects. If you are a hacker passionate about database internals, feel free to play with our code.

You can submit issues for any problems you find. We also highly appreciate any of your pull requests.

This week in Databend #41