Databend is an open source elastic and reliable Modern Cloud Data Warehouse, it offers blazing fast query and combines elasticity, simplicity, low cost of the cloud, built to make the Data Cloud easy.

Big changes

Below is a list of some major changes that we don't want you to miss.

Features

  • undrop database (#5770)
  • support async insert mode to imporve throughput (#5567)
  • support alter table cluster key (#5718)

meta

  • add gc out of drop retention time data schema API and unit tests (#5746)
  • protobuf message has to persist MIN_COMPATIBLE_VER (#5785)

functions

  • support function timezone() (#5840)
  • add function NULLIF (#5772)

new planner

  • introduce InterpreterFactoryV2 for new planner (#5729)
  • support udf (#5751)
  • CREATE VIEW (#5816)
  • CREATE DATABASE (#5804) / DROP DATABASE (#5846)
  • CREATE USER (#5802) / ALTER USER (#5823) / DROP USER (#5813)

Improvement

  • refactor location parse logic (#5790)
  • show tenant quota for current tenant (#5750)
  • support errorcode hint in new planner (#5756)
  • don't load credential while reading stage (#5783)
  • move Clickhouse HTTP handler to its own port (#5797)

new parser

  • support cross join (#5730)
  • flatten join lists (#5742)

Build / Testing / CI

  • enable logic test (#5836)

Bug fixes

  • fix compressed buf not consumed correctly (#5727)
  • support abort for pipeline executor stream (#5803)
  • lz4raw compression of zero len buffer (#5806)
  • fix server hang when sync work panic (#5814)

Tips

Let's learn a weekly tip from Databend.

Databend Supports Async Insert Mode

When thousands of clients concurrently insert a small batch of data, each insert will be executed as follows:

Parser -> Planner -> Interpreter -> Pipeline

It's inefficient because of I/O depth and cache locality.

To solve the problem, we want to buffer small inserts into batches in server which sacrifices tiny latency for better insert throughput, smaller block count and larger DataBlock in storage.

After doing this, inserts into the same table will be parsed and planned individual. The insert data will be convert to DataBlock and buffered. When some conditions are triggered, the buffered DataBlock will be interpreted once which also is beneficial for pipelines.

Learn more:

Changlogs

You can check the changelogs of Databend nightly to learn about our latest developments.

Contributors

Thanks a lot to the contributors for their excellent work this week.

andylokandyb41shBohuTANGdantengskydevillove084drmingdrmer
andylokandyb41shBohuTANGdantengskydevillove084drmingdrmer
everpcpcfkunerflaneur2020hantmacjunnplusleiysky
everpcpcfkunerflaneur2020hantmacjunnplusleiysky
lichuangLiuYuHuimergify[bot]RinChanNOWWWsoyeric128sundy-li
lichuangLiuYuHuimergify[bot]RinChanNOWWWsoyeric128sundy-li
TCeasonXuanwoxudong963ygf11youngsofunZeaLoVe
TCeasonXuanwoxudong963ygf11youngsofunZeaLoVe
zhang2014zhyass
zhang2014zhyass

Meet Us

Please join the DatafuseLabs Community if you are interested in Databend.

We are looking forward to seeing you try our code. We have a strong team behind you to ensure a smooth experience in trying our code for your projects. If you are a hacker passionate about database internals, feel free to play with our code.

You can submit issues for any problems you find. We also highly appreciate any of your pull requests.