This week in Databend #69
Databend is a powerful cloud data warehouse. Built for elasticity and efficiency. Free and open. Also available in the cloud: https://app.databend.com .
What's Changed
Below is a list of some major changes that we don't want you to miss.
Exciting New Features ✨
multiple catalog
- implement drop user defined catalog (#8820)
meta
- add cli command to delete a key and to expire a key (#8858)
planner
query
- optimize count(Nullable(col)) (#8805)
- support unset settings (#8870)
- add distinct count aggregator and column distinct count (#8825)
storage
- compact segments in reversed order (#8806)
new expression
- geo functions (#8481)
- add methods to get memory size of
ValueType
s (#8875) - add a global builtin function registry (#8912)
Code Refactor 🎉
memory tracker
- send pointer addresses to mem tracker (#8879)
- add
StatBuffer
to provide fine grained mem allocation stats buffer (#8880)
new expression
- allow sparse column id for constant folder (#8821)
Build/Testing/CI Infra Changes 🔌
- separate sqllogic test with handler (#8836)
Thoughtful Bug Fix 🔧
base
- support track processor async task (#8871)
http handler
- avoid dropping runtime when task on it not finished (#8894)
query
- remove useless memcpy when group long string (#8851)
storage
- snapshot removed unsafely during meta commit failure (#8850)
News
Let's take a look at what's new at Datafuse Labs & Databend each week.
Preview of New Expressions: Geo Functions
By supporting Geo functions, Databend will have the ability to perform operations on geographic inputs.
With the merging of #8481, geo functions like great_circle_distance
, geo_distance
, great_circle_angle
and point_in_ellipses
are already supported in the new expression system.
Databend is currently actively working on the migration to the new expressions, so keep an eye on the expression branch for progress!
Learn More
Unset Settings
The merging of #8870 brings the ability to unset settings to Databend
UNSET
means to restore one or more settings to their default values. The settings will also be reset to the initial SESSION level if they were set to GLOBAL level.
UNSET <setting_name> | ( <setting_name> [, <setting_name> ...])
Learn More
Issues
Meet issues you may be interested in and try to solve it.
Add Compression Option to Create Table
Compression helps to reduce the size of databases. For IO-intensive loads, compression may provide some performance improvements.
Databend plans to introduce compression
option in the create table
statement and support compression algorithms such as LZ4 (default) and Snappy.
create table t1(a int) [compression="LZ4|SNAPPY"]
Issue 8903: feat: add compression option to create table
If you find it interesting, try to solve it or participate in discussions and PR reviews. Or you can click on https://link.databend.rs/i-m-feeling-lucky to pick up a good first issue, good luck!
Changelogs
You can check the changelogs of Databend nightly to learn about our latest developments.
- v0.8.125-nightly
- v0.8.124-nightly
- v0.8.123-nightly
- v0.8.122-nightly
- v0.8.121-nightly
- v0.8.120-nightly
- v0.8.119-nightly
- v0.8.118-nightly
- v0.8.117-nightly
- v0.8.116-nightly
Contributors
Thanks a lot to the contributors for their excellent work this week.
andylokandy | ariesdevil | b41sh | BohuTANG | Chasen-Zhang | ClSlaid |
dantengsky | drmingdrmer | everpcpc | kemingy | lichuang | mergify[bot] |
RinChanNOWWW | soyeric128 | sundy-li | TCeason | wubx | Xuanwo |
xudong963 | youngsofun | zhang2014 | zhyass |
Meet Us
Please join the DatafuseLabs Community if you are interested in Databend.
We are looking forward to seeing you try our code. We have a strong team behind you to ensure a smooth experience in trying our code for your projects. If you are a hacker passionate about database internals, feel free to play with our code.
You can submit issues for any problems you find. We also highly appreciate any of your pull requests.
- Databend Website
- Weekly (A weekly newsletter about Databend)
- GitHub Discussions (Feature/Bug reports, Contributions)
- Twitter (Get the news fast)
- Slack Channel (For live discussion with the Community)