Databend aimed to be an open source elastic and reliable cloud warehouse, it offers blazing fast query and combines elasticity, simplicity, low cost of the cloud, built to make the Data Cloud easy.
Below is a list of some major changes that we don't want you to miss.
- databend-meta: add rename_table for meta api: remove old table record and add a new one in a sled-tree-transaction. by @Junnplus, (#4288)
- databend-query: support COPY from external location(S3): load files from a storage location (Amazon S3). by @BohuTANG, (#4170 & #4241)
- bendctl: add package purge subcommand by @linyihai, (#4245)
- databend-query: use database_id/table_id as data file prefix: no more having all parquet files in one big directory. by @dantengsky, (#4273)
- databend-query: support aggregate sum/avg booleans by @everpcpc, (#4237)
- databend-query: zero extra cost of async trait: use GAT feature instead of
#[async_trait]in some hot path. by @sundy-li, (#4269)
- databend-query: implement new processor for system tables & github & memory & null engine by @zhang2014, (#4166 & #4272)
- With change to wiremock from httpmock and a series of other PRs, @PsiACE removed ~60 dependencies for clippy and test.
- dal_context: use ObserveReader to calculate metrics: calculate the read cost time correctly. by @Xuanwo, (#4298)
Let's learn a weekly tip from Databend.
Loading files from S3 External Location
Here's an example of reading 5 lines from a file:
COPY INTO ontime FROM 's3://databend-external/t_ontime/t_ontime.csv' CREDENTIALS=(aws_key_id='<your-key-id>' aws_secret_key='<your-secret-key>') FILE_FORMAT = (type = "CSV" field_delimiter = '\t' record_delimiter = '\n' skip_header = 1) SIZE_LIMIT=5; /* only read 5 rows */ /* Check. */ SELECT * FROM ontime;
Note To learn more, check out:
You can check the changelogs of Databend nightly to learn about our latest developments.
From open source, for open source. Our team is also committed to contributing to the Rust ecosystem and upstream dependencies.
OpenDAL announced v0.1
We announced that Open Data Access Layer is in v0.1. Let's look at what's different together.
- Observe reader support (opendal#92)
- Object Stream (opendal#69)
- List support (opendal#76)
- Auto Region (opendal#59)
Big changes are happening at OpenRaft
Although no new releases have been made recently, we have seen some important changes with @schreter's help.
- Refactor storage APIs to allow more clear ownership of data. (openraft#199)
- Make NodeId type configurable via RaftTypeConfig. (openraft#220)
Feel free to visit Insights/Pulse to observe all changes, and it would be great if you would like to try it out in advance.
Please join the DatafuseLabs Community if you are interested in Databend.
We are looking forward to seeing you try our code. We have a strong team behind you to ensure a smooth experience in trying our code for your projects. If you are a hacker passionate about database internals, feel free to play with our code.
You can submit issues for any problems you find. We also highly appreciate any of your pull requests.