Databend is an open source elastic and reliable Modern Cloud Data Warehouse, it offers blazing fast query and combines elasticity, simplicity, low cost of the cloud, built to make the Data Cloud easy.
Below is a list of some major changes that we don't want you to miss.
- support correlated subquery (#5593)
select ... at(#5617 & #5628)
- exchange protocol version with client (#5645)
- add decompress support for COPY INTO and streaming loading (#5655)
explainfor new planner (#5587)
- support tuple in new planner (#5640)
- support cross join (#5715)
- refine table optimize (#5589)
- parse integer to u64 (#5692)
FixedKeyu128, u256, u512 in group query (#5678)
- utilize with
HashMethodFixedKeysin hash join (#5693)
- record count of tables for a tenant in KV space (#5708)
Build / Testing / CI
- add mock module, add test of out retention time data (#5707)
- retry while meeting error during load_credential (#5590)
- deny the root login from others host (#5588)
Let's learn a weekly tip from Databend.
COPY INTO Databend with
After PR 5655, Databend started to support decompression reads in
COPY INTO and streaming leading.
Loading Compressed Files from Amazon S3
Try to load data from a gzip compressed csv and insert into
COPY INTO mytable FROM s3://mybucket/data.csv.gz credentials=(aws_key_id='<AWS_ACCESS_KEY_ID>' aws_secret_key='<AWS_SECRET_ACCESS_KEY>') FILE_FORMAT = (type = "CSV" field_delimiter = ',' record_delimiter = '\n' skip_header = 1 compression = GZIP) size_limit=10;
COMPRESSION option is a string that represents the compression algorithm.
| Values | Notes | | ------------- | --------------------------------------------------------------- | | `AUTO` | Auto detect compression via file extensions | | `GZIP` | | | `BZ2` | | | `BROTLI` | Must be specified if loading/unloading Brotli-compressed files. | | `ZSTD` | Zstandard v0.8 (and higher) is supported. | | `DEFLATE` | Deflate-compressed files (with zlib header, RFC1950). | | `RAW_DEFLATE` | Deflate-compressed files (without any header, RFC1951). | | `NONE` | Indicates that the files have not been compressed. |
You can check the changelogs of Databend nightly to learn about our latest developments.
Thanks a lot to the contributors for their excellent work this week.
Please join the DatafuseLabs Community if you are interested in Databend.
We are looking forward to seeing you try our code. We have a strong team behind you to ensure a smooth experience in trying our code for your projects. If you are a hacker passionate about database internals, feel free to play with our code.
You can submit issues for any problems you find. We also highly appreciate any of your pull requests.