Commit Graph

27 Commits

Author SHA1 Message Date
Jinqing Kuang ac23a4fc55
feat(stream): support stream processing for virtual tables (#30144)
* enh: add client processing

* enh: add mnode vtables processing

* enh: add mnode vtable processing

* enh: add normal child vtable support

* fix: compile issues

* fix: compile issues

* fix: create stream issues

* fix: multi stream scan issue

* fix: remove debug info

* fix: agg task and task level issues

* fix: correct task output type

* fix: split vtablescan from agg

* fix: memory leak issues

* fix: add limitations

* Update 09-error-code.md

* Update 09-error-code.md

* fix: remove usless case

* feat(stream): extract original table data in source scan task

Implemented functionality in the source task to extract data
corresponding to the virtual table from the original table using WAL.
The extracted data is then sent to the downstream merge task for further
processing.

* feat(stream): multi-way merge using loser tree in virtual merge task

Implemented multi-way merge in the merge task using a loser tree to
combine data from multiple original table into a single virtual table.
The merged virtual table data is then pushed downstream for further
processing.  Introduced memory limit handling during the merge process
with configurable behavior when the memory limit is reached.

* fix(test): remove useless cases

---------

Co-authored-by: dapan1121 <wpan@taosdata.com>
Co-authored-by: Pan Wei <72057773+dapan1121@users.noreply.github.com>
2025-03-15 17:06:48 +08:00
wangjiaming0909 d35bdba180 fix parser/planner/nodes ret check ci 2024-07-26 17:57:29 +08:00
wangjiaming0909 4d5cd9cd1a planner/nodes/parser ret check 2024-07-26 17:57:27 +08:00
Xiaoyu Wang df0d200f59 feat: the compact command adds 'start with end with' clause 2023-03-07 15:24:04 +08:00
Xiaoyu Wang ca43c0ab55 feat: support writing streams to existing tables 2023-01-17 16:47:43 +08:00
Xiaoyu Wang 80dc658dbb Merge remote-tracking branch 'origin/3.0' into enh/3.0_planner_optimize 2022-11-04 18:03:21 +08:00
Xiaoyu Wang 8513dc1067 enh: insert optimize 2022-11-04 15:21:38 +08:00
Xiaoyu Wang 02ac6fa552 enh: add unit test and delete useless code 2022-10-26 17:01:55 +08:00
Xiaoyu Wang 18fa48ae57 feat: support batch loading of csv files 2022-10-11 15:54:01 +08:00
Xiaoyu Wang 8a010a58f3 enh: added memory allocators for parser and planner 2022-09-20 16:13:08 +08:00
Xiaoyu Wang eef595318d feat: system table visible permission 2022-08-24 19:08:50 +08:00
Xiaoyu Wang b420bf4c3d enh: insert client optimize 2022-08-15 14:45:59 +08:00
Xiaoyu Wang 555a575c58 enh: insert client optimize 2022-08-12 10:46:55 +08:00
Xiaoyu Wang 8fc86a31d0 enh: asynchronous catalog interface optimization 2022-08-04 20:53:43 +08:00
Xiaoyu Wang 37ec05a715 fix: memory leak problems of parser and planner 2022-07-20 17:20:29 +08:00
Xiaoyu Wang 5988fc0e4a feat: sql command 'show tags from table_name' 2022-07-19 15:31:44 +08:00
Xiaoyu Wang ae076af4f1 feat: sql system info function 'database()', 'client_version()', 'server_version()', 'server_status()', 'current_user()' and 'user()' 2022-06-29 11:41:32 +08:00
Xiaoyu Wang 1676bfe5cf fix: some problems of parser 2022-06-14 15:43:56 +08:00
Xiaoyu Wang 380a071551 fix: handle the memory leak of parser 2022-06-13 14:54:38 +08:00
Xiaoyu Wang d1e0327d19 fix: handle the memory leak of parser 2022-06-11 21:32:29 +08:00
afwerar 73724363f5 fix(os): add win ctest 2022-05-28 21:38:35 +08:00
Xiaoyu Wang 768beb6482 feat: parser adapts asynchronous interface 2022-05-28 16:45:49 +08:00
Xiaoyu Wang b5db96b717 feat: parser adapts asynchronous interface 2022-05-28 15:11:37 +08:00
Xiaoyu Wang d0c11c5bba feat: parser adapts asynchronous interface 2022-05-27 17:00:37 +08:00
Xiaoyu Wang 2745113a77 fix: some problems of parser 2022-05-10 20:59:10 +08:00
Xiaoyu Wang 3ec702605b enh: refactor unit test of parser and planner 2022-04-28 21:02:11 +08:00
Xiaoyu Wang 6d449c8224 enh: refactor db and table options 2022-04-27 18:18:37 +08:00