Commit Graph

137 Commits

Author SHA1 Message Date
liuyao 2de6736160
feat(stream): add new trigger continuous_window_close (#30125)
* opt stream build twa result

* opt force window close memory

* feat(stream):optimize new interval and scan operator

* adj log

* opt code

* opt code

* fill history

* fix issue for fill history

* add ci

* feat(stream): add new stream nonblock interval operator

* adjust code

* use new scan operator

* use new scan operator

* add log

* fix issue

* recover stream scan next function

* fix issue

* fix issue

* fix issue

* ignore disorder data

* fix issue

* fix issue for interval sliding

* fix issue

* fix ci issue

* fix ci issue

* fix ci issue

* add semi && final nonblock interval operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* refactor(stream): track the msgId for each upstream tasks.

* fix(stream): fix race condition.

* fix(stream): update the task last msgId when putting into input queue succ.

* fix issue

* fix issue

* put recalculate data to rocksdb

* fix issue

* fix issue

* enh(query)[TD-33071]: add support for saving and restoring tsdbReader scan progress

- Implement functionality to save scan progress during tsdbReader operations
- Enable resuming scans from the last saved position

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* add rec interval check

* enh(stream):add recalculate tasks.

* enh(stream): support the re-calculate the tasks.

* fix issue && do refactor

* do refactor

* fix issue

* fix issue

* update backend opt

* add new interface

* add new session operator

* support blob

* add new session operator

* fix issue

* add rec state for rec task

* fix invalid read

* add new session window

* enh(stream): update the stream tasks backend.

* new session operator

* add pull data

* fix(stream): fix error in expand stream backend.

* fix issue

* fix issue

* fix issue

* merge code

* fix issue

* fix(stream): check for null ptr.

* fix(stream): add more check.

* fix issue

* fix issue

* fix issue

* add debug code

* fix issue

* fix issue

* fix issue

* set rec end flag

* fix(stream): opt re-calculate stream tasks.

* fix issue

* fix issue

* add new operator

* enh(stream): dispatch recalculate block to agg tasks.

* fix issue

* fix issue

* fix(stream): adjust filter.

* fix issue

* refactor

* refactor(stream): adjust the recalculate end block.

* fix issue

* fix(stream): set correct create trigger block.

* fix issue

* fix(stream): fix error in build recalculate end block.

* fix(stream): check null ptr.

* add stream client && fix issue

* fix mem leak

* fix(stream): free msg.

* add stream client

* fix(stream): fix error.

* add stream client && fix issue

* add stream client

* refactor(stream): set the recalculate task complete.

* add wend and group_id for session window dest table

* feat(stream): refactor and set the recalcul agg task complete.

* add cfg for adapter

* fix issue

* add state && event operator

* feat(stream): support fill-history task.

* add new fill operator

* fix(stream): set correct backend when updating fill-history task to recalculate task.

* add new fill operator

* fix(stream): s2025-03-06 11:10:31.272
et ops always open in scan tsdb

* fix(stream):set the correct taskType for sink task.

* new fill operator

* adj stream fill operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* adj test

* fix issue

* fix(stream): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix: ut com error

* fix(stream): fix mem leak and adjust operator type check rule

* fix(stream): fix mem leak and adjust test case

* refactor code

* fix(stream): free items.

* fix(stream): free fix memory leak.

* fix(stream): fix syntax error.

* fix: ignore unexpect block

* fix: adjust op type

---------

Co-authored-by: Haojun Liao <hjliao@taosdata.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
2025-03-14 20:14:01 +08:00
wangjiaming f50fcb85b8
feat(decimal): support decimal data type (#30060)
* decimal: create table

* decimal: add test case decimal.py

* decimal: add decimal.c

* support input decimal

* decimal test

* refactor svalue

* fix test cases

* add decimal unit test

* add decimal test cmake

* support insert and query decimal type

* define wide integer, support decimal128

* support decimal128 divide

* set decimal type expr res types

* scalar decimal

* convert to decimal

* fix decimal64/128 from str and to str

* fix decimal from str and decimal to str

* decimal simple conversion

* unit test for decimal

* decimal conversion and unit tests

* decimal + - * /

* decimal scalar ops and comparision

* start to refactor GET_TYPED_DATA

* support decimal max func, cast func

* refactor GET_TYPED_DATA interface

* decimal scalar comparision

* start to implement sum for decimal

* support sum and avg for decimal type

* decimal tests

* add decimal test

* decimal add test cases

* decimal use int256/int128

* decimal testing

* fix decimal table meta and add tests for decimal col streams

* fix create stream and create tsma

* test insert decimal values

* decimal from str

* test decimal input

* test parse decimal from string

* add taos_fetch_field_e api

* decimal insert tests

* test decimal operators

* decimal operator test

* feat:support decimal in raw block

* decimal operator tests

* decimal test

* feat:support decimal in raw block

* feat:support decimal in raw block

* feat:add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* decimal test operators

* decimal operator test

* test decimal operators

* test decimal compare operators

* decimal unary operator test

* decimal col with decimal col oper test

* test decimal col filtering

* fix decimal float operator test

* decimal test where filtering

* fix decimal filtering

* fix decimal order by

* fix decimal op test

* test decimal agg funcs

* test decimal functions

* remove assert

* fix ci build for ret check

* fix decimal windows build

* fix ci ret check

* skip decimal ret check

* skip decimal ret check

* fix decimal tests

* fix decimal ci test

* decimal test

* fix(tmq): heap user after free

* fix(tmq): double free

* fix(tmq): double free

* fix decimal tests

* fix(decimal): decimal test ci build

* fix(decimal): windows build

* fix(decimal): decimal test build

* fix(decimal): fix decimal build and tests

* fix(decimal): fix decimal tests

* fix(decimal): fix taos_fetch_fields_e api

* fix(decimal): fix decimal taos_fetch_fields_e api

* fix(decimal): rebase 3.0

* fix(decimal): fix decimal functions

* fix(decimal): fix decimal test case memory leak

* fix(decimal): fix decimal tests

* fix(decimal): fix decimal test case

* fix(decimal): fix decimal tests

* feat(decimal): fix unit tests

* feat(decimal): fix deicmal unit test

---------

Co-authored-by: wangmm0220 <wangmm0220@gmail.com>
Co-authored-by: yihaoDeng <yhdeng@taosdata.com>
2025-03-14 18:08:07 +08:00
Zhixiao Bao af7e34e189
feat(keep): support keep on super table level. (#30097)
* Feat: support use keep while create super table.

* Test(keep): add test for create super table with keep option.

* Feat(keep): Add tmsg for create keep.

* Feat(keep): support alter table option keep.

* Fix(keep): Add baisc test for alter table option.

* Fix(keep): memory leek.

* Feat(keep): add keep to metaEntry&metaCache and fix earliestTs with stn keep.

* Test(keep): add some cases for select with stb keep.

* Fix: fix ci core while alter stb.

* Feat(keep): delete expired data in super table level.

* Feat: remove get stb keep while query.

* Fix : build error.

* Revert "Fix : build error."

This reverts commit 0ed66e4e848e5528d57f5a7c9fa7600c4742a71c.

* Revert "Feat(keep): delete expired data in super table level."

This reverts commit 36330f6b4c587bc126d3f75480d81e637d7601e6.

* Fix : build errors.

* Feat : support restart taosd.

* Fix : alter table comment problems.

* Test : add tests for super table keep.

* Fix: change sdb stb reserve size.

* Test: add more tests.

* Feat: Disable normal tables and sub tables from setting the keep parameter

* Fix: add more checks to avoid unknown address.

* Docs: Add docs for stable keep.

* Fix: some review changes.

* Fix: review errors.
2025-03-14 16:10:13 +08:00
Shengliang Guan 2bee60a41a Merge branch 'main' into merge/mainto3.0 2025-02-20 18:12:02 +08:00
54liuyao 870fe1c071 fix(tsma):drop invalid state for tsma 2025-02-20 11:52:04 +08:00
Jinqing Kuang fc2d2672b9 feat(stream)[TS-5469]: add more stream event notifications
- Add event notifications for all window types
- Implement window invalidation notifications
- Add statistics for event notification delivery
2025-02-16 11:07:16 +08:00
dapan1121 89cea7859a fix: invalid subtable in tbname in clause issue 2025-01-21 17:41:37 +08:00
54liuyao f6500b3551 ci(test):delete invalid code&& add ci 2024-12-26 11:16:02 +08:00
Hongze Cheng 2b4046ecf8 Merge branch '3.0' of https://github.com/taosdata/TDengine into enh/TD-29367-3.0 2024-12-15 11:28:37 +08:00
Hongze Cheng a6463884f0 more code 2024-12-13 16:47:05 +08:00
Hongze Cheng 25b9de5d51 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TS-4994-3.0 2024-12-03 14:32:03 +08:00
Hongze Cheng 3ae5a72a4c more code 2024-11-27 17:53:53 +08:00
Hongze Cheng 6b236018d5 more code and case change 2024-11-27 14:40:58 +08:00
yihaoDeng 42065bf2e8 Merge remote-tracking branch 'origin/3.0' into enh/addDiskUsage 2024-11-14 17:44:29 +08:00
54liuyao f452ca47fe delete window state when drop table 2024-11-08 09:42:56 +08:00
yihaoDeng a6436882a2 support disk usage 2024-11-06 15:25:50 +08:00
yihaoDeng dc576905f0 support disk usage 2024-11-05 19:57:13 +08:00
54liuyao e9550a6b2c feat(stream):save force window close scan range 2024-10-18 15:40:37 +08:00
54liuyao a62b5a3efd fix issue 2024-10-17 14:55:07 +08:00
54liuyao 3e49f40c74 stream twa 2024-10-16 12:16:59 +08:00
54liuyao d13819bbf4 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-09-30 09:05:46 +08:00
54liuyao 7517228dde fix(stream):fix issue for stream count window state buff 2024-09-29 14:16:16 +08:00
54liuyao 813cab6e33 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-09-19 14:45:38 +08:00
Hongze Cheng c0b3cc17fc
Merge pull request #27899 from taosdata/enh/TD-32121-3.0
enh: retrieveRows no more if got all data
2024-09-19 09:12:14 +08:00
54liuyao c4566f98b8 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-09-18 16:26:37 +08:00
Haojun Liao 5f486d6976 Merge branch '3.0' into fix/3_liaohj 2024-09-14 17:29:29 +08:00
Shungang Li 0fd4928dc2 enh: retrieveRows no more if got all data 2024-09-14 17:02:38 +08:00
54liuyao b649f1f1c4 fix(stream):adj build file state res 2024-09-14 10:51:57 +08:00
Haojun Liao 1a90e9612d refactor: check return value. 2024-09-13 23:04:41 +08:00
54liuyao 3491396d92 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-08-08 19:30:46 +08:00
Hongze Cheng dfc004ac37 fix: handle error code 2024-08-08 10:22:06 +08:00
54liuyao bdaf0d419e stream interp force window close 2024-08-06 16:14:15 +08:00
54liuyao b26750c200 add ci 2024-08-02 13:22:14 +08:00
54liuyao 0bd8c57049 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-07-29 09:24:40 +08:00
54liuyao b5f432bd50 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-07-26 16:44:18 +08:00
wangmm0220 3f2278ebcf fix:[TD-31017]process return value in vnode for tmq 2024-07-24 17:56:29 +08:00
Haojun Liao 7497f193c7 fix(stream): check return value, and do some internal refactor. 2024-07-23 14:47:29 +08:00
54liuyao 99f4484328 adj stream operator result 2024-07-18 11:26:50 +08:00
54liuyao 4e6b898c9a adj stream operator result 2024-07-18 10:37:14 +08:00
54liuyao 113cb57326 adj stream operator result 2024-07-17 19:35:35 +08:00
Haojun Liao 665fbd6f22
Merge pull request #26551 from taosdata/fix/TD-30967
feat: adj result of function
2024-07-17 09:09:23 +08:00
Haojun Liao 94f596c4ce refactor: remove the invalid return value. 2024-07-16 18:52:01 +08:00
54liuyao 3cdb957f22 adj function of stream operator result 2024-07-15 15:32:11 +08:00
54liuyao 9b067d075f adj stream function return 2024-07-15 13:38:34 +08:00
54liuyao 5b982354e1 adj bloomfilter 2024-07-12 15:30:03 +08:00
54liuyao 1c9011e820 stream time slice 2024-07-12 11:03:45 +08:00
54liuyao ecfb5e5f4c adj par name catch 2024-06-20 09:28:04 +08:00
Haojun Liao 2534ce5071 fix(stream): fix a typo. 2024-06-19 10:29:01 +08:00
Haojun Liao 985d619569 fix(stream): fix syntax error. 2024-06-17 23:14:33 +08:00
Haojun Liao 3baa4169e6 Merge branch '3.0' into fix/3_liaohj 2024-05-20 10:31:53 +08:00