Commit Graph

409 Commits

Author SHA1 Message Date
liuyao fac36002c1
ci(stream): add ci and fix issue (#30270)
* ci(stream): add ci and fix issue

* ci: adj test

* ci: add test and fix issue

* ci: adj test

* ci(stream): add test for fill operator and fix issue

* fix: fix issue

* fix: fix issue

* fix: adjust defalut rec value
2025-03-20 10:34:31 +08:00
WANG MINGMING 3fafecb242
feat[TS-6137]: support sliding in force_window_close (#30203)
* feat[TS-6137]: support sliding in force_window_close

* feat[TS-6137]: support sliding in force_window_close

* feat(stream): force window close support interval sliding

* feat[TS-6137]: support sliding in force_window_close

* feat[TS-6137]: support sliding in force_window_close

* feat[TS-6137]: support sliding in force_window_close

---------

Co-authored-by: 54liuyao <54liuyao@163.com>
2025-03-19 10:22:06 +08:00
liuyao 2de6736160
feat(stream): add new trigger continuous_window_close (#30125)
* opt stream build twa result

* opt force window close memory

* feat(stream):optimize new interval and scan operator

* adj log

* opt code

* opt code

* fill history

* fix issue for fill history

* add ci

* feat(stream): add new stream nonblock interval operator

* adjust code

* use new scan operator

* use new scan operator

* add log

* fix issue

* recover stream scan next function

* fix issue

* fix issue

* fix issue

* ignore disorder data

* fix issue

* fix issue for interval sliding

* fix issue

* fix ci issue

* fix ci issue

* fix ci issue

* add semi && final nonblock interval operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* refactor(stream): track the msgId for each upstream tasks.

* fix(stream): fix race condition.

* fix(stream): update the task last msgId when putting into input queue succ.

* fix issue

* fix issue

* put recalculate data to rocksdb

* fix issue

* fix issue

* enh(query)[TD-33071]: add support for saving and restoring tsdbReader scan progress

- Implement functionality to save scan progress during tsdbReader operations
- Enable resuming scans from the last saved position

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* add rec interval check

* enh(stream):add recalculate tasks.

* enh(stream): support the re-calculate the tasks.

* fix issue && do refactor

* do refactor

* fix issue

* fix issue

* update backend opt

* add new interface

* add new session operator

* support blob

* add new session operator

* fix issue

* add rec state for rec task

* fix invalid read

* add new session window

* enh(stream): update the stream tasks backend.

* new session operator

* add pull data

* fix(stream): fix error in expand stream backend.

* fix issue

* fix issue

* fix issue

* merge code

* fix issue

* fix(stream): check for null ptr.

* fix(stream): add more check.

* fix issue

* fix issue

* fix issue

* add debug code

* fix issue

* fix issue

* fix issue

* set rec end flag

* fix(stream): opt re-calculate stream tasks.

* fix issue

* fix issue

* add new operator

* enh(stream): dispatch recalculate block to agg tasks.

* fix issue

* fix issue

* fix(stream): adjust filter.

* fix issue

* refactor

* refactor(stream): adjust the recalculate end block.

* fix issue

* fix(stream): set correct create trigger block.

* fix issue

* fix(stream): fix error in build recalculate end block.

* fix(stream): check null ptr.

* add stream client && fix issue

* fix mem leak

* fix(stream): free msg.

* add stream client

* fix(stream): fix error.

* add stream client && fix issue

* add stream client

* refactor(stream): set the recalculate task complete.

* add wend and group_id for session window dest table

* feat(stream): refactor and set the recalcul agg task complete.

* add cfg for adapter

* fix issue

* add state && event operator

* feat(stream): support fill-history task.

* add new fill operator

* fix(stream): set correct backend when updating fill-history task to recalculate task.

* add new fill operator

* fix(stream): s2025-03-06 11:10:31.272
et ops always open in scan tsdb

* fix(stream):set the correct taskType for sink task.

* new fill operator

* adj stream fill operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* adj test

* fix issue

* fix(stream): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix: ut com error

* fix(stream): fix mem leak and adjust operator type check rule

* fix(stream): fix mem leak and adjust test case

* refactor code

* fix(stream): free items.

* fix(stream): free fix memory leak.

* fix(stream): fix syntax error.

* fix: ignore unexpect block

* fix: adjust op type

---------

Co-authored-by: Haojun Liao <hjliao@taosdata.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
2025-03-14 20:14:01 +08:00
Simon Guan 6826edf5e0 Merge branch 'main' into merge/mainto3.0 2025-03-03 14:01:43 +08:00
Haojun Liao d279aaa22c test(stream): sleep a little bit longer. 2025-02-27 11:44:50 +08:00
Shengliang Guan 3a1d06f4d7
Merge pull request #29354 from taosdata/enh/TS-5255/colsFunc2
enh: ts5255/cols func2
2025-02-24 14:19:37 +08:00
facetosea 00473a9c64 fix: aliasName too long 2025-02-18 13:06:13 +08:00
Shengliang Guan d6bd52ece7
Merge pull request #29750 from taosdata/enh/streamqueue
refactor(stream): add more operation handled in ctrl queue, and limit the stream input q size
2025-02-14 15:02:23 +08:00
Haojun Liao 03d702ea55 test(stream): wait for task ready. 2025-02-12 14:41:58 +08:00
factosea 13c93bc2c9 test case 2025-02-10 18:02:28 +08:00
54liuyao daad609cc1 fix(stream):stream unsupported having clause 2025-01-06 15:30:44 +08:00
Shengliang Guan e53600fee2
Merge pull request #29380 from taosdata/fix/tag
fix(stream): check if the altered tag value table
2024-12-29 22:16:11 +08:00
Haojun Liao 8978b7d97f test: add test cases. 2024-12-29 17:52:43 +08:00
Haojun Liao a334bf1991 test: update the sim, set the output as uint, instead of int to avoid overflow. 2024-12-27 22:04:35 +08:00
54liuyao e8f63cded9 fix stream state operator issue 2024-12-27 22:04:28 +08:00
54liuyao e7a6bc195a adj ci 2024-12-20 08:38:57 +08:00
54liuyao b3b62326b2 Merge branch '3.0' of https://github.com/taosdata/TDengine into 3.0 2024-12-19 11:52:42 +08:00
54liuyao ce69387a43 ci(stream):adjust stream ci test 2024-12-19 10:32:32 +08:00
54liuyao 116c703b1c add test 2024-12-18 09:47:36 +08:00
54liuyao 63a6428e34 fix(stream):adjust status of ins_streams 2024-12-17 17:59:53 +08:00
54liuyao b6aa299a3c opt stream build twa result 2024-12-05 08:57:12 +08:00
54liuyao 803ed61ca0 ci(stream):add long time test for stream 2024-12-02 09:29:55 +08:00
54liuyao 58fdcae4d7 Merge branch 'main' of https://github.com/taosdata/TDengine into fix/TD-33020 2024-11-29 15:01:58 +08:00
Haojun Liao 6580e7751f test: add test cases. 2024-11-28 16:57:11 +08:00
Haojun Liao 05e9032fb9 test(stream): add test cases. 2024-11-28 15:29:43 +08:00
54liuyao 23d67933cd add ci 2024-11-27 16:52:01 +08:00
54liuyao 9d668150d5 fix stream issue 2024-11-27 16:24:46 +08:00
54liuyao 39564a41ae fix(stream):build create table request for force_window_close 2024-11-25 13:54:31 +08:00
54liuyao 53ae014354 fix issue for streamscan 2024-11-20 19:13:05 +08:00
54liuyao ba31bf2c52 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-11-06 09:42:50 +08:00
Haojun Liao 0614efdffd fix(stream): check the right return code for concurrent checkpoint trans. 2024-11-04 20:06:36 +08:00
54liuyao 158eda13f9 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-10-31 17:49:58 +08:00
Haojun Liao a5501f3046 test: add test cases. 2024-10-29 10:07:08 +08:00
Haojun Liao 4bd1e0c3cf test: add test cases. 2024-10-29 00:20:04 +08:00
54liuyao 3fdb1f4b7c add stream check 2024-10-28 13:18:59 +08:00
54liuyao ff46074b19 unsupported sliding 2024-10-25 15:43:28 +08:00
54liuyao f79dcffc8c Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-10-25 14:14:43 +08:00
54liuyao c466bea5e0 add stream doc 2024-10-25 14:03:03 +08:00
54liuyao 9475664494 add stream doc 2024-10-25 13:39:26 +08:00
Shengliang Guan 058d605c5b
Merge pull request #28473 from taosdata/fix/TS-5546-main
avoid free memory incorrectly
2024-10-24 14:09:21 +08:00
54liuyao 20c54a24f0 avoid free memory incorrectly 2024-10-24 11:25:43 +08:00
54liuyao 4d8bcd906b add ci 2024-10-23 17:33:54 +08:00
54liuyao 657c29b4e0 add ci 2024-10-22 14:07:23 +08:00
54liuyao 1ffc1a20ff add ci 2024-10-21 19:16:52 +08:00
54liuyao 118d079cbf adj ci 2024-10-21 15:08:59 +08:00
54liuyao 1195396e4f adjust ci 2024-10-21 09:54:30 +08:00
54liuyao f04151d772 fix(stream):add force window close check 2024-10-18 17:38:14 +08:00
54liuyao bef7fb6045 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-10-18 16:45:30 +08:00
54liuyao e64f55f5b1 adj ci 2024-10-17 18:06:16 +08:00
54liuyao 24b963d621 add ci 2024-10-17 16:43:47 +08:00