Commit Graph

5889 Commits

Author SHA1 Message Date
Yihao Deng 2151c5cae0
update telemInterval (#29703) 2025-03-17 17:42:24 +08:00
Haojun Liao ab92886820
fix(stream): reduce the consensus checkpoint id trans. (#30105)
* fix(stream): reduce the consensus checkpoint id trans.

* refactor(stream): add some logs.

* refactor(stream): set the max checkpoint exec time 30min.

* refactor(stream): add checkpoint-consensus trans conflict check.

* refactor(stream): remove unused local variables.

* fix(stream): fix syntax error.

* fix(stream): 1. fix free memory error 2. continue if put result into dst hashmap failed.

* fix issue

* fix issue

* fix(mnd): follower mnode not processes the timer event.

* fix(stream): print correct error msg.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): truncate long subtable name

* fix(stream): add buffer len.

* refactor(stream): update some logs.

* fix issue

* refactor(stream): update some logs.

* refactor(stream): update some logs.

* fix(stream): check return value.

* fix(stream): fix syntax error.

* fix(stream): check return value.

* fix(stream): update the timer check in mnode.

* fix(stream): add restart stage tracking.

* fix(stream): track the start task stage for meta.

* fix(stream): fix error in log.

* refactor(stream): adjust log info.

* fix mem issue

* fix(stream): check the number of required tasks for consensus checkpointId.

* fix(stream): lock the whole start procedure.

* fix(stream): add lock during start all tasks.

* fix(stream): update logs.

* fix(stream): update logs.

* fix(stream): update logs.

* fix(stream): fix dead-lock.

* fix(stream): fix syntax error.

* fix(stream): not drop the scan-history task.

* fix(stream): fix syntax error.

* fix(stream): wait for executor stop before restarting.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): disable some logs.

* fix(stream): reset the start info if no task left.

---------

Co-authored-by: 54liuyao <54liuyao@163.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
2025-03-17 10:20:17 +08:00
Simon Guan 6fa7a7bb99
Merge pull request #29996 from taosdata/fix/droptask
feat(stream): add timeout parameter.
2025-03-05 18:49:23 +08:00
Haojun Liao 24882459e3 chore: merge main 2025-03-05 13:40:44 +08:00
yihaoDeng d1ded07b43 fix(index): fix return error 2025-03-05 11:07:59 +08:00
yihaoDeng a4b738b3c6 enh(index): opt error code 2025-03-05 11:04:39 +08:00
Simon Guan 6656460320
Merge pull request #29988 from taosdata/fix/chkptq
fix(stream):check the maximum allowed forecast rows, add one test case
2025-03-05 09:29:01 +08:00
Haojun Liao 2eea385485 fix(analytics): check return value. 2025-03-04 15:29:54 +08:00
Haojun Liao c186d31c7d fix(analytics): check return value. 2025-03-04 14:30:48 +08:00
Haojun Liao 19f09d679d feat(stream): add timeout parameter. 2025-03-03 23:05:14 +08:00
Haojun Liao f563a35b3f fix(stream):check the maximum allowed forecast rows, add one test case, and do some internal refactor. 2025-03-03 17:27:05 +08:00
Haojun Liao ac667164ed refactor: do some internal refactor. 2025-03-03 16:20:58 +08:00
Simon Guan eee9976a44
Merge pull request #29958 from taosdata/fix/TD-33872-improve-status-sync-heartbeat
fix: [td-33872] improve status sync heartbeat
2025-03-03 13:51:44 +08:00
Haojun Liao f8cc9648f6 fix(stream): fix memory leak. 2025-03-01 15:33:31 +08:00
dmchen 3359483836 fix:decouple get mload 2025-03-01 00:22:40 +08:00
dmchen 774c512148 fix:add-status-timestamp 2025-03-01 00:22:37 +08:00
dmchen 48787ba1aa fix/move-status-to-new-queue 2025-03-01 00:22:35 +08:00
Simon Guan 24fae6adbb
Merge pull request #29900 from taosdata/fix/checkfh
refactor(stream): scan wal is driven by time, instead of insert events.
2025-02-28 13:42:43 +08:00
dmchen 4dd0622594 fix:decouple get mload 2025-02-27 19:07:33 +08:00
dmchen 7eeb800213 fix:add-status-timestamp 2025-02-27 15:29:52 +08:00
dmchen 1f6c4df422 fix/move-status-to-new-queue 2025-02-27 15:26:06 +08:00
Haojun Liao 7b56df0adc refactor(stream): create task epset update trans out of lock, to avoid blocking mnode-read thread. 2025-02-27 14:51:26 +08:00
Shengliang Guan e429264432
Merge pull request #29884 from taosdata/enh/killtrans
enh(stream): auto kill checkpoint trans that last long time
2025-02-25 11:29:49 +08:00
Haojun Liao 91271f3fa4 refactor: do some internal refactor. 2025-02-24 13:53:28 +08:00
Haojun Liao 4d21d5e055 fix(stream): fix memory leak. 2025-02-24 13:50:38 +08:00
Haojun Liao 97071f1d29 other: add some logs. 2025-02-24 09:42:15 +08:00
Haojun Liao ea549364d4 fix(stream): free array and check the return values. 2025-02-21 16:38:26 +08:00
Haojun Liao 6d5984421f refactor(stream): update the long trans threshold. 2025-02-20 17:27:41 +08:00
Haojun Liao 0a672f1b96 refactor(stream): update logs. 2025-02-20 16:55:10 +08:00
Haojun Liao 99d6086c5a enh(stream): kill too long checkpoint trans. 2025-02-20 15:07:24 +08:00
Haojun Liao 3048654484 refactor(stream): kill too long checkpoint trans. 2025-02-20 10:01:30 +08:00
Haojun Liao 7d617a7cde
fix(stream): fix the invalid start task condition
1. Get the basic checkpointId, and wait for 10 s, it's allowed to go.  2. All tasks have the same checkpointId, go immedidately.
2025-02-19 00:39:01 +08:00
Haojun Liao 267f7d3b08 refactor(stream): limit the maximum number of consensus checkpoint trans. 2025-02-11 10:31:11 +08:00
Haojun Liao d08a81cc98 fix(stream): fix bug in limiting concurrent checkpoints 2025-02-10 19:24:37 +08:00
Haojun Liao 6c7a8e9774 refactor(stream): allow ready stream to start checkpoint procedure. 2025-02-10 19:24:36 +08:00
Shengliang Guan dae17cd1b0
Merge pull request #29718 from taosdata/enh/streamqueue
refactor(stream): notify close stream in main thread
2025-02-10 18:44:27 +08:00
Haojun Liao ec954bf408 refactor(stream): record the dropped stream. 2025-02-09 21:13:10 +08:00
xiao-77 adfab1f2b6 Fix some review errors. 2025-02-08 18:51:39 +08:00
xiao-77 4011d01148 Fix some review errors. 2025-02-08 17:42:48 +08:00
xiao-77 7707e5d8dd Fix some review errors. 2025-02-08 17:37:15 +08:00
xiao-77 a2f11f21c9 Fix review problems. 2025-02-08 17:07:55 +08:00
xiao-77 72f228fa97 Fix build errors. 2025-02-07 18:52:54 +08:00
xiao-77 2cbb7a7814 Fix sdb config compare errors. 2025-02-07 18:49:00 +08:00
Haojun Liao 2f6977dfda Merge branch 'main' into enh/analysis 2025-01-23 18:55:32 +08:00
Haojun Liao 55ca679571 fix(analysis): fix compiling error on windows. 2025-01-23 09:06:55 +08:00
Haojun Liao 989c20078a refactor: do some internal refactor. 2025-01-23 00:32:46 +08:00
Haojun Liao 883b1b79a2 enh(analysis): add the algorithm in telemetry report. 2025-01-23 00:29:18 +08:00
Haojun Liao d6dc966ed1 fix(analysis): fix compiler error on windows. 2025-01-22 15:12:43 +08:00
Haojun Liao 9f2c448e36 fix(stream): fix error in check node update trans. 2025-01-22 14:05:40 +08:00
Haojun Liao 30358f1297 fix(stream): fix error in check node update trans. 2025-01-22 12:52:30 +08:00