Commit Graph

334 Commits

Author SHA1 Message Date
wangmm0220 a16189d2a9 fix:confilct from 3.0 2023-11-15 19:14:27 +08:00
wangmm0220 8a05bad899 fix:snode error 2023-11-10 14:14:22 +08:00
Haojun Liao 8c2f6bab0e fix(stream): exec scan-history in timer. 2023-11-07 16:40:03 +08:00
Haojun Liao 4d8548e938 refactor: do some internal refactor. 2023-11-06 23:42:16 +08:00
Haojun Liao d9474016fc fix(stream): exec scan-history in timer. 2023-11-06 18:55:53 +08:00
Haojun Liao dc8a90c864 fix(stream): fix memory leak. 2023-11-02 08:13:11 +08:00
Haojun Liao 79e971385f fix(stream): fix memory leak. 2023-11-02 08:11:51 +08:00
Haojun Liao df7633650e fix(stream): fix memory leak. 2023-11-01 20:35:45 +08:00
Haojun Liao e1b3e00b3d fix(stream): fix memory leak. 2023-11-01 20:10:20 +08:00
Haojun Liao 9a9c1d689d fix(stream): add lock log. 2023-11-01 19:38:12 +08:00
Haojun Liao 5ff89bc098 fix(stream): add lock log. 2023-11-01 13:19:53 +08:00
Haojun Liao c7285c0ede fix(stream): add expired epset node list for stream tasks in hb to mnode. 2023-10-27 23:41:57 +08:00
Haojun Liao 4c267b538f fix(stream): add expired epset node list for stream tasks in hb to mnode. 2023-10-27 23:05:41 +08:00
Haojun Liao 52d8e05c0c fix(stream): resolve the nodeUpdate msg compatible issue. 2023-10-27 09:59:52 +08:00
Haojun Liao 454624516c fix(stream): handle the repeatly issued nodeUpdate msg from dnode. 2023-10-27 09:59:51 +08:00
Haojun Liao 48ff521968 fix(stream): update the nodeEp even during checkpoint trans. 2023-10-27 09:55:02 +08:00
Haojun Liao 579dd8e6b2 refactor(stream): do some internal refactor. 2023-10-27 09:54:56 +08:00
Haojun Liao 6f3c0d17cc refactor(stream): add state machine to manage the state of stream tasks. 2023-10-27 09:54:49 +08:00
Haojun Liao 596eb073b7 refactor(stream): do some internal refactor. 2023-10-27 09:54:48 +08:00
Haojun Liao 985371e841 fix(stream): resolve the nodeUpdate msg compatible issue. 2023-10-25 10:01:20 +08:00
Haojun Liao 03165d5327 fix(stream): handle the repeatly issued nodeUpdate msg from dnode. 2023-10-25 09:55:02 +08:00
Haojun Liao 49014d4b68 fix(stream): update the nodeEp even during checkpoint trans. 2023-10-21 01:42:43 +08:00
Haojun Liao 402aefd95d refactor(stream): do some internal refactor. 2023-10-19 15:09:47 +08:00
Haojun Liao f4caeca24a refactor(stream): add state machine to manage the state of stream tasks. 2023-10-19 10:34:20 +08:00
Haojun Liao 54ec679b58 refactor(stream): do some internal refactor. 2023-10-18 15:34:53 +08:00
Haojun Liao af2c0c95e9 refactor: do some internal refactor. 2023-10-09 20:10:22 +08:00
Haojun Liao cf14772dd9 enh(stream): optimize the scan wal perf 2023-10-09 20:09:21 +08:00
Haojun Liao d5abb21335 fix(stream): fix invalid check. 2023-10-09 20:09:20 +08:00
Haojun Liao dfe8641c0c refactor: optimize the fill-history task launch policy, and do some other internal refactor. 2023-10-09 20:08:52 +08:00
Haojun Liao f2848de12a fix(stream): reduce the counter when remove stream tasks. 2023-10-09 20:08:52 +08:00
Haojun Liao 0db357401d fix(stream): add new status for meta. 2023-10-09 20:08:12 +08:00
Haojun Liao 3912d80c02 fix(stream): do some internal refactor. 2023-10-09 20:08:11 +08:00
Haojun Liao 65f66f0986 fix(stream): transfer state when all downstream rsp already. 2023-10-09 20:08:11 +08:00
Haojun Liao 52763ca2a3 refactor: do some internal refactor. 2023-10-09 20:08:07 +08:00
Haojun Liao c05a8de3e8 fix(stream): rsp when checkpoint source failed. 2023-10-09 20:08:06 +08:00
Haojun Liao b4bc4fac98 fix(stream): disable auto rsp for checkpoint source. 2023-10-09 20:08:06 +08:00
Haojun Liao 46b17aa9f1 fix(stream): fix race condition. 2023-10-09 20:08:06 +08:00
Haojun Liao 5eee76df65 fix(stream): add msg identifier check. 2023-10-09 20:08:05 +08:00
Haojun Liao 192c47a7c2 fix(stream): add some logs. 2023-10-09 20:08:05 +08:00
Haojun Liao 693254f602 fix(stream): remove invalid free. 2023-10-09 20:08:04 +08:00
Haojun Liao f3bd5c411d fix(stream): remove the blocking flag after successfully retry sending msg. 2023-10-09 20:08:04 +08:00
Haojun Liao ed9e035095 fix(stream): clear the failure vnode id list before send msg. 2023-10-09 20:08:04 +08:00
Haojun Liao e0d19af000 fix(stream): refactor re-try dispatch msg for stream tasks. 2023-10-09 20:08:03 +08:00
Haojun Liao 2b16976902 refactor(stream): refactor send msg. 2023-10-09 20:08:03 +08:00
Haojun Liao e5535fa1f5 fix(stream): handle dispatch checkpoint-trigger failure. 2023-10-09 20:08:01 +08:00
Haojun Liao 65cc4040f3 fix(stream): update logs. 2023-10-09 20:08:00 +08:00
Haojun Liao 6d3a134077 fix(stream): ignore the dispatch failure, and set the correct rsp counter. 2023-10-09 20:07:54 +08:00
Haojun Liao ef9efc8a8e refactor: do refactor the sink task. 2023-10-09 20:07:53 +08:00
Haojun Liao e4ac835554 fix(stream): disable token bucket. 2023-10-09 20:07:53 +08:00
Haojun Liao 1f45bd82d2 other: merge stream fix in main. 2023-10-09 20:07:01 +08:00
Haojun Liao 734f648891 fix(stream): set the output normal for stream dispatch when encountering the transfer state. 2023-10-09 20:05:38 +08:00
Haojun Liao ab17bfdf09 refactor: do some internal refactor. 2023-10-07 19:16:55 +08:00
Haojun Liao 41ef1a45c1 enh(stream): optimize the scan wal perf 2023-10-04 22:06:46 +08:00
Haojun Liao 5564eb215c fix(stream): fix invalid check. 2023-10-03 00:38:24 +08:00
Haojun Liao e13efd5b04 refactor: optimize the fill-history task launch policy, and do some other internal refactor. 2023-09-27 14:00:48 +08:00
Haojun Liao 37bf387453 fix(stream): reduce the counter when remove stream tasks. 2023-09-27 09:36:20 +08:00
Haojun Liao f3f18b0fa3 fix(stream): add new status for meta. 2023-09-26 12:15:12 +08:00
Haojun Liao d214a2f6bd fix(stream): do some internal refactor. 2023-09-26 09:32:57 +08:00
Haojun Liao c9576d7a85 fix(stream): transfer state when all downstream rsp already. 2023-09-25 17:31:38 +08:00
Haojun Liao 4ea737571d refactor: do some internal refactor. 2023-09-24 01:26:51 +08:00
Haojun Liao acd3321494 fix(stream): rsp when checkpoint source failed. 2023-09-23 14:13:33 +08:00
Haojun Liao 7d5f76c6f1 fix(stream): disable auto rsp for checkpoint source. 2023-09-23 11:31:14 +08:00
Haojun Liao fcbd747afc fix(stream): fix race condition. 2023-09-22 19:35:12 +08:00
Haojun Liao 5b12e276cf fix(stream): add msg identifier check. 2023-09-22 18:13:18 +08:00
Haojun Liao bb1a2322ca fix(stream): add some logs. 2023-09-22 17:27:10 +08:00
Haojun Liao 07ec6f38b8 fix(stream): remove invalid free. 2023-09-22 16:53:22 +08:00
Haojun Liao 5a695f632c fix(stream): remove the blocking flag after successfully retry sending msg. 2023-09-22 16:44:55 +08:00
Haojun Liao 6c357104a5 fix(stream): clear the failure vnode id list before send msg. 2023-09-22 16:19:25 +08:00
Haojun Liao 619a9f6d08 fix(stream): refactor re-try dispatch msg for stream tasks. 2023-09-22 15:57:55 +08:00
Haojun Liao 898aea5cfb refactor(stream): refactor send msg. 2023-09-22 14:52:18 +08:00
Haojun Liao b075b4438b fix(stream): handle dispatch checkpoint-trigger failure. 2023-09-21 23:15:18 +08:00
Haojun Liao 50d5fb6641 fix(stream): update logs. 2023-09-21 19:14:34 +08:00
Haojun Liao 4925b67d4a fix(stream): ignore the dispatch failure, and set the correct rsp counter. 2023-09-21 17:22:09 +08:00
Haojun Liao 17586af0ce refactor: do refactor the sink task. 2023-09-21 13:32:57 +08:00
Haojun Liao 9f9b712332 fix(stream): disable token bucket. 2023-09-21 01:19:58 +08:00
Haojun Liao 6633fb971b fix(stream): update logs. 2023-09-20 18:18:31 +08:00
Haojun Liao c7b8d572f6 fix(stream): set the output normal for stream dispatch when encountering the transfer state. 2023-09-18 18:25:22 +08:00
Haojun Liao 09a15bf340 refactor: do some internal refactor. 2023-09-12 22:07:52 +08:00
Haojun Liao f6a097d96f refactor: do some internal refactor. 2023-09-12 21:58:22 +08:00
Haojun Liao 13c20facd0 fix(stream): fix syntax error. 2023-09-12 18:37:30 +08:00
Haojun Liao e457d94ac6 refactor: do some internal refactor. 2023-09-12 18:09:02 +08:00
Haojun Liao 35b97cbd19 fix(stream): set correct token in bucket. 2023-09-08 23:50:41 +08:00
Haojun Liao 4f85a2620a fix(stream): remove invalid free. 2023-09-05 09:27:16 +08:00
Haojun Liao d8518de5e0 fix(stream):fix error in timerActive. 2023-09-01 20:20:39 +08:00
Haojun Liao 3fc3dafca6 refactor: do some internal refactor. 2023-08-31 13:46:48 +08:00
Haojun Liao d99d8875e1 fix(stream): if stream task is running, do not recheck its status. 2023-08-30 17:24:13 +08:00
liuyao 0a72e576bd op stream pause
pause data dispatch

opt stream pause
2023-08-30 14:53:49 +08:00
Haojun Liao c3e9590740 fix(stream): fix error. 2023-08-29 15:11:01 +08:00
Haojun Liao 60d1df0e25
Merge branch '3.0' into enh/triggerCheckPoint2 2023-08-25 15:49:28 +08:00
Haojun Liao ce1a3a379c fix(stream): handle the case when the task not in ready state, and the checkpoint msg arrived. 2023-08-25 10:12:12 +08:00
Haojun Liao f7dada53b9
Merge pull request #22559 from taosdata/fix/liaohj
other: merge main.
2023-08-24 19:01:18 +08:00
Haojun Liao 230d8c2d80 fix(stream): fix error in stream computing after vnode is transferred to other dnodes. 2023-08-23 17:50:44 +08:00
Haojun Liao b0c5fa8290 fix(stream): reset the task status. 2023-08-23 14:10:41 +08:00
Haojun Liao def0877980 fix(stream): add timer to check stream task status. 2023-08-22 10:46:49 +08:00
Haojun Liao 718bb4a881 fix(stream): fix the invalid free. 2023-08-18 22:40:04 +08:00
Haojun Liao ea6574b3c7 other: merge 3.0 2023-08-17 00:07:55 +08:00
Haojun Liao 7628be236d other: merge main. 2023-08-16 09:03:54 +08:00
Haojun Liao 39079ab64a fix(stream): load term from sync model to decide if the node is restart or not 2023-08-15 15:52:06 +08:00
Haojun Liao 0acb00c516 fix(stream): add log. 2023-08-15 11:48:56 +08:00
Haojun Liao b1d35f43ba enh(stream): using term from sync module to denote the upstream node is restarted/replica-modified/vnode-transferred or not. 2023-08-15 09:25:05 +08:00
Haojun Liao a334547726 fix(stream): fix an syntax error. 2023-08-14 19:28:36 +08:00
Haojun Liao 1a08ffc79f fix(stream): check error. 2023-08-14 19:27:09 +08:00
Haojun Liao 2e6263b43f fix(stream): fix memory leak. 2023-08-14 17:16:11 +08:00
Haojun Liao 2806fe1c56 fix(stream): ignore the related stream task destory msg in transfer state. 2023-08-14 10:58:24 +08:00
Haojun Liao 98f40325e9 fix(stream): fix memory leak. 2023-08-13 20:02:15 +08:00
Haojun Liao 0d38f389ab refactor: do some internal refactor. 2023-08-13 16:25:35 +08:00
Haojun Liao 0e189f70a1 refactor 2023-08-12 19:44:44 +08:00
Haojun Liao 03c26a9d6d refactor 2023-08-12 19:16:20 +08:00
Haojun Liao afe9b848a5 fix(tsdb): check the 2023-08-12 18:54:39 +08:00
Haojun Liao 09da6c6840 refactor: do some internal refactor. 2023-08-12 18:51:20 +08:00
Haojun Liao 75e6fb0f16 refactor: do some internal refactor. 2023-08-12 18:47:01 +08:00
Haojun Liao 30c2a9c619 refactor: do some internal refactor. 2023-08-12 18:27:51 +08:00
Haojun Liao 6a9826cf6e enh(stream): update the stream task epset. 2023-08-11 10:37:25 +08:00
Haojun Liao f3c5f20ee2 enh(stream): add timer to check the node changing. 2023-08-09 10:25:16 +08:00
Haojun Liao f8ee055641 other: merge 3.0 2023-08-08 00:19:08 +08:00
Haojun Liao aea1690cd1 fix(stream): fix the stream task after checking if it is in the timer activities. 2023-08-07 17:49:36 +08:00
Haojun Liao 13543f7daa feature(stream): handle task update in tq module. 2023-08-02 10:33:41 +08:00
Haojun Liao 164bfd5408 enh(stream): prepare update trans. 2023-08-01 16:12:35 +08:00
Haojun Liao 75b1520be0 enh(stream): create task update info 2023-08-01 14:04:07 +08:00
Haojun Liao 9f943213ed enh(stream): update the hb info. 2023-07-28 14:23:53 +08:00
Haojun Liao 74834aea11 enh(stream): prepare hb to mnode from stream meta. 2023-07-28 13:07:16 +08:00
Haojun Liao c116b69941 fix(stream): fix memory leak. 2023-07-25 09:25:18 +08:00
Haojun Liao 1e34f84f78 other: merge 3.0 2023-07-24 15:23:03 +08:00
Haojun Liao 9679cc4527 fix(stream): fix race condition. 2023-07-22 10:19:51 +08:00
Haojun Liao f4864b8713 fix(stream): fix memory leak. 2023-07-22 01:02:58 +08:00
Haojun Liao 1b2636028a fix(stream): fix memory leak. 2023-07-21 23:05:42 +08:00
Haojun Liao 16d7707b90 fix(stream): align the scan real time data for stream task. 2023-07-21 19:33:43 +08:00
Haojun Liao 97c44e7c18 refactor: do some internal refactor. 2023-07-19 18:42:02 +08:00
Haojun Liao c6a322e0ef refactor(stream): back pressure is active, which is trigger by dispatch rsp. 2023-07-16 12:47:08 +08:00
Haojun Liao eb7e6152ae refactor(stream): do some internal refactor. 2023-07-14 17:21:20 +08:00
Haojun Liao 2fd72500e2 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-07-13 16:32:37 +08:00
Haojun Liao 813f4cb363 refactor: do some internal refactor. 2023-07-13 16:32:25 +08:00
yihaoDeng f3f3893872 fix compile error 2023-07-13 10:05:29 +08:00
Haojun Liao 08a4fb06ae fix(stream): dispatch checkpoint msg to downstream by puting message into input queue. 2023-07-13 09:17:20 +08:00
Haojun Liao d392bb60ba refactor(stream): set the initial version from the checkpoint. 2023-07-12 11:38:13 +08:00
Haojun Liao 87bedef2cd fix(stream): fix error in checkpointing. 2023-07-10 19:20:05 +08:00
Haojun Liao 385a5c9651 fix(stream): fix error in generating checkpoint. 2023-07-10 18:20:50 +08:00
Haojun Liao d6e7e6e174 fix(stream): fix typo. 2023-07-10 14:57:28 +08:00
Haojun Liao 16ee72ab06 fix(stream): free msg after send checkpoint rsp. 2023-07-10 13:57:51 +08:00
Haojun Liao f6515e2a2c fix(stream): fix error in checkpointing. 2023-07-10 11:43:37 +08:00
Haojun Liao a5c19427e7 fix(stream): send correct rsp to mnode. 2023-07-08 09:39:44 +08:00
Haojun Liao 47877898d0 enh(stream): generate the checkpoint framework. 2023-07-07 16:07:52 +08:00
Haojun Liao d7101f7109 enh(stream): support dispatch checkpoint msg. 2023-07-06 18:26:22 +08:00
Haojun Liao 03dbcbf055 refactor: do some internal refactor. 2023-07-05 15:55:55 +08:00
Haojun Liao 2dbccc1573 refactor: do some internal refactor. 2023-07-03 17:52:17 +08:00
Haojun Liao 57fcd553f8 refactor: do some internal refactor. 2023-06-28 13:13:13 +08:00
Haojun Liao 3289ad62be other: merge 3.0 2023-06-27 11:33:40 +08:00
Haojun Liao b73444b291 fix(stream): start stream task in case of scan history completing. 2023-06-27 09:44:51 +08:00
liuyao 6b8549dea5 set table name 2023-06-26 14:14:46 +08:00
Haojun Liao 0322fdc1fc fix(stream): fix memory leak. 2023-06-24 23:43:43 +08:00