xxq250
/
homework-jianmu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
64,588 Commits 3 Branches 0 Tags 479 MiB
Commit Graph

1221 Commits

Author SHA1 Message Date
dapan1121 228e6c6acd Merge remote-tracking branch 'origin/main' into 3.0 2023-09-01 10:01:10 +08:00
Haojun Liao e8294ed8dc fix(stream): fix bug in multi-replica vnode redistribute. 2023-09-01 03:10:12 +08:00
Haojun Liao e2bb64eb18 refactor: do some internal refactor. 2023-08-31 14:23:46 +08:00
Haojun Liao 3fc3dafca6 refactor: do some internal refactor. 2023-08-31 13:46:48 +08:00
Haojun Liao e53b5d4392 refactor: do some internal refactor. 2023-08-31 11:01:53 +08:00
wangmm0220 23c4d7eb0a fix:dead lock 2023-08-30 18:10:19 +08:00
Haojun Liao d99d8875e1 fix(stream): if stream task is running, do not recheck its status. 2023-08-30 17:24:13 +08:00
Haojun Liao c5b279f045 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-08-30 16:01:08 +08:00
Haojun Liao 24b1a050fc other: merge 3.0 2023-08-30 16:00:14 +08:00
liuyao 0a72e576bd op stream pause 
pause data dispatch

opt stream pause
 2023-08-30 14:53:49 +08:00
dapan1121 f6321d0c4f Merge remote-tracking branch 'origin/main' into fix/m23.1 2023-08-30 11:03:33 +08:00
Haojun Liao 3a7a220d43 fix(stream): fix error found by ci. 2023-08-30 10:14:01 +08:00
Haojun Liao 6e4acddd96
Merge pull request #22603 from taosdata/fix/TD-25962 
fix:unregister push manager if delete subscribe to avoid tqProcessPol…
 2023-08-29 10:58:49 +08:00
wangmm0220 2e04d513a4 fix:unregister push manager if delete subscribe to avoid tqProcessPollPush 2023-08-28 16:41:28 +08:00
Haojun Liao 05bb164675 fix(stream): release tasks. 2023-08-28 15:56:02 +08:00
Haojun Liao 391103bc5a fix(stream): do dome internal refactor. 2023-08-28 15:54:01 +08:00
Haojun Liao 33672f2f35 fix(stream): release stream tasks. 2023-08-28 15:52:54 +08:00
Haojun Liao b75ae217f6 fix(stream): notify close for snode. 2023-08-28 09:53:08 +08:00
dapan1121 1f40da6fef Merge remote-tracking branch 'origin/main' into 3.0 2023-08-28 09:28:55 +08:00
Haojun Liao 6b327886e9 fix(stream): add null ptr check. 2023-08-27 00:27:04 +08:00
Haojun Liao e013ba6795 fix(stream): fix the counter. 2023-08-25 17:32:34 +08:00
Haojun Liao ab2aad2b2c fix(stream): update the task status even it is in stop status. 2023-08-25 15:32:55 +08:00
Haojun Liao 4c7ad54c9c fix(stream): not restart tasks when in restoring procedure. 2023-08-25 11:28:28 +08:00
Haojun Liao 38bb123701 fix(stream): release task. 2023-08-25 10:33:15 +08:00
Haojun Liao ce1a3a379c fix(stream): handle the case when the task not in ready state, and the checkpoint msg arrived. 2023-08-25 10:12:12 +08:00
Haojun Liao f7ea875af1 fix(stream): add some logs. 2023-08-24 23:27:48 +08:00
Haojun Liao 82e356a9b5 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 
# Conflicts:
#	source/dnode/vnode/src/tq/tq.c
 2023-08-24 18:04:44 +08:00
Haojun Liao 70396aa5a5 fix(stream): wakeup the hb timer every 200ms, to speeup the close db procedure. 2023-08-24 17:54:50 +08:00
yihaoDeng 090515c684 refactor code 2023-08-24 17:20:04 +08:00
wangmm0220 b83cc11043 fix:[TD-25651] reset epoch if consumer changed to avoid consumeing no data 2023-08-24 16:38:53 +08:00
wangmm0220 49216ffa54 fix:transaction in tmq 2023-08-24 16:32:15 +08:00
Haojun Liao 230d8c2d80 fix(stream): fix error in stream computing after vnode is transferred to other dnodes. 2023-08-23 17:50:44 +08:00
Haojun Liao b0c5fa8290 fix(stream): reset the task status. 2023-08-23 14:10:41 +08:00
Haojun Liao af80d94a88 refactor: do some internal refactor. 2023-08-22 15:43:50 +08:00
Haojun Liao cc8b2263cb fix(stream): fix error in heartbeat from vnode. 2023-08-22 13:37:38 +08:00
Haojun Liao def0877980 fix(stream): add timer to check stream task status. 2023-08-22 10:46:49 +08:00
Haojun Liao 873c22abb1 fix(stream): reset update stream tasks. 2023-08-21 19:32:27 +08:00
Haojun Liao 8b27f98ee4 refactor: do some internal refactor. 2023-08-21 19:08:27 +08:00
yihaoDeng 387b4d365e refactor checkpoint 2023-08-21 19:07:15 +08:00
Haojun Liao 29bb5854cc fix(stream): fix error in check stream state. 2023-08-21 11:48:55 +08:00
Haojun Liao b8101afbe2 fix(stream): refactor the tasks restart mechanism. 2023-08-18 22:36:38 +08:00
Haojun Liao edbad45772 other: add logs. 2023-08-18 13:45:08 +08:00
Haojun Liao a8b53c2e69 other: add logs. 2023-08-18 10:51:59 +08:00
Haojun Liao 633ecf66e5 other: add logs. 2023-08-18 10:39:32 +08:00
Haojun Liao 9ad8cf48dc other: add logs. 2023-08-18 10:38:24 +08:00
Haojun Liao 87785b1d9c refactor: add some logs. 2023-08-18 10:30:00 +08:00
Haojun Liao ea6574b3c7 other: merge 3.0 2023-08-17 00:07:55 +08:00
yihaoDeng 993ae84398 close backend 2023-08-16 07:17:15 +00:00
Haojun Liao ad9f3cbc43 other: merge main. 2023-08-16 09:14:10 +08:00
Haojun Liao 7628be236d other: merge main. 2023-08-16 09:03:54 +08:00
Haojun Liao 39079ab64a fix(stream): load term from sync model to decide if the node is restart or not 2023-08-15 15:52:06 +08:00
Haojun Liao b1d35f43ba enh(stream): using term from sync module to denote the upstream node is restarted/replica-modified/vnode-transferred or not. 2023-08-15 09:25:05 +08:00
Haojun Liao a1e554fbf3 refactor: exec directly not asynchnoized. 2023-08-14 15:05:13 +08:00
Haojun Liao 91710b0c0f fix(stream): 2023-08-14 14:56:17 +08:00
Haojun Liao ead04964fd Merge branch 'main' into fix/liaohj 2023-08-14 12:11:15 +08:00
Haojun Liao 2806fe1c56 fix(stream): ignore the related stream task destory msg in transfer state. 2023-08-14 10:58:24 +08:00
Haojun Liao 01ac99a651
Merge pull request #22413 from taosdata/fix/liaohj 
refactor: refactor the transfer state procedure.
 2023-08-13 23:28:18 +08:00
Haojun Liao 0d38f389ab refactor: do some internal refactor. 2023-08-13 16:25:35 +08:00
Haojun Liao f24b22000f enh(stream): stop stream task before redistribute vnodes. 2023-08-11 23:46:53 +08:00
Haojun Liao 87c7e4c0fe fix(stream): not launch stream tasks when restarting the stream tasks. 2023-08-11 16:19:46 +08:00
Haojun Liao e984affe90 fix(stream): disable the stream in the followers nodes. 2023-08-11 15:58:15 +08:00
wangmm0220 25e32bbf7c fix:offset error in tmq & add test cases 2023-08-11 14:25:12 +08:00
Haojun Liao 9322f80343 fix(stream): register the rsp callback. 2023-08-11 13:31:50 +08:00
Haojun Liao 6a9826cf6e enh(stream): update the stream task epset. 2023-08-11 10:37:25 +08:00
Haojun Liao d9bc881fb3
Merge pull request #22384 from taosdata/mark/tmq 
fix:offset error in tmq & add test cases
 2023-08-11 10:00:08 +08:00
wangmm0220 78fd70202c fix:offset error in tmq & add test cases 2023-08-10 16:18:53 +08:00
Haojun Liao a713b37f35 fix(stream): add more logs. 2023-08-10 14:07:29 +08:00
Haojun Liao 161fd6902a fix(stream): fix the acquire task. 2023-08-10 12:45:20 +08:00
Haojun Liao f633c7fe2b other: merge 3.0. 2023-08-10 11:56:30 +08:00
Haojun Liao b658608541
Merge pull request #22388 from taosdata/fix/3_liaohj 
other: merge main.
 2023-08-10 11:43:02 +08:00
Haojun Liao c10b323188 other: merge main. 2023-08-10 09:26:33 +08:00
Haojun Liao 4393375e47 fix(stream): set correct task id. 2023-08-10 02:16:04 +08:00
Haojun Liao 0757e88ced fix(stream): set the correct hash keys. 2023-08-09 23:52:14 +08:00
wangmm0220 1f5532fd67 fix:offset error in tmq & add test cases 2023-08-09 19:41:22 +08:00
Haojun Liao 1e8579e8c5 fix(stream): fix other cases. 2023-08-09 16:04:48 +08:00
Haojun Liao 4c92997328 fix(stream): use streamId&taskId to identify the stream task in the stream meta. 2023-08-09 15:52:23 +08:00
Haojun Liao 3cdb16352c fix(stream): avoid launching check downstream when failed to add stream task into stream meta. 2023-08-09 12:03:14 +08:00
Haojun Liao f3c5f20ee2 enh(stream): add timer to check the node changing. 2023-08-09 10:25:16 +08:00
Haojun Liao 17c6143471 fix(stream): check more status when handling the state transfer. 2023-08-08 15:08:11 +08:00
Haojun Liao f8ee055641 other: merge 3.0 2023-08-08 00:19:08 +08:00
Haojun Liao ad3e231b2f
Merge pull request #22350 from taosdata/fix/3_liaohj 
other: merge main and fix bugs in stop stream tasks.
 2023-08-07 22:57:13 +08:00
Haojun Liao aea1690cd1 fix(stream): fix the stream task after checking if it is in the timer activities. 2023-08-07 17:49:36 +08:00
Haojun Liao 225c0e61dc hen(stream): support the rollback and checkdown stream tasks. 2023-08-07 16:13:39 +08:00
wangmm0220 9ecabb30f5 fix:ref is not there 2023-08-07 10:27:44 +08:00
wangmm0220 5f02e2ddfb fix:ref is not there 2023-08-07 09:31:29 +08:00
Haojun Liao 8a11a0418f other: merge 3.0 2023-08-07 09:19:26 +08:00
wangmm0220 d9a1695257 fix:add log 2023-08-04 18:07:44 +08:00
Haojun Liao 13543f7daa feature(stream): handle task update in tq module. 2023-08-02 10:33:41 +08:00
Haojun Liao e1e5e9bb0f enh(stream): check stream tasks dag. 2023-08-02 09:44:39 +08:00
Haojun Liao ae9745ecf2
Merge pull request #22258 from taosdata/mark/tmq-3.0 
fix:add committed & seek process logic
 2023-08-01 09:52:09 +08:00
Haojun Liao 2640e41705
Merge pull request #22261 from taosdata/fix/3_liaohj 
fix(stream): set the correct end key of delete block.
 2023-07-31 22:55:50 +08:00
wangmm0220 cddf27e8b4 fix:add committed & seek process logic 2023-07-31 18:48:50 +08:00
Haojun Liao d45596bb33 fix(stream): kill task when pause the stream execution. 2023-07-31 18:07:37 +08:00
wangmm0220 6179618b17 fix:add committed & seek process logic 2023-07-31 17:56:04 +08:00
wangmm0220 8093ef2dc8 Merge branch 'main' of https://github.com/taosdata/TDengine into mark/tmq 2023-07-31 16:14:25 +08:00
Haojun Liao 9981cecbed refactor: do some internal refactor. 2023-07-31 15:24:35 +08:00
wade zhang bae028d65c
Merge pull request #22230 from taosdata/mark/tmq 
fix:semaphore always wait in tmq commit logic
 2023-07-31 09:10:13 +08:00
wade zhang c26165975d
Merge pull request #22227 from taosdata/fix/TS-3728 
fix:add macro ASSERT_NOT_CORE to control ASSERT not core in release version
 2023-07-30 14:28:56 +08:00
Haojun Liao 8dfef8768a other: merge 3.0 2023-07-29 22:54:16 +08:00
Haojun Liao 0bd2548937 fix(stream): initialize the filter window initial range. 2023-07-29 16:41:01 +08:00
Haojun Liao f6e07d1fdb other: merge 3.0 2023-07-29 01:32:02 +08:00
Haojun Liao 936afeb5ac refactor: do internal refactor to simple the pause/resume in case of fill history exists. 2023-07-28 19:13:03 +08:00
Haojun Liao d5974a8f25 refactor(stream): refactor the pause/resume for fill history execution. 2023-07-28 18:05:32 +08:00
wangmm0220 6e673072cd fix:conflicts 2023-07-28 14:57:13 +08:00
Haojun Liao 9f943213ed enh(stream): update the hb info. 2023-07-28 14:23:53 +08:00
Haojun Liao 74834aea11 enh(stream): prepare hb to mnode from stream meta. 2023-07-28 13:07:16 +08:00
Haojun Liao 2a78cd2d23
Merge pull request #22215 from taosdata/fix/liaohj 
other: merge main.
 2023-07-28 13:06:17 +08:00
wangmm0220 2f6c565392 fix:pHandle not load if taosd restart 2023-07-28 11:48:53 +08:00
Haojun Liao 7f3d1dc464 fix(stream): fix memory leak. 2023-07-27 23:06:53 +08:00
Haojun Liao fbb33be85d other: add some comments. 2023-07-27 18:59:38 +08:00
Haojun Liao 22a5a18fff fix(stream): 1. set correct timewindow after step2. 2. handle the case when the task is failed to be added into the meta store. 2023-07-27 18:57:36 +08:00
Haojun Liao b145771a1b other: merge 3.0 2023-07-27 16:38:57 +08:00
Haojun Liao 3d403dca5a other: merge main. 2023-07-27 16:23:41 +08:00
wangmm0220 75eacd56f1 fix:add log 2023-07-27 15:58:32 +08:00
Haojun Liao 40fe3ef4f6 fix(stream): refactor the step2 wal scan. 2023-07-27 14:49:20 +08:00
Haojun Liao e89f530db2 fix(stream): scan wal in step2 2023-07-27 10:55:59 +08:00
Haojun Liao 3c387f6e91 fix(stream): fix syntax error. 2023-07-26 17:00:41 +08:00
Haojun Liao 432c943c25 fix(stream): do filter before the update check. 2023-07-26 16:34:39 +08:00
Haojun Liao cdffabcdad fix(stream): fix memory leak. 2023-07-26 14:37:04 +08:00
Haojun Liao 728112ed89 fix(stream): set correct upstream epinfo. 2023-07-26 10:26:59 +08:00
Haojun Liao 1c1bf63e1f other: merge 3.0 2023-07-25 18:17:11 +08:00
Haojun Liao 0bdccd2681 refactor: do some internal refactor. 2023-07-25 17:31:54 +08:00
Haojun Liao d29f835a63 fix(stream): set the correct step2 scan time window range. 2023-07-25 15:09:22 +08:00
Haojun Liao a2694b9ce9 fix(stream): fix the invalid check of step2 . 2023-07-24 19:44:28 +08:00
wangmm0220 9628a9f74e fix:open info log in tmq & ignore wal apply ver when read wal 2023-07-24 15:53:02 +08:00
Haojun Liao 1e34f84f78 other: merge 3.0 2023-07-24 15:23:03 +08:00
Haojun Liao 9bae91bcba fix(stream): add some logs, and remove the invalid loop. 2023-07-24 13:37:44 +08:00
Haojun Liao 1367552f38 fix(stream): refactor the halt status to check more status. 2023-07-24 10:04:15 +08:00
Haojun Liao 0b6e1a12bb refactor: do some internal refactor. 2023-07-23 19:09:43 +08:00
Haojun Liao e387b2f932 refactor: do some internal refactor. 2023-07-23 18:59:34 +08:00
Haojun Liao 4e0f7ffbf8 fix(stream): set the correct flag of pause/resume. 2023-07-23 18:49:56 +08:00
Haojun Liao bfc4a07207 fix(stream): fix memory leak. 2023-07-22 00:20:02 +08:00
Haojun Liao 1b2636028a fix(stream): fix memory leak. 2023-07-21 23:05:42 +08:00
Haojun Liao 16d7707b90 fix(stream): align the scan real time data for stream task. 2023-07-21 19:33:43 +08:00
wangmm0220 bb86f5c580 fix:commit offset should not plus 1 2023-07-21 11:13:05 +08:00
Haojun Liao 39ca9b537f fix(stream): fix syntax error 2023-07-21 09:43:08 +08:00
Haojun Liao c9d75eab5a refactor: do some internal refactor. 2023-07-21 09:16:13 +08:00
Haojun Liao db46d31ca5 fix(stream): refactor the halt function. 2023-07-21 01:32:42 +08:00
Haojun Liao 9353f7c8cf refactor: do some internal refactor. 2023-07-20 16:35:02 +08:00
Haojun Liao c9fa170e65 refactor: do some internal refactor. 2023-07-20 16:31:46 +08:00
Haojun Liao 45cb478b31 refactor: do some internal refactor. 2023-07-20 15:36:46 +08:00
Haojun Liao aed99da6c1 refactor: refactor the stream task starting order. 2023-07-20 12:14:53 +08:00
Haojun Liao 97c44e7c18 refactor: do some internal refactor. 2023-07-19 18:42:02 +08:00
wangmm0220 d7d81d82a0 feat:add committed & position & commite_offset interface 2023-07-19 18:27:19 +08:00
wangmm0220 5cb35f2fa6 feat:add committed & position & commite_offset interface 2023-07-19 17:29:39 +08:00
Haojun Liao a32c650248 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-07-19 10:05:48 +08:00
Haojun Liao 2ae95217bd fix(stream): open inputQ by default for task deployed on snode. 2023-07-19 10:05:35 +08:00
liuyao fd02b9a71e
Merge branch '3.0' into enh/triggerCheckPoint2 2023-07-19 10:02:06 +08:00
Haojun Liao 9513f76172 fix(stream): set downstream task ready state. 2023-07-18 19:02:24 +08:00
Haojun Liao 901b7d8ecc fix(stream): if a source task is set to be non-normal, no data should be put into inputQ anymore. 2023-07-18 15:18:57 +08:00
Haojun Liao 92106312d2 fix(stream): not dispatch checkpoint block result to downstream. 2023-07-18 09:43:38 +08:00
Haojun Liao d7f7a7d6cf fix(stream): fix syntax error 2023-07-17 12:53:36 +08:00
Haojun Liao b2a905bd91
Merge branch '3.0' into enh/triggerCheckPoint2 2023-07-17 11:50:12 +08:00
Haojun Liao a9757637d9 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-07-16 12:47:24 +08:00
Haojun Liao c6a322e0ef refactor(stream): back pressure is active, which is trigger by dispatch rsp. 2023-07-16 12:47:08 +08:00
yihaoDeng e18c9ea672 add checkpoint 2023-07-15 04:26:34 +00:00
yihaoDeng 3c4631ef1f add checkpoint 2023-07-14 10:06:24 +00:00
Haojun Liao eb7e6152ae refactor(stream): do some internal refactor. 2023-07-14 17:21:20 +08:00
Haojun Liao e0ea0f2df4
Merge pull request #22020 from taosdata/fix/liaohj 
other: merge main
 2023-07-13 17:55:28 +08:00
Haojun Liao 813f4cb363 refactor: do some internal refactor. 2023-07-13 16:32:25 +08:00
Haojun Liao 2f90a53548 fix(stream): set checkpoint id when expanding task. 2023-07-13 10:16:57 +08:00
wangmm0220 cf64d4c9c5 fix:set get_assignment offset to first version of response block 2023-07-12 17:22:23 +08:00
Haojun Liao d392bb60ba refactor(stream): set the initial version from the checkpoint. 2023-07-12 11:38:13 +08:00
Haojun Liao fed3803739 fix(stream): fix memory leak. 2023-07-11 19:29:52 +08:00
Haojun Liao 2e386a6fdf fix(stream): check if stream task is in normal state. 2023-07-11 15:23:49 +08:00
Haojun Liao 6d78816d22 fix(stream): fix syntax error. 2023-07-11 14:06:25 +08:00
Haojun Liao 09b764494d fix(stream): set correct checkpoint version when starting stream tasks. 2023-07-11 14:03:43 +08:00
Haojun Liao 3ffdbe923d other: merge 3.0 2023-07-11 09:12:57 +08:00
Haojun Liao 04f5bcd23a fix(stream): remove invalid set of task status . 2023-07-11 08:51:13 +08:00
Haojun Liao 87bedef2cd fix(stream): fix error in checkpointing. 2023-07-10 19:20:05 +08:00
Haojun Liao 385a5c9651 fix(stream): fix error in generating checkpoint. 2023-07-10 18:20:50 +08:00
wangmm0220 35ba132a29 fix:conflicts from main 2023-07-09 12:52:18 +08:00
wangmm0220 f9f656b21a fix:return offset stored if get assignment in init mode 2023-07-08 13:25:39 +08:00
Haojun Liao 4f534ce6e9 fix(stream): fix error in generating checkpoint. 2023-07-08 10:29:09 +08:00
Haojun Liao a5c19427e7 fix(stream): send correct rsp to mnode. 2023-07-08 09:39:44 +08:00
Haojun Liao 9c30abf95b fix(stream): fix error in generating checkpoint. 2023-07-07 21:38:53 +08:00
wangmm0220 4436eb7e0f fix:set commitOffset is send msg success & optimize log & add test cases for TD-25129 2023-07-07 19:12:18 +08:00
Haojun Liao 128f67784a fix(stream): fix error in parsing checkpoint msg. 2023-07-07 18:17:50 +08:00
Haojun Liao 8298f30e56 fix(stream): fix bug in checkpoint. 2023-07-07 17:16:08 +08:00
wangmm0220 106e273157 fix:bugs in get_assignment 2023-07-07 16:12:15 +08:00
Haojun Liao 47877898d0 enh(stream): generate the checkpoint framework. 2023-07-07 16:07:52 +08:00
wangmm0220 0928bd5510 fix:send rsp offset to client if unregister push mgr 2023-07-06 19:35:48 +08:00
Haojun Liao d7101f7109 enh(stream): support dispatch checkpoint msg. 2023-07-06 18:26:22 +08:00
wangmm0220 11e6856d3e fix:init char array 2023-07-06 13:37:58 +08:00
Haojun Liao 5a9461a69a other: merge 3.0 2023-07-06 11:52:21 +08:00
Haojun Liao f4a3567935 other: merge 3.0 2023-07-06 11:24:30 +08:00
wangmm0220 34a62c3f38 fix:vginfo assigment not in same time,we should get vginfo by vg,not all vg 2023-07-05 22:55:41 +08:00
Haojun Liao 74f5b3f80e
Merge pull request #21959 from taosdata/refact/fillhistory 
refactor: do some internal refactor.
 2023-07-05 17:47:00 +08:00
wangmm0220 bdfeb32923 fix:bugs in tmq_get_topic_assignment 2023-07-05 16:04:34 +08:00
Haojun Liao 03dbcbf055 refactor: do some internal refactor. 2023-07-05 15:55:55 +08:00
Haojun Liao 9b3e34d589
Merge pull request #21735 from taosdata/refact/fillhistory 
refactor: refactor the fill history operation
 2023-07-05 10:48:51 +08:00
Haojun Liao ff2bf35603 fix(stream): fix error in set the version range for secondary scan. 2023-07-04 15:08:39 +08:00
Haojun Liao 20d099dfa6
Merge pull request #21901 from taosdata/fix/liaohj 
fix: fix coverity issue.
 2023-07-03 15:15:21 +08:00
wangmm0220 58c7011e0e feat:support only meta in tmq subscriptions 2023-06-30 11:42:40 +08:00
Haojun Liao 622e1cb608 fix: fix coverity issue. 2023-06-29 15:54:09 +08:00
liuyao 2fc5eeb810 session win range 2023-06-28 17:20:08 +08:00
Haojun Liao 8867a57262 fix(stream): fix syntax error. 2023-06-28 14:48:59 +08:00
Haojun Liao 57fcd553f8 refactor: do some internal refactor. 2023-06-28 13:13:13 +08:00
Haojun Liao 3710ea4aca refactor: do some internal refactor. 2023-06-28 09:41:08 +08:00
Haojun Liao a19e63fd23 fix(stream): fix error in handling fill history. 2023-06-27 23:00:14 +08:00
Yihao Deng 4de7db06f9
Merge branch '3.0' into enh/triggerCheckPoint2 2023-06-27 17:57:14 +08:00
Haojun Liao 4bb78df27c fix(stream): reduce the sleep time. 2023-06-27 16:44:35 +08:00
Haojun Liao 32fddeff18 fix(stream): fix error retrieve data from source task.And optimize the test cases. 2023-06-27 16:24:47 +08:00
Haojun Liao 3289ad62be other: merge 3.0 2023-06-27 11:33:40 +08:00
Haojun Liao b73444b291 fix(stream): start stream task in case of scan history completing. 2023-06-27 09:44:51 +08:00
dapan1121 4dae9f4eff merge main to 3.0 2023-06-26 18:43:00 +08:00
Haojun Liao 402c091df0 fix(stream): fix sma error. 2023-06-25 19:41:44 +08:00
liuyao e236bf818c set fill history info 2023-06-25 17:47:10 +08:00
Haojun Liao 7c23630fe4 fix(stream): set correct stream error code. 2023-06-25 09:26:37 +08:00
Haojun Liao ea7b8e91d3 fix(stream): fix memory leak. 2023-06-21 12:03:26 +08:00
liuyao 74bea44300 stream op transform 2023-06-20 17:30:59 +08:00
liuyao aafbdcb492 set task status 2023-06-20 15:20:04 +08:00
liuyao 870d75bf82 fill history pause&resume 2023-06-20 11:34:09 +08:00
liuyao 41c193945f fill history pause&resume 2023-06-19 17:28:19 +08:00
yihaoDeng fd85a8495a Merge branch 'dev30' into enh/triggerCheckPoint2 2023-06-17 14:17:21 +08:00
Haojun Liao 64365666b7 fix(stream): update the info, and do some internal refactor. 2023-06-16 16:04:43 +08:00
Haojun Liao 4c82558eab Merge remote-tracking branch 'origin/refact/fillhistory' into refact/fillhistory 2023-06-16 09:55:24 +08:00
Haojun Liao 933801269a fix(stream): fix memory leak and failed to close vnode. 2023-06-16 09:55:11 +08:00
liuyao 0aa141ec77 use stream task state 2023-06-15 18:24:40 +08:00
Haojun Liao 74e0138f72 Merge branch '3.0' into refact/fillhistory 2023-06-15 15:25:00 +08:00
wangmm0220 22776e60d5 fix:control memory size if consumer group is too much 2023-06-15 14:51:04 +08:00
liuyao 693942b821 trans state 2023-06-15 14:13:17 +08:00
Haojun Liao 1b9eca5ce0 other: merge main. 2023-06-15 13:03:17 +08:00
Haojun Liao 13028744df refactor: do some internal refactor. 2023-06-15 10:24:21 +08:00
Haojun Liao 53377c2c1f fix(stream): wait for stream task completed. 2023-06-15 01:14:41 +08:00
Yihao Deng 7ded6fc7a2
Merge branch '3.0' into enh/triggerCheckPoint2 2023-06-14 16:30:38 +08:00
liuyao 9a3708e17b pause&resume fill history 2023-06-14 15:54:07 +08:00
Haojun Liao 7c6fbd7772 enh(stream): do some internal refactor and support secondary scan for history data. 2023-06-14 14:10:09 +08:00
Haojun Liao b44447e6f3 enh(stream): support restore from disk. 2023-06-14 10:19:03 +08:00
wangmm0220 8070223aba fix:lock error 2023-06-13 18:27:06 +08:00
Haojun Liao 97da2a8fac fix(stream): set the fill history status. 2023-06-13 16:10:55 +08:00
wangmm0220 8a81278024 fix:load pHandle if necessary wher tq restore 2023-06-13 15:14:11 +08:00
Haojun Liao fb24ed161d enh(stream): set correct task status. 2023-06-12 22:48:23 +08:00
yihaoDeng 83dde6a212 add trigger checkpoint 2023-06-12 14:33:08 +00:00
Haojun Liao 78a240be8f enh(stream): add dispatch msg. 2023-06-12 13:12:45 +08:00
wangmm0220 1903b815d4 fix:modify offset description to string 2023-06-12 09:45:06 +08:00
wangmm0220 6657b8d849 fix:modify offset description to string 2023-06-12 09:44:00 +08:00
yihaoDeng a2030dc378 add trigger checkpoint 2023-06-09 12:12:51 +00:00
Haojun Liao 69c9eda7af fix(stream): fix race condition. 2023-06-09 11:37:38 +08:00
wangmm0220 3f46a9bbc9 feat:add parameters for consumer & add offset rows for subscription 2023-06-08 18:14:02 +08:00
Haojun Liao 2b6ace653a refactor: do some internal refactor. 2023-06-08 16:14:27 +08:00
yihaoDeng 1f1f37312d add trigger checkpoint 2023-06-07 14:32:48 +00:00
Haojun Liao 06cf3588da refactor: do some internal refactor. 2023-06-07 15:03:04 +08:00
Haojun Liao 6a3c935b4a fix(stream): fix error in fill history scan. 2023-06-07 09:15:55 +08:00
Haojun Liao 0dd933013c enh(stream): make history task for stream running. 2023-06-05 17:58:00 +08:00
wangmm0220 f2ee657b62 fix:check wal not exist again before register to push & move push logic from write thread to query thread 2023-06-05 17:21:47 +08:00
Haojun Liao fcc706c45c enh(stream): refactor the fill history task. 2023-06-05 15:59:01 +08:00
Haojun Liao 077e1a1ed4 Merge branch '3.0' into refact/fillhistory 2023-06-05 11:41:13 +08:00
Haojun Liao 044e56fc97
Merge branch '3.0' into fix/resume_mem 2023-06-03 12:19:17 +08:00
Haojun Liao cca69f3038 Merge branch '3.0' into fix/resume_mem 2023-06-02 18:40:48 +08:00
wangmm0220 56f7a137ee fix:conflict 2023-06-02 15:47:53 +08:00
wangmm0220 b2f0d88639 fix:conflict 2023-06-02 15:43:35 +08:00
yihaoDeng f4bde33dc3 avoid mem leak 2023-06-02 06:06:05 +00:00
Haojun Liao cce87a036f other: merge 3.0. 2023-06-02 13:34:20 +08:00
Haojun Liao d45aa7ef84
Merge pull request #21555 from taosdata/fix/liaohj_main 
other: merge main into 3.0.
 2023-06-02 13:11:12 +08:00
Haojun Liao f1bd829977 refactor: do some internal refactor. 2023-06-02 13:10:54 +08:00
Haojun Liao 27b7d1ec88 refactor: do some internal refactor. 2023-06-02 11:17:44 +08:00
yihaoDeng 4d0ce625c4 Merge branch '3.0' of https://github.com/taosdata/TDengine into feature/TD-19042 2023-06-01 22:55:30 +08:00
wangmm0220 a7418d439b Merge branch '3.0' of https://github.com/taosdata/TDengine into feature/TD-19042 2023-06-01 22:50:51 +08:00
wangmm0220 56782a5d41 fix:open task case & modify mqRebVgReq encode/decode style 2023-06-01 18:29:16 +08:00
Haojun Liao 5d98a319c7
Merge pull request #21515 from taosdata/feature/TD-19042 
feat: support create topic as stable with conditions
 2023-06-01 09:56:55 +08:00
Haojun Liao 5301c5756a other: merge main into 3.0. 2023-06-01 09:29:18 +08:00
Haojun Liao 6d04c4e2fd refactor: do some internal refactor. 2023-05-31 19:05:40 +08:00
Haojun Liao 9e021087a9 refactor: do some internal refactor. 2023-05-31 19:04:53 +08:00
Haojun Liao a3abe25539 refactor: do some internal refactor. 2023-05-31 19:03:37 +08:00
Haojun Liao e237b4ac39 refactor: do some internal refactor. 2023-05-31 09:19:05 +08:00
wangmm0220 689ee05527 Merge branch '3.0' of https://github.com/taosdata/TDengine into feature/TD-19042 2023-05-30 15:45:46 +08:00
wangmm0220 41ed0138b4 fix:compile error 2023-05-29 17:25:11 +08:00
wangmm0220 07bdb95b1a fix:compile error 2023-05-29 17:10:11 +08:00
wangmm0220 5eb1c559e5 fix:save offset if latest 2023-05-29 15:51:00 +08:00
liuyao b541a0a053 set sched status 2023-05-29 15:42:47 +08:00
wangmm0220 78db3520a6 fix:conflict 2023-05-29 14:57:03 +08:00
wangmm0220 a71878d07f feat: support create topic as stable with conditions 2023-05-29 11:46:59 +08:00
Haojun Liao 1e786624b2
Merge pull request #21501 from taosdata/refact/fillhistory 
fix(stream): set the skip offset when resume stream task.
 2023-05-27 22:45:54 +08:00
Haojun Liao b38e253019 fix(stream): set the skip offset when resume stream task. 2023-05-26 22:51:58 +08:00
Haojun Liao 87957e2bb0
Merge pull request #21478 from taosdata/fix/TD-23423 
fix tsdb read error version
 2023-05-26 16:56:09 +08:00
Haojun Liao f61f37c936 fix(tmq): set the correct offset rsp when no poll occuring. 2023-05-26 14:20:01 +08:00
Haojun Liao 1e845acabe fix(tmq): allow seek before get assignment 2023-05-26 11:34:12 +08:00
Haojun Liao cea8334427
Merge pull request #21449 from taosdata/refact/fillhistory 
refactor: remove the module dependency.  TD-24279
 2023-05-26 09:01:45 +08:00
liuyao 75eb724c37 fix tsdb read error version 2023-05-26 08:55:23 +08:00
Haojun Liao da9f33a483 fix: fix error in set function ptr. 2023-05-25 09:56:34 +08:00
Haojun Liao 4a42f6b983 fix: fix error in api call. 2023-05-25 00:50:03 +08:00
wangmm0220 8537e4e80e feat:add tag filter for stable subscribe 2023-05-24 18:26:55 +08:00
Haojun Liao 1f7f326bed refactor: do some internal refactor and set the api function ptr. 2023-05-24 13:22:05 +08:00
liuyao 180abf2229 set agg task status 2023-05-24 09:34:40 +08:00
liuyao b5d01f9bab set agg pause status 2023-05-24 09:19:28 +08:00
Haojun Liao 39a1fa8f78 refactor: do some internal refactor. 2023-05-23 19:20:25 +08:00
wangmm0220 ed612088c0 fix:lost data because tsdbreader or task is killed 2023-05-23 10:32:13 +08:00
wangmm0220 d8387b95ff fix:set task status to 0 in inti 2023-05-22 20:14:43 +08:00
wangmm0220 591b51f624 fix:set task code success after kill 2023-05-22 12:42:21 +08:00
Haojun Liao 899e4d3350 refactor: do some internal refactor. 2023-05-20 18:13:58 +08:00
Haojun Liao de44c9160c refactor: do some internal refactor. 2023-05-20 18:05:39 +08:00
wangmm0220 7b735c67d0 fix:move sleep logic from rpc thread to tmq thread & fix some error 2023-05-20 16:36:25 +08:00
wangmm0220 8183be4aff fix:move sleep logic from rpc thread to tmq thread & fix some error 2023-05-20 16:13:52 +08:00
wangmm0220 04858fae22 fix:tsdbreader is free by mistake 2023-05-20 12:20:54 +08:00
wangmm0220 1ddbdad33c fix:set task status killed when vnode receive subscribe msg from mnode 2023-05-19 21:40:33 +08:00
wangmm0220 33d6df8717 fix:set task status killed when vnode receive subscribe msg from mnode 2023-05-19 19:11:49 +08:00
wangmm0220 c7c255e967 fix:set task status killed when vnode receive subscribe msg from mnode 2023-05-19 18:06:05 +08:00
wangmm0220 ff2b545b27 fix:set task status killed when vnode receive subscribe msg from mnode 2023-05-19 17:16:47 +08:00
wangmm0220 f7ab8dabf0 fix:[TD-23972] push subscribe msg to vnode even though consumer not change 2023-05-19 15:14:39 +08:00
wangmm0220 238254e49e fix:add task status check in doQueueScan & checkout consumer in tqProcessPollReq 2023-05-19 14:06:43 +08:00
Haojun Liao d8525123a2 enh(stream): add API to retrieve last ts for multi-tables. 2023-05-19 11:03:41 +08:00
Haojun Liao c4c5bbb65c other: merge 3.0 2023-05-18 17:28:40 +08:00
liuyao da3fc5111e fix pause agg task 2023-05-18 15:29:41 +08:00
Haojun Liao 81f821a6fc refactor: do some internal refactor. 2023-05-18 10:51:43 +08:00
wangmm0220 54f4860139 fix:rollback [TD-23972] push subscribe msg to vnode even though consumer not change 2023-05-17 19:44:56 +08:00
Haojun Liao b9bda9b30f fix(stream): set the correct start offset for resumed stream task. 2023-05-17 19:36:42 +08:00
wangmm0220 405cdfa7bc fix:error in pHandle lock 2023-05-17 19:02:16 +08:00
wangmm0220 1d9dd153ea fix:error in pHandle lock 2023-05-17 18:57:45 +08:00
Haojun Liao a4d9210b07 fix(tmq): add filter for taosX poll rsp. 2023-05-17 16:40:44 +08:00
wangmm0220 4bb25c280f fix:add log 2023-05-17 15:54:36 +08:00
Haojun Liao 041ffa91bd other: merge main. 2023-05-17 15:42:15 +08:00
Haojun Liao f55f8d245c fix(tmq): avoid the offset is revised by pool rsp which is issued before invoking tmq_offset_seek function. 2023-05-17 15:19:51 +08:00
Haojun Liao 366cd1871a
Merge pull request #21314 from taosdata/fix/liaohj_main 
fix(stream):extract delete msg from wal.
 2023-05-17 14:52:49 +08:00
Haojun Liao 3896e148fd log(tmq): add some logs. 2023-05-17 11:37:38 +08:00
Haojun Liao f3f52c0232 fix(stream): add some logs and set the wal offset to be the ended version when finishing the step2 recover stage. 2023-05-17 10:43:23 +08:00
wangmm0220 c52a4ea4f0 fix:merge from 3.0 2023-05-17 01:11:55 +08:00
wangmm0220 90099ea1c0 fix:compile error in windows 2023-05-16 18:41:35 +08:00
wangmm0220 e6860cada8 fix:compile error in windows 2023-05-16 18:35:10 +08:00
wangmm0220 a52939127f fix:dead lock 2023-05-16 17:58:31 +08:00
wangmm0220 e6e0ac7603 fix:alwalys in exec if consumer Id mismatch & check result for wal seek ver 2023-05-16 14:01:38 +08:00
Haojun Liao e35d145734 fix(stream): fix memory leak. 2023-05-16 13:59:13 +08:00
Haojun Liao 30e7cb5860 fix(tmq): avoid return delete msg for table subscription. 2023-05-16 10:42:02 +08:00
Haojun Liao 5e16db4e19 fix(stream):extract delete msg from wal. 2023-05-16 08:54:03 +08:00
wangmm0220 62520af2e5 fix:core dump 2023-05-16 00:32:21 +08:00
wangmm0220 36454c73ed fix:add lock to pHandle for safety 2023-05-15 18:38:14 +08:00
Haojun Liao 170c603995 other: merge 3.0 2023-05-13 20:16:55 +08:00
Haojun Liao 99148d6719 other: merge 3.0 2023-05-13 20:02:48 +08:00
Haojun Liao 236926b9ee other: merge enh/rocksdbstate 2023-05-13 19:49:28 +08:00
Haojun Liao 40ffe0cd83 fix(tmq): check handle status before close sub. 2023-05-13 16:45:21 +08:00
Haojun Liao b383651d8b fix(tmq): fix the invalid write and set the flag when closing taosx sub. 2023-05-13 16:17:26 +08:00
Haojun Liao 99422c87c3 fix(tq): set the offset to be the consume offset, instead of the committed offset. And update the table sim. 2023-05-12 16:14:56 +08:00
Haojun Liao 7ef16e1922 other: merge main. 2023-05-12 11:49:53 +08:00
liuyao 8b7b804606 opt partition by tbname 2023-05-11 18:47:27 +08:00
Haojun Liao 26bea33ccc fix(tmq): add null check for print log. 2023-05-11 17:56:19 +08:00
Haojun Liao 2b736ffd60 fix(stream): avoid duplicated task id in task id list. 2023-05-11 11:47:57 +08:00
Haojun Liao 8a1b0ed052 refactor: do some internal refactor. 2023-05-11 10:35:09 +08:00
yihaoDeng 5708fcb5cb Merge branch 'enh/3.0' into enh/rocksdbSstateMerge 2023-05-10 08:02:16 +00:00
Yihao Deng c6e5879f44
Merge branch 'main' into enh/rocksdbSstate 2023-05-10 10:04:28 +08:00
yihaoDeng 8356533eb1 Merge branch 'enh/standVer' into enh/rocksdbSstateMerge 2023-05-09 12:13:16 +00:00
dapan1121 1f973123c4
Merge pull request #21177 from taosdata/fix/TD-23972 
fix:[TD-23972] push subscribe msg to vnode even though consumer not c…
 2023-05-08 20:29:31 +08:00
Yihao Deng cb223acf82
Merge branch 'main' into enh/rocksdbSstate 2023-05-08 09:45:27 +08:00
Haojun Liao 9392c03b7f fix(stream): add some logs. 2023-05-06 19:05:55 +08:00
wangmm0220 279fe0803f fix:[TD-23972] push subscribe msg to vnode even though consumer not change 2023-05-06 18:35:24 +08:00
Haojun Liao da6212bab0 other: merge main. 2023-05-06 12:51:38 +08:00
wangmm0220 55eddbfb5e fix:[TD-23972] push subscribe msg to vnode even though consumer not change 2023-05-06 10:54:05 +08:00
Haojun Liao 1fa4327f6c Merge branch 'main' into fix/liaohj_main 2023-05-06 10:15:23 +08:00
Haojun Liao 29fcd1b83a refactor: do some internal refactor. 2023-05-06 10:08:05 +08:00
Haojun Liao 8958aabe4b fix(stream): opt task scan performance. 2023-05-05 19:33:30 +08:00
Haojun Liao 8b12d4d3da fix(stream): secure the delete task operation. TD-1198 2023-05-05 15:02:03 +08:00