xxq250
/
homework-jianmu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
61,108 Commits 3 Branches 0 Tags 479 MiB
Commit Graph

2210 Commits

Author SHA1 Message Date
yihaoDeng d6840ae07f fix transfer crash 2023-10-18 14:22:52 +08:00
yihaoDeng 1b755dd4e7 fix transfer crash 2023-10-18 14:22:33 +08:00
yihaoDeng e49b9b2276 fix transfer crash 2023-10-18 11:34:11 +08:00
Haojun Liao 2904cfd4d5 Merge branch 'main' into refact/streamsm 2023-10-18 10:29:09 +08:00
Haojun Liao 5115a1ac88
Merge pull request #23304 from taosdata/fix/TD-26771 
double free
 2023-10-18 10:09:17 +08:00
yihaoDeng d67fdb5e46 fix transfer crash 2023-10-17 21:08:31 +08:00
yihaoDeng 2004c1a346 fix transfer crash 2023-10-17 18:19:52 +08:00
Haojun Liao 64ed7e49a2 enh(stream): sm for stream tasks. 2023-10-17 13:47:55 +08:00
yihaoDeng 77542cc461 fix stream snap deadlock 2023-10-17 11:23:39 +08:00
Haojun Liao 7b7ce10475 refactor(stream): do some internal refactor. 2023-10-17 11:23:04 +08:00
Haojun Liao 14035c763c refactor: do some internal refactor. 2023-10-17 09:57:13 +08:00
Haojun Liao fdb6ec1fa3 enh(stream): add parameter to limit the stream sink task. 2023-10-16 22:52:39 +08:00
liuyao 990d36654d release buff 2023-10-16 19:49:45 +08:00
Haojun Liao a1aa2c9e0c fix(stream):add more check for test cases. 2023-10-16 18:47:01 +08:00
Haojun Liao 67bee17e5e other: merge main. 2023-10-16 09:55:41 +08:00
Haojun Liao f90cf54714 fix(stream): fix error in creating update node trans. 2023-10-15 21:56:49 +08:00
Haojun Liao 6611f216ac enh(stream): add some logs. 2023-10-13 19:45:29 +08:00
liuyao 7269dedb39 Merge branch '3.0' of https://github.com/taosdata/TDengine into 3.0 2023-10-13 11:35:01 +08:00
liuyao 429125be93 session state recover 2023-10-13 11:32:35 +08:00
Haojun Liao 33d9f772be other: merge 3.0 2023-10-13 09:06:13 +08:00
Haojun Liao 84c5359d5b other: merge 3.0 2023-10-13 08:58:26 +08:00
yihaoDeng d6ff282006 refactor backend 2023-10-12 20:42:36 +08:00
yihaoDeng 8c1a9f7c58 refactor backend 2023-10-12 20:28:40 +08:00
yihaoDeng 0ba7e2278a refactor backend 2023-10-12 20:21:11 +08:00
yihaoDeng a56d831a14 refactor backend 2023-10-12 20:16:41 +08:00
yihaoDeng 861d26e356 refactor backend 2023-10-12 15:24:50 +08:00
yihaoDeng f704d4b82e refactor backend 2023-10-12 11:45:18 +08:00
yihaoDeng bfd205b3f7 refactor backend 2023-10-12 11:31:39 +08:00
Haojun Liao eb4449999f other: merge 3.0 2023-10-12 10:33:48 +08:00
yihaoDeng 4c94fbb44f refactor backend 2023-10-11 20:26:19 +08:00
yihaoDeng f264ffdcd6 refactor backend 2023-10-11 20:12:07 +08:00
yihaoDeng 83b52a4c1d refactor backend 2023-10-11 20:02:40 +08:00
yihaoDeng 031506aef8 refactor backend 2023-10-11 19:46:35 +08:00
yihaoDeng 4f47162512 refactor backend 2023-10-11 16:36:43 +08:00
yihaoDeng 722d72659b refactor backend 2023-10-11 16:10:16 +08:00
yihaoDeng 0baef335d7 refactor backend 2023-10-11 16:03:04 +08:00
yihaoDeng 4cff121a38 refactor backend 2023-10-11 14:46:41 +08:00
liuyao e9fc079d26 opt bloom filter 2023-10-11 10:14:11 +08:00
Haojun Liao 1d4f0c8599
Merge pull request #23201 from taosdata/enh/TD-26646 
Enh/td 26646
 2023-10-10 22:59:37 +08:00
yihaoDeng 28421cd35e refactor backend 2023-10-10 20:51:41 +08:00
Haojun Liao 05dfadd8f8 fix(stream): set initial version for fill-history task. disable reentry of scan-history task function. 2023-10-10 15:52:48 +08:00
Haojun Liao 5f9b858fc1 other: merge main. 2023-10-10 15:38:46 +08:00
yihaoDeng b36163a197 add session iter 2023-10-10 15:12:21 +08:00
yihaoDeng 825341d90a add sess iter 2023-10-10 14:35:02 +08:00
Haojun Liao adf6f65d09 fix(stream): set initial version for fill-history task. disable reentry of scan-history task function. 2023-10-10 14:00:30 +08:00
Alex Duan 9da96ebf63
Merge pull request #23017 from taosdata/case/TD-26408-MAIN 
fix: add case for lost data after  split vgroup (main)
 2023-10-10 00:54:34 -05:00
Alex Duan e566027cef
Merge pull request #23019 from taosdata/case/TD-26408-3.0 
fix: add case for lost data after split 3.0
 2023-10-10 00:53:56 -05:00
yihaoDeng 6adf7532a8 add sess iter 2023-10-10 11:35:18 +08:00
yihaoDeng 7ac4ef5beb add sess iter 2023-10-10 10:41:46 +08:00
yihaoDeng 0fc6ffe6b6 change test case 2023-10-09 20:51:26 +08:00
Haojun Liao a25d94ab6f fix(stream): retry when error occurs during timer. 2023-10-09 20:10:28 +08:00
Haojun Liao 7d39164133 fix(stream): disable retrieving results during checkpoint procedure 2023-10-09 20:10:27 +08:00
Haojun Liao af4618f958 fix(stream): remove invalid clear of rspmsg list. 2023-10-09 20:10:27 +08:00
Haojun Liao 847baa8b71 fix(stream): fix the pause failure bug. 2023-10-09 20:10:27 +08:00
Haojun Liao 21fb502690 enh(stream): add trans to reset task status to avoid being frozen in checkpoint status due to doing checkpoint failure of partial tasks. 2023-10-09 20:10:26 +08:00
Haojun Liao 64bde93f85 fix(stream): not idle for sink task when no tokens are available. 2023-10-09 20:10:25 +08:00
Haojun Liao 9a7167aaba fix(stream): sleep for 10ms 2023-10-09 20:10:24 +08:00
Haojun Liao 9af8ce21ed fix(stream): wait for a while when no available token. 2023-10-09 20:10:23 +08:00
Haojun Liao 4780474877 fix(stream): continue 2023-10-09 20:10:23 +08:00
Haojun Liao da833ccb30 refactor: inc the dump batch. 2023-10-09 20:10:23 +08:00
Haojun Liao 2936ac9b89 fix(stream): add missing refactor. 2023-10-09 20:10:22 +08:00
Haojun Liao af2c0c95e9 refactor: do some internal refactor. 2023-10-09 20:10:22 +08:00
Haojun Liao 940d6a3cb4 fix(stream): check items in queue and qall, stead of only the queue items. 2023-10-09 20:10:21 +08:00
Haojun Liao e95aea84ae fix(stream): fix syntax error. 2023-10-09 20:10:21 +08:00
Haojun Liao 59e284c332 fix(stream): quit from loop when input queue is full. 2023-10-09 20:10:21 +08:00
Haojun Liao 9d5a3b8d78 other: merge stream fix. 2023-10-09 20:10:13 +08:00
Haojun Liao c5997bee5b fix(stream): update the current processing version. 2023-10-09 20:09:23 +08:00
Haojun Liao 28008f8263 enh(stream): add more info for sink task. 2023-10-09 20:09:22 +08:00
Haojun Liao b3b9e89d30 fix(stream): fix the invalid check. 2023-10-09 20:09:22 +08:00
Haojun Liao cf14772dd9 enh(stream): optimize the scan wal perf 2023-10-09 20:09:21 +08:00
Haojun Liao d2160efa05 refactor: wait for 100ms before scan wal again. 2023-10-09 20:09:21 +08:00
Haojun Liao 932d93ab35 refactor: wait for more time for future data 2023-10-09 20:09:21 +08:00
Haojun Liao 7682c033ad fix(stream): add null check. 2023-10-09 20:09:21 +08:00
Haojun Liao 0ab5d4b342 refactor: do some internal refactor. 2023-10-09 20:09:20 +08:00
Haojun Liao c295ec322a refactor: do some internal refactor. 2023-10-09 20:09:20 +08:00
Haojun Liao d5abb21335 fix(stream): fix invalid check. 2023-10-09 20:09:20 +08:00
Haojun Liao 583ac1fc45 refactor: do some internal refactor. 2023-10-09 20:09:20 +08:00
Haojun Liao 83f84d92bc refactor: do some internal refactor (discard the checkpoint source msg during restoring). 2023-10-09 20:09:10 +08:00
Haojun Liao bb4ba54f28 fix(stream): enable new time range for stream task. 2023-10-09 20:09:09 +08:00
Haojun Liao c759d90bf6 enh(stream): optimize the stream status when creating without fill-history option. 2023-10-09 20:09:09 +08:00
Haojun Liao db897fb03a fix(stream): opt perf. 2023-10-09 20:09:09 +08:00
Haojun Liao 0a2b70fd7f fix(stream): fix syntax error. 2023-10-09 20:09:06 +08:00
Haojun Liao 8477271240 fix(stream): fix dead lock. 2023-10-09 20:09:00 +08:00
Haojun Liao 12eabb0c2c log(stream): update logs. 2023-10-09 20:09:00 +08:00
Haojun Liao b716f0d1fd fix(stream): fix dead lock. 2023-10-09 20:08:57 +08:00
Haojun Liao 60a720efbb log: update logs. 2023-10-09 20:08:53 +08:00
Haojun Liao d10b07d23d refactor: do some internal refactor. 2023-10-09 20:08:53 +08:00
Haojun Liao 6d2dbd549b fix(stream): avoid repeatly free obj. 2023-10-09 20:08:53 +08:00
Haojun Liao 6b5770aab7 log(stream): update the log. 2023-10-09 20:08:52 +08:00
Haojun Liao dfe8641c0c refactor: optimize the fill-history task launch policy, and do some other internal refactor. 2023-10-09 20:08:52 +08:00
Haojun Liao 38164435f9 refactor: do some internal refactor. 2023-10-09 20:08:52 +08:00
Haojun Liao f2848de12a fix(stream): reduce the counter when remove stream tasks. 2023-10-09 20:08:52 +08:00
Haojun Liao ecb3b44026 fix(stream): limit sink quota. 2023-10-09 20:08:52 +08:00
Haojun Liao 2c0e54f4f7 fix(stream): update the retry interval. 2023-10-09 20:08:13 +08:00
Haojun Liao 7f8da725bc enh(stream): add quota limitation for sink task. 2023-10-09 20:08:13 +08:00
Haojun Liao a248f3d088 refactor: do some internal refactor. 2023-10-09 20:08:12 +08:00
Haojun Liao 0db357401d fix(stream): add new status for meta. 2023-10-09 20:08:12 +08:00
Haojun Liao 3912d80c02 fix(stream): do some internal refactor. 2023-10-09 20:08:11 +08:00
Haojun Liao feb09c8361 fix(stream): fix error in pause scan-history data. 2023-10-09 20:08:11 +08:00
Haojun Liao 6a98b11bef fix(stream): fix invalid read and memory leak. 2023-10-09 20:08:11 +08:00
Haojun Liao 65f66f0986 fix(stream): transfer state when all downstream rsp already. 2023-10-09 20:08:11 +08:00
Haojun Liao a32b56f381 fix(stream): fix memory leak. 2023-10-09 20:08:10 +08:00
Haojun Liao 4af7812f0b fix(stream): fix the invalid counter for ready tasks in vnode, fix an invalid read. 2023-10-09 20:08:10 +08:00
Haojun Liao 4fe1382cd9 fix(stream): adjust nodeUpdate check interval. 2023-10-09 20:08:09 +08:00
Haojun Liao f4f55fcbce fix(stream): record the tasks started info, and do some internal refactor. 2023-10-09 20:08:09 +08:00
Haojun Liao 68b003c804 fix(stream): fix race condition. 2023-10-09 20:08:09 +08:00
Haojun Liao fc96ec6bae fix(stream): do not wait for the checkpoint finish before stop tasks. 2023-10-09 20:08:09 +08:00
Haojun Liao 7167c3c5de fix(stream): add new timer for each task. 2023-10-09 20:08:08 +08:00
Haojun Liao 7336228c16 refactor: do some internal refactor. 2023-10-09 20:08:08 +08:00
Haojun Liao 477f57aa0e refactor: do some internal refactor. 2023-10-09 20:08:08 +08:00
Haojun Liao 52763ca2a3 refactor: do some internal refactor. 2023-10-09 20:08:07 +08:00
Haojun Liao 3baddbe67a fix(stream): update the checkpoint record start time. 2023-10-09 20:08:07 +08:00
Haojun Liao a6ffb845c8 fix(stream): fix invalid checkpoint start time. 2023-10-09 20:08:07 +08:00
Haojun Liao c05a8de3e8 fix(stream): rsp when checkpoint source failed. 2023-10-09 20:08:06 +08:00
Haojun Liao b4bc4fac98 fix(stream): disable auto rsp for checkpoint source. 2023-10-09 20:08:06 +08:00
Haojun Liao e671e87234 fix(stream): fix invalid assert. 2023-10-09 20:08:06 +08:00
Haojun Liao d1a6b8bd22 refactor: do some internal refactor. 2023-10-09 20:08:06 +08:00
Haojun Liao 46b17aa9f1 fix(stream): fix race condition. 2023-10-09 20:08:06 +08:00
Haojun Liao e49409a11e fix(stream): add some logs. 2023-10-09 20:08:05 +08:00
Haojun Liao 5eee76df65 fix(stream): add msg identifier check. 2023-10-09 20:08:05 +08:00
Haojun Liao 192c47a7c2 fix(stream): add some logs. 2023-10-09 20:08:05 +08:00
Haojun Liao 693254f602 fix(stream): remove invalid free. 2023-10-09 20:08:04 +08:00
Haojun Liao f3bd5c411d fix(stream): remove the blocking flag after successfully retry sending msg. 2023-10-09 20:08:04 +08:00
Haojun Liao ed9e035095 fix(stream): clear the failure vnode id list before send msg. 2023-10-09 20:08:04 +08:00
Haojun Liao 3726aa723e fix(stream): fix invalid read. 2023-10-09 20:08:04 +08:00
Haojun Liao e0d19af000 fix(stream): refactor re-try dispatch msg for stream tasks. 2023-10-09 20:08:03 +08:00
Haojun Liao 2b16976902 refactor(stream): refactor send msg. 2023-10-09 20:08:03 +08:00
Haojun Liao 27e00b1f19 fix(stream): add some logs, and remove invalid assert. 2023-10-09 20:08:02 +08:00
Haojun Liao 12687eb287 fix(stream): add check info detailed information. 2023-10-09 20:08:02 +08:00
Haojun Liao ab7c18f604 refactor: do some internal refactor. 2023-10-09 20:08:01 +08:00
Haojun Liao e5535fa1f5 fix(stream): handle dispatch checkpoint-trigger failure. 2023-10-09 20:08:01 +08:00
Haojun Liao 65cc4040f3 fix(stream): update logs. 2023-10-09 20:08:00 +08:00
Haojun Liao 6d3a134077 fix(stream): ignore the dispatch failure, and set the correct rsp counter. 2023-10-09 20:07:54 +08:00
Haojun Liao 7f4d9b6e41 fix(stream): wait for the task checkpoint before stop. 2023-10-09 20:07:53 +08:00
Haojun Liao ef9efc8a8e refactor: do refactor the sink task. 2023-10-09 20:07:53 +08:00
Haojun Liao e4ac835554 fix(stream): disable token bucket. 2023-10-09 20:07:53 +08:00
Haojun Liao a3d13d6634 fix(stream): remove invalid cached task info in buffer. 2023-10-09 20:07:08 +08:00
Haojun Liao 1f45bd82d2 other: merge stream fix in main. 2023-10-09 20:07:01 +08:00
Haojun Liao d752fb1ed6 fix(stream): exec task in task queuee threads. 2023-10-09 20:06:11 +08:00
Haojun Liao 1ebafe4a1f fix(stream): add more check. 2023-10-09 20:06:11 +08:00
Haojun Liao c403a15f36 fix(stream): check the status before exec stream task when handling the dispatch requests. 2023-10-09 20:06:11 +08:00
Haojun Liao d76e3ac718 fix(stream): add task status check. 2023-10-09 20:06:10 +08:00
Haojun Liao e27111ecd1 enh(stream): add node stage check. 2023-10-09 20:06:09 +08:00
Haojun Liao ce85945583 fix(stream): set correct size for results generated by scan history stream tasks. 2023-10-09 20:06:05 +08:00
Haojun Liao 053643be19 fix(stream): add timestamp. 2023-10-09 20:06:05 +08:00
Haojun Liao b38f9e0288 fix(stream): update logs. 2023-10-09 20:06:05 +08:00
Haojun Liao 5c0b8ea804 enh(stream): log the checkpoint time. 2023-10-09 20:06:04 +08:00
Haojun Liao b95ad74c7f fix(stream): not handle the check msg for follower tasks. 2023-10-09 20:06:04 +08:00
Haojun Liao e6fb9ee46c fix(stream): reset task counter. 2023-10-09 20:06:04 +08:00
Haojun Liao c3a3cf21fb fix(stream): add some logs. 2023-10-09 20:05:39 +08:00
Haojun Liao 08b37dfc13 fix(stream): fix memory leak. 2023-10-09 20:05:38 +08:00
Haojun Liao 734f648891 fix(stream): set the output normal for stream dispatch when encountering the transfer state. 2023-10-09 20:05:38 +08:00
Haojun Liao 636eccc160 fix(stream): add null check 2023-10-09 20:05:37 +08:00
Haojun Liao 57bf1cadc4 fix(stream): fix error in decode stream task. 2023-10-09 20:05:37 +08:00
Haojun Liao 9d6199adf7 fix(stream): fix invalid read. 2023-10-09 20:05:37 +08:00
Haojun Liao 0459a4628c fix(stream): fix stream task id error. 2023-10-09 20:05:36 +08:00
Haojun Liao 7ccc273206 other: merge stream fix in main. 2023-10-09 20:05:29 +08:00
Haojun Liao 98ef566eb4 fix(stream): remove fill-history sink task. 2023-10-09 20:04:17 +08:00
Haojun Liao e84eeee6b8 refactor: do some internal refactor. 2023-10-09 20:04:17 +08:00
Haojun Liao fe7864b976 fix(stream): fix error in drop task. 2023-10-09 20:04:17 +08:00
Haojun Liao c33e521aa1 fix(stream): update some logs. 2023-10-09 20:04:17 +08:00
Haojun Liao 39e0c57323 fix(stream): set the correct guard flag. 2023-10-09 20:04:16 +08:00
Haojun Liao 3d1f29ff21 fix(stream): fix memory leak. 2023-10-09 20:04:16 +08:00
yihaoDeng c6e9134cea add sess iter 2023-10-09 20:02:23 +08:00
Haojun Liao 6fa54789cb fix(stream): disable follower send hb to mnode. and do some internal refactor. 2023-10-09 19:59:31 +08:00
Haojun Liao 1256eafddb other: merge stream fix. 2023-10-09 19:59:24 +08:00
Haojun Liao 283118bf68 fix(stream):fix missing unlock. 2023-10-09 19:56:17 +08:00
Haojun Liao aaa12ed4f8 refactor: do some internal refactor about the sched status set 2023-10-09 19:56:17 +08:00
Haojun Liao 01d104706e refactor(stream): remove unnecessary streamschedTask function to improve the performance. 2023-10-09 19:56:17 +08:00
Haojun Liao a678e722ce fix(stream): limit the token bucket capacity 2023-10-09 19:56:17 +08:00
Haojun Liao b0ef091430 fix(stream): retry when error occurs during timer. 2023-10-09 16:40:05 +08:00
Haojun Liao be0a2e4573 fix(stream): disable retrieving results during checkpoint procedure 2023-10-09 16:33:28 +08:00
yihaoDeng 2aa88dfe9b fix mem leak 2023-10-09 15:53:42 +08:00
Haojun Liao 8266c4ff77 fix(stream): remove invalid clear of rspmsg list. 2023-10-09 15:26:42 +08:00
Haojun Liao d87db2f829 fix(stream): fix the pause failure bug. 2023-10-09 14:48:11 +08:00
yihaoDeng 28b546a4ad fix mem leak 2023-10-09 14:25:57 +08:00
yihaoDeng be42143362 Merge branch '3.0' into case/TD-26408-3.0 2023-10-09 14:24:31 +08:00
yihaoDeng 3f9784f475 fix mem leak 2023-10-09 14:07:22 +08:00
yihaoDeng 0b92a6b1ea fix mem leak 2023-10-09 11:16:16 +08:00
Haojun Liao 74ff0c5b50 Merge branch 'main' into fix/liaohj 2023-10-09 10:14:53 +08:00
Haojun Liao ee6516f3c4 enh(stream): add trans to reset task status to avoid being frozen in checkpoint status due to doing checkpoint failure of partial tasks. 2023-10-09 10:13:53 +08:00
Haojun Liao 4b8cd3bccd fix(stream): not idle for sink task when no tokens are available. 2023-10-09 10:07:19 +08:00
yihaoDeng f49a2cbeb0 refact task backend 2023-10-08 18:14:17 +08:00
liuyao 1cbb8ef444 init bloomfilter array 2023-10-08 14:43:44 +08:00
yihaoDeng b5994b31e2 fix mem leak 2023-10-08 13:33:02 +08:00
yihaoDeng e3f18495e0 fix mem leak 2023-10-08 11:22:26 +08:00
yihaoDeng 2fa91341a7 refact task backend 2023-10-08 10:35:49 +08:00
Haojun Liao 23fb26798f fix(stream): sleep for 10ms 2023-10-07 19:47:49 +08:00
Haojun Liao e4e4c5a36e fix(stream): wait for a while when no available token. 2023-10-07 19:41:49 +08:00
Haojun Liao 45bb3f50d7 fix(stream): continue 2023-10-07 19:25:40 +08:00
liuyao 95ffe6fa23 stream recover 2023-10-07 19:24:21 +08:00
Haojun Liao 62cdcbc7e3 refactor: inc the dump batch. 2023-10-07 19:20:48 +08:00
Haojun Liao 4d6f143146 fix(stream): add missing refactor. 2023-10-07 19:18:26 +08:00
Haojun Liao ab17bfdf09 refactor: do some internal refactor. 2023-10-07 19:16:55 +08:00
Haojun Liao 0e8a8aabc1 fix(stream): check items in queue and qall, stead of only the queue items. 2023-10-07 19:16:25 +08:00
liuyao 1e323c0408 fix issue 2023-10-07 19:11:46 +08:00
yihaoDeng 98099ebbae fix mem leak 2023-10-07 18:41:27 +08:00
liuyao 19bbf10579 fix issue 2023-10-07 17:59:53 +08:00
yihaoDeng 97fdcbc60b fix mem leak 2023-10-07 17:48:24 +08:00
liuyao 1aaf873134 fix issue 2023-10-07 17:43:50 +08:00
yihaoDeng bdc4afec83 refact task backend 2023-10-07 17:17:30 +08:00
yihaoDeng e41da13d5b refact task backend 2023-10-07 17:07:26 +08:00
yihaoDeng 3ebc7eef5f refact task backend 2023-10-07 17:03:12 +08:00
yihaoDeng c146327da8 refact task backend 2023-10-07 16:48:36 +08:00
Haojun Liao d11c0a113c fix(stream): fix syntax error. 2023-10-07 14:56:09 +08:00
Haojun Liao 7ce3d2c200 fix(stream): quit from loop when input queue is full. 2023-10-07 14:54:11 +08:00
yihaoDeng 66315fe405 refact task backend 2023-10-07 13:53:44 +08:00
yihaoDeng 82e0242889 refact task backend 2023-10-07 13:43:52 +08:00
yihaoDeng fa70221238 refact task backend 2023-10-07 12:17:55 +08:00
liuyao 460ee86fdf fix issue 2023-10-07 11:55:40 +08:00
liuyao 59d47e1e1f fix issuse 2023-10-07 10:34:35 +08:00
Haojun Liao 115a284ac0 enh(stream): update the checkpoint interval 2023-10-07 09:56:03 +08:00
Haojun Liao 011627d50a fix(stream): update the current processing version. 2023-10-07 09:07:44 +08:00
Haojun Liao c20dd002fb enh(stream): add more info for sink task. 2023-10-07 01:14:00 +08:00
Haojun Liao 98659ad323 fix(stream): fix the invalid check. 2023-10-05 00:09:47 +08:00
Haojun Liao 41ef1a45c1 enh(stream): optimize the scan wal perf 2023-10-04 22:06:46 +08:00
Haojun Liao 45e413f8d5 refactor: wait for 100ms before scan wal again. 2023-10-03 20:57:19 +08:00
Haojun Liao 132fb8e82e refactor: wait for more time for future data 2023-10-03 15:42:00 +08:00
Haojun Liao 38aff641ee fix(stream): add null check. 2023-10-03 02:39:57 +08:00
Haojun Liao 58f410a9df refactor: do some internal refactor. 2023-10-03 02:37:52 +08:00
Haojun Liao a543f4ca97 refactor: do some internal refactor. 2023-10-03 01:52:03 +08:00
Haojun Liao 5564eb215c fix(stream): fix invalid check. 2023-10-03 00:38:24 +08:00
Haojun Liao 5718158fc0 refactor: do some internal refactor. 2023-10-01 22:27:29 +08:00
yihaoDeng b8e265e502 add cvt state 2023-09-30 09:05:28 +00:00
Haojun Liao 31acbc6fe2 refactor: do some internal refactor (discard the checkpoint source msg during restoring). 2023-09-28 22:44:13 +08:00
yihaoDeng eac16b9d1b add cvt state 2023-09-28 11:59:53 +00:00
yihaoDeng 4f66dc538f add cvt state 2023-09-28 11:52:18 +00:00
Haojun Liao f2d8431619 fix(stream): enable new time range for stream task. 2023-09-28 18:26:52 +08:00
yihaoDeng c5bb37a5cb refact task backend 2023-09-28 17:30:49 +08:00
yihaoDeng bb265887e6 refact task backend 2023-09-28 17:07:19 +08:00
yihaoDeng 54e3ac2c1e refact task backend 2023-09-28 15:46:12 +08:00
Haojun Liao 80ec77cbb6 enh(stream): optimize the stream status when creating without fill-history option. 2023-09-28 11:23:03 +08:00
Haojun Liao 22e485d132 fix(stream): opt perf. 2023-09-28 09:25:23 +08:00
Haojun Liao f525778aef fix(stream): fix syntax error. 2023-09-28 00:53:36 +08:00
Haojun Liao 8c998673ad fix(stream): fix dead lock. 2023-09-27 17:15:49 +08:00
Haojun Liao 2dade996bb log(stream): update logs. 2023-09-27 15:37:12 +08:00
Haojun Liao 8f2d217834 log: update logs. 2023-09-27 14:48:53 +08:00
yihaoDeng d4f87378fd avoid first tag index conflict 2023-09-27 14:44:09 +08:00
Haojun Liao 11641e1115 refactor: do some internal refactor. 2023-09-27 14:18:01 +08:00
Haojun Liao 34d547bfff fix(stream): avoid repeatly free obj. 2023-09-27 14:05:14 +08:00
Haojun Liao ae020fcd2d log(stream): update the log. 2023-09-27 14:03:57 +08:00
Haojun Liao e13efd5b04 refactor: optimize the fill-history task launch policy, and do some other internal refactor. 2023-09-27 14:00:48 +08:00
Haojun Liao a7e6933ce7 refactor: do some internal refactor. 2023-09-27 09:46:15 +08:00
Haojun Liao 37bf387453 fix(stream): reduce the counter when remove stream tasks. 2023-09-27 09:36:20 +08:00
Haojun Liao 9ab9de0ad3 fix(stream): limit sink quota. 2023-09-27 00:30:38 +08:00
Haojun Liao 27dcbcb96f fix(stream): update the retry interval. 2023-09-26 23:42:02 +08:00
Haojun Liao aec15ae39f enh(stream): add quota limitation for sink task. 2023-09-26 23:29:54 +08:00
liuyao c95dc167b7 add ci 2023-09-26 17:19:21 +08:00
Haojun Liao 890d658aea refactor: do some internal refactor. 2023-09-26 14:11:45 +08:00
Haojun Liao f3f18b0fa3 fix(stream): add new status for meta. 2023-09-26 12:15:12 +08:00
liuyao 82e18e32b0 add ci 2023-09-26 10:41:12 +08:00
Haojun Liao d214a2f6bd fix(stream): do some internal refactor. 2023-09-26 09:32:57 +08:00
Haojun Liao 6cc3618ed9 fix(stream): fix error in pause scan-history data. 2023-09-26 01:09:55 +08:00
Haojun Liao 5a4a0aa0e3 fix(stream): fix invalid read and memory leak. 2023-09-25 23:10:33 +08:00
yihaoDeng ed172a939c rename cfg default name 2023-09-25 18:09:35 +08:00
Haojun Liao c9576d7a85 fix(stream): transfer state when all downstream rsp already. 2023-09-25 17:31:38 +08:00
Haojun Liao c615c513e8 fix(stream): fix memory leak. 2023-09-25 16:01:17 +08:00
Haojun Liao fcf4cc7169 fix(stream): fix the invalid counter for ready tasks in vnode, fix an invalid read. 2023-09-25 15:58:24 +08:00
Haojun Liao 1af95969b2 fix(stream): adjust nodeUpdate check interval. 2023-09-25 10:40:30 +08:00
Haojun Liao eb4078b7f3 fix(stream): record the tasks started info, and do some internal refactor. 2023-09-25 09:59:05 +08:00
Haojun Liao c987f61300 fix(stream): fix race condition. 2023-09-25 01:37:27 +08:00
Haojun Liao da82a5f19e fix(stream): do not wait for the checkpoint finish before stop tasks. 2023-09-24 22:03:07 +08:00
Haojun Liao a74b462ffb fix(stream): add new timer for each task. 2023-09-24 02:25:53 +08:00
Haojun Liao b7dcd94c64 refactor: do some internal refactor. 2023-09-24 01:56:52 +08:00
Haojun Liao 3a75c21e66 refactor: do some internal refactor. 2023-09-24 01:55:59 +08:00
Haojun Liao 4ea737571d refactor: do some internal refactor. 2023-09-24 01:26:51 +08:00
Haojun Liao 7f265e181d fix(stream): update the checkpoint record start time. 2023-09-23 18:05:14 +08:00
Haojun Liao 6ef68fdf06 fix(stream): fix invalid checkpoint start time. 2023-09-23 17:57:09 +08:00
Haojun Liao acd3321494 fix(stream): rsp when checkpoint source failed. 2023-09-23 14:13:33 +08:00
Haojun Liao 7d5f76c6f1 fix(stream): disable auto rsp for checkpoint source. 2023-09-23 11:31:14 +08:00
Haojun Liao 5155519559 fix(stream): fix invalid assert. 2023-09-22 23:06:48 +08:00
Haojun Liao cf6c255f22 refactor: do some internal refactor. 2023-09-22 22:50:21 +08:00
yihaoDeng 8890f5c950 add cvt state 2023-09-22 12:51:29 +00:00
Haojun Liao fcbd747afc fix(stream): fix race condition. 2023-09-22 19:35:12 +08:00
liuyao 5cc1ae1266 fix issue 2023-09-22 19:25:19 +08:00
Haojun Liao 1ca05fb3af fix(stream): add some logs. 2023-09-22 18:56:31 +08:00
Haojun Liao 5b12e276cf fix(stream): add msg identifier check. 2023-09-22 18:13:18 +08:00
Haojun Liao bb1a2322ca fix(stream): add some logs. 2023-09-22 17:27:10 +08:00
yihaoDeng 11da631bb5 add cvt state 2023-09-22 09:08:42 +00:00
Haojun Liao 07ec6f38b8 fix(stream): remove invalid free. 2023-09-22 16:53:22 +08:00
Haojun Liao 5a695f632c fix(stream): remove the blocking flag after successfully retry sending msg. 2023-09-22 16:44:55 +08:00
yihaoDeng 970b1cb840 add cvt state 2023-09-22 08:38:41 +00:00
yihaoDeng 173604ee1a add cvt state 2023-09-22 08:32:14 +00:00
yihaoDeng 19e52c364e add cvt state 2023-09-22 08:26:03 +00:00
Haojun Liao 6c357104a5 fix(stream): clear the failure vnode id list before send msg. 2023-09-22 16:19:25 +08:00
Haojun Liao 0f5f21ba09 fix(stream): fix invalid read. 2023-09-22 16:05:12 +08:00
Haojun Liao 619a9f6d08 fix(stream): refactor re-try dispatch msg for stream tasks. 2023-09-22 15:57:55 +08:00
liuyao 4f8e6d7795 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-26174 2023-09-22 15:00:52 +08:00
liuyao 410a177637 fix issue 2023-09-22 15:00:26 +08:00
Haojun Liao 898aea5cfb refactor(stream): refactor send msg. 2023-09-22 14:52:18 +08:00
Haojun Liao 87ea745861 fix(stream): add some logs, and remove invalid assert. 2023-09-22 10:53:03 +08:00
Haojun Liao f7bcff862d fix(stream): add check info detailed information. 2023-09-22 10:23:56 +08:00
wangmm0220 5555371fbc fix:memory leak 2023-09-22 10:04:53 +08:00
liuyao 4979fc5283 fix issue 2023-09-22 09:40:08 +08:00
wangmm0220 803b17a2db fix:memory leak 2023-09-22 09:31:58 +08:00
Haojun Liao b1ba716299 refactor: do some internal refactor. 2023-09-22 09:24:36 +08:00
Haojun Liao b075b4438b fix(stream): handle dispatch checkpoint-trigger failure. 2023-09-21 23:15:18 +08:00
Haojun Liao 50d5fb6641 fix(stream): update logs. 2023-09-21 19:14:34 +08:00
yihaoDeng 470849982e add cvt func 2023-09-21 09:41:30 +00:00
Haojun Liao 4925b67d4a fix(stream): ignore the dispatch failure, and set the correct rsp counter. 2023-09-21 17:22:09 +08:00
Haojun Liao 2efd155adf fix(stream): wait for the task checkpoint before stop. 2023-09-21 15:53:21 +08:00
Haojun Liao 17586af0ce refactor: do refactor the sink task. 2023-09-21 13:32:57 +08:00
yihaoDeng 77ef632233 refactor code 2023-09-21 01:36:12 +00:00
Haojun Liao 9f9b712332 fix(stream): disable token bucket. 2023-09-21 01:19:58 +08:00
yihaoDeng ee8c09667e refactor backend 2023-09-20 12:53:58 +00:00
Haojun Liao 92f18989df fix(stream): remove invalid cached task info in buffer. 2023-09-20 19:19:04 +08:00
Haojun Liao 6633fb971b fix(stream): update logs. 2023-09-20 18:18:31 +08:00
liuyao b4be5fe895 fix mem leak 2023-09-20 17:35:11 +08:00
Haojun Liao 619b377dd9 fix(stream): exec task in task queuee threads. 2023-09-20 17:30:41 +08:00
Haojun Liao 49165f8c3c fix(stream): add more check. 2023-09-20 16:49:55 +08:00
Haojun Liao 8329ec79f2 fix(stream): check the status before exec stream task when handling the dispatch requests. 2023-09-20 16:32:27 +08:00
Haojun Liao 2a781c509f fix(stream): add task status check. 2023-09-20 15:32:15 +08:00
liuyao 1cebda368a session&state operator buff 2023-09-20 15:06:08 +08:00
Haojun Liao b4419bda65 enh(stream): add node stage check. 2023-09-20 10:35:01 +08:00
Haojun Liao 1decaaee1e fix(stream): set correct size for results generated by scan history stream tasks. 2023-09-19 16:49:04 +08:00
Haojun Liao 309630eb11 fix(stream): add timestamp. 2023-09-19 14:22:05 +08:00
Haojun Liao 38b3a7c1bd fix(stream): update logs. 2023-09-19 11:05:49 +08:00
Haojun Liao b7efedf4aa enh(stream): log the checkpoint time. 2023-09-19 09:53:33 +08:00
Haojun Liao e76bab7122 fix(stream): not handle the check msg for follower tasks. 2023-09-19 09:43:14 +08:00
Haojun Liao 9a9fffa577 fix(stream): reset task counter. 2023-09-19 00:52:35 +08:00
Haojun Liao 377fdfacf6 fix(stream): add some logs. 2023-09-18 23:51:08 +08:00
Haojun Liao 2c9d54af37 fix(stream): fix memory leak. 2023-09-18 19:58:46 +08:00
Haojun Liao c7b8d572f6 fix(stream): set the output normal for stream dispatch when encountering the transfer state. 2023-09-18 18:25:22 +08:00
Haojun Liao c33ef4ce88 fix(stream): add null check 2023-09-18 15:14:51 +08:00
Haojun Liao 8aa9960760
Merge pull request #22944 from taosdata/fix/liaohj 
refactor: do some internal refactor, and fix some bugs.
 2023-09-17 21:25:33 +08:00
Haojun Liao 3dfdda3283 fix(stream): fix error in decode stream task. 2023-09-17 18:07:26 +08:00
Haojun Liao 301258784a fix(stream): fix invalid read. 2023-09-17 13:59:06 +08:00
Haojun Liao 01144c58a7 fix(stream): fix stream task id error. 2023-09-17 01:19:59 +08:00
Haojun Liao 4846194bbb refactor: do some internal refactor, remove the related fill-history task by sending mesg to replay it in follower node. 2023-09-16 21:37:30 +08:00
Haojun Liao caa7542cea
Merge pull request #22913 from taosdata/fix/liaohj 
fix(stream): fix error in drop task.
 2023-09-16 21:36:38 +08:00
Haojun Liao 91e3d70796 fix(stream): remove fill-history sink task. 2023-09-15 17:46:17 +08:00
yihaoDeng e958e18726 fix race crash and add log 2023-09-15 15:02:05 +08:00
yihaoDeng 6820fb0c5d fix race crash and add log 2023-09-15 14:59:44 +08:00
Haojun Liao 2bfd6e3355 refactor: do some internal refactor. 2023-09-15 09:25:22 +08:00
Haojun Liao 117de7ab7d fix(stream): fix error in drop task. 2023-09-15 00:51:13 +08:00
Haojun Liao 500fcad36c
Merge pull request #22892 from taosdata/fix/liaohj 
refactor: do some internal refactor.
 2023-09-15 00:46:46 +08:00
Haojun Liao 73c2782d78 fix(stream): update some logs. 2023-09-14 23:17:23 +08:00
Haojun Liao 303e8caf0c fix(stream): set the correct guard flag. 2023-09-14 22:44:16 +08:00
Haojun Liao 27f626769e fix(stream): fix memory leak. 2023-09-14 19:33:05 +08:00
Haojun Liao f2c3e0a6e8 fix(stream): disable follower send hb to mnode. and do some internal refactor. 2023-09-14 17:22:31 +08:00
Haojun Liao fce5be0eb4 fix(stream): fix dead lock. 2023-09-14 14:09:05 +08:00
Haojun Liao d9f6cf3966 fix(stream):fix missing unlock. 2023-09-13 14:11:41 +08:00
Haojun Liao 7305c8dbe1 refactor: do some internal refactor about the sched status set 2023-09-13 14:00:34 +08:00
Haojun Liao ca2fea3e1b refactor(stream): remove unnecessary streamschedTask function to improve the performance. 2023-09-13 11:02:38 +08:00
Haojun Liao 0943c1c5b1 fix(stream): limit the token bucket capacity 2023-09-13 09:19:59 +08:00
Haojun Liao 99477babca
Merge pull request #22872 from taosdata/fix/coverityScanDyh 
fix coverity scan problem
 2023-09-13 09:18:27 +08:00
Haojun Liao 09a15bf340 refactor: do some internal refactor. 2023-09-12 22:07:52 +08:00
Haojun Liao 60f5056ab7 other: merge 3.0 2023-09-12 22:00:40 +08:00
Haojun Liao f6a097d96f refactor: do some internal refactor. 2023-09-12 21:58:22 +08:00
Haojun Liao 4325a0a433
Merge pull request #22866 from taosdata/fix/3_liaohj 
refactor: do  some internal refactor.
 2023-09-12 21:56:44 +08:00
yihaoDeng 8fc33dd74d fix coverity scan problem 2023-09-12 18:52:16 +08:00
Haojun Liao 13c20facd0 fix(stream): fix syntax error. 2023-09-12 18:37:30 +08:00
Haojun Liao 66676b977e fix(stream): set correct number items in outputQ 2023-09-12 18:16:07 +08:00
Haojun Liao e457d94ac6 refactor: do some internal refactor. 2023-09-12 18:09:02 +08:00
Haojun Liao 4d33387e80
Merge pull request #22814 from taosdata/fix/fixTransferCrash2 
fix merge error
 2023-09-12 16:10:07 +08:00
Haojun Liao 78a5680cf4 fix(stream): add some logs. 2023-09-12 15:59:09 +08:00
yihaoDeng 3e48f3639c fix transfer crash 2023-09-12 14:46:32 +08:00
yihaoDeng 6ce85f5cae fix transfer crash 2023-09-12 10:44:56 +08:00
Haojun Liao f602aa965f fix(stream): fix coverity scan issues. 2023-09-12 09:16:18 +08:00
Haojun Liao 45880f70d4 fix(stream): fix display error in logs. 2023-09-11 13:57:33 +08:00
Haojun Liao 3b79ab3266 fix(stream): reset the init ts. 2023-09-11 13:50:55 +08:00
Haojun Liao 50ef500f51 refactor(stream): do sink module refactor. 2023-09-11 10:09:31 +08:00
Haojun Liao a9321248a9 fix(stream): set the correct flag when there is one token existed. 2023-09-10 18:34:03 +08:00
Haojun Liao 4ffb4a5b1f refactor: disable some logs. 2023-09-10 11:02:32 +08:00
Haojun Liao 85a5c45098 fix(stream): limit the already processed data. 2023-09-10 00:01:25 +08:00
Haojun Liao 6f5c855a4b fix(stream): fix msg lost bug. 2023-09-09 15:08:36 +08:00
Haojun Liao 47fd144447 other: add some logs. 2023-09-09 11:19:51 +08:00
Haojun Liao e4918e85eb fix(stream): limit the sink rate 2023-09-09 00:22:22 +08:00
Haojun Liao bf79c2c993 fix(stream): limit the sink rate 2023-09-09 00:11:13 +08:00
Haojun Liao 35b97cbd19 fix(stream): set correct token in bucket. 2023-09-08 23:50:41 +08:00
Haojun Liao 328377c25f refactor: add some logs. 2023-09-08 23:04:39 +08:00
Haojun Liao b317c8ebff fix(stream): disable merge submit blocks. 2023-09-08 12:46:15 +08:00
Haojun Liao dc821057e5 fix(stream): update the log. 2023-09-07 10:33:38 +08:00
Haojun Liao e407d7dc6b fix(stream): update log. 2023-09-07 10:09:05 +08:00
Haojun Liao 956047a441 fix(stream): count down the token in bucket. 2023-09-07 10:02:13 +08:00
Haojun Liao b240e91c91 enh(stream): add token bucket to limit the rate of sink operation. 2023-09-07 09:58:48 +08:00
liuyao d7caed1c44 opt stream state 2023-09-06 16:34:25 +08:00
Haojun Liao 8783ebcb02 enh(stream): add some logs and record the submit package performance. 2023-09-06 14:34:36 +08:00
Haojun Liao c86eeb3938 fix(stream): add task status check when waiting for the table create. 2023-09-06 10:58:33 +08:00
Haojun Liao 0441c7a870 Merge branch 'fix/3_liaohj' of github.com:taosdata/tdengine into fix/3_liaohj 2023-09-05 10:40:48 +08:00
Haojun Liao 5b3cb8ec42 fix(stream): fix memory leak. 2023-09-05 09:27:16 +08:00
Haojun Liao 4f85a2620a fix(stream): remove invalid free. 2023-09-05 09:27:16 +08:00
Haojun Liao 77eed06bcd
Merge branch '3.0' into fix/3_liaohj 2023-09-04 18:51:07 +08:00
Haojun Liao 5869750782 fix(stream): release stream task with max_delay option. 2023-09-04 18:33:33 +08:00
Haojun Liao 325a177170 fix(stream): disable the fill-history task when restart stream tasks. 2023-09-04 17:18:04 +08:00
Haojun Liao aeea699c9c Merge branch '3.0' into fix/3_liaohj 2023-09-04 15:53:39 +08:00
Haojun Liao 375d0abea5 fix(stream): merge main. 2023-09-04 13:12:53 +08:00
Haojun Liao 21dab8ceab other: merge 3.0 2023-09-04 13:08:19 +08:00
Haojun Liao d8518de5e0 fix(stream):fix error in timerActive. 2023-09-01 20:20:39 +08:00
Minglei Jin 5159bf3e2a fix(stream): fix epset errors 2023-09-01 15:33:23 +08:00
dapan1121 cc17848b78 fix: merge 3.0 to main 2023-09-01 13:24:47 +08:00
Haojun Liao bce46f1be8 fix(stream): transfer state should wait for the checkpoint generating completed. 2023-09-01 09:50:45 +08:00
Haojun Liao 9612704fa6 fix(stream): add assert 2023-08-31 21:43:44 +08:00
Haojun Liao 95da66e3e8 fix(stream): fix error. 2023-08-31 20:50:12 +08:00
Haojun Liao bef0d4bff3 fix(stream): add assert. 2023-08-31 20:01:59 +08:00
Haojun Liao e816d0efa5
Merge pull request #22669 from taosdata/fix/TD-26069 
fix invalid insert
 2023-08-31 18:29:25 +08:00
yihaoDeng 1833be028f fix invalid insert 2023-08-31 15:28:31 +08:00
Haojun Liao e2bb64eb18 refactor: do some internal refactor. 2023-08-31 14:23:46 +08:00
Haojun Liao 69f95f9324 other: merge 3.0 2023-08-31 13:56:34 +08:00
Haojun Liao 2285b834b0 refactor: do some internal refactor. 2023-08-31 13:53:12 +08:00
Haojun Liao 3fc3dafca6 refactor: do some internal refactor. 2023-08-31 13:46:48 +08:00
Haojun Liao 3ed78ddadd
Merge pull request #22662 from taosdata/fix/3_liaohj 
refactor: do some internal refactor.
 2023-08-31 13:46:14 +08:00
Haojun Liao 8645f763b5
Merge pull request #22661 from taosdata/enh/reserve 
stream change ver
 2023-08-31 13:35:23 +08:00
Haojun Liao ecff0acb73 fix(stream): fix a syntax error. 2023-08-31 11:48:25 +08:00
yihaoDeng 83a5e2be4c stream change ver 2023-08-31 11:42:12 +08:00
Haojun Liao e53b5d4392 refactor: do some internal refactor. 2023-08-31 11:01:53 +08:00
yihaoDeng 05ca71d5de stream change ver 2023-08-31 09:28:43 +08:00
Haojun Liao 65eaa9b15f fix(stream): add task release. 2023-08-30 19:06:12 +08:00
Haojun Liao d99d8875e1 fix(stream): if stream task is running, do not recheck its status. 2023-08-30 17:24:13 +08:00
yihaoDeng 57608e7fff fix merge error 2023-08-30 17:12:37 +08:00
Haojun Liao 997d247aa6 fix(stream): remove invalid memset. 2023-08-30 16:58:22 +08:00
Haojun Liao d26e5373b9 fix(stream): fix syntax error. 2023-08-30 16:02:46 +08:00
Haojun Liao c5b279f045 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-08-30 16:01:08 +08:00
Haojun Liao 24b1a050fc other: merge 3.0 2023-08-30 16:00:14 +08:00
liuyao fc3468fe39 add log 2023-08-30 15:58:51 +08:00
liuyao 0a72e576bd op stream pause 
pause data dispatch

opt stream pause
 2023-08-30 14:53:49 +08:00
Haojun Liao 646b339bde fix(stream): not enable the pause before receive all finish rsp 2023-08-30 14:28:45 +08:00
dapan1121 f6321d0c4f Merge remote-tracking branch 'origin/main' into fix/m23.1 2023-08-30 11:03:33 +08:00
Haojun Liao 3a7a220d43 fix(stream): fix error found by ci. 2023-08-30 10:14:01 +08:00
Haojun Liao 204fb97f68 fix(stream): fix error in snode, fix memory leaks, fix repeatly create bug in sink task. 2023-08-29 17:34:31 +08:00
Haojun Liao c3e9590740 fix(stream): fix error. 2023-08-29 15:11:01 +08:00
yihaoDeng 8ed2836a2b refactor code 2023-08-29 09:52:03 +08:00
yihaoDeng a2901411b0 refactor code 2023-08-28 18:52:10 +08:00
yihaoDeng 1687f68156 refactor code 2023-08-28 16:13:27 +08:00
Haojun Liao 391103bc5a fix(stream): do dome internal refactor. 2023-08-28 15:54:01 +08:00
dapan1121 b00aa1a992
Merge pull request #22458 from taosdata/fix/TD-25719 
fix: fix non-root users cannot create log files even if they have write
 2023-08-28 13:28:59 +08:00
Haojun Liao b75ae217f6 fix(stream): notify close for snode. 2023-08-28 09:53:08 +08:00
Haojun Liao 88a59eee37 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-08-27 00:18:53 +08:00
Haojun Liao 46529d383c fix(stream): build checkpoint for sink tasks. 2023-08-27 00:18:41 +08:00
yihaoDeng 998dac0ffb fix mem leak 2023-08-26 21:54:36 +08:00
yihaoDeng 7d237d4310 fix UAF 2023-08-26 17:37:09 +08:00
yihaoDeng bed5765013 fix UAF 2023-08-26 16:15:26 +08:00
yihaoDeng 0348ef6e22 fix UAF 2023-08-26 16:13:17 +08:00
Haojun Liao b7c36bbfe4 fix(stream): fix the status error. 2023-08-26 00:49:23 +08:00
yihaoDeng 22b8d90fbc fix invalid read 2023-08-25 18:34:11 +08:00
Haojun Liao ef3e2737f4 fix(stream): fix syntax error. 2023-08-25 17:39:31 +08:00
yihaoDeng a02b35ed86 fix compile error 2023-08-25 09:02:56 +00:00
Haojun Liao 60d1df0e25
Merge branch '3.0' into enh/triggerCheckPoint2 2023-08-25 15:49:28 +08:00
Haojun Liao 35193020dd fix(stream): add logs. 2023-08-25 10:18:06 +08:00
Haojun Liao ce1a3a379c fix(stream): handle the case when the task not in ready state, and the checkpoint msg arrived. 2023-08-25 10:12:12 +08:00
Ganlin Zhao af130cc92f Revert "add seperate function for createLogs" 
This reverts commit 13d2d72bb6.
 2023-08-25 09:55:50 +08:00
Haojun Liao f7ea875af1 fix(stream): add some logs. 2023-08-24 23:27:48 +08:00
Haojun Liao f7dada53b9
Merge pull request #22559 from taosdata/fix/liaohj 
other: merge main.
 2023-08-24 19:01:18 +08:00
Haojun Liao 218e2cc5f3 fix(stream): invoke the timer when not sending hb. 2023-08-24 18:27:09 +08:00
Haojun Liao 82e356a9b5 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 
# Conflicts:
#	source/dnode/vnode/src/tq/tq.c
 2023-08-24 18:04:44 +08:00
Haojun Liao 70396aa5a5 fix(stream): wakeup the hb timer every 200ms, to speeup the close db procedure. 2023-08-24 17:54:50 +08:00
yihaoDeng 8883d66227 Merge branch 'enh/triggerCheckPoint2' of https://github.com/taosdata/TDengine into enh/triggerCheckPoint2 2023-08-24 17:20:12 +08:00
yihaoDeng 090515c684 refactor code 2023-08-24 17:20:04 +08:00
Ganlin Zhao 13d2d72bb6 add seperate function for createLogs 2023-08-24 14:56:14 +08:00
Haojun Liao a750eb9699 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-08-24 11:43:32 +08:00
Haojun Liao 957c7f872b fix(stream): update the agg task status check, when transferring state. 2023-08-24 11:43:18 +08:00
liuyao 1e108c4178 mem leak 2023-08-24 11:34:43 +08:00
liuyao 7ebce2814f mem leak 2023-08-24 11:19:59 +08:00
yihaoDeng 3afea998ac refactor log 2023-08-24 10:15:15 +08:00
Ganlin Zhao 962a0cd403 fix 2023-08-23 23:34:02 +08:00
yihaoDeng 0959758bd5 fix transfer error 2023-08-23 21:43:10 +08:00
Haojun Liao 8e07703dd2 fix(stream): reset the timerActive value. 2023-08-23 18:26:07 +08:00
Haojun Liao 809d3087ff Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 
# Conflicts:
#	source/dnode/mnode/impl/src/mndStream.c
 2023-08-23 17:53:50 +08:00
Haojun Liao 230d8c2d80 fix(stream): fix error in stream computing after vnode is transferred to other dnodes. 2023-08-23 17:50:44 +08:00
yihaoDeng 1ffc13d391 fix mem leak 2023-08-23 17:02:12 +08:00
Haojun Liao f8a3cdb987 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-08-23 14:10:56 +08:00
Haojun Liao b0c5fa8290 fix(stream): reset the task status. 2023-08-23 14:10:41 +08:00
yihaoDeng ab407c58b6 add log 2023-08-23 10:54:50 +08:00
yihaoDeng 12b10dfc76 fix mem leak 2023-08-23 09:08:37 +08:00
yihaoDeng 5fe7563a13 fix mem leak 2023-08-22 21:13:02 +08:00
yihaoDeng ba5c3648db fix mem leak 2023-08-22 21:12:54 +08:00
Haojun Liao 57ef43ba18 fix(stream): set sink task status to enable the checkpoint. 2023-08-22 18:16:46 +08:00
yihaoDeng ab9490101f Merge branch 'enh/3.0' into enh/triggerCheckPoint2 2023-08-22 18:04:04 +08:00
Haojun Liao af80d94a88 refactor: do some internal refactor. 2023-08-22 15:43:50 +08:00
Haojun Liao e6b49b45d9 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-08-22 13:37:50 +08:00
Haojun Liao cc8b2263cb fix(stream): fix error in heartbeat from vnode. 2023-08-22 13:37:38 +08:00
yihaoDeng 2c1ca6d325 Merge branch 'enh/3.0' into enh/triggerCheckPoint2 2023-08-22 10:51:39 +08:00
yihaoDeng d106b5a87e Merge branch 'enh/triggerCheckPoint2' of https://github.com/taosdata/TDengine into enh/triggerCheckPoint2 2023-08-22 10:48:30 +08:00
yihaoDeng a0fd56c84f fix invalid cmp 2023-08-22 10:48:15 +08:00
Haojun Liao 64556c9f1d Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-08-22 10:47:02 +08:00
Haojun Liao def0877980 fix(stream): add timer to check stream task status. 2023-08-22 10:46:49 +08:00
yihaoDeng 0304ad88ff fix invalid cmp 2023-08-22 09:49:04 +08:00
yihaoDeng fdd62e53d8 fix invalid cmp 2023-08-22 09:11:56 +08:00
yihaoDeng 2268114c4a fix invalid cmp 2023-08-22 09:10:48 +08:00
yihaoDeng 55f1d464ab fix invalid cmp 2023-08-22 09:06:42 +08:00
yihaoDeng 3e944d228b refactor checkpoint 2023-08-21 21:48:06 +08:00
Haojun Liao 082882b71e fix(stream): reset the upstream stage info. 2023-08-21 19:29:03 +08:00
Haojun Liao 8b27f98ee4 refactor: do some internal refactor. 2023-08-21 19:08:27 +08:00
yihaoDeng 387b4d365e refactor checkpoint 2023-08-21 19:07:15 +08:00
yihaoDeng bf3c401a68 refactor checkpoint 2023-08-21 12:52:00 +08:00
Haojun Liao 99024e9d78 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-08-21 11:49:08 +08:00
Haojun Liao 29bb5854cc fix(stream): fix error in check stream state. 2023-08-21 11:48:55 +08:00
yihaoDeng 9ef69dd455 refactor checkpoint 2023-08-21 10:27:47 +08:00
yihaoDeng 1c5346b13c refactor checkpoint 2023-08-19 16:10:03 +08:00
yihaoDeng 52788aca1c fix checkpoint error 2023-08-19 12:25:22 +08:00
yihaoDeng fbdea723b0 fix checkpoint error 2023-08-19 11:58:12 +08:00
Haojun Liao 718bb4a881 fix(stream): fix the invalid free. 2023-08-18 22:40:04 +08:00
yihaoDeng 8e4411c144 fix checkpoint error 2023-08-18 21:42:35 +08:00
Haojun Liao 4b10393c81 fix(stream): fix memory leak and do some internal refactor. 2023-08-18 18:02:08 +08:00
yihaoDeng 3a3385bb84 fix mem leak 2023-08-18 17:01:57 +08:00
yihaoDeng 0537323bac fix get error 2023-08-18 16:54:33 +08:00
Haojun Liao c26d625152
Merge branch '3.0' into enh/triggerCheckPoint2 2023-08-18 15:51:07 +08:00
liuyao 9afd5155ed Merge branch 'enh/triggerCheckPoint2' of https://github.com/taosdata/TDengine into enh/triggerCheckPoint2 2023-08-18 15:33:37 +08:00
liuyao 87c65f3b68 Merge remote-tracking branch 'origin/3.0' into enh/triggerCheckPoint2 2023-08-18 15:27:02 +08:00
Haojun Liao 10872240ae
Merge pull request #22476 from taosdata/fix/3_liaohj 
fix(stream): pause when outputQ is blocked.
 2023-08-18 15:12:54 +08:00
Haojun Liao 3ec28ca5e1 Merge branch 'enh/triggerCheckPoint2' of github.com:taosdata/tdengine into enh/triggerCheckPoint2 2023-08-18 15:11:19 +08:00
Haojun Liao edbad45772 other: add logs. 2023-08-18 13:45:08 +08:00
yihaoDeng ac10f0f9b4 add trace log 2023-08-18 12:06:41 +08:00
Haojun Liao a8b53c2e69 other: add logs. 2023-08-18 10:51:59 +08:00
Haojun Liao 2a9668da67 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 
# Conflicts:
#	source/dnode/mnode/impl/src/mndStream.c
 2023-08-18 10:30:30 +08:00
Haojun Liao 87785b1d9c refactor: add some logs. 2023-08-18 10:30:00 +08:00
yihaoDeng 002d5979c1 add trace log 2023-08-17 21:13:03 +08:00
yihaoDeng ebd4cac36b Merge branch 'enh/triggerCheckPoint2' of https://github.com/taosdata/TDengine into enh/triggerCheckPoint2 2023-08-17 18:58:18 +08:00
yihaoDeng 46574c385f fix mem leak 2023-08-17 18:52:33 +08:00
Haojun Liao a18989df88 fix(stream): add lost codes caused by merge 3.0. 2023-08-17 18:51:39 +08:00
Haojun Liao 9e9d83d4a1 refactor: do some internal refactor 2023-08-17 11:57:04 +08:00
Haojun Liao bb4f8515bd fix(stream): pause when outputQ is blocked. 2023-08-17 11:53:17 +08:00
Haojun Liao 58c0a3d8ef fix(stream): fix syntax error. 2023-08-17 09:13:37 +08:00
Haojun Liao eb76b6f175 fix(stream): fix syntax error. 2023-08-17 09:11:32 +08:00
liuyao 5ce7688a93 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/ly_refactor 2023-08-17 09:06:46 +08:00
Haojun Liao 6746339609 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 
# Conflicts:
#	source/libs/stream/src/streamSnapshot.c
 2023-08-17 00:08:44 +08:00
Haojun Liao ea6574b3c7 other: merge 3.0 2023-08-17 00:07:55 +08:00
yihaoDeng 114df7c349 fix transfer error 2023-08-16 11:37:15 +00:00
Haojun Liao 59d5858b95 fix(stream): fix a typo 2023-08-16 19:22:45 +08:00
Haojun Liao 9a76ae72d6 fix(stream): add the back pressure for sink node. 2023-08-16 19:11:43 +08:00
Haojun Liao ec84d7fd81 fix(stream): add the back pressure for sink node. 2023-08-16 19:00:35 +08:00
yihaoDeng ab7e2b3122 Merge branch 'enh/chkpTransfer' into enh/triggerCheckPoint2 2023-08-16 09:00:30 +00:00
Haojun Liao d63758a1c5 fix(stream): fix error in execute task on a tranferred node. 2023-08-16 15:24:24 +08:00
yihaoDeng 993ae84398 close backend 2023-08-16 07:17:15 +00:00
yihaoDeng 7815e44655 fix invalid free 2023-08-16 09:48:11 +08:00
Haojun Liao ad9f3cbc43 other: merge main. 2023-08-16 09:14:10 +08:00
Haojun Liao 7628be236d other: merge main. 2023-08-16 09:03:54 +08:00
yihaoDeng 7fd918962f fix invalid free 2023-08-16 00:53:10 +00:00
Haojun Liao 9551fcf7bb fix(tsdb): check the 2023-08-15 20:57:12 +08:00
Haojun Liao 13640f5f53 other 2023-08-15 20:46:06 +08:00
Haojun Liao 6d7b6fbb74 other: add some logs. 2023-08-15 20:01:53 +08:00
liuyao 19dfcf3a59 refactor stream scan log 2023-08-15 16:29:48 +08:00
Haojun Liao 39079ab64a fix(stream): load term from sync model to decide if the node is restart or not 2023-08-15 15:52:06 +08:00
Haojun Liao 71c3c710e6 fix(stream): continue process when met with trans-state msg. 2023-08-15 14:23:20 +08:00
Haojun Liao 0acb00c516 fix(stream): add log. 2023-08-15 11:48:56 +08:00
Haojun Liao c9f3e0b0e1 fix(stream): fix an syntax error. 2023-08-15 09:34:19 +08:00
Haojun Liao fe5e48b87d fix(stream): remove redundant include. 2023-08-15 09:26:35 +08:00
Haojun Liao b1d35f43ba enh(stream): using term from sync module to denote the upstream node is restarted/replica-modified/vnode-transferred or not. 2023-08-15 09:25:05 +08:00
Haojun Liao a334547726 fix(stream): fix an syntax error. 2023-08-14 19:28:36 +08:00
Haojun Liao 1a08ffc79f fix(stream): check error. 2023-08-14 19:27:09 +08:00
Haojun Liao 5d3232d275 fix(stream): fix an syntax error. 2023-08-14 18:14:32 +08:00
Haojun Liao 2e6263b43f fix(stream): fix memory leak. 2023-08-14 17:16:11 +08:00
Haojun Liao a1e554fbf3 refactor: exec directly not asynchnoized. 2023-08-14 15:05:13 +08:00
Haojun Liao 91710b0c0f fix(stream): 2023-08-14 14:56:17 +08:00
Haojun Liao e8c9a019a4 fix(stream): remove the invalid set of scheduler status. 2023-08-14 11:41:24 +08:00
Haojun Liao 2806fe1c56 fix(stream): ignore the related stream task destory msg in transfer state. 2023-08-14 10:58:24 +08:00
Haojun Liao 98f40325e9 fix(stream): fix memory leak. 2023-08-13 20:02:15 +08:00
Haojun Liao a81cc9aebf fix(stream): fix the error when no agg tasks exist. 2023-08-13 17:31:09 +08:00
Haojun Liao 0d38f389ab refactor: do some internal refactor. 2023-08-13 16:25:35 +08:00
Haojun Liao 0e189f70a1 refactor 2023-08-12 19:44:44 +08:00
Haojun Liao 03c26a9d6d refactor 2023-08-12 19:16:20 +08:00
Haojun Liao afe9b848a5 fix(tsdb): check the 2023-08-12 18:54:39 +08:00
Haojun Liao 09da6c6840 refactor: do some internal refactor. 2023-08-12 18:51:20 +08:00
Haojun Liao 75e6fb0f16 refactor: do some internal refactor. 2023-08-12 18:47:01 +08:00
Haojun Liao 075b5e9481 refactor: do some internal refactor. 2023-08-12 18:41:54 +08:00
Haojun Liao a0534ee715 refactor: do some internal refactor. 2023-08-12 18:40:09 +08:00
Haojun Liao 8f9de93cf0 refactor: do some internal refactor. 2023-08-12 18:37:36 +08:00
Haojun Liao 30c2a9c619 refactor: do some internal refactor. 2023-08-12 18:27:51 +08:00
Haojun Liao 0d0e30b764 fix(stream): fix the error. 2023-08-12 17:20:10 +08:00
Haojun Liao ce721a0146 fix(stream): fix error. 2023-08-12 16:32:05 +08:00
Haojun Liao a89ce1a20b fix(stream): transfer state by using data block. 2023-08-12 01:24:07 +08:00
Haojun Liao f24b22000f enh(stream): stop stream task before redistribute vnodes. 2023-08-11 23:46:53 +08:00
Haojun Liao 87c7e4c0fe fix(stream): not launch stream tasks when restarting the stream tasks. 2023-08-11 16:19:46 +08:00
yihaoDeng 6e6eb05d5c Merge branch 'enh/triggerCheckPoint2' into enh/chkpTransfer 2023-08-11 08:03:02 +00:00
Haojun Liao e984affe90 fix(stream): disable the stream in the followers nodes. 2023-08-11 15:58:15 +08:00
Haojun Liao 4afe603c6e fix(stream): fix error. 2023-08-11 14:17:09 +08:00
yihaoDeng 99be2836b2 Merge branch 'enh/triggerCheckPoint2' into enh/chkpTransfer 2023-08-11 02:50:12 +00:00
Haojun Liao 1e636a2ef9 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-08-11 10:37:40 +08:00
Haojun Liao 6a9826cf6e enh(stream): update the stream task epset. 2023-08-11 10:37:25 +08:00
yihaoDeng ba4ce9db02 Merge branch 'enh/triggerCheckPoint2' into enh/chkpTransfer 2023-08-11 02:15:07 +00:00
yihaoDeng 32983a11e1 add log and fix crash 2023-08-10 20:34:00 +08:00
Haojun Liao 8890fa5781 fix(stream): execute the stream task directly, instead of executing it in a asynchronized way. 2023-08-10 15:13:49 +08:00
yihaoDeng b2fb94d7f8 Merge branch 'enh/chkpTransfer' of https://github.com/taosdata/TDengine into enh/chkpTransfer 2023-08-10 14:57:25 +08:00
yihaoDeng ea00f676fc Merge branch 'enh/chkpTransfer' of https://github.com/taosdata/TDengine into enh/chkpTransfer 2023-08-10 14:57:12 +08:00
yihaoDeng 100d2240c3 support reopen stream state 2023-08-10 06:35:25 +00:00
Haojun Liao a713b37f35 fix(stream): add more logs. 2023-08-10 14:07:29 +08:00
Haojun Liao f633c7fe2b other: merge 3.0. 2023-08-10 11:56:30 +08:00
Haojun Liao b658608541
Merge pull request #22388 from taosdata/fix/3_liaohj 
other: merge main.
 2023-08-10 11:43:02 +08:00
yihaoDeng ef247cdb1d support reopen stream state 2023-08-10 01:49:26 +00:00
Haojun Liao c10b323188 other: merge main. 2023-08-10 09:26:33 +08:00
Haojun Liao 4393375e47 fix(stream): set correct task id. 2023-08-10 02:16:04 +08:00
Haojun Liao 0cf8144952 fix(stream): fix the invalid key used by remove data in stream meta hash table. 2023-08-09 22:28:23 +08:00
yihaoDeng a20b299f9b support reopen stream state 2023-08-09 13:06:10 +00:00
yihaoDeng 19ac9054d0 support reopen stream state 2023-08-09 12:47:49 +00:00
Haojun Liao d34b9af054 enh(stream): create update stream trans. 2023-08-09 19:38:13 +08:00
yihaoDeng 1db15da4e1 support reopen stream state 2023-08-09 09:50:49 +00:00
yihaoDeng 6fdcd82a33 support reopen stream state 2023-08-09 09:19:03 +00:00
yihaoDeng f3cf907d5a support reopen stream state 2023-08-09 09:07:13 +00:00
yihaoDeng a336a7b1a4 support reopen stream state 2023-08-09 09:06:22 +00:00
yihaoDeng a619b8f5be rm duplicate para 2023-08-09 08:58:49 +00:00
Haojun Liao 1e8579e8c5 fix(stream): fix other cases. 2023-08-09 16:04:48 +08:00
Haojun Liao 4c92997328 fix(stream): use streamId&taskId to identify the stream task in the stream meta. 2023-08-09 15:52:23 +08:00
yihaoDeng 92b247aae8 support reopen stream state 2023-08-09 07:36:00 +00:00
yihaoDeng bbcfa9ab03 support reopen stream state 2023-08-09 06:10:40 +00:00
Haojun Liao 3cdb16352c fix(stream): avoid launching check downstream when failed to add stream task into stream meta. 2023-08-09 12:03:14 +08:00
Haojun Liao f3c5f20ee2 enh(stream): add timer to check the node changing. 2023-08-09 10:25:16 +08:00
yihaoDeng c43a6b272c support reopen stream state 2023-08-09 02:09:50 +00:00
yihaoDeng ac11537fc8 support reopen stream state 2023-08-08 07:21:47 +00:00
Haojun Liao 17c6143471 fix(stream): check more status when handling the state transfer. 2023-08-08 15:08:11 +08:00
Haojun Liao a0dcec5890 enh(stream): add function handler for hb message. 2023-08-08 11:23:19 +08:00