Commit Graph

684 Commits

Author SHA1 Message Date
wangmm0220 ce2a3e4be2 fix:test case error 2024-01-26 15:35:29 +08:00
wangmm0220 bd2bef2428 fix:add test case 2024-01-26 15:05:31 +08:00
Haojun Liao 649d26ce72 fix(stream): seek to right place to start wal read. 2024-01-26 14:14:04 +08:00
Haojun Liao 49aff4571b fix(stream): fix memory leak. 2024-01-26 11:33:07 +08:00
Haojun Liao 9b3b03caab refactor: do some internal refactor. 2024-01-26 10:22:16 +08:00
Haojun Liao bc7fe1fe6a refactor: do some internal refactor. 2024-01-26 10:08:20 +08:00
Haojun Liao bd1d1cddaf refactor: do some internal refactor. 2024-01-26 10:03:54 +08:00
Haojun Liao d4bab8c09b refactor: do some internal refactor. 2024-01-25 16:53:30 +08:00
wangmm0220 2798ff824c feat:[TD-28247]add grant for subscribe and stream 2024-01-25 16:07:09 +08:00
Haojun Liao dc1ea9f9a1 fix(stream): clear htask info when unregistering the task. 2024-01-24 16:23:47 +08:00
Haojun Liao 8a7e38ad15 fix(stream): fix checkpoint failure check. 2024-01-24 13:10:53 +08:00
Haojun Liao 0cd84aa587 fix(stream): gen checkpoint for single task. 2024-01-23 17:51:50 +08:00
kailixu 5208829f23 Merge branch '3.0' into feat/TD-27463-3.0 2024-01-22 13:20:00 +08:00
Haojun Liao db474626e6 fix(stream): fix memory leak. 2024-01-22 11:49:16 +08:00
Haojun Liao 2cc584ff44 fix(stream): fix error in check vgId. 2024-01-22 11:05:42 +08:00
Haojun Liao 4a5ab10b3d fix(stream): fix memory leak. 2024-01-19 20:28:43 +08:00
Haojun Liao d973f66cea fix(stream): do checkpoint after fill-history task completed. 2024-01-19 16:18:02 +08:00
wangmm0220 7246fedfef Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/ly_multi_marks 2024-01-18 19:08:35 +08:00
Haojun Liao 996e2939a9 refactor: do some internal refactor. 2024-01-18 15:58:46 +08:00
Haojun Liao 76e6c453e7 enh(stream): send msg to mnode when the transferring state completed. 2024-01-18 15:43:36 +08:00
Haojun Liao f8d0c52483 fix(stream): remove this unused drop history task msg and corresponding functions. 2024-01-18 14:32:48 +08:00
kailixu bf0205b54d Merge branch '3.0' into feat/TD-27463-3.0 2024-01-17 11:29:47 +08:00
Haojun Liao 3d98fc6a67 other: merge 3.0 2024-01-16 18:29:33 +08:00
Haojun Liao 0280f129bd fix(stream):add some logs. 2024-01-16 18:21:29 +08:00
Haojun Liao 882b916940 fix(stream): adjust stream trans conflict check. 2024-01-16 16:11:04 +08:00
Haojun Liao 8ede6d87ce refactor: do some internal refactor. 2024-01-16 15:40:40 +08:00
Yihao Deng 7a09ba2630 fix pause/resume error 2024-01-15 11:56:38 +00:00
Haojun Liao 1296700092
Merge pull request #24433 from taosdata/fix/TD-28188
fix:[TD-28188]the epset of snode is error because epset is not initil…
2024-01-15 17:05:11 +08:00
wangmm0220 2ab54a1335 fix:conflicts from 3.0 2024-01-15 16:54:52 +08:00
kailixu 414e15a810 chore: merge 3.0 2024-01-15 10:42:45 +08:00
kailixu ddefc5f46a chore: merge 3.0 2024-01-11 20:05:23 +08:00
wangmm0220 89ca3418af fix:[TD-28188]the epset of snode is error because epset is not initilized 2024-01-11 17:38:15 +08:00
Haojun Liao 9703018f56 fix(stream): add checkpoint status check. 2024-01-11 16:47:51 +08:00
Haojun Liao 5f7ce21530 refactor: do some internal refactor. 2024-01-11 11:59:29 +08:00
Haojun Liao 58694c67dd refactor: do some internal refactor. 2024-01-11 11:50:52 +08:00
Haojun Liao 3fea717373 fix(stream): discard the orphan tasks. 2024-01-03 19:28:49 +08:00
Haojun Liao b06ccc0ca7 fix(stream): cleanup the checkpoint trans & task info in execInfo 2024-01-03 19:18:06 +08:00
Haojun Liao 185fa75376 enh(stream): add task into timer to be idle during exec when the corresponding output queue is full. 2024-01-03 17:59:42 +08:00
Haojun Liao 284ef7d085 fix(stream): remove the related checkpoint trans when trying to drop stream. 2024-01-03 14:44:04 +08:00
Haojun Liao e7a0044502 Merge branch '3.0' into fix/3_liaohj 2024-01-02 13:02:55 +08:00
Haojun Liao 7abeb5a27a fix(stream): update the stream error code. 2023-12-29 17:44:10 +08:00
Haojun Liao 62cfcefbb0 refactor: remove stream-scan-history event for stream task. 2023-12-26 00:34:04 +08:00
Haojun Liao b175a4b7b3 refactor(stream): async stop tasks, and do some internal refactor. 2023-12-22 17:56:16 +08:00
Haojun Liao 0fab9a1827 other: fix syntax error. 2023-12-22 00:29:49 +08:00
kailixu 3005ab1953 chore: more code 2023-12-21 19:45:09 +08:00
Haojun Liao 84583580e2 fix(stream): transfer the checkpoint trans id. 2023-12-21 19:12:49 +08:00
Haojun Liao ad5d78a6ca fix(stream): fix deadlock. 2023-12-20 19:12:47 +08:00
Haojun Liao 64eee4bbd2 fix(stream): add conflict check for nodeUpdate 2023-12-20 19:06:32 +08:00
wangmm0220 dd0afef6f2 fix:set triggerParams=0 in mid interval 2023-12-20 11:00:25 +08:00
Haojun Liao 92045f5485 fix(stream): add snode check, and handle the task status reset from checkpoint when it is in the snode. 2023-12-19 18:34:30 +08:00
wangmm0220 78c8ebae7f fix:remove useless code in split logic plan & random choose vnode in stream task & add test case 2023-12-19 16:35:04 +08:00
Haojun Liao bba5a4f08b other: merge main into 3.0 2023-12-18 18:26:42 +08:00
WANG MINGMING 6f23b63d9e
Merge branch '3.0' into feat/ly_multi_marks 2023-12-18 16:22:56 +08:00
kailixu 9fa90fc953 Merge branch '3.0' into feat/TD-27463-3.0 2023-12-18 14:20:10 +08:00
Haojun Liao edc3841a2c fix(stream): avoid invalid read. 2023-12-18 09:26:56 +08:00
Haojun Liao f9a5485d52 fix(stream): refactor the stream trans conflict detection strategy. 2023-12-16 01:46:29 +08:00
Haojun Liao aedc5e6b05 fix(stream): set the correct dst stable name. 2023-12-15 17:16:25 +08:00
Haojun Liao 0449b5f1bd other: add some logs. 2023-12-14 15:43:59 +08:00
kailixu d7e16bb91d chore: merge 3.0 2023-12-14 10:09:32 +08:00
Haojun Liao 105542bb21 fix(stream):handle the case of delete orphaned tasks. 2023-12-13 17:44:00 +08:00
wangmm0220 7ba6135f2c fix:error in split plan 2023-12-12 16:37:22 +08:00
kailixu 780f3d9cad Merge branch '3.0' into feat/TD-27463-3.0 2023-12-06 19:14:24 +08:00
kailixu 67fd9993b0 feat: support uniq grant 2023-12-06 15:30:26 +08:00
Haojun Liao 4d9211dcf1
Merge pull request #23899 from taosdata/fix/liaohj
other: merge the patch of stream from 3.0 branch.
2023-12-01 15:49:54 +08:00
Haojun Liao d7a19a9179 fix(stream): fix invalid decode hbmsg and memory leaks caused by invalid hb msg. 2023-12-01 14:45:18 +08:00
Haojun Liao 7e63943c4e
Merge pull request #23860 from taosdata/fix/TD-27455
fix:tmqVnodeSplit-column.py failed in arm64 because of rebalance after unsubscribe while split vnode
2023-12-01 14:18:58 +08:00
Haojun Liao 33ad05e2c8 fix(stream): fix syntax error. 2023-12-01 13:08:25 +08:00
Haojun Liao 12f32f0ae7 fix(stream): fix syntax error. 2023-12-01 11:57:16 +08:00
Haojun Liao dba58f5e38 fix(stream): fix syntax error. 2023-12-01 11:54:13 +08:00
Haojun Liao 18273b8d1b fix(stream): ignore the in_progress code. 2023-12-01 10:20:45 +08:00
Haojun Liao 2972f5f420 fix(stream):return the error code if create stream failed. 2023-12-01 10:20:45 +08:00
Haojun Liao 86618ffd97 refactor: do some internal refactor. 2023-12-01 10:20:38 +08:00
Haojun Liao 31ae46925d fix(stream): fix invalid free. 2023-12-01 10:20:32 +08:00
Haojun Liao c937f42d88 fix(stream): fix deadlock. 2023-12-01 10:19:54 +08:00
Haojun Liao 8515a2f9b6 fix(stream): fix status check for stream-scan-history status when drop stream task msg is missing. 2023-12-01 10:19:18 +08:00
Haojun Liao 9d735e7f8b fix(stream): reduce the sink throughput. and fix the bug in status check. 2023-12-01 10:18:25 +08:00
Haojun Liao 523573844a fix(stream): fix syntax error. 2023-12-01 10:18:24 +08:00
Haojun Liao eb2f5ad074 fix(stream): fix error. 2023-12-01 10:18:22 +08:00
Haojun Liao 2b03d64850 fix(stream): drop the related fill-history task & set the task to be ready, if the drop fill-history task msg is missing due to the vnode restarting. 2023-12-01 10:18:15 +08:00
Haojun Liao 60fa7cccbf fix(stream): fix memory leak. 2023-12-01 10:17:14 +08:00
wangmm0220 4fc19a6343 fix:snode can only be created 1 2023-11-30 15:49:31 +08:00
Haojun Liao 999afd97e5 other: merge 3.0 2023-11-30 14:36:18 +08:00
Haojun Liao de82aba2e5 fix(stream): ignore the in_progress code. 2023-11-30 14:32:14 +08:00
Haojun Liao 71860bfb46 fix(stream):return the error code if create stream failed. 2023-11-30 13:57:00 +08:00
yihaoDeng 5c32680cd8 Merge branch '3.0' into enh/refactorBackend 2023-11-30 10:31:27 +08:00
WANG MINGMING 9b72d890c8
Merge branch '3.0' into fix/TD-27455 2023-11-30 09:37:23 +08:00
Haojun Liao 130614c938
Merge pull request #23831 from taosdata/fix/3_liaohj
enh(stream/tsdb): do multiple optimization and fix some bugs.
2023-11-30 09:20:15 +08:00
wangmm0220 4f168b4b2d fix:tmqVnodeSplit-column.py failed in arm64 because of rebalance after unsubscribe while split vnode 2023-11-29 17:22:43 +08:00
yihaoDeng 786383118c fix drop stream case 2023-11-28 10:39:17 +08:00
Haojun Liao 6bb30822c2 refactor: do some internal refactor. 2023-11-28 10:04:22 +08:00
Haojun Liao 9649db9003 fix(stream): fix invalid free. 2023-11-28 10:04:14 +08:00
Haojun Liao 8b406c0a19
Merge pull request #23830 from taosdata/fix/liaohj
fix(stream): fix invalid free.
2023-11-28 09:52:28 +08:00
Haojun Liao b419650116 refactor: do some internal refactor. 2023-11-27 18:51:30 +08:00
Haojun Liao 91eea02258 fix(stream): fix invalid free. 2023-11-27 18:49:22 +08:00
wangmm0220 d007d31c4d fix:process snode when drop stream 2023-11-27 18:15:09 +08:00
yihaoDeng 630f4f9d80 Merge branch '3.0' into enh/refactorBackend 2023-11-27 13:53:33 +08:00
yihaoDeng e483570cd7 Merge branch '3.0' into enh/refactorBackend 2023-11-27 11:10:56 +08:00
yihaoDeng e9bec34c44 Merge branch '3.0' into enh/refactorBackend 2023-11-27 10:01:12 +08:00
Haojun Liao 788194c748 fix(stream): fix deadlock. 2023-11-24 22:53:55 +08:00
wangmm0220 ff0828fcc2 fix:add O_CLOSEXEC for .running & process snode when drop stream 2023-11-24 18:30:58 +08:00
yihaoDeng 6416a6153a Merge branch '3.0' into enh/refactorBackend 2023-11-24 16:29:09 +08:00
Haojun Liao f28cf740b0 fix(stream): fix status check for stream-scan-history status when drop stream task msg is missing. 2023-11-24 14:44:00 +08:00
Haojun Liao d51baab9c2
Merge pull request #23785 from taosdata/fix/liaohj
other: merge fix from 3.0 to main.
2023-11-24 14:43:02 +08:00
yihaoDeng 23ae62d268 Merge branch '3.0' into enh/refactorBackend 2023-11-24 11:10:39 +08:00
Haojun Liao 2e922fefc6 fix(stream): reduce the sink throughput. and fix the bug in status check. 2023-11-23 18:04:09 +08:00
yihaoDeng 6129ebc294 Merge branch '3.0' into enh/refactorBackend 2023-11-23 17:41:37 +08:00
Haojun Liao 810eb8e0e3 fix(stream): fix syntax error. 2023-11-23 16:25:12 +08:00
Haojun Liao de4cc2009d fix(stream): fix error. 2023-11-23 16:04:06 +08:00
Haojun Liao 0a90c33b4d fix(stream): drop the related fill-history task & set the task to be ready, if the drop fill-history task msg is missing due to the vnode restarting. 2023-11-23 15:49:07 +08:00
Haojun Liao 0654b9299a fix(stream): fix memory leak. 2023-11-23 11:44:25 +08:00
Haojun Liao 8ad071e69a other: merge main. 2023-11-23 10:25:51 +08:00
Haojun Liao ff9c0c0df4 fix(stream): fix error in check trans conflict. 2023-11-23 09:38:11 +08:00
Haojun Liao 6fca8c170c enh(stream): add sink_quota/scan-history-idle-duration column for stream tasks. 2023-11-23 09:33:44 +08:00
Haojun Liao 8204db13aa enh(stream): add sink_quota column for stream tasks. 2023-11-23 09:33:42 +08:00
Haojun Liao 41aa175099 fix(stream): add return. 2023-11-23 09:33:36 +08:00
Haojun Liao 7b1f586d4f
Merge pull request #23762 from taosdata/fix/3_liaohj
fix(stream): fix error in generating token in bucket.
2023-11-23 09:32:14 +08:00
Haojun Liao a2a68ce256 fix(stream): fix memory leak. 2023-11-22 18:28:38 +08:00
Haojun Liao d88b32e93d fix(stream): fix memory leak. 2023-11-22 18:18:43 +08:00
Haojun Liao 9988720a1e fix(stream): fix the invalid check of trans conflict. 2023-11-22 16:39:29 +08:00
Haojun Liao ebb1993900 fix(stream): fix error in check trans conflict. 2023-11-22 16:20:59 +08:00
Haojun Liao 91e7000433 Merge branch 'enh/refactorBackend' of github.com:taosdata/tdengine into enh/refactorBackend 2023-11-22 15:29:31 +08:00
Haojun Liao c96f7cf182 fix(stream): scheduler the checkpoint generating. 2023-11-22 15:17:21 +08:00
yihaoDeng 0eba15b4a4 Merge branch '3.0' into enh/refactorBackend 2023-11-22 10:57:41 +08:00
Haojun Liao 3907268f40 fix(stream): register checkpoint trans. 2023-11-22 08:40:40 +08:00
Haojun Liao d355b78884 fix(stream): fix invalid read. 2023-11-22 08:40:28 +08:00
Haojun Liao 43e6722fd8 refactor: refactor the stream trans conflict check 2023-11-22 08:38:32 +08:00
yihaoDeng 947775155b fix transfer crash 2023-11-21 20:51:11 +08:00
Haojun Liao 785d91c17b
Merge pull request #23661 from taosdata/feat/TD-26060
fix: errors in snode in conditions like fill history/checkpoint/restart
2023-11-21 19:02:38 +08:00
Haojun Liao 8c4558c5c2
Merge pull request #23763 from taosdata/fix/pause_stream
fix(stream): fix error in generating token in bucket.
2023-11-21 15:46:25 +08:00
Haojun Liao 1077378d51 other: merge main. 2023-11-21 10:14:30 +08:00
yihaoDeng 939aa7a3cc Merge branch '3.0' into enh/refactorBackend 2023-11-20 17:33:30 +08:00
dmchen 2d7c4890da TD-27416 2023-11-20 16:50:43 +08:00
dmchen 376f78788e TD-27416 2023-11-20 08:43:12 +00:00
Haojun Liao ec919415b8 enh(stream): add sink_quota/scan-history-idle-duration column for stream tasks. 2023-11-20 12:03:23 +08:00
Haojun Liao 56edf1b52c enh(stream): add sink_quota column for stream tasks. 2023-11-20 12:03:23 +08:00
Haojun Liao 7cfa45e95e enh(stream): add sink_quota/scan-history-idle-duration column for stream tasks. 2023-11-20 11:59:49 +08:00
Haojun Liao 3912e49309 enh(stream): add sink_quota column for stream tasks. 2023-11-20 11:46:11 +08:00
yihaoDeng 734bf07317 Merge branch '3.0' into enh/refactorBackend 2023-11-20 10:48:49 +08:00
yihaoDeng 20de2f3cb5 Merge branch '3.0' into enh/refactorBackend 2023-11-17 17:15:14 +08:00
wade zhang 43a8a76719
Merge pull request #23713 from taosdata/FIX/TD-27336-3.0
fix/TD-27336
2023-11-17 08:59:50 +08:00
yihaoDeng 7905f22aae fix stream backend convert 2023-11-16 17:35:17 +08:00
yihaoDeng 8023f2bebd fix stream backend convert 2023-11-16 15:30:01 +08:00
dmchen 8dbb7ff2d4 copy sql before destroy 2023-11-16 10:28:11 +08:00
dmchen 1e2c9b03a1 copy sql before destroy 2023-11-16 02:22:01 +00:00
dm chen b18aa087da Revert "create steam"
This reverts commit cd6df48df5.
2023-11-16 10:09:46 +08:00
dmchen cd6df48df5 create steam 2023-11-16 01:49:31 +00:00
wangmm0220 a16189d2a9 fix:confilct from 3.0 2023-11-15 19:14:27 +08:00
dmchen 9e37fd8fc6 create stream 2023-11-15 18:48:41 +08:00
dmchen 82498a9b4e create stream 2023-11-15 10:44:53 +00:00
Haojun Liao 1b89d4f404 fix(stream): add return. 2023-11-15 18:06:16 +08:00