Commit Graph

848 Commits

Author SHA1 Message Date
Haojun Liao fe7565330b
Merge branch '3.0' into fix/3_liaohj 2024-08-08 11:20:12 +08:00
Haojun Liao 1e0b6b08b2 fix(stream):set accept code for trans. 2024-08-07 09:54:35 +08:00
Haojun Liao e9ac0d6cd4 other: merge 3.0 2024-08-05 13:38:05 +08:00
Haojun Liao 099064af5e other: merge 3.0 2024-08-02 13:29:35 +08:00
Haojun Liao 9b81f536a1
Merge pull request #26938 from taosdata/fix/create_tb
fix(stream): fix memory leak.
2024-08-02 12:55:17 +08:00
Haojun Liao 079f6358aa refactor: do some internal refactor. 2024-08-02 11:49:21 +08:00
Haojun Liao 42c11e9e36 refactor: do some internal refactor. 2024-08-02 11:38:30 +08:00
Haojun Liao 7f04e2cfb9 fix(stream): fix memory leak. 2024-08-02 10:47:06 +08:00
Haojun Liao 27cadbbcb6 other: merge 3.0 2024-08-02 09:16:03 +08:00
Haojun Liao 2075bfd0f6 fix(stream): check return value. 2024-08-01 17:17:32 +08:00
Haojun Liao aefb9d275e fix(stream): add ts in HbMsg. 2024-08-01 15:22:37 +08:00
Haojun Liao 91dcf63933 other: merge 3.0 2024-07-31 15:01:13 +08:00
Haojun Liao 52b3e1be6f fix(stream): drop orphan tasks/reset task in trans done by write-queue. 2024-07-31 14:08:22 +08:00
dapan1121 4e62fda958
Merge pull request #26877 from taosdata/fix/3.0/TD-31133
fix:[TD-31133] Fix crash when using partition by with having clause.
2024-07-31 10:31:57 +08:00
sima 880921e165 fix:[TD-31133] Fix crash when using partition by with having clause. 2024-07-30 15:55:04 +08:00
Haojun Liao 0bb263076e fix(stream): check return value 2024-07-30 10:13:02 +08:00
dapan1121 05ccb32992
Merge pull request #26816 from taosdata/fix_tsma_test
fix tsma test for invalid paramters when creating tsma
2024-07-28 20:31:11 +08:00
Haojun Liao 5096e25032 fix(stream): set correct name. 2024-07-28 12:14:39 +08:00
wangjiaming0909 49ef380e97 fix tsma test for invalid paramters when creating tsma 2024-07-27 22:40:36 +08:00
kailixu 2fd3bf87fc Merge branch '3.0' into enh/TD-30988-3.0 2024-07-24 16:08:34 +08:00
Haojun Liao 3e6d3c5416 refactor: do some internal refactor. 2024-07-24 14:35:55 +08:00
Haojun Liao 3cb462cdf9 refactor: do some internal refactor. 2024-07-24 14:34:45 +08:00
Haojun Liao 1237120730 fix(stream): ignore error. 2024-07-24 14:31:16 +08:00
Haojun Liao 4d8064c950 fix(tsdb):fix invalid return code. 2024-07-23 19:16:06 +08:00
kailixu 66dda35f75 Merge branch '3.0' into enh/TD-30988-3.0 2024-07-23 18:06:44 +08:00
Haojun Liao 7497f193c7 fix(stream): check return value, and do some internal refactor. 2024-07-23 14:47:29 +08:00
Haojun Liao 5f9f6310a4 fix(stream): check the correct return value. 2024-07-23 09:57:13 +08:00
kailixu 280c5d4b3d enh: return error code 2024-07-22 16:33:42 +08:00
Haojun Liao be162fb62f fix(stream): check return value. 2024-07-22 13:31:57 +08:00
Haojun Liao ebe1d09d74 fix(stream): check return value. 2024-07-22 10:07:18 +08:00
Haojun Liao cb42806148 fix(stream): update some logs. 2024-07-12 13:31:39 +08:00
Haojun Liao 8c024c8554 fix(stream): add some logs. 2024-07-12 11:43:14 +08:00
Haojun Liao 4e684c400d fix(stream): ignore the invalid_vgroup_id error for checkpoint_trans. 2024-07-11 15:12:05 +08:00
Haojun Liao 8db4722be4 fix(stream):fix syntax error. 2024-07-09 16:08:35 +08:00
Haojun Liao acde36a25b Merge branch '3.0' into fix/3_liaohj 2024-07-09 14:43:35 +08:00
Haojun Liao 0cbf88f021 fix(stream): fix memory leak. 2024-07-08 14:51:08 +08:00
Haojun Liao 6e15c16cf7 fix(stream): check the set consensus-checkpointId ts. 2024-07-08 14:33:28 +08:00
Haojun Liao 647f9f47ef refactor: do some internal refactor. 2024-07-08 10:32:23 +08:00
Haojun Liao ce4153b6fc fix(stream): use hb to send the consens-checkpointid req. 2024-07-08 09:05:33 +08:00
Haojun Liao 674acd0e9f refactor(stream): delay checkpointInterval to generate the checkpoint after stream started. 2024-07-05 18:53:35 +08:00
Haojun Liao 6a1555e893 refactor: do some internal refactor. 2024-07-05 18:06:00 +08:00
Haojun Liao 8e6bb176c2 fix(stream): use trans to set the consensus-checkpoint id 2024-07-05 16:17:20 +08:00
Haojun Liao 78993d9c55 fix(stream): update logs. 2024-07-05 09:00:01 +08:00
Haojun Liao db4a00c74e fix(stream): not restart for reset task status. 2024-07-05 08:55:40 +08:00
Haojun Liao 47b0a0464e fix(stream): send checkpoint-source-rsp to mnode before reset task status. 2024-07-04 15:21:16 +08:00
Haojun Liao 653f7a1a43 fix(stream): refactor the checkpoint consensus policy. 2024-07-04 13:55:50 +08:00
Haojun Liao 27cb3638c2 fix(stream): add more check before launching update stream task nodeEp 2024-07-03 09:20:37 +08:00
Haojun Liao c73a003ee3 fix(stream): set the initial checkpointId for a new created stream. 2024-07-02 19:13:20 +08:00
Haojun Liao c7d4f45b7f fix(stream): remove invalid assert. 2024-07-02 15:33:25 +08:00
Haojun Liao 2c86e3940f fix(stream): fix memory leak. 2024-07-01 17:43:10 +08:00
Haojun Liao 025437df0c fix(stream): fix error in checkpoint consensus. 2024-06-27 19:25:18 +08:00
Haojun Liao 48bf5eb4d8 other: merge 3.0 2024-06-27 19:23:06 +08:00
Hongze Cheng b761bd2e38 enh: port main change to 3.0 2024-06-26 10:32:15 +08:00
Haojun Liao 1818edcb2b fix(stream): consensus the start checkpoint id, and extract the streamhb related functions into a new file. 2024-06-26 10:18:32 +08:00
Haojun Liao 53f9af06ff fix(stream): fix invalid write. 2024-06-21 15:44:38 +08:00
Haojun Liao 53b51b9b71 fix(stream): fix syntax error. 2024-06-21 15:23:15 +08:00
Haojun Liao e7105edaa4 fix(stream): disable pause if task is un-init. 2024-06-21 14:14:44 +08:00
Haojun Liao dc2497791a fix(stream): accept invalid vgroup id error code when handling the update checkpoint info. 2024-06-12 10:11:55 +08:00
Haojun Liao b22679c941 fix(stream): do NOT remove the stream if the stream object is not accessiable, since the deploy trans may not completed yet. 2024-06-11 17:13:44 +08:00
Haojun Liao 781b644a8c fix(stream):fix memory leak and update test cases. 2024-06-11 16:34:47 +08:00
Haojun Liao 3b537367c9 fix(stream): fix memory leak. 2024-06-11 15:21:48 +08:00
Haojun Liao 0245d82e54 fix(stream): make sure that the unit test case can work. 2024-06-11 14:26:29 +08:00
Haojun Liao 00eb621825 fix(stream): do scan checkpoint-report in write queue. 2024-06-11 14:09:50 +08:00
Haojun Liao 68aac5dee1 other: fix syntax error. 2024-06-11 11:12:24 +08:00
Haojun Liao cbf8b363fc fix(stream): update checkpoint info by using trans. 2024-06-10 00:46:01 +08:00
Haojun Liao d0675a660e fix(stream): init the node list and task list when starting mnode. 2024-06-07 17:53:53 +08:00
Haojun Liao 34205ba62c refactor: do some internal refactor. 2024-06-07 10:09:40 +08:00
Haojun Liao 88e77d6bc6 fix(stream): remove expired stream in buf. 2024-06-06 23:54:48 +08:00
Haojun Liao 0e9f720cc4 fix(stream):add logs. 2024-06-06 11:27:20 +08:00
Haojun Liao 1e33a1f408 fix(stream): fix memory leak. 2024-06-05 11:43:39 +08:00
Haojun Liao 82293406f1 fix(stream): fix syntax error. 2024-06-05 09:14:33 +08:00
Haojun Liao 29648be30d fix(stream): add the new node info when adding stream tasks. 2024-06-04 23:28:08 +08:00
Haojun Liao 4fc21cc8b0 refactor: do some internal refactor. 2024-06-04 10:08:18 +08:00
Haojun Liao f13dc1858a fix(stream): disable stream checkpoint when related fill-history task exists. 2024-06-03 09:54:28 +08:00
Haojun Liao ee4e4c0e6d refactor: do some internal refactor. 2024-05-31 09:17:20 +08:00
Haojun Liao 0946f25e5f fix(stream): fix syntax error. 2024-05-29 22:36:22 +08:00
Haojun Liao becc4cb368 enh(stream): limit the concurrent started checkpoint tasks. 2024-05-29 19:29:26 +08:00
Haojun Liao 89fb56e77d other: merge 3.0 2024-05-24 11:41:51 +08:00
Haojun Liao 0c61fb20d6 fix(stream): check if outputQ is full before execute the scan history task. if it is full, idle for 5 sec. 2024-05-23 17:22:29 +08:00
Haojun Liao 317742acc1 fix(stream): adjust buf size. 2024-05-22 13:29:38 +08:00
Haojun Liao 565d1f408c fix(stream): adjust buf size, and avoid overflow 2024-05-22 13:26:49 +08:00
Haojun Liao 77852fa702 fix(stream): avoid data overflow and fix test cases. 2024-05-22 13:20:45 +08:00
Haojun Liao be39044b66 fix(stream): add more info 2024-05-22 11:04:54 +08:00
kailixu 04c0c8b6cd enh(grant): refine error code 2024-05-21 15:26:29 +08:00
Haojun Liao f6af165d9a refactor(stream): add more info in info-meta 2024-05-21 10:09:55 +08:00
Haojun Liao d73fd5492a fix(test): update test case. 2024-04-30 14:24:20 +08:00
Haojun Liao 4c27af9b13 refactor: do some internal refactor. 2024-04-30 13:25:39 +08:00
Haojun Liao d3e676b084 fix(cos): fix syntax error. 2024-04-30 10:14:32 +08:00
Haojun Liao 9c39d11ea0 other: add some logs for drop streams. 2024-04-30 09:55:56 +08:00
Haojun Liao 9ba8e6bd34 refactor: add logs for drop streams. 2024-04-30 09:26:06 +08:00
Haojun Liao 3c5ca17737 Merge branch '3.0' into fix/3_liaohj 2024-04-28 13:57:16 +08:00
Haojun Liao ed962186a1 enh(stream): add attrs for stream tasks. 2024-04-28 10:22:09 +08:00
Haojun Liao b2d8260f14 refactor: do some internal refactor. 2024-04-28 10:05:22 +08:00
Haojun Liao 5f5595a6ab
Merge pull request #25517 from taosdata/fix/3_liaohj
fix(stream): add task node into update list if it is timeout for more than 100sec.
2024-04-26 17:00:23 +08:00
Haojun Liao 8b53f76691 fix(stream): discard the repeatly recv checkpoint-source msg. 2024-04-26 15:56:34 +08:00
54liuyao 3300bf04c1 opt bloom filter 2024-04-24 17:27:24 +08:00
kailixu a0a7c05faf fix: if not exists logic for topic/stream/table 2024-04-23 18:42:05 +08:00
Haojun Liao 4802cd1ddf Merge branch '3.0' into fix/3_liaohj 2024-04-18 17:53:30 +08:00
Haojun Liao ddf98ae0c2 refactor: do some internal refactor. 2024-04-17 18:04:02 +08:00
Yihao Deng 04e78d0ce7 Merge branch 'feat/3.0' into feat/compress_3.0 2024-04-17 03:23:02 +00:00
dapan1121 5936d3a1b0
Merge pull request #25300 from taosdata/draft/tsma_before_ci
feat: support tsma
2024-04-17 10:00:17 +08:00
Yihao Deng 6f9de676f0 Merge branch 'fix/compress_3.0' into feat/compress_3.0 2024-04-16 09:27:26 +00:00
Haojun Liao 5ae8b68b17 fix(stream): fix deadlock. 2024-04-16 15:37:51 +08:00
wangjiaming0909 6ce0616ffd fix ci and TODOs 2024-04-16 14:25:34 +08:00
wangjiaming0909 a75c67efc8 rebase with stream sub table name no group id 2024-04-16 14:18:10 +08:00
Haojun Liao 810eb22eb2 fix(stream): generate the checkpoint id after checking the max checkpoint id from vnode. 2024-04-16 14:15:18 +08:00
wangjiaming0909 f77cd43d26 ins_tsmas 2024-04-16 14:13:10 +08:00
wangjiaming0909 6324d1293a drop tsma 2024-04-16 14:13:10 +08:00
wangjiaming0909 8e2977d083 create tsma request 2024-04-16 14:13:10 +08:00
Haojun Liao 13bd296d58 Merge branch '3.0' into fix/3_liaohj 2024-04-15 18:53:48 +08:00
Haojun Liao 1ac192c069 fix(stream): keep the original tsdb scan version range. 2024-04-15 18:41:38 +08:00
Haojun Liao 665107ad3a enh(stream): add more info in meta table. 2024-04-15 14:23:37 +08:00
yihaoDeng 4948bd690d 3.0 merge compress 2024-04-15 11:38:44 +08:00
54liuyao 1d42e927c6 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TS-4243-3.0 2024-04-11 15:35:42 +08:00
Haojun Liao fabb986aa8 fix(stream): fix stack overflow, caused by print epset. 2024-04-10 19:54:33 +08:00
54liuyao 204c5acefe Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/ly-TS-4243-3.0 2024-04-02 09:10:52 +08:00
yihaoDeng 53ecfe511a Merge branch '3.0' into feature/compressData 2024-04-01 08:04:16 +00:00
yihaoDeng 9e11fe45a9 fix ci error 2024-03-31 00:08:23 +00:00
wangmm0220 8e2a2414c0 fix:same subtable same partition by leads to same table name in stream 2024-03-28 13:48:46 +08:00
Haojun Liao e776cde461 fix(stream):fix unrelease stream obj. 2024-03-27 09:02:21 +08:00
Haojun Liao f3c306d582 fix(stream): add some comments. 2024-03-26 23:28:20 +08:00
Haojun Liao 3c7fe5fd01 fix(stream): check null ptr. 2024-03-26 23:27:38 +08:00
Haojun Liao 46280bfee9 fix(stream): add some comments. 2024-03-26 19:40:47 +08:00
Haojun Liao 530e0133b9 fix(stream): acquire the stream task in exec buffer if not in mnode store. 2024-03-26 19:39:58 +08:00
54liuyao a02535c3ab Merge branch 'feat/TS-4243-3.0' of https://github.com/taosdata/TDengine into feat/ly-TS-4243-3.0 2024-03-25 11:15:21 +08:00
54liuyao a60f390cdc stream primary key 2024-03-21 10:50:07 +08:00
Haojun Liao a9ba534f48 fix(stream): adjust conflict level when creating stream. 2024-03-20 18:31:57 +08:00
Haojun Liao 7bdabc309b Merge branch '3.0' into fix/3_liaohj 2024-03-18 14:16:15 +08:00
Haojun Liao a576be6ae5 fix(stream): set the correct rows info for stream tasks when retreiving task info. 2024-03-15 15:19:20 +08:00
Haojun Liao 3aa2e24d78
Merge pull request #25006 from taosdata/fix/nullcheck
fix(stream): set the dest epset from mnode, and set the retry error
2024-03-05 19:46:41 +08:00
Haojun Liao cdfbbc2985 fix(stream): set the dest epset from mnode, and set the retry error code for trans. 2024-03-05 13:28:34 +08:00
Haojun Liao 8791a163f8 other: merge 3.0 2024-02-29 22:31:52 +08:00
Haojun Liao db92bf84d8 other: merge 3.0 2024-02-29 22:21:50 +08:00
Haojun Liao dafac591f7 enh(stream): add rsp for req-checkpoint 2024-02-27 10:53:25 +08:00
Haojun Liao 6e891b0145 fix(stream): add into buffer before start the create stream trans. 2024-02-26 10:22:34 +08:00
Haojun Liao 1cf7790b1c
Merge pull request #24816 from taosdata/fix/nullcheck
fix(stream): send hb correctly after the leader of mnode switching in a cluster.
2024-02-22 08:45:34 +08:00
Haojun Liao 8740c3cb4a fix(stream): send hb rsp explicitly. 2024-02-21 16:17:06 +08:00
wangmm0220 2049fa688a fix:add log 2024-02-21 16:01:02 +08:00
wangmm0220 22b1db5191 fix:stream load error 2024-02-20 14:49:52 +08:00
Haojun Liao 13c3654bf5 other: merge 3.0 2024-02-20 10:51:04 +08:00
Hongze Cheng ffedfbaf1e
Merge pull request #24777 from taosdata/feat/TD-27463-3.0
fix: grant check for stream/topic/subscription
2024-02-19 19:36:22 +08:00
kailixu 5b69e22e9b fix: grant check for streams 2024-02-19 16:29:40 +08:00
wangmm0220 ac9582b24a fix:[TD-28567] do not add md5 to the end of subtable if create stream by sma 2024-02-19 13:58:58 +08:00
wangmm0220 a8e45440c0 fix:conflicts from 3.0 2024-02-18 18:02:14 +08:00
Haojun Liao c4f9bee629 fix(stream): add into buffer pool before start trans, to avoid false alarm on orphan task. 2024-02-18 15:46:37 +08:00
kailixu f76a61c777 feat: support uniq grant 2024-02-13 23:17:05 +08:00
Haojun Liao ecdcc6dee1
Merge pull request #24738 from taosdata/fix/3_liaohj
test(stream): add unit test case for the mnode stream module.
2024-02-05 13:32:46 +08:00
Haojun Liao 3533f7ecea Merge branch '3.0' into fix/3_liaohj 2024-02-05 09:23:29 +08:00
Haojun Liao 739485a419 test(stream): add unit test case for the mnode stream module. 2024-02-05 09:10:04 +08:00
kailixu 231cf000da Merge branch 'feat/TD-27463-3.0a' into feat/TD-27463-3.0 2024-02-02 00:07:15 +08:00