Commit Graph

2637 Commits

Author SHA1 Message Date
Haojun Liao 96e1487787 refactor: do some internal refactor. 2024-05-22 16:57:09 +08:00
Haojun Liao 142bc9046d fix(stream): fix memory leak. 2024-05-22 15:13:00 +08:00
Haojun Liao be39044b66 fix(stream): add more info 2024-05-22 11:04:54 +08:00
wangmm0220 f21eeae56c fix:[TD-30180]error in judging if groupId already added to the end of table name 2024-05-21 16:57:00 +08:00
Haojun Liao 3baa4169e6 Merge branch '3.0' into fix/3_liaohj 2024-05-20 10:31:53 +08:00
Hongze Cheng 9527b66db1
Merge pull request #25841 from taosdata/feat/ly_opt_checkpointYh
Feat/ly_opt_checkpointYh
2024-05-20 10:10:28 +08:00
Haojun Liao 130f7a292d fix(stream): reset the error code when trying to restore checkpoint. 2024-05-19 22:36:09 +08:00
Haojun Liao f74fad7e60 fix(query): fix error in refactor. 2024-05-19 17:17:40 +08:00
Haojun Liao 0fe2686b63 fix(query): fix bugs caused by refactor. 2024-05-19 14:55:51 +08:00
Haojun Liao 7a90e68667 fix(query): fix bugs caused by refactor. 2024-05-19 13:55:46 +08:00
Haojun Liao 82c10fd709 fix(query): fix bugs caused by refactor. 2024-05-19 13:40:09 +08:00
Haojun Liao 1e5098f676 fix(query): fix bugs caused by refactor. 2024-05-19 13:32:30 +08:00
Yihao Deng bb47a8df33 opt checkpoint 2024-05-17 15:30:04 +08:00
Yihao Deng b0377a2e3a opt checkpoint 2024-05-17 14:21:04 +08:00
wade zhang b299ff0449
Merge pull request #25798 from taosdata/fix/backendInvalidFree
Fix/backendInvalidFree
2024-05-17 13:46:18 +08:00
Yihao Deng a7b434075b opt checkpoint 2024-05-16 20:16:59 +08:00
Yihao Deng 2359e0e61b opt checkpoint 2024-05-16 18:07:57 +08:00
Yihao Deng ef5d9316bb Merge branch 'enh/refactoBackendCp' into feat/ly_opt_checkpoint 2024-05-16 16:30:27 +08:00
54liuyao 7b50663240 optimize stream operator checkpoint 2024-05-16 16:24:58 +08:00
Yihao Deng f33bf898b4 fix invalid free 2024-05-15 16:56:02 +08:00
Yihao Deng 01e9ae6299 fix invalid free 2024-05-15 16:53:42 +08:00
Haojun Liao 2b44211928 fix(test): update test case and fix a typo. 2024-05-15 14:52:59 +08:00
Haojun Liao abd90c733a fix(stream): remove backend files only after the in-memory task is dropped. 2024-05-15 14:24:35 +08:00
Yihao Deng b1c50e23b6 refactor checkpoint 2024-05-15 11:38:43 +08:00
Haojun Liao 042ed3caff fix(stream): fix invalid unlock. 2024-05-15 10:08:56 +08:00
Haojun Liao 610aa80e65 fix(stream): init backend for fill-history task. 2024-05-15 00:08:38 +08:00
Haojun Liao 4eacf86843 fix(tsdb): fix deadlock when stopping reader. 2024-05-14 19:20:19 +08:00
Haojun Liao 241f8b3d2e fix(tsdb): fix deadlock when stopping reader. 2024-05-14 18:05:13 +08:00
Haojun Liao 1ccf743215 fix(stream): fix syntax error. 2024-05-14 16:19:21 +08:00
Haojun Liao 4fa8cfbffe fix(stream): fix syntax error. 2024-05-14 15:42:37 +08:00
Haojun Liao 6898eba6ed fix(stream): expand stream tasks are divided into two phase, the first is to build stream task and then expand stream task before start stream tasks. 2024-05-14 14:34:31 +08:00
Haojun Liao fb248b2682 fix(stream): not save the stream task for rsma. 2024-05-13 17:26:39 +08:00
Haojun Liao 3dfffe9113 fix(stream): async load and start stream tasks. 2024-05-13 15:31:26 +08:00
Haojun Liao d214dd5cdd fix(stream): fix error in downloading remote backup checkpoint data. 2024-05-11 16:19:11 +08:00
Haojun Liao f8d8061260 refactor: do some internal refactor. 2024-05-11 11:08:51 +08:00
Haojun Liao d7e1f37cd4 Merge branch '3.0' into fix/3_liaohj 2024-05-11 10:28:12 +08:00
Haojun Liao 370c255128 refactor: do some internal refactor. 2024-05-11 10:27:55 +08:00
Haojun Liao 11ed1f54b3 refactor: do some internal refactor. 2024-05-11 10:09:23 +08:00
Haojun Liao 7f93ec2c53 fix(stream): update the checkpoint info for followers, and some internal refactor. 2024-05-11 09:37:02 +08:00
Haojun Liao 845fcfa247
Merge pull request #25700 from taosdata/fix/3_liaohj
refactor: do some internal refactor.
2024-05-09 15:14:46 +08:00
Haojun Liao 6721ec3cd8 refactor: do some internal refactor. 2024-05-09 14:06:25 +08:00
Haojun Liao 2ee8f7da32
Merge pull request #25695 from taosdata/fix/3_liaohj
refactor: do some internal refactor.
2024-05-09 13:36:56 +08:00
Haojun Liao db848f7e6b refactor: do some internal refactor. 2024-05-09 10:15:34 +08:00
Haojun Liao 6693efaa85 refactor: do some internal refactor. 2024-05-09 10:11:37 +08:00
Haojun Liao a450deb60b
Merge pull request #25679 from taosdata/fix/3_liaohj
fix(query): set correct row length for rowid sort.
2024-05-09 10:10:27 +08:00
Haojun Liao 4f59754a63 refactor: do some internal refactor. 2024-05-08 16:22:02 +08:00
Haojun Liao a1a198630a
Merge pull request #25666 from taosdata/fix/3_liaohj
refactor: do some internal refactor.
2024-05-08 15:33:53 +08:00
Haojun Liao bc04093d11 refactor: do some internal refactor. 2024-05-08 13:54:25 +08:00
Haojun Liao 9ce8f6953d refactor: do some internal refactor. 2024-05-08 13:52:20 +08:00
Haojun Liao b26656fd1c
Merge pull request #25661 from taosdata/fix/3_liaohj
refactor: do some internal refactor.
2024-05-08 13:03:13 +08:00
Haojun Liao c43fbf6fee refactor: do some internal refactor. 2024-05-08 11:21:24 +08:00
Haojun Liao cdeb4462fb refactor: do some internal refactor. 2024-05-07 23:43:21 +08:00
Haojun Liao 55a630911d
Merge pull request #25653 from taosdata/fix/3_liaohj
refactor: do some internal refactor.
2024-05-07 23:12:30 +08:00
Haojun Liao b6debd985b fix(stream): always return success if repeatly recv checkpoint source msg. 2024-05-07 16:33:08 +08:00
54liuyao a5213e4cf9 adj delete mark 2024-05-07 15:01:45 +08:00
Haojun Liao 0e0f98eb8d refactor: do some internal refactor. 2024-05-07 10:50:44 +08:00
Haojun Liao bbd423f666 refactor: do some internal refactor. 2024-05-07 09:46:12 +08:00
Haojun Liao 3b43b8f8c7
Merge pull request #25646 from taosdata/fix/3_liaohj
fix(stream): set global close flag in the streamMeta.
2024-05-06 22:19:23 +08:00
Haojun Liao 18c66a5491 refactor: do some internal refactor. 2024-05-06 18:17:38 +08:00
Haojun Liao c982f6ac80 refactor: do some internal refactor. 2024-05-06 17:08:11 +08:00
Haojun Liao 7ad7ef7cfc refactor: do some internal refactor. 2024-05-06 17:03:41 +08:00
Haojun Liao 46e9fe6f97 fix(stream): set global close flag in the streamMeta. 2024-05-06 17:03:09 +08:00
Haojun Liao d879c7c967 fix(stream): set global close flag in the streamMeta. 2024-05-06 16:40:49 +08:00
Yihao Deng 8061688e79 fix rocskdb init failed 2024-05-06 03:22:28 +00:00
Haojun Liao 05a204dd6c refactor: do some internal refactor. 2024-04-30 19:30:56 +08:00
Haojun Liao 732e52d64d refactor: do some internal refactor, and add some logs. 2024-04-30 19:11:58 +08:00
Haojun Liao 94a0f9ab8f other: merge 3.0 2024-04-30 17:12:46 +08:00
Haojun Liao ed610d293f fix(stream): close task if it's in checkdown stream procedure. 2024-04-30 17:09:37 +08:00
Haojun Liao 629502e12e refactor: do some internal refactor. 2024-04-30 17:00:51 +08:00
Haojun Liao f2bc93b78d
Merge pull request #25598 from taosdata/fix/3_liaohj
refactor(stream): opt stream sink perf.
2024-04-30 16:59:22 +08:00
Haojun Liao 973ca3be80 refactor: do some internal refactor. 2024-04-30 09:51:40 +08:00
Haojun Liao cd6261c8d4 refactor: do some internal refactor. 2024-04-30 09:49:11 +08:00
Yihao Deng 3fd6770c81 avoid invalid read 2024-04-29 05:52:54 +00:00
Yihao Deng b54198ba4a avoid invalid read 2024-04-29 03:18:38 +00:00
Yihao Deng 97aadd3aa7 avoid invalid read 2024-04-29 03:17:33 +00:00
Yihao Deng 2beb46c9df Merge branch '3.0' into fix/TD-29844 2024-04-29 01:27:20 +00:00
Haojun Liao 6c93fe5593 fix(stream): fix dead lock. 2024-04-28 18:35:00 +08:00
Haojun Liao b990632e8d fix(stream): fix dead lock. 2024-04-28 18:29:18 +08:00
Haojun Liao bf5d523116 fix(stream): disable the exec of complete check status in timer thread. 2024-04-28 17:57:34 +08:00
Yihao Deng 9e19c52924 Remove unnecessary memory free in uploadCheckpointData function 2024-04-28 06:50:54 +00:00
Yihao Deng fac8530580 fix invalid read 2024-04-28 06:50:33 +00:00
Yihao Deng af567ea2cb fix invalid read 2024-04-28 06:47:31 +00:00
Haojun Liao 57ee97814f fix(stream): fix failed to launch timer bug. 2024-04-28 14:29:18 +08:00
Haojun Liao 3c5ca17737 Merge branch '3.0' into fix/3_liaohj 2024-04-28 13:57:16 +08:00
Haojun Liao 6ea4823f1e fix(stream): update the timeout measurement. 2024-04-28 11:14:10 +08:00
Haojun Liao b2d8260f14 refactor: do some internal refactor. 2024-04-28 10:05:22 +08:00
Haojun Liao 8adaeb59a3 refactor: do some internal refactor. 2024-04-28 09:50:21 +08:00
Haojun Liao 384efdc185
Merge pull request #25531 from taosdata/fix/3_liaohj
fix(stream):add ref for task in check rsp monitor timer.
2024-04-28 09:22:00 +08:00
Haojun Liao a33e7b2eb3 refactor: do some internal refactor. 2024-04-26 22:22:54 +08:00
Haojun Liao 2d1e9ba631 fix(stream):add ref for task in check rsp monitor timer. 2024-04-26 19:11:57 +08:00
Haojun Liao 324072fed8
Merge pull request #25525 from taosdata/fix/3_liaohj
fix(stream): update stream task meta info.
2024-04-26 18:15:09 +08:00
Haojun Liao b1b3246066 fix(stream): update the checkpoint executed transId range. 2024-04-26 17:03:01 +08:00
Haojun Liao 5f5595a6ab
Merge pull request #25517 from taosdata/fix/3_liaohj
fix(stream): add task node into update list if it is timeout for more than 100sec.
2024-04-26 17:00:23 +08:00
Haojun Liao 8b53f76691 fix(stream): discard the repeatly recv checkpoint-source msg. 2024-04-26 15:56:34 +08:00
Haojun Liao 3961903fea refactor: do some internal refactor. 2024-04-26 14:00:03 +08:00
Haojun Liao 88af030966 refactor: do some internal refactor. 2024-04-26 13:53:41 +08:00
Haojun Liao 19b0de6582 fix(stream): add task node into update list if it is timeout for more than 100sec. 2024-04-26 13:49:40 +08:00
Hongze Cheng e678d2508f
Merge pull request #25462 from taosdata/enh/supportTmqInterface
Enh/support tmq interface
2024-04-26 13:05:33 +08:00
Haojun Liao c565e31638
Merge pull request #25510 from taosdata/fix/3_liaohj
fix(stream): handle disorder node Update trans Id.
2024-04-26 12:44:00 +08:00
Haojun Liao e5069669cb refactor: do some internal refactor. 2024-04-26 11:16:40 +08:00
Haojun Liao a0e68c2bfb refactor: do some internal refactor. 2024-04-26 11:14:53 +08:00
Haojun Liao ae4e2bf671 fix(stream): suppress some warnings by ide. 2024-04-26 10:00:54 +08:00
Haojun Liao fae53efed9 fix(stream): add some logs for retry for notready/timeout downstream tasks. and do some internal refactor. 2024-04-26 09:58:20 +08:00
Haojun Liao 77961ea791 fix(stream): handle disorder node Update trans Id. 2024-04-26 09:45:42 +08:00
Haojun Liao 5df720910e
Merge pull request #25497 from taosdata/fix/TD-29672
save stream tbname
2024-04-26 09:14:24 +08:00
Haojun Liao b3d2ccec83
Merge pull request #25483 from taosdata/fix/3_liaohj
fix(query): add ts in cache_scan_operator if pk exists and only retri…
2024-04-25 22:32:26 +08:00
54liuyao 0551c1189d save stream tbname 2024-04-25 18:46:13 +08:00
54liuyao 220a31d02f save stream tbname 2024-04-25 17:48:32 +08:00
54liuyao dbe3b25618 save stream tbname 2024-04-25 17:37:59 +08:00
Haojun Liao addd2d2a84 fix(stream): fix syntax error. 2024-04-25 17:06:15 +08:00
Haojun Liao cd8baa7ba0 fix(stream): add log for update task epset, set correct update flag if epset updated. 2024-04-25 16:39:11 +08:00
Yihao Deng d06cf49182 Fix memory leak in copyDataAt function 2024-04-25 07:44:31 +00:00
Haojun Liao d1ecfe5cf3 refactor: do some internal refactor. 2024-04-25 14:24:20 +08:00
Haojun Liao b698b8be96 fix(stream): pass the address of tmr_h id 2024-04-25 09:57:08 +08:00
54liuyao 2a72ceadd1 Merge branch '3.0' of https://github.com/taosdata/TDengine into 3.0 2024-04-24 18:57:22 +08:00
54liuyao 85cd290aa5 opt bloom filter 2024-04-24 18:40:39 +08:00
54liuyao 3300bf04c1 opt bloom filter 2024-04-24 17:27:24 +08:00
Haojun Liao a37667968f refactor: do some internal refactor. 2024-04-23 19:49:14 +08:00
Haojun Liao 82cde46614 fix(stream): fix memory leak. 2024-04-23 19:48:50 +08:00
Haojun Liao 10c792efa2 fix(stream):add related fill-history task to be done, when check-rsp moniting quit. 2024-04-23 16:55:47 +08:00
Haojun Liao 1adf59e7d8 fix(stream): do check when task is paused, and do some internal refactor. 2024-04-23 13:53:23 +08:00
Haojun Liao 34048937a4 fix(stream): check req id to remove expired check rsp . 2024-04-23 10:03:01 +08:00
Haojun Liao 315b86cf66 fix(stream): save the epset if task epset updated. 2024-04-22 23:52:45 +08:00
Haojun Liao cf7ffb82e0 fix(stream): init before send msg, to avoid race condition. 2024-04-22 17:18:01 +08:00
Haojun Liao 97c1f88b22 refactor: do some internal refactor. 2024-04-22 14:06:11 +08:00
Haojun Liao 6c76790e6c fix(stream): fix a typo 2024-04-22 14:03:13 +08:00
Haojun Liao a8fac441be fix(stream):update the check-rsp prcedure, to avoid repeatly start check-rsp procedure. 2024-04-22 14:02:31 +08:00
Haojun Liao b903872a4e fix(stream): correct count down the ready tasks. 2024-04-19 19:02:34 +08:00
Haojun Liao adfc1fb927 Merge branch '3.0' into fix/3_liaohj 2024-04-19 16:24:53 +08:00
Haojun Liao 3baa47f266 fix(stream): record the task check downstream failed info. 2024-04-19 16:21:05 +08:00
Hongze Cheng fc5c84422e
Merge pull request #25256 from taosdata/feat/TD-24834
Feat/td 24834
2024-04-19 13:34:37 +08:00
Haojun Liao 3769da535e fix(stream): re-send checkmsg to downstream for fixed dispatch type. 2024-04-19 10:00:47 +08:00
Haojun Liao 815d12fd77 fix(stream): set check downstream completed for sink task. 2024-04-18 18:36:02 +08:00
Haojun Liao 4802cd1ddf Merge branch '3.0' into fix/3_liaohj 2024-04-18 17:53:30 +08:00
Hongze Cheng 614f1f5a9d
Merge pull request #25379 from taosdata/feat/addCompress
Feat/add compress
2024-04-18 17:10:34 +08:00
Haojun Liao e7dee4ac63 refactor(stream): refactor check msg retry procedure and be tolerant to the dnode disconnection during check downstream tasks. 2024-04-18 15:55:21 +08:00
Haojun Liao ac84a5f82b refactor: do some internal refactor. 2024-04-17 21:57:09 +08:00
Haojun Liao ddf98ae0c2 refactor: do some internal refactor. 2024-04-17 18:04:02 +08:00
Yihao Deng a461b24ac9 add compress 2024-04-17 09:37:23 +00:00
dm chen 1196ea72da
Merge branch '3.0' into feat/TD-24834 2024-04-17 17:13:29 +08:00
Yihao Deng 07008747ed add compress 2024-04-17 09:09:30 +00:00
Yihao Deng 2547f29de9 add compress 2024-04-17 08:59:38 +00:00
Yihao Deng f8dcf806d2 add compress 2024-04-17 08:58:10 +00:00
Yihao Deng 7cec8836b1 add compress 2024-04-17 08:39:43 +00:00
Yihao Deng 15f9e1e29e add compress 2024-04-17 08:38:46 +00:00
Yihao Deng 1363cd0a71 add compress 2024-04-17 08:33:40 +00:00
Yihao Deng 0f629bebda add compress 2024-04-17 07:35:00 +00:00
Yihao Deng 1d9b65226e add compress 2024-04-17 07:02:55 +00:00
kailixu 74afb8deae chore: solve conflicts 2024-04-17 10:52:34 +08:00
Haojun Liao 13e14d0f55 fix(stream): add return. 2024-04-17 10:48:22 +08:00