Commit Graph

1181 Commits

Author SHA1 Message Date
Haojun Liao 2b6ace653a refactor: do some internal refactor. 2023-06-08 16:14:27 +08:00
yihaoDeng 1f1f37312d add trigger checkpoint 2023-06-07 14:32:48 +00:00
Haojun Liao 06cf3588da refactor: do some internal refactor. 2023-06-07 15:03:04 +08:00
Haojun Liao 6a3c935b4a fix(stream): fix error in fill history scan. 2023-06-07 09:15:55 +08:00
Haojun Liao 0dd933013c enh(stream): make history task for stream running. 2023-06-05 17:58:00 +08:00
wangmm0220 f2ee657b62 fix:check wal not exist again before register to push & move push logic from write thread to query thread 2023-06-05 17:21:47 +08:00
Haojun Liao fcc706c45c enh(stream): refactor the fill history task. 2023-06-05 15:59:01 +08:00
Haojun Liao 077e1a1ed4 Merge branch '3.0' into refact/fillhistory 2023-06-05 11:41:13 +08:00
Haojun Liao 044e56fc97
Merge branch '3.0' into fix/resume_mem 2023-06-03 12:19:17 +08:00
Haojun Liao cca69f3038 Merge branch '3.0' into fix/resume_mem 2023-06-02 18:40:48 +08:00
wangmm0220 56f7a137ee fix:conflict 2023-06-02 15:47:53 +08:00
wangmm0220 b2f0d88639 fix:conflict 2023-06-02 15:43:35 +08:00
yihaoDeng f4bde33dc3 avoid mem leak 2023-06-02 06:06:05 +00:00
Haojun Liao cce87a036f other: merge 3.0. 2023-06-02 13:34:20 +08:00
Haojun Liao d45aa7ef84
Merge pull request #21555 from taosdata/fix/liaohj_main
other: merge main into 3.0.
2023-06-02 13:11:12 +08:00
Haojun Liao f1bd829977 refactor: do some internal refactor. 2023-06-02 13:10:54 +08:00
Haojun Liao 27b7d1ec88 refactor: do some internal refactor. 2023-06-02 11:17:44 +08:00
yihaoDeng 4d0ce625c4 Merge branch '3.0' of https://github.com/taosdata/TDengine into feature/TD-19042 2023-06-01 22:55:30 +08:00
wangmm0220 a7418d439b Merge branch '3.0' of https://github.com/taosdata/TDengine into feature/TD-19042 2023-06-01 22:50:51 +08:00
wangmm0220 56782a5d41 fix:open task case & modify mqRebVgReq encode/decode style 2023-06-01 18:29:16 +08:00
Haojun Liao 5d98a319c7
Merge pull request #21515 from taosdata/feature/TD-19042
feat: support create topic as stable with conditions
2023-06-01 09:56:55 +08:00
Haojun Liao 5301c5756a other: merge main into 3.0. 2023-06-01 09:29:18 +08:00
Haojun Liao 6d04c4e2fd refactor: do some internal refactor. 2023-05-31 19:05:40 +08:00
Haojun Liao 9e021087a9 refactor: do some internal refactor. 2023-05-31 19:04:53 +08:00
Haojun Liao a3abe25539 refactor: do some internal refactor. 2023-05-31 19:03:37 +08:00
Haojun Liao e237b4ac39 refactor: do some internal refactor. 2023-05-31 09:19:05 +08:00
wangmm0220 689ee05527 Merge branch '3.0' of https://github.com/taosdata/TDengine into feature/TD-19042 2023-05-30 15:45:46 +08:00
wangmm0220 41ed0138b4 fix:compile error 2023-05-29 17:25:11 +08:00
wangmm0220 07bdb95b1a fix:compile error 2023-05-29 17:10:11 +08:00
wangmm0220 5eb1c559e5 fix:save offset if latest 2023-05-29 15:51:00 +08:00
liuyao b541a0a053 set sched status 2023-05-29 15:42:47 +08:00
wangmm0220 78db3520a6 fix:conflict 2023-05-29 14:57:03 +08:00
wangmm0220 a71878d07f feat: support create topic as stable with conditions 2023-05-29 11:46:59 +08:00
Haojun Liao 1e786624b2
Merge pull request #21501 from taosdata/refact/fillhistory
fix(stream): set the skip offset when resume stream task.
2023-05-27 22:45:54 +08:00
Haojun Liao b38e253019 fix(stream): set the skip offset when resume stream task. 2023-05-26 22:51:58 +08:00
Haojun Liao 87957e2bb0
Merge pull request #21478 from taosdata/fix/TD-23423
fix tsdb read error version
2023-05-26 16:56:09 +08:00
Haojun Liao f61f37c936 fix(tmq): set the correct offset rsp when no poll occuring. 2023-05-26 14:20:01 +08:00
Haojun Liao 1e845acabe fix(tmq): allow seek before get assignment 2023-05-26 11:34:12 +08:00
Haojun Liao cea8334427
Merge pull request #21449 from taosdata/refact/fillhistory
refactor: remove the module dependency.  TD-24279
2023-05-26 09:01:45 +08:00
liuyao 75eb724c37 fix tsdb read error version 2023-05-26 08:55:23 +08:00
Haojun Liao da9f33a483 fix: fix error in set function ptr. 2023-05-25 09:56:34 +08:00
Haojun Liao 4a42f6b983 fix: fix error in api call. 2023-05-25 00:50:03 +08:00
wangmm0220 8537e4e80e feat:add tag filter for stable subscribe 2023-05-24 18:26:55 +08:00
Haojun Liao 1f7f326bed refactor: do some internal refactor and set the api function ptr. 2023-05-24 13:22:05 +08:00
liuyao 180abf2229 set agg task status 2023-05-24 09:34:40 +08:00
liuyao b5d01f9bab set agg pause status 2023-05-24 09:19:28 +08:00
Haojun Liao 39a1fa8f78 refactor: do some internal refactor. 2023-05-23 19:20:25 +08:00
wangmm0220 ed612088c0 fix:lost data because tsdbreader or task is killed 2023-05-23 10:32:13 +08:00
wangmm0220 d8387b95ff fix:set task status to 0 in inti 2023-05-22 20:14:43 +08:00
wangmm0220 591b51f624 fix:set task code success after kill 2023-05-22 12:42:21 +08:00
Haojun Liao 899e4d3350 refactor: do some internal refactor. 2023-05-20 18:13:58 +08:00
Haojun Liao de44c9160c refactor: do some internal refactor. 2023-05-20 18:05:39 +08:00
wangmm0220 7b735c67d0 fix:move sleep logic from rpc thread to tmq thread & fix some error 2023-05-20 16:36:25 +08:00
wangmm0220 8183be4aff fix:move sleep logic from rpc thread to tmq thread & fix some error 2023-05-20 16:13:52 +08:00
wangmm0220 04858fae22 fix:tsdbreader is free by mistake 2023-05-20 12:20:54 +08:00
wangmm0220 1ddbdad33c fix:set task status killed when vnode receive subscribe msg from mnode 2023-05-19 21:40:33 +08:00
wangmm0220 33d6df8717 fix:set task status killed when vnode receive subscribe msg from mnode 2023-05-19 19:11:49 +08:00
wangmm0220 c7c255e967 fix:set task status killed when vnode receive subscribe msg from mnode 2023-05-19 18:06:05 +08:00
wangmm0220 ff2b545b27 fix:set task status killed when vnode receive subscribe msg from mnode 2023-05-19 17:16:47 +08:00
wangmm0220 f7ab8dabf0 fix:[TD-23972] push subscribe msg to vnode even though consumer not change 2023-05-19 15:14:39 +08:00
wangmm0220 238254e49e fix:add task status check in doQueueScan & checkout consumer in tqProcessPollReq 2023-05-19 14:06:43 +08:00
Haojun Liao d8525123a2 enh(stream): add API to retrieve last ts for multi-tables. 2023-05-19 11:03:41 +08:00
Haojun Liao c4c5bbb65c other: merge 3.0 2023-05-18 17:28:40 +08:00
liuyao da3fc5111e fix pause agg task 2023-05-18 15:29:41 +08:00
Haojun Liao 81f821a6fc refactor: do some internal refactor. 2023-05-18 10:51:43 +08:00
wangmm0220 54f4860139 fix:rollback [TD-23972] push subscribe msg to vnode even though consumer not change 2023-05-17 19:44:56 +08:00
Haojun Liao b9bda9b30f fix(stream): set the correct start offset for resumed stream task. 2023-05-17 19:36:42 +08:00
wangmm0220 405cdfa7bc fix:error in pHandle lock 2023-05-17 19:02:16 +08:00
wangmm0220 1d9dd153ea fix:error in pHandle lock 2023-05-17 18:57:45 +08:00
Haojun Liao a4d9210b07 fix(tmq): add filter for taosX poll rsp. 2023-05-17 16:40:44 +08:00
wangmm0220 4bb25c280f fix:add log 2023-05-17 15:54:36 +08:00
Haojun Liao 041ffa91bd other: merge main. 2023-05-17 15:42:15 +08:00
Haojun Liao f55f8d245c fix(tmq): avoid the offset is revised by pool rsp which is issued before invoking tmq_offset_seek function. 2023-05-17 15:19:51 +08:00
Haojun Liao 366cd1871a
Merge pull request #21314 from taosdata/fix/liaohj_main
fix(stream):extract delete msg from wal.
2023-05-17 14:52:49 +08:00
Haojun Liao 3896e148fd log(tmq): add some logs. 2023-05-17 11:37:38 +08:00
Haojun Liao f3f52c0232 fix(stream): add some logs and set the wal offset to be the ended version when finishing the step2 recover stage. 2023-05-17 10:43:23 +08:00
wangmm0220 c52a4ea4f0 fix:merge from 3.0 2023-05-17 01:11:55 +08:00
wangmm0220 90099ea1c0 fix:compile error in windows 2023-05-16 18:41:35 +08:00
wangmm0220 e6860cada8 fix:compile error in windows 2023-05-16 18:35:10 +08:00
wangmm0220 a52939127f fix:dead lock 2023-05-16 17:58:31 +08:00
wangmm0220 e6e0ac7603 fix:alwalys in exec if consumer Id mismatch & check result for wal seek ver 2023-05-16 14:01:38 +08:00
Haojun Liao e35d145734 fix(stream): fix memory leak. 2023-05-16 13:59:13 +08:00
Haojun Liao 30e7cb5860 fix(tmq): avoid return delete msg for table subscription. 2023-05-16 10:42:02 +08:00
Haojun Liao 5e16db4e19 fix(stream):extract delete msg from wal. 2023-05-16 08:54:03 +08:00
wangmm0220 62520af2e5 fix:core dump 2023-05-16 00:32:21 +08:00
wangmm0220 36454c73ed fix:add lock to pHandle for safety 2023-05-15 18:38:14 +08:00
Haojun Liao 170c603995 other: merge 3.0 2023-05-13 20:16:55 +08:00
Haojun Liao 99148d6719 other: merge 3.0 2023-05-13 20:02:48 +08:00
Haojun Liao 236926b9ee other: merge enh/rocksdbstate 2023-05-13 19:49:28 +08:00
Haojun Liao 40ffe0cd83 fix(tmq): check handle status before close sub. 2023-05-13 16:45:21 +08:00
Haojun Liao b383651d8b fix(tmq): fix the invalid write and set the flag when closing taosx sub. 2023-05-13 16:17:26 +08:00
Haojun Liao 99422c87c3 fix(tq): set the offset to be the consume offset, instead of the committed offset. And update the table sim. 2023-05-12 16:14:56 +08:00
Haojun Liao 7ef16e1922 other: merge main. 2023-05-12 11:49:53 +08:00
liuyao 8b7b804606 opt partition by tbname 2023-05-11 18:47:27 +08:00
Haojun Liao 26bea33ccc fix(tmq): add null check for print log. 2023-05-11 17:56:19 +08:00
Haojun Liao 2b736ffd60 fix(stream): avoid duplicated task id in task id list. 2023-05-11 11:47:57 +08:00
Haojun Liao 8a1b0ed052 refactor: do some internal refactor. 2023-05-11 10:35:09 +08:00
yihaoDeng 5708fcb5cb Merge branch 'enh/3.0' into enh/rocksdbSstateMerge 2023-05-10 08:02:16 +00:00
Yihao Deng c6e5879f44
Merge branch 'main' into enh/rocksdbSstate 2023-05-10 10:04:28 +08:00
yihaoDeng 8356533eb1 Merge branch 'enh/standVer' into enh/rocksdbSstateMerge 2023-05-09 12:13:16 +00:00
dapan1121 1f973123c4
Merge pull request #21177 from taosdata/fix/TD-23972
fix:[TD-23972] push subscribe msg to vnode even though consumer not c…
2023-05-08 20:29:31 +08:00
Yihao Deng cb223acf82
Merge branch 'main' into enh/rocksdbSstate 2023-05-08 09:45:27 +08:00
Haojun Liao 9392c03b7f fix(stream): add some logs. 2023-05-06 19:05:55 +08:00
wangmm0220 279fe0803f fix:[TD-23972] push subscribe msg to vnode even though consumer not change 2023-05-06 18:35:24 +08:00
Haojun Liao da6212bab0 other: merge main. 2023-05-06 12:51:38 +08:00
wangmm0220 55eddbfb5e fix:[TD-23972] push subscribe msg to vnode even though consumer not change 2023-05-06 10:54:05 +08:00
Haojun Liao 1fa4327f6c Merge branch 'main' into fix/liaohj_main 2023-05-06 10:15:23 +08:00
Haojun Liao 29fcd1b83a refactor: do some internal refactor. 2023-05-06 10:08:05 +08:00
Haojun Liao 8958aabe4b fix(stream): opt task scan performance. 2023-05-05 19:33:30 +08:00
Haojun Liao 8b12d4d3da fix(stream): secure the delete task operation. TD-1198 2023-05-05 15:02:03 +08:00
Haojun Liao 11dccf12de
Merge pull request #21131 from taosdata/fix/liaohj_main
refactor: do some internal refactor.
2023-05-04 11:26:00 +08:00
Haojun Liao 851193a033
Merge pull request #21060 from taosdata/fix/TS-3222
opti:change push mgr to consume msg for subscribe
2023-05-04 10:16:41 +08:00
liuyao d8a1f15d10 fix resum bug 2023-05-04 09:28:06 +08:00
Haojun Liao 3c61932ecc refactor: do some internal refactor. 2023-05-01 19:28:54 +08:00
Haojun Liao 777ed1761f refactor: update logs. 2023-04-29 18:24:46 +08:00
Haojun Liao bbb8aaa0dc refactor: add some logs. 2023-04-29 02:37:10 +08:00
Haojun Liao b69137df44 refactor(log): add some logs. 2023-04-29 00:00:25 +08:00
Haojun Liao c647ded0da fix(stream): add some logs. 2023-04-28 19:47:49 +08:00
Haojun Liao b7c8346b44 other: merge other branch. 2023-04-28 13:06:47 +08:00
Haojun Liao f9a64cbcd6 other: add some logs. 2023-04-28 12:58:03 +08:00
liuyao 2984c5a758 merge main 2023-04-28 10:34:46 +08:00
wangmm0220 cce868d140 fix:offset encode assert error 2023-04-28 09:45:16 +08:00
wangmm0220 1ee1b0422c fix:change push mgr from SArray to Hash 2023-04-27 23:44:27 +08:00
wangmm0220 db5b5c828e fix:add log for msg push 2023-04-27 18:35:10 +08:00
Haojun Liao 91f9b58f9a refactor: do some internal refactor. 2023-04-27 16:06:06 +08:00
wangmm0220 8677b56a4f fix:assert error in tqProcessSubmitReqForSubscribe if put pHandle to array twice 2023-04-27 15:44:44 +08:00
Haojun Liao b7040ed15e enh(stream): support resume from the latest data, and do some internal refactor. 2023-04-27 15:07:54 +08:00
wangmm0220 224d87b131 fix:cosume null if rebalance 2023-04-26 17:58:14 +08:00
54liuyao b79313ecd7 pause&resume 2023-04-25 16:59:09 +08:00
wangmm0220 d1e5d6e0f9 fix:pHandle->msg is not null if rebalance 2023-04-25 16:23:58 +08:00
wangmm0220 41bec8560a opti:change push mgr to consume msg for subscribe 2023-04-25 09:58:28 +08:00
wangmm0220 8572f4a32f opti:change push mgr to consume msg for subscribe 2023-04-24 19:41:30 +08:00
wangmm0220 77e03bfd78 opti:change push mgr to consume msg for subscribe 2023-04-23 20:14:49 +08:00
liuyao 641c707b37 pause&resume 2023-04-23 17:37:10 +08:00
liuyao 5bd8fe5441 feat:add pause&resume 2023-04-23 15:09:01 +08:00
54liuyao 5089528a02 feat:stream pause&resume 2023-04-23 15:08:51 +08:00
Haojun Liao 9cf89c6b6f enh(tmq): add check for consumer Id; 2023-04-23 11:51:43 +08:00
yihaoDeng 917b07348c Merge branch 'fix/rocksdbState' into enh/rocksRevert 2023-04-23 02:37:26 +00:00
Haojun Liao 3dbe208712 enh(tmq): fix memory error and add retrieve wal info as required. 2023-04-22 15:34:46 +08:00
wangmm0220 6efbcfec4b Merge branch 'main' of https://github.com/taosdata/TDengine into fix/TS-3222 2023-04-21 17:46:35 +08:00
Haojun Liao 6c641cff39 fix(stream): fix error in start stream tasks. 2023-04-21 09:37:16 +08:00
Haojun Liao 471abd9160 log: update the log. 2023-04-20 23:39:13 +08:00
Haojun Liao 5e6c06e253 fix(stream): fix the race condition during create new stream tasks. 2023-04-20 23:21:18 +08:00
Haojun Liao f9801ba9c5 enh(stream): stop stream asap. 2023-04-20 18:58:07 +08:00
Haojun Liao 742b5ee08c other: update wal logs level. 2023-04-20 13:42:16 +08:00
Haojun Liao c654f1145d enh(stream): add new msg for seek, and do some internal refactor. 2023-04-20 10:59:16 +08:00
wangmm0220 594d68b8a4 fix:remove lock for consume handler 2023-04-19 11:46:59 +08:00
wangmm0220 14fd2e7904 fix:remove lock for consume handler 2023-04-19 11:44:52 +08:00
wangmm0220 25542c26c7 fix:remove tmPushMsg in consumer 2023-04-19 10:43:13 +08:00
Haojun Liao 7a8dd0db45 refactor: update the log. 2023-04-18 16:08:51 +08:00
Haojun Liao 76becf59b7
Merge pull request #20947 from taosdata/feature/3_liaohj
fix(stream): set the correct initial checkpoint versions.
2023-04-18 11:26:48 +08:00
Haojun Liao 2ae05b8cf6 fix(stream): set the correct initial offset value. 2023-04-18 08:56:31 +08:00
liuyao 27d14f742d fix:stream memory leak 2023-04-17 17:27:02 +08:00
Haojun Liao be90d2c511 fix(stream): disable stream task when no tasks exist. 2023-04-16 23:07:54 +08:00
Haojun Liao 1350af5267 fix(stream): set the correct initial checkpoint version to restore the operators state and add check for the initial destination tables. 2023-04-16 22:48:22 +08:00
Haojun Liao ab4b08a4b5 Merge branch 'feature/3_liaohj' into enh/rocksRevert 2023-04-14 17:44:50 +08:00
Haojun Liao e0cb8aa534 fix(stream): don't the initial task status and do some internal refactor. 2023-04-14 17:43:37 +08:00
Haojun Liao eb0e1f848a fix(stream): remove unused tqreader, do some internal refactor, set the meta value for streamtask. 2023-04-14 12:00:09 +08:00
Haojun Liao b57c813681 other: merge feature/3_liaohj 2023-04-14 10:36:54 +08:00
Haojun Liao f8b672f09f other: merge main. 2023-04-14 09:07:18 +08:00
Haojun Liao 4d83118ff7 fix(stream): fix memory leak. 2023-04-13 23:49:14 +08:00
Haojun Liao 70b45a4cf0 fix(stream): all data should be extracted from wal. 2023-04-13 23:22:09 +08:00
Haojun Liao 5ab54481f0 refactor: do some internal refactor. 2023-04-11 19:24:34 +08:00
Haojun Liao f083697dba refactor(tq): do some internal refactor. 2023-04-11 19:04:33 +08:00
yihaoDeng b24b91886a Merge branch 'dev3' into enh/rocksRevert 2023-04-11 10:59:00 +00:00
Xiaoyu Wang 4c04b155d5 merge main 2023-04-11 15:56:28 +08:00
Haojun Liao 18479d8115 refactor: do some internall refactor. 2023-04-11 15:42:24 +08:00
Haojun Liao 15cceb5a5a refactor: do some internal refactors. 2023-04-11 11:53:10 +08:00
Haojun Liao a4ba5401f9 enh(stream): set the start version of all operators. 2023-04-11 10:01:36 +08:00
Haojun Liao 7419c0bfe9 enh(stream): the stream will start after vnode restore being completed. 2023-04-10 16:56:14 +08:00
Haojun Liao f5b52749c2 refactor: do some internal refactor. 2023-04-09 01:39:09 +08:00
Haojun Liao 871cfb4760
Merge pull request #20775 from taosdata/fix/TD-23450
fix:tq memory leak
2023-04-06 11:48:45 +08:00
Haojun Liao 2b2059ef18
Merge pull request #20534 from taosdata/fix/TD-22671
fix:error in TD-23218 & remove useless logic
2023-04-06 11:47:24 +08:00
yihaoDeng 5aff5aa93b add backpressure 2023-04-04 12:38:56 +00:00
54liuyao 519e2f1cfd fix:tq memory leak 2023-04-04 18:21:37 +08:00
yihaoDeng 628bb62c8a add backpressure 2023-04-04 07:20:05 +00:00
Xiaoyu Wang e9d7a099cc merge main 2023-04-04 14:50:58 +08:00
Haojun Liao d48be5c0ce refactor: do some internal refactor. 2023-04-03 18:23:35 +08:00
Haojun Liao a3ac8b647c refactor: do some internal refactor. 2023-04-03 18:20:35 +08:00
Haojun Liao 482c319bee refactor: do some internal refactor. 2023-04-03 18:07:28 +08:00
Haojun Liao 3ef7d43dae refactor: do some internal refactor. 2023-04-03 17:47:48 +08:00
wangmm0220 2d6436c57d fix:modify log & sleep 500ms if data is inserting while consume 2023-04-03 11:24:22 +08:00
wangmm0220 97b5c7abee fix:merge from main 2023-03-31 16:20:21 +08:00
Haojun Liao 5f4cb41e28 fix(tmq): set correct start offset value. 2023-03-29 23:45:41 +08:00
Haojun Liao 56c98d7768 refactor: do some internal refactor. 2023-03-29 19:35:04 +08:00
wangmm0220 2243184a4d fix:conflicts 2023-03-29 17:53:05 +08:00
wangmm0220 9d9ae749d1 opti:tmq logic & fix CI cases 2023-03-29 17:27:04 +08:00
wangmm0220 d0c55ef572 fix:conflicts 2023-03-28 18:12:13 +08:00
Haojun Liao a883e57cf2 Merge branch 'main' into fix/liaohj 2023-03-28 15:07:32 +08:00
Haojun Liao 68f310dd48 refactor: do some internal refactor. 2023-03-28 15:05:10 +08:00
Haojun Liao 4fd3064fd9 refactor: do some internal refactor. 2023-03-28 14:43:37 +08:00
Haojun Liao fca79dbc53 Merge branch 'fix/liaohj' into main 2023-03-28 11:20:01 +08:00
Haojun Liao c158087d66 fix(tmq): kill the ongoing tsdb scans while transferring the ownership of vnode to other consumers. 2023-03-28 09:53:49 +08:00
Haojun Liao 5c128e2292 refactor: do some internal refactor. 2023-03-28 09:27:48 +08:00
wangmm0220 f1d67b6059 fix:conflict 2023-03-25 19:03:23 +08:00
wangmm0220 750cd8c5a4 fix:ci error 2023-03-25 16:33:47 +08:00
wangmm0220 9f5e435140 fix:conflict from main 2023-03-25 11:49:37 +08:00
Haojun Liao b22cc82270 others: merge main. 2023-03-24 15:43:13 +08:00
wangmm0220 5d06474f2e fix:send data batch if consume wal where subscribe db 2023-03-24 14:43:29 +08:00
wangmm0220 bf78f63e22 fix:send data batch if consume wal where subscribe db 2023-03-24 10:10:52 +08:00
wangmm0220 5f9c7e9a7d fix:[TS-2961] consume error change from snapshot to wal for taosx 2023-03-22 20:14:19 +08:00
wangmm0220 6fae38619d fix:error in TD-23218 & remove useless logic 2023-03-22 19:36:12 +08:00
wangmm0220 dd4cf0597b fix:error in optimize consume logic 2023-03-22 17:04:02 +08:00
dapan1121 5b02af6a91 fix: merge main to 3.0 2023-03-22 16:08:16 +08:00
Haojun Liao 69ca2b2f7b fix(tmq): do some internal refactor. 2023-03-21 19:40:36 +08:00
Haojun Liao c3acab2070 fix(tmq): do some internal refactor. 2023-03-21 19:38:34 +08:00
Haojun Liao ca29dd344f other: merge main. 2023-03-21 14:04:52 +08:00
Haojun Liao 18b121c69c fix(tmq): fix the invalid write and do some internal refactor. 2023-03-21 13:45:43 +08:00
Haojun Liao ea0cc2c584
Merge pull request #20554 from taosdata/fix/TS-2828
fix: add retry if insert error with meta change
2023-03-21 09:01:22 +08:00
wangmm0220 f7b7320f5f fix:set offset if read wal none 2023-03-20 16:57:45 +08:00
wangmm0220 f80078030c fix:error in TD-23218 & remove useless logic 2023-03-20 15:40:01 +08:00
Haojun Liao 2517f9bb83 fix(tmq): add some logs. 2023-03-20 15:27:16 +08:00
Haojun Liao 4cb1c51d4e fix(tmq): update some logs. 2023-03-20 13:27:58 +08:00
wangmm0220 a5144f0ba2 fix:set offset if read wal none 2023-03-20 10:10:07 +08:00
Haojun Liao 7e712a14b6 fix(tmq): adjust log. 2023-03-19 19:06:28 +08:00
Haojun Liao ae5155edcf fix(tmq): do some internal refactor. 2023-03-19 10:49:30 +08:00
wangmm0220 ccba1d07c6 fix:error in TD-23218 & remove useless logic 2023-03-18 18:42:49 +08:00
wangmm0220 7616a283e3 fix:error in TD-23218 & remove useless logic 2023-03-18 18:23:38 +08:00
wangmm0220 4fdef7421c fix:add log for taosCreateLog error 2023-03-17 10:50:54 +08:00
yihaoDeng 3ac11ce87a enh: change log 2023-03-16 06:17:45 +00:00
yihaoDeng ef092c1cce enh: add queue limit 2023-03-15 11:54:36 +00:00
Haojun Liao 32ddecc347 other: merge main. 2023-03-14 12:13:09 +08:00
Haojun Liao 731a2138df fix(tmq): fix error in tmq. 2023-03-13 17:08:17 +08:00
Haojun Liao 7f2aae69d4 fix(tmq): fix error in taosx 2023-03-13 01:54:29 +08:00
Haojun Liao 564e9bb833 Merge branch 'main' into fix/liaohj 2023-03-12 14:38:38 +08:00
wangmm0220 0e2cf0ed73 fix:deal with ASSERT in tmq 2023-03-11 16:51:34 +08:00
wangmm0220 9cc7a9f504 fix:deal with ASSERT in tmq 2023-03-11 16:12:11 +08:00
Haojun Liao 05ddf68247 fix(tmq): fix invalid free. 2023-03-10 14:27:34 +08:00
Haojun Liao d8807d3d5a fix(tmq): fix memory leak. 2023-03-10 14:23:16 +08:00
Haojun Liao 691cb08e88 refactor: remove some unused attributes. 2023-03-10 09:23:03 +08:00
Haojun Liao 09e84884be refactor: remove invalid assert 2023-03-09 23:31:29 +08:00
Haojun Liao 03aa391859 fix(tmq): add lock before close the tsdbreader. 2023-03-09 23:03:51 +08:00
54liuyao 5acf665b7e fix:get dest table version 2023-03-09 19:36:10 +08:00
Haojun Liao d54ae0c840 fix(tmq): free reader after check the reader status. 2023-03-07 17:49:20 +08:00
Haojun Liao e8c11c58f0 fix(tq): add some logs. 2023-03-03 10:20:52 +08:00
wangmm0220 986ae9c4d0 fix:conflicts 2023-02-28 15:11:16 +08:00
wangmm0220 c638564179 fix:confilcts from main 2023-02-26 23:09:57 +08:00
Haojun Liao b46098793d refactor: do some internal refactor and add some logs for tmq. 2023-02-26 12:16:45 +08:00
Haojun Liao 0369ff2231 other: merge main. 2023-02-25 15:42:32 +08:00
Haojun Liao 514369c611 other: merge main 2023-02-25 11:06:13 +08:00
Haojun Liao d6fca036e5 refactor: do some internal refactor. 2023-02-25 11:02:42 +08:00
Xiaoyu Wang d6dd087f84 merge main 2023-02-25 10:29:20 +08:00
Xiaoyu Wang 5856a39827
Merge pull request #20123 from taosdata/fix/3.0_merge_main
merge main
2023-02-25 10:01:26 +08:00
wangmm0220 54a546931c fix:optimize version logic in tmq and remove useless code 2023-02-24 19:22:08 +08:00
Haojun Liao 097723f446
Merge pull request #20151 from taosdata/feature/3_liaohj
fix(query): expand output buffer.
2023-02-24 18:56:51 +08:00
Haojun Liao 22aa03307c fix(query): fix syntax errors on centos. 2023-02-24 11:38:54 +08:00
Haojun Liao 7821bbe0d0 refactor: add some logs. 2023-02-24 10:08:25 +08:00
Haojun Liao cc393f3939
Merge pull request #20143 from taosdata/feature/3_liaohj
refactor: do some internal refactor.
2023-02-24 09:58:44 +08:00
Haojun Liao bbb571a383 refactor: do some internal refactor. 2023-02-24 00:06:18 +08:00
Haojun Liao dbb1e9bd59 refactor: do some internal refactor. 2023-02-23 21:59:55 +08:00
Xiaoyu Wang 1ab144609e merge main 2023-02-23 14:58:22 +08:00
54liuyao 72e7f617e2 fix:disable stream 2023-02-23 14:24:45 +08:00
Xiaoyu Wang c597de099f merge main 2023-02-20 13:41:27 +08:00
Haojun Liao 7fe250412f refactor:do some internal refactor. 2023-02-20 10:13:11 +08:00
Haojun Liao 8d187efc18 refactor:do some internal refactor. 2023-02-20 10:04:08 +08:00
Xiaoyu Wang dc984d92f7 feat: compatible with older versions of wal 2023-02-17 09:21:07 +08:00
Liu Jicong 2a4708d82a fix: save fill history state 2023-02-16 17:01:09 +08:00
Liu Jicong b147ba2812 fix: handle insufficient resource 2023-02-14 16:18:40 +08:00
wangmm0220 c8d49b603d fix:roll back htol length for block raw data 2023-02-02 17:23:01 +08:00
Liu Jicong 041d0d817a fix: tq open 2023-02-02 09:11:17 +08:00
wangmm0220 8a21d7f307 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-21899 2023-02-01 16:13:06 +08:00
wangmm0220 94a3fa4ff3 fix:htol(length) & set vgId,uid correct for auto create table in taosx 2023-01-30 09:55:22 +08:00
Hongze Cheng 5e7b6602c5 Merge branch 'main' of https://github.com/taosdata/TDengine into fix/3.0_merge_main 2023-01-29 14:05:15 +08:00
Liu Jicong f5c4ca3380 fix: wal ref 2023-01-28 17:17:55 +08:00
Liu Jicong 2a71d47668 fix: wal ref 2023-01-28 16:11:17 +08:00
Minglei Jin 625bf9992f
Merge branch '3.0' into fix/long_query 2023-01-04 15:15:51 +08:00
Liu Jicong e30c00875c enh: close tsdb reader 2023-01-04 15:14:00 +08:00
Xiaoyu Wang fdde287ca9 merge main 2023-01-04 11:43:20 +08:00
Liu Jicong 66b9fa01f7 enh: optimize drop when scan exec 2022-12-29 15:51:12 +08:00
Shengliang Guan 78c14a2c88 Merge remote-tracking branch 'origin/3.0' into merge/mainto3.0_1228 2022-12-28 14:59:02 +08:00
Liu Jicong b2cec0f1e9 fix: skip msg 2022-12-27 14:02:23 +08:00
Haojun Liao 723efba79b
Merge branch '3.0' into refact/submit_req 2022-12-26 13:25:25 +08:00
Shengliang Guan 664dcb6b4e Merge branch 'main' into merge/mainto3.0_1226 2022-12-26 09:26:54 +08:00
Liu Jicong 0d56da7508 fix: skip msg 2022-12-23 19:04:39 +08:00
Xiaoyu Wang 0c269c317b merge 3.0 2022-12-23 17:50:05 +08:00
Liu Jicong 2a7de0cd46 refactor: remove assert 2022-12-21 13:54:14 +08:00
Haojun Liao fdade6d6a3
Merge branch '3.0' into refact/submit_req 2022-12-19 11:24:00 +08:00
Liu Jicong be03f7aa8b Merge branch 'main' into feature/stream 2022-12-16 14:42:44 +08:00
Liu Jicong 4402bcef94 enh(stream): forbid source db replica > 1 2022-12-16 08:53:33 +08:00
Haojun Liao 48f7fc5c3d
Merge branch '3.0' into refact/submit_req 2022-12-12 18:11:17 +08:00
Shengliang Guan aa512f1d55
Merge pull request #18874 from taosdata/fix/TD-21117
fix: control rpc qitem memory
2022-12-10 15:58:31 +08:00
Shengliang Guan 50b66df39f fix: control rpc qitem memory 2022-12-10 14:02:57 +08:00
Liu Jicong a6fff95795 fix: close wal ref 2022-12-09 17:18:21 +08:00
Liu Jicong 74a253eac4 refactor 2022-12-08 13:34:42 +08:00
Shengliang Guan 6d94afe48f
Revert "refact: adjust some assert cases" 2022-12-07 21:51:17 +08:00
Shengliang Guan d5126d469a refact: replcase ASSERT with tAssert 2022-12-07 18:42:48 +08:00
Shengliang Guan f69a188f7e refact: replcase ASSERT with tAssert 2022-12-07 18:37:40 +08:00
Liu Jicong 9e7bbf622a refactor: support submitreq2 2022-12-07 17:54:09 +08:00
Liu Jicong 45b93280c9 refactor: support submitreq2 2022-12-07 17:22:38 +08:00
Liu Jicong b133a56219 refactor: support submitreq2 2022-12-07 15:19:34 +08:00
Hongze Cheng 899d7b0bcc Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/submit_req 2022-12-06 14:37:22 +08:00
Liu Jicong 619bbdfd82 refactor: support submitreq2 2022-12-06 05:58:28 +08:00
Shengliang Guan d9c261cd13 refact: remove some error codes 2022-12-03 10:17:02 +08:00
Hongze Cheng 1cf9974b50 Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/submit_req 2022-12-02 10:10:21 +08:00
Liu Jicong 3424b184d3 refactor(stream): distributed checkpoint 2022-12-02 03:48:45 +08:00
Hongze Cheng f848460f21 Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/submit_req 2022-11-30 13:28:17 +08:00
Liu Jicong 548684e5ac fix(stream): delete multiple row 2022-11-29 23:39:58 +08:00
Hongze Cheng 0fdf4193e0 Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/submit_req 2022-11-27 11:22:21 +08:00
Liu Jicong 48894c8bdb enh(stream): new api for stream queue 2022-11-25 18:03:40 +08:00
Hongze Cheng 91a5f8958f Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/submit_req 2022-11-24 09:23:45 +08:00
Shengliang Guan 962aa4aff4
Merge pull request #18383 from taosdata/feature/stream
fix(stream): memory leak
2022-11-24 08:17:40 +08:00
Hongze Cheng c1db8a9e4d refact code 2022-11-23 20:13:27 +08:00
Liu Jicong 3b84053893 fix(stream): memory leak 2022-11-23 12:32:02 +08:00
dapan1121 19852ba85b Merge remote-tracking branch 'origin/3.0' into enh/msgRefactor2 2022-11-22 10:45:11 +08:00
Liu Jicong 9358429e78 Merge branch '3.0' into feature/stream 2022-11-21 20:54:21 +08:00
dapan1121 c66390d401 enh: refact tmq pull message 2022-11-21 20:38:42 +08:00
Liu Jicong 6478fcb2c1 fix(stream): check task 2022-11-15 22:32:52 +08:00
Liu Jicong a9bf05bc01 refactor: stream meta ref count 2022-11-15 19:11:42 +08:00
Liu Jicong 76cd3122d1 fix: memory leak 2022-11-15 14:28:20 +08:00
Liu Jicong efde22f081 fix: memory leak 2022-11-15 11:59:29 +08:00
Liu Jicong 2386e0b513 refactor(tmq): add more strict drop condition check 2022-11-11 14:37:31 +08:00
Liu Jicong 2fbc8e6306 refactor(stream): stream deploy and state transfer 2022-11-03 13:22:23 +08:00
Liu Jicong 94e7c79ca0 fix(tmq): handle adding subtable for subscribing stb 2022-11-01 18:18:26 +08:00
Liu Jicong 5e253cc8ea refactor(stream): fill history 2022-11-01 11:56:14 +08:00
Liu Jicong 2e640c38a2 refactor: stream and tmq message 2022-10-31 18:37:28 +08:00
Liu Jicong b26cc9c318 feat(snode) 2022-10-28 17:03:17 +08:00
Liu Jicong dcf2c980c5 refactor(mnode): remove unused code 2022-10-27 15:32:20 +08:00
Liu Jicong 42422b7a65 fix(taosx): reset latest offset 2022-10-26 21:20:35 +08:00
Liu Jicong a16b010e9f fix: memory leak 2022-10-26 11:35:15 +08:00
Liu Jicong 8d8fd2b2bc refactor(stream): recover and fill history 2022-10-26 10:20:33 +08:00
Liu Jicong 75e5e490fd refactor(stream): recover and fill history 2022-10-26 00:31:00 +08:00
Benguang Zhao 00a066443f Merge branch '3.0' into FIX/TD-19593-3.0 2022-10-19 09:38:36 +08:00
Benguang Zhao 3b0968d7f9 enh: enable the -Wformat=2 compiler option 2022-10-19 09:24:15 +08:00
Liu Jicong ca2012a521 refactor: tmq commit 2022-10-18 16:56:07 +08:00
Liu Jicong 73662b3058 refactor: table sink 2022-10-17 14:48:56 +08:00
Liu Jicong 980fdd6e91 fix: error handling 2022-10-17 10:27:15 +08:00
Liu Jicong 6f8d4d6955 fix(tmq): push msg 2022-10-14 13:31:05 +08:00
Liu Jicong d507678b35 refactor(stream): set state buffer size 2022-10-13 11:18:02 +08:00
Liu Jicong b1594a6857 fix coverity scan 2022-10-10 11:44:17 +08:00
Liu Jicong bb1165f1e9 optimize projection return 2022-09-29 12:14:41 +08:00
Liu Jicong a2a8fe0211 rsp consumer when taosd stop 2022-09-29 01:37:45 +08:00
Liu Jicong b0a46942c2 rsp to consumer if offset move forward 2022-09-28 23:47:47 +08:00
Liu Jicong fab0adde99 fix memory error 2022-09-28 23:06:45 +08:00
Liu Jicong cde4085a6d optimize msg batch 2022-09-28 20:24:02 +08:00
Liu Jicong 01c94a775b feat(tmq): push optimization 2022-09-28 14:55:19 +08:00
Liu Jicong 0e8aaf4c9e feat(tmq): push optimization 2022-09-28 10:38:49 +08:00
Liu Jicong 7ade2b3b8d fix double free in sma 2022-09-21 18:54:49 +08:00
Liu Jicong 72fba5a8ce fix(stream): check delete uid 2022-09-21 11:54:44 +08:00
Shengliang Guan 69e1edb344
Merge pull request #16898 from taosdata/fix/TD-18740
feat:Complete the automatic table creation function for taosX
2022-09-20 11:35:18 +08:00
Liu Jicong 756a2d3d5d merge 3.0 2022-09-19 19:29:20 +08:00
Liu Jicong 86c0fb56e0 enhance stream backend for sma 2022-09-19 17:27:25 +08:00
wangmm0220 e3bc2487b3 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-18740 2022-09-16 19:00:49 +08:00
wangmm0220 dfc5ce23a0 fix:error in auto create table for taosX 2022-09-16 17:01:41 +08:00
Liu Jicong 367552b914 fix delete 2022-09-15 10:12:53 +08:00
Liu Jicong 004a84ac81 fix memory leak 2022-09-02 16:41:32 +08:00
Liu Jicong d653da7971 feat(stream): support delete data 2022-09-02 15:46:41 +08:00
Liu Jicong 1fe4ab2341 feat(taox): support auto create sub table 2022-09-01 11:45:00 +08:00
Liu Jicong c4dcc994fb remove queue scan 2022-08-31 11:35:25 +08:00
wangmm0220 bf3231ab9f fix: cols num error in tmq for query 2022-08-29 17:30:50 +08:00
wangmm0220 6b61d3c23b fix:conflicts from 3.0 2022-08-25 16:57:19 +08:00
wangmm0220 4cc2d361e1 fix:error in getting snapshot for taosX 2022-08-25 16:55:12 +08:00
wangmm0220 c7bf08d5af feat: get snapshot data for taosX 2022-08-24 17:42:33 +08:00
Liu Jicong 24261cc90b refactor(stream): refine stream backend interface 2022-08-24 15:53:29 +08:00
wangmm0220 2afc23a120 fix:conflicts from 3.0 2022-08-24 13:59:51 +08:00
Liu Jicong d7b804e02e fix stream load task 2022-08-24 10:42:34 +08:00
wangmm0220 f477e94dda fix:error in snapshot from taosx 2022-08-24 10:24:19 +08:00
Liu Jicong 2d628491b2 feat(stream): support tdb state backend 2022-08-23 19:30:04 +08:00
wangmm0220 fbc69a10c4 Merge branch '3.0' into feature/TD-14761 2022-08-18 16:45:06 +08:00
wangmm0220 d2727b6252 feat:merge from 3.0 2022-08-18 10:45:20 +08:00
Liu Jicong 853e6e2988 refactor(mnode): drop stream task 2022-08-17 19:19:58 +08:00
Liu Jicong 357b21cfeb refactor(tmq): use tdb to store check info 2022-08-17 13:58:56 +08:00
wangmm0220 78c9f245b1 fix:conflicts from 3.0 2022-08-16 15:18:11 +08:00
Liu Jicong 18f6fb6123 enh(tmq): speed up consumer recover 2022-08-10 16:04:03 +08:00
wangmm0220 c381ffd38c fix:error in tmq for snapshot 2022-08-09 19:06:24 +08:00
Liu Jicong d9a4ea305d fix(stream): memory error 2022-08-09 14:11:41 +08:00
Liu Jicong c411b9aebb fix(stream): memory leak 2022-08-08 18:09:36 +08:00
Liu Jicong bb0ce39511 refactor(stream): add stream meta 2022-08-08 15:05:54 +08:00
wangmm0220 507adf9e7b fix:error in tmp for snapshot 2022-08-08 13:49:24 +08:00
wangmm0220 1309c489e2 fix:error in tmq for snapshot 2022-08-06 19:31:53 +08:00
wangmm0220 f28155714e fix:confilcts from 3.0 2022-08-06 12:31:15 +08:00
wangmm0220 45e7597df3 feat:get data from snapshot for taosx 2022-08-05 21:12:18 +08:00
Liu Jicong 7c9d57bd89 refactor(stream): unify sink and dispatch 2022-08-05 20:16:47 +08:00
Liu Jicong 6f8683de7e enh(tmq): retry consumer close 2022-08-04 20:37:02 +08:00
Liu Jicong 031414001d fix schedule 2022-08-04 17:46:13 +08:00
Liu Jicong e64e41679e fix schedule 2022-08-04 17:00:45 +08:00
Liu Jicong d6cc88ed4b refactor(stream): exec scheduler 2022-08-04 16:19:53 +08:00
wangmm0220 c3cd858a63 feat:add snapshot for tmq in stable and db 2022-08-04 15:01:59 +08:00
Liu Jicong c089d3ba08 merge 3.0 2022-08-02 22:53:49 +08:00
Liu Jicong d43eec5d16 enh(stream): directly dispatch 2022-08-02 22:23:33 +08:00
Shengliang Guan 56f2e49764 refactor: adjust logs 2022-08-02 17:12:07 +08:00
Shengliang Guan bd31c9da5a
Merge pull request #15657 from taosdata/fix/dnode
fix: write snapshot after apply queue is empty
2022-08-01 20:13:08 +08:00
Liu Jicong b084dd8ee6 feat(tq): transport snapshot 2022-08-01 19:12:37 +08:00
Shengliang Guan 99f29f6455 refactor: adjust log 2022-08-01 17:23:52 +08:00
Liu Jicong bb6fabcc11 refactor(stream): use vnode meta 2022-07-30 20:20:12 +08:00
Liu Jicong 6d67b17113 enh(wal): add lock to guarantee read behaviour 2022-07-30 16:35:49 +08:00
Liu Jicong a9d11b58ad enh(stream): recover 2022-07-28 16:19:50 +08:00
Liu Jicong 00acf4520c refactor(stream): remove option 2022-07-27 16:49:03 +08:00
wangmm0220 915d3c4203 fix:conflicts from 3.0 2022-07-26 17:47:10 +08:00
wangmm0220 ba1962cacf feat:add tmq get data interface 2022-07-26 17:43:22 +08:00
Liu Jicong c2164dd8aa fix(wal): snapshot end with no deleting 2022-07-26 14:13:03 +08:00
Liu Jicong 222e925644 feat(wal): ref 2022-07-25 16:40:19 +08:00
Shengliang Guan 5b654779ba Merge pull request #15372 from taosdata/fix/tsim
test: regression case
2022-07-25 15:23:41 +08:00
Liu Jicong 072fe10013
Merge pull request #15361 from taosdata/feature/tq
fix(tmq): correctly set reader status
2022-07-23 20:40:14 +08:00
Liu Jicong 095f6aa4e0 fix(tmq): correctly set reader status 2022-07-23 20:07:16 +08:00
plum-lihui 19acda27b2 test: add comment for tq 2022-07-23 13:32:25 +08:00
Liu Jicong e1046c7a98 fix: memory leak 2022-07-22 18:11:29 +08:00
Liu Jicong 2441ec22bd feat(tmq): check alter for ntb 2022-07-22 16:05:28 +08:00
Liu Jicong b1f3428c05 refactor(stream): do not merge output 2022-07-22 10:43:49 +08:00
Liu Jicong b49c61cfb4 merge 3.0 2022-07-21 20:44:08 +08:00
Haojun Liao 8a2fb51129
Merge pull request #15230 from taosdata/feature/3_liaohj
fix(query): add query involved column info
2022-07-21 20:29:25 +08:00
Liu Jicong eaf11ca1ac refactor(tmq): disable parallel 2022-07-21 20:15:52 +08:00
Haojun Liao c2d588a118 other:merge 3.0 2022-07-21 11:29:34 +08:00
Haojun Liao ee34c3bb3e fix(query): add query involved column info 2022-07-21 11:20:30 +08:00
Cary Xu 65871c2d73 Merge branch '3.0' into feature/TD-11274-3.0 2022-07-21 09:44:20 +08:00
Cary Xu 0f5983ad88 fix: merge dup rows in client 2022-07-20 20:30:54 +08:00
Liu Jicong fb52cfa816 fix(stream): memory error 2022-07-20 19:31:56 +08:00
Liu Jicong b74560537c fix(stream): check task exist 2022-07-20 16:25:20 +08:00
Haojun Liao 282630a394 Merge branch '3.0' into feature/3_liaohj 2022-07-19 17:42:45 +08:00
Haojun Liao cd2eaf0152 fix(query): set the output column number for subscribe. 2022-07-19 16:01:28 +08:00
54liuyao 8e41fc912b feat(stream): add num of children 2022-07-19 14:17:53 +08:00
Liu Jicong 500553d581 refactor: add debug log 2022-07-18 11:41:56 +08:00
Liu Jicong d75e9b8f6a merge from 3.0 2022-07-14 15:35:00 +08:00
Liu Jicong 9ea0ad57ec refactor(stream): destroy stream task 2022-07-14 15:32:21 +08:00
wangmm0220 cd051d40c3 fix:confilct from 3.0 2022-07-13 16:03:39 +08:00
Shengliang Guan 148e3a5f97
Merge pull request #14818 from taosdata/fix/mnode
enh: adjust vnode fetch queue number
2022-07-12 20:43:05 +08:00
Shengliang Guan dad12c24fb refactor: comment out to avoid crash of tmqShow.py 2022-07-12 19:56:38 +08:00
Liu Jicong 0fccdace56 fix(tmq): reset offset 2022-07-12 16:39:05 +08:00
Liu Jicong 7cf99d5359 refactor(tmq): prepare only needed 2022-07-12 15:11:14 +08:00
Liu Jicong d637ffe22c refactor(tmq): prepare only needed 2022-07-12 14:10:22 +08:00
wangmm0220 f9d5c809c9 fix:conflict from 3.0 2022-07-11 19:06:36 +08:00
Liu Jicong c6052fadba feat(wal): add append interface 2022-07-11 16:22:51 +08:00
wangmm0220 0a193cf811 feat:get meta ok 2022-07-11 16:12:07 +08:00
Liu Jicong 219d966b25 fix(wal): read after rolling 2022-07-10 21:02:25 +08:00
Liu Jicong 9189f886a9 feat(tmq): support consume from tsdb then wal 2022-07-10 16:34:45 +08:00
Liu Jicong bafa54778a refactor(stream) 2022-07-10 14:10:39 +08:00
Liu Jicong 2225411edc refactor(stream) 2022-07-10 13:57:17 +08:00
Liu Jicong e9077c036f merge from 3.0 2022-07-10 11:24:37 +08:00
Liu Jicong a2107992e2 refactor(stream) 2022-07-10 11:18:15 +08:00
Shengliang Guan c53a84b733 Merge remote-tracking branch 'origin/3.0' into fix/valgrind 2022-07-09 20:13:26 +08:00
Liu Jicong 0fd4f3b519 fix(tmq): invalid free 2022-07-08 23:05:32 +08:00
Liu Jicong 8d01ae00cb fix(tmq): offset 2022-07-08 21:37:15 +08:00
Liu Jicong a9e75c87aa refactor(stream): internal refactor 2022-07-08 19:30:40 +08:00
Shengliang Guan a5e8ddbc56 refactor: adjust logs 2022-07-08 18:00:19 +08:00
Liu Jicong 3cd5a35054 refactor(stream): internal refactor 2022-07-08 17:48:34 +08:00
Liu Jicong 866e4c4be0 refactor(stream): internal refactor 2022-07-07 17:56:52 +08:00
Liu Jicong 1ec57798d2 ci: remove unstable case 2022-07-07 11:45:28 +08:00
Liu Jicong 1e7d9ada19 fix(stream): clean up timer 2022-07-07 10:55:22 +08:00
Liu Jicong db23b07ae1 merge from 3.0 2022-07-06 21:12:46 +08:00
Liu Jicong 8c5dbde918 refactor(stream): stream reader created in scanner 2022-07-06 14:20:07 +08:00
Hongze Cheng ddb198de23 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/tsdb_refact 2022-07-04 03:42:50 +00:00
Liu Jicong 31bbef82d1 fix(tmq): seek snapshot offset 2022-07-02 20:18:40 +08:00
Liu Jicong 4a2b266612 feat(stream): support ignore expired flag 2022-07-02 10:59:29 +08:00
Liu Jicong 52d09b4173 refactor(wal) 2022-07-01 14:39:21 +08:00
Liu Jicong 7a65589525 feat(wal): add api to check log existance 2022-07-01 10:15:05 +08:00
Liu Jicong 4f107908db fix(tmq): reset window 2022-06-30 19:51:48 +08:00
Liu Jicong 94f07836d6 fix(tmq): set scan rsp offset correctly 2022-06-30 17:25:19 +08:00
Liu Jicong 4695dc2dae feat(tmq): add snapshot test 2022-06-30 14:41:50 +08:00
Liu Jicong 94c3eb29d2 fix(tmq): rsp error offset 2022-06-29 23:25:04 +08:00
Liu Jicong e74187248b fix(tmq): offset 2022-06-29 22:29:02 +08:00
Liu Jicong 0f0fe6276a merge from 3.0 2022-06-29 21:33:10 +08:00
Liu Jicong c2955a807b feat(tmq): offset support snapshot 2022-06-29 21:04:51 +08:00
Haojun Liao 4a59e1300b other: merge 3.0. 2022-06-28 15:22:32 +08:00
WANG MINGMING f0b50add27
Merge pull request #14184 from taosdata/feature/TD-13041
feat:sort table group if needed & optimize partition by tag
2022-06-25 09:41:30 +08:00
Liu Jicong f53521cb8c test(stream): partition by tbname 2022-06-24 16:50:11 +08:00
wangmm0220 0d9ed6d60a fix:merge from 3.0 2022-06-24 13:51:33 +08:00
Liu Jicong 323ee6b1b0 enh(tmq): add demo for tmq_get_raw_meta 2022-06-23 17:26:52 +08:00
Liu Jicong 14048a5e4c fix(tmq): read repeat data after reading snapshot 2022-06-23 17:09:06 +08:00
Liu Jicong 71bda7e658 feat(tmq): add with meta config 2022-06-23 15:56:54 +08:00
Liu Jicong 2af4a4d250 merge from 3.0 2022-06-23 14:15:33 +08:00
Liu Jicong 4ed47a0296 feat(tmq): scan tsdb as snapshot 2022-06-23 13:35:24 +08:00
Liu Jicong 14a488c92c feat(stream): stream support multiple type input 2022-06-22 21:21:39 +08:00
Liu Jicong fb596625d3 feat(stream): support retrieve 2022-06-22 19:12:31 +08:00
Liu Jicong 3f6ce2da73 feat(tmq): support topic with meta 2022-06-21 15:18:09 +08:00
Liu Jicong c409760f75 feat(stream): drop stream 2022-06-21 11:32:36 +08:00
Liu Jicong 450675f672 feat(stream): support snode 2022-06-20 20:18:34 +08:00
Liu Jicong 53ef66961a feat(stream): support snode 2022-06-20 14:29:18 +08:00
Hongze Cheng 21ea4e3a0b more work 2022-06-20 02:38:08 +00:00
Liu Jicong d7660484cc fix(tmq): performance issue 2022-06-17 15:10:00 +08:00
Liu Jicong 18993d50e5 refactor(tmq): offset storage 2022-06-17 13:16:46 +08:00
Liu Jicong 79494364ae refactor(tmq): offset storage 2022-06-16 22:00:33 +08:00
Liu Jicong 5572abe1b4 feat(stream): drop task 2022-06-15 21:41:59 +08:00
Liu Jicong 8eb08fa0d4 feat(stream): trigger mode add max delay 2022-06-15 16:11:20 +08:00
Liu Jicong 5881ef2690 refactor: convert datablock to submit block 2022-06-14 15:08:37 +08:00
Liu Jicong 601e454a24 enh(tmq): put offset store into vnode 2022-06-14 13:58:40 +08:00
Liu Jicong 0ef33c05bc enh(tmq): avoid uninitialized memory 2022-06-11 21:36:15 +08:00
Liu Jicong 6f4f6c14fa enh(tmq): add log 2022-06-11 14:02:48 +08:00
Liu Jicong 73b8f02ac7 fix(stream): set child id 2022-06-10 12:00:38 +08:00
Liu Jicong 9e4b3adc7c fix(stream): msg dispatch 2022-06-09 11:45:06 +08:00
Liu Jicong cbbf5f5d4d refactor(stream) 2022-06-07 21:35:49 +08:00
Liu Jicong 858868d76a refactor(stream): distributed execution 2022-06-07 10:54:18 +08:00
Liu Jicong 1b26da1bae refactor(stream): distributed execution 2022-06-06 22:18:50 +08:00
Liu Jicong 2677a3c8d9 refactor(stream) 2022-06-06 20:15:36 +08:00
Liu Jicong f2048b00cb fix(tmq): subscribe stb 2022-06-02 11:01:05 +08:00
Liu Jicong e1fd4a6640 feat(tmq): get_db api 2022-06-02 10:18:11 +08:00
Liu Jicong f3b680f630 refactor(tmq): push mode 2022-06-01 22:22:14 +08:00
Liu Jicong 135836c729 fix(tmq): tq deserialize msg 2022-06-01 19:57:03 +08:00
Liu Jicong e1a5000e50 refactor(tmq): tq sink and push 2022-06-01 17:26:15 +08:00
Liu Jicong 98f62165ae refactor(tmq): tq meta 2022-06-01 13:37:33 +08:00
Liu Jicong 66899bd8d0 fix(tmq): auto unsubscribe when consumer close 2022-06-01 10:27:41 +08:00
Liu Jicong 4169eb559a fix(tmq): skip log 2022-05-31 17:02:43 +08:00
Liu Jicong 451d19f986 feat(tmq): create topic new grammar 2022-05-31 16:20:40 +08:00
Liu Jicong 2248bc4207 Merge branch '3.0' into feature/topic_grammar 2022-05-30 23:31:22 +08:00
Liu Jicong 1aa97bc117 fix(tmq): update tq for auto create table 2022-05-28 19:48:17 +08:00
Liu Jicong 05a0774b6d feat(tmq): add sub type 2022-05-27 14:06:28 +08:00
Haojun Liao d87bb5f3fa refactor(query): set the block groupId for stream block. 2022-05-27 10:03:45 +08:00
Liu Jicong d3a2e98d60 fix(stream): double free 2022-05-26 20:52:27 +08:00
Liu Jicong a0f3138888 fix(stream): tb name error 2022-05-26 11:16:35 +08:00
Liu Jicong a9c54ae701 enh(tmq): support restart 2022-05-25 17:22:13 +08:00
Liu Jicong 2de2eafaa8 feat(tmq): use tdb as persistent storage 2022-05-24 20:35:37 +08:00
Liu Jicong 0ad9c4cebd fix: init once 2022-05-23 20:58:40 +08:00
Liu Jicong 3629958b43 feat(tmq): serializer and deserializer for tq exec 2022-05-23 20:31:35 +08:00
Liu Jicong 930b850035 refacotr: remove tq meta store 2022-05-23 19:51:40 +08:00
Liu Jicong a393dee9d2 refactor(stream) 2022-05-23 16:51:12 +08:00
Liu Jicong 30671eaadf fix(tmq): skip dropped table for db subscription 2022-05-22 20:35:57 +08:00
Liu Jicong 8e3701df01 fix(tmq): memory leak 2022-05-21 21:36:26 +08:00
Liu Jicong d8479d7a0d merge from 3.0 2022-05-20 18:55:40 +08:00
Liu Jicong 7391a756f6 refactor(stream) 2022-05-20 18:53:21 +08:00
yihaoDeng afda7c637b fix: tq mem leak 2022-05-20 15:58:25 +08:00
Liu Jicong 71984b4268 refactor(stream): exec 2022-05-20 05:43:36 +08:00
Liu Jicong 2174b5e48e enh(tq): update tb uid when droping table 2022-05-19 16:46:12 +08:00
Liu Jicong 2f9c63ae3f enh(tq): update tb uid when droping table 2022-05-19 15:37:30 +08:00
Liu Jicong 2216438c72 fix(stream): skip update tb uid for db topic 2022-05-19 10:26:42 +08:00
Liu Jicong a696ed583a test(stream): _wstartts should be reverse quoted 2022-05-18 21:23:06 +08:00
Liu Jicong 4e640b6d67 enh(tmq): update tb list for existing topic 2022-05-18 16:27:50 +08:00
Liu Jicong 8ceae5854d fix(tmq): fix memory leak 2022-05-17 17:01:07 +08:00
Liu Jicong 2aec989bd7 enh(tmq): add drop msg to vnode 2022-05-17 05:13:27 +08:00
Liu Jicong a2ee1c0bf6 enh(tmq): add drop msg to vnode 2022-05-17 04:29:17 +08:00
Liu Jicong b3fea32a7e refactor(stream) 2022-05-16 22:43:04 +08:00
Liu Jicong f77f910264 refactor(stream) 2022-05-16 02:11:47 +08:00
Cary Xu 069ce051f0 feat: move sma module from tsdb to vnode 2022-05-15 20:17:20 +08:00
Liu Jicong a6015d2f39 refactor(stream) 2022-05-15 08:14:23 +08:00
Liu Jicong d7e34a6642 fix(tmq): set config 2022-05-13 18:39:37 +08:00
liuyao 426200a71b
Merge pull request #12384 from taosdata/feature/TD-15379
feat(stream): stream reprocess window
2022-05-13 09:19:22 +08:00
Liu Jicong acea83e535 fix(wal): wal read error 2022-05-13 01:22:29 +08:00
54liuyao 42f27e99d0 steam reprocess window 2022-05-12 22:15:13 +08:00
Liu Jicong 88d1ea6222
Merge pull request #12378 from taosdata/feature/tq
feat(tmq): add config msg.with.table.name
2022-05-12 17:00:13 +08:00
Liu Jicong 06853043bd feat(tmq): add config msg.with.table.name 2022-05-12 14:57:58 +08:00
Liu Jicong f1d15f15cf
Merge pull request #12366 from taosdata/feature/tq
feat(tmq): support get tb name
2022-05-12 12:07:36 +08:00
Liu Jicong 7ab2e76c35 feat(tmq): support get tb name 2022-05-12 11:31:56 +08:00
Hongze Cheng 97e74a811f
Merge pull request #12338 from taosdata/feature/data_format
fix: database options
2022-05-11 16:57:33 +08:00
Liu Jicong 714dd4f497 enh(tmq): only read committed log 2022-05-11 14:42:20 +08:00
Hongze Cheng ac27d62f33 refact data code 2022-05-11 02:29:53 +00:00
Liu Jicong 951d102a99
Merge pull request #12313 from taosdata/feature/tq
feat(tmq): change default config
2022-05-10 18:09:37 +08:00
Liu Jicong a6b258a45f feat(tmq): change default config 2022-05-10 17:12:11 +08:00
Shengliang Guan 5f33d88cb2 refactor: remove rpc client in executor and scanoperator 2022-05-09 17:03:44 +08:00
Shengliang Guan 0dfac36f5e Merge remote-tracking branch 'origin/3.0' into feature/dnode 2022-05-09 16:40:16 +08:00
Liu Jicong 68b9424f8a test(stream): more stream case 2022-05-08 11:58:19 +08:00
Liu Jicong d17b478e96 feat(stream): auto create ctb 2022-05-08 02:37:23 +08:00
Liu Jicong 3ed1ab6acb enh(stream): auto create ctb 2022-05-07 23:19:05 +08:00
Shengliang Guan 963813b5e6 Merge remote-tracking branch 'origin/3.0' into feature/dnode 2022-05-07 20:19:54 +08:00
Shengliang Guan c8865f3a05 refactor: remove rpc client in executor and scanoperator 2022-05-07 20:19:40 +08:00
Hongze Cheng 323ce214b6 refact encode APIs 2022-05-07 10:03:06 +00:00
Haojun Liao 2b803d8b93 enh(query): set uid for the result datablock of stream scanner when handling the submit block. 2022-05-06 23:40:26 +08:00
Liu Jicong a5706ea7e5 feat(stream): insert data 2022-05-05 18:57:09 +08:00
Liu Jicong de939d589f fix: memory error 2022-04-29 20:11:58 +08:00
Liu Jicong 3ae3fd65e8 fix: memory error 2022-04-28 16:31:35 +08:00
Liu Jicong 7b892a0f28 fix: client memory leak 2022-04-26 21:50:36 +08:00
Liu Jicong 84b19fbea2
Merge pull request #11908 from taosdata/feature/tq
enh(tmq): use sem_timewait to reduce busy loop
2022-04-26 19:47:23 +08:00
Liu Jicong 9723145c1d enh(tmq): use sem_timewait to reduce busy loop 2022-04-26 19:21:31 +08:00
Hongze Cheng 42c6f01980 refact vnode includes 2022-04-26 11:04:26 +00:00