Commit Graph

383 Commits

Author SHA1 Message Date
wangmm0220 604ae220f3 fix:change datablock to old version for compatibility 2024-02-22 17:48:33 +08:00
wangmm0220 6e13e4aa63 fix:add excluded msg for meta in tmq 2024-02-22 10:58:32 +08:00
wangmm0220 77e2500a81 fix:conflicts from 3.0 2024-02-19 18:46:31 +08:00
wangmm0220 478e1a67ae fix:[TD-28590]add logic for consume excluded 2024-02-06 18:14:06 +08:00
wangmm0220 871ebfa7ba Merge branch '3.0' of https://github.com/taosdata/TDengine into mark/3.0 2024-02-06 14:02:46 +08:00
wangmm0220 5796f411e4 Merge branch 'mark/3.0' of https://github.com/taosdata/TDengine into mark/3.0 2024-02-05 17:08:05 +08:00
wangmm0220 a656d75ca7 opti:consume data excluded from some source 2024-02-05 17:07:50 +08:00
wangmm0220 d369289ea2 fix:[TD-28514] memory leak 2024-02-01 16:34:05 +08:00
wangmm0220 bd2bef2428 fix:add test case 2024-01-26 15:05:31 +08:00
wangmm0220 2798ff824c feat:[TD-28247]add grant for subscribe and stream 2024-01-25 16:07:09 +08:00
wangmm0220 1b85adaa3a opti:[TD-28118] raw block data for tmq 2024-01-18 14:17:08 +08:00
wangmm0220 d5d1dd9127 opti:[TD-28118] raw block data for tmq 2024-01-17 20:08:26 +08:00
wangmm0220 d3659db2ff fix:[TD-28202]move tq timer to write thread in mnode 2024-01-11 19:22:54 +08:00
wangmm0220 53ce344b36 fix:set error msg if build consumer error 2023-12-28 17:09:52 +08:00
wangmm0220 e1eda752e7 fix:[TS-4242] tmq status is always RECOVER 2023-11-23 20:44:10 +08:00
yihaoDeng 55e66eac3c fix:print log if assert & check epoch when update epoch 2023-10-31 20:47:08 +08:00
wangmm0220 e92460105c fix:print log if assert & check epoch when update epoch 2023-10-31 15:36:32 +08:00
wangmm0220 770c88200f fix:print log if assert & check epoch when update epoch 2023-10-30 09:19:31 +08:00
wangmm0220 473142fab7 fix:conflicts from 3.0 2023-10-16 11:35:56 +08:00
Haojun Liao cdd480d186
Merge pull request #23249 from taosdata/feat/TD-26052
fix:core dump if tmq is null
2023-10-13 16:50:48 +08:00
wangmm0220 9cfb15d4f4 fix:core dump if tmq is null 2023-10-13 14:04:00 +08:00
Haojun Liao eb4449999f other: merge 3.0 2023-10-12 10:33:48 +08:00
wangmm0220 33045e63ae feat:[TD-26056] add replay logic 2023-10-09 17:35:40 +08:00
wangmm0220 5f7b6f19ba feat:[TD-26056] add replay logic 2023-10-08 19:05:59 +08:00
wangmm0220 3f2d890560 feat:[TD-26056] add replay logic 2023-10-08 16:06:15 +08:00
wangmm0220 9c470e9297 opti:set default offset to latest in subscription & return error if use snapshot when consume column 2023-10-07 15:25:24 +08:00
Haojun Liao 68c329e1f7
Merge pull request #23015 from taosdata/mark/tmq-3.0
fix:add interface tmq_get_connect to get TAOS from tmq
2023-09-22 14:02:19 +08:00
wangmm0220 63a6aacc18 fix:add interface tmq_get_connect to get TAOS from tmq 2023-09-22 11:44:37 +08:00
wangmm0220 70b5509217 fix:enable split if there are topics 2023-09-21 18:16:05 +08:00
wangmm0220 06afd0bd33 fix:add interface tmq_get_connect to get TAOS from tmq 2023-09-21 15:13:38 +08:00
wangmm0220 7869b693f2 add test case for tmq in vnode transform & fix core in race condition for pTq->pExecStore 2023-09-18 14:09:28 +08:00
wangmm0220 ef81299494 fix:heap use after free 2023-09-14 13:41:14 +08:00
wangmm0220 a833d5d769 fix:need update epset if unsubscribe topic and topicNumGet is 0 at the same time 2023-09-14 10:39:22 +08:00
wangmm0220 f729150497 fix:opti poll callbak logic & send epoch as -1 to get epset if vnode transformed 2023-09-13 18:32:45 +08:00
wangmm0220 d707f69a6c opti:ask ep logic in tmq 2023-09-13 09:52:24 +08:00
wangmm0220 f4ec83025d opti:optimize code logic & fix python test case error 2023-09-06 19:46:17 +08:00
wangmm0220 629de12bb1 fix:do not return error if commit nothing 2023-09-06 18:29:28 +08:00
wangmm0220 cc62961337 fix:heap use after free of pParamSet since the first commitCb is called before send another commit message 2023-09-06 14:45:07 +08:00
wangmm0220 897fd5b8b5 fix:tmq close error because of same committed offset 2023-09-06 10:15:41 +08:00
wangmm0220 53c4f4a147 opti:commit logic 2023-09-05 10:35:05 +08:00
wangmm0220 9a2da3adee fix:set vg status idle if reveive poll callback 2023-09-04 18:31:07 +08:00
wangmm0220 90e938e48f fix:set vg status idle if reveive poll callback 2023-09-04 16:53:12 +08:00
wangmm0220 f0fbe08e1d fix:core dump if tmq is null 2023-09-01 16:45:46 +08:00
wangmm0220 b8f041f5ea fix:use vgstatus before if rebalance 2023-09-01 10:05:46 +08:00
wangmm0220 ac6d031e62 fix:do not send unsubscribe msg to taosd if subscribe failed 2023-08-21 17:56:35 +08:00
wangmm0220 78fd70202c fix:offset error in tmq & add test cases 2023-08-10 16:18:53 +08:00
wade zhang bae028d65c
Merge pull request #22230 from taosdata/mark/tmq
fix:semaphore always wait in tmq commit logic
2023-07-31 09:10:13 +08:00
wangmm0220 6e673072cd fix:conflicts 2023-07-28 14:57:13 +08:00
Haojun Liao 3d403dca5a other: merge main. 2023-07-27 16:23:41 +08:00
wangmm0220 75eacd56f1 fix:add log 2023-07-27 15:58:32 +08:00
wangmm0220 2d41a42976 Merge branch 'main' of https://github.com/taosdata/TDengine into fix/TS-3728 2023-07-27 15:31:16 +08:00
wangmm0220 5fa580960e fix:semaphore always wait in tmq commit logic 2023-07-27 13:53:42 +08:00
wangmm0220 68eb1cfc1e fix:semaphore always wait in tmq commit logic 2023-07-27 10:03:18 +08:00
wangmm0220 11e690cb67 fix:semaphore always wait in tmq commit logic 2023-07-27 09:57:52 +08:00
wangmm0220 be4e47e9a7 fix:add macro ASSERT_NOT_CORE to control ASSERT not core in release version 2023-07-25 16:15:33 +08:00
Haojun Liao c2588d71c3
Merge pull request #22173 from taosdata/fix/TS-3728
fix:avoid request offset type is 0
2023-07-25 14:59:30 +08:00
wangmm0220 6a0e994169 fix:avoid request offset type is 0 2023-07-24 19:24:05 +08:00
wangmm0220 9628a9f74e fix:open info log in tmq & ignore wal apply ver when read wal 2023-07-24 15:53:02 +08:00
wangmm0220 99c3ebc282 fix:optimize log & change return value for async interface 2023-07-20 18:28:21 +08:00
wangmm0220 e1c4cca33d feat:add committed & position & commite_offset interface 2023-07-20 00:07:04 +08:00
wangmm0220 5cb35f2fa6 feat:add committed & position & commite_offset interface 2023-07-19 17:29:39 +08:00
wangmm0220 5c344b0159 fix:conflicts from main 2023-07-19 09:05:05 +08:00
wangmm0220 e812a659a8 fix:add tmq_position() interface & optimize commit logic 2023-07-18 19:30:40 +08:00
plum-lihui 745ab09b1b test: add tmq seek case 2023-07-18 14:30:19 +08:00
wangmm0220 bbdcbcb75b fix:modify commit version to next validate version 2023-07-14 16:43:28 +08:00
Haojun Liao f41bcd73b0 other: merge main. 2023-07-13 11:02:09 +08:00
wangmm0220 cf64d4c9c5 fix:set get_assignment offset to first version of response block 2023-07-12 17:22:23 +08:00
wangmm0220 8dd7f36993 fix:set firset version to reqOffset of response 2023-07-11 16:19:38 +08:00
wangmm0220 574010e067 fix:set firset version to reqOffset of response 2023-07-11 16:06:19 +08:00
wangmm0220 cb62d4cb97 fix:set firset version to reqOffset of response 2023-07-11 14:49:53 +08:00
Haojun Liao 27eabb3da9 fix(tmq): fix tmq syntax error. 2023-07-11 12:05:49 +08:00
Haojun Liao 3ffdbe923d other: merge 3.0 2023-07-11 09:12:57 +08:00
wangmm0220 31a8af9e50 fix:do not commit offset if seek offset 2023-07-10 19:12:50 +08:00
wangmm0220 8e011c46c9 fix:can not do assignment if in tsdb mode 2023-07-10 15:07:20 +08:00
wangmm0220 dabaefe3e2 Merge branch 'main' of https://github.com/taosdata/TDengine into mark/tmq 2023-07-10 14:47:10 +08:00
wangmm0220 9648117454 fix:can not do assignment if in tsdb mode 2023-07-10 11:28:30 +08:00
Haojun Liao 516fcc53ad
Merge pull request #21943 from taosdata/fix/TD-25071
fix:[TD-25071] pSub is null & return wal not exist if no data
2023-07-10 10:21:36 +08:00
wangmm0220 8904f3857b fix:seek failed in initilized status 2023-07-09 19:16:19 +08:00
wangmm0220 ee50ed4847 fix:seek failed in initilized statue 2023-07-09 15:31:19 +08:00
wangmm0220 35ba132a29 fix:conflicts from main 2023-07-09 12:52:18 +08:00
wangmm0220 0929be690d fix:seek to offset - 1 2023-07-09 11:54:53 +08:00
wangmm0220 3ad486e95b fix:seek failed if type is earliest 2023-07-09 11:34:49 +08:00
wangmm0220 4436eb7e0f fix:set commitOffset is send msg success & optimize log & add test cases for TD-25129 2023-07-07 19:12:18 +08:00
wangmm0220 106e273157 fix:bugs in get_assignment 2023-07-07 16:12:15 +08:00
wangmm0220 0928bd5510 fix:send rsp offset to client if unregister push mgr 2023-07-06 19:35:48 +08:00
wangmm0220 b2d41ce907 Merge branch 'main' of https://github.com/taosdata/TDengine into fix/TD-25071 2023-07-06 19:34:22 +08:00
wangmm0220 8e29de0c4d Merge branch 'mark/tmq' of https://github.com/taosdata/TDengine into mark/tmq 2023-07-06 17:03:15 +08:00
wangmm0220 ba9ab1f20b fix:null pointer error 2023-07-06 17:02:58 +08:00
wangmm0220 47ae92af93 fix:conflicts from main 2023-07-06 13:40:04 +08:00
wangmm0220 11e6856d3e fix:init char array 2023-07-06 13:37:58 +08:00
wangmm0220 34a62c3f38 fix:vginfo assigment not in same time,we should get vginfo by vg,not all vg 2023-07-05 22:55:41 +08:00
wangmm0220 bdfeb32923 fix:bugs in tmq_get_topic_assignment 2023-07-05 16:04:34 +08:00
wangmm0220 6b5daf6c9f fix:add lock for tmq->clientTopic 2023-07-04 14:59:36 +08:00
Haojun Liao 82342ebdc7
Merge pull request #21934 from taosdata/feature/TD-20678
feat:subscribe only meta info
2023-07-04 11:28:27 +08:00
wangmm0220 58c7011e0e feat:support only meta in tmq subscriptions 2023-06-30 11:42:40 +08:00
dapan1121 c8435de64a
Merge branch '3.0' into fix/m23.0 2023-06-29 14:03:56 +08:00
dapan1121 3519e9170e Merge remote-tracking branch 'origin/main' into 3.0 2023-06-29 11:13:45 +08:00
wangmm0220 e8d53c2a96 fix:add log 2023-06-27 10:50:57 +08:00
wangmm0220 7ebbe0603b fix:coverity scan error 2023-06-26 18:26:10 +08:00
wangmm0220 5b88dfb90b fix:remove report offset flag 2023-06-25 15:24:52 +08:00
wangmm0220 610aa0c8ba fix:modify log 2023-06-25 15:14:34 +08:00
wangmm0220 d3dedbe676 fix:report current offset in tmq 2023-06-25 14:52:45 +08:00
wangmm0220 c28d7d9bf8 fix:open info log in tmq client & modify parametes in show consumers 2023-06-14 12:01:54 +08:00
wangmm0220 fe155898fe fix:modify offset description to string 2023-06-12 16:34:53 +08:00
wangmm0220 5c5e76f894 fix:modify offset description to string 2023-06-12 11:17:32 +08:00
wangmm0220 ae85e5e0f6 fix:modify offset description to string 2023-06-12 10:29:33 +08:00
wangmm0220 6657b8d849 fix:modify offset description to string 2023-06-12 09:44:00 +08:00
wangmm0220 aa610b27a4 feat:add parameters for consumer & add offset rows for subscription 2023-06-09 18:51:10 +08:00
wangmm0220 3f46a9bbc9 feat:add parameters for consumer & add offset rows for subscription 2023-06-08 18:14:02 +08:00
t_max 163f5abc4c fix: no data after seek 2023-06-02 17:22:49 +08:00
t_max 27d8870e18 fix: no data after seek 2023-06-02 17:09:03 +08:00
Haojun Liao 5301c5756a other: merge main into 3.0. 2023-06-01 09:29:18 +08:00
Haojun Liao 1e845acabe fix(tmq): allow seek before get assignment 2023-05-26 11:34:12 +08:00
t_max 2aad823709 fix: free assignment crash on Windows 2023-05-22 10:53:10 +08:00
wangmm0220 8183be4aff fix:move sleep logic from rpc thread to tmq thread & fix some error 2023-05-20 16:13:52 +08:00
wangmm0220 bca1af3f5d fix:change tmq subscribe try time to 60mins 2023-05-18 21:18:44 +08:00
wangmm0220 50bdfef7b6 fix:tmq ref err & vg error 2023-05-18 18:34:42 +08:00
wangmm0220 7db9f1e58e fix:tmq ref err & vg error 2023-05-18 18:21:45 +08:00
wangmm0220 c7e6883ac8 fix:tmq ref err & vg error 2023-05-18 18:10:33 +08:00
Haojun Liao a4d9210b07 fix(tmq): add filter for taosX poll rsp. 2023-05-17 16:40:44 +08:00
Haojun Liao f55f8d245c fix(tmq): avoid the offset is revised by pool rsp which is issued before invoking tmq_offset_seek function. 2023-05-17 15:19:51 +08:00
Haojun Liao 573a86ed99 feature(tmq): add new API to extract offset from result set. 2023-05-16 09:51:00 +08:00
Haojun Liao 7ef16e1922 other: merge main. 2023-05-12 11:49:53 +08:00
Haojun Liao c6e62d7fb8 fix(tmq): update the local offset when retrieving offset from vnode. 2023-05-11 17:40:17 +08:00
Haojun Liao b16f26a12f fix(tmq): add some logs. 2023-05-11 11:56:17 +08:00
Haojun Liao 09a04052ec refactor: do some internal refactor. 2023-05-04 16:15:14 +08:00
wangmm0220 8d84e8f8a5 fix:[TD-23788] client wait if task status error in taosx transform data 2023-04-28 19:29:17 +08:00
wangmm0220 cce868d140 fix:offset encode assert error 2023-04-28 09:45:16 +08:00
wangmm0220 db5b5c828e fix:add log for msg push 2023-04-27 18:35:10 +08:00
wangmm0220 6a889ae80e Merge branch 'main' of https://github.com/taosdata/TDengine into fix/TS-3222 2023-04-27 09:48:04 +08:00
wangmm0220 224d87b131 fix:cosume null if rebalance 2023-04-26 17:58:14 +08:00
Haojun Liao 2a45ef88b0 fix(query): fix memory leak. 2023-04-26 00:40:33 +08:00
wangmm0220 d1e5d6e0f9 fix:pHandle->msg is not null if rebalance 2023-04-25 16:23:58 +08:00
wangmm0220 1c63408b3e opti:change push mgr to consume msg for subscribe 2023-04-24 20:18:20 +08:00
wangmm0220 8572f4a32f opti:change push mgr to consume msg for subscribe 2023-04-24 19:41:30 +08:00
Haojun Liao 62dad38d5c refactor: do some internal refactor and add the sample code. 2023-04-24 15:25:49 +08:00
Haojun Liao 9cf89c6b6f enh(tmq): add check for consumer Id; 2023-04-23 11:51:43 +08:00
Haojun Liao 232d8f1fda
Merge pull request #21012 from taosdata/fix/TD-23714
fix: topic name supports capital
2023-04-23 09:30:19 +08:00
Haojun Liao 3dbe208712 enh(tmq): fix memory error and add retrieve wal info as required. 2023-04-22 15:34:46 +08:00
wangmm0220 d288e44f92 fix:ci error 2023-04-21 17:18:40 +08:00
wangmm0220 7ea860ad11 fix:[TD-23714] topic support capital 2023-04-21 17:14:41 +08:00
wangmm0220 b1561c95c3 fix:add test cases for escape in schemaless 2023-04-20 15:31:18 +08:00
Haojun Liao c654f1145d enh(stream): add new msg for seek, and do some internal refactor. 2023-04-20 10:59:16 +08:00
Haojun Liao 95346cd4d8 test:update the test cases. 2023-04-19 15:17:46 +08:00
Haojun Liao d5c57ca244 enh(tmq): support the seek. 2023-04-19 14:46:48 +08:00
Haojun Liao 3f12156c53 enh(tmq): support seek in tmq. 2023-04-19 13:56:42 +08:00
wangmm0220 5b3087e48b opti:escape logic in schemaless 2023-04-19 00:55:26 +08:00
Haojun Liao 9627e8105b enh(tmq): support seek offset. 2023-04-18 13:52:47 +08:00
Haojun Liao f8b672f09f other: merge main. 2023-04-14 09:07:18 +08:00
Xiaoyu Wang 4c04b155d5 merge main 2023-04-11 15:56:28 +08:00