Commit Graph

383 Commits

Author SHA1 Message Date
wangmm0220 2d41a42976 Merge branch 'main' of https://github.com/taosdata/TDengine into fix/TS-3728 2023-07-27 15:31:16 +08:00
wangmm0220 5fa580960e fix:semaphore always wait in tmq commit logic 2023-07-27 13:53:42 +08:00
wangmm0220 68eb1cfc1e fix:semaphore always wait in tmq commit logic 2023-07-27 10:03:18 +08:00
wangmm0220 11e690cb67 fix:semaphore always wait in tmq commit logic 2023-07-27 09:57:52 +08:00
wangmm0220 be4e47e9a7 fix:add macro ASSERT_NOT_CORE to control ASSERT not core in release version 2023-07-25 16:15:33 +08:00
Haojun Liao c2588d71c3
Merge pull request #22173 from taosdata/fix/TS-3728
fix:avoid request offset type is 0
2023-07-25 14:59:30 +08:00
wangmm0220 6a0e994169 fix:avoid request offset type is 0 2023-07-24 19:24:05 +08:00
wangmm0220 9628a9f74e fix:open info log in tmq & ignore wal apply ver when read wal 2023-07-24 15:53:02 +08:00
wangmm0220 99c3ebc282 fix:optimize log & change return value for async interface 2023-07-20 18:28:21 +08:00
wangmm0220 e1c4cca33d feat:add committed & position & commite_offset interface 2023-07-20 00:07:04 +08:00
wangmm0220 5cb35f2fa6 feat:add committed & position & commite_offset interface 2023-07-19 17:29:39 +08:00
wangmm0220 5c344b0159 fix:conflicts from main 2023-07-19 09:05:05 +08:00
wangmm0220 e812a659a8 fix:add tmq_position() interface & optimize commit logic 2023-07-18 19:30:40 +08:00
plum-lihui 745ab09b1b test: add tmq seek case 2023-07-18 14:30:19 +08:00
wangmm0220 bbdcbcb75b fix:modify commit version to next validate version 2023-07-14 16:43:28 +08:00
Haojun Liao f41bcd73b0 other: merge main. 2023-07-13 11:02:09 +08:00
wangmm0220 cf64d4c9c5 fix:set get_assignment offset to first version of response block 2023-07-12 17:22:23 +08:00
wangmm0220 8dd7f36993 fix:set firset version to reqOffset of response 2023-07-11 16:19:38 +08:00
wangmm0220 574010e067 fix:set firset version to reqOffset of response 2023-07-11 16:06:19 +08:00
wangmm0220 cb62d4cb97 fix:set firset version to reqOffset of response 2023-07-11 14:49:53 +08:00
Haojun Liao 27eabb3da9 fix(tmq): fix tmq syntax error. 2023-07-11 12:05:49 +08:00
Haojun Liao 3ffdbe923d other: merge 3.0 2023-07-11 09:12:57 +08:00
wangmm0220 31a8af9e50 fix:do not commit offset if seek offset 2023-07-10 19:12:50 +08:00
wangmm0220 8e011c46c9 fix:can not do assignment if in tsdb mode 2023-07-10 15:07:20 +08:00
wangmm0220 dabaefe3e2 Merge branch 'main' of https://github.com/taosdata/TDengine into mark/tmq 2023-07-10 14:47:10 +08:00
wangmm0220 9648117454 fix:can not do assignment if in tsdb mode 2023-07-10 11:28:30 +08:00
Haojun Liao 516fcc53ad
Merge pull request #21943 from taosdata/fix/TD-25071
fix:[TD-25071] pSub is null & return wal not exist if no data
2023-07-10 10:21:36 +08:00
wangmm0220 8904f3857b fix:seek failed in initilized status 2023-07-09 19:16:19 +08:00
wangmm0220 ee50ed4847 fix:seek failed in initilized statue 2023-07-09 15:31:19 +08:00
wangmm0220 35ba132a29 fix:conflicts from main 2023-07-09 12:52:18 +08:00
wangmm0220 0929be690d fix:seek to offset - 1 2023-07-09 11:54:53 +08:00
wangmm0220 3ad486e95b fix:seek failed if type is earliest 2023-07-09 11:34:49 +08:00
wangmm0220 4436eb7e0f fix:set commitOffset is send msg success & optimize log & add test cases for TD-25129 2023-07-07 19:12:18 +08:00
wangmm0220 106e273157 fix:bugs in get_assignment 2023-07-07 16:12:15 +08:00
wangmm0220 0928bd5510 fix:send rsp offset to client if unregister push mgr 2023-07-06 19:35:48 +08:00
wangmm0220 b2d41ce907 Merge branch 'main' of https://github.com/taosdata/TDengine into fix/TD-25071 2023-07-06 19:34:22 +08:00
wangmm0220 8e29de0c4d Merge branch 'mark/tmq' of https://github.com/taosdata/TDengine into mark/tmq 2023-07-06 17:03:15 +08:00
wangmm0220 ba9ab1f20b fix:null pointer error 2023-07-06 17:02:58 +08:00
wangmm0220 47ae92af93 fix:conflicts from main 2023-07-06 13:40:04 +08:00
wangmm0220 11e6856d3e fix:init char array 2023-07-06 13:37:58 +08:00
wangmm0220 34a62c3f38 fix:vginfo assigment not in same time,we should get vginfo by vg,not all vg 2023-07-05 22:55:41 +08:00
wangmm0220 bdfeb32923 fix:bugs in tmq_get_topic_assignment 2023-07-05 16:04:34 +08:00
wangmm0220 6b5daf6c9f fix:add lock for tmq->clientTopic 2023-07-04 14:59:36 +08:00
Haojun Liao 82342ebdc7
Merge pull request #21934 from taosdata/feature/TD-20678
feat:subscribe only meta info
2023-07-04 11:28:27 +08:00
wangmm0220 58c7011e0e feat:support only meta in tmq subscriptions 2023-06-30 11:42:40 +08:00
dapan1121 c8435de64a
Merge branch '3.0' into fix/m23.0 2023-06-29 14:03:56 +08:00
dapan1121 3519e9170e Merge remote-tracking branch 'origin/main' into 3.0 2023-06-29 11:13:45 +08:00
wangmm0220 e8d53c2a96 fix:add log 2023-06-27 10:50:57 +08:00
wangmm0220 7ebbe0603b fix:coverity scan error 2023-06-26 18:26:10 +08:00
wangmm0220 5b88dfb90b fix:remove report offset flag 2023-06-25 15:24:52 +08:00
wangmm0220 610aa0c8ba fix:modify log 2023-06-25 15:14:34 +08:00
wangmm0220 d3dedbe676 fix:report current offset in tmq 2023-06-25 14:52:45 +08:00
wangmm0220 c28d7d9bf8 fix:open info log in tmq client & modify parametes in show consumers 2023-06-14 12:01:54 +08:00
wangmm0220 fe155898fe fix:modify offset description to string 2023-06-12 16:34:53 +08:00
wangmm0220 5c5e76f894 fix:modify offset description to string 2023-06-12 11:17:32 +08:00
wangmm0220 ae85e5e0f6 fix:modify offset description to string 2023-06-12 10:29:33 +08:00
wangmm0220 6657b8d849 fix:modify offset description to string 2023-06-12 09:44:00 +08:00
wangmm0220 aa610b27a4 feat:add parameters for consumer & add offset rows for subscription 2023-06-09 18:51:10 +08:00
wangmm0220 3f46a9bbc9 feat:add parameters for consumer & add offset rows for subscription 2023-06-08 18:14:02 +08:00
t_max 163f5abc4c fix: no data after seek 2023-06-02 17:22:49 +08:00
t_max 27d8870e18 fix: no data after seek 2023-06-02 17:09:03 +08:00
Haojun Liao 5301c5756a other: merge main into 3.0. 2023-06-01 09:29:18 +08:00
Haojun Liao 1e845acabe fix(tmq): allow seek before get assignment 2023-05-26 11:34:12 +08:00
t_max 2aad823709 fix: free assignment crash on Windows 2023-05-22 10:53:10 +08:00
wangmm0220 8183be4aff fix:move sleep logic from rpc thread to tmq thread & fix some error 2023-05-20 16:13:52 +08:00
wangmm0220 bca1af3f5d fix:change tmq subscribe try time to 60mins 2023-05-18 21:18:44 +08:00
wangmm0220 50bdfef7b6 fix:tmq ref err & vg error 2023-05-18 18:34:42 +08:00
wangmm0220 7db9f1e58e fix:tmq ref err & vg error 2023-05-18 18:21:45 +08:00
wangmm0220 c7e6883ac8 fix:tmq ref err & vg error 2023-05-18 18:10:33 +08:00
Haojun Liao a4d9210b07 fix(tmq): add filter for taosX poll rsp. 2023-05-17 16:40:44 +08:00
Haojun Liao f55f8d245c fix(tmq): avoid the offset is revised by pool rsp which is issued before invoking tmq_offset_seek function. 2023-05-17 15:19:51 +08:00
Haojun Liao 573a86ed99 feature(tmq): add new API to extract offset from result set. 2023-05-16 09:51:00 +08:00
Haojun Liao 7ef16e1922 other: merge main. 2023-05-12 11:49:53 +08:00
Haojun Liao c6e62d7fb8 fix(tmq): update the local offset when retrieving offset from vnode. 2023-05-11 17:40:17 +08:00
Haojun Liao b16f26a12f fix(tmq): add some logs. 2023-05-11 11:56:17 +08:00
Haojun Liao 09a04052ec refactor: do some internal refactor. 2023-05-04 16:15:14 +08:00
wangmm0220 8d84e8f8a5 fix:[TD-23788] client wait if task status error in taosx transform data 2023-04-28 19:29:17 +08:00
wangmm0220 cce868d140 fix:offset encode assert error 2023-04-28 09:45:16 +08:00
wangmm0220 db5b5c828e fix:add log for msg push 2023-04-27 18:35:10 +08:00
wangmm0220 6a889ae80e Merge branch 'main' of https://github.com/taosdata/TDengine into fix/TS-3222 2023-04-27 09:48:04 +08:00
wangmm0220 224d87b131 fix:cosume null if rebalance 2023-04-26 17:58:14 +08:00
Haojun Liao 2a45ef88b0 fix(query): fix memory leak. 2023-04-26 00:40:33 +08:00
wangmm0220 d1e5d6e0f9 fix:pHandle->msg is not null if rebalance 2023-04-25 16:23:58 +08:00
wangmm0220 1c63408b3e opti:change push mgr to consume msg for subscribe 2023-04-24 20:18:20 +08:00
wangmm0220 8572f4a32f opti:change push mgr to consume msg for subscribe 2023-04-24 19:41:30 +08:00
Haojun Liao 62dad38d5c refactor: do some internal refactor and add the sample code. 2023-04-24 15:25:49 +08:00
Haojun Liao 9cf89c6b6f enh(tmq): add check for consumer Id; 2023-04-23 11:51:43 +08:00
Haojun Liao 232d8f1fda
Merge pull request #21012 from taosdata/fix/TD-23714
fix: topic name supports capital
2023-04-23 09:30:19 +08:00
Haojun Liao 3dbe208712 enh(tmq): fix memory error and add retrieve wal info as required. 2023-04-22 15:34:46 +08:00
wangmm0220 d288e44f92 fix:ci error 2023-04-21 17:18:40 +08:00
wangmm0220 7ea860ad11 fix:[TD-23714] topic support capital 2023-04-21 17:14:41 +08:00
wangmm0220 b1561c95c3 fix:add test cases for escape in schemaless 2023-04-20 15:31:18 +08:00
Haojun Liao c654f1145d enh(stream): add new msg for seek, and do some internal refactor. 2023-04-20 10:59:16 +08:00
Haojun Liao 95346cd4d8 test:update the test cases. 2023-04-19 15:17:46 +08:00
Haojun Liao d5c57ca244 enh(tmq): support the seek. 2023-04-19 14:46:48 +08:00
Haojun Liao 3f12156c53 enh(tmq): support seek in tmq. 2023-04-19 13:56:42 +08:00
wangmm0220 5b3087e48b opti:escape logic in schemaless 2023-04-19 00:55:26 +08:00
Haojun Liao 9627e8105b enh(tmq): support seek offset. 2023-04-18 13:52:47 +08:00
Haojun Liao f8b672f09f other: merge main. 2023-04-14 09:07:18 +08:00
Xiaoyu Wang 4c04b155d5 merge main 2023-04-11 15:56:28 +08:00
Haojun Liao 497c9ea9c8 Merge branch 'main' into feature/3_liaohj 2023-04-10 17:57:26 +08:00
Haojun Liao f5b52749c2 refactor: do some internal refactor. 2023-04-09 01:39:09 +08:00
wangmm0220 4f23578a79 fix:add err log 2023-04-07 19:53:11 +08:00
wangmm0220 7a58f448a5 fix:time_wait in doAskEp 2023-04-06 19:16:31 +08:00
wangmm0220 fa154172b4 fix:time_wait in doAskEp 2023-04-06 18:15:03 +08:00
Haojun Liao 2b2059ef18
Merge pull request #20534 from taosdata/fix/TD-22671
fix:error in TD-23218 & remove useless logic
2023-04-06 11:47:24 +08:00
Xiaoyu Wang 650dc5cafb merge main 2023-04-04 19:35:54 +08:00
Xiaoyu Wang e9d7a099cc merge main 2023-04-04 14:50:58 +08:00
Haojun Liao a3ac8b647c refactor: do some internal refactor. 2023-04-03 18:20:35 +08:00
Haojun Liao 3ef7d43dae refactor: do some internal refactor. 2023-04-03 17:47:48 +08:00
wangmm0220 2d6436c57d fix:modify log & sleep 500ms if data is inserting while consume 2023-04-03 11:24:22 +08:00
Haojun Liao 23e8edd148 fix(tmq): set the default commit callback fn. 2023-04-01 00:14:01 +08:00
Haojun Liao 066a173d28 fix(tmq): fix the invalid write. 2023-03-31 18:39:45 +08:00
Haojun Liao ed21ef0420 fix(tmq): fix sync commit error. 2023-03-31 17:06:33 +08:00
Haojun Liao 32109e7322 fix(tmq): fix the syntax error. 2023-03-31 16:21:46 +08:00
Haojun Liao 504a2ac512 Merge remote-tracking branch 'origin/feature/3_liaohj' into feature/3_liaohj 2023-03-31 16:13:08 +08:00
Haojun Liao 05b97544b0 fix(tmq): fix the syntax error. 2023-03-31 16:12:30 +08:00
Haojun Liao 82d98f9c97
Merge branch 'main' into feature/3_liaohj 2023-03-31 16:07:40 +08:00
Haojun Liao a88de7e18b fix(tmq): build sync api based on async api. 2023-03-31 15:55:16 +08:00
Haojun Liao d2d3fc14e8
Merge pull request #20700 from taosdata/fix/TS-2940
opti:disable set enable.heartbeat.background
2023-03-31 11:33:01 +08:00
wangmm0220 a5c8713517 opti:disable set enable.heartbeat.background 2023-03-30 09:40:16 +08:00
Haojun Liao 5f4cb41e28 fix(tmq): set correct start offset value. 2023-03-29 23:45:41 +08:00
Haojun Liao 8f6ac6d771 fix(tmq): wait for 2mins when subscribe topics. 2023-03-24 15:36:42 +08:00
dapan1121 5b02af6a91 fix: merge main to 3.0 2023-03-22 16:08:16 +08:00
Haojun Liao c3acab2070 fix(tmq): do some internal refactor. 2023-03-21 19:38:34 +08:00
Haojun Liao 18b121c69c fix(tmq): fix the invalid write and do some internal refactor. 2023-03-21 13:45:43 +08:00
Haojun Liao e31796893d fix(tmq): diable commit when close conumer if auto commit is disabled. 2023-03-20 18:34:01 +08:00
Haojun Liao 2517f9bb83 fix(tmq): add some logs. 2023-03-20 15:27:16 +08:00
Haojun Liao 7e712a14b6 fix(tmq): adjust log. 2023-03-19 19:06:28 +08:00
Haojun Liao fad23f8cb0 fix(tmq): adjust log. 2023-03-19 14:03:01 +08:00
Haojun Liao 49df168a76 fix(tmq): adjust logs. 2023-03-19 11:51:16 +08:00
Haojun Liao a9e54166be fix(tmq): adjust log. 2023-03-19 11:45:53 +08:00
Haojun Liao e5e4f507c2 fix(tmq): adjust the total rows counting. 2023-03-19 11:39:31 +08:00
Haojun Liao 8a8f1daabd refactor: update the logs. 2023-03-18 18:43:13 +08:00
Haojun Liao 4f15a32839 fix(client): fix potential memory leak. 2023-03-17 14:29:12 +08:00
Haojun Liao 5142ccf6c9 fix(tmq): release the tmq properly, and add some logs for subscription. 2023-03-16 18:18:59 +08:00
Haojun Liao 08ac102114 refactor: do some internal refactor. 2023-03-16 15:45:48 +08:00
Haojun Liao 989ae86a05 fix(tmq): adjust the time out value check. 2023-03-16 14:32:40 +08:00
Haojun Liao a1682664f2 fix(tmq): disable non-leader vnode responsing the poll request. 2023-03-16 10:12:34 +08:00
Haojun Liao 32ddecc347 other: merge main. 2023-03-14 12:13:09 +08:00
Haojun Liao 9cdd52de84 fix(tmq): fix dead lock. 2023-03-14 11:50:02 +08:00
Haojun Liao 8f1f423c90 fix(tmq): fix the error in tmq. 2023-03-14 00:37:09 +08:00
Haojun Liao 8a9746d2bf fix(tmq): fix the invalid read. 2023-03-13 18:29:04 +08:00
Haojun Liao 731a2138df fix(tmq): fix error in tmq. 2023-03-13 17:08:17 +08:00
Haojun Liao 7f2aae69d4 fix(tmq): fix error in taosx 2023-03-13 01:54:29 +08:00
wangmm0220 5b75ee4396 opti:modify tmq logic 2023-03-11 19:13:50 +08:00
Haojun Liao 26925a1c13 refactor: add some logs for tmq. 2023-03-11 18:01:56 +08:00
Haojun Liao 85ee1318d2 refactor: add some logs. 2023-03-11 17:44:35 +08:00
Haojun Liao af423685ca fix(tmq): handle the commit failure. 2023-03-11 17:42:01 +08:00
Haojun Liao 9e860f02ac refactor: do some internal refactor. 2023-03-11 11:25:37 +08:00
Haojun Liao 1e01f90283 refactor(tmq): do some internal refactor. 2023-03-11 09:13:26 +08:00
Haojun Liao 9d680a0995 fix(tmq): fix race condition. 2023-03-11 09:09:03 +08:00
Haojun Liao 1f71cf3a8b fix(tmq): fix invalid read. 2023-03-10 14:18:55 +08:00
Haojun Liao 3513de5ab4 fix(tmq): fix race condition. 2023-03-10 14:11:58 +08:00
Haojun Liao 9ca77572d9 fix(tmq): fix race condition. 2023-03-10 14:09:42 +08:00
Haojun Liao b31b7c3f64 fix(tmq): fix the invalid read. 2023-03-10 10:27:21 +08:00
Haojun Liao da550029d8 fix(tmq): fix memory error and adjust some logs. 2023-03-06 11:52:52 +08:00
Haojun Liao 6fafa8443e fix(query): sleep a little bit when errors occur. 2023-03-05 15:32:00 +08:00
Haojun Liao 22a85734c6 fix(mq): add more chek for balance couner to avoid the negative value emerges. 2023-03-05 13:23:50 +08:00
Haojun Liao dfeed5a495 refactor: disable 0 topics; 2023-03-04 22:14:30 +08:00
Haojun Liao 33f2d07605 refactor: sleep for a while. 2023-03-04 16:38:50 +08:00
Haojun Liao c159fd0747 refactor: disable sleep temporarily. 2023-03-04 16:37:44 +08:00
Haojun Liao 53a0ff0be3 refactor: add some logs. 2023-03-04 16:19:45 +08:00
Haojun Liao 3645c52b25 refactor: update some logs. 2023-03-04 00:34:53 +08:00
wangmm0220 6c68b36d7f fix:change ask ep to 20s 2023-03-03 18:22:26 +08:00
Haojun Liao e8c11c58f0 fix(tq): add some logs. 2023-03-03 10:20:52 +08:00
Xiaoyu Wang d6dd087f84 merge main 2023-02-25 10:29:20 +08:00
Xiaoyu Wang 5856a39827
Merge pull request #20123 from taosdata/fix/3.0_merge_main
merge main
2023-02-25 10:01:26 +08:00
Haojun Liao 22aa03307c fix(query): fix syntax errors on centos. 2023-02-24 11:38:54 +08:00
Haojun Liao dbb1e9bd59 refactor: do some internal refactor. 2023-02-23 21:59:55 +08:00
Xiaoyu Wang 1ab144609e merge main 2023-02-23 14:58:22 +08:00
Haojun Liao 8532de31cf refactor: remove assert. 2023-02-22 14:32:39 +08:00
Haojun Liao 4b7c88bcb1 refactor: do some internal refactor. 2023-02-22 08:52:20 +08:00
Haojun Liao 37fea8cf3d fix(tmq): update the epset when the leader of vnode changed. 2023-02-21 23:04:58 +08:00
Haojun Liao e10ec84154 refactor: do some internal refactor. 2023-02-20 14:52:16 +08:00
Xiaoyu Wang c597de099f merge main 2023-02-20 13:41:27 +08:00
Haojun Liao 50ae5e7427 refactor: do some internal refactor and add some logs. 2023-02-18 15:51:02 +08:00
Haojun Liao a2a7dffb5d refactor: do some internal refactor and add some logs. 2023-02-18 13:52:07 +08:00
Haojun Liao abe37a16e3 refactor: do some internal refactor and add some logs. 2023-02-18 12:40:09 +08:00
Haojun Liao 6836dc1c80 refactor: do some internal refactor and add some logs. 2023-02-18 12:03:11 +08:00
Haojun Liao 7d915626c4 refactor: do some internal refactor and fix race condition. 2023-02-17 23:06:19 +08:00
Xiaoyu Wang 5d499e52f9 merge main 2023-02-03 17:13:11 +08:00
Liu Jicong 1e17f1d482 fix: mem leak 2023-02-02 09:42:06 +08:00
Shengliang Guan 295eee4703
Merge pull request #19416 from taosdata/refact/submit_req_marks
fix:ASSERT in client mode & converity error
2023-01-07 11:01:01 +08:00
wangmm0220 ce3fbae952 fix:ASSERT in client mode & converity error 2023-01-06 14:37:38 +08:00
Xiaoyu Wang fdde287ca9 merge main 2023-01-04 11:43:20 +08:00
Liu Jicong 617d8d4013 fix mem leak 2023-01-03 11:18:48 +08:00
Shengliang Guan 664dcb6b4e Merge branch 'main' into merge/mainto3.0_1226 2022-12-26 09:26:54 +08:00
wangmm0220 51a6aa51cd fix:modify delete msg type to tmq meta 2022-12-21 20:23:00 +08:00
Liu Jicong 50729f0cc4 refactor: remove assert 2022-12-21 14:16:28 +08:00
yihaoDeng 03c53c9531 fix mem leak 2022-12-16 22:37:24 +08:00
yihaoDeng 9542b95d62 fix mem leak 2022-12-16 22:32:44 +08:00
Liu Jicong be03f7aa8b Merge branch 'main' into feature/stream 2022-12-16 14:42:44 +08:00
Liu Jicong 4402bcef94 enh(stream): forbid source db replica > 1 2022-12-16 08:53:33 +08:00
Shengliang Guan aa512f1d55
Merge pull request #18874 from taosdata/fix/TD-21117
fix: control rpc qitem memory
2022-12-10 15:58:31 +08:00
Shengliang Guan 50b66df39f fix: control rpc qitem memory 2022-12-10 14:02:57 +08:00
Liu Jicong 5b8cc2234b fix: mem leak 2022-12-09 14:38:29 +08:00
Liu Jicong 1e51c1b49c add debug log 2022-12-09 09:41:06 +08:00
Liu Jicong 332ecd1d92 fix: mem leak 2022-12-08 10:07:55 +08:00
Shengliang Guan 6d94afe48f
Revert "refact: adjust some assert cases" 2022-12-07 21:51:17 +08:00