Commit Graph

507 Commits

Author SHA1 Message Date
Xiaoyu Wang 0c269c317b merge 3.0 2022-12-23 17:50:05 +08:00
Liu Jicong 2a7de0cd46 refactor: remove assert 2022-12-21 13:54:14 +08:00
Haojun Liao fdade6d6a3
Merge branch '3.0' into refact/submit_req 2022-12-19 11:24:00 +08:00
Liu Jicong be03f7aa8b Merge branch 'main' into feature/stream 2022-12-16 14:42:44 +08:00
Liu Jicong 4402bcef94 enh(stream): forbid source db replica > 1 2022-12-16 08:53:33 +08:00
Haojun Liao 48f7fc5c3d
Merge branch '3.0' into refact/submit_req 2022-12-12 18:11:17 +08:00
Shengliang Guan aa512f1d55
Merge pull request #18874 from taosdata/fix/TD-21117
fix: control rpc qitem memory
2022-12-10 15:58:31 +08:00
Shengliang Guan 50b66df39f fix: control rpc qitem memory 2022-12-10 14:02:57 +08:00
Liu Jicong a6fff95795 fix: close wal ref 2022-12-09 17:18:21 +08:00
Liu Jicong 74a253eac4 refactor 2022-12-08 13:34:42 +08:00
Shengliang Guan 6d94afe48f
Revert "refact: adjust some assert cases" 2022-12-07 21:51:17 +08:00
Shengliang Guan d5126d469a refact: replcase ASSERT with tAssert 2022-12-07 18:42:48 +08:00
Shengliang Guan f69a188f7e refact: replcase ASSERT with tAssert 2022-12-07 18:37:40 +08:00
Liu Jicong 9e7bbf622a refactor: support submitreq2 2022-12-07 17:54:09 +08:00
Liu Jicong 45b93280c9 refactor: support submitreq2 2022-12-07 17:22:38 +08:00
Liu Jicong b133a56219 refactor: support submitreq2 2022-12-07 15:19:34 +08:00
Hongze Cheng 899d7b0bcc Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/submit_req 2022-12-06 14:37:22 +08:00
Liu Jicong 619bbdfd82 refactor: support submitreq2 2022-12-06 05:58:28 +08:00
Shengliang Guan d9c261cd13 refact: remove some error codes 2022-12-03 10:17:02 +08:00
Hongze Cheng 1cf9974b50 Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/submit_req 2022-12-02 10:10:21 +08:00
Liu Jicong 3424b184d3 refactor(stream): distributed checkpoint 2022-12-02 03:48:45 +08:00
Hongze Cheng f848460f21 Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/submit_req 2022-11-30 13:28:17 +08:00
Liu Jicong 548684e5ac fix(stream): delete multiple row 2022-11-29 23:39:58 +08:00
Hongze Cheng 0fdf4193e0 Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/submit_req 2022-11-27 11:22:21 +08:00
Liu Jicong 48894c8bdb enh(stream): new api for stream queue 2022-11-25 18:03:40 +08:00
Hongze Cheng 91a5f8958f Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/submit_req 2022-11-24 09:23:45 +08:00
Shengliang Guan 962aa4aff4
Merge pull request #18383 from taosdata/feature/stream
fix(stream): memory leak
2022-11-24 08:17:40 +08:00
Hongze Cheng c1db8a9e4d refact code 2022-11-23 20:13:27 +08:00
Liu Jicong 3b84053893 fix(stream): memory leak 2022-11-23 12:32:02 +08:00
dapan1121 19852ba85b Merge remote-tracking branch 'origin/3.0' into enh/msgRefactor2 2022-11-22 10:45:11 +08:00
Liu Jicong 9358429e78 Merge branch '3.0' into feature/stream 2022-11-21 20:54:21 +08:00
dapan1121 c66390d401 enh: refact tmq pull message 2022-11-21 20:38:42 +08:00
Liu Jicong 6478fcb2c1 fix(stream): check task 2022-11-15 22:32:52 +08:00
Liu Jicong a9bf05bc01 refactor: stream meta ref count 2022-11-15 19:11:42 +08:00
Liu Jicong 76cd3122d1 fix: memory leak 2022-11-15 14:28:20 +08:00
Liu Jicong efde22f081 fix: memory leak 2022-11-15 11:59:29 +08:00
Liu Jicong 2386e0b513 refactor(tmq): add more strict drop condition check 2022-11-11 14:37:31 +08:00
Liu Jicong 2fbc8e6306 refactor(stream): stream deploy and state transfer 2022-11-03 13:22:23 +08:00
Liu Jicong 94e7c79ca0 fix(tmq): handle adding subtable for subscribing stb 2022-11-01 18:18:26 +08:00
Liu Jicong 5e253cc8ea refactor(stream): fill history 2022-11-01 11:56:14 +08:00
Liu Jicong 2e640c38a2 refactor: stream and tmq message 2022-10-31 18:37:28 +08:00
Liu Jicong b26cc9c318 feat(snode) 2022-10-28 17:03:17 +08:00
Liu Jicong dcf2c980c5 refactor(mnode): remove unused code 2022-10-27 15:32:20 +08:00
Liu Jicong 42422b7a65 fix(taosx): reset latest offset 2022-10-26 21:20:35 +08:00
Liu Jicong a16b010e9f fix: memory leak 2022-10-26 11:35:15 +08:00
Liu Jicong 8d8fd2b2bc refactor(stream): recover and fill history 2022-10-26 10:20:33 +08:00
Liu Jicong 75e5e490fd refactor(stream): recover and fill history 2022-10-26 00:31:00 +08:00
Benguang Zhao 00a066443f Merge branch '3.0' into FIX/TD-19593-3.0 2022-10-19 09:38:36 +08:00
Benguang Zhao 3b0968d7f9 enh: enable the -Wformat=2 compiler option 2022-10-19 09:24:15 +08:00
Liu Jicong ca2012a521 refactor: tmq commit 2022-10-18 16:56:07 +08:00
Liu Jicong 73662b3058 refactor: table sink 2022-10-17 14:48:56 +08:00
Liu Jicong 980fdd6e91 fix: error handling 2022-10-17 10:27:15 +08:00
Liu Jicong 6f8d4d6955 fix(tmq): push msg 2022-10-14 13:31:05 +08:00
Liu Jicong d507678b35 refactor(stream): set state buffer size 2022-10-13 11:18:02 +08:00
Liu Jicong b1594a6857 fix coverity scan 2022-10-10 11:44:17 +08:00
Liu Jicong bb1165f1e9 optimize projection return 2022-09-29 12:14:41 +08:00
Liu Jicong a2a8fe0211 rsp consumer when taosd stop 2022-09-29 01:37:45 +08:00
Liu Jicong b0a46942c2 rsp to consumer if offset move forward 2022-09-28 23:47:47 +08:00
Liu Jicong fab0adde99 fix memory error 2022-09-28 23:06:45 +08:00
Liu Jicong cde4085a6d optimize msg batch 2022-09-28 20:24:02 +08:00
Liu Jicong 01c94a775b feat(tmq): push optimization 2022-09-28 14:55:19 +08:00
Liu Jicong 0e8aaf4c9e feat(tmq): push optimization 2022-09-28 10:38:49 +08:00
Liu Jicong 7ade2b3b8d fix double free in sma 2022-09-21 18:54:49 +08:00
Liu Jicong 72fba5a8ce fix(stream): check delete uid 2022-09-21 11:54:44 +08:00
Shengliang Guan 69e1edb344
Merge pull request #16898 from taosdata/fix/TD-18740
feat:Complete the automatic table creation function for taosX
2022-09-20 11:35:18 +08:00
Liu Jicong 756a2d3d5d merge 3.0 2022-09-19 19:29:20 +08:00
Liu Jicong 86c0fb56e0 enhance stream backend for sma 2022-09-19 17:27:25 +08:00
wangmm0220 e3bc2487b3 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-18740 2022-09-16 19:00:49 +08:00
wangmm0220 dfc5ce23a0 fix:error in auto create table for taosX 2022-09-16 17:01:41 +08:00
Liu Jicong 367552b914 fix delete 2022-09-15 10:12:53 +08:00
Liu Jicong 004a84ac81 fix memory leak 2022-09-02 16:41:32 +08:00
Liu Jicong d653da7971 feat(stream): support delete data 2022-09-02 15:46:41 +08:00
Liu Jicong 1fe4ab2341 feat(taox): support auto create sub table 2022-09-01 11:45:00 +08:00
Liu Jicong c4dcc994fb remove queue scan 2022-08-31 11:35:25 +08:00
wangmm0220 bf3231ab9f fix: cols num error in tmq for query 2022-08-29 17:30:50 +08:00
wangmm0220 6b61d3c23b fix:conflicts from 3.0 2022-08-25 16:57:19 +08:00
wangmm0220 4cc2d361e1 fix:error in getting snapshot for taosX 2022-08-25 16:55:12 +08:00
wangmm0220 c7bf08d5af feat: get snapshot data for taosX 2022-08-24 17:42:33 +08:00
Liu Jicong 24261cc90b refactor(stream): refine stream backend interface 2022-08-24 15:53:29 +08:00
wangmm0220 2afc23a120 fix:conflicts from 3.0 2022-08-24 13:59:51 +08:00
Liu Jicong d7b804e02e fix stream load task 2022-08-24 10:42:34 +08:00
wangmm0220 f477e94dda fix:error in snapshot from taosx 2022-08-24 10:24:19 +08:00
Liu Jicong 2d628491b2 feat(stream): support tdb state backend 2022-08-23 19:30:04 +08:00
wangmm0220 fbc69a10c4 Merge branch '3.0' into feature/TD-14761 2022-08-18 16:45:06 +08:00
wangmm0220 d2727b6252 feat:merge from 3.0 2022-08-18 10:45:20 +08:00
Liu Jicong 853e6e2988 refactor(mnode): drop stream task 2022-08-17 19:19:58 +08:00
Liu Jicong 357b21cfeb refactor(tmq): use tdb to store check info 2022-08-17 13:58:56 +08:00
wangmm0220 78c9f245b1 fix:conflicts from 3.0 2022-08-16 15:18:11 +08:00
Liu Jicong 18f6fb6123 enh(tmq): speed up consumer recover 2022-08-10 16:04:03 +08:00
wangmm0220 c381ffd38c fix:error in tmq for snapshot 2022-08-09 19:06:24 +08:00
Liu Jicong d9a4ea305d fix(stream): memory error 2022-08-09 14:11:41 +08:00
Liu Jicong c411b9aebb fix(stream): memory leak 2022-08-08 18:09:36 +08:00
Liu Jicong bb0ce39511 refactor(stream): add stream meta 2022-08-08 15:05:54 +08:00
wangmm0220 507adf9e7b fix:error in tmp for snapshot 2022-08-08 13:49:24 +08:00
wangmm0220 1309c489e2 fix:error in tmq for snapshot 2022-08-06 19:31:53 +08:00
wangmm0220 f28155714e fix:confilcts from 3.0 2022-08-06 12:31:15 +08:00
wangmm0220 45e7597df3 feat:get data from snapshot for taosx 2022-08-05 21:12:18 +08:00
Liu Jicong 7c9d57bd89 refactor(stream): unify sink and dispatch 2022-08-05 20:16:47 +08:00
Liu Jicong 6f8683de7e enh(tmq): retry consumer close 2022-08-04 20:37:02 +08:00
Liu Jicong 031414001d fix schedule 2022-08-04 17:46:13 +08:00
Liu Jicong e64e41679e fix schedule 2022-08-04 17:00:45 +08:00
Liu Jicong d6cc88ed4b refactor(stream): exec scheduler 2022-08-04 16:19:53 +08:00
wangmm0220 c3cd858a63 feat:add snapshot for tmq in stable and db 2022-08-04 15:01:59 +08:00
Liu Jicong c089d3ba08 merge 3.0 2022-08-02 22:53:49 +08:00
Liu Jicong d43eec5d16 enh(stream): directly dispatch 2022-08-02 22:23:33 +08:00
Shengliang Guan 56f2e49764 refactor: adjust logs 2022-08-02 17:12:07 +08:00
Shengliang Guan bd31c9da5a
Merge pull request #15657 from taosdata/fix/dnode
fix: write snapshot after apply queue is empty
2022-08-01 20:13:08 +08:00
Liu Jicong b084dd8ee6 feat(tq): transport snapshot 2022-08-01 19:12:37 +08:00
Shengliang Guan 99f29f6455 refactor: adjust log 2022-08-01 17:23:52 +08:00
Liu Jicong bb6fabcc11 refactor(stream): use vnode meta 2022-07-30 20:20:12 +08:00
Liu Jicong 6d67b17113 enh(wal): add lock to guarantee read behaviour 2022-07-30 16:35:49 +08:00
Liu Jicong a9d11b58ad enh(stream): recover 2022-07-28 16:19:50 +08:00
Liu Jicong 00acf4520c refactor(stream): remove option 2022-07-27 16:49:03 +08:00
wangmm0220 915d3c4203 fix:conflicts from 3.0 2022-07-26 17:47:10 +08:00
wangmm0220 ba1962cacf feat:add tmq get data interface 2022-07-26 17:43:22 +08:00
Liu Jicong c2164dd8aa fix(wal): snapshot end with no deleting 2022-07-26 14:13:03 +08:00
Liu Jicong 222e925644 feat(wal): ref 2022-07-25 16:40:19 +08:00
Shengliang Guan 5b654779ba Merge pull request #15372 from taosdata/fix/tsim
test: regression case
2022-07-25 15:23:41 +08:00
Liu Jicong 072fe10013
Merge pull request #15361 from taosdata/feature/tq
fix(tmq): correctly set reader status
2022-07-23 20:40:14 +08:00
Liu Jicong 095f6aa4e0 fix(tmq): correctly set reader status 2022-07-23 20:07:16 +08:00
plum-lihui 19acda27b2 test: add comment for tq 2022-07-23 13:32:25 +08:00
Liu Jicong e1046c7a98 fix: memory leak 2022-07-22 18:11:29 +08:00
Liu Jicong 2441ec22bd feat(tmq): check alter for ntb 2022-07-22 16:05:28 +08:00
Liu Jicong b1f3428c05 refactor(stream): do not merge output 2022-07-22 10:43:49 +08:00
Liu Jicong b49c61cfb4 merge 3.0 2022-07-21 20:44:08 +08:00
Haojun Liao 8a2fb51129
Merge pull request #15230 from taosdata/feature/3_liaohj
fix(query): add query involved column info
2022-07-21 20:29:25 +08:00
Liu Jicong eaf11ca1ac refactor(tmq): disable parallel 2022-07-21 20:15:52 +08:00
Haojun Liao c2d588a118 other:merge 3.0 2022-07-21 11:29:34 +08:00
Haojun Liao ee34c3bb3e fix(query): add query involved column info 2022-07-21 11:20:30 +08:00
Cary Xu 65871c2d73 Merge branch '3.0' into feature/TD-11274-3.0 2022-07-21 09:44:20 +08:00
Cary Xu 0f5983ad88 fix: merge dup rows in client 2022-07-20 20:30:54 +08:00
Liu Jicong fb52cfa816 fix(stream): memory error 2022-07-20 19:31:56 +08:00
Liu Jicong b74560537c fix(stream): check task exist 2022-07-20 16:25:20 +08:00
Haojun Liao 282630a394 Merge branch '3.0' into feature/3_liaohj 2022-07-19 17:42:45 +08:00
Haojun Liao cd2eaf0152 fix(query): set the output column number for subscribe. 2022-07-19 16:01:28 +08:00
54liuyao 8e41fc912b feat(stream): add num of children 2022-07-19 14:17:53 +08:00
Liu Jicong 500553d581 refactor: add debug log 2022-07-18 11:41:56 +08:00
Liu Jicong d75e9b8f6a merge from 3.0 2022-07-14 15:35:00 +08:00
Liu Jicong 9ea0ad57ec refactor(stream): destroy stream task 2022-07-14 15:32:21 +08:00
wangmm0220 cd051d40c3 fix:confilct from 3.0 2022-07-13 16:03:39 +08:00
Shengliang Guan 148e3a5f97
Merge pull request #14818 from taosdata/fix/mnode
enh: adjust vnode fetch queue number
2022-07-12 20:43:05 +08:00
Shengliang Guan dad12c24fb refactor: comment out to avoid crash of tmqShow.py 2022-07-12 19:56:38 +08:00
Liu Jicong 0fccdace56 fix(tmq): reset offset 2022-07-12 16:39:05 +08:00
Liu Jicong 7cf99d5359 refactor(tmq): prepare only needed 2022-07-12 15:11:14 +08:00
Liu Jicong d637ffe22c refactor(tmq): prepare only needed 2022-07-12 14:10:22 +08:00
wangmm0220 f9d5c809c9 fix:conflict from 3.0 2022-07-11 19:06:36 +08:00
Liu Jicong c6052fadba feat(wal): add append interface 2022-07-11 16:22:51 +08:00
wangmm0220 0a193cf811 feat:get meta ok 2022-07-11 16:12:07 +08:00
Liu Jicong 219d966b25 fix(wal): read after rolling 2022-07-10 21:02:25 +08:00
Liu Jicong 9189f886a9 feat(tmq): support consume from tsdb then wal 2022-07-10 16:34:45 +08:00