Commit Graph

3486 Commits

Author SHA1 Message Date
Haojun Liao f1359076c1
Merge pull request #22450 from taosdata/fix/3_liaohj
other: merge main.
2023-08-16 16:48:33 +08:00
Haojun Liao d63758a1c5 fix(stream): fix error in execute task on a tranferred node. 2023-08-16 15:24:24 +08:00
dapan1121 143c657571 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-08-16 14:50:31 +08:00
dapan1121 b113d02476
Merge pull request #22438 from taosdata/feat/TD-23299
feat: add irate distributed execution
2023-08-16 14:44:33 +08:00
dapan1121 91d6734b34 fix: join condition push down issue 2023-08-16 14:28:39 +08:00
dapan1121 a6ecb7253e
Merge pull request #22390 from taosdata/feat/3.0/TD-25365
feat: optimize select agg_func partition by tag slimit
2023-08-16 13:32:58 +08:00
wangjiaming0909 e587cc50e6 feat: optimize select agg_func partition by tag slimit 2023-08-16 11:20:56 +08:00
wangjiaming0909 cb027cc5c5 feat: support alter table add column with comment 2023-08-16 11:07:40 +08:00
dmchen de5aedd8f3 vnode.json 2023-08-16 10:20:48 +08:00
Haojun Liao ad9f3cbc43 other: merge main. 2023-08-16 09:14:10 +08:00
slzhou 4fdaaba3ce Merge branch '3.0' of github.com:taosdata/TDengine into szhou/tag-scan-opt 2023-08-15 17:18:25 +08:00
Benguang Zhao 239a79d405 enh: add some vnode info in SStatusReq 2023-08-15 17:06:33 +08:00
wangjiaming0909 dcbef83dea feat: support column comment 2023-08-15 16:30:02 +08:00
shenglian zhou 84e472ad03 enhance: tag cond col list only once and tag scan derive from scan 2023-08-15 16:10:54 +08:00
Haojun Liao 39079ab64a fix(stream): load term from sync model to decide if the node is restart or not 2023-08-15 15:52:06 +08:00
Ganlin Zhao 01a7dfbc34 feat: add irate distributed execution 2023-08-15 12:59:30 +08:00
Haojun Liao b1d35f43ba enh(stream): using term from sync module to denote the upstream node is restarted/replica-modified/vnode-transferred or not. 2023-08-15 09:25:05 +08:00
Haojun Liao ff17763ed6
Merge pull request #22417 from taosdata/mark/tmq
opti:wal logic
2023-08-15 09:16:29 +08:00
dapan1121 f9f74eaaf4
Merge pull request #22406 from taosdata/fix/TD-24684
fix: remove ins_modules table
2023-08-15 08:42:08 +08:00
slzhou 012248b681 fix: move the only ctb idx flag to logical plan 2023-08-14 19:26:53 +08:00
dapan1121 6a78053b51 feat: add hint supporting 2023-08-14 17:21:15 +08:00
Haojun Liao e8c9a019a4 fix(stream): remove the invalid set of scheduler status. 2023-08-14 11:41:24 +08:00
dapan1121 6ce28e8886 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-08-14 10:49:56 +08:00
slzhou 646e52ae7f Merge branch '3.0' of github.com:taosdata/TDengine into szhou/tag-scan-opt 2023-08-14 10:19:36 +08:00
wangmm0220 dd0bc0e050 fix:heap use after free 2023-08-14 09:42:55 +08:00
Haojun Liao 0d38f389ab refactor: do some internal refactor. 2023-08-13 16:25:35 +08:00
Haojun Liao afe9b848a5 fix(tsdb): check the 2023-08-12 18:54:39 +08:00
Haojun Liao 09da6c6840 refactor: do some internal refactor. 2023-08-12 18:51:20 +08:00
Haojun Liao 75e6fb0f16 refactor: do some internal refactor. 2023-08-12 18:47:01 +08:00
Haojun Liao 8f9de93cf0 refactor: do some internal refactor. 2023-08-12 18:37:36 +08:00
Haojun Liao ce721a0146 fix(stream): fix error. 2023-08-12 16:32:05 +08:00
Haojun Liao a89ce1a20b fix(stream): transfer state by using data block. 2023-08-12 01:24:07 +08:00
wangmm0220 f1baed79b4 opti:rm useless code 2023-08-11 20:22:51 +08:00
wangmm0220 deda4b9eed opti:wal logic 2023-08-11 19:26:17 +08:00
dapan1121 c9656d3f77 fix: fix memory issues 2023-08-11 17:39:41 +08:00
wangmm0220 7b22d37f12 fix:lost data when consumer db 2023-08-11 17:25:43 +08:00
slzhou 42a4f364c1 Merge branch 'szhou/tag-scan-opt' of github.com:taosdata/TDengine into szhou/tag-scan-opt 2023-08-11 16:48:49 +08:00
yihaoDeng 6e6eb05d5c Merge branch 'enh/triggerCheckPoint2' into enh/chkpTransfer 2023-08-11 08:03:02 +00:00
Haojun Liao e984affe90 fix(stream): disable the stream in the followers nodes. 2023-08-11 15:58:15 +08:00
dapan1121 8987553d9c fix: remove ins_modules 2023-08-11 15:56:38 +08:00
shenglian zhou 1c7f854a71 enhance: add only meta ctb index to tag scan physi node 2023-08-11 14:47:28 +08:00
yihaoDeng 99be2836b2 Merge branch 'enh/triggerCheckPoint2' into enh/chkpTransfer 2023-08-11 02:50:12 +00:00
Haojun Liao 6a9826cf6e enh(stream): update the stream task epset. 2023-08-11 10:37:25 +08:00
yihaoDeng ba4ce9db02 Merge branch 'enh/triggerCheckPoint2' into enh/chkpTransfer 2023-08-11 02:15:07 +00:00
shenglian zhou b2e615d4e7 enhance: tag scan cursor based block 2023-08-10 17:30:01 +08:00
dm chen dd271bca6d
Merge pull request #22394 from taosdata/3.0
merge 3.0 to feat/TD-22970
2023-08-10 14:14:22 +08:00
Haojun Liao f633c7fe2b other: merge 3.0. 2023-08-10 11:56:30 +08:00
Haojun Liao b658608541
Merge pull request #22388 from taosdata/fix/3_liaohj
other: merge main.
2023-08-10 11:43:02 +08:00
yihaoDeng ef247cdb1d support reopen stream state 2023-08-10 01:49:26 +00:00
Haojun Liao 7e3f1d92fc Merge branch 'main' into fix/liaohj 2023-08-09 19:43:24 +08:00
Haojun Liao 1e8579e8c5 fix(stream): fix other cases. 2023-08-09 16:04:48 +08:00
Haojun Liao 4c92997328 fix(stream): use streamId&taskId to identify the stream task in the stream meta. 2023-08-09 15:52:23 +08:00
Haojun Liao 1d649bccb6
Merge pull request #22372 from taosdata/fix/TD-25626
fix taosd cannot quit
2023-08-09 15:46:14 +08:00
Haojun Liao 3cdb16352c fix(stream): avoid launching check downstream when failed to add stream task into stream meta. 2023-08-09 12:03:14 +08:00
dapan1121 1aac643a9b enh: support single vnode join 2023-08-09 10:30:35 +08:00
Haojun Liao f3c5f20ee2 enh(stream): add timer to check the node changing. 2023-08-09 10:25:16 +08:00
yihaoDeng fb7dec00e1 support reopen stream state 2023-08-09 01:50:49 +00:00
yihaoDeng cfb5247e30 change var name 2023-08-09 00:46:18 +00:00
Benguang Zhao e4d16e594c enh: check if disk space sufficient at primary dir with tfs 2023-08-08 17:53:11 +08:00
Haojun Liao af3a87560a refactor: do some refactor. 2023-08-08 09:11:41 +08:00
yihaoDeng b6c991f896 support reopen stream state 2023-08-07 08:59:04 +00:00
yihaoDeng eeb97351e8 support reopen stream state 2023-08-07 08:51:01 +00:00
Haojun Liao 225c0e61dc hen(stream): support the rollback and checkdown stream tasks. 2023-08-07 16:13:39 +08:00
dapan1121 0546bdceb2 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-08-07 11:23:15 +08:00
Haojun Liao 8a11a0418f other: merge 3.0 2023-08-07 09:19:26 +08:00
yihaoDeng 6dc93a8dec add stream ver 2023-08-04 13:52:20 +08:00
dm chen 5b03ab1705
Merge branch '3.0' into feat/TD-22970 2023-08-03 17:43:51 +08:00
yihaoDeng f9d0874e45 support reopen stream state 2023-08-03 08:24:50 +00:00
Haojun Liao 13543f7daa feature(stream): handle task update in tq module. 2023-08-02 10:33:41 +08:00
Haojun Liao ad4c5916e7 enh(stream): update the stream task. 2023-08-01 18:14:58 +08:00
Haojun Liao 164bfd5408 enh(stream): prepare update trans. 2023-08-01 16:12:35 +08:00
Haojun Liao 6365d5000b
Merge pull request #22271 from taosdata/fix/3_liaohj
fix(stream): dump results to sink node before paused.
2023-08-01 15:50:17 +08:00
Haojun Liao 75b1520be0 enh(stream): create task update info 2023-08-01 14:04:07 +08:00
Haojun Liao f6ae9cf9bb fix(stream): fetch all data before paused and dump to sink node. 2023-08-01 10:16:14 +08:00
Haojun Liao 36ef9a537f
Merge pull request #22256 from taosdata/fix/TD-25457
check update data
2023-07-31 22:56:05 +08:00
Haojun Liao d45596bb33 fix(stream): kill task when pause the stream execution. 2023-07-31 18:07:37 +08:00
liuyao 32b59af4b1 check update data 2023-07-31 17:32:24 +08:00
Haojun Liao 9981cecbed refactor: do some internal refactor. 2023-07-31 15:24:35 +08:00
Haojun Liao e61aa83594 enh(stream): handle the stream hb. 2023-07-31 14:07:18 +08:00
Haojun Liao 8dfef8768a other: merge 3.0 2023-07-29 22:54:16 +08:00
Haojun Liao f6e07d1fdb other: merge 3.0 2023-07-29 01:32:02 +08:00
Haojun Liao 936afeb5ac refactor: do internal refactor to simple the pause/resume in case of fill history exists. 2023-07-28 19:13:03 +08:00
Haojun Liao d5974a8f25 refactor(stream): refactor the pause/resume for fill history execution. 2023-07-28 18:05:32 +08:00
dapan1121 b7126b9cc6 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-07-28 18:01:11 +08:00
Haojun Liao 9f943213ed enh(stream): update the hb info. 2023-07-28 14:23:53 +08:00
Haojun Liao 74834aea11 enh(stream): prepare hb to mnode from stream meta. 2023-07-28 13:07:16 +08:00
Haojun Liao 2a78cd2d23
Merge pull request #22215 from taosdata/fix/liaohj
other: merge main.
2023-07-28 13:06:17 +08:00
Haojun Liao ccc0f13649
Merge pull request #22214 from taosdata/fix/3_liaohj
fix(stream): scan wal in step2
2023-07-28 00:42:15 +08:00
Haojun Liao 22a5a18fff fix(stream): 1. set correct timewindow after step2. 2. handle the case when the task is failed to be added into the meta store. 2023-07-27 18:57:36 +08:00
Haojun Liao b145771a1b other: merge 3.0 2023-07-27 16:38:57 +08:00
Haojun Liao 3d403dca5a other: merge main. 2023-07-27 16:23:41 +08:00
Haojun Liao 40fe3ef4f6 fix(stream): refactor the step2 wal scan. 2023-07-27 14:49:20 +08:00
Shungang Li 167ecea705 fix: check term for role time
1. time of mnode becomes current role in this term
2. show 0 for offline mnodes
2023-07-27 14:16:54 +08:00
Haojun Liao e89f530db2 fix(stream): scan wal in step2 2023-07-27 10:55:59 +08:00
dapan1121 e83a46b88f enh: cache necessary file for seq mode 2023-07-26 19:17:59 +08:00
dapan1121 4e5fbb52ee Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-07-26 13:56:46 +08:00
dapan1121 6a0061ccfb fix: invalid task status update issue 2023-07-26 13:54:58 +08:00
Haojun Liao 728112ed89 fix(stream): set correct upstream epinfo. 2023-07-26 10:26:59 +08:00
Haojun Liao 1c1bf63e1f other: merge 3.0 2023-07-25 18:17:11 +08:00
Haojun Liao 0bdccd2681 refactor: do some internal refactor. 2023-07-25 17:31:54 +08:00
Haojun Liao a2694b9ce9 fix(stream): fix the invalid check of step2 . 2023-07-24 19:44:28 +08:00
Haojun Liao 1e34f84f78 other: merge 3.0 2023-07-24 15:23:03 +08:00
dapan1121 4f75eb5655 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-07-24 11:43:15 +08:00
Haojun Liao 1367552f38 fix(stream): refactor the halt status to check more status. 2023-07-24 10:04:15 +08:00
Haojun Liao 4e0f7ffbf8 fix(stream): set the correct flag of pause/resume. 2023-07-23 18:49:56 +08:00
Haojun Liao 1b2636028a fix(stream): fix memory leak. 2023-07-21 23:05:42 +08:00
Haojun Liao 16d7707b90 fix(stream): align the scan real time data for stream task. 2023-07-21 19:33:43 +08:00
dmchen 7d480cf3ac compatible old sync config 2023-07-21 10:31:53 +08:00
Haojun Liao d0da988b7d Merge branch '3.0' into fix/3_liaohj 2023-07-21 09:47:44 +08:00
dapan1121 2ef01728ca
Merge pull request #22089 from wangjiaming0909/fix/3.0/TD-23763
feat: optimize partition by tbname slimit
2023-07-21 09:29:36 +08:00
Haojun Liao db46d31ca5 fix(stream): refactor the halt function. 2023-07-21 01:32:42 +08:00
wangjiaming0909 7d850c1a5e feat: optimize partition by tbname slimit 2023-07-20 15:07:55 +08:00
Haojun Liao 87fa8f0c4c Merge branch '3.0' into fix/3_liaohj 2023-07-20 12:15:23 +08:00
Haojun Liao aed99da6c1 refactor: refactor the stream task starting order. 2023-07-20 12:14:53 +08:00
yihaoDeng 6e5be0d30e fix stream state transfer 2023-07-20 03:49:32 +00:00
Haojun Liao 97c44e7c18 refactor: do some internal refactor. 2023-07-19 18:42:02 +08:00
dapan1121 cbe03ef212
Merge pull request #22069 from taosdata/enh/TS-2288
enh: add scope for show xx variables
2023-07-19 18:01:11 +08:00
dm chen 141004bf77
Merge branch '3.0' into feat/TD-22970 2023-07-19 10:57:40 +08:00
dapan1121 c45359c989 enh: add batch table scan 2023-07-19 10:37:30 +08:00
Haojun Liao a32c650248 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-07-19 10:05:48 +08:00
Haojun Liao 2ae95217bd fix(stream): open inputQ by default for task deployed on snode. 2023-07-19 10:05:35 +08:00
liuyao fd02b9a71e
Merge branch '3.0' into enh/triggerCheckPoint2 2023-07-19 10:02:06 +08:00
wangmm0220 5c344b0159 fix:conflicts from main 2023-07-19 09:05:05 +08:00
dmchen 3711fdd865 feat/TD-22970 2023-07-18 16:09:38 +08:00
Haojun Liao d2c875e774 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-07-18 15:19:09 +08:00
Haojun Liao 901b7d8ecc fix(stream): if a source task is set to be non-normal, no data should be put into inputQ anymore. 2023-07-18 15:18:57 +08:00
dapan1121 294376de0b enh: add batch table scan 2023-07-18 14:44:53 +08:00
Benguang Zhao 24fed51137 enh: tfsRename dir on primary disk at last 2023-07-18 13:58:43 +08:00
yihaoDeng 40d59990d8 fix mem leak 2023-07-18 04:01:20 +00:00
Benguang Zhao 5624ecc8df fix: prepare vnode dir recursively in vnodeCreate 2023-07-18 11:32:23 +08:00
Benguang Zhao 6b6b7d8d8f enh: alloc disk of vnode primary dir in vmAllocPrimaryDisk 2023-07-18 11:32:23 +08:00
wangmm0220 872222eb77 fix:[TD-25222] error in drop topic if topic num is too long 2023-07-17 19:52:07 +08:00
Ganlin Zhao 929db3d647 Merge branch '3.0' into enh/TS-2288 2023-07-17 16:09:07 +08:00
Haojun Liao 08f43b9d00 fix(stream): fix checkpoint block not fetched bug. 2023-07-17 15:31:54 +08:00
Haojun Liao b2a905bd91
Merge branch '3.0' into enh/triggerCheckPoint2 2023-07-17 11:50:12 +08:00
dapan1121 a1e18ac898 fix: fix global cache issue 2023-07-17 09:25:28 +08:00
Haojun Liao a9757637d9 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-07-16 12:47:24 +08:00
Haojun Liao c6a322e0ef refactor(stream): back pressure is active, which is trigger by dispatch rsp. 2023-07-16 12:47:08 +08:00
yihaoDeng e18c9ea672 add checkpoint 2023-07-15 04:26:34 +00:00
dapan1121 188df1d675 fix: fix bugs 2023-07-14 19:36:56 +08:00
yihaoDeng ed931c1100 Merge branch 'fix/newCheckpoint' into enh/triggerCheckPoint2 2023-07-14 10:11:20 +00:00
yihaoDeng 3c4631ef1f add checkpoint 2023-07-14 10:06:24 +00:00
Haojun Liao 2b85ea62aa refactor: do some internal refactor. 2023-07-14 17:48:47 +08:00
Haojun Liao eb7e6152ae refactor(stream): do some internal refactor. 2023-07-14 17:21:20 +08:00
wangmm0220 6762a01d88 Merge branch 'main' of https://github.com/taosdata/TDengine into mark/tmq 2023-07-14 16:44:00 +08:00
wangmm0220 bbdcbcb75b fix:modify commit version to next validate version 2023-07-14 16:43:28 +08:00
wangjiaming0909 b850c1af47 fix: state/session/event window with order by _wstart/_wend has no effect 2023-07-14 16:03:59 +08:00
Ganlin Zhao cdb3453812 add show local variables scope 2023-07-14 14:38:35 +08:00
dapan1121 aca9c5fe19 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-07-14 13:31:07 +08:00
dapan1121 2312f10372 enh: support concurrent fetch from group cache 2023-07-14 13:30:13 +08:00