Commit Graph

4169 Commits

Author SHA1 Message Date
dapan1121 81439ba0cf fix: memory invalid read and perf issue 2023-08-09 13:31:38 +08:00
Haojun Liao 3cdb16352c fix(stream): avoid launching check downstream when failed to add stream task into stream meta. 2023-08-09 12:03:14 +08:00
dapan1121 1d0624bbf1 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-08-09 10:32:13 +08:00
dapan1121 1aac643a9b enh: support single vnode join 2023-08-09 10:30:35 +08:00
Haojun Liao db92546cf7 fix(stream): remove invalid filter. 2023-08-08 16:09:33 +08:00
Haojun Liao 17c6143471 fix(stream): check more status when handling the state transfer. 2023-08-08 15:08:11 +08:00
dapan1121 68e0fb924a
Merge pull request #22165 from taosdata/feat/3.0/TD-25271
feature: optimize interval with limit
2023-08-08 10:34:02 +08:00
Haojun Liao db8ce3de3b other: merge main. 2023-08-08 10:17:51 +08:00
Haojun Liao 327e13c3f7 fix(stream): fix the syntax error. 2023-08-08 10:07:28 +08:00
Haojun Liao ac00e1d520 fix(stream): fetch all data before paused and dump to sink node. 2023-08-08 09:54:43 +08:00
Haojun Liao 0e3fd5277d fix(stream): dump results to sink node before paused. 2023-08-08 09:54:39 +08:00
Haojun Liao ca2183acfd refactor: do some internal refactor. 2023-08-08 09:50:40 +08:00
Haojun Liao 143f39b6f7 fix(stream): set the correct end key of delete block. 2023-08-08 09:49:43 +08:00
dapan1121 a95295774a enh: end table scan 2023-08-07 19:35:33 +08:00
dapan1121 443c6ef318
Merge pull request #22333 from taosdata/fix/TD-25546
fix: fix tsort crash caused by not enough disk space
2023-08-07 19:02:11 +08:00
dapan1121 0546bdceb2 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-08-07 11:23:15 +08:00
dapan1121 da27a306db fix: mutex init issue 2023-08-07 11:22:25 +08:00
Haojun Liao d3bc7c3c94 refactor: do some internal refactor. 2023-08-07 09:20:53 +08:00
Haojun Liao 8a11a0418f other: merge 3.0 2023-08-07 09:19:26 +08:00
Ganlin Zhao 9f3af66ae0 fix: fix tsort crash caused by not enough disk space 2023-08-04 16:03:06 +08:00
wangjiaming0909 fba43e1748 feature: optimize interval with limit 2023-08-04 15:04:04 +08:00
slzhou 7676afff19 fix: stream use column node 2023-08-03 21:43:08 +08:00
slzhou 658c64f16f fix: make statekey of state window operator not target node 2023-08-03 21:43:08 +08:00
wangjiaming0909 632f5fbe8f feat: optimize table merge scan when 1 child table 2023-08-03 15:33:25 +08:00
wade zhang 211af11f39
Merge pull request #22303 from taosdata/fix/TD-25538
reset state window end key
2023-08-03 14:03:54 +08:00
liuyao 74c69bbb6e reset state window end key 2023-08-03 11:23:46 +08:00
liuyao 3464979489 set dummy delete data request 2023-08-03 09:03:48 +08:00
liuyao 71b000f08c reload state window state 2023-08-01 18:22:40 +08:00
Haojun Liao 6365d5000b
Merge pull request #22271 from taosdata/fix/3_liaohj
fix(stream): dump results to sink node before paused.
2023-08-01 15:50:17 +08:00
Haojun Liao f6ae9cf9bb fix(stream): fetch all data before paused and dump to sink node. 2023-08-01 10:16:14 +08:00
Haojun Liao b83860372c fix(stream): dump results to sink node before paused. 2023-08-01 00:50:30 +08:00
Haojun Liao 36ef9a537f
Merge pull request #22256 from taosdata/fix/TD-25457
check update data
2023-07-31 22:56:05 +08:00
Haojun Liao 2640e41705
Merge pull request #22261 from taosdata/fix/3_liaohj
fix(stream): set the correct end key of delete block.
2023-07-31 22:55:50 +08:00
liuyao 32b59af4b1 check update data 2023-07-31 17:32:24 +08:00
Haojun Liao 9981cecbed refactor: do some internal refactor. 2023-07-31 15:24:35 +08:00
Haojun Liao fe21943c9c fix(stream): set the correct end key of delete block. 2023-07-31 14:23:46 +08:00
Haojun Liao f359a0dca3 fix(stream): set the correct end key of delete block. 2023-07-31 14:22:46 +08:00
liuyao 09c806539f fix double free 2023-07-30 17:21:42 +08:00
Haojun Liao 8dfef8768a other: merge 3.0 2023-07-29 22:54:16 +08:00
Haojun Liao 80ad5a0abb
Merge pull request #22243 from taosdata/fix/3_liaohj
fix(stream): initialize the filter window initial range.
2023-07-29 21:10:42 +08:00
Haojun Liao 399088e2bd fix(stream): remove invalid code. 2023-07-29 19:23:35 +08:00
Haojun Liao 94551e6e46 fix(stream): update the load block position. 2023-07-29 18:48:22 +08:00
Haojun Liao e0c310215c
Merge pull request #22237 from taosdata/fix/TD-25420
init scan state
2023-07-29 16:55:43 +08:00
Haojun Liao 71e16d1eeb
Merge pull request #22238 from taosdata/fix/TD-25423
save compact window info
2023-07-29 16:55:27 +08:00
Haojun Liao 0bd2548937 fix(stream): initialize the filter window initial range. 2023-07-29 16:41:01 +08:00
Haojun Liao f6e07d1fdb other: merge 3.0 2023-07-29 01:32:02 +08:00
Haojun Liao 92d1dbd9ad fix(stream): set the time window filter before generating create table result. 2023-07-29 01:23:06 +08:00
liuyao 73ed6fe649 save compace window info 2023-07-28 19:30:26 +08:00
dapan1121 b7126b9cc6 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-07-28 18:01:11 +08:00
dapan1121 9ed1417bba enh: remove group cache 2023-07-28 18:00:33 +08:00
liuyao d712124921 init scan state 2023-07-28 17:25:56 +08:00
dapan1121 9c7b53083b
Merge pull request #22223 from taosdata/fix/td-25378
fix: fix memleak during create initial sources for block merge sort
2023-07-28 13:09:34 +08:00
Haojun Liao 2a78cd2d23
Merge pull request #22215 from taosdata/fix/liaohj
other: merge main.
2023-07-28 13:06:17 +08:00
slzhou bcf4c1ebae fix: fix memleak during create initial sources for block merge sort 2023-07-28 09:54:56 +08:00
Haojun Liao 688884de08
Merge pull request #22220 from taosdata/fix/ly_stream_fill
fix stream fill issue
2023-07-28 09:27:21 +08:00
Haojun Liao ccc0f13649
Merge pull request #22214 from taosdata/fix/3_liaohj
fix(stream): scan wal in step2
2023-07-28 00:42:15 +08:00
Haojun Liao 22a5a18fff fix(stream): 1. set correct timewindow after step2. 2. handle the case when the task is failed to be added into the meta store. 2023-07-27 18:57:36 +08:00
liuyao f667a09cc6 fix stream fill issue 2023-07-27 18:56:25 +08:00
liuyao 5d21f50340 adjust history task trigger mode 2023-07-27 17:45:28 +08:00
Haojun Liao b145771a1b other: merge 3.0 2023-07-27 16:38:57 +08:00
Haojun Liao 3d403dca5a other: merge main. 2023-07-27 16:23:41 +08:00
Haojun Liao 3fdbd5383f Merge remote-tracking branch 'origin/fix/3_liaohj' into fix/3_liaohj 2023-07-27 16:11:07 +08:00
Haojun Liao 6ff50d4eaf fix(stream): update the step2 scan wal files. 2023-07-27 16:10:55 +08:00
liuyao 7c9898cfa5 adj stream scan 2023-07-27 15:56:06 +08:00
liuyao c54bd55c68 op stream scan 2023-07-27 15:16:22 +08:00
liuyao c72274504e op stream scan 2023-07-27 15:13:57 +08:00
liuyao 0b53bdbae6 op stream scan 2023-07-27 15:09:25 +08:00
liuyao 0ac7ebe9c7 op stream scan 2023-07-27 14:56:06 +08:00
Haojun Liao 40fe3ef4f6 fix(stream): refactor the step2 wal scan. 2023-07-27 14:49:20 +08:00
liuyao 1894608b80 optimize session operator reload state 2023-07-27 09:59:58 +08:00
Haojun Liao 05504023f5
Merge pull request #22200 from taosdata/fix/ly_stream_test
stream fill operator mem leak
2023-07-27 09:38:22 +08:00
dapan1121 e83a46b88f enh: cache necessary file for seq mode 2023-07-26 19:17:59 +08:00
liuyao c368db30c3 stream fill operator mem leak 2023-07-26 17:38:37 +08:00
Haojun Liao 432c943c25 fix(stream): do filter before the update check. 2023-07-26 16:34:39 +08:00
Haojun Liao 610f9d19c1
Merge pull request #22188 from taosdata/fix/3_liaohj
fix(stream): set the correct step2 scan time window range.
2023-07-26 16:34:10 +08:00
Haojun Liao df57768a3e Merge branch '3.0' into fix/3_liaohj 2023-07-26 14:45:11 +08:00
dapan1121 4e5fbb52ee Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-07-26 13:56:46 +08:00
dapan1121 6a0061ccfb fix: invalid task status update issue 2023-07-26 13:54:58 +08:00
Haojun Liao 4d6b9d4a15 fix(stream): split delete msg for real-time scan from wal. 2023-07-26 09:08:17 +08:00
Haojun Liao 1c1bf63e1f other: merge 3.0 2023-07-25 18:17:11 +08:00
dapan1121 66042669a1
Merge pull request #22176 from taosdata/szhou/enhance-limit-proc
fix: choose heap sort for table merge scan with long row and limit
2023-07-25 17:48:11 +08:00
slzhou f4c28d848b fix: table merge scan use heap sort when there is limit 2023-07-25 15:23:00 +08:00
Haojun Liao d29f835a63 fix(stream): set the correct step2 scan time window range. 2023-07-25 15:09:22 +08:00
slzhou ca2ad71bb8 fix: return null when there are no input stream blocks for priority queue sort 2023-07-25 13:08:37 +08:00
slzhou a205eba2ab fix: choose heap sort for table merge scan when row size is big and has limit 2023-07-25 09:48:08 +08:00
dapan1121 fb63294289 fix: hash join res issue 2023-07-25 09:03:52 +08:00
dapan1121 10f9c40dba
Merge pull request #22171 from taosdata/szhou/enhance-limit-proc
enhance: skip data blocks with the merge limit ts when create initial…
2023-07-25 08:42:35 +08:00
Haojun Liao 01f11bf5a6 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2
# Conflicts:
#	source/common/src/tglobal.c
#	source/libs/executor/src/timewindowoperator.c
2023-07-24 15:25:43 +08:00
Haojun Liao 1e34f84f78 other: merge 3.0 2023-07-24 15:23:03 +08:00
slzhou 1d24b96e82 enhance: skip data blocks with the merge limit ts when create initial source 2023-07-24 15:16:39 +08:00
liuyao 992f1cb328 merge code 2023-07-24 15:00:48 +08:00
dapan1121 6fbf251978
Merge pull request #22138 from taosdata/enh/TS-3625
fix: fix(value, null) output behavior
2023-07-24 13:17:19 +08:00
dapan1121 4f75eb5655 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-07-24 11:43:15 +08:00
dapan1121 ebf1a91fb5 enh: read/write from disk cache 2023-07-24 11:39:00 +08:00
Haojun Liao 321be1f83f
Merge pull request #22135 from taosdata/fix/3_liaohj
refactor: do some internal refactor.
2023-07-22 15:06:28 +08:00
dapan1121 ae8caa8275 enh: add disk cache 2023-07-21 19:25:38 +08:00
dapan1121 3725cafe00 enh: add disk cache 2023-07-21 17:30:09 +08:00
liuyao 4451b2185a transform stream state 2023-07-21 10:32:53 +08:00
liuyao d45563684b
Merge branch '3.0' into enh/triggerCheckPoint2 2023-07-21 10:23:41 +08:00
Ganlin Zhao 62b73db8bf fix: fix(value, null) output behavior 2023-07-21 10:19:41 +08:00
Ganlin Zhao 624b6b094f fix: fix(value, null) output behavior 2023-07-21 10:14:12 +08:00
Haojun Liao d0da988b7d Merge branch '3.0' into fix/3_liaohj 2023-07-21 09:47:44 +08:00
dapan1121 2ef01728ca
Merge pull request #22089 from wangjiaming0909/fix/3.0/TD-23763
feat: optimize partition by tbname slimit
2023-07-21 09:29:36 +08:00
dapan1121 fde8eabfd7
Merge pull request #22111 from taosdata/szhou/tms-dev-2
enhance: table merge scan optimization
2023-07-20 16:28:29 +08:00
Haojun Liao 45cb478b31 refactor: do some internal refactor. 2023-07-20 15:36:46 +08:00
wangjiaming0909 7d850c1a5e feat: optimize partition by tbname slimit 2023-07-20 15:07:55 +08:00
slzhou 1a0031a43c enhance: uinfo to qdebug and increase pages num to 2048 2023-07-20 13:53:13 +08:00
slzhou 5d9f6fd481 fix: fix limit/offset bugs 2023-07-20 12:29:34 +08:00
dapan1121 7221f447df fix: group cache table index issue 2023-07-20 11:00:29 +08:00
dapan1121 3f58bc5578 fix: group cache last block index issue 2023-07-20 10:55:15 +08:00
dapan1121 58c5d5e70e fix: group cache block cache issue 2023-07-20 10:21:19 +08:00
slzhou 30129f64da enhance: merge blocks of the same table before sort 2023-07-20 09:28:19 +08:00
liuyao 8a6ed67f9d fill history 2023-07-20 09:13:30 +08:00
dapan1121 b9b8fa1ba2 fix: table list issue 2023-07-19 19:41:54 +08:00
dapan1121 a41888a9ef fix: batch fetch issue 2023-07-19 19:14:30 +08:00
liuyao af18afba9d fill history 2023-07-19 17:19:55 +08:00
liuyao c7e4aa4f77 set win range 2023-07-19 15:51:22 +08:00
dapan1121 d125bb0952 fix: merge join reset params issue 2023-07-19 14:59:05 +08:00
liuyao 8481725517 remove redundant results 2023-07-19 14:58:28 +08:00
dapan1121 8b62c75c26 fix: merge join ignore param issue 2023-07-19 14:42:18 +08:00
slzhou d1a1976aa4 enhance: add limit to merge sort 2023-07-19 13:59:43 +08:00
dapan1121 38be8f2a8e fix: reset rows before reuse block 2023-07-19 13:42:15 +08:00
dapan1121 6960872515 enh: add merge join reset 2023-07-19 11:56:13 +08:00
dapan1121 b2cea6ab84 fix: block info capicity issue 2023-07-19 11:42:54 +08:00
dapan1121 6d7912003d fix: block data memory issue 2023-07-19 11:34:44 +08:00
dapan1121 c45359c989 enh: add batch table scan 2023-07-19 10:37:30 +08:00
liuyao fd02b9a71e
Merge branch '3.0' into enh/triggerCheckPoint2 2023-07-19 10:02:06 +08:00
wangmm0220 5c344b0159 fix:conflicts from main 2023-07-19 09:05:05 +08:00
dapan1121 04fea1e38f enh: add batch fetch process 2023-07-18 19:35:45 +08:00
liuyao caa4503352 reset state key memory 2023-07-18 18:02:39 +08:00
liuyao bada18f5f0 heap use after free 2023-07-18 17:59:42 +08:00
slzhou 4348b9c51d fix: use uinfo to show on stdout 2023-07-18 16:47:31 +08:00
slzhou f79fc81d9c fix: add to page buf in the same way as single source sort 2023-07-18 15:46:38 +08:00
liuyao 14e591c802 mem leak 2023-07-18 14:51:32 +08:00
dapan1121 294376de0b enh: add batch table scan 2023-07-18 14:44:53 +08:00
yihaoDeng ad0b0c3821 fix mem leak 2023-07-18 06:04:41 +00:00
dapan1121 c2245a3546
Merge pull request #22100 from taosdata/fix/TD-25241
fix: fix interval in nested query filter not applied
2023-07-18 13:52:35 +08:00
liuyao a4676fc74b init winkey 2023-07-18 10:43:35 +08:00
slzhou 97a6e89d11 enhance: refactor create initial sources 2023-07-18 10:38:40 +08:00
Ganlin Zhao 94b13df641 fix: fix interval in nested query filter not applied 2023-07-18 10:36:07 +08:00
slzhou ba2b404295 enhance: optimize msortComparFn for table merge scan 2023-07-17 21:07:55 +08:00
yihaoDeng 1b8ec19509 fix compile err on windows 2023-07-17 11:44:40 +00:00
liuyao d45adeaa4e add checksum 2023-07-17 18:04:17 +08:00
slzhou 1b9754f1bc enhance: add log with info level. change to debug later 2023-07-17 16:18:06 +08:00
Haojun Liao 08f43b9d00 fix(stream): fix checkpoint block not fetched bug. 2023-07-17 15:31:54 +08:00
dapan1121 c22619ab67 fix: add perf debug info 2023-07-17 15:26:59 +08:00
slzhou 655233fd4f fix: create initial source with blocks 2023-07-17 15:05:30 +08:00
dapan1121 2e0b4236e3 fix: hash join row index issue 2023-07-17 14:45:45 +08:00
liuyao e80ae266ab refact 2023-07-17 14:37:54 +08:00
dapan1121 efbb6cb193 fix: add debug line 2023-07-17 13:50:19 +08:00