Commit Graph

25158 Commits

Author SHA1 Message Date
wade zhang 12ac393fb0
Merge pull request #22287 from taosdata/fix/3_liaohj
other: merge 3.0
2023-08-02 15:10:04 +08:00
Minglei Jin cff85550f9 Merge branch '3.0' into 3.0_main 2023-08-02 14:10:45 +08:00
wade zhang c24b52ad82
Merge pull request #22277 from taosdata/fix/TD-25400
enh(tsdb/cache/read): lazy init table map hash and entries
2023-08-02 14:08:43 +08:00
Haojun Liao 0feffc687b fix(tsdb): check uid in multiple stt statistics blocks. 2023-08-02 12:44:45 +08:00
Minglei Jin 0e51277521 cache/reader: cleanup contiue nex row 2023-08-02 12:43:57 +08:00
Haojun Liao 2c357f1958 fix(stream): enable filter table based on table statistics. 2023-08-02 11:18:35 +08:00
Minglei Jin 7fca057a81 cache/reader: move pLastCols into single block 2023-08-02 11:10:12 +08:00
Minglei Jin 95cc540d9b cache/reader: clean uid in all loop 2023-08-02 10:48:55 +08:00
Minglei Jin ed76525c4f cache/reader: cleanup uid in signle loop 2023-08-02 10:28:13 +08:00
Minglei Jin 29b2f32821 cache reader: cleanup stt init 2023-08-02 09:49:41 +08:00
Haojun Liao 53aefa531f Merge branch '3.0' into fix/3_liaohj 2023-08-02 09:45:21 +08:00
Haojun Liao 6b09566549
Merge pull request #22275 from taosdata/enh/addKeepAliveParam
add keep alive parameter
2023-08-02 09:23:03 +08:00
wade zhang d673643aff
Merge pull request #22284 from taosdata/fix/TD-25138-3.0
fix: null pointer check for mutex on windows
2023-08-01 23:04:57 +08:00
Hongze Cheng 4ba4e83b5b fix: retention file corruption 2023-08-01 20:19:13 +08:00
kailixu 42da90e459 chore: code revert 2023-08-01 19:45:14 +08:00
kailixu 2a22e55705 fix: null pointer check for mutex on windows 2023-08-01 19:42:05 +08:00
Haojun Liao 3296aecf06 Merge branch '3.0' into fix/3_liaohj 2023-08-01 19:24:29 +08:00
Minglei Jin 7d294cac5f tsdb/cache: load current fileset's brinblk once 2023-08-01 18:56:10 +08:00
Minglei Jin a593e887fa cache: remove nFileset from fs state 2023-08-01 18:37:23 +08:00
liuyao 71b000f08c reload state window state 2023-08-01 18:22:40 +08:00
Minglei Jin 7ad628417a fix mem issue: stack-use-after-scope 2023-08-01 17:57:38 +08:00
Minglei Jin 4716fc6400 cache/fs: remove fs state's tsdb handle 2023-08-01 17:49:05 +08:00
Minglei Jin 1d5532019f tsdb/merge-tree: cleanup comment 2023-08-01 17:25:15 +08:00
Minglei Jin f72f129d41 cache/skyline: cleanup del array 2023-08-01 17:23:53 +08:00
Minglei Jin c997662ba0 tsdb/cache: build skyline only if there's del data 2023-08-01 16:50:00 +08:00
Minglei Jin 94b9599f95 fix null pointer dereference 2023-08-01 16:39:14 +08:00
Haojun Liao 637f09359a other: merge main. 2023-08-01 16:15:34 +08:00
Minglei Jin 72ac5a4e3b tsdb/cache: use read util's uid compare functor 2023-08-01 16:08:20 +08:00
wade zhang 26375e834a
Merge pull request #22276 from taosdata/fix/TD-25408
Fix: TD-25408
2023-08-01 16:06:03 +08:00
Minglei Jin c0a0e07597 cache/read: lazy init uidList when laoding from tsdb 2023-08-01 16:02:09 +08:00
Minglei Jin b3fae65d90 enh(tsdb/cache/read): lazy init table map hash and entries 2023-08-01 15:53:42 +08:00
Haojun Liao 6365d5000b
Merge pull request #22271 from taosdata/fix/3_liaohj
fix(stream): dump results to sink node before paused.
2023-08-01 15:50:17 +08:00
Haojun Liao fb98ede512
Merge pull request #22266 from taosdata/fix/3_liaohj
fix(stream): dump results to sink node before paused.
2023-08-01 15:49:59 +08:00
Hongze Cheng 78c23f407e fix compile issue 2023-08-01 15:04:14 +08:00
Hongze Cheng 07c40943a4 fix: TD-25408 2023-08-01 15:01:06 +08:00
yihaoDeng 2caaae392a add keep alive parameter 2023-08-01 06:38:30 +00:00
Haojun Liao f1e0e64918 fix(stream): add more check. 2023-08-01 14:31:41 +08:00
wade zhang 347e78ecb5
Merge pull request #22272 from taosdata/fix/TD-25466
Fix/td 25466
2023-08-01 14:22:24 +08:00
Hongze Cheng 1b39147496 more refact 2023-08-01 11:36:37 +08:00
Hongze Cheng 0e46041c1a fix: tsdbRetention memory leak 2023-08-01 11:21:40 +08:00
wade zhang 6da5276259
Merge pull request #22264 from taosdata/szhou/fix-ts3749
fix: restore user privilege processing if the rows generated by the u…
2023-08-01 11:21:11 +08:00
Haojun Liao e11cda17c0 fix(stream): remove invalid check. 2023-08-01 10:30:24 +08:00
Haojun Liao f6ae9cf9bb fix(stream): fetch all data before paused and dump to sink node. 2023-08-01 10:16:14 +08:00
Haojun Liao ae9745ecf2
Merge pull request #22258 from taosdata/mark/tmq-3.0
fix:add committed & seek process logic
2023-08-01 09:52:09 +08:00
slzhou 785326c494 fix: dedicated max privileges rows to 2048 2023-08-01 09:46:08 +08:00
Haojun Liao ea875c58f2
Merge pull request #22263 from taosdata/mark/tmq-main
fix:add committed & seek process logic
2023-08-01 09:43:23 +08:00
slzhou 411b5d4cf4 fix: increase privileges max block row number to 4096 2023-08-01 09:41:59 +08:00
Haojun Liao b83860372c fix(stream): dump results to sink node before paused. 2023-08-01 00:50:30 +08:00
slzhou 65e028ea5a enhance: add log 2023-07-31 23:27:35 +08:00
Haojun Liao 36ef9a537f
Merge pull request #22256 from taosdata/fix/TD-25457
check update data
2023-07-31 22:56:05 +08:00
Haojun Liao 2640e41705
Merge pull request #22261 from taosdata/fix/3_liaohj
fix(stream): set the correct end key of delete block.
2023-07-31 22:55:50 +08:00
Haojun Liao 7e8ecceada
Merge pull request #22260 from taosdata/fix/3_liaohj
fix(stream): kill task when pause the stream execution.
2023-07-31 22:55:36 +08:00
Haojun Liao 8c1c17e31e fix(stream): ignore the sink task transfer state. 2023-07-31 19:23:53 +08:00
slzhou 215e358ce3 fix: restore user privilege processing if the rows generated by the user exceeds max block rows 2023-07-31 19:20:54 +08:00
Haojun Liao d72580dad7 fix(stream): transfer the state for agg tasks. 2023-07-31 19:12:23 +08:00
wangmm0220 68ae64cdb2 fix:add committed & seek process logic 2023-07-31 18:53:25 +08:00
wangmm0220 cddf27e8b4 fix:add committed & seek process logic 2023-07-31 18:48:50 +08:00
Haojun Liao d45596bb33 fix(stream): kill task when pause the stream execution. 2023-07-31 18:07:37 +08:00
Haojun Liao 0b732e512a
Merge pull request #22251 from taosdata/fix/3_liaohj
fix(stream): set the correct end key of delete block.
2023-07-31 18:06:48 +08:00
wangmm0220 6179618b17 fix:add committed & seek process logic 2023-07-31 17:56:04 +08:00
wade zhang 9675384b05
Merge pull request #22252 from taosdata/fix/TD-25391
fix(tsdb/read2): reset stt reader when suspended
2023-07-31 17:36:43 +08:00
liuyao 32b59af4b1 check update data 2023-07-31 17:32:24 +08:00
wade zhang 64d63b0584
Merge pull request #22212 from taosdata/fix/TD-24809-3.0
fix: taos print version
2023-07-31 16:31:03 +08:00
wangmm0220 8093ef2dc8 Merge branch 'main' of https://github.com/taosdata/TDengine into mark/tmq 2023-07-31 16:14:25 +08:00
Haojun Liao 9981cecbed refactor: do some internal refactor. 2023-07-31 15:24:35 +08:00
Minglei Jin 079d7ff69e fix(tsdb/read2): reset stt reader when suspended 2023-07-31 15:21:47 +08:00
wade zhang 79646fff10
Merge pull request #22246 from taosdata/szhou/fix-ts3744
fix: increase condition size
2023-07-31 14:48:06 +08:00
Haojun Liao fe21943c9c fix(stream): set the correct end key of delete block. 2023-07-31 14:23:46 +08:00
Haojun Liao f359a0dca3 fix(stream): set the correct end key of delete block. 2023-07-31 14:22:46 +08:00
slzhou 6151b790d9 fix: increase the condition size of user privileges to 48k 2023-07-31 13:25:56 +08:00
wade zhang c7fa5ed6ef
Merge pull request #22247 from taosdata/fix/TD-25436
fix(tsdb/cache): check stt data with empty brin index
2023-07-31 11:24:37 +08:00
Minglei Jin 135beebc0a fix(tsdb/cache): check stt data with empty brin index 2023-07-31 09:15:08 +08:00
wade zhang e5f3835e48
Merge pull request #22229 from taosdata/fix/TS-3724
fix:modify dot to underline in schemaless supertable name
2023-07-31 09:10:34 +08:00
wade zhang bae028d65c
Merge pull request #22230 from taosdata/mark/tmq
fix:semaphore always wait in tmq commit logic
2023-07-31 09:10:13 +08:00
kailixu ff39aadb34 fix: mac OS for community 2023-07-31 08:42:17 +08:00
liuyao 09c806539f fix double free 2023-07-30 17:21:42 +08:00
wade zhang 3a59b95211
Merge pull request #22226 from taosdata/wjm/fix/TD-25374
fix: window fill ts range check with int64 overflow
2023-07-30 14:32:31 +08:00
wade zhang c26165975d
Merge pull request #22227 from taosdata/fix/TS-3728
fix:add macro ASSERT_NOT_CORE to control ASSERT not core in release version
2023-07-30 14:28:56 +08:00
Haojun Liao c3c9e32ddd
Merge pull request #22244 from taosdata/fix/3_liaohj
other: merge 3.0
2023-07-29 22:38:51 +08:00
Haojun Liao 80ad5a0abb
Merge pull request #22243 from taosdata/fix/3_liaohj
fix(stream): initialize the filter window initial range.
2023-07-29 21:10:42 +08:00
Haojun Liao 399088e2bd fix(stream): remove invalid code. 2023-07-29 19:23:35 +08:00
Haojun Liao 94551e6e46 fix(stream): update the load block position. 2023-07-29 18:48:22 +08:00
Haojun Liao e0c310215c
Merge pull request #22237 from taosdata/fix/TD-25420
init scan state
2023-07-29 16:55:43 +08:00
Haojun Liao 71e16d1eeb
Merge pull request #22238 from taosdata/fix/TD-25423
save compact window info
2023-07-29 16:55:27 +08:00
Haojun Liao 0bd2548937 fix(stream): initialize the filter window initial range. 2023-07-29 16:41:01 +08:00
kailixu 2deeed87ae fix: community for mac os 2023-07-29 13:18:17 +08:00
Haojun Liao f6e07d1fdb other: merge 3.0 2023-07-29 01:32:02 +08:00
Haojun Liao 92d1dbd9ad fix(stream): set the time window filter before generating create table result. 2023-07-29 01:23:06 +08:00
wade zhang c19feb2fa1
Merge pull request #22236 from taosdata/fix/TD-25416
fix(tsdb/cache/del): use lru mutex to fix race with committing
2023-07-28 23:17:45 +08:00
Haojun Liao 6c9fa4d7c8 fix(stream): check for pause when handling the fill history data. 2023-07-28 19:58:52 +08:00
liuyao 73ed6fe649 save compace window info 2023-07-28 19:30:26 +08:00
Haojun Liao 936afeb5ac refactor: do internal refactor to simple the pause/resume in case of fill history exists. 2023-07-28 19:13:03 +08:00
Haojun Liao d5974a8f25 refactor(stream): refactor the pause/resume for fill history execution. 2023-07-28 18:05:32 +08:00
liuyao d712124921 init scan state 2023-07-28 17:25:56 +08:00
Minglei Jin 1fcbf3432c tsdb/cache/del: remove duplicate lruMutex lock 2023-07-28 17:20:20 +08:00
wangmm0220 d982a7cb80 fix:heap use after free for wal 2023-07-28 16:59:35 +08:00
Minglei Jin b0f65dfa62 fix(tsdb/cache/del): use lru mutex to fix race with committing 2023-07-28 16:32:07 +08:00
wangmm0220 ce7db7bf90 fix:subscribe time & return -1 if wal not exist 2023-07-28 15:07:15 +08:00
wangmm0220 1f8e5eae6a fix:conflicts 2023-07-28 15:04:12 +08:00
wangmm0220 6e673072cd fix:conflicts 2023-07-28 14:57:13 +08:00
wangjiaming0909 6713882b48 fix: window fill ts range check with int64 overflow 2023-07-28 14:17:21 +08:00
dapan1121 9c7b53083b
Merge pull request #22223 from taosdata/fix/td-25378
fix: fix memleak during create initial sources for block merge sort
2023-07-28 13:09:34 +08:00
Haojun Liao 2a78cd2d23
Merge pull request #22215 from taosdata/fix/liaohj
other: merge main.
2023-07-28 13:06:17 +08:00
wangmm0220 2f6c565392 fix:pHandle not load if taosd restart 2023-07-28 11:48:53 +08:00
Haojun Liao 0a7e4c75f7 Merge remote-tracking branch 'origin/fix/liaohj' into fix/liaohj 2023-07-28 10:59:46 +08:00
Haojun Liao 6459587ae7 other: fix merge error. 2023-07-28 10:59:34 +08:00
slzhou bcf4c1ebae fix: fix memleak during create initial sources for block merge sort 2023-07-28 09:54:56 +08:00
Haojun Liao 688884de08
Merge pull request #22220 from taosdata/fix/ly_stream_fill
fix stream fill issue
2023-07-28 09:27:21 +08:00
wade zhang b3e5b5bd24
Merge pull request #22217 from taosdata/fix/TD-25386
fix(tsdb/cache): move to stt next row if data file exausted
2023-07-28 08:56:29 +08:00
Haojun Liao ccc0f13649
Merge pull request #22214 from taosdata/fix/3_liaohj
fix(stream): scan wal in step2
2023-07-28 00:42:15 +08:00
Haojun Liao 7f3d1dc464 fix(stream): fix memory leak. 2023-07-27 23:06:53 +08:00
wangmm0220 77d43ef5fd fix:tmq ci error 2023-07-27 20:28:45 +08:00
Haojun Liao 9f301bdd79
Merge pull request #22218 from taosdata/fix/TD-24620
adjust history task trigger mode
2023-07-27 19:29:56 +08:00
Haojun Liao fbb33be85d other: add some comments. 2023-07-27 18:59:38 +08:00
Haojun Liao 22a5a18fff fix(stream): 1. set correct timewindow after step2. 2. handle the case when the task is failed to be added into the meta store. 2023-07-27 18:57:36 +08:00
liuyao f667a09cc6 fix stream fill issue 2023-07-27 18:56:25 +08:00
liuyao 5d21f50340 adjust history task trigger mode 2023-07-27 17:45:28 +08:00
Minglei Jin cc9b9bd0b9 fix(tsdb/cache): move to stt next row if data file exausted 2023-07-27 17:19:42 +08:00
Haojun Liao 6319cb8dd4 fix: fix syntax error. 2023-07-27 17:16:45 +08:00
Shungang Li fb226c2cdb enh: check cluster ttlChangeOnWrite consistency 2023-07-27 16:49:56 +08:00
Haojun Liao b145771a1b other: merge 3.0 2023-07-27 16:38:57 +08:00
Haojun Liao 3d403dca5a other: merge main. 2023-07-27 16:23:41 +08:00
Haojun Liao 3fdbd5383f Merge remote-tracking branch 'origin/fix/3_liaohj' into fix/3_liaohj 2023-07-27 16:11:07 +08:00
Haojun Liao 6ff50d4eaf fix(stream): update the step2 scan wal files. 2023-07-27 16:10:55 +08:00
wangmm0220 75eacd56f1 fix:add log 2023-07-27 15:58:32 +08:00
liuyao 7c9898cfa5 adj stream scan 2023-07-27 15:56:06 +08:00
wade zhang a72c11c8de
Merge pull request #22205 from taosdata/fix/TD-25315
fix(tsdb/commit2): new maxDelKey for commit context
2023-07-27 15:48:46 +08:00
wade zhang cc3cc73cfc
Merge pull request #22199 from taosdata/fix/TS-3708
fix/TS-3708: check term for role time
2023-07-27 15:43:23 +08:00
wangmm0220 2d41a42976 Merge branch 'main' of https://github.com/taosdata/TDengine into fix/TS-3728 2023-07-27 15:31:16 +08:00
liuyao c54bd55c68 op stream scan 2023-07-27 15:16:22 +08:00
liuyao c72274504e op stream scan 2023-07-27 15:13:57 +08:00
liuyao 0b53bdbae6 op stream scan 2023-07-27 15:09:25 +08:00
liuyao 0ac7ebe9c7 op stream scan 2023-07-27 14:56:06 +08:00
Haojun Liao 40fe3ef4f6 fix(stream): refactor the step2 wal scan. 2023-07-27 14:49:20 +08:00
Shungang Li 167ecea705 fix: check term for role time
1. time of mnode becomes current role in this term
2. show 0 for offline mnodes
2023-07-27 14:16:54 +08:00
Haojun Liao d9fe1a3107
Merge pull request #22209 from taosdata/fix/TD-25392
optimize session operator reload state
2023-07-27 14:10:46 +08:00
Minglei Jin fe495f7013 tsdb/commit2: fix commit minKey for del 2023-07-27 13:54:48 +08:00
wangmm0220 5fa580960e fix:semaphore always wait in tmq commit logic 2023-07-27 13:53:42 +08:00
Haojun Liao e89f530db2 fix(stream): scan wal in step2 2023-07-27 10:55:59 +08:00
Minglei Jin 8a2764c698 tsdb/commit2: use max fileset's maxKey as initial maxDelKey 2023-07-27 10:51:17 +08:00
Haojun Liao 2ef67b326a
Merge pull request #22207 from taosdata/fix/3_liaohj
fix(stream): set the value of ratio of vnode stream thread factor to be 4.0 by default.
2023-07-27 10:36:37 +08:00
wade zhang 866f469bc1
Merge pull request #22203 from taosdata/FIX/TD-25382-3.0
fix: not to walEndSnapshot in syncNodeTimerRoutine
2023-07-27 10:15:55 +08:00
wangmm0220 68eb1cfc1e fix:semaphore always wait in tmq commit logic 2023-07-27 10:03:18 +08:00
liuyao 1894608b80 optimize session operator reload state 2023-07-27 09:59:58 +08:00
wangmm0220 11e690cb67 fix:semaphore always wait in tmq commit logic 2023-07-27 09:57:52 +08:00
Minglei Jin dd32f12f92 tsdb/commit2: reset nextKey to max to stop commit 2023-07-27 09:50:32 +08:00
Haojun Liao 05504023f5
Merge pull request #22200 from taosdata/fix/ly_stream_test
stream fill operator mem leak
2023-07-27 09:38:22 +08:00
Minglei Jin 992d623ae5 fix(tsdb/commit2): initialize ctx's maxDelKey with TSKEY_MIN 2023-07-27 09:34:47 +08:00
Haojun Liao a4e1901309 fix(stream): set the value of ratio of vnode stream thread factor to be 4.0 by default. 2023-07-27 09:32:15 +08:00
Minglei Jin 580b7124cf fix(tsdb/commit2): new maxDelKey for commit context 2023-07-27 09:16:28 +08:00