Commit Graph

7058 Commits

Author SHA1 Message Date
wangmm0220 1f5532fd67 fix:offset error in tmq & add test cases 2023-08-09 19:41:22 +08:00
Haojun Liao 1e8579e8c5 fix(stream): fix other cases. 2023-08-09 16:04:48 +08:00
Haojun Liao 4c92997328 fix(stream): use streamId&taskId to identify the stream task in the stream meta. 2023-08-09 15:52:23 +08:00
Hui Li 584427dd44
Merge pull request #22374 from taosdata/szhou/restore-minglei-main
fix: restore stt/block merge back
2023-08-09 14:40:03 +08:00
Haojun Liao 3cdb16352c fix(stream): avoid launching check downstream when failed to add stream task into stream meta. 2023-08-09 12:03:14 +08:00
shenglian zhou e9e06d1eae fix: restore stt block/data block merge back 2023-08-09 10:36:54 +08:00
shenglian zhou 2b1478f46e Revert "fix(tsdb/read2): reset stt reader when suspended"
This reverts commit 079d7ff69e.
2023-08-09 10:36:26 +08:00
Haojun Liao bb39d94f9c
Merge pull request #22357 from taosdata/fix/liaohj
other: merge fix from 3.0 to main.
2023-08-09 09:05:54 +08:00
Haojun Liao 17c6143471 fix(stream): check more status when handling the state transfer. 2023-08-08 15:08:11 +08:00
Hongze Cheng c8f106106a fix: update assert issue 2023-08-08 14:49:05 +08:00
Hui Li 05e356cd81
Merge pull request #22346 from taosdata/fix/TD-25612
fix: snapshot invalid read
2023-08-08 09:19:55 +08:00
Hongze Cheng a545d45b5e fix: snapshot invalid read 2023-08-07 16:12:03 +08:00
Hongze Cheng 661f2aca98 fix: memory leak 2023-08-07 14:13:44 +08:00
wade zhang abe15cc68e
Merge pull request #22315 from taosdata/szhou/fix-td25385-main
fix: keep stt iterator valid in the list
2023-08-04 08:02:16 +08:00
slzhou fef091b13d fix: keep the stt iterator valid 2023-08-03 21:50:50 +08:00
shenglian zhou b85c1ae8a1 fix: remove the core 2023-08-03 21:50:50 +08:00
shenglian zhou 5322e38dfe fix: add init add check it 2023-08-03 21:50:50 +08:00
shenglian zhou 97b15ed718 fix: delete SLDataItem if stt level files num is smaller 2023-08-03 21:50:50 +08:00
slzhou 1cb1451e41 fix: set nextIndex when get neighbouring index 2023-08-03 18:26:29 +08:00
wade zhang 13c713d33b
Merge pull request #22290 from taosdata/3.0_main
merge 3.0 into branch main
2023-08-02 16:04:08 +08:00
Hongze Cheng f90f0b7047 disable infinite count loop 2023-08-02 15:51:51 +08:00
Minglei Jin cff85550f9 Merge branch '3.0' into 3.0_main 2023-08-02 14:10:45 +08:00
wade zhang c24b52ad82
Merge pull request #22277 from taosdata/fix/TD-25400
enh(tsdb/cache/read): lazy init table map hash and entries
2023-08-02 14:08:43 +08:00
Haojun Liao 0feffc687b fix(tsdb): check uid in multiple stt statistics blocks. 2023-08-02 12:44:45 +08:00
Minglei Jin 0e51277521 cache/reader: cleanup contiue nex row 2023-08-02 12:43:57 +08:00
Haojun Liao 2c357f1958 fix(stream): enable filter table based on table statistics. 2023-08-02 11:18:35 +08:00
Minglei Jin 7fca057a81 cache/reader: move pLastCols into single block 2023-08-02 11:10:12 +08:00
Minglei Jin 95cc540d9b cache/reader: clean uid in all loop 2023-08-02 10:48:55 +08:00
Minglei Jin ed76525c4f cache/reader: cleanup uid in signle loop 2023-08-02 10:28:13 +08:00
Minglei Jin 29b2f32821 cache reader: cleanup stt init 2023-08-02 09:49:41 +08:00
Haojun Liao 53aefa531f Merge branch '3.0' into fix/3_liaohj 2023-08-02 09:45:21 +08:00
Hongze Cheng 4ba4e83b5b fix: retention file corruption 2023-08-01 20:19:13 +08:00
Minglei Jin 7d294cac5f tsdb/cache: load current fileset's brinblk once 2023-08-01 18:56:10 +08:00
Minglei Jin a593e887fa cache: remove nFileset from fs state 2023-08-01 18:37:23 +08:00
Minglei Jin 7ad628417a fix mem issue: stack-use-after-scope 2023-08-01 17:57:38 +08:00
Minglei Jin 4716fc6400 cache/fs: remove fs state's tsdb handle 2023-08-01 17:49:05 +08:00
Minglei Jin 1d5532019f tsdb/merge-tree: cleanup comment 2023-08-01 17:25:15 +08:00
Minglei Jin f72f129d41 cache/skyline: cleanup del array 2023-08-01 17:23:53 +08:00
Minglei Jin c997662ba0 tsdb/cache: build skyline only if there's del data 2023-08-01 16:50:00 +08:00
Minglei Jin 94b9599f95 fix null pointer dereference 2023-08-01 16:39:14 +08:00
Haojun Liao 637f09359a other: merge main. 2023-08-01 16:15:34 +08:00
Minglei Jin 72ac5a4e3b tsdb/cache: use read util's uid compare functor 2023-08-01 16:08:20 +08:00
Minglei Jin c0a0e07597 cache/read: lazy init uidList when laoding from tsdb 2023-08-01 16:02:09 +08:00
Minglei Jin b3fae65d90 enh(tsdb/cache/read): lazy init table map hash and entries 2023-08-01 15:53:42 +08:00
Hongze Cheng 78c23f407e fix compile issue 2023-08-01 15:04:14 +08:00
Hongze Cheng 07c40943a4 fix: TD-25408 2023-08-01 15:01:06 +08:00
wade zhang 347e78ecb5
Merge pull request #22272 from taosdata/fix/TD-25466
Fix/td 25466
2023-08-01 14:22:24 +08:00
Hongze Cheng 1b39147496 more refact 2023-08-01 11:36:37 +08:00
Hongze Cheng 0e46041c1a fix: tsdbRetention memory leak 2023-08-01 11:21:40 +08:00
Haojun Liao ae9745ecf2
Merge pull request #22258 from taosdata/mark/tmq-3.0
fix:add committed & seek process logic
2023-08-01 09:52:09 +08:00
Haojun Liao ea875c58f2
Merge pull request #22263 from taosdata/mark/tmq-main
fix:add committed & seek process logic
2023-08-01 09:43:23 +08:00
Haojun Liao 36ef9a537f
Merge pull request #22256 from taosdata/fix/TD-25457
check update data
2023-07-31 22:56:05 +08:00
Haojun Liao 2640e41705
Merge pull request #22261 from taosdata/fix/3_liaohj
fix(stream): set the correct end key of delete block.
2023-07-31 22:55:50 +08:00
wangmm0220 68ae64cdb2 fix:add committed & seek process logic 2023-07-31 18:53:25 +08:00
wangmm0220 cddf27e8b4 fix:add committed & seek process logic 2023-07-31 18:48:50 +08:00
Haojun Liao d45596bb33 fix(stream): kill task when pause the stream execution. 2023-07-31 18:07:37 +08:00
wangmm0220 6179618b17 fix:add committed & seek process logic 2023-07-31 17:56:04 +08:00
liuyao 32b59af4b1 check update data 2023-07-31 17:32:24 +08:00
wangmm0220 8093ef2dc8 Merge branch 'main' of https://github.com/taosdata/TDengine into mark/tmq 2023-07-31 16:14:25 +08:00
Haojun Liao 9981cecbed refactor: do some internal refactor. 2023-07-31 15:24:35 +08:00
Minglei Jin 079d7ff69e fix(tsdb/read2): reset stt reader when suspended 2023-07-31 15:21:47 +08:00
wade zhang c7fa5ed6ef
Merge pull request #22247 from taosdata/fix/TD-25436
fix(tsdb/cache): check stt data with empty brin index
2023-07-31 11:24:37 +08:00
Minglei Jin 135beebc0a fix(tsdb/cache): check stt data with empty brin index 2023-07-31 09:15:08 +08:00
wade zhang bae028d65c
Merge pull request #22230 from taosdata/mark/tmq
fix:semaphore always wait in tmq commit logic
2023-07-31 09:10:13 +08:00
wade zhang c26165975d
Merge pull request #22227 from taosdata/fix/TS-3728
fix:add macro ASSERT_NOT_CORE to control ASSERT not core in release version
2023-07-30 14:28:56 +08:00
Haojun Liao 0bd2548937 fix(stream): initialize the filter window initial range. 2023-07-29 16:41:01 +08:00
Haojun Liao f6e07d1fdb other: merge 3.0 2023-07-29 01:32:02 +08:00
wade zhang c19feb2fa1
Merge pull request #22236 from taosdata/fix/TD-25416
fix(tsdb/cache/del): use lru mutex to fix race with committing
2023-07-28 23:17:45 +08:00
Haojun Liao 936afeb5ac refactor: do internal refactor to simple the pause/resume in case of fill history exists. 2023-07-28 19:13:03 +08:00
Haojun Liao d5974a8f25 refactor(stream): refactor the pause/resume for fill history execution. 2023-07-28 18:05:32 +08:00
Minglei Jin 1fcbf3432c tsdb/cache/del: remove duplicate lruMutex lock 2023-07-28 17:20:20 +08:00
wangmm0220 d982a7cb80 fix:heap use after free for wal 2023-07-28 16:59:35 +08:00
Minglei Jin b0f65dfa62 fix(tsdb/cache/del): use lru mutex to fix race with committing 2023-07-28 16:32:07 +08:00
wangmm0220 6e673072cd fix:conflicts 2023-07-28 14:57:13 +08:00
Haojun Liao 2a78cd2d23
Merge pull request #22215 from taosdata/fix/liaohj
other: merge main.
2023-07-28 13:06:17 +08:00
wangmm0220 2f6c565392 fix:pHandle not load if taosd restart 2023-07-28 11:48:53 +08:00
wade zhang b3e5b5bd24
Merge pull request #22217 from taosdata/fix/TD-25386
fix(tsdb/cache): move to stt next row if data file exausted
2023-07-28 08:56:29 +08:00
Haojun Liao ccc0f13649
Merge pull request #22214 from taosdata/fix/3_liaohj
fix(stream): scan wal in step2
2023-07-28 00:42:15 +08:00
Haojun Liao 7f3d1dc464 fix(stream): fix memory leak. 2023-07-27 23:06:53 +08:00
wangmm0220 77d43ef5fd fix:tmq ci error 2023-07-27 20:28:45 +08:00
Haojun Liao fbb33be85d other: add some comments. 2023-07-27 18:59:38 +08:00
Haojun Liao 22a5a18fff fix(stream): 1. set correct timewindow after step2. 2. handle the case when the task is failed to be added into the meta store. 2023-07-27 18:57:36 +08:00
Minglei Jin cc9b9bd0b9 fix(tsdb/cache): move to stt next row if data file exausted 2023-07-27 17:19:42 +08:00
Haojun Liao b145771a1b other: merge 3.0 2023-07-27 16:38:57 +08:00
Haojun Liao 3d403dca5a other: merge main. 2023-07-27 16:23:41 +08:00
Haojun Liao 6ff50d4eaf fix(stream): update the step2 scan wal files. 2023-07-27 16:10:55 +08:00
wangmm0220 75eacd56f1 fix:add log 2023-07-27 15:58:32 +08:00
wade zhang a72c11c8de
Merge pull request #22205 from taosdata/fix/TD-25315
fix(tsdb/commit2): new maxDelKey for commit context
2023-07-27 15:48:46 +08:00
wangmm0220 2d41a42976 Merge branch 'main' of https://github.com/taosdata/TDengine into fix/TS-3728 2023-07-27 15:31:16 +08:00
Haojun Liao 40fe3ef4f6 fix(stream): refactor the step2 wal scan. 2023-07-27 14:49:20 +08:00
Minglei Jin fe495f7013 tsdb/commit2: fix commit minKey for del 2023-07-27 13:54:48 +08:00
Haojun Liao e89f530db2 fix(stream): scan wal in step2 2023-07-27 10:55:59 +08:00
Minglei Jin 8a2764c698 tsdb/commit2: use max fileset's maxKey as initial maxDelKey 2023-07-27 10:51:17 +08:00
Minglei Jin dd32f12f92 tsdb/commit2: reset nextKey to max to stop commit 2023-07-27 09:50:32 +08:00
Minglei Jin 992d623ae5 fix(tsdb/commit2): initialize ctx's maxDelKey with TSKEY_MIN 2023-07-27 09:34:47 +08:00
Minglei Jin 580b7124cf fix(tsdb/commit2): new maxDelKey for commit context 2023-07-27 09:16:28 +08:00
Haojun Liao 087d54a807
Merge pull request #22198 from taosdata/fix/3_liaohj
fix(stream): do filter before the update check.
2023-07-26 18:57:48 +08:00
Haojun Liao c57defa1c9
Merge pull request #22195 from taosdata/szhou/fix-td25369
fix: stt and data block merge
2023-07-26 17:11:18 +08:00
Haojun Liao 6a6830761d fix(tsdb): check for null ptr of pScanInfo 2023-07-26 17:08:29 +08:00
Haojun Liao 3c387f6e91 fix(stream): fix syntax error. 2023-07-26 17:00:41 +08:00