Commit Graph

386 Commits

Author SHA1 Message Date
wangmm0220 8a81278024 fix:load pHandle if necessary wher tq restore 2023-06-13 15:14:11 +08:00
Haojun Liao b5b4cd2a05 fix(query): load del info with upper version limitation. 2023-06-02 23:15:27 +08:00
Haojun Liao 5301c5756a other: merge main into 3.0. 2023-06-01 09:29:18 +08:00
Haojun Liao b38e253019 fix(stream): set the skip offset when resume stream task. 2023-05-26 22:51:58 +08:00
wangmm0220 a3e214b9e8 fix:put poll to push manager if wal not exist when offset is latest 2023-05-26 19:17:04 +08:00
Haojun Liao 5d5b2bb16a refactor: do some internal refactor. 2023-05-18 12:59:19 +08:00
Haojun Liao 041ffa91bd other: merge main. 2023-05-17 15:42:15 +08:00
Haojun Liao 366cd1871a
Merge pull request #21314 from taosdata/fix/liaohj_main
fix(stream):extract delete msg from wal.
2023-05-17 14:52:49 +08:00
wangmm0220 c52a4ea4f0 fix:merge from 3.0 2023-05-17 01:11:55 +08:00
wangmm0220 e6e0ac7603 fix:alwalys in exec if consumer Id mismatch & check result for wal seek ver 2023-05-16 14:01:38 +08:00
Haojun Liao 30e7cb5860 fix(tmq): avoid return delete msg for table subscription. 2023-05-16 10:42:02 +08:00
Haojun Liao 5e16db4e19 fix(stream):extract delete msg from wal. 2023-05-16 08:54:03 +08:00
Haojun Liao d585f34ea9 fix(tmq): avoid seek to previous position. 2023-05-15 18:48:50 +08:00
wangmm0220 36454c73ed fix:add lock to pHandle for safety 2023-05-15 18:38:14 +08:00
Haojun Liao 236926b9ee other: merge enh/rocksdbstate 2023-05-13 19:49:28 +08:00
Haojun Liao 1975e94501 Merge branch 'main' into enh/rocksdbSstate 2023-05-13 19:41:03 +08:00
Benguang Zhao b8933389bb enh: keep the trailing empty wal file in walCheckAdnRepairMeta 2023-05-11 09:42:46 +08:00
Benguang Zhao 7433bad888 enh: tidy up walEndSnapshot 2023-05-11 09:14:37 +08:00
Benguang Zhao 2a558323be fix: set absent closeTs with mtime of log files during walCheckAndRepairMeta 2023-05-10 20:35:08 +08:00
yihaoDeng 5708fcb5cb Merge branch 'enh/3.0' into enh/rocksdbSstateMerge 2023-05-10 08:02:16 +00:00
Yihao Deng c6e5879f44
Merge branch 'main' into enh/rocksdbSstate 2023-05-10 10:04:28 +08:00
yihaoDeng 8356533eb1 Merge branch 'enh/standVer' into enh/rocksdbSstateMerge 2023-05-09 12:13:16 +00:00
wangmm0220 587f750c94 fix:[TD-24058]send poll result to client if no data 5 times to avoid lost data 2023-05-09 17:42:11 +08:00
wangmm0220 f9142c0ddb fix:[TD24010] lost data if apply ver is small than commit ver 2023-05-08 16:49:56 +08:00
Yihao Deng cb223acf82
Merge branch 'main' into enh/rocksdbSstate 2023-05-08 09:45:27 +08:00
wangmm0220 261f2736f7 fix:[TD-24010] subscribeStb.py failed in windows CI 2023-05-06 19:17:36 +08:00
Haojun Liao 6b305bf745 refactor: do some internal refactor. 2023-05-06 13:35:18 +08:00
Haojun Liao da6212bab0 other: merge main. 2023-05-06 12:51:38 +08:00
Haojun Liao 29fcd1b83a refactor: do some internal refactor. 2023-05-06 10:08:05 +08:00
Haojun Liao ace193ca9b fix(stream): set the seek version if the reader's version is earlier than the first version in wal. 2023-04-29 23:34:49 +08:00
wangmm0220 1c63408b3e opti:change push mgr to consume msg for subscribe 2023-04-24 20:18:20 +08:00
Haojun Liao c654f1145d enh(stream): add new msg for seek, and do some internal refactor. 2023-04-20 10:59:16 +08:00
wangmm0220 4d9d1b520d fix:add log for wal 2023-04-19 18:16:57 +08:00
wangmm0220 6f94281ab7 fix:add log for wal 2023-04-19 17:30:58 +08:00
Haojun Liao f8b672f09f other: merge main. 2023-04-14 09:07:18 +08:00
Xiaoyu Wang 8b8ac0cd77 merge main 2023-04-12 10:43:28 +08:00
Haojun Liao f6315af0c9 other: merge main. 2023-04-11 19:41:17 +08:00
Xiaoyu Wang 4c04b155d5 merge main 2023-04-11 15:56:28 +08:00
wangmm0220 c40c695ea7 fix:optimize log & change the length of tag if tag is null in schemaless 2023-04-11 13:49:18 +08:00
Haojun Liao 7419c0bfe9 enh(stream): the stream will start after vnode restore being completed. 2023-04-10 16:56:14 +08:00
Benguang Zhao 22d92cd3dc feat: WAL cleanup not affected by topic consumption anymore 2023-03-23 18:59:14 +08:00
Benguang Zhao 75b922c2be feat: retentionSize as 0 means unlimitted in size 2023-03-22 15:01:01 +08:00
Benguang Zhao 0b101e4952 feat: support WAL_RETENTION_PERIOD and WAL_RETENTION_SIZE for alter database 2023-03-22 11:40:24 +08:00
wangmm0220 7616a283e3 fix:error in TD-23218 & remove useless logic 2023-03-18 18:23:38 +08:00
Benguang Zhao 22e236f4e6 fix: check pWal->cfg.retentionPeriod properly 2023-03-07 15:15:46 +08:00
Benguang Zhao ca23365d68 enh: fsync WAL idx and log when walRollImpl 2023-03-02 20:40:34 +08:00
wangmm0220 c638564179 fix:confilcts from main 2023-02-26 23:09:57 +08:00
Xiaoyu Wang d6dd087f84 merge main 2023-02-25 10:29:20 +08:00
wangmm0220 54a546931c fix:optimize version logic in tmq and remove useless code 2023-02-24 19:22:08 +08:00
Xiaoyu Wang d04d610a85
Merge pull request #20061 from taosdata/FIX/TD-22611-main
enh: refactor syncBeginSnapshot and walBeginSnapshot for logRetention
2023-02-24 13:12:06 +08:00
Benguang Zhao e45367f924 fix: resolve coverity scan issues in sync and wal 2023-02-23 14:16:21 +08:00
wangmm0220 b28c0ad952 fix:remove useless logic in tmq 2023-02-21 18:49:39 +08:00
Benguang Zhao 0c7a4bfad9 enh: refactor syncBeginSnapshot and walBeginSnapshot for logRetention 2023-02-20 19:19:49 +08:00
Xiaoyu Wang c597de099f merge main 2023-02-20 13:41:27 +08:00
Benguang Zhao bcf6d5b641 fix: clear some asserts in walRead.c 2023-02-16 16:06:12 +08:00
Haojun Liao fd379310f6
Merge pull request #19979 from taosdata/fix/tq_oom
fix: handle insufficient resource
2023-02-16 09:17:52 +08:00
Liu Jicong e4a365c265 remove assert 2023-02-14 16:36:55 +08:00
Haojun Liao c597a1e4ce refactor: do some internal refactor. 2023-02-13 15:46:47 +08:00
Xiaoyu Wang 3382a24a69 merge main 2023-02-10 18:03:11 +08:00
Benguang Zhao 5ce5dded75 enh: reset wal reader after walReadVer 2023-02-08 14:41:15 +08:00
Hongze Cheng 0f00862d8b Merge branch 'fix/TD-22070' of https://github.com/taosdata/TDengine into fix/3.0_merge_main 2023-01-30 10:17:59 +08:00
Shengliang Guan 98e174606c
Merge pull request #19578 from taosdata/FIX/TD-21942-main
enh: trim the last WAL idx file which is not fsynced
2023-01-30 09:31:28 +08:00
Hongze Cheng 5e7b6602c5 Merge branch 'main' of https://github.com/taosdata/TDengine into fix/3.0_merge_main 2023-01-29 14:05:15 +08:00
Liu Jicong f5c4ca3380 fix: wal ref 2023-01-28 17:17:55 +08:00
Liu Jicong 2a71d47668 fix: wal ref 2023-01-28 16:11:17 +08:00
Benguang Zhao 95493432ed enh: trim the last WAL idx file which is not fsynced 2023-01-16 14:12:26 +08:00
Xiaoyu Wang 9a0c2805f4 merge main 2023-01-13 13:54:04 +08:00
Benguang Zhao 97ce96a2fa enh: not fsync idx file in walFsync 2023-01-11 19:24:51 +08:00
Benguang Zhao df6e9631e1 fix: synchronize access within walFsync 2023-01-10 11:24:05 +08:00
Shengliang Guan 398161eb5f Merge branch 'main' into merge/mainto3.0_0108 2023-01-09 10:00:53 +08:00
Benguang Zhao 2890a8cb96 fix: return error on failing to truncate raft log 2022-12-28 20:34:40 +08:00
Benguang Zhao 0c4ade9373 fix: update sync node commitIndex only if matchTerm equals currentTerm upon heartbeat 2022-12-28 20:25:52 +08:00
Benguang Zhao c4fad84c7c enh: reset commitVer in WAL on restore 2022-12-28 20:25:52 +08:00
Haojun Liao fbc9e77004
Merge branch '3.0' into refact/submit_req 2022-12-26 10:26:17 +08:00
kailixu ed3a0c1dc1 chore: vgId format uniform 2022-12-24 23:44:01 +08:00
Xiaoyu Wang 0c269c317b merge 3.0 2022-12-23 17:50:05 +08:00
Liu Jicong 6cbcbd1402 remove assert 2022-12-21 15:37:50 +08:00
Liu Jicong 50729f0cc4 refactor: remove assert 2022-12-21 14:16:28 +08:00
Liu Jicong 2a7de0cd46 refactor: remove assert 2022-12-21 13:54:14 +08:00
Shengliang Guan 41dcd32b8e refact: adjust some logs 2022-12-20 15:01:03 +08:00
Haojun Liao 48f7fc5c3d
Merge branch '3.0' into refact/submit_req 2022-12-12 18:11:17 +08:00
Liu Jicong a6fff95795 fix: close wal ref 2022-12-09 17:18:21 +08:00
Liu Jicong 271822ebc4 refactor: support submitreq2 2022-12-08 01:55:47 +08:00
Shengliang Guan 6d94afe48f
Revert "refact: adjust some assert cases" 2022-12-07 21:51:17 +08:00
Shengliang Guan d5126d469a refact: replcase ASSERT with tAssert 2022-12-07 18:42:48 +08:00
Shengliang Guan f69a188f7e refact: replcase ASSERT with tAssert 2022-12-07 18:37:40 +08:00
Benguang Zhao 3f1a0cefbb fix: walSaveMeta ahead of deleting WAL files as gc in walEndSnapshot
delete WAL files in descending order during walRollback
2022-12-06 00:51:45 +08:00
Shengliang Guan d9c261cd13 refact: remove some error codes 2022-12-03 10:17:02 +08:00
Benguang Zhao 68f8ebac9d Merge branch 3.0 to FIX/TD-19334-3.0 2022-11-25 13:10:06 +08:00
Benguang Zhao f68e41a40e enh: initialize log buffer again after receiving a complete snapshot 2022-11-23 15:57:40 +08:00
Benguang Zhao 95f8e96eb5 fix: align firstVer with lastVer too if no WAL logs exist 2022-11-22 23:42:08 +08:00
Liu Jicong 1344671869 fix(wal): add debug log 2022-11-22 17:24:43 +08:00
Liu Jicong 6f5d90658e fix(wal): add debug log 2022-11-22 16:40:18 +08:00
Minghao Li b600b4c8bc refactor(sync): modify wal pInfo++, to find which case can not pass 2022-11-21 16:18:50 +08:00
Benguang Zhao 051b73c1ed Merge branch 3.0 to FIX/TD-19334-3.0 2022-11-17 17:28:11 +08:00
Shengliang Guan 2b5257f905 fix: memory lead found in asan 2022-11-16 20:45:15 +08:00
Shengliang Guan c8f9707d5a
Merge pull request #18203 from taosdata/feature/stream
enh(stream): add tbname map cache
2022-11-16 17:15:30 +08:00
Shengliang Guan 201ce10fbc fix: restore wal info changes 2022-11-16 15:45:40 +08:00
Liu Jicong ab57d069d4 enh(stream): add tbname map cache 2022-11-16 15:18:31 +08:00
Shengliang Guan b62a85dfdc
Merge pull request #18192 from taosdata/feature/stream
fix(stream): timer refer stream task
2022-11-16 12:40:50 +08:00