Commit Graph

368 Commits

Author SHA1 Message Date
Benguang Zhao 2a558323be fix: set absent closeTs with mtime of log files during walCheckAndRepairMeta 2023-05-10 20:35:08 +08:00
yihaoDeng 5708fcb5cb Merge branch 'enh/3.0' into enh/rocksdbSstateMerge 2023-05-10 08:02:16 +00:00
Yihao Deng c6e5879f44
Merge branch 'main' into enh/rocksdbSstate 2023-05-10 10:04:28 +08:00
yihaoDeng 8356533eb1 Merge branch 'enh/standVer' into enh/rocksdbSstateMerge 2023-05-09 12:13:16 +00:00
wangmm0220 587f750c94 fix:[TD-24058]send poll result to client if no data 5 times to avoid lost data 2023-05-09 17:42:11 +08:00
wangmm0220 f9142c0ddb fix:[TD24010] lost data if apply ver is small than commit ver 2023-05-08 16:49:56 +08:00
Yihao Deng cb223acf82
Merge branch 'main' into enh/rocksdbSstate 2023-05-08 09:45:27 +08:00
wangmm0220 261f2736f7 fix:[TD-24010] subscribeStb.py failed in windows CI 2023-05-06 19:17:36 +08:00
Haojun Liao 6b305bf745 refactor: do some internal refactor. 2023-05-06 13:35:18 +08:00
Haojun Liao da6212bab0 other: merge main. 2023-05-06 12:51:38 +08:00
Haojun Liao 29fcd1b83a refactor: do some internal refactor. 2023-05-06 10:08:05 +08:00
Haojun Liao ace193ca9b fix(stream): set the seek version if the reader's version is earlier than the first version in wal. 2023-04-29 23:34:49 +08:00
wangmm0220 1c63408b3e opti:change push mgr to consume msg for subscribe 2023-04-24 20:18:20 +08:00
Haojun Liao c654f1145d enh(stream): add new msg for seek, and do some internal refactor. 2023-04-20 10:59:16 +08:00
wangmm0220 4d9d1b520d fix:add log for wal 2023-04-19 18:16:57 +08:00
wangmm0220 6f94281ab7 fix:add log for wal 2023-04-19 17:30:58 +08:00
Haojun Liao f8b672f09f other: merge main. 2023-04-14 09:07:18 +08:00
Xiaoyu Wang 8b8ac0cd77 merge main 2023-04-12 10:43:28 +08:00
Haojun Liao f6315af0c9 other: merge main. 2023-04-11 19:41:17 +08:00
Xiaoyu Wang 4c04b155d5 merge main 2023-04-11 15:56:28 +08:00
wangmm0220 c40c695ea7 fix:optimize log & change the length of tag if tag is null in schemaless 2023-04-11 13:49:18 +08:00
Haojun Liao 7419c0bfe9 enh(stream): the stream will start after vnode restore being completed. 2023-04-10 16:56:14 +08:00
Benguang Zhao 22d92cd3dc feat: WAL cleanup not affected by topic consumption anymore 2023-03-23 18:59:14 +08:00
Benguang Zhao 75b922c2be feat: retentionSize as 0 means unlimitted in size 2023-03-22 15:01:01 +08:00
Benguang Zhao 0b101e4952 feat: support WAL_RETENTION_PERIOD and WAL_RETENTION_SIZE for alter database 2023-03-22 11:40:24 +08:00
wangmm0220 7616a283e3 fix:error in TD-23218 & remove useless logic 2023-03-18 18:23:38 +08:00
Benguang Zhao 22e236f4e6 fix: check pWal->cfg.retentionPeriod properly 2023-03-07 15:15:46 +08:00
Benguang Zhao ca23365d68 enh: fsync WAL idx and log when walRollImpl 2023-03-02 20:40:34 +08:00
wangmm0220 c638564179 fix:confilcts from main 2023-02-26 23:09:57 +08:00
Xiaoyu Wang d6dd087f84 merge main 2023-02-25 10:29:20 +08:00
wangmm0220 54a546931c fix:optimize version logic in tmq and remove useless code 2023-02-24 19:22:08 +08:00
Xiaoyu Wang d04d610a85
Merge pull request #20061 from taosdata/FIX/TD-22611-main
enh: refactor syncBeginSnapshot and walBeginSnapshot for logRetention
2023-02-24 13:12:06 +08:00
Benguang Zhao e45367f924 fix: resolve coverity scan issues in sync and wal 2023-02-23 14:16:21 +08:00
wangmm0220 b28c0ad952 fix:remove useless logic in tmq 2023-02-21 18:49:39 +08:00
Benguang Zhao 0c7a4bfad9 enh: refactor syncBeginSnapshot and walBeginSnapshot for logRetention 2023-02-20 19:19:49 +08:00
Xiaoyu Wang c597de099f merge main 2023-02-20 13:41:27 +08:00
Benguang Zhao bcf6d5b641 fix: clear some asserts in walRead.c 2023-02-16 16:06:12 +08:00
Haojun Liao fd379310f6
Merge pull request #19979 from taosdata/fix/tq_oom
fix: handle insufficient resource
2023-02-16 09:17:52 +08:00
Liu Jicong e4a365c265 remove assert 2023-02-14 16:36:55 +08:00
Haojun Liao c597a1e4ce refactor: do some internal refactor. 2023-02-13 15:46:47 +08:00
Xiaoyu Wang 3382a24a69 merge main 2023-02-10 18:03:11 +08:00
Benguang Zhao 5ce5dded75 enh: reset wal reader after walReadVer 2023-02-08 14:41:15 +08:00
Hongze Cheng 0f00862d8b Merge branch 'fix/TD-22070' of https://github.com/taosdata/TDengine into fix/3.0_merge_main 2023-01-30 10:17:59 +08:00
Shengliang Guan 98e174606c
Merge pull request #19578 from taosdata/FIX/TD-21942-main
enh: trim the last WAL idx file which is not fsynced
2023-01-30 09:31:28 +08:00
Hongze Cheng 5e7b6602c5 Merge branch 'main' of https://github.com/taosdata/TDengine into fix/3.0_merge_main 2023-01-29 14:05:15 +08:00
Liu Jicong f5c4ca3380 fix: wal ref 2023-01-28 17:17:55 +08:00
Liu Jicong 2a71d47668 fix: wal ref 2023-01-28 16:11:17 +08:00
Benguang Zhao 95493432ed enh: trim the last WAL idx file which is not fsynced 2023-01-16 14:12:26 +08:00
Xiaoyu Wang 9a0c2805f4 merge main 2023-01-13 13:54:04 +08:00
Benguang Zhao 97ce96a2fa enh: not fsync idx file in walFsync 2023-01-11 19:24:51 +08:00
Benguang Zhao df6e9631e1 fix: synchronize access within walFsync 2023-01-10 11:24:05 +08:00
Shengliang Guan 398161eb5f Merge branch 'main' into merge/mainto3.0_0108 2023-01-09 10:00:53 +08:00
Benguang Zhao 2890a8cb96 fix: return error on failing to truncate raft log 2022-12-28 20:34:40 +08:00
Benguang Zhao 0c4ade9373 fix: update sync node commitIndex only if matchTerm equals currentTerm upon heartbeat 2022-12-28 20:25:52 +08:00
Benguang Zhao c4fad84c7c enh: reset commitVer in WAL on restore 2022-12-28 20:25:52 +08:00
Haojun Liao fbc9e77004
Merge branch '3.0' into refact/submit_req 2022-12-26 10:26:17 +08:00
kailixu ed3a0c1dc1 chore: vgId format uniform 2022-12-24 23:44:01 +08:00
Xiaoyu Wang 0c269c317b merge 3.0 2022-12-23 17:50:05 +08:00
Liu Jicong 6cbcbd1402 remove assert 2022-12-21 15:37:50 +08:00
Liu Jicong 50729f0cc4 refactor: remove assert 2022-12-21 14:16:28 +08:00
Liu Jicong 2a7de0cd46 refactor: remove assert 2022-12-21 13:54:14 +08:00
Shengliang Guan 41dcd32b8e refact: adjust some logs 2022-12-20 15:01:03 +08:00
Haojun Liao 48f7fc5c3d
Merge branch '3.0' into refact/submit_req 2022-12-12 18:11:17 +08:00
Liu Jicong a6fff95795 fix: close wal ref 2022-12-09 17:18:21 +08:00
Liu Jicong 271822ebc4 refactor: support submitreq2 2022-12-08 01:55:47 +08:00
Shengliang Guan 6d94afe48f
Revert "refact: adjust some assert cases" 2022-12-07 21:51:17 +08:00
Shengliang Guan d5126d469a refact: replcase ASSERT with tAssert 2022-12-07 18:42:48 +08:00
Shengliang Guan f69a188f7e refact: replcase ASSERT with tAssert 2022-12-07 18:37:40 +08:00
Benguang Zhao 3f1a0cefbb fix: walSaveMeta ahead of deleting WAL files as gc in walEndSnapshot
delete WAL files in descending order during walRollback
2022-12-06 00:51:45 +08:00
Shengliang Guan d9c261cd13 refact: remove some error codes 2022-12-03 10:17:02 +08:00
Benguang Zhao 68f8ebac9d Merge branch 3.0 to FIX/TD-19334-3.0 2022-11-25 13:10:06 +08:00
Benguang Zhao f68e41a40e enh: initialize log buffer again after receiving a complete snapshot 2022-11-23 15:57:40 +08:00
Benguang Zhao 95f8e96eb5 fix: align firstVer with lastVer too if no WAL logs exist 2022-11-22 23:42:08 +08:00
Liu Jicong 1344671869 fix(wal): add debug log 2022-11-22 17:24:43 +08:00
Liu Jicong 6f5d90658e fix(wal): add debug log 2022-11-22 16:40:18 +08:00
Minghao Li b600b4c8bc refactor(sync): modify wal pInfo++, to find which case can not pass 2022-11-21 16:18:50 +08:00
Benguang Zhao 051b73c1ed Merge branch 3.0 to FIX/TD-19334-3.0 2022-11-17 17:28:11 +08:00
Shengliang Guan 2b5257f905 fix: memory lead found in asan 2022-11-16 20:45:15 +08:00
Shengliang Guan c8f9707d5a
Merge pull request #18203 from taosdata/feature/stream
enh(stream): add tbname map cache
2022-11-16 17:15:30 +08:00
Shengliang Guan 201ce10fbc fix: restore wal info changes 2022-11-16 15:45:40 +08:00
Liu Jicong ab57d069d4 enh(stream): add tbname map cache 2022-11-16 15:18:31 +08:00
Shengliang Guan b62a85dfdc
Merge pull request #18192 from taosdata/feature/stream
fix(stream): timer refer stream task
2022-11-16 12:40:50 +08:00
Liu Jicong 015c00aaca fix(stream): timer refer stream task 2022-11-16 11:16:32 +08:00
Shengliang Guan 555bf7d4cb
Merge pull request #18181 from taosdata/feature/stream
fix(stream): check task
2022-11-16 08:11:30 +08:00
Liu Jicong 6478fcb2c1 fix(stream): check task 2022-11-15 22:32:52 +08:00
Shengliang Guan ede50530f3
Merge pull request #18178 from taosdata/feature/wal
fix(wal): set commit ver and snapshot ver after restore
2022-11-15 21:26:03 +08:00
Liu Jicong 20e31aacca fix(wal): set commit ver and snapshot ver after restore 2022-11-15 19:48:16 +08:00
Liu Jicong a9bf05bc01 refactor: stream meta ref count 2022-11-15 19:11:42 +08:00
Liu Jicong efde22f081 fix: memory leak 2022-11-15 11:59:29 +08:00
Liu Jicong 7c8b2867ce enh(stream): add stream task info into information schema 2022-11-14 19:23:42 +08:00
Liu Jicong aa9e935854 enh(stream): set path max length to 1024 2022-11-14 14:46:50 +08:00
Liu Jicong bc469f7f7b fix(stream): delete tb should be checked in write thread 2022-11-08 11:30:29 +08:00
Benguang Zhao 5878c9a31a fix: impl ring log buffer 2022-11-07 14:27:22 +08:00
Shengliang Guan 77a6cf503e
Merge pull request #17824 from taosdata/FIX/TS-1984-3.0
fix: set firstTrial as false if error encountered in loop of walScanLogGetLastVer
2022-11-02 15:49:48 +08:00
benguang.zhao 8a5130a88d fix: set firstTrial as false if error encountered in loop of walScanLogGetLastVer 2022-11-02 10:21:40 +08:00
Shengliang Guan cec6642700
Merge pull request #17811 from taosdata/feature/stream
fix(tmq): handle adding subtable for subscribing stb
2022-11-01 21:09:43 +08:00
Liu Jicong f0a66602da fix(taosx): set version 2022-11-01 19:28:42 +08:00
Shengliang Guan 871757240a
Merge pull request #17777 from taosdata/FIX/TS-1984-3.0
fix: fsync wal files and meta if data size not synced yet beyond a threshold
2022-11-01 18:53:10 +08:00
Liu Jicong 94e7c79ca0 fix(tmq): handle adding subtable for subscribing stb 2022-11-01 18:18:26 +08:00
benguang.zhao 4e46b78aa0 fix: update info of WAL entry found within loop in walScanLogGetLastVer 2022-11-01 17:56:24 +08:00