Commit Graph

126 Commits

Author SHA1 Message Date
Xiaoyu Wang 9a0c2805f4 merge main 2023-01-13 13:54:04 +08:00
Benguang Zhao 97ce96a2fa enh: not fsync idx file in walFsync 2023-01-11 19:24:51 +08:00
Benguang Zhao df6e9631e1 fix: synchronize access within walFsync 2023-01-10 11:24:05 +08:00
Shengliang Guan 398161eb5f Merge branch 'main' into merge/mainto3.0_0108 2023-01-09 10:00:53 +08:00
Benguang Zhao 2890a8cb96 fix: return error on failing to truncate raft log 2022-12-28 20:34:40 +08:00
Liu Jicong 50729f0cc4 refactor: remove assert 2022-12-21 14:16:28 +08:00
Shengliang Guan 6d94afe48f
Revert "refact: adjust some assert cases" 2022-12-07 21:51:17 +08:00
Shengliang Guan d5126d469a refact: replcase ASSERT with tAssert 2022-12-07 18:42:48 +08:00
Shengliang Guan f69a188f7e refact: replcase ASSERT with tAssert 2022-12-07 18:37:40 +08:00
Benguang Zhao 3f1a0cefbb fix: walSaveMeta ahead of deleting WAL files as gc in walEndSnapshot
delete WAL files in descending order during walRollback
2022-12-06 00:51:45 +08:00
Benguang Zhao 68f8ebac9d Merge branch 3.0 to FIX/TD-19334-3.0 2022-11-25 13:10:06 +08:00
Benguang Zhao f68e41a40e enh: initialize log buffer again after receiving a complete snapshot 2022-11-23 15:57:40 +08:00
Liu Jicong 1344671869 fix(wal): add debug log 2022-11-22 17:24:43 +08:00
Liu Jicong 6f5d90658e fix(wal): add debug log 2022-11-22 16:40:18 +08:00
Minghao Li b600b4c8bc refactor(sync): modify wal pInfo++, to find which case can not pass 2022-11-21 16:18:50 +08:00
Benguang Zhao 051b73c1ed Merge branch 3.0 to FIX/TD-19334-3.0 2022-11-17 17:28:11 +08:00
Shengliang Guan c8f9707d5a
Merge pull request #18203 from taosdata/feature/stream
enh(stream): add tbname map cache
2022-11-16 17:15:30 +08:00
Shengliang Guan 201ce10fbc fix: restore wal info changes 2022-11-16 15:45:40 +08:00
Liu Jicong ab57d069d4 enh(stream): add tbname map cache 2022-11-16 15:18:31 +08:00
Shengliang Guan b62a85dfdc
Merge pull request #18192 from taosdata/feature/stream
fix(stream): timer refer stream task
2022-11-16 12:40:50 +08:00
Liu Jicong 015c00aaca fix(stream): timer refer stream task 2022-11-16 11:16:32 +08:00
Shengliang Guan 555bf7d4cb
Merge pull request #18181 from taosdata/feature/stream
fix(stream): check task
2022-11-16 08:11:30 +08:00
Liu Jicong 6478fcb2c1 fix(stream): check task 2022-11-15 22:32:52 +08:00
Shengliang Guan ede50530f3
Merge pull request #18178 from taosdata/feature/wal
fix(wal): set commit ver and snapshot ver after restore
2022-11-15 21:26:03 +08:00
Liu Jicong 20e31aacca fix(wal): set commit ver and snapshot ver after restore 2022-11-15 19:48:16 +08:00
Liu Jicong efde22f081 fix: memory leak 2022-11-15 11:59:29 +08:00
Liu Jicong 7c8b2867ce enh(stream): add stream task info into information schema 2022-11-14 19:23:42 +08:00
Liu Jicong aa9e935854 enh(stream): set path max length to 1024 2022-11-14 14:46:50 +08:00
Liu Jicong bc469f7f7b fix(stream): delete tb should be checked in write thread 2022-11-08 11:30:29 +08:00
Benguang Zhao 5878c9a31a fix: impl ring log buffer 2022-11-07 14:27:22 +08:00
Shengliang Guan 871757240a
Merge pull request #17777 from taosdata/FIX/TS-1984-3.0
fix: fsync wal files and meta if data size not synced yet beyond a threshold
2022-11-01 18:53:10 +08:00
benguang.zhao 4237cafb21 enh: rename walRecoverSizeLimit to walFsyncDataSizeLimit 2022-11-01 17:19:33 +08:00
Shengliang Guan 26778f29da enh: add config for persist wal at the beginning of vnodeCommit 2022-11-01 10:42:13 +08:00
benguang.zhao cccfa17027 fix: fsync wal files and their meta if data size unsynced exceeds a limit 2022-10-31 23:58:49 +08:00
benguang.zhao 1d0eae197d fix: not allow to rollback WAL across snapshot point 2022-10-31 21:06:23 +08:00
Liu Jicong 25bb914ac1 merge 3.0 2022-10-21 16:30:43 +08:00
Liu Jicong 1c26f1b53e fix(wal): reference 2022-10-21 16:13:55 +08:00
Shengliang Guan 05d04dcd2d Merge remote-tracking branch 'origin/3.0' into enh/TD-19461 2022-10-21 11:07:48 +08:00
Shengliang Guan 6c3cd3e19e test: adjust log 2022-10-21 10:39:29 +08:00
Benguang Zhao 06f5d91d21 enh: provide more info on misalignment detection of idx entries after appending 2022-10-20 15:17:52 +08:00
Benguang Zhao 3b0968d7f9 enh: enable the -Wformat=2 compiler option 2022-10-19 09:24:15 +08:00
Shengliang Guan 3cbfd2d160
Merge pull request #17392 from taosdata/FIX/TD-19183-3.0
enh: not allowed to start on disk space unavailable
2022-10-18 17:46:14 +08:00
Benguang Zhao cf2b6ccbfb enh: write idx ahead of writing its log entry in WAL 2022-10-18 11:23:24 +08:00
Benguang Zhao af9aa9d783 enh: assert on alignment of idx entries in walWriteIndex 2022-10-18 11:23:13 +08:00
Benguang Zhao f0427c9322 fix: ftrunc on appending WAL log failure for recovery in walWriteImpl 2022-10-18 11:22:21 +08:00
Liu Jicong 9a3ac0ca57 Merge branch '3.0' into feature/stream 2022-10-17 10:17:45 +08:00
Liu Jicong 6f9153c6e3 fix: error handling 2022-10-17 10:13:10 +08:00
Benguang Zhao 616eb5f903 Merge branch '3.0' into FIX/TD-19239-3.0 2022-10-16 00:29:01 +08:00
Benguang Zhao b6c31ee67d fix: fsync the current wal log and idx files at first to ensure validity of meta in walSaveMeta 2022-10-15 23:57:37 +08:00
Benguang Zhao 7fb827debd fix: refactor walScanLogGetLastVer, walCheckAndRepairMeta, and walCheckAndRepairIdx.
* search for the last entry in the contiguous range of valid WAL from the last pos fsynced as firstTrial
    * find the last entry before the last pos fsynced backwards as second trial
    * reserve sufficient space for computing CRC32 checksum, esp. of WAL record body
    * rebuild meta info to resolve potential misalignment between lists of meta and actual log files
    * retract commitIndex and appliedIndex to lastLogIndex if needed
    * put an upper size limit on possibly corrupted WAL range to be recovered
2022-10-15 23:56:31 +08:00