Xiaoyu Wang
9a0c2805f4
merge main
2023-01-13 13:54:04 +08:00
Benguang Zhao
97ce96a2fa
enh: not fsync idx file in walFsync
2023-01-11 19:24:51 +08:00
Benguang Zhao
df6e9631e1
fix: synchronize access within walFsync
2023-01-10 11:24:05 +08:00
Shengliang Guan
398161eb5f
Merge branch 'main' into merge/mainto3.0_0108
2023-01-09 10:00:53 +08:00
Benguang Zhao
2890a8cb96
fix: return error on failing to truncate raft log
2022-12-28 20:34:40 +08:00
Liu Jicong
50729f0cc4
refactor: remove assert
2022-12-21 14:16:28 +08:00
Shengliang Guan
6d94afe48f
Revert "refact: adjust some assert cases"
2022-12-07 21:51:17 +08:00
Shengliang Guan
d5126d469a
refact: replcase ASSERT with tAssert
2022-12-07 18:42:48 +08:00
Shengliang Guan
f69a188f7e
refact: replcase ASSERT with tAssert
2022-12-07 18:37:40 +08:00
Benguang Zhao
3f1a0cefbb
fix: walSaveMeta ahead of deleting WAL files as gc in walEndSnapshot
...
delete WAL files in descending order during walRollback
2022-12-06 00:51:45 +08:00
Benguang Zhao
68f8ebac9d
Merge branch 3.0 to FIX/TD-19334-3.0
2022-11-25 13:10:06 +08:00
Benguang Zhao
f68e41a40e
enh: initialize log buffer again after receiving a complete snapshot
2022-11-23 15:57:40 +08:00
Liu Jicong
1344671869
fix(wal): add debug log
2022-11-22 17:24:43 +08:00
Liu Jicong
6f5d90658e
fix(wal): add debug log
2022-11-22 16:40:18 +08:00
Minghao Li
b600b4c8bc
refactor(sync): modify wal pInfo++, to find which case can not pass
2022-11-21 16:18:50 +08:00
Benguang Zhao
051b73c1ed
Merge branch 3.0 to FIX/TD-19334-3.0
2022-11-17 17:28:11 +08:00
Shengliang Guan
c8f9707d5a
Merge pull request #18203 from taosdata/feature/stream
...
enh(stream): add tbname map cache
2022-11-16 17:15:30 +08:00
Shengliang Guan
201ce10fbc
fix: restore wal info changes
2022-11-16 15:45:40 +08:00
Liu Jicong
ab57d069d4
enh(stream): add tbname map cache
2022-11-16 15:18:31 +08:00
Shengliang Guan
b62a85dfdc
Merge pull request #18192 from taosdata/feature/stream
...
fix(stream): timer refer stream task
2022-11-16 12:40:50 +08:00
Liu Jicong
015c00aaca
fix(stream): timer refer stream task
2022-11-16 11:16:32 +08:00
Shengliang Guan
555bf7d4cb
Merge pull request #18181 from taosdata/feature/stream
...
fix(stream): check task
2022-11-16 08:11:30 +08:00
Liu Jicong
6478fcb2c1
fix(stream): check task
2022-11-15 22:32:52 +08:00
Shengliang Guan
ede50530f3
Merge pull request #18178 from taosdata/feature/wal
...
fix(wal): set commit ver and snapshot ver after restore
2022-11-15 21:26:03 +08:00
Liu Jicong
20e31aacca
fix(wal): set commit ver and snapshot ver after restore
2022-11-15 19:48:16 +08:00
Liu Jicong
efde22f081
fix: memory leak
2022-11-15 11:59:29 +08:00
Liu Jicong
7c8b2867ce
enh(stream): add stream task info into information schema
2022-11-14 19:23:42 +08:00
Liu Jicong
aa9e935854
enh(stream): set path max length to 1024
2022-11-14 14:46:50 +08:00
Liu Jicong
bc469f7f7b
fix(stream): delete tb should be checked in write thread
2022-11-08 11:30:29 +08:00
Benguang Zhao
5878c9a31a
fix: impl ring log buffer
2022-11-07 14:27:22 +08:00
Shengliang Guan
871757240a
Merge pull request #17777 from taosdata/FIX/TS-1984-3.0
...
fix: fsync wal files and meta if data size not synced yet beyond a threshold
2022-11-01 18:53:10 +08:00
benguang.zhao
4237cafb21
enh: rename walRecoverSizeLimit to walFsyncDataSizeLimit
2022-11-01 17:19:33 +08:00
Shengliang Guan
26778f29da
enh: add config for persist wal at the beginning of vnodeCommit
2022-11-01 10:42:13 +08:00
benguang.zhao
cccfa17027
fix: fsync wal files and their meta if data size unsynced exceeds a limit
2022-10-31 23:58:49 +08:00
benguang.zhao
1d0eae197d
fix: not allow to rollback WAL across snapshot point
2022-10-31 21:06:23 +08:00
Liu Jicong
25bb914ac1
merge 3.0
2022-10-21 16:30:43 +08:00
Liu Jicong
1c26f1b53e
fix(wal): reference
2022-10-21 16:13:55 +08:00
Shengliang Guan
05d04dcd2d
Merge remote-tracking branch 'origin/3.0' into enh/TD-19461
2022-10-21 11:07:48 +08:00
Shengliang Guan
6c3cd3e19e
test: adjust log
2022-10-21 10:39:29 +08:00
Benguang Zhao
06f5d91d21
enh: provide more info on misalignment detection of idx entries after appending
2022-10-20 15:17:52 +08:00
Benguang Zhao
3b0968d7f9
enh: enable the -Wformat=2 compiler option
2022-10-19 09:24:15 +08:00
Shengliang Guan
3cbfd2d160
Merge pull request #17392 from taosdata/FIX/TD-19183-3.0
...
enh: not allowed to start on disk space unavailable
2022-10-18 17:46:14 +08:00
Benguang Zhao
cf2b6ccbfb
enh: write idx ahead of writing its log entry in WAL
2022-10-18 11:23:24 +08:00
Benguang Zhao
af9aa9d783
enh: assert on alignment of idx entries in walWriteIndex
2022-10-18 11:23:13 +08:00
Benguang Zhao
f0427c9322
fix: ftrunc on appending WAL log failure for recovery in walWriteImpl
2022-10-18 11:22:21 +08:00
Liu Jicong
9a3ac0ca57
Merge branch '3.0' into feature/stream
2022-10-17 10:17:45 +08:00
Liu Jicong
6f9153c6e3
fix: error handling
2022-10-17 10:13:10 +08:00
Benguang Zhao
616eb5f903
Merge branch '3.0' into FIX/TD-19239-3.0
2022-10-16 00:29:01 +08:00
Benguang Zhao
b6c31ee67d
fix: fsync the current wal log and idx files at first to ensure validity of meta in walSaveMeta
2022-10-15 23:57:37 +08:00
Benguang Zhao
7fb827debd
fix: refactor walScanLogGetLastVer, walCheckAndRepairMeta, and walCheckAndRepairIdx.
...
* search for the last entry in the contiguous range of valid WAL from the last pos fsynced as firstTrial
* find the last entry before the last pos fsynced backwards as second trial
* reserve sufficient space for computing CRC32 checksum, esp. of WAL record body
* rebuild meta info to resolve potential misalignment between lists of meta and actual log files
* retract commitIndex and appliedIndex to lastLogIndex if needed
* put an upper size limit on possibly corrupted WAL range to be recovered
2022-10-15 23:56:31 +08:00