Commit Graph

116 Commits

Author SHA1 Message Date
Benguang Zhao 68f8ebac9d Merge branch 3.0 to FIX/TD-19334-3.0 2022-11-25 13:10:06 +08:00
Benguang Zhao f68e41a40e enh: initialize log buffer again after receiving a complete snapshot 2022-11-23 15:57:40 +08:00
Liu Jicong 1344671869 fix(wal): add debug log 2022-11-22 17:24:43 +08:00
Liu Jicong 6f5d90658e fix(wal): add debug log 2022-11-22 16:40:18 +08:00
Minghao Li b600b4c8bc refactor(sync): modify wal pInfo++, to find which case can not pass 2022-11-21 16:18:50 +08:00
Benguang Zhao 051b73c1ed Merge branch 3.0 to FIX/TD-19334-3.0 2022-11-17 17:28:11 +08:00
Shengliang Guan c8f9707d5a
Merge pull request #18203 from taosdata/feature/stream
enh(stream): add tbname map cache
2022-11-16 17:15:30 +08:00
Shengliang Guan 201ce10fbc fix: restore wal info changes 2022-11-16 15:45:40 +08:00
Liu Jicong ab57d069d4 enh(stream): add tbname map cache 2022-11-16 15:18:31 +08:00
Shengliang Guan b62a85dfdc
Merge pull request #18192 from taosdata/feature/stream
fix(stream): timer refer stream task
2022-11-16 12:40:50 +08:00
Liu Jicong 015c00aaca fix(stream): timer refer stream task 2022-11-16 11:16:32 +08:00
Shengliang Guan 555bf7d4cb
Merge pull request #18181 from taosdata/feature/stream
fix(stream): check task
2022-11-16 08:11:30 +08:00
Liu Jicong 6478fcb2c1 fix(stream): check task 2022-11-15 22:32:52 +08:00
Shengliang Guan ede50530f3
Merge pull request #18178 from taosdata/feature/wal
fix(wal): set commit ver and snapshot ver after restore
2022-11-15 21:26:03 +08:00
Liu Jicong 20e31aacca fix(wal): set commit ver and snapshot ver after restore 2022-11-15 19:48:16 +08:00
Liu Jicong efde22f081 fix: memory leak 2022-11-15 11:59:29 +08:00
Liu Jicong 7c8b2867ce enh(stream): add stream task info into information schema 2022-11-14 19:23:42 +08:00
Liu Jicong aa9e935854 enh(stream): set path max length to 1024 2022-11-14 14:46:50 +08:00
Liu Jicong bc469f7f7b fix(stream): delete tb should be checked in write thread 2022-11-08 11:30:29 +08:00
Benguang Zhao 5878c9a31a fix: impl ring log buffer 2022-11-07 14:27:22 +08:00
Shengliang Guan 871757240a
Merge pull request #17777 from taosdata/FIX/TS-1984-3.0
fix: fsync wal files and meta if data size not synced yet beyond a threshold
2022-11-01 18:53:10 +08:00
benguang.zhao 4237cafb21 enh: rename walRecoverSizeLimit to walFsyncDataSizeLimit 2022-11-01 17:19:33 +08:00
Shengliang Guan 26778f29da enh: add config for persist wal at the beginning of vnodeCommit 2022-11-01 10:42:13 +08:00
benguang.zhao cccfa17027 fix: fsync wal files and their meta if data size unsynced exceeds a limit 2022-10-31 23:58:49 +08:00
benguang.zhao 1d0eae197d fix: not allow to rollback WAL across snapshot point 2022-10-31 21:06:23 +08:00
Liu Jicong 25bb914ac1 merge 3.0 2022-10-21 16:30:43 +08:00
Liu Jicong 1c26f1b53e fix(wal): reference 2022-10-21 16:13:55 +08:00
Shengliang Guan 05d04dcd2d Merge remote-tracking branch 'origin/3.0' into enh/TD-19461 2022-10-21 11:07:48 +08:00
Shengliang Guan 6c3cd3e19e test: adjust log 2022-10-21 10:39:29 +08:00
Benguang Zhao 06f5d91d21 enh: provide more info on misalignment detection of idx entries after appending 2022-10-20 15:17:52 +08:00
Benguang Zhao 3b0968d7f9 enh: enable the -Wformat=2 compiler option 2022-10-19 09:24:15 +08:00
Shengliang Guan 3cbfd2d160
Merge pull request #17392 from taosdata/FIX/TD-19183-3.0
enh: not allowed to start on disk space unavailable
2022-10-18 17:46:14 +08:00
Benguang Zhao cf2b6ccbfb enh: write idx ahead of writing its log entry in WAL 2022-10-18 11:23:24 +08:00
Benguang Zhao af9aa9d783 enh: assert on alignment of idx entries in walWriteIndex 2022-10-18 11:23:13 +08:00
Benguang Zhao f0427c9322 fix: ftrunc on appending WAL log failure for recovery in walWriteImpl 2022-10-18 11:22:21 +08:00
Liu Jicong 9a3ac0ca57 Merge branch '3.0' into feature/stream 2022-10-17 10:17:45 +08:00
Liu Jicong 6f9153c6e3 fix: error handling 2022-10-17 10:13:10 +08:00
Benguang Zhao 616eb5f903 Merge branch '3.0' into FIX/TD-19239-3.0 2022-10-16 00:29:01 +08:00
Benguang Zhao b6c31ee67d fix: fsync the current wal log and idx files at first to ensure validity of meta in walSaveMeta 2022-10-15 23:57:37 +08:00
Benguang Zhao 7fb827debd fix: refactor walScanLogGetLastVer, walCheckAndRepairMeta, and walCheckAndRepairIdx.
* search for the last entry in the contiguous range of valid WAL from the last pos fsynced as firstTrial
    * find the last entry before the last pos fsynced backwards as second trial
    * reserve sufficient space for computing CRC32 checksum, esp. of WAL record body
    * rebuild meta info to resolve potential misalignment between lists of meta and actual log files
    * retract commitIndex and appliedIndex to lastLogIndex if needed
    * put an upper size limit on possibly corrupted WAL range to be recovered
2022-10-15 23:56:31 +08:00
Liu Jicong 6f8d4d6955 fix(tmq): push msg 2022-10-14 13:31:05 +08:00
Liu Jicong b1594a6857 fix coverity scan 2022-10-10 11:44:17 +08:00
Liu Jicong 18f6fb6123 enh(tmq): speed up consumer recover 2022-08-10 16:04:03 +08:00
Liu Jicong f0f4748320 fix(util): concurrently hash iterate 2022-08-05 16:14:19 +08:00
Liu Jicong b084dd8ee6 feat(tq): transport snapshot 2022-08-01 19:12:37 +08:00
Liu Jicong bb6fabcc11 refactor(stream): use vnode meta 2022-07-30 20:20:12 +08:00
Liu Jicong 6d67b17113 enh(wal): add lock to guarantee read behaviour 2022-07-30 16:35:49 +08:00
Liu Jicong 0c942e622a enh(wal): remove file failure handling 2022-07-28 13:37:57 +08:00
Liu Jicong 3fa5791889 fix(stream): dispatch 2022-07-27 10:52:25 +08:00
Liu Jicong ecb7a9c133 fix(wal): snapshot end with no deleting 2022-07-26 14:53:39 +08:00