Commit Graph

85 Commits

Author SHA1 Message Date
Shengliang Guan 3cbfd2d160
Merge pull request #17392 from taosdata/FIX/TD-19183-3.0
enh: not allowed to start on disk space unavailable
2022-10-18 17:46:14 +08:00
Benguang Zhao cf2b6ccbfb enh: write idx ahead of writing its log entry in WAL 2022-10-18 11:23:24 +08:00
Benguang Zhao af9aa9d783 enh: assert on alignment of idx entries in walWriteIndex 2022-10-18 11:23:13 +08:00
Benguang Zhao f0427c9322 fix: ftrunc on appending WAL log failure for recovery in walWriteImpl 2022-10-18 11:22:21 +08:00
Liu Jicong 9a3ac0ca57 Merge branch '3.0' into feature/stream 2022-10-17 10:17:45 +08:00
Liu Jicong 6f9153c6e3 fix: error handling 2022-10-17 10:13:10 +08:00
Benguang Zhao 616eb5f903 Merge branch '3.0' into FIX/TD-19239-3.0 2022-10-16 00:29:01 +08:00
Benguang Zhao b6c31ee67d fix: fsync the current wal log and idx files at first to ensure validity of meta in walSaveMeta 2022-10-15 23:57:37 +08:00
Benguang Zhao 7fb827debd fix: refactor walScanLogGetLastVer, walCheckAndRepairMeta, and walCheckAndRepairIdx.
* search for the last entry in the contiguous range of valid WAL from the last pos fsynced as firstTrial
    * find the last entry before the last pos fsynced backwards as second trial
    * reserve sufficient space for computing CRC32 checksum, esp. of WAL record body
    * rebuild meta info to resolve potential misalignment between lists of meta and actual log files
    * retract commitIndex and appliedIndex to lastLogIndex if needed
    * put an upper size limit on possibly corrupted WAL range to be recovered
2022-10-15 23:56:31 +08:00
Liu Jicong 6f8d4d6955 fix(tmq): push msg 2022-10-14 13:31:05 +08:00
Liu Jicong b1594a6857 fix coverity scan 2022-10-10 11:44:17 +08:00
Liu Jicong 18f6fb6123 enh(tmq): speed up consumer recover 2022-08-10 16:04:03 +08:00
Liu Jicong f0f4748320 fix(util): concurrently hash iterate 2022-08-05 16:14:19 +08:00
Liu Jicong b084dd8ee6 feat(tq): transport snapshot 2022-08-01 19:12:37 +08:00
Liu Jicong bb6fabcc11 refactor(stream): use vnode meta 2022-07-30 20:20:12 +08:00
Liu Jicong 6d67b17113 enh(wal): add lock to guarantee read behaviour 2022-07-30 16:35:49 +08:00
Liu Jicong 0c942e622a enh(wal): remove file failure handling 2022-07-28 13:37:57 +08:00
Liu Jicong 3fa5791889 fix(stream): dispatch 2022-07-27 10:52:25 +08:00
Liu Jicong ecb7a9c133 fix(wal): snapshot end with no deleting 2022-07-26 14:53:39 +08:00
Liu Jicong c2164dd8aa fix(wal): snapshot end with no deleting 2022-07-26 14:13:03 +08:00
Liu Jicong 0b8e9af8ec refactor(wal) 2022-07-26 13:33:26 +08:00
Shengliang Guan 5b654779ba Merge pull request #15372 from taosdata/fix/tsim
test: regression case
2022-07-25 15:23:41 +08:00
Liu Jicong bf37f3fa68 feat(wal): remove wal log size limitation 2022-07-20 16:15:53 +08:00
Liu Jicong a3133b9f6d feat(wal): log applied ver 2022-07-20 10:55:28 +08:00
Liu Jicong c6052fadba feat(wal): add append interface 2022-07-11 16:22:51 +08:00
Liu Jicong 9fd5ec9020 fix(wal): rollback 2022-07-11 13:38:50 +08:00
Shengliang Guan de644c7d43 refactor: adjust logs 2022-07-08 18:33:52 +08:00
Shengliang Guan 361c21a35c refactor: adjust logs 2022-07-08 17:27:06 +08:00
Shengliang Guan dca93b18cc refactor: adjust logs 2022-07-08 16:57:54 +08:00
Liu Jicong ca3d918004 fix(wal): mutex 2022-07-07 09:58:23 +08:00
Liu Jicong 8c5dbde918 refactor(stream): stream reader created in scanner 2022-07-06 14:20:07 +08:00
Shengliang Guan 3cc215bdcc fix: crash while write wal in multi-threaded way 2022-07-04 17:54:00 +08:00
Liu Jicong 52d09b4173 refactor(wal) 2022-07-01 14:39:21 +08:00
Liu Jicong 81ed902d7f fix(stream): stream rsp to retrieve msg if data is empty 2022-06-25 14:03:06 +08:00
Liu Jicong f53521cb8c test(stream): partition by tbname 2022-06-24 16:50:11 +08:00
Liu Jicong ef554e9884
Merge pull request #14191 from taosdata/feature/stream
fix(wal): set first version to -1 if all log rollbacked
2022-06-24 11:45:24 +08:00
Liu Jicong a1cc2571ee fix(wal): set first version to -1 if all log rollbacked 2022-06-24 11:13:40 +08:00
Minghao Li 983bf5b865 fix(wal): read handle version 2022-06-23 15:26:42 +08:00
Liu Jicong 04f11e2646 feat(wal): provide is empty api 2022-06-22 19:40:37 +08:00
Liu Jicong 5572abe1b4 feat(stream): drop task 2022-06-15 21:41:59 +08:00
Liu Jicong 7ab3a1e42c feat(wal): support restore from snapshot 2022-06-09 23:08:06 +08:00
Liu Jicong b23fa19bc4 feat(stream): sink table can be other db 2022-06-09 16:50:11 +08:00
Liu Jicong 0ead238adb feat(tmq): support check col and tag modifiable 2022-06-02 15:53:30 +08:00
Liu Jicong 361332505c feat(wal): support restore from snapshot 2022-06-02 11:38:04 +08:00
afwerar 5acf419019 fix(os): win not close file ctest error 2022-05-21 23:47:39 +08:00
Liu Jicong af2d2db5d3 fix(wal): int overflow 2022-05-12 20:56:29 +08:00
Liu Jicong 1dbe0650e0 enh(wal): set errno code 2022-05-12 14:20:50 +08:00
Liu Jicong 6b18948884 enh(wal): set error code 2022-05-07 00:14:52 +08:00
Liu Jicong fa03303bad reject wal write if exceeds limit 2022-04-26 17:19:09 +08:00
Liu Jicong 28325c9080 reject wal write if exceeds limit 2022-04-26 17:18:10 +08:00