Commit Graph

64 Commits

Author SHA1 Message Date
Shengliang Guan 26778f29da enh: add config for persist wal at the beginning of vnodeCommit 2022-11-01 10:42:13 +08:00
benguang.zhao 07db554fab enh: persist wal at the beginning of vnodeCommit 2022-11-01 00:00:18 +08:00
Liu Jicong 75e5e490fd refactor(stream): recover and fill history 2022-10-26 00:31:00 +08:00
Benguang Zhao 7fb827debd fix: refactor walScanLogGetLastVer, walCheckAndRepairMeta, and walCheckAndRepairIdx.
* search for the last entry in the contiguous range of valid WAL from the last pos fsynced as firstTrial
    * find the last entry before the last pos fsynced backwards as second trial
    * reserve sufficient space for computing CRC32 checksum, esp. of WAL record body
    * rebuild meta info to resolve potential misalignment between lists of meta and actual log files
    * retract commitIndex and appliedIndex to lastLogIndex if needed
    * put an upper size limit on possibly corrupted WAL range to be recovered
2022-10-15 23:56:31 +08:00
Liu Jicong b084dd8ee6 feat(tq): transport snapshot 2022-08-01 19:12:37 +08:00
Liu Jicong bb6fabcc11 refactor(stream): use vnode meta 2022-07-30 20:20:12 +08:00
Liu Jicong 0c942e622a enh(wal): remove file failure handling 2022-07-28 13:37:57 +08:00
Liu Jicong 0b8e9af8ec refactor(wal) 2022-07-26 13:33:26 +08:00
Shengliang Guan 5b654779ba Merge pull request #15372 from taosdata/fix/tsim
test: regression case
2022-07-25 15:23:41 +08:00
Liu Jicong 095f6aa4e0 fix(tmq): correctly set reader status 2022-07-23 20:07:16 +08:00
Liu Jicong bf37f3fa68 feat(wal): remove wal log size limitation 2022-07-20 16:15:53 +08:00
Liu Jicong a3133b9f6d feat(wal): log applied ver 2022-07-20 10:55:28 +08:00
Liu Jicong c6052fadba feat(wal): add append interface 2022-07-11 16:22:51 +08:00
Liu Jicong 219d966b25 fix(wal): read after rolling 2022-07-10 21:02:25 +08:00
Liu Jicong e9077c036f merge from 3.0 2022-07-10 11:24:37 +08:00
Liu Jicong a2107992e2 refactor(stream) 2022-07-10 11:18:15 +08:00
Shengliang Guan c53a84b733 Merge remote-tracking branch 'origin/3.0' into fix/valgrind 2022-07-09 20:13:26 +08:00
Liu Jicong 3cd5a35054 refactor(stream): internal refactor 2022-07-08 17:48:34 +08:00
Shengliang Guan 361c21a35c refactor: adjust logs 2022-07-08 17:27:06 +08:00
Liu Jicong 4a648d711f feat(wal): support filter 2022-07-07 13:56:47 +08:00
Liu Jicong 1ec57798d2 ci: remove unstable case 2022-07-07 11:45:28 +08:00
Liu Jicong 8c5dbde918 refactor(stream): stream reader created in scanner 2022-07-06 14:20:07 +08:00
Liu Jicong 52d09b4173 refactor(wal) 2022-07-01 14:39:21 +08:00
Liu Jicong 7a65589525 feat(wal): add api to check log existance 2022-07-01 10:15:05 +08:00
Liu Jicong 04f11e2646 feat(wal): provide is empty api 2022-06-22 19:40:37 +08:00
Liu Jicong 7ab3a1e42c feat(wal): support restore from snapshot 2022-06-09 23:08:06 +08:00
Liu Jicong 361332505c feat(wal): support restore from snapshot 2022-06-02 11:38:04 +08:00
Liu Jicong 714dd4f497 enh(tmq): only read committed log 2022-05-11 14:42:20 +08:00
Liu Jicong 9723145c1d enh(tmq): use sem_timewait to reduce busy loop 2022-04-26 19:21:31 +08:00
Liu Jicong 575f64197d enh(wal): skip read for specific msg 2022-04-24 16:51:32 +08:00
Liu Jicong 3d972f81a2 refacor(wal): rename len to bodyLen 2022-04-24 13:42:54 +08:00
Hongze Cheng a6d12f069e insert return result 2022-04-16 06:32:34 +00:00
Liu Jicong 3e5da8bedb fix 2022-03-31 15:52:54 +08:00
Liu Jicong 96eb6a9bcf wal add sync info 2022-03-26 15:47:58 +08:00
Liu Jicong 7bd71bcb81 wal add sync info 2022-03-26 15:36:49 +08:00
afwerar 79057240bd [TD-13759]<fix>: redefine thread api. 2022-03-20 00:47:45 +08:00
Shengliang Guan 1d9119df87 record num of logs for monitoring 2022-03-04 14:34:35 +08:00
afwerar 5a37fdf28f [TD-13062]<fix>: cross platform file system. 2022-02-25 01:47:20 +08:00
Liu Jicong c62304531a fix query error 2022-01-26 09:56:32 +08:00
Liu Jicong f7726ce8a4 add tq process consumer msg 2022-01-20 14:38:55 +08:00
Liu Jicong 47f1558e7c add wal handle meta corrupt 2022-01-11 15:40:10 +08:00
Liu Jicong 6beee204d8 add tq query 2021-12-21 20:07:32 +08:00
Liu Jicong dfc8c4e0b7 format code 2021-12-16 15:04:13 +08:00
Liu Jicong 3d141cf109 refactor wal and tq 2021-12-15 17:17:50 +08:00
Liu Jicong 5874463cb1 refactor wal 2021-12-14 18:14:45 +08:00
Liu Jicong 43a2bcdce3 fix wal read handle 2021-12-14 14:38:52 +08:00
Liu Jicong dc24592929 add wal read handle 2021-12-13 18:33:30 +08:00
Liu Jicong b2083ea62f fix memory leak 2021-12-13 13:58:54 +08:00
Liu Jicong 9e4b890727 add test for wal snapshot 2021-12-11 17:18:18 +08:00
Liu Jicong 03a54b5a35 make tfinit called once 2021-12-10 10:30:57 +08:00