Commit Graph

452 Commits

Author SHA1 Message Date
Minglei Jin 383e90e43f fix unit test case 2024-07-19 17:51:41 +08:00
Minglei Jin a0b59d3348 use new macros to return terrno 2024-07-19 16:13:09 +08:00
Minglei Jin fcce9ebafa return wal fsync error code 2024-07-19 15:20:38 +08:00
Minglei Jin 6ce5320750 wal/mgmg: use new return macros for wal mgmt 2024-07-18 14:40:49 +08:00
Minglei Jin 180411434a wal/ref: use new return macros for ref 2024-07-18 14:14:51 +08:00
Minglei Jin 964397ea63 move log macros into internal header 2024-07-18 10:29:25 +08:00
Minglei Jin f485e4c5e8 remove wal seek module 2024-07-18 10:16:12 +08:00
Minglei Jin d2be5e0464 refactor wal's meta & reader 2024-07-18 09:51:31 +08:00
Minglei Jin 5e4b7b35a5 fix compile error 2024-07-18 08:04:19 +08:00
Minglei Jin 10dc3b35e0 remove struct dot initialization for cpp 2024-07-17 17:39:13 +08:00
Minglei Jin e2371f9d91 remove unused internal api: walSeekWriteVer 2024-07-17 14:52:42 +08:00
Minglei Jin e6c86f5586 fix(wal/write): remove walWrite which is only used in unit testing 2024-07-17 14:45:23 +08:00
wangmm0220 e78a75d183 opti:[TS-4593] transform offset from file to tdb in tmq 2024-07-15 09:48:04 +08:00
Minglei Jin 930c5d2adf cleanup wal level 0 before ref removing 2024-06-17 08:57:53 +08:00
Minglei Jin d47fe255ae remova level 0's empty wals when closing vnode 2024-06-14 17:09:48 +08:00
Minglei Jin d6b2bcba68 enh/wal_level: new level 0 to disable wal writing 2024-06-05 15:19:55 +08:00
dmchen 8875311f23 remove log output 2024-04-10 09:04:53 +00:00
dmchen 0e5053b3de mem leak 2024-04-09 10:34:28 +00:00
dmchen d06bfd9cea free and log 2024-04-02 02:30:56 +00:00
dmchen a1b7444a48 refactor 2024-04-01 09:39:29 +00:00
dmchen b9deb71fc5 basic function 2024-03-29 02:48:49 +00:00
Haojun Liao 3236ef7bae feat(stream): return the stream source task exec delay, and do some internal refactor. 2024-02-02 11:26:47 +08:00
Haojun Liao 719d1d1b90 refactor: do some internal refactor. 2024-01-18 14:34:18 +08:00
Haojun Liao dc0c5539a0 fix(stream): remove invalid timer controllers. 2024-01-10 08:48:49 +08:00
Haojun Liao 8ac02adc0b refactor: do some internal refactor. 2023-12-29 17:42:23 +08:00
Benguang Zhao c74c4fe208 enh: clean WAL logs based on size if one follower is offline. 2023-12-13 10:10:02 +08:00
factosea f01241f720 Merge branch 'fix/xsren/TD-22212/taosQueryParam3.0' into enh/xsren/TD-22212/taosQueryParam3.0 2023-12-06 17:39:25 +08:00
Benguang Zhao d9c84b4b72 enh: tolerate out-of-order of closeTS in walEndSnapshot 2023-11-30 11:59:41 +08:00
facetosea 45170a73b3 close file before create in rollback 2023-11-21 19:54:33 +08:00
facetosea 845493294d osfile 2023-11-21 16:31:31 +08:00
xsren 355ec86926 fix error when taos_query_a param is null
c_test missing on linux

mem leak

wal_test  pageBufferTest on linux

tfsTest on linux
2023-10-26 16:50:45 +08:00
Benguang Zhao 05ba5e1ed0 refact: adjust logging msg in walLogEntriesComplete 2023-10-19 11:18:57 +08:00
wangmm0220 05c016b667 fix:[TD-26671] judge if pointer is null to avoid core dump 2023-10-11 13:55:54 +08:00
wangmm0220 f7d1e0ea82 fix:[TD-26671] judge if pointer is null to avoid core dump 2023-10-11 11:08:14 +08:00
Haojun Liao 10271aee26 fix(stream): fix syntax error in unit test. 2023-09-10 14:12:03 +08:00
Haojun Liao 85a5c45098 fix(stream): limit the already processed data. 2023-09-10 00:01:25 +08:00
Haojun Liao 9a30573b8b fix(stream): disable scan wal when halt is set. 2023-09-09 16:12:08 +08:00
Haojun Liao aa8909b267 other: add some logs. 2023-09-09 15:18:54 +08:00
Haojun Liao ad9f3cbc43 other: merge main. 2023-08-16 09:14:10 +08:00
wade zhang 1a1a12b7b8
Merge pull request #22327 from taosdata/fix/TD-25010-3
feat(tsdb/cos): s3 migration
2023-08-16 08:02:20 +08:00
Haojun Liao ff17763ed6
Merge pull request #22417 from taosdata/mark/tmq
opti:wal logic
2023-08-15 09:16:29 +08:00
wade zhang f63d1f98b2
Merge pull request #22150 from taosdata/test/toolsBuild
test: add build
2023-08-14 11:22:30 +08:00
wangmm0220 dd0bc0e050 fix:heap use after free 2023-08-14 09:42:55 +08:00
wangmm0220 f1baed79b4 opti:rm useless code 2023-08-11 20:22:51 +08:00
wangmm0220 deda4b9eed opti:wal logic 2023-08-11 19:26:17 +08:00
wangmm0220 7b22d37f12 fix:lost data when consumer db 2023-08-11 17:25:43 +08:00
wangmm0220 25e32bbf7c fix:offset error in tmq & add test cases 2023-08-11 14:25:12 +08:00
wangmm0220 78fd70202c fix:offset error in tmq & add test cases 2023-08-10 16:18:53 +08:00
wangmm0220 1f5532fd67 fix:offset error in tmq & add test cases 2023-08-09 19:41:22 +08:00
Minglei Jin fac7e521e9 s3/evict: fetch atime from stat file 2023-08-07 15:59:37 +08:00
wade zhang bae028d65c
Merge pull request #22230 from taosdata/mark/tmq
fix:semaphore always wait in tmq commit logic
2023-07-31 09:10:13 +08:00
wangmm0220 acae6df6cc Merge branch '3.0' of https://github.com/taosdata/TDengine into mark/tmq 2023-07-28 15:07:48 +08:00
wangmm0220 ce7db7bf90 fix:subscribe time & return -1 if wal not exist 2023-07-28 15:07:15 +08:00
Haojun Liao 3d403dca5a other: merge main. 2023-07-27 16:23:41 +08:00
wangmm0220 9628a9f74e fix:open info log in tmq & ignore wal apply ver when read wal 2023-07-24 15:53:02 +08:00
Alex Duan 41aa6a0c67 test: test build taos-tools 2023-07-23 18:17:18 +08:00
wangmm0220 bbdcbcb75b fix:modify commit version to next validate version 2023-07-14 16:43:28 +08:00
Haojun Liao 3ffdbe923d other: merge 3.0 2023-07-11 09:12:57 +08:00
wangmm0220 47ae92af93 fix:conflicts from main 2023-07-06 13:40:04 +08:00
Benguang Zhao 9b477eed81 enh: terminate on failure to recover WAL from writing errors, or to commit vnode 2023-07-05 17:54:56 +08:00
wangmm0220 a641a07085 fix:[TD-25071] pSub is null & return wal not exist if no data 2023-07-04 11:13:03 +08:00
Benguang Zhao 60dc26156b enh: tolerate WAL_MAGIC mismatch as long as checksums being valid 2023-06-29 18:34:17 +08:00
Benguang Zhao b434ac2fc9 enh: improve walScanLogGetLastVer 2023-06-29 16:36:28 +08:00
dapan1121 4dae9f4eff merge main to 3.0 2023-06-26 18:43:00 +08:00
wangmm0220 c5dfe169a6 fix:error 2023-06-16 19:33:03 +08:00
wangmm0220 22776e60d5 fix:control memory size if consumer group is too much 2023-06-15 14:51:04 +08:00
wangmm0220 8a81278024 fix:load pHandle if necessary wher tq restore 2023-06-13 15:14:11 +08:00
Haojun Liao b5b4cd2a05 fix(query): load del info with upper version limitation. 2023-06-02 23:15:27 +08:00
Haojun Liao 5301c5756a other: merge main into 3.0. 2023-06-01 09:29:18 +08:00
Haojun Liao b38e253019 fix(stream): set the skip offset when resume stream task. 2023-05-26 22:51:58 +08:00
wangmm0220 a3e214b9e8 fix:put poll to push manager if wal not exist when offset is latest 2023-05-26 19:17:04 +08:00
Haojun Liao 5d5b2bb16a refactor: do some internal refactor. 2023-05-18 12:59:19 +08:00
Haojun Liao 041ffa91bd other: merge main. 2023-05-17 15:42:15 +08:00
Haojun Liao 366cd1871a
Merge pull request #21314 from taosdata/fix/liaohj_main
fix(stream):extract delete msg from wal.
2023-05-17 14:52:49 +08:00
wangmm0220 c52a4ea4f0 fix:merge from 3.0 2023-05-17 01:11:55 +08:00
wangmm0220 e6e0ac7603 fix:alwalys in exec if consumer Id mismatch & check result for wal seek ver 2023-05-16 14:01:38 +08:00
Haojun Liao 30e7cb5860 fix(tmq): avoid return delete msg for table subscription. 2023-05-16 10:42:02 +08:00
Haojun Liao 5e16db4e19 fix(stream):extract delete msg from wal. 2023-05-16 08:54:03 +08:00
Haojun Liao d585f34ea9 fix(tmq): avoid seek to previous position. 2023-05-15 18:48:50 +08:00
wangmm0220 36454c73ed fix:add lock to pHandle for safety 2023-05-15 18:38:14 +08:00
Haojun Liao 236926b9ee other: merge enh/rocksdbstate 2023-05-13 19:49:28 +08:00
Haojun Liao 1975e94501 Merge branch 'main' into enh/rocksdbSstate 2023-05-13 19:41:03 +08:00
Benguang Zhao b8933389bb enh: keep the trailing empty wal file in walCheckAdnRepairMeta 2023-05-11 09:42:46 +08:00
Benguang Zhao 7433bad888 enh: tidy up walEndSnapshot 2023-05-11 09:14:37 +08:00
Benguang Zhao 2a558323be fix: set absent closeTs with mtime of log files during walCheckAndRepairMeta 2023-05-10 20:35:08 +08:00
yihaoDeng 5708fcb5cb Merge branch 'enh/3.0' into enh/rocksdbSstateMerge 2023-05-10 08:02:16 +00:00
Yihao Deng c6e5879f44
Merge branch 'main' into enh/rocksdbSstate 2023-05-10 10:04:28 +08:00
yihaoDeng 8356533eb1 Merge branch 'enh/standVer' into enh/rocksdbSstateMerge 2023-05-09 12:13:16 +00:00
wangmm0220 587f750c94 fix:[TD-24058]send poll result to client if no data 5 times to avoid lost data 2023-05-09 17:42:11 +08:00
wangmm0220 f9142c0ddb fix:[TD24010] lost data if apply ver is small than commit ver 2023-05-08 16:49:56 +08:00
Yihao Deng cb223acf82
Merge branch 'main' into enh/rocksdbSstate 2023-05-08 09:45:27 +08:00
wangmm0220 261f2736f7 fix:[TD-24010] subscribeStb.py failed in windows CI 2023-05-06 19:17:36 +08:00
Haojun Liao 6b305bf745 refactor: do some internal refactor. 2023-05-06 13:35:18 +08:00
Haojun Liao da6212bab0 other: merge main. 2023-05-06 12:51:38 +08:00
Haojun Liao 29fcd1b83a refactor: do some internal refactor. 2023-05-06 10:08:05 +08:00
Haojun Liao ace193ca9b fix(stream): set the seek version if the reader's version is earlier than the first version in wal. 2023-04-29 23:34:49 +08:00
wangmm0220 1c63408b3e opti:change push mgr to consume msg for subscribe 2023-04-24 20:18:20 +08:00
Haojun Liao c654f1145d enh(stream): add new msg for seek, and do some internal refactor. 2023-04-20 10:59:16 +08:00
wangmm0220 4d9d1b520d fix:add log for wal 2023-04-19 18:16:57 +08:00
wangmm0220 6f94281ab7 fix:add log for wal 2023-04-19 17:30:58 +08:00