Commit Graph

8347 Commits

Author SHA1 Message Date
Minglei Jin fe824c51e2 tsdb/cos: not remove cp files 2023-12-11 19:36:06 +08:00
dmchen a861e55764 revert dependency 2023-12-11 11:24:31 +00:00
slzhou 2743228217 Merge branch '3.0' of github.com:taosdata/TDengine into feat/TS-4229 2023-12-11 14:15:17 +08:00
Haojun Liao 39ea7d7567
Merge pull request #23980 from taosdata/fix/3_liaohj
fix(tsdb): opt read stt file
2023-12-09 21:28:52 +08:00
Haojun Liao 9acd4af1f9 fix(tsdb):fix capacity rows value. 2023-12-09 01:49:55 +08:00
Haojun Liao 5528c2bf67 fix(tsdb): do some internal refactor. 2023-12-08 23:47:49 +08:00
Haojun Liao f2592f7399 fix(tsdb): prepare the memory buffer. 2023-12-08 23:44:05 +08:00
Haojun Liao da636ab527 fix(tsdb): add null check. 2023-12-08 18:55:20 +08:00
Haojun Liao 67d967e22f fix(tsdb): check the overlap between stt blocks and data file blocks. 2023-12-08 18:48:00 +08:00
Hongze Cheng 853fb6f119
Merge pull request #23987 from taosdata/fix/little_fix_3.0
fix: little fix
2023-12-08 17:29:24 +08:00
Hongze Cheng f830fc2f4c more dev 2023-12-08 17:26:44 +08:00
Haojun Liao 63b34c7acf fix(tsdb): add check for not load stt file blocks. 2023-12-08 16:09:58 +08:00
Minglei Jin 2dbb718101 Merge branch '3.0' into enh/TD-27119 2023-12-08 14:48:17 +08:00
Hongze Cheng a0487529d0 more fix 2023-12-08 14:37:21 +08:00
wade zhang 638e4c1f00
Merge pull request #23568 from taosdata/FEAT/TD-27125-3.0
feat: support tsdb snap replication of raw data mode
2023-12-08 14:36:16 +08:00
Haojun Liao 9c72ce846e fix(tsdb): add more condition for not load data. 2023-12-08 14:23:37 +08:00
Hongze Cheng 16068bab41
Merge pull request #23964 from taosdata/fix/TD-27707
fix(tsdb/cache): del from mem with time range
2023-12-08 14:22:08 +08:00
Hongze Cheng b9d056c6a8 fix: little fix 2023-12-08 14:11:36 +08:00
dmchen 18dbc47949 break dependency 2023-12-08 04:24:33 +00:00
Haojun Liao da1207ef74 fix(tsdb): add return flag for stt files. 2023-12-08 11:07:45 +08:00
Minglei Jin d4dbb55669 fix(tsdb/cache): remove malloc zero bytes 2023-12-08 11:04:04 +08:00
Haojun Liao 2e13b0b97d refactor: do some internal refactor. 2023-12-08 10:43:17 +08:00
Haojun Liao 9477146b30 fix(tsdb): fix syntax error. 2023-12-08 10:29:38 +08:00
Haojun Liao 4ffaff4d50 other: merge 3.0 2023-12-08 10:14:42 +08:00
Haojun Liao 446c14da72 fix(tsdb): opt read stt file 2023-12-08 10:03:36 +08:00
dapan1121 a2ee41233e
Merge pull request #23835 from taosdata/szhou/tms-duration
feat: tsdb output in duration order
2023-12-08 09:07:41 +08:00
Haojun Liao 7c17d6f313 fix(tsdb): opt read performance by check clean stt block if only row number required. 2023-12-07 23:12:54 +08:00
dmchen 348bef49ec message handle 2023-12-07 11:42:32 +00:00
Hongze Cheng aee070918b more dev 2023-12-07 17:37:47 +08:00
Benguang Zhao 1255b46469 fix: destroy dataReaderArr properly in tsdbSnapRAWReaderClose 2023-12-07 15:29:42 +08:00
Benguang Zhao ebd3b69745 enh: control playload size of tsdb snap replication with TSDB_SNAP_DATA_PAYLOAD_SIZE 2023-12-07 15:29:42 +08:00
Benguang Zhao 5cdf2b0b04 fixup: remove call to obsolete funcs of BgTask in tsdbSnapRAWWriter open and close 2023-12-07 15:29:42 +08:00
Benguang Zhao 28e1d83662 fix: adjust tsdbReadFile call in tsdbDataFileRAWReadBlockData 2023-12-07 15:29:42 +08:00
Benguang Zhao 52672657c1 feat: toggle tsdb snap replication mode by snap info handshake ahead of time 2023-12-07 15:29:42 +08:00
Benguang Zhao ef34176e37 fix: close data file readers in tsbSnapRAWReaderClose 2023-12-07 15:29:42 +08:00
Benguang Zhao 6c4c0242fd refact: adjust logging format in tsdbSnapRAWReaderOpen 2023-12-07 15:29:42 +08:00
Benguang Zhao f136266972 feat: support snap replication by file blocks 2023-12-07 15:29:42 +08:00
Benguang Zhao 1dc9019baa feat: add tsdbSnapRAWRead into vnodeSnapRead 2023-12-07 15:29:42 +08:00
Benguang Zhao ae60e1f810 refact: add tsdbSnapRAWReaderIterNext 2023-12-07 15:29:42 +08:00
Benguang Zhao a504007ade refact: alloc just enough memory for snap data in tsdbSnapRAWReadData 2023-12-07 15:29:42 +08:00
Benguang Zhao 6c419423de feat: impl tsdb snapshot reader and writer for raw files 2023-12-07 15:29:42 +08:00
Benguang Zhao a4c504169d feat: add file tsdbFSetRAW.c 2023-12-07 15:29:42 +08:00
Benguang Zhao da646b5a5b feat: add file tsdbDataFileRAW.c 2023-12-07 15:29:42 +08:00
Benguang Zhao e05914119e feat: add file tsdbSnapshotRAW.c 2023-12-07 15:29:42 +08:00
Benguang Zhao fe258e226d feat: add data type of SNAP_DATA_RAW for snap replication 2023-12-07 15:29:42 +08:00
Benguang Zhao de9c9d4f20 refact: improve code of tsdbPartitionInfoSerialize 2023-12-07 15:28:14 +08:00
Benguang Zhao 1a9b08fa08 refact: relocate func syncSnapInfoDataRealloc 2023-12-07 15:28:14 +08:00
Benguang Zhao f68804322c refact: improve code of tsdbSnapGetDetails as tsdbSnapPrepDescription 2023-12-07 15:28:14 +08:00
Benguang Zhao ed39c9a572 refact: rename snap range to fset range in tsdbSnapInfo.c 2023-12-07 15:28:14 +08:00
Benguang Zhao 3550347f0c refact: use tsdbFSDestroyCopyRangedSnapshot and tsdbFSDestroyRefRangedSnapshot 2023-12-07 15:28:14 +08:00
Benguang Zhao 962febef02 refact: rename tsdb snap range to tsdb fset range 2023-12-07 15:28:14 +08:00
Benguang Zhao ed916b1a80 refact: rename tsdb snap partition to fset partition 2023-12-07 15:28:14 +08:00
Benguang Zhao 20b5cf8d49 refact: remove redundent includes from tsdbSnapInfo.c 2023-12-07 15:28:14 +08:00
Benguang Zhao 4487eeacb5 refact: improve funcs dealing with SnapInfo for vnode snap reader and writer 2023-12-07 15:28:14 +08:00
Benguang Zhao b8b53a4b0e refact: put source code of snap info into tsdbSnapInfo.c 2023-12-07 15:28:06 +08:00
Benguang Zhao 3ad68bdfea enh: add tsdbSnapInfo.c 2023-12-07 15:20:55 +08:00
Benguang Zhao ddcabcfa4a refact: remove unused fn tSerializeSnapRangeArray 2023-12-07 15:20:55 +08:00
Benguang Zhao e429526789 refact: rename STSnapRange to STFileSetRange 2023-12-07 15:20:55 +08:00
dmchen 27a86a6faa debug info and default setting 2023-12-07 03:13:23 +00:00
dapan1121 30eb035447
Merge pull request #23940 from taosdata/fix/TS-4327
fix: last/last_row crash with dropped column
2023-12-06 17:57:24 +08:00
Minglei Jin a9309cb295 fix(tsdb/cache): del from mem with time range 2023-12-06 17:52:15 +08:00
Hongze Cheng 04e84ba152 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TS-4229 2023-12-06 16:43:00 +08:00
dmchen dc9408ed5f fix/TS-4251 2023-12-06 07:48:40 +00:00
shenglian zhou a2e7c78d27 refactor: rename variables 2023-12-06 09:39:38 +08:00
Haojun Liao 85c7dc76e9
Merge pull request #23948 from taosdata/mark/3.0
fix:offset out of range if poll offset is read from offsetRestored wh…
2023-12-06 09:12:38 +08:00
wangmm0220 cc51fd47d0 fix:offset out of range if poll offset is read from offsetRestored while wal is deleted 2023-12-05 19:09:29 +08:00
yihaoDeng 87c78919a9 add stream backend copy 2023-12-05 17:09:03 +08:00
Minglei Jin e9cc54d87b cos_cp/load: load checkpoint from json 2023-12-05 16:37:28 +08:00
wangjiaming0909 a6c876bbc1 fix: last/last_row crash with dropped column 2023-12-05 14:16:01 +08:00
Minglei Jin e0a14bd359 fix(tsdb/cache): remove debug log 2023-12-05 14:05:16 +08:00
Haojun Liao 9d451ab56e
Merge pull request #23934 from taosdata/fix/3_liaohj
refactor: do some internal refactor.
2023-12-05 12:58:18 +08:00
Haojun Liao cb8dfae0cc refactor: do some internal refactor. 2023-12-05 10:47:19 +08:00
slzhou e690076331 Merge branch '3.0' of github.com:taosdata/TDengine into szhou/tms-duration 2023-12-05 09:53:29 +08:00
Haojun Liao cc5890d56a
Merge pull request #23924 from taosdata/fix/3_liaohj
enh(tsdb): extract rows for the given tables in all stt files.
2023-12-04 19:21:19 +08:00
Hongze Cheng 26774f87c9
Merge pull request #23920 from taosdata/fix/TD-27619
fix: bg task memory waste
2023-12-04 18:10:30 +08:00
Haojun Liao 7145a7b8a1 fix(tsdb): calculate the rows in next fileset. 2023-12-04 18:03:32 +08:00
Haojun Liao 6d82900a32 fix(tsdb): fix error in block dist. 2023-12-04 18:00:41 +08:00
Hongze Cheng ac2d9ebc27 fix: memory leak 2023-12-04 16:48:43 +08:00
slzhou 8c290587b7 fix: typo error 2023-12-04 16:19:07 +08:00
Haojun Liao 24d9337632 enh(tsdb): extract rows for the given tables in all stt files. 2023-12-04 15:38:24 +08:00
slzhou a517271f39 Merge branch 'szhou/tms-duration-2' into szhou/tms-duration 2023-12-04 14:56:19 +08:00
slzhou 07e43fe56b fix: two intervals are intesected or not 2023-12-04 14:54:15 +08:00
Hongze Cheng cea4be6f3c fix: bg task memory waste 2023-12-04 13:40:28 +08:00
54liuyao 38900d672e Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-27419 2023-12-04 09:12:53 +08:00
Haojun Liao 48af3a9559
Merge pull request #23909 from taosdata/fix/TD-27455
fix:[TD-27644]heartbeat closed before snode close leading to snode is…
2023-12-04 09:10:00 +08:00
Haojun Liao 5467bba86b fix(tsdb): do some internal refactor. 2023-12-04 09:03:40 +08:00
Haojun Liao 7b0cf47fd6 fix(tsdb): do some internal refactor. 2023-12-04 08:58:10 +08:00
Haojun Liao 29a1c6d518 fix(tsdb): do some internal refactor. 2023-12-04 08:55:53 +08:00
wangmm0220 f06117dbc9 fix:[TD-27644]heartbeat closed before snode close leading to snode is hanged 2023-12-01 17:32:29 +08:00
Haojun Liao 3194ff41aa
Merge pull request #23900 from taosdata/fix/3_liaohj
fix(stream): merge fix from main to 3.0
2023-12-01 16:39:38 +08:00
shenglian zhou bff88e0639 fix: set duration order api 2023-12-01 16:27:36 +08:00
slzhou 82629c2658 fix: notify when there are no memory pre fileset 2023-12-01 15:11:43 +08:00
Haojun Liao 7e63943c4e
Merge pull request #23860 from taosdata/fix/TD-27455
fix:tmqVnodeSplit-column.py failed in arm64 because of rebalance after unsubscribe while split vnode
2023-12-01 14:18:58 +08:00
wade zhang 041add545c
Merge pull request #23844 from taosdata/FIX/TS-4306-3.0
enh: limit size of snap replication msg by data length
2023-12-01 14:06:36 +08:00
slzhou 4d755562af fix: resolve conflicts 2023-12-01 13:02:14 +08:00
Haojun Liao eac3cbf564 fix(stream): merge 3.0 2023-12-01 11:12:57 +08:00
Haojun Liao e008d0f440 fix(tsdb): fix invalid read. 2023-12-01 10:20:45 +08:00
Haojun Liao fbb604d1f5 fix(tsdb): fix bugs in suspend/resume. adjust code to conduct the test more easily. td-27579 2023-12-01 10:20:44 +08:00
Haojun Liao 1c9ff8030b fix(tsdb):clear the tsdb stt file reader flag when suspending the tsdbReader 2023-12-01 10:20:39 +08:00
Haojun Liao 17d4a22e47 fix(stream): add logs. 2023-12-01 10:19:54 +08:00