Commit Graph

13092 Commits

Author SHA1 Message Date
Haojun Liao 0449b5f1bd other: add some logs. 2023-12-14 15:43:59 +08:00
dmchen 1ca0858a97 compact detail log 2023-12-14 07:11:00 +00:00
wade zhang c95c567376
Merge pull request #24046 from taosdata/fix/TS-4251-3.0
revert dependency
2023-12-14 14:03:04 +08:00
Haojun Liao 05ca8f1784
Merge pull request #24056 from taosdata/fix/3_liaohj
refactor: do some internal refactor.
2023-12-14 13:52:04 +08:00
Haojun Liao 7cf0add513 fix(stream): fix error caused by refactor. 2023-12-14 11:28:03 +08:00
dmchen 606a6fe891 field encode 2023-12-14 03:24:53 +00:00
dapan1121 7803a717c2
Merge pull request #24051 from taosdata/fix/TD-27839-3.0
enh: add check and reduce offset dependencies in lib
2023-12-14 10:19:02 +08:00
Haojun Liao f0d63a977a refactor: do some internal refactor. 2023-12-14 10:01:57 +08:00
Haojun Liao 8e042e34cb refactor: do some internal refactor. 2023-12-14 09:48:14 +08:00
Haojun Liao 81f96603b0 refactor: do some internal refactor. 2023-12-14 09:33:51 +08:00
Cary Xu 46f27a77f1
Update dmMgmt.h 2023-12-13 23:05:46 +08:00
Haojun Liao 26af397431
Merge pull request #24052 from taosdata/fix/3_liaohj
fix(tsdb): fix error in tsdb read.
2023-12-13 22:35:08 +08:00
Haojun Liao 105542bb21 fix(stream):handle the case of delete orphaned tasks. 2023-12-13 17:44:00 +08:00
Haojun Liao 88a8e4e9f0 fix(tsdb): fix syntax error. 2023-12-13 17:01:55 +08:00
Haojun Liao 840b457308 fix(tsdb): fix error in tsdb read. 2023-12-13 17:00:39 +08:00
kailixu f685b6c34a enh: reduce offset dependencies in lib 2023-12-13 16:56:45 +08:00
dmchen 04366f2e68 revert dependency 2023-12-13 15:29:32 +08:00
wade zhang fe0248afbc
Merge pull request #24029 from taosdata/fix/TS-4251-3.0
Fix/ts 4251 3.0
2023-12-13 11:05:51 +08:00
wade zhang 3be9af9765
Merge pull request #24039 from taosdata/FIX/TD-27843-3.0
enh: resend new snap replication msg on all acked if not finished yet
2023-12-13 09:51:31 +08:00
dmchen e9cf639adb db is null 2023-12-13 01:04:34 +00:00
dmchen 71020e4812 show command db field is null 2023-12-13 00:53:33 +00:00
dapan1121 8528fb7a15
Merge pull request #23963 from taosdata/enh/xsren/TD-22212/taosQueryParam3.0
enh: td-22212 taos query param3.0
2023-12-13 08:24:39 +08:00
dmchen 4c9763d2c1 mem leak 2023-12-12 13:12:04 +00:00
Benguang Zhao b1dd76b924 fix: assert on offset leq file.size in tsdbDataFileRAWWriterCloseCommit 2023-12-12 18:33:42 +08:00
dmchen 7e2ce6381f ci case 2023-12-12 07:24:13 +00:00
Hongze Cheng 613ca2c370
Merge pull request #23982 from taosdata/fix/PI-30-3.0
fix(tsdb/cache): remove malloc zero bytes
2023-12-12 15:22:38 +08:00
factosea d25275713e ctest 2023-12-12 13:59:56 +08:00
dmchen ea9eba5621 fix ci case 2023-12-12 05:34:41 +00:00
dmchen dad76804e8 mem leak 2023-12-12 12:19:43 +08:00
dmchen 1547d95742 break dependency 2023-12-12 12:19:12 +08:00
dmchen 67e9f695c9 debug info and default setting 2023-12-12 12:18:35 +08:00
dmchen daa2c2238a fix/TS-4251 2023-12-12 12:17:49 +08:00
facetosea 33897447cb ctest on linux exclude 2023-12-12 10:37:39 +08:00
Minglei Jin c4bf9a92bc Merge branch '3.0' into enh/TD-27119 2023-12-11 19:38:04 +08:00
Minglei Jin fe824c51e2 tsdb/cos: not remove cp files 2023-12-11 19:36:06 +08:00
dmchen a861e55764 revert dependency 2023-12-11 11:24:31 +00:00
slzhou 2743228217 Merge branch '3.0' of github.com:taosdata/TDengine into feat/TS-4229 2023-12-11 14:15:17 +08:00
Haojun Liao 39ea7d7567
Merge pull request #23980 from taosdata/fix/3_liaohj
fix(tsdb): opt read stt file
2023-12-09 21:28:52 +08:00
Haojun Liao 9acd4af1f9 fix(tsdb):fix capacity rows value. 2023-12-09 01:49:55 +08:00
Haojun Liao 5528c2bf67 fix(tsdb): do some internal refactor. 2023-12-08 23:47:49 +08:00
Haojun Liao f2592f7399 fix(tsdb): prepare the memory buffer. 2023-12-08 23:44:05 +08:00
Haojun Liao da636ab527 fix(tsdb): add null check. 2023-12-08 18:55:20 +08:00
Haojun Liao 67d967e22f fix(tsdb): check the overlap between stt blocks and data file blocks. 2023-12-08 18:48:00 +08:00
dmchen e8617812ae update progress rsp 2023-12-08 10:17:04 +00:00
Hongze Cheng 853fb6f119
Merge pull request #23987 from taosdata/fix/little_fix_3.0
fix: little fix
2023-12-08 17:29:24 +08:00
Hongze Cheng f830fc2f4c more dev 2023-12-08 17:26:44 +08:00
Haojun Liao 63b34c7acf fix(tsdb): add check for not load stt file blocks. 2023-12-08 16:09:58 +08:00
dmchen 0be39ebe77 mem leak 2023-12-08 07:26:20 +00:00
Minglei Jin 2dbb718101 Merge branch '3.0' into enh/TD-27119 2023-12-08 14:48:17 +08:00
Hongze Cheng a0487529d0 more fix 2023-12-08 14:37:21 +08:00
wade zhang 638e4c1f00
Merge pull request #23568 from taosdata/FEAT/TD-27125-3.0
feat: support tsdb snap replication of raw data mode
2023-12-08 14:36:16 +08:00
Haojun Liao 9c72ce846e fix(tsdb): add more condition for not load data. 2023-12-08 14:23:37 +08:00
Hongze Cheng 16068bab41
Merge pull request #23964 from taosdata/fix/TD-27707
fix(tsdb/cache): del from mem with time range
2023-12-08 14:22:08 +08:00
Hongze Cheng b9d056c6a8 fix: little fix 2023-12-08 14:11:36 +08:00
dmchen 18dbc47949 break dependency 2023-12-08 04:24:33 +00:00
Haojun Liao da1207ef74 fix(tsdb): add return flag for stt files. 2023-12-08 11:07:45 +08:00
Minglei Jin d4dbb55669 fix(tsdb/cache): remove malloc zero bytes 2023-12-08 11:04:04 +08:00
Haojun Liao 2e13b0b97d refactor: do some internal refactor. 2023-12-08 10:43:17 +08:00
Haojun Liao 9477146b30 fix(tsdb): fix syntax error. 2023-12-08 10:29:38 +08:00
Haojun Liao 4ffaff4d50 other: merge 3.0 2023-12-08 10:14:42 +08:00
Haojun Liao 446c14da72 fix(tsdb): opt read stt file 2023-12-08 10:03:36 +08:00
dapan1121 a2ee41233e
Merge pull request #23835 from taosdata/szhou/tms-duration
feat: tsdb output in duration order
2023-12-08 09:07:41 +08:00
Haojun Liao 7c17d6f313 fix(tsdb): opt read performance by check clean stt block if only row number required. 2023-12-07 23:12:54 +08:00
dmchen 348bef49ec message handle 2023-12-07 11:42:32 +00:00
Hongze Cheng aee070918b more dev 2023-12-07 17:37:47 +08:00
Benguang Zhao 1255b46469 fix: destroy dataReaderArr properly in tsdbSnapRAWReaderClose 2023-12-07 15:29:42 +08:00
Benguang Zhao ebd3b69745 enh: control playload size of tsdb snap replication with TSDB_SNAP_DATA_PAYLOAD_SIZE 2023-12-07 15:29:42 +08:00
Benguang Zhao 5cdf2b0b04 fixup: remove call to obsolete funcs of BgTask in tsdbSnapRAWWriter open and close 2023-12-07 15:29:42 +08:00
Benguang Zhao 28e1d83662 fix: adjust tsdbReadFile call in tsdbDataFileRAWReadBlockData 2023-12-07 15:29:42 +08:00
Benguang Zhao 52672657c1 feat: toggle tsdb snap replication mode by snap info handshake ahead of time 2023-12-07 15:29:42 +08:00
Benguang Zhao ef34176e37 fix: close data file readers in tsbSnapRAWReaderClose 2023-12-07 15:29:42 +08:00
Benguang Zhao 6c4c0242fd refact: adjust logging format in tsdbSnapRAWReaderOpen 2023-12-07 15:29:42 +08:00
Benguang Zhao f136266972 feat: support snap replication by file blocks 2023-12-07 15:29:42 +08:00
Benguang Zhao 1dc9019baa feat: add tsdbSnapRAWRead into vnodeSnapRead 2023-12-07 15:29:42 +08:00
Benguang Zhao ae60e1f810 refact: add tsdbSnapRAWReaderIterNext 2023-12-07 15:29:42 +08:00
Benguang Zhao a504007ade refact: alloc just enough memory for snap data in tsdbSnapRAWReadData 2023-12-07 15:29:42 +08:00
Benguang Zhao 6c419423de feat: impl tsdb snapshot reader and writer for raw files 2023-12-07 15:29:42 +08:00
Benguang Zhao a4c504169d feat: add file tsdbFSetRAW.c 2023-12-07 15:29:42 +08:00
Benguang Zhao da646b5a5b feat: add file tsdbDataFileRAW.c 2023-12-07 15:29:42 +08:00
Benguang Zhao e05914119e feat: add file tsdbSnapshotRAW.c 2023-12-07 15:29:42 +08:00
Benguang Zhao fe258e226d feat: add data type of SNAP_DATA_RAW for snap replication 2023-12-07 15:29:42 +08:00
Benguang Zhao de9c9d4f20 refact: improve code of tsdbPartitionInfoSerialize 2023-12-07 15:28:14 +08:00
Benguang Zhao 1a9b08fa08 refact: relocate func syncSnapInfoDataRealloc 2023-12-07 15:28:14 +08:00
Benguang Zhao f68804322c refact: improve code of tsdbSnapGetDetails as tsdbSnapPrepDescription 2023-12-07 15:28:14 +08:00
Benguang Zhao ed39c9a572 refact: rename snap range to fset range in tsdbSnapInfo.c 2023-12-07 15:28:14 +08:00
Benguang Zhao 3550347f0c refact: use tsdbFSDestroyCopyRangedSnapshot and tsdbFSDestroyRefRangedSnapshot 2023-12-07 15:28:14 +08:00
Benguang Zhao 962febef02 refact: rename tsdb snap range to tsdb fset range 2023-12-07 15:28:14 +08:00
Benguang Zhao ed916b1a80 refact: rename tsdb snap partition to fset partition 2023-12-07 15:28:14 +08:00
Benguang Zhao 20b5cf8d49 refact: remove redundent includes from tsdbSnapInfo.c 2023-12-07 15:28:14 +08:00
Benguang Zhao 4487eeacb5 refact: improve funcs dealing with SnapInfo for vnode snap reader and writer 2023-12-07 15:28:14 +08:00
Benguang Zhao b8b53a4b0e refact: put source code of snap info into tsdbSnapInfo.c 2023-12-07 15:28:06 +08:00
Benguang Zhao 3ad68bdfea enh: add tsdbSnapInfo.c 2023-12-07 15:20:55 +08:00
Benguang Zhao ddcabcfa4a refact: remove unused fn tSerializeSnapRangeArray 2023-12-07 15:20:55 +08:00
Benguang Zhao e429526789 refact: rename STSnapRange to STFileSetRange 2023-12-07 15:20:55 +08:00
dmchen 27a86a6faa debug info and default setting 2023-12-07 03:13:23 +00:00
dapan1121 30eb035447
Merge pull request #23940 from taosdata/fix/TS-4327
fix: last/last_row crash with dropped column
2023-12-06 17:57:24 +08:00
Minglei Jin a9309cb295 fix(tsdb/cache): del from mem with time range 2023-12-06 17:52:15 +08:00
factosea f01241f720 Merge branch 'fix/xsren/TD-22212/taosQueryParam3.0' into enh/xsren/TD-22212/taosQueryParam3.0 2023-12-06 17:39:25 +08:00
Hongze Cheng 04e84ba152 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TS-4229 2023-12-06 16:43:00 +08:00
dmchen dc9408ed5f fix/TS-4251 2023-12-06 07:48:40 +00:00
shenglian zhou a2e7c78d27 refactor: rename variables 2023-12-06 09:39:38 +08:00
Haojun Liao 85c7dc76e9
Merge pull request #23948 from taosdata/mark/3.0
fix:offset out of range if poll offset is read from offsetRestored wh…
2023-12-06 09:12:38 +08:00
wangmm0220 cc51fd47d0 fix:offset out of range if poll offset is read from offsetRestored while wal is deleted 2023-12-05 19:09:29 +08:00
yihaoDeng 87c78919a9 add stream backend copy 2023-12-05 17:09:03 +08:00
Minglei Jin e9cc54d87b cos_cp/load: load checkpoint from json 2023-12-05 16:37:28 +08:00
wangjiaming0909 a6c876bbc1 fix: last/last_row crash with dropped column 2023-12-05 14:16:01 +08:00
Minglei Jin e0a14bd359 fix(tsdb/cache): remove debug log 2023-12-05 14:05:16 +08:00
Haojun Liao 9d451ab56e
Merge pull request #23934 from taosdata/fix/3_liaohj
refactor: do some internal refactor.
2023-12-05 12:58:18 +08:00
wangmm0220 79c8a1494d fix:disable snode unit test temporary 2023-12-05 11:26:12 +08:00
Haojun Liao cb8dfae0cc refactor: do some internal refactor. 2023-12-05 10:47:19 +08:00
slzhou e690076331 Merge branch '3.0' of github.com:taosdata/TDengine into szhou/tms-duration 2023-12-05 09:53:29 +08:00
Haojun Liao cc5890d56a
Merge pull request #23924 from taosdata/fix/3_liaohj
enh(tsdb): extract rows for the given tables in all stt files.
2023-12-04 19:21:19 +08:00
Hongze Cheng 26774f87c9
Merge pull request #23920 from taosdata/fix/TD-27619
fix: bg task memory waste
2023-12-04 18:10:30 +08:00
Haojun Liao 7145a7b8a1 fix(tsdb): calculate the rows in next fileset. 2023-12-04 18:03:32 +08:00
Haojun Liao 6d82900a32 fix(tsdb): fix error in block dist. 2023-12-04 18:00:41 +08:00
Hongze Cheng ac2d9ebc27 fix: memory leak 2023-12-04 16:48:43 +08:00
slzhou 8c290587b7 fix: typo error 2023-12-04 16:19:07 +08:00
Haojun Liao 24d9337632 enh(tsdb): extract rows for the given tables in all stt files. 2023-12-04 15:38:24 +08:00
slzhou a517271f39 Merge branch 'szhou/tms-duration-2' into szhou/tms-duration 2023-12-04 14:56:19 +08:00
slzhou 07e43fe56b fix: two intervals are intesected or not 2023-12-04 14:54:15 +08:00
Hongze Cheng cea4be6f3c fix: bg task memory waste 2023-12-04 13:40:28 +08:00
54liuyao 38900d672e Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-27419 2023-12-04 09:12:53 +08:00
Haojun Liao 48af3a9559
Merge pull request #23909 from taosdata/fix/TD-27455
fix:[TD-27644]heartbeat closed before snode close leading to snode is…
2023-12-04 09:10:00 +08:00
Haojun Liao 5467bba86b fix(tsdb): do some internal refactor. 2023-12-04 09:03:40 +08:00
Haojun Liao 7b0cf47fd6 fix(tsdb): do some internal refactor. 2023-12-04 08:58:10 +08:00
Haojun Liao 29a1c6d518 fix(tsdb): do some internal refactor. 2023-12-04 08:55:53 +08:00
wangmm0220 f06117dbc9 fix:[TD-27644]heartbeat closed before snode close leading to snode is hanged 2023-12-01 17:32:29 +08:00
Haojun Liao 3194ff41aa
Merge pull request #23900 from taosdata/fix/3_liaohj
fix(stream): merge fix from main to 3.0
2023-12-01 16:39:38 +08:00
shenglian zhou bff88e0639 fix: set duration order api 2023-12-01 16:27:36 +08:00
Haojun Liao 4d9211dcf1
Merge pull request #23899 from taosdata/fix/liaohj
other: merge the patch of stream from 3.0 branch.
2023-12-01 15:49:54 +08:00
slzhou 82629c2658 fix: notify when there are no memory pre fileset 2023-12-01 15:11:43 +08:00
Haojun Liao d7a19a9179 fix(stream): fix invalid decode hbmsg and memory leaks caused by invalid hb msg. 2023-12-01 14:45:18 +08:00
Haojun Liao 7e63943c4e
Merge pull request #23860 from taosdata/fix/TD-27455
fix:tmqVnodeSplit-column.py failed in arm64 because of rebalance after unsubscribe while split vnode
2023-12-01 14:18:58 +08:00
wade zhang 041add545c
Merge pull request #23844 from taosdata/FIX/TS-4306-3.0
enh: limit size of snap replication msg by data length
2023-12-01 14:06:36 +08:00
Haojun Liao 33ad05e2c8 fix(stream): fix syntax error. 2023-12-01 13:08:25 +08:00
slzhou 4d755562af fix: resolve conflicts 2023-12-01 13:02:14 +08:00
Haojun Liao 12f32f0ae7 fix(stream): fix syntax error. 2023-12-01 11:57:16 +08:00
Haojun Liao dba58f5e38 fix(stream): fix syntax error. 2023-12-01 11:54:13 +08:00
Haojun Liao eac3cbf564 fix(stream): merge 3.0 2023-12-01 11:12:57 +08:00
Haojun Liao 18273b8d1b fix(stream): ignore the in_progress code. 2023-12-01 10:20:45 +08:00
Haojun Liao 2972f5f420 fix(stream):return the error code if create stream failed. 2023-12-01 10:20:45 +08:00
Haojun Liao e008d0f440 fix(tsdb): fix invalid read. 2023-12-01 10:20:45 +08:00
Haojun Liao fbb604d1f5 fix(tsdb): fix bugs in suspend/resume. adjust code to conduct the test more easily. td-27579 2023-12-01 10:20:44 +08:00
Haojun Liao 1c9ff8030b fix(tsdb):clear the tsdb stt file reader flag when suspending the tsdbReader 2023-12-01 10:20:39 +08:00
Haojun Liao 86618ffd97 refactor: do some internal refactor. 2023-12-01 10:20:38 +08:00
Haojun Liao 31ae46925d fix(stream): fix invalid free. 2023-12-01 10:20:32 +08:00
Haojun Liao 17d4a22e47 fix(stream): add logs. 2023-12-01 10:19:54 +08:00
Haojun Liao b9c727575d fix(stream): reset the scheduler status. 2023-12-01 10:19:54 +08:00
Haojun Liao c937f42d88 fix(stream): fix deadlock. 2023-12-01 10:19:54 +08:00
Haojun Liao 8515a2f9b6 fix(stream): fix status check for stream-scan-history status when drop stream task msg is missing. 2023-12-01 10:19:18 +08:00