Commit Graph

13228 Commits

Author SHA1 Message Date
Hongze Cheng aee070918b more dev 2023-12-07 17:37:47 +08:00
Benguang Zhao 1255b46469 fix: destroy dataReaderArr properly in tsdbSnapRAWReaderClose 2023-12-07 15:29:42 +08:00
Benguang Zhao ebd3b69745 enh: control playload size of tsdb snap replication with TSDB_SNAP_DATA_PAYLOAD_SIZE 2023-12-07 15:29:42 +08:00
Benguang Zhao 5cdf2b0b04 fixup: remove call to obsolete funcs of BgTask in tsdbSnapRAWWriter open and close 2023-12-07 15:29:42 +08:00
Benguang Zhao 28e1d83662 fix: adjust tsdbReadFile call in tsdbDataFileRAWReadBlockData 2023-12-07 15:29:42 +08:00
Benguang Zhao 52672657c1 feat: toggle tsdb snap replication mode by snap info handshake ahead of time 2023-12-07 15:29:42 +08:00
Benguang Zhao ef34176e37 fix: close data file readers in tsbSnapRAWReaderClose 2023-12-07 15:29:42 +08:00
Benguang Zhao 6c4c0242fd refact: adjust logging format in tsdbSnapRAWReaderOpen 2023-12-07 15:29:42 +08:00
Benguang Zhao f136266972 feat: support snap replication by file blocks 2023-12-07 15:29:42 +08:00
Benguang Zhao 1dc9019baa feat: add tsdbSnapRAWRead into vnodeSnapRead 2023-12-07 15:29:42 +08:00
Benguang Zhao ae60e1f810 refact: add tsdbSnapRAWReaderIterNext 2023-12-07 15:29:42 +08:00
Benguang Zhao a504007ade refact: alloc just enough memory for snap data in tsdbSnapRAWReadData 2023-12-07 15:29:42 +08:00
Benguang Zhao 6c419423de feat: impl tsdb snapshot reader and writer for raw files 2023-12-07 15:29:42 +08:00
Benguang Zhao a4c504169d feat: add file tsdbFSetRAW.c 2023-12-07 15:29:42 +08:00
Benguang Zhao da646b5a5b feat: add file tsdbDataFileRAW.c 2023-12-07 15:29:42 +08:00
Benguang Zhao e05914119e feat: add file tsdbSnapshotRAW.c 2023-12-07 15:29:42 +08:00
Benguang Zhao fe258e226d feat: add data type of SNAP_DATA_RAW for snap replication 2023-12-07 15:29:42 +08:00
Benguang Zhao de9c9d4f20 refact: improve code of tsdbPartitionInfoSerialize 2023-12-07 15:28:14 +08:00
Benguang Zhao 1a9b08fa08 refact: relocate func syncSnapInfoDataRealloc 2023-12-07 15:28:14 +08:00
Benguang Zhao f68804322c refact: improve code of tsdbSnapGetDetails as tsdbSnapPrepDescription 2023-12-07 15:28:14 +08:00
Benguang Zhao ed39c9a572 refact: rename snap range to fset range in tsdbSnapInfo.c 2023-12-07 15:28:14 +08:00
Benguang Zhao 3550347f0c refact: use tsdbFSDestroyCopyRangedSnapshot and tsdbFSDestroyRefRangedSnapshot 2023-12-07 15:28:14 +08:00
Benguang Zhao 962febef02 refact: rename tsdb snap range to tsdb fset range 2023-12-07 15:28:14 +08:00
Benguang Zhao ed916b1a80 refact: rename tsdb snap partition to fset partition 2023-12-07 15:28:14 +08:00
Benguang Zhao 20b5cf8d49 refact: remove redundent includes from tsdbSnapInfo.c 2023-12-07 15:28:14 +08:00
Benguang Zhao 4487eeacb5 refact: improve funcs dealing with SnapInfo for vnode snap reader and writer 2023-12-07 15:28:14 +08:00
Benguang Zhao b8b53a4b0e refact: put source code of snap info into tsdbSnapInfo.c 2023-12-07 15:28:06 +08:00
Benguang Zhao 3ad68bdfea enh: add tsdbSnapInfo.c 2023-12-07 15:20:55 +08:00
Benguang Zhao ddcabcfa4a refact: remove unused fn tSerializeSnapRangeArray 2023-12-07 15:20:55 +08:00
Benguang Zhao e429526789 refact: rename STSnapRange to STFileSetRange 2023-12-07 15:20:55 +08:00
dmchen 27a86a6faa debug info and default setting 2023-12-07 03:13:23 +00:00
dapan1121 30eb035447
Merge pull request #23940 from taosdata/fix/TS-4327
fix: last/last_row crash with dropped column
2023-12-06 17:57:24 +08:00
Minglei Jin a9309cb295 fix(tsdb/cache): del from mem with time range 2023-12-06 17:52:15 +08:00
factosea f01241f720 Merge branch 'fix/xsren/TD-22212/taosQueryParam3.0' into enh/xsren/TD-22212/taosQueryParam3.0 2023-12-06 17:39:25 +08:00
Hongze Cheng 04e84ba152 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TS-4229 2023-12-06 16:43:00 +08:00
dmchen dc9408ed5f fix/TS-4251 2023-12-06 07:48:40 +00:00
shenglian zhou a2e7c78d27 refactor: rename variables 2023-12-06 09:39:38 +08:00
Haojun Liao 85c7dc76e9
Merge pull request #23948 from taosdata/mark/3.0
fix:offset out of range if poll offset is read from offsetRestored wh…
2023-12-06 09:12:38 +08:00
wangmm0220 cc51fd47d0 fix:offset out of range if poll offset is read from offsetRestored while wal is deleted 2023-12-05 19:09:29 +08:00
yihaoDeng 87c78919a9 add stream backend copy 2023-12-05 17:09:03 +08:00
Minglei Jin e9cc54d87b cos_cp/load: load checkpoint from json 2023-12-05 16:37:28 +08:00
wangjiaming0909 a6c876bbc1 fix: last/last_row crash with dropped column 2023-12-05 14:16:01 +08:00
Minglei Jin e0a14bd359 fix(tsdb/cache): remove debug log 2023-12-05 14:05:16 +08:00
Haojun Liao 9d451ab56e
Merge pull request #23934 from taosdata/fix/3_liaohj
refactor: do some internal refactor.
2023-12-05 12:58:18 +08:00
wangmm0220 79c8a1494d fix:disable snode unit test temporary 2023-12-05 11:26:12 +08:00
Haojun Liao cb8dfae0cc refactor: do some internal refactor. 2023-12-05 10:47:19 +08:00
slzhou e690076331 Merge branch '3.0' of github.com:taosdata/TDengine into szhou/tms-duration 2023-12-05 09:53:29 +08:00
Haojun Liao cc5890d56a
Merge pull request #23924 from taosdata/fix/3_liaohj
enh(tsdb): extract rows for the given tables in all stt files.
2023-12-04 19:21:19 +08:00
Hongze Cheng 26774f87c9
Merge pull request #23920 from taosdata/fix/TD-27619
fix: bg task memory waste
2023-12-04 18:10:30 +08:00
Haojun Liao 7145a7b8a1 fix(tsdb): calculate the rows in next fileset. 2023-12-04 18:03:32 +08:00
Haojun Liao 6d82900a32 fix(tsdb): fix error in block dist. 2023-12-04 18:00:41 +08:00
Hongze Cheng ac2d9ebc27 fix: memory leak 2023-12-04 16:48:43 +08:00
slzhou 8c290587b7 fix: typo error 2023-12-04 16:19:07 +08:00
Haojun Liao 24d9337632 enh(tsdb): extract rows for the given tables in all stt files. 2023-12-04 15:38:24 +08:00
slzhou a517271f39 Merge branch 'szhou/tms-duration-2' into szhou/tms-duration 2023-12-04 14:56:19 +08:00
slzhou 07e43fe56b fix: two intervals are intesected or not 2023-12-04 14:54:15 +08:00
Hongze Cheng cea4be6f3c fix: bg task memory waste 2023-12-04 13:40:28 +08:00
54liuyao 38900d672e Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-27419 2023-12-04 09:12:53 +08:00
Haojun Liao 48af3a9559
Merge pull request #23909 from taosdata/fix/TD-27455
fix:[TD-27644]heartbeat closed before snode close leading to snode is…
2023-12-04 09:10:00 +08:00
Haojun Liao 5467bba86b fix(tsdb): do some internal refactor. 2023-12-04 09:03:40 +08:00
Haojun Liao 7b0cf47fd6 fix(tsdb): do some internal refactor. 2023-12-04 08:58:10 +08:00
Haojun Liao 29a1c6d518 fix(tsdb): do some internal refactor. 2023-12-04 08:55:53 +08:00
wangmm0220 f06117dbc9 fix:[TD-27644]heartbeat closed before snode close leading to snode is hanged 2023-12-01 17:32:29 +08:00
Haojun Liao 3194ff41aa
Merge pull request #23900 from taosdata/fix/3_liaohj
fix(stream): merge fix from main to 3.0
2023-12-01 16:39:38 +08:00
shenglian zhou bff88e0639 fix: set duration order api 2023-12-01 16:27:36 +08:00
Haojun Liao 4d9211dcf1
Merge pull request #23899 from taosdata/fix/liaohj
other: merge the patch of stream from 3.0 branch.
2023-12-01 15:49:54 +08:00
slzhou 82629c2658 fix: notify when there are no memory pre fileset 2023-12-01 15:11:43 +08:00
Haojun Liao d7a19a9179 fix(stream): fix invalid decode hbmsg and memory leaks caused by invalid hb msg. 2023-12-01 14:45:18 +08:00
Haojun Liao 7e63943c4e
Merge pull request #23860 from taosdata/fix/TD-27455
fix:tmqVnodeSplit-column.py failed in arm64 because of rebalance after unsubscribe while split vnode
2023-12-01 14:18:58 +08:00
wade zhang 041add545c
Merge pull request #23844 from taosdata/FIX/TS-4306-3.0
enh: limit size of snap replication msg by data length
2023-12-01 14:06:36 +08:00
Haojun Liao 33ad05e2c8 fix(stream): fix syntax error. 2023-12-01 13:08:25 +08:00
slzhou 4d755562af fix: resolve conflicts 2023-12-01 13:02:14 +08:00
Haojun Liao 12f32f0ae7 fix(stream): fix syntax error. 2023-12-01 11:57:16 +08:00
Haojun Liao dba58f5e38 fix(stream): fix syntax error. 2023-12-01 11:54:13 +08:00
Haojun Liao eac3cbf564 fix(stream): merge 3.0 2023-12-01 11:12:57 +08:00
Haojun Liao 18273b8d1b fix(stream): ignore the in_progress code. 2023-12-01 10:20:45 +08:00
Haojun Liao 2972f5f420 fix(stream):return the error code if create stream failed. 2023-12-01 10:20:45 +08:00
Haojun Liao e008d0f440 fix(tsdb): fix invalid read. 2023-12-01 10:20:45 +08:00
Haojun Liao fbb604d1f5 fix(tsdb): fix bugs in suspend/resume. adjust code to conduct the test more easily. td-27579 2023-12-01 10:20:44 +08:00
Haojun Liao 1c9ff8030b fix(tsdb):clear the tsdb stt file reader flag when suspending the tsdbReader 2023-12-01 10:20:39 +08:00
Haojun Liao 86618ffd97 refactor: do some internal refactor. 2023-12-01 10:20:38 +08:00
Haojun Liao 31ae46925d fix(stream): fix invalid free. 2023-12-01 10:20:32 +08:00
Haojun Liao 17d4a22e47 fix(stream): add logs. 2023-12-01 10:19:54 +08:00
Haojun Liao b9c727575d fix(stream): reset the scheduler status. 2023-12-01 10:19:54 +08:00
Haojun Liao c937f42d88 fix(stream): fix deadlock. 2023-12-01 10:19:54 +08:00
Haojun Liao 8515a2f9b6 fix(stream): fix status check for stream-scan-history status when drop stream task msg is missing. 2023-12-01 10:19:18 +08:00
Haojun Liao 9d735e7f8b fix(stream): reduce the sink throughput. and fix the bug in status check. 2023-12-01 10:18:25 +08:00
Haojun Liao 523573844a fix(stream): fix syntax error. 2023-12-01 10:18:24 +08:00
Haojun Liao eb2f5ad074 fix(stream): fix error. 2023-12-01 10:18:22 +08:00
Haojun Liao 2b03d64850 fix(stream): drop the related fill-history task & set the task to be ready, if the drop fill-history task msg is missing due to the vnode restarting. 2023-12-01 10:18:15 +08:00
Haojun Liao 60fa7cccbf fix(stream): fix memory leak. 2023-12-01 10:17:14 +08:00
dapan1121 32030ca130
Merge pull request #23886 from taosdata/revert-23730-enh/TD-26976-3.0
Revert "enh:  grant and active codes"
2023-11-30 19:04:50 +08:00
Minglei Jin 49ecb1093f cos/put: new arg withcp to control resumable uploading 2023-11-30 17:20:21 +08:00
wangmm0220 4fc19a6343 fix:snode can only be created 1 2023-11-30 15:49:31 +08:00
Haojun Liao 999afd97e5 other: merge 3.0 2023-11-30 14:36:18 +08:00
54liuyao 271311a770 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-27419 2023-11-30 14:33:48 +08:00
Haojun Liao de82aba2e5 fix(stream): ignore the in_progress code. 2023-11-30 14:32:14 +08:00
Haojun Liao 71860bfb46 fix(stream):return the error code if create stream failed. 2023-11-30 13:57:00 +08:00
Hongze Cheng 0bfa86f188
Merge pull request #23864 from taosdata/feat/TD-27461-3.0
feat: backgroud task priority
2023-11-30 13:51:57 +08:00
Haojun Liao e9ead3bd28
Merge pull request #23341 from taosdata/enh/refactorBackend
Enh/refactor backend
2023-11-30 13:48:18 +08:00
Hongze Cheng 8ee4e2c1fe
Merge pull request #23862 from taosdata/fix/TD-27571
fix(tsdb/cache): flush dirty lru entries before del
2023-11-30 13:46:50 +08:00
Cary Xu f96044e09f
Revert "enh: grant and active codes" 2023-11-30 13:39:54 +08:00
slzhou 80ee5bfb3e fix: the initial value of first fileset start / end key 2023-11-30 11:08:18 +08:00
yihaoDeng 5c32680cd8 Merge branch '3.0' into enh/refactorBackend 2023-11-30 10:31:27 +08:00
WANG MINGMING 9b72d890c8
Merge branch '3.0' into fix/TD-27455 2023-11-30 09:37:23 +08:00
Hongze Cheng 547e673955 fix: coredump caused by typo 2023-11-30 09:20:18 +08:00
Haojun Liao 130614c938
Merge pull request #23831 from taosdata/fix/3_liaohj
enh(stream/tsdb): do multiple optimization and fix some bugs.
2023-11-30 09:20:15 +08:00
Minglei Jin ab2c7abca1 tsdb/cache-del: commit before get from rocks 2023-11-30 09:15:58 +08:00
Haojun Liao 28e17a61b9 fix(tsdb): fix invalid read. 2023-11-29 19:28:53 +08:00
Haojun Liao 66f25712fd fix(tsdb): fix bugs in suspend/resume. adjust code to conduct the test more easily. td-27579 2023-11-29 18:07:15 +08:00
Hongze Cheng 65ccc4aa48 feat: backgroud task priority 2023-11-29 18:01:31 +08:00
Minglei Jin 2c9c117529 fix(tsdb/cache): flush dirty lru entries before del 2023-11-29 17:37:40 +08:00
wangmm0220 4f168b4b2d fix:tmqVnodeSplit-column.py failed in arm64 because of rebalance after unsubscribe while split vnode 2023-11-29 17:22:43 +08:00
Benguang Zhao e66a1d6c9a enh: limit snap replication msg by size of block data in tsdbSnapReadTimeSeriesData 2023-11-29 16:01:11 +08:00
slzhou 1a3e280ea8 fix: use imem maxKey/minKey 2023-11-29 13:50:53 +08:00
slzhou 09e314c3ae fix: pStatus->pTableIter is set to null when getting mem table time range 2023-11-29 12:26:32 +08:00
54liuyao ae386c6e34 opt expired data 2023-11-29 11:16:56 +08:00
shenglian zhou 4691a7767a restore buildBlockFromBufferSequentially back 2023-11-29 10:29:40 +08:00
shenglian zhou 8a79df1113 enhance: prepare duration for next fileset 2023-11-29 10:15:43 +08:00
shenglian zhou 765b8384c0 Merge branch 'szhou/tms-duration' of github.com:taosdata/TDengine into szhou/tms-duration-2 2023-11-29 08:18:27 +08:00
slzhou b2fa6209d9 fix: descending traverse wrong endkey 2023-11-29 08:09:00 +08:00
Haojun Liao a5c8eaacaa fix(tsdb):clear the tsdb stt file reader flag when suspending the tsdbReader 2023-11-28 23:52:19 +08:00
kailixu b54f5b47ad fix: reserve enough buffer for user 2023-11-28 17:58:54 +08:00
shenglian zhou 5a75d74421 fix:mem table has data in the fileset duration 2023-11-28 15:58:33 +08:00
slzhou 4fb0f83f07 fix: core dump without resetTableListIndex 2023-11-28 14:50:50 +08:00
shenglian zhou 89cd2b6f17 enhance: tsdb output duration order 2023-11-28 14:39:08 +08:00
yihaoDeng 08eeee0794 Merge branch '3.0' into enh/refactorBackend 2023-11-28 11:52:49 +08:00
yihaoDeng 786383118c fix drop stream case 2023-11-28 10:39:17 +08:00
shenglian zhou cdb7eb462d Revert "Revert "fix: pass compilation""
This reverts commit 644dbad920.
2023-11-28 10:31:18 +08:00
shenglian zhou df4f7d6956 Revert "Revert "enhance: duration notification""
This reverts commit 3331e25aaf.
2023-11-28 10:31:18 +08:00
Haojun Liao 6bb30822c2 refactor: do some internal refactor. 2023-11-28 10:04:22 +08:00
Haojun Liao 9649db9003 fix(stream): fix invalid free. 2023-11-28 10:04:14 +08:00
Haojun Liao 8b406c0a19
Merge pull request #23830 from taosdata/fix/liaohj
fix(stream): fix invalid free.
2023-11-28 09:52:28 +08:00
shenglian zhou 3331e25aaf Revert "enhance: duration notification"
This reverts commit 9dd306e1ef.
2023-11-28 08:29:05 +08:00
shenglian zhou 644dbad920 Revert "fix: pass compilation"
This reverts commit 1a7ee3643e.
2023-11-28 08:28:49 +08:00
yihaoDeng a4d475b582 fix rsma 2023-11-27 20:27:53 +08:00
Haojun Liao b419650116 refactor: do some internal refactor. 2023-11-27 18:51:30 +08:00
Haojun Liao 91eea02258 fix(stream): fix invalid free. 2023-11-27 18:49:22 +08:00
wangmm0220 02575e0f28 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-27455 2023-11-27 18:15:53 +08:00
wangmm0220 d007d31c4d fix:process snode when drop stream 2023-11-27 18:15:09 +08:00
wangmm0220 06e2fbfb1c fix:conflicts from 3.0 2023-11-27 17:53:53 +08:00
wangmm0220 70c5d36103 opti:extract common logic from snode & vnode to tqCommon 2023-11-27 17:49:09 +08:00
yihaoDeng 630f4f9d80 Merge branch '3.0' into enh/refactorBackend 2023-11-27 13:53:33 +08:00
yihaoDeng e483570cd7 Merge branch '3.0' into enh/refactorBackend 2023-11-27 11:10:56 +08:00
Haojun Liao e85c6349a4
Merge pull request #23818 from taosdata/fix/TD-27455
fix:add O_CLOSEXEC for .running & process snode when drop stream
2023-11-27 11:01:24 +08:00
yihaoDeng e9bec34c44 Merge branch '3.0' into enh/refactorBackend 2023-11-27 10:01:12 +08:00
dapan1121 d3ac79aeb7
Merge pull request #23808 from taosdata/fix/PI-23-3.0
fix: add check in mnode/vnode when alter table col/tag
2023-11-26 17:31:38 +08:00
Haojun Liao 9194c0c0cd fix(stream): add logs. 2023-11-25 00:56:18 +08:00
Haojun Liao 30c776cc0a fix(stream): reset the scheduler status. 2023-11-25 00:54:32 +08:00
Haojun Liao 788194c748 fix(stream): fix deadlock. 2023-11-24 22:53:55 +08:00
wangmm0220 ff0828fcc2 fix:add O_CLOSEXEC for .running & process snode when drop stream 2023-11-24 18:30:58 +08:00
wangmm0220 f8477204e1 fix:[TD-27475] send msg to snode wher drop stream to avoid pVgObj is NULL 2023-11-24 18:00:19 +08:00
yihaoDeng 6416a6153a Merge branch '3.0' into enh/refactorBackend 2023-11-24 16:29:09 +08:00
Haojun Liao f28cf740b0 fix(stream): fix status check for stream-scan-history status when drop stream task msg is missing. 2023-11-24 14:44:00 +08:00
Haojun Liao d51baab9c2
Merge pull request #23785 from taosdata/fix/liaohj
other: merge fix from 3.0 to main.
2023-11-24 14:43:02 +08:00
Haojun Liao d5e123db78
Merge pull request #23791 from taosdata/fix/3_liaohj
refactor: do some internal refactor.
2023-11-24 14:42:43 +08:00
yihaoDeng 23ae62d268 Merge branch '3.0' into enh/refactorBackend 2023-11-24 11:10:39 +08:00
Hongze Cheng 24932d2d41
Merge pull request #23799 from taosdata/fix/TD-27423
fix(tsdb/skip-row): remove s3 size condition
2023-11-24 10:48:03 +08:00
dapan1121 26fe05bb30
Merge pull request #23805 from taosdata/fix/slowReport
fix: slow down report when error occurs
2023-11-24 09:18:27 +08:00
Minglei Jin b1841ee97b tsdb/reader-writer: fix pgno iteration with cache setting 2023-11-24 09:06:43 +08:00
wade zhang 95b5fbbb1a
Merge pull request #23801 from taosdata/enh/PI-25-3.0
enh: trigger vnode commit at exit if meta changed
2023-11-24 08:59:24 +08:00
kailixu ac47d3ea7d enh: more check for column bytes 2023-11-24 06:51:57 +08:00
kailixu c4f1babdcf enh: naming optimize 2023-11-24 06:42:30 +08:00
kailixu dc977ca200 fix: stash drop column 2023-11-23 23:09:49 +08:00
kailixu 9ce28483a0 fix: add check in mnode/vnode when alter table col/tag 2023-11-23 21:22:41 +08:00
wangmm0220 d249892b2a fix:conflicts from 3.0 2023-11-23 20:29:46 +08:00
wangmm0220 55a4181e5e fix:add O_CLOEXEC for .running lock file to prevent inherited by child processes(rsync) 2023-11-23 20:16:54 +08:00
dmchen 522c848489 clean code 2023-11-23 11:53:14 +00:00
dmchen 75810bbdbb audit 2023-11-23 11:26:14 +00:00
dmchen 682ad9dd96 msg epset 2023-11-23 10:16:18 +00:00
Haojun Liao 2e922fefc6 fix(stream): reduce the sink throughput. and fix the bug in status check. 2023-11-23 18:04:09 +08:00
wade zhang 0e8130b195
Merge pull request #23787 from taosdata/FIX/TD-27473-3.0
enh: skip request to restore a vnode of single replica in failed mode
2023-11-23 17:54:08 +08:00
yihaoDeng 6129ebc294 Merge branch '3.0' into enh/refactorBackend 2023-11-23 17:41:37 +08:00
Minglei Jin 3c3763a774 tsdb/read-by-column: fix last column index 2023-11-23 17:30:49 +08:00
Minglei Jin 9334fbcb06 tsdb/read-by-column: zero hint size when i's nonzero 2023-11-23 17:14:26 +08:00
dapan1121 699c14e781 fix: slow down report when error occurs 2023-11-23 16:51:07 +08:00
kailixu 14f163a795 Merge branch '3.0' into fix/PI-23-3.0 2023-11-23 16:49:19 +08:00
Benguang Zhao f7f9546d1e enh: trigger vnode commit at exit if meta changed, for dbs of single replica 2023-11-23 16:32:57 +08:00
Minglei Jin 3034ca2875 tsdb/open-file: log not exist file path 2023-11-23 16:25:39 +08:00
Haojun Liao 810eb8e0e3 fix(stream): fix syntax error. 2023-11-23 16:25:12 +08:00
Minglei Jin f076bfdfca fix(tsdb/skip-row): remove s3 size condition 2023-11-23 16:19:26 +08:00
Minglei Jin 460638d294 enh(tsdb/open-file): new s3 failed to connect error code 2023-11-23 16:09:02 +08:00
Haojun Liao de4cc2009d fix(stream): fix error. 2023-11-23 16:04:06 +08:00
dmchen cf4b3ec49f clean code 2023-11-23 08:02:37 +00:00
Minglei Jin c6721be112 fix stt read compilation 2023-11-23 15:49:42 +08:00
Haojun Liao 0a90c33b4d fix(stream): drop the related fill-history task & set the task to be ready, if the drop fill-history task msg is missing due to the vnode restarting. 2023-11-23 15:49:07 +08:00
Minglei Jin 08e5525012 tsdb/read-file-page: new arg for hint size 2023-11-23 15:37:26 +08:00
dmchen bbbdec5db7 update progress 2023-11-23 07:36:33 +00:00
Haojun Liao 421a154d2a
Merge pull request #23768 from taosdata/fix/fixChkpRespMemleak
fix mem leak
2023-11-23 15:29:30 +08:00
Minglei Jin 023821a80b enh(tsdb/read-by-column): calc hint size for preloading 2023-11-23 15:16:36 +08:00
wade zhang 45bbf89a19
Merge pull request #23786 from taosdata/FIX/TD-27456-3.0
enh: not allow to acquire a vnode in failed mode from vmAcquireVnode
2023-11-23 14:20:23 +08:00
kailixu 6e3a62803c fix: add check in mnd when alter table col/tag 2023-11-23 14:08:31 +08:00
yihaoDeng 6a0501967d Merge branch '3.0' into fix/fixChkpRespMemleak 2023-11-23 14:06:04 +08:00
Benguang Zhao d35bf6186f enh: skip restoring a vnode of single replica in failed mode from vmProcessCreateVnodeReq 2023-11-23 13:16:37 +08:00
dmchen 4e1a355052 mnode for compact 2023-11-23 04:43:28 +00:00
Haojun Liao 0654b9299a fix(stream): fix memory leak. 2023-11-23 11:44:25 +08:00
Haojun Liao 8ad071e69a other: merge main. 2023-11-23 10:25:51 +08:00
Benguang Zhao 8d70d023b9 enh: adjust a logging msg in vmRestoreVnodeInThread 2023-11-23 10:12:58 +08:00
Benguang Zhao 917246e236 fix: allow to drop a vnode in failed mode 2023-11-23 10:09:41 +08:00
Benguang Zhao 0a4fac7704 enh: not allow to acquire a vnode in failed mode from vmAcquireVnode 2023-11-23 09:48:42 +08:00
Haojun Liao 630b259d7f fix(stream): fix error in stream. 2023-11-23 09:41:07 +08:00
Haojun Liao ff9c0c0df4 fix(stream): fix error in check trans conflict. 2023-11-23 09:38:11 +08:00
Haojun Liao f3355000a3 refactor: do some internal refactor. 2023-11-23 09:34:20 +08:00
Haojun Liao 6fca8c170c enh(stream): add sink_quota/scan-history-idle-duration column for stream tasks. 2023-11-23 09:33:44 +08:00
Haojun Liao 8204db13aa enh(stream): add sink_quota column for stream tasks. 2023-11-23 09:33:42 +08:00
Haojun Liao 3edb25a49e fix(stream): add check for assert. 2023-11-23 09:33:40 +08:00
Haojun Liao 41aa175099 fix(stream): add return. 2023-11-23 09:33:36 +08:00
Haojun Liao 7b1f586d4f
Merge pull request #23762 from taosdata/fix/3_liaohj
fix(stream): fix error in generating token in bucket.
2023-11-23 09:32:14 +08:00
wangmm0220 15d75a0497 fix:spelling errors 2023-11-23 09:17:54 +08:00
Hongze Cheng 44866b7b12 fix: possible race condition coredump 2023-11-22 20:35:18 +08:00
wangmm0220 6dc91d12d5 fix:add msg type 2023-11-22 20:23:58 +08:00
wangmm0220 cfab06239b Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-26060 2023-11-22 19:37:44 +08:00
Haojun Liao a2a68ce256 fix(stream): fix memory leak. 2023-11-22 18:28:38 +08:00
Haojun Liao d88b32e93d fix(stream): fix memory leak. 2023-11-22 18:18:43 +08:00
wangmm0220 9a09e318ca opti:common message process logic in vnode and snode 2023-11-22 16:53:33 +08:00
Haojun Liao 9988720a1e fix(stream): fix the invalid check of trans conflict. 2023-11-22 16:39:29 +08:00
Haojun Liao ebb1993900 fix(stream): fix error in check trans conflict. 2023-11-22 16:20:59 +08:00
Haojun Liao 91e7000433 Merge branch 'enh/refactorBackend' of github.com:taosdata/tdengine into enh/refactorBackend 2023-11-22 15:29:31 +08:00
Haojun Liao c96f7cf182 fix(stream): scheduler the checkpoint generating. 2023-11-22 15:17:21 +08:00
slzhou 1a7ee3643e fix: pass compilation 2023-11-22 14:09:09 +08:00
shenglian zhou 9dd306e1ef enhance: duration notification 2023-11-22 13:56:16 +08:00
Haojun Liao 9a4222e189
Merge pull request #23767 from taosdata/fix/TD-27414
fix:test case for windows
2023-11-22 11:02:08 +08:00
dapan1121 1b6a3c2069
Merge pull request #23771 from taosdata/enh/xsren/TD-27022/windows
Enh/xsren/td 27022/windows
2023-11-22 10:58:39 +08:00
yihaoDeng 0eba15b4a4 Merge branch '3.0' into enh/refactorBackend 2023-11-22 10:57:41 +08:00
yihaoDeng 0c0d9cdd2c Merge branch '3.0' into fix/fixChkpRespMemleak 2023-11-22 10:42:30 +08:00
Haojun Liao b712f5e7dc refactor: do some internal refactor. 2023-11-22 08:41:17 +08:00
Haojun Liao 3907268f40 fix(stream): register checkpoint trans. 2023-11-22 08:40:40 +08:00
Haojun Liao d355b78884 fix(stream): fix invalid read. 2023-11-22 08:40:28 +08:00
Haojun Liao 7a0dfdb0cc fix(tsdb): fix invalid read. 2023-11-22 08:40:10 +08:00
Haojun Liao 0a2fb5fe9a refactor: do some internal refactor. 2023-11-22 08:38:39 +08:00
Haojun Liao 43e6722fd8 refactor: refactor the stream trans conflict check 2023-11-22 08:38:32 +08:00
yihaoDeng 947775155b fix transfer crash 2023-11-21 20:51:11 +08:00
Haojun Liao 785d91c17b
Merge pull request #23661 from taosdata/feat/TD-26060
fix: errors in snode in conditions like fill history/checkpoint/restart
2023-11-21 19:02:38 +08:00
facetosea 845493294d osfile 2023-11-21 16:31:31 +08:00
wade zhang c4a1398050
Merge pull request #23761 from taosdata/enh/TD-27426-3.0
enh: create replica of the same vgroup one by one for alter db replica
2023-11-21 15:58:58 +08:00
Haojun Liao 8c4558c5c2
Merge pull request #23763 from taosdata/fix/pause_stream
fix(stream): fix error in generating token in bucket.
2023-11-21 15:46:25 +08:00
yihaoDeng a92602b667 fix mem leak 2023-11-21 15:23:28 +08:00
wangmm0220 b3ca673fe9 fix:test case for windows 2023-11-21 14:35:41 +08:00
wade zhang 4f5359f684
Merge pull request #23756 from taosdata/FIX/TD-27416-3.0
TD-27416
2023-11-21 12:00:18 +08:00
Haojun Liao 1077378d51 other: merge main. 2023-11-21 10:14:30 +08:00
Haojun Liao 4b3495cd4b refactor: do some internal refactor. 2023-11-21 10:11:02 +08:00
Haojun Liao 2b099b11dd refactor: do some internal refactor. 2023-11-21 10:09:59 +08:00
dapan1121 415d162358
Merge pull request #23730 from taosdata/enh/TD-26976-3.0
enh:  grant and active codes
2023-11-21 10:04:20 +08:00
Benguang Zhao 1343c3b274 enh: create replica of the same vgroup one by one for alter db replica 2023-11-21 09:17:34 +08:00
yihaoDeng 939aa7a3cc Merge branch '3.0' into enh/refactorBackend 2023-11-20 17:33:30 +08:00
dmchen 2d7c4890da TD-27416 2023-11-20 16:50:43 +08:00
dmchen 376f78788e TD-27416 2023-11-20 08:43:12 +00:00
Hongze Cheng 5f849320d1
Merge pull request #23751 from taosdata/fix/TD-27405
fix(tsdb/merge): clear file set before return 0
2023-11-20 16:10:38 +08:00
Minglei Jin d12bc41236 tsdb/merge: clear file set before return 0 2023-11-20 14:30:25 +08:00
Haojun Liao ec919415b8 enh(stream): add sink_quota/scan-history-idle-duration column for stream tasks. 2023-11-20 12:03:23 +08:00
Haojun Liao 56edf1b52c enh(stream): add sink_quota column for stream tasks. 2023-11-20 12:03:23 +08:00
Haojun Liao 7cfa45e95e enh(stream): add sink_quota/scan-history-idle-duration column for stream tasks. 2023-11-20 11:59:49 +08:00
Haojun Liao 3912e49309 enh(stream): add sink_quota column for stream tasks. 2023-11-20 11:46:11 +08:00
yihaoDeng 734bf07317 Merge branch '3.0' into enh/refactorBackend 2023-11-20 10:48:49 +08:00
Haojun Liao 1b4f187ded
Merge pull request #23743 from taosdata/fix/pause_stream
fix(stream): remove invalid assert
2023-11-17 23:48:47 +08:00
Haojun Liao 709502ba9e fix(stream): record the check downstream failure. 2023-11-17 22:05:36 +08:00
wangmm0220 3241f9bda5 fix:memory leak 2023-11-17 17:26:52 +08:00
yihaoDeng 20de2f3cb5 Merge branch '3.0' into enh/refactorBackend 2023-11-17 17:15:14 +08:00
kailixu 3b62d555ab enh: code optimization for active code 2023-11-17 15:50:08 +08:00
Haojun Liao 0fac706fa6
Merge pull request #23732 from taosdata/fix/pause_stream
fix(stream): add check for assert.
2023-11-17 11:33:19 +08:00
Haojun Liao 2a1a7c82e2 Merge branch '3.0' into fix/3_liaohj 2023-11-17 10:17:14 +08:00
wade zhang 440a750d64
Merge pull request #23711 from taosdata/fix/TD-27336-main
fix/TD-27336
2023-11-17 09:00:06 +08:00
wade zhang 43a8a76719
Merge pull request #23713 from taosdata/FIX/TD-27336-3.0
fix/TD-27336
2023-11-17 08:59:50 +08:00
kailixu 14a2be8ed2 enh: adjust test case 2023-11-17 08:41:47 +08:00
Haojun Liao 8fe79685a2 fix(stream): add check for assert. 2023-11-16 23:45:38 +08:00
Haojun Liao c070e79d36 fix(stream): add check for assert. 2023-11-16 23:18:07 +08:00
wangmm0220 a5de1b032b fix:snode upport stream task 2023-11-16 21:19:58 +08:00
kailixu 63a17295a7 enh: support spread active codes 2023-11-16 20:09:56 +08:00
kailixu 9e12ad408e enh: support spread active codes 2023-11-16 20:01:54 +08:00
kailixu 50126e4425 enh: support spread active codes 2023-11-16 19:52:05 +08:00
Minglei Jin 65a27dd9c8 fix(tsdb/cos-merge): skip stt merging & not block commit 2023-11-16 18:58:58 +08:00
yihaoDeng 7905f22aae fix stream backend convert 2023-11-16 17:35:17 +08:00
yihaoDeng 8023f2bebd fix stream backend convert 2023-11-16 15:30:01 +08:00
dmchen 8dbb7ff2d4 copy sql before destroy 2023-11-16 10:28:11 +08:00
dmchen 1e2c9b03a1 copy sql before destroy 2023-11-16 02:22:01 +00:00
dm chen b18aa087da Revert "create steam"
This reverts commit cd6df48df5.
2023-11-16 10:09:46 +08:00
dmchen cd6df48df5 create steam 2023-11-16 01:49:31 +00:00
wade zhang cd8f542143
Merge pull request #23696 from taosdata/fix/TD-27311
TD-27311
2023-11-16 08:59:27 +08:00
Haojun Liao f45c3277c3
Merge pull request #23710 from taosdata/fix/3_liaohj
fix(stream): check the status before pause
2023-11-15 22:06:56 +08:00
dapan1121 dc71252096 Merge remote-tracking branch 'origin/3.0' into merge/32m 2023-11-15 19:43:19 +08:00
wade zhang a405a981be
Merge pull request #23500 from taosdata/fix/transaction-retry-main
fix/transaction-retry
2023-11-15 19:19:02 +08:00
wade zhang 297fea94df
Merge pull request #23700 from taosdata/FIX/transaction-retry-3.0
Fix/transaction retry 3.0
2023-11-15 19:18:41 +08:00
wangmm0220 a16189d2a9 fix:confilct from 3.0 2023-11-15 19:14:27 +08:00
dmchen 9e37fd8fc6 create stream 2023-11-15 18:48:41 +08:00
dmchen 82498a9b4e create stream 2023-11-15 10:44:53 +00:00
dmchen 54d235dad0 fix/TD-27336 2023-11-15 18:21:55 +08:00
wangmm0220 45782f278b fix:send update msg to vnodes if snode restart 2023-11-15 18:13:35 +08:00
dmchen 51abd736e0 fix/TD-27336 2023-11-15 10:13:00 +00:00
Haojun Liao 1b89d4f404 fix(stream): add return. 2023-11-15 18:06:16 +08:00
Haojun Liao 6529e56020 fix(stream): check the status before pause 2023-11-15 18:06:16 +08:00
Minglei Jin 3b7e92f7a2 enh(mnode/split): not support split with s3 2023-11-15 17:34:26 +08:00
Haojun Liao 974d0bbb48 fix(stream): add return. 2023-11-15 16:53:51 +08:00
Haojun Liao 115246c877 fix(stream): check the status before pause 2023-11-15 16:46:09 +08:00
kailixu 188a4ce6a5 enh: the latest active code take effect 2023-11-15 16:33:05 +08:00
Haojun Liao 9d2256ec9c refactor: do some internal refactor. 2023-11-15 16:03:23 +08:00
Haojun Liao 1b1ad3068d fix(stream): check node status before start pause trans. 2023-11-15 16:02:56 +08:00
Haojun Liao f299bdb387 fix(stream): register checkpoint trans. 2023-11-15 14:29:01 +08:00
Haojun Liao 2ca663ce66
Merge pull request #23593 from taosdata/feat/TD-21898
stream event window
2023-11-15 14:18:07 +08:00
dapan1121 9457cf0b46
Merge pull request #23702 from taosdata/merge/m23
merge: from main to 3.0
2023-11-15 13:54:21 +08:00
Hongze Cheng 082ddbddf6 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TS-4229 2023-11-15 13:41:28 +08:00
Hongze Cheng 8050d73202 first version 2023-11-15 13:41:00 +08:00
Haojun Liao 86f2673849
Merge pull request #23703 from taosdata/fix/drop_stream
fix(tsdb): fix invalid read.
2023-11-15 13:35:46 +08:00
wangmm0220 8a9ded764d fix:add finish history msg to snode 2023-11-15 11:50:14 +08:00
dapan1121 dfb3feb5dc Merge remote-tracking branch 'origin/main' into merge/m23 2023-11-15 11:39:18 +08:00
Haojun Liao f1b1b8911b fix(tsdb): fix invalid read. 2023-11-15 11:26:43 +08:00
wade zhang bef2991211
Merge pull request #23691 from taosdata/fix/TD-27303-main
TD-27303
2023-11-15 11:23:00 +08:00
wade zhang d800d59bf0
Merge pull request #23693 from taosdata/FIX/TD-27303-3.0
TD-27303
2023-11-15 11:22:47 +08:00
Haojun Liao 51897a679f fix(stream): fix invalid read. 2023-11-15 11:21:31 +08:00
Haojun Liao 62c7877176 fix(tsdb): fix invalid read. 2023-11-15 11:13:06 +08:00
Hongze Cheng ed6be3b8c5
Merge pull request #23695 from taosdata/fix/TD-27313
fix(cos/list): free object array in exception
2023-11-15 11:09:38 +08:00
dmchen 46b5f8631c partial retry 2023-11-15 10:26:01 +08:00
dmchen 85729babf8 retry in outside 2023-11-15 10:25:29 +08:00
dmchen f93d569552 redo 2023-11-15 10:24:59 +08:00
dmchen 20e5702183 revert 2023-11-15 10:24:15 +08:00
dmchen 481c5981d6 fix/transaction-retry 2023-11-15 10:23:46 +08:00
dmchen 1e50b11374 TD-27311 2023-11-15 01:31:51 +00:00
Haojun Liao 3294395617
Merge pull request #23686 from taosdata/fix/3_liaohj
fix(stream): send msg by using epset extract from mnode
2023-11-14 22:11:46 +08:00
Minglei Jin 7f65e92b81 tsdb/retention: not allow migrate backward 2023-11-14 19:27:18 +08:00
dmchen d8277d5505 TD-27303 2023-11-14 19:18:06 +08:00
dmchen 056993b519 TD-27303 2023-11-14 11:10:22 +00:00
yihaoDeng f4d6cf7c74 Merge branch '3.0' into enh/refactorBackend 2023-11-14 18:12:48 +08:00
Haojun Liao 61552e30d9 fix(stream): send msg by using epset extract from mnode, instead of cached epset in streamObj. 2023-11-14 17:13:40 +08:00
Haojun Liao b390ee859c fix(stream): send msg by using epset extract from mnode, instead of cached epset in streamObj. 2023-11-14 17:04:54 +08:00
Hongze Cheng f7e63ad1ca
Merge pull request #23680 from taosdata/fix/TD-27273
fix(tsdb/merge): skip merge if data file cannot be written
2023-11-14 16:08:12 +08:00
Haojun Liao 18333d7347 fix(stream): send msg by using epset extract from mnode, instead of cached epset in streamObj. 2023-11-14 15:48:00 +08:00
Minglei Jin cb3fe4d6de fix(tsdb/merge): skip merge if data file cannot be written 2023-11-14 14:15:02 +08:00
Haojun Liao 436c7871a3
Merge pull request #23673 from taosdata/fix/3_liaohj
fix(stream): fix deadlock and do some internal refactor.
2023-11-14 11:26:54 +08:00
wade zhang 9167a07e89
Merge pull request #23657 from taosdata/enh/TD-27210
enh: add humanStr convert func
2023-11-14 10:04:54 +08:00
wade zhang 2cbce6c442
Merge pull request #23578 from taosdata/FIX/TD-27171-3.0
enh: improve user error of redistribute vgroup
2023-11-14 10:04:33 +08:00
Haojun Liao 9f6b0d25c5 refactor: do some internal refactor. 2023-11-13 23:59:05 +08:00
Haojun Liao f99e22b91f fix(stream): fix deadlock. 2023-11-13 23:56:42 +08:00
Haojun Liao 9a852b723d refactor: refactor the stream trans conflict check 2023-11-13 23:26:16 +08:00
Haojun Liao cfa7c19dd5
Merge pull request #23667 from taosdata/fix/liaohj
fix(stream): fix deadlock.
2023-11-13 22:15:06 +08:00
wangmm0220 56381dea90 fix:conflict from 3.0 2023-11-13 18:42:59 +08:00
wangmm0220 1cbe568be7 fix:add logic when snode restart 2023-11-13 18:39:42 +08:00
Haojun Liao aad8a2eb87 fix(stream): fix deadlock. 2023-11-13 18:20:30 +08:00
Minglei Jin 6e583de80c fix(config/block-size): remove invalid range (-1, 1024) 2023-11-13 17:46:09 +08:00
Minglei Jin acd816a364 fix(tsdb/retention): move fset destruction out of for loop 2023-11-13 16:59:26 +08:00
Shungang Li a96e21aabd enh: refactor dynamic config 2023-11-13 16:28:42 +08:00
Shungang Li 06c4aa431f enh: add humanStr convert func 2023-11-13 16:28:38 +08:00
liuyao 37c102a115 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898 2023-11-13 16:06:17 +08:00
yihaoDeng de7f2a2d1a Merge branch '3.0' into enh/refactorBackend 2023-11-13 11:11:28 +08:00
wade zhang 05d7274d98
Merge pull request #23639 from taosdata/fix/TS-4249-main
TS-4249
2023-11-13 10:03:18 +08:00
wade zhang 8e5de50050
Merge pull request #23641 from taosdata/FIX/TS-4249-3.0
TS-4249
2023-11-13 10:03:03 +08:00
wangmm0220 6a44bd54ac fix:load task in snode 2023-11-13 09:12:15 +08:00
dapan1121 a51df86e43
Merge pull request #23636 from taosdata/enh/TD-25601-3.0
enh: support delete msg for rsma
2023-11-13 08:47:42 +08:00
Haojun Liao 9ee9174608 other: merge 3.0 2023-11-11 15:48:31 +08:00
Haojun Liao 3bacd7516e fix(stream): extact stream nodes list if not initialized. 2023-11-11 15:40:35 +08:00
Haojun Liao 15430f4d50 refactor: 2023-11-11 15:40:33 +08:00
Haojun Liao 5fcef5bd89 refactor:add some logs. 2023-11-11 15:40:32 +08:00