Commit Graph

11231 Commits

Author SHA1 Message Date
Haojun Liao 1f792f09c4 fix(stream): remove invalid code. 2023-08-14 10:39:47 +08:00
slzhou 646e52ae7f Merge branch '3.0' of github.com:taosdata/TDengine into szhou/tag-scan-opt 2023-08-14 10:19:36 +08:00
yihaoDeng 246efd83c6 refactor chpt transfer 2023-08-14 02:03:41 +00:00
wangmm0220 dd0bc0e050 fix:heap use after free 2023-08-14 09:42:55 +08:00
wade zhang 1b51f53c15
Merge pull request #22407 from taosdata/fix/TD-25671
fix: infinite tomb data sync loop
2023-08-14 09:25:47 +08:00
Haojun Liao 01ac99a651
Merge pull request #22413 from taosdata/fix/liaohj
refactor: refactor the transfer state procedure.
2023-08-13 23:28:18 +08:00
Haojun Liao 98f40325e9 fix(stream): fix memory leak. 2023-08-13 20:02:15 +08:00
Haojun Liao 0d38f389ab refactor: do some internal refactor. 2023-08-13 16:25:35 +08:00
Haojun Liao ce721a0146 fix(stream): fix error. 2023-08-12 16:32:05 +08:00
wangmm0220 efb472d54a fix:heap use after free 2023-08-12 10:15:40 +08:00
Haojun Liao f24b22000f enh(stream): stop stream task before redistribute vnodes. 2023-08-11 23:46:53 +08:00
wangmm0220 767563752a opti:wal logic 2023-08-11 20:21:13 +08:00
wangmm0220 deda4b9eed opti:wal logic 2023-08-11 19:26:17 +08:00
Hongze Cheng 847bc0da08 fix: infinite tomb data sync loop 2023-08-11 18:10:16 +08:00
slzhou 47d2f9ad6d fix: first run without tag cond 2023-08-11 17:52:52 +08:00
wangmm0220 7b22d37f12 fix:lost data when consumer db 2023-08-11 17:25:43 +08:00
slzhou 42a4f364c1 Merge branch 'szhou/tag-scan-opt' of github.com:taosdata/TDengine into szhou/tag-scan-opt 2023-08-11 16:48:49 +08:00
Minglei Jin 84452c8dee cos: use static sdk 2023-08-11 16:36:53 +08:00
Haojun Liao 87c7e4c0fe fix(stream): not launch stream tasks when restarting the stream tasks. 2023-08-11 16:19:46 +08:00
yihaoDeng 6e6eb05d5c Merge branch 'enh/triggerCheckPoint2' into enh/chkpTransfer 2023-08-11 08:03:02 +00:00
Haojun Liao e984affe90 fix(stream): disable the stream in the followers nodes. 2023-08-11 15:58:15 +08:00
dapan1121 8987553d9c fix: remove ins_modules 2023-08-11 15:56:38 +08:00
wangmm0220 25e32bbf7c fix:offset error in tmq & add test cases 2023-08-11 14:25:12 +08:00
Haojun Liao ba7825141f fix(stream): enable the checkpoint . 2023-08-11 13:33:14 +08:00
Haojun Liao 9322f80343 fix(stream): register the rsp callback. 2023-08-11 13:31:50 +08:00
kailixu 64a1636dd2 chore: trigger CI 2023-08-11 12:54:45 +08:00
kailixu 1d089fe085 fix: dup read lock on windows 2023-08-11 11:34:57 +08:00
yihaoDeng 99be2836b2 Merge branch 'enh/triggerCheckPoint2' into enh/chkpTransfer 2023-08-11 02:50:12 +00:00
Haojun Liao 1e636a2ef9 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-08-11 10:37:40 +08:00
Haojun Liao 6a9826cf6e enh(stream): update the stream task epset. 2023-08-11 10:37:25 +08:00
yihaoDeng ba4ce9db02 Merge branch 'enh/triggerCheckPoint2' into enh/chkpTransfer 2023-08-11 02:15:07 +00:00
Haojun Liao d9bc881fb3
Merge pull request #22384 from taosdata/mark/tmq
fix:offset error in tmq & add test cases
2023-08-11 10:00:08 +08:00
Minglei Jin 4513acfee9 cos: use static libs for mxml, apr, apu, curl 2023-08-11 09:54:54 +08:00
yihaoDeng 32983a11e1 add log and fix crash 2023-08-10 20:34:00 +08:00
shenglian zhou b2e615d4e7 enhance: tag scan cursor based block 2023-08-10 17:30:01 +08:00
wangmm0220 78fd70202c fix:offset error in tmq & add test cases 2023-08-10 16:18:53 +08:00
Minglei Jin c04ada3573 cos: link with static libs 2023-08-10 15:28:22 +08:00
Minglei Jin cd63e81450 cos: separate building phase for apr & apr-util 2023-08-10 14:18:12 +08:00
Haojun Liao a713b37f35 fix(stream): add more logs. 2023-08-10 14:07:29 +08:00
Haojun Liao 161fd6902a fix(stream): fix the acquire task. 2023-08-10 12:45:20 +08:00
Minglei Jin 33967b243b
Merge branch '3.0' into fix/TD-25010-3 2023-08-10 12:14:01 +08:00
Haojun Liao f633c7fe2b other: merge 3.0. 2023-08-10 11:56:30 +08:00
Haojun Liao b658608541
Merge pull request #22388 from taosdata/fix/3_liaohj
other: merge main.
2023-08-10 11:43:02 +08:00
yihaoDeng ef247cdb1d support reopen stream state 2023-08-10 01:49:26 +00:00
Haojun Liao c10b323188 other: merge main. 2023-08-10 09:26:33 +08:00
Haojun Liao 4393375e47 fix(stream): set correct task id. 2023-08-10 02:16:04 +08:00
Haojun Liao 0757e88ced fix(stream): set the correct hash keys. 2023-08-09 23:52:14 +08:00
Haojun Liao 7e3f1d92fc Merge branch 'main' into fix/liaohj 2023-08-09 19:43:24 +08:00
wangmm0220 1f5532fd67 fix:offset error in tmq & add test cases 2023-08-09 19:41:22 +08:00
Haojun Liao d34b9af054 enh(stream): create update stream trans. 2023-08-09 19:38:13 +08:00
Hui Li 10f5d223bb
Merge pull request #22370 from taosdata/FIX/TD-25605-3.0
enh: check if disk space sufficient at primary dir with tfs
2023-08-09 18:32:07 +08:00
Minglei Jin a853d9d40c mxml: use ~/local/include 2023-08-09 18:21:55 +08:00
Haojun Liao 1e8579e8c5 fix(stream): fix other cases. 2023-08-09 16:04:48 +08:00
Haojun Liao 4c92997328 fix(stream): use streamId&taskId to identify the stream task in the stream meta. 2023-08-09 15:52:23 +08:00
Haojun Liao 1d649bccb6
Merge pull request #22372 from taosdata/fix/TD-25626
fix taosd cannot quit
2023-08-09 15:46:14 +08:00
Hui Li 584427dd44
Merge pull request #22374 from taosdata/szhou/restore-minglei-main
fix: restore stt/block merge back
2023-08-09 14:40:03 +08:00
Minglei Jin b08d5b4d42 mxml: add include dir to vnode 2023-08-09 14:02:10 +08:00
Minglei Jin 7e2859ed43 apr: use tarball to avoid ./buildconf 2023-08-09 13:03:37 +08:00
Haojun Liao 3cdb16352c fix(stream): avoid launching check downstream when failed to add stream task into stream meta. 2023-08-09 12:03:14 +08:00
Minglei Jin 2e0519b960 apr: fix apr & apr-util project names 2023-08-09 11:01:42 +08:00
Minglei Jin 7114ad63e5 apr:makefile for apr & apr-util 2023-08-09 10:54:17 +08:00
Minglei Jin 4d1155a5cf curl: makefile for curl 2023-08-09 10:43:27 +08:00
shenglian zhou e9e06d1eae fix: restore stt block/data block merge back 2023-08-09 10:36:54 +08:00
shenglian zhou 2b1478f46e Revert "fix(tsdb/read2): reset stt reader when suspended"
This reverts commit 079d7ff69e.
2023-08-09 10:36:26 +08:00
Haojun Liao f3c5f20ee2 enh(stream): add timer to check the node changing. 2023-08-09 10:25:16 +08:00
Minglei Jin 16e015253b s3/mxml: remove os external dependency 2023-08-09 10:10:25 +08:00
yihaoDeng fb7dec00e1 support reopen stream state 2023-08-09 01:50:49 +00:00
Haojun Liao bb39d94f9c
Merge pull request #22357 from taosdata/fix/liaohj
other: merge fix from 3.0 to main.
2023-08-09 09:05:54 +08:00
yihaoDeng cfb5247e30 change var name 2023-08-09 00:46:18 +00:00
Hui Li 8d7e8f5734
Merge pull request #22336 from taosdata/fix/TD-25445
fix/TD-25445: alter ttlChangeOnWrite note info
2023-08-08 18:19:02 +08:00
Benguang Zhao e4d16e594c enh: check if disk space sufficient at primary dir with tfs 2023-08-08 17:53:11 +08:00
wangmm0220 d322cfce50 fix:add hashIteratorCancel for hash iterator 2023-08-08 16:28:51 +08:00
Haojun Liao 17c6143471 fix(stream): check more status when handling the state transfer. 2023-08-08 15:08:11 +08:00
Hongze Cheng c8f106106a fix: update assert issue 2023-08-08 14:49:05 +08:00
Hui Li 4f03ae2c65
Merge pull request #22335 from taosdata/temp/open_case
open split vgroup case
2023-08-08 11:49:28 +08:00
Haojun Liao a0dcec5890 enh(stream): add function handler for hb message. 2023-08-08 11:23:19 +08:00
dapan1121 44717ddbea
Merge pull request #22288 from taosdata/fix/3.0/TD-25499
fix: memory leak when drop db which acquired by user with grant/revoke
2023-08-08 09:29:55 +08:00
Hui Li 05e356cd81
Merge pull request #22346 from taosdata/fix/TD-25612
fix: snapshot invalid read
2023-08-08 09:19:55 +08:00
Haojun Liao af3a87560a refactor: do some refactor. 2023-08-08 09:11:41 +08:00
Haojun Liao 458143c8b7
Merge pull request #22348 from taosdata/szhou/restore-tsdb-minglei
fix: restore stt block and data block merge back
2023-08-08 08:58:02 +08:00
Haojun Liao f8ee055641 other: merge 3.0 2023-08-08 00:19:08 +08:00
Haojun Liao ad3e231b2f
Merge pull request #22350 from taosdata/fix/3_liaohj
other: merge main and fix bugs in stop stream tasks.
2023-08-07 22:57:13 +08:00
wangmm0220 864d07e1bc feat:[TD-24559]support geomety type in schemaless 2023-08-07 19:39:06 +08:00
dapan1121 11f6f203ed
Merge pull request #22331 from taosdata/fix/TD-25515
fix: ins_indexes vgroup_id using null value instead of -1 for invalid
2023-08-07 19:03:01 +08:00
Haojun Liao aea1690cd1 fix(stream): fix the stream task after checking if it is in the timer activities. 2023-08-07 17:49:36 +08:00
shenglian zhou ec1270f75c fix: restore stt block/data block merge back 2023-08-07 17:17:12 +08:00
Minglei Jin 9e4da6c089 s3/config: parsing s3 configuration 2023-08-07 17:14:58 +08:00
shenglian zhou 9ddab11d98 Revert "fix(tsdb/read2): reset stt reader when suspended"
This reverts commit 079d7ff69e.
2023-08-07 17:13:49 +08:00
yihaoDeng b6c991f896 support reopen stream state 2023-08-07 08:59:04 +00:00
yihaoDeng eeb97351e8 support reopen stream state 2023-08-07 08:51:01 +00:00
Haojun Liao e6b80e6c52 Merge branch '3.0' into fix/3_liaohj 2023-08-07 16:26:31 +08:00
Haojun Liao 225c0e61dc hen(stream): support the rollback and checkdown stream tasks. 2023-08-07 16:13:39 +08:00
Hongze Cheng a545d45b5e fix: snapshot invalid read 2023-08-07 16:12:03 +08:00
Minglei Jin fac7e521e9 s3/evict: fetch atime from stat file 2023-08-07 15:59:37 +08:00
Hongze Cheng 661f2aca98 fix: memory leak 2023-08-07 14:13:44 +08:00
wangmm0220 5daf38c0d5 fix:ref is not there 2023-08-07 11:29:54 +08:00
Benguang Zhao 0e4f1942f1 fix: resolve Vgroup refcount leak in drop topic 2023-08-07 11:13:52 +08:00
Haojun Liao 76288f311f
Merge pull request #22311 from taosdata/fix/TD-25543
delete atomic_exchange_64
2023-08-07 10:34:50 +08:00
wangmm0220 9ecabb30f5 fix:ref is not there 2023-08-07 10:27:44 +08:00
wangmm0220 5f02e2ddfb fix:ref is not there 2023-08-07 09:31:29 +08:00
Haojun Liao 8a11a0418f other: merge 3.0 2023-08-07 09:19:26 +08:00
wangmm0220 d9a1695257 fix:add log 2023-08-04 18:07:44 +08:00
Hui Li f19826fa9d
Merge pull request #22313 from taosdata/fix/addStreamVer
add stream verion
2023-08-04 17:36:21 +08:00
Shungang Li 5154d0e1a3 fix: alter ttlChangeOnWrite note info 2023-08-04 17:24:35 +08:00
liuyao 944c1f68d2 delete atomic_exchange_64 2023-08-04 16:49:02 +08:00
Minglei Jin 65d8af19ed s3: new api s3Size 2023-08-04 16:35:35 +08:00
Minglei Jin 87783a9650 s3query/get: pull object to local 2023-08-04 15:37:52 +08:00
Ganlin Zhao f7a2e0cdab fix: ins_indexes vgroup_id using null value instead of -1 for invalid
value
2023-08-04 15:21:23 +08:00
yihaoDeng 56ed422f88 add stream ver 2023-08-04 14:09:00 +08:00
yihaoDeng 6dc93a8dec add stream ver 2023-08-04 13:52:20 +08:00
Minglei Jin ebd09ca532 tsdb/write: use keep1 as minKey instead of keep2 2023-08-04 13:44:17 +08:00
Minglei Jin 88d755be76 feat(tsdb/cos): s3 migration 2023-08-04 13:31:00 +08:00
wade zhang abe15cc68e
Merge pull request #22315 from taosdata/szhou/fix-td25385-main
fix: keep stt iterator valid in the list
2023-08-04 08:02:16 +08:00
slzhou fef091b13d fix: keep the stt iterator valid 2023-08-03 21:50:50 +08:00
shenglian zhou b85c1ae8a1 fix: remove the core 2023-08-03 21:50:50 +08:00
shenglian zhou 5322e38dfe fix: add init add check it 2023-08-03 21:50:50 +08:00
shenglian zhou 97b15ed718 fix: delete SLDataItem if stt level files num is smaller 2023-08-03 21:50:50 +08:00
yihaoDeng f7dc9497ef add stream verion 2023-08-03 20:28:34 +08:00
slzhou 1cb1451e41 fix: set nextIndex when get neighbouring index 2023-08-03 18:26:29 +08:00
yihaoDeng f9d0874e45 support reopen stream state 2023-08-03 08:24:50 +00:00
wade zhang 68f2018246
Merge pull request #22289 from taosdata/szhou/fix-25385
fix: keep stt iterator valid
2023-08-02 20:22:54 +08:00
wade zhang 13c713d33b
Merge pull request #22290 from taosdata/3.0_main
merge 3.0 into branch main
2023-08-02 16:04:08 +08:00
Hongze Cheng f90f0b7047 disable infinite count loop 2023-08-02 15:51:51 +08:00
Minglei Jin cff85550f9 Merge branch '3.0' into 3.0_main 2023-08-02 14:10:45 +08:00
wade zhang c24b52ad82
Merge pull request #22277 from taosdata/fix/TD-25400
enh(tsdb/cache/read): lazy init table map hash and entries
2023-08-02 14:08:43 +08:00
Haojun Liao 0feffc687b fix(tsdb): check uid in multiple stt statistics blocks. 2023-08-02 12:44:45 +08:00
Minglei Jin 0e51277521 cache/reader: cleanup contiue nex row 2023-08-02 12:43:57 +08:00
Haojun Liao 2c357f1958 fix(stream): enable filter table based on table statistics. 2023-08-02 11:18:35 +08:00
yihaoDeng 4aef18733f fix state mem leak 2023-08-02 03:16:24 +00:00
Minglei Jin 7fca057a81 cache/reader: move pLastCols into single block 2023-08-02 11:10:12 +08:00
slzhou dd8e893508 fix: keep the stt iterator valid 2023-08-02 11:02:27 +08:00
Minglei Jin 95cc540d9b cache/reader: clean uid in all loop 2023-08-02 10:48:55 +08:00
shenglian zhou 38e797b4ff fix: remove the core 2023-08-02 10:43:26 +08:00
shenglian zhou d5ca51e94d fix: add init add check it 2023-08-02 10:43:26 +08:00
shenglian zhou 6bf95c0c4d fix: delete SLDataItem if stt level files num is smaller 2023-08-02 10:43:26 +08:00
Haojun Liao 13543f7daa feature(stream): handle task update in tq module. 2023-08-02 10:33:41 +08:00
Minglei Jin ed76525c4f cache/reader: cleanup uid in signle loop 2023-08-02 10:28:13 +08:00
wangjiaming0909 72dc46600e fix: memory leak when drop db which acquired by user with grant/revoke 2023-08-02 10:09:50 +08:00
Minglei Jin 29b2f32821 cache reader: cleanup stt init 2023-08-02 09:49:41 +08:00
Haojun Liao 53aefa531f Merge branch '3.0' into fix/3_liaohj 2023-08-02 09:45:21 +08:00
Haojun Liao e1e5e9bb0f enh(stream): check stream tasks dag. 2023-08-02 09:44:39 +08:00
Hongze Cheng 4ba4e83b5b fix: retention file corruption 2023-08-01 20:19:13 +08:00
Haojun Liao c016a58623 enh(stream): do some internal refactor. 2023-08-01 19:22:59 +08:00
Minglei Jin 7d294cac5f tsdb/cache: load current fileset's brinblk once 2023-08-01 18:56:10 +08:00
Minglei Jin a593e887fa cache: remove nFileset from fs state 2023-08-01 18:37:23 +08:00
Haojun Liao ad4c5916e7 enh(stream): update the stream task. 2023-08-01 18:14:58 +08:00
Minglei Jin 7ad628417a fix mem issue: stack-use-after-scope 2023-08-01 17:57:38 +08:00
Minglei Jin 4716fc6400 cache/fs: remove fs state's tsdb handle 2023-08-01 17:49:05 +08:00
Minglei Jin 1d5532019f tsdb/merge-tree: cleanup comment 2023-08-01 17:25:15 +08:00
Minglei Jin f72f129d41 cache/skyline: cleanup del array 2023-08-01 17:23:53 +08:00