Commit Graph

10576 Commits

Author SHA1 Message Date
Haojun Liao 56f1c80018 fix(stream): disable error injection. 2025-02-22 14:18:44 +08:00
Haojun Liao 4d62c60aaf fix(stream): fix syntax error. 2025-02-22 14:17:19 +08:00
Haojun Liao 6c92474afd fix(stream): fix syntax error. 2025-02-22 14:16:46 +08:00
Haojun Liao a9b5d0fbf0 fix(stream): update function return value. 2025-02-22 14:13:42 +08:00
Haojun Liao 9bd8b532ad fix(stream): update function return value. 2025-02-22 14:12:38 +08:00
Haojun Liao 0ec8e1e202 refactor(stream): remove comments. 2025-02-22 12:38:19 +08:00
Haojun Liao 0f89421b62 refactor(stream): scan wal is driven by time, instead of insert events. 2025-02-22 12:38:03 +08:00
Haojun Liao b5fdb44193 refactor(stream): remove comments. 2025-02-22 12:35:37 +08:00
Haojun Liao 09b6642e45 refactor(stream): scan wal is driven by time, instead of insert events. 2025-02-22 12:33:32 +08:00
Hongze Cheng 93e17112e1 Merge branch '3.0' of https://github.com/taosdata/TDengine into enh/TS-5445-3.0 2025-02-21 15:22:22 +08:00
wangmm0220 e594c2bbd5 feat:[TD-33798]modify metadata logic for tmq 2025-02-21 11:49:17 +08:00
Haojun Liao 2228a9fc33
Merge branch 'main' into fix/checkfh 2025-02-21 11:11:36 +08:00
Haojun Liao ecc667c22b
Revert "refactor(stream): drop task in async ways" 2025-02-21 01:10:18 +08:00
wangmm0220 7b5edade95 fix:[TD-337958]send batch metadata & remove data if subscribe only meta 2025-02-20 19:50:37 +08:00
Haojun Liao 9d5735703b Merge branch 'fix/checkfh' into fix/checkpoint 2025-02-20 18:28:46 +08:00
Haojun Liao bea8fe42db fix(stream): check status before scan history. 2025-02-20 18:28:19 +08:00
Haojun Liao 66697ee188 fix(stream): check status before scan history. 2025-02-20 18:22:45 +08:00
Shengliang Guan 2bee60a41a Merge branch 'main' into merge/mainto3.0 2025-02-20 18:12:02 +08:00
Hongze Cheng b980bb35bc more code 2025-02-20 18:08:20 +08:00
Haojun Liao 6766e6db53 refactor(stream): add checkpoint queue to handle chkpt. 2025-02-20 16:13:33 +08:00
Haojun Liao 55f03f2187 fix(stream): try starting task even the inputQ is full. 2025-02-20 16:07:05 +08:00
Shengliang Guan 42bafc0d47
Merge pull request #29848 from taosdata/fix/ly_stream
fix(tsma):drop invalid state for tsma
2025-02-20 15:04:03 +08:00
Shengliang Guan 0a9f5086f0
Merge pull request #29775 from taosdata/fix/analysis
refactor(stream): drop task in async ways
2025-02-20 13:38:53 +08:00
54liuyao 870fe1c071 fix(tsma):drop invalid state for tsma 2025-02-20 11:52:04 +08:00
Shengliang Guan f16ae7abe1
Merge pull request #29809 from taosdata/merge/mainto3.0
merge: from main to 3.0 branch
2025-02-20 10:27:20 +08:00
Shengliang Guan 4e064b5aec
Merge pull request #29823 from taosdata/feat/TS-5928
fix:[TS-5928]add consumer parameters
2025-02-19 19:22:26 +08:00
Shengliang Guan d13f9448eb
Merge pull request #29730 from taosdata/feat/3.0/TS-5584
Feat(sync):Add restore progress to the "show vnodes" command, and add…
2025-02-19 16:30:27 +08:00
wangmm0220 fc3995a10c fix:[TS-5776]add test case 2025-02-19 11:12:13 +08:00
Shengliang Guan 6735e5e948
Merge pull request #29822 from taosdata/kjq/enh-code-clarity
enh(stream): add options to control message and frame size
2025-02-18 18:59:28 +08:00
Shengliang Guan 8bee0e7030
Merge pull request #29666 from taosdata/feat/TS-5776
fix:[TS-5776]add raw type from consumer
2025-02-18 17:53:52 +08:00
wangmm0220 bdfa67e4a9 fix:[TS-5928]add consumer parameters 2025-02-18 17:32:00 +08:00
Jinqing Kuang a48095d467 enh(streams): add options to control message and frame size
Add options to allow users to specify message and frame size limits for
event notifications.
2025-02-18 17:20:38 +08:00
Zhixiao Bao 05112546fd
Merge branch '3.0' into feat/3.0/TS-5584 2025-02-18 16:07:47 +08:00
wangmm0220 f824a75f4d Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TS-5776 2025-02-18 15:28:10 +08:00
wangmm0220 07e4265dde fix:[TS-5776]error in create table time 2025-02-18 15:21:51 +08:00
Shengliang Guan 92eb6038e2
Merge pull request #29762 from taosdata/feat/TS-5469-3.0
feat(stream)[TS-5469]: add more stream event notifications
2025-02-18 15:14:58 +08:00
wangmm0220 9dcac195cb fix:[TS-5776]error in ci 2025-02-18 11:20:42 +08:00
Shengliang Guan 63fc5dea34 Merge branch '3.0' into merge/mainto3.0 2025-02-18 10:29:16 +08:00
wangmm0220 6f2ccfc3b2 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TS-5776 2025-02-17 17:41:25 +08:00
wangmm0220 bf5dc63a30 fix:[TS-5776]add test case 2025-02-17 17:40:54 +08:00
Shengliang Guan 6dedab4bba
Merge pull request #29784 from taosdata/fix/TS-6017-3.0m
fix: timeseries statistics and test case
2025-02-17 10:25:44 +08:00
wangmm0220 3da00b7c74 fix:[TS-5776]add raw type from consumer 2025-02-17 10:13:49 +08:00
Haojun Liao c4cd6dd62d fix(stream): drop task in synch model in write thread. 2025-02-17 10:06:09 +08:00
Shengliang Guan 596e381e0e
Merge pull request #29779 from taosdata/fix/metanull
fix(stream): access value after null ptr check.
2025-02-17 09:36:09 +08:00
Jinqing Kuang fc2d2672b9 feat(stream)[TS-5469]: add more stream event notifications
- Add event notifications for all window types
- Implement window invalidation notifications
- Add statistics for event notification delivery
2025-02-16 11:07:16 +08:00
wangmm0220 47a6836c27 fix:[TS-5776]add raw type from consumer 2025-02-16 00:15:01 +08:00
Haojun Liao 465e5eabd7 fix(stream): drop task in synch model in write thread. 2025-02-15 23:20:05 +08:00
Haojun Liao 0a101fc3c1 fix(stream): add missing release 2025-02-15 00:05:24 +08:00
Haojun Liao 57a2471cc6 fix(stream): add missing release 2025-02-15 00:03:33 +08:00
Haojun Liao 8848ae61a4 fix(stream): add missing release 2025-02-15 00:00:45 +08:00
Haojun Liao 25cdfa5ee9 refactor(stream): add long exec stream queue for history tasks in step1 and re-calculate task execution. 2025-02-14 18:34:20 +08:00
kailixu 5fa20d8cb3 Merge branch 'main' into fix/TS-6017-3.0m 2025-02-14 17:02:47 +08:00
wangmm0220 39f77e45fc fix:[TS-5776]add raw type from consumer 2025-02-14 16:54:53 +08:00
kailixu 6885b942b1 fix: timeseries statistics and test case 2025-02-14 16:53:32 +08:00
wangmm0220 f16f21897e fix:[TS-5776]add raw type from consumer 2025-02-14 16:12:00 +08:00
Haojun Liao 73ec4af4a7 Merge branch 'main' into fix/analysis 2025-02-14 15:37:11 +08:00
wangmm0220 d67de02a31 fix:[TS-5776]add raw type from consumer 2025-02-14 15:09:32 +08:00
Shengliang Guan d6bd52ece7
Merge pull request #29750 from taosdata/enh/streamqueue
refactor(stream): add more operation handled in ctrl queue, and limit the stream input q size
2025-02-14 15:02:23 +08:00
Haojun Liao 03041d0f61 other: merge main 2025-02-14 10:18:19 +08:00
Haojun Liao 43d45e9f9a refactor(stream): drop task in async ways 2025-02-13 23:43:07 +08:00
Haojun Liao fd96b096ea refactor(stream): drop task in async ways 2025-02-13 23:41:06 +08:00
kailixu e3aaab405d fix: timeseries statistics and test case 2025-02-13 18:55:28 +08:00
kailixu 61a606a410 fix: timeseries calculation when create table 2025-02-13 14:03:54 +08:00
wangmm0220 daea0ec61a fix:[TS-5776]add raw type from consumer 2025-02-13 13:54:54 +08:00
Zhixiao Bao e71c03a90f
Merge branch '3.0' into feat/3.0/TS-5584 2025-02-13 10:17:58 +08:00
Shengliang Guan 67b678f9f0
Merge pull request #29746 from taosdata/merge/mainto3.02
merge: from main to 3.0 branch
2025-02-12 13:57:36 +08:00
Shengliang Guan 54c8a2be0a
Merge pull request #29310 from taosdata/fix/TS-5805-check-arb
fix/TS-5805-check-arb
2025-02-12 13:46:06 +08:00
xiao-77 bdcec392b4 Merge remote-tracking branch 'origin/3.0' into feat/3.0/TS-5584 2025-02-12 09:56:23 +08:00
wangmm0220 878bac1556 fix:[TS-5776]avoid memcpy fo DataRspObj 2025-02-11 18:04:53 +08:00
Shengliang Guan cd52401303 Merge branch 'main' into merge/mainto3.02 2025-02-11 15:15:35 +08:00
Haojun Liao 81b718ec69 refactor(stream): handle retrieve chkpt trigger in ctrl queue. 2025-02-11 13:07:06 +08:00
Haojun Liao 9ad6fd170b fix(stream): update the process queue. 2025-02-10 19:24:36 +08:00
Shengliang Guan dae17cd1b0
Merge pull request #29718 from taosdata/enh/streamqueue
refactor(stream): notify close stream in main thread
2025-02-10 18:44:27 +08:00
Hongze Cheng 3d081041ba fix: invalid pointer 2025-02-10 18:38:27 +08:00
xiao-77 fbe65197d6 Feat(sync):Add restore progress to the "show vnodes" command, and add the applied index to the "show vgroups" command. 2025-02-10 15:58:25 +08:00
Hongze Cheng d537b2f8b6 adjust api code 2025-02-10 15:56:23 +08:00
Hongze Cheng b2bfbebfb1 Merge branch '3.0' of https://github.com/taosdata/TDengine into enh/TS-5794-3.0 2025-02-10 14:29:13 +08:00
dongming chen 240a8b5d5b
Merge pull request #29723 from taosdata/3.0
3.0
2025-02-10 09:56:12 +08:00
Haojun Liao 694b6385d5 refactor(stream): adjust reset and mark failed Id. 2025-02-09 21:40:33 +08:00
Haojun Liao a2dcba10a7 refactor(stream): 1) notify closing in main thread. 2) not saving the halt status. 2025-02-09 01:42:24 +08:00
Jinqing Kuang d32cf8bcb8 fix(query)[TS-5984]. correct data block filtering logic in tsdbReader
When scanning a table with a primary key, tsdbReader could load
irrelevant data blocks outside the query time range, causing errors
when returning results.

This fix improves filtering logic to ensure only valid blocks are
read and adds test verification.
2025-02-08 17:32:25 +08:00
Hongze Cheng 499cf0c294 more code 2025-02-08 14:34:22 +08:00
Hongze Cheng f2dafd9362 more code 2025-02-08 11:24:45 +08:00
Shengliang Guan 1c8b1520d6 Merge branch 'main' into merge/mainto3.02 2025-02-08 11:12:14 +08:00
Hongze Cheng 708fa269b5 more code 2025-02-08 11:10:17 +08:00
Hongze Cheng cfb8cdb991 more code 2025-02-07 18:52:53 +08:00
wangmm0220 94b74cc20f Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TS-5776 2025-02-07 18:25:08 +08:00
Shengliang Guan a004d95996 Merge branch '3.0' into merge/mainto3.02 2025-02-06 13:46:36 +08:00
yihaoDeng 14bc7b03bc feat:add test logic 2025-02-05 18:20:02 +08:00
Haojun Liao a5cdf3320c refactor(stream): handle stream check in ctrl queue. 2025-02-05 15:03:30 +08:00
Haojun Liao 0f04f47337 refactor(stream): dispatch req handled in ctrl queue. 2025-02-05 15:03:24 +08:00
Haojun Liao fa2229c453 refactor(stream): dispatch rsp handled in ctrl queue. 2025-02-05 15:03:10 +08:00
Haojun Liao 5e130c4d53 enh(stream): add ctrl queue to handle hb rsp, to avoid hb rsp not being confirmed if all stream threads are occupied. 2025-02-05 14:57:15 +08:00
Shengliang Guan e1cd2735a6
Merge pull request #29639 from taosdata/feat/3.0/TS-5795
Enh(tsdb):print fid while data file corrupted.
2025-02-05 09:41:28 +08:00
Haojun Liao 79284e7414 Merge branch 'main' into fix/disp_lost 2025-01-24 17:15:22 +08:00
Haojun Liao 093233203d
Revert "refactor(stream): do some internal refactor ." 2025-01-24 17:11:15 +08:00
wangmm0220 b5edb79d48 fix:[TS-5776]add raw type from consumer 2025-01-24 16:55:53 +08:00
Shengliang Guan 3089e3d904
Merge pull request #29544 from taosdata/fix/tag
refactor(stream): do some internal refactor .
2025-01-24 16:27:22 +08:00
wangmm0220 902d067776 fix:[TS-5776]add raw type from consumer 2025-01-24 14:56:00 +08:00
wangmm0220 7fad4bceb0 fix:[TS-5776]add raw type from consumer 2025-01-24 13:44:56 +08:00
Pan Wei d6633a27ac
Merge pull request #29651 from taosdata/fix/addMetaDebugLog.m
fix: add table meta validation error and debug log
2025-01-24 09:54:36 +08:00
dapan1121 bd450fe751 fix: add meta debug log 2025-01-23 16:12:54 +08:00
Haojun Liao 963e397984 fix(stream): update logs. 2025-01-23 14:45:47 +08:00
Hongze Cheng 3405a007bd enh: accelerate tsdb snapshot when data are all in stt file 2025-01-23 14:23:54 +08:00
Haojun Liao 6cc3b5da0e fix(stream): add upper bound for the time of waiting for creating table. 2025-01-23 13:37:41 +08:00
Haojun Liao e5e57923cd fix(stream): add upper bound for the time of waiting for creating table. 2025-01-23 13:18:56 +08:00
xiao-77 362aac2de1 Enh(tsdb):print fid while data file corrupted. 2025-01-22 16:46:10 +08:00
wangmm0220 7c5c9f6245 fix:[TS-5776]add raw type from consumer 2025-01-22 16:38:37 +08:00
Shengliang Guan 8c0600649f
Merge pull request #29623 from taosdata/merge/mainto3.0
merge: from main to 3.0 branch
2025-01-22 16:36:17 +08:00
dapan1121 89cea7859a fix: invalid subtable in tbname in clause issue 2025-01-21 17:41:37 +08:00
Jinqing Kuang eb5d463490 feat(stream)[TS-5469]. add support for window event notifications in stream processing
- Introduce new syntax to specify notification type and destination address
- Collect relevant event information during window computations
- Implement websocket-based notification delivery to the specified address
2025-01-21 17:39:46 +08:00
Haojun Liao 84eaed0bbb fix(stream): fix invalid read. 2025-01-19 00:13:58 +08:00
Haojun Liao b4a6523aa6 fix(stream): fix invalid read. 2025-01-18 01:35:24 +08:00
Haojun Liao c93fe999ba refactor(stream): injection error. 2025-01-18 01:26:49 +08:00
Haojun Liao 6274eea7c4 fix(stream): adjust the free stream meta position and check the close flag of streamMeta before starting scan wal. 2025-01-17 19:15:08 +08:00
Haojun Liao 3223cc0287 other: add todo 2025-01-09 18:37:05 +08:00
Shengliang Guan 081c902c64
Merge pull request #29520 from taosdata/fix/TD-32471-main
fix:[TD-32471]set error code to terrno if tmq_consumer_poll return NULL
2025-01-09 13:33:48 +08:00
Hongze Cheng dbfb910bff
Merge pull request #29522 from taosdata/fix/TS-5853-main
enh: add strict message check for submit message
2025-01-09 13:18:38 +08:00
Hongze Cheng c2528b8535 more code 2025-01-09 11:11:19 +08:00
Haojun Liao 30039bb53b enh(stream): add ctrl queue to handle hb rsp, to avoid hb rsp not being confirmed if all stream threads are occupied. 2025-01-09 09:17:08 +08:00
Hongze Cheng a881103c86 more code 2025-01-08 18:13:10 +08:00
Shengliang Guan 37eb686d7c
Merge pull request #29502 from taosdata/fix/main/TD-33457
Fix(cfg): resolve dynamic configuration update failure after rolling …
2025-01-08 17:31:13 +08:00
Hongze Cheng 00b0a46060 enh: add strict message check for submit message 2025-01-08 17:29:32 +08:00
wangmm0220 ba9892bae0 fix:[TD-32471]set error code to terrno if tmq_consumer_poll return NULL 2025-01-08 16:50:38 +08:00
xiao-77 77359f26b3 Fix(cfg): resolve dynamic configuration update failure after rolling upgrade to 3.3.5.0 2025-01-07 15:26:01 +08:00
Hongze Cheng bb01c4d763 fix: compile issue 2025-01-07 15:13:26 +08:00
Hongze Cheng 4587beb849 fix: meta entry snapshot problem for old version 2025-01-07 14:09:29 +08:00
Shengliang Guan b81ade633f
Merge pull request #29478 from taosdata/fix/TD-33440-3.0m
fix: eliminate duplicated read locks in one thread
2025-01-06 13:34:19 +08:00
kailixu c69215576d fix: eliminate duplicated read locks in one thread 2025-01-03 19:00:06 +08:00
Hongze Cheng 55a27061da fix: possible snapshot.py fail 2025-01-03 15:51:59 +08:00
Shengliang Guan 0bbbab1bfa
Merge pull request #29447 from taosdata/fix/tag
fix(stream): fix dead-lock caused by invalid repeatly unlock.
2025-01-02 14:02:35 +08:00
Haojun Liao 055441d8fb fix(stream): fix dead-lock caused by invalid repeatly unlock. 2025-01-02 09:55:05 +08:00
wangmm0220 85aad90d3a fix:[TD-33396]add log for tmq 2024-12-31 11:22:56 +08:00
wangmm0220 b63787aa09 fix:[TD-33396]add log for tmq 2024-12-30 17:23:11 +08:00
Shengliang Guan e53600fee2
Merge pull request #29380 from taosdata/fix/tag
fix(stream): check if the altered tag value table
2024-12-29 22:16:11 +08:00
Haojun Liao a73ad9642a fix(stream): check if the altered tag value table should be added into stream/tq scan table list. 2024-12-28 21:25:40 +08:00
Hongze Cheng a72c46a7fc more cases 2024-12-27 17:44:05 +08:00
Shengliang Guan 912fcb13bf
Merge pull request #29351 from taosdata/merge/mainto3.0
merge: from main to 3.0 branch
2024-12-27 09:06:35 +08:00
Shengliang Guan bc17e0aa9a
Merge pull request #29328 from taosdata/fix/TS-5812-main
fix[TS-5812]:add params check
2024-12-26 11:35:06 +08:00
54liuyao f6500b3551 ci(test):delete invalid code&& add ci 2024-12-26 11:16:02 +08:00
Shengliang Guan 173e6191f3
Merge pull request #29331 from taosdata/fix/fixMetaDeadlock
fix dead lock
2024-12-26 10:04:12 +08:00
yihaoDeng ae168b2560 fix dead lock 2024-12-25 19:14:52 +08:00
wangmm0220 25dc5d7b5c fix[TS-5812]:add params check 2024-12-25 17:53:34 +08:00
wangmm0220 a95304a857 fix[TS-5812]:add params check 2024-12-25 11:59:48 +08:00
dmchen 0da606c7fb show code 2024-12-24 12:14:47 +00:00
Hongze Cheng a89dd27ec1
Merge pull request #29309 from taosdata/enh/TD-33263-3.0-1
ehn: remove useless code
2024-12-24 20:03:08 +08:00
Hongze Cheng 91a9bc30e2
Merge pull request #29228 from taosdata/fix/TD-33284-compact-coverage
Fix/td 33284 compact coverage
2024-12-24 19:01:42 +08:00
Hongze Cheng b32f05bb7c ehn: remove useless code 2024-12-24 18:28:37 +08:00
Shengliang Guan 3c807c59af
Merge pull request #29302 from taosdata/merge/mainto3.0
merge: from main to 3.0
2024-12-24 17:45:15 +08:00
Shengliang Guan 8e5d6d1acd
Merge pull request #29293 from taosdata/fix/code_error
fix:error rewrite if terrno is not 0
2024-12-24 13:35:00 +08:00