Commit Graph

293 Commits

Author SHA1 Message Date
Haojun Liao db92bf84d8 other: merge 3.0 2024-02-29 22:21:50 +08:00
Haojun Liao dafac591f7 enh(stream): add rsp for req-checkpoint 2024-02-27 10:53:25 +08:00
Haojun Liao d8ffa65b62 fix(stream): fix the compatible issue when the fill-history exists. 2024-02-23 09:12:27 +08:00
Haojun Liao af1f61cd39 fix(stream): fix the compatible issue when the fill-history exists. 2024-02-22 19:05:30 +08:00
Haojun Liao d6f5ae226b refactor: do some internal refactor. 2024-02-21 09:42:23 +08:00
Haojun Liao 8d848a5bf0 fix(stream): add null ptr check. 2024-02-21 09:40:22 +08:00
Haojun Liao 13c3654bf5 other: merge 3.0 2024-02-20 10:51:04 +08:00
54liuyao af1893d109 merge 3.0 2024-02-19 18:30:12 +08:00
Haojun Liao bf242a4f64 fix(stream): handle pause event in waiting list. 2024-02-19 15:57:21 +08:00
wangmm0220 ac9582b24a fix:[TD-28567] do not add md5 to the end of subtable if create stream by sma 2024-02-19 13:58:58 +08:00
Haojun Liao 730edf24f9 fix(stream): fix syntax error. 2024-02-19 13:38:14 +08:00
Haojun Liao d3e8adf2eb enh(stream): async handle pause event 2024-02-19 13:36:57 +08:00
54liuyao b53dc5602e Merge branch '3.0' of https://github.com/taosdata/TDengine into 3.0 2024-02-19 10:58:20 +08:00
wangmm0220 a8e45440c0 fix:conflicts from 3.0 2024-02-18 18:02:14 +08:00
Haojun Liao 7a18e5910c enh(stream): halt the initial stream tasks when fill-history is set for count window agg 2024-02-05 17:51:09 +08:00
Haojun Liao cb2ea4a721 fix(stream): take ver snapshot for all vgroups before launching stream with fill-history option opened. 2024-02-01 11:37:57 +08:00
wangmm0220 a9fcc67321 fix:conflicts from 3.0 2024-02-01 11:13:03 +08:00
Haojun Liao d4bab8c09b refactor: do some internal refactor. 2024-01-25 16:53:30 +08:00
Haojun Liao dc1ea9f9a1 fix(stream): clear htask info when unregistering the task. 2024-01-24 16:23:47 +08:00
Haojun Liao b8856931d8 fix(stream): fix syntax error. 2024-01-24 13:11:36 +08:00
Haojun Liao 8a7e38ad15 fix(stream): fix checkpoint failure check. 2024-01-24 13:10:53 +08:00
Haojun Liao d973f66cea fix(stream): do checkpoint after fill-history task completed. 2024-01-19 16:18:02 +08:00
wangmm0220 7246fedfef Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/ly_multi_marks 2024-01-18 19:08:35 +08:00
Haojun Liao 76e6c453e7 enh(stream): send msg to mnode when the transferring state completed. 2024-01-18 15:43:36 +08:00
Haojun Liao fc16d26da2
Merge pull request #24488 from taosdata/fix/3_liaohj
fix(stream): fix some errors and do some internal refactor.
2024-01-17 14:25:32 +08:00
Haojun Liao 8ede6d87ce refactor: do some internal refactor. 2024-01-16 15:40:40 +08:00
jiajingbin d59ede5196 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-28032 2024-01-16 15:05:13 +08:00
Haojun Liao 63494de7f9 refactor: do some internal refactor. 2024-01-16 10:28:07 +08:00
wangmm0220 2ab54a1335 fix:conflicts from 3.0 2024-01-15 16:54:52 +08:00
Haojun Liao 143e6a13af fix(stream): fix deadlock in pause. 2024-01-15 14:37:24 +08:00
Haojun Liao e012bc4bde refactor: do some internal refactor. 2024-01-15 11:28:36 +08:00
Haojun Liao 6d37e596ec refactor: do some internal refactor. 2024-01-04 18:47:49 +08:00
Haojun Liao 185fa75376 enh(stream): add task into timer to be idle during exec when the corresponding output queue is full. 2024-01-03 17:59:42 +08:00
wangmm0220 8add4dc177 fix:[TD-28032]concat subtable and hash val as new subtable name to avoid data from multi time line write to one time line 2024-01-02 18:37:29 +08:00
Haojun Liao 84583580e2 fix(stream): transfer the checkpoint trans id. 2023-12-21 19:12:49 +08:00
wangmm0220 81b5f59fa5 Merge branch 'feat/ly_multi_marks' of https://github.com/taosdata/TDengine into feat/ly_multi_marks 2023-12-19 10:54:12 +08:00
wangmm0220 3a0af1ea49 fix:aggCnt must bigger than 1 & move msg code to the end to be compatible with old version 2023-12-19 10:53:46 +08:00
Haojun Liao 4f852704f5 fix(stream): send checkpoint complete msg to mnode when dropping it. 2023-12-17 02:06:01 +08:00
yihaoDeng 08eeee0794 Merge branch '3.0' into enh/refactorBackend 2023-11-28 11:52:49 +08:00
wangmm0220 06e2fbfb1c fix:conflicts from 3.0 2023-11-27 17:53:53 +08:00
wangmm0220 70c5d36103 opti:extract common logic from snode & vnode to tqCommon 2023-11-27 17:49:09 +08:00
yihaoDeng e9bec34c44 Merge branch '3.0' into enh/refactorBackend 2023-11-27 10:01:12 +08:00
wangmm0220 d249892b2a fix:conflicts from 3.0 2023-11-23 20:29:46 +08:00
Haojun Liao 2e922fefc6 fix(stream): reduce the sink throughput. and fix the bug in status check. 2023-11-23 18:04:09 +08:00
yihaoDeng 6129ebc294 Merge branch '3.0' into enh/refactorBackend 2023-11-23 17:41:37 +08:00
wangmm0220 15d75a0497 fix:spelling errors 2023-11-23 09:17:54 +08:00
wangmm0220 9a09e318ca opti:common message process logic in vnode and snode 2023-11-22 16:53:33 +08:00
yihaoDeng a92602b667 fix mem leak 2023-11-21 15:23:28 +08:00
yihaoDeng de7f2a2d1a Merge branch '3.0' into enh/refactorBackend 2023-11-13 11:11:28 +08:00
Haojun Liao 97772e9aab fix(stream): the checkpoint version can only be updated when generating checkpoint. 2023-11-11 15:40:05 +08:00
yihaoDeng 3abf0ae4b2 Merge branch 'feat/TD-26060' into enh/refactorBackend 2023-11-08 17:31:16 +08:00
Haojun Liao 884770dab1 fix(stream): fix syntax error. 2023-11-07 17:12:06 +08:00
Haojun Liao e13f003b90 refactor: do some internal refactor. 2023-11-05 01:23:23 +08:00
yihaoDeng 4c8edb5545 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-01 16:07:49 +08:00
yihaoDeng ee6b620597 fix mem leak 2023-10-30 17:29:32 +08:00
Haojun Liao c7285c0ede fix(stream): add expired epset node list for stream tasks in hb to mnode. 2023-10-27 23:41:57 +08:00
Haojun Liao 0d4892f25b fix(stream): fix bugs caused by refactor sm. 2023-10-27 09:55:00 +08:00
Haojun Liao a31613ae95 refactor: add ref in log. 2023-10-27 09:54:59 +08:00
Haojun Liao e1f6043eed fix(stream): fix bugs caused by refactor sm. 2023-10-27 09:54:57 +08:00
Haojun Liao 1eac7f44c0 fix(stream): fix error in stream. 2023-10-27 09:54:56 +08:00
Haojun Liao 579dd8e6b2 refactor(stream): do some internal refactor. 2023-10-27 09:54:56 +08:00
Haojun Liao 6f3c0d17cc refactor(stream): add state machine to manage the state of stream tasks. 2023-10-27 09:54:49 +08:00
Haojun Liao 596eb073b7 refactor(stream): do some internal refactor. 2023-10-27 09:54:48 +08:00
Haojun Liao 2409a17517 refactor(stream): do some internal refactor. 2023-10-27 09:54:44 +08:00
Haojun Liao 164159f293 enh(stream): add parameter to limit the stream sink task. 2023-10-27 09:54:07 +08:00
Haojun Liao a4f6042966 fix(stream): fix error in creating update node trans. 2023-10-27 09:52:14 +08:00
yihaoDeng db0f6258fe fix stream case error 2023-10-25 19:38:22 +08:00
yihaoDeng 1a4cfd11a5 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-10-20 15:28:10 +08:00
yihaoDeng 2004c1a346 fix transfer crash 2023-10-17 18:19:52 +08:00
yihaoDeng a56d831a14 refactor backend 2023-10-12 20:16:41 +08:00
yihaoDeng 4c94fbb44f refactor backend 2023-10-11 20:26:19 +08:00
yihaoDeng f264ffdcd6 refactor backend 2023-10-11 20:12:07 +08:00
Haojun Liao 21fb502690 enh(stream): add trans to reset task status to avoid being frozen in checkpoint status due to doing checkpoint failure of partial tasks. 2023-10-09 20:10:26 +08:00
Haojun Liao 2936ac9b89 fix(stream): add missing refactor. 2023-10-09 20:10:22 +08:00
Haojun Liao bb4ba54f28 fix(stream): enable new time range for stream task. 2023-10-09 20:09:09 +08:00
Haojun Liao c759d90bf6 enh(stream): optimize the stream status when creating without fill-history option. 2023-10-09 20:09:09 +08:00
Haojun Liao db897fb03a fix(stream): opt perf. 2023-10-09 20:09:09 +08:00
Haojun Liao dfe8641c0c refactor: optimize the fill-history task launch policy, and do some other internal refactor. 2023-10-09 20:08:52 +08:00
Haojun Liao 7f8da725bc enh(stream): add quota limitation for sink task. 2023-10-09 20:08:13 +08:00
Haojun Liao 0db357401d fix(stream): add new status for meta. 2023-10-09 20:08:12 +08:00
Haojun Liao fc96ec6bae fix(stream): do not wait for the checkpoint finish before stop tasks. 2023-10-09 20:08:09 +08:00
Haojun Liao 7167c3c5de fix(stream): add new timer for each task. 2023-10-09 20:08:08 +08:00
Haojun Liao 52763ca2a3 refactor: do some internal refactor. 2023-10-09 20:08:07 +08:00
Haojun Liao 3726aa723e fix(stream): fix invalid read. 2023-10-09 20:08:04 +08:00
Haojun Liao e0d19af000 fix(stream): refactor re-try dispatch msg for stream tasks. 2023-10-09 20:08:03 +08:00
Haojun Liao 2b16976902 refactor(stream): refactor send msg. 2023-10-09 20:08:03 +08:00
Haojun Liao 7f4d9b6e41 fix(stream): wait for the task checkpoint before stop. 2023-10-09 20:07:53 +08:00
Haojun Liao 1f45bd82d2 other: merge stream fix in main. 2023-10-09 20:07:01 +08:00
Haojun Liao 636eccc160 fix(stream): add null check 2023-10-09 20:05:37 +08:00
Haojun Liao 57bf1cadc4 fix(stream): fix error in decode stream task. 2023-10-09 20:05:37 +08:00
Haojun Liao 0459a4628c fix(stream): fix stream task id error. 2023-10-09 20:05:36 +08:00
Haojun Liao 7ccc273206 other: merge stream fix in main. 2023-10-09 20:05:29 +08:00
Haojun Liao 98ef566eb4 fix(stream): remove fill-history sink task. 2023-10-09 20:04:17 +08:00
Haojun Liao e84eeee6b8 refactor: do some internal refactor. 2023-10-09 20:04:17 +08:00
Haojun Liao fe7864b976 fix(stream): fix error in drop task. 2023-10-09 20:04:17 +08:00
Haojun Liao 1256eafddb other: merge stream fix. 2023-10-09 19:59:24 +08:00
Haojun Liao aaa12ed4f8 refactor: do some internal refactor about the sched status set 2023-10-09 19:56:17 +08:00
Haojun Liao a678e722ce fix(stream): limit the token bucket capacity 2023-10-09 19:56:17 +08:00
Haojun Liao ee6516f3c4 enh(stream): add trans to reset task status to avoid being frozen in checkpoint status due to doing checkpoint failure of partial tasks. 2023-10-09 10:13:53 +08:00
Haojun Liao 4d6f143146 fix(stream): add missing refactor. 2023-10-07 19:18:26 +08:00