Haojun Liao
|
579dd8e6b2
|
refactor(stream): do some internal refactor.
|
2023-10-27 09:54:56 +08:00 |
|
Haojun Liao
|
6f3c0d17cc
|
refactor(stream): add state machine to manage the state of stream tasks.
|
2023-10-27 09:54:49 +08:00 |
|
Haojun Liao
|
596eb073b7
|
refactor(stream): do some internal refactor.
|
2023-10-27 09:54:48 +08:00 |
|
Haojun Liao
|
91e3fd66e6
|
enh(stream): sm for stream tasks.
|
2023-10-27 09:54:44 +08:00 |
|
Haojun Liao
|
2409a17517
|
refactor(stream): do some internal refactor.
|
2023-10-27 09:54:44 +08:00 |
|
Haojun Liao
|
c9bdb56d79
|
refactor: do some internal refactor.
|
2023-10-27 09:54:09 +08:00 |
|
Haojun Liao
|
164159f293
|
enh(stream): add parameter to limit the stream sink task.
|
2023-10-27 09:54:07 +08:00 |
|
Haojun Liao
|
62c58ecd8b
|
fix(stream):add more check for test cases.
|
2023-10-27 09:54:06 +08:00 |
|
Haojun Liao
|
a4f6042966
|
fix(stream): fix error in creating update node trans.
|
2023-10-27 09:52:14 +08:00 |
|
Haojun Liao
|
ffa798c4fa
|
enh(stream): add some logs.
|
2023-10-27 09:52:14 +08:00 |
wade zhang
|
628667f22c
|
Merge pull request #23231 from taosdata/FEAT/TS-3816-3.0
feat: support restoring vnodes on replaced disks as needed
|
2023-10-20 09:02:25 +08:00 |
Haojun Liao
|
614bdd9fb9
|
Merge pull request #23220 from taosdata/feat/TD-26644
opt bloom filter
|
2023-10-19 15:34:43 +08:00 |
|
Haojun Liao
|
da8f6a4c2f
|
Merge pull request #23248 from taosdata/feat/TD-26647
recover session buff state
|
2023-10-19 15:34:03 +08:00 |
Benguang Zhao
|
c4e9069a66
|
fix: set nextProcessedVer properly in tqProcessTaskScanHistory
|
2023-10-19 11:18:57 +08:00 |
liuyao
|
7269dedb39
|
Merge branch '3.0' of https://github.com/taosdata/TDengine into 3.0
|
2023-10-13 11:35:01 +08:00 |
|
liuyao
|
429125be93
|
session state recover
|
2023-10-13 11:32:35 +08:00 |
|
Haojun Liao
|
33d9f772be
|
other: merge 3.0
|
2023-10-13 09:06:13 +08:00 |
|
Haojun Liao
|
84c5359d5b
|
other: merge 3.0
|
2023-10-13 08:58:26 +08:00 |
|
Haojun Liao
|
eb4449999f
|
other: merge 3.0
|
2023-10-12 10:33:48 +08:00 |
|
liuyao
|
e9fc079d26
|
opt bloom filter
|
2023-10-11 10:14:11 +08:00 |
|
Haojun Liao
|
1d4f0c8599
|
Merge pull request #23201 from taosdata/enh/TD-26646
Enh/td 26646
|
2023-10-10 22:59:37 +08:00 |
|
Haojun Liao
|
05dfadd8f8
|
fix(stream): set initial version for fill-history task. disable reentry of scan-history task function.
|
2023-10-10 15:52:48 +08:00 |
|
Haojun Liao
|
5f9b858fc1
|
other: merge main.
|
2023-10-10 15:38:46 +08:00 |
yihaoDeng
|
b36163a197
|
add session iter
|
2023-10-10 15:12:21 +08:00 |
|
yihaoDeng
|
825341d90a
|
add sess iter
|
2023-10-10 14:35:02 +08:00 |
|
Haojun Liao
|
adf6f65d09
|
fix(stream): set initial version for fill-history task. disable reentry of scan-history task function.
|
2023-10-10 14:00:30 +08:00 |
Alex Duan
|
9da96ebf63
|
Merge pull request #23017 from taosdata/case/TD-26408-MAIN
fix: add case for lost data after split vgroup (main)
|
2023-10-10 00:54:34 -05:00 |
|
Alex Duan
|
e566027cef
|
Merge pull request #23019 from taosdata/case/TD-26408-3.0
fix: add case for lost data after split 3.0
|
2023-10-10 00:53:56 -05:00 |
|
yihaoDeng
|
6adf7532a8
|
add sess iter
|
2023-10-10 11:35:18 +08:00 |
|
yihaoDeng
|
7ac4ef5beb
|
add sess iter
|
2023-10-10 10:41:46 +08:00 |
|
yihaoDeng
|
0fc6ffe6b6
|
change test case
|
2023-10-09 20:51:26 +08:00 |
|
Haojun Liao
|
a25d94ab6f
|
fix(stream): retry when error occurs during timer.
|
2023-10-09 20:10:28 +08:00 |
|
Haojun Liao
|
7d39164133
|
fix(stream): disable retrieving results during checkpoint procedure
|
2023-10-09 20:10:27 +08:00 |
|
Haojun Liao
|
af4618f958
|
fix(stream): remove invalid clear of rspmsg list.
|
2023-10-09 20:10:27 +08:00 |
|
Haojun Liao
|
847baa8b71
|
fix(stream): fix the pause failure bug.
|
2023-10-09 20:10:27 +08:00 |
|
Haojun Liao
|
21fb502690
|
enh(stream): add trans to reset task status to avoid being frozen in checkpoint status due to doing checkpoint failure of partial tasks.
|
2023-10-09 20:10:26 +08:00 |
|
Haojun Liao
|
64bde93f85
|
fix(stream): not idle for sink task when no tokens are available.
|
2023-10-09 20:10:25 +08:00 |
|
Haojun Liao
|
9a7167aaba
|
fix(stream): sleep for 10ms
|
2023-10-09 20:10:24 +08:00 |
|
Haojun Liao
|
9af8ce21ed
|
fix(stream): wait for a while when no available token.
|
2023-10-09 20:10:23 +08:00 |
|
Haojun Liao
|
4780474877
|
fix(stream): continue
|
2023-10-09 20:10:23 +08:00 |
|
Haojun Liao
|
da833ccb30
|
refactor: inc the dump batch.
|
2023-10-09 20:10:23 +08:00 |
|
Haojun Liao
|
2936ac9b89
|
fix(stream): add missing refactor.
|
2023-10-09 20:10:22 +08:00 |
|
Haojun Liao
|
af2c0c95e9
|
refactor: do some internal refactor.
|
2023-10-09 20:10:22 +08:00 |
|
Haojun Liao
|
940d6a3cb4
|
fix(stream): check items in queue and qall, stead of only the queue items.
|
2023-10-09 20:10:21 +08:00 |
|
Haojun Liao
|
e95aea84ae
|
fix(stream): fix syntax error.
|
2023-10-09 20:10:21 +08:00 |
|
Haojun Liao
|
59e284c332
|
fix(stream): quit from loop when input queue is full.
|
2023-10-09 20:10:21 +08:00 |
|
Haojun Liao
|
9d5a3b8d78
|
other: merge stream fix.
|
2023-10-09 20:10:13 +08:00 |
|
Haojun Liao
|
c5997bee5b
|
fix(stream): update the current processing version.
|
2023-10-09 20:09:23 +08:00 |
|
Haojun Liao
|
28008f8263
|
enh(stream): add more info for sink task.
|
2023-10-09 20:09:22 +08:00 |
|
Haojun Liao
|
b3b9e89d30
|
fix(stream): fix the invalid check.
|
2023-10-09 20:09:22 +08:00 |
|
Haojun Liao
|
cf14772dd9
|
enh(stream): optimize the scan wal perf
|
2023-10-09 20:09:21 +08:00 |
|
Haojun Liao
|
d2160efa05
|
refactor: wait for 100ms before scan wal again.
|
2023-10-09 20:09:21 +08:00 |
|
Haojun Liao
|
932d93ab35
|
refactor: wait for more time for future data
|
2023-10-09 20:09:21 +08:00 |
|
Haojun Liao
|
7682c033ad
|
fix(stream): add null check.
|
2023-10-09 20:09:21 +08:00 |
|
Haojun Liao
|
0ab5d4b342
|
refactor: do some internal refactor.
|
2023-10-09 20:09:20 +08:00 |
|
Haojun Liao
|
c295ec322a
|
refactor: do some internal refactor.
|
2023-10-09 20:09:20 +08:00 |
|
Haojun Liao
|
d5abb21335
|
fix(stream): fix invalid check.
|
2023-10-09 20:09:20 +08:00 |
|
Haojun Liao
|
583ac1fc45
|
refactor: do some internal refactor.
|
2023-10-09 20:09:20 +08:00 |
|
Haojun Liao
|
83f84d92bc
|
refactor: do some internal refactor (discard the checkpoint source msg during restoring).
|
2023-10-09 20:09:10 +08:00 |
|
Haojun Liao
|
bb4ba54f28
|
fix(stream): enable new time range for stream task.
|
2023-10-09 20:09:09 +08:00 |
|
Haojun Liao
|
c759d90bf6
|
enh(stream): optimize the stream status when creating without fill-history option.
|
2023-10-09 20:09:09 +08:00 |
|
Haojun Liao
|
db897fb03a
|
fix(stream): opt perf.
|
2023-10-09 20:09:09 +08:00 |
|
Haojun Liao
|
0a2b70fd7f
|
fix(stream): fix syntax error.
|
2023-10-09 20:09:06 +08:00 |
|
Haojun Liao
|
8477271240
|
fix(stream): fix dead lock.
|
2023-10-09 20:09:00 +08:00 |
|
Haojun Liao
|
12eabb0c2c
|
log(stream): update logs.
|
2023-10-09 20:09:00 +08:00 |
|
Haojun Liao
|
b716f0d1fd
|
fix(stream): fix dead lock.
|
2023-10-09 20:08:57 +08:00 |
|
Haojun Liao
|
60a720efbb
|
log: update logs.
|
2023-10-09 20:08:53 +08:00 |
|
Haojun Liao
|
d10b07d23d
|
refactor: do some internal refactor.
|
2023-10-09 20:08:53 +08:00 |
|
Haojun Liao
|
6d2dbd549b
|
fix(stream): avoid repeatly free obj.
|
2023-10-09 20:08:53 +08:00 |
|
Haojun Liao
|
6b5770aab7
|
log(stream): update the log.
|
2023-10-09 20:08:52 +08:00 |
|
Haojun Liao
|
dfe8641c0c
|
refactor: optimize the fill-history task launch policy, and do some other internal refactor.
|
2023-10-09 20:08:52 +08:00 |
|
Haojun Liao
|
38164435f9
|
refactor: do some internal refactor.
|
2023-10-09 20:08:52 +08:00 |
|
Haojun Liao
|
f2848de12a
|
fix(stream): reduce the counter when remove stream tasks.
|
2023-10-09 20:08:52 +08:00 |
|
Haojun Liao
|
ecb3b44026
|
fix(stream): limit sink quota.
|
2023-10-09 20:08:52 +08:00 |
|
Haojun Liao
|
2c0e54f4f7
|
fix(stream): update the retry interval.
|
2023-10-09 20:08:13 +08:00 |
|
Haojun Liao
|
7f8da725bc
|
enh(stream): add quota limitation for sink task.
|
2023-10-09 20:08:13 +08:00 |
|
Haojun Liao
|
a248f3d088
|
refactor: do some internal refactor.
|
2023-10-09 20:08:12 +08:00 |
|
Haojun Liao
|
0db357401d
|
fix(stream): add new status for meta.
|
2023-10-09 20:08:12 +08:00 |
|
Haojun Liao
|
3912d80c02
|
fix(stream): do some internal refactor.
|
2023-10-09 20:08:11 +08:00 |
|
Haojun Liao
|
feb09c8361
|
fix(stream): fix error in pause scan-history data.
|
2023-10-09 20:08:11 +08:00 |
|
Haojun Liao
|
6a98b11bef
|
fix(stream): fix invalid read and memory leak.
|
2023-10-09 20:08:11 +08:00 |
|
Haojun Liao
|
65f66f0986
|
fix(stream): transfer state when all downstream rsp already.
|
2023-10-09 20:08:11 +08:00 |
|
Haojun Liao
|
a32b56f381
|
fix(stream): fix memory leak.
|
2023-10-09 20:08:10 +08:00 |
|
Haojun Liao
|
4af7812f0b
|
fix(stream): fix the invalid counter for ready tasks in vnode, fix an invalid read.
|
2023-10-09 20:08:10 +08:00 |
|
Haojun Liao
|
4fe1382cd9
|
fix(stream): adjust nodeUpdate check interval.
|
2023-10-09 20:08:09 +08:00 |
|
Haojun Liao
|
f4f55fcbce
|
fix(stream): record the tasks started info, and do some internal refactor.
|
2023-10-09 20:08:09 +08:00 |
|
Haojun Liao
|
68b003c804
|
fix(stream): fix race condition.
|
2023-10-09 20:08:09 +08:00 |
|
Haojun Liao
|
fc96ec6bae
|
fix(stream): do not wait for the checkpoint finish before stop tasks.
|
2023-10-09 20:08:09 +08:00 |
|
Haojun Liao
|
7167c3c5de
|
fix(stream): add new timer for each task.
|
2023-10-09 20:08:08 +08:00 |
|
Haojun Liao
|
7336228c16
|
refactor: do some internal refactor.
|
2023-10-09 20:08:08 +08:00 |
|
Haojun Liao
|
477f57aa0e
|
refactor: do some internal refactor.
|
2023-10-09 20:08:08 +08:00 |
|
Haojun Liao
|
52763ca2a3
|
refactor: do some internal refactor.
|
2023-10-09 20:08:07 +08:00 |
|
Haojun Liao
|
3baddbe67a
|
fix(stream): update the checkpoint record start time.
|
2023-10-09 20:08:07 +08:00 |
|
Haojun Liao
|
a6ffb845c8
|
fix(stream): fix invalid checkpoint start time.
|
2023-10-09 20:08:07 +08:00 |
|
Haojun Liao
|
c05a8de3e8
|
fix(stream): rsp when checkpoint source failed.
|
2023-10-09 20:08:06 +08:00 |
|
Haojun Liao
|
b4bc4fac98
|
fix(stream): disable auto rsp for checkpoint source.
|
2023-10-09 20:08:06 +08:00 |
|
Haojun Liao
|
e671e87234
|
fix(stream): fix invalid assert.
|
2023-10-09 20:08:06 +08:00 |
|
Haojun Liao
|
d1a6b8bd22
|
refactor: do some internal refactor.
|
2023-10-09 20:08:06 +08:00 |
|
Haojun Liao
|
46b17aa9f1
|
fix(stream): fix race condition.
|
2023-10-09 20:08:06 +08:00 |
|
Haojun Liao
|
e49409a11e
|
fix(stream): add some logs.
|
2023-10-09 20:08:05 +08:00 |
|
Haojun Liao
|
5eee76df65
|
fix(stream): add msg identifier check.
|
2023-10-09 20:08:05 +08:00 |
|
Haojun Liao
|
192c47a7c2
|
fix(stream): add some logs.
|
2023-10-09 20:08:05 +08:00 |
|
Haojun Liao
|
693254f602
|
fix(stream): remove invalid free.
|
2023-10-09 20:08:04 +08:00 |
|
Haojun Liao
|
f3bd5c411d
|
fix(stream): remove the blocking flag after successfully retry sending msg.
|
2023-10-09 20:08:04 +08:00 |
|
Haojun Liao
|
ed9e035095
|
fix(stream): clear the failure vnode id list before send msg.
|
2023-10-09 20:08:04 +08:00 |
|
Haojun Liao
|
3726aa723e
|
fix(stream): fix invalid read.
|
2023-10-09 20:08:04 +08:00 |
|
Haojun Liao
|
e0d19af000
|
fix(stream): refactor re-try dispatch msg for stream tasks.
|
2023-10-09 20:08:03 +08:00 |
|
Haojun Liao
|
2b16976902
|
refactor(stream): refactor send msg.
|
2023-10-09 20:08:03 +08:00 |
|
Haojun Liao
|
27e00b1f19
|
fix(stream): add some logs, and remove invalid assert.
|
2023-10-09 20:08:02 +08:00 |
|
Haojun Liao
|
12687eb287
|
fix(stream): add check info detailed information.
|
2023-10-09 20:08:02 +08:00 |
|
Haojun Liao
|
ab7c18f604
|
refactor: do some internal refactor.
|
2023-10-09 20:08:01 +08:00 |
|
Haojun Liao
|
e5535fa1f5
|
fix(stream): handle dispatch checkpoint-trigger failure.
|
2023-10-09 20:08:01 +08:00 |
|
Haojun Liao
|
65cc4040f3
|
fix(stream): update logs.
|
2023-10-09 20:08:00 +08:00 |
|
Haojun Liao
|
6d3a134077
|
fix(stream): ignore the dispatch failure, and set the correct rsp counter.
|
2023-10-09 20:07:54 +08:00 |
|
Haojun Liao
|
7f4d9b6e41
|
fix(stream): wait for the task checkpoint before stop.
|
2023-10-09 20:07:53 +08:00 |
|
Haojun Liao
|
ef9efc8a8e
|
refactor: do refactor the sink task.
|
2023-10-09 20:07:53 +08:00 |
|
Haojun Liao
|
e4ac835554
|
fix(stream): disable token bucket.
|
2023-10-09 20:07:53 +08:00 |
|
Haojun Liao
|
a3d13d6634
|
fix(stream): remove invalid cached task info in buffer.
|
2023-10-09 20:07:08 +08:00 |
|
Haojun Liao
|
1f45bd82d2
|
other: merge stream fix in main.
|
2023-10-09 20:07:01 +08:00 |
|
Haojun Liao
|
d752fb1ed6
|
fix(stream): exec task in task queuee threads.
|
2023-10-09 20:06:11 +08:00 |
|
Haojun Liao
|
1ebafe4a1f
|
fix(stream): add more check.
|
2023-10-09 20:06:11 +08:00 |
|
Haojun Liao
|
c403a15f36
|
fix(stream): check the status before exec stream task when handling the dispatch requests.
|
2023-10-09 20:06:11 +08:00 |
|
Haojun Liao
|
d76e3ac718
|
fix(stream): add task status check.
|
2023-10-09 20:06:10 +08:00 |
|
Haojun Liao
|
e27111ecd1
|
enh(stream): add node stage check.
|
2023-10-09 20:06:09 +08:00 |
|
Haojun Liao
|
ce85945583
|
fix(stream): set correct size for results generated by scan history stream tasks.
|
2023-10-09 20:06:05 +08:00 |
|
Haojun Liao
|
053643be19
|
fix(stream): add timestamp.
|
2023-10-09 20:06:05 +08:00 |
|
Haojun Liao
|
b38f9e0288
|
fix(stream): update logs.
|
2023-10-09 20:06:05 +08:00 |
|
Haojun Liao
|
5c0b8ea804
|
enh(stream): log the checkpoint time.
|
2023-10-09 20:06:04 +08:00 |
|
Haojun Liao
|
b95ad74c7f
|
fix(stream): not handle the check msg for follower tasks.
|
2023-10-09 20:06:04 +08:00 |
|
Haojun Liao
|
e6fb9ee46c
|
fix(stream): reset task counter.
|
2023-10-09 20:06:04 +08:00 |
|
Haojun Liao
|
c3a3cf21fb
|
fix(stream): add some logs.
|
2023-10-09 20:05:39 +08:00 |
|
Haojun Liao
|
08b37dfc13
|
fix(stream): fix memory leak.
|
2023-10-09 20:05:38 +08:00 |
|
Haojun Liao
|
734f648891
|
fix(stream): set the output normal for stream dispatch when encountering the transfer state.
|
2023-10-09 20:05:38 +08:00 |
|
Haojun Liao
|
636eccc160
|
fix(stream): add null check
|
2023-10-09 20:05:37 +08:00 |
|
Haojun Liao
|
57bf1cadc4
|
fix(stream): fix error in decode stream task.
|
2023-10-09 20:05:37 +08:00 |
|
Haojun Liao
|
9d6199adf7
|
fix(stream): fix invalid read.
|
2023-10-09 20:05:37 +08:00 |
|
Haojun Liao
|
0459a4628c
|
fix(stream): fix stream task id error.
|
2023-10-09 20:05:36 +08:00 |
|
Haojun Liao
|
7ccc273206
|
other: merge stream fix in main.
|
2023-10-09 20:05:29 +08:00 |
|
Haojun Liao
|
98ef566eb4
|
fix(stream): remove fill-history sink task.
|
2023-10-09 20:04:17 +08:00 |
|
Haojun Liao
|
e84eeee6b8
|
refactor: do some internal refactor.
|
2023-10-09 20:04:17 +08:00 |
|
Haojun Liao
|
fe7864b976
|
fix(stream): fix error in drop task.
|
2023-10-09 20:04:17 +08:00 |
|
Haojun Liao
|
c33e521aa1
|
fix(stream): update some logs.
|
2023-10-09 20:04:17 +08:00 |
|
Haojun Liao
|
39e0c57323
|
fix(stream): set the correct guard flag.
|
2023-10-09 20:04:16 +08:00 |
|
Haojun Liao
|
3d1f29ff21
|
fix(stream): fix memory leak.
|
2023-10-09 20:04:16 +08:00 |
|
yihaoDeng
|
c6e9134cea
|
add sess iter
|
2023-10-09 20:02:23 +08:00 |
|
Haojun Liao
|
6fa54789cb
|
fix(stream): disable follower send hb to mnode. and do some internal refactor.
|
2023-10-09 19:59:31 +08:00 |
|
Haojun Liao
|
1256eafddb
|
other: merge stream fix.
|
2023-10-09 19:59:24 +08:00 |
|
Haojun Liao
|
283118bf68
|
fix(stream):fix missing unlock.
|
2023-10-09 19:56:17 +08:00 |
|
Haojun Liao
|
aaa12ed4f8
|
refactor: do some internal refactor about the sched status set
|
2023-10-09 19:56:17 +08:00 |
|
Haojun Liao
|
01d104706e
|
refactor(stream): remove unnecessary streamschedTask function to improve the performance.
|
2023-10-09 19:56:17 +08:00 |