Haojun Liao
|
7ccc273206
|
other: merge stream fix in main.
|
2023-10-09 20:05:29 +08:00 |
|
Haojun Liao
|
98ef566eb4
|
fix(stream): remove fill-history sink task.
|
2023-10-09 20:04:17 +08:00 |
|
Haojun Liao
|
e84eeee6b8
|
refactor: do some internal refactor.
|
2023-10-09 20:04:17 +08:00 |
|
Haojun Liao
|
fe7864b976
|
fix(stream): fix error in drop task.
|
2023-10-09 20:04:17 +08:00 |
|
Haojun Liao
|
1256eafddb
|
other: merge stream fix.
|
2023-10-09 19:59:24 +08:00 |
|
Haojun Liao
|
aaa12ed4f8
|
refactor: do some internal refactor about the sched status set
|
2023-10-09 19:56:17 +08:00 |
|
Haojun Liao
|
a678e722ce
|
fix(stream): limit the token bucket capacity
|
2023-10-09 19:56:17 +08:00 |
|
Haojun Liao
|
ee6516f3c4
|
enh(stream): add trans to reset task status to avoid being frozen in checkpoint status due to doing checkpoint failure of partial tasks.
|
2023-10-09 10:13:53 +08:00 |
|
Haojun Liao
|
4d6f143146
|
fix(stream): add missing refactor.
|
2023-10-07 19:18:26 +08:00 |
yihaoDeng
|
fa70221238
|
refact task backend
|
2023-10-07 12:17:55 +08:00 |
|
Haojun Liao
|
f2d8431619
|
fix(stream): enable new time range for stream task.
|
2023-09-28 18:26:52 +08:00 |
|
yihaoDeng
|
54e3ac2c1e
|
refact task backend
|
2023-09-28 15:46:12 +08:00 |
|
Haojun Liao
|
80ec77cbb6
|
enh(stream): optimize the stream status when creating without fill-history option.
|
2023-09-28 11:23:03 +08:00 |
|
Haojun Liao
|
22e485d132
|
fix(stream): opt perf.
|
2023-09-28 09:25:23 +08:00 |
|
Haojun Liao
|
e13efd5b04
|
refactor: optimize the fill-history task launch policy, and do some other internal refactor.
|
2023-09-27 14:00:48 +08:00 |
|
Haojun Liao
|
aec15ae39f
|
enh(stream): add quota limitation for sink task.
|
2023-09-26 23:29:54 +08:00 |
|
Haojun Liao
|
f3f18b0fa3
|
fix(stream): add new status for meta.
|
2023-09-26 12:15:12 +08:00 |
|
Haojun Liao
|
da82a5f19e
|
fix(stream): do not wait for the checkpoint finish before stop tasks.
|
2023-09-24 22:03:07 +08:00 |
|
Haojun Liao
|
a74b462ffb
|
fix(stream): add new timer for each task.
|
2023-09-24 02:25:53 +08:00 |
|
Haojun Liao
|
4ea737571d
|
refactor: do some internal refactor.
|
2023-09-24 01:26:51 +08:00 |
yihaoDeng
|
11da631bb5
|
add cvt state
|
2023-09-22 09:08:42 +00:00 |
|
Haojun Liao
|
0f5f21ba09
|
fix(stream): fix invalid read.
|
2023-09-22 16:05:12 +08:00 |
|
Haojun Liao
|
619a9f6d08
|
fix(stream): refactor re-try dispatch msg for stream tasks.
|
2023-09-22 15:57:55 +08:00 |
|
Haojun Liao
|
898aea5cfb
|
refactor(stream): refactor send msg.
|
2023-09-22 14:52:18 +08:00 |
|
yihaoDeng
|
470849982e
|
add cvt func
|
2023-09-21 09:41:30 +00:00 |
|
Haojun Liao
|
2efd155adf
|
fix(stream): wait for the task checkpoint before stop.
|
2023-09-21 15:53:21 +08:00 |
|
Haojun Liao
|
6633fb971b
|
fix(stream): update logs.
|
2023-09-20 18:18:31 +08:00 |
|
Haojun Liao
|
c33ef4ce88
|
fix(stream): add null check
|
2023-09-18 15:14:51 +08:00 |
|
Haojun Liao
|
3dfdda3283
|
fix(stream): fix error in decode stream task.
|
2023-09-17 18:07:26 +08:00 |
|
Haojun Liao
|
01144c58a7
|
fix(stream): fix stream task id error.
|
2023-09-17 01:19:59 +08:00 |
|
Haojun Liao
|
4846194bbb
|
refactor: do some internal refactor, remove the related fill-history task by sending mesg to replay it in follower node.
|
2023-09-16 21:37:30 +08:00 |
|
Haojun Liao
|
91e3d70796
|
fix(stream): remove fill-history sink task.
|
2023-09-15 17:46:17 +08:00 |
|
Haojun Liao
|
2bfd6e3355
|
refactor: do some internal refactor.
|
2023-09-15 09:25:22 +08:00 |
|
Haojun Liao
|
117de7ab7d
|
fix(stream): fix error in drop task.
|
2023-09-15 00:51:13 +08:00 |
|
Haojun Liao
|
fce5be0eb4
|
fix(stream): fix dead lock.
|
2023-09-14 14:09:05 +08:00 |
|
Haojun Liao
|
7305c8dbe1
|
refactor: do some internal refactor about the sched status set
|
2023-09-13 14:00:34 +08:00 |
|
Haojun Liao
|
0943c1c5b1
|
fix(stream): limit the token bucket capacity
|
2023-09-13 09:19:59 +08:00 |
|
Haojun Liao
|
3b79ab3266
|
fix(stream): reset the init ts.
|
2023-09-11 13:50:55 +08:00 |
|
Haojun Liao
|
85a5c45098
|
fix(stream): limit the already processed data.
|
2023-09-10 00:01:25 +08:00 |
|
Haojun Liao
|
47fd144447
|
other: add some logs.
|
2023-09-09 11:19:51 +08:00 |
|
Haojun Liao
|
e4918e85eb
|
fix(stream): limit the sink rate
|
2023-09-09 00:22:22 +08:00 |
|
Haojun Liao
|
bf79c2c993
|
fix(stream): limit the sink rate
|
2023-09-09 00:11:13 +08:00 |
|
Haojun Liao
|
b240e91c91
|
enh(stream): add token bucket to limit the rate of sink operation.
|
2023-09-07 09:58:48 +08:00 |
|
Haojun Liao
|
8783ebcb02
|
enh(stream): add some logs and record the submit package performance.
|
2023-09-06 14:34:36 +08:00 |
Haojun Liao
|
e816d0efa5
|
Merge pull request #22669 from taosdata/fix/TD-26069
fix invalid insert
|
2023-08-31 18:29:25 +08:00 |
yihaoDeng
|
1833be028f
|
fix invalid insert
|
2023-08-31 15:28:31 +08:00 |
|
Haojun Liao
|
69f95f9324
|
other: merge 3.0
|
2023-08-31 13:56:34 +08:00 |
|
Haojun Liao
|
3fc3dafca6
|
refactor: do some internal refactor.
|
2023-08-31 13:46:48 +08:00 |
|
yihaoDeng
|
83a5e2be4c
|
stream change ver
|
2023-08-31 11:42:12 +08:00 |
|
yihaoDeng
|
05ca71d5de
|
stream change ver
|
2023-08-31 09:28:43 +08:00 |
|
Haojun Liao
|
391103bc5a
|
fix(stream): do dome internal refactor.
|
2023-08-28 15:54:01 +08:00 |
|
Haojun Liao
|
60d1df0e25
|
Merge branch '3.0' into enh/triggerCheckPoint2
|
2023-08-25 15:49:28 +08:00 |
|
Haojun Liao
|
f7dada53b9
|
Merge pull request #22559 from taosdata/fix/liaohj
other: merge main.
|
2023-08-24 19:01:18 +08:00 |
|
Haojun Liao
|
8e07703dd2
|
fix(stream): reset the timerActive value.
|
2023-08-23 18:26:07 +08:00 |
|
Haojun Liao
|
230d8c2d80
|
fix(stream): fix error in stream computing after vnode is transferred to other dnodes.
|
2023-08-23 17:50:44 +08:00 |
|
yihaoDeng
|
ab9490101f
|
Merge branch 'enh/3.0' into enh/triggerCheckPoint2
|
2023-08-22 18:04:04 +08:00 |
|
Haojun Liao
|
af80d94a88
|
refactor: do some internal refactor.
|
2023-08-22 15:43:50 +08:00 |
|
Haojun Liao
|
cc8b2263cb
|
fix(stream): fix error in heartbeat from vnode.
|
2023-08-22 13:37:38 +08:00 |
|
Haojun Liao
|
8b27f98ee4
|
refactor: do some internal refactor.
|
2023-08-21 19:08:27 +08:00 |
|
Haojun Liao
|
29bb5854cc
|
fix(stream): fix error in check stream state.
|
2023-08-21 11:48:55 +08:00 |
|
yihaoDeng
|
8e4411c144
|
fix checkpoint error
|
2023-08-18 21:42:35 +08:00 |
|
Haojun Liao
|
4b10393c81
|
fix(stream): fix memory leak and do some internal refactor.
|
2023-08-18 18:02:08 +08:00 |
|
Haojun Liao
|
a8b53c2e69
|
other: add logs.
|
2023-08-18 10:51:59 +08:00 |
|
Haojun Liao
|
87785b1d9c
|
refactor: add some logs.
|
2023-08-18 10:30:00 +08:00 |
|
Haojun Liao
|
d63758a1c5
|
fix(stream): fix error in execute task on a tranferred node.
|
2023-08-16 15:24:24 +08:00 |
|
Haojun Liao
|
6d7b6fbb74
|
other: add some logs.
|
2023-08-15 20:01:53 +08:00 |
|
Haojun Liao
|
fe5e48b87d
|
fix(stream): remove redundant include.
|
2023-08-15 09:26:35 +08:00 |
|
Haojun Liao
|
b1d35f43ba
|
enh(stream): using term from sync module to denote the upstream node is restarted/replica-modified/vnode-transferred or not.
|
2023-08-15 09:25:05 +08:00 |
|
Haojun Liao
|
87c7e4c0fe
|
fix(stream): not launch stream tasks when restarting the stream tasks.
|
2023-08-11 16:19:46 +08:00 |
|
Haojun Liao
|
e984affe90
|
fix(stream): disable the stream in the followers nodes.
|
2023-08-11 15:58:15 +08:00 |
|
Haojun Liao
|
4afe603c6e
|
fix(stream): fix error.
|
2023-08-11 14:17:09 +08:00 |
|
Haojun Liao
|
6a9826cf6e
|
enh(stream): update the stream task epset.
|
2023-08-11 10:37:25 +08:00 |
|
Haojun Liao
|
f633c7fe2b
|
other: merge 3.0.
|
2023-08-10 11:56:30 +08:00 |
|
Haojun Liao
|
c10b323188
|
other: merge main.
|
2023-08-10 09:26:33 +08:00 |
|
Haojun Liao
|
3cdb16352c
|
fix(stream): avoid launching check downstream when failed to add stream task into stream meta.
|
2023-08-09 12:03:14 +08:00 |
|
Haojun Liao
|
af3a87560a
|
refactor: do some refactor.
|
2023-08-08 09:11:41 +08:00 |
|
Haojun Liao
|
f8ee055641
|
other: merge 3.0
|
2023-08-08 00:19:08 +08:00 |
|
Haojun Liao
|
aea1690cd1
|
fix(stream): fix the stream task after checking if it is in the timer activities.
|
2023-08-07 17:49:36 +08:00 |
|
Haojun Liao
|
225c0e61dc
|
hen(stream): support the rollback and checkdown stream tasks.
|
2023-08-07 16:13:39 +08:00 |
|
yihaoDeng
|
6dc93a8dec
|
add stream ver
|
2023-08-04 13:52:20 +08:00 |
|
Haojun Liao
|
e61aa83594
|
enh(stream): handle the stream hb.
|
2023-07-31 14:07:18 +08:00 |
|
Haojun Liao
|
8dfef8768a
|
other: merge 3.0
|
2023-07-29 22:54:16 +08:00 |
|
Haojun Liao
|
cdffabcdad
|
fix(stream): fix memory leak.
|
2023-07-26 14:37:04 +08:00 |
|
Haojun Liao
|
728112ed89
|
fix(stream): set correct upstream epinfo.
|
2023-07-26 10:26:59 +08:00 |
|
Haojun Liao
|
c116b69941
|
fix(stream): fix memory leak.
|
2023-07-25 09:25:18 +08:00 |
|
Haojun Liao
|
1e34f84f78
|
other: merge 3.0
|
2023-07-24 15:23:03 +08:00 |
|
Haojun Liao
|
f4864b8713
|
fix(stream): fix memory leak.
|
2023-07-22 01:02:58 +08:00 |
|
Haojun Liao
|
1b2636028a
|
fix(stream): fix memory leak.
|
2023-07-21 23:05:42 +08:00 |
|
Haojun Liao
|
97c44e7c18
|
refactor: do some internal refactor.
|
2023-07-19 18:42:02 +08:00 |
|
Haojun Liao
|
d2c875e774
|
Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2
|
2023-07-18 15:19:09 +08:00 |
|
Haojun Liao
|
901b7d8ecc
|
fix(stream): if a source task is set to be non-normal, no data should be put into inputQ anymore.
|
2023-07-18 15:18:57 +08:00 |
|
yihaoDeng
|
40d59990d8
|
fix mem leak
|
2023-07-18 04:01:20 +00:00 |
|
Haojun Liao
|
2f24f776fb
|
fix(stream): fix error in generating checkpoint for state_window.
|
2023-07-18 09:04:28 +08:00 |
|
Haojun Liao
|
b2a905bd91
|
Merge branch '3.0' into enh/triggerCheckPoint2
|
2023-07-17 11:50:12 +08:00 |
|
Haojun Liao
|
eb7e6152ae
|
refactor(stream): do some internal refactor.
|
2023-07-14 17:21:20 +08:00 |
|
Haojun Liao
|
813f4cb363
|
refactor: do some internal refactor.
|
2023-07-13 16:32:25 +08:00 |
|
Haojun Liao
|
2f90a53548
|
fix(stream): set checkpoint id when expanding task.
|
2023-07-13 10:16:57 +08:00 |
|
Haojun Liao
|
d392bb60ba
|
refactor(stream): set the initial version from the checkpoint.
|
2023-07-12 11:38:13 +08:00 |
|
Haojun Liao
|
fed3803739
|
fix(stream): fix memory leak.
|
2023-07-11 19:29:52 +08:00 |
|
Haojun Liao
|
87bedef2cd
|
fix(stream): fix error in checkpointing.
|
2023-07-10 19:20:05 +08:00 |
|
Haojun Liao
|
5940bbfb33
|
fix(stream): set correct datablock.
|
2023-07-10 10:42:38 +08:00 |
|
Haojun Liao
|
f4a3567935
|
other: merge 3.0
|
2023-07-06 11:24:30 +08:00 |
|
yihaoDeng
|
bb520b56e7
|
add checkpoint
|
2023-06-28 09:58:07 +00:00 |
|
Haojun Liao
|
437eb93a4d
|
fix(stream): fix error while fill history exists.
|
2023-06-26 18:44:59 +08:00 |
liuyao
|
aafbdcb492
|
set task status
|
2023-06-20 15:20:04 +08:00 |
|
Haojun Liao
|
b44447e6f3
|
enh(stream): support restore from disk.
|
2023-06-14 10:19:03 +08:00 |
|
Haojun Liao
|
fb24ed161d
|
enh(stream): set correct task status.
|
2023-06-12 22:48:23 +08:00 |
|
Haojun Liao
|
53246ed6bb
|
other: merge other branch.
|
2023-06-09 09:41:10 +08:00 |
|
Haojun Liao
|
06cf3588da
|
refactor: do some internal refactor.
|
2023-06-07 15:03:04 +08:00 |
|
liuyao
|
1e9c4d5fac
|
opt stream block dispatch
|
2023-06-07 10:45:27 +08:00 |
|
Haojun Liao
|
6a3c935b4a
|
fix(stream): fix error in fill history scan.
|
2023-06-07 09:15:55 +08:00 |
|
Haojun Liao
|
eda06081ff
|
enh(stream): refactor and serialize the attributes of history tasks.
|
2023-05-30 16:55:37 +08:00 |
|
Haojun Liao
|
2977a4d07f
|
refactor: do some internal refactor.
|
2023-05-30 09:41:19 +08:00 |
|
Haojun Liao
|
3628888ecc
|
refactor: do some internal refactor.
|
2023-05-29 18:02:45 +08:00 |
|
Haojun Liao
|
cd2d225c93
|
refactor: do some internal refactor.
|
2023-05-29 17:50:39 +08:00 |
|
Haojun Liao
|
f2887278e2
|
refactor: do some internal refactor.
|
2023-05-29 15:44:45 +08:00 |
|
Haojun Liao
|
236926b9ee
|
other: merge enh/rocksdbstate
|
2023-05-13 19:49:28 +08:00 |
|
liuyao
|
8b7b804606
|
opt partition by tbname
|
2023-05-11 18:47:27 +08:00 |
|
yihaoDeng
|
08c3a63341
|
Merge branch 'enh/standVer' into enh/rocksdbSstateMerge
|
2023-05-06 12:37:54 +00:00 |
|
Haojun Liao
|
09a04052ec
|
refactor: do some internal refactor.
|
2023-05-04 16:15:14 +08:00 |
|
yihaoDeng
|
36a4413bf1
|
merge rocksdb inst
|
2023-04-27 14:38:33 +00:00 |
|
Haojun Liao
|
eb0e1f848a
|
fix(stream): remove unused tqreader, do some internal refactor, set the meta value for streamtask.
|
2023-04-14 12:00:09 +08:00 |
|
Haojun Liao
|
4d83118ff7
|
fix(stream): fix memory leak.
|
2023-04-13 23:49:14 +08:00 |
|
Haojun Liao
|
70b45a4cf0
|
fix(stream): all data should be extracted from wal.
|
2023-04-13 23:22:09 +08:00 |
|
Haojun Liao
|
5ab54481f0
|
refactor: do some internal refactor.
|
2023-04-11 19:24:34 +08:00 |
|
Haojun Liao
|
18479d8115
|
refactor: do some internall refactor.
|
2023-04-11 15:42:24 +08:00 |
|
Haojun Liao
|
15cceb5a5a
|
refactor: do some internal refactors.
|
2023-04-11 11:53:10 +08:00 |
|
Haojun Liao
|
f5b52749c2
|
refactor: do some internal refactor.
|
2023-04-09 01:39:09 +08:00 |
Liu Jicong
|
bbab03274a
|
fix memory leak
|
2022-12-27 18:02:43 +08:00 |
|
Liu Jicong
|
035e6b13c8
|
enh(stream): internal optimize
|
2022-10-21 10:29:05 +08:00 |
|
54liuyao
|
7f16081e29
|
feat(stream): stream task and meta
|
2022-10-19 10:49:42 +08:00 |
|
Liu Jicong
|
2d628491b2
|
feat(stream): support tdb state backend
|
2022-08-23 19:30:04 +08:00 |
|
Liu Jicong
|
2b092f6551
|
fix(query): memory leak
|
2022-08-18 18:39:21 +08:00 |
|
Liu Jicong
|
963287798d
|
refactor(mnode): drop stream task
|
2022-08-17 23:18:39 +08:00 |
|
Liu Jicong
|
853e6e2988
|
refactor(mnode): drop stream task
|
2022-08-17 19:19:58 +08:00 |
|
Liu Jicong
|
c411b9aebb
|
fix(stream): memory leak
|
2022-08-08 18:09:36 +08:00 |
|
Liu Jicong
|
43026ac31d
|
refactor(stream)
|
2022-08-08 11:29:50 +08:00 |
|
Liu Jicong
|
7c9d57bd89
|
refactor(stream): unify sink and dispatch
|
2022-08-05 20:16:47 +08:00 |
|
Liu Jicong
|
d6cc88ed4b
|
refactor(stream): exec scheduler
|
2022-08-04 16:19:53 +08:00 |
|
Liu Jicong
|
00acf4520c
|
refactor(stream): remove option
|
2022-07-27 16:49:03 +08:00 |
|
54liuyao
|
8e41fc912b
|
feat(stream): add num of children
|
2022-07-19 14:17:53 +08:00 |
|
Liu Jicong
|
9394e3380f
|
feat(stream): recover from failure
|
2022-07-15 17:58:53 +08:00 |
|
Liu Jicong
|
9ea0ad57ec
|
refactor(stream): destroy stream task
|
2022-07-14 15:32:21 +08:00 |
|
Liu Jicong
|
c2955a807b
|
feat(tmq): offset support snapshot
|
2022-06-29 21:04:51 +08:00 |
|
Liu Jicong
|
14a488c92c
|
feat(stream): stream support multiple type input
|
2022-06-22 21:21:39 +08:00 |
|
Liu Jicong
|
fb596625d3
|
feat(stream): support retrieve
|
2022-06-22 19:12:31 +08:00 |
|
Liu Jicong
|
c409760f75
|
feat(stream): drop stream
|
2022-06-21 11:32:36 +08:00 |
|
Liu Jicong
|
53ef66961a
|
feat(stream): support snode
|
2022-06-20 14:29:18 +08:00 |
|
Liu Jicong
|
342a1ae42d
|
fix(stream): build ctb name
|
2022-06-17 21:00:10 +08:00 |
|
Liu Jicong
|
8eb08fa0d4
|
feat(stream): trigger mode add max delay
|
2022-06-15 16:11:20 +08:00 |
|
Liu Jicong
|
73b8f02ac7
|
fix(stream): set child id
|
2022-06-10 12:00:38 +08:00 |
|
Liu Jicong
|
2677a3c8d9
|
refactor(stream)
|
2022-06-06 20:15:36 +08:00 |