325a177170
fix(stream): disable the fill-history task when restart stream tasks.
2023-09-04 17:18:04 +08:00
5159bf3e2a
fix(stream): fix epset errors
2023-09-01 15:33:23 +08:00
cc17848b78
fix: merge 3.0 to main
2023-09-01 13:24:47 +08:00
95da66e3e8
fix(stream): fix error.
2023-08-31 20:50:12 +08:00
3fc3dafca6
refactor: do some internal refactor.
2023-08-31 13:46:48 +08:00
d99d8875e1
fix(stream): if stream task is running, do not recheck its status.
2023-08-30 17:24:13 +08:00
57608e7fff
fix merge error
2023-08-30 17:12:37 +08:00
997d247aa6
fix(stream): remove invalid memset.
2023-08-30 16:58:22 +08:00
c5b279f045
Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2
2023-08-30 16:01:08 +08:00
24b1a050fc
other: merge 3.0
2023-08-30 16:00:14 +08:00
0a72e576bd
op stream pause
...
pause data dispatch
opt stream pause
2023-08-30 14:53:49 +08:00
f6321d0c4f
Merge remote-tracking branch 'origin/main' into fix/m23.1
2023-08-30 11:03:33 +08:00
3a7a220d43
fix(stream): fix error found by ci.
2023-08-30 10:14:01 +08:00
204fb97f68
fix(stream): fix error in snode, fix memory leaks, fix repeatly create bug in sink task.
2023-08-29 17:34:31 +08:00
8ed2836a2b
refactor code
2023-08-29 09:52:03 +08:00
a2901411b0
refactor code
2023-08-28 18:52:10 +08:00
1687f68156
refactor code
2023-08-28 16:13:27 +08:00
391103bc5a
fix(stream): do dome internal refactor.
2023-08-28 15:54:01 +08:00
b75ae217f6
fix(stream): notify close for snode.
2023-08-28 09:53:08 +08:00
998dac0ffb
fix mem leak
2023-08-26 21:54:36 +08:00
7d237d4310
fix UAF
2023-08-26 17:37:09 +08:00
bed5765013
fix UAF
2023-08-26 16:15:26 +08:00
0348ef6e22
fix UAF
2023-08-26 16:13:17 +08:00
af130cc92f
Revert "add seperate function for createLogs"
...
This reverts commit 13d2d72bb6
2023-08-25 09:55:50 +08:00
218e2cc5f3
fix(stream): invoke the timer when not sending hb.
2023-08-24 18:27:09 +08:00
82e356a9b5
Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2
...
# Conflicts:
# source/dnode/vnode/src/tq/tq.c
2023-08-24 18:04:44 +08:00
70396aa5a5
fix(stream): wakeup the hb timer every 200ms, to speeup the close db procedure.
2023-08-24 17:54:50 +08:00
090515c684
refactor code
2023-08-24 17:20:04 +08:00
13d2d72bb6
add seperate function for createLogs
2023-08-24 14:56:14 +08:00
962a0cd403
fix
2023-08-23 23:34:02 +08:00
0959758bd5
fix transfer error
2023-08-23 21:43:10 +08:00
230d8c2d80
fix(stream): fix error in stream computing after vnode is transferred to other dnodes.
2023-08-23 17:50:44 +08:00
12b10dfc76
fix mem leak
2023-08-23 09:08:37 +08:00
5fe7563a13
fix mem leak
2023-08-22 21:13:02 +08:00
ba5c3648db
fix mem leak
2023-08-22 21:12:54 +08:00
ab9490101f
Merge branch 'enh/3.0' into enh/triggerCheckPoint2
2023-08-22 18:04:04 +08:00
af80d94a88
refactor: do some internal refactor.
2023-08-22 15:43:50 +08:00
e6b49b45d9
Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2
2023-08-22 13:37:50 +08:00
cc8b2263cb
fix(stream): fix error in heartbeat from vnode.
2023-08-22 13:37:38 +08:00
d106b5a87e
Merge branch 'enh/triggerCheckPoint2' of https://github.com/taosdata/TDengine into enh/triggerCheckPoint2
2023-08-22 10:48:30 +08:00
a0fd56c84f
fix invalid cmp
2023-08-22 10:48:15 +08:00
def0877980
fix(stream): add timer to check stream task status.
2023-08-22 10:46:49 +08:00
082882b71e
fix(stream): reset the upstream stage info.
2023-08-21 19:29:03 +08:00
387b4d365e
refactor checkpoint
2023-08-21 19:07:15 +08:00
29bb5854cc
fix(stream): fix error in check stream state.
2023-08-21 11:48:55 +08:00
8e4411c144
fix checkpoint error
2023-08-18 21:42:35 +08:00
114df7c349
fix transfer error
2023-08-16 11:37:15 +00:00
ab7e2b3122
Merge branch 'enh/chkpTransfer' into enh/triggerCheckPoint2
2023-08-16 09:00:30 +00:00
d63758a1c5
fix(stream): fix error in execute task on a tranferred node.
2023-08-16 15:24:24 +08:00
993ae84398
close backend
2023-08-16 07:17:15 +00:00
39079ab64a
fix(stream): load term from sync model to decide if the node is restart or not
2023-08-15 15:52:06 +08:00
b1d35f43ba
enh(stream): using term from sync module to denote the upstream node is restarted/replica-modified/vnode-transferred or not.
2023-08-15 09:25:05 +08:00
99be2836b2
Merge branch 'enh/triggerCheckPoint2' into enh/chkpTransfer
2023-08-11 02:50:12 +00:00
1e636a2ef9
Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2
2023-08-11 10:37:40 +08:00
6a9826cf6e
enh(stream): update the stream task epset.
2023-08-11 10:37:25 +08:00
ba4ce9db02
Merge branch 'enh/triggerCheckPoint2' into enh/chkpTransfer
2023-08-11 02:15:07 +00:00
32983a11e1
add log and fix crash
2023-08-10 20:34:00 +08:00
a713b37f35
fix(stream): add more logs.
2023-08-10 14:07:29 +08:00
f633c7fe2b
other: merge 3.0.
2023-08-10 11:56:30 +08:00
ef247cdb1d
support reopen stream state
2023-08-10 01:49:26 +00:00
c10b323188
other: merge main.
2023-08-10 09:26:33 +08:00
0cf8144952
fix(stream): fix the invalid key used by remove data in stream meta hash table.
2023-08-09 22:28:23 +08:00
d34b9af054
enh(stream): create update stream trans.
2023-08-09 19:38:13 +08:00
1e8579e8c5
fix(stream): fix other cases.
2023-08-09 16:04:48 +08:00
4c92997328
fix(stream): use streamId&taskId to identify the stream task in the stream meta.
2023-08-09 15:52:23 +08:00
f3c5f20ee2
enh(stream): add timer to check the node changing.
2023-08-09 10:25:16 +08:00
17c6143471
fix(stream): check more status when handling the state transfer.
2023-08-08 15:08:11 +08:00
a0dcec5890
enh(stream): add function handler for hb message.
2023-08-08 11:23:19 +08:00
f8ee055641
other: merge 3.0
2023-08-08 00:19:08 +08:00
80e78e054d
support reopen stream state
2023-08-07 11:39:32 +00:00
6c8c575549
support reopen stream state
2023-08-07 10:48:04 +00:00
aea1690cd1
fix(stream): fix the stream task after checking if it is in the timer activities.
2023-08-07 17:49:36 +08:00
b6c991f896
support reopen stream state
2023-08-07 08:59:04 +00:00
eeb97351e8
support reopen stream state
2023-08-07 08:51:01 +00:00
225c0e61dc
hen(stream): support the rollback and checkdown stream tasks.
2023-08-07 16:13:39 +08:00
a4da6e9a88
fix mem leak
2023-08-04 15:59:34 +08:00
81908be32f
add stream ver
2023-08-04 14:02:03 +08:00
6dc93a8dec
add stream ver
2023-08-04 13:52:20 +08:00
f9d0874e45
support reopen stream state
2023-08-03 08:24:50 +00:00
ad4c5916e7
enh(stream): update the stream task.
2023-08-01 18:14:58 +08:00
cc5ff44604
enh(stream): handle the stream hb.
2023-07-30 19:25:00 +08:00
8dfef8768a
other: merge 3.0
2023-07-29 22:54:16 +08:00
9f943213ed
enh(stream): update the hb info.
2023-07-28 14:23:53 +08:00
74834aea11
enh(stream): prepare hb to mnode from stream meta.
2023-07-28 13:07:16 +08:00
22a5a18fff
fix(stream): 1. set correct timewindow after step2. 2. handle the case when the task is failed to be added into the meta store.
2023-07-27 18:57:36 +08:00
cdffabcdad
fix(stream): fix memory leak.
2023-07-26 14:37:04 +08:00
063673726a
fix(stream): fix syntax error
2023-07-25 18:08:22 +08:00
2cc4e0a727
fix(stream): fix syntax error
2023-07-25 17:56:40 +08:00
0bdccd2681
refactor: do some internal refactor.
2023-07-25 17:31:54 +08:00
aed99da6c1
refactor: refactor the stream task starting order.
2023-07-20 12:14:53 +08:00
40d59990d8
fix mem leak
2023-07-18 04:01:20 +00:00
2f24f776fb
fix(stream): fix error in generating checkpoint for state_window.
2023-07-18 09:04:28 +08:00
b2a905bd91
Merge branch '3.0' into enh/triggerCheckPoint2
2023-07-17 11:50:12 +08:00
dea2e73d2e
add checkpoint
2023-07-17 02:22:21 +00:00
b4de0892e6
add checkpoint
2023-07-15 09:35:33 +00:00
e18c9ea672
add checkpoint
2023-07-15 04:26:34 +00:00
aad669e33c
add checkpoint
2023-07-14 14:46:13 +00:00
3020e27114
fix mem leak
2023-07-14 10:20:37 +00:00
3c4631ef1f
add checkpoint
2023-07-14 10:06:24 +00:00
2f90a53548
fix(stream): set checkpoint id when expanding task.
2023-07-13 10:16:57 +08:00
bb72f07a94
rm expire checkpoint
2023-07-12 16:38:46 +08:00
fed3803739
fix(stream): fix memory leak.
2023-07-11 19:29:52 +08:00
e547ed3699
rm expire checkpoint
2023-07-11 16:30:14 +08:00
d7101f7109
enh(stream): support dispatch checkpoint msg.
2023-07-06 18:26:22 +08:00
5a9461a69a
other: merge 3.0
2023-07-06 11:52:21 +08:00
f4a3567935
other: merge 3.0
2023-07-06 11:24:30 +08:00
03dbcbf055
refactor: do some internal refactor.
2023-07-05 15:55:55 +08:00
247d3ac7ee
vnode snapshot read
2023-06-30 12:05:05 +00:00
7d30a6e27a
vnode snapshot read
2023-06-30 11:48:46 +00:00
57fcd553f8
refactor: do some internal refactor.
2023-06-28 13:13:13 +08:00
a19e63fd23
fix(stream): fix error in handling fill history.
2023-06-27 23:00:14 +08:00
402c091df0
fix(stream): fix sma error.
2023-06-25 19:41:44 +08:00
0eeaab9cd3
Merge branch 'refact/fillhistory1' into refact/fillhistory
2023-06-19 13:53:37 +08:00
fd85a8495a
Merge branch 'dev30' into enh/triggerCheckPoint2
2023-06-17 14:17:21 +08:00
b6fe5d917b
refactor checkpoint
2023-06-17 14:05:49 +08:00
1e05f5c693
support fill history
2023-06-16 10:00:16 +08:00
0ef8afbdaa
support fill history
2023-06-16 09:32:38 +08:00
9f9171719a
support fill history
2023-06-15 21:29:52 +08:00
74e0138f72
Merge branch '3.0' into refact/fillhistory
2023-06-15 15:25:00 +08:00
52ca94ea6d
trigger checkpoint
2023-06-14 16:20:14 +08:00
b44447e6f3
enh(stream): support restore from disk.
2023-06-14 10:19:03 +08:00
53246ed6bb
other: merge other branch.
2023-06-09 09:41:10 +08:00
9828ca9d7f
add trigger checkpoint
2023-06-08 09:28:34 +00:00
06cf3588da
refactor: do some internal refactor.
2023-06-07 15:03:04 +08:00
66f483e51a
fix invalid free
2023-06-06 02:32:06 +00:00
0dd933013c
enh(stream): make history task for stream running.
2023-06-05 17:58:00 +08:00
fcc706c45c
enh(stream): refactor the fill history task.
2023-06-05 15:59:01 +08:00
74d5f02969
fix invalid free
2023-06-05 04:56:07 +00:00
1571ea844a
Avoid creating the same ID task multiple times
2023-06-01 11:25:55 +00:00
0315e89591
Avoid creating the same ID task multiple times
2023-06-01 11:13:01 +00:00
1e845acabe
fix(tmq): allow seek before get assignment
2023-05-26 11:34:12 +08:00
899e4d3350
refactor: do some internal refactor.
2023-05-20 18:13:58 +08:00
6ded3284da
refactor: do some internal refactor.
2023-05-20 11:37:17 +08:00
9c0d2c87c6
Merge branch 'enh/3.0' into enh/rocksdbSstateMerge
2023-05-15 01:52:42 +00:00
236926b9ee
other: merge enh/rocksdbstate
2023-05-13 19:49:28 +08:00
6cf1adf1b1
factor code
2023-05-11 13:01:45 +00:00
0710fc39af
factor code
2023-05-11 09:04:14 +00:00
f652f71d68
factor code
2023-05-11 06:37:57 +00:00
2b736ffd60
fix(stream): avoid duplicated task id in task id list.
2023-05-11 11:47:57 +08:00
c6e5879f44
Merge branch 'main' into enh/rocksdbSstate
2023-05-10 10:04:28 +08:00
8356533eb1
Merge branch 'enh/standVer' into enh/rocksdbSstateMerge
2023-05-09 12:13:16 +00:00
8224d49702
fix:after restarting taosd, stream does not work.
2023-05-09 17:24:11 +08:00
55014aa003
Merge branch 'enh/rocksdbSstate' into enh/rocksdbSstateMerge
2023-05-09 08:58:51 +00:00
b5fdaa327d
avoid deadlock
2023-05-09 07:00:29 +00:00
08c3a63341
Merge branch 'enh/standVer' into enh/rocksdbSstateMerge
2023-05-06 12:37:54 +00:00
8be29dc98e
failed to open rocksdb cf
2023-05-06 10:35:23 +00:00
aca5760ceb
fix(query): fix syntax error.
2023-05-06 13:17:31 +08:00
da6212bab0
other: merge main.
2023-05-06 12:51:38 +08:00
8958aabe4b
fix(stream): opt task scan performance.
2023-05-05 19:33:30 +08:00
f7a7743af7
fix case failure
2023-05-05 08:32:33 +00:00
8b12d4d3da
fix(stream): secure the delete task operation. TD-1198
2023-05-05 15:02:03 +08:00
2b7a816b86
fix(tmq): fix syntax error.
2023-05-04 16:59:26 +08:00
23c49594b7
other: merge main.
2023-05-04 16:41:53 +08:00
c27d01a510
Merge branch 'enh/addTtlToStream' into enh/rocksdbSstateMerge
2023-05-04 08:22:31 +00:00
3c61932ecc
refactor: do some internal refactor.
2023-05-01 19:28:54 +08:00
b69137df44
refactor(log): add some logs.
2023-04-29 00:00:25 +08:00
dda1c8803c
Merge branch 'enh/rocksdbSstate' into enh/rocksdbSstateMerge
2023-04-28 09:17:22 +00:00
36a4413bf1
merge rocksdb inst
2023-04-27 14:38:33 +00:00
b7040ed15e
enh(stream): support resume from the latest data, and do some internal refactor.
2023-04-27 15:07:54 +08:00
917b07348c
Merge branch 'fix/rocksdbState' into enh/rocksRevert
2023-04-23 02:37:26 +00:00
6c641cff39
fix(stream): fix error in start stream tasks.
2023-04-21 09:37:16 +08:00
5e6c06e253
fix(stream): fix the race condition during create new stream tasks.
2023-04-20 23:21:18 +08:00
b60b1796f7
fix(stream): add lock during check wal to create new stream task.
2023-04-20 15:53:41 +08:00
9f16802451
fix(stream): set the initial value.
2023-04-18 16:42:40 +08:00
76becf59b7
Merge pull request #20947 from taosdata/feature/3_liaohj
...
fix(stream): set the correct initial checkpoint versions.
2023-04-18 11:26:48 +08:00
930b267a75
fix(stream): set the correct start offset for stream task.
2023-04-17 22:59:00 +08:00
27d14f742d
fix:stream memory leak
2023-04-17 17:27:02 +08:00
a3b02a80c0
fix(stream): update the version when open stream tasks.
2023-04-15 00:10:08 +08:00
e05d61456a
fix(stream): disable the status check.
2023-04-14 22:45:22 +08:00
e0cb8aa534
fix(stream): don't the initial task status and do some internal refactor.
2023-04-14 17:43:37 +08:00
eb0e1f848a
fix(stream): remove unused tqreader, do some internal refactor, set the meta value for streamtask.
2023-04-14 12:00:09 +08:00
f8b672f09f
other: merge main.
2023-04-14 09:07:18 +08:00
70b45a4cf0
fix(stream): all data should be extracted from wal.
2023-04-13 23:22:09 +08:00
5ab54481f0
refactor: do some internal refactor.
2023-04-11 19:24:34 +08:00
18479d8115
refactor: do some internall refactor.
2023-04-11 15:42:24 +08:00
a4ba5401f9
enh(stream): set the start version of all operators.
2023-04-11 10:01:36 +08:00
7419c0bfe9
enh(stream): the stream will start after vnode restore being completed.
2023-04-10 16:56:14 +08:00
f5b52749c2
refactor: do some internal refactor.
2023-04-09 01:39:09 +08:00
e2646734ad
enh: add queue limit
2023-03-15 12:01:16 +00:00
ef092c1cce
enh: add queue limit
2023-03-15 11:54:36 +00:00
22aa03307c
fix(query): fix syntax errors on centos.
2023-02-24 11:38:54 +08:00
1d58f85a54
feat:coverity scan
2023-02-22 10:10:04 +08:00
2a4708d82a
fix: save fill history state
2023-02-16 17:01:09 +08:00
0cd6bd3d86
Merge branch 'main' into enh/stream_buffer_param
2023-02-13 09:48:02 +08:00
c8acfc4d54
fix: stream meta
2023-02-10 10:48:26 +08:00
98906bb272
remove assert
2023-01-29 15:24:31 +08:00
0c6f6d3a42
fix: task db
2023-01-28 10:59:09 +08:00
4031dc1af4
Merge branch main into FIX/TD-21043-main
2023-01-02 16:41:46 +08:00
3edde0eadf
fix: tdbAbort on metaClose or streamMetaClose instead of tdbTxnClose
2023-01-01 21:07:32 +08:00
4601e67e67
fix: reset stream status when load from disk
2022-12-30 16:59:48 +08:00
6fc47beb71
enh: streamMetaCommit in sync with vnodeCommit
2022-12-28 14:34:23 +08:00
6cb65f336d
Merge branch 'v3.0' into feat/async_commit
2022-12-02 14:49:58 +08:00
3424b184d3
refactor(stream): distributed checkpoint
2022-12-02 03:48:45 +08:00
78335f83d2
tdb/api: migrate txn related api changes of meta, sma, tq, and stream meta/state
2022-11-17 20:02:13 +08:00
015c00aaca
fix(stream): timer refer stream task
2022-11-16 11:16:32 +08:00
a9bf05bc01
refactor: stream meta ref count
2022-11-15 19:11:42 +08:00
64f22d560c
refactor(stream): drop stream task
2022-11-10 16:36:49 +08:00
1a4cf6964b
fix: add post commit to tq & stream
2022-11-07 20:34:23 +08:00
8d8fd2b2bc
refactor(stream): recover and fill history
2022-10-26 10:20:33 +08:00
52b4b510a8
merge 3.0
2022-10-26 09:18:29 +08:00
75e5e490fd
refactor(stream): recover and fill history
2022-10-26 00:31:00 +08:00
5571ebee0f
Merge pull request #17542 from taosdata/feature/stream
...
enh(stream): internal optimize
2022-10-21 14:31:42 +08:00
26d4af6968
fix: memory leak
2022-10-21 11:42:30 +08:00
e0b5b9263e
refactor: remove unused code
2022-10-21 10:42:24 +08:00
035e6b13c8
enh(stream): internal optimize
2022-10-21 10:29:05 +08:00
86a858e04d
fix(stream): coverity scan
2022-10-20 11:41:20 +08:00
eb7de2e571
Merge pull request #17443 from taosdata/fix/commit_txn
...
fix: commit txn
2022-10-19 13:15:15 +08:00
7f16081e29
feat(stream): stream task and meta
2022-10-19 10:49:42 +08:00
82132cdfdd
meta: new metaFinishCommit api & rollback parameter with metaOpen
2022-10-18 18:34:39 +08:00
a57a305035
fix(stream): memory leak
2022-10-14 14:43:22 +08:00
d05b134a48
os: Mac package ( #17201 )
...
* os: Mac package
* os: Mac package
* os: Mac package
* os: Mac package
* os: Mac package
* os: Mac package
* os: Mac package
* os: Mac package
* os: Mac package
* os: Mac package
* os: Mac package
* os: Mac package
2022-10-08 10:12:17 +08:00
04e5c75bba
fix: cleanup memory with streamLoadTasks
2022-08-26 14:51:30 +08:00
d7b804e02e
fix stream load task
2022-08-24 10:42:34 +08:00
c296dd0a63
fix compile
2022-08-24 01:48:00 +08:00
2d628491b2
feat(stream): support tdb state backend
2022-08-23 19:30:04 +08:00
963287798d
refactor(mnode): drop stream task
2022-08-17 23:18:39 +08:00
853e6e2988
refactor(mnode): drop stream task
2022-08-17 19:19:58 +08:00
357b21cfeb
refactor(tmq): use tdb to store check info
2022-08-17 13:58:56 +08:00
ad032a0a90
enh(stream): stream recover
2022-08-09 10:36:27 +08:00
bb0ce39511
refactor(stream): add stream meta
2022-08-08 15:05:54 +08:00
43026ac31d
refactor(stream)
2022-08-08 11:29:50 +08:00
7c9d57bd89
refactor(stream): unify sink and dispatch
2022-08-05 20:16:47 +08:00
d43eec5d16
enh(stream): directly dispatch
2022-08-02 22:23:33 +08:00
Haojun Liao
Minglei Jin
dapan1121
yihaoDeng
liuyao
Ganlin Zhao
yihaoDeng
Haojun Liao
Liu Jicong
Benguang Zhao
Hongze Cheng
Zhiqiang Wang