Commit Graph

207 Commits

Author SHA1 Message Date
Haojun Liao ea6574b3c7 other: merge 3.0 2023-08-17 00:07:55 +08:00
Haojun Liao ec84d7fd81 fix(stream): add the back pressure for sink node. 2023-08-16 19:00:35 +08:00
Haojun Liao 0acb00c516 fix(stream): add log. 2023-08-15 11:48:56 +08:00
Haojun Liao b1d35f43ba enh(stream): using term from sync module to denote the upstream node is restarted/replica-modified/vnode-transferred or not. 2023-08-15 09:25:05 +08:00
Haojun Liao 0d38f389ab refactor: do some internal refactor. 2023-08-13 16:25:35 +08:00
Haojun Liao 0e189f70a1 refactor 2023-08-12 19:44:44 +08:00
Haojun Liao 09da6c6840 refactor: do some internal refactor. 2023-08-12 18:51:20 +08:00
Haojun Liao 75e6fb0f16 refactor: do some internal refactor. 2023-08-12 18:47:01 +08:00
Haojun Liao 075b5e9481 refactor: do some internal refactor. 2023-08-12 18:41:54 +08:00
Haojun Liao a0534ee715 refactor: do some internal refactor. 2023-08-12 18:40:09 +08:00
Haojun Liao 8f9de93cf0 refactor: do some internal refactor. 2023-08-12 18:37:36 +08:00
Haojun Liao 30c2a9c619 refactor: do some internal refactor. 2023-08-12 18:27:51 +08:00
Haojun Liao ce721a0146 fix(stream): fix error. 2023-08-12 16:32:05 +08:00
Haojun Liao 8dfef8768a other: merge 3.0 2023-07-29 22:54:16 +08:00
Haojun Liao 0bd2548937 fix(stream): initialize the filter window initial range. 2023-07-29 16:41:01 +08:00
Haojun Liao 74834aea11 enh(stream): prepare hb to mnode from stream meta. 2023-07-28 13:07:16 +08:00
Haojun Liao e89f530db2 fix(stream): scan wal in step2 2023-07-27 10:55:59 +08:00
Haojun Liao 0bdccd2681 refactor: do some internal refactor. 2023-07-25 17:31:54 +08:00
Haojun Liao d29f835a63 fix(stream): set the correct step2 scan time window range. 2023-07-25 15:09:22 +08:00
Haojun Liao 1e34f84f78 other: merge 3.0 2023-07-24 15:23:03 +08:00
Haojun Liao 4e0f7ffbf8 fix(stream): set the correct flag of pause/resume. 2023-07-23 18:49:56 +08:00
Haojun Liao 16d7707b90 fix(stream): align the scan real time data for stream task. 2023-07-21 19:33:43 +08:00
Haojun Liao 97c44e7c18 refactor: do some internal refactor. 2023-07-19 18:42:02 +08:00
Haojun Liao 902f9d1f94 fix(stream): set not-ready initial value. 2023-07-18 18:29:35 +08:00
Haojun Liao 2f24f776fb fix(stream): fix error in generating checkpoint for state_window. 2023-07-18 09:04:28 +08:00
Haojun Liao b2a905bd91
Merge branch '3.0' into enh/triggerCheckPoint2 2023-07-17 11:50:12 +08:00
Haojun Liao 8c79157822 fix(stream): close the inputQ if checkpoint msg received. 2023-07-14 18:35:41 +08:00
Haojun Liao eb7e6152ae refactor(stream): do some internal refactor. 2023-07-14 17:21:20 +08:00
Haojun Liao 813f4cb363 refactor: do some internal refactor. 2023-07-13 16:32:25 +08:00
Haojun Liao 08a4fb06ae fix(stream): dispatch checkpoint msg to downstream by puting message into input queue. 2023-07-13 09:17:20 +08:00
Haojun Liao d392bb60ba refactor(stream): set the initial version from the checkpoint. 2023-07-12 11:38:13 +08:00
Haojun Liao fed3803739 fix(stream): fix memory leak. 2023-07-11 19:29:52 +08:00
yihaoDeng e263d5f55f add checkpoint 2023-07-10 17:33:48 +08:00
Haojun Liao 47877898d0 enh(stream): generate the checkpoint framework. 2023-07-07 16:07:52 +08:00
Haojun Liao d7101f7109 enh(stream): support dispatch checkpoint msg. 2023-07-06 18:26:22 +08:00
Haojun Liao 03dbcbf055 refactor: do some internal refactor. 2023-07-05 15:55:55 +08:00
Haojun Liao 77530e1956 refactor: record the downstream input Queue blocking time. 2023-07-04 10:58:11 +08:00
Haojun Liao 2dbccc1573 refactor: do some internal refactor. 2023-07-03 17:52:17 +08:00
Haojun Liao 78739e4bdb fix(stream): fix syntax error. 2023-06-28 14:21:11 +08:00
Haojun Liao 57fcd553f8 refactor: do some internal refactor. 2023-06-28 13:13:13 +08:00
Haojun Liao a19e63fd23 fix(stream): fix error in handling fill history. 2023-06-27 23:00:14 +08:00
Haojun Liao 32fddeff18 fix(stream): fix error retrieve data from source task.And optimize the test cases. 2023-06-27 16:24:47 +08:00
Haojun Liao b73444b291 fix(stream): start stream task in case of scan history completing. 2023-06-27 09:44:51 +08:00
Haojun Liao 64365666b7 fix(stream): update the info, and do some internal refactor. 2023-06-16 16:04:43 +08:00
Haojun Liao fb24ed161d enh(stream): set correct task status. 2023-06-12 22:48:23 +08:00
Haojun Liao 69c9eda7af fix(stream): fix race condition. 2023-06-09 11:37:38 +08:00
Haojun Liao 6c13491428 refactor: do some internal refactor. 2023-06-08 16:55:41 +08:00
Haojun Liao cb26dd9fa2 refactor(stream): do some internal refactor. 2023-06-08 10:17:21 +08:00
Haojun Liao 06cf3588da refactor: do some internal refactor. 2023-06-07 15:03:04 +08:00
Haojun Liao 0dd933013c enh(stream): make history task for stream running. 2023-06-05 17:58:00 +08:00
Haojun Liao e8549ce511 refactor: do some internal refactor. 2023-06-05 16:19:37 +08:00
Haojun Liao 27b7d1ec88 refactor: do some internal refactor. 2023-06-02 11:17:44 +08:00
Haojun Liao a3abe25539 refactor: do some internal refactor. 2023-05-31 19:03:37 +08:00
Haojun Liao 65b81a803f refactor: do some internal refactor and add some logs. 2023-05-31 18:05:53 +08:00
Haojun Liao fc20ca8423 fix(stream): set the correct rps flag as the dispatch rsp. 2023-05-30 18:42:23 +08:00
Haojun Liao d9cc9f2157 fix(stream): remove invalid free. 2023-05-30 18:03:21 +08:00
Haojun Liao 937f67f07b fix(stream): fix invalid free. 2023-05-30 18:02:42 +08:00
Haojun Liao e96ed81ec0 fix(stream): add input queue capacity check. 2023-05-30 17:29:01 +08:00
Haojun Liao de7231e59e fix(stream): fix memory leak in handling dispatch msg when output buffer is full. 2023-05-30 15:35:59 +08:00
Haojun Liao dcb91082f9 fix(stream): fix error. 2023-05-20 23:37:40 +08:00
Haojun Liao de44c9160c refactor: do some internal refactor. 2023-05-20 18:05:39 +08:00
Haojun Liao 3cef1e7c42 fix(stream): add some logs. 2023-05-20 15:45:07 +08:00
Haojun Liao aa7ea60bca fix(stream): update the free strategy. 2023-05-20 15:43:48 +08:00
Haojun Liao 336102b8c5 refactor: do some internal refactor. 2023-05-20 13:34:49 +08:00
Haojun Liao 6ded3284da refactor: do some internal refactor. 2023-05-20 11:37:17 +08:00
Haojun Liao a9b7b8a5fd other: add some logs. 2023-05-19 19:26:15 +08:00
Haojun Liao 90687bae1c refactor: do some internal refactor. 2023-05-19 15:30:50 +08:00
yihaoDeng 9380b22394 merge 3.0 2023-05-18 05:46:38 +00:00
Haojun Liao 48dd3753aa
Merge pull request #21353 from taosdata/feature/3_liaohj
merge main, and do some internal refactor.
2023-05-17 22:34:34 +08:00
Haojun Liao 92feb89a09 refactor: do some internal refactor. 2023-05-17 18:08:04 +08:00
Haojun Liao 041ffa91bd other: merge main. 2023-05-17 15:42:15 +08:00
liuyao 695de9d3ac fix stream queue mem leak 2023-05-17 15:19:14 +08:00
Haojun Liao e1700d00dc fix(stream): avoid memory leak. 2023-05-16 09:26:19 +08:00
Haojun Liao 4c64260a7a refactor: do internal refactor. 2023-05-15 14:15:10 +08:00
Haojun Liao 3b6e052c6b enh(stream): avoid clone submit block. 2023-05-15 10:11:08 +08:00
Haojun Liao 3d00f6ce54 refactor: do some internal refactor. 2023-05-14 22:08:17 +08:00
Haojun Liao 8e15c64499 refactor: do some internal refactor. 2023-05-13 23:22:36 +08:00
Haojun Liao 8277d608f4 other: update logs. 2023-05-11 16:53:19 +08:00
Haojun Liao dbd79cc80c refactor: do some internal refactor. 2023-05-10 22:22:01 +08:00
Haojun Liao 31ac1e3eaa fix(stream): revise the input queue capacity. 2023-05-08 00:00:55 +08:00
Haojun Liao d026e7ef16 refactor: update some logs. 2023-05-07 22:55:49 +08:00
Haojun Liao c141abfbb8 fix(stream): set correct msg size; 2023-05-06 20:03:07 +08:00
Haojun Liao 13f3ca4226 fix(stream): add input queue size limitation. 2023-05-06 18:47:29 +08:00
Haojun Liao da6212bab0 other: merge main. 2023-05-06 12:51:38 +08:00
liuyao 74ab689719 opt batch num 2023-05-05 08:59:07 +08:00
liuyao d8a1f15d10 fix resum bug 2023-05-04 09:28:06 +08:00
Haojun Liao 3b814f2478 fix(stream): do some internal refactor. 2023-05-01 23:21:29 +08:00
Haojun Liao 777ed1761f refactor: update logs. 2023-04-29 18:24:46 +08:00
Haojun Liao bbb8aaa0dc refactor: add some logs. 2023-04-29 02:37:10 +08:00
Haojun Liao 6780833cd2 Merge remote-tracking branch 'origin/enh/rocksdbSstate' into enh/rocksdbSstate 2023-04-27 15:08:17 +08:00
Haojun Liao b7040ed15e enh(stream): support resume from the latest data, and do some internal refactor. 2023-04-27 15:07:54 +08:00
liuyao 8b5251004f inc queue cap 2023-04-27 14:04:46 +08:00
liuyao dbdc31539c inc queue cap 2023-04-27 10:46:08 +08:00
liuyao 4c4a4775de opt stream 2023-04-26 19:13:58 +08:00
Haojun Liao 48c36a8c35 Merge branch 'main' into enh/rocksdbSstate 2023-04-24 14:44:13 +08:00
Haojun Liao 623ea6df5f fix(stream): add task status check. 2023-04-23 17:42:56 +08:00
yihaoDeng 917b07348c Merge branch 'fix/rocksdbState' into enh/rocksRevert 2023-04-23 02:37:26 +00:00
Haojun Liao dc733352db enh(stream): set the max input queue size to be 3000. 2023-04-20 17:38:10 +08:00
Haojun Liao ac137b4b33 refactor: do some internal refactor. 2023-04-17 09:46:58 +08:00
Haojun Liao 10b3fd9426 other: merge main. 2023-04-17 09:19:48 +08:00