Commit Graph

248 Commits

Author SHA1 Message Date
Haojun Liao b0c5fa8290 fix(stream): reset the task status. 2023-08-23 14:10:41 +08:00
Haojun Liao c26d625152
Merge branch '3.0' into enh/triggerCheckPoint2 2023-08-18 15:51:07 +08:00
Haojun Liao edbad45772 other: add logs. 2023-08-18 13:45:08 +08:00
Haojun Liao bb4f8515bd fix(stream): pause when outputQ is blocked. 2023-08-17 11:53:17 +08:00
Haojun Liao ea6574b3c7 other: merge 3.0 2023-08-17 00:07:55 +08:00
Haojun Liao 0d38f389ab refactor: do some internal refactor. 2023-08-13 16:25:35 +08:00
Haojun Liao 03c26a9d6d refactor 2023-08-12 19:16:20 +08:00
Haojun Liao 30c2a9c619 refactor: do some internal refactor. 2023-08-12 18:27:51 +08:00
Haojun Liao 8a11a0418f other: merge 3.0 2023-08-07 09:19:26 +08:00
Haojun Liao d45596bb33 fix(stream): kill task when pause the stream execution. 2023-07-31 18:07:37 +08:00
Haojun Liao 8dfef8768a other: merge 3.0 2023-07-29 22:54:16 +08:00
Haojun Liao 1e34f84f78 other: merge 3.0 2023-07-24 15:23:03 +08:00
Haojun Liao f4864b8713 fix(stream): fix memory leak. 2023-07-22 01:02:58 +08:00
Haojun Liao 16d7707b90 fix(stream): align the scan real time data for stream task. 2023-07-21 19:33:43 +08:00
yihaoDeng aad669e33c add checkpoint 2023-07-14 14:46:13 +00:00
Haojun Liao 2b85ea62aa refactor: do some internal refactor. 2023-07-14 17:48:47 +08:00
Haojun Liao eb7e6152ae refactor(stream): do some internal refactor. 2023-07-14 17:21:20 +08:00
Haojun Liao 08a4fb06ae fix(stream): dispatch checkpoint msg to downstream by puting message into input queue. 2023-07-13 09:17:20 +08:00
Haojun Liao d392bb60ba refactor(stream): set the initial version from the checkpoint. 2023-07-12 11:38:13 +08:00
Haojun Liao a049e310f6 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-07-12 10:16:26 +08:00
Haojun Liao 6f2fc4fab3 fix(stream): commit task meta after do vnode-wide checkpoint. 2023-07-12 10:16:14 +08:00
yihaoDeng e547ed3699 rm expire checkpoint 2023-07-11 16:30:14 +08:00
Haojun Liao 87bedef2cd fix(stream): fix error in checkpointing. 2023-07-10 19:20:05 +08:00
Haojun Liao a5c19427e7 fix(stream): send correct rsp to mnode. 2023-07-08 09:39:44 +08:00
Haojun Liao 47877898d0 enh(stream): generate the checkpoint framework. 2023-07-07 16:07:52 +08:00
Haojun Liao d7101f7109 enh(stream): support dispatch checkpoint msg. 2023-07-06 18:26:22 +08:00
Haojun Liao 5a9461a69a other: merge 3.0 2023-07-06 11:52:21 +08:00
Haojun Liao f4a3567935 other: merge 3.0 2023-07-06 11:24:30 +08:00
Haojun Liao 03dbcbf055 refactor: do some internal refactor. 2023-07-05 15:55:55 +08:00
yihaoDeng 7d30a6e27a vnode snapshot read 2023-06-30 11:48:46 +00:00
yihaoDeng 5ccfae87d9 add checkpoint 2023-06-19 20:49:04 +08:00
yihaoDeng 548e439aa2 add checkpoint 2023-06-19 20:48:49 +08:00
yihaoDeng 0ef8afbdaa support fill history 2023-06-16 09:32:38 +08:00
yihaoDeng 9f9171719a support fill history 2023-06-15 21:29:52 +08:00
Yihao Deng 7ded6fc7a2
Merge branch '3.0' into enh/triggerCheckPoint2 2023-06-14 16:30:38 +08:00
Haojun Liao 11f0c3b336 refactor: do some internal refactor. 2023-06-14 10:46:46 +08:00
Haojun Liao 53246ed6bb other: merge other branch. 2023-06-09 09:41:10 +08:00
yihaoDeng d5c0ca3163 opt batch write 2023-06-08 13:36:52 +00:00
yihaoDeng 9828ca9d7f add trigger checkpoint 2023-06-08 09:28:34 +00:00
Haojun Liao cb26dd9fa2 refactor(stream): do some internal refactor. 2023-06-08 10:17:21 +08:00
Haojun Liao 06cf3588da refactor: do some internal refactor. 2023-06-07 15:03:04 +08:00
Haojun Liao 6a3c935b4a fix(stream): fix error in fill history scan. 2023-06-07 09:15:55 +08:00
Haojun Liao 0dd933013c enh(stream): make history task for stream running. 2023-06-05 17:58:00 +08:00
yihaoDeng 74d5f02969 fix invalid free 2023-06-05 04:56:07 +00:00
Haojun Liao e67b532d8a refactor: refactor the module dependency. 2023-05-23 18:29:23 +08:00
Haojun Liao de44c9160c refactor: do some internal refactor. 2023-05-20 18:05:39 +08:00
Haojun Liao aa7ea60bca fix(stream): update the free strategy. 2023-05-20 15:43:48 +08:00
Haojun Liao 336102b8c5 refactor: do some internal refactor. 2023-05-20 13:34:49 +08:00
Haojun Liao 90687bae1c refactor: do some internal refactor. 2023-05-19 15:30:50 +08:00
yihaoDeng eb3ab4fcea factor code 2023-05-11 13:34:17 +00:00
yihaoDeng 4e8f5f5d32 factor code 2023-05-11 09:58:53 +00:00
yihaoDeng afaee9c386 factor code 2023-05-11 09:51:01 +00:00
yihaoDeng 4715aacd41 factor code 2023-05-11 09:44:40 +00:00
yihaoDeng 04c7c99a67 factor code 2023-05-11 09:17:05 +00:00
yihaoDeng 8be29dc98e failed to open rocksdb cf 2023-05-06 10:35:23 +00:00
yihaoDeng d7e86aca05 fix case failure 2023-05-05 15:14:16 +00:00
Haojun Liao 23c49594b7 other: merge main. 2023-05-04 16:41:53 +08:00
yihaoDeng c27d01a510 Merge branch 'enh/addTtlToStream' into enh/rocksdbSstateMerge 2023-05-04 08:22:31 +00:00
Haojun Liao 3c61932ecc refactor: do some internal refactor. 2023-05-01 19:28:54 +08:00
yihaoDeng 36a4413bf1 merge rocksdb inst 2023-04-27 14:38:33 +00:00
yihaoDeng 2d5218246d add ttl to stream state key 2023-04-24 08:18:02 +00:00
yihaoDeng 3abdbec813 add ttl to stream state key 2023-04-24 08:17:57 +00:00
yihaoDeng b640a0330d change write opt 2023-04-19 23:55:09 +00:00
yihaoDeng efa7b48c1b Merge branch 'enh/rocksRevert' of https://github.com/taosdata/TDengine into enh/rocksRevert 2023-04-14 09:40:25 +00:00
yihaoDeng 4c70cfdfad startGroupTableMe 2023-04-14 09:36:08 +00:00
Haojun Liao 5850a3ab01 Merge remote-tracking branch 'origin/enh/rocksRevert' into enh/rocksRevert 2023-04-14 10:37:17 +08:00
Haojun Liao b57c813681 other: merge feature/3_liaohj 2023-04-14 10:36:54 +08:00
yihaoDeng c0ebdb92d3 startGroupTableMe 2023-04-14 02:10:33 +00:00
yihaoDeng 36eef2fdef refactor code 2023-04-13 09:57:04 +00:00
yihaoDeng ed4aaade8c add backend 2023-04-12 03:03:51 +00:00
yihaoDeng ff8cb09f63 refactor backend 2023-04-11 10:47:15 +00:00
Haojun Liao f5b52749c2 refactor: do some internal refactor. 2023-04-09 01:39:09 +08:00
yihaoDeng c082536242 opt write snapshot 2023-04-07 07:26:15 +00:00
yihaoDeng b701198dbb add backend 2023-03-22 13:19:56 +00:00
Liu Jicong 3424b184d3 refactor(stream): distributed checkpoint 2022-12-02 03:48:45 +08:00
Liu Jicong 48894c8bdb enh(stream): new api for stream queue 2022-11-25 18:03:40 +08:00
Liu Jicong bc469f7f7b fix(stream): delete tb should be checked in write thread 2022-11-08 11:30:29 +08:00
Liu Jicong 2fbc8e6306 refactor(stream): stream deploy and state transfer 2022-11-03 13:22:23 +08:00
Liu Jicong 5e253cc8ea refactor(stream): fill history 2022-11-01 11:56:14 +08:00
Liu Jicong 75e5e490fd refactor(stream): recover and fill history 2022-10-26 00:31:00 +08:00
Liu Jicong 853e6e2988 refactor(mnode): drop stream task 2022-08-17 19:19:58 +08:00
Liu Jicong 18f6fb6123 enh(tmq): speed up consumer recover 2022-08-10 16:04:03 +08:00
Liu Jicong fc225a580d enh(stream): recover process 2022-08-09 21:37:21 +08:00
Liu Jicong a9d11b58ad enh(stream): recover 2022-07-28 16:19:50 +08:00
Liu Jicong 3fa5791889 fix(stream): dispatch 2022-07-27 10:52:25 +08:00
jiacy-jcy 5200a48107 fix: update 2022-07-19 16:54:19 +08:00
Liu Jicong 280ed4f65d feat(stream): pipelined exec for recovering 2022-07-15 21:29:53 +08:00
Liu Jicong 8a6e4b877a refactor(stream): simple batch 2022-07-13 16:37:33 +08:00
Liu Jicong 8c5dbde918 refactor(stream): stream reader created in scanner 2022-07-06 14:20:07 +08:00
Liu Jicong f2f5d68f3b fix(stream): memory leak 2022-06-23 22:05:00 +08:00
Liu Jicong fb596625d3 feat(stream): support retrieve 2022-06-22 19:12:31 +08:00
Liu Jicong 8eb08fa0d4 feat(stream): trigger mode add max delay 2022-06-15 16:11:20 +08:00
Liu Jicong cbbf5f5d4d refactor(stream) 2022-06-07 21:35:49 +08:00
Liu Jicong 46bea47b62 enh(stream): direct sink if not dispatch 2022-06-07 19:58:06 +08:00
Liu Jicong 1a84d07188 fix(stream): msg deserialize 2022-06-07 11:36:40 +08:00
Liu Jicong 1b26da1bae refactor(stream): distributed execution 2022-06-06 22:18:50 +08:00
Liu Jicong 2677a3c8d9 refactor(stream) 2022-06-06 20:15:36 +08:00
Liu Jicong 59a4eec186 stream extract to module 2022-03-25 20:19:09 +08:00