Commit Graph

209 Commits

Author SHA1 Message Date
Haojun Liao 4c92997328 fix(stream): use streamId&taskId to identify the stream task in the stream meta. 2023-08-09 15:52:23 +08:00
Haojun Liao 3cdb16352c fix(stream): avoid launching check downstream when failed to add stream task into stream meta. 2023-08-09 12:03:14 +08:00
Haojun Liao 225c0e61dc hen(stream): support the rollback and checkdown stream tasks. 2023-08-07 16:13:39 +08:00
Haojun Liao 8a11a0418f other: merge 3.0 2023-08-07 09:19:26 +08:00
liuyao 32b59af4b1 check update data 2023-07-31 17:32:24 +08:00
Haojun Liao 8dfef8768a other: merge 3.0 2023-07-29 22:54:16 +08:00
Haojun Liao 0bd2548937 fix(stream): initialize the filter window initial range. 2023-07-29 16:41:01 +08:00
Haojun Liao 22a5a18fff fix(stream): 1. set correct timewindow after step2. 2. handle the case when the task is failed to be added into the meta store. 2023-07-27 18:57:36 +08:00
Haojun Liao e89f530db2 fix(stream): scan wal in step2 2023-07-27 10:55:59 +08:00
Haojun Liao cdffabcdad fix(stream): fix memory leak. 2023-07-26 14:37:04 +08:00
Haojun Liao 0bdccd2681 refactor: do some internal refactor. 2023-07-25 17:31:54 +08:00
Haojun Liao 1e34f84f78 other: merge 3.0 2023-07-24 15:23:03 +08:00
Haojun Liao 1b2636028a fix(stream): fix memory leak. 2023-07-21 23:05:42 +08:00
Haojun Liao 16d7707b90 fix(stream): align the scan real time data for stream task. 2023-07-21 19:33:43 +08:00
Haojun Liao aed99da6c1 refactor: refactor the stream task starting order. 2023-07-20 12:14:53 +08:00
Haojun Liao 97c44e7c18 refactor: do some internal refactor. 2023-07-19 18:42:02 +08:00
Haojun Liao 2ae95217bd fix(stream): open inputQ by default for task deployed on snode. 2023-07-19 10:05:35 +08:00
Haojun Liao 901b7d8ecc fix(stream): if a source task is set to be non-normal, no data should be put into inputQ anymore. 2023-07-18 15:18:57 +08:00
Haojun Liao 813f4cb363 refactor: do some internal refactor. 2023-07-13 16:32:25 +08:00
Haojun Liao 2f90a53548 fix(stream): set checkpoint id when expanding task. 2023-07-13 10:16:57 +08:00
Haojun Liao 09b764494d fix(stream): set correct checkpoint version when starting stream tasks. 2023-07-11 14:03:43 +08:00
Haojun Liao 5a9461a69a other: merge 3.0 2023-07-06 11:52:21 +08:00
Haojun Liao f4a3567935 other: merge 3.0 2023-07-06 11:24:30 +08:00
Haojun Liao 03dbcbf055 refactor: do some internal refactor. 2023-07-05 15:55:55 +08:00
yihaoDeng 0bf6675f39 add checkpoint 2023-06-28 09:58:29 +00:00
Haojun Liao 57fcd553f8 refactor: do some internal refactor. 2023-06-28 13:13:13 +08:00
Haojun Liao b73444b291 fix(stream): start stream task in case of scan history completing. 2023-06-27 09:44:51 +08:00
liuyao e236bf818c set fill history info 2023-06-25 17:47:10 +08:00
Haojun Liao 64365666b7 fix(stream): update the info, and do some internal refactor. 2023-06-16 16:04:43 +08:00
liuyao 693942b821 trans state 2023-06-15 14:13:17 +08:00
Haojun Liao 11f0c3b336 refactor: do some internal refactor. 2023-06-14 10:46:46 +08:00
Haojun Liao 78a240be8f enh(stream): add dispatch msg. 2023-06-12 13:12:45 +08:00
Haojun Liao 06cf3588da refactor: do some internal refactor. 2023-06-07 15:03:04 +08:00
Haojun Liao 1090afd108 fix(stream): add new file. 2023-05-25 18:46:53 +08:00
Haojun Liao c3be7b14be fix: set correct function ptr. 2023-05-25 17:51:03 +08:00
Haojun Liao da9f33a483 fix: fix error in set function ptr. 2023-05-25 09:56:34 +08:00
Haojun Liao fe09705855 fix: set the init function pointer. 2023-05-24 17:13:20 +08:00
Haojun Liao e67b532d8a refactor: refactor the module dependency. 2023-05-23 18:29:23 +08:00
Haojun Liao dcb91082f9 fix(stream): fix error. 2023-05-20 23:37:40 +08:00
Haojun Liao de44c9160c refactor: do some internal refactor. 2023-05-20 18:05:39 +08:00
yihaoDeng 08c3a63341 Merge branch 'enh/standVer' into enh/rocksdbSstateMerge 2023-05-06 12:37:54 +00:00
Haojun Liao 8958aabe4b fix(stream): opt task scan performance. 2023-05-05 19:33:30 +08:00
Haojun Liao ab4b08a4b5 Merge branch 'feature/3_liaohj' into enh/rocksRevert 2023-04-14 17:44:50 +08:00
Haojun Liao eb0e1f848a fix(stream): remove unused tqreader, do some internal refactor, set the meta value for streamtask. 2023-04-14 12:00:09 +08:00
Haojun Liao b57c813681 other: merge feature/3_liaohj 2023-04-14 10:36:54 +08:00
Haojun Liao 70b45a4cf0 fix(stream): all data should be extracted from wal. 2023-04-13 23:22:09 +08:00
Haojun Liao 5ab54481f0 refactor: do some internal refactor. 2023-04-11 19:24:34 +08:00
yihaoDeng b24b91886a Merge branch 'dev3' into enh/rocksRevert 2023-04-11 10:59:00 +00:00
Haojun Liao 18479d8115 refactor: do some internall refactor. 2023-04-11 15:42:24 +08:00
Haojun Liao 15cceb5a5a refactor: do some internal refactors. 2023-04-11 11:53:10 +08:00
Haojun Liao f5b52749c2 refactor: do some internal refactor. 2023-04-09 01:39:09 +08:00
yihaoDeng 628bb62c8a add backpressure 2023-04-04 07:20:05 +00:00
dapan1121 5b02af6a91 fix: merge main to 3.0 2023-03-22 16:08:16 +08:00
Haojun Liao ae5155edcf fix(tmq): do some internal refactor. 2023-03-19 10:49:30 +08:00
Haojun Liao 22aa03307c fix(query): fix syntax errors on centos. 2023-02-24 11:38:54 +08:00
Liu Jicong 869ab0d394 fix: snode timer 2023-02-01 17:27:56 +08:00
Benguang Zhao 6fc47beb71 enh: streamMetaCommit in sync with vnodeCommit 2022-12-28 14:34:23 +08:00
Liu Jicong 3424b184d3 refactor(stream): distributed checkpoint 2022-12-02 03:48:45 +08:00
Liu Jicong a9bf05bc01 refactor: stream meta ref count 2022-11-15 19:11:42 +08:00
Liu Jicong 31f574359d fix:log empty msg 2022-11-03 14:29:51 +08:00
Liu Jicong 2fbc8e6306 refactor(stream): stream deploy and state transfer 2022-11-03 13:22:23 +08:00
Liu Jicong 2e640c38a2 refactor: stream and tmq message 2022-10-31 18:37:28 +08:00
Liu Jicong b26cc9c318 feat(snode) 2022-10-28 17:03:17 +08:00
Liu Jicong 7c9d57bd89 refactor(stream): unify sink and dispatch 2022-08-05 20:16:47 +08:00
Liu Jicong 08a4a7ab0f fix 2022-08-02 23:44:47 +08:00
Liu Jicong d43eec5d16 enh(stream): directly dispatch 2022-08-02 22:23:33 +08:00
Liu Jicong a9d11b58ad enh(stream): recover 2022-07-28 16:19:50 +08:00
Liu Jicong 280ed4f65d feat(stream): pipelined exec for recovering 2022-07-15 21:29:53 +08:00
Shengliang Guan a5e8ddbc56 refactor: adjust logs 2022-07-08 18:00:19 +08:00
54liuyao 10e90ce973 feat(stream): auto pull data 2022-06-25 16:44:14 +08:00
Liu Jicong fb596625d3 feat(stream): support retrieve 2022-06-22 19:12:31 +08:00
Liu Jicong c409760f75 feat(stream): drop stream 2022-06-21 11:32:36 +08:00
Liu Jicong 450675f672 feat(stream): support snode 2022-06-20 20:18:34 +08:00
Liu Jicong 53ef66961a feat(stream): support snode 2022-06-20 14:29:18 +08:00
Liu Jicong 94d7cbe622 feat(stream): snode process stream msg 2022-06-07 16:24:50 +08:00
Liu Jicong d5d8581bcf refactor(stream): remove out-of-date msg 2022-05-26 21:45:59 +08:00
Liu Jicong a393dee9d2 refactor(stream) 2022-05-23 16:51:12 +08:00
Hongze Cheng 323ce214b6 refact encode APIs 2022-05-07 10:03:06 +00:00
afwerar 2662a3b69d [TD-13254]<fix>: make 3.0 grant. 2022-04-10 19:08:16 +08:00
afwerar 222db126bc [TD-13758]<fix>: redefine memory api. 2022-03-26 00:29:53 +08:00
Liu Jicong 03b2162309 refactor stream 2022-03-25 21:03:12 +08:00
Liu Jicong 59a4eec186 stream extract to module 2022-03-25 20:19:09 +08:00
Liu Jicong af67c49f0e
Merge branch '3.0' into feature/tq 2022-03-22 19:25:55 +08:00
Liu Jicong d3104e78ec stream task exec 2022-03-22 18:00:03 +08:00
Xiaoyu Wang ef125eaa40 merge 3.0 2022-03-22 04:29:54 -04:00
Liu Jicong a36cfba9b9
Merge pull request #10879 from taosdata/feature/tq
Feature/tq
2022-03-21 21:54:30 +08:00
Shengliang Guan 8738ef6980
Merge pull request #10875 from taosdata/feature/shm
refact dnode queue
2022-03-21 19:37:41 +08:00
Shengliang Guan 220fdfabe2 refact dnode queue 2022-03-21 19:08:25 +08:00
Haojun Liao 1c1c95edee [td-13039] merge 3.0 2022-03-21 18:34:42 +08:00
Liu Jicong 1751c0285a add parallel 2022-03-21 16:47:59 +08:00
Liu Jicong 70cddceb60 assign task to vg 2022-03-21 16:35:27 +08:00
Xiaoyu Wang 9ca80596e4 merge 3.0 2022-03-21 02:00:30 -04:00
Shengliang Guan 0f45064d8d Merge remote-tracking branch 'origin/feature/tq' into feature/shm 2022-03-21 10:15:17 +08:00
Liu Jicong cc0c0293ea vnode process task exec 2022-03-18 17:55:39 +08:00
Shengliang Guan 846129fd38 shm 2022-03-18 14:54:08 +08:00
Liu Jicong 7f4c88b756 task deploy and task exec 2022-03-16 19:42:28 +08:00
Liu Jicong 18f254a017 task deploy and task exec 2022-03-16 18:29:31 +08:00
Liu Jicong bd60e92f29 add stream scheduler 2022-03-15 20:04:19 +08:00
Liu Jicong eced27c4f2 add uuid 2022-03-15 15:54:32 +08:00
Liu Jicong 2bcc139443 add stream msg routing 2022-03-15 14:56:33 +08:00
Liu Jicong 36d4ef0af6 refactor stream worker 2022-03-14 20:32:19 +08:00
Liu Jicong e766f3fad4 add mnode stream 2022-03-10 17:15:45 +08:00
Liu Jicong 402a5e3b61 add scheduler 2022-03-07 16:30:28 +08:00
Shengliang Guan dc30a83f24 monitor 2022-03-02 17:48:39 +08:00
Shengliang Guan aac15d1822 minor changes 2022-01-11 19:24:48 -08:00
Shengliang Guan bcad6bb528 restore from wal 2022-01-04 03:44:44 -08:00
Shengliang Guan 736c90adfd add qnode test 2021-12-29 04:31:04 -08:00
Shengliang Guan d60941d4e3 add bnode 2021-12-28 00:20:48 -08:00
Shengliang Guan b63266b05e add qnode snode bnode 2021-12-27 02:43:27 -08:00