Commit Graph

404 Commits

Author SHA1 Message Date
yihaoDeng 46574c385f fix mem leak 2023-08-17 18:52:33 +08:00
Haojun Liao a18989df88 fix(stream): add lost codes caused by merge 3.0. 2023-08-17 18:51:39 +08:00
Haojun Liao 6746339609 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2
# Conflicts:
#	source/libs/stream/src/streamSnapshot.c
2023-08-17 00:08:44 +08:00
Haojun Liao ea6574b3c7 other: merge 3.0 2023-08-17 00:07:55 +08:00
yihaoDeng ab7e2b3122 Merge branch 'enh/chkpTransfer' into enh/triggerCheckPoint2 2023-08-16 09:00:30 +00:00
kailixu 242bf77f4c chore: code optimization 2023-08-15 16:13:51 +08:00
kailixu 8a84dce641 enh: disable udf/stream/rsma/tsma for windows 2023-08-15 16:04:30 +08:00
Haojun Liao 39079ab64a fix(stream): load term from sync model to decide if the node is restart or not 2023-08-15 15:52:06 +08:00
kailixu 27db6cfd67 enh: disable udf/stream for mnd on windows 2023-08-15 14:25:31 +08:00
Haojun Liao 1f792f09c4 fix(stream): remove invalid code. 2023-08-14 10:39:47 +08:00
Haojun Liao f24b22000f enh(stream): stop stream task before redistribute vnodes. 2023-08-11 23:46:53 +08:00
yihaoDeng 6e6eb05d5c Merge branch 'enh/triggerCheckPoint2' into enh/chkpTransfer 2023-08-11 08:03:02 +00:00
Haojun Liao e984affe90 fix(stream): disable the stream in the followers nodes. 2023-08-11 15:58:15 +08:00
Haojun Liao ba7825141f fix(stream): enable the checkpoint . 2023-08-11 13:33:14 +08:00
Haojun Liao 9322f80343 fix(stream): register the rsp callback. 2023-08-11 13:31:50 +08:00
yihaoDeng 99be2836b2 Merge branch 'enh/triggerCheckPoint2' into enh/chkpTransfer 2023-08-11 02:50:12 +00:00
Haojun Liao 6a9826cf6e enh(stream): update the stream task epset. 2023-08-11 10:37:25 +08:00
yihaoDeng ba4ce9db02 Merge branch 'enh/triggerCheckPoint2' into enh/chkpTransfer 2023-08-11 02:15:07 +00:00
Haojun Liao f633c7fe2b other: merge 3.0. 2023-08-10 11:56:30 +08:00
Haojun Liao d34b9af054 enh(stream): create update stream trans. 2023-08-09 19:38:13 +08:00
Haojun Liao 4c92997328 fix(stream): use streamId&taskId to identify the stream task in the stream meta. 2023-08-09 15:52:23 +08:00
Haojun Liao f3c5f20ee2 enh(stream): add timer to check the node changing. 2023-08-09 10:25:16 +08:00
Haojun Liao a0dcec5890 enh(stream): add function handler for hb message. 2023-08-08 11:23:19 +08:00
Haojun Liao af3a87560a refactor: do some refactor. 2023-08-08 09:11:41 +08:00
Haojun Liao 76288f311f
Merge pull request #22311 from taosdata/fix/TD-25543
delete atomic_exchange_64
2023-08-07 10:34:50 +08:00
liuyao 944c1f68d2 delete atomic_exchange_64 2023-08-04 16:49:02 +08:00
yihaoDeng 56ed422f88 add stream ver 2023-08-04 14:09:00 +08:00
yihaoDeng 6dc93a8dec add stream ver 2023-08-04 13:52:20 +08:00
yihaoDeng f7dc9497ef add stream verion 2023-08-03 20:28:34 +08:00
yihaoDeng 4aef18733f fix state mem leak 2023-08-02 03:16:24 +00:00
Haojun Liao 13543f7daa feature(stream): handle task update in tq module. 2023-08-02 10:33:41 +08:00
Haojun Liao e1e5e9bb0f enh(stream): check stream tasks dag. 2023-08-02 09:44:39 +08:00
Haojun Liao c016a58623 enh(stream): do some internal refactor. 2023-08-01 19:22:59 +08:00
Haojun Liao ad4c5916e7 enh(stream): update the stream task. 2023-08-01 18:14:58 +08:00
Haojun Liao 164bfd5408 enh(stream): prepare update trans. 2023-08-01 16:12:35 +08:00
Haojun Liao 75b1520be0 enh(stream): create task update info 2023-08-01 14:04:07 +08:00
Haojun Liao e61aa83594 enh(stream): handle the stream hb. 2023-07-31 14:07:18 +08:00
Haojun Liao cc5ff44604 enh(stream): handle the stream hb. 2023-07-30 19:25:00 +08:00
Haojun Liao 8dfef8768a other: merge 3.0 2023-07-29 22:54:16 +08:00
Haojun Liao 3d403dca5a other: merge main. 2023-07-27 16:23:41 +08:00
yihaoDeng 8ab545ac77 fillhistory task not trigger checkpoint 2023-07-18 01:16:48 +00:00
wangmm0220 da9e30434a fix:add sdbFetchCancel to release hash node 2023-07-17 20:21:23 +08:00
yihaoDeng 3c4631ef1f add checkpoint 2023-07-14 10:06:24 +00:00
yihaoDeng dd4740e5a4 add checkpoint 2023-07-14 03:45:30 +00:00
yihaoDeng 44170b9b16 fix recover error 2023-07-14 10:36:27 +08:00
yihaoDeng e588640e02 fix recover error 2023-07-13 20:46:58 +08:00
yihaoDeng e263d5f55f add checkpoint 2023-07-10 17:33:48 +08:00
yihaoDeng 4d838d3dff add checkpoint 2023-07-10 15:51:35 +08:00
yihaoDeng 2dc041929a add checkpoint 2023-07-07 10:13:31 +00:00
Haojun Liao d7101f7109 enh(stream): support dispatch checkpoint msg. 2023-07-06 18:26:22 +08:00
yihaoDeng 483f9ab896 add other 2023-07-06 03:39:28 +00:00
Haojun Liao f4a3567935 other: merge 3.0 2023-07-06 11:24:30 +08:00
liuyao bc5e709d31 set task status of ins_stream_tasks 2023-07-03 14:00:30 +08:00
liuyao 863da2ae05 set task status of ins_stream_tasks 2023-07-03 11:10:38 +08:00
Yihao Deng 4de7db06f9
Merge branch '3.0' into enh/triggerCheckPoint2 2023-06-27 17:57:14 +08:00
Haojun Liao 32fddeff18 fix(stream): fix error retrieve data from source task.And optimize the test cases. 2023-06-27 16:24:47 +08:00
Haojun Liao 3289ad62be other: merge 3.0 2023-06-27 11:33:40 +08:00
Haojun Liao daafe240ed fix(stream): fix memory leak. 2023-06-26 14:18:47 +08:00
Benguang Zhao e8649670aa enh: not allow to split vgroup when streams or topics exist 2023-06-26 11:57:24 +08:00
Haojun Liao 000325f716 fix(stream): fix memory leak. 2023-06-25 19:30:21 +08:00
liuyao 41c193945f fill history pause&resume 2023-06-19 17:28:19 +08:00
yihaoDeng b6fe5d917b refactor checkpoint 2023-06-17 14:05:49 +08:00
yihaoDeng 8937192799 refactor checkpoint 2023-06-16 18:04:57 +08:00
yihaoDeng b0a3a8d619 refactor checkpoint 2023-06-16 17:28:50 +08:00
Haojun Liao 74e0138f72 Merge branch '3.0' into refact/fillhistory 2023-06-15 15:25:00 +08:00
yihaoDeng bd0168e562 trigger checkpoint 2023-06-14 21:43:47 +08:00
yihaoDeng 551cd7cdc8 trigger checkpoint 2023-06-14 21:14:00 +08:00
Yihao Deng 7ded6fc7a2
Merge branch '3.0' into enh/triggerCheckPoint2 2023-06-14 16:30:38 +08:00
yihaoDeng 52ca94ea6d trigger checkpoint 2023-06-14 16:20:14 +08:00
yihaoDeng 83dde6a212 add trigger checkpoint 2023-06-12 14:33:08 +00:00
Benguang Zhao 3689b41f3a Merge branch '3.0' into FIX/TD-24182-3.0 2023-06-12 09:39:49 +08:00
yihaoDeng a0c2897124 add trigger checkpoint 2023-06-09 10:46:03 +00:00
yihaoDeng 88d765c71a add trigger checkpoint 2023-06-07 14:32:56 +00:00
yihaoDeng 1f1f37312d add trigger checkpoint 2023-06-07 14:32:48 +00:00
Haojun Liao 06cf3588da refactor: do some internal refactor. 2023-06-07 15:03:04 +08:00
Haojun Liao fcc706c45c enh(stream): refactor the fill history task. 2023-06-05 15:59:01 +08:00
Haojun Liao eda06081ff enh(stream): refactor and serialize the attributes of history tasks. 2023-05-30 16:55:37 +08:00
Haojun Liao be8fd9e48b enh(stream): create additional task for history data processing. 2023-05-30 14:47:10 +08:00
Haojun Liao 2977a4d07f refactor: do some internal refactor. 2023-05-30 09:41:19 +08:00
Haojun Liao f2887278e2 refactor: do some internal refactor. 2023-05-29 15:44:45 +08:00
Benguang Zhao 30104f300c enh: refactor func names of mndTransPerformFinishedStage and mndTrancCheckConflict 2023-05-25 15:44:26 +08:00
liuyao da3fc5111e fix pause agg task 2023-05-18 15:29:41 +08:00
liuyao 3c81137ffe pause source task 2023-05-16 13:43:46 +08:00
54liuyao 255af84139 resume option 2023-04-25 19:01:19 +08:00
54liuyao b79313ecd7 pause&resume 2023-04-25 16:59:09 +08:00
liuyao 641c707b37 pause&resume 2023-04-23 17:37:10 +08:00
54liuyao 5089528a02 feat:stream pause&resume 2023-04-23 15:08:51 +08:00
Haojun Liao f8b672f09f other: merge main. 2023-04-14 09:07:18 +08:00
Haojun Liao 18479d8115 refactor: do some internall refactor. 2023-04-11 15:42:24 +08:00
Haojun Liao 7419c0bfe9 enh(stream): the stream will start after vnode restore being completed. 2023-04-10 16:56:14 +08:00
Haojun Liao f5b52749c2 refactor: do some internal refactor. 2023-04-09 01:39:09 +08:00
yihaoDeng 07ca8eb8e0 enh: add trace to transaction 2023-03-16 10:25:46 +00:00
54liuyao abbc69a2ed fix:asan error 2023-02-28 17:43:47 +08:00
Xiaoyu Wang 58fc265b80 merge main 2023-02-24 13:26:45 +08:00
Xiaoyu Wang d06dedd5cc
Merge pull request #20103 from taosdata/fix/TD-22668
fix: coverity scan issues
2023-02-24 10:53:49 +08:00
54liuyao d43355e77f fix:check null column 2023-02-23 17:31:52 +08:00
Xiaoyu Wang 1ab144609e merge main 2023-02-23 14:58:22 +08:00
Minglei Jin 0a782ef78c fix(mnd): coverity scan issues 2023-02-22 14:50:41 +08:00
54liuyao 1d58f85a54 feat:coverity scan 2023-02-22 10:10:04 +08:00
Xiaoyu Wang 7337f5ff7d
Merge pull request #20052 from taosdata/fix/3.0_merge_main
merge main
2023-02-21 16:27:19 +08:00
54liuyao f34509cc6e feat:a maximum of 60 streams can be created 2023-02-20 14:18:21 +08:00
Xiaoyu Wang c597de099f merge main 2023-02-20 13:41:27 +08:00
Haojun Liao 751c1cb5d2 refactor:do some internal refactor. 2023-02-20 10:12:27 +08:00
Haojun Liao 8d187efc18 refactor:do some internal refactor. 2023-02-20 10:04:08 +08:00
54liuyao 33caf8e625 fix:check stream 2023-02-16 13:48:47 +08:00
Haojun Liao c9ce01c878 Merge branch 'main' into feature/3_liaohj 2023-02-13 17:09:27 +08:00
Haojun Liao c597a1e4ce refactor: do some internal refactor. 2023-02-13 15:46:47 +08:00
Xiaoyu Wang 3382a24a69 merge main 2023-02-10 18:03:11 +08:00
Xiaoyu Wang b2e60da1cf merge main 2023-02-10 10:34:55 +08:00
Xiaoyu Wang 98e2fa2d7b feat: add ignore update option for create stream 2023-02-08 18:46:12 +08:00
Liu Jicong ae4ad7d851 Merge branch 'main' into enh/stream_buffer_param 2023-02-08 16:09:35 +08:00
Xiaoyu Wang 09925437e3 merge main 2023-02-08 09:51:52 +08:00
Liu Jicong 55f78b4109 fix case 2023-02-06 16:41:30 +08:00
Liu Jicong e615053f4a enh: stream number limit 2023-02-06 11:32:04 +08:00
54liuyao e03deeaed8 feat:add stream delete mark 2023-01-17 16:21:51 +08:00
54liuyao 3ec1560faa create stream and use existing super table 2023-01-13 10:26:49 +08:00
54liuyao d5ade95b31 feat(stream):user define tag 2023-01-10 10:13:56 +08:00
54liuyao 4734795fe1 feat(stream):stream insert data into an existing table 2022-12-28 17:09:54 +08:00
Shengliang Guan 664dcb6b4e Merge branch 'main' into merge/mainto3.0_1226 2022-12-26 09:26:54 +08:00
Shengliang Guan cb5ec76cce enh: check conflict before create trans 2022-12-23 13:58:39 +08:00
Liu Jicong 27c7e23397 refactor: remove assert 2022-12-21 14:26:27 +08:00
Liu Jicong 4ab9638f9a enh(stream): forbid source db replica > 1 2022-12-16 15:41:08 +08:00
Liu Jicong 4402bcef94 enh(stream): forbid source db replica > 1 2022-12-16 08:53:33 +08:00
Shengliang Guan 6d94afe48f
Revert "refact: adjust some assert cases" 2022-12-07 21:51:17 +08:00
Shengliang Guan d5126d469a refact: replcase ASSERT with tAssert 2022-12-07 18:42:48 +08:00
Shengliang Guan f69a188f7e refact: replcase ASSERT with tAssert 2022-12-07 18:37:40 +08:00
Shengliang Guan 99657ccf78 fix: compatibility after update stream 2022-12-02 08:00:10 +08:00
Shengliang Guan b8cc174211 Merge remote-tracking branch 'origin/3.0' into feature/stream 2022-12-02 07:59:39 +08:00
Shengliang Guan ffb1d52e55 fix: compatility after update stream 2022-12-02 07:57:22 +08:00
Shengliang Guan a366696478
Merge pull request #18596 from taosdata/feature/stream
refactor(stream): distributed checkpoint
2022-12-02 07:51:10 +08:00
Liu Jicong 3424b184d3 refactor(stream): distributed checkpoint 2022-12-02 03:48:45 +08:00
Shengliang Guan 2d48563959 fix: crash if failed to decode sdb row 2022-12-01 16:04:39 +08:00
Shengliang Guan 74f83a362b fix: memory while use taosx 2022-11-27 22:23:40 +08:00
Shengliang Guan 853c22de28 fix: memory leak while create stream 2022-11-16 10:55:19 +08:00
Liu Jicong a9bf05bc01 refactor: stream meta ref count 2022-11-15 19:11:42 +08:00
Liu Jicong 7c8b2867ce enh(stream): add stream task info into information schema 2022-11-14 19:23:42 +08:00
Shengliang Guan 30058f7619 enh: optimize the error code when create and delete a database in a dropping state 2022-11-08 10:46:32 +08:00
Liu Jicong 2fbc8e6306 refactor(stream): stream deploy and state transfer 2022-11-03 13:22:23 +08:00
Liu Jicong 2e640c38a2 refactor: stream and tmq message 2022-10-31 18:37:28 +08:00
Liu Jicong 26d4af6968 fix: memory leak 2022-10-21 11:42:30 +08:00
Liu Jicong 035e6b13c8 enh(stream): internal optimize 2022-10-21 10:29:05 +08:00
Liu Jicong fbfae317b3 refactor 2022-10-11 02:09:15 +08:00
Liu Jicong eafe8666f7 merge 3.0 2022-10-09 17:00:18 +08:00
Liu Jicong 2f9aeeb5fc feat(stream): support define table name 2022-10-09 16:54:27 +08:00
Shengliang Guan 4e65accad2 fix: coverity issues 2022-10-08 17:02:34 +08:00
Shengliang Guan e98335bdfa fix: coverity issues 2022-10-08 11:29:46 +08:00
Shengliang Guan 31e424d5ab fix: coverity issues 2022-09-30 14:59:31 +08:00
Shengliang Guan dfb2b0678f fix: coverity issues 2022-09-30 14:03:46 +08:00
Shengliang Guan 3810b5d4fa fix: add more info log for mnode 2022-09-23 15:42:36 +08:00
Shengliang Guan 43ef21f67b enh: add oper name to show transaction 2022-09-22 16:19:33 +08:00
Shengliang Guan 30b2bd98b8 fix: STrans struct dbname2 naming ambiguity 2022-09-19 10:39:51 +08:00
Liu Jicong e484090b75 fix(stream): fix handling invalid option 2022-08-25 16:27:18 +08:00
Liu Jicong a2757eb4f3 fix(mnode): memory leak 2022-08-18 14:26:11 +08:00
Liu Jicong b8a3654cd6 enh(stream): show stream 2022-08-11 13:37:32 +08:00
Liu Jicong 7f4b46e003 feat(stream): session window trigger delete 2022-08-10 21:26:05 +08:00
Liu Jicong 9e5d347383 enh(tmq): enable background heartbeat by default 2022-08-10 18:22:22 +08:00
Liu Jicong 95f696b30d enh(tmq): prevent drop db when topic exists 2022-08-10 17:24:47 +08:00
Liu Jicong d9a4ea305d fix(stream): memory error 2022-08-09 14:11:41 +08:00
Liu Jicong 43026ac31d refactor(stream) 2022-08-08 11:29:50 +08:00
Liu Jicong 7c9d57bd89 refactor(stream): unify sink and dispatch 2022-08-05 20:16:47 +08:00
54liuyao c1522632dc fix(stream): amend error message 2022-08-04 13:39:16 +08:00
Liu Jicong 280ed4f65d feat(stream): pipelined exec for recovering 2022-07-15 21:29:53 +08:00
Liu Jicong 9394e3380f feat(stream): recover from failure 2022-07-15 17:58:53 +08:00
Liu Jicong 9ea0ad57ec refactor(stream): destroy stream task 2022-07-14 15:32:21 +08:00
Liu Jicong 31bbef82d1 fix(tmq): seek snapshot offset 2022-07-02 20:18:40 +08:00
Shengliang Guan f249bae82d Merge remote-tracking branch 'origin/3.0' into fix/tsim 2022-07-02 14:14:01 +08:00
Shengliang Guan 0c510fcc8c fix: drop db has stream exist 2022-07-02 11:55:59 +08:00
Liu Jicong 4a2b266612 feat(stream): support ignore expired flag 2022-07-02 10:59:29 +08:00
Liu Jicong 996f995644 fix(sma): drop stream when drop sma 2022-06-25 18:14:42 +08:00
Liu Jicong bdd4134fc1
Revert "fix(sma): drop stream when drop sma" 2022-06-25 18:11:11 +08:00
Liu Jicong 3210fdc367 merge from 3.0 2022-06-25 17:49:25 +08:00
Liu Jicong 47ae534c0f fix(sma): drop stream when drop sma 2022-06-25 17:44:54 +08:00
Shengliang Guan 1fb2c664a4
Merge pull request #14223 from taosdata/fix/tsim
refactor: rename auth to privilege
2022-06-25 15:26:59 +08:00
Liu Jicong e6f88b6286
Merge pull request #14225 from taosdata/feature/stream
enh(query): window included in serialized retrieve table rsp
2022-06-25 11:35:18 +08:00
Shengliang Guan 23653fd843 enh: privilege for topic and stream 2022-06-25 11:22:34 +08:00
Liu Jicong 5040cb9541 enh(query): window included in serialized retrieve table rsp 2022-06-25 10:43:19 +08:00
Shengliang Guan 09ce47ae9f refactor: rename auth to privilege 2022-06-25 10:27:27 +08:00
Liu Jicong f53521cb8c test(stream): partition by tbname 2022-06-24 16:50:11 +08:00
Liu Jicong b884ca9db7 refactor(stream): seperate stream scheduling and persistence 2022-06-24 14:19:12 +08:00
Liu Jicong c409760f75 feat(stream): drop stream 2022-06-21 11:32:36 +08:00
Liu Jicong 212144a3e6 test(stream): window close not work for stb 2022-06-20 21:50:13 +08:00
Shengliang Guan 5ae355077c enh(mnode): transaction conflict supports two db 2022-06-17 11:11:29 +08:00
Shengliang Guan 80e0b830fe refactor: remove duplicate conninfo in SRpcMsg 2022-06-17 10:40:42 +08:00
Liu Jicong 07288e26b2 refactor(stream): change msg name 2022-06-16 17:08:31 +08:00
Liu Jicong 44e733645c ci: remove unstable case 2022-06-16 16:35:55 +08:00
Liu Jicong 32d8e60ce2 refactor(sma): create 2022-06-16 16:22:58 +08:00
Liu Jicong 928f45709d feat(stream): drop stream 2022-06-16 16:01:22 +08:00
Liu Jicong 803249837a feat(stream): drop stream 2022-06-16 10:29:30 +08:00
Liu Jicong 683bc24646
Merge pull request #13876 from taosdata/feature/stream
feat(stream): drop task
2022-06-15 22:59:43 +08:00
Liu Jicong 5572abe1b4 feat(stream): drop task 2022-06-15 21:41:59 +08:00
Shengliang Guan 1b15c3439a
Merge pull request #13859 from taosdata/fix/dnode
feat: increase enable and sysinfo limits for user privilege
2022-06-15 18:24:32 +08:00
Shengliang Guan 6711d79434 feat: increase enable and sysinfo limits for user privilege 2022-06-15 17:40:02 +08:00
Liu Jicong d7a6c4976c
fix(stream): max delay should not be seen by operator (#13864) 2022-06-15 17:27:01 +08:00
Liu Jicong d200fc710b fix(stream): max delay should not be seen by operator 2022-06-15 17:25:47 +08:00
Liu Jicong 8eb08fa0d4 feat(stream): trigger mode add max delay 2022-06-15 16:11:20 +08:00
Xiaoyu Wang 380a071551 fix: handle the memory leak of parser 2022-06-13 14:54:38 +08:00
Liu Jicong a7fba4b224 refactor(stream): stream scheduler 2022-06-10 21:57:59 +08:00
Liu Jicong 94d7cbe622 feat(stream): snode process stream msg 2022-06-07 16:24:50 +08:00
Shengliang Guan 980e289cdf refactor: add alter-confirm while alter db 2022-06-05 14:59:53 +08:00
Shengliang Guan 1013b06107 refactor: make trans support global conflict 2022-05-31 19:24:26 +08:00