xxq250
/
homework-jianmu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
63,621 Commits 3 Branches 0 Tags 479 MiB
Commit Graph

2410 Commits

Author SHA1 Message Date
Minglei Jin 49ecb1093f cos/put: new arg withcp to control resumable uploading 2023-11-30 17:20:21 +08:00
Haojun Liao 30d78c6be1 fix(stream): make sure open operator only once for scan-history task. 2023-11-30 16:10:09 +08:00
54liuyao 271311a770 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-27419 2023-11-30 14:33:48 +08:00
yihaoDeng 5c32680cd8 Merge branch '3.0' into enh/refactorBackend 2023-11-30 10:31:27 +08:00
Haojun Liao 130614c938
Merge pull request #23831 from taosdata/fix/3_liaohj 
enh(stream/tsdb): do multiple optimization and fix some bugs.
 2023-11-30 09:20:15 +08:00
yihaoDeng 08cc976975 fix crash 2023-11-29 19:57:30 +08:00
yihaoDeng 03b698d8ae change default config 2023-11-29 14:50:07 +08:00
54liuyao ae386c6e34 opt expired data 2023-11-29 11:16:56 +08:00
yihaoDeng cc1eb2769f fix crash while quit 2023-11-28 20:29:42 +08:00
yihaoDeng 08eeee0794 Merge branch '3.0' into enh/refactorBackend 2023-11-28 11:52:49 +08:00
Haojun Liao c5aba60899 fix(stream): disable the time slice usage limitation for stream task. 2023-11-28 09:28:43 +08:00
yihaoDeng a4d475b582 fix rsma 2023-11-27 20:27:53 +08:00
wangmm0220 06e2fbfb1c fix:conflicts from 3.0 2023-11-27 17:53:53 +08:00
wangmm0220 70c5d36103 opti:extract common logic from snode & vnode to tqCommon 2023-11-27 17:49:09 +08:00
Haojun Liao 6111138567 fix(stream): fix error. 2023-11-27 17:23:34 +08:00
yihaoDeng e9bec34c44 Merge branch '3.0' into enh/refactorBackend 2023-11-27 10:01:12 +08:00
yihaoDeng 6416a6153a Merge branch '3.0' into enh/refactorBackend 2023-11-24 16:29:09 +08:00
Haojun Liao 3deab3cfbf fix(stream):set failure for the related fill-history task. 2023-11-24 15:37:30 +08:00
yihaoDeng 23ae62d268 Merge branch '3.0' into enh/refactorBackend 2023-11-24 11:10:39 +08:00
yihaoDeng 50f996bbf7 Merge branch '3.0' into enh/refactorBackend 2023-11-23 21:07:58 +08:00
wangmm0220 d249892b2a fix:conflicts from 3.0 2023-11-23 20:29:46 +08:00
Haojun Liao 2e922fefc6 fix(stream): reduce the sink throughput. and fix the bug in status check. 2023-11-23 18:04:09 +08:00
yihaoDeng 6129ebc294 Merge branch '3.0' into enh/refactorBackend 2023-11-23 17:41:37 +08:00
Haojun Liao 0a90c33b4d fix(stream): drop the related fill-history task & set the task to be ready, if the drop fill-history task msg is missing due to the vnode restarting. 2023-11-23 15:49:07 +08:00
Haojun Liao de47ea6c81 other: merge 3.0. 2023-11-23 15:42:40 +08:00
Haojun Liao 1a1319908d refactor: do some internal refactor. 2023-11-23 14:25:52 +08:00
yihaoDeng 6a0501967d Merge branch '3.0' into fix/fixChkpRespMemleak 2023-11-23 14:06:04 +08:00
Haojun Liao 8ad071e69a other: merge main. 2023-11-23 10:25:51 +08:00
Haojun Liao f3355000a3 refactor: do some internal refactor. 2023-11-23 09:34:20 +08:00
Haojun Liao 00f8103262 fix(stream): fix error in generating token in bucket. 2023-11-23 09:34:19 +08:00
Haojun Liao b7347a1d64 fix(stream): remove invalid assert 2023-11-23 09:33:41 +08:00
Haojun Liao 466c80e370 fix(stream): fix bug in delay send. 2023-11-23 09:33:38 +08:00
Haojun Liao 957ce07dc4 fix(stream): fix syntax error. 2023-11-23 09:33:38 +08:00
Haojun Liao 298755cc4e fix(stream): wait for task to be normal and then send data block. 2023-11-23 09:33:37 +08:00
Haojun Liao 7b1f586d4f
Merge pull request #23762 from taosdata/fix/3_liaohj 
fix(stream): fix error in generating token in bucket.
 2023-11-23 09:32:14 +08:00
wangmm0220 15d75a0497 fix:spelling errors 2023-11-23 09:17:54 +08:00
Haojun Liao 4ea49fe285 fix(stream): fix the race condition in dispatching data. 2023-11-22 23:49:16 +08:00
wangmm0220 9a09e318ca opti:common message process logic in vnode and snode 2023-11-22 16:53:33 +08:00
Haojun Liao 91e7000433 Merge branch 'enh/refactorBackend' of github.com:taosdata/tdengine into enh/refactorBackend 2023-11-22 15:29:31 +08:00
Haojun Liao c96f7cf182 fix(stream): scheduler the checkpoint generating. 2023-11-22 15:17:21 +08:00
yihaoDeng 0eba15b4a4 Merge branch '3.0' into enh/refactorBackend 2023-11-22 10:57:41 +08:00
yihaoDeng 0c0d9cdd2c Merge branch '3.0' into fix/fixChkpRespMemleak 2023-11-22 10:42:30 +08:00
Haojun Liao 81d3e5737e fix(stream): fix send msg error. 2023-11-22 08:41:55 +08:00
Haojun Liao c70c54a099 fix(stream): add missing pushed data. 2023-11-22 08:41:35 +08:00
Haojun Liao b712f5e7dc refactor: do some internal refactor. 2023-11-22 08:41:17 +08:00
Haojun Liao d355b78884 fix(stream): fix invalid read. 2023-11-22 08:40:28 +08:00
Haojun Liao 75420bfefb fix(stream): fix the checkpoint msg failure. 2023-11-22 08:40:01 +08:00
yihaoDeng 947775155b fix transfer crash 2023-11-21 20:51:11 +08:00
Haojun Liao 785d91c17b
Merge pull request #23661 from taosdata/feat/TD-26060 
fix: errors in snode in conditions like fill history/checkpoint/restart
 2023-11-21 19:02:38 +08:00
yihaoDeng a92602b667 fix mem leak 2023-11-21 15:23:28 +08:00
Haojun Liao 2b099b11dd refactor: do some internal refactor. 2023-11-21 10:09:59 +08:00
yihaoDeng 939aa7a3cc Merge branch '3.0' into enh/refactorBackend 2023-11-20 17:33:30 +08:00
yihaoDeng 0d724244dc Merge branch '3.0' into enh/refactorBackend 2023-11-20 16:55:14 +08:00
Haojun Liao 9f8a63099c fix(stream): fix error in generating token in bucket. 2023-11-20 15:26:50 +08:00
Haojun Liao 7cfa45e95e enh(stream): add sink_quota/scan-history-idle-duration column for stream tasks. 2023-11-20 11:59:49 +08:00
yihaoDeng 734bf07317 Merge branch '3.0' into enh/refactorBackend 2023-11-20 10:48:49 +08:00
Haojun Liao 1b4f187ded
Merge pull request #23743 from taosdata/fix/pause_stream 
fix(stream): remove invalid assert
 2023-11-17 23:48:47 +08:00
Haojun Liao 709502ba9e fix(stream): record the check downstream failure. 2023-11-17 22:05:36 +08:00
wangmm0220 3241f9bda5 fix:memory leak 2023-11-17 17:26:52 +08:00
yihaoDeng 20de2f3cb5 Merge branch '3.0' into enh/refactorBackend 2023-11-17 17:15:14 +08:00
yihaoDeng e7830bfdfd fix stream backend convert 2023-11-17 16:50:45 +08:00
Haojun Liao 753716f1e6 fix(stream): remove invalid assert 2023-11-17 16:36:17 +08:00
yihaoDeng 286413abbb fix stream backend convert 2023-11-17 16:14:51 +08:00
Haojun Liao b12e5050ea fix(stream): remove invalid assert 2023-11-17 15:57:46 +08:00
yihaoDeng ccc74ef558 fix stream backend convert 2023-11-17 15:39:03 +08:00
yihaoDeng 1dbdb32dd7 fix stream backend convert 2023-11-17 11:56:22 +08:00
yihaoDeng c7c1bdef6e fix stream backend convert 2023-11-17 11:44:21 +08:00
yihaoDeng e455c4799d fix stream backend convert 2023-11-17 10:33:43 +08:00
yihaoDeng b6994413d0 fix stream backend convert 2023-11-17 10:13:39 +08:00
Haojun Liao 27f3723cd3
Merge pull request #23723 from taosdata/fix/3_liaohj 
fix(stream): wait for task to be normal and then send data block.
 2023-11-16 22:58:50 +08:00
yihaoDeng 52d7a26c10 fix stream backend convert 2023-11-16 21:22:47 +08:00
wangmm0220 a5de1b032b fix:snode upport stream task 2023-11-16 21:19:58 +08:00
Haojun Liao 3100edae5d
Merge pull request #23727 from taosdata/fix/pause_stream 
fix(stream): wait for task to be normal and then send data block.
 2023-11-16 19:51:33 +08:00
Haojun Liao bc779edda9 fix(stream): fix bug in delay send. 2023-11-16 17:57:13 +08:00
Haojun Liao f0f4b988a1 fix(stream): fix syntax error. 2023-11-16 17:57:13 +08:00
Haojun Liao a69b870260 fix(stream): wait for task to be normal and then send data block. 2023-11-16 17:57:13 +08:00
Haojun Liao 811d661d83 fix(stream): fix bug in delay send. 2023-11-16 17:34:47 +08:00
Haojun Liao 37d181bb9c fix(stream): fix syntax error. 2023-11-16 17:00:50 +08:00
Haojun Liao c0d3c15528 fix(stream): wait for task to be normal and then send data block. 2023-11-16 16:34:47 +08:00
yihaoDeng f5ef2e727f fix stream backend convert 2023-11-16 14:41:04 +08:00
yihaoDeng cee33ffb95 fix stream backend convert 2023-11-16 11:22:18 +08:00
yihaoDeng 77261068e0 fix stream backend convert 2023-11-16 11:06:55 +08:00
wangmm0220 a16189d2a9 fix:confilct from 3.0 2023-11-15 19:14:27 +08:00
Haojun Liao f7ee65d059 fix(stream): fix send msg error. 2023-11-15 16:31:07 +08:00
Haojun Liao d7fb9c5b6e fix(stream): add missing pushed data. 2023-11-15 16:10:26 +08:00
Haojun Liao 9d2256ec9c refactor: do some internal refactor. 2023-11-15 16:03:23 +08:00
Haojun Liao 2ca663ce66
Merge pull request #23593 from taosdata/feat/TD-21898 
stream event window
 2023-11-15 14:18:07 +08:00
dapan1121 dfb3feb5dc Merge remote-tracking branch 'origin/main' into merge/m23 2023-11-15 11:39:18 +08:00
Haojun Liao 51897a679f fix(stream): fix invalid read. 2023-11-15 11:21:31 +08:00
Haojun Liao 4af844ee61 fix(stream): fix the checkpoint msg failure. 2023-11-14 23:35:09 +08:00
yihaoDeng b74105727d fix mem leak 2023-11-14 20:58:35 +08:00
yihaoDeng 20e1bd9cff fix invalid read/write 2023-11-14 20:46:56 +08:00
yihaoDeng f4d6cf7c74 Merge branch '3.0' into enh/refactorBackend 2023-11-14 18:12:48 +08:00
yihaoDeng aa2c09ef5b rm dup data 2023-11-14 18:09:00 +08:00
yihaoDeng 4476e46a5f rm dup data 2023-11-14 17:43:13 +08:00
dapan1121 478c8ef87f
Merge pull request #23665 from taosdata/fix/TD-27281-3.0 
fix: extend buffer of trace
 2023-11-14 13:50:50 +08:00
Haojun Liao 436c7871a3
Merge pull request #23673 from taosdata/fix/3_liaohj 
fix(stream): fix deadlock and do some internal refactor.
 2023-11-14 11:26:54 +08:00
Haojun Liao e46a13c788
Merge pull request #23669 from taosdata/enh/stream_delay 
opt max delay
 2023-11-14 09:31:17 +08:00
Haojun Liao f99e22b91f fix(stream): fix deadlock. 2023-11-13 23:56:42 +08:00
yihaoDeng 68b6cc1d9d Merge branch '3.0' into enh/refactorBackend 2023-11-13 19:07:58 +08:00
wangmm0220 56381dea90 fix:conflict from 3.0 2023-11-13 18:42:59 +08:00
wangmm0220 1cbe568be7 fix:add logic when snode restart 2023-11-13 18:39:42 +08:00
Haojun Liao aad8a2eb87 fix(stream): fix deadlock. 2023-11-13 18:20:30 +08:00
kailixu dd663a7650 fix: extend buffer of trace 2023-11-13 18:16:48 +08:00
yihaoDeng 4104840cea Merge branch '3.0' into enh/refactorBackend 2023-11-13 18:07:38 +08:00
liuyao 37c102a115 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898 2023-11-13 16:06:17 +08:00
yihaoDeng 1786f7077d Merge branch '3.0' into enh/refactorBackend 2023-11-13 14:35:23 +08:00
yihaoDeng de7f2a2d1a Merge branch '3.0' into enh/refactorBackend 2023-11-13 11:11:28 +08:00
yihaoDeng cf3e01e942 Merge branch '3.0' into enh/refactorBackend 2023-11-13 10:34:44 +08:00
Haojun Liao 4ff8907b58 fix: fix syntax error. 2023-11-11 23:05:20 +08:00
Haojun Liao 9ee9174608 other: merge 3.0 2023-11-11 15:48:31 +08:00
Haojun Liao 3bacd7516e fix(stream): extact stream nodes list if not initialized. 2023-11-11 15:40:35 +08:00
Haojun Liao 15430f4d50 refactor: 2023-11-11 15:40:33 +08:00
Haojun Liao 5fcef5bd89 refactor:add some logs. 2023-11-11 15:40:32 +08:00
Haojun Liao 7a23df4b1a fix(stream): set the correct updated nodeId. 2023-11-11 15:40:32 +08:00
Haojun Liao 8e5db21c3f fix(test): fix syntax error. 2023-11-11 15:40:13 +08:00
Haojun Liao af08a189c1 fix(stream): disable concurrently restart stream tasks. 2023-11-11 15:40:10 +08:00
Haojun Liao e1de1de421 fix(stream): fix the invalid free. 2023-11-11 15:40:07 +08:00
Haojun Liao 97772e9aab fix(stream): the checkpoint version can only be updated when generating checkpoint. 2023-11-11 15:40:05 +08:00
Haojun Liao 0463c0d755 refactor: wait for a while when no quota available. 2023-11-11 15:38:41 +08:00
Haojun Liao 22808ce1b7 fix(stream): update the fill-time for quota limitation. 2023-11-11 15:38:32 +08:00
yihaoDeng 1e8d46c405 Merge branch '3.0' into enh/refactorBackend 2023-11-10 18:37:25 +08:00
Haojun Liao 438c303b0d other: merge main. 2023-11-10 17:05:36 +08:00
Haojun Liao 08a9244230 fix(stream): extact stream nodes list if not initialized. 2023-11-10 16:24:20 +08:00
yihaoDeng a3fdb6dd59 Merge branch '3.0' into enh/refactorBackend 2023-11-10 16:06:08 +08:00
Haojun Liao d0307e5876 refactor: 2023-11-10 15:30:21 +08:00
Haojun Liao 0b36081158 refactor:add some logs. 2023-11-10 14:36:18 +08:00
wangmm0220 8a05bad899 fix:snode error 2023-11-10 14:14:22 +08:00
Haojun Liao 1f9a58361d fix(stream): set the correct updated nodeId. 2023-11-10 13:36:35 +08:00
Haojun Liao 5824caa8f5
Merge pull request #23631 from taosdata/fix/liaohj 
multiple fix
 2023-11-10 13:09:40 +08:00
yihaoDeng ad119ea4c0 Merge branch '3.0' into enh/refactorBackend 2023-11-10 12:07:33 +08:00
Haojun Liao ecf6c1c09d fix(test): fix syntax error. 2023-11-10 10:10:38 +08:00
yihaoDeng 54fe42d372 Merge branch '3.0' into enh/refactorBackend 2023-11-10 09:24:40 +08:00
Haojun Liao 9978667c48 fix(stream): disable concurrently restart stream tasks. 2023-11-10 09:24:08 +08:00
liuyao 6d2d5823dc Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898 2023-11-10 09:24:05 +08:00
yihaoDeng 0669037de0 fix mem leak 2023-11-09 20:37:08 +08:00
yihaoDeng 4a99bdc2b0 support s3 2023-11-09 20:27:30 +08:00
yihaoDeng 5f9b922a04 support s3 2023-11-09 20:15:08 +08:00
Haojun Liao da83f27124
Merge pull request #23610 from taosdata/fix/TD-27216 
change chkp gen way
 2023-11-09 18:55:31 +08:00
Haojun Liao 7f3a50dfde fix(stream): fix the invalid free. 2023-11-09 18:14:27 +08:00
yihaoDeng 757425bff8 Merge branch '3.0' into enh/refactorBackend 2023-11-09 17:58:49 +08:00
yihaoDeng dd2252598e upload by rsync 2023-11-09 17:39:34 +08:00
Yihao Deng 0453991cdf
Merge branch '3.0' into fix/TD-27216-3 2023-11-09 17:12:56 +08:00
Haojun Liao 13c6d1b23a fix(stream): the checkpoint version can only be updated when generating checkpoint. 2023-11-09 17:11:37 +08:00
dapan1121 8314cb6013
Merge pull request #23494 from taosdata/enh/TD-23769-3.0x 
enh: checkpoint for rsma
 2023-11-09 14:51:37 +08:00
yihaoDeng f74edc7618 Merge branch '3.0' into enh/refactorBackend 2023-11-09 06:14:08 +00:00
liuyao e56a3e8a1e Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898 2023-11-09 13:59:39 +08:00
kailixu 1a950afdb8 enh: test case for rsma snapshot 2023-11-09 12:09:27 +08:00
Haojun Liao 257ab4f270
Merge pull request #23526 from taosdata/feat/TD-26060 
feat:checkpoint backup function
 2023-11-09 11:46:28 +08:00
yihaoDeng 679155f771 change chkpid gen way 2023-11-09 11:25:52 +08:00
yihaoDeng a215332e2f change chkpid gen way 2023-11-09 11:24:18 +08:00
yihaoDeng 50666987f0 change chkpid gen way 2023-11-09 11:23:17 +08:00
Haojun Liao ef646873ef
Merge pull request #23596 from taosdata/fix/streamTransferError 
fix stream transfer err
 2023-11-09 10:51:22 +08:00
kailixu edef4de7b2 chore: revert the code 2023-11-08 20:57:57 +08:00
kailixu 72ecb0431c enh: rsma checkpoint 2023-11-08 20:45:44 +08:00
kailixu e72582a1fa Merge branch 'fix/streamTransferError' into enh/TD-23769-3.0x 2023-11-08 20:45:21 +08:00
yihaoDeng 3abf0ae4b2 Merge branch 'feat/TD-26060' into enh/refactorBackend 2023-11-08 17:31:16 +08:00
yihaoDeng 92fa1043b8 Merge branch 'enh/refactorBackend' of https://github.com/taosdata/TDengine into enh/refactorBackend 2023-11-08 17:12:32 +08:00
yihaoDeng 2de5c4a97d add bg thread 2023-11-08 17:11:11 +08:00
yihaoDeng 76fcc49c6d fix stream transfer error 2023-11-08 17:10:54 +08:00
Haojun Liao f949333e8b refactor:do some internal refactor. 2023-11-08 16:57:55 +08:00
Haojun Liao 470f244032 Merge remote-tracking branch 'origin/enh/refactorBackend' into enh/refactorBackend 
# Conflicts:
#	include/libs/stream/tstream.h
 2023-11-08 15:21:21 +08:00
Haojun Liao 99dbbe61d7 refactor(stream): do some internal refactor. 2023-11-08 15:20:47 +08:00
yihaoDeng 6a82cd7fbf Merge branch 'main' of https://github.com/taosdata/TDengine into fix/fixStreamTransferErrMain 2023-11-08 15:14:31 +08:00
yihaoDeng b3c056e6cb fix stream transfer err 2023-11-08 15:12:47 +08:00
wangmm0220 c4c1423243 fix:error 2023-11-08 15:01:04 +08:00
yihaoDeng 7e03fa6664 fix stream transfer err 2023-11-08 14:57:20 +08:00
yihaoDeng 2e7d041e0a fix stream transfer err 2023-11-08 14:48:28 +08:00
liuyao 340fb3e727 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898 2023-11-08 14:16:59 +08:00
yihaoDeng 7b0891981e merge 3.0 2023-11-08 14:14:01 +08:00
wangmm0220 7d19fcf19f Merge branch 'feat/TD-26060' of https://github.com/taosdata/TDengine into feat/TD-26060 2023-11-08 11:59:03 +08:00
wangmm0220 047eb14f67 fix:error 2023-11-08 11:57:49 +08:00
yihaoDeng fb77d811f9 Merge branch 'enh/refactorBackend' of https://github.com/taosdata/TDengine into enh/refactorBackend 2023-11-08 10:48:36 +08:00
Haojun Liao 17bd6badec refactor(stream): do checkpoint for each task. 2023-11-08 10:47:54 +08:00
yihaoDeng 7ada7f34ac fix:merge from 3.0 2023-11-08 10:45:15 +08:00
wangmm0220 07aa749fcd fix:adapt to windows 2023-11-08 10:28:45 +08:00
yihaoDeng 594da2d603 merge 3.0 2023-11-08 10:04:46 +08:00
kailixu b9f389a49f Merge branch '3.0' into enh/TD-23769-3.0x 2023-11-08 08:36:35 +08:00
kailixu f2d24306b1 enh: rsma tasks share one checkpoint and fix memory leak 2023-11-08 08:35:52 +08:00
Haojun Liao e052d3cd62 fix(stream): check null. 2023-11-08 00:10:08 +08:00
kailixu 6b738884a4 chore: rsma checkpoint 2023-11-07 19:59:05 +08:00
wangmm0220 f0b2ed567c fix:upload checkpoint to s3 2023-11-07 18:49:12 +08:00
Haojun Liao 73d75aac25 fix(stream): add ref for task. 2023-11-07 18:43:35 +08:00
Haojun Liao 26b96b149d refactor: wait for a while when no quota available. 2023-11-07 18:43:35 +08:00
Haojun Liao 0979ff944b fix(stream): update the fill-time for quota limitation. 2023-11-07 18:43:34 +08:00
Haojun Liao 2bdde122fc refactor: wait a while when no token available. 2023-11-07 18:43:34 +08:00
Haojun Liao a0d81734f1 fix(stream): add ref for task. 2023-11-07 18:18:58 +08:00
Haojun Liao 43b14ba8b4 refactor: wait for a while when no quota available. 2023-11-07 17:49:49 +08:00
Haojun Liao 06d02b862e fix(stream): update the fill-time for quota limitation. 2023-11-07 17:33:30 +08:00
Haojun Liao 0b59681158 refactor: wait a while when no token available. 2023-11-07 17:18:58 +08:00
Haojun Liao 884770dab1 fix(stream): fix syntax error. 2023-11-07 17:12:06 +08:00
Haojun Liao 344c3c5b13 fix(stream): fix syntax error. 2023-11-07 17:11:36 +08:00
Haojun Liao 9c49de69f8 refactor: do some internal refactor. 2023-11-07 17:05:20 +08:00
Haojun Liao ed0c580d11 refactor: do some internal refactor. 2023-11-07 17:04:55 +08:00
Haojun Liao 1301b92844 fix(stream): fix errors in scan-history, introduced by refactor 2023-11-07 16:41:15 +08:00
Haojun Liao c9a1cc50fb refactor: do some internal refactor. 2023-11-07 16:40:04 +08:00
Haojun Liao 9125fe44c6 fix(stream): fix error. 2023-11-07 16:40:04 +08:00
Haojun Liao 8c2f6bab0e fix(stream): exec scan-history in timer. 2023-11-07 16:40:03 +08:00
Haojun Liao 7370daa6e0 fix(stream): record the start failure tasks. 2023-11-07 16:39:56 +08:00
Haojun Liao e38e526dda fix(stream):reset the starting flag for the followers. 2023-11-07 16:39:12 +08:00
Haojun Liao 207b53f3f6 fix(stream): fix errors in scan-history, introduced by refactor 2023-11-07 16:19:09 +08:00
yihaoDeng 4f53ffd76f refactor backend 2023-11-07 16:01:57 +08:00
yihaoDeng 1ee82f5e2c refactor backend 2023-11-07 15:50:42 +08:00
liuyao 467c27c758 recover flush mark 2023-11-07 15:45:38 +08:00
yihaoDeng 8f7ad2bd90 refactor backend 2023-11-07 15:20:58 +08:00
yihaoDeng 042522ac7a refactor backend 2023-11-07 15:18:24 +08:00
yihaoDeng e015134d6d refactor backend 2023-11-07 14:46:52 +08:00
liuyao e972ab16fe recover flush mark 2023-11-07 14:11:08 +08:00
yihaoDeng e5449bf161 refactor backend 2023-11-07 11:49:56 +08:00
yihaoDeng 95cfc5eb46 refactor checkpoint 2023-11-07 09:55:55 +08:00
yihaoDeng 021859361f refactor checkpoint 2023-11-07 09:22:57 +08:00
Haojun Liao 96f9264906 fix(stream): add null ptr check. 2023-11-07 00:38:02 +08:00
Haojun Liao 1312b9ea36 fix(stream): fix error. 2023-11-06 23:44:01 +08:00
Haojun Liao 4d8548e938 refactor: do some internal refactor. 2023-11-06 23:42:16 +08:00
yihaoDeng c9bf4f6507 refactor checkpoint 2023-11-06 20:19:11 +08:00
yihaoDeng 819be65db0 refactor checkpoint 2023-11-06 19:59:11 +08:00
kailixu f5d796a081 chore: print stream state 2023-11-06 19:53:09 +08:00
Haojun Liao 312dbc1caa refactor: do some internal refactor. 2023-11-06 19:49:25 +08:00
yihaoDeng 9ca1151b67 refactor checkpoint 2023-11-06 19:47:47 +08:00
Haojun Liao 1f09d3a46b fix(stream): fix error. 2023-11-06 19:46:47 +08:00
Haojun Liao d9474016fc fix(stream): exec scan-history in timer. 2023-11-06 18:55:53 +08:00
yihaoDeng aee1b700a6 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-06 18:08:03 +08:00
yihaoDeng ba52769867 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-06 17:56:09 +08:00
Haojun Liao 6b1889284b fix(stream): record the start failure tasks. 2023-11-06 15:45:44 +08:00
yihaoDeng feeead50c5 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-06 13:26:38 +08:00
Haojun Liao 69863febe6 fix(stream):reset the starting flag for the followers. 2023-11-06 13:17:13 +08:00
Haojun Liao d41fd27189 fix(stream): restart tasks in stream threads, instead of write thread. 2023-11-06 11:57:21 +08:00
Haojun Liao 0e44950a37 enh(stream): execute scan history by using time slice 1s. 2023-11-06 09:21:33 +08:00
Haojun Liao 1f756f58f6 fix(stream): add need update node into update node list. 2023-11-06 09:21:33 +08:00
Haojun Liao 680eb5674e refactor: do some internal refactor. 2023-11-06 09:15:49 +08:00
Haojun Liao d440da0d8c enh(stream): execute scan history by using time slice 1s. 2023-11-05 01:26:53 +08:00
Haojun Liao c71f6713a7 fix(stream): add need update node into update node list. 2023-11-05 01:24:23 +08:00
Haojun Liao e13f003b90 refactor: do some internal refactor. 2023-11-05 01:23:23 +08:00
kailixu 411151d671 fix: buffer overflow/buffer use after free/memory leak 2023-11-04 08:44:26 +08:00
yihaoDeng f038f88c07 Merge branch 'feat/TD-26060' into enh/refactorBackend 2023-11-03 18:29:18 +08:00
wangmm0220 5a6c3f2b04 fix:add windows support 2023-11-03 17:45:37 +08:00
kailixu 77eadc59dc Merge branch '3.0' into enh/TD-23769-3.0x 2023-11-03 16:26:07 +08:00
kailixu a48968e5e8 enh: rsma checkpoint 2023-11-03 14:33:45 +08:00
Haojun Liao 53b2158c54 refactor: do some internal refactor. 2023-11-03 13:51:17 +08:00
wangmm0220 0a1f1085fc Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-26060 2023-11-03 11:33:50 +08:00
wangmm0220 3ac11eba0e fix:modify upload checkpoint interface 2023-11-03 11:33:18 +08:00
yihaoDeng d82d51cf8f Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-03 10:40:00 +08:00
yihaoDeng 24e961976f Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-03 09:31:57 +08:00
Haojun Liao 9d77351bb2 refactor: do some internal refactor. 2023-11-03 09:30:50 +08:00
yihaoDeng 069dd8de84 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-03 09:28:30 +08:00
liuyao 31ad73f6d1 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898 2023-11-03 09:12:17 +08:00
yihaoDeng 0a73832fd2 add checkpoint delta 2023-11-02 20:39:55 +08:00
kailixu 6c944bb192 enh: skip rsma_task during load stream tasks 2023-11-02 20:32:54 +08:00
wangmm0220 159873692c feat:checkpoint bakeup using rsync 2023-11-02 20:13:29 +08:00
Haojun Liao b3d57e4a2d fix(stream): not transfer state if state is not appropriate. 2023-11-02 19:31:35 +08:00
Haojun Liao 7a32b3a209 fix(stream): not transfer state if state is not appropriate. 2023-11-02 19:12:36 +08:00
Haojun Liao 706f1e4744 fix(stream): fix deadlock in transfer state. 2023-11-02 18:18:04 +08:00
Haojun Liao 39175dc03b fix(stream): fix deadlock in transfer state. 2023-11-02 18:07:36 +08:00
Haojun Liao c48c801e19 fix(stream): return error code. 2023-11-02 11:02:40 +08:00
Haojun Liao 95efa07e92 fix(stream): add full unsupported event filtering. 2023-11-02 11:02:39 +08:00
Haojun Liao 05b3d51803 fix(stream): add qualified status in assert 2023-11-02 11:02:39 +08:00
Haojun Liao 420825c62f fix(stream): return error code. 2023-11-02 10:22:03 +08:00
yihaoDeng 7e7a39f077 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-02 10:15:34 +08:00
yihaoDeng b911492270 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-02 09:46:36 +08:00
Haojun Liao 351a31302f fix(stream): add full unsupported event filtering. 2023-11-02 09:43:48 +08:00
Haojun Liao e14d42c01e fix(stream): add qualified status in assert 2023-11-02 08:46:18 +08:00
Haojun Liao dc8a90c864 fix(stream): fix memory leak. 2023-11-02 08:13:11 +08:00
Haojun Liao 79e971385f fix(stream): fix memory leak. 2023-11-02 08:11:51 +08:00
Haojun Liao df7633650e fix(stream): fix memory leak. 2023-11-01 20:35:45 +08:00
Haojun Liao e1b3e00b3d fix(stream): fix memory leak. 2023-11-01 20:10:20 +08:00
Haojun Liao 6dc468c55f fix(stream): fix dead-lock 2023-11-01 19:38:13 +08:00
Haojun Liao fa7ebf63ea refactor: do some internal refactor. 2023-11-01 19:38:13 +08:00
Haojun Liao 9156e8449d fix(stream): fix deference null ptr. 2023-11-01 19:38:13 +08:00
Haojun Liao 7cf90dde5c refactor: do some internal refactor. 2023-11-01 19:38:13 +08:00
Haojun Liao 00f029e44f refactor: add some logs. 2023-11-01 19:38:12 +08:00
Haojun Liao 385e699cc2 fix(stream): do some internal refactor. 2023-11-01 19:38:12 +08:00
Haojun Liao bd12ae88e1 fix(stream):fix the bug when event in waiting list not fulfilled. 2023-11-01 19:38:12 +08:00
Haojun Liao 9a9c1d689d fix(stream): add lock log. 2023-11-01 19:38:12 +08:00
Haojun Liao e7609d8e56 fix(stream): fix dead-lock 2023-11-01 19:09:21 +08:00
Haojun Liao ae25f09c45 refactor: do some internal refactor. 2023-11-01 18:03:44 +08:00
Haojun Liao 899d13b4ae fix(stream): fix deference null ptr. 2023-11-01 17:25:59 +08:00
Haojun Liao 246ed4e022 refactor: do some internal refactor. 2023-11-01 17:19:21 +08:00
liuyao 505c44d9aa Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898 2023-11-01 16:50:46 +08:00
Haojun Liao 12bc4f32cf refactor: add some logs. 2023-11-01 16:12:15 +08:00
yihaoDeng 4c8edb5545 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-01 16:07:49 +08:00
Haojun Liao 778c29b808 fix(stream): do some internal refactor. 2023-11-01 14:07:18 +08:00
Haojun Liao e4aeea176b fix(stream):fix the bug when event in waiting list not fulfilled. 2023-11-01 13:59:51 +08:00
Haojun Liao 5ff89bc098 fix(stream): add lock log. 2023-11-01 13:19:53 +08:00
Haojun Liao 3f561fb8e4 fix(stream): remove invalid assert. 2023-11-01 11:43:02 +08:00
Haojun Liao 3e6c89d8eb fix(stream): add unsupport trans. 2023-11-01 11:43:02 +08:00
kailixu 1530fe0c49 Merge branch '3.0' into enh/TD-23769-3.0x 2023-11-01 09:52:16 +08:00
kailixu ad1e6accd4 chore: build checkpoint for rsma 2023-11-01 09:45:58 +08:00
Haojun Liao 0fb4cfd94f fix(stream): remove invalid assert. 2023-11-01 00:33:17 +08:00
Haojun Liao 639d5cde47 fix(stream): add unsupport trans. 2023-10-31 23:14:02 +08:00
yihaoDeng bb8252df16 fix mem leak 2023-10-31 20:59:24 +08:00
Haojun Liao 294c59150c fix(stream): add jump out loop by checking task status. 2023-10-31 19:31:28 +08:00
Haojun Liao 4694859325 fix(stream): add jump out loop by checking task status. 2023-10-31 19:31:00 +08:00
Haojun Liao 0b18192b23 fix(stream): fix error in finishing event handling. 2023-10-31 16:40:16 +08:00
Haojun Liao 762cfef498 fix(stream): adjust the stop condition for stream tasks. 2023-10-31 16:40:16 +08:00
Haojun Liao 4c0242a58d fix(stream): fix error in finishing event handling. 2023-10-31 16:39:14 +08:00
Haojun Liao e01c7f3cfd fix(stream): adjust the stop condition for stream tasks. 2023-10-31 11:51:47 +08:00
Haojun Liao 88469568b8 fix(stream): handle failure during checkpoint 2023-10-31 10:45:14 +08:00
Haojun Liao 635243f01a refactor: do some internal refactor. 2023-10-31 10:45:12 +08:00
Haojun Liao 0c982093cd fix(stream): enable the set of fill-history task to be un-init. 2023-10-31 10:45:12 +08:00
Haojun Liao 86dfc52a6e fix(stream): add event check when handling the end of event. 2023-10-31 10:45:12 +08:00
Haojun Liao 72f3e4446d fix(stream): handle failure during checkpoint 2023-10-31 10:38:17 +08:00
yihaoDeng 7fad855247 fix mem leak 2023-10-30 18:16:12 +08:00
yihaoDeng ee6b620597 fix mem leak 2023-10-30 17:29:32 +08:00
Haojun Liao fdf3f210ac refactor: do some internal refactor. 2023-10-30 14:39:24 +08:00
Haojun Liao 93232a0e14 fix(stream): enable the set of fill-history task to be un-init. 2023-10-30 11:48:51 +08:00
Haojun Liao 19c5882cbe fix(stream): add event check when handling the end of event. 2023-10-30 10:22:24 +08:00
Haojun Liao 980f6fc46e fix(stream): pause the task when they are still in uninit status. 2023-10-30 09:32:40 +08:00
Haojun Liao 1bbad532a7 fix(stream): add more check. 2023-10-30 09:32:40 +08:00
Haojun Liao 076b533ba3 fix(stream): fix race condition when state transferring. 2023-10-30 09:32:40 +08:00
Haojun Liao 85befa10b9 fix(stream): avoid clear the restart flag . 2023-10-30 09:32:39 +08:00
Haojun Liao f708ddb792 fix(stream): pause the task when they are still in uninit status. 2023-10-30 09:09:21 +08:00
Haojun Liao 790d869494 fix(stream): add more check. 2023-10-29 22:57:33 +08:00
Haojun Liao 3c07c3ec2f fix(stream): fix race condition when state transferring. 2023-10-29 02:27:27 +08:00
Haojun Liao c7285c0ede fix(stream): add expired epset node list for stream tasks in hb to mnode. 2023-10-27 23:41:57 +08:00
Haojun Liao 4c267b538f fix(stream): add expired epset node list for stream tasks in hb to mnode. 2023-10-27 23:05:41 +08:00
Haojun Liao 62cc33715c fix(stream): fix deadlock. 2023-10-27 15:05:36 +08:00
Haojun Liao 0c2079a963 fix(stream): fix deadlock. 2023-10-27 15:03:21 +08:00
yihaoDeng ebdf6d1cc6 fix mem leak 2023-10-27 14:18:56 +08:00
Haojun Liao afe7fdf672 fix(stream): avoid clear the restart flag . 2023-10-27 14:12:06 +08:00
Haojun Liao dcca6b0868 other: merge 3.0. 2023-10-27 11:41:28 +08:00
Haojun Liao 29b3a7a5e9 refactor: do some internal refactor. 2023-10-27 09:59:53 +08:00
Haojun Liao 9c9d02b181 fix(stream): disable follower nodes exec reset. 2023-10-27 09:59:52 +08:00
Haojun Liao 52d8e05c0c fix(stream): resolve the nodeUpdate msg compatible issue. 2023-10-27 09:59:52 +08:00
Haojun Liao 454624516c fix(stream): handle the repeatly issued nodeUpdate msg from dnode. 2023-10-27 09:59:51 +08:00
Haojun Liao 9c9cd08e2e fix(stream): not clear role flag. 2023-10-27 09:59:44 +08:00
Haojun Liao 33dc51cc86 fix(stream): add some logs. 2023-10-27 09:59:44 +08:00
Haojun Liao 1ccbb77981 fix(stream): fix dead lock when shutdown all tasks. 2023-10-27 09:59:37 +08:00
Haojun Liao 9a1a8feec2 fix(stream): do send check info if downstream is not leader. 2023-10-27 09:58:53 +08:00
Haojun Liao 2493c21ed3 fix(stream): fix the invalid write in sma 2023-10-27 09:58:52 +08:00
Haojun Liao e8520997b0 fix(stream): disable continue check for downstream tasks. 2023-10-27 09:58:52 +08:00
Haojun Liao 3bcffcf1da other: merge stream refactor from main. 2023-10-27 09:56:02 +08:00
Haojun Liao b0ef072556 fix(stream): clear flag for checkpoint. 2023-10-27 09:55:07 +08:00
Haojun Liao f682ad0479 refactor: do some internal refactor. 2023-10-27 09:55:06 +08:00
Haojun Liao efede8669f refactor: do some internal refactor. 2023-10-27 09:55:06 +08:00
Haojun Liao 9ebf806789 fix(stream): clear the flag. 2023-10-27 09:55:05 +08:00
Haojun Liao 48ff521968 fix(stream): update the nodeEp even during checkpoint trans. 2023-10-27 09:55:02 +08:00
Haojun Liao 0d4892f25b fix(stream): fix bugs caused by refactor sm. 2023-10-27 09:55:00 +08:00
Haojun Liao a31613ae95 refactor: add ref in log. 2023-10-27 09:54:59 +08:00
Haojun Liao 226b6596d6 fix(stream): fix bugs caused by refactor. 2023-10-27 09:54:58 +08:00
Haojun Liao 091238e7be fix(stream): fix bugs caused by refactor sm. 2023-10-27 09:54:58 +08:00
Haojun Liao e1f6043eed fix(stream): fix bugs caused by refactor sm. 2023-10-27 09:54:57 +08:00
Haojun Liao e359da7b19 fix(stream): fix bugs caused by refactor. 2023-10-27 09:54:57 +08:00
Haojun Liao 8729bdac1e fix(stream): fix bugs caused by refactor. 2023-10-27 09:54:57 +08:00
Haojun Liao 1eac7f44c0 fix(stream): fix error in stream. 2023-10-27 09:54:56 +08:00
Haojun Liao 579dd8e6b2 refactor(stream): do some internal refactor. 2023-10-27 09:54:56 +08:00
Haojun Liao 6f3c0d17cc refactor(stream): add state machine to manage the state of stream tasks. 2023-10-27 09:54:49 +08:00
Haojun Liao 596eb073b7 refactor(stream): do some internal refactor. 2023-10-27 09:54:48 +08:00
Haojun Liao 91e3fd66e6 enh(stream): sm for stream tasks. 2023-10-27 09:54:44 +08:00
Haojun Liao 2409a17517 refactor(stream): do some internal refactor. 2023-10-27 09:54:44 +08:00
Haojun Liao c9bdb56d79 refactor: do some internal refactor. 2023-10-27 09:54:09 +08:00
Haojun Liao 164159f293 enh(stream): add parameter to limit the stream sink task. 2023-10-27 09:54:07 +08:00
Haojun Liao 62c58ecd8b fix(stream):add more check for test cases. 2023-10-27 09:54:06 +08:00
Haojun Liao a4f6042966 fix(stream): fix error in creating update node trans. 2023-10-27 09:52:14 +08:00
Haojun Liao ffa798c4fa enh(stream): add some logs. 2023-10-27 09:52:14 +08:00
yihaoDeng 7e1e68d8f5 fix stream case error 2023-10-26 18:27:09 +08:00
yihaoDeng 87db93e72e fix stream case error 2023-10-26 11:36:29 +08:00
liuyao d0e4a5d0f7 opt max delay 2023-10-25 20:29:25 +08:00
yihaoDeng db0f6258fe fix stream case error 2023-10-25 19:38:22 +08:00
liuyao f51c985914 stream event window 2023-10-25 17:50:16 +08:00
Haojun Liao f36b24ed24 fix(stream): disable follower nodes exec reset. 2023-10-25 14:16:36 +08:00
Haojun Liao 985371e841 fix(stream): resolve the nodeUpdate msg compatible issue. 2023-10-25 10:01:20 +08:00
Haojun Liao 03165d5327 fix(stream): handle the repeatly issued nodeUpdate msg from dnode. 2023-10-25 09:55:02 +08:00
Haojun Liao 26aaa4a83c other: merge main. 2023-10-25 00:39:14 +08:00
Haojun Liao 5b42204dba fix(stream): not clear role flag. 2023-10-25 00:32:46 +08:00
Haojun Liao 13d979a2ee fix(stream): add some logs. 2023-10-25 00:03:46 +08:00
Haojun Liao 8cde39eebd fix(stream): fix dead lock when shutdown all tasks. 2023-10-24 19:11:24 +08:00
yihaoDeng d43b3b4a32 refactor stream backend 2023-10-24 16:17:54 +08:00
yihaoDeng b3c54a0f06 refactor stream backend 2023-10-24 16:04:43 +08:00
Haojun Liao 6ffb5ee476 fix(stream): do send check info if downstream is not leader. 2023-10-24 15:49:50 +08:00
Haojun Liao f1498f8929 fix(stream): fix the invalid write in sma 2023-10-24 15:24:34 +08:00
Haojun Liao 0f9328330b fix(stream): disable continue check for downstream tasks. 2023-10-24 11:56:36 +08:00
yihaoDeng d46cf878eb refactor stream backend 2023-10-24 11:53:52 +08:00
Haojun Liao 7a8b046a35 fix(sma): fix bugs in sma and do some internal refactor. 2023-10-24 09:40:38 +08:00
yihaoDeng 09192acf96 fix mem leak 2023-10-23 10:20:57 +08:00
Haojun Liao 059e94e428 fix(stream): clear flag for checkpoint. 2023-10-22 01:25:00 +08:00
Haojun Liao 4826b34a2b refactor: do some internal refactor. 2023-10-21 16:32:44 +08:00
Haojun Liao 06cd67e26e refactor: do some internal refactor. 2023-10-21 16:20:03 +08:00
Haojun Liao 21e98e358e fix(stream): clear the flag. 2023-10-21 02:41:03 +08:00
Haojun Liao d1095e2c4c Merge remote-tracking branch 'origin/refact/streamsm' into refact/streamsm 2023-10-21 01:42:59 +08:00
Haojun Liao 49014d4b68 fix(stream): update the nodeEp even during checkpoint trans. 2023-10-21 01:42:43 +08:00
yihaoDeng aaf7e070ee Merge branch 'refact/streamsm' of https://github.com/taosdata/TDengine into refact/streamsm 2023-10-20 20:49:17 +08:00
yihaoDeng e95735eea4 fix mem leak 2023-10-20 20:49:01 +08:00
Haojun Liao c68c0eda79 fix(stream): fix bugs caused by refactor sm. 2023-10-20 16:27:39 +08:00
Haojun Liao 4feebffb05 refactor: add ref in log. 2023-10-20 15:59:43 +08:00
Haojun Liao c4268ca395 fix(stream): fix bugs caused by refactor. 2023-10-20 15:54:30 +08:00
yihaoDeng 1c3c58062d Merge branch 'enh/new3.0' into enh/refactorBackend 2023-10-20 15:40:47 +08:00
yihaoDeng 1a4cfd11a5 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-10-20 15:28:10 +08:00
Haojun Liao 3b8c85f632 fix(stream): fix bugs caused by refactor sm. 2023-10-20 14:19:57 +08:00
Haojun Liao 270a847809 fix(stream): fix bugs caused by refactor sm. 2023-10-20 10:23:48 +08:00
wade zhang 628667f22c
Merge pull request #23231 from taosdata/FEAT/TS-3816-3.0 
feat: support restoring vnodes on replaced disks as needed
 2023-10-20 09:02:25 +08:00
Haojun Liao 4d5d078fbe fix(stream): fix bugs caused by refactor. 2023-10-19 18:00:54 +08:00
Haojun Liao d5059826a8 fix(stream): fix bugs caused by refactor. 2023-10-19 17:59:04 +08:00
Haojun Liao c1cebae6ba fix(stream): fix error in stream. 2023-10-19 16:20:27 +08:00
Haojun Liao 614bdd9fb9
Merge pull request #23220 from taosdata/feat/TD-26644 
opt bloom filter
 2023-10-19 15:34:43 +08:00
Haojun Liao da8f6a4c2f
Merge pull request #23248 from taosdata/feat/TD-26647 
recover session buff state
 2023-10-19 15:34:03 +08:00
Haojun Liao 402aefd95d refactor(stream): do some internal refactor. 2023-10-19 15:09:47 +08:00
Benguang Zhao c4e9069a66 fix: set nextProcessedVer properly in tqProcessTaskScanHistory 2023-10-19 11:18:57 +08:00
Haojun Liao f4caeca24a refactor(stream): add state machine to manage the state of stream tasks. 2023-10-19 10:34:20 +08:00
Haojun Liao 54ec679b58 refactor(stream): do some internal refactor. 2023-10-18 15:34:53 +08:00
yihaoDeng c6ba2ca205 fix transfer crash 2023-10-18 14:49:12 +08:00