2410 Commits

Author	SHA1	Message	Date
wangmm0220	56381dea90	fix:conflict from 3.0	2023-11-13 18:42:59 +08:00
wangmm0220	1cbe568be7	fix:add logic when snode restart	2023-11-13 18:39:42 +08:00
Haojun Liao	aad8a2eb87	fix(stream): fix deadlock.	2023-11-13 18:20:30 +08:00
kailixu	dd663a7650	fix: extend buffer of trace	2023-11-13 18:16:48 +08:00
yihaoDeng	4104840cea	Merge branch '3.0' into enh/refactorBackend	2023-11-13 18:07:38 +08:00
liuyao	37c102a115	Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898	2023-11-13 16:06:17 +08:00
yihaoDeng	1786f7077d	Merge branch '3.0' into enh/refactorBackend	2023-11-13 14:35:23 +08:00
yihaoDeng	de7f2a2d1a	Merge branch '3.0' into enh/refactorBackend	2023-11-13 11:11:28 +08:00
yihaoDeng	cf3e01e942	Merge branch '3.0' into enh/refactorBackend	2023-11-13 10:34:44 +08:00
Haojun Liao	4ff8907b58	fix: fix syntax error.	2023-11-11 23:05:20 +08:00
Haojun Liao	9ee9174608	other: merge 3.0	2023-11-11 15:48:31 +08:00
Haojun Liao	3bacd7516e	fix(stream): extact stream nodes list if not initialized.	2023-11-11 15:40:35 +08:00
Haojun Liao	15430f4d50	refactor:	2023-11-11 15:40:33 +08:00
Haojun Liao	5fcef5bd89	refactor:add some logs.	2023-11-11 15:40:32 +08:00
Haojun Liao	7a23df4b1a	fix(stream): set the correct updated nodeId.	2023-11-11 15:40:32 +08:00
Haojun Liao	8e5db21c3f	fix(test): fix syntax error.	2023-11-11 15:40:13 +08:00
Haojun Liao	af08a189c1	fix(stream): disable concurrently restart stream tasks.	2023-11-11 15:40:10 +08:00
Haojun Liao	e1de1de421	fix(stream): fix the invalid free.	2023-11-11 15:40:07 +08:00
Haojun Liao	97772e9aab	fix(stream): the checkpoint version can only be updated when generating checkpoint.	2023-11-11 15:40:05 +08:00
Haojun Liao	0463c0d755	refactor: wait for a while when no quota available.	2023-11-11 15:38:41 +08:00
Haojun Liao	22808ce1b7	fix(stream): update the fill-time for quota limitation.	2023-11-11 15:38:32 +08:00
yihaoDeng	1e8d46c405	Merge branch '3.0' into enh/refactorBackend	2023-11-10 18:37:25 +08:00
Haojun Liao	438c303b0d	other: merge main.	2023-11-10 17:05:36 +08:00
Haojun Liao	08a9244230	fix(stream): extact stream nodes list if not initialized.	2023-11-10 16:24:20 +08:00
yihaoDeng	a3fdb6dd59	Merge branch '3.0' into enh/refactorBackend	2023-11-10 16:06:08 +08:00
Haojun Liao	d0307e5876	refactor:	2023-11-10 15:30:21 +08:00
Haojun Liao	0b36081158	refactor:add some logs.	2023-11-10 14:36:18 +08:00
wangmm0220	8a05bad899	fix:snode error	2023-11-10 14:14:22 +08:00
Haojun Liao	1f9a58361d	fix(stream): set the correct updated nodeId.	2023-11-10 13:36:35 +08:00
Haojun Liao	5824caa8f5	Merge pull request #23631 from taosdata/fix/liaohj ... multiple fix	2023-11-10 13:09:40 +08:00
yihaoDeng	ad119ea4c0	Merge branch '3.0' into enh/refactorBackend	2023-11-10 12:07:33 +08:00
Haojun Liao	ecf6c1c09d	fix(test): fix syntax error.	2023-11-10 10:10:38 +08:00
yihaoDeng	54fe42d372	Merge branch '3.0' into enh/refactorBackend	2023-11-10 09:24:40 +08:00
Haojun Liao	9978667c48	fix(stream): disable concurrently restart stream tasks.	2023-11-10 09:24:08 +08:00
liuyao	6d2d5823dc	Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898	2023-11-10 09:24:05 +08:00
yihaoDeng	0669037de0	fix mem leak	2023-11-09 20:37:08 +08:00
yihaoDeng	4a99bdc2b0	support s3	2023-11-09 20:27:30 +08:00
yihaoDeng	5f9b922a04	support s3	2023-11-09 20:15:08 +08:00
Haojun Liao	da83f27124	Merge pull request #23610 from taosdata/fix/TD-27216 ... change chkp gen way	2023-11-09 18:55:31 +08:00
Haojun Liao	7f3a50dfde	fix(stream): fix the invalid free.	2023-11-09 18:14:27 +08:00
yihaoDeng	757425bff8	Merge branch '3.0' into enh/refactorBackend	2023-11-09 17:58:49 +08:00
yihaoDeng	dd2252598e	upload by rsync	2023-11-09 17:39:34 +08:00
Yihao Deng	0453991cdf	Merge branch '3.0' into fix/TD-27216-3	2023-11-09 17:12:56 +08:00
Haojun Liao	13c6d1b23a	fix(stream): the checkpoint version can only be updated when generating checkpoint.	2023-11-09 17:11:37 +08:00
dapan1121	8314cb6013	Merge pull request #23494 from taosdata/enh/TD-23769-3.0x ... enh: checkpoint for rsma	2023-11-09 14:51:37 +08:00
yihaoDeng	f74edc7618	Merge branch '3.0' into enh/refactorBackend	2023-11-09 06:14:08 +00:00
liuyao	e56a3e8a1e	Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898	2023-11-09 13:59:39 +08:00
kailixu	1a950afdb8	enh: test case for rsma snapshot	2023-11-09 12:09:27 +08:00
Haojun Liao	257ab4f270	Merge pull request #23526 from taosdata/feat/TD-26060 ... feat:checkpoint backup function	2023-11-09 11:46:28 +08:00
yihaoDeng	679155f771	change chkpid gen way	2023-11-09 11:25:52 +08:00
yihaoDeng	a215332e2f	change chkpid gen way	2023-11-09 11:24:18 +08:00
yihaoDeng	50666987f0	change chkpid gen way	2023-11-09 11:23:17 +08:00
Haojun Liao	ef646873ef	Merge pull request #23596 from taosdata/fix/streamTransferError ... fix stream transfer err	2023-11-09 10:51:22 +08:00
kailixu	edef4de7b2	chore: revert the code	2023-11-08 20:57:57 +08:00
kailixu	72ecb0431c	enh: rsma checkpoint	2023-11-08 20:45:44 +08:00
kailixu	e72582a1fa	Merge branch 'fix/streamTransferError' into enh/TD-23769-3.0x	2023-11-08 20:45:21 +08:00
yihaoDeng	3abf0ae4b2	Merge branch 'feat/TD-26060' into enh/refactorBackend	2023-11-08 17:31:16 +08:00
yihaoDeng	92fa1043b8	Merge branch 'enh/refactorBackend' of https://github.com/taosdata/TDengine into enh/refactorBackend	2023-11-08 17:12:32 +08:00
yihaoDeng	2de5c4a97d	add bg thread	2023-11-08 17:11:11 +08:00
yihaoDeng	76fcc49c6d	fix stream transfer error	2023-11-08 17:10:54 +08:00
Haojun Liao	f949333e8b	refactor:do some internal refactor.	2023-11-08 16:57:55 +08:00
Haojun Liao	470f244032	Merge remote-tracking branch 'origin/enh/refactorBackend' into enh/refactorBackend ... # Conflicts: # include/libs/stream/tstream.h	2023-11-08 15:21:21 +08:00
Haojun Liao	99dbbe61d7	refactor(stream): do some internal refactor.	2023-11-08 15:20:47 +08:00
yihaoDeng	6a82cd7fbf	Merge branch 'main' of https://github.com/taosdata/TDengine into fix/fixStreamTransferErrMain	2023-11-08 15:14:31 +08:00
yihaoDeng	b3c056e6cb	fix stream transfer err	2023-11-08 15:12:47 +08:00
wangmm0220	c4c1423243	fix:error	2023-11-08 15:01:04 +08:00
yihaoDeng	7e03fa6664	fix stream transfer err	2023-11-08 14:57:20 +08:00
yihaoDeng	2e7d041e0a	fix stream transfer err	2023-11-08 14:48:28 +08:00
liuyao	340fb3e727	Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898	2023-11-08 14:16:59 +08:00
yihaoDeng	7b0891981e	merge 3.0	2023-11-08 14:14:01 +08:00
wangmm0220	7d19fcf19f	Merge branch 'feat/TD-26060' of https://github.com/taosdata/TDengine into feat/TD-26060	2023-11-08 11:59:03 +08:00
wangmm0220	047eb14f67	fix:error	2023-11-08 11:57:49 +08:00
yihaoDeng	fb77d811f9	Merge branch 'enh/refactorBackend' of https://github.com/taosdata/TDengine into enh/refactorBackend	2023-11-08 10:48:36 +08:00
Haojun Liao	17bd6badec	refactor(stream): do checkpoint for each task.	2023-11-08 10:47:54 +08:00
yihaoDeng	7ada7f34ac	fix:merge from 3.0	2023-11-08 10:45:15 +08:00
wangmm0220	07aa749fcd	fix:adapt to windows	2023-11-08 10:28:45 +08:00
yihaoDeng	594da2d603	merge 3.0	2023-11-08 10:04:46 +08:00
kailixu	b9f389a49f	Merge branch '3.0' into enh/TD-23769-3.0x	2023-11-08 08:36:35 +08:00
kailixu	f2d24306b1	enh: rsma tasks share one checkpoint and fix memory leak	2023-11-08 08:35:52 +08:00
Haojun Liao	e052d3cd62	fix(stream): check null.	2023-11-08 00:10:08 +08:00
kailixu	6b738884a4	chore: rsma checkpoint	2023-11-07 19:59:05 +08:00
wangmm0220	f0b2ed567c	fix:upload checkpoint to s3	2023-11-07 18:49:12 +08:00
Haojun Liao	73d75aac25	fix(stream): add ref for task.	2023-11-07 18:43:35 +08:00
Haojun Liao	26b96b149d	refactor: wait for a while when no quota available.	2023-11-07 18:43:35 +08:00
Haojun Liao	0979ff944b	fix(stream): update the fill-time for quota limitation.	2023-11-07 18:43:34 +08:00
Haojun Liao	2bdde122fc	refactor: wait a while when no token available.	2023-11-07 18:43:34 +08:00
Haojun Liao	a0d81734f1	fix(stream): add ref for task.	2023-11-07 18:18:58 +08:00
Haojun Liao	43b14ba8b4	refactor: wait for a while when no quota available.	2023-11-07 17:49:49 +08:00
Haojun Liao	06d02b862e	fix(stream): update the fill-time for quota limitation.	2023-11-07 17:33:30 +08:00
Haojun Liao	0b59681158	refactor: wait a while when no token available.	2023-11-07 17:18:58 +08:00
Haojun Liao	884770dab1	fix(stream): fix syntax error.	2023-11-07 17:12:06 +08:00
Haojun Liao	344c3c5b13	fix(stream): fix syntax error.	2023-11-07 17:11:36 +08:00
Haojun Liao	9c49de69f8	refactor: do some internal refactor.	2023-11-07 17:05:20 +08:00
Haojun Liao	ed0c580d11	refactor: do some internal refactor.	2023-11-07 17:04:55 +08:00
Haojun Liao	1301b92844	fix(stream): fix errors in scan-history, introduced by refactor	2023-11-07 16:41:15 +08:00
Haojun Liao	c9a1cc50fb	refactor: do some internal refactor.	2023-11-07 16:40:04 +08:00
Haojun Liao	9125fe44c6	fix(stream): fix error.	2023-11-07 16:40:04 +08:00
Haojun Liao	8c2f6bab0e	fix(stream): exec scan-history in timer.	2023-11-07 16:40:03 +08:00
Haojun Liao	7370daa6e0	fix(stream): record the start failure tasks.	2023-11-07 16:39:56 +08:00
Haojun Liao	e38e526dda	fix(stream):reset the starting flag for the followers.	2023-11-07 16:39:12 +08:00
Haojun Liao	207b53f3f6	fix(stream): fix errors in scan-history, introduced by refactor	2023-11-07 16:19:09 +08:00
yihaoDeng	4f53ffd76f	refactor backend	2023-11-07 16:01:57 +08:00
yihaoDeng	1ee82f5e2c	refactor backend	2023-11-07 15:50:42 +08:00
liuyao	467c27c758	recover flush mark	2023-11-07 15:45:38 +08:00
yihaoDeng	8f7ad2bd90	refactor backend	2023-11-07 15:20:58 +08:00
yihaoDeng	042522ac7a	refactor backend	2023-11-07 15:18:24 +08:00
yihaoDeng	e015134d6d	refactor backend	2023-11-07 14:46:52 +08:00
liuyao	e972ab16fe	recover flush mark	2023-11-07 14:11:08 +08:00
yihaoDeng	e5449bf161	refactor backend	2023-11-07 11:49:56 +08:00
yihaoDeng	95cfc5eb46	refactor checkpoint	2023-11-07 09:55:55 +08:00
yihaoDeng	021859361f	refactor checkpoint	2023-11-07 09:22:57 +08:00
Haojun Liao	96f9264906	fix(stream): add null ptr check.	2023-11-07 00:38:02 +08:00
Haojun Liao	1312b9ea36	fix(stream): fix error.	2023-11-06 23:44:01 +08:00
Haojun Liao	4d8548e938	refactor: do some internal refactor.	2023-11-06 23:42:16 +08:00
yihaoDeng	c9bf4f6507	refactor checkpoint	2023-11-06 20:19:11 +08:00
yihaoDeng	819be65db0	refactor checkpoint	2023-11-06 19:59:11 +08:00
kailixu	f5d796a081	chore: print stream state	2023-11-06 19:53:09 +08:00
Haojun Liao	312dbc1caa	refactor: do some internal refactor.	2023-11-06 19:49:25 +08:00
yihaoDeng	9ca1151b67	refactor checkpoint	2023-11-06 19:47:47 +08:00
Haojun Liao	1f09d3a46b	fix(stream): fix error.	2023-11-06 19:46:47 +08:00
Haojun Liao	d9474016fc	fix(stream): exec scan-history in timer.	2023-11-06 18:55:53 +08:00
yihaoDeng	aee1b700a6	Merge branch 'enh/new3.0' into enh/refactorBackend	2023-11-06 18:08:03 +08:00
yihaoDeng	ba52769867	Merge branch 'enh/new3.0' into enh/refactorBackend	2023-11-06 17:56:09 +08:00
Haojun Liao	6b1889284b	fix(stream): record the start failure tasks.	2023-11-06 15:45:44 +08:00
yihaoDeng	feeead50c5	Merge branch 'enh/new3.0' into enh/refactorBackend	2023-11-06 13:26:38 +08:00
Haojun Liao	69863febe6	fix(stream):reset the starting flag for the followers.	2023-11-06 13:17:13 +08:00
Haojun Liao	d41fd27189	fix(stream): restart tasks in stream threads, instead of write thread.	2023-11-06 11:57:21 +08:00
Haojun Liao	0e44950a37	enh(stream): execute scan history by using time slice 1s.	2023-11-06 09:21:33 +08:00
Haojun Liao	1f756f58f6	fix(stream): add need update node into update node list.	2023-11-06 09:21:33 +08:00
Haojun Liao	680eb5674e	refactor: do some internal refactor.	2023-11-06 09:15:49 +08:00
Haojun Liao	d440da0d8c	enh(stream): execute scan history by using time slice 1s.	2023-11-05 01:26:53 +08:00
Haojun Liao	c71f6713a7	fix(stream): add need update node into update node list.	2023-11-05 01:24:23 +08:00
Haojun Liao	e13f003b90	refactor: do some internal refactor.	2023-11-05 01:23:23 +08:00
kailixu	411151d671	fix: buffer overflow/buffer use after free/memory leak	2023-11-04 08:44:26 +08:00
yihaoDeng	f038f88c07	Merge branch 'feat/TD-26060' into enh/refactorBackend	2023-11-03 18:29:18 +08:00
wangmm0220	5a6c3f2b04	fix:add windows support	2023-11-03 17:45:37 +08:00
kailixu	77eadc59dc	Merge branch '3.0' into enh/TD-23769-3.0x	2023-11-03 16:26:07 +08:00
kailixu	a48968e5e8	enh: rsma checkpoint	2023-11-03 14:33:45 +08:00
Haojun Liao	53b2158c54	refactor: do some internal refactor.	2023-11-03 13:51:17 +08:00
wangmm0220	0a1f1085fc	Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-26060	2023-11-03 11:33:50 +08:00
wangmm0220	3ac11eba0e	fix:modify upload checkpoint interface	2023-11-03 11:33:18 +08:00
yihaoDeng	d82d51cf8f	Merge branch 'enh/new3.0' into enh/refactorBackend	2023-11-03 10:40:00 +08:00
yihaoDeng	24e961976f	Merge branch 'enh/new3.0' into enh/refactorBackend	2023-11-03 09:31:57 +08:00
Haojun Liao	9d77351bb2	refactor: do some internal refactor.	2023-11-03 09:30:50 +08:00
yihaoDeng	069dd8de84	Merge branch 'enh/new3.0' into enh/refactorBackend	2023-11-03 09:28:30 +08:00
liuyao	31ad73f6d1	Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898	2023-11-03 09:12:17 +08:00
yihaoDeng	0a73832fd2	add checkpoint delta	2023-11-02 20:39:55 +08:00
kailixu	6c944bb192	enh: skip rsma_task during load stream tasks	2023-11-02 20:32:54 +08:00
wangmm0220	159873692c	feat:checkpoint bakeup using rsync	2023-11-02 20:13:29 +08:00
Haojun Liao	b3d57e4a2d	fix(stream): not transfer state if state is not appropriate.	2023-11-02 19:31:35 +08:00
Haojun Liao	7a32b3a209	fix(stream): not transfer state if state is not appropriate.	2023-11-02 19:12:36 +08:00
Haojun Liao	706f1e4744	fix(stream): fix deadlock in transfer state.	2023-11-02 18:18:04 +08:00
Haojun Liao	39175dc03b	fix(stream): fix deadlock in transfer state.	2023-11-02 18:07:36 +08:00
Haojun Liao	c48c801e19	fix(stream): return error code.	2023-11-02 11:02:40 +08:00
Haojun Liao	95efa07e92	fix(stream): add full unsupported event filtering.	2023-11-02 11:02:39 +08:00
Haojun Liao	05b3d51803	fix(stream): add qualified status in assert	2023-11-02 11:02:39 +08:00
Haojun Liao	420825c62f	fix(stream): return error code.	2023-11-02 10:22:03 +08:00
yihaoDeng	7e7a39f077	Merge branch 'enh/new3.0' into enh/refactorBackend	2023-11-02 10:15:34 +08:00
yihaoDeng	b911492270	Merge branch 'enh/new3.0' into enh/refactorBackend	2023-11-02 09:46:36 +08:00
Haojun Liao	351a31302f	fix(stream): add full unsupported event filtering.	2023-11-02 09:43:48 +08:00
Haojun Liao	e14d42c01e	fix(stream): add qualified status in assert	2023-11-02 08:46:18 +08:00
Haojun Liao	dc8a90c864	fix(stream): fix memory leak.	2023-11-02 08:13:11 +08:00
Haojun Liao	79e971385f	fix(stream): fix memory leak.	2023-11-02 08:11:51 +08:00
Haojun Liao	df7633650e	fix(stream): fix memory leak.	2023-11-01 20:35:45 +08:00
Haojun Liao	e1b3e00b3d	fix(stream): fix memory leak.	2023-11-01 20:10:20 +08:00
Haojun Liao	6dc468c55f	fix(stream): fix dead-lock	2023-11-01 19:38:13 +08:00
Haojun Liao	fa7ebf63ea	refactor: do some internal refactor.	2023-11-01 19:38:13 +08:00
Haojun Liao	9156e8449d	fix(stream): fix deference null ptr.	2023-11-01 19:38:13 +08:00
Haojun Liao	7cf90dde5c	refactor: do some internal refactor.	2023-11-01 19:38:13 +08:00
Haojun Liao	00f029e44f	refactor: add some logs.	2023-11-01 19:38:12 +08:00
Haojun Liao	385e699cc2	fix(stream): do some internal refactor.	2023-11-01 19:38:12 +08:00
Haojun Liao	bd12ae88e1	fix(stream):fix the bug when event in waiting list not fulfilled.	2023-11-01 19:38:12 +08:00
Haojun Liao	9a9c1d689d	fix(stream): add lock log.	2023-11-01 19:38:12 +08:00
Haojun Liao	e7609d8e56	fix(stream): fix dead-lock	2023-11-01 19:09:21 +08:00
Haojun Liao	ae25f09c45	refactor: do some internal refactor.	2023-11-01 18:03:44 +08:00
Haojun Liao	899d13b4ae	fix(stream): fix deference null ptr.	2023-11-01 17:25:59 +08:00
Haojun Liao	246ed4e022	refactor: do some internal refactor.	2023-11-01 17:19:21 +08:00
liuyao	505c44d9aa	Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898	2023-11-01 16:50:46 +08:00
Haojun Liao	12bc4f32cf	refactor: add some logs.	2023-11-01 16:12:15 +08:00
yihaoDeng	4c8edb5545	Merge branch 'enh/new3.0' into enh/refactorBackend	2023-11-01 16:07:49 +08:00
Haojun Liao	778c29b808	fix(stream): do some internal refactor.	2023-11-01 14:07:18 +08:00
Haojun Liao	e4aeea176b	fix(stream):fix the bug when event in waiting list not fulfilled.	2023-11-01 13:59:51 +08:00
Haojun Liao	5ff89bc098	fix(stream): add lock log.	2023-11-01 13:19:53 +08:00
Haojun Liao	3f561fb8e4	fix(stream): remove invalid assert.	2023-11-01 11:43:02 +08:00
Haojun Liao	3e6c89d8eb	fix(stream): add unsupport trans.	2023-11-01 11:43:02 +08:00
kailixu	1530fe0c49	Merge branch '3.0' into enh/TD-23769-3.0x	2023-11-01 09:52:16 +08:00
kailixu	ad1e6accd4	chore: build checkpoint for rsma	2023-11-01 09:45:58 +08:00
Haojun Liao	0fb4cfd94f	fix(stream): remove invalid assert.	2023-11-01 00:33:17 +08:00
Haojun Liao	639d5cde47	fix(stream): add unsupport trans.	2023-10-31 23:14:02 +08:00
yihaoDeng	bb8252df16	fix mem leak	2023-10-31 20:59:24 +08:00
Haojun Liao	294c59150c	fix(stream): add jump out loop by checking task status.	2023-10-31 19:31:28 +08:00
Haojun Liao	4694859325	fix(stream): add jump out loop by checking task status.	2023-10-31 19:31:00 +08:00
Haojun Liao	0b18192b23	fix(stream): fix error in finishing event handling.	2023-10-31 16:40:16 +08:00
Haojun Liao	762cfef498	fix(stream): adjust the stop condition for stream tasks.	2023-10-31 16:40:16 +08:00
Haojun Liao	4c0242a58d	fix(stream): fix error in finishing event handling.	2023-10-31 16:39:14 +08:00
Haojun Liao	e01c7f3cfd	fix(stream): adjust the stop condition for stream tasks.	2023-10-31 11:51:47 +08:00
Haojun Liao	88469568b8	fix(stream): handle failure during checkpoint	2023-10-31 10:45:14 +08:00
Haojun Liao	635243f01a	refactor: do some internal refactor.	2023-10-31 10:45:12 +08:00
Haojun Liao	0c982093cd	fix(stream): enable the set of fill-history task to be un-init.	2023-10-31 10:45:12 +08:00
Haojun Liao	86dfc52a6e	fix(stream): add event check when handling the end of event.	2023-10-31 10:45:12 +08:00
Haojun Liao	72f3e4446d	fix(stream): handle failure during checkpoint	2023-10-31 10:38:17 +08:00
yihaoDeng	7fad855247	fix mem leak	2023-10-30 18:16:12 +08:00
yihaoDeng	ee6b620597	fix mem leak	2023-10-30 17:29:32 +08:00
Haojun Liao	fdf3f210ac	refactor: do some internal refactor.	2023-10-30 14:39:24 +08:00
Haojun Liao	93232a0e14	fix(stream): enable the set of fill-history task to be un-init.	2023-10-30 11:48:51 +08:00
Haojun Liao	19c5882cbe	fix(stream): add event check when handling the end of event.	2023-10-30 10:22:24 +08:00
Haojun Liao	980f6fc46e	fix(stream): pause the task when they are still in uninit status.	2023-10-30 09:32:40 +08:00
Haojun Liao	1bbad532a7	fix(stream): add more check.	2023-10-30 09:32:40 +08:00
Haojun Liao	076b533ba3	fix(stream): fix race condition when state transferring.	2023-10-30 09:32:40 +08:00
Haojun Liao	85befa10b9	fix(stream): avoid clear the restart flag .	2023-10-30 09:32:39 +08:00
Haojun Liao	f708ddb792	fix(stream): pause the task when they are still in uninit status.	2023-10-30 09:09:21 +08:00
Haojun Liao	790d869494	fix(stream): add more check.	2023-10-29 22:57:33 +08:00
Haojun Liao	3c07c3ec2f	fix(stream): fix race condition when state transferring.	2023-10-29 02:27:27 +08:00
Haojun Liao	c7285c0ede	fix(stream): add expired epset node list for stream tasks in hb to mnode.	2023-10-27 23:41:57 +08:00
Haojun Liao	4c267b538f	fix(stream): add expired epset node list for stream tasks in hb to mnode.	2023-10-27 23:05:41 +08:00
Haojun Liao	62cc33715c	fix(stream): fix deadlock.	2023-10-27 15:05:36 +08:00
Haojun Liao	0c2079a963	fix(stream): fix deadlock.	2023-10-27 15:03:21 +08:00
yihaoDeng	ebdf6d1cc6	fix mem leak	2023-10-27 14:18:56 +08:00
Haojun Liao	afe7fdf672	fix(stream): avoid clear the restart flag .	2023-10-27 14:12:06 +08:00
Haojun Liao	dcca6b0868	other: merge 3.0.	2023-10-27 11:41:28 +08:00
Haojun Liao	29b3a7a5e9	refactor: do some internal refactor.	2023-10-27 09:59:53 +08:00
Haojun Liao	9c9d02b181	fix(stream): disable follower nodes exec reset.	2023-10-27 09:59:52 +08:00
Haojun Liao	52d8e05c0c	fix(stream): resolve the nodeUpdate msg compatible issue.	2023-10-27 09:59:52 +08:00
Haojun Liao	454624516c	fix(stream): handle the repeatly issued nodeUpdate msg from dnode.	2023-10-27 09:59:51 +08:00
Haojun Liao	9c9cd08e2e	fix(stream): not clear role flag.	2023-10-27 09:59:44 +08:00
Haojun Liao	33dc51cc86	fix(stream): add some logs.	2023-10-27 09:59:44 +08:00
Haojun Liao	1ccbb77981	fix(stream): fix dead lock when shutdown all tasks.	2023-10-27 09:59:37 +08:00
Haojun Liao	9a1a8feec2	fix(stream): do send check info if downstream is not leader.	2023-10-27 09:58:53 +08:00
Haojun Liao	2493c21ed3	fix(stream): fix the invalid write in sma	2023-10-27 09:58:52 +08:00
Haojun Liao	e8520997b0	fix(stream): disable continue check for downstream tasks.	2023-10-27 09:58:52 +08:00
Haojun Liao	3bcffcf1da	other: merge stream refactor from main.	2023-10-27 09:56:02 +08:00
Haojun Liao	b0ef072556	fix(stream): clear flag for checkpoint.	2023-10-27 09:55:07 +08:00
Haojun Liao	f682ad0479	refactor: do some internal refactor.	2023-10-27 09:55:06 +08:00
Haojun Liao	efede8669f	refactor: do some internal refactor.	2023-10-27 09:55:06 +08:00
Haojun Liao	9ebf806789	fix(stream): clear the flag.	2023-10-27 09:55:05 +08:00
Haojun Liao	48ff521968	fix(stream): update the nodeEp even during checkpoint trans.	2023-10-27 09:55:02 +08:00
Haojun Liao	0d4892f25b	fix(stream): fix bugs caused by refactor sm.	2023-10-27 09:55:00 +08:00
Haojun Liao	a31613ae95	refactor: add ref in log.	2023-10-27 09:54:59 +08:00
Haojun Liao	226b6596d6	fix(stream): fix bugs caused by refactor.	2023-10-27 09:54:58 +08:00
Haojun Liao	091238e7be	fix(stream): fix bugs caused by refactor sm.	2023-10-27 09:54:58 +08:00
Haojun Liao	e1f6043eed	fix(stream): fix bugs caused by refactor sm.	2023-10-27 09:54:57 +08:00
Haojun Liao	e359da7b19	fix(stream): fix bugs caused by refactor.	2023-10-27 09:54:57 +08:00
Haojun Liao	8729bdac1e	fix(stream): fix bugs caused by refactor.	2023-10-27 09:54:57 +08:00
Haojun Liao	1eac7f44c0	fix(stream): fix error in stream.	2023-10-27 09:54:56 +08:00
Haojun Liao	579dd8e6b2	refactor(stream): do some internal refactor.	2023-10-27 09:54:56 +08:00
Haojun Liao	6f3c0d17cc	refactor(stream): add state machine to manage the state of stream tasks.	2023-10-27 09:54:49 +08:00
Haojun Liao	596eb073b7	refactor(stream): do some internal refactor.	2023-10-27 09:54:48 +08:00
Haojun Liao	91e3fd66e6	enh(stream): sm for stream tasks.	2023-10-27 09:54:44 +08:00
Haojun Liao	2409a17517	refactor(stream): do some internal refactor.	2023-10-27 09:54:44 +08:00
Haojun Liao	c9bdb56d79	refactor: do some internal refactor.	2023-10-27 09:54:09 +08:00
Haojun Liao	164159f293	enh(stream): add parameter to limit the stream sink task.	2023-10-27 09:54:07 +08:00
Haojun Liao	62c58ecd8b	fix(stream):add more check for test cases.	2023-10-27 09:54:06 +08:00
Haojun Liao	a4f6042966	fix(stream): fix error in creating update node trans.	2023-10-27 09:52:14 +08:00
Haojun Liao	ffa798c4fa	enh(stream): add some logs.	2023-10-27 09:52:14 +08:00
yihaoDeng	7e1e68d8f5	fix stream case error	2023-10-26 18:27:09 +08:00
yihaoDeng	87db93e72e	fix stream case error	2023-10-26 11:36:29 +08:00
liuyao	d0e4a5d0f7	opt max delay	2023-10-25 20:29:25 +08:00
yihaoDeng	db0f6258fe	fix stream case error	2023-10-25 19:38:22 +08:00
liuyao	f51c985914	stream event window	2023-10-25 17:50:16 +08:00
Haojun Liao	f36b24ed24	fix(stream): disable follower nodes exec reset.	2023-10-25 14:16:36 +08:00
Haojun Liao	985371e841	fix(stream): resolve the nodeUpdate msg compatible issue.	2023-10-25 10:01:20 +08:00
Haojun Liao	03165d5327	fix(stream): handle the repeatly issued nodeUpdate msg from dnode.	2023-10-25 09:55:02 +08:00
Haojun Liao	26aaa4a83c	other: merge main.	2023-10-25 00:39:14 +08:00
Haojun Liao	5b42204dba	fix(stream): not clear role flag.	2023-10-25 00:32:46 +08:00
Haojun Liao	13d979a2ee	fix(stream): add some logs.	2023-10-25 00:03:46 +08:00
Haojun Liao	8cde39eebd	fix(stream): fix dead lock when shutdown all tasks.	2023-10-24 19:11:24 +08:00
yihaoDeng	d43b3b4a32	refactor stream backend	2023-10-24 16:17:54 +08:00
yihaoDeng	b3c54a0f06	refactor stream backend	2023-10-24 16:04:43 +08:00
Haojun Liao	6ffb5ee476	fix(stream): do send check info if downstream is not leader.	2023-10-24 15:49:50 +08:00
Haojun Liao	f1498f8929	fix(stream): fix the invalid write in sma	2023-10-24 15:24:34 +08:00
Haojun Liao	0f9328330b	fix(stream): disable continue check for downstream tasks.	2023-10-24 11:56:36 +08:00
yihaoDeng	d46cf878eb	refactor stream backend	2023-10-24 11:53:52 +08:00
Haojun Liao	7a8b046a35	fix(sma): fix bugs in sma and do some internal refactor.	2023-10-24 09:40:38 +08:00
yihaoDeng	09192acf96	fix mem leak	2023-10-23 10:20:57 +08:00
Haojun Liao	059e94e428	fix(stream): clear flag for checkpoint.	2023-10-22 01:25:00 +08:00
Haojun Liao	4826b34a2b	refactor: do some internal refactor.	2023-10-21 16:32:44 +08:00
Haojun Liao	06cd67e26e	refactor: do some internal refactor.	2023-10-21 16:20:03 +08:00
Haojun Liao	21e98e358e	fix(stream): clear the flag.	2023-10-21 02:41:03 +08:00
Haojun Liao	d1095e2c4c	Merge remote-tracking branch 'origin/refact/streamsm' into refact/streamsm	2023-10-21 01:42:59 +08:00
Haojun Liao	49014d4b68	fix(stream): update the nodeEp even during checkpoint trans.	2023-10-21 01:42:43 +08:00
yihaoDeng	aaf7e070ee	Merge branch 'refact/streamsm' of https://github.com/taosdata/TDengine into refact/streamsm	2023-10-20 20:49:17 +08:00
yihaoDeng	e95735eea4	fix mem leak	2023-10-20 20:49:01 +08:00
Haojun Liao	c68c0eda79	fix(stream): fix bugs caused by refactor sm.	2023-10-20 16:27:39 +08:00
Haojun Liao	4feebffb05	refactor: add ref in log.	2023-10-20 15:59:43 +08:00
Haojun Liao	c4268ca395	fix(stream): fix bugs caused by refactor.	2023-10-20 15:54:30 +08:00
yihaoDeng	1c3c58062d	Merge branch 'enh/new3.0' into enh/refactorBackend	2023-10-20 15:40:47 +08:00
yihaoDeng	1a4cfd11a5	Merge branch 'enh/new3.0' into enh/refactorBackend	2023-10-20 15:28:10 +08:00
Haojun Liao	3b8c85f632	fix(stream): fix bugs caused by refactor sm.	2023-10-20 14:19:57 +08:00
Haojun Liao	270a847809	fix(stream): fix bugs caused by refactor sm.	2023-10-20 10:23:48 +08:00
wade zhang	628667f22c	Merge pull request #23231 from taosdata/FEAT/TS-3816-3.0 ... feat: support restoring vnodes on replaced disks as needed	2023-10-20 09:02:25 +08:00
Haojun Liao	4d5d078fbe	fix(stream): fix bugs caused by refactor.	2023-10-19 18:00:54 +08:00
Haojun Liao	d5059826a8	fix(stream): fix bugs caused by refactor.	2023-10-19 17:59:04 +08:00
Haojun Liao	c1cebae6ba	fix(stream): fix error in stream.	2023-10-19 16:20:27 +08:00
Haojun Liao	614bdd9fb9	Merge pull request #23220 from taosdata/feat/TD-26644 ... opt bloom filter	2023-10-19 15:34:43 +08:00
Haojun Liao	da8f6a4c2f	Merge pull request #23248 from taosdata/feat/TD-26647 ... recover session buff state	2023-10-19 15:34:03 +08:00
Haojun Liao	402aefd95d	refactor(stream): do some internal refactor.	2023-10-19 15:09:47 +08:00
Benguang Zhao	c4e9069a66	fix: set nextProcessedVer properly in tqProcessTaskScanHistory	2023-10-19 11:18:57 +08:00
Haojun Liao	f4caeca24a	refactor(stream): add state machine to manage the state of stream tasks.	2023-10-19 10:34:20 +08:00
Haojun Liao	54ec679b58	refactor(stream): do some internal refactor.	2023-10-18 15:34:53 +08:00
yihaoDeng	c6ba2ca205	fix transfer crash	2023-10-18 14:49:12 +08:00
yihaoDeng	d6840ae07f	fix transfer crash	2023-10-18 14:22:52 +08:00
yihaoDeng	1b755dd4e7	fix transfer crash	2023-10-18 14:22:33 +08:00
yihaoDeng	e49b9b2276	fix transfer crash	2023-10-18 11:34:11 +08:00
Haojun Liao	2904cfd4d5	Merge branch 'main' into refact/streamsm	2023-10-18 10:29:09 +08:00
Haojun Liao	5115a1ac88	Merge pull request #23304 from taosdata/fix/TD-26771 ... double free	2023-10-18 10:09:17 +08:00
yihaoDeng	d67fdb5e46	fix transfer crash	2023-10-17 21:08:31 +08:00
yihaoDeng	2004c1a346	fix transfer crash	2023-10-17 18:19:52 +08:00
Haojun Liao	64ed7e49a2	enh(stream): sm for stream tasks.	2023-10-17 13:47:55 +08:00
yihaoDeng	77542cc461	fix stream snap deadlock	2023-10-17 11:23:39 +08:00
Haojun Liao	7b7ce10475	refactor(stream): do some internal refactor.	2023-10-17 11:23:04 +08:00
Haojun Liao	14035c763c	refactor: do some internal refactor.	2023-10-17 09:57:13 +08:00
Haojun Liao	fdb6ec1fa3	enh(stream): add parameter to limit the stream sink task.	2023-10-16 22:52:39 +08:00
liuyao	990d36654d	release buff	2023-10-16 19:49:45 +08:00
Haojun Liao	a1aa2c9e0c	fix(stream):add more check for test cases.	2023-10-16 18:47:01 +08:00
Haojun Liao	67bee17e5e	other: merge main.	2023-10-16 09:55:41 +08:00
Haojun Liao	f90cf54714	fix(stream): fix error in creating update node trans.	2023-10-15 21:56:49 +08:00
Haojun Liao	6611f216ac	enh(stream): add some logs.	2023-10-13 19:45:29 +08:00
liuyao	7269dedb39	Merge branch '3.0' of https://github.com/taosdata/TDengine into 3.0	2023-10-13 11:35:01 +08:00
liuyao	429125be93	session state recover	2023-10-13 11:32:35 +08:00
Haojun Liao	33d9f772be	other: merge 3.0	2023-10-13 09:06:13 +08:00
Haojun Liao	84c5359d5b	other: merge 3.0	2023-10-13 08:58:26 +08:00
yihaoDeng	d6ff282006	refactor backend	2023-10-12 20:42:36 +08:00
yihaoDeng	8c1a9f7c58	refactor backend	2023-10-12 20:28:40 +08:00
yihaoDeng	0ba7e2278a	refactor backend	2023-10-12 20:21:11 +08:00
yihaoDeng	a56d831a14	refactor backend	2023-10-12 20:16:41 +08:00
yihaoDeng	861d26e356	refactor backend	2023-10-12 15:24:50 +08:00
yihaoDeng	f704d4b82e	refactor backend	2023-10-12 11:45:18 +08:00
yihaoDeng	bfd205b3f7	refactor backend	2023-10-12 11:31:39 +08:00
Haojun Liao	eb4449999f	other: merge 3.0	2023-10-12 10:33:48 +08:00
yihaoDeng	4c94fbb44f	refactor backend	2023-10-11 20:26:19 +08:00
yihaoDeng	f264ffdcd6	refactor backend	2023-10-11 20:12:07 +08:00
yihaoDeng	83b52a4c1d	refactor backend	2023-10-11 20:02:40 +08:00
yihaoDeng	031506aef8	refactor backend	2023-10-11 19:46:35 +08:00
yihaoDeng	4f47162512	refactor backend	2023-10-11 16:36:43 +08:00
yihaoDeng	722d72659b	refactor backend	2023-10-11 16:10:16 +08:00
yihaoDeng	0baef335d7	refactor backend	2023-10-11 16:03:04 +08:00
yihaoDeng	4cff121a38	refactor backend	2023-10-11 14:46:41 +08:00
liuyao	e9fc079d26	opt bloom filter	2023-10-11 10:14:11 +08:00
Haojun Liao	1d4f0c8599	Merge pull request #23201 from taosdata/enh/TD-26646 ... Enh/td 26646	2023-10-10 22:59:37 +08:00
yihaoDeng	28421cd35e	refactor backend	2023-10-10 20:51:41 +08:00
Haojun Liao	05dfadd8f8	fix(stream): set initial version for fill-history task. disable reentry of scan-history task function.	2023-10-10 15:52:48 +08:00
Haojun Liao	5f9b858fc1	other: merge main.	2023-10-10 15:38:46 +08:00
yihaoDeng	b36163a197	add session iter	2023-10-10 15:12:21 +08:00
yihaoDeng	825341d90a	add sess iter	2023-10-10 14:35:02 +08:00
Haojun Liao	adf6f65d09	fix(stream): set initial version for fill-history task. disable reentry of scan-history task function.	2023-10-10 14:00:30 +08:00
Alex Duan	9da96ebf63	Merge pull request #23017 from taosdata/case/TD-26408-MAIN ... fix: add case for lost data after split vgroup (main)	2023-10-10 00:54:34 -05:00
Alex Duan	e566027cef	Merge pull request #23019 from taosdata/case/TD-26408-3.0 ... fix: add case for lost data after split 3.0	2023-10-10 00:53:56 -05:00
yihaoDeng	6adf7532a8	add sess iter	2023-10-10 11:35:18 +08:00
yihaoDeng	7ac4ef5beb	add sess iter	2023-10-10 10:41:46 +08:00
yihaoDeng	0fc6ffe6b6	change test case	2023-10-09 20:51:26 +08:00
Haojun Liao	a25d94ab6f	fix(stream): retry when error occurs during timer.	2023-10-09 20:10:28 +08:00
Haojun Liao	7d39164133	fix(stream): disable retrieving results during checkpoint procedure	2023-10-09 20:10:27 +08:00
Haojun Liao	af4618f958	fix(stream): remove invalid clear of rspmsg list.	2023-10-09 20:10:27 +08:00
Haojun Liao	847baa8b71	fix(stream): fix the pause failure bug.	2023-10-09 20:10:27 +08:00
Haojun Liao	21fb502690	enh(stream): add trans to reset task status to avoid being frozen in checkpoint status due to doing checkpoint failure of partial tasks.	2023-10-09 20:10:26 +08:00
Haojun Liao	64bde93f85	fix(stream): not idle for sink task when no tokens are available.	2023-10-09 20:10:25 +08:00
Haojun Liao	9a7167aaba	fix(stream): sleep for 10ms	2023-10-09 20:10:24 +08:00
Haojun Liao	9af8ce21ed	fix(stream): wait for a while when no available token.	2023-10-09 20:10:23 +08:00
Haojun Liao	4780474877	fix(stream): continue	2023-10-09 20:10:23 +08:00
Haojun Liao	da833ccb30	refactor: inc the dump batch.	2023-10-09 20:10:23 +08:00
Haojun Liao	2936ac9b89	fix(stream): add missing refactor.	2023-10-09 20:10:22 +08:00
Haojun Liao	af2c0c95e9	refactor: do some internal refactor.	2023-10-09 20:10:22 +08:00
Haojun Liao	940d6a3cb4	fix(stream): check items in queue and qall, stead of only the queue items.	2023-10-09 20:10:21 +08:00
Haojun Liao	e95aea84ae	fix(stream): fix syntax error.	2023-10-09 20:10:21 +08:00
Haojun Liao	59e284c332	fix(stream): quit from loop when input queue is full.	2023-10-09 20:10:21 +08:00
Haojun Liao	9d5a3b8d78	other: merge stream fix.	2023-10-09 20:10:13 +08:00
Haojun Liao	c5997bee5b	fix(stream): update the current processing version.	2023-10-09 20:09:23 +08:00
Haojun Liao	28008f8263	enh(stream): add more info for sink task.	2023-10-09 20:09:22 +08:00
Haojun Liao	b3b9e89d30	fix(stream): fix the invalid check.	2023-10-09 20:09:22 +08:00
Haojun Liao	cf14772dd9	enh(stream): optimize the scan wal perf	2023-10-09 20:09:21 +08:00
Haojun Liao	d2160efa05	refactor: wait for 100ms before scan wal again.	2023-10-09 20:09:21 +08:00
Haojun Liao	932d93ab35	refactor: wait for more time for future data	2023-10-09 20:09:21 +08:00
Haojun Liao	7682c033ad	fix(stream): add null check.	2023-10-09 20:09:21 +08:00
Haojun Liao	0ab5d4b342	refactor: do some internal refactor.	2023-10-09 20:09:20 +08:00
Haojun Liao	c295ec322a	refactor: do some internal refactor.	2023-10-09 20:09:20 +08:00
Haojun Liao	d5abb21335	fix(stream): fix invalid check.	2023-10-09 20:09:20 +08:00
Haojun Liao	583ac1fc45	refactor: do some internal refactor.	2023-10-09 20:09:20 +08:00
Haojun Liao	83f84d92bc	refactor: do some internal refactor (discard the checkpoint source msg during restoring).	2023-10-09 20:09:10 +08:00
Haojun Liao	bb4ba54f28	fix(stream): enable new time range for stream task.	2023-10-09 20:09:09 +08:00
Haojun Liao	c759d90bf6	enh(stream): optimize the stream status when creating without fill-history option.	2023-10-09 20:09:09 +08:00
Haojun Liao	db897fb03a	fix(stream): opt perf.	2023-10-09 20:09:09 +08:00
Haojun Liao	0a2b70fd7f	fix(stream): fix syntax error.	2023-10-09 20:09:06 +08:00
Haojun Liao	8477271240	fix(stream): fix dead lock.	2023-10-09 20:09:00 +08:00
Haojun Liao	12eabb0c2c	log(stream): update logs.	2023-10-09 20:09:00 +08:00
Haojun Liao	b716f0d1fd	fix(stream): fix dead lock.	2023-10-09 20:08:57 +08:00
Haojun Liao	60a720efbb	log: update logs.	2023-10-09 20:08:53 +08:00
Haojun Liao	d10b07d23d	refactor: do some internal refactor.	2023-10-09 20:08:53 +08:00
Haojun Liao	6d2dbd549b	fix(stream): avoid repeatly free obj.	2023-10-09 20:08:53 +08:00
Haojun Liao	6b5770aab7	log(stream): update the log.	2023-10-09 20:08:52 +08:00
Haojun Liao	dfe8641c0c	refactor: optimize the fill-history task launch policy, and do some other internal refactor.	2023-10-09 20:08:52 +08:00
Haojun Liao	38164435f9	refactor: do some internal refactor.	2023-10-09 20:08:52 +08:00
Haojun Liao	f2848de12a	fix(stream): reduce the counter when remove stream tasks.	2023-10-09 20:08:52 +08:00
Haojun Liao	ecb3b44026	fix(stream): limit sink quota.	2023-10-09 20:08:52 +08:00
Haojun Liao	2c0e54f4f7	fix(stream): update the retry interval.	2023-10-09 20:08:13 +08:00
Haojun Liao	7f8da725bc	enh(stream): add quota limitation for sink task.	2023-10-09 20:08:13 +08:00
Haojun Liao	a248f3d088	refactor: do some internal refactor.	2023-10-09 20:08:12 +08:00
Haojun Liao	0db357401d	fix(stream): add new status for meta.	2023-10-09 20:08:12 +08:00
Haojun Liao	3912d80c02	fix(stream): do some internal refactor.	2023-10-09 20:08:11 +08:00
Haojun Liao	feb09c8361	fix(stream): fix error in pause scan-history data.	2023-10-09 20:08:11 +08:00
Haojun Liao	6a98b11bef	fix(stream): fix invalid read and memory leak.	2023-10-09 20:08:11 +08:00
Haojun Liao	65f66f0986	fix(stream): transfer state when all downstream rsp already.	2023-10-09 20:08:11 +08:00
Haojun Liao	a32b56f381	fix(stream): fix memory leak.	2023-10-09 20:08:10 +08:00
Haojun Liao	4af7812f0b	fix(stream): fix the invalid counter for ready tasks in vnode, fix an invalid read.	2023-10-09 20:08:10 +08:00
Haojun Liao	4fe1382cd9	fix(stream): adjust nodeUpdate check interval.	2023-10-09 20:08:09 +08:00
Haojun Liao	f4f55fcbce	fix(stream): record the tasks started info, and do some internal refactor.	2023-10-09 20:08:09 +08:00
Haojun Liao	68b003c804	fix(stream): fix race condition.	2023-10-09 20:08:09 +08:00
Haojun Liao	fc96ec6bae	fix(stream): do not wait for the checkpoint finish before stop tasks.	2023-10-09 20:08:09 +08:00
Haojun Liao	7167c3c5de	fix(stream): add new timer for each task.	2023-10-09 20:08:08 +08:00
Haojun Liao	7336228c16	refactor: do some internal refactor.	2023-10-09 20:08:08 +08:00
Haojun Liao	477f57aa0e	refactor: do some internal refactor.	2023-10-09 20:08:08 +08:00
Haojun Liao	52763ca2a3	refactor: do some internal refactor.	2023-10-09 20:08:07 +08:00
Haojun Liao	3baddbe67a	fix(stream): update the checkpoint record start time.	2023-10-09 20:08:07 +08:00
Haojun Liao	a6ffb845c8	fix(stream): fix invalid checkpoint start time.	2023-10-09 20:08:07 +08:00
Haojun Liao	c05a8de3e8	fix(stream): rsp when checkpoint source failed.	2023-10-09 20:08:06 +08:00
Haojun Liao	b4bc4fac98	fix(stream): disable auto rsp for checkpoint source.	2023-10-09 20:08:06 +08:00
Haojun Liao	e671e87234	fix(stream): fix invalid assert.	2023-10-09 20:08:06 +08:00
Haojun Liao	d1a6b8bd22	refactor: do some internal refactor.	2023-10-09 20:08:06 +08:00
Haojun Liao	46b17aa9f1	fix(stream): fix race condition.	2023-10-09 20:08:06 +08:00
Haojun Liao	e49409a11e	fix(stream): add some logs.	2023-10-09 20:08:05 +08:00
Haojun Liao	5eee76df65	fix(stream): add msg identifier check.	2023-10-09 20:08:05 +08:00
Haojun Liao	192c47a7c2	fix(stream): add some logs.	2023-10-09 20:08:05 +08:00
Haojun Liao	693254f602	fix(stream): remove invalid free.	2023-10-09 20:08:04 +08:00
Haojun Liao	f3bd5c411d	fix(stream): remove the blocking flag after successfully retry sending msg.	2023-10-09 20:08:04 +08:00
Haojun Liao	ed9e035095	fix(stream): clear the failure vnode id list before send msg.	2023-10-09 20:08:04 +08:00
Haojun Liao	3726aa723e	fix(stream): fix invalid read.	2023-10-09 20:08:04 +08:00
Haojun Liao	e0d19af000	fix(stream): refactor re-try dispatch msg for stream tasks.	2023-10-09 20:08:03 +08:00
Haojun Liao	2b16976902	refactor(stream): refactor send msg.	2023-10-09 20:08:03 +08:00
Haojun Liao	27e00b1f19	fix(stream): add some logs, and remove invalid assert.	2023-10-09 20:08:02 +08:00
Haojun Liao	12687eb287	fix(stream): add check info detailed information.	2023-10-09 20:08:02 +08:00
Haojun Liao	ab7c18f604	refactor: do some internal refactor.	2023-10-09 20:08:01 +08:00
Haojun Liao	e5535fa1f5	fix(stream): handle dispatch checkpoint-trigger failure.	2023-10-09 20:08:01 +08:00
Haojun Liao	65cc4040f3	fix(stream): update logs.	2023-10-09 20:08:00 +08:00
Haojun Liao	6d3a134077	fix(stream): ignore the dispatch failure, and set the correct rsp counter.	2023-10-09 20:07:54 +08:00
Haojun Liao	7f4d9b6e41	fix(stream): wait for the task checkpoint before stop.	2023-10-09 20:07:53 +08:00
Haojun Liao	ef9efc8a8e	refactor: do refactor the sink task.	2023-10-09 20:07:53 +08:00
Haojun Liao	e4ac835554	fix(stream): disable token bucket.	2023-10-09 20:07:53 +08:00
Haojun Liao	a3d13d6634	fix(stream): remove invalid cached task info in buffer.	2023-10-09 20:07:08 +08:00
Haojun Liao	1f45bd82d2	other: merge stream fix in main.	2023-10-09 20:07:01 +08:00
Haojun Liao	d752fb1ed6	fix(stream): exec task in task queuee threads.	2023-10-09 20:06:11 +08:00
Haojun Liao	1ebafe4a1f	fix(stream): add more check.	2023-10-09 20:06:11 +08:00
Haojun Liao	c403a15f36	fix(stream): check the status before exec stream task when handling the dispatch requests.	2023-10-09 20:06:11 +08:00
Haojun Liao	d76e3ac718	fix(stream): add task status check.	2023-10-09 20:06:10 +08:00
Haojun Liao	e27111ecd1	enh(stream): add node stage check.	2023-10-09 20:06:09 +08:00
Haojun Liao	ce85945583	fix(stream): set correct size for results generated by scan history stream tasks.	2023-10-09 20:06:05 +08:00
Haojun Liao	053643be19	fix(stream): add timestamp.	2023-10-09 20:06:05 +08:00
Haojun Liao	b38f9e0288	fix(stream): update logs.	2023-10-09 20:06:05 +08:00
Haojun Liao	5c0b8ea804	enh(stream): log the checkpoint time.	2023-10-09 20:06:04 +08:00
Haojun Liao	b95ad74c7f	fix(stream): not handle the check msg for follower tasks.	2023-10-09 20:06:04 +08:00
Haojun Liao	e6fb9ee46c	fix(stream): reset task counter.	2023-10-09 20:06:04 +08:00
Haojun Liao	c3a3cf21fb	fix(stream): add some logs.	2023-10-09 20:05:39 +08:00
Haojun Liao	08b37dfc13	fix(stream): fix memory leak.	2023-10-09 20:05:38 +08:00
Haojun Liao	734f648891	fix(stream): set the output normal for stream dispatch when encountering the transfer state.	2023-10-09 20:05:38 +08:00
Haojun Liao	636eccc160	fix(stream): add null check	2023-10-09 20:05:37 +08:00
Haojun Liao	57bf1cadc4	fix(stream): fix error in decode stream task.	2023-10-09 20:05:37 +08:00
Haojun Liao	9d6199adf7	fix(stream): fix invalid read.	2023-10-09 20:05:37 +08:00
Haojun Liao	0459a4628c	fix(stream): fix stream task id error.	2023-10-09 20:05:36 +08:00
Haojun Liao	7ccc273206	other: merge stream fix in main.	2023-10-09 20:05:29 +08:00
Haojun Liao	98ef566eb4	fix(stream): remove fill-history sink task.	2023-10-09 20:04:17 +08:00
Haojun Liao	e84eeee6b8	refactor: do some internal refactor.	2023-10-09 20:04:17 +08:00
Haojun Liao	fe7864b976	fix(stream): fix error in drop task.	2023-10-09 20:04:17 +08:00
Haojun Liao	c33e521aa1	fix(stream): update some logs.	2023-10-09 20:04:17 +08:00
Haojun Liao	39e0c57323	fix(stream): set the correct guard flag.	2023-10-09 20:04:16 +08:00
Haojun Liao	3d1f29ff21	fix(stream): fix memory leak.	2023-10-09 20:04:16 +08:00
yihaoDeng	c6e9134cea	add sess iter	2023-10-09 20:02:23 +08:00
Haojun Liao	6fa54789cb	fix(stream): disable follower send hb to mnode. and do some internal refactor.	2023-10-09 19:59:31 +08:00
Haojun Liao	1256eafddb	other: merge stream fix.	2023-10-09 19:59:24 +08:00
Haojun Liao	283118bf68	fix(stream):fix missing unlock.	2023-10-09 19:56:17 +08:00
Haojun Liao	aaa12ed4f8	refactor: do some internal refactor about the sched status set	2023-10-09 19:56:17 +08:00
Haojun Liao	01d104706e	refactor(stream): remove unnecessary streamschedTask function to improve the performance.	2023-10-09 19:56:17 +08:00
Haojun Liao	a678e722ce	fix(stream): limit the token bucket capacity	2023-10-09 19:56:17 +08:00
Haojun Liao	b0ef091430	fix(stream): retry when error occurs during timer.	2023-10-09 16:40:05 +08:00
Haojun Liao	be0a2e4573	fix(stream): disable retrieving results during checkpoint procedure	2023-10-09 16:33:28 +08:00
yihaoDeng	2aa88dfe9b	fix mem leak	2023-10-09 15:53:42 +08:00
Haojun Liao	8266c4ff77	fix(stream): remove invalid clear of rspmsg list.	2023-10-09 15:26:42 +08:00
Haojun Liao	d87db2f829	fix(stream): fix the pause failure bug.	2023-10-09 14:48:11 +08:00
yihaoDeng	28b546a4ad	fix mem leak	2023-10-09 14:25:57 +08:00
yihaoDeng	be42143362	Merge branch '3.0' into case/TD-26408-3.0	2023-10-09 14:24:31 +08:00
yihaoDeng	3f9784f475	fix mem leak	2023-10-09 14:07:22 +08:00
yihaoDeng	0b92a6b1ea	fix mem leak	2023-10-09 11:16:16 +08:00
Haojun Liao	74ff0c5b50	Merge branch 'main' into fix/liaohj	2023-10-09 10:14:53 +08:00
Haojun Liao	ee6516f3c4	enh(stream): add trans to reset task status to avoid being frozen in checkpoint status due to doing checkpoint failure of partial tasks.	2023-10-09 10:13:53 +08:00
Haojun Liao	4b8cd3bccd	fix(stream): not idle for sink task when no tokens are available.	2023-10-09 10:07:19 +08:00
yihaoDeng	f49a2cbeb0	refact task backend	2023-10-08 18:14:17 +08:00
liuyao	1cbb8ef444	init bloomfilter array	2023-10-08 14:43:44 +08:00
yihaoDeng	b5994b31e2	fix mem leak	2023-10-08 13:33:02 +08:00
yihaoDeng	e3f18495e0	fix mem leak	2023-10-08 11:22:26 +08:00
yihaoDeng	2fa91341a7	refact task backend	2023-10-08 10:35:49 +08:00
Haojun Liao	23fb26798f	fix(stream): sleep for 10ms	2023-10-07 19:47:49 +08:00
Haojun Liao	e4e4c5a36e	fix(stream): wait for a while when no available token.	2023-10-07 19:41:49 +08:00
Haojun Liao	45bb3f50d7	fix(stream): continue	2023-10-07 19:25:40 +08:00
liuyao	95ffe6fa23	stream recover	2023-10-07 19:24:21 +08:00
Haojun Liao	62cdcbc7e3	refactor: inc the dump batch.	2023-10-07 19:20:48 +08:00
Haojun Liao	4d6f143146	fix(stream): add missing refactor.	2023-10-07 19:18:26 +08:00
Haojun Liao	ab17bfdf09	refactor: do some internal refactor.	2023-10-07 19:16:55 +08:00
Haojun Liao	0e8a8aabc1	fix(stream): check items in queue and qall, stead of only the queue items.	2023-10-07 19:16:25 +08:00
liuyao	1e323c0408	fix issue	2023-10-07 19:11:46 +08:00
yihaoDeng	98099ebbae	fix mem leak	2023-10-07 18:41:27 +08:00
liuyao	19bbf10579	fix issue	2023-10-07 17:59:53 +08:00
yihaoDeng	97fdcbc60b	fix mem leak	2023-10-07 17:48:24 +08:00
liuyao	1aaf873134	fix issue	2023-10-07 17:43:50 +08:00