Commit Graph

623 Commits

Author SHA1 Message Date
Haojun Liao 025437df0c fix(stream): fix error in checkpoint consensus. 2024-06-27 19:25:18 +08:00
Haojun Liao 48bf5eb4d8 other: merge 3.0 2024-06-27 19:23:06 +08:00
Yihao Deng 1004ac6924 add self check 2024-06-27 05:28:17 +00:00
Yihao Deng f2fc09cd02 add self check 2024-06-27 01:48:59 +00:00
Minglei Jin 3014c66965 fix(streamMeta): commit tdb after load tasks 2024-06-26 11:21:31 +08:00
Haojun Liao 1818edcb2b fix(stream): consensus the start checkpoint id, and extract the streamhb related functions into a new file. 2024-06-26 10:18:32 +08:00
Yihao Deng 33aef6ddc5 add self check info 2024-06-25 07:12:09 +00:00
Haojun Liao 2cb9644471 fix(stream): update the checkpoint info only when the status is normal. 2024-06-21 09:23:29 +08:00
Haojun Liao c4a2756907 fix(stream): not revise the stream id for fill-history task. 2024-06-17 23:13:54 +08:00
Haojun Liao f57ff96408 fix(stream): do some refactor. 2024-06-17 19:56:18 +08:00
Haojun Liao 60fac80345 fix(stream): add some logs. 2024-06-17 19:34:09 +08:00
Haojun Liao 0f4ffdf49a fix(stream): add some info. 2024-06-17 18:57:31 +08:00
Haojun Liao bdbcdf3465 refactor: add two assert. 2024-06-17 16:58:58 +08:00
Haojun Liao 848b1e19b5 fix(stream): fix some typo. 2024-06-17 14:39:46 +08:00
Haojun Liao 42e0dff589 fix(stream): move the expansion of the stream task related fill-history task to the place where the fill-history task must have been deoployed already. 2024-06-14 17:03:04 +08:00
Haojun Liao a41a04dc37 fix(stream): remove the related history task correctly. 2024-06-11 18:35:59 +08:00
Haojun Liao 3b3ed1c30c fix(stream): add monitor for checkpoint-ready msg recv status. 2024-06-03 16:45:37 +08:00
Haojun Liao ee4e4c0e6d refactor: do some internal refactor. 2024-05-31 09:17:20 +08:00
Haojun Liao cc318d7e2d enh(stream): sink task does not initialize the backend. 2024-05-28 11:34:04 +08:00
Haojun Liao bd514fe4d3 other: merge rows. 2024-05-28 11:13:09 +08:00
Haojun Liao be107b204c enh(stream): support retrieve checkpoint data remotely when start stream tasks in vnodes. 2024-05-28 09:54:22 +08:00
Haojun Liao 01fb950cbc refactor: add some logs. 2024-05-24 11:44:17 +08:00
Haojun Liao 28032dfd39 fix(stream): fix memory leak if write failed. 2024-05-24 11:39:02 +08:00
Haojun Liao 69567799eb fix(stream): initialization tasks before start all tasks. 2024-05-23 15:45:59 +08:00
Haojun Liao 61b08259ba fix(stream): initialization fill-history tasks before start all other stream tasks. 2024-05-23 15:38:16 +08:00
Haojun Liao 96e1487787 refactor: do some internal refactor. 2024-05-22 16:57:09 +08:00
Haojun Liao be39044b66 fix(stream): add more info 2024-05-22 11:04:54 +08:00
Haojun Liao 130f7a292d fix(stream): reset the error code when trying to restore checkpoint. 2024-05-19 22:36:09 +08:00
Haojun Liao abd90c733a fix(stream): remove backend files only after the in-memory task is dropped. 2024-05-15 14:24:35 +08:00
Haojun Liao 042ed3caff fix(stream): fix invalid unlock. 2024-05-15 10:08:56 +08:00
Haojun Liao 610aa80e65 fix(stream): init backend for fill-history task. 2024-05-15 00:08:38 +08:00
Haojun Liao 4eacf86843 fix(tsdb): fix deadlock when stopping reader. 2024-05-14 19:20:19 +08:00
Haojun Liao 241f8b3d2e fix(tsdb): fix deadlock when stopping reader. 2024-05-14 18:05:13 +08:00
Haojun Liao 6898eba6ed fix(stream): expand stream tasks are divided into two phase, the first is to build stream task and then expand stream task before start stream tasks. 2024-05-14 14:34:31 +08:00
Haojun Liao 7f93ec2c53 fix(stream): update the checkpoint info for followers, and some internal refactor. 2024-05-11 09:37:02 +08:00
Haojun Liao 6721ec3cd8 refactor: do some internal refactor. 2024-05-09 14:06:25 +08:00
Haojun Liao 6693efaa85 refactor: do some internal refactor. 2024-05-09 10:11:37 +08:00
Haojun Liao 4f59754a63 refactor: do some internal refactor. 2024-05-08 16:22:02 +08:00
Haojun Liao bc04093d11 refactor: do some internal refactor. 2024-05-08 13:54:25 +08:00
Haojun Liao 9ce8f6953d refactor: do some internal refactor. 2024-05-08 13:52:20 +08:00
Haojun Liao cdeb4462fb refactor: do some internal refactor. 2024-05-07 23:43:21 +08:00
Haojun Liao c982f6ac80 refactor: do some internal refactor. 2024-05-06 17:08:11 +08:00
Haojun Liao 7ad7ef7cfc refactor: do some internal refactor. 2024-05-06 17:03:41 +08:00
Haojun Liao 46e9fe6f97 fix(stream): set global close flag in the streamMeta. 2024-05-06 17:03:09 +08:00
Haojun Liao d879c7c967 fix(stream): set global close flag in the streamMeta. 2024-05-06 16:40:49 +08:00
Haojun Liao 05a204dd6c refactor: do some internal refactor. 2024-04-30 19:30:56 +08:00
Haojun Liao 732e52d64d refactor: do some internal refactor, and add some logs. 2024-04-30 19:11:58 +08:00
Haojun Liao ed610d293f fix(stream): close task if it's in checkdown stream procedure. 2024-04-30 17:09:37 +08:00
Haojun Liao 6c93fe5593 fix(stream): fix dead lock. 2024-04-28 18:35:00 +08:00
Haojun Liao b990632e8d fix(stream): fix dead lock. 2024-04-28 18:29:18 +08:00
Haojun Liao bf5d523116 fix(stream): disable the exec of complete check status in timer thread. 2024-04-28 17:57:34 +08:00
Haojun Liao 8adaeb59a3 refactor: do some internal refactor. 2024-04-28 09:50:21 +08:00
Haojun Liao fae53efed9 fix(stream): add some logs for retry for notready/timeout downstream tasks. and do some internal refactor. 2024-04-26 09:58:20 +08:00
Haojun Liao adfc1fb927 Merge branch '3.0' into fix/3_liaohj 2024-04-19 16:24:53 +08:00
Haojun Liao 3baa47f266 fix(stream): record the task check downstream failed info. 2024-04-19 16:21:05 +08:00
Hongze Cheng fc5c84422e
Merge pull request #25256 from taosdata/feat/TD-24834
Feat/td 24834
2024-04-19 13:34:37 +08:00
Haojun Liao e7dee4ac63 refactor(stream): refactor check msg retry procedure and be tolerant to the dnode disconnection during check downstream tasks. 2024-04-18 15:55:21 +08:00
Haojun Liao ac84a5f82b refactor: do some internal refactor. 2024-04-17 21:57:09 +08:00
Haojun Liao ddf98ae0c2 refactor: do some internal refactor. 2024-04-17 18:04:02 +08:00
dm chen aedec5ead7
Merge branch '3.0' into feat/TD-24834 2024-04-16 11:35:49 +08:00
Haojun Liao 665107ad3a enh(stream): add more info in meta table. 2024-04-15 14:23:37 +08:00
Haojun Liao 15c18af221 fix(stream): fix init error. 2024-04-13 18:27:38 +08:00
kailixu 3738ce9e10 chore: solve conflicts of sql.c 2024-04-11 11:05:58 +08:00
Haojun Liao f23a8a37bc refactor: do some internal refactor. 2024-04-10 16:43:02 +08:00
kailixu b4c8375cb9 chore: solve conflicts 2024-04-07 14:53:05 +08:00
dmchen b9deb71fc5 basic function 2024-03-29 02:48:49 +00:00
wangmm0220 8e2a2414c0 fix:same subtable same partition by leads to same table name in stream 2024-03-28 13:48:46 +08:00
Haojun Liao 5157d5b97d fix(stream): fix invalid read. 2024-03-27 15:53:24 +08:00
Haojun Liao a11b2c614e fix(stream): not return error when failing to load stream task meta. 2024-03-25 23:49:26 +08:00
Haojun Liao 6763ac9110 fix(stream):adjust current offset by using the scanning version, not the processed version in WAL. 2024-03-18 09:12:53 +08:00
Haojun Liao 8791a163f8 other: merge 3.0 2024-02-29 22:31:52 +08:00
Haojun Liao db92bf84d8 other: merge 3.0 2024-02-29 22:21:50 +08:00
yihaoDeng c54e4aeaad add test case 2024-02-23 09:30:53 +00:00
Haojun Liao b27b635c10 fix(stream): allow hb message rsp. 2024-02-21 13:55:06 +08:00
Haojun Liao 7e866c5527 refactor: do some internal refactor. 2024-02-19 11:32:04 +08:00
Haojun Liao 7a18e5910c enh(stream): halt the initial stream tasks when fill-history is set for count window agg 2024-02-05 17:51:09 +08:00
Haojun Liao cb2ea4a721 fix(stream): take ver snapshot for all vgroups before launching stream with fill-history option opened. 2024-02-01 11:37:57 +08:00
Haojun Liao 4970040459 refactor: do some internal refactor. 2024-01-30 15:22:49 +08:00
Haojun Liao 3730e43f0e refactor: disable some logs. 2024-01-29 17:24:58 +08:00
Haojun Liao d4bab8c09b refactor: do some internal refactor. 2024-01-25 16:53:30 +08:00
Haojun Liao dc1ea9f9a1 fix(stream): clear htask info when unregistering the task. 2024-01-24 16:23:47 +08:00
Haojun Liao 1a45d40607 fix(stream): fix deadlock. 2024-01-24 14:37:46 +08:00
Haojun Liao 8a7e38ad15 fix(stream): fix checkpoint failure check. 2024-01-24 13:10:53 +08:00
Haojun Liao 8620e88965 fix(stream): enable wait for timer for meta quit safely. 2024-01-18 14:17:08 +08:00
Haojun Liao fc16d26da2
Merge pull request #24488 from taosdata/fix/3_liaohj
fix(stream): fix some errors and do some internal refactor.
2024-01-17 14:25:32 +08:00
Haojun Liao 8ede6d87ce refactor: do some internal refactor. 2024-01-16 15:40:40 +08:00
jiajingbin d59ede5196 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-28032 2024-01-16 15:05:13 +08:00
Haojun Liao 80793c94c2 fix(stream): reset the tasks status before start all tasks, and do some other internal refactor. 2024-01-16 14:32:45 +08:00
Haojun Liao 63494de7f9 refactor: do some internal refactor. 2024-01-16 10:28:07 +08:00
Haojun Liao 43081b540b fix(stream): reset restart counter to be 0 if all tasks are ready. 2024-01-15 16:16:28 +08:00
Haojun Liao e012bc4bde refactor: do some internal refactor. 2024-01-15 11:28:36 +08:00
Haojun Liao 00e2bdec23 refactor: do some internal refactor. 2024-01-12 19:08:46 +08:00
Haojun Liao 5fa8f85c42 fix(stream): remove invalid release. 2024-01-12 11:28:41 +08:00
Haojun Liao 58694c67dd refactor: do some internal refactor. 2024-01-11 11:50:52 +08:00
Haojun Liao bff0ad89a4 other: merge 3.0 2024-01-11 09:51:12 +08:00
Haojun Liao 1c562bd535 refactor: enable the restart of stream tasks. 2024-01-11 09:41:40 +08:00
Haojun Liao 9294c8d4ee fix(stream): enable the hb to mnode for stream meta when it becomes the leader. 2024-01-11 09:20:34 +08:00
wangmm0220 df69a89da2 fix:[TD-28185]add pause & resume logic for snode 2024-01-10 18:46:10 +08:00
Haojun Liao 9d0d0d9ddc Merge branch '3.0' into fix/3_liaohj 2024-01-10 09:57:46 +08:00
Haojun Liao dc0c5539a0 fix(stream): remove invalid timer controllers. 2024-01-10 08:48:49 +08:00