Commit Graph

796 Commits

Author SHA1 Message Date
Haojun Liao 3628888ecc refactor: do some internal refactor. 2023-05-29 18:02:45 +08:00
Haojun Liao cd2d225c93 refactor: do some internal refactor. 2023-05-29 17:50:39 +08:00
yihaoDeng 1782ecd57e Merge branch 'fix/shared_link_rocksdb' into enh/addCompileError 2023-05-29 08:20:11 +00:00
Haojun Liao f2887278e2 refactor: do some internal refactor. 2023-05-29 15:44:45 +08:00
yihaoDeng 26cbcf996c Merge branch 'enh/3.0' into fix/shared_link_rocksdb 2023-05-29 07:31:14 +00:00
yihaoDeng 342ce14c7d Merge branch 'enh/3.0' into fix/shared_link_rocksdb 2023-05-29 02:37:47 +00:00
yihaoDeng 58cc631c22 Merge branch 'enh/3.0' into fix/shared_link_rocksdb 2023-05-29 02:29:56 +00:00
yihaoDeng 0be1858b4b Merge branch 'enh/dev3.0' into enh/addCompileError 2023-05-29 01:50:43 +00:00
yihaoDeng de739ce086 Merge branch 'enh/dev3.0' into enh/addCompileError 2023-05-29 01:11:13 +00:00
yihaoDeng 2f5fe2dec6 Merge branch 'enh/dev3.0' into enh/addCompileError 2023-05-26 14:01:05 +00:00
yihaoDeng 4b931d3007 Merge branch 'enh/dev3.0' into enh/addCompileError 2023-05-26 13:59:16 +00:00
yihaoDeng a71bffd719 Merge branch 'enh/dev3.0' into enh/addCompileError 2023-05-26 13:03:35 +00:00
yihaoDeng 57a4e56a61 change parameter 2023-05-26 20:30:03 +08:00
yihaoDeng 34d21bc03f change parameter 2023-05-26 20:11:33 +08:00
liuyao 5ebe75599e fix pause issue 2023-05-26 14:58:39 +08:00
Haojun Liao 1e845acabe fix(tmq): allow seek before get assignment 2023-05-26 11:34:12 +08:00
yihaoDeng 52c384c7c9 change parameter 2023-05-26 09:06:19 +08:00
Haojun Liao cea8334427
Merge pull request #21449 from taosdata/refact/fillhistory
refactor: remove the module dependency.  TD-24279
2023-05-26 09:01:45 +08:00
Haojun Liao 30de31e374 fix(test): fix unit test link error. 2023-05-25 18:17:09 +08:00
yihaoDeng cd44bd9f3f change parameter 2023-05-25 18:16:33 +08:00
yihaoDeng 9cb4a114e2 change parameter 2023-05-25 16:20:00 +08:00
yihaoDeng 2f90fda13d change parameter 2023-05-25 16:15:10 +08:00
Haojun Liao 0966213847 fix(arch): set the correct module api dependency. 2023-05-25 14:21:40 +08:00
Haojun Liao 01f6f74fb9 fix: update the cmake file. 2023-05-24 22:27:45 +08:00
Hongze Cheng 4d526ed0cd change more 2023-05-24 18:59:09 +08:00
Haojun Liao fe09705855 fix: set the init function pointer. 2023-05-24 17:13:20 +08:00
Haojun Liao a566180770 fix: fix syntax error. 2023-05-24 15:16:50 +08:00
Haojun Liao 1f7f326bed refactor: do some internal refactor and set the api function ptr. 2023-05-24 13:22:05 +08:00
Haojun Liao e67b532d8a refactor: refactor the module dependency. 2023-05-23 18:29:23 +08:00
Haojun Liao f520de0dab refactor: adjust the module dependency. 2023-05-22 10:07:01 +08:00
Haojun Liao 9423001fea
Merge pull request #21396 from taosdata/refact/fillhistory
Refact/fillhistory
2023-05-21 01:12:00 +08:00
Haojun Liao dcb91082f9 fix(stream): fix error. 2023-05-20 23:37:40 +08:00
Haojun Liao cb75e5a863 fix(stream): fix memory leak. 2023-05-20 22:07:04 +08:00
Haojun Liao 899e4d3350 refactor: do some internal refactor. 2023-05-20 18:13:58 +08:00
Haojun Liao de44c9160c refactor: do some internal refactor. 2023-05-20 18:05:39 +08:00
Haojun Liao 3cef1e7c42 fix(stream): add some logs. 2023-05-20 15:45:07 +08:00
Haojun Liao aa7ea60bca fix(stream): update the free strategy. 2023-05-20 15:43:48 +08:00
Haojun Liao 84146e6aaa other: merge main. 2023-05-20 15:16:51 +08:00
Haojun Liao 6932c59510 other: merge main. 2023-05-20 15:09:30 +08:00
Haojun Liao 10692fde5e other: merge main. 2023-05-20 14:53:28 +08:00
Haojun Liao afc384bead fix:remove invalid assert. 2023-05-20 14:44:34 +08:00
Haojun Liao 484788a60f fix:add null ptr check. 2023-05-20 14:29:51 +08:00
Haojun Liao 4974ac78f7 fix: fix invalid ref. 2023-05-20 13:42:09 +08:00
Haojun Liao 336102b8c5 refactor: do some internal refactor. 2023-05-20 13:34:49 +08:00
Haojun Liao 5cbad1da3e refactor: do some internal refactor. 2023-05-20 12:20:04 +08:00
Haojun Liao 6ded3284da refactor: do some internal refactor. 2023-05-20 11:37:17 +08:00
Haojun Liao 8ab065d01e other: update some logs. 2023-05-19 23:53:23 +08:00
Haojun Liao 56e5a483dd other: update log. 2023-05-19 20:00:32 +08:00
Haojun Liao a9b7b8a5fd other: add some logs. 2023-05-19 19:26:15 +08:00
Haojun Liao e2ec8d738c other: add some logs. 2023-05-19 18:08:01 +08:00
Haojun Liao 90687bae1c refactor: do some internal refactor. 2023-05-19 15:30:50 +08:00
liuyao e1291bd8b4 opt stream ci test 2023-05-19 13:28:58 +08:00
Haojun Liao d8525123a2 enh(stream): add API to retrieve last ts for multi-tables. 2023-05-19 11:03:41 +08:00
Haojun Liao c4c5bbb65c other: merge 3.0 2023-05-18 17:28:40 +08:00
Haojun Liao dbd52ecffa
Merge pull request #21364 from taosdata/enh/rocksdbSstate
state merge into 3.0
2023-05-18 17:25:44 +08:00
Haojun Liao 84a81716b3
Merge branch '3.0' into feature/3_liaohj 2023-05-18 17:00:56 +08:00
Haojun Liao 8a328b6636 fix(stream): fix error in stream read. 2023-05-18 16:59:18 +08:00
liuyao da3fc5111e fix pause agg task 2023-05-18 15:29:41 +08:00
yihaoDeng 9380b22394 merge 3.0 2023-05-18 05:46:38 +00:00
Haojun Liao 48dd3753aa
Merge pull request #21353 from taosdata/feature/3_liaohj
merge main, and do some internal refactor.
2023-05-17 22:34:34 +08:00
Haojun Liao 92feb89a09 refactor: do some internal refactor. 2023-05-17 18:08:04 +08:00
Haojun Liao ffee5ebc61 Merge branch '3.0' into feature/3_liaohj 2023-05-17 17:47:44 +08:00
Haojun Liao 041ffa91bd other: merge main. 2023-05-17 15:42:15 +08:00
liuyao 695de9d3ac fix stream queue mem leak 2023-05-17 15:19:14 +08:00
Haojun Liao f3f52c0232 fix(stream): add some logs and set the wal offset to be the ended version when finishing the step2 recover stage. 2023-05-17 10:43:23 +08:00
Haojun Liao e35d145734 fix(stream): fix memory leak. 2023-05-16 13:59:13 +08:00
liuyao 3c81137ffe pause source task 2023-05-16 13:43:46 +08:00
Haojun Liao e1700d00dc fix(stream): avoid memory leak. 2023-05-16 09:26:19 +08:00
liuyao 502efb3d01 op stream selectivity buff 2023-05-15 16:26:24 +08:00
yihaoDeng a001e3c424 refactor code 2023-05-15 07:22:29 +00:00
Haojun Liao 4c64260a7a refactor: do internal refactor. 2023-05-15 14:15:10 +08:00
yihaoDeng 4015f544db refactor code 2023-05-15 05:45:17 +00:00
yihaoDeng 42562d388f refactor code 2023-05-15 05:29:56 +00:00
liuyao a6570a4327 Merge branch 'enh/rocksdbSstate' of https://github.com/taosdata/TDengine into enh/rocksdbSstate 2023-05-15 11:19:55 +08:00
liuyao 9258dcd740 fix:mem leak 2023-05-15 11:19:17 +08:00
Haojun Liao 3b6e052c6b enh(stream): avoid clone submit block. 2023-05-15 10:11:08 +08:00
yihaoDeng 9c0d2c87c6 Merge branch 'enh/3.0' into enh/rocksdbSstateMerge 2023-05-15 01:52:42 +00:00
Haojun Liao 3d00f6ce54 refactor: do some internal refactor. 2023-05-14 22:08:17 +08:00
yihaoDeng 41ebd07472 factor code 2023-05-13 23:59:57 +00:00
Haojun Liao 8e15c64499 refactor: do some internal refactor. 2023-05-13 23:22:36 +08:00
Haojun Liao 236926b9ee other: merge enh/rocksdbstate 2023-05-13 19:49:28 +08:00
Haojun Liao 23f5f7f6da Merge remote-tracking branch 'origin/enh/rocksdbSstate' into enh/rocksdbSstate 2023-05-12 19:55:37 +08:00
Haojun Liao b78fc86550 fix(stream): remove some logs. 2023-05-12 19:55:22 +08:00
yihaoDeng cfc6bd9f9f factor code 2023-05-12 02:10:40 +00:00
yihaoDeng eb3ab4fcea factor code 2023-05-11 13:34:17 +00:00
yihaoDeng 6cf1adf1b1 factor code 2023-05-11 13:01:45 +00:00
liuyao 8b7b804606 opt partition by tbname 2023-05-11 18:47:27 +08:00
yihaoDeng 4e8f5f5d32 factor code 2023-05-11 09:58:53 +00:00
yihaoDeng afaee9c386 factor code 2023-05-11 09:51:01 +00:00
yihaoDeng 4715aacd41 factor code 2023-05-11 09:44:40 +00:00
yihaoDeng 59ff6bd301 factor code 2023-05-11 09:32:44 +00:00
yihaoDeng 04c7c99a67 factor code 2023-05-11 09:17:05 +00:00
yihaoDeng 0710fc39af factor code 2023-05-11 09:04:14 +00:00
Haojun Liao 8277d608f4 other: update logs. 2023-05-11 16:53:19 +08:00
yihaoDeng e6474caddf factor code 2023-05-11 08:51:41 +00:00
Haojun Liao b2d141fe40 fix(stream): limit the batch size. 2023-05-11 15:07:07 +08:00
yihaoDeng f652f71d68 factor code 2023-05-11 06:37:57 +00:00
Haojun Liao 2b736ffd60 fix(stream): avoid duplicated task id in task id list. 2023-05-11 11:47:57 +08:00
Haojun Liao b02b3117b5 refactor: do some internal refactor. 2023-05-11 10:14:29 +08:00
yihaoDeng d473fcf6b0 refactor code 2023-05-11 01:19:54 +00:00
yihaoDeng c7602a4c1a refactor code 2023-05-11 01:19:40 +00:00
Haojun Liao dbd79cc80c refactor: do some internal refactor. 2023-05-10 22:22:01 +08:00
yihaoDeng b4bd6a4f1c refactor code 2023-05-10 10:49:10 +00:00
Yihao Deng c6e5879f44
Merge branch 'main' into enh/rocksdbSstate 2023-05-10 10:04:28 +08:00
liuyao 62ccd4bf16 avoid delete data 2023-05-10 09:11:04 +08:00
yihaoDeng 8356533eb1 Merge branch 'enh/standVer' into enh/rocksdbSstateMerge 2023-05-09 12:13:16 +00:00
liuyao 8224d49702 fix:after restarting taosd, stream does not work. 2023-05-09 17:24:11 +08:00
yihaoDeng 55014aa003 Merge branch 'enh/rocksdbSstate' into enh/rocksdbSstateMerge 2023-05-09 08:58:51 +00:00
yihaoDeng b5fdaa327d avoid deadlock 2023-05-09 07:00:29 +00:00
Yihao Deng cb223acf82
Merge branch 'main' into enh/rocksdbSstate 2023-05-08 09:45:27 +08:00
Haojun Liao 31ac1e3eaa fix(stream): revise the input queue capacity. 2023-05-08 00:00:55 +08:00
Haojun Liao d026e7ef16 refactor: update some logs. 2023-05-07 22:55:49 +08:00
yihaoDeng 163a862f5f fix mem leak 2023-05-06 13:57:33 +00:00
yihaoDeng 08c3a63341 Merge branch 'enh/standVer' into enh/rocksdbSstateMerge 2023-05-06 12:37:54 +00:00
Haojun Liao c141abfbb8 fix(stream): set correct msg size; 2023-05-06 20:03:07 +08:00
Haojun Liao 13f3ca4226 fix(stream): add input queue size limitation. 2023-05-06 18:47:29 +08:00
yihaoDeng 8be29dc98e failed to open rocksdb cf 2023-05-06 10:35:23 +00:00
Haojun Liao aca5760ceb fix(query): fix syntax error. 2023-05-06 13:17:31 +08:00
Haojun Liao da6212bab0 other: merge main. 2023-05-06 12:51:38 +08:00
Haojun Liao 1fa4327f6c Merge branch 'main' into fix/liaohj_main 2023-05-06 10:15:23 +08:00
liuyao 76bf0aea56 opt stream input queue 2023-05-06 10:14:27 +08:00
Haojun Liao 3afc5bdfcf refactor: add some logs. 2023-05-06 10:09:51 +08:00
Haojun Liao 29fcd1b83a refactor: do some internal refactor. 2023-05-06 10:08:05 +08:00
Haojun Liao 5d4a3116d5
Merge pull request #21139 from taosdata/fix/liaohj_main
refactor: do some internal refactor.
2023-05-06 09:49:26 +08:00
yihaoDeng d7e86aca05 fix case failure 2023-05-05 15:14:16 +00:00
Haojun Liao 8958aabe4b fix(stream): opt task scan performance. 2023-05-05 19:33:30 +08:00
yihaoDeng f7a7743af7 fix case failure 2023-05-05 08:32:33 +00:00
Haojun Liao 8b12d4d3da fix(stream): secure the delete task operation. TD-1198 2023-05-05 15:02:03 +08:00
yihaoDeng a1b9dcf595 fix mem leak 2023-05-05 02:20:11 +00:00
liuyao 74ab689719 opt batch num 2023-05-05 08:59:07 +08:00
yihaoDeng f20b28bc9f fix mem leak 2023-05-04 14:29:56 +00:00
yihaoDeng 6b0bda6708 fix mem leak 2023-05-04 13:38:19 +00:00
yihaoDeng c2de8c91dd fix mem leak 2023-05-04 11:52:29 +00:00
yihaoDeng f9c49f4938 add compile opt on windows 2023-05-04 10:22:13 +00:00
Haojun Liao 2b7a816b86 fix(tmq): fix syntax error. 2023-05-04 16:59:26 +08:00
Haojun Liao 23c49594b7 other: merge main. 2023-05-04 16:41:53 +08:00
yihaoDeng c27d01a510 Merge branch 'enh/addTtlToStream' into enh/rocksdbSstateMerge 2023-05-04 08:22:31 +00:00
Haojun Liao 09a04052ec refactor: do some internal refactor. 2023-05-04 16:15:14 +08:00
liuyao d8a1f15d10 fix resum bug 2023-05-04 09:28:06 +08:00
Haojun Liao 3b814f2478 fix(stream): do some internal refactor. 2023-05-01 23:21:29 +08:00
Haojun Liao a1263b8b04 fix(query):fix syntax error. 2023-05-01 19:40:26 +08:00
Haojun Liao 3c61932ecc refactor: do some internal refactor. 2023-05-01 19:28:54 +08:00
Haojun Liao 6c86847b12 refactor: do some internal refactor. 2023-05-01 14:58:59 +08:00
Haojun Liao 777ed1761f refactor: update logs. 2023-04-29 18:24:46 +08:00
Haojun Liao bbb8aaa0dc refactor: add some logs. 2023-04-29 02:37:10 +08:00
Haojun Liao b69137df44 refactor(log): add some logs. 2023-04-29 00:00:25 +08:00
Haojun Liao c647ded0da fix(stream): add some logs. 2023-04-28 19:47:49 +08:00
yihaoDeng dda1c8803c Merge branch 'enh/rocksdbSstate' into enh/rocksdbSstateMerge 2023-04-28 09:17:22 +00:00
yihaoDeng afc6023a8a merge rocksdb inst 2023-04-28 06:32:33 +00:00
Haojun Liao b7c8346b44 other: merge other branch. 2023-04-28 13:06:47 +08:00
Haojun Liao f9a64cbcd6 other: add some logs. 2023-04-28 12:58:03 +08:00
yihaoDeng c4f36d32ae merge rocksdb inst 2023-04-28 01:31:37 +00:00
yihaoDeng 984a69671e merge rocksdb inst 2023-04-27 14:46:33 +00:00
yihaoDeng 36a4413bf1 merge rocksdb inst 2023-04-27 14:38:33 +00:00
jiajingbin ebf9a13fe7 inc batch num 2023-04-27 18:15:17 +08:00
Haojun Liao 6780833cd2 Merge remote-tracking branch 'origin/enh/rocksdbSstate' into enh/rocksdbSstate 2023-04-27 15:08:17 +08:00
Haojun Liao b7040ed15e enh(stream): support resume from the latest data, and do some internal refactor. 2023-04-27 15:07:54 +08:00
liuyao 8b5251004f inc queue cap 2023-04-27 14:04:46 +08:00
liuyao dbdc31539c inc queue cap 2023-04-27 10:46:08 +08:00
liuyao 4c4a4775de opt stream 2023-04-26 19:13:58 +08:00
liuyao 863d30e5a9 opt tq read 2023-04-26 18:58:19 +08:00
liuyao 2533b5c5c2 op stream 2023-04-26 17:00:33 +08:00
yihaoDeng 10582fb5e0 opt write opt 2023-04-25 09:49:28 +00:00
yihaoDeng 84cae91f9f rocksdb_iter_value 2023-04-25 09:39:57 +00:00
yihaoDeng c64a7d3e5f add ttl to stream state key 2023-04-25 08:54:03 +00:00
yihaoDeng 05df0f6ed7 add ttl to stream state key 2023-04-25 02:36:44 +00:00
yihaoDeng 81bd7661bd add ttl to stream state key 2023-04-24 14:20:01 +00:00
yihaoDeng f15b806e15 opt write opt 2023-04-24 09:08:48 +00:00
yihaoDeng e1535124d1 opt write opt 2023-04-24 09:08:07 +00:00
yihaoDeng 23803c04ed opt write opt 2023-04-24 08:49:56 +00:00
yihaoDeng 2d5218246d add ttl to stream state key 2023-04-24 08:18:02 +00:00
yihaoDeng 3abdbec813 add ttl to stream state key 2023-04-24 08:17:57 +00:00
liuyao eca7b68fa0 ignore update 2023-04-24 15:01:56 +08:00
Haojun Liao 48c36a8c35 Merge branch 'main' into enh/rocksdbSstate 2023-04-24 14:44:13 +08:00
yihaoDeng ae3730c318 add ttl to stream state key 2023-04-23 13:29:18 +00:00
Haojun Liao 623ea6df5f fix(stream): add task status check. 2023-04-23 17:42:56 +08:00
yihaoDeng 917b07348c Merge branch 'fix/rocksdbState' into enh/rocksRevert 2023-04-23 02:37:26 +00:00
yihaoDeng 54f3bc2543 change write opt 2023-04-21 10:06:37 +00:00
liuyao 426a794e7e add ini 2023-04-21 17:47:00 +08:00
yihaoDeng c71bbb0109 change write opt 2023-04-21 06:24:46 +00:00
yihaoDeng 98d6ff5a5d change write opt 2023-04-21 05:55:10 +00:00
yihaoDeng 57c4544bf0 change write opt 2023-04-21 05:54:52 +00:00
Haojun Liao b6f4cac619 fix(stream): set the correct offset version. 2023-04-21 13:35:24 +08:00
Haojun Liao 6c641cff39 fix(stream): fix error in start stream tasks. 2023-04-21 09:37:16 +08:00
Haojun Liao 5e6c06e253 fix(stream): fix the race condition during create new stream tasks. 2023-04-20 23:21:18 +08:00
Haojun Liao dc733352db enh(stream): set the max input queue size to be 3000. 2023-04-20 17:38:10 +08:00
Haojun Liao 8323ad8670 enh(stream): add more check to stop stream asap. 2023-04-20 17:29:08 +08:00
Haojun Liao b60b1796f7 fix(stream): add lock during check wal to create new stream task. 2023-04-20 15:53:41 +08:00
yihaoDeng 3272d14e33 change write opt 2023-04-20 04:05:35 +00:00
yihaoDeng aa613285e7 change write opt 2023-04-20 00:21:16 +00:00
yihaoDeng 9d6e8b3fa7 change write opt 2023-04-20 00:18:57 +00:00
yihaoDeng b640a0330d change write opt 2023-04-19 23:55:09 +00:00
yihaoDeng 6ff0b09e5e change write opt 2023-04-19 23:46:08 +00:00
yihaoDeng 22afdaa6a4 change write opt 2023-04-19 14:15:17 +00:00
yihaoDeng 10e44c63e1 change write opt 2023-04-19 13:12:53 +00:00
yihaoDeng 5efa5f6b9f change write opt 2023-04-19 13:01:13 +00:00
54liuyao 5b7ccc42ca inc sleep time 2023-04-19 13:38:30 +08:00
liuyao ae117bd7c3 fill state 2023-04-19 09:15:12 +08:00
Haojun Liao 9f16802451 fix(stream): set the initial value. 2023-04-18 16:42:40 +08:00
Haojun Liao 76becf59b7
Merge pull request #20947 from taosdata/feature/3_liaohj
fix(stream): set the correct initial checkpoint versions.
2023-04-18 11:26:48 +08:00