Commit Graph

521 Commits

Author SHA1 Message Date
Haojun Liao 4879ae3d68 refactor(query): optimize load last block. 2022-08-21 21:36:15 +08:00
Haojun Liao 29a1245be4 fix(query): use the separate last block delete index. 2022-08-21 20:15:53 +08:00
Haojun Liao 7904686915 fix(query): support delete in last 2022-08-21 18:59:21 +08:00
Haojun Liao d305e975c4 fix(query): remove printf. 2022-08-21 15:56:46 +08:00
Haojun Liao fe918b3f69 fix(query): set correct column data. 2022-08-21 15:56:06 +08:00
Haojun Liao 5a1ba3d016 fix(query): increase the output buffer size. 2022-08-20 23:26:58 +08:00
Haojun Liao 6d8a70ebf6 fix(query): remove invalid free. 2022-08-20 20:22:39 +08:00
Haojun Liao df0d6f1823 fix(query): fix memory leak. 2022-08-20 19:05:55 +08:00
Haojun Liao 0dfd032068 fix(query): reset block data when no blocks are available. 2022-08-20 16:57:44 +08:00
Haojun Liao 09ffcd2e13 fix(query): fix bug in tsdbread 2022-08-20 12:41:58 +08:00
Haojun Liao 515e8d36b0 fix(query): fix bug in merge buffer data and last block. 2022-08-20 11:14:00 +08:00
Haojun Liao 5eae03fcab fix(query): handle the reverse case. 2022-08-18 22:57:26 +08:00
Haojun Liao eaef3dffaf fix(query): fix error in query last block. 2022-08-18 22:42:16 +08:00
Haojun Liao e75c400045 fix(query): fix bug in query last block. 2022-08-18 18:48:50 +08:00
Hongze Cheng e9bb654a4a Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/tsdb_last 2022-08-18 17:14:51 +08:00
Haojun Liao 8eb86676d0 fix(query): support query last file. 2022-08-18 17:09:44 +08:00
wangmm0220 fbc69a10c4 Merge branch '3.0' into feature/TD-14761 2022-08-18 16:45:06 +08:00
Haojun Liao 9dc5b4cb0c fix(query): support read from last file. 2022-08-18 00:08:09 +08:00
Haojun Liao 5b396276ea Merge branch 'refact/tsdb_last' of github.com:taosdata/tdengine into refact/tsdb_last 2022-08-17 07:54:31 +08:00
Haojun Liao 559286db71 refactor(query): support read data from last files. 2022-08-17 07:51:09 +08:00
wangmm0220 78c9f245b1 fix:conflicts from 3.0 2022-08-16 15:18:11 +08:00
Hongze Cheng 3359b8e620 more optimize 2022-08-15 10:16:07 +00:00
Hongze Cheng e1b638aeb5 Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/tsdb_last 2022-08-15 03:02:54 +00:00
Cary Xu ae7550eb86 enh: rsma exec in async mode 2022-08-12 21:00:48 +08:00
Minglei Jin 6058b8dff5
Merge pull request #16003 from taosdata/fix/TD-18321
fix: use mem schema if pReader->pSchema is null
2022-08-11 19:18:30 +08:00
Haojun Liao 02300b30fa
Merge pull request #15995 from taosdata/feature/3_liaohj
fix(query): add check when invoking sma filter.
2022-08-11 18:52:51 +08:00
Minglei Jin 203a8073ca fix: use mem schema if pReader->pSchema is null 2022-08-11 18:19:44 +08:00
Haojun Liao 3a6285c478 fix(query): add check when invoking sma filter. 2022-08-11 17:40:55 +08:00
Cary Xu 1d6d0aa2db enh: row iter and fetch optimization 2022-08-10 17:26:44 +08:00
Haojun Liao 6250c1b157
Merge pull request #15851 from taosdata/feature/3.0_query_optimize
ehn(query): optimize the query performance.
2022-08-10 16:51:46 +08:00
Ganlin Zhao 249e2b12f6 fix(query): fix interp + fill interpolation after last datablock 2022-08-10 14:21:31 +08:00
Haojun Liao 41d2c92231 fix(query): copy null value. 2022-08-09 16:54:05 +08:00
Haojun Liao 29c84833d3 fix(query):fix invalid free. 2022-08-09 10:34:01 +08:00
Haojun Liao 30a64bdcd9 fix(query): set correct schema. 2022-08-09 10:16:00 +08:00
Hongze Cheng 4f06c4bbdd make it compile 2022-08-08 07:32:02 +00:00
Hongze Cheng 8ae2ab1c6e more work 2022-08-08 05:34:01 +00:00
Hongze Cheng 33c3f34f34 more work 2022-08-08 03:22:24 +00:00
Hongze Cheng 71691eb7da make code compile 2022-08-07 13:14:43 +00:00
Hongze Cheng 7c32e099d0 more work 2022-08-06 14:20:30 +00:00
Hongze Cheng 04804e384a more 2022-08-06 12:24:46 +00:00
Hongze Cheng 3436d52316 more refact 2022-08-06 12:23:29 +00:00
Hongze Cheng 3324f4e5b7 more work 2022-08-06 12:04:42 +00:00
wangmm0220 f28155714e fix:confilcts from 3.0 2022-08-06 12:31:15 +08:00
wangmm0220 45e7597df3 feat:get data from snapshot for taosx 2022-08-05 21:12:18 +08:00
Hongze Cheng 51538a1710 more work 2022-08-05 09:47:26 +00:00
Haojun Liao 52173e14e1 fix(query): add check for null ptr. 2022-08-05 16:10:45 +08:00
Haojun Liao 0e4748771a enh(query): support combine multi datablock to pass to upstream operators 2022-08-05 14:10:45 +08:00
Haojun Liao 271528f4e2 refactor: opt tsdb performance. 2022-08-04 18:12:44 +08:00
Haojun Liao 4cd7af0ecf other: merge 3.0 2022-08-03 19:02:43 +08:00
Cary Xu 71419d330d other: adjust rsma debug logs 2022-08-03 14:58:54 +08:00
Cary Xu 335da8fc65 other: add tsma debug log 2022-08-03 14:33:29 +08:00
Haojun Liao bc600a4942 refactor: do some internal refactor optimize the building block performance. 2022-08-03 14:08:03 +08:00
Haojun Liao 64db9afeb7 refactor: do some internal refactor. 2022-08-03 10:52:06 +08:00
Shengliang Guan efb0127fbf refactor: adjust logs 2022-08-02 16:48:49 +08:00
Shengliang Guan d5aea7dac2 refactor: adjust sync log 2022-08-02 16:39:12 +08:00
Minglei Jin 711c75652c
Merge pull request #15671 from taosdata/fix/TD-18074
fix: new pSchema if reader's schema is null
2022-08-02 14:12:30 +08:00
Haojun Liao bb869800f6
Merge pull request #15661 from taosdata/feature/3_liaohj
refactor: optimize the build block perf.
2022-08-02 13:04:00 +08:00
Minglei Jin 5cf4dac071 fix: new pSchema if reader's schema is null 2022-08-02 12:56:33 +08:00
Shengliang Guan bd31c9da5a
Merge pull request #15657 from taosdata/fix/dnode
fix: write snapshot after apply queue is empty
2022-08-01 20:13:08 +08:00
Haojun Liao bbcba1855d Merge branch '3.0' into feature/3_liaohj 2022-08-01 20:09:42 +08:00
Shengliang Guan 99f29f6455 refactor: adjust log 2022-08-01 17:23:52 +08:00
54liuyao b3d104349f feat(stream):optimize stream update scan 2022-08-01 17:09:46 +08:00
Haojun Liao 7db6497d28 refactor: update some logs. 2022-08-01 11:19:06 +08:00
Minglei Jin c2faef7377
Merge pull request #15555 from taosdata/fix/TD-17968
fix: update schema to newest version to parsing rows
2022-07-30 16:04:04 +08:00
Minglei Jin e57f12d581 fix: use colId to merge rows 2022-07-30 15:23:28 +08:00
Minglei Jin 56d2697308 fix: new tRowMergerInit2 for ts row merging 2022-07-30 12:46:40 +08:00
Haojun Liao 2261d689e4 fix(query): opt read data from file block. 2022-07-29 21:09:33 +08:00
Haojun Liao 0c1e30dc56 fix(query): step forward index 2022-07-29 19:34:13 +08:00
Minglei Jin 0c111cd2bf fix: update schema to newest version to parsing rows 2022-07-29 18:15:44 +08:00
Haojun Liao 60c5b2beac fix(query): optimize the row merge procedure for files. 2022-07-29 17:53:30 +08:00
Haojun Liao aeed234b2e fix(query):fix memory leak. 2022-07-26 20:40:39 +08:00
Haojun Liao bd8fa53371 fix(query): set current SBlock before move to next block. 2022-07-26 15:52:47 +08:00
Haojun Liao 6600540fd9 refactor: optimize the memory consumption during tsdbread 2022-07-26 11:43:45 +08:00
Haojun Liao 231f439979 refactor: opt memory consumption for tsdbread. 2022-07-26 10:19:12 +08:00
Haojun Liao 6d86ef0c33 other: merge 3.0 2022-07-25 11:22:09 +08:00
Haojun Liao 9bddf99d7d refactor(query): do some internal refactor. 2022-07-25 11:17:45 +08:00
Cary Xu 00a9598ac8 enh: rsma level 2/3 submitReq msg use wal version 2022-07-22 20:39:13 +08:00
Hongze Cheng 8cdef36879 fix: sam cases 2022-07-22 04:56:15 +00:00
Hongze Cheng cd762afe76 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/hzcheng_3.0 2022-07-22 04:50:24 +00:00
Hongze Cheng b27272d784 fix memory leak 2022-07-22 01:40:02 +00:00
Haojun Liao 7522c2edf3 refactor: do some internal refactor. 2022-07-21 20:44:36 +08:00
Hongze Cheng bce29321c6 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/hzcheng_3.0 2022-07-21 12:40:03 +00:00
Hongze Cheng 6a7a3da3db fix some crash 2022-07-21 11:56:18 +00:00
Hongze Cheng 684dd82358 fix read concurrency 2022-07-21 11:42:42 +00:00
Haojun Liao 8e42bf85e8 fix(query): fix border check. 2022-07-21 15:47:22 +08:00
Hongze Cheng f195decac9 more fix 2022-07-21 06:27:32 +00:00
Haojun Liao 2a437ee38a fix(query): close del handle when successing in read del file content. 2022-07-21 11:35:31 +08:00
Haojun Liao c2d588a118 other:merge 3.0 2022-07-21 11:29:34 +08:00
Haojun Liao ef0d705c5f fix(query): close del file read handle. 2022-07-21 11:27:20 +08:00
Minglei Jin 37bb5caf29
Merge pull request #15205 from taosdata/fix/TD-17582
fix: use suid from pReader
2022-07-20 19:44:40 +08:00
Minglei Jin 5cc829e45f fix: use suid from pReader 2022-07-20 18:47:46 +08:00
Minglei Jin 982b11f6eb
Merge pull request #15184 from taosdata/fix/TD-17582
fix: use Ex version of metaGetTbTSchema to retrieve schema
2022-07-20 16:51:10 +08:00
Minglei Jin 527aa3584d fix: get suid from uid to be used to retrieve schema 2022-07-20 15:26:47 +08:00
Minglei Jin eefad2bd9d fix: use Ex version of metaGetTbTSchema to retrieve schema 2022-07-20 14:28:13 +08:00
WANG MINGMING 258f4fb20a
Merge branch '3.0' into feature/TD-14761 2022-07-20 11:20:42 +08:00
Minglei Jin e422d7dbf5 fix: remove assert if schema is null 2022-07-20 10:29:21 +08:00
wangmm0220 bc0e8e3061 fix:invalidate in telnet schemaless 2022-07-20 09:48:57 +08:00
Haojun Liao a0428f5685 fix(query): add null ptr check. 2022-07-19 18:05:26 +08:00
Hongze Cheng 024f2221b7 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/hzcheng_3.0 2022-07-19 08:31:16 +00:00
Hongze Cheng 657c2dcaa4 more r/w concurrency 2022-07-19 07:21:15 +00:00
Hongze Cheng 58135a1aeb tsdb r/w concurrency 2022-07-19 06:19:01 +00:00
Haojun Liao 5e754a7662 fix(query): add check for deleting record 2022-07-19 14:00:45 +08:00
dapan1121 077fbd6787
Merge pull request #15010 from taosdata/fix/TD-17460
fix: fix taosd mem leak
2022-07-18 09:18:43 +08:00
dapan1121 9ec14ece62 fix: fix taosd mem leak 2022-07-16 18:58:29 +08:00
Hongze Cheng 989159f0ba vnod snapshot done 2022-07-15 13:02:24 +00:00
Haojun Liao fd2dec280b fix(query): fix memory leak. 2022-07-15 11:18:26 +08:00
Haojun Liao 707f0a837a fix(query): set correct fill output object. 2022-07-14 20:11:29 +08:00
Haojun Liao 75aca0b031 fix(query): set correct fill output column index, fix some memory leak, and do some internal refactor, 2022-07-14 13:53:23 +08:00
Haojun Liao 083de75afc fix(query):fix memory leak. 2022-07-13 23:15:58 +08:00
Haojun Liao 2c1efc3963 other: merge 3.0 2022-07-12 21:10:10 +08:00
Liu Jicong a34d344225
Merge pull request #14808 from taosdata/feature/stream
refactor(tmq): prepare only needed
2022-07-12 18:09:31 +08:00
Haojun Liao 85ddb63e53 fix(query): free resources in tsdbread 2022-07-12 17:44:10 +08:00
Liu Jicong 8e3f5135a6 fix(tsdb): close fd 2022-07-12 17:28:14 +08:00
Cary Xu 6c5e8bae55 fix: use correct keep2 when set least ts for query 2022-07-12 14:14:42 +08:00
Haojun Liao 4527447b60 fix(query): handle the indefinit function output with partition by 2022-07-11 11:13:49 +08:00
Haojun Liao 7dc7f478de refactor: do some internal refactor and fix some memory leak. 2022-07-10 17:34:21 +08:00
Haojun Liao 40380ef9c1 refactor: do internal refactor. 2022-07-10 10:15:27 +08:00
Haojun Liao fceee02622
Merge branch '3.0' into feature/3_liaohj 2022-07-09 11:11:11 +08:00
Haojun Liao 3e81daf578 fix(query): fix memory leak. 2022-07-08 17:28:37 +08:00
dapan1121 50a6ef7f39
Merge pull request #14635 from taosdata/feature/insertselect
feat: support insert from query res
2022-07-08 16:59:21 +08:00
Haojun Liao 1711d646c1
Merge pull request #14671 from taosdata/feature/3_liaohj
enh(query): add block sma for int type column data.
2022-07-08 16:23:13 +08:00
Shengliang Guan 09c095a91c fix: asan error 2022-07-08 11:51:35 +08:00
Haojun Liao c339800611 fix(query): handle delete duration generating data block. 2022-07-08 11:02:12 +08:00
dapan1121 0925064d24 Merge branch 'feature/3.0_debug_wxy' into feature/insertselect 2022-07-08 10:28:53 +08:00
Haojun Liao 5c2e98544d fix(query): check if the data block overlaps with the delete skyline 2022-07-07 22:40:20 +08:00
Haojun Liao 211985f03e refactor: do some internal refactor. 2022-07-07 17:23:54 +08:00
Haojun Liao 557873c1bc other: merge 3.0 2022-07-07 16:50:00 +08:00
Haojun Liao 41be3e4d04 refactor: do some internal refactor. 2022-07-07 15:56:43 +08:00
Haojun Liao 57bf509dee refactor: do some internal refactor. 2022-07-07 15:40:04 +08:00
Haojun Liao d9e5172357 fix(query): set the correct sma data 2022-07-07 15:32:56 +08:00
Haojun Liao 4d2bc796e7 enh(query): add block sma for int type column data. 2022-07-07 11:26:58 +08:00
dapan1121 d4696ef31f Merge branch '3.0' into feature/insertselect 2022-07-07 09:02:09 +08:00
yihaoDeng 1e5ddcb92f merge 3.0 2022-07-06 18:29:11 +08:00
dapan1121 0d4fb5bb80 feat: support insert from query 2022-07-06 16:29:51 +08:00
Haojun Liao ddb71ed095 refactor: do some internal refactor. 2022-07-06 13:33:21 +08:00
Cary Xu 442a781781 Merge branch 'feat/tsdb_refact' of github.com:taosdata/TDengine into feat/tsdb_refact 2022-07-05 19:31:19 +08:00
Cary Xu eedaeda846 enh: rsma level 1/2 utilize separated version 2022-07-05 19:30:37 +08:00
Hongze Cheng 004a6571b0 Merge branch 'feat/tsdb_refact' of https://github.com/taosdata/TDengine into feat/tsdb_refact 2022-07-05 09:31:54 +00:00
Hongze Cheng 08a7d9d38f fix ci coredump 2022-07-05 09:31:51 +00:00
Haojun Liao c9ffc42ddb fix(query): set initial value for neighbor block check. 2022-07-05 17:02:21 +08:00
Haojun Liao 5b20f00383 refactor: do some internal refactor. 2022-07-05 14:38:45 +08:00
Haojun Liao 3bae30412c refactor: do some internal refactor. 2022-07-05 13:57:07 +08:00
Haojun Liao c7559a81ca refactor: do some internal refactor. 2022-07-04 23:28:27 +08:00
Minglei Jin 8d00f1139a tsdbRead: fix None display to NULL instead of 0 2022-07-04 14:46:00 +08:00
Haojun Liao 16d18b8046 refactor: do some internal refactor. 2022-07-04 14:10:29 +08:00
Hongze Cheng 38ca993452 make merge compile 2022-07-04 05:21:02 +00:00
Hongze Cheng ddb198de23 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/tsdb_refact 2022-07-04 03:42:50 +00:00
Haojun Liao a7cbb93a38 refactor: do some internal refactor. 2022-07-03 23:00:12 +08:00
Haojun Liao ba8ab92632 refactor: do some internal refactor. 2022-07-02 23:37:31 +08:00
Haojun Liao 42d635a73d refactor: do some internal refactor. 2022-07-02 20:39:23 +08:00
Haojun Liao a0e38fc703 refactor: do some internal refactor. 2022-07-02 15:22:49 +08:00
Haojun Liao 42d0af5cf7 refactor: do some internal refactor. 2022-07-02 14:17:27 +08:00
Haojun Liao 99fb3c678c refactor: do some internal refactor. 2022-07-02 12:05:03 +08:00
Hongze Cheng 7d50bfcb0d more code 2022-07-01 15:10:46 +00:00
Haojun Liao 34e4980a51 refactor(query): do some internal refactor. 2022-07-01 15:19:02 +08:00
Haojun Liao 0db8d77a70 refactor: do some internal refactor. 2022-06-30 23:38:18 +08:00
Liu Jicong 8dece648e3 feat(tmq): enable withTbName config for snapshot 2022-06-30 15:38:27 +08:00
Haojun Liao d377a89afd refactor: do some internal rafactor. 2022-06-30 15:17:04 +08:00
Haojun Liao 0758751b0c refactor: do some internal refactor. 2022-06-30 14:50:59 +08:00
Liu Jicong 4695dc2dae feat(tmq): add snapshot test 2022-06-30 14:41:50 +08:00
Hongze Cheng 319e745dd5 more work 2022-06-30 06:02:47 +00:00
Haojun Liao c5951374d4 refactor: do internal refactor. 2022-06-29 20:25:03 +08:00
Hongze Cheng aefc151c03 fix some code 2022-06-29 10:37:35 +00:00
Haojun Liao 3a566d5e5e refactor(query): do some internal refactor. 2022-06-29 17:14:00 +08:00
Haojun Liao 6464a3d272 refactor(query): do some internal refactor. 2022-06-29 14:18:03 +08:00
Haojun Liao f2b83dfb4a fix(query):check null ptr. 2022-06-29 10:35:07 +08:00
Haojun Liao b8bc052aa4 refactor(query): do some internal refactor. 2022-06-29 09:40:31 +08:00
Haojun Liao 4a59e1300b other: merge 3.0. 2022-06-28 15:22:32 +08:00
Haojun Liao 6a32f81265 fix(query): retrieve in-memory buffer data. 2022-06-28 11:37:26 +08:00
Haojun Liao c638c33a76 enh(query): retrieve in-memory data. 2022-06-28 10:32:42 +08:00
Cary Xu e6dd04f573 refactor: rsma restore 2022-06-27 14:47:14 +08:00
Haojun Liao dbd8c300fa refactor: do tsdbread refactor. 2022-06-27 10:46:20 +08:00
Cary Xu 0597f3b9cd refactor: rsma resource release 2022-06-26 18:44:49 +08:00
slzhou a9f094613a fix: restore table merge scan operator 2022-06-25 13:52:52 +08:00
wangmm0220 a95f98f7e7 feat:merge from 3.0 2022-06-23 20:50:48 +08:00
wangmm0220 3799db4366 feat:sort table group if needed 2022-06-23 20:46:11 +08:00
wangmm0220 26cceaf172 feat:sort table group if needed 2022-06-23 19:58:12 +08:00
Haojun Liao bf0caa2038 Merge branch '3.0' into feature/3_liaohj 2022-06-22 20:32:02 +08:00
shenglian zhou c1f5f825b7 feat: prepare for multiple group tsdb reads 2022-06-22 18:51:16 +08:00
Haojun Liao 5c1e7f3114 Merge branch '3.0' into feature/3_liaohj 2022-06-22 17:23:57 +08:00
Haojun Liao 14ddf539e0 refactor(query): do some internal refactor. 2022-06-22 17:21:04 +08:00
afwerar 9dff7b93c6 test: fix sim test error 2022-06-21 20:04:55 +08:00
Haojun Liao 659e429b87 feature(query): support show table block distribution. 2022-06-20 12:54:46 +08:00
Hongze Cheng 21ea4e3a0b more work 2022-06-20 02:38:08 +00:00
Hongze Cheng 64554461f6 more work 2022-06-17 12:22:45 +00:00
Hongze Cheng 095ca03f0d more work 2022-06-17 06:29:06 +00:00
Hongze Cheng 75816b7e7a more work 2022-06-16 12:53:25 +00:00
Hongze Cheng 813005eb7b query start 2022-06-16 11:32:05 +00:00
Hongze Cheng 760905f632 refact 2022-06-16 11:14:56 +00:00
Hongze Cheng 5bd3732adf refact more code 2022-06-16 09:53:07 +00:00
Hongze Cheng 5485c7fc9f refact tsdb read 2022-06-16 09:18:30 +00:00
Hongze Cheng d59b0254d3 more work 2022-06-16 08:58:00 +00:00
Hongze Cheng 07792850cb refact code 2022-06-16 06:29:28 +00:00
Hongze Cheng b0b30215af more work 2022-06-16 04:00:15 +00:00
Hongze Cheng 47b11afe10 more work 2022-06-16 01:32:48 +00:00
Hongze Cheng 0ef7eec754 more work 2022-06-13 12:12:54 +00:00
Hongze Cheng c94ac9a1e4 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/tsdb_refact 2022-06-13 09:11:11 +00:00
shenglian zhou dc56c636ea Merge branch '3.0' of github.com:taosdata/TDengine into szhou/feature/multiwaymerge 2022-06-13 15:08:26 +08:00
shenglian zhou 140e15bc8d fix: merge interval operator optimization and explain analyze multimerge operator 2022-06-13 15:07:54 +08:00
Hongze Cheng 2c63430804 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/tsdb_refact 2022-06-13 06:10:51 +00:00