Commit Graph

833 Commits

Author SHA1 Message Date
dapan1121 225bf28536
Merge pull request #19744 from taosdata/fix/TD-22009
enhance: optimize building table delete skyline performance
2023-02-03 19:00:47 +08:00
Haojun Liao 845530e125 other: add some logs. 2023-02-03 17:33:35 +08:00
Xiaoyu Wang 5d499e52f9 merge main 2023-02-03 17:13:11 +08:00
Haojun Liao f803fa44b5 fix(query): set the correct number of rows when calculating using sma for min/max function. 2023-02-03 16:50:13 +08:00
Haojun Liao e29ca4f5d8 refactor(query): use real number of stt instead of default value. 2023-02-03 10:04:15 +08:00
Haojun Liao baaa57567e refactor: do some internal refactor. 2023-02-03 09:13:13 +08:00
Haojun Liao e9490a3235 refactor(query): disable tag filter cache by default. 2023-02-02 17:38:21 +08:00
Haojun Liao edca89b731 refactor: opt query perf. 2023-02-02 15:15:46 +08:00
Haojun Liao fd6ea6ba2f other: merge main. 2023-02-02 14:12:24 +08:00
dapan1121 9a261ab2d7 improvement: optimize build table delete skyline performance 2023-02-02 08:45:38 +08:00
Haojun Liao 9fe1512c55 refactor: add some logs. 2023-02-01 22:41:39 +08:00
Haojun Liao 8e4c6ccbb3 fix(query): move the reset to other place. 2023-02-01 18:15:35 +08:00
Minglei Jin dca720fc7d fix(query): protect read snap with reader mutex when closing reader 2023-02-01 15:09:21 +08:00
Xiaoyu Wang fc829aec52
Merge pull request #19689 from taosdata/fix/TD-22030
fix(query): reset snap to null after untaking mem snap
2023-01-31 16:51:07 +08:00
dapan1121 c8394cbf6e fix: add get tb from mem time info 2023-01-31 16:38:29 +08:00
Minglei Jin 0257c4a90b fix(query): release reader's lock if not composed block 2023-01-31 15:04:48 +08:00
Minglei Jin 251fdd35ba fix(tsdb/read): wrap tsdb reader's lock with acquire/release 2023-01-30 19:26:36 +08:00
Haojun Liao 1741e98a4e fix(query): reset the del file index when beginning last file check. 2023-01-30 10:13:54 +08:00
Minglei Jin 101772fe1b fix(query): reset snap to null after untaking mem snap 2023-01-29 13:43:37 +08:00
Xiaoyu Wang 6e247e8911
Merge pull request #19679 from taosdata/fix/TD-22030
fix(query): reset iter, iiter's hasVal to false when suspending reader
2023-01-29 11:04:41 +08:00
Minglei Jin ac3e6323e2 fix(query): reset all block scan info when suspending reader 2023-01-29 09:33:32 +08:00
Xiaoyu Wang eb86f5ae71 merge main 2023-01-28 18:10:08 +08:00
Xiaoyu Wang 24157e78d6 merge main 2023-01-28 17:52:58 +08:00
Minglei Jin b4807d9aa9 fix(query): reset iter, iiter's hasVal to false when suspending reader 2023-01-28 14:06:37 +08:00
kailixu 39b69ba39e fix: row schema version for row merger 2023-01-21 13:44:25 +08:00
kailixu 70fb18e611 fix: tsdb ReadSnap take/untake 2023-01-17 15:11:11 +08:00
Xiaoyu Wang 9a0c2805f4 merge main 2023-01-13 13:54:04 +08:00
Haojun Liao 53f27901a5 Merge branch 'main' into feature/3_liaohj 2023-01-11 15:42:34 +08:00
Minglei Jin 5024fc1e9f tsdb/suspend: fix table block scan info 2023-01-10 18:40:15 +08:00
Minglei Jin 4032340e92 tsdb/reader: reset reader status with lock protected 2023-01-10 16:45:58 +08:00
Haojun Liao 08321f3c6d enh(query): update the log. 2023-01-10 13:35:25 +08:00
Haojun Liao b2a70c0a34 enh(query): jump out of loop once queried tables are all found. 2023-01-10 13:33:27 +08:00
Minglei Jin e48bdbfd0f tsdb/log-query: new proactive flag for mem table unref 2023-01-10 11:52:40 +08:00
Hongze Cheng bff8c216cc more code 2023-01-10 11:36:40 +08:00
Haojun Liao e244a484f8 fix(query): add additional check for imem/mem 2023-01-10 10:49:35 +08:00
Minglei Jin d69ec98723 tsdb/read: reseek with try lock mutex 2023-01-10 10:38:44 +08:00
Hongze Cheng f848324d54 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/long_query 2023-01-10 09:42:45 +08:00
Haojun Liao 6a6d53b896
fix(query): fix error for retrieve data only in last files. (#19457)
* fix: add test cases;

* fix(query): fix error for retrieve data only in last files.
2023-01-09 20:55:34 +08:00
Haojun Liao 16f0900fe9 fix(query): fix error for retrieve data only in last files. 2023-01-09 19:26:39 +08:00
Hongze Cheng ad9e8a9aa3 more code 2023-01-09 19:03:09 +08:00
Hongze Cheng 2e443c395e more code 2023-01-09 15:51:30 +08:00
Shengliang Guan 398161eb5f Merge branch 'main' into merge/mainto3.0_0108 2023-01-09 10:00:53 +08:00
Haojun Liao 95a6720b2a fix(query): do not merge rows in last file when no data blocks exist. 2023-01-08 23:02:16 +08:00
Minglei Jin 57c303b4f1 Merge branch 'fix/long_query' of https://github.com/taosdata/TDengine into fix/long_query 2023-01-04 17:58:06 +08:00
Minglei Jin 6591b98245 fix(tsdb): comment qlist out to please CI 2023-01-04 17:57:50 +08:00
Liu Jicong 5ee7e7aed9 fix: init tsdb read snap 2023-01-04 17:55:00 +08:00
Minglei Jin 625bf9992f
Merge branch '3.0' into fix/long_query 2023-01-04 15:15:51 +08:00
Minglei Jin b020383046 fix(tsdb): comment qlist out to please CI 2023-01-03 18:30:50 +08:00
Minglei Jin 07f16b1660 Merge branch 'v3.0' into fix/long_query 2023-01-03 17:21:10 +08:00
Alex Duan b68e440cc2 feat: support total vgroups count for table 2023-01-03 14:13:51 +08:00
Minglei Jin e4e72dd9e9 fix(tsdb/read): new tsdbReleaseDataBlock api from release reader's lock 2023-01-03 09:43:29 +08:00
Minglei Jin d3c97947f3 fix(tsdb/read): use int32_t instead of size_t for taosHashGetSize 2022-12-30 16:04:18 +08:00
Minglei Jin 2ed9c19da4 Merge branch 'v3.0' into fix/long_query 2022-12-29 17:43:51 +08:00
Shengliang Guan 78c14a2c88 Merge remote-tracking branch 'origin/3.0' into merge/mainto3.0_1228 2022-12-28 14:59:02 +08:00
Shengliang Guan c844922aef
Merge pull request #19172 from taosdata/feature/3_liaohj
refactor: do some internal refactor.
2022-12-27 16:07:23 +08:00
Haojun Liao 8a4fb244fb fix(query): fix coverity issues. 2022-12-26 14:20:05 +08:00
Haojun Liao f54f6201be fix(query): fix coverity issues. 2022-12-26 14:07:22 +08:00
Haojun Liao 9107c3149b fix(query): fix syntax error. 2022-12-26 13:49:16 +08:00
Haojun Liao 1a07451bc3 refactor: do some internal refactor. 2022-12-26 13:45:25 +08:00
Haojun Liao 723efba79b
Merge branch '3.0' into refact/submit_req 2022-12-26 13:25:25 +08:00
Haojun Liao a5165ab5e7 refactor: do some internal refactor. 2022-12-26 10:58:17 +08:00
Shengliang Guan 028cc52846
Merge pull request #19116 from taosdata/feature/3_liaohj
refactor: do some internal refactor.
2022-12-26 09:12:35 +08:00
Haojun Liao 1edf1e1799 refactor: do some internal refactor. 2022-12-23 23:02:40 +08:00
dapan1121 d39e34ce7c
Merge pull request #19071 from taosdata/fix/TS-2276-main
Fix(tsdb): table block distributed totalSize is zero
2022-12-23 11:54:13 +08:00
wangmm0220 fd76fdb9e0 opti:taosx for write_raw_block 2022-12-22 16:50:21 +08:00
Haojun Liao 8210c49a85 fix(query): fix memory leak. 2022-12-22 15:47:02 +08:00
Alex Duan 950529d148 fix(query): show distributed table modify average calc ++ 2022-12-21 19:03:48 +08:00
Alex Duan 8dfbcc5a83 fix(query): getFileBlockDistInfo totalSize set use index 0 2022-12-21 17:19:00 +08:00
Alex Duan ebf35651f3 fix(query): getFileBlockDistInfo totalSize set 2022-12-21 17:01:59 +08:00
Haojun Liao d5a770c98a fix(query): add comment. 2022-12-21 16:25:18 +08:00
Haojun Liao d1b85f893e fix(query): update the blockInfo 2022-12-21 16:24:37 +08:00
Haojun Liao fdade6d6a3
Merge branch '3.0' into refact/submit_req 2022-12-19 11:24:00 +08:00
Haojun Liao b5f611328a fix(query): add check for blockdata. 2022-12-14 20:31:21 +08:00
Shengliang Guan 04750d9078
Merge pull request #18905 from taosdata/feature/3_liaohj
refactor: do multiple refactor and improve some query perf.
2022-12-14 08:01:42 +08:00
Haojun Liao 48f7fc5c3d
Merge branch '3.0' into refact/submit_req 2022-12-12 18:11:17 +08:00
Haojun Liao ba50a7c30d refactor(query): add additional attribute to denote if current block has already loaded data. 2022-12-12 14:25:26 +08:00
Shengliang Guan 9e8f4f5e71
Merge pull request #18862 from taosdata/feature/3_liaohj
fix(query): return with error code if the table does not exist
2022-12-10 15:56:30 +08:00
Haojun Liao eeefd1a4b5 fix(query): does not return error code if queried table does not exists. 2022-12-10 14:35:56 +08:00
Haojun Liao 44dc2564eb fix(query): return with error code if the table does not exist during creating tsdbReader 2022-12-09 21:07:03 +08:00
Haojun Liao 27ebafdbaa other: merge 3.0 2022-12-09 10:19:05 +08:00
54liuyao 6c9d6e2aa8 fix:read the deleted data 2022-12-08 15:42:37 +08:00
Haojun Liao d3aa0be66e fix(query): add duplicated row merge for column format data in memory. 2022-12-07 20:36:42 +08:00
Haojun Liao 35d13a91c2 fix(query): enable merge duplicated rows. 2022-12-07 18:40:18 +08:00
Alex Duan cc285d4900 fix(query): SMA ensure support info pColAgg capacity over column numfor this table 2022-12-07 18:00:32 +08:00
Alex Duan 15b8b28972 fix(query): SMA ensure support info pColAgg capacity over column numfor this table 2022-12-07 17:52:49 +08:00
Hongze Cheng c0494b6453 fix another memory problem 2022-12-07 10:21:01 +08:00
Hongze Cheng 2d85c7c451 Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/submit_req 2022-12-07 09:04:00 +08:00
Haojun Liao 78aea791df fix(query): remove invalid free. 2022-12-06 16:38:31 +08:00
Hongze Cheng 259b7e8a8a fix compile issue 2022-12-06 14:53:02 +08:00
Hongze Cheng 899d7b0bcc Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/submit_req 2022-12-06 14:37:22 +08:00
Haojun Liao 114943ddb0 fix(query): set the correct flag when no neighbors exist. 2022-12-05 22:28:26 +08:00
Haojun Liao d5b493677b fix(query): set column sma into the correct position. 2022-12-05 20:39:37 +08:00
Haojun Liao 9e04f42635 fix(query): check for neighbor block when merging data block. 2022-12-05 20:20:48 +08:00
Haojun Liao 136937d69a fix(query): insert null column sma firstly. 2022-12-05 17:05:07 +08:00
Shengliang Guan af3bd5bc9a
Merge pull request #18670 from taosdata/fix/liao_cov
enh(query): avoid repeatly load del file.
2022-12-04 13:20:14 +08:00
Haojun Liao 0d687a533e fix(query): check for null ptr. 2022-12-04 11:35:54 +08:00
Haojun Liao fca20668f5 fix(query): check for 0 tables in tsdbreader 2022-12-03 23:52:38 +08:00
Haojun Liao 803762c4e0 enh(query): avoid repeatly load del file. 2022-12-03 22:57:00 +08:00
Shengliang Guan d9c261cd13 refact: remove some error codes 2022-12-03 10:17:02 +08:00
Shengliang Guan 6d70f48802
Merge pull request #18592 from taosdata/feature/3_liaohj
enh(query): optimize query perf.
2022-12-02 21:00:35 +08:00
Haojun Liao 8152541998 fix(query): set correct tsrow index. 2022-12-02 18:16:12 +08:00
Haojun Liao 27ed740eb5 refactor: do some internal refactor. 2022-12-02 17:52:49 +08:00
Haojun Liao 42a512c39f refactor: do some internal refactor. 2022-12-02 15:52:32 +08:00
Haojun Liao 6d9e229a72 refactor: fix the in-memory row merge. 2022-12-02 11:40:03 +08:00
Haojun Liao 24614d03bd fix(query): set correct sma retrieve procedure. 2022-12-01 21:35:01 +08:00
Hongze Cheng f8b18c4f18 fix some code 2022-12-01 20:48:55 +08:00
Haojun Liao efcc3b06bb fix(query): fix memory leak. 2022-12-01 20:01:09 +08:00
Haojun Liao 8c62d5f9fa fix(query): fix bug in tsdbread. 2022-12-01 18:34:26 +08:00
Haojun Liao f8a2ab8338 refactor: do some internal refactor. 2022-12-01 17:24:26 +08:00
Hongze Cheng 73f8dc6a5b more code 2022-12-01 14:24:10 +08:00
Hongze Cheng 86b0dc98c3 Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/submit_req 2022-11-30 21:17:51 +08:00
Hongze Cheng 4b5671a631 Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/submit_req 2022-11-30 19:28:52 +08:00
Haojun Liao 124fb5fc50 refactor: do some internal refactor. 2022-11-30 17:52:12 +08:00
Haojun Liao e45023f3e9 fix(query): add an null ptr checking before copying memory. 2022-11-30 12:23:08 +08:00
Hongze Cheng f5270f6a0c adjust api 2022-11-30 10:30:03 +08:00
Haojun Liao 445c4f28f0 enh(query): optimize the perf. 2022-11-29 23:34:22 +08:00
Hongze Cheng f812957d3a Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/submit_req 2022-11-28 16:35:28 +08:00
Hongze Cheng 0f8016995c more code 2022-11-28 13:41:06 +08:00
Haojun Liao 6e81e8cc73 refactor: do some internal refactor. 2022-11-28 12:32:40 +08:00
Haojun Liao 0948216ca9 fix(query): fix dead lock. 2022-11-26 21:13:34 +08:00
Haojun Liao 9a208b2d09 other: merge 3.0 2022-11-24 17:08:15 +08:00
slzhou 9a85050ff3 fix: direct return file block only rows <= capacity 2022-11-23 15:43:57 +08:00
slzhou c1ba62b26f fix: build from buf first when buf data is in block gap and only capacity caused load 2022-11-23 14:52:07 +08:00
slzhou 0161374a8b Merge branch '3.0' of github.com:taosdata/TDengine into szhou/fixbugs 2022-11-22 22:59:06 +08:00
slzhou 272a161a61 fix: build data from mem/imem first when large nchar column incur fileBlockShouldLoad since SDataBlockToLoadInfo.moreThanCapacity 2022-11-22 22:06:39 +08:00
Shengliang Guan d588cd37f8
Merge pull request #18357 from taosdata/fix/liao_cov
fix(query): fix memory leak.
2022-11-22 20:49:33 +08:00
Haojun Liao 7c7b2f1cef fix(query): fix memory leak. 2022-11-22 18:06:01 +08:00
Haojun Liao b70a616a2d refactor: do some internal refactor. 2022-11-22 15:27:33 +08:00
Haojun Liao 4d206fd3b2 refactor: do some internal refactor. 2022-11-22 14:59:17 +08:00
Haojun Liao 93ad9e9d3d refactor: do some internal refactor. 2022-11-22 14:29:30 +08:00
dapan1121 44462ef869
Merge pull request #18321 from taosdata/szhou/fixbugs
fix: fix error for select count(c2),count(ts) from table caused by null ts column aggregation
2022-11-22 13:17:11 +08:00
Haojun Liao 0af77e6ae2 refactor: do some internal refactor. 2022-11-22 12:43:00 +08:00
Haojun Liao fbb0a1e921 fix(query): set correct tsdbreader during the creation of tableblockscan info 2022-11-22 11:08:55 +08:00
Haojun Liao 56edf57c12 refactor: do some internal refactor. 2022-11-22 09:52:53 +08:00
slzhou bb51d3cac7 fix: fix error for select count(c2),count(ts) from table caused by invalid ts column aggregation 2022-11-21 23:11:00 +08:00
Haojun Liao 8fee089aaf refactor: do some internal refactor. 2022-11-21 22:48:25 +08:00
Haojun Liao 6d5a256774 fix(query): fix some memory leaks. 2022-11-21 22:40:20 +08:00
Haojun Liao a464053692 fix(query): set correct size of array list. 2022-11-17 00:46:06 +08:00
Haojun Liao 9c459a24e2 fix(query): fix memory leak. 2022-11-17 00:38:35 +08:00
Haojun Liao a54275a7b5 fix(query): support null columns sma optimization. 2022-11-16 23:46:59 +08:00
Haojun Liao cf56c38e7a fix(query): fix errors found by asan. 2022-11-14 17:10:25 +08:00
Haojun Liao 9f5cf450de refactor: do some internal refactor. 2022-11-10 15:37:18 +08:00
Haojun Liao 411371ba48 refactor: do some internal refactor. 2022-11-09 14:16:21 +08:00
Haojun Liao 0c427b5f5b refactor: do some internal refactor. 2022-11-09 10:39:32 +08:00
Haojun Liao 68f2f92116 fix(query): fix the syntax errors. 2022-11-09 09:07:21 +08:00
Haojun Liao d238f387ce other: merge 3.0 2022-11-09 00:00:14 +08:00
Haojun Liao 663138ba1e enh(query): optimize the query perf. 2022-11-08 18:33:57 +08:00
dapan1121 d5874419ec
Merge pull request #17935 from taosdata/fix/TD-20246
fix: fix tsdbRead crash caused by reuse after free
2022-11-08 17:48:25 +08:00
Haojun Liao cc3e97da36 fix(query): fix an typo. 2022-11-08 17:41:29 +08:00
Haojun Liao 6e1a2b1e54 enh(query): improve the perf. 2022-11-08 17:38:09 +08:00