Commit Graph

521 Commits

Author SHA1 Message Date
Haojun Liao a170796f66 fix(query): fix invalid read/write. 2022-09-02 14:34:30 +08:00
Haojun Liao cac1fd36b7 Merge remote-tracking branch 'origin/refact/tsdb_optimize' into refact/tsdb_optimize
# Conflicts:
#	source/dnode/vnode/src/inc/tsdb.h
#	source/dnode/vnode/src/tsdb/tsdbMergeTree.c
#	source/dnode/vnode/src/tsdb/tsdbRead.c
2022-09-02 11:23:01 +08:00
Haojun Liao 720f3b2482 refactor: do some internal refactor. 2022-09-02 11:20:12 +08:00
Hongze Cheng 0eb29455ae refact code 2022-09-02 11:19:34 +08:00
Hongze Cheng aec4297eb4 refact code 2022-09-02 11:16:23 +08:00
Hongze Cheng ff4fda8442 refact code 2022-09-02 10:43:49 +08:00
Hongze Cheng 1132a7f719 refact code 2022-09-02 10:17:40 +08:00
Haojun Liao 52c5992c87 other:merge 3.0 2022-09-02 09:58:16 +08:00
Haojun Liao d1b7d07790 refactor:do some internal refactor. 2022-09-02 09:54:22 +08:00
Haojun Liao fed9efa840 fix(query): check memory malloc failure. 2022-09-01 18:46:50 +08:00
Haojun Liao b2cf2818eb refactor(query): do some internal refactor. 2022-09-01 18:23:44 +08:00
Haojun Liao a3011a0194 refactor(query): do some internal refactor. 2022-09-01 17:51:26 +08:00
Haojun Liao 72e0f09dd2 refactor(query): do some internal refactor. 2022-09-01 15:50:41 +08:00
Haojun Liao e65927bc5b fix(query): fix bug in merge multiple last files. 2022-09-01 15:11:40 +08:00
Haojun Liao 1cfd3e74f9 enh(query): support merge multiple last files. 2022-09-01 11:56:56 +08:00
Haojun Liao fdea9db9d3 fix(query): fix macro definition error. 2022-08-30 09:58:30 +08:00
Hongze Cheng 90d7ba0f4f Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/tsdb_optimize 2022-08-29 11:02:53 +08:00
wade zhang 2ad31384f6
Merge pull request #15818 from taosdata/feature/TD-14761
feat: get meta and data from tmq using snapshot
2022-08-29 09:49:35 +08:00
Haojun Liao 655dbb47c1 refactor(query): do some internal refactor. 2022-08-26 19:19:48 +08:00
Haojun Liao 60d9acb7b4 feature(query): support last function cache and retrieve data. 2022-08-26 15:27:19 +08:00
Hongze Cheng c677d5494b make it compile 2022-08-26 13:03:12 +08:00
Hongze Cheng b833c3ef89 Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/tsdb_optimize 2022-08-26 13:01:15 +08:00
Hongze Cheng 4a1447b666 make it compile 2022-08-26 10:12:36 +08:00
Hongze Cheng 5bbfc54504 Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/tsdb_optimize 2022-08-26 10:08:12 +08:00
Haojun Liao 5ea4c0a7c8 fix(query): update the uid list when tablemap is changed. 2022-08-25 23:04:07 +08:00
Hongze Cheng eb2fb724b1 more code 2022-08-25 18:15:20 +08:00
Haojun Liao 755176d28f fix(query): add the null ptr check. 2022-08-25 17:32:27 +08:00
wangmm0220 6b61d3c23b fix:conflicts from 3.0 2022-08-25 16:57:19 +08:00
Haojun Liao 0ec1c52a19 fix(query): set ascending order of check last block. 2022-08-25 16:34:00 +08:00
Haojun Liao 739457ad28 refactor: do some internal refactor. 2022-08-25 16:22:22 +08:00
Haojun Liao 0d14fa9aad fix(query): reset the table accesss index. 2022-08-25 16:06:54 +08:00
Haojun Liao b09ee742d5 refactor(query): check table in uid order when handling the last block, and some internal refactor. 2022-08-25 15:31:48 +08:00
Haojun Liao a23f5f59ea refactor(query): do some internal refactor. 2022-08-24 17:09:33 +08:00
wangmm0220 2afc23a120 fix:conflicts from 3.0 2022-08-24 13:59:51 +08:00
Hongze Cheng a4d4b10404 make it compile 2022-08-24 11:58:10 +08:00
Hongze Cheng 0920cb911a Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/tsdb_optimize 2022-08-24 11:52:40 +08:00
Haojun Liao 370b5136c2 fix(query): fix the last block check condition. 2022-08-24 10:38:26 +08:00
Haojun Liao 20b06a7ec7 refactor(query): do some internal refactor. 2022-08-23 19:44:59 +08:00
Haojun Liao 2889b8d913 refactor(query): do some internal refactor. 2022-08-23 17:28:08 +08:00
Hongze Cheng 478d5ebb64 more code 2022-08-23 13:49:10 +08:00
wangmm0220 1134319570 Merge branch '3.0' of https://github.com/taosdata/TDengine into feature/TD-14761 2022-08-23 10:45:51 +08:00
Haojun Liao 821a12a048 fix(query): add check before retrieve data. 2022-08-22 18:17:47 +08:00
Haojun Liao 9cdf2bb3a5 fix(query): init the blockdata before load data. 2022-08-22 17:43:55 +08:00
wangmm0220 43862ccd74 Merge branch '3.0' of https://github.com/taosdata/TDengine into feature/TD-14761 2022-08-22 16:53:53 +08:00
Haojun Liao 4322f59a22 fix(query): fix bug in descending order scan in lastblock. 2022-08-22 14:28:13 +08:00
Haojun Liao e61ee31de6 refactor(query): do some internal refactor. 2022-08-22 13:26:09 +08:00
Haojun Liao 530c775d97 refactor(query): do some internal refactor. 2022-08-22 11:57:10 +08:00
Haojun Liao e4881423b8 fix(query): add one more condition check when merge file block and last block. 2022-08-22 11:09:12 +08:00
Haojun Liao f029df4876 fix(query): reset index when moving to next file. 2022-08-21 22:17:12 +08:00
Haojun Liao a7ba94939a fix(query): set init value for current last block index. 2022-08-21 22:02:01 +08:00
Haojun Liao 4879ae3d68 refactor(query): optimize load last block. 2022-08-21 21:36:15 +08:00
Haojun Liao 29a1245be4 fix(query): use the separate last block delete index. 2022-08-21 20:15:53 +08:00
Haojun Liao 7904686915 fix(query): support delete in last 2022-08-21 18:59:21 +08:00
Haojun Liao d305e975c4 fix(query): remove printf. 2022-08-21 15:56:46 +08:00
Haojun Liao fe918b3f69 fix(query): set correct column data. 2022-08-21 15:56:06 +08:00
Haojun Liao 5a1ba3d016 fix(query): increase the output buffer size. 2022-08-20 23:26:58 +08:00
Haojun Liao 6d8a70ebf6 fix(query): remove invalid free. 2022-08-20 20:22:39 +08:00
Haojun Liao df0d6f1823 fix(query): fix memory leak. 2022-08-20 19:05:55 +08:00
Haojun Liao 0dfd032068 fix(query): reset block data when no blocks are available. 2022-08-20 16:57:44 +08:00
Haojun Liao 09ffcd2e13 fix(query): fix bug in tsdbread 2022-08-20 12:41:58 +08:00
Haojun Liao 515e8d36b0 fix(query): fix bug in merge buffer data and last block. 2022-08-20 11:14:00 +08:00
Haojun Liao 5eae03fcab fix(query): handle the reverse case. 2022-08-18 22:57:26 +08:00
Haojun Liao eaef3dffaf fix(query): fix error in query last block. 2022-08-18 22:42:16 +08:00
Haojun Liao e75c400045 fix(query): fix bug in query last block. 2022-08-18 18:48:50 +08:00
Hongze Cheng e9bb654a4a Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/tsdb_last 2022-08-18 17:14:51 +08:00
Haojun Liao 8eb86676d0 fix(query): support query last file. 2022-08-18 17:09:44 +08:00
wangmm0220 fbc69a10c4 Merge branch '3.0' into feature/TD-14761 2022-08-18 16:45:06 +08:00
Haojun Liao 9dc5b4cb0c fix(query): support read from last file. 2022-08-18 00:08:09 +08:00
Haojun Liao 5b396276ea Merge branch 'refact/tsdb_last' of github.com:taosdata/tdengine into refact/tsdb_last 2022-08-17 07:54:31 +08:00
Haojun Liao 559286db71 refactor(query): support read data from last files. 2022-08-17 07:51:09 +08:00
wangmm0220 78c9f245b1 fix:conflicts from 3.0 2022-08-16 15:18:11 +08:00
Hongze Cheng 3359b8e620 more optimize 2022-08-15 10:16:07 +00:00
Hongze Cheng e1b638aeb5 Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/tsdb_last 2022-08-15 03:02:54 +00:00
Cary Xu ae7550eb86 enh: rsma exec in async mode 2022-08-12 21:00:48 +08:00
Minglei Jin 6058b8dff5
Merge pull request #16003 from taosdata/fix/TD-18321
fix: use mem schema if pReader->pSchema is null
2022-08-11 19:18:30 +08:00
Haojun Liao 02300b30fa
Merge pull request #15995 from taosdata/feature/3_liaohj
fix(query): add check when invoking sma filter.
2022-08-11 18:52:51 +08:00
Minglei Jin 203a8073ca fix: use mem schema if pReader->pSchema is null 2022-08-11 18:19:44 +08:00
Haojun Liao 3a6285c478 fix(query): add check when invoking sma filter. 2022-08-11 17:40:55 +08:00
Cary Xu 1d6d0aa2db enh: row iter and fetch optimization 2022-08-10 17:26:44 +08:00
Haojun Liao 6250c1b157
Merge pull request #15851 from taosdata/feature/3.0_query_optimize
ehn(query): optimize the query performance.
2022-08-10 16:51:46 +08:00
Ganlin Zhao 249e2b12f6 fix(query): fix interp + fill interpolation after last datablock 2022-08-10 14:21:31 +08:00
Haojun Liao 41d2c92231 fix(query): copy null value. 2022-08-09 16:54:05 +08:00
Haojun Liao 29c84833d3 fix(query):fix invalid free. 2022-08-09 10:34:01 +08:00
Haojun Liao 30a64bdcd9 fix(query): set correct schema. 2022-08-09 10:16:00 +08:00
Hongze Cheng 4f06c4bbdd make it compile 2022-08-08 07:32:02 +00:00
Hongze Cheng 8ae2ab1c6e more work 2022-08-08 05:34:01 +00:00
Hongze Cheng 33c3f34f34 more work 2022-08-08 03:22:24 +00:00
Hongze Cheng 71691eb7da make code compile 2022-08-07 13:14:43 +00:00
Hongze Cheng 7c32e099d0 more work 2022-08-06 14:20:30 +00:00
Hongze Cheng 04804e384a more 2022-08-06 12:24:46 +00:00
Hongze Cheng 3436d52316 more refact 2022-08-06 12:23:29 +00:00
Hongze Cheng 3324f4e5b7 more work 2022-08-06 12:04:42 +00:00
wangmm0220 f28155714e fix:confilcts from 3.0 2022-08-06 12:31:15 +08:00
wangmm0220 45e7597df3 feat:get data from snapshot for taosx 2022-08-05 21:12:18 +08:00
Hongze Cheng 51538a1710 more work 2022-08-05 09:47:26 +00:00
Haojun Liao 52173e14e1 fix(query): add check for null ptr. 2022-08-05 16:10:45 +08:00
Haojun Liao 0e4748771a enh(query): support combine multi datablock to pass to upstream operators 2022-08-05 14:10:45 +08:00
Haojun Liao 271528f4e2 refactor: opt tsdb performance. 2022-08-04 18:12:44 +08:00
Haojun Liao 4cd7af0ecf other: merge 3.0 2022-08-03 19:02:43 +08:00
Cary Xu 71419d330d other: adjust rsma debug logs 2022-08-03 14:58:54 +08:00
Cary Xu 335da8fc65 other: add tsma debug log 2022-08-03 14:33:29 +08:00
Haojun Liao bc600a4942 refactor: do some internal refactor optimize the building block performance. 2022-08-03 14:08:03 +08:00
Haojun Liao 64db9afeb7 refactor: do some internal refactor. 2022-08-03 10:52:06 +08:00
Shengliang Guan efb0127fbf refactor: adjust logs 2022-08-02 16:48:49 +08:00
Shengliang Guan d5aea7dac2 refactor: adjust sync log 2022-08-02 16:39:12 +08:00
Minglei Jin 711c75652c
Merge pull request #15671 from taosdata/fix/TD-18074
fix: new pSchema if reader's schema is null
2022-08-02 14:12:30 +08:00
Haojun Liao bb869800f6
Merge pull request #15661 from taosdata/feature/3_liaohj
refactor: optimize the build block perf.
2022-08-02 13:04:00 +08:00
Minglei Jin 5cf4dac071 fix: new pSchema if reader's schema is null 2022-08-02 12:56:33 +08:00
Shengliang Guan bd31c9da5a
Merge pull request #15657 from taosdata/fix/dnode
fix: write snapshot after apply queue is empty
2022-08-01 20:13:08 +08:00
Haojun Liao bbcba1855d Merge branch '3.0' into feature/3_liaohj 2022-08-01 20:09:42 +08:00
Shengliang Guan 99f29f6455 refactor: adjust log 2022-08-01 17:23:52 +08:00
54liuyao b3d104349f feat(stream):optimize stream update scan 2022-08-01 17:09:46 +08:00
Haojun Liao 7db6497d28 refactor: update some logs. 2022-08-01 11:19:06 +08:00
Minglei Jin c2faef7377
Merge pull request #15555 from taosdata/fix/TD-17968
fix: update schema to newest version to parsing rows
2022-07-30 16:04:04 +08:00
Minglei Jin e57f12d581 fix: use colId to merge rows 2022-07-30 15:23:28 +08:00
Minglei Jin 56d2697308 fix: new tRowMergerInit2 for ts row merging 2022-07-30 12:46:40 +08:00
Haojun Liao 2261d689e4 fix(query): opt read data from file block. 2022-07-29 21:09:33 +08:00
Haojun Liao 0c1e30dc56 fix(query): step forward index 2022-07-29 19:34:13 +08:00
Minglei Jin 0c111cd2bf fix: update schema to newest version to parsing rows 2022-07-29 18:15:44 +08:00
Haojun Liao 60c5b2beac fix(query): optimize the row merge procedure for files. 2022-07-29 17:53:30 +08:00
Haojun Liao aeed234b2e fix(query):fix memory leak. 2022-07-26 20:40:39 +08:00
Haojun Liao bd8fa53371 fix(query): set current SBlock before move to next block. 2022-07-26 15:52:47 +08:00
Haojun Liao 6600540fd9 refactor: optimize the memory consumption during tsdbread 2022-07-26 11:43:45 +08:00
Haojun Liao 231f439979 refactor: opt memory consumption for tsdbread. 2022-07-26 10:19:12 +08:00
Haojun Liao 6d86ef0c33 other: merge 3.0 2022-07-25 11:22:09 +08:00
Haojun Liao 9bddf99d7d refactor(query): do some internal refactor. 2022-07-25 11:17:45 +08:00
Cary Xu 00a9598ac8 enh: rsma level 2/3 submitReq msg use wal version 2022-07-22 20:39:13 +08:00
Hongze Cheng 8cdef36879 fix: sam cases 2022-07-22 04:56:15 +00:00
Hongze Cheng cd762afe76 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/hzcheng_3.0 2022-07-22 04:50:24 +00:00
Hongze Cheng b27272d784 fix memory leak 2022-07-22 01:40:02 +00:00
Haojun Liao 7522c2edf3 refactor: do some internal refactor. 2022-07-21 20:44:36 +08:00
Hongze Cheng bce29321c6 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/hzcheng_3.0 2022-07-21 12:40:03 +00:00
Hongze Cheng 6a7a3da3db fix some crash 2022-07-21 11:56:18 +00:00
Hongze Cheng 684dd82358 fix read concurrency 2022-07-21 11:42:42 +00:00
Haojun Liao 8e42bf85e8 fix(query): fix border check. 2022-07-21 15:47:22 +08:00
Hongze Cheng f195decac9 more fix 2022-07-21 06:27:32 +00:00
Haojun Liao 2a437ee38a fix(query): close del handle when successing in read del file content. 2022-07-21 11:35:31 +08:00
Haojun Liao c2d588a118 other:merge 3.0 2022-07-21 11:29:34 +08:00
Haojun Liao ef0d705c5f fix(query): close del file read handle. 2022-07-21 11:27:20 +08:00
Minglei Jin 37bb5caf29
Merge pull request #15205 from taosdata/fix/TD-17582
fix: use suid from pReader
2022-07-20 19:44:40 +08:00
Minglei Jin 5cc829e45f fix: use suid from pReader 2022-07-20 18:47:46 +08:00
Minglei Jin 982b11f6eb
Merge pull request #15184 from taosdata/fix/TD-17582
fix: use Ex version of metaGetTbTSchema to retrieve schema
2022-07-20 16:51:10 +08:00
Minglei Jin 527aa3584d fix: get suid from uid to be used to retrieve schema 2022-07-20 15:26:47 +08:00
Minglei Jin eefad2bd9d fix: use Ex version of metaGetTbTSchema to retrieve schema 2022-07-20 14:28:13 +08:00
WANG MINGMING 258f4fb20a
Merge branch '3.0' into feature/TD-14761 2022-07-20 11:20:42 +08:00
Minglei Jin e422d7dbf5 fix: remove assert if schema is null 2022-07-20 10:29:21 +08:00
wangmm0220 bc0e8e3061 fix:invalidate in telnet schemaless 2022-07-20 09:48:57 +08:00
Haojun Liao a0428f5685 fix(query): add null ptr check. 2022-07-19 18:05:26 +08:00
Hongze Cheng 024f2221b7 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/hzcheng_3.0 2022-07-19 08:31:16 +00:00
Hongze Cheng 657c2dcaa4 more r/w concurrency 2022-07-19 07:21:15 +00:00