Commit Graph

575 Commits

Author SHA1 Message Date
dapan1121 8ef01584ee
Merge pull request #22629 from taosdata/feat/TD-25620
feat: optimize partition node, replace with sort node
2023-09-14 17:26:15 +08:00
Haojun Liao 24524e1a8d fix(stream): fix the invalid length of tag column name. 2023-09-11 19:10:54 +08:00
Haojun Liao 50ef500f51 refactor(stream): do sink module refactor. 2023-09-11 10:09:31 +08:00
wangjiaming0909 4a131000e4 fix: data error of partition by col + interval 2023-09-07 10:49:09 +08:00
dapan1121 2e7897239a fix: trim block and partition by constant function issue 2023-08-30 16:51:44 +08:00
dapan1121 0cce255a02 Merge remote-tracking branch 'origin/3.0' into fix/m23.0 2023-08-28 10:06:53 +08:00
Haojun Liao 88f6596965
Merge pull request #22464 from taosdata/feat/TD-18789
feat:support varbinary type
2023-08-28 09:45:57 +08:00
dapan1121 1f40da6fef Merge remote-tracking branch 'origin/main' into 3.0 2023-08-28 09:28:55 +08:00
liuyao b142979f7f mem leak 2023-08-24 15:00:09 +08:00
dapan1121 3045a623e1 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-08-17 14:16:05 +08:00
wangmm0220 b3f945c4c5 fix:conflict 2023-08-16 19:35:05 +08:00
wangmm0220 f266d51764 feat:support varbinary type 2023-08-16 19:05:07 +08:00
dapan1121 143c657571 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-08-16 14:50:31 +08:00
liuyao 919ffbaf72 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/ly_refactor 2023-08-16 10:04:07 +08:00
yihaoDeng 7d1e4a9894 rebuild index at tag0 2023-08-15 08:25:33 +00:00
liuyao 9d1e5a512d add stream task id 2023-08-14 17:52:39 +08:00
dapan1121 6ce28e8886 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-08-14 10:49:56 +08:00
slzhou ff36f5f6ca fix: reset rowsize since it is set during blockDataAppendColInfo 2023-08-08 13:23:35 +08:00
dapan1121 4f75eb5655 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-07-24 11:43:15 +08:00
slzhou b9aeda263c fix: fix ubsan error of null argument 2023-07-19 09:48:52 +08:00
dapan1121 779224f0a6 fix: fix bugs 2023-07-06 19:33:31 +08:00
dapan1121 c85bbef2e9 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-07-05 11:13:57 +08:00
dapan1121 9f0da17c49 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-06-28 19:40:55 +08:00
Haojun Liao ed900d6ff6
Merge branch '3.0' into refact/fillhistory 2023-06-28 13:48:26 +08:00
wangjiaming0909 57ddb3437d feature: optimize order by limit using priority queue 2023-06-27 17:50:50 +08:00
Haojun Liao 3289ad62be other: merge 3.0 2023-06-27 11:33:40 +08:00
dapan1121 4dae9f4eff merge main to 3.0 2023-06-26 18:43:00 +08:00
dapan1121 9972aa81d6 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-06-25 17:31:32 +08:00
dapan1121 24a5a3f41b fix: compile issues 2023-06-25 17:31:04 +08:00
Haojun Liao 0322fdc1fc fix(stream): fix memory leak. 2023-06-24 23:43:43 +08:00
Haojun Liao ceb78fdad2
Merge branch '3.0' into refact/fillhistory 2023-06-24 18:53:48 +08:00
dapan1121 7e555e437b enh: add hash join 2023-06-20 19:34:16 +08:00
dapan1121 1ca24fea6b
Merge pull request #21742 from wangjiaming0909/feature/3.0/TD-24656
feature: add input, output ts order for execution plans
2023-06-18 19:24:29 +08:00
wangjiaming0909 fb4f174fbc feature: add input, output ts order for execution plans 2023-06-16 10:26:09 +08:00
Haojun Liao 74e0138f72 Merge branch '3.0' into refact/fillhistory 2023-06-15 15:25:00 +08:00
Haojun Liao 1b9eca5ce0 other: merge main. 2023-06-15 13:03:17 +08:00
Haojun Liao 55724157c4 refactor: remove one function. 2023-06-15 10:40:25 +08:00
slzhou a36557f009 fix: calculate the rows that can be put in one page 2023-06-14 13:44:11 +08:00
Haojun Liao 53246ed6bb other: merge other branch. 2023-06-09 09:41:10 +08:00
Haojun Liao 2b6ace653a refactor: do some internal refactor. 2023-06-08 16:14:27 +08:00
Haojun Liao f582705feb refactor: do some internal refactor. 2023-06-08 14:18:06 +08:00
liuyao 1e9c4d5fac opt stream block dispatch 2023-06-07 10:45:27 +08:00
dapan1121 a0428387a9 enh: corrent column data reassign issue 2023-06-06 16:44:15 +08:00
dapan1121 02b1257b01 enh: reassign data column in blocks 2023-06-06 11:17:02 +08:00
dapan1121 7381209618 enh: optimize filter performance 2023-06-05 16:08:11 +08:00
dapan1121 532e02fb2a enh: improve filter performance 2023-06-05 10:47:15 +08:00
liuyao 1f81642303 fix sma sliding window 2023-05-25 09:12:10 +08:00
Dingle Zhang 984211f3b4
Feature/3.0 geometry (#21037)
* Add GEOMETRY data type and make sql.c able to parse it. The GEMETRY works like BINARY so far.

* add GEOMETRY type into gConvertTypes to fix some issues like DELETE calling

* change some test cases to make sure no same timestamp is inserted, and add my smoketest.sh

* Add a function MakePoint() and introduce a lib geometry

* implement sql functions GeomFromText() and AsText()

* Use GEOS *_r funcions instead for thread safety

* Handle with TSDB_DATA_TYPE_GEOMETRY when INSERT geometry data by converting WKT.
Add geosWrapper to wrap the basic GEOS functions for TDEngine.

* refactor AsText and MakePoint functions to be like GeomFromText

* Show WKT when print geometry data in screen
Dump hex data when dump geometry data in a file

* define TYPE_BYTES item for TSDB_DATA_TYPE_GEOMETRY, which casued some strange issues.

* set number of decimals of WKT to 6

* Implement SQL function Intersects()

* refactor geometry sql functions

* Add geosErrMsgeHandler() to get the GEOS error detail

* use threadlocal to instantiate SGeosContext
call destroyGeosContext() only if the thread exists

* remove SGeosContext *context param for all geometry functions since we use thread local one,
so that all caller do not need to know the context.

* Modify Intersects() to call PreparedIntersects() when one of param is a constant, which has higher performance.

* rename prepareFn() to initCtxFn() to avoid confusion with PreparedFn

* Add prefix "ST_" for all geometry functions

* move getThreadLocalGeosCtx() and destroyThreadLocalGeosCtx() into util,
so that all unit test tools can compile

* Add unit test for geometry lib, only test MakePoint so far

* refactor and enhance existing cases in geomFuncTest

* implement NULL type and NULL value test for geomFuncTest

* add test on geomFromText()

* add unit test on AsText() in geomFuncTest

* combine some makePointFunction test items

* add intersectsFunctionTwoColumns test
refactor on callGeomFromTextWrapper functions

* enhance intersectsFunction test to add cases like input constant , NULL type, NULL value, or wrong content

* add more cases into intersectsFunction test

* Add basic test on geometry in system test

* Add ST_GeomFromText and ST_AsText function test in system test on geometry

* add ST_Intersects function test in system test on geometry

* support to check expectedErrno in system test on geometry

* adjust geomTest unit test and geometry system test

* add geometry data type and functions in doc english version

* implement touchesFunction() in geometry lib
refactor geometry relation functions model

* separate gemFuncTest into several src files

* add unit test on touchesFunction

* support sql function ST_Touches()
add system test on ST_Touches

* add docs for ST_Touches()

* Add ST_Contains()

* Add ST_Covers()

* Add ST_Equals()

* add swapAllowed param for geomRelationFunction()
read geom2 earlier intead of at doGeosRelation()

* Add ST_ContainsProperly()

* build on windows

* Merge from 3.0 to 3.0_geometry

* change macro definition TSDB_DATA_TYPE_GEOMETRY as the last one for compatibility

* change '\\NULL' to 'NULL' back in shellDumpFieldToFile()

* add /usr/local/include into include directory

* add /usr/local/inlcude and /usr/local/lib in cmake.platform for DARWIN
2023-05-24 15:36:46 +08:00
Haojun Liao 9423001fea
Merge pull request #21396 from taosdata/refact/fillhistory
Refact/fillhistory
2023-05-21 01:12:00 +08:00
Haojun Liao 8ab065d01e other: update some logs. 2023-05-19 23:53:23 +08:00
Ganlin Zhao 709b59a3a6 fix: fix assert crash in tsort.c 2023-05-19 17:27:15 +08:00
dapan1121 8fe63e1e24
Merge pull request #21180 from taosdata/szhou/fix-ts3326
fix: clear null bitmap bit when set value
2023-05-06 15:58:57 +08:00
slzhou 25170e1a75 fix: restore the removal of show datablock to console 2023-05-06 13:32:58 +08:00
slzhou 6b0580eedf fix: clear null bitmap bit when set value 2023-05-06 13:30:49 +08:00
Haojun Liao 09a04052ec refactor: do some internal refactor. 2023-05-04 16:15:14 +08:00
Haojun Liao db1d75bb48 refactor: do some internal refactor. 2023-04-27 09:28:20 +08:00
Haojun Liao 02ec67ad9e
Merge pull request #20882 from taosdata/fix/TS-3081
fix:open test cases for tmq & add log if rebalance error
2023-04-14 17:56:58 +08:00
wangmm0220 e985f15cf0 fix:memset nullBitmap of SSDataBlock to 0 in udf 2023-04-14 09:36:36 +08:00
wangmm0220 7648e03e31 fix:doBitmapMerge error if remaind bytes is not 0 2023-04-13 10:03:30 +08:00
Xiaoyu Wang 4c04b155d5 merge main 2023-04-11 15:56:28 +08:00
Ganlin Zhao 2d0dad7a37 fix: fix nullbitmap shift error when trimming data blocks. 2023-04-07 19:35:46 +08:00
Ganlin Zhao 1c6bf2d1d3 fix: fix nullbitmap shift error when trimming data blocks. 2023-04-07 18:08:49 +08:00
Xiaoyu Wang c134432c45 merge main 2023-04-06 09:31:45 +08:00
Ganlin Zhao 23ec9287c9 fix: fix tsim crash on windows due to invalid input to strftime 2023-04-04 10:33:41 +08:00
Ganlin Zhao 6c64b760a0 fix: fix tsim crash on windows due to invalid input to strftime 2023-04-03 17:58:43 +08:00
Haojun Liao addabf3ba7 refactor: push down the tableListinfo to scanner. 2023-04-02 15:25:31 +08:00
dapan1121 9952be43f6 Merge remote-tracking branch 'origin/main' into fix/cencVer 2023-03-29 15:50:14 +08:00
dapan1121 0327a8eaa3 fix: block rows number type 2023-03-29 10:46:56 +08:00
dapan1121 22ad3129b9 fix: query schema is old issue 2023-03-28 15:21:47 +08:00
dapan1121 61571d9296 enh: optimize count performance 2023-03-27 19:40:29 +08:00
54liuyao 87ba6cb019 fix:fill history check update 2023-03-08 19:19:09 +08:00
54liuyao 8a23f0ce3b fix:set timestamp precision 2023-03-01 18:59:27 +08:00
Xiaoyu Wang 58fc265b80 merge main 2023-02-24 13:26:45 +08:00
Haojun Liao 97024f1468 fix(query): fix coverity issue. 2023-02-24 09:48:34 +08:00
Xiaoyu Wang 1ab144609e merge main 2023-02-23 14:58:22 +08:00
dapan1121 6aa05c4370
Merge pull request #20067 from taosdata/fix/TS-2687
fix: add merge join operator result limitation
2023-02-23 10:40:23 +08:00
dapan1121 968c538be5 fix: add merge join operator result limitation 2023-02-21 08:54:05 +08:00
Xiaoyu Wang c597de099f merge main 2023-02-20 13:41:27 +08:00
Haojun Liao 0a9f8347cc fix(query):suppress the syntax error in internal. 2023-02-20 10:51:04 +08:00
Haojun Liao 751c1cb5d2 refactor:do some internal refactor. 2023-02-20 10:12:27 +08:00
Haojun Liao 8d187efc18 refactor:do some internal refactor. 2023-02-20 10:04:08 +08:00
Haojun Liao c597a1e4ce refactor: do some internal refactor. 2023-02-13 15:46:47 +08:00
Haojun Liao 72287a3b5f fix(query): set correct tag value during tag filter and do some internal refactor. 2023-02-10 16:03:11 +08:00
Haojun Liao 3822225231 other: merge main. 2023-02-06 16:24:15 +08:00
wangmm0220 762f6466f3 fix:conflicts from 3.0 2023-02-06 09:23:29 +08:00
Xiaoyu Wang 5d499e52f9 merge main 2023-02-03 17:13:11 +08:00
wangmm0220 c8d49b603d fix:roll back htol length for block raw data 2023-02-02 17:23:01 +08:00
wangmm0220 936b15629f fix:remove htonl length in block encode 2023-01-31 15:26:58 +08:00
wangmm0220 2ad5869659 fix:add log for block encode 2023-01-31 14:25:39 +08:00
Haojun Liao 7803104b7e fix(query): do some internal refactor, and identify a bug. 2023-01-20 22:50:35 +08:00
Haojun Liao 07cf336fa0 refactor: add an array list api. 2023-01-17 09:43:33 +08:00
Xiaoyu Wang 9a0c2805f4 merge main 2023-01-13 13:54:04 +08:00
Haojun Liao e0dbe22199 fix(query): fix uninitialized memory buffer accessed. 2023-01-11 10:27:23 +08:00
Haojun Liao 75f4e64326 enh(query): adjust the memset rule when creating new datablock. 2023-01-10 23:45:49 +08:00
Haojun Liao 3eadb9b921 enh(query): disable the memset when allocate the memory for dataBlock. 2023-01-10 23:08:25 +08:00
54liuyao d5ade95b31 feat(stream):user define tag 2023-01-10 10:13:56 +08:00
Shengliang Guan b9f95b6a12
Merge pull request #19390 from taosdata/merge/mainto3.0_0105
merge from main to 3.0
2023-01-06 08:59:32 +08:00
Shengliang Guan 3016f35e9f Merge branch '3.0' into merge/mainto3.0_0105 2023-01-05 16:08:23 +08:00
Ganlin Zhao a2cc08145a fix: fix coverity issues 2023-01-05 14:11:05 +08:00
dapan1121 8a773d6b7f
Merge pull request #19325 from taosdata/enh/TD-21206
fix: fix ASSERTs
2023-01-04 16:10:12 +08:00
Ganlin Zhao 795fcab242 fix error 2023-01-04 14:01:04 +08:00
Ganlin Zhao 0f877eea43 fix error 2023-01-04 11:29:35 +08:00
Ganlin Zhao 4b81962269 fix asserts 2023-01-04 09:16:00 +08:00
Haojun Liao 00c2d382b1 fix(query): opt filter perf. 2023-01-03 22:45:02 +08:00
kailixu d34dc4bf5f chore: print format 2023-01-03 10:29:24 +08:00
kailixu 88b4a904ac chore: print format 2023-01-03 10:27:58 +08:00
kailixu 8905a4839e fix: tsma result submit for row/column update 2023-01-02 15:57:36 +08:00
Liu Jicong 834495d62a refact: adapt to new sml api 2022-12-26 10:14:59 +08:00
dapan1121 f6123395c0 Merge branch 'refact/submit_req' of github.com:taosdata/TDengine into refact/submit_req 2022-12-20 18:05:03 +08:00
dapan1121 32e8202a25 enh: refact insert into select processing 2022-12-20 16:55:11 +08:00
kailixu adc23925bf refact: optimization of method buildSubmitReqFromDataBlock 2022-12-20 12:07:56 +08:00
Haojun Liao fdade6d6a3
Merge branch '3.0' into refact/submit_req 2022-12-19 11:24:00 +08:00
Shengliang Guan 04750d9078
Merge pull request #18905 from taosdata/feature/3_liaohj
refactor: do multiple refactor and improve some query perf.
2022-12-14 08:01:42 +08:00
Haojun Liao e4ad989070 fix(query): disable assert. 2022-12-13 00:17:11 +08:00
Haojun Liao e3e42cdfbd fix(query): set update ts flag for stream. 2022-12-13 00:16:40 +08:00
Haojun Liao 48f7fc5c3d
Merge branch '3.0' into refact/submit_req 2022-12-12 18:11:17 +08:00
kailixu fcb99d4586 Merge branch 'refact/submit_req' of github.com:taosdata/TDengine into refact/submit_req 2022-12-12 16:56:00 +08:00
Haojun Liao ba50a7c30d refactor(query): add additional attribute to denote if current block has already loaded data. 2022-12-12 14:25:26 +08:00
54liuyao 8caca97568 feat:add delete mark for sma 2022-12-09 11:34:58 +08:00
WANG MINGMING 35b5f185a2
Merge pull request #18833 from taosdata/refact/submit_req_marks
opti:performance of schemaless
2022-12-09 05:04:48 +08:00
kailixu 5dbbaa3b77 refact: add more check for rsma 2022-12-08 15:06:18 +08:00
kailixu a2092892f7 refact: tsma/rsma code optimization 2022-12-08 14:43:48 +08:00
kailixu 6f377e3234 refact: tsma/rsma data format 2022-12-08 14:39:59 +08:00
wangmm0220 443aa2ee3f fix:modify influxdb parse logic for sml 2022-12-07 11:05:33 +08:00
Hongze Cheng 899d7b0bcc Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/submit_req 2022-12-06 14:37:22 +08:00
Haojun Liao 392fe988a0 fix(query): remove invalid assert for mac&win. 2022-12-03 20:16:56 +08:00
kailixu e360094f8a chore: code format 2022-12-03 20:15:31 +08:00
kailixu 42b17760a2 refact: tsma result adaption 2022-12-03 20:06:55 +08:00
kailixu 11684ceb82 refact: resource mgmt when build block for rsma 2022-12-03 15:00:40 +08:00
kailixu e918982a5a refact: rsma adaption for new SSubmitReq 2022-12-03 14:23:58 +08:00
kailixu d3a88ea517 refactor: row and submit msg of rsma/tsdb 2022-12-03 13:33:30 +08:00
Haojun Liao fa43f86fe1 refactor: add an assert. 2022-12-03 12:43:03 +08:00
Haojun Liao b6199d8d99 enh(query): update the aligned bits for colInfodata 2022-12-03 12:37:42 +08:00
Shengliang Guan 6d70f48802
Merge pull request #18592 from taosdata/feature/3_liaohj
enh(query): optimize query perf.
2022-12-02 21:00:35 +08:00
Haojun Liao 83782fb27b fix(query): fix invalid tag fill and clear nullbitmp 2022-12-02 10:07:48 +08:00
Haojun Liao f8a2ab8338 refactor: do some internal refactor. 2022-12-01 17:24:26 +08:00
Haojun Liao cefe4be1ab refactor: do some internal refactor. 2022-12-01 00:34:06 +08:00
Haojun Liao ae81633144 enh(query): optimize query perf. 2022-11-30 14:55:27 +08:00
Haojun Liao 445c4f28f0 enh(query): optimize the perf. 2022-11-29 23:34:22 +08:00
Shengliang Guan d65e55558f
Merge pull request #18540 from taosdata/main
merge from main to 3.0 branch
2022-11-29 14:43:36 +08:00
MysticBoy 4b668a4153
修改可能导致读取额外内容 2022-11-28 17:47:37 +08:00
Haojun Liao 6e81e8cc73 refactor: do some internal refactor. 2022-11-28 12:32:40 +08:00
Haojun Liao 203ea1f9c8 fix(query): fix error 2022-11-27 14:57:44 +08:00
Haojun Liao 7fe4cf7bbe
Merge branch '3.0' into feature/3_liaohj 2022-11-25 22:24:59 +08:00
Ganlin Zhao 0ea961b659 fix: [ASAN] fix nullpointer issue in tdatablock.c 2022-11-24 21:24:11 +08:00
Haojun Liao 9a208b2d09 other: merge 3.0 2022-11-24 17:08:15 +08:00
Shengliang Guan 335a572d86
Revert "Revert "enh: row optimize2"" 2022-11-24 10:12:25 +08:00
Hongze Cheng ffbb04d40a
Revert "enh: row optimize2" 2022-11-24 09:21:26 +08:00
Shengliang Guan d31be656e3
Merge pull request #18401 from taosdata/fix/TD-20605
fix: [ASAN] fix null pointer in tdatablock.c
2022-11-24 08:04:01 +08:00
Haojun Liao 611ab8b394 refactor: make sure the memory is aligned to 32 bytes. 2022-11-23 21:52:51 +08:00