Commit Graph

9921 Commits

Author SHA1 Message Date
Haojun Liao 4a42f6b983 fix: fix error in api call. 2023-05-25 00:50:03 +08:00
Haojun Liao a9b18d203d fix:fix error. 2023-05-24 23:49:26 +08:00
Haojun Liao ab907e3344 fix(query): fix syntax error. 2023-05-24 21:53:40 +08:00
Hongze Cheng 4d526ed0cd change more 2023-05-24 18:59:09 +08:00
Minglei Jin fc5d06681e cache/last_row: treat none as null with last_row 2023-05-24 18:55:51 +08:00
wangmm0220 8537e4e80e feat:add tag filter for stable subscribe 2023-05-24 18:26:55 +08:00
Haojun Liao fe09705855 fix: set the init function pointer. 2023-05-24 17:13:20 +08:00
Dingle Zhang 984211f3b4
Feature/3.0 geometry (#21037)
* Add GEOMETRY data type and make sql.c able to parse it. The GEMETRY works like BINARY so far.

* add GEOMETRY type into gConvertTypes to fix some issues like DELETE calling

* change some test cases to make sure no same timestamp is inserted, and add my smoketest.sh

* Add a function MakePoint() and introduce a lib geometry

* implement sql functions GeomFromText() and AsText()

* Use GEOS *_r funcions instead for thread safety

* Handle with TSDB_DATA_TYPE_GEOMETRY when INSERT geometry data by converting WKT.
Add geosWrapper to wrap the basic GEOS functions for TDEngine.

* refactor AsText and MakePoint functions to be like GeomFromText

* Show WKT when print geometry data in screen
Dump hex data when dump geometry data in a file

* define TYPE_BYTES item for TSDB_DATA_TYPE_GEOMETRY, which casued some strange issues.

* set number of decimals of WKT to 6

* Implement SQL function Intersects()

* refactor geometry sql functions

* Add geosErrMsgeHandler() to get the GEOS error detail

* use threadlocal to instantiate SGeosContext
call destroyGeosContext() only if the thread exists

* remove SGeosContext *context param for all geometry functions since we use thread local one,
so that all caller do not need to know the context.

* Modify Intersects() to call PreparedIntersects() when one of param is a constant, which has higher performance.

* rename prepareFn() to initCtxFn() to avoid confusion with PreparedFn

* Add prefix "ST_" for all geometry functions

* move getThreadLocalGeosCtx() and destroyThreadLocalGeosCtx() into util,
so that all unit test tools can compile

* Add unit test for geometry lib, only test MakePoint so far

* refactor and enhance existing cases in geomFuncTest

* implement NULL type and NULL value test for geomFuncTest

* add test on geomFromText()

* add unit test on AsText() in geomFuncTest

* combine some makePointFunction test items

* add intersectsFunctionTwoColumns test
refactor on callGeomFromTextWrapper functions

* enhance intersectsFunction test to add cases like input constant , NULL type, NULL value, or wrong content

* add more cases into intersectsFunction test

* Add basic test on geometry in system test

* Add ST_GeomFromText and ST_AsText function test in system test on geometry

* add ST_Intersects function test in system test on geometry

* support to check expectedErrno in system test on geometry

* adjust geomTest unit test and geometry system test

* add geometry data type and functions in doc english version

* implement touchesFunction() in geometry lib
refactor geometry relation functions model

* separate gemFuncTest into several src files

* add unit test on touchesFunction

* support sql function ST_Touches()
add system test on ST_Touches

* add docs for ST_Touches()

* Add ST_Contains()

* Add ST_Covers()

* Add ST_Equals()

* add swapAllowed param for geomRelationFunction()
read geom2 earlier intead of at doGeosRelation()

* Add ST_ContainsProperly()

* build on windows

* Merge from 3.0 to 3.0_geometry

* change macro definition TSDB_DATA_TYPE_GEOMETRY as the last one for compatibility

* change '\\NULL' to 'NULL' back in shellDumpFieldToFile()

* add /usr/local/include into include directory

* add /usr/local/inlcude and /usr/local/lib in cmake.platform for DARWIN
2023-05-24 15:36:46 +08:00
Ganlin Zhao 89bea7100f fix: fix count var type error 2023-05-24 00:31:05 -07:00
Haojun Liao 99f669434a
Merge pull request #21282 from taosdata/feature/TD-23117
feat:[TD-23117] add schema for ins_topics
2023-05-24 14:54:42 +08:00
dapan1121 cbbbaca553
Merge pull request #21437 from taosdata/fix/TD-23955-3.0
fix: delete index filter for desc query
2023-05-24 14:05:07 +08:00
wade zhang 398bf8f322
Merge pull request #21446 from taosdata/fix/confirm-msg-when-add-learner
comfirm msg when add learner
2023-05-24 13:54:15 +08:00
Haojun Liao b2ce45b07b refactor: do some internal refactor. 2023-05-24 13:39:20 +08:00
Haojun Liao a2f29aa0f5 refactor: do some internal refactor. 2023-05-24 13:34:46 +08:00
Haojun Liao 1f7f326bed refactor: do some internal refactor and set the api function ptr. 2023-05-24 13:22:05 +08:00
kailixu ef899a0cc3 fix: delete index filter for desc query 2023-05-24 11:28:45 +08:00
dmchen ab3d54aae3 comfirm msg when add learner 2023-05-24 11:08:41 +08:00
Minglei Jin 26bfa4363d fix(cache/read): associate tbname with primary ts column 2023-05-24 11:01:47 +08:00
Hongze Cheng d6d0bf0e41 enh: add varchar sma 2023-05-24 10:47:26 +08:00
liuyao 180abf2229 set agg task status 2023-05-24 09:34:40 +08:00
liuyao b5d01f9bab set agg pause status 2023-05-24 09:19:28 +08:00
Haojun Liao 39a1fa8f78 refactor: do some internal refactor. 2023-05-23 19:20:25 +08:00
Haojun Liao 66477a28ca refactor: do some internal refactor. 2023-05-23 19:10:50 +08:00
Haojun Liao 3d084adf1f Merge remote-tracking branch 'origin/refact/fillhistory' into refact/fillhistory
# Conflicts:
#	source/dnode/vnode/inc/vnode.h
#	source/libs/executor/src/executil.c
#	source/libs/index/CMakeLists.txt
#	source/libs/index/src/indexFilter.c
2023-05-23 18:32:47 +08:00
kailixu 34cf6aad30 fix: alter dnode option has no effect 2023-05-23 18:29:53 +08:00
Haojun Liao e67b532d8a refactor: refactor the module dependency. 2023-05-23 18:29:23 +08:00
kailixu 82b8c3c3d1 fix: delete index filter for desc query 2023-05-23 18:21:21 +08:00
kailixu 18e83173c8 fix: delete index filter for desc query 2023-05-23 17:21:45 +08:00
wade zhang 68a98d4d09
Merge pull request #21429 from taosdata/fix/TD-24306
fix(cache): sychronize commit, read and write threads with writebatch
2023-05-23 16:20:59 +08:00
Minglei Jin 6d620a7d7c fix(cache/read): set none to null instead of skipping column 2023-05-23 14:51:26 +08:00
Minglei Jin 81048f93b0 fix(cache): sychronize commit, read and write threads with writebatch 2023-05-23 13:44:33 +08:00
yihaoDeng c15b1e3520 opt compile 2023-05-23 11:46:02 +08:00
wangmm0220 ed612088c0 fix:lost data because tsdbreader or task is killed 2023-05-23 10:32:13 +08:00
kailixu a171c0c860 Merge branch '3.0' into fix/TD-21457-3.0 2023-05-23 09:27:31 +08:00
wade zhang 9e73a69316
Merge pull request #21389 from taosdata/fix/TD-24306
fix(cache/commit): skip batch write if empty
2023-05-23 08:22:33 +08:00
kailixu a3bf49d22f chore: revert 2023-05-22 20:35:52 +08:00
kailixu c75aa54a3d fix: tsdb read and append _wstart for tsma 2023-05-22 20:33:29 +08:00
wangmm0220 d8387b95ff fix:set task status to 0 in inti 2023-05-22 20:14:43 +08:00
Haojun Liao f3491310a9
Merge pull request #21411 from taosdata/fix/TD-24144
fix:remove get schema from previous for taosx
2023-05-22 18:37:20 +08:00
Minglei Jin 75ccd024a2 cache/writebatch: separate read & write batches 2023-05-22 18:06:58 +08:00
dapan1121 08ff4f4540 fix: crash issue 2023-05-22 17:06:01 +08:00
kailixu f18fc19d76 fix: tsdb read and tsma _wstart append 2023-05-22 16:13:50 +08:00
Hui Li 0f4fb1c31d
Merge pull request #21407 from taosdata/mark/tmq
fix:set task code success after kill
2023-05-22 16:10:18 +08:00
wangmm0220 ae515851c4 fix:remove get schema from previous for taosx 2023-05-22 15:32:30 +08:00
Minglei Jin 21adc23af4 cache/loading: skip partial loading tables 2023-05-22 15:21:19 +08:00
wangmm0220 591b51f624 fix:set task code success after kill 2023-05-22 12:42:21 +08:00
Haojun Liao f520de0dab refactor: adjust the module dependency. 2023-05-22 10:07:01 +08:00
Hui Li 3f824e2045
Merge pull request #21355 from taosdata/mark/tmq
fix:error in pHandle lock
2023-05-22 08:57:45 +08:00
Haojun Liao 9423001fea
Merge pull request #21396 from taosdata/refact/fillhistory
Refact/fillhistory
2023-05-21 01:12:00 +08:00
Haojun Liao dcb91082f9 fix(stream): fix error. 2023-05-20 23:37:40 +08:00
Haojun Liao 899e4d3350 refactor: do some internal refactor. 2023-05-20 18:13:58 +08:00
Haojun Liao de44c9160c refactor: do some internal refactor. 2023-05-20 18:05:39 +08:00
wangmm0220 7b735c67d0 fix:move sleep logic from rpc thread to tmq thread & fix some error 2023-05-20 16:36:25 +08:00
wangmm0220 8183be4aff fix:move sleep logic from rpc thread to tmq thread & fix some error 2023-05-20 16:13:52 +08:00
dapan1121 6e5ec3d097
Merge pull request #21358 from taosdata/feat/TD-23362-3.0
feat: support update active code by SQL
2023-05-20 14:09:20 +08:00
wangmm0220 04858fae22 fix:tsdbreader is free by mistake 2023-05-20 12:20:54 +08:00
wangmm0220 1ddbdad33c fix:set task status killed when vnode receive subscribe msg from mnode 2023-05-19 21:40:33 +08:00
wangmm0220 33d6df8717 fix:set task status killed when vnode receive subscribe msg from mnode 2023-05-19 19:11:49 +08:00
dapan1121 778b7b84fd
Merge pull request #21371 from taosdata/fix/TD-24234
fix: distinct wrong result issue
2023-05-19 18:15:04 +08:00
Haojun Liao e2ec8d738c other: add some logs. 2023-05-19 18:08:01 +08:00
wangmm0220 c7c255e967 fix:set task status killed when vnode receive subscribe msg from mnode 2023-05-19 18:06:05 +08:00
Haojun Liao bcaa44e290
Merge pull request #21385 from taosdata/fix/liaohj_main
fix(query): add null check.
2023-05-19 17:55:02 +08:00
wangmm0220 ff2b545b27 fix:set task status killed when vnode receive subscribe msg from mnode 2023-05-19 17:16:47 +08:00
Minglei Jin a5c1b9a054 cache/commit: skip batch write if empty 2023-05-19 17:10:24 +08:00
dapan1121 42c65898ea
Merge pull request #21335 from taosdata/feat/TD-22746
feat: support client meta control
2023-05-19 16:55:08 +08:00
dapan1121 6e6853413c fix: db cfg memory issue 2023-05-19 16:48:05 +08:00
Minglei Jin 962db65775 Merge branch '3.0' into fix/TD-23549 2023-05-19 16:19:57 +08:00
kailixu 8fd4b27c43 chore: code optimization 2023-05-19 16:10:44 +08:00
kailixu 301f7f7a41 Merge branch 'feat/TD-23362-3.0' of https://github.com/taosdata/TDengine into feat/TD-23362-3.0 2023-05-19 16:06:36 +08:00
kailixu 8571e6f6c5 chore: code optimization 2023-05-19 16:06:20 +08:00
Haojun Liao be35194d16 fix(query): add null check. 2023-05-19 15:54:46 +08:00
wangmm0220 f7ab8dabf0 fix:[TD-23972] push subscribe msg to vnode even though consumer not change 2023-05-19 15:14:39 +08:00
wangmm0220 238254e49e fix:add task status check in doQueueScan & checkout consumer in tqProcessPollReq 2023-05-19 14:06:43 +08:00
Haojun Liao d8525123a2 enh(stream): add API to retrieve last ts for multi-tables. 2023-05-19 11:03:41 +08:00
wade zhang 208c9756cf
Merge pull request #21331 from taosdata/fix/TD-24097
enh(cache/batchread): load columns in one trip
2023-05-19 10:20:47 +08:00
kailixu de93895859 chore: code optimization 2023-05-19 09:25:26 +08:00
wade zhang 1ade314043
Merge pull request #21325 from taosdata/feat/force-drop-dnode
Feat/force drop dnode
2023-05-19 09:23:54 +08:00
dapan1121 e882601b67 fix: last table scan issue 2023-05-19 09:15:48 +08:00
wade zhang 7374ea2f0b
Merge pull request #21322 from taosdata/fix/status-resp-return-null-when-only-one-ep
status resp return null when only one ep
2023-05-19 09:04:41 +08:00
kailixu c770a5c306 chore: code optimization 2023-05-19 06:59:28 +08:00
wangmm0220 bca1af3f5d fix:change tmq subscribe try time to 60mins 2023-05-18 21:18:44 +08:00
kailixu e3f194ed9b chore: code optimization 2023-05-18 19:13:04 +08:00
Minglei Jin 35f74ce2e8 cache/schema: not cache schema 2023-05-18 18:35:06 +08:00
kailixu 1de90208f7 test: test case for alter dnode active code 2023-05-18 18:02:48 +08:00
Haojun Liao c4c5bbb65c other: merge 3.0 2023-05-18 17:28:40 +08:00
Haojun Liao dbd52ecffa
Merge pull request #21364 from taosdata/enh/rocksdbSstate
state merge into 3.0
2023-05-18 17:25:44 +08:00
liuyao da3fc5111e fix pause agg task 2023-05-18 15:29:41 +08:00
Minglei Jin f95867d255 cache/del: fix lru key length 2023-05-18 14:48:08 +08:00
Minglei Jin 3b316051bf cache/schema: fix schema double free 2023-05-18 13:51:07 +08:00
yihaoDeng 9380b22394 merge 3.0 2023-05-18 05:46:38 +00:00
kailixu 0e891c5c14 chore: more code 2023-05-18 13:30:45 +08:00
Minglei Jin 0f8b7c87f7 cache/delete: fix rocks key length 2023-05-18 13:15:25 +08:00
kailixu ab06dd04f4 feat: support update active code in sdb 2023-05-18 13:01:28 +08:00
kailixu 5098d44068 feat: support update active code in sdb 2023-05-18 12:59:44 +08:00
Haojun Liao 5d5b2bb16a refactor: do some internal refactor. 2023-05-18 12:59:19 +08:00
Haojun Liao ce61b2ab00 fix: fix syntax error in tqRead 2023-05-18 11:35:45 +08:00
Haojun Liao c3491dae16 refactor: do some interal refactor. 2023-05-18 10:56:40 +08:00
Haojun Liao 81f821a6fc refactor: do some internal refactor. 2023-05-18 10:51:43 +08:00
dapan1121 f5ebc17fae
Merge branch '3.0' into feat/TD-22746 2023-05-18 09:10:49 +08:00
dmchen 32c6ef4b6e fix ci 2023-05-17 23:09:16 +08:00
kailixu cf0d3ddde3 feat: support update active code by SQL 2023-05-17 20:27:12 +08:00
wangmm0220 54f4860139 fix:rollback [TD-23972] push subscribe msg to vnode even though consumer not change 2023-05-17 19:44:56 +08:00
Haojun Liao b9bda9b30f fix(stream): set the correct start offset for resumed stream task. 2023-05-17 19:36:42 +08:00
wangmm0220 405cdfa7bc fix:error in pHandle lock 2023-05-17 19:02:16 +08:00
wangmm0220 1d9dd153ea fix:error in pHandle lock 2023-05-17 18:57:45 +08:00
Minglei Jin 765cc72618 cache/memtable: update cache when flag is on 2023-05-17 18:56:03 +08:00
dmchen c7911c6640 fix ci 2023-05-17 18:33:35 +08:00
Haojun Liao 73b1a4bddf fix(stream): fix syntax error. 2023-05-17 18:29:52 +08:00
Haojun Liao 92feb89a09 refactor: do some internal refactor. 2023-05-17 18:08:04 +08:00
Minglei Jin 5778fa676f Merge branch '3.0' into fix/TD-24097 2023-05-17 18:04:10 +08:00
Haojun Liao ffee5ebc61 Merge branch '3.0' into feature/3_liaohj 2023-05-17 17:47:44 +08:00
Haojun Liao a4d9210b07 fix(tmq): add filter for taosX poll rsp. 2023-05-17 16:40:44 +08:00
Minglei Jin f428a68ec1 cache/update: update lru when deleting or updating 2023-05-17 16:08:48 +08:00
wangmm0220 4bb25c280f fix:add log 2023-05-17 15:54:36 +08:00
Haojun Liao 041ffa91bd other: merge main. 2023-05-17 15:42:15 +08:00
Minglei Jin b892b1fdfe cache/flag: fix cache flag compiling issue 2023-05-17 15:26:10 +08:00
Haojun Liao f55f8d245c fix(tmq): avoid the offset is revised by pool rsp which is issued before invoking tmq_offset_seek function. 2023-05-17 15:19:51 +08:00
Haojun Liao 366cd1871a
Merge pull request #21314 from taosdata/fix/liaohj_main
fix(stream):extract delete msg from wal.
2023-05-17 14:52:49 +08:00
Minglei Jin e95613ec10 cache/commit: update & commit cache if cache not off 2023-05-17 14:32:02 +08:00
Minglei Jin 86e3c1d20d cache/stt: remove ts from merge tree loading 2023-05-17 14:05:28 +08:00
Haojun Liao 3896e148fd log(tmq): add some logs. 2023-05-17 11:37:38 +08:00
dapan1121 fb7a06e06e
Merge pull request #21271 from taosdata/enh/tsbsPerf.4
Enh/tsbs perf.4
2023-05-17 11:00:14 +08:00
Haojun Liao f3f52c0232 fix(stream): add some logs and set the wal offset to be the ended version when finishing the step2 recover stage. 2023-05-17 10:43:23 +08:00
Haojun Liao d457b84aba
Merge pull request #21311 from taosdata/mark/tmq
fix:add lock to pHandle for safety
2023-05-17 09:11:39 +08:00
Minglei Jin 756fefaa29 cache/batchread: load columns in one trip 2023-05-16 18:56:05 +08:00
wangmm0220 90099ea1c0 fix:compile error in windows 2023-05-16 18:41:35 +08:00
wangmm0220 e6860cada8 fix:compile error in windows 2023-05-16 18:35:10 +08:00
wangmm0220 a52939127f fix:dead lock 2023-05-16 17:58:31 +08:00
dapan1121 0eb94d97d3
Merge pull request #21320 from taosdata/fix/TD-24167
pause source task
2023-05-16 15:39:59 +08:00
dapan1121 9033d34852
Merge pull request #21319 from taosdata/fix/TD-24178
fix: file fd not clear issue
2023-05-16 14:34:05 +08:00
dapan1121 e7df9b8fa0 enh: optimize copy group result 2023-05-16 14:12:48 +08:00
dmchen 9ff56a4e2b unsafe command 2023-05-16 14:02:54 +08:00
wangmm0220 e6e0ac7603 fix:alwalys in exec if consumer Id mismatch & check result for wal seek ver 2023-05-16 14:01:38 +08:00
Haojun Liao e35d145734 fix(stream): fix memory leak. 2023-05-16 13:59:13 +08:00
liuyao 3c81137ffe pause source task 2023-05-16 13:43:46 +08:00
dmchen 8e9b253574 status resp return null when only one ep 2023-05-16 11:31:23 +08:00
Haojun Liao cbb02458ed fix(stream): fix the seek condition. 2023-05-16 11:19:11 +08:00
Hui Li ea50ba248e
Merge pull request #21309 from taosdata/FIX/TD-24069-main
fix: ignore crash signals in dmLogCrash
2023-05-16 11:04:21 +08:00
dapan1121 ee2521a6ab Merge remote-tracking branch 'origin/3.0' into feat/TD-22746 2023-05-16 10:53:21 +08:00
dapan1121 dc67223deb fix: file fd not clear issue 2023-05-16 10:47:49 +08:00
Haojun Liao 30e7cb5860 fix(tmq): avoid return delete msg for table subscription. 2023-05-16 10:42:02 +08:00
Haojun Liao d1a746e4e7
Merge pull request #21295 from taosdata/enh/rocksdbSstateMerge
Enh/rocksdb sstate merge
2023-05-16 10:15:45 +08:00
Haojun Liao 5e16db4e19 fix(stream):extract delete msg from wal. 2023-05-16 08:54:03 +08:00
wangmm0220 62520af2e5 fix:core dump 2023-05-16 00:32:21 +08:00
Benguang Zhao b479d510da fix: ignore crash signals in dmLogCrash 2023-05-15 19:39:03 +08:00
Haojun Liao d585f34ea9 fix(tmq): avoid seek to previous position. 2023-05-15 18:48:50 +08:00
wangmm0220 36454c73ed fix:add lock to pHandle for safety 2023-05-15 18:38:14 +08:00
dapan1121 143d3b2ee9
Merge pull request #21291 from taosdata/feat/TD-24115-3.0D
feat: create dnode limited by grant of cpu cores
2023-05-15 18:01:03 +08:00
Haojun Liao 04c8c933c8 fix(stream): add initial offset check. 2023-05-15 17:15:03 +08:00
Haojun Liao 17592ea294 refactor: do some internal refactor. 2023-05-15 13:49:26 +08:00