Commit Graph

3183 Commits

Author SHA1 Message Date
liuyao 9a3708e17b pause&resume fill history 2023-06-14 15:54:07 +08:00
Haojun Liao 7c6fbd7772 enh(stream): do some internal refactor and support secondary scan for history data. 2023-06-14 14:10:09 +08:00
Ganlin Zhao e79d63d766 add error code 2023-06-14 13:34:25 +08:00
Haojun Liao 11f0c3b336 refactor: do some internal refactor. 2023-06-14 10:46:46 +08:00
Haojun Liao b44447e6f3 enh(stream): support restore from disk. 2023-06-14 10:19:03 +08:00
Haojun Liao 4fac83ecf2
Merge pull request #21709 from taosdata/feature/TD-24749
feat:control memory size if consumer group is too much
2023-06-14 10:15:20 +08:00
Haojun Liao 97da2a8fac fix(stream): set the fill history status. 2023-06-13 16:10:55 +08:00
wangmm0220 d8d6b6dd21 Merge branch '3.0' of https://github.com/taosdata/TDengine into feature/TD-24749 2023-06-13 15:15:05 +08:00
wangmm0220 8a81278024 fix:load pHandle if necessary wher tq restore 2023-06-13 15:14:11 +08:00
dapan1121 73d3c21e21 Merge remote-tracking branch 'origin/3.0' into feat/TD-24499 2023-06-13 11:44:41 +08:00
Haojun Liao fb24ed161d enh(stream): set correct task status. 2023-06-12 22:48:23 +08:00
dapan1121 170182fecd feat: support fill history with sub request 2023-06-12 16:10:54 +08:00
Haojun Liao 78a240be8f enh(stream): add dispatch msg. 2023-06-12 13:12:45 +08:00
Benguang Zhao 3689b41f3a Merge branch '3.0' into FIX/TD-24182-3.0 2023-06-12 09:39:49 +08:00
Haojun Liao 69c9eda7af fix(stream): fix race condition. 2023-06-09 11:37:38 +08:00
Haojun Liao 53246ed6bb other: merge other branch. 2023-06-09 09:41:10 +08:00
Haojun Liao c9aa59cbb9 refactor: do some internal refactor. 2023-06-08 18:59:25 +08:00
Haojun Liao 6c13491428 refactor: do some internal refactor. 2023-06-08 16:55:41 +08:00
Haojun Liao 30eba472c3
Merge pull request #21602 from taosdata/fix/TD-24600-dv3
fix/td-24600-dv3
2023-06-08 13:42:02 +08:00
Haojun Liao cb26dd9fa2 refactor(stream): do some internal refactor. 2023-06-08 10:17:21 +08:00
Haojun Liao 06cf3588da refactor: do some internal refactor. 2023-06-07 15:03:04 +08:00
liuyao 1e9c4d5fac opt stream block dispatch 2023-06-07 10:45:27 +08:00
Haojun Liao 6a3c935b4a fix(stream): fix error in fill history scan. 2023-06-07 09:15:55 +08:00
dapan1121 65d323d9c7
Merge pull request #21222 from taosdata/fix/TD-24001
enh: optimize timeline function validation
2023-06-06 14:22:53 +08:00
Haojun Liao 0dd933013c enh(stream): make history task for stream running. 2023-06-05 17:58:00 +08:00
Haojun Liao fcc706c45c enh(stream): refactor the fill history task. 2023-06-05 15:59:01 +08:00
Minglei Jin d02746d86f fix(query/sys): pause/resume with sys scan operator 2023-06-05 15:24:39 +08:00
yihaoDeng 74d5f02969 fix invalid free 2023-06-05 04:56:07 +00:00
Haojun Liao 077e1a1ed4 Merge branch '3.0' into refact/fillhistory 2023-06-05 11:41:13 +08:00
Haojun Liao 6a9c03309e
Merge pull request #21553 from taosdata/enh/changeRocksParam
change log level
2023-06-02 14:23:50 +08:00
dapan1121 ddd3a53e56 Merge remote-tracking branch 'origin/3.0' into fix/TD-24001 2023-06-02 13:57:34 +08:00
Haojun Liao d45aa7ef84
Merge pull request #21555 from taosdata/fix/liaohj_main
other: merge main into 3.0.
2023-06-02 13:11:12 +08:00
yihaoDeng 32c3cfd51a Avoid creating the same ID task multiple times 2023-06-01 11:13:27 +00:00
Haojun Liao 5301c5756a other: merge main into 3.0. 2023-06-01 09:29:18 +08:00
Benguang Zhao 28efd7dcd7 enh: check trans conflicts in mndTransValidate 2023-05-31 10:53:36 +08:00
Haojun Liao eda06081ff enh(stream): refactor and serialize the attributes of history tasks. 2023-05-30 16:55:37 +08:00
Haojun Liao d9c364d663 Merge branch '3.0' into refact/fillhistory 2023-05-30 15:51:12 +08:00
Haojun Liao be8fd9e48b enh(stream): create additional task for history data processing. 2023-05-30 14:47:10 +08:00
wangmm0220 d490ca753f Merge branch '3.0' of https://github.com/taosdata/TDengine into feature/TD-19042 2023-05-30 09:48:49 +08:00
Haojun Liao 20a0054c4a
Merge pull request #21472 from taosdata/enh/addCompileError
change parameter
2023-05-30 09:36:36 +08:00
Haojun Liao 3628888ecc refactor: do some internal refactor. 2023-05-29 18:02:45 +08:00
wangmm0220 41ed0138b4 fix:compile error 2023-05-29 17:25:11 +08:00
wangmm0220 07bdb95b1a fix:compile error 2023-05-29 17:10:11 +08:00
yihaoDeng 1782ecd57e Merge branch 'fix/shared_link_rocksdb' into enh/addCompileError 2023-05-29 08:20:11 +00:00
Haojun Liao f2887278e2 refactor: do some internal refactor. 2023-05-29 15:44:45 +08:00
wangmm0220 78db3520a6 fix:conflict 2023-05-29 14:57:03 +08:00
wangmm0220 a71878d07f feat: support create topic as stable with conditions 2023-05-29 11:46:59 +08:00
Haojun Liao 8f61e2d6ef refactor: do some internal refactor. 2023-05-29 10:50:50 +08:00
Haojun Liao 953a69a829 refactor: do some internal refactor. 2023-05-29 10:12:27 +08:00
Haojun Liao b38e253019 fix(stream): set the skip offset when resume stream task. 2023-05-26 22:51:58 +08:00
yihaoDeng 479a117266 Merge branch 'enh/dev3.0' into enh/addCompileError 2023-05-26 13:03:47 +00:00
yihaoDeng a71bffd719 Merge branch 'enh/dev3.0' into enh/addCompileError 2023-05-26 13:03:35 +00:00
yihaoDeng 34d21bc03f change parameter 2023-05-26 20:11:33 +08:00
wangmm0220 a3e214b9e8 fix:put poll to push manager if wal not exist when offset is latest 2023-05-26 19:17:04 +08:00
Haojun Liao cea8334427
Merge pull request #21449 from taosdata/refact/fillhistory
refactor: remove the module dependency.  TD-24279
2023-05-26 09:01:45 +08:00
dapan1121 e70c433a59 enh: support time line query from union/union all subquery 2023-05-26 08:44:23 +08:00
Haojun Liao e24ce2111f fix: set function ptr. 2023-05-26 00:00:07 +08:00
dapan1121 2713f4f6d4 feat: support create topic as stable with conditions 2023-05-25 19:01:58 +08:00
Haojun Liao c3be7b14be fix: set correct function ptr. 2023-05-25 17:51:03 +08:00
Haojun Liao 0966213847 fix(arch): set the correct module api dependency. 2023-05-25 14:21:40 +08:00
Haojun Liao 4a42f6b983 fix: fix error in api call. 2023-05-25 00:50:03 +08:00
Haojun Liao a9b18d203d fix:fix error. 2023-05-24 23:49:26 +08:00
Haojun Liao ab907e3344 fix(query): fix syntax error. 2023-05-24 21:53:40 +08:00
wangmm0220 8537e4e80e feat:add tag filter for stable subscribe 2023-05-24 18:26:55 +08:00
Haojun Liao fe09705855 fix: set the init function pointer. 2023-05-24 17:13:20 +08:00
Dingle Zhang 984211f3b4
Feature/3.0 geometry (#21037)
* Add GEOMETRY data type and make sql.c able to parse it. The GEMETRY works like BINARY so far.

* add GEOMETRY type into gConvertTypes to fix some issues like DELETE calling

* change some test cases to make sure no same timestamp is inserted, and add my smoketest.sh

* Add a function MakePoint() and introduce a lib geometry

* implement sql functions GeomFromText() and AsText()

* Use GEOS *_r funcions instead for thread safety

* Handle with TSDB_DATA_TYPE_GEOMETRY when INSERT geometry data by converting WKT.
Add geosWrapper to wrap the basic GEOS functions for TDEngine.

* refactor AsText and MakePoint functions to be like GeomFromText

* Show WKT when print geometry data in screen
Dump hex data when dump geometry data in a file

* define TYPE_BYTES item for TSDB_DATA_TYPE_GEOMETRY, which casued some strange issues.

* set number of decimals of WKT to 6

* Implement SQL function Intersects()

* refactor geometry sql functions

* Add geosErrMsgeHandler() to get the GEOS error detail

* use threadlocal to instantiate SGeosContext
call destroyGeosContext() only if the thread exists

* remove SGeosContext *context param for all geometry functions since we use thread local one,
so that all caller do not need to know the context.

* Modify Intersects() to call PreparedIntersects() when one of param is a constant, which has higher performance.

* rename prepareFn() to initCtxFn() to avoid confusion with PreparedFn

* Add prefix "ST_" for all geometry functions

* move getThreadLocalGeosCtx() and destroyThreadLocalGeosCtx() into util,
so that all unit test tools can compile

* Add unit test for geometry lib, only test MakePoint so far

* refactor and enhance existing cases in geomFuncTest

* implement NULL type and NULL value test for geomFuncTest

* add test on geomFromText()

* add unit test on AsText() in geomFuncTest

* combine some makePointFunction test items

* add intersectsFunctionTwoColumns test
refactor on callGeomFromTextWrapper functions

* enhance intersectsFunction test to add cases like input constant , NULL type, NULL value, or wrong content

* add more cases into intersectsFunction test

* Add basic test on geometry in system test

* Add ST_GeomFromText and ST_AsText function test in system test on geometry

* add ST_Intersects function test in system test on geometry

* support to check expectedErrno in system test on geometry

* adjust geomTest unit test and geometry system test

* add geometry data type and functions in doc english version

* implement touchesFunction() in geometry lib
refactor geometry relation functions model

* separate gemFuncTest into several src files

* add unit test on touchesFunction

* support sql function ST_Touches()
add system test on ST_Touches

* add docs for ST_Touches()

* Add ST_Contains()

* Add ST_Covers()

* Add ST_Equals()

* add swapAllowed param for geomRelationFunction()
read geom2 earlier intead of at doGeosRelation()

* Add ST_ContainsProperly()

* build on windows

* Merge from 3.0 to 3.0_geometry

* change macro definition TSDB_DATA_TYPE_GEOMETRY as the last one for compatibility

* change '\\NULL' to 'NULL' back in shellDumpFieldToFile()

* add /usr/local/include into include directory

* add /usr/local/inlcude and /usr/local/lib in cmake.platform for DARWIN
2023-05-24 15:36:46 +08:00
Haojun Liao a566180770 fix: fix syntax error. 2023-05-24 15:16:50 +08:00
Haojun Liao 1f7f326bed refactor: do some internal refactor and set the api function ptr. 2023-05-24 13:22:05 +08:00
Haojun Liao 66477a28ca refactor: do some internal refactor. 2023-05-23 19:10:50 +08:00
Haojun Liao 85afbf0d35 refactor: do some internal refactor. 2023-05-23 18:58:54 +08:00
Haojun Liao a63473b1cc refacotor: do some internal refactor. 2023-05-23 18:54:32 +08:00
Haojun Liao c66c0c4dfd refactor: do some internal refactor. 2023-05-23 18:42:52 +08:00
Haojun Liao 3d084adf1f Merge remote-tracking branch 'origin/refact/fillhistory' into refact/fillhistory
# Conflicts:
#	source/dnode/vnode/inc/vnode.h
#	source/libs/executor/src/executil.c
#	source/libs/index/CMakeLists.txt
#	source/libs/index/src/indexFilter.c
2023-05-23 18:32:47 +08:00
Haojun Liao e67b532d8a refactor: refactor the module dependency. 2023-05-23 18:29:23 +08:00
yihaoDeng c15b1e3520 opt compile 2023-05-23 11:46:02 +08:00
dapan1121 4c88fd08de Merge remote-tracking branch 'origin/3.0' into fix/TD-24001 2023-05-23 09:25:37 +08:00
Hui Li 3f824e2045
Merge pull request #21355 from taosdata/mark/tmq
fix:error in pHandle lock
2023-05-22 08:57:45 +08:00
Haojun Liao 9423001fea
Merge pull request #21396 from taosdata/refact/fillhistory
Refact/fillhistory
2023-05-21 01:12:00 +08:00
Haojun Liao de44c9160c refactor: do some internal refactor. 2023-05-20 18:05:39 +08:00
wangmm0220 04858fae22 fix:tsdbreader is free by mistake 2023-05-20 12:20:54 +08:00
Haojun Liao 6ded3284da refactor: do some internal refactor. 2023-05-20 11:37:17 +08:00
Haojun Liao 200c1a367b
Merge pull request #21378 from taosdata/fix/nodisk
fix(query): do some internal refactor, and fix the bug when calender duration exists in sliding and offset.
2023-05-19 19:30:04 +08:00
wangmm0220 ff2b545b27 fix:set task status killed when vnode receive subscribe msg from mnode 2023-05-19 17:16:47 +08:00
dapan1121 42c65898ea
Merge pull request #21335 from taosdata/feat/TD-22746
feat: support client meta control
2023-05-19 16:55:08 +08:00
slzhou f9da4abdcb fix: change more variable names 2023-05-19 15:38:52 +08:00
shenglian zhou f36b0be17d fix: join eq conditions not only for tag 2023-05-19 14:32:03 +08:00
wade zhang 1ade314043
Merge pull request #21325 from taosdata/feat/force-drop-dnode
Feat/force drop dnode
2023-05-19 09:23:54 +08:00
Haojun Liao 1e020a46c9 other: merge main. 2023-05-19 09:07:35 +08:00
Haojun Liao c4c5bbb65c other: merge 3.0 2023-05-18 17:28:40 +08:00
yihaoDeng 9380b22394 merge 3.0 2023-05-18 05:46:38 +00:00
slzhou 304423e27b Merge branch 'main' of github.com:taosdata/TDengine into szhou/fix-ts3405 2023-05-18 13:27:58 +08:00
Haojun Liao 5d5b2bb16a refactor: do some internal refactor. 2023-05-18 12:59:19 +08:00
dapan1121 f5ebc17fae
Merge branch '3.0' into feat/TD-22746 2023-05-18 09:10:49 +08:00
Haojun Liao ffee5ebc61 Merge branch '3.0' into feature/3_liaohj 2023-05-17 17:47:44 +08:00
Haojun Liao 041ffa91bd other: merge main. 2023-05-17 15:42:15 +08:00
Haojun Liao 366cd1871a
Merge pull request #21314 from taosdata/fix/liaohj_main
fix(stream):extract delete msg from wal.
2023-05-17 14:52:49 +08:00
dapan1121 fb7a06e06e
Merge pull request #21271 from taosdata/enh/tsbsPerf.4
Enh/tsbs perf.4
2023-05-17 11:00:14 +08:00
dapan1121 df973ce28e fix: remove debug info 2023-05-16 18:37:00 +08:00
Haojun Liao e35d145734 fix(stream): fix memory leak. 2023-05-16 13:59:13 +08:00
dapan1121 ee2521a6ab Merge remote-tracking branch 'origin/3.0' into feat/TD-22746 2023-05-16 10:53:21 +08:00