Commit Graph

468 Commits

Author SHA1 Message Date
WANG MINGMING 3bc6635de3
feat[TS-6115]: support deletion of topics with active consumers. (#30232)
* feat[TS-6115]: drop topic & consumer group in force

* feat[TS-6115]: drop topic & consumer group in force

* feat[TS-6115]: drop topic & consumer group in force
2025-03-18 14:11:00 +08:00
WANG MINGMING 930b57b770
feat: support fill_history 1 async when creating stream (#30185)
* feat:[TS-5617]use last_row cache mode if creating stream in fill_history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* fix: heap use after free

* feat: add log

* fix: ci case error

* fix: compile error in windows

* fix: ci case error

* fix: heap user after free

* fix: memory leak

* fix: ci case error

* fix: ci case error

* fix: ci case error

* feat: support fill_history 1 async when creating stream

* feat: support fill_history 1 async when creating stream

* fix: compile error

* feat[TS-5617]: support fill_history 1 async when creating stream

---------

Co-authored-by: yihaoDeng <yhdeng@taosdata.com>
2025-03-18 10:01:33 +08:00
Jing Sima 410324746b
feat:[TS-4897] virtual table (#30098)
* feat: [TS-4897] Support create/drop/alter/show/describe vtable

* feat: [TS-4897] Support vtable's query

* feat: [TS-4897] Support create virtual supertable

* feat: [TS-4897] Support explain analyze / where / count(*) and only select ts of vtable.

* feat: [TS-4897] Add create test and fix bugs

* feat: [TS-4897] Add alter/drop test and fix bugs

* feat: [TS-4897] Add describe/show test and fix bugs

* feat: [TS-4897] Add auth test and fix bugs

* feat: [TS-4897] Fix meta/catalog/cache bugs

* feat: [TS-4897] Support select tag from virtual child table

* feat: [TS-4897] Add select test and fix plenty of bugs

* feat: [TS-4897] Add optimize rule for vtable scan / support create vtable cross database / remove enterprise constraint / fix bugs.

* feat: [TS-4897] Fix 'schema is old'

* feat: [TS-4897] Support virtual stable query

* feat: [TS-4897] Add tests and Fix bugs

* feat: [TS-4897] resolve conflict.
2025-03-15 14:10:46 +08:00
liuyao 2de6736160
feat(stream): add new trigger continuous_window_close (#30125)
* opt stream build twa result

* opt force window close memory

* feat(stream):optimize new interval and scan operator

* adj log

* opt code

* opt code

* fill history

* fix issue for fill history

* add ci

* feat(stream): add new stream nonblock interval operator

* adjust code

* use new scan operator

* use new scan operator

* add log

* fix issue

* recover stream scan next function

* fix issue

* fix issue

* fix issue

* ignore disorder data

* fix issue

* fix issue for interval sliding

* fix issue

* fix ci issue

* fix ci issue

* fix ci issue

* add semi && final nonblock interval operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* refactor(stream): track the msgId for each upstream tasks.

* fix(stream): fix race condition.

* fix(stream): update the task last msgId when putting into input queue succ.

* fix issue

* fix issue

* put recalculate data to rocksdb

* fix issue

* fix issue

* enh(query)[TD-33071]: add support for saving and restoring tsdbReader scan progress

- Implement functionality to save scan progress during tsdbReader operations
- Enable resuming scans from the last saved position

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* add rec interval check

* enh(stream):add recalculate tasks.

* enh(stream): support the re-calculate the tasks.

* fix issue && do refactor

* do refactor

* fix issue

* fix issue

* update backend opt

* add new interface

* add new session operator

* support blob

* add new session operator

* fix issue

* add rec state for rec task

* fix invalid read

* add new session window

* enh(stream): update the stream tasks backend.

* new session operator

* add pull data

* fix(stream): fix error in expand stream backend.

* fix issue

* fix issue

* fix issue

* merge code

* fix issue

* fix(stream): check for null ptr.

* fix(stream): add more check.

* fix issue

* fix issue

* fix issue

* add debug code

* fix issue

* fix issue

* fix issue

* set rec end flag

* fix(stream): opt re-calculate stream tasks.

* fix issue

* fix issue

* add new operator

* enh(stream): dispatch recalculate block to agg tasks.

* fix issue

* fix issue

* fix(stream): adjust filter.

* fix issue

* refactor

* refactor(stream): adjust the recalculate end block.

* fix issue

* fix(stream): set correct create trigger block.

* fix issue

* fix(stream): fix error in build recalculate end block.

* fix(stream): check null ptr.

* add stream client && fix issue

* fix mem leak

* fix(stream): free msg.

* add stream client

* fix(stream): fix error.

* add stream client && fix issue

* add stream client

* refactor(stream): set the recalculate task complete.

* add wend and group_id for session window dest table

* feat(stream): refactor and set the recalcul agg task complete.

* add cfg for adapter

* fix issue

* add state && event operator

* feat(stream): support fill-history task.

* add new fill operator

* fix(stream): set correct backend when updating fill-history task to recalculate task.

* add new fill operator

* fix(stream): s2025-03-06 11:10:31.272
et ops always open in scan tsdb

* fix(stream):set the correct taskType for sink task.

* new fill operator

* adj stream fill operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* adj test

* fix issue

* fix(stream): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix: ut com error

* fix(stream): fix mem leak and adjust operator type check rule

* fix(stream): fix mem leak and adjust test case

* refactor code

* fix(stream): free items.

* fix(stream): free fix memory leak.

* fix(stream): fix syntax error.

* fix: ignore unexpect block

* fix: adjust op type

---------

Co-authored-by: Haojun Liao <hjliao@taosdata.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
2025-03-14 20:14:01 +08:00
wangjiaming f50fcb85b8
feat(decimal): support decimal data type (#30060)
* decimal: create table

* decimal: add test case decimal.py

* decimal: add decimal.c

* support input decimal

* decimal test

* refactor svalue

* fix test cases

* add decimal unit test

* add decimal test cmake

* support insert and query decimal type

* define wide integer, support decimal128

* support decimal128 divide

* set decimal type expr res types

* scalar decimal

* convert to decimal

* fix decimal64/128 from str and to str

* fix decimal from str and decimal to str

* decimal simple conversion

* unit test for decimal

* decimal conversion and unit tests

* decimal + - * /

* decimal scalar ops and comparision

* start to refactor GET_TYPED_DATA

* support decimal max func, cast func

* refactor GET_TYPED_DATA interface

* decimal scalar comparision

* start to implement sum for decimal

* support sum and avg for decimal type

* decimal tests

* add decimal test

* decimal add test cases

* decimal use int256/int128

* decimal testing

* fix decimal table meta and add tests for decimal col streams

* fix create stream and create tsma

* test insert decimal values

* decimal from str

* test decimal input

* test parse decimal from string

* add taos_fetch_field_e api

* decimal insert tests

* test decimal operators

* decimal operator test

* feat:support decimal in raw block

* decimal operator tests

* decimal test

* feat:support decimal in raw block

* feat:support decimal in raw block

* feat:add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* decimal test operators

* decimal operator test

* test decimal operators

* test decimal compare operators

* decimal unary operator test

* decimal col with decimal col oper test

* test decimal col filtering

* fix decimal float operator test

* decimal test where filtering

* fix decimal filtering

* fix decimal order by

* fix decimal op test

* test decimal agg funcs

* test decimal functions

* remove assert

* fix ci build for ret check

* fix decimal windows build

* fix ci ret check

* skip decimal ret check

* skip decimal ret check

* fix decimal tests

* fix decimal ci test

* decimal test

* fix(tmq): heap user after free

* fix(tmq): double free

* fix(tmq): double free

* fix decimal tests

* fix(decimal): decimal test ci build

* fix(decimal): windows build

* fix(decimal): decimal test build

* fix(decimal): fix decimal build and tests

* fix(decimal): fix decimal tests

* fix(decimal): fix taos_fetch_fields_e api

* fix(decimal): fix decimal taos_fetch_fields_e api

* fix(decimal): rebase 3.0

* fix(decimal): fix decimal functions

* fix(decimal): fix decimal test case memory leak

* fix(decimal): fix decimal tests

* fix(decimal): fix decimal test case

* fix(decimal): fix decimal tests

* feat(decimal): fix unit tests

* feat(decimal): fix deicmal unit test

---------

Co-authored-by: wangmm0220 <wangmm0220@gmail.com>
Co-authored-by: yihaoDeng <yhdeng@taosdata.com>
2025-03-14 18:08:07 +08:00
Zhixiao Bao af7e34e189
feat(keep): support keep on super table level. (#30097)
* Feat: support use keep while create super table.

* Test(keep): add test for create super table with keep option.

* Feat(keep): Add tmsg for create keep.

* Feat(keep): support alter table option keep.

* Fix(keep): Add baisc test for alter table option.

* Fix(keep): memory leek.

* Feat(keep): add keep to metaEntry&metaCache and fix earliestTs with stn keep.

* Test(keep): add some cases for select with stb keep.

* Fix: fix ci core while alter stb.

* Feat(keep): delete expired data in super table level.

* Feat: remove get stb keep while query.

* Fix : build error.

* Revert "Fix : build error."

This reverts commit 0ed66e4e848e5528d57f5a7c9fa7600c4742a71c.

* Revert "Feat(keep): delete expired data in super table level."

This reverts commit 36330f6b4c587bc126d3f75480d81e637d7601e6.

* Fix : build errors.

* Feat : support restart taosd.

* Fix : alter table comment problems.

* Test : add tests for super table keep.

* Fix: change sdb stb reserve size.

* Test: add more tests.

* Feat: Disable normal tables and sub tables from setting the keep parameter

* Fix: add more checks to avoid unknown address.

* Docs: Add docs for stable keep.

* Fix: some review changes.

* Fix: review errors.
2025-03-14 16:10:13 +08:00
xinsheng Ren 4edddd6038
Merge branch '3.0' into enh/TS-5650/showLike 2025-03-07 10:29:35 +08:00
dongming chen e5fb75797a
Merge branch '3.0' into feat/TS-5927-long-password 2025-03-05 12:15:12 +08:00
factosea e91014f807 feat: show variables like 2025-03-02 22:48:05 +08:00
Hongze Cheng 0d2ca243b1 Merge branch '3.0' of https://github.com/taosdata/TDengine into enh/TS-5445-3.0 2025-02-27 17:18:33 +08:00
Simon Guan 5d777f9bcf
Merge pull request #29654 from taosdata/feat/TS-5805-force-assign-leader
Feat/ts 5805 force assign leader
2025-02-27 15:42:27 +08:00
Hongze Cheng 541f98037f Merge branch '3.0' of https://github.com/taosdata/TDengine into enh/TS-5445-3.0 2025-02-27 14:06:53 +08:00
dongming chen 0ce087f2b3
Merge branch '3.0' into feat/TS-5805-force-assign-leader 2025-02-26 13:57:36 +08:00
Hongze Cheng b26166b48d support meta_only option 2025-02-25 17:57:25 +08:00
Hongze Cheng d48547ed64 more code 2025-02-25 16:48:43 +08:00
wangmm0220 9b6a094cac feat:[TS-5613] modify sql.y to move bool_expression to common expression 2025-02-25 09:32:17 +08:00
wangmm0220 ee0f7a4d38 feat:[TS-5613]support bool in cast 2025-02-24 17:45:07 +08:00
wangmm0220 47e23e3d99 feat:[TS-5613]support bool in castfsd 2025-02-24 17:37:01 +08:00
Shengliang Guan 5ad05a662a Merge branch '3.0' into feat/TS-5470-3.0 2025-02-24 16:12:47 +08:00
Shengliang Guan 3a1d06f4d7
Merge pull request #29354 from taosdata/enh/TS-5255/colsFunc2
enh: ts5255/cols func2
2025-02-24 14:19:37 +08:00
Jinqing Kuang 51ffff2079 feat(query)[TS-5470]: add syntax to specify minimum duration for event and state windows
Introduce the `true for` syntax to allow users to specify the minimum
duration for event and state windows. Add corresponding tests to
validate the feature. Updated the user manual with usage instructions.
2025-02-24 08:02:48 +08:00
dmchen 538ce1948c retry case 2025-02-17 02:09:57 +00:00
Shengliang Guan 14aee31c22
Merge pull request #29757 from taosdata/feat/TS-5486
feat: support interp function with time range and time interval
2025-02-14 14:43:52 +08:00
dmchen ee0c280fe6 feat/TS-5805-force-assign-leader-token 2025-02-12 08:20:56 +00:00
dmchen 977372d376 feat/TS-5805-force-assign-leader-fix-cases 2025-02-12 06:50:37 +00:00
dongming chen 0350f46ba4
Merge pull request #29756 from taosdata/3.0
3.0
2025-02-12 14:00:02 +08:00
dapan1121 bf13c90a61 feat: support range fill with interval limitation 2025-02-11 19:14:15 +08:00
Shengliang Guan cd52401303 Merge branch 'main' into merge/mainto3.02 2025-02-11 15:15:35 +08:00
kailixu 27963d0978 fix: keep_time_offset format of show create database 2025-02-10 14:21:44 +08:00
Shengliang Guan 60283e67de Merge branch '3.0' into merge/mainto3.02 2025-02-08 11:11:10 +08:00
Jinqing Kuang 62a872bb49 fix: resolve compilation issues
- Remove redundant third-party library builds
- Fix parsing conflicts in sql.y
2025-02-06 15:20:53 +08:00
factosea 1cc17dfacc enh: regexp 2025-02-06 15:14:38 +08:00
Shengliang Guan a004d95996 Merge branch '3.0' into merge/mainto3.02 2025-02-06 13:46:36 +08:00
dmchen b07f4a9bac feat/TS-5805-force-assign-leader-change-sql 2025-01-24 09:08:47 +00:00
dmchen 9608344833 feat/TS-5805-force-assign-leader-add-sql 2025-01-24 03:47:45 +00:00
dmchen d9401fd75d feat/TS-5805-add-sql-command 2025-01-23 09:37:07 +00:00
factosea 65e0990dc0 merge 3.0 2025-01-23 14:16:56 +08:00
dapan1121 84c37d501a enh: support ? in limit clause 2025-01-22 11:45:12 +08:00
Jinqing Kuang eb5d463490 feat(stream)[TS-5469]. add support for window event notifications in stream processing
- Introduce new syntax to specify notification type and destination address
- Collect relevant event information during window computations
- Implement websocket-based notification delivery to the specified address
2025-01-21 17:39:46 +08:00
Shengliang Guan 3e1692d2c5
Merge pull request #29410 from freemine/freemine.lemon.warning.as.error.with.jointest
Freemine.lemon.warning.as.error.with.jointest
2025-01-07 10:04:12 +08:00
freemine c2e5910c23 inner join [ON ...]; window join [ON ...] WINDOW_OFFSET ... 2025-01-03 22:26:47 +08:00
freemine 86da050f25 fail as early as in parser stage 2025-01-02 19:43:45 +08:00
freemine 3d9b243f85 revert: only [LEFT|RIGHT] is allowed for non-inner-join 2024-12-31 15:17:56 +08:00
Shengliang Guan fcca5e7c2e
Merge pull request #29381 from taosdata/fix/liaohj
test: add test cases and fix related bugs.
2024-12-31 11:10:34 +08:00
freemine 26ba35c883 re-enable FULL <non-inner-join> at sql grammar level 2024-12-31 07:37:55 +08:00
freemine 12efc52af3 typo: JOIN_TYPE_NONE -> JOIN_TYPE_INNER 2024-12-30 20:15:33 +08:00
freemine 4acf170690 remove LEFT as default for non-inner-joins 2024-12-30 19:30:43 +08:00
freemine 69826e9569 Merge remote-tracking branch 'tdengine/3.0' into freemine.lemon.warning.as.error 2024-12-30 14:41:38 +08:00
kailixu 243195c37a test: add test case for auto compact 2024-12-29 15:21:49 +08:00
kailixu de3558a4a7 test: add test case for auto compact 2024-12-29 15:14:51 +08:00