xxq250
/
homework-jianmu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Merge branch '3.0' into enh/TD-32413/snprintf

This commit is contained in:
xinsheng Ren 2024-10-11 16:07:25 +08:00 committed by GitHub
parent 47a6f3e6bc cd7734735b
commit 9f645a0879
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
120 changed files with 9726 additions and 6236 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
.gitignore vendored
View File

@ -138,3 +138,24 @@ tags
*CMakeCache*
*CMakeFiles*
.history/
*.txt
*.tcl
*.pc
contrib/geos
contrib/libuv
contrib/pcre2
contrib/zlib
deps_tmp_CMakeLists.txt.in
*.a
*.ctest
pcre2-config
pcre2_test.sh
pcre2_grep_test.sh
pcre2_chartables.c
geos-config
config.h
pcre2.h
zconf.h
version.h
geos_c.h

2
cmake/stub_CMakeLists.txt.in
View File

@ -2,7 +2,7 @@
# stub
ExternalProject_Add(stub
GIT_REPOSITORY https://github.com/coolxv/cpp-stub.git
GIT_TAG 5e903b8e
GIT_TAG 3137465194014d66a8402941e80d2bccc6346f51
GIT_SUBMODULES "src"
SOURCE_DIR "${TD_CONTRIB_DIR}/cpp-stub"
BINARY_DIR "${TD_CONTRIB_DIR}/cpp-stub/src"

4
docs/zh/04-get-started/05-cloud.md
View File

@ -15,7 +15,7 @@ TDengine Cloud 大幅减轻了用户在部署、运维等方面的人力负担
要在 TDengine Cloud 注册新用户，请遵循以下简易步骤完成注册流程：
1. 打开浏览器，访问 TDengine Cloud 的首页：https://cloud.taosdata.com，在右边的“注册”部分，填入自己的姓名以及企业邮箱地址，点击“获取验证码”按钮。
1. 打开浏览器，访问 [TDengine Cloud](https://cloud.taosdata.com)，在右边的“注册”部分，填入自己的姓名以及企业邮箱地址，点击“获取验证码”按钮。
2. 检查企业邮箱，找到主题为“你的 TDengine Cloud 注册账户验证码”的邮件。从邮件内容中复制 6 位验证码，并将其粘贴到注册页面上的“验证码”输入框中。接着，点击“注册 TDengine Cloud”按钮，进入客户信息补全页面。
@ -32,4 +32,4 @@ TDengine Cloud 大幅减轻了用户在部署、运维等方面的人力负担
3. 第 3 步，创建实例。在此步骤中，你需要填写实例的区域、名称、是否选择高可用选项以及计费方案等必填信息。确认无误后，点击“创建”按钮。大约等待 1min，新的TDengine 实例便会创建完成。随后，你可以在控制台中对该实例进行各种操作，如查询数据、创建订阅、创建流等。
TDengine Cloud 提供多种级别的计费方案，包括入门版、基础版、标准版、专业版和旗舰版，以满足不同客户的需求。如果你觉得现有计费方案无法满足自己的特定需求，请联系 TDengine Cloud 的客户支持团队，他们将为你量身定制计费方案。注册后，你将获得一定的免费额度，以便体验服务
TDengine Cloud 提供多种级别的计费方案，包括入门版、基础版、标准版、专业版和旗舰版，以满足不同客户的需求。如果你觉得现有计费方案无法满足自己的特定需求，请联系 TDengine Cloud 的客户支持团队，他们将为你量身定制计费方案。注册后，你将获得一定的免费额度，以便体验服务

4
docs/zh/08-operation/03-deployment.md
View File

@ -206,11 +206,11 @@ http {
### 部署 taosX
如果想使用 TDengine 的数据接入能力，需要部署 taosX 服务，关于它的详细说明和部署请参考[taosX 参考手册](../../reference/components/taosx)。
如果想使用 TDengine 的数据接入能力，需要部署 taosX 服务，关于它的详细说明和部署请参考企业版参考手册。
### 部署 taosX-Agent
有些数据源如 Pi, OPC 等，因为网络条件和数据源访问的限制，taosX 无法直接访问数据源，这种情况下需要部署一个代理服务 taosX-Agent，关于它的详细说明和部署请参考[taosX-Agent 参考手册](../../reference/components/taosx-agent)。
有些数据源如 Pi, OPC 等，因为网络条件和数据源访问的限制，taosX 无法直接访问数据源，这种情况下需要部署一个代理服务 taosX-Agent，关于它的详细说明和部署请参考企业版参考手册。
### 部署 taos-Explorer

2
docs/zh/08-operation/12-multi.md
View File

@ -70,7 +70,7 @@ dataDir /mnt/data6 2 0
|参数名称 | 参数含义 |
|:-------------|:-----------------------------------------------|
|s3EndPoint | 用户所在地域的 COS 服务域名，支持 http 和 https，bucket 的区域需要与 endpoint 的保持一致，否则无法访问。例如：http://cos.ap-beijing.myqcloud.com |
|s3EndPoint | 用户所在地域的 COS 服务域名，支持 http 和 https，bucket 的区域需要与 endpoint 的保持一致，否则无法访问。 |
|s3AccessKey |冒号分隔的用户 SecretId:SecretKey。例如：AKIDsQmwsfKxTo2A6nGVXZN0UlofKn6JRRSJ:lIdoy99ygEacU7iHfogaN2Xq0yumSm1E |
|s3BucketName | 存储桶名称，减号后面是用户注册 COS 服务的 AppId。其中 AppId 是 COS 特有，AWS 和阿里云都没有，配置时需要作为 bucket name 的一部分，使用减号分隔。参数值均为字符串类型，但不需要引号。例如：test0711-1309024725 |
|s3UploadDelaySec | data 文件持续多长时间不再变动后上传至 s3，单位：秒。最小值：1；最大值：2592000 (30天），默认值 60 秒 |

2
docs/zh/08-operation/18-dual.md
View File

@ -83,7 +83,7 @@ taosx replica start
```shell
taosx replica start -f td1:6030 -t td2:6030
```
该示例命令会自动创建除 information_schema、performance_schema、log、audit 库之外的同步任务。可以使用 http://td2:6041 指定该 endpoint 使用 websocket 接口（默认是原生接口）。也可以指定数据库同步：taosx replica start -f td1:6030 -t td2:6030 db1 仅创建指定的数据库同步任务。
该示例命令会自动创建除 information_schema、performance_schema、log、audit 库之外的同步任务。可以使用 `http://td2:6041` 指定该 endpoint 使用 websocket 接口（默认是原生接口）。也可以指定数据库同步：taosx replica start -f td1:6030 -t td2:6030 db1 仅创建指定的数据库同步任务。
2. 方法二

2
docs/zh/14-reference/03-taos-sql/10-function.md
View File

@ -1458,7 +1458,7 @@ WEEKDAY(expr)
**适用于**: 表和超级表。
**使用说明**：
- 返回值 0 代表周日，1 代表周一 ... 6 代表周六。
- 返回值 0 代表周一，1 代表周二 ... 6 代表周日。
- 若 `expr` 为 NULL，返回 NULL。
- 输入时间戳的精度由所查询表的精度确定, 若未指定表, 则精度为毫秒.

2
docs/zh/14-reference/03-taos-sql/14-stream.md
View File

@ -99,7 +99,7 @@ PARTITION 子句中，为 tbname 定义了一个别名 tname, 在PARTITION 子
## 流式计算读取历史数据
正常情况下，流式计算不会处理创建前已经写入源表中的数据，若要处理已经写入的数据，可以在创建流时设置 fill_history 1 选项，这样创建的流式计算会自动处理创建前、创建中、创建后写入的数据。例如：
正常情况下，流式计算不会处理创建前已经写入源表中的数据，若要处理已经写入的数据，可以在创建流时设置 fill_history 1 选项，这样创建的流式计算会自动处理创建前、创建中、创建后写入的数据。流计算处理历史数据的最大窗口数是2000万，超过限制会报错。例如：
```sql
create stream if not exists s1 fill_history 1 into st1 as select count(*) from t1 interval(10s)

11
docs/zh/14-reference/03-taos-sql/24-show.md
View File

@ -30,6 +30,16 @@ SHOW CLUSTER ALIVE;
查询当前集群的状态是否可用，返回值： 0：不可用 1：完全可用 2：部分可用（集群中部分节点下线，但其它节点仍可以正常使用）
## SHOW CLUSTER MACHINES
```sql
SHOW CLUSTER MACHINES; // 从 TDengine 3.2.3.0 版本开始支持
```
显示集群的机器码等信息。
注：企业版独有
## SHOW CONNECTIONS
```sql
@ -99,6 +109,7 @@ SHOW FUNCTIONS;
```sql
SHOW LICENCES;
SHOW GRANTS;
SHOW GRANTS FULL; // 从 TDengine 3.2.3.0 版本开始支持
```
显示企业版许可授权的信息。

89
docs/zh/28-releases/03-notes/3.3.3.0.md Normal file
View File

@ -0,0 +1,89 @@
---
title: 3.3.3.0 版本说明
sidebar_label: 3.3.3.0
description: 3.3.3.0 版本说明
---
### 新特性/优化
1. 增加函数支持：pi、truncate/tunc、exp、ln、mod、rand、sign、degress、radians、char、ascii、position、trim、replace、repeat、substring、substr、substring_index、week、weekday、weekofyear、dayofweek、stddev_pop、var_pop
2. 多级存储支持微软对象存储 Azure Blob [企业版]
3. 支持 MongoDB 数据源 [企业版]
4. TDengine支持macOS企业版客户端 [企业版]
5. taosX日志默认不写入syslog [企业版]
6. 服务端记录所有慢查询信息到log库
7. show cluster machines 查询结果中添加服务端版本号
8. 删除保留关键字LEVEL/ENCODE/COMPRESS, 可以作为列名/表名/数据库名等使用
9. 禁止动态修改临时目录
10. round 函数：支持四舍五入的精度
11. timediff 函数：返回时间戳 expr1 - expr2 的结果，支持负数结果，并近似到时间单位 time_unit 指定的精度
12. max/min 函数：支持字符串作为输入参数，当输入参数为字符串类型时，返回最大字符串值
13. 提升了 taosX 数据同步的性能 [企业版]
14. 统一慢查询和普通日志的保存策略
15. 优化订阅功能中offset文件过大且加载过慢的问题
16. 升级 JDBC driver 至 3.2.5
17. Kafka connector 提交改为同步方式
18. 新增参数：表查询不存在时是否报异常
19. 扩大 tsma 的 interval 的最大窗口到 1 年
20. 支持从 CSV 文件批量建表
### 修复问题
1. 服务器重启后 mqtt 写入任务未自动重新启动 [企业版]
2. 修复windows上agent和PI connector连接错误导致任务终止的问题 [企业版]
3. 重启taosadapter后,mqtt 任务异常，无法同步数据 [企业版]
4. Explorer 上 mqtt 写入任务状态不正确 [企业版]
5. 重启 taosx 后，运行中的任务没有被重新调度 [企业版]
6. 对于 taosx 的 MongoDB DataIn 任务，优化了日志的输出信息 [企业版]
7. 修复在 local.toml 文件为空时备份失败的问题 [企业版]
8. taosx replica 任务在网络断开恢复后数据同步异常的问题 [企业版]
9. 修复3.1版本同步到3.3版本meta数据不同步问题 [企业版]
10. taosadapter 异常重启后, mqtt任务无法自动恢复的问题[企业版]
11. Kafka DataIn 任务状态正常，但任务停止消费数据 [企业版]
12. 修复 last + interval 查询导致 crash 的问题
13. 在数据写入时热更stt_trigger导致taosd崩溃
14. 修改root密码后taoskeeper反复重启
15. 云服务中服务重启后流计算无法继续
16. tsdb加载缓存导致taosd死锁
17. 查询时fill值指定为0报错
18. last_row 查询结果与预期不符
19. 用户指定 information_schema 库登录时导致 taosd 崩溃
20. group by 语句支持位置参数和别名语法
21. 元数据克隆函数对象错误导致 crash
22. 缓存更新时数据填充的游标错误导致的 taosd 异常退出
23. STDDEV 函数计算结果随机不正确的问题
24. 多级存储以及加密场景下无法添加离线节点
25. taos CLI 无法输入大于20字节长度的密码
26. 拼接 sql 写入报错: int data overflow
27. 大量查询并发场景下元数据的一致性
28. 尝试解决手动点击停止按钮，任务无法停止的问题
29. 解决了列或者标签中包含逗号或引号时导出 CSV 文件错误的问题
30. 多线程并发的对同一张子表进行 describe 导致 crash
31. 数据浏览器的查询结果未按照配置的时区展示的问题
32. 创建 websocket 连接时 taosadapter 内存泄漏的问题
33. 社区版无法在 redhat 操作系统正常启动
34. 无效断言在两个 stt 分别位于数据块时间轴分布空洞场景引发的 crash
35. S3 长时间拉取数据会失败的问题
36. 使用结果集函数判断时间线的有序性
37. 解决超级表投影查询慢的问题
38. 修复: 多节点环境，事务返回码与事务状态不一致导致事务堆积
39. 修复：在stt_trigger = 1时，delete数据之后重新出现的错误
40. 节点恢复阶段taosd内存占用过高导致OOM
41. limit过小时的判断错误
42. AVEVA historian 数据源连通性及获取示例数据功能
43. taosd 服务无法正常停止的问题
44. last 函数查询全空列导致 crash
45. 查询 cid 不在当前版本的 schema 时触发 assert 导致 coredump
46. 写入消息体过大时 OOM 的问题，增加 syncLogBufferMemoryAllowed 参数控制
47. timezone 设置为 UTC 且数据包含“1970”时,查询报错
48. 取消mnode节点的消息同步的等待超时
49. 集群节点异常恢复后，集群授权状态失效
50. 查询空指针问题
51. 在restore vnode时，将vgroup暂时设置成单副本
52. 可以使用SQL保留字作为列名创建超级表，也可以使用SQL保留字作为标签名
53. 高并发查询期间内存持续增长的问题
54. 流计算在checkpoint处理过程中，由于断言失败导致的crash
55. 修复了 3.3.2.8.0820 版本中可以查询到 information_schema 中不存在的表perf_queries问题
56. 节点恢复阶段taosd内存占用过高导致OOM
57. 修复在schema变更情况下，通过taosx导入数据时，taosd出现crash的问题
58. 可能的内存泄漏
59. 客户端内存泄漏
60. 开源用户修改stt_trigger值升级后无法修改其他数据库选项
61. NOT IN (NULL) 查询结果不正确
62. taos shell和taosBenchmark不能成功连接云服务实例

2
docs/zh/28-releases/03-notes/index.md
View File

@ -3,5 +3,5 @@ title: 版本说明
sidebar_label: 版本说明
description: 各版本版本说明
---
[3.3.3.0](./3.3.3.0)
[3.3.2.0](./3.3.2.0)

2
include/common/ttime.h
View File

@ -64,7 +64,7 @@ static FORCE_INLINE int64_t taosGetTimestampToday(int32_t precision) {
: 1000000000;
time_t t = taosTime(NULL);
struct tm tm;
(void) taosLocalTime(&t, &tm, NULL);
(void) taosLocalTime(&t, &tm, NULL, 0);
tm.tm_hour = 0;
tm.tm_min = 0;
tm.tm_sec = 0;

664
include/common/ttokendef.h
View File

@ -69,335 +69,341 @@
#define TK_NK_DOT 51
#define TK_WITH 52
#define TK_ENCRYPT_KEY 53
#define TK_DNODE 54
#define TK_PORT 55
#define TK_DNODES 56
#define TK_RESTORE 57
#define TK_NK_IPTOKEN 58
#define TK_FORCE 59
#define TK_UNSAFE 60
#define TK_CLUSTER 61
#define TK_LOCAL 62
#define TK_QNODE 63
#define TK_BNODE 64
#define TK_SNODE 65
#define TK_MNODE 66
#define TK_VNODE 67
#define TK_DATABASE 68
#define TK_USE 69
#define TK_FLUSH 70
#define TK_TRIM 71
#define TK_S3MIGRATE 72
#define TK_COMPACT 73
#define TK_IF 74
#define TK_NOT 75
#define TK_EXISTS 76
#define TK_BUFFER 77
#define TK_CACHEMODEL 78
#define TK_CACHESIZE 79
#define TK_COMP 80
#define TK_DURATION 81
#define TK_NK_VARIABLE 82
#define TK_MAXROWS 83
#define TK_MINROWS 84
#define TK_KEEP 85
#define TK_PAGES 86
#define TK_PAGESIZE 87
#define TK_TSDB_PAGESIZE 88
#define TK_PRECISION 89
#define TK_REPLICA 90
#define TK_VGROUPS 91
#define TK_SINGLE_STABLE 92
#define TK_RETENTIONS 93
#define TK_SCHEMALESS 94
#define TK_WAL_LEVEL 95
#define TK_WAL_FSYNC_PERIOD 96
#define TK_WAL_RETENTION_PERIOD 97
#define TK_WAL_RETENTION_SIZE 98
#define TK_WAL_ROLL_PERIOD 99
#define TK_WAL_SEGMENT_SIZE 100
#define TK_STT_TRIGGER 101
#define TK_TABLE_PREFIX 102
#define TK_TABLE_SUFFIX 103
#define TK_S3_CHUNKSIZE 104
#define TK_S3_KEEPLOCAL 105
#define TK_S3_COMPACT 106
#define TK_KEEP_TIME_OFFSET 107
#define TK_ENCRYPT_ALGORITHM 108
#define TK_NK_COLON 109
#define TK_BWLIMIT 110
#define TK_START 111
#define TK_TIMESTAMP 112
#define TK_END 113
#define TK_TABLE 114
#define TK_NK_LP 115
#define TK_NK_RP 116
#define TK_USING 117
#define TK_FILE 118
#define TK_STABLE 119
#define TK_COLUMN 120
#define TK_MODIFY 121
#define TK_RENAME 122
#define TK_TAG 123
#define TK_SET 124
#define TK_NK_EQ 125
#define TK_TAGS 126
#define TK_BOOL 127
#define TK_TINYINT 128
#define TK_SMALLINT 129
#define TK_INT 130
#define TK_INTEGER 131
#define TK_BIGINT 132
#define TK_FLOAT 133
#define TK_DOUBLE 134
#define TK_BINARY 135
#define TK_NCHAR 136
#define TK_UNSIGNED 137
#define TK_JSON 138
#define TK_VARCHAR 139
#define TK_MEDIUMBLOB 140
#define TK_BLOB 141
#define TK_VARBINARY 142
#define TK_GEOMETRY 143
#define TK_DECIMAL 144
#define TK_COMMENT 145
#define TK_MAX_DELAY 146
#define TK_WATERMARK 147
#define TK_ROLLUP 148
#define TK_TTL 149
#define TK_SMA 150
#define TK_DELETE_MARK 151
#define TK_FIRST 152
#define TK_LAST 153
#define TK_SHOW 154
#define TK_FULL 155
#define TK_PRIVILEGES 156
#define TK_DATABASES 157
#define TK_TABLES 158
#define TK_STABLES 159
#define TK_MNODES 160
#define TK_QNODES 161
#define TK_ARBGROUPS 162
#define TK_FUNCTIONS 163
#define TK_INDEXES 164
#define TK_ACCOUNTS 165
#define TK_APPS 166
#define TK_CONNECTIONS 167
#define TK_LICENCES 168
#define TK_GRANTS 169
#define TK_LOGS 170
#define TK_MACHINES 171
#define TK_ENCRYPTIONS 172
#define TK_QUERIES 173
#define TK_SCORES 174
#define TK_TOPICS 175
#define TK_VARIABLES 176
#define TK_BNODES 177
#define TK_SNODES 178
#define TK_TRANSACTIONS 179
#define TK_DISTRIBUTED 180
#define TK_CONSUMERS 181
#define TK_SUBSCRIPTIONS 182
#define TK_VNODES 183
#define TK_ALIVE 184
#define TK_VIEWS 185
#define TK_VIEW 186
#define TK_COMPACTS 187
#define TK_NORMAL 188
#define TK_CHILD 189
#define TK_LIKE 190
#define TK_TBNAME 191
#define TK_QTAGS 192
#define TK_AS 193
#define TK_SYSTEM 194
#define TK_TSMA 195
#define TK_INTERVAL 196
#define TK_RECURSIVE 197
#define TK_TSMAS 198
#define TK_FUNCTION 199
#define TK_INDEX 200
#define TK_COUNT 201
#define TK_LAST_ROW 202
#define TK_META 203
#define TK_ONLY 204
#define TK_TOPIC 205
#define TK_CONSUMER 206
#define TK_GROUP 207
#define TK_DESC 208
#define TK_DESCRIBE 209
#define TK_RESET 210
#define TK_QUERY 211
#define TK_CACHE 212
#define TK_EXPLAIN 213
#define TK_ANALYZE 214
#define TK_VERBOSE 215
#define TK_NK_BOOL 216
#define TK_RATIO 217
#define TK_NK_FLOAT 218
#define TK_OUTPUTTYPE 219
#define TK_AGGREGATE 220
#define TK_BUFSIZE 221
#define TK_LANGUAGE 222
#define TK_REPLACE 223
#define TK_STREAM 224
#define TK_INTO 225
#define TK_PAUSE 226
#define TK_RESUME 227
#define TK_PRIMARY 228
#define TK_KEY 229
#define TK_TRIGGER 230
#define TK_AT_ONCE 231
#define TK_WINDOW_CLOSE 232
#define TK_IGNORE 233
#define TK_EXPIRED 234
#define TK_FILL_HISTORY 235
#define TK_UPDATE 236
#define TK_SUBTABLE 237
#define TK_UNTREATED 238
#define TK_KILL 239
#define TK_CONNECTION 240
#define TK_TRANSACTION 241
#define TK_BALANCE 242
#define TK_VGROUP 243
#define TK_LEADER 244
#define TK_MERGE 245
#define TK_REDISTRIBUTE 246
#define TK_SPLIT 247
#define TK_DELETE 248
#define TK_INSERT 249
#define TK_NK_BIN 250
#define TK_NK_HEX 251
#define TK_NULL 252
#define TK_NK_QUESTION 253
#define TK_NK_ALIAS 254
#define TK_NK_ARROW 255
#define TK_ROWTS 256
#define TK_QSTART 257
#define TK_QEND 258
#define TK_QDURATION 259
#define TK_WSTART 260
#define TK_WEND 261
#define TK_WDURATION 262
#define TK_IROWTS 263
#define TK_ISFILLED 264
#define TK_CAST 265
#define TK_POSITION 266
#define TK_IN 267
#define TK_FOR 268
#define TK_NOW 269
#define TK_TODAY 270
#define TK_RAND 271
#define TK_SUBSTR 272
#define TK_SUBSTRING 273
#define TK_BOTH 274
#define TK_TRAILING 275
#define TK_LEADING 276
#define TK_TIMEZONE 277
#define TK_CLIENT_VERSION 278
#define TK_SERVER_VERSION 279
#define TK_SERVER_STATUS 280
#define TK_CURRENT_USER 281
#define TK_PI 282
#define TK_CASE 283
#define TK_WHEN 284
#define TK_THEN 285
#define TK_ELSE 286
#define TK_BETWEEN 287
#define TK_IS 288
#define TK_NK_LT 289
#define TK_NK_GT 290
#define TK_NK_LE 291
#define TK_NK_GE 292
#define TK_NK_NE 293
#define TK_MATCH 294
#define TK_NMATCH 295
#define TK_CONTAINS 296
#define TK_JOIN 297
#define TK_INNER 298
#define TK_LEFT 299
#define TK_RIGHT 300
#define TK_OUTER 301
#define TK_SEMI 302
#define TK_ANTI 303
#define TK_ASOF 304
#define TK_WINDOW 305
#define TK_WINDOW_OFFSET 306
#define TK_JLIMIT 307
#define TK_SELECT 308
#define TK_NK_HINT 309
#define TK_DISTINCT 310
#define TK_WHERE 311
#define TK_PARTITION 312
#define TK_BY 313
#define TK_SESSION 314
#define TK_STATE_WINDOW 315
#define TK_EVENT_WINDOW 316
#define TK_COUNT_WINDOW 317
#define TK_SLIDING 318
#define TK_FILL 319
#define TK_VALUE 320
#define TK_VALUE_F 321
#define TK_NONE 322
#define TK_PREV 323
#define TK_NULL_F 324
#define TK_LINEAR 325
#define TK_NEXT 326
#define TK_HAVING 327
#define TK_RANGE 328
#define TK_EVERY 329
#define TK_ORDER 330
#define TK_SLIMIT 331
#define TK_SOFFSET 332
#define TK_LIMIT 333
#define TK_OFFSET 334
#define TK_ASC 335
#define TK_NULLS 336
#define TK_ABORT 337
#define TK_AFTER 338
#define TK_ATTACH 339
#define TK_BEFORE 340
#define TK_BEGIN 341
#define TK_BITAND 342
#define TK_BITNOT 343
#define TK_BITOR 344
#define TK_BLOCKS 345
#define TK_CHANGE 346
#define TK_COMMA 347
#define TK_CONCAT 348
#define TK_CONFLICT 349
#define TK_COPY 350
#define TK_DEFERRED 351
#define TK_DELIMITERS 352
#define TK_DETACH 353
#define TK_DIVIDE 354
#define TK_DOT 355
#define TK_EACH 356
#define TK_FAIL 357
#define TK_GLOB 358
#define TK_ID 359
#define TK_IMMEDIATE 360
#define TK_IMPORT 361
#define TK_INITIALLY 362
#define TK_INSTEAD 363
#define TK_ISNULL 364
#define TK_MODULES 365
#define TK_NK_BITNOT 366
#define TK_NK_SEMI 367
#define TK_NOTNULL 368
#define TK_OF 369
#define TK_PLUS 370
#define TK_PRIVILEGE 371
#define TK_RAISE 372
#define TK_RESTRICT 373
#define TK_ROW 374
#define TK_STAR 375
#define TK_STATEMENT 376
#define TK_STRICT 377
#define TK_STRING 378
#define TK_TIMES 379
#define TK_VALUES 380
#define TK_VARIABLE 381
#define TK_WAL 382
#define TK_ANODE 54
#define TK_UPDATE 55
#define TK_ANODES 56
#define TK_DNODE 57
#define TK_PORT 58
#define TK_DNODES 59
#define TK_RESTORE 60
#define TK_NK_IPTOKEN 61
#define TK_FORCE 62
#define TK_UNSAFE 63
#define TK_CLUSTER 64
#define TK_LOCAL 65
#define TK_QNODE 66
#define TK_BNODE 67
#define TK_SNODE 68
#define TK_MNODE 69
#define TK_VNODE 70
#define TK_DATABASE 71
#define TK_USE 72
#define TK_FLUSH 73
#define TK_TRIM 74
#define TK_S3MIGRATE 75
#define TK_COMPACT 76
#define TK_IF 77
#define TK_NOT 78
#define TK_EXISTS 79
#define TK_BUFFER 80
#define TK_CACHEMODEL 81
#define TK_CACHESIZE 82
#define TK_COMP 83
#define TK_DURATION 84
#define TK_NK_VARIABLE 85
#define TK_MAXROWS 86
#define TK_MINROWS 87
#define TK_KEEP 88
#define TK_PAGES 89
#define TK_PAGESIZE 90
#define TK_TSDB_PAGESIZE 91
#define TK_PRECISION 92
#define TK_REPLICA 93
#define TK_VGROUPS 94
#define TK_SINGLE_STABLE 95
#define TK_RETENTIONS 96
#define TK_SCHEMALESS 97
#define TK_WAL_LEVEL 98
#define TK_WAL_FSYNC_PERIOD 99
#define TK_WAL_RETENTION_PERIOD 100
#define TK_WAL_RETENTION_SIZE 101
#define TK_WAL_ROLL_PERIOD 102
#define TK_WAL_SEGMENT_SIZE 103
#define TK_STT_TRIGGER 104
#define TK_TABLE_PREFIX 105
#define TK_TABLE_SUFFIX 106
#define TK_S3_CHUNKSIZE 107
#define TK_S3_KEEPLOCAL 108
#define TK_S3_COMPACT 109
#define TK_KEEP_TIME_OFFSET 110
#define TK_ENCRYPT_ALGORITHM 111
#define TK_NK_COLON 112
#define TK_BWLIMIT 113
#define TK_START 114
#define TK_TIMESTAMP 115
#define TK_END 116
#define TK_TABLE 117
#define TK_NK_LP 118
#define TK_NK_RP 119
#define TK_USING 120
#define TK_FILE 121
#define TK_STABLE 122
#define TK_COLUMN 123
#define TK_MODIFY 124
#define TK_RENAME 125
#define TK_TAG 126
#define TK_SET 127
#define TK_NK_EQ 128
#define TK_TAGS 129
#define TK_BOOL 130
#define TK_TINYINT 131
#define TK_SMALLINT 132
#define TK_INT 133
#define TK_INTEGER 134
#define TK_BIGINT 135
#define TK_FLOAT 136
#define TK_DOUBLE 137
#define TK_BINARY 138
#define TK_NCHAR 139
#define TK_UNSIGNED 140
#define TK_JSON 141
#define TK_VARCHAR 142
#define TK_MEDIUMBLOB 143
#define TK_BLOB 144
#define TK_VARBINARY 145
#define TK_GEOMETRY 146
#define TK_DECIMAL 147
#define TK_COMMENT 148
#define TK_MAX_DELAY 149
#define TK_WATERMARK 150
#define TK_ROLLUP 151
#define TK_TTL 152
#define TK_SMA 153
#define TK_DELETE_MARK 154
#define TK_FIRST 155
#define TK_LAST 156
#define TK_SHOW 157
#define TK_FULL 158
#define TK_PRIVILEGES 159
#define TK_DATABASES 160
#define TK_TABLES 161
#define TK_STABLES 162
#define TK_MNODES 163
#define TK_QNODES 164
#define TK_ARBGROUPS 165
#define TK_FUNCTIONS 166
#define TK_INDEXES 167
#define TK_ACCOUNTS 168
#define TK_APPS 169
#define TK_CONNECTIONS 170
#define TK_LICENCES 171
#define TK_GRANTS 172
#define TK_LOGS 173
#define TK_MACHINES 174
#define TK_ENCRYPTIONS 175
#define TK_QUERIES 176
#define TK_SCORES 177
#define TK_TOPICS 178
#define TK_VARIABLES 179
#define TK_BNODES 180
#define TK_SNODES 181
#define TK_TRANSACTIONS 182
#define TK_DISTRIBUTED 183
#define TK_CONSUMERS 184
#define TK_SUBSCRIPTIONS 185
#define TK_VNODES 186
#define TK_ALIVE 187
#define TK_VIEWS 188
#define TK_VIEW 189
#define TK_COMPACTS 190
#define TK_NORMAL 191
#define TK_CHILD 192
#define TK_LIKE 193
#define TK_TBNAME 194
#define TK_QTAGS 195
#define TK_AS 196
#define TK_SYSTEM 197
#define TK_TSMA 198
#define TK_INTERVAL 199
#define TK_RECURSIVE 200
#define TK_TSMAS 201
#define TK_FUNCTION 202
#define TK_INDEX 203
#define TK_COUNT 204
#define TK_LAST_ROW 205
#define TK_META 206
#define TK_ONLY 207
#define TK_TOPIC 208
#define TK_CONSUMER 209
#define TK_GROUP 210
#define TK_DESC 211
#define TK_DESCRIBE 212
#define TK_RESET 213
#define TK_QUERY 214
#define TK_CACHE 215
#define TK_EXPLAIN 216
#define TK_ANALYZE 217
#define TK_VERBOSE 218
#define TK_NK_BOOL 219
#define TK_RATIO 220
#define TK_NK_FLOAT 221
#define TK_OUTPUTTYPE 222
#define TK_AGGREGATE 223
#define TK_BUFSIZE 224
#define TK_LANGUAGE 225
#define TK_REPLACE 226
#define TK_STREAM 227
#define TK_INTO 228
#define TK_PAUSE 229
#define TK_RESUME 230
#define TK_PRIMARY 231
#define TK_KEY 232
#define TK_TRIGGER 233
#define TK_AT_ONCE 234
#define TK_WINDOW_CLOSE 235
#define TK_IGNORE 236
#define TK_EXPIRED 237
#define TK_FILL_HISTORY 238
#define TK_SUBTABLE 239
#define TK_UNTREATED 240
#define TK_KILL 241
#define TK_CONNECTION 242
#define TK_TRANSACTION 243
#define TK_BALANCE 244
#define TK_VGROUP 245
#define TK_LEADER 246
#define TK_MERGE 247
#define TK_REDISTRIBUTE 248
#define TK_SPLIT 249
#define TK_DELETE 250
#define TK_INSERT 251
#define TK_NK_BIN 252
#define TK_NK_HEX 253
#define TK_NULL 254
#define TK_NK_QUESTION 255
#define TK_NK_ALIAS 256
#define TK_NK_ARROW 257
#define TK_ROWTS 258
#define TK_QSTART 259
#define TK_QEND 260
#define TK_QDURATION 261
#define TK_WSTART 262
#define TK_WEND 263
#define TK_WDURATION 264
#define TK_IROWTS 265
#define TK_ISFILLED 266
#define TK_FLOW 267
#define TK_FHIGH 268
#define TK_FROWTS 269
#define TK_CAST 270
#define TK_POSITION 271
#define TK_IN 272
#define TK_FOR 273
#define TK_NOW 274
#define TK_TODAY 275
#define TK_RAND 276
#define TK_SUBSTR 277
#define TK_SUBSTRING 278
#define TK_BOTH 279
#define TK_TRAILING 280
#define TK_LEADING 281
#define TK_TIMEZONE 282
#define TK_CLIENT_VERSION 283
#define TK_SERVER_VERSION 284
#define TK_SERVER_STATUS 285
#define TK_CURRENT_USER 286
#define TK_PI 287
#define TK_CASE 288
#define TK_WHEN 289
#define TK_THEN 290
#define TK_ELSE 291
#define TK_BETWEEN 292
#define TK_IS 293
#define TK_NK_LT 294
#define TK_NK_GT 295
#define TK_NK_LE 296
#define TK_NK_GE 297
#define TK_NK_NE 298
#define TK_MATCH 299
#define TK_NMATCH 300
#define TK_CONTAINS 301
#define TK_JOIN 302
#define TK_INNER 303
#define TK_LEFT 304
#define TK_RIGHT 305
#define TK_OUTER 306
#define TK_SEMI 307
#define TK_ANTI 308
#define TK_ASOF 309
#define TK_WINDOW 310
#define TK_WINDOW_OFFSET 311
#define TK_JLIMIT 312
#define TK_SELECT 313
#define TK_NK_HINT 314
#define TK_DISTINCT 315
#define TK_WHERE 316
#define TK_PARTITION 317
#define TK_BY 318
#define TK_SESSION 319
#define TK_STATE_WINDOW 320
#define TK_EVENT_WINDOW 321
#define TK_COUNT_WINDOW 322
#define TK_ANOMALY_WINDOW 323
#define TK_SLIDING 324
#define TK_FILL 325
#define TK_VALUE 326
#define TK_VALUE_F 327
#define TK_NONE 328
#define TK_PREV 329
#define TK_NULL_F 330
#define TK_LINEAR 331
#define TK_NEXT 332
#define TK_HAVING 333
#define TK_RANGE 334
#define TK_EVERY 335
#define TK_ORDER 336
#define TK_SLIMIT 337
#define TK_SOFFSET 338
#define TK_LIMIT 339
#define TK_OFFSET 340
#define TK_ASC 341
#define TK_NULLS 342
#define TK_ABORT 343
#define TK_AFTER 344
#define TK_ATTACH 345
#define TK_BEFORE 346
#define TK_BEGIN 347
#define TK_BITAND 348
#define TK_BITNOT 349
#define TK_BITOR 350
#define TK_BLOCKS 351
#define TK_CHANGE 352
#define TK_COMMA 353
#define TK_CONCAT 354
#define TK_CONFLICT 355
#define TK_COPY 356
#define TK_DEFERRED 357
#define TK_DELIMITERS 358
#define TK_DETACH 359
#define TK_DIVIDE 360
#define TK_DOT 361
#define TK_EACH 362
#define TK_FAIL 363
#define TK_GLOB 364
#define TK_ID 365
#define TK_IMMEDIATE 366
#define TK_IMPORT 367
#define TK_INITIALLY 368
#define TK_INSTEAD 369
#define TK_ISNULL 370
#define TK_MODULES 371
#define TK_NK_BITNOT 372
#define TK_NK_SEMI 373
#define TK_NOTNULL 374
#define TK_OF 375
#define TK_PLUS 376
#define TK_PRIVILEGE 377
#define TK_RAISE 378
#define TK_RESTRICT 379
#define TK_ROW 380
#define TK_STAR 381
#define TK_STATEMENT 382
#define TK_STRICT 383
#define TK_STRING 384
#define TK_TIMES 385
#define TK_VALUES 386
#define TK_VARIABLE 387
#define TK_WAL 388
#define TK_NK_SPACE 600
#define TK_NK_COMMENT 601

2
include/libs/nodes/cmdnodes.h
View File

@ -320,7 +320,7 @@ typedef struct SAlterDnodeStmt {
typedef struct {
ENodeType type;
char url[TSDB_ANAL_ANODE_URL_LEN];
char url[TSDB_ANAL_ANODE_URL_LEN + 3];
} SCreateAnodeStmt;
typedef struct {

3
include/os/osDef.h
View File

@ -247,9 +247,12 @@ void syslog(int unused, const char *format, ...);
#define TD_DIRSEP_CHAR '/'
#endif
#define TD_FQDN_LEN 128
#define TD_LOCALE_LEN 64
#define TD_CHARSET_LEN 64
#define TD_TIMEZONE_LEN 96
#define TD_TIME_STR_LEN 128
#define TD_IP_LEN 64
#ifdef __cplusplus
}

2
include/os/osString.h
View File

@ -88,7 +88,7 @@ bool taosMbsToUcs4(const char *mbs, size_t mbs_len, TdUcs4 *ucs4, int32_t ucs
int32_t tasoUcs4Compare(TdUcs4 *f1_ucs4, TdUcs4 *f2_ucs4, int32_t bytes);
int32_t tasoUcs4Copy(TdUcs4 *target_ucs4, TdUcs4 *source_ucs4, int32_t len_ucs4);
bool taosValidateEncodec(const char *encodec);
int32_t taosHexEncode(const unsigned char *src, char *dst, int32_t len);
int32_t taosHexEncode(const unsigned char *src, char *dst, int32_t len, int32_t bufSize);
int32_t taosHexDecode(const char *src, char *dst, int32_t len);
int32_t taosWcharWidth(TdWchar wchar);

2
include/os/osTime.h
View File

@ -91,7 +91,7 @@ static FORCE_INLINE int64_t taosGetMonoTimestampMs() {
}
char *taosStrpTime(const char *buf, const char *fmt, struct tm *tm);
struct tm *taosLocalTime(const time_t *timep, struct tm *result, char *buf);
struct tm *taosLocalTime(const time_t *timep, struct tm *result, char *buf, int32_t bufSize);
struct tm *taosLocalTimeNolock(struct tm *result, const time_t *timep, int dst);
time_t taosTime(time_t *t);
time_t taosMktime(struct tm *timep);

7
include/util/taoserror.h
View File

@ -482,9 +482,10 @@ int32_t taosGetErrSize();
#define TSDB_CODE_MND_ANODE_TOO_LONG_URL TAOS_DEF_ERROR_CODE(0, 0x0432)
#define TSDB_CODE_MND_ANODE_INVALID_PROTOCOL TAOS_DEF_ERROR_CODE(0, 0x0433)
#define TSDB_CODE_MND_ANODE_INVALID_VERSION TAOS_DEF_ERROR_CODE(0, 0x0434)
#define TSDB_CODE_MND_ANODE_TOO_MANY_ALGO TAOS_DEF_ERROR_CODE(0, 0x0435)
#define TSDB_CODE_MND_ANODE_TOO_LONG_ALGO_NAME TAOS_DEF_ERROR_CODE(0, 0x0436)
#define TSDB_CODE_MND_ANODE_TOO_MANY_ALGO_TYPE TAOS_DEF_ERROR_CODE(0, 0x0437)
#define TSDB_CODE_MND_ANODE_INVALID_ALGO_TYPE TAOS_DEF_ERROR_CODE(0, 0x0435)
#define TSDB_CODE_MND_ANODE_TOO_MANY_ALGO TAOS_DEF_ERROR_CODE(0, 0x0436)
#define TSDB_CODE_MND_ANODE_TOO_LONG_ALGO_NAME TAOS_DEF_ERROR_CODE(0, 0x0437)
#define TSDB_CODE_MND_ANODE_TOO_MANY_ALGO_TYPE TAOS_DEF_ERROR_CODE(0, 0x0438)
// analysis
#define TSDB_CODE_ANAL_URL_RSP_IS_NULL TAOS_DEF_ERROR_CODE(0, 0x0440)

2
include/util/tdef.h
View File

@ -284,7 +284,7 @@ typedef enum ELogicConditionType {
#define TSDB_CLUSTER_ID_LEN 40
#define TSDB_MACHINE_ID_LEN 24
#define TSDB_FQDN_LEN 128
#define TSDB_FQDN_LEN TD_FQDN_LEN
#define TSDB_EP_LEN (TSDB_FQDN_LEN + 6)
#define TSDB_IPv4ADDR_LEN 16
#define TSDB_FILENAME_LEN 128

1
include/util/tjson.h
View File

@ -71,6 +71,7 @@ int32_t tjsonGetObjectValueString(const SJson* pJson, char** pStringValue);
void tjsonGetObjectValueBigInt(const SJson* pJson, int64_t* pVal);
void tjsonGetObjectValueDouble(const SJson* pJson, double* pVal);
int32_t tjsonGetStringValue(const SJson* pJson, const char* pName, char* pVal);
int32_t tjsonGetStringValue2(const SJson* pJson, const char* pName, char* pVal, int32_t maxLen);
int32_t tjsonDupStringValue(const SJson* pJson, const char* pName, char** pVal);
int32_t tjsonGetBigIntValue(const SJson* pJson, const char* pName, int64_t* pVal);
int32_t tjsonGetIntValue(const SJson* pJson, const char* pName, int32_t* pVal);

33
packaging/tools/com.taosdata.taos-explorer.plist Normal file
View File

@ -0,0 +1,33 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.tdengine.taos-explorer</string>
<key>ProgramArguments</key>
<array>
<string>/usr/local/bin/taos-explorer</string>
</array>
<key>ProcessType</key>
<string>Interactive</string>
<key>Disabled</key>
<false/>
<key>RunAtLoad</key>
<false/>
<key>LaunchOnlyOnce</key>
<false/>
<key>SessionCreate</key>
<true/>
<key>ExitTimeOut</key>
<integer>600</integer>
<key>KeepAlive</key>
<dict>
<key>SuccessfulExit</key>
<false/>
<key>AfterInitialDemand</key>
<true/>
</dict>
<key>Program</key>
<string>/usr/local/bin/taos-explorer</string>
</dict>
</plist>

26
packaging/tools/remove.sh
View File

@ -206,10 +206,17 @@ function clean_log() {
}
function clean_service_on_launchctl() {
${csudouser}launchctl unload -w /Library/LaunchDaemons/com.taosdata.taosd.plist > /dev/null 2>&1 || :
${csudo}rm /Library/LaunchDaemons/com.taosdata.taosd.plist > /dev/null 2>&1 || :
${csudouser}launchctl unload -w /Library/LaunchDaemons/com.taosdata.${clientName2}adapter.plist > /dev/null 2>&1 || :
${csudo}rm /Library/LaunchDaemons/com.taosdata.${clientName2}adapter.plist > /dev/null 2>&1 || :
${csudo}launchctl unload -w /Library/LaunchDaemons/com.taosdata.taosd.plist || :
${csudo}launchctl unload -w /Library/LaunchDaemons/com.taosdata.${PREFIX}adapter.plist || :
${csudo}launchctl unload -w /Library/LaunchDaemons/com.taosdata.${PREFIX}keeper.plist || :
${csudo}launchctl unload -w /Library/LaunchDaemons/com.taosdata.${PREFIX}-explorer.plist || :
${csudo}launchctl remove com.tdengine.taosd || :
${csudo}launchctl remove com.tdengine.${PREFIX}adapter || :
${csudo}launchctl remove com.tdengine.${PREFIX}keeper || :
${csudo}launchctl remove com.tdengine.${PREFIX}-explorer || :
${csudo}rm /Library/LaunchDaemons/com.taosdata.* > /dev/null 2>&1 || :
}
function remove_data_and_config() {
@ -250,6 +257,12 @@ if [ -e ${install_main_dir}/uninstall_${PREFIX}x.sh ]; then
fi
fi
if [ "$osType" = "Darwin" ]; then
clean_service_on_launchctl
${csudo}rm -rf /Applications/TDengine.app
fi
remove_bin
clean_header
# Remove lib file
@ -282,10 +295,7 @@ elif echo $osinfo | grep -qwi "centos"; then
# echo "this is centos system"
${csudo}rpm -e --noscripts tdengine >/dev/null 2>&1 || :
fi
if [ "$osType" = "Darwin" ]; then
clean_service_on_launchctl
${csudo}rm -rf /Applications/TDengine.app
fi
command -v systemctl >/dev/null 2>&1 && ${csudo}systemctl daemon-reload >/dev/null 2>&1 || true
echo

4
packaging/tools/tdengine.iss
View File

@ -71,8 +71,8 @@ Source: {#MyAppSourceDir}\taosdump.exe; DestDir: "{app}"; DestName: "{#CusPrompt
Filename: {sys}\sc.exe; Parameters: "create taosd start= DEMAND binPath= ""C:\\TDengine\\taosd.exe --win_service""" ; Flags: runhidden
Filename: {sys}\sc.exe; Parameters: "create taosadapter start= DEMAND binPath= ""C:\\TDengine\\taosadapter.exe""" ; Flags: runhidden
Filename: "C:\Windows\System32\odbcconf.exe"; Parameters: "/S /F win_odbcinst.ini"; WorkingDir: "{app}\taos_odbc\x64"; Flags: runhidden; StatusMsg: "Configuring ODBC x64"
Filename: "C:\Windows\SysWOW64\odbcconf.exe"; Parameters: "/S /F win_odbcinst.ini"; WorkingDir: "{app}\taos_odbc\x86"; Flags: runhidden; StatusMsg: "Configuring ODBC x86"
Filename: "C:\Windows\System32\odbcconf.exe"; Parameters: "/S /F win_odbc_install.ini"; WorkingDir: "{app}\taos_odbc\x64"; Flags: runhidden; StatusMsg: "Configuring ODBC x64"
Filename: "C:\Windows\SysWOW64\odbcconf.exe"; Parameters: "/S /F win_odbc_install.ini"; WorkingDir: "{app}\taos_odbc\x86"; Flags: runhidden; StatusMsg: "Configuring ODBC x86"
[UninstallRun]
RunOnceId: "stoptaosd"; Filename: {sys}\sc.exe; Parameters: "stop taosd" ; Flags: runhidden

9
source/client/src/clientTmq.c
View File

@ -134,6 +134,7 @@ struct tmq_t {
// poll info
int64_t pollCnt;
int64_t totalRows;
int8_t pollFlag;
// timer
tmr_h hbLiveTimer;
@ -287,7 +288,6 @@ typedef struct {
static TdThreadOnce tmqInit = PTHREAD_ONCE_INIT; // initialize only once
volatile int32_t tmqInitRes = 0; // initialize rsp code
static SMqMgmt tmqMgmt = {0};
static int8_t pollFlag = 0;
tmq_conf_t* tmq_conf_new() {
tmq_conf_t* conf = taosMemoryCalloc(1, sizeof(tmq_conf_t));
@ -977,7 +977,7 @@ void tmqSendHbReq(void* param, void* tmrId) {
SMqHbReq req = {0};
req.consumerId = tmq->consumerId;
req.epoch = tmq->epoch;
req.pollFlag = atomic_load_8(&pollFlag);
req.pollFlag = atomic_load_8(&tmq->pollFlag);
req.topics = taosArrayInit(taosArrayGetSize(tmq->clientTopics), sizeof(TopicOffsetRows));
if (req.topics == NULL) {
goto END;
@ -1057,7 +1057,7 @@ void tmqSendHbReq(void* param, void* tmrId) {
if (code != 0) {
tqErrorC("tmqSendHbReq asyncSendMsgToServer failed");
}
(void)atomic_val_compare_exchange_8(&pollFlag, 1, 0);
(void)atomic_val_compare_exchange_8(&tmq->pollFlag, 1, 0);
END:
tDestroySMqHbReq(&req);
@ -1640,6 +1640,7 @@ tmq_t* tmq_consumer_new(tmq_conf_t* conf, char* errstr, int32_t errstrLen) {
pTmq->status = TMQ_CONSUMER_STATUS__INIT;
pTmq->pollCnt = 0;
pTmq->epoch = 0;
pTmq->pollFlag = 0;
// set conf
tstrncpy(pTmq->clientId, conf->clientId, TSDB_CLIENT_ID_LEN);
@ -2441,7 +2442,7 @@ TAOS_RES* tmq_consumer_poll(tmq_t* tmq, int64_t timeout) {
return NULL;
}
(void)atomic_val_compare_exchange_8(&pollFlag, 0, 1);
(void)atomic_val_compare_exchange_8(&tmq->pollFlag, 0, 1);
while (1) {
tmqHandleAllDelayedTask(tmq);

4
source/common/CMakeLists.txt
View File

@ -47,10 +47,6 @@ target_link_libraries(
INTERFACE api
)
if(${BUILD_WITH_ANALYSIS})
add_definitions(-DUSE_ANAL)
endif()
if(${BUILD_S3})
if(${BUILD_WITH_S3})

2
source/common/src/rsync.c
View File

@ -94,7 +94,7 @@ static int32_t generateConfigFile(char* confDir) {
#endif
);
uDebug("[rsync] conf:%s", confContent);
if (taosWriteFile(pFile, confContent, strlen(confContent)) != TSDB_CODE_SUCCESS) {
if (taosWriteFile(pFile, confContent, strlen(confContent)) <= 0) {
uError("[rsync] write conf file error," ERRNO_ERR_FORMAT, ERRNO_ERR_DATA);
(void)taosCloseFile(&pFile);
code = terrno;

17
source/common/src/systable.c
View File

@ -398,6 +398,21 @@ static const SSysDbTableSchema userCompactsDetailSchema[] = {
{.name = "finished", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false},
{.name = "start_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false},
};
static const SSysDbTableSchema anodesSchema[] = {
{.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false},
{.name = "url", .bytes = TSDB_ANAL_ANODE_URL_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true},
{.name = "status", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true},
{.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = true},
{.name = "update_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = true},
};
static const SSysDbTableSchema anodesFullSchema[] = {
{.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false},
{.name = "type", .bytes = TSDB_ANAL_ALGO_TYPE_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true},
{.name = "algo", .bytes = TSDB_ANAL_ALGO_NAME_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true},
};
static const SSysDbTableSchema tsmaSchema[] = {
{.name = "tsma_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false},
{.name = "db_name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false},
@ -472,6 +487,8 @@ static const SSysTableMeta infosMeta[] = {
{TSDB_INS_TABLE_ARBGROUPS, arbGroupsSchema, tListLen(arbGroupsSchema), true},
{TSDB_INS_TABLE_ENCRYPTIONS, encryptionsSchema, tListLen(encryptionsSchema), true},
{TSDB_INS_TABLE_TSMAS, tsmaSchema, tListLen(tsmaSchema), false},
{TSDB_INS_TABLE_ANODES, anodesSchema, tListLen(anodesSchema), true},
{TSDB_INS_TABLE_ANODES_FULL, anodesFullSchema, tListLen(anodesFullSchema), true},
};
static const SSysDbTableSchema connectionsSchema[] = {

15
source/common/src/tdatablock.c
View File

@ -2446,7 +2446,7 @@ _error:
return NULL;
}
static char* formatTimestamp(char* buf, int64_t val, int precision) {
static char* formatTimestamp(char* buf, int32_t bufSize, int64_t val, int precision) {
time_t tt;
int32_t ms = 0;
if (precision == TSDB_TIME_PRECISION_NANO) {
@ -2479,11 +2479,11 @@ static char* formatTimestamp(char* buf, int64_t val, int precision) {
}
}
struct tm ptm = {0};
if (taosLocalTime(&tt, &ptm, buf) == NULL) {
if (taosLocalTime(&tt, &ptm, buf, bufSize) == NULL) {
return buf;
}
size_t pos = strftime(buf, 35, "%Y-%m-%d %H:%M:%S", &ptm);
size_t pos = strftime(buf, bufSize, "%Y-%m-%d %H:%M:%S", &ptm);
if (precision == TSDB_TIME_PRECISION_NANO) {
sprintf(buf + pos, ".%09d", ms);
} else if (precision == TSDB_TIME_PRECISION_MICRO) {
@ -2497,10 +2497,11 @@ static char* formatTimestamp(char* buf, int64_t val, int precision) {
// for debug
int32_t dumpBlockData(SSDataBlock* pDataBlock, const char* flag, char** pDataBuf, const char* taskIdStr) {
int32_t lino = 0;
int32_t size = 2048 * 1024;
int32_t code = 0;
char* dumpBuf = NULL;
char pBuf[128] = {0};
char pBuf[TD_TIME_STR_LEN] = {0};
int32_t rows = pDataBlock->info.rows;
int32_t len = 0;
@ -2530,6 +2531,7 @@ int32_t dumpBlockData(SSDataBlock* pDataBlock, const char* flag, char** pDataBuf
SColumnInfoData* pColInfoData = taosArrayGet(pDataBlock->pDataBlock, k);
if (pColInfoData == NULL) {
code = terrno;
lino = __LINE__;
goto _exit;
}
@ -2543,7 +2545,7 @@ int32_t dumpBlockData(SSDataBlock* pDataBlock, const char* flag, char** pDataBuf
switch (pColInfoData->info.type) {
case TSDB_DATA_TYPE_TIMESTAMP:
memset(pBuf, 0, sizeof(pBuf));
(void)formatTimestamp(pBuf, *(uint64_t*)var, pColInfoData->info.precision);
(void)formatTimestamp(pBuf, sizeof(pBuf), *(uint64_t*)var, pColInfoData->info.precision);
len += tsnprintf(dumpBuf + len, size - len, " %25s |", pBuf);
if (len >= size - 1) goto _exit;
break;
@ -2609,6 +2611,7 @@ int32_t dumpBlockData(SSDataBlock* pDataBlock, const char* flag, char** pDataBuf
code = taosUcs4ToMbs((TdUcs4*)varDataVal(pData), dataSize, pBuf);
if (code < 0) {
uError("func %s failed to convert to ucs charset since %s", __func__, tstrerror(code));
lino = __LINE__;
goto _exit;
}
len += tsnprintf(dumpBuf + len, size - len, " %15s |", pBuf);
@ -2626,7 +2629,7 @@ _exit:
*pDataBuf = dumpBuf;
dumpBuf = NULL;
} else {
uError("%s failed at line %d since %s", __func__, __LINE__, tstrerror(code));
uError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
if (dumpBuf) {
taosMemoryFree(dumpBuf);
}

1
source/common/src/tglobal.c
View File

@ -1725,6 +1725,7 @@ int32_t taosReadDataFolder(const char *cfgDir, const char **envCmd, const char *
TAOS_CHECK_RETURN(cfgInit(&pCfg));
TAOS_CHECK_GOTO(cfgAddDir(pCfg, "dataDir", tsDataDir, CFG_SCOPE_SERVER, CFG_DYN_NONE), NULL, _exit);
TAOS_CHECK_GOTO(cfgAddInt32(pCfg, "debugFlag", dDebugFlag, 0, 255, CFG_SCOPE_SERVER, CFG_DYN_SERVER), NULL, _exit);
TAOS_CHECK_GOTO(cfgAddInt32(pCfg, "dDebugFlag", dDebugFlag, 0, 255, CFG_SCOPE_SERVER, CFG_DYN_SERVER) ,NULL, _exit);
if ((code = taosLoadCfg(pCfg, envCmd, cfgDir, envFile, apolloUrl)) != 0) {

264
source/common/src/tmsg.c
View File

@ -40,6 +40,7 @@
#define TD_MSG_RANGE_CODE_
#include "tmsgdef.h"
#include "tanal.h"
#include "tcol.h"
#include "tlog.h"
@ -1453,6 +1454,7 @@ int32_t tSerializeSStatusReq(void *buf, int32_t bufLen, SStatusReq *pReq) {
}
TAOS_CHECK_EXIT(tEncodeI64(&encoder, pReq->ipWhiteVer));
TAOS_CHECK_EXIT(tEncodeI64(&encoder, pReq->analVer));
TAOS_CHECK_EXIT(tSerializeSMonitorParas(&encoder, &pReq->clusterCfg.monitorParas));
tEndEncode(&encoder);
@ -1576,6 +1578,10 @@ int32_t tDeserializeSStatusReq(void *buf, int32_t bufLen, SStatusReq *pReq) {
TAOS_CHECK_EXIT(tDecodeI64(&decoder, &pReq->ipWhiteVer));
}
if (!tDecodeIsEnd(&decoder)) {
TAOS_CHECK_EXIT(tDecodeI64(&decoder, &pReq->analVer));
}
if (!tDecodeIsEnd(&decoder)) {
TAOS_CHECK_EXIT(tDeserializeSMonitorParas(&decoder, &pReq->clusterCfg.monitorParas));
}
@ -1652,6 +1658,7 @@ int32_t tSerializeSStatusRsp(void *buf, int32_t bufLen, SStatusRsp *pRsp) {
TAOS_CHECK_EXIT(tEncodeI32(&encoder, pRsp->statusSeq));
TAOS_CHECK_EXIT(tEncodeI64(&encoder, pRsp->ipWhiteVer));
TAOS_CHECK_EXIT(tEncodeI64(&encoder, pRsp->analVer));
tEndEncode(&encoder);
_exit:
@ -1703,6 +1710,11 @@ int32_t tDeserializeSStatusRsp(void *buf, int32_t bufLen, SStatusRsp *pRsp) {
if (!tDecodeIsEnd(&decoder)) {
TAOS_CHECK_EXIT(tDecodeI64(&decoder, &pRsp->ipWhiteVer));
}
if (!tDecodeIsEnd(&decoder)) {
TAOS_CHECK_EXIT(tDecodeI64(&decoder, &pRsp->analVer));
}
tEndDecode(&decoder);
_exit:
tDecoderClear(&decoder);
@ -2044,6 +2056,156 @@ _exit:
return code;
}
int32_t tSerializeRetrieveAnalAlgoReq(void *buf, int32_t bufLen, SRetrieveAnalAlgoReq *pReq) {
SEncoder encoder = {0};
int32_t code = 0;
int32_t lino;
int32_t tlen;
tEncoderInit(&encoder, buf, bufLen);
TAOS_CHECK_EXIT(tStartEncode(&encoder));
TAOS_CHECK_EXIT(tEncodeI32(&encoder, pReq->dnodeId));
TAOS_CHECK_EXIT(tEncodeI64(&encoder, pReq->analVer));
tEndEncode(&encoder);
_exit:
if (code) {
tlen = code;
} else {
tlen = encoder.pos;
}
tEncoderClear(&encoder);
return tlen;
}
int32_t tDeserializeRetrieveAnalAlgoReq(void *buf, int32_t bufLen, SRetrieveAnalAlgoReq *pReq) {
SDecoder decoder = {0};
int32_t code = 0;
int32_t lino;
tDecoderInit(&decoder, buf, bufLen);
TAOS_CHECK_EXIT(tStartDecode(&decoder));
TAOS_CHECK_EXIT(tDecodeI32(&decoder, &pReq->dnodeId));
TAOS_CHECK_EXIT(tDecodeI64(&decoder, &pReq->analVer));
tEndDecode(&decoder);
_exit:
tDecoderClear(&decoder);
return code;
}
int32_t tSerializeRetrieveAnalAlgoRsp(void *buf, int32_t bufLen, SRetrieveAnalAlgoRsp *pRsp) {
SEncoder encoder = {0};
int32_t code = 0;
int32_t lino;
int32_t tlen;
tEncoderInit(&encoder, buf, bufLen);
int32_t numOfAlgos = 0;
void *pIter = taosHashIterate(pRsp->hash, NULL);
while (pIter != NULL) {
SAnalUrl *pUrl = pIter;
size_t nameLen = 0;
const char *name = taosHashGetKey(pIter, &nameLen);
if (nameLen > 0 && nameLen <= TSDB_ANAL_ALGO_KEY_LEN && pUrl->urlLen > 0) {
numOfAlgos++;
}
pIter = taosHashIterate(pRsp->hash, pIter);
}
TAOS_CHECK_EXIT(tStartEncode(&encoder));
TAOS_CHECK_EXIT(tEncodeI64(&encoder, pRsp->ver));
TAOS_CHECK_EXIT(tEncodeI32(&encoder, numOfAlgos));
pIter = taosHashIterate(pRsp->hash, NULL);
while (pIter != NULL) {
SAnalUrl *pUrl = pIter;
size_t nameLen = 0;
const char *name = taosHashGetKey(pIter, &nameLen);
if (nameLen > 0 && pUrl->urlLen > 0) {
TAOS_CHECK_EXIT(tEncodeI32(&encoder, nameLen));
TAOS_CHECK_EXIT(tEncodeBinary(&encoder, (const uint8_t *)name, nameLen));
TAOS_CHECK_EXIT(tEncodeI32(&encoder, pUrl->anode));
TAOS_CHECK_EXIT(tEncodeI32(&encoder, pUrl->type));
TAOS_CHECK_EXIT(tEncodeI32(&encoder, pUrl->urlLen));
TAOS_CHECK_EXIT(tEncodeBinary(&encoder, (const uint8_t *)pUrl->url, pUrl->urlLen));
}
pIter = taosHashIterate(pRsp->hash, pIter);
}
tEndEncode(&encoder);
_exit:
if (code) {
tlen = code;
} else {
tlen = encoder.pos;
}
tEncoderClear(&encoder);
return tlen;
}
int32_t tDeserializeRetrieveAnalAlgoRsp(void *buf, int32_t bufLen, SRetrieveAnalAlgoRsp *pRsp) {
if (pRsp->hash == NULL) {
pRsp->hash = taosHashInit(64, MurmurHash3_32, true, HASH_ENTRY_LOCK);
if (pRsp->hash == NULL) {
terrno = TSDB_CODE_OUT_OF_BUFFER;
return terrno;
}
}
SDecoder decoder = {0};
int32_t code = 0;
int32_t lino;
tDecoderInit(&decoder, buf, bufLen);
int32_t numOfAlgos = 0;
int32_t nameLen;
int32_t type;
char name[TSDB_ANAL_ALGO_KEY_LEN];
SAnalUrl url = {0};
TAOS_CHECK_EXIT(tStartDecode(&decoder));
TAOS_CHECK_EXIT(tDecodeI64(&decoder, &pRsp->ver));
TAOS_CHECK_EXIT(tDecodeI32(&decoder, &numOfAlgos));
for (int32_t f = 0; f < numOfAlgos; ++f) {
TAOS_CHECK_EXIT(tDecodeI32(&decoder, &nameLen));
if (nameLen > 0 && nameLen <= TSDB_ANAL_ALGO_NAME_LEN) {
TAOS_CHECK_EXIT(tDecodeCStrTo(&decoder, name));
}
TAOS_CHECK_EXIT(tDecodeI32(&decoder, &url.anode));
TAOS_CHECK_EXIT(tDecodeI32(&decoder, &type));
url.type = (EAnalAlgoType)type;
TAOS_CHECK_EXIT(tDecodeI32(&decoder, &url.urlLen));
if (url.urlLen > 0) {
TAOS_CHECK_EXIT(tDecodeBinaryAlloc(&decoder, (void **)&url.url, NULL) < 0);
}
TAOS_CHECK_EXIT(taosHashPut(pRsp->hash, name, nameLen, &url, sizeof(SAnalUrl)));
}
tEndDecode(&decoder);
_exit:
tDecoderClear(&decoder);
return code;
}
void tFreeRetrieveAnalAlgoRsp(SRetrieveAnalAlgoRsp *pRsp) {
void *pIter = taosHashIterate(pRsp->hash, NULL);
while (pIter != NULL) {
SAnalUrl *pUrl = (SAnalUrl *)pIter;
taosMemoryFree(pUrl->url);
pIter = taosHashIterate(pRsp->hash, pIter);
}
taosHashCleanup(pRsp->hash);
pRsp->hash = NULL;
}
void tFreeSCreateUserReq(SCreateUserReq *pReq) {
FREESQL();
taosMemoryFreeClear(pReq->pIpRanges);
@ -2961,6 +3123,108 @@ _exit:
return code;
}
int32_t tSerializeSMCreateAnodeReq(void *buf, int32_t bufLen, SMCreateAnodeReq *pReq) {
SEncoder encoder = {0};
int32_t code = 0;
int32_t lino;
int32_t tlen;
tEncoderInit(&encoder, buf, bufLen);
TAOS_CHECK_EXIT(tStartEncode(&encoder));
TAOS_CHECK_EXIT(tEncodeI32(&encoder, pReq->urlLen));
if (pReq->urlLen > 0) {
TAOS_CHECK_EXIT(tEncodeBinary(&encoder, (const uint8_t *)pReq->url, pReq->urlLen));
}
ENCODESQL();
tEndEncode(&encoder);
_exit:
if (code) {
tlen = code;
} else {
tlen = encoder.pos;
}
tEncoderClear(&encoder);
return tlen;
}
int32_t tDeserializeSMCreateAnodeReq(void *buf, int32_t bufLen, SMCreateAnodeReq *pReq) {
SDecoder decoder = {0};
int32_t code = 0;
int32_t lino;
tDecoderInit(&decoder, buf, bufLen);
TAOS_CHECK_EXIT(tStartDecode(&decoder));
TAOS_CHECK_EXIT(tDecodeI32(&decoder, &pReq->urlLen));
if (pReq->urlLen > 0) {
TAOS_CHECK_EXIT(tDecodeBinaryAlloc(&decoder, (void **)&pReq->url, NULL));
}
DECODESQL();
tEndDecode(&decoder);
_exit:
tDecoderClear(&decoder);
return code;
}
void tFreeSMCreateAnodeReq(SMCreateAnodeReq *pReq) {
taosMemoryFreeClear(pReq->url);
FREESQL();
}
int32_t tSerializeSMDropAnodeReq(void *buf, int32_t bufLen, SMDropAnodeReq *pReq) {
SEncoder encoder = {0};
int32_t code = 0;
int32_t lino;
int32_t tlen;
tEncoderInit(&encoder, buf, bufLen);
TAOS_CHECK_EXIT(tStartEncode(&encoder));
TAOS_CHECK_EXIT(tEncodeI32(&encoder, pReq->anodeId));
ENCODESQL();
tEndEncode(&encoder);
_exit:
if (code) {
tlen = code;
} else {
tlen = encoder.pos;
}
tEncoderClear(&encoder);
return tlen;
}
int32_t tDeserializeSMDropAnodeReq(void *buf, int32_t bufLen, SMDropAnodeReq *pReq) {
SDecoder decoder = {0};
int32_t code = 0;
int32_t lino;
tDecoderInit(&decoder, buf, bufLen);
TAOS_CHECK_EXIT(tStartDecode(&decoder));
TAOS_CHECK_EXIT(tDecodeI32(&decoder, &pReq->anodeId));
DECODESQL();
tEndDecode(&decoder);
_exit:
tDecoderClear(&decoder);
return code;
}
void tFreeSMDropAnodeReq(SMDropAnodeReq *pReq) { FREESQL(); }
int32_t tSerializeSMUpdateAnodeReq(void *buf, int32_t bufLen, SMUpdateAnodeReq *pReq) {
return tSerializeSMDropAnodeReq(buf, bufLen, pReq);
}
int32_t tDeserializeSMUpdateAnodeReq(void *buf, int32_t bufLen, SMUpdateAnodeReq *pReq) {
return tDeserializeSMDropAnodeReq(buf, bufLen, pReq);
}
void tFreeSMUpdateAnodeReq(SMUpdateAnodeReq *pReq) { tFreeSMDropAnodeReq(pReq); }
int32_t tSerializeSCreateDnodeReq(void *buf, int32_t bufLen, SCreateDnodeReq *pReq) {
SEncoder encoder = {0};
int32_t code = 0;

2
source/common/src/tname.c
View File

@ -33,7 +33,7 @@ int64_t taosGetIntervalStartTimestamp(int64_t startTime, int64_t slidingTime, in
}
struct tm tm;
time_t t = (time_t)start;
taosLocalTime(&t, &tm);
taosLocalTime(&t, &tm, NULL, 0);
tm.tm_sec = 0;
tm.tm_min = 0;
tm.tm_hour = 0;

20
source/common/src/ttime.c
View File

@ -693,7 +693,7 @@ int64_t taosTimeAdd(int64_t t, int64_t duration, char unit, int32_t precision) {
struct tm tm;
time_t tt = (time_t)(t / TSDB_TICK_PER_SECOND(precision));
struct tm* ptm = taosLocalTime(&tt, &tm, NULL);
struct tm* ptm = taosLocalTime(&tt, &tm, NULL, 0);
int32_t mon = tm.tm_year * 12 + tm.tm_mon + (int32_t)numOfMonth;
tm.tm_year = mon / 12;
tm.tm_mon = mon % 12;
@ -754,11 +754,11 @@ int32_t taosTimeCountIntervalForFill(int64_t skey, int64_t ekey, int64_t interva
struct tm tm;
time_t t = (time_t)skey;
struct tm* ptm = taosLocalTime(&t, &tm, NULL);
struct tm* ptm = taosLocalTime(&t, &tm, NULL, 0);
int32_t smon = tm.tm_year * 12 + tm.tm_mon;
t = (time_t)ekey;
ptm = taosLocalTime(&t, &tm, NULL);
ptm = taosLocalTime(&t, &tm, NULL, 0);
int32_t emon = tm.tm_year * 12 + tm.tm_mon;
if (unit == 'y') {
@ -782,7 +782,7 @@ int64_t taosTimeTruncate(int64_t ts, const SInterval* pInterval) {
start /= (int64_t)(TSDB_TICK_PER_SECOND(precision));
struct tm tm;
time_t tt = (time_t)start;
struct tm* ptm = taosLocalTime(&tt, &tm, NULL);
struct tm* ptm = taosLocalTime(&tt, &tm, NULL, 0);
tm.tm_sec = 0;
tm.tm_min = 0;
tm.tm_hour = 0;
@ -911,13 +911,13 @@ int64_t taosTimeGetIntervalEnd(int64_t intervalStart, const SInterval* pInterval
// 2020-07-03 17:48:42
// and the parameter can also be a variable.
const char* fmtts(int64_t ts) {
static char buf[96] = {0};
static char buf[TD_TIME_STR_LEN] = {0};
size_t pos = 0;
struct tm tm;
if (ts > -62135625943 && ts < 32503651200) {
time_t t = (time_t)ts;
if (taosLocalTime(&t, &tm, buf) == NULL) {
if (taosLocalTime(&t, &tm, buf, sizeof(buf)) == NULL) {
return buf;
}
pos += strftime(buf + pos, sizeof(buf), "s=%Y-%m-%d %H:%M:%S", &tm);
@ -925,7 +925,7 @@ const char* fmtts(int64_t ts) {
if (ts > -62135625943000 && ts < 32503651200000) {
time_t t = (time_t)(ts / 1000);
if (taosLocalTime(&t, &tm, buf) == NULL) {
if (taosLocalTime(&t, &tm, buf, sizeof(buf)) == NULL) {
return buf;
}
if (pos > 0) {
@ -939,7 +939,7 @@ const char* fmtts(int64_t ts) {
{
time_t t = (time_t)(ts / 1000000);
if (taosLocalTime(&t, &tm, buf) == NULL) {
if (taosLocalTime(&t, &tm, buf, sizeof(buf)) == NULL) {
return buf;
}
if (pos > 0) {
@ -993,7 +993,7 @@ int32_t taosFormatUtcTime(char* buf, int32_t bufLen, int64_t t, int32_t precisio
TAOS_RETURN(TSDB_CODE_INVALID_PARA);
}
if (NULL == taosLocalTime(&quot, &ptm, buf)) {
if (NULL == taosLocalTime(&quot, &ptm, buf, bufLen)) {
TAOS_RETURN(TAOS_SYSTEM_ERROR(errno));
}
int32_t length = (int32_t)strftime(ts, 40, "%Y-%m-%dT%H:%M:%S", &ptm);
@ -1007,7 +1007,7 @@ int32_t taosFormatUtcTime(char* buf, int32_t bufLen, int64_t t, int32_t precisio
int32_t taosTs2Tm(int64_t ts, int32_t precision, struct STm* tm) {
tm->fsec = ts % TICK_PER_SECOND[precision] * (TICK_PER_SECOND[TSDB_TIME_PRECISION_NANO] / TICK_PER_SECOND[precision]);
time_t t = ts / TICK_PER_SECOND[precision];
if (NULL == taosLocalTime(&t, &tm->tm, NULL)) {
if (NULL == taosLocalTime(&t, &tm->tm, NULL, 0)) {
TAOS_RETURN(TAOS_SYSTEM_ERROR(errno));
}
return TSDB_CODE_SUCCESS;

45
source/dnode/mgmt/mgmt_dnode/src/dmHandle.c
View File

@ -18,6 +18,7 @@
#include "dmInt.h"
#include "monitor.h"
#include "systable.h"
#include "tanal.h"
#include "tchecksum.h"
extern SConfig *tsCfg;
@ -39,6 +40,7 @@ static void dmUpdateDnodeCfg(SDnodeMgmt *pMgmt, SDnodeCfg *pCfg) {
(void)taosThreadRwlockUnlock(&pMgmt->pData->lock);
}
}
static void dmMayShouldUpdateIpWhiteList(SDnodeMgmt *pMgmt, int64_t ver) {
int32_t code = 0;
dDebug("ip-white-list on dnode ver: %" PRId64 ", status ver: %" PRId64 "", pMgmt->pData->ipWhiteVer, ver);
@ -84,6 +86,47 @@ static void dmMayShouldUpdateIpWhiteList(SDnodeMgmt *pMgmt, int64_t ver) {
dError("failed to send retrieve ip white list request since:%s", tstrerror(code));
}
}
static void dmMayShouldUpdateAnalFunc(SDnodeMgmt *pMgmt, int64_t newVer) {
int32_t code = 0;
int64_t oldVer = taosAnalGetVersion();
if (oldVer == newVer) return;
dDebug("analysis on dnode ver:%" PRId64 ", status ver:%" PRId64, oldVer, newVer);
SRetrieveAnalAlgoReq req = {.dnodeId = pMgmt->pData->dnodeId, .analVer = oldVer};
int32_t contLen = tSerializeRetrieveAnalAlgoReq(NULL, 0, &req);
if (contLen < 0) {
dError("failed to serialize analysis function ver request since %s", tstrerror(contLen));
return;
}
void *pHead = rpcMallocCont(contLen);
contLen = tSerializeRetrieveAnalAlgoReq(pHead, contLen, &req);
if (contLen < 0) {
rpcFreeCont(pHead);
dError("failed to serialize analysis function ver request since %s", tstrerror(contLen));
return;
}
SRpcMsg rpcMsg = {
.pCont = pHead,
.contLen = contLen,
.msgType = TDMT_MND_RETRIEVE_ANAL_ALGO,
.info.ahandle = (void *)0x9527,
.info.refId = 0,
.info.noResp = 0,
.info.handle = 0,
};
SEpSet epset = {0};
(void)dmGetMnodeEpSet(pMgmt->pData, &epset);
code = rpcSendRequest(pMgmt->msgCb.clientRpc, &epset, &rpcMsg, NULL);
if (code != 0) {
dError("failed to send retrieve analysis func ver request since %s", tstrerror(code));
}
}
static void dmProcessStatusRsp(SDnodeMgmt *pMgmt, SRpcMsg *pRsp) {
const STraceId *trace = &pRsp->info.traceId;
dGTrace("status rsp received from mnode, statusSeq:%d code:0x%x", pMgmt->statusSeq, pRsp->code);
@ -111,6 +154,7 @@ static void dmProcessStatusRsp(SDnodeMgmt *pMgmt, SRpcMsg *pRsp) {
dmUpdateEps(pMgmt->pData, statusRsp.pDnodeEps);
}
dmMayShouldUpdateIpWhiteList(pMgmt, statusRsp.ipWhiteVer);
dmMayShouldUpdateAnalFunc(pMgmt, statusRsp.analVer);
}
tFreeSStatusRsp(&statusRsp);
}
@ -172,6 +216,7 @@ void dmSendStatusReq(SDnodeMgmt *pMgmt) {
pMgmt->statusSeq++;
req.statusSeq = pMgmt->statusSeq;
req.ipWhiteVer = pMgmt->pData->ipWhiteVer;
req.analVer = taosAnalGetVersion();
int32_t contLen = tSerializeSStatusReq(NULL, 0, &req);
if (contLen < 0) {

18
source/dnode/mgmt/node_mgmt/src/dmTransport.c
View File

@ -17,6 +17,7 @@
#include "dmMgmt.h"
#include "qworker.h"
#include "tversion.h"
#include "tanal.h"
static inline void dmSendRsp(SRpcMsg *pMsg) {
if (rpcSendResponse(pMsg) != 0) {
@ -105,6 +106,17 @@ static bool dmIsForbiddenIp(int8_t forbidden, char *user, uint32_t clientIp) {
return false;
}
}
static void dmUpdateAnalFunc(SDnodeData *pData, void *pTrans, SRpcMsg *pRpc) {
SRetrieveAnalAlgoRsp rsp = {0};
if (tDeserializeRetrieveAnalAlgoRsp(pRpc->pCont, pRpc->contLen, &rsp) == 0) {
taosAnalUpdate(rsp.ver, rsp.hash);
rsp.hash = NULL;
}
tFreeRetrieveAnalAlgoRsp(&rsp);
rpcFreeCont(pRpc->pCont);
}
static void dmProcessRpcMsg(SDnode *pDnode, SRpcMsg *pRpc, SEpSet *pEpSet) {
SDnodeTrans *pTrans = &pDnode->trans;
int32_t code = -1;
@ -150,10 +162,12 @@ static void dmProcessRpcMsg(SDnode *pDnode, SRpcMsg *pRpc, SEpSet *pEpSet) {
dmSetMnodeEpSet(&pDnode->data, pEpSet);
}
break;
case TDMT_MND_RETRIEVE_IP_WHITE_RSP: {
case TDMT_MND_RETRIEVE_IP_WHITE_RSP:
dmUpdateRpcIpWhite(&pDnode->data, pTrans->serverRpc, pRpc);
return;
} break;
case TDMT_MND_RETRIEVE_ANAL_ALGO_RSP:
dmUpdateAnalFunc(&pDnode->data, pTrans->serverRpc, pRpc);
return;
default:
break;
}

4
source/dnode/mnode/impl/CMakeLists.txt
View File

@ -14,6 +14,10 @@ IF (TD_ENTERPRISE)
ELSEIF(${BUILD_WITH_COS})
add_definitions(-DUSE_COS)
ENDIF()
IF(${BUILD_WITH_ANALYSIS})
add_definitions(-DUSE_ANAL)
ENDIF()
ENDIF ()
add_library(mnode STATIC ${MNODE_SRC})

23
source/dnode/mnode/impl/src/mndAnode.c
View File

@ -43,7 +43,7 @@ static void mndCancelGetNextAnode(SMnode *pMnode, void *pIter);
static int32_t mndRetrieveAnodesFull(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBlock, int32_t rows);
static void mndCancelGetNextAnodeFull(SMnode *pMnode, void *pIter);
static int32_t mndGetAnodeAlgoList(const char *url, SAnodeObj *pObj);
static int32_t mndGetAnodeStatus(SAnodeObj *pObj, char *status);
static int32_t mndGetAnodeStatus(SAnodeObj *pObj, char *status, int32_t statusLen);
int32_t mndInitAnode(SMnode *pMnode) {
SSdbTable table = {
@ -603,7 +603,7 @@ static int32_t mndRetrieveAnodes(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pB
if (code != 0) goto _end;
status[0] = 0;
if (mndGetAnodeStatus(pObj, status) == 0) {
if (mndGetAnodeStatus(pObj, status, 64) == 0) {
STR_TO_VARSTR(buf, status);
} else {
STR_TO_VARSTR(buf, "offline");
@ -698,7 +698,7 @@ static int32_t mndDecodeAlgoList(SJson *pJson, SAnodeObj *pObj) {
code = tjsonGetDoubleValue(pJson, "protocol", &tmp);
if (code < 0) return TSDB_CODE_INVALID_JSON_FORMAT;
protocol = (int32_t)(tmp * 1000);
if (protocol != 100) return TSDB_CODE_MND_ANODE_INVALID_PROTOCOL;
if (protocol != 100 && protocol != 1000) return TSDB_CODE_MND_ANODE_INVALID_PROTOCOL;
code = tjsonGetDoubleValue(pJson, "version", &tmp);
pObj->version = (int32_t)(tmp * 1000);
@ -722,10 +722,10 @@ static int32_t mndDecodeAlgoList(SJson *pJson, SAnodeObj *pObj) {
SJson *detail = tjsonGetArrayItem(details, d);
if (detail == NULL) return TSDB_CODE_INVALID_JSON_FORMAT;
code = tjsonGetStringValue(detail, "type", buf);
code = tjsonGetStringValue2(detail, "type", buf, sizeof(buf));
if (code < 0) return TSDB_CODE_INVALID_JSON_FORMAT;
EAnalAlgoType type = taosAnalAlgoInt(buf);
if (type < 0 || type >= ANAL_ALGO_TYPE_END) continue;
if (type < 0 || type >= ANAL_ALGO_TYPE_END) return TSDB_CODE_MND_ANODE_INVALID_ALGO_TYPE;
SJson *algos = tjsonGetObjectItem(detail, "algo");
if (algos == NULL) return TSDB_CODE_INVALID_JSON_FORMAT;
@ -734,13 +734,12 @@ static int32_t mndDecodeAlgoList(SJson *pJson, SAnodeObj *pObj) {
SJson *algo = tjsonGetArrayItem(algos, a);
if (algo == NULL) return TSDB_CODE_INVALID_JSON_FORMAT;
code = tjsonGetStringValue(algo, "name", buf);
if (code < 0) return TSDB_CODE_INVALID_JSON_FORMAT;
code = tjsonGetStringValue2(algo, "name", buf, sizeof(buf));
if (code < 0) return TSDB_CODE_MND_ANODE_TOO_LONG_ALGO_NAME;
SAnodeAlgo algoObj = {0};
algoObj.nameLen = strlen(buf) + 1;
if (algoObj.nameLen > TSDB_ANAL_ALGO_NAME_LEN) return TSDB_CODE_MND_ANODE_TOO_LONG_ALGO_NAME;
if (algoObj.nameLen <= 1) return TSDB_CODE_OUT_OF_MEMORY;
if (algoObj.nameLen <= 1) return TSDB_CODE_INVALID_JSON_FORMAT;
algoObj.name = taosMemoryCalloc(algoObj.nameLen, 1);
tstrncpy(algoObj.name, buf, algoObj.nameLen);
@ -764,7 +763,7 @@ static int32_t mndGetAnodeAlgoList(const char *url, SAnodeObj *pObj) {
TAOS_RETURN(code);
}
static int32_t mndGetAnodeStatus(SAnodeObj *pObj, char *status) {
static int32_t mndGetAnodeStatus(SAnodeObj *pObj, char *status, int32_t statusLen) {
int32_t code = 0;
int32_t protocol = 0;
double tmp = 0;
@ -780,12 +779,12 @@ static int32_t mndGetAnodeStatus(SAnodeObj *pObj, char *status) {
goto _OVER;
}
protocol = (int32_t)(tmp * 1000);
if (protocol != 100) {
if (protocol != 100 && protocol != 1000) {
code = TSDB_CODE_MND_ANODE_INVALID_PROTOCOL;
goto _OVER;
}
code = tjsonGetStringValue(pJson, "status", status);
code = tjsonGetStringValue2(pJson, "status", status, statusLen);
if (code < 0) {
code = TSDB_CODE_INVALID_JSON_FORMAT;
goto _OVER;

7
source/dnode/mnode/impl/src/mndArbGroup.c
View File

@ -445,7 +445,7 @@ static int32_t mndProcessArbHbTimer(SRpcMsg *pReq) {
SDnodeObj *pDnode = mndAcquireDnode(pMnode, dnodeId);
if (pDnode == NULL) {
mError("dnodeId:%d, timer failed to send arb-hb request, failed find dnode", dnodeId);
mError("dnodeId:%d, timer failed to acquire dnode", dnodeId);
taosArrayDestroy(hbMembers);
continue;
}
@ -453,7 +453,10 @@ static int32_t mndProcessArbHbTimer(SRpcMsg *pReq) {
int64_t mndTerm = mndGetTerm(pMnode);
if (mndIsDnodeOnline(pDnode, nowMs)) {
TAOS_CHECK_RETURN(mndSendArbHeartBeatReq(pDnode, arbToken, mndTerm, hbMembers));
int32_t sendCode = mndSendArbHeartBeatReq(pDnode, arbToken, mndTerm, hbMembers);
if (TSDB_CODE_SUCCESS != sendCode) {
mError("dnodeId:%d, timer failed to send arb-hb request", dnodeId);
}
}
mndReleaseDnode(pMnode, pDnode);

2
source/dnode/mnode/impl/src/mndMain.c
View File

@ -575,6 +575,8 @@ static int32_t mndOpenSdb(SMnode *pMnode) {
code = sdbReadFile(pMnode->pSdb);
}
mInfo("vgId:1, mnode sdb is opened, with applied index:%" PRId64, pMnode->pSdb->commitIndex);
atomic_store_64(&pMnode->applied, pMnode->pSdb->commitIndex);
return code;
}

2
source/dnode/mnode/impl/src/mndSync.c
View File

@ -637,7 +637,7 @@ void mndSyncStop(SMnode *pMnode) {
(void)taosThreadMutexLock(&pMgmt->lock);
if (pMgmt->transId != 0) {
mInfo("vgId:1, is stopped and post sem, trans:%d", pMgmt->transId);
mInfo("vgId:1, trans:%d, is stopped and post sem", pMgmt->transId);
pMgmt->transId = 0;
pMgmt->transSec = 0;
pMgmt->errCode = TSDB_CODE_APP_IS_STOPPING;

18
source/dnode/mnode/impl/src/mndTrans.c
View File

@ -589,6 +589,7 @@ STrans *mndAcquireTrans(SMnode *pMnode, int32_t transId) {
void mndReleaseTrans(SMnode *pMnode, STrans *pTrans) {
SSdb *pSdb = pMnode->pSdb;
if (pTrans != NULL) mInfo("vgId:1, trans:%d, release transaction", pTrans->id);
sdbRelease(pSdb, pTrans);
}
@ -1131,10 +1132,11 @@ static void mndTransSendRpcRsp(SMnode *pMnode, STrans *pTrans) {
if (!sendRsp) {
return;
} else {
mInfo("trans:%d, send rsp, stage:%s failedTimes:%d code:0x%x", pTrans->id, mndTransStr(pTrans->stage),
pTrans->failedTimes, code);
mInfo("vgId:1, trans:%d, start to send rsp, stage:%s failedTimes:%d code:0x%x", pTrans->id,
mndTransStr(pTrans->stage), pTrans->failedTimes, code);
}
mInfo("vgId:1, trans:%d, start to lock rpc array", pTrans->id);
taosWLockLatch(&pTrans->lockRpcArray);
int32_t size = taosArrayGetSize(pTrans->pRpcArray);
if (size <= 0) {
@ -1155,8 +1157,8 @@ static void mndTransSendRpcRsp(SMnode *pMnode, STrans *pTrans) {
if (i != 0 && code == 0) {
code = TSDB_CODE_MNODE_NOT_FOUND;
}
mInfo("trans:%d, client:%d send rsp, code:0x%x stage:%s app:%p", pTrans->id, i, code, mndTransStr(pTrans->stage),
pInfo->ahandle);
mInfo("vgId:1, trans:%d, client:%d start to send rsp, code:0x%x stage:%s app:%p", pTrans->id, i, code,
mndTransStr(pTrans->stage), pInfo->ahandle);
SRpcMsg rspMsg = {.code = code, .info = *pInfo};
@ -1199,6 +1201,9 @@ static void mndTransSendRpcRsp(SMnode *pMnode, STrans *pTrans) {
}
tmsgSendRsp(&rspMsg);
mInfo("vgId:1, trans:%d, client:%d send rsp finished, code:0x%x stage:%s app:%p", pTrans->id, i, code,
mndTransStr(pTrans->stage), pInfo->ahandle);
}
}
taosArrayClear(pTrans->pRpcArray);
@ -1495,7 +1500,7 @@ static int32_t mndTransExecuteActionsSerial(SMnode *pMnode, STrans *pTrans, SArr
return code;
}
mInfo("trans:%d, execute %d actions serial, current redoAction:%d", pTrans->id, numOfActions, pTrans->actionPos);
mInfo("trans:%d, execute %d actions serial, current action:%d", pTrans->id, numOfActions, pTrans->actionPos);
for (int32_t action = pTrans->actionPos; action < numOfActions; ++action) {
STransAction *pAction = taosArrayGet(pActions, action);
@ -1768,7 +1773,8 @@ static bool mndTransPerformRollbackStage(SMnode *pMnode, STrans *pTrans, bool to
if (code == 0) {
pTrans->stage = TRN_STAGE_UNDO_ACTION;
mInfo("trans:%d, stage from rollback to undoAction", pTrans->id);
pTrans->actionPos = 0;
mInfo("trans:%d, stage from rollback to undoAction, actionPos:%d", pTrans->id, pTrans->actionPos);
continueExec = true;
} else {
pTrans->failedTimes++;

9
source/dnode/mnode/sdb/src/sdb.c
View File

@ -168,11 +168,10 @@ static int32_t sdbCreateDir(SSdb *pSdb) {
}
void sdbSetApplyInfo(SSdb *pSdb, int64_t index, int64_t term, int64_t config) {
#if 1
mTrace("mnode apply info changed from index:%" PRId64 " term:%" PRId64 " config:%" PRId64 " to index:%" PRId64
" term:%" PRId64 " config:%" PRId64,
pSdb->applyIndex, pSdb->applyTerm, pSdb->applyConfig, index, term, config);
#endif
mInfo("vgId:1, mnode apply info changed from index:%" PRId64 " term:%" PRId64 " config:%" PRId64 " to index:%" PRId64
" term:%" PRId64 " config:%" PRId64,
pSdb->applyIndex, pSdb->applyTerm, pSdb->applyConfig, index, term, config);
pSdb->applyIndex = index;
pSdb->applyTerm = term;
pSdb->applyConfig = config;

5
source/dnode/mnode/sdb/src/sdbFile.c
View File

@ -207,6 +207,8 @@ static int32_t sdbWriteFileHead(SSdb *pSdb, TdFilePtr pFile) {
return terrno;
}
mInfo("vgId:1, write sdb file with sdb applyIndex:%" PRId64 " term:%" PRId64 " config:%" PRId64, pSdb->applyIndex,
pSdb->applyTerm, pSdb->applyConfig);
if (taosWriteFile(pFile, &pSdb->applyIndex, sizeof(int64_t)) != sizeof(int64_t)) {
return terrno;
}
@ -607,6 +609,9 @@ int32_t sdbWriteFile(SSdb *pSdb, int32_t delta) {
}
if (code != 0) {
mError("failed to write sdb file since %s", tstrerror(code));
} else {
mInfo("write sdb file success, apply index:%" PRId64 " term:%" PRId64 " config:%" PRId64, pSdb->applyIndex,
pSdb->applyTerm, pSdb->applyConfig);
}
(void)taosThreadMutexUnlock(&pSdb->filelock);
return code;

1
source/dnode/mnode/sdb/src/sdbRow.c
View File

@ -42,6 +42,7 @@ void sdbFreeRow(SSdb *pSdb, SSdbRow *pRow, bool callFunc) {
// remove attached object such as trans
SdbDeleteFp deleteFp = pSdb->deleteFps[pRow->type];
if (deleteFp != NULL) {
mInfo("vgId:1, deleteFp:%p, type:%s", deleteFp, sdbTableName(pRow->type));
(void)(*deleteFp)(pSdb, pRow->pObj, callFunc);
}

14
source/dnode/vnode/src/tsdb/tsdbCache.c
View File

@ -1767,10 +1767,14 @@ int32_t tsdbCacheGetBatch(STsdb *pTsdb, tb_uid_t uid, SArray *pLastArray, SCache
SLastCol *pLastCol = h ? (SLastCol *)taosLRUCacheValue(pCache, h) : NULL;
if (h && pLastCol->cacheStatus != TSDB_LAST_CACHE_NO_CACHE) {
SLastCol lastCol = *pLastCol;
TAOS_CHECK_GOTO(tsdbCacheReallocSLastCol(&lastCol, NULL), NULL, _exit);
if (TSDB_CODE_SUCCESS != (code = tsdbCacheReallocSLastCol(&lastCol, NULL))) {
tsdbLRUCacheRelease(pCache, h, false);
TAOS_CHECK_GOTO(code, NULL, _exit);
}
if (taosArrayPush(pLastArray, &lastCol) == NULL) {
code = terrno;
tsdbLRUCacheRelease(pCache, h, false);
goto _exit;
}
} else {
@ -1780,28 +1784,33 @@ int32_t tsdbCacheGetBatch(STsdb *pTsdb, tb_uid_t uid, SArray *pLastArray, SCache
if (taosArrayPush(pLastArray, &noneCol) == NULL) {
code = terrno;
tsdbLRUCacheRelease(pCache, h, false);
goto _exit;
}
if (!remainCols) {
if ((remainCols = taosArrayInit(numKeys, sizeof(SIdxKey))) == NULL) {
code = terrno;
tsdbLRUCacheRelease(pCache, h, false);
goto _exit;
}
}
if (!ignoreFromRocks) {
if ((ignoreFromRocks = taosArrayInit(numKeys, sizeof(bool))) == NULL) {
code = terrno;
tsdbLRUCacheRelease(pCache, h, false);
goto _exit;
}
}
if (taosArrayPush(remainCols, &(SIdxKey){i, key}) == NULL) {
code = terrno;
tsdbLRUCacheRelease(pCache, h, false);
goto _exit;
}
bool ignoreRocks = pLastCol ? (pLastCol->cacheStatus == TSDB_LAST_CACHE_NO_CACHE) : false;
if (taosArrayPush(ignoreFromRocks, &ignoreRocks) == NULL) {
code = terrno;
tsdbLRUCacheRelease(pCache, h, false);
goto _exit;
}
}
@ -1822,6 +1831,7 @@ int32_t tsdbCacheGetBatch(STsdb *pTsdb, tb_uid_t uid, SArray *pLastArray, SCache
SLastCol lastCol = *pLastCol;
code = tsdbCacheReallocSLastCol(&lastCol, NULL);
if (code) {
tsdbLRUCacheRelease(pCache, h, false);
(void)taosThreadMutexUnlock(&pTsdb->lruMutex);
TAOS_RETURN(code);
}
@ -3600,4 +3610,4 @@ void tsdbCacheSetPageS3(SLRUCache *pCache, STsdbFD *pFD, int64_t pgno, uint8_t *
(void)taosThreadMutexUnlock(&pFD->pTsdb->pgMutex);
tsdbCacheRelease(pFD->pTsdb->pgCache, handle);
}
}

17
source/dnode/vnode/src/tsdb/tsdbSnapshot.c
View File

@ -623,6 +623,7 @@ static int32_t tsdbSnapWriteFileSetOpenReader(STsdbSnapWriter* writer) {
int32_t lino = 0;
if (writer->ctx->fset) {
#if 0
// open data reader
SDataFileReaderConfig dataFileReaderConfig = {
.tsdb = writer->tsdb,
@ -650,6 +651,7 @@ static int32_t tsdbSnapWriteFileSetOpenReader(STsdbSnapWriter* writer) {
code = tsdbDataFileReaderOpen(NULL, &dataFileReaderConfig, &writer->ctx->dataReader);
TSDB_CHECK_CODE(code, lino, _exit);
#endif
// open stt reader array
SSttLvl* lvl;
@ -791,6 +793,15 @@ static int32_t tsdbSnapWriteFileSetOpenWriter(STsdbSnapWriter* writer) {
.did = writer->ctx->did,
.level = 0,
};
// merge stt files to either data or a new stt file
if (writer->ctx->fset) {
for (int32_t ftype = 0; ftype < TSDB_FTYPE_MAX; ++ftype) {
if (writer->ctx->fset->farr[ftype] != NULL) {
config.files[ftype].exist = true;
config.files[ftype].file = writer->ctx->fset->farr[ftype]->f[0];
}
}
}
code = tsdbFSetWriterOpen(&config, &writer->ctx->fsetWriter);
TSDB_CHECK_CODE(code, lino, _exit);
@ -842,6 +853,8 @@ static int32_t tsdbSnapWriteFileSetBegin(STsdbSnapWriter* writer, int32_t fid) {
_exit:
if (code) {
TSDB_ERROR_LOG(TD_VID(writer->tsdb->pVnode), lino, code);
} else {
tsdbInfo("vgId:%d %s succeeded, fid:%d", TD_VID(writer->tsdb->pVnode), __func__, fid);
}
return code;
}
@ -922,6 +935,8 @@ static int32_t tsdbSnapWriteFileSetEnd(STsdbSnapWriter* writer) {
_exit:
if (code) {
TSDB_ERROR_LOG(TD_VID(writer->tsdb->pVnode), lino, code);
} else {
tsdbInfo("vgId:%d %s succeeded, fid:%d", TD_VID(writer->tsdb->pVnode), __func__, writer->ctx->fid);
}
return code;
}
@ -1175,7 +1190,7 @@ _exit:
if (code) {
TSDB_ERROR_LOG(TD_VID(tsdb->pVnode), lino, code);
} else {
tsdbInfo("vgId:%d %s done", TD_VID(tsdb->pVnode), __func__);
tsdbInfo("vgId:%d %s done, rollback:%d", TD_VID(tsdb->pVnode), __func__, rollback);
}
return code;
}

5
source/dnode/vnode/src/vnd/vnodeCommit.c
View File

@ -102,9 +102,8 @@ static int32_t vnodeGetBufPoolToUse(SVnode *pVnode) {
ts.tv_sec = tv.tv_sec;
}
int32_t rc = taosThreadCondTimedWait(&pVnode->poolNotEmpty, &pVnode->mutex, &ts);
if (rc && rc != ETIMEDOUT) {
code = TAOS_SYSTEM_ERROR(rc);
code = taosThreadCondTimedWait(&pVnode->poolNotEmpty, &pVnode->mutex, &ts);
if (code && code != TSDB_CODE_TIMEOUT_ERROR) {
TSDB_CHECK_CODE(code, lino, _exit);
}
}

2
source/libs/catalog/src/ctgAsync.c
View File

@ -2916,7 +2916,7 @@ int32_t ctgHandleGetTbTSMARsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf
if (META_TYPE_BOTH_TABLE == pOut->metaType) {
// rewrite tsma fetch table with it's super table name
(void)sprintf(pFetch->tsmaSourceTbName.tname, "%s", pOut->tbName);
(void)snprintf(pFetch->tsmaSourceTbName.tname, sizeof(pFetch->tsmaSourceTbName.tname), "%s", pOut->tbName);
}
CTG_ERR_JRET(ctgGetTbTSMAFromMnode(pCtg, pConn, &pFetch->tsmaSourceTbName, NULL, tReq, TDMT_MND_GET_TABLE_TSMA));

18
source/libs/catalog/src/ctgDbg.c
View File

@ -176,22 +176,22 @@ int32_t ctgdLaunchAsyncCall(SCatalog *pCtg, SRequestConnInfo *pConn, uint64_t re
taosArrayPush(req.pTableMeta, &name);
taosArrayPush(req.pTableHash, &name);
strcpy(dbFName, "1.db1");
tstrncpy(dbFName, "1.db1", sizeof(dbFName));
taosArrayPush(req.pDbVgroup, dbFName);
taosArrayPush(req.pDbCfg, dbFName);
taosArrayPush(req.pDbInfo, dbFName);
strcpy(dbFName, "1.db2");
tstrncpy(dbFName, "1.db2", sizeof(dbFName));
taosArrayPush(req.pDbVgroup, dbFName);
taosArrayPush(req.pDbCfg, dbFName);
taosArrayPush(req.pDbInfo, dbFName);
strcpy(funcName, "udf1");
tstrncpy(funcName, "udf1", sizeof(funcName));
taosArrayPush(req.pUdf, funcName);
strcpy(funcName, "udf2");
tstrncpy(funcName, "udf2", sizeof(funcName));
taosArrayPush(req.pUdf, funcName);
strcpy(user.user, "root");
strcpy(user.dbFName, "1.db1");
tstrncpy(user.user, "root", sizeof(user.user));
tstrncpy(user.dbFName, "1.db1", sizeof(user.dbFName));
user.type = AUTH_TYPE_READ;
taosArrayPush(req.pUser, &user);
user.type = AUTH_TYPE_WRITE;
@ -199,8 +199,8 @@ int32_t ctgdLaunchAsyncCall(SCatalog *pCtg, SRequestConnInfo *pConn, uint64_t re
user.type = AUTH_TYPE_OTHER;
taosArrayPush(req.pUser, &user);
strcpy(user.user, "user1");
strcpy(user.dbFName, "1.db2");
tstrncpy(user.user, "user1", sizeof(user.user));
tstrncpy(user.dbFName, "1.db2", sizeof(user.dbFName));
user.type = AUTH_TYPE_READ;
taosArrayPush(req.pUser, &user);
user.type = AUTH_TYPE_WRITE;
@ -335,7 +335,7 @@ int32_t ctgdHandleDbgCommand(char *command) {
CTG_RET(TSDB_CODE_INVALID_PARA);
}
bool enable = atoi(param);
bool enable = taosStr2Int32(param, NULL, 10);
int32_t code = ctgdEnableDebug(option, enable);

4
source/libs/catalog/src/ctgRemote.c
View File

@ -1303,7 +1303,7 @@ int32_t ctgGetTbMetaFromMnodeImpl(SCatalog* pCtg, SRequestConnInfo* pConn, const
int32_t msgLen = 0;
int32_t reqType = TDMT_MND_TABLE_META;
char tbFName[TSDB_TABLE_FNAME_LEN];
(void)sprintf(tbFName, "%s.%s", dbFName, tbName);
(void)snprintf(tbFName, sizeof(tbFName), "%s.%s", dbFName, tbName);
void* (*mallocFp)(int64_t) = pTask ? (MallocType)taosMemoryMalloc : (MallocType)rpcMallocCont;
ctgDebug("try to get table meta from mnode, tbFName:%s", tbFName);
@ -1369,7 +1369,7 @@ int32_t ctgGetTbMetaFromVnode(SCatalog* pCtg, SRequestConnInfo* pConn, const SNa
(void)tNameGetFullDbName(pTableName, dbFName);
int32_t reqType = (pTask && pTask->type == CTG_TASK_GET_TB_NAME ? TDMT_VND_TABLE_NAME : TDMT_VND_TABLE_META);
char tbFName[TSDB_TABLE_FNAME_LEN];
(void)sprintf(tbFName, "%s.%s", dbFName, pTableName->tname);
(void)snprintf(tbFName, sizeof(tbFName), "%s.%s", dbFName, pTableName->tname);
void* (*mallocFp)(int64_t) = pTask ? (MallocType)taosMemoryMalloc : (MallocType)rpcMallocCont;
SEp* pEp = &vgroupInfo->epSet.eps[vgroupInfo->epSet.inUse];

4
source/libs/catalog/src/ctgUtil.c
View File

@ -1386,7 +1386,7 @@ int32_t ctgGetVgInfosFromHashValue(SCatalog* pCtg, SEpSet* pMgmgEpSet, SCtgTaskR
}
char tbFullName[TSDB_TABLE_FNAME_LEN];
(void)sprintf(tbFullName, "%s.", dbFName);
(void)snprintf(tbFullName, sizeof(tbFullName), "%s.", dbFName);
int32_t offset = strlen(tbFullName);
SName* pName = NULL;
int32_t tbNameLen = 0;
@ -2070,7 +2070,7 @@ int32_t ctgChkSetTbAuthRes(SCatalog* pCtg, SCtgAuthReq* req, SCtgAuthRsp* res) {
continue;
}
(void)sprintf(tbFName, "%s.%s", dbFName, stbName);
(void)snprintf(tbFName, sizeof(tbFName), "%s.%s", dbFName, stbName);
continue;
}

4
source/libs/executor/CMakeLists.txt
View File

@ -5,6 +5,10 @@ if(${TD_DARWIN})
target_compile_options(executor PRIVATE -Wno-error=deprecated-non-prototype)
endif(${TD_DARWIN})
IF(${BUILD_WITH_ANALYSIS})
add_definitions(-DUSE_ANAL)
ENDIF()
target_link_libraries(executor
PRIVATE os util common function parser planner qcom scalar nodes index wal tdb geometry
)

609
source/libs/executor/src/anomalywindowoperator.c Normal file
View File

@ -0,0 +1,609 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include "executorInt.h"
#include "filter.h"
#include "function.h"
#include "functionMgt.h"
#include "operator.h"
#include "querytask.h"
#include "tanal.h"
#include "tcommon.h"
#include "tcompare.h"
#include "tdatablock.h"
#include "tjson.h"
#include "ttime.h"
#ifdef USE_ANAL
typedef struct {
SArray* blocks; // SSDataBlock*
SArray* windows; // STimeWindow
uint64_t groupId;
int64_t numOfRows;
int32_t curWinIndex;
STimeWindow curWin;
SResultRow* pResultRow;
} SAnomalyWindowSupp;
typedef struct {
SOptrBasicInfo binfo;
SAggSupporter aggSup;
SExprSupp scalarSup;
int32_t tsSlotId;
STimeWindowAggSupp twAggSup;
char algoName[TSDB_ANAL_ALGO_NAME_LEN];
char algoUrl[TSDB_ANAL_ALGO_URL_LEN];
char anomalyOpt[TSDB_ANAL_ALGO_OPTION_LEN];
SAnomalyWindowSupp anomalySup;
SWindowRowsSup anomalyWinRowSup;
SColumn anomalyCol;
SStateKeys anomalyKey;
} SAnomalyWindowOperatorInfo;
static void anomalyDestroyOperatorInfo(void* param);
static int32_t anomalyAggregateNext(SOperatorInfo* pOperator, SSDataBlock** ppRes);
static void anomalyAggregateBlocks(SOperatorInfo* pOperator);
static int32_t anomalyCacheBlock(SAnomalyWindowOperatorInfo* pInfo, SSDataBlock* pBlock);
int32_t createAnomalywindowOperatorInfo(SOperatorInfo* downstream, SPhysiNode* physiNode, SExecTaskInfo* pTaskInfo,
SOperatorInfo** pOptrInfo) {
QRY_PARAM_CHECK(pOptrInfo);
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
SAnomalyWindowOperatorInfo* pInfo = taosMemoryCalloc(1, sizeof(SAnomalyWindowOperatorInfo));
SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo));
SAnomalyWindowPhysiNode* pAnomalyNode = (SAnomalyWindowPhysiNode*)physiNode;
SColumnNode* pColNode = (SColumnNode*)(pAnomalyNode->pAnomalyKey);
if (pInfo == NULL || pOperator == NULL) {
code = terrno;
goto _error;
}
if (!taosAnalGetOptStr(pAnomalyNode->anomalyOpt, "algo", pInfo->algoName, sizeof(pInfo->algoName))) {
qError("failed to get anomaly_window algorithm name from %s", pAnomalyNode->anomalyOpt);
code = TSDB_CODE_ANAL_ALGO_NOT_FOUND;
goto _error;
}
if (taosAnalGetAlgoUrl(pInfo->algoName, ANAL_ALGO_TYPE_ANOMALY_DETECT, pInfo->algoUrl, sizeof(pInfo->algoUrl)) != 0) {
qError("failed to get anomaly_window algorithm url from %s", pInfo->algoName);
code = TSDB_CODE_ANAL_ALGO_NOT_LOAD;
goto _error;
}
pOperator->exprSupp.hasWindowOrGroup = true;
pInfo->tsSlotId = ((SColumnNode*)pAnomalyNode->window.pTspk)->slotId;
strncpy(pInfo->anomalyOpt, pAnomalyNode->anomalyOpt, sizeof(pInfo->anomalyOpt));
if (pAnomalyNode->window.pExprs != NULL) {
int32_t numOfScalarExpr = 0;
SExprInfo* pScalarExprInfo = NULL;
code = createExprInfo(pAnomalyNode->window.pExprs, NULL, &pScalarExprInfo, &numOfScalarExpr);
QUERY_CHECK_CODE(code, lino, _error);
code = initExprSupp(&pInfo->scalarSup, pScalarExprInfo, numOfScalarExpr, &pTaskInfo->storageAPI.functionStore);
QUERY_CHECK_CODE(code, lino, _error);
}
size_t keyBufSize = 0;
int32_t num = 0;
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pAnomalyNode->window.pFuncs, NULL, &pExprInfo, &num);
QUERY_CHECK_CODE(code, lino, _error);
initResultSizeInfo(&pOperator->resultInfo, 4096);
code = initAggSup(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, num, keyBufSize, pTaskInfo->id.str,
pTaskInfo->streamInfo.pState, &pTaskInfo->storageAPI.functionStore);
QUERY_CHECK_CODE(code, lino, _error);
SSDataBlock* pResBlock = createDataBlockFromDescNode(pAnomalyNode->window.node.pOutputDataBlockDesc);
QUERY_CHECK_NULL(pResBlock, code, lino, _error, terrno);
initBasicInfo(&pInfo->binfo, pResBlock);
code = blockDataEnsureCapacity(pResBlock, pOperator->resultInfo.capacity);
QUERY_CHECK_CODE(code, lino, _error);
initResultRowInfo(&pInfo->binfo.resultRowInfo);
pInfo->binfo.inputTsOrder = pAnomalyNode->window.node.inputTsOrder;
pInfo->binfo.outputTsOrder = pAnomalyNode->window.node.outputTsOrder;
pInfo->anomalyCol = extractColumnFromColumnNode(pColNode);
pInfo->anomalyKey.type = pInfo->anomalyCol.type;
pInfo->anomalyKey.bytes = pInfo->anomalyCol.bytes;
pInfo->anomalyKey.pData = taosMemoryCalloc(1, pInfo->anomalyCol.bytes);
if (pInfo->anomalyKey.pData == NULL) {
goto _error;
}
int32_t itemSize = sizeof(int32_t) + pInfo->aggSup.resultRowSize + pInfo->anomalyKey.bytes;
pInfo->anomalySup.pResultRow = taosMemoryCalloc(1, itemSize);
pInfo->anomalySup.blocks = taosArrayInit(16, sizeof(SSDataBlock*));
pInfo->anomalySup.windows = taosArrayInit(16, sizeof(STimeWindow));
if (pInfo->anomalySup.windows == NULL || pInfo->anomalySup.blocks == NULL || pInfo->anomalySup.pResultRow == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
goto _error;
}
code = filterInitFromNode((SNode*)pAnomalyNode->window.node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
QUERY_CHECK_CODE(code, lino, _error);
code = initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pTaskInfo->window);
QUERY_CHECK_CODE(code, lino, _error);
setOperatorInfo(pOperator, "AnomalyWindowOperator", QUERY_NODE_PHYSICAL_PLAN_MERGE_ANOMALY, true, OP_NOT_OPENED,
pInfo, pTaskInfo);
pOperator->fpSet = createOperatorFpSet(optrDummyOpenFn, anomalyAggregateNext, NULL, anomalyDestroyOperatorInfo,
optrDefaultBufFn, NULL, optrDefaultGetNextExtFn, NULL);
code = appendDownstream(pOperator, &downstream, 1);
QUERY_CHECK_CODE(code, lino, _error);
*pOptrInfo = pOperator;
qDebug("anomaly_window operator is created, algo:%s url:%s opt:%s", pInfo->algoName, pInfo->algoUrl,
pInfo->anomalyOpt);
return TSDB_CODE_SUCCESS;
_error:
if (pInfo != NULL) {
anomalyDestroyOperatorInfo(pInfo);
}
destroyOperatorAndDownstreams(pOperator, &downstream, 1);
pTaskInfo->code = code;
qError("failed to create anomaly_window operator, algo:%s code:0x%x", pInfo->algoName, code);
return code;
}
static int32_t anomalyAggregateNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
SAnomalyWindowOperatorInfo* pInfo = pOperator->info;
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
SOptrBasicInfo* pBInfo = &pInfo->binfo;
SAnomalyWindowSupp* pSupp = &pInfo->anomalySup;
SSDataBlock* pRes = pInfo->binfo.pRes;
int64_t st = taosGetTimestampUs();
int32_t numOfBlocks = taosArrayGetSize(pSupp->blocks);
blockDataCleanup(pRes);
while (1) {
SSDataBlock* pBlock = getNextBlockFromDownstream(pOperator, 0);
if (pBlock == NULL) {
break;
}
if (pSupp->groupId == 0 || pSupp->groupId == pBlock->info.id.groupId) {
pSupp->groupId = pBlock->info.id.groupId;
numOfBlocks++;
qDebug("group:%" PRId64 ", blocks:%d, cache block rows:%" PRId64, pSupp->groupId, numOfBlocks, pBlock->info.rows);
code = anomalyCacheBlock(pInfo, pBlock);
QUERY_CHECK_CODE(code, lino, _end);
} else {
qDebug("group:%" PRId64 ", read finish for new group coming, blocks:%d", pSupp->groupId, numOfBlocks);
anomalyAggregateBlocks(pOperator);
pSupp->groupId = pBlock->info.id.groupId;
numOfBlocks = 1;
qDebug("group:%" PRId64 ", new group, cache block rows:%" PRId64, pSupp->groupId, pBlock->info.rows);
code = anomalyCacheBlock(pInfo, pBlock);
QUERY_CHECK_CODE(code, lino, _end);
}
if (pRes->info.rows > 0) {
(*ppRes) = pRes;
qDebug("group:%" PRId64 ", return to upstream, blocks:%d", pRes->info.id.groupId, numOfBlocks);
return code;
}
}
if (numOfBlocks > 0) {
qDebug("group:%" PRId64 ", read finish, blocks:%d", pInfo->anomalySup.groupId, numOfBlocks);
anomalyAggregateBlocks(pOperator);
}
int64_t cost = taosGetTimestampUs() - st;
qDebug("all groups finished, cost:%" PRId64 "us", cost);
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
(*ppRes) = (pBInfo->pRes->info.rows == 0) ? NULL : pBInfo->pRes;
return code;
}
static void anomalyDestroyOperatorInfo(void* param) {
SAnomalyWindowOperatorInfo* pInfo = (SAnomalyWindowOperatorInfo*)param;
if (pInfo == NULL) return;
qDebug("anomaly_window operator is destroyed, algo:%s", pInfo->algoName);
cleanupBasicInfo(&pInfo->binfo);
cleanupAggSup(&pInfo->aggSup);
cleanupExprSupp(&pInfo->scalarSup);
colDataDestroy(&pInfo->twAggSup.timeWindowData);
for (int32_t i = 0; i < taosArrayGetSize(pInfo->anomalySup.blocks); ++i) {
SSDataBlock* pBlock = taosArrayGetP(pInfo->anomalySup.blocks, i);
blockDataDestroy(pBlock);
}
taosArrayDestroy(pInfo->anomalySup.blocks);
taosArrayDestroy(pInfo->anomalySup.windows);
taosMemoryFreeClear(pInfo->anomalySup.pResultRow);
taosMemoryFreeClear(pInfo->anomalyKey.pData);
taosMemoryFreeClear(param);
}
static int32_t anomalyCacheBlock(SAnomalyWindowOperatorInfo* pInfo, SSDataBlock* pSrc) {
SSDataBlock* pDst = NULL;
int32_t code = createOneDataBlock(pSrc, true, &pDst);
if (code != 0) return code;
if (pDst == NULL) return TSDB_CODE_OUT_OF_MEMORY;
if (taosArrayPush(pInfo->anomalySup.blocks, &pDst) == NULL) return TSDB_CODE_OUT_OF_MEMORY;
return 0;
}
static int32_t anomalyFindWindow(SAnomalyWindowSupp* pSupp, TSKEY key) {
for (int32_t i = pSupp->curWinIndex; i < taosArrayGetSize(pSupp->windows); ++i) {
STimeWindow* pWindow = taosArrayGet(pSupp->windows, i);
if (key >= pWindow->skey && key < pWindow->ekey) {
pSupp->curWin = *pWindow;
pSupp->curWinIndex = i;
return 0;
}
}
return -1;
}
static int32_t anomalyParseJson(SJson* pJson, SArray* pWindows) {
int32_t code = 0;
int32_t rows = 0;
STimeWindow win = {0};
taosArrayClear(pWindows);
tjsonGetInt32ValueFromDouble(pJson, "rows", rows, code);
if (code < 0) return TSDB_CODE_INVALID_JSON_FORMAT;
if (rows <= 0) return 0;
SJson* res = tjsonGetObjectItem(pJson, "res");
if (res == NULL) return TSDB_CODE_INVALID_JSON_FORMAT;
int32_t ressize = tjsonGetArraySize(res);
if (ressize != rows) return TSDB_CODE_INVALID_JSON_FORMAT;
for (int32_t i = 0; i < rows; ++i) {
SJson* row = tjsonGetArrayItem(res, i);
if (row == NULL) return TSDB_CODE_INVALID_JSON_FORMAT;
int32_t colsize = tjsonGetArraySize(row);
if (colsize != 2) return TSDB_CODE_INVALID_JSON_FORMAT;
SJson* start = tjsonGetArrayItem(row, 0);
SJson* end = tjsonGetArrayItem(row, 1);
if (start == NULL || end == NULL) return TSDB_CODE_INVALID_JSON_FORMAT;
tjsonGetObjectValueBigInt(start, &win.skey);
tjsonGetObjectValueBigInt(end, &win.ekey);
if (win.skey >= win.ekey) {
win.ekey = win.skey + 1;
}
if (taosArrayPush(pWindows, &win) == NULL) return TSDB_CODE_OUT_OF_BUFFER;
}
int32_t numOfWins = taosArrayGetSize(pWindows);
qDebug("anomaly window recevied, total:%d", numOfWins);
for (int32_t i = 0; i < numOfWins; ++i) {
STimeWindow* pWindow = taosArrayGet(pWindows, i);
qDebug("anomaly win:%d [%" PRId64 ", %" PRId64 ")", i, pWindow->skey, pWindow->ekey);
}
return 0;
}
static int32_t anomalyAnalysisWindow(SOperatorInfo* pOperator) {
SAnomalyWindowOperatorInfo* pInfo = pOperator->info;
SAnomalyWindowSupp* pSupp = &pInfo->anomalySup;
SJson* pJson = NULL;
SAnalBuf analBuf = {.bufType = ANAL_BUF_TYPE_JSON};
char dataBuf[64] = {0};
int32_t code = 0;
int64_t ts = 0;
// int64_t ts = taosGetTimestampMs();
snprintf(analBuf.fileName, sizeof(analBuf.fileName), "%s/tdengine-anomaly-%" PRId64 "-%" PRId64, tsTempDir, ts,
pSupp->groupId);
code = tsosAnalBufOpen(&analBuf, 2);
if (code != 0) goto _OVER;
const char* prec = TSDB_TIME_PRECISION_MILLI_STR;
if (pInfo->anomalyCol.precision == TSDB_TIME_PRECISION_MICRO) prec = TSDB_TIME_PRECISION_MICRO_STR;
if (pInfo->anomalyCol.precision == TSDB_TIME_PRECISION_NANO) prec = TSDB_TIME_PRECISION_NANO_STR;
code = taosAnalBufWriteOptStr(&analBuf, "algo", pInfo->algoName);
if (code != 0) goto _OVER;
code = taosAnalBufWriteOptStr(&analBuf, "prec", prec);
if (code != 0) goto _OVER;
code = taosAnalBufWriteColMeta(&analBuf, 0, TSDB_DATA_TYPE_TIMESTAMP, "ts");
if (code != 0) goto _OVER;
code = taosAnalBufWriteColMeta(&analBuf, 1, pInfo->anomalyCol.type, "val");
if (code != 0) goto _OVER;
code = taosAnalBufWriteDataBegin(&analBuf);
if (code != 0) goto _OVER;
int32_t numOfBlocks = (int32_t)taosArrayGetSize(pSupp->blocks);
// timestamp
code = taosAnalBufWriteColBegin(&analBuf, 0);
if (code != 0) goto _OVER;
for (int32_t i = 0; i < numOfBlocks; ++i) {
SSDataBlock* pBlock = taosArrayGetP(pSupp->blocks, i);
if (pBlock == NULL) break;
SColumnInfoData* pTsCol = taosArrayGet(pBlock->pDataBlock, pInfo->tsSlotId);
if (pTsCol == NULL) break;
for (int32_t j = 0; j < pBlock->info.rows; ++j) {
code = taosAnalBufWriteColData(&analBuf, 0, TSDB_DATA_TYPE_TIMESTAMP, &((TSKEY*)pTsCol->pData)[j]);
if (code != 0) goto _OVER;
}
}
code = taosAnalBufWriteColEnd(&analBuf, 0);
if (code != 0) goto _OVER;
// data
code = taosAnalBufWriteColBegin(&analBuf, 1);
if (code != 0) goto _OVER;
for (int32_t i = 0; i < numOfBlocks; ++i) {
SSDataBlock* pBlock = taosArrayGetP(pSupp->blocks, i);
if (pBlock == NULL) break;
SColumnInfoData* pValCol = taosArrayGet(pBlock->pDataBlock, pInfo->anomalyCol.slotId);
if (pValCol == NULL) break;
for (int32_t j = 0; j < pBlock->info.rows; ++j) {
code = taosAnalBufWriteColData(&analBuf, 1, pValCol->info.type, colDataGetData(pValCol, j));
if (code != 0) goto _OVER;
if (code != 0) goto _OVER;
}
}
code = taosAnalBufWriteColEnd(&analBuf, 1);
if (code != 0) goto _OVER;
code = taosAnalBufWriteDataEnd(&analBuf);
if (code != 0) goto _OVER;
code = taosAnalBufWriteOptStr(&analBuf, "option", pInfo->anomalyOpt);
if (code != 0) goto _OVER;
code = taosAnalBufClose(&analBuf);
if (code != 0) goto _OVER;
pJson = taosAnalSendReqRetJson(pInfo->algoUrl, ANAL_HTTP_TYPE_POST, &analBuf);
if (pJson == NULL) {
code = terrno;
goto _OVER;
}
code = anomalyParseJson(pJson, pSupp->windows);
if (code != 0) goto _OVER;
_OVER:
if (code != 0) {
qError("failed to analysis window since %s", tstrerror(code));
}
taosAnalBufDestroy(&analBuf);
if (pJson != NULL) tjsonDelete(pJson);
return code;
}
static void anomalyAggregateRows(SOperatorInfo* pOperator, SSDataBlock* pBlock) {
SAnomalyWindowOperatorInfo* pInfo = pOperator->info;
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
SExprSupp* pExprSup = &pOperator->exprSupp;
SAnomalyWindowSupp* pSupp = &pInfo->anomalySup;
SWindowRowsSup* pRowSup = &pInfo->anomalyWinRowSup;
SResultRow* pResRow = pSupp->pResultRow;
int32_t numOfOutput = pOperator->exprSupp.numOfExprs;
if (setResultRowInitCtx(pResRow, pExprSup->pCtx, pExprSup->numOfExprs, pExprSup->rowEntryInfoOffset) == 0) {
updateTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pSupp->curWin, 0);
applyAggFunctionOnPartialTuples(pTaskInfo, pExprSup->pCtx, &pInfo->twAggSup.timeWindowData, pRowSup->startRowIndex,
pRowSup->numOfRows, pBlock->info.rows, numOfOutput);
}
}
static void anomalyBuildResult(SOperatorInfo* pOperator) {
SAnomalyWindowOperatorInfo* pInfo = pOperator->info;
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
SExprSupp* pExprSup = &pOperator->exprSupp;
SSDataBlock* pRes = pInfo->binfo.pRes;
SResultRow* pResRow = pInfo->anomalySup.pResultRow;
doUpdateNumOfRows(pExprSup->pCtx, pResRow, pExprSup->numOfExprs, pExprSup->rowEntryInfoOffset);
copyResultrowToDataBlock(pExprSup->pExprInfo, pExprSup->numOfExprs, pResRow, pExprSup->pCtx, pRes,
pExprSup->rowEntryInfoOffset, pTaskInfo);
pRes->info.rows += pResRow->numOfRows;
clearResultRowInitFlag(pExprSup->pCtx, pExprSup->numOfExprs);
}
static void anomalyAggregateBlocks(SOperatorInfo* pOperator) {
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
SAnomalyWindowOperatorInfo* pInfo = pOperator->info;
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
SExprSupp* pExprSup = &pOperator->exprSupp;
SSDataBlock* pRes = pInfo->binfo.pRes;
SAnomalyWindowSupp* pSupp = &pInfo->anomalySup;
SWindowRowsSup* pRowSup = &pInfo->anomalyWinRowSup;
SResultRow* pResRow = pSupp->pResultRow;
int32_t numOfOutput = pOperator->exprSupp.numOfExprs;
int32_t rowsInWin = 0;
int32_t rowsInBlock = 0;
const int64_t gid = pSupp->groupId;
const int32_t order = pInfo->binfo.inputTsOrder;
int32_t numOfBlocks = (int32_t)taosArrayGetSize(pSupp->blocks);
if (numOfBlocks == 0) goto _OVER;
qDebug("group:%" PRId64 ", aggregate blocks, blocks:%d", pSupp->groupId, numOfBlocks);
pRes->info.id.groupId = pSupp->groupId;
code = anomalyAnalysisWindow(pOperator);
QUERY_CHECK_CODE(code, lino, _OVER);
int32_t numOfWins = taosArrayGetSize(pSupp->windows);
qDebug("group:%" PRId64 ", wins:%d, rows:%" PRId64, pSupp->groupId, numOfWins, pSupp->numOfRows);
for (int32_t w = 0; w < numOfWins; ++w) {
STimeWindow* pWindow = taosArrayGet(pSupp->windows, w);
if (w == 0) {
pSupp->curWin = *pWindow;
pRowSup->win.skey = pSupp->curWin.skey;
}
qDebug("group:%" PRId64 ", win:%d [%" PRId64 ", %" PRId64 ")", pSupp->groupId, w, pWindow->skey, pWindow->ekey);
}
if (numOfWins <= 0) goto _OVER;
if (numOfWins > pRes->info.capacity) {
code = blockDataEnsureCapacity(pRes, numOfWins);
QUERY_CHECK_CODE(code, lino, _OVER);
}
for (int32_t b = 0; b < numOfBlocks; ++b) {
SSDataBlock* pBlock = taosArrayGetP(pSupp->blocks, b);
if (pBlock == NULL) break;
pRes->info.scanFlag = pBlock->info.scanFlag;
code = setInputDataBlock(pExprSup, pBlock, order, MAIN_SCAN, true);
if (code != 0) break;
code = blockDataUpdateTsWindow(pBlock, pInfo->tsSlotId);
if (code != 0) break;
// there is an scalar expression that needs to be calculated right before apply the group aggregation.
if (pInfo->scalarSup.pExprInfo != NULL) {
code = projectApplyFunctions(pInfo->scalarSup.pExprInfo, pBlock, pBlock, pInfo->scalarSup.pCtx,
pInfo->scalarSup.numOfExprs, NULL);
if (code != 0) break;
}
SColumnInfoData* pValCol = taosArrayGet(pBlock->pDataBlock, pInfo->anomalyCol.slotId);
if (pValCol == NULL) break;
SColumnInfoData* pTsCol = taosArrayGet(pBlock->pDataBlock, pInfo->tsSlotId);
if (pTsCol == NULL) break;
TSKEY* tsList = (TSKEY*)pTsCol->pData;
bool lastBlock = (b == numOfBlocks - 1);
qTrace("group:%" PRId64 ", block:%d win:%d, riwin:%d riblock:%d, rows:%" PRId64, pSupp->groupId, b,
pSupp->curWinIndex, rowsInWin, rowsInBlock, pBlock->info.rows);
for (int32_t r = 0; r < pBlock->info.rows; ++r) {
TSKEY key = tsList[r];
bool keyInWin = (key >= pSupp->curWin.skey && key < pSupp->curWin.ekey);
bool lastRow = (r == pBlock->info.rows - 1);
if (keyInWin) {
if (r < 5) {
qTrace("group:%" PRId64 ", block:%d win:%d, row:%d ts:%" PRId64 ", riwin:%d riblock:%d", pSupp->groupId, b,
pSupp->curWinIndex, r, key, rowsInWin, rowsInBlock);
}
if (rowsInBlock == 0) {
doKeepNewWindowStartInfo(pRowSup, tsList, r, gid);
}
doKeepTuple(pRowSup, tsList[r], gid);
rowsInBlock++;
rowsInWin++;
} else {
if (rowsInBlock > 0) {
qTrace("group:%" PRId64 ", block:%d win:%d, row:%d ts:%" PRId64 ", riwin:%d riblock:%d, agg", pSupp->groupId,
b, pSupp->curWinIndex, r, key, rowsInWin, rowsInBlock);
anomalyAggregateRows(pOperator, pBlock);
rowsInBlock = 0;
}
if (rowsInWin > 0) {
qTrace("group:%" PRId64 ", block:%d win:%d, row:%d ts:%" PRId64 ", riwin:%d riblock:%d, build result",
pSupp->groupId, b, pSupp->curWinIndex, r, key, rowsInWin, rowsInBlock);
anomalyBuildResult(pOperator);
rowsInWin = 0;
}
if (anomalyFindWindow(pSupp, tsList[r]) == 0) {
qTrace("group:%" PRId64 ", block:%d win:%d, row:%d ts:%" PRId64 ", riwin:%d riblock:%d, new window detect",
pSupp->groupId, b, pSupp->curWinIndex, r, key, rowsInWin, rowsInBlock);
doKeepNewWindowStartInfo(pRowSup, tsList, r, gid);
doKeepTuple(pRowSup, tsList[r], gid);
rowsInBlock = 1;
rowsInWin = 1;
} else {
qTrace("group:%" PRId64 ", block:%d win:%d, row:%d ts:%" PRId64 ", riwin:%d riblock:%d, window not found",
pSupp->groupId, b, pSupp->curWinIndex, r, key, rowsInWin, rowsInBlock);
rowsInBlock = 0;
rowsInWin = 0;
}
}
if (lastRow && rowsInBlock > 0) {
qTrace("group:%" PRId64 ", block:%d win:%d, row:%d ts:%" PRId64 ", riwin:%d riblock:%d, agg since lastrow",
pSupp->groupId, b, pSupp->curWinIndex, r, key, rowsInWin, rowsInBlock);
anomalyAggregateRows(pOperator, pBlock);
rowsInBlock = 0;
}
}
if (lastBlock && rowsInWin > 0) {
qTrace("group:%" PRId64 ", block:%d win:%d, riwin:%d riblock:%d, build result since lastblock", pSupp->groupId, b,
pSupp->curWinIndex, rowsInWin, rowsInBlock);
anomalyBuildResult(pOperator);
rowsInWin = 0;
}
}
code = doFilter(pRes, pOperator->exprSupp.pFilterInfo, NULL);
QUERY_CHECK_CODE(code, lino, _OVER);
_OVER:
for (int32_t i = 0; i < numOfBlocks; ++i) {
SSDataBlock* pBlock = taosArrayGetP(pSupp->blocks, i);
qDebug("%s, clear block, pBlock:%p pBlock->pDataBlock:%p", __func__, pBlock, pBlock->pDataBlock);
blockDataDestroy(pBlock);
}
taosArrayClear(pSupp->blocks);
taosArrayClear(pSupp->windows);
pSupp->numOfRows = 0;
pSupp->curWin.ekey = 0;
pSupp->curWin.skey = 0;
pSupp->curWinIndex = 0;
}
#else
int32_t createAnomalywindowOperatorInfo(SOperatorInfo* downstream, SPhysiNode* physiNode, SExecTaskInfo* pTaskInfo,
SOperatorInfo** pOptrInfo) {
return TSDB_CODE_OPS_NOT_SUPPORT;
}
void destroyForecastInfo(void* param) {}
#endif

10
source/libs/executor/src/executil.c
View File

@ -1794,9 +1794,13 @@ int32_t createExprFromOneNode(SExprInfo* pExp, SNode* pNode, int16_t slotId) {
pExp->pExpr->nodeType = QUERY_NODE_FUNCTION;
SFunctionNode* pFuncNode = (SFunctionNode*)pNode;
SDataType* pType = &pFuncNode->node.resType;
pExp->base.resSchema =
createResSchema(pType->type, pType->bytes, slotId, pType->scale, pType->precision, pFuncNode->node.aliasName);
SDataType* pType = &pFuncNode->node.resType;
const char* pName = pFuncNode->node.aliasName;
if (pFuncNode->funcType == FUNCTION_TYPE_FORECAST_LOW || pFuncNode->funcType == FUNCTION_TYPE_FORECAST_HIGH ||
pFuncNode->funcType == FUNCTION_TYPE_FORECAST_ROWTS) {
pName = pFuncNode->functionName;
}
pExp->base.resSchema = createResSchema(pType->type, pType->bytes, slotId, pType->scale, pType->precision, pName);
tExprNode* pExprNode = pExp->pExpr;

4
source/libs/executor/src/executor.c
View File

@ -1635,7 +1635,9 @@ int32_t getTableListInfo(const SExecTaskInfo* pTaskInfo, SArray** pList) {
int32_t qStreamOperatorReleaseState(qTaskInfo_t tInfo) {
SExecTaskInfo* pTaskInfo = (SExecTaskInfo*)tInfo;
pTaskInfo->pRoot->fpSet.releaseStreamStateFn(pTaskInfo->pRoot);
if (pTaskInfo->pRoot->fpSet.releaseStreamStateFn != NULL) {
pTaskInfo->pRoot->fpSet.releaseStreamStateFn(pTaskInfo->pRoot);
}
return 0;
}

663
source/libs/executor/src/forecastoperator.c Normal file
View File

@ -0,0 +1,663 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include "executorInt.h"
#include "filter.h"
#include "function.h"
#include "functionMgt.h"
#include "operator.h"
#include "querytask.h"
#include "storageapi.h"
#include "tanal.h"
#include "tcommon.h"
#include "tcompare.h"
#include "tdatablock.h"
#include "tfill.h"
#include "ttime.h"
#ifdef USE_ANAL
typedef struct {
char algoName[TSDB_ANAL_ALGO_NAME_LEN];
char algoUrl[TSDB_ANAL_ALGO_URL_LEN];
char algoOpt[TSDB_ANAL_ALGO_OPTION_LEN];
int64_t maxTs;
int64_t minTs;
int64_t numOfRows;
uint64_t groupId;
int32_t numOfBlocks;
int32_t optRows;
int16_t resTsSlot;
int16_t resValSlot;
int16_t resLowSlot;
int16_t resHighSlot;
int16_t inputTsSlot;
int16_t inputValSlot;
int8_t inputValType;
int8_t inputPrecision;
SAnalBuf analBuf;
} SForecastSupp;
typedef struct SForecastOperatorInfo {
SSDataBlock* pRes;
SExprSupp scalarSup; // scalar calculation
SForecastSupp forecastSupp;
} SForecastOperatorInfo;
static void destroyForecastInfo(void* param);
static FORCE_INLINE int32_t forecastEnsureBlockCapacity(SSDataBlock* pBlock, int32_t newRowsNum) {
if (pBlock->info.rows < pBlock->info.capacity) {
return TSDB_CODE_SUCCESS;
}
int32_t code = blockDataEnsureCapacity(pBlock, newRowsNum);
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, __LINE__, tstrerror(code));
return code;
}
return TSDB_CODE_SUCCESS;
}
static int32_t forecastCacheBlock(SForecastSupp* pSupp, SSDataBlock* pBlock) {
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
SAnalBuf* pBuf = &pSupp->analBuf;
qDebug("block:%d, %p rows:%" PRId64, pSupp->numOfBlocks, pBlock, pBlock->info.rows);
pSupp->numOfBlocks++;
for (int32_t j = 0; j < pBlock->info.rows; ++j) {
SColumnInfoData* pValCol = taosArrayGet(pBlock->pDataBlock, pSupp->inputValSlot);
SColumnInfoData* pTsCol = taosArrayGet(pBlock->pDataBlock, pSupp->inputTsSlot);
if (pTsCol == NULL || pValCol == NULL) break;
int64_t ts = ((TSKEY*)pTsCol->pData)[j];
char* val = colDataGetData(pValCol, j);
int16_t valType = pValCol->info.type;
pSupp->minTs = MIN(pSupp->minTs, ts);
pSupp->maxTs = MAX(pSupp->maxTs, ts);
pSupp->numOfRows++;
code = taosAnalBufWriteColData(pBuf, 0, TSDB_DATA_TYPE_TIMESTAMP, &ts);
if (TSDB_CODE_SUCCESS != code) return code;
code = taosAnalBufWriteColData(pBuf, 1, valType, val);
if (TSDB_CODE_SUCCESS != code) return code;
}
return 0;
}
static int32_t forecastCloseBuf(SForecastSupp* pSupp) {
SAnalBuf* pBuf = &pSupp->analBuf;
int32_t code = 0;
for (int32_t i = 0; i < 2; ++i) {
code = taosAnalBufWriteColEnd(pBuf, i);
if (code != 0) return code;
}
code = taosAnalBufWriteDataEnd(pBuf);
if (code != 0) return code;
int32_t len = strlen(pSupp->algoOpt);
int64_t every = (pSupp->maxTs - pSupp->minTs) / (pSupp->numOfRows + 1);
int64_t start = pSupp->maxTs + every;
bool hasStart = taosAnalGetOptStr(pSupp->algoOpt, "start", NULL, 0);
if (!hasStart) {
qDebug("forecast start not found from %s, use %" PRId64, pSupp->algoOpt, start);
code = taosAnalBufWriteOptInt(pBuf, "start", start);
if (code != 0) return code;
}
bool hasEvery = taosAnalGetOptStr(pSupp->algoOpt, "every", NULL, 0);
if (!hasEvery) {
qDebug("forecast every not found from %s, use %" PRId64, pSupp->algoOpt, every);
code = taosAnalBufWriteOptInt(pBuf, "every", every);
if (code != 0) return code;
}
code = taosAnalBufWriteOptStr(pBuf, "option", pSupp->algoOpt);
if (code != 0) return code;
code = taosAnalBufClose(pBuf);
return code;
}
static int32_t forecastAnalysis(SForecastSupp* pSupp, SSDataBlock* pBlock) {
SAnalBuf* pBuf = &pSupp->analBuf;
int32_t resCurRow = pBlock->info.rows;
int8_t tmpI8;
int16_t tmpI16;
int32_t tmpI32;
int64_t tmpI64;
float tmpFloat;
double tmpDouble;
int32_t code = 0;
SColumnInfoData* pResValCol = taosArrayGet(pBlock->pDataBlock, pSupp->resValSlot);
if (NULL == pResValCol) return TSDB_CODE_OUT_OF_RANGE;
SColumnInfoData* pResTsCol = (pSupp->resTsSlot != -1 ? taosArrayGet(pBlock->pDataBlock, pSupp->resTsSlot) : NULL);
SColumnInfoData* pResLowCol = (pSupp->resLowSlot != -1 ? taosArrayGet(pBlock->pDataBlock, pSupp->resLowSlot) : NULL);
SColumnInfoData* pResHighCol =
(pSupp->resHighSlot != -1 ? taosArrayGet(pBlock->pDataBlock, pSupp->resHighSlot) : NULL);
SJson* pJson = taosAnalSendReqRetJson(pSupp->algoUrl, ANAL_HTTP_TYPE_POST, pBuf);
if (pJson == NULL) return terrno;
int32_t rows = 0;
tjsonGetInt32ValueFromDouble(pJson, "rows", rows, code);
if (code < 0) goto _OVER;
if (rows <= 0) goto _OVER;
SJson* res = tjsonGetObjectItem(pJson, "res");
if (res == NULL) goto _OVER;
int32_t ressize = tjsonGetArraySize(res);
bool returnConf = (pSupp->resHighSlot != -1 || pSupp->resLowSlot != -1);
if (returnConf) {
if (ressize != 4) goto _OVER;
} else if (ressize != 2) {
goto _OVER;
}
if (pResTsCol != NULL) {
resCurRow = pBlock->info.rows;
SJson* tsJsonArray = tjsonGetArrayItem(res, 0);
if (tsJsonArray == NULL) goto _OVER;
int32_t tsSize = tjsonGetArraySize(tsJsonArray);
if (tsSize != rows) goto _OVER;
for (int32_t i = 0; i < tsSize; ++i) {
SJson* tsJson = tjsonGetArrayItem(tsJsonArray, i);
tjsonGetObjectValueBigInt(tsJson, &tmpI64);
colDataSetInt64(pResTsCol, resCurRow, &tmpI64);
resCurRow++;
}
}
if (pResLowCol != NULL) {
resCurRow = pBlock->info.rows;
SJson* lowJsonArray = tjsonGetArrayItem(res, 2);
if (lowJsonArray == NULL) goto _OVER;
int32_t lowSize = tjsonGetArraySize(lowJsonArray);
if (lowSize != rows) goto _OVER;
for (int32_t i = 0; i < lowSize; ++i) {
SJson* lowJson = tjsonGetArrayItem(lowJsonArray, i);
tjsonGetObjectValueDouble(lowJson, &tmpDouble);
tmpFloat = (float)tmpDouble;
colDataSetFloat(pResLowCol, resCurRow, &tmpFloat);
resCurRow++;
}
}
if (pResHighCol != NULL) {
resCurRow = pBlock->info.rows;
SJson* highJsonArray = tjsonGetArrayItem(res, 3);
if (highJsonArray == NULL) goto _OVER;
int32_t highSize = tjsonGetArraySize(highJsonArray);
if (highSize != rows) goto _OVER;
for (int32_t i = 0; i < highSize; ++i) {
SJson* highJson = tjsonGetArrayItem(highJsonArray, i);
tjsonGetObjectValueDouble(highJson, &tmpDouble);
tmpFloat = (float)tmpDouble;
colDataSetFloat(pResHighCol, resCurRow, &tmpFloat);
resCurRow++;
}
}
resCurRow = pBlock->info.rows;
SJson* valJsonArray = tjsonGetArrayItem(res, 1);
if (valJsonArray == NULL) goto _OVER;
int32_t valSize = tjsonGetArraySize(valJsonArray);
if (valSize != rows) goto _OVER;
for (int32_t i = 0; i < valSize; ++i) {
SJson* valJson = tjsonGetArrayItem(valJsonArray, i);
tjsonGetObjectValueDouble(valJson, &tmpDouble);
switch (pSupp->inputValType) {
case TSDB_DATA_TYPE_BOOL:
case TSDB_DATA_TYPE_UTINYINT:
case TSDB_DATA_TYPE_TINYINT: {
tmpI8 = (int8_t)tmpDouble;
colDataSetInt8(pResValCol, resCurRow, &tmpI8);
break;
}
case TSDB_DATA_TYPE_USMALLINT:
case TSDB_DATA_TYPE_SMALLINT: {
tmpI16 = (int16_t)tmpDouble;
colDataSetInt16(pResValCol, resCurRow, &tmpI16);
break;
}
case TSDB_DATA_TYPE_INT:
case TSDB_DATA_TYPE_UINT: {
tmpI32 = (int32_t)tmpDouble;
colDataSetInt32(pResValCol, resCurRow, &tmpI32);
break;
}
case TSDB_DATA_TYPE_TIMESTAMP:
case TSDB_DATA_TYPE_UBIGINT:
case TSDB_DATA_TYPE_BIGINT: {
tmpI64 = (int64_t)tmpDouble;
colDataSetInt64(pResValCol, resCurRow, &tmpI64);
break;
}
case TSDB_DATA_TYPE_FLOAT: {
tmpFloat = (float)tmpDouble;
colDataSetFloat(pResValCol, resCurRow, &tmpFloat);
break;
}
case TSDB_DATA_TYPE_DOUBLE: {
colDataSetDouble(pResValCol, resCurRow, &tmpDouble);
break;
}
default:
code = TSDB_CODE_FUNC_FUNTION_PARA_TYPE;
goto _OVER;
}
resCurRow++;
}
// for (int32_t i = rows; i < pSupp->optRows; ++i) {
// colDataSetNNULL(pResValCol, rows, (pSupp->optRows - rows));
// if (pResTsCol != NULL) {
// colDataSetNNULL(pResTsCol, rows, (pSupp->optRows - rows));
// }
// if (pResLowCol != NULL) {
// colDataSetNNULL(pResLowCol, rows, (pSupp->optRows - rows));
// }
// if (pResHighCol != NULL) {
// colDataSetNNULL(pResHighCol, rows, (pSupp->optRows - rows));
// }
// }
// if (rows == pSupp->optRows) {
// pResValCol->hasNull = false;
// }
pBlock->info.rows += rows;
if (pJson != NULL) tjsonDelete(pJson);
return 0;
_OVER:
if (pJson != NULL) tjsonDelete(pJson);
if (code == 0) {
code = TSDB_CODE_INVALID_JSON_FORMAT;
}
qError("failed to perform forecast finalize since %s", tstrerror(code));
return TSDB_CODE_INVALID_JSON_FORMAT;
}
static int32_t forecastAggregateBlocks(SForecastSupp* pSupp, SSDataBlock* pResBlock) {
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
SAnalBuf* pBuf = &pSupp->analBuf;
code = forecastCloseBuf(pSupp);
QUERY_CHECK_CODE(code, lino, _end);
code = forecastEnsureBlockCapacity(pResBlock, 1);
QUERY_CHECK_CODE(code, lino, _end);
code = forecastAnalysis(pSupp, pResBlock);
QUERY_CHECK_CODE(code, lino, _end);
uInfo("block:%d, forecast finalize", pSupp->numOfBlocks);
_end:
pSupp->numOfBlocks = 0;
taosAnalBufDestroy(&pSupp->analBuf);
return code;
}
static int32_t forecastNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
SForecastOperatorInfo* pInfo = pOperator->info;
SSDataBlock* pResBlock = pInfo->pRes;
SForecastSupp* pSupp = &pInfo->forecastSupp;
SAnalBuf* pBuf = &pSupp->analBuf;
int64_t st = taosGetTimestampUs();
int32_t numOfBlocks = pSupp->numOfBlocks;
blockDataCleanup(pResBlock);
while (1) {
SSDataBlock* pBlock = getNextBlockFromDownstream(pOperator, 0);
if (pBlock == NULL) {
break;
}
if (pSupp->groupId == 0 || pSupp->groupId == pBlock->info.id.groupId) {
pSupp->groupId = pBlock->info.id.groupId;
numOfBlocks++;
qDebug("group:%" PRId64 ", blocks:%d, cache block rows:%" PRId64, pSupp->groupId, numOfBlocks, pBlock->info.rows);
code = forecastCacheBlock(pSupp, pBlock);
QUERY_CHECK_CODE(code, lino, _end);
} else {
qDebug("group:%" PRId64 ", read finish for new group coming, blocks:%d", pSupp->groupId, numOfBlocks);
forecastAggregateBlocks(pSupp, pResBlock);
pSupp->groupId = pBlock->info.id.groupId;
numOfBlocks = 1;
qDebug("group:%" PRId64 ", new group, cache block rows:%" PRId64, pSupp->groupId, pBlock->info.rows);
code = forecastCacheBlock(pSupp, pBlock);
QUERY_CHECK_CODE(code, lino, _end);
}
if (pResBlock->info.rows > 0) {
(*ppRes) = pResBlock;
qDebug("group:%" PRId64 ", return to upstream, blocks:%d", pResBlock->info.id.groupId, numOfBlocks);
return code;
}
}
if (numOfBlocks > 0) {
qDebug("group:%" PRId64 ", read finish, blocks:%d", pSupp->groupId, numOfBlocks);
forecastAggregateBlocks(pSupp, pResBlock);
}
int64_t cost = taosGetTimestampUs() - st;
qDebug("all groups finished, cost:%" PRId64 "us", cost);
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
(*ppRes) = (pResBlock->info.rows == 0) ? NULL : pResBlock;
return code;
}
static int32_t forecastParseOutput(SForecastSupp* pSupp, SExprSupp* pExprSup) {
pSupp->resLowSlot = -1;
pSupp->resHighSlot = -1;
pSupp->resTsSlot = -1;
pSupp->resValSlot = -1;
for (int32_t j = 0; j < pExprSup->numOfExprs; ++j) {
SExprInfo* pExprInfo = &pExprSup->pExprInfo[j];
int32_t dstSlot = pExprInfo->base.resSchema.slotId;
if (pExprInfo->pExpr->_function.functionType == FUNCTION_TYPE_FORECAST) {
pSupp->resValSlot = dstSlot;
} else if (pExprInfo->pExpr->_function.functionType == FUNCTION_TYPE_FORECAST_ROWTS) {
pSupp->resTsSlot = dstSlot;
} else if (pExprInfo->pExpr->_function.functionType == FUNCTION_TYPE_FORECAST_LOW) {
pSupp->resLowSlot = dstSlot;
} else if (pExprInfo->pExpr->_function.functionType == FUNCTION_TYPE_FORECAST_HIGH) {
pSupp->resHighSlot = dstSlot;
} else {
}
}
return 0;
}
static int32_t forecastParseInput(SForecastSupp* pSupp, SNodeList* pFuncs) {
SNode* pNode = NULL;
pSupp->inputTsSlot = -1;
pSupp->inputValSlot = -1;
pSupp->inputValType = -1;
pSupp->inputPrecision = -1;
FOREACH(pNode, pFuncs) {
if ((nodeType(pNode) == QUERY_NODE_TARGET) && (nodeType(((STargetNode*)pNode)->pExpr) == QUERY_NODE_FUNCTION)) {
SFunctionNode* pFunc = (SFunctionNode*)((STargetNode*)pNode)->pExpr;
int32_t numOfParam = LIST_LENGTH(pFunc->pParameterList);
if (pFunc->funcType == FUNCTION_TYPE_FORECAST) {
if (numOfParam == 3) {
SNode* p1 = nodesListGetNode(pFunc->pParameterList, 0);
SNode* p2 = nodesListGetNode(pFunc->pParameterList, 1);
SNode* p3 = nodesListGetNode(pFunc->pParameterList, 2);
if (p1 == NULL || p2 == NULL || p3 == NULL) return TSDB_CODE_PLAN_INTERNAL_ERROR;
if (p1->type != QUERY_NODE_COLUMN) return TSDB_CODE_PLAN_INTERNAL_ERROR;
if (p2->type != QUERY_NODE_VALUE) return TSDB_CODE_PLAN_INTERNAL_ERROR;
if (p3->type != QUERY_NODE_COLUMN) return TSDB_CODE_PLAN_INTERNAL_ERROR;
SColumnNode* pValNode = (SColumnNode*)p1;
SValueNode* pOptNode = (SValueNode*)p2;
SColumnNode* pTsNode = (SColumnNode*)p3;
pSupp->inputTsSlot = pTsNode->slotId;
pSupp->inputPrecision = pTsNode->node.resType.precision;
pSupp->inputValSlot = pValNode->slotId;
pSupp->inputValType = pValNode->node.resType.type;
tstrncpy(pSupp->algoOpt, pOptNode->literal, sizeof(pSupp->algoOpt));
} else if (numOfParam == 2) {
SNode* p1 = nodesListGetNode(pFunc->pParameterList, 0);
SNode* p2 = nodesListGetNode(pFunc->pParameterList, 1);
if (p1 == NULL || p2 == NULL) return TSDB_CODE_PLAN_INTERNAL_ERROR;
if (p1->type != QUERY_NODE_COLUMN) return TSDB_CODE_PLAN_INTERNAL_ERROR;
if (p2->type != QUERY_NODE_COLUMN) return TSDB_CODE_PLAN_INTERNAL_ERROR;
SColumnNode* pValNode = (SColumnNode*)p1;
SColumnNode* pTsNode = (SColumnNode*)p2;
pSupp->inputTsSlot = pTsNode->slotId;
pSupp->inputPrecision = pTsNode->node.resType.precision;
pSupp->inputValSlot = pValNode->slotId;
pSupp->inputValType = pValNode->node.resType.type;
tstrncpy(pSupp->algoOpt, "algo=arima", TSDB_ANAL_ALGO_OPTION_LEN);
} else {
return TSDB_CODE_PLAN_INTERNAL_ERROR;
}
}
}
}
return 0;
}
static int32_t forecastParseAlgo(SForecastSupp* pSupp) {
pSupp->maxTs = 0;
pSupp->minTs = INT64_MAX;
pSupp->numOfRows = 0;
if (!taosAnalGetOptStr(pSupp->algoOpt, "algo", pSupp->algoName, sizeof(pSupp->algoName))) {
qError("failed to get forecast algorithm name from %s", pSupp->algoOpt);
return TSDB_CODE_ANAL_ALGO_NOT_FOUND;
}
if (taosAnalGetAlgoUrl(pSupp->algoName, ANAL_ALGO_TYPE_FORECAST, pSupp->algoUrl, sizeof(pSupp->algoUrl)) != 0) {
qError("failed to get forecast algorithm url from %s", pSupp->algoName);
return TSDB_CODE_ANAL_ALGO_NOT_LOAD;
}
return 0;
}
static int32_t forecastCreateBuf(SForecastSupp* pSupp) {
SAnalBuf* pBuf = &pSupp->analBuf;
int64_t ts = 0; // taosGetTimestampMs();
pBuf->bufType = ANAL_BUF_TYPE_JSON_COL;
snprintf(pBuf->fileName, sizeof(pBuf->fileName), "%s/tdengine-forecast-%" PRId64, tsTempDir, ts);
int32_t code = tsosAnalBufOpen(pBuf, 2);
if (code != 0) goto _OVER;
code = taosAnalBufWriteOptStr(pBuf, "algo", pSupp->algoName);
if (code != 0) goto _OVER;
bool returnConf = (pSupp->resHighSlot == -1 || pSupp->resLowSlot == -1);
code = taosAnalBufWriteOptStr(pBuf, "return_conf", returnConf ? "true" : "false");
if (code != 0) goto _OVER;
bool hasAlpha = taosAnalGetOptStr(pSupp->algoOpt, "alpha", NULL, 0);
if (!hasAlpha) {
qDebug("forecast alpha not found from %s, use default:%f", pSupp->algoOpt, ANAL_FORECAST_DEFAULT_ALPHA);
code = taosAnalBufWriteOptFloat(pBuf, "alpha", ANAL_FORECAST_DEFAULT_ALPHA);
if (code != 0) goto _OVER;
}
char tmpOpt[32] = {0};
bool hasParam = taosAnalGetOptStr(pSupp->algoOpt, "param", tmpOpt, sizeof(tmpOpt));
if (!hasParam) {
qDebug("forecast param not found from %s, use default:%s", pSupp->algoOpt, ANAL_FORECAST_DEFAULT_PARAM);
code = taosAnalBufWriteOptStr(pBuf, "param", ANAL_FORECAST_DEFAULT_PARAM);
if (code != 0) goto _OVER;
}
bool hasPeriod = taosAnalGetOptInt(pSupp->algoOpt, "period", NULL);
if (!hasPeriod) {
qDebug("forecast period not found from %s, use default:%d", pSupp->algoOpt, ANAL_FORECAST_DEFAULT_PERIOD);
code = taosAnalBufWriteOptInt(pBuf, "period", ANAL_FORECAST_DEFAULT_PERIOD);
if (code != 0) goto _OVER;
}
bool hasRows = taosAnalGetOptInt(pSupp->algoOpt, "rows", &pSupp->optRows);
if (!hasRows) {
pSupp->optRows = ANAL_FORECAST_DEFAULT_ROWS;
qDebug("forecast rows not found from %s, use default:%d", pSupp->algoOpt, pSupp->optRows);
code = taosAnalBufWriteOptInt(pBuf, "forecast_rows", pSupp->optRows);
if (code != 0) goto _OVER;
}
const char* prec = TSDB_TIME_PRECISION_MILLI_STR;
if (pSupp->inputPrecision == TSDB_TIME_PRECISION_MICRO) prec = TSDB_TIME_PRECISION_MICRO_STR;
if (pSupp->inputPrecision == TSDB_TIME_PRECISION_NANO) prec = TSDB_TIME_PRECISION_NANO_STR;
code = taosAnalBufWriteOptStr(pBuf, "prec", prec);
if (code != 0) goto _OVER;
if (returnConf) {
bool hasConf = taosAnalGetOptStr(pSupp->algoOpt, "conf", NULL, 0);
if (!hasConf) {
qDebug("forecast conf not found from %s, use default:%d", pSupp->algoOpt, ANAL_FORECAST_DEFAULT_CONF);
code = taosAnalBufWriteOptInt(pBuf, "conf", ANAL_FORECAST_DEFAULT_CONF);
if (code != 0) goto _OVER;
}
}
code = taosAnalBufWriteColMeta(pBuf, 0, TSDB_DATA_TYPE_TIMESTAMP, "ts");
if (code != 0) goto _OVER;
code = taosAnalBufWriteColMeta(pBuf, 1, pSupp->inputValType, "val");
if (code != 0) goto _OVER;
code = taosAnalBufWriteDataBegin(pBuf);
if (code != 0) goto _OVER;
for (int32_t i = 0; i < 2; ++i) {
code = taosAnalBufWriteColBegin(pBuf, i);
if (code != 0) goto _OVER;
}
_OVER:
if (code != 0) {
taosAnalBufClose(pBuf);
taosAnalBufDestroy(pBuf);
}
return code;
}
int32_t createForecastOperatorInfo(SOperatorInfo* downstream, SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo,
SOperatorInfo** pOptrInfo) {
QRY_PARAM_CHECK(pOptrInfo);
int32_t code = 0;
int32_t lino = 0;
SForecastOperatorInfo* pInfo = taosMemoryCalloc(1, sizeof(SForecastOperatorInfo));
SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo));
if (pOperator == NULL || pInfo == NULL) {
code = terrno;
goto _error;
}
SForecastSupp* pSupp = &pInfo->forecastSupp;
SForecastFuncPhysiNode* pForecastPhyNode = (SForecastFuncPhysiNode*)pPhyNode;
SExprSupp* pExprSup = &pOperator->exprSupp;
int32_t numOfExprs = 0;
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pForecastPhyNode->pFuncs, NULL, &pExprInfo, &numOfExprs);
QUERY_CHECK_CODE(code, lino, _error);
code = initExprSupp(pExprSup, pExprInfo, numOfExprs, &pTaskInfo->storageAPI.functionStore);
QUERY_CHECK_CODE(code, lino, _error);
if (pForecastPhyNode->pExprs != NULL) {
int32_t num = 0;
SExprInfo* pScalarExprInfo = NULL;
code = createExprInfo(pForecastPhyNode->pExprs, NULL, &pScalarExprInfo, &num);
QUERY_CHECK_CODE(code, lino, _error);
code = initExprSupp(&pInfo->scalarSup, pScalarExprInfo, num, &pTaskInfo->storageAPI.functionStore);
QUERY_CHECK_CODE(code, lino, _error);
}
code = filterInitFromNode((SNode*)pForecastPhyNode->node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
QUERY_CHECK_CODE(code, lino, _error);
code = forecastParseInput(pSupp, pForecastPhyNode->pFuncs);
QUERY_CHECK_CODE(code, lino, _error);
code = forecastParseOutput(pSupp, pExprSup);
QUERY_CHECK_CODE(code, lino, _error);
code = forecastParseAlgo(pSupp);
QUERY_CHECK_CODE(code, lino, _error);
code = forecastCreateBuf(pSupp);
QUERY_CHECK_CODE(code, lino, _error);
initResultSizeInfo(&pOperator->resultInfo, 4096);
pInfo->pRes = createDataBlockFromDescNode(pPhyNode->pOutputDataBlockDesc);
QUERY_CHECK_NULL(pInfo->pRes, code, lino, _error, terrno);
setOperatorInfo(pOperator, "ForecastOperator", QUERY_NODE_PHYSICAL_PLAN_FORECAST_FUNC, false, OP_NOT_OPENED, pInfo,
pTaskInfo);
pOperator->fpSet = createOperatorFpSet(optrDummyOpenFn, forecastNext, NULL, destroyForecastInfo, optrDefaultBufFn,
NULL, optrDefaultGetNextExtFn, NULL);
code = blockDataEnsureCapacity(pInfo->pRes, pOperator->resultInfo.capacity);
QUERY_CHECK_CODE(code, lino, _error);
code = appendDownstream(pOperator, &downstream, 1);
QUERY_CHECK_CODE(code, lino, _error);
*pOptrInfo = pOperator;
qDebug("forecast env is initialized, option:%s", pSupp->algoOpt);
return TSDB_CODE_SUCCESS;
_error:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
}
if (pInfo != NULL) destroyForecastInfo(pInfo);
destroyOperatorAndDownstreams(pOperator, &downstream, 1);
pTaskInfo->code = code;
return code;
}
static void destroyForecastInfo(void* param) {
SForecastOperatorInfo* pInfo = (SForecastOperatorInfo*)param;
blockDataDestroy(pInfo->pRes);
pInfo->pRes = NULL;
cleanupExprSupp(&pInfo->scalarSup);
taosAnalBufDestroy(&pInfo->forecastSupp.analBuf);
taosMemoryFreeClear(param);
}
#else
int32_t createForecastOperatorInfo(SOperatorInfo* downstream, SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo,
SOperatorInfo** pOptrInfo) {
return TSDB_CODE_OPS_NOT_SUPPORT;
}
#endif

17
source/libs/executor/src/groupcacheoperator.c
View File

@ -78,14 +78,15 @@ static void logGroupCacheExecInfo(SGroupCacheOperatorInfo* pGrpCacheOperator) {
if (pGrpCacheOperator->downstreamNum <= 0 || NULL == pGrpCacheOperator->execInfo.pDownstreamBlkNum) {
return;
}
char* buf = taosMemoryMalloc(pGrpCacheOperator->downstreamNum * 32 + 100);
int32_t bufSize = pGrpCacheOperator->downstreamNum * 32 + 100;
char* buf = taosMemoryMalloc(bufSize);
if (NULL == buf) {
return;
}
int32_t offset = sprintf(buf, "groupCache exec info, downstreamBlkNum:");
int32_t offset = snprintf(buf, bufSize, "groupCache exec info, downstreamBlkNum:");
for (int32_t i = 0; i < pGrpCacheOperator->downstreamNum; ++i) {
offset += sprintf(buf + offset, " %" PRId64 , pGrpCacheOperator->execInfo.pDownstreamBlkNum[i]);
offset += snprintf(buf + offset, bufSize, " %" PRId64 , pGrpCacheOperator->execInfo.pDownstreamBlkNum[i]);
}
qDebug("%s", buf);
taosMemoryFree(buf);
@ -234,7 +235,7 @@ static int32_t acquireFdFromFileCtx(SGcFileCacheCtx* pFileCtx, int32_t fileId, S
SGroupCacheFileInfo* pTmp = taosHashGet(pFileCtx->pCacheFile, &fileId, sizeof(fileId));
if (NULL == pTmp) {
(void)sprintf(&pFileCtx->baseFilename[pFileCtx->baseNameLen], "_%d", fileId);
(void)snprintf(&pFileCtx->baseFilename[pFileCtx->baseNameLen], sizeof(pFileCtx->baseFilename) - pFileCtx->baseNameLen, "_%d", fileId);
SGroupCacheFileInfo newFile = {0};
if (taosHashPut(pFileCtx->pCacheFile, &fileId, sizeof(fileId), &newFile, sizeof(newFile))) {
@ -439,7 +440,7 @@ static FORCE_INLINE void chkRemoveVgroupCurrFile(SGcFileCacheCtx* pFileCtx, int3
#if 0
/* debug only */
sprintf(&pFileCtx->baseFilename[pFileCtx->baseNameLen], "_%d", pFileCtx->fileId);
snprintf(&pFileCtx->baseFilename[pFileCtx->baseNameLen], sizeof(pFileCtx->baseFilename) - pFileCtx->baseNameLen, "_%d", pFileCtx->fileId);
taosRemoveFile(pFileCtx->baseFilename);
/* debug only */
#endif
@ -813,7 +814,7 @@ static int32_t addFileRefTableNum(SGcFileCacheCtx* pFileCtx, int32_t fileId, int
SGroupCacheFileInfo* pTmp = taosHashGet(pFileCtx->pCacheFile, &fileId, sizeof(fileId));
if (NULL == pTmp) {
(void)sprintf(&pFileCtx->baseFilename[pFileCtx->baseNameLen], "_%u", fileId);
(void)snprintf(&pFileCtx->baseFilename[pFileCtx->baseNameLen], sizeof(pFileCtx->baseFilename) - pFileCtx->baseNameLen, "_%u", fileId);
SGroupCacheFileInfo newFile = {0};
newFile.groupNum = 1;
@ -1377,7 +1378,7 @@ static void freeRemoveGroupCacheData(void* p) {
#if 0
/* debug only */
sprintf(&pFileCtx->baseFilename[pFileCtx->baseNameLen], "_%d", pGroup->fileId);
snprintf(&pFileCtx->baseFilename[pFileCtx->baseNameLen], sizeof(pFileCtx->baseFilename) - pFileCtx->baseNameLen, "_%d", pGroup->fileId);
taosRemoveFile(pFileCtx->baseFilename);
/* debug only */
#endif

4
source/libs/executor/src/operator.c
View File

@ -619,6 +619,8 @@ int32_t createOperator(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo, SReadHand
code = createIndefinitOutputOperatorInfo(ops[0], pPhyNode, pTaskInfo, &pOptr);
} else if (QUERY_NODE_PHYSICAL_PLAN_INTERP_FUNC == type) {
code = createTimeSliceOperatorInfo(ops[0], pPhyNode, pTaskInfo, &pOptr);
} else if (QUERY_NODE_PHYSICAL_PLAN_FORECAST_FUNC == type) {
code = createForecastOperatorInfo(ops[0], pPhyNode, pTaskInfo, &pOptr);
} else if (QUERY_NODE_PHYSICAL_PLAN_MERGE_EVENT == type) {
code = createEventwindowOperatorInfo(ops[0], pPhyNode, pTaskInfo, &pOptr);
} else if (QUERY_NODE_PHYSICAL_PLAN_GROUP_CACHE == type) {
@ -629,6 +631,8 @@ int32_t createOperator(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo, SReadHand
code = createStreamCountAggOperatorInfo(ops[0], pPhyNode, pTaskInfo, pHandle, &pOptr);
} else if (QUERY_NODE_PHYSICAL_PLAN_MERGE_COUNT == type) {
code = createCountwindowOperatorInfo(ops[0], pPhyNode, pTaskInfo, &pOptr);
} else if (QUERY_NODE_PHYSICAL_PLAN_MERGE_ANOMALY == type) {
code = createAnomalywindowOperatorInfo(ops[0], pPhyNode, pTaskInfo, &pOptr);
} else {
code = TSDB_CODE_INVALID_PARA;
pTaskInfo->code = code;

2
source/libs/executor/src/streamtimewindowoperator.c
View File

@ -49,7 +49,7 @@
#define STREAM_SESSION_OP_CHECKPOINT_NAME "StreamSessionOperator_Checkpoint"
#define STREAM_STATE_OP_CHECKPOINT_NAME "StreamStateOperator_Checkpoint"
#define MAX_STREAM_HISTORY_RESULT 100000000
#define MAX_STREAM_HISTORY_RESULT 20000000
typedef struct SStateWindowInfo {
SResultWindowInfo winInfo;

90
source/libs/function/src/builtins.c
View File

@ -16,9 +16,10 @@
#include "builtins.h"
#include "builtinsimpl.h"
#include "cJSON.h"
#include "geomFunc.h"
#include "querynodes.h"
#include "scalar.h"
#include "geomFunc.h"
#include "tanal.h"
#include "taoserror.h"
#include "ttime.h"
@ -208,10 +209,10 @@ static int32_t countTrailingSpaces(const SValueNode* pVal, bool isLtrim) {
}
static int32_t addTimezoneParam(SNodeList* pList) {
char buf[6] = {0};
char buf[TD_TIME_STR_LEN] = {0};
time_t t = taosTime(NULL);
struct tm tmInfo;
if (taosLocalTime(&t, &tmInfo, buf) != NULL) {
if (taosLocalTime(&t, &tmInfo, buf, sizeof(buf)) != NULL) {
(void)strftime(buf, sizeof(buf), "%z", &tmInfo);
}
int32_t len = (int32_t)strlen(buf);
@ -2078,6 +2079,47 @@ static int32_t translateMode(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
return translateUniqueMode(pFunc, pErrBuf, len, false);
}
static int32_t translateForecast(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
int32_t numOfParams = LIST_LENGTH(pFunc->pParameterList);
if (2 != numOfParams && 1 != numOfParams) {
return invaildFuncParaNumErrMsg(pErrBuf, len, "FORECAST require 1 or 2 parameters");
}
uint8_t valType = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->type;
if (!IS_MATHABLE_TYPE(valType)) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, "FORECAST only support mathable column");
}
if (numOfParams == 2) {
uint8_t optionType = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 1))->type;
if (TSDB_DATA_TYPE_BINARY != optionType) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, "FORECAST option should be varchar");
}
SNode* pOption = nodesListGetNode(pFunc->pParameterList, 1);
if (QUERY_NODE_VALUE != nodeType(pOption)) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, "FORECAST option should be value");
}
SValueNode* pValue = (SValueNode*)pOption;
if (!taosAnalGetOptStr(pValue->literal, "algo", NULL, 0) != 0) {
return invaildFuncParaValueErrMsg(pErrBuf, len, "FORECAST option should include algo field");
}
pValue->notReserved = true;
}
pFunc->node.resType = (SDataType){.bytes = tDataTypes[valType].bytes, .type = valType};
return TSDB_CODE_SUCCESS;
}
static int32_t translateForecastConf(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
pFunc->node.resType = (SDataType){.bytes = tDataTypes[TSDB_DATA_TYPE_FLOAT].bytes, .type = TSDB_DATA_TYPE_FLOAT};
return TSDB_CODE_SUCCESS;
}
static EFuncReturnRows forecastEstReturnRows(SFunctionNode* pFunc) { return FUNC_RETURN_ROWS_N; }
static int32_t translateDiff(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
int32_t numOfParams = LIST_LENGTH(pFunc->pParameterList);
if (numOfParams > 2) {
@ -4797,6 +4839,48 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
.sprocessFunc = randFunction,
.finalizeFunc = NULL
},
{
.name = "forecast",
.type = FUNCTION_TYPE_FORECAST,
.classification = FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_IMPLICIT_TS_FUNC |
FUNC_MGT_FORBID_STREAM_FUNC | FUNC_MGT_FORBID_SYSTABLE_FUNC | FUNC_MGT_KEEP_ORDER_FUNC | FUNC_MGT_PRIMARY_KEY_FUNC,
.translateFunc = translateForecast,
.getEnvFunc = getSelectivityFuncEnv,
.initFunc = functionSetup,
.processFunc = NULL,
.finalizeFunc = NULL,
.estimateReturnRowsFunc = forecastEstReturnRows,
},
{
.name = "_frowts",
.type = FUNCTION_TYPE_FORECAST_ROWTS,
.classification = FUNC_MGT_PSEUDO_COLUMN_FUNC | FUNC_MGT_FORECAST_PC_FUNC | FUNC_MGT_KEEP_ORDER_FUNC,
.translateFunc = translateTimePseudoColumn,
.getEnvFunc = getTimePseudoFuncEnv,
.initFunc = NULL,
.sprocessFunc = NULL,
.finalizeFunc = NULL
},
{
.name = "_flow",
.type = FUNCTION_TYPE_FORECAST_LOW,
.classification = FUNC_MGT_PSEUDO_COLUMN_FUNC | FUNC_MGT_FORECAST_PC_FUNC | FUNC_MGT_KEEP_ORDER_FUNC,
.translateFunc = translateForecastConf,
.getEnvFunc = getForecastConfEnv,
.initFunc = NULL,
.sprocessFunc = NULL,
.finalizeFunc = NULL
},
{
.name = "_fhigh",
.type = FUNCTION_TYPE_FORECAST_HIGH,
.classification = FUNC_MGT_PSEUDO_COLUMN_FUNC | FUNC_MGT_FORECAST_PC_FUNC | FUNC_MGT_KEEP_ORDER_FUNC,
.translateFunc = translateForecastConf,
.getEnvFunc = getForecastConfEnv,
.initFunc = NULL,
.sprocessFunc = NULL,
.finalizeFunc = NULL
},
};
// clang-format on

6
source/libs/function/src/builtinsimpl.c
View File

@ -18,6 +18,7 @@
#include "function.h"
#include "query.h"
#include "querynodes.h"
#include "tanal.h"
#include "tcompare.h"
#include "tdatablock.h"
#include "tdigest.h"
@ -3578,6 +3579,11 @@ bool funcInputGetNextRowIndex(SInputColumnInfoData* pInput, int32_t from, bool f
}
}
bool getForecastConfEnv(SFunctionNode* UNUSED_PARAM(pFunc), SFuncExecEnv* pEnv) {
pEnv->calcMemSize = sizeof(float);
return true;
}
int32_t diffResultIsNull(SqlFunctionCtx* pCtx, SFuncInputRow* pRow){
SResultRowEntryInfo* pResInfo = GET_RES_INFO(pCtx);
SDiffInfo* pDiffInfo = GET_ROWCELL_INTERBUF(pResInfo);

9
source/libs/function/src/functionMgt.c
View File

@ -232,6 +232,15 @@ bool fmIsInterpFunc(int32_t funcId) {
bool fmIsInterpPseudoColumnFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_INTERP_PC_FUNC); }
bool fmIsForecastFunc(int32_t funcId) {
if (funcId < 0 || funcId >= funcMgtBuiltinsNum) {
return false;
}
return FUNCTION_TYPE_FORECAST == funcMgtBuiltins[funcId].type;
}
bool fmIsForecastPseudoColumnFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_FORECAST_PC_FUNC); }
bool fmIsLastRowFunc(int32_t funcId) {
if (funcId < 0 || funcId >= funcMgtBuiltinsNum) {
return false;

5
source/libs/function/src/tudf.c
View File

@ -158,11 +158,12 @@ static int32_t udfSpawnUdfd(SUdfdData *pData) {
char *taosFqdnEnvItem = NULL;
char *taosFqdn = getenv("TAOS_FQDN");
if (taosFqdn != NULL) {
int len = strlen("TAOS_FQDN=") + strlen(taosFqdn) + 1;
int subLen = strlen(taosFqdn);
int len = strlen("TAOS_FQDN=") + subLen + 1;
taosFqdnEnvItem = taosMemoryMalloc(len);
if (taosFqdnEnvItem != NULL) {
tstrncpy(taosFqdnEnvItem, "TAOS_FQDN=", len);
TAOS_STRNCAT(taosFqdnEnvItem, taosFqdn, strlen(taosFqdn));
TAOS_STRNCAT(taosFqdnEnvItem, taosFqdn, subLen);
fnInfo("[UDFD]Succsess to set TAOS_FQDN:%s", taosFqdn);
} else {
fnError("[UDFD]Failed to allocate memory for TAOS_FQDN");

21
source/libs/nodes/src/nodesCloneFuncs.c
View File

@ -368,6 +368,13 @@ static int32_t countWindowNodeCopy(const SCountWindowNode* pSrc, SCountWindowNod
return TSDB_CODE_SUCCESS;
}
static int32_t anomalyWindowNodeCopy(const SAnomalyWindowNode* pSrc, SAnomalyWindowNode* pDst) {
CLONE_NODE_FIELD(pCol);
CLONE_NODE_FIELD(pExpr);
COPY_CHAR_ARRAY_FIELD(anomalyOpt);
return TSDB_CODE_SUCCESS;
}
static int32_t sessionWindowNodeCopy(const SSessionWindowNode* pSrc, SSessionWindowNode* pDst) {
CLONE_NODE_FIELD_EX(pCol, SColumnNode*);
CLONE_NODE_FIELD_EX(pGap, SValueNode*);
@ -622,6 +629,8 @@ static int32_t logicWindowCopy(const SWindowLogicNode* pSrc, SWindowLogicNode* p
COPY_SCALAR_FIELD(windowAlgo);
COPY_SCALAR_FIELD(windowCount);
COPY_SCALAR_FIELD(windowSliding);
CLONE_NODE_FIELD(pAnomalyExpr);
COPY_CHAR_ARRAY_FIELD(anomalyOpt);
return TSDB_CODE_SUCCESS;
}
@ -674,6 +683,12 @@ static int32_t logicInterpFuncCopy(const SInterpFuncLogicNode* pSrc, SInterpFunc
return TSDB_CODE_SUCCESS;
}
static int32_t logicForecastFuncCopy(const SForecastFuncLogicNode* pSrc, SForecastFuncLogicNode* pDst) {
COPY_BASE_OBJECT_FIELD(node, logicNodeCopy);
CLONE_NODE_LIST_FIELD(pFuncs);
return TSDB_CODE_SUCCESS;
}
static int32_t logicGroupCacheCopy(const SGroupCacheLogicNode* pSrc, SGroupCacheLogicNode* pDst) {
COPY_BASE_OBJECT_FIELD(node, logicNodeCopy);
COPY_SCALAR_FIELD(grpColsMayBeNull);
@ -937,6 +952,9 @@ int32_t nodesCloneNode(const SNode* pNode, SNode** ppNode) {
case QUERY_NODE_COUNT_WINDOW:
code = countWindowNodeCopy((const SCountWindowNode*)pNode, (SCountWindowNode*)pDst);
break;
case QUERY_NODE_ANOMALY_WINDOW:
code = anomalyWindowNodeCopy((const SAnomalyWindowNode*)pNode, (SAnomalyWindowNode*)pDst);
break;
case QUERY_NODE_SESSION_WINDOW:
code = sessionWindowNodeCopy((const SSessionWindowNode*)pNode, (SSessionWindowNode*)pDst);
break;
@ -1021,6 +1039,9 @@ int32_t nodesCloneNode(const SNode* pNode, SNode** ppNode) {
case QUERY_NODE_LOGIC_PLAN_INTERP_FUNC:
code = logicInterpFuncCopy((const SInterpFuncLogicNode*)pNode, (SInterpFuncLogicNode*)pDst);
break;
case QUERY_NODE_LOGIC_PLAN_FORECAST_FUNC:
code = logicForecastFuncCopy((const SForecastFuncLogicNode*)pNode, (SForecastFuncLogicNode*)pDst);
break;
case QUERY_NODE_LOGIC_PLAN_GROUP_CACHE:
code = logicGroupCacheCopy((const SGroupCacheLogicNode*)pNode, (SGroupCacheLogicNode*)pDst);
break;

216
source/libs/nodes/src/nodesCodeFuncs.c
View File

@ -97,6 +97,8 @@ const char* nodesNodeName(ENodeType type) {
return "WindowOffset";
case QUERY_NODE_COUNT_WINDOW:
return "CountWindow";
case QUERY_NODE_ANOMALY_WINDOW:
return "AnomalyWindow";
case QUERY_NODE_SET_OPERATOR:
return "SetOperator";
case QUERY_NODE_SELECT_STMT:
@ -153,6 +155,12 @@ const char* nodesNodeName(ENodeType type) {
return "CreateQnodeStmt";
case QUERY_NODE_DROP_QNODE_STMT:
return "DropQnodeStmt";
case QUERY_NODE_CREATE_ANODE_STMT:
return "CreateAnodeStmt";
case QUERY_NODE_DROP_ANODE_STMT:
return "DropAnodeStmt";
case QUERY_NODE_UPDATE_ANODE_STMT:
return "UpdateAnodeStmt";
case QUERY_NODE_CREATE_SNODE_STMT:
return "CreateSnodeStmt";
case QUERY_NODE_DROP_SNODE_STMT:
@ -213,6 +221,10 @@ const char* nodesNodeName(ENodeType type) {
return "ShowModulesStmt";
case QUERY_NODE_SHOW_QNODES_STMT:
return "ShowQnodesStmt";
case QUERY_NODE_SHOW_ANODES_STMT:
return "ShowAnodesStmt";
case QUERY_NODE_SHOW_ANODES_FULL_STMT:
return "ShowAnodesFullStmt";
case QUERY_NODE_SHOW_SNODES_STMT:
return "ShowSnodesStmt";
case QUERY_NODE_SHOW_BNODES_STMT:
@ -328,6 +340,8 @@ const char* nodesNodeName(ENodeType type) {
return "LogicIndefRowsFunc";
case QUERY_NODE_LOGIC_PLAN_INTERP_FUNC:
return "LogicInterpFunc";
case QUERY_NODE_LOGIC_PLAN_FORECAST_FUNC:
return "LogicForecastFunc";
case QUERY_NODE_LOGIC_PLAN_GROUP_CACHE:
return "LogicGroupCache";
case QUERY_NODE_LOGIC_PLAN_DYN_QUERY_CTRL:
@ -362,6 +376,10 @@ const char* nodesNodeName(ENodeType type) {
return "PhysiMergeCountWindow";
case QUERY_NODE_PHYSICAL_PLAN_STREAM_COUNT:
return "PhysiStreamCountWindow";
case QUERY_NODE_PHYSICAL_PLAN_MERGE_ANOMALY:
return "PhysiMergeAnomalyWindow";
case QUERY_NODE_PHYSICAL_PLAN_STREAM_ANOMALY:
return "PhysiStreamAnomalyWindow";
case QUERY_NODE_PHYSICAL_PLAN_PROJECT:
return "PhysiProject";
case QUERY_NODE_PHYSICAL_PLAN_MERGE_JOIN:
@ -413,6 +431,8 @@ const char* nodesNodeName(ENodeType type) {
return "PhysiIndefRowsFunc";
case QUERY_NODE_PHYSICAL_PLAN_INTERP_FUNC:
return "PhysiInterpFunc";
case QUERY_NODE_PHYSICAL_PLAN_FORECAST_FUNC:
return "PhysiForecastFunc";
case QUERY_NODE_PHYSICAL_PLAN_DISPATCH:
return "PhysiDispatch";
case QUERY_NODE_PHYSICAL_PLAN_INSERT:
@ -1260,6 +1280,30 @@ static int32_t jsonToLogicInterpFuncNode(const SJson* pJson, void* pObj) {
return code;
}
static const char* jkForecastFuncLogicPlanFuncs = "Funcs";
static int32_t logicForecastFuncNodeToJson(const void* pObj, SJson* pJson) {
const SForecastFuncLogicNode* pNode = (const SForecastFuncLogicNode*)pObj;
int32_t code = logicPlanNodeToJson(pObj, pJson);
if (TSDB_CODE_SUCCESS == code) {
code = nodeListToJson(pJson, jkForecastFuncLogicPlanFuncs, pNode->pFuncs);
}
return code;
}
static int32_t jsonToLogicForecastFuncNode(const SJson* pJson, void* pObj) {
SForecastFuncLogicNode* pNode = (SForecastFuncLogicNode*)pObj;
int32_t code = jsonToLogicPlanNode(pJson, pObj);
if (TSDB_CODE_SUCCESS == code) {
code = jsonToNodeList(pJson, jkForecastFuncLogicPlanFuncs, &pNode->pFuncs);
}
return code;
}
static const char* jkGroupCacheLogicPlanGrpColsMayBeNull = "GroupColsMayBeNull";
static const char* jkGroupCacheLogicPlanGroupByUid = "GroupByUid";
static const char* jkGroupCacheLogicPlanGlobalGroup = "GlobalGroup";
@ -3011,6 +3055,36 @@ static int32_t jsonToPhysiCountWindowNode(const SJson* pJson, void* pObj) {
return code;
}
static const char* jkAnomalyWindowPhysiPlanAnomalyKey = "AnomalyKey";
static const char* jkAnomalyWindowPhysiPlanAnomalyOption = "AnomalyOpt";
static int32_t physiAnomalyWindowNodeToJson(const void* pObj, SJson* pJson) {
const SAnomalyWindowPhysiNode* pNode = (const SAnomalyWindowPhysiNode*)pObj;
int32_t code = physiWindowNodeToJson(pObj, pJson);
if (TSDB_CODE_SUCCESS == code) {
code = tjsonAddObject(pJson, jkAnomalyWindowPhysiPlanAnomalyKey, nodeToJson, pNode->pAnomalyKey);
}
if (TSDB_CODE_SUCCESS == code) {
code = tjsonAddStringToObject(pJson, jkAnomalyWindowPhysiPlanAnomalyOption, pNode->anomalyOpt);
}
return code;
}
static int32_t jsonToPhysiAnomalyWindowNode(const SJson* pJson, void* pObj) {
SAnomalyWindowPhysiNode* pNode = (SAnomalyWindowPhysiNode*)pObj;
int32_t code = jsonToPhysiWindowNode(pJson, pObj);
if (TSDB_CODE_SUCCESS == code) {
code = jsonToNodeObject(pJson, jkAnomalyWindowPhysiPlanAnomalyKey, &pNode->pAnomalyKey);
}
if (TSDB_CODE_SUCCESS == code) {
code = tjsonGetStringValue(pJson, jkAnomalyWindowPhysiPlanAnomalyOption, pNode->anomalyOpt);
}
return code;
}
static const char* jkPartitionPhysiPlanExprs = "Exprs";
static const char* jkPartitionPhysiPlanPartitionKeys = "PartitionKeys";
static const char* jkPartitionPhysiPlanTargets = "Targets";
@ -3198,6 +3272,37 @@ static int32_t jsonToPhysiInterpFuncNode(const SJson* pJson, void* pObj) {
return code;
}
static const char* jkForecastFuncPhysiPlanExprs = "Exprs";
static const char* jkForecastFuncPhysiPlanFuncs = "Funcs";
static int32_t physiForecastFuncNodeToJson(const void* pObj, SJson* pJson) {
const SForecastFuncPhysiNode* pNode = (const SForecastFuncPhysiNode*)pObj;
int32_t code = physicPlanNodeToJson(pObj, pJson);
if (TSDB_CODE_SUCCESS == code) {
code = nodeListToJson(pJson, jkForecastFuncPhysiPlanExprs, pNode->pExprs);
}
if (TSDB_CODE_SUCCESS == code) {
code = nodeListToJson(pJson, jkForecastFuncPhysiPlanFuncs, pNode->pFuncs);
}
return code;
}
static int32_t jsonToPhysiForecastFuncNode(const SJson* pJson, void* pObj) {
SForecastFuncPhysiNode* pNode = (SForecastFuncPhysiNode*)pObj;
int32_t code = jsonToPhysicPlanNode(pJson, pObj);
if (TSDB_CODE_SUCCESS == code) {
code = jsonToNodeList(pJson, jkForecastFuncPhysiPlanExprs, &pNode->pExprs);
}
if (TSDB_CODE_SUCCESS == code) {
code = jsonToNodeList(pJson, jkForecastFuncPhysiPlanFuncs, &pNode->pFuncs);
}
return code;
}
static const char* jkDataSinkInputDataBlockDesc = "InputDataBlockDesc";
static int32_t physicDataSinkNodeToJson(const void* pObj, SJson* pJson) {
@ -3953,9 +4058,10 @@ static int32_t datumToJson(const void* pObj, SJson* pJson) {
break;
case TSDB_DATA_TYPE_NCHAR: {
// cJSON only support utf-8 encoding. Convert memory content to hex string.
char* buf = taosMemoryCalloc(varDataLen(pNode->datum.p) * 2 + 1, sizeof(char));
int32_t bufSize = varDataLen(pNode->datum.p) * 2 + 1;
char* buf = taosMemoryCalloc(bufSize, sizeof(char));
if (!buf) return terrno;
code = taosHexEncode(varDataVal(pNode->datum.p), buf, varDataLen(pNode->datum.p));
code = taosHexEncode(varDataVal(pNode->datum.p), buf, varDataLen(pNode->datum.p), bufSize);
if (code != TSDB_CODE_SUCCESS) {
taosMemoryFree(buf);
return TSDB_CODE_TSC_INVALID_VALUE;
@ -3971,9 +4077,10 @@ static int32_t datumToJson(const void* pObj, SJson* pJson) {
break;
case TSDB_DATA_TYPE_JSON: {
int32_t len = getJsonValueLen(pNode->datum.p);
char* buf = taosMemoryCalloc(len * 2 + 1, sizeof(char));
int32_t bufSize = len * 2 + 1;
char* buf = taosMemoryCalloc(bufSize, sizeof(char));
if (!buf) return terrno;
code = taosHexEncode(pNode->datum.p, buf, len);
code = taosHexEncode(pNode->datum.p, buf, len, bufSize);
if (code != TSDB_CODE_SUCCESS) {
taosMemoryFree(buf);
return TSDB_CODE_TSC_INVALID_VALUE;
@ -4763,6 +4870,36 @@ static int32_t jsonToCountWindowNode(const SJson* pJson, void* pObj) {
return code;
}
static const char* jkAnomalyWindowTsPrimaryKey = "AnomalyTsPrimaryKey";
static const char* jkAnomalyWindowExpr = "AnomalyWindowExpr";
static const char* jkAnomalyWindowOption = "AnomalyWindowOpt";
static int32_t anomalyWindowNodeToJson(const void* pObj, SJson* pJson) {
const SAnomalyWindowNode* pNode = (const SAnomalyWindowNode*)pObj;
int32_t code = tjsonAddObject(pJson, jkAnomalyWindowTsPrimaryKey, nodeToJson, pNode->pCol);
if (TSDB_CODE_SUCCESS == code) {
code = tjsonAddObject(pJson, jkAnomalyWindowExpr, nodeToJson, pNode->pExpr);
}
if (TSDB_CODE_SUCCESS == code) {
code = tjsonAddStringToObject(pJson, jkAnomalyWindowOption, pNode->anomalyOpt);
}
return code;
}
static int32_t jsonToAnomalyWindowNode(const SJson* pJson, void* pObj) {
SAnomalyWindowNode* pNode = (SAnomalyWindowNode*)pObj;
int32_t code = jsonToNodeObject(pJson, jkAnomalyWindowTsPrimaryKey, &pNode->pCol);
if (TSDB_CODE_SUCCESS == code) {
code = jsonToNodeObject(pJson, jkAnomalyWindowExpr, (SNode**)&pNode->pExpr);
}
if (TSDB_CODE_SUCCESS == code) {
code = tjsonGetStringValue(pJson, jkAnomalyWindowOption, pNode->anomalyOpt);
}
return code;
}
static const char* jkIntervalWindowInterval = "Interval";
static const char* jkIntervalWindowOffset = "Offset";
static const char* jkIntervalWindowSliding = "Sliding";
@ -6449,6 +6586,39 @@ static int32_t dropQnodeStmtToJson(const void* pObj, SJson* pJson) { return drop
static int32_t jsonToDropQnodeStmt(const SJson* pJson, void* pObj) { return jsonToDropComponentNodeStmt(pJson, pObj); }
static const char* jkCreateAnodeStmtUrl = "Url";
static const char* jkUpdateDropANodeStmtId = "AnodeId";
static int32_t createAnodeStmtToJson(const void* pObj, SJson* pJson) {
const SCreateAnodeStmt* pNode = (const SCreateAnodeStmt*)pObj;
return tjsonAddStringToObject(pJson, jkCreateAnodeStmtUrl, pNode->url);
}
static int32_t jsonToCreateAnodeStmt(const SJson* pJson, void* pObj) {
SCreateAnodeStmt* pNode = (SCreateAnodeStmt*)pObj;
return tjsonGetStringValue(pJson, jkCreateAnodeStmtUrl, pNode->url);
}
static int32_t updateAnodeStmtToJson(const void* pObj, SJson* pJson) {
const SUpdateAnodeStmt* pNode = (const SUpdateAnodeStmt*)pObj;
return tjsonAddIntegerToObject(pJson, jkUpdateDropANodeStmtId, pNode->anodeId);
}
static int32_t jsonToUpdateAnodeStmt(const SJson* pJson, void* pObj) {
SUpdateAnodeStmt* pNode = (SUpdateAnodeStmt*)pObj;
return tjsonGetIntValue(pJson, jkUpdateDropANodeStmtId, &pNode->anodeId);
}
static int32_t dropAnodeStmtToJson(const void* pObj, SJson* pJson) {
const SDropAnodeStmt* pNode = (const SDropAnodeStmt*)pObj;
return tjsonAddIntegerToObject(pJson, jkUpdateDropANodeStmtId, pNode->anodeId);
}
static int32_t jsonToDropAnodeStmt(const SJson* pJson, void* pObj) {
SDropAnodeStmt* pNode = (SDropAnodeStmt*)pObj;
return tjsonGetIntValue(pJson, jkUpdateDropANodeStmtId, &pNode->anodeId);
}
static int32_t createSnodeStmtToJson(const void* pObj, SJson* pJson) {
return createComponentNodeStmtToJson(pObj, pJson);
}
@ -7014,6 +7184,14 @@ static int32_t showQnodesStmtToJson(const void* pObj, SJson* pJson) { return sho
static int32_t jsonToShowQnodesStmt(const SJson* pJson, void* pObj) { return jsonToShowStmt(pJson, pObj); }
static int32_t showAnodesStmtToJson(const void* pObj, SJson* pJson) { return showStmtToJson(pObj, pJson); }
static int32_t jsonToShowAnodesStmt(const SJson* pJson, void* pObj) { return jsonToShowStmt(pJson, pObj); }
static int32_t showAnodesFullStmtToJson(const void* pObj, SJson* pJson) { return showStmtToJson(pObj, pJson); }
static int32_t jsonToShowAnodesFullStmt(const SJson* pJson, void* pObj) { return jsonToShowStmt(pJson, pObj); }
static int32_t showArbGroupsStmtToJson(const void* pObj, SJson* pJson) { return showStmtToJson(pObj, pJson); }
static int32_t jsonToShowArbGroupsStmt(const SJson* pJson, void* pObj) { return jsonToShowStmt(pJson, pObj); }
@ -7550,6 +7728,8 @@ static int32_t specificNodeToJson(const void* pObj, SJson* pJson) {
return windowOffsetNodeToJson(pObj, pJson);
case QUERY_NODE_COUNT_WINDOW:
return countWindowNodeToJson(pObj, pJson);
case QUERY_NODE_ANOMALY_WINDOW:
return anomalyWindowNodeToJson(pObj, pJson);
case QUERY_NODE_SET_OPERATOR:
return setOperatorToJson(pObj, pJson);
case QUERY_NODE_SELECT_STMT:
@ -7602,6 +7782,12 @@ static int32_t specificNodeToJson(const void* pObj, SJson* pJson) {
return createQnodeStmtToJson(pObj, pJson);
case QUERY_NODE_DROP_QNODE_STMT:
return dropQnodeStmtToJson(pObj, pJson);
case QUERY_NODE_CREATE_ANODE_STMT:
return createAnodeStmtToJson(pObj, pJson);
case QUERY_NODE_DROP_ANODE_STMT:
return dropAnodeStmtToJson(pObj, pJson);
case QUERY_NODE_UPDATE_ANODE_STMT:
return updateAnodeStmtToJson(pObj, pJson);
case QUERY_NODE_CREATE_SNODE_STMT:
return createSnodeStmtToJson(pObj, pJson);
case QUERY_NODE_DROP_SNODE_STMT:
@ -7652,6 +7838,10 @@ static int32_t specificNodeToJson(const void* pObj, SJson* pJson) {
return showMnodesStmtToJson(pObj, pJson);
case QUERY_NODE_SHOW_QNODES_STMT:
return showQnodesStmtToJson(pObj, pJson);
case QUERY_NODE_SHOW_ANODES_STMT:
return showAnodesStmtToJson(pObj, pJson);
case QUERY_NODE_SHOW_ANODES_FULL_STMT:
return showAnodesFullStmtToJson(pObj, pJson);
case QUERY_NODE_SHOW_ARBGROUPS_STMT:
return showArbGroupsStmtToJson(pObj, pJson);
case QUERY_NODE_SHOW_CLUSTER_STMT:
@ -7741,6 +7931,8 @@ static int32_t specificNodeToJson(const void* pObj, SJson* pJson) {
return logicIndefRowsFuncNodeToJson(pObj, pJson);
case QUERY_NODE_LOGIC_PLAN_INTERP_FUNC:
return logicInterpFuncNodeToJson(pObj, pJson);
case QUERY_NODE_LOGIC_PLAN_FORECAST_FUNC:
return logicForecastFuncNodeToJson(pObj, pJson);
case QUERY_NODE_LOGIC_PLAN_GROUP_CACHE:
return logicGroupCacheNodeToJson(pObj, pJson);
case QUERY_NODE_LOGIC_PLAN_DYN_QUERY_CTRL:
@ -7801,6 +7993,8 @@ static int32_t specificNodeToJson(const void* pObj, SJson* pJson) {
case QUERY_NODE_PHYSICAL_PLAN_MERGE_COUNT:
case QUERY_NODE_PHYSICAL_PLAN_STREAM_COUNT:
return physiCountWindowNodeToJson(pObj, pJson);
case QUERY_NODE_PHYSICAL_PLAN_MERGE_ANOMALY:
return physiAnomalyWindowNodeToJson(pObj, pJson);
case QUERY_NODE_PHYSICAL_PLAN_PARTITION:
return physiPartitionNodeToJson(pObj, pJson);
case QUERY_NODE_PHYSICAL_PLAN_STREAM_PARTITION:
@ -7809,6 +8003,8 @@ static int32_t specificNodeToJson(const void* pObj, SJson* pJson) {
return physiIndefRowsFuncNodeToJson(pObj, pJson);
case QUERY_NODE_PHYSICAL_PLAN_INTERP_FUNC:
return physiInterpFuncNodeToJson(pObj, pJson);
case QUERY_NODE_PHYSICAL_PLAN_FORECAST_FUNC:
return physiForecastFuncNodeToJson(pObj, pJson);
case QUERY_NODE_PHYSICAL_PLAN_DISPATCH:
return physiDispatchNodeToJson(pObj, pJson);
case QUERY_NODE_PHYSICAL_PLAN_INSERT:
@ -7900,6 +8096,8 @@ static int32_t jsonToSpecificNode(const SJson* pJson, void* pObj) {
return jsonToWindowOffsetNode(pJson, pObj);
case QUERY_NODE_COUNT_WINDOW:
return jsonToCountWindowNode(pJson, pObj);
case QUERY_NODE_ANOMALY_WINDOW:
return jsonToAnomalyWindowNode(pJson, pObj);
case QUERY_NODE_SET_OPERATOR:
return jsonToSetOperator(pJson, pObj);
case QUERY_NODE_SELECT_STMT:
@ -8002,6 +8200,10 @@ static int32_t jsonToSpecificNode(const SJson* pJson, void* pObj) {
return jsonToShowMnodesStmt(pJson, pObj);
case QUERY_NODE_SHOW_QNODES_STMT:
return jsonToShowQnodesStmt(pJson, pObj);
case QUERY_NODE_SHOW_ANODES_STMT:
return jsonToShowAnodesStmt(pJson, pObj);
case QUERY_NODE_SHOW_ANODES_FULL_STMT:
return jsonToShowAnodesFullStmt(pJson, pObj);
case QUERY_NODE_SHOW_ARBGROUPS_STMT:
return jsonToShowArbGroupsStmt(pJson, pObj);
case QUERY_NODE_SHOW_CLUSTER_STMT:
@ -8099,6 +8301,8 @@ static int32_t jsonToSpecificNode(const SJson* pJson, void* pObj) {
return jsonToLogicIndefRowsFuncNode(pJson, pObj);
case QUERY_NODE_LOGIC_PLAN_INTERP_FUNC:
return jsonToLogicInterpFuncNode(pJson, pObj);
case QUERY_NODE_LOGIC_PLAN_FORECAST_FUNC:
return jsonToLogicForecastFuncNode(pJson, pObj);
case QUERY_NODE_LOGIC_PLAN_GROUP_CACHE:
return jsonToLogicGroupCacheNode(pJson, pObj);
case QUERY_NODE_LOGIC_PLAN_DYN_QUERY_CTRL:
@ -8159,6 +8363,8 @@ static int32_t jsonToSpecificNode(const SJson* pJson, void* pObj) {
case QUERY_NODE_PHYSICAL_PLAN_MERGE_COUNT:
case QUERY_NODE_PHYSICAL_PLAN_STREAM_COUNT:
return jsonToPhysiCountWindowNode(pJson, pObj);
case QUERY_NODE_PHYSICAL_PLAN_MERGE_ANOMALY:
return jsonToPhysiAnomalyWindowNode(pJson, pObj);
case QUERY_NODE_PHYSICAL_PLAN_PARTITION:
return jsonToPhysiPartitionNode(pJson, pObj);
case QUERY_NODE_PHYSICAL_PLAN_STREAM_PARTITION:
@ -8167,6 +8373,8 @@ static int32_t jsonToSpecificNode(const SJson* pJson, void* pObj) {
return jsonToPhysiIndefRowsFuncNode(pJson, pObj);
case QUERY_NODE_PHYSICAL_PLAN_INTERP_FUNC:
return jsonToPhysiInterpFuncNode(pJson, pObj);
case QUERY_NODE_PHYSICAL_PLAN_FORECAST_FUNC:
return jsonToPhysiForecastFuncNode(pJson, pObj);
case QUERY_NODE_PHYSICAL_PLAN_DISPATCH:
return jsonToPhysiDispatchNode(pJson, pObj);
case QUERY_NODE_PHYSICAL_PLAN_QUERY_INSERT:

96
source/libs/nodes/src/nodesMsgFuncs.c
View File

@ -3539,6 +3539,46 @@ static int32_t msgToPhysiCountWindowNode(STlvDecoder* pDecoder, void* pObj) {
return code;
}
enum { PHY_ANOMALY_CODE_WINDOW = 1, PHY_ANOMALY_CODE_KEY, PHY_ANOMALY_CODE_WINDOW_OPTION };
static int32_t physiAnomalyWindowNodeToMsg(const void* pObj, STlvEncoder* pEncoder) {
const SAnomalyWindowPhysiNode* pNode = (const SAnomalyWindowPhysiNode*)pObj;
int32_t code = tlvEncodeObj(pEncoder, PHY_ANOMALY_CODE_WINDOW, physiWindowNodeToMsg, &pNode->window);
if (TSDB_CODE_SUCCESS == code) {
code = tlvEncodeObj(pEncoder, PHY_ANOMALY_CODE_KEY, nodeToMsg, pNode->pAnomalyKey);
}
if (TSDB_CODE_SUCCESS == code) {
code = tlvEncodeCStr(pEncoder, PHY_ANOMALY_CODE_WINDOW_OPTION, pNode->anomalyOpt);
}
return code;
}
static int32_t msgToPhysiAnomalyWindowNode(STlvDecoder* pDecoder, void* pObj) {
SAnomalyWindowPhysiNode* pNode = (SAnomalyWindowPhysiNode*)pObj;
int32_t code = TSDB_CODE_SUCCESS;
STlv* pTlv = NULL;
tlvForEach(pDecoder, pTlv, code) {
switch (pTlv->type) {
case PHY_ANOMALY_CODE_WINDOW:
code = tlvDecodeObjFromTlv(pTlv, msgToPhysiWindowNode, &pNode->window);
break;
case PHY_ANOMALY_CODE_KEY:
code = msgToNodeFromTlv(pTlv, (void**)&pNode->pAnomalyKey);
break;
case PHY_ANOMALY_CODE_WINDOW_OPTION:
code = tlvDecodeCStr(pTlv, pNode->anomalyOpt, sizeof(pNode->anomalyOpt));
break;
default:
break;
}
}
return code;
}
enum {
PHY_PARTITION_CODE_BASE_NODE = 1,
PHY_PARTITION_CODE_EXPR,
@ -3770,6 +3810,50 @@ static int32_t msgToPhysiInterpFuncNode(STlvDecoder* pDecoder, void* pObj) {
return code;
}
enum {
PHY_FORECAST_FUNC_CODE_BASE_NODE = 1,
PHY_FORECAST_FUNC_CODE_EXPR,
PHY_FORECAST_FUNC_CODE_FUNCS,
};
static int32_t physiForecastFuncNodeToMsg(const void* pObj, STlvEncoder* pEncoder) {
const SForecastFuncPhysiNode* pNode = (const SForecastFuncPhysiNode*)pObj;
int32_t code = tlvEncodeObj(pEncoder, PHY_FORECAST_FUNC_CODE_BASE_NODE, physiNodeToMsg, &pNode->node);
if (TSDB_CODE_SUCCESS == code) {
code = tlvEncodeObj(pEncoder, PHY_FORECAST_FUNC_CODE_EXPR, nodeListToMsg, pNode->pExprs);
}
if (TSDB_CODE_SUCCESS == code) {
code = tlvEncodeObj(pEncoder, PHY_FORECAST_FUNC_CODE_FUNCS, nodeListToMsg, pNode->pFuncs);
}
return code;
}
static int32_t msgToPhysiForecastFuncNode(STlvDecoder* pDecoder, void* pObj) {
SForecastFuncPhysiNode* pNode = (SForecastFuncPhysiNode*)pObj;
int32_t code = TSDB_CODE_SUCCESS;
STlv* pTlv = NULL;
tlvForEach(pDecoder, pTlv, code) {
switch (pTlv->type) {
case PHY_FORECAST_FUNC_CODE_BASE_NODE:
code = tlvDecodeObjFromTlv(pTlv, msgToPhysiNode, &pNode->node);
break;
case PHY_FORECAST_FUNC_CODE_EXPR:
code = msgToNodeListFromTlv(pTlv, (void**)&pNode->pExprs);
break;
case PHY_FORECAST_FUNC_CODE_FUNCS:
code = msgToNodeListFromTlv(pTlv, (void**)&pNode->pFuncs);
break;
default:
break;
}
}
return code;
}
enum { PHY_DATA_SINK_CODE_INPUT_DESC = 1 };
static int32_t physicDataSinkNodeToMsg(const void* pObj, STlvEncoder* pEncoder) {
@ -4536,6 +4620,9 @@ static int32_t specificNodeToMsg(const void* pObj, STlvEncoder* pEncoder) {
case QUERY_NODE_PHYSICAL_PLAN_STREAM_COUNT:
code = physiCountWindowNodeToMsg(pObj, pEncoder);
break;
case QUERY_NODE_PHYSICAL_PLAN_MERGE_ANOMALY:
code = physiAnomalyWindowNodeToMsg(pObj, pEncoder);
break;
case QUERY_NODE_PHYSICAL_PLAN_PARTITION:
code = physiPartitionNodeToMsg(pObj, pEncoder);
break;
@ -4548,6 +4635,9 @@ static int32_t specificNodeToMsg(const void* pObj, STlvEncoder* pEncoder) {
case QUERY_NODE_PHYSICAL_PLAN_INTERP_FUNC:
code = physiInterpFuncNodeToMsg(pObj, pEncoder);
break;
case QUERY_NODE_PHYSICAL_PLAN_FORECAST_FUNC:
code = physiForecastFuncNodeToMsg(pObj, pEncoder);
break;
case QUERY_NODE_PHYSICAL_PLAN_DISPATCH:
code = physiDispatchNodeToMsg(pObj, pEncoder);
break;
@ -4698,6 +4788,9 @@ static int32_t msgToSpecificNode(STlvDecoder* pDecoder, void* pObj) {
case QUERY_NODE_PHYSICAL_PLAN_STREAM_COUNT:
code = msgToPhysiCountWindowNode(pDecoder, pObj);
break;
case QUERY_NODE_PHYSICAL_PLAN_MERGE_ANOMALY:
code = msgToPhysiAnomalyWindowNode(pDecoder, pObj);
break;
case QUERY_NODE_PHYSICAL_PLAN_PARTITION:
code = msgToPhysiPartitionNode(pDecoder, pObj);
break;
@ -4710,6 +4803,9 @@ static int32_t msgToSpecificNode(STlvDecoder* pDecoder, void* pObj) {
case QUERY_NODE_PHYSICAL_PLAN_INTERP_FUNC:
code = msgToPhysiInterpFuncNode(pDecoder, pObj);
break;
case QUERY_NODE_PHYSICAL_PLAN_FORECAST_FUNC:
code = msgToPhysiForecastFuncNode(pDecoder, pObj);
break;
case QUERY_NODE_PHYSICAL_PLAN_DISPATCH:
code = msgToPhysiDispatchNode(pDecoder, pObj);
break;

16
source/libs/nodes/src/nodesTraverseFuncs.c
View File

@ -181,6 +181,14 @@ static EDealRes dispatchExpr(SNode* pNode, ETraversalOrder order, FNodeWalker wa
res = walkExpr(pEvent->pCol, order, walker, pContext);
break;
}
case QUERY_NODE_ANOMALY_WINDOW: {
SAnomalyWindowNode* pAnomaly = (SAnomalyWindowNode*)pNode;
res = walkExpr(pAnomaly->pExpr, order, walker, pContext);
if (DEAL_RES_ERROR != res && DEAL_RES_END != res) {
res = walkExpr(pAnomaly->pCol, order, walker, pContext);
}
break;
}
default:
break;
}
@ -392,6 +400,14 @@ static EDealRes rewriteExpr(SNode** pRawNode, ETraversalOrder order, FNodeRewrit
res = rewriteExpr(&pEvent->pCol, order, rewriter, pContext);
break;
}
case QUERY_NODE_ANOMALY_WINDOW: {
SAnomalyWindowNode* pAnomaly = (SAnomalyWindowNode*)pNode;
res = rewriteExpr(&pAnomaly->pExpr, order, rewriter, pContext);
if (DEAL_RES_ERROR != res && DEAL_RES_END != res) {
res = rewriteExpr(&pAnomaly->pCol, order, rewriter, pContext);
}
break;
}
default:
break;
}

44
source/libs/nodes/src/nodesUtilFuncs.c
View File

@ -419,6 +419,8 @@ int32_t nodesMakeNode(ENodeType type, SNode** ppNodeOut) {
code = makeNode(type, sizeof(SEventWindowNode), &pNode); break;
case QUERY_NODE_COUNT_WINDOW:
code = makeNode(type, sizeof(SCountWindowNode), &pNode); break;
case QUERY_NODE_ANOMALY_WINDOW:
code = makeNode(type, sizeof(SAnomalyWindowNode), &pNode); break;
case QUERY_NODE_HINT:
code = makeNode(type, sizeof(SHintNode), &pNode); break;
case QUERY_NODE_VIEW:
@ -474,6 +476,12 @@ int32_t nodesMakeNode(ENodeType type, SNode** ppNodeOut) {
code = makeNode(type, sizeof(SDropDnodeStmt), &pNode); break;
case QUERY_NODE_ALTER_DNODE_STMT:
code = makeNode(type, sizeof(SAlterDnodeStmt), &pNode); break;
case QUERY_NODE_CREATE_ANODE_STMT:
code = makeNode(type, sizeof(SCreateAnodeStmt), &pNode); break;
case QUERY_NODE_DROP_ANODE_STMT:
code = makeNode(type, sizeof(SDropAnodeStmt), &pNode); break;
case QUERY_NODE_UPDATE_ANODE_STMT:
code = makeNode(type, sizeof(SUpdateAnodeStmt), &pNode); break;
case QUERY_NODE_CREATE_INDEX_STMT:
code = makeNode(type, sizeof(SCreateIndexStmt), &pNode); break;
case QUERY_NODE_DROP_INDEX_STMT:
@ -540,6 +548,8 @@ int32_t nodesMakeNode(ENodeType type, SNode** ppNodeOut) {
case QUERY_NODE_SHOW_MNODES_STMT:
case QUERY_NODE_SHOW_MODULES_STMT:
case QUERY_NODE_SHOW_QNODES_STMT:
case QUERY_NODE_SHOW_ANODES_STMT:
case QUERY_NODE_SHOW_ANODES_FULL_STMT:
case QUERY_NODE_SHOW_SNODES_STMT:
case QUERY_NODE_SHOW_BNODES_STMT:
case QUERY_NODE_SHOW_ARBGROUPS_STMT:
@ -647,6 +657,8 @@ int32_t nodesMakeNode(ENodeType type, SNode** ppNodeOut) {
code = makeNode(type, sizeof(SIndefRowsFuncLogicNode), &pNode); break;
case QUERY_NODE_LOGIC_PLAN_INTERP_FUNC:
code = makeNode(type, sizeof(SInterpFuncLogicNode), &pNode); break;
case QUERY_NODE_LOGIC_PLAN_FORECAST_FUNC:
code = makeNode(type, sizeof(SForecastFuncLogicNode), &pNode); break;
case QUERY_NODE_LOGIC_PLAN_GROUP_CACHE:
code = makeNode(type, sizeof(SGroupCacheLogicNode), &pNode); break;
case QUERY_NODE_LOGIC_PLAN_DYN_QUERY_CTRL:
@ -722,6 +734,8 @@ int32_t nodesMakeNode(ENodeType type, SNode** ppNodeOut) {
code = makeNode(type, sizeof(SStreamEventWinodwPhysiNode), &pNode); break;
case QUERY_NODE_PHYSICAL_PLAN_MERGE_COUNT:
code = makeNode(type, sizeof(SCountWinodwPhysiNode), &pNode); break;
case QUERY_NODE_PHYSICAL_PLAN_MERGE_ANOMALY:
code = makeNode(type, sizeof(SAnomalyWindowPhysiNode), &pNode); break;
case QUERY_NODE_PHYSICAL_PLAN_STREAM_COUNT:
code = makeNode(type, sizeof(SStreamCountWinodwPhysiNode), &pNode); break;
case QUERY_NODE_PHYSICAL_PLAN_PARTITION:
@ -732,6 +746,8 @@ int32_t nodesMakeNode(ENodeType type, SNode** ppNodeOut) {
code = makeNode(type, sizeof(SIndefRowsFuncPhysiNode), &pNode); break;
case QUERY_NODE_PHYSICAL_PLAN_INTERP_FUNC:
code = makeNode(type, sizeof(SInterpFuncLogicNode), &pNode); break;
case QUERY_NODE_PHYSICAL_PLAN_FORECAST_FUNC:
code = makeNode(type, sizeof(SForecastFuncLogicNode), &pNode); break;
case QUERY_NODE_PHYSICAL_PLAN_DISPATCH:
code = makeNode(type, sizeof(SDataDispatcherNode), &pNode); break;
case QUERY_NODE_PHYSICAL_PLAN_INSERT:
@ -1019,6 +1035,11 @@ void nodesDestroyNode(SNode* pNode) {
nodesDestroyNode(pEvent->pCol);
break;
}
case QUERY_NODE_ANOMALY_WINDOW: {
SAnomalyWindowNode* pAnomaly = (SAnomalyWindowNode*)pNode;
nodesDestroyNode(pAnomaly->pCol);
break;
}
case QUERY_NODE_HINT: {
SHintNode* pHint = (SHintNode*)pNode;
destroyHintValue(pHint->option, pHint->value);
@ -1167,6 +1188,9 @@ void nodesDestroyNode(SNode* pNode) {
case QUERY_NODE_CREATE_DNODE_STMT: // no pointer field
case QUERY_NODE_DROP_DNODE_STMT: // no pointer field
case QUERY_NODE_ALTER_DNODE_STMT: // no pointer field
case QUERY_NODE_CREATE_ANODE_STMT: // no pointer field
case QUERY_NODE_UPDATE_ANODE_STMT: // no pointer field
case QUERY_NODE_DROP_ANODE_STMT: // no pointer field
break;
case QUERY_NODE_CREATE_INDEX_STMT: {
SCreateIndexStmt* pStmt = (SCreateIndexStmt*)pNode;
@ -1252,6 +1276,8 @@ void nodesDestroyNode(SNode* pNode) {
case QUERY_NODE_SHOW_MNODES_STMT:
case QUERY_NODE_SHOW_MODULES_STMT:
case QUERY_NODE_SHOW_QNODES_STMT:
case QUERY_NODE_SHOW_ANODES_STMT:
case QUERY_NODE_SHOW_ANODES_FULL_STMT:
case QUERY_NODE_SHOW_SNODES_STMT:
case QUERY_NODE_SHOW_BNODES_STMT:
case QUERY_NODE_SHOW_ARBGROUPS_STMT:
@ -1500,6 +1526,12 @@ void nodesDestroyNode(SNode* pNode) {
nodesDestroyNode(pLogicNode->pTimeSeries);
break;
}
case QUERY_NODE_LOGIC_PLAN_FORECAST_FUNC: {
SForecastFuncLogicNode* pLogicNode = (SForecastFuncLogicNode*)pNode;
destroyLogicNode((SLogicNode*)pLogicNode);
nodesDestroyList(pLogicNode->pFuncs);
break;
}
case QUERY_NODE_LOGIC_PLAN_GROUP_CACHE: {
SGroupCacheLogicNode* pLogicNode = (SGroupCacheLogicNode*)pNode;
destroyLogicNode((SLogicNode*)pLogicNode);
@ -1663,6 +1695,11 @@ void nodesDestroyNode(SNode* pNode) {
destroyWinodwPhysiNode((SWindowPhysiNode*)pPhyNode);
break;
}
case QUERY_NODE_PHYSICAL_PLAN_MERGE_ANOMALY: {
SAnomalyWindowPhysiNode* pPhyNode = (SAnomalyWindowPhysiNode*)pNode;
destroyWinodwPhysiNode((SWindowPhysiNode*)pPhyNode);
break;
}
case QUERY_NODE_PHYSICAL_PLAN_PARTITION: {
destroyPartitionPhysiNode((SPartitionPhysiNode*)pNode);
break;
@ -1690,6 +1727,13 @@ void nodesDestroyNode(SNode* pNode) {
nodesDestroyNode(pPhyNode->pTimeSeries);
break;
}
case QUERY_NODE_PHYSICAL_PLAN_FORECAST_FUNC: {
SForecastFuncPhysiNode* pPhyNode = (SForecastFuncPhysiNode*)pNode;
destroyPhysiNode((SPhysiNode*)pPhyNode);
nodesDestroyList(pPhyNode->pExprs);
nodesDestroyList(pPhyNode->pFuncs);
break;
}
case QUERY_NODE_PHYSICAL_PLAN_DISPATCH:
destroyDataSinkNode((SDataSinkNode*)pNode);
break;

15
source/libs/parser/inc/sql.y
View File

@ -157,6 +157,12 @@ with_clause_opt(A) ::= WITH search_condition(B).
/************************************************ create encrypt_key *********************************************/
cmd ::= CREATE ENCRYPT_KEY NK_STRING(A). { pCxt->pRootNode = createEncryptKeyStmt(pCxt, &A); }
/************************************************ create drop update anode ***************************************/
cmd ::= CREATE ANODE NK_STRING(A). { pCxt->pRootNode = createCreateAnodeStmt(pCxt, &A); }
cmd ::= UPDATE ANODE NK_INTEGER(A). { pCxt->pRootNode = createUpdateAnodeStmt(pCxt, &A, false); }
cmd ::= UPDATE ALL ANODES. { pCxt->pRootNode = createUpdateAnodeStmt(pCxt, NULL, true); }
cmd ::= DROP ANODE NK_INTEGER(A). { pCxt->pRootNode = createDropAnodeStmt(pCxt, &A); }
/************************************************ create/drop/alter/restore dnode *********************************************/
cmd ::= CREATE DNODE dnode_endpoint(A). { pCxt->pRootNode = createCreateDnodeStmt(pCxt, &A, NULL); }
cmd ::= CREATE DNODE dnode_endpoint(A) PORT NK_INTEGER(B). { pCxt->pRootNode = createCreateDnodeStmt(pCxt, &A, &B); }
@ -524,6 +530,8 @@ cmd ::= SHOW db_name_cond_opt(A) VGROUPS.
cmd ::= SHOW MNODES. { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_MNODES_STMT); }
//cmd ::= SHOW MODULES. { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_MODULES_STMT); }
cmd ::= SHOW QNODES. { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_QNODES_STMT); }
cmd ::= SHOW ANODES. { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_ANODES_STMT); }
cmd ::= SHOW ANODES FULL. { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_ANODES_FULL_STMT); }
cmd ::= SHOW ARBGROUPS. { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_ARBGROUPS_STMT); }
cmd ::= SHOW FUNCTIONS. { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_FUNCTIONS_STMT); }
cmd ::= SHOW INDEXES FROM table_name_cond(A) from_db_opt(B). { pCxt->pRootNode = createShowStmtWithCond(pCxt, QUERY_NODE_SHOW_INDEXES_STMT, B, A, OP_TYPE_EQUAL); }
@ -1187,6 +1195,9 @@ pseudo_column(A) ::= WDURATION(B).
pseudo_column(A) ::= IROWTS(B). { A = createRawExprNode(pCxt, &B, createFunctionNode(pCxt, &B, NULL)); }
pseudo_column(A) ::= ISFILLED(B). { A = createRawExprNode(pCxt, &B, createFunctionNode(pCxt, &B, NULL)); }
pseudo_column(A) ::= QTAGS(B). { A = createRawExprNode(pCxt, &B, createFunctionNode(pCxt, &B, NULL)); }
pseudo_column(A) ::= FLOW(B). { A = createRawExprNode(pCxt, &B, createFunctionNode(pCxt, &B, NULL)); }
pseudo_column(A) ::= FHIGH(B). { A = createRawExprNode(pCxt, &B, createFunctionNode(pCxt, &B, NULL)); }
pseudo_column(A) ::= FROWTS(B). { A = createRawExprNode(pCxt, &B, createFunctionNode(pCxt, &B, NULL)); }
function_expression(A) ::= function_name(B) NK_LP expression_list(C) NK_RP(D). { A = createRawExprNodeExt(pCxt, &B, &D, createFunctionNode(pCxt, &B, C)); }
function_expression(A) ::= star_func(B) NK_LP star_func_para_list(C) NK_RP(D). { A = createRawExprNodeExt(pCxt, &B, &D, createFunctionNode(pCxt, &B, C)); }
@ -1505,6 +1516,10 @@ twindow_clause_opt(A) ::=
COUNT_WINDOW NK_LP NK_INTEGER(B) NK_RP. { A = createCountWindowNode(pCxt, &B, &B); }
twindow_clause_opt(A) ::=
COUNT_WINDOW NK_LP NK_INTEGER(B) NK_COMMA NK_INTEGER(C) NK_RP. { A = createCountWindowNode(pCxt, &B, &C); }
twindow_clause_opt(A) ::=
ANOMALY_WINDOW NK_LP expr_or_subquery(B) NK_RP. { A = createAnomalyWindowNode(pCxt, releaseRawExprNode(pCxt, B), NULL); }
twindow_clause_opt(A) ::=
ANOMALY_WINDOW NK_LP expr_or_subquery(B) NK_COMMA NK_STRING(C) NK_RP. { A = createAnomalyWindowNode(pCxt, releaseRawExprNode(pCxt, B), &C); }
sliding_opt(A) ::= . { A = NULL; }
sliding_opt(A) ::= SLIDING NK_LP interval_sliding_duration_literal(B) NK_RP. { A = releaseRawExprNode(pCxt, B); }

60
source/libs/parser/src/parAstCreater.c
View File

@ -1367,6 +1367,25 @@ _err:
return NULL;
}
SNode* createAnomalyWindowNode(SAstCreateContext* pCxt, SNode* pExpr, const SToken* pFuncOpt) {
SAnomalyWindowNode* pAnomaly = NULL;
CHECK_PARSER_STATUS(pCxt);
pCxt->errCode = nodesMakeNode(QUERY_NODE_ANOMALY_WINDOW, (SNode**)&pAnomaly);
CHECK_MAKE_NODE(pAnomaly);
pAnomaly->pCol = createPrimaryKeyCol(pCxt, NULL);
CHECK_MAKE_NODE(pAnomaly->pCol);
pAnomaly->pExpr = pExpr;
if (pFuncOpt == NULL) {
tstrncpy(pAnomaly->anomalyOpt, "algo=iqr", TSDB_ANAL_ALGO_OPTION_LEN);
} else {
(void)trimString(pFuncOpt->z, pFuncOpt->n, pAnomaly->anomalyOpt, sizeof(pAnomaly->anomalyOpt));
}
return (SNode*)pAnomaly;
_err:
nodesDestroyNode((SNode*)pAnomaly);
return NULL;
}
SNode* createIntervalWindowNode(SAstCreateContext* pCxt, SNode* pInterval, SNode* pOffset, SNode* pSliding,
SNode* pFill) {
SIntervalWindowNode* interval = NULL;
@ -2997,6 +3016,47 @@ _err:
return NULL;
}
SNode* createCreateAnodeStmt(SAstCreateContext* pCxt, const SToken* pUrl) {
CHECK_PARSER_STATUS(pCxt);
SCreateAnodeStmt* pStmt = NULL;
pCxt->errCode = nodesMakeNode(QUERY_NODE_CREATE_ANODE_STMT, (SNode**)&pStmt);
CHECK_MAKE_NODE(pStmt);
(void)trimString(pUrl->z, pUrl->n, pStmt->url, sizeof(pStmt->url));
return (SNode*)pStmt;
_err:
return NULL;
}
SNode* createDropAnodeStmt(SAstCreateContext* pCxt, const SToken* pAnode) {
CHECK_PARSER_STATUS(pCxt);
SUpdateAnodeStmt* pStmt = NULL;
pCxt->errCode = nodesMakeNode(QUERY_NODE_DROP_ANODE_STMT, (SNode**)&pStmt);
CHECK_MAKE_NODE(pStmt);
if (NULL != pAnode) {
pStmt->anodeId = taosStr2Int32(pAnode->z, NULL, 10);
} else {
pStmt->anodeId = -1;
}
return (SNode*)pStmt;
_err:
return NULL;
}
SNode* createUpdateAnodeStmt(SAstCreateContext* pCxt, const SToken* pAnode, bool updateAll) {
CHECK_PARSER_STATUS(pCxt);
SUpdateAnodeStmt* pStmt = NULL;
pCxt->errCode = nodesMakeNode(QUERY_NODE_UPDATE_ANODE_STMT, (SNode**)&pStmt);
CHECK_MAKE_NODE(pStmt);
if (NULL != pAnode) {
pStmt->anodeId = taosStr2Int32(pAnode->z, NULL, 10);
} else {
pStmt->anodeId = -1;
}
return (SNode*)pStmt;
_err:
return NULL;
}
SNode* createEncryptKeyStmt(SAstCreateContext* pCxt, const SToken* pValue) {
SToken config;
config.type = TK_NK_STRING;

20
source/libs/parser/src/parAstParser.c
View File

@ -555,6 +555,22 @@ static int32_t collectMetaKeyFromShowSnodes(SCollectMetaKeyCxt* pCxt, SShowStmt*
return TSDB_CODE_SUCCESS;
}
static int32_t collectMetaKeyFromShowAnodes(SCollectMetaKeyCxt* pCxt, SShowStmt* pStmt) {
if (pCxt->pParseCxt->enableSysInfo) {
return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_ANODES,
pCxt->pMetaCache);
}
return TSDB_CODE_SUCCESS;
}
static int32_t collectMetaKeyFromShowAnodesFull(SCollectMetaKeyCxt* pCxt, SShowStmt* pStmt) {
if (pCxt->pParseCxt->enableSysInfo) {
return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_ANODES_FULL,
pCxt->pMetaCache);
}
return TSDB_CODE_SUCCESS;
}
static int32_t collectMetaKeyFromShowBnodes(SCollectMetaKeyCxt* pCxt, SShowStmt* pStmt) {
if (pCxt->pParseCxt->enableSysInfo) {
return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_BNODES,
@ -983,6 +999,10 @@ static int32_t collectMetaKeyFromQuery(SCollectMetaKeyCxt* pCxt, SNode* pStmt) {
return collectMetaKeyFromShowQnodes(pCxt, (SShowStmt*)pStmt);
case QUERY_NODE_SHOW_SNODES_STMT:
return collectMetaKeyFromShowSnodes(pCxt, (SShowStmt*)pStmt);
case QUERY_NODE_SHOW_ANODES_STMT:
return collectMetaKeyFromShowAnodes(pCxt, (SShowStmt*)pStmt);
case QUERY_NODE_SHOW_ANODES_FULL_STMT:
return collectMetaKeyFromShowAnodesFull(pCxt, (SShowStmt*)pStmt);
case QUERY_NODE_SHOW_BNODES_STMT:
return collectMetaKeyFromShowBnodes(pCxt, (SShowStmt*)pStmt);
case QUERY_NODE_SHOW_ARBGROUPS_STMT:

2
source/libs/parser/src/parAuthenticator.c
View File

@ -358,6 +358,8 @@ static int32_t authQuery(SAuthCxt* pCxt, SNode* pStmt) {
case QUERY_NODE_SHOW_MNODES_STMT:
case QUERY_NODE_SHOW_MODULES_STMT:
case QUERY_NODE_SHOW_QNODES_STMT:
case QUERY_NODE_SHOW_ANODES_STMT:
case QUERY_NODE_SHOW_ANODES_FULL_STMT:
case QUERY_NODE_SHOW_SNODES_STMT:
case QUERY_NODE_SHOW_BNODES_STMT:
case QUERY_NODE_SHOW_CLUSTER_STMT:

6
source/libs/parser/src/parTokenizer.c
View File

@ -38,6 +38,9 @@ static SKeyword keywordTable[] = {
{"ANALYZE", TK_ANALYZE},
{"AND", TK_AND},
{"ANTI", TK_ANTI},
{"ANODE", TK_ANODE},
{"ANODES", TK_ANODES},
{"ANOMALY_WINDOW", TK_ANOMALY_WINDOW},
// {"ANY", TK_ANY},
{"APPS", TK_APPS},
{"AS", TK_AS},
@ -332,6 +335,9 @@ static SKeyword keywordTable[] = {
{"_WDURATION", TK_WDURATION},
{"_WEND", TK_WEND},
{"_WSTART", TK_WSTART},
{"_FLOW", TK_FLOW},
{"_FHIGH", TK_FHIGH},
{"_FROWTS", TK_FROWTS},
{"ALIVE", TK_ALIVE},
{"VARBINARY", TK_VARBINARY},
{"S3_CHUNKSIZE", TK_S3_CHUNKSIZE},

243
source/libs/parser/src/parTranslater.c
View File

@ -24,6 +24,7 @@
#include "parUtil.h"
#include "scalar.h"
#include "systable.h"
#include "tanal.h"
#include "tcol.h"
#include "tglobal.h"
#include "ttime.h"
@ -348,6 +349,20 @@ static const SSysTableShowAdapter sysTableShowAdapter[] = {
.numOfShowCols = 1,
.pShowCols = {"*"}
},
{
.showType = QUERY_NODE_SHOW_ANODES_STMT,
.pDbName = TSDB_INFORMATION_SCHEMA_DB,
.pTableName = TSDB_INS_TABLE_ANODES,
.numOfShowCols = 1,
.pShowCols = {"*"}
},
{
.showType = QUERY_NODE_SHOW_ANODES_FULL_STMT,
.pDbName = TSDB_INFORMATION_SCHEMA_DB,
.pTableName = TSDB_INS_TABLE_ANODES_FULL,
.numOfShowCols = 1,
.pShowCols = {"*"}
},
};
// clang-format on
@ -1035,6 +1050,14 @@ static bool isInterpPseudoColumnFunc(const SNode* pNode) {
return (QUERY_NODE_FUNCTION == nodeType(pNode) && fmIsInterpPseudoColumnFunc(((SFunctionNode*)pNode)->funcId));
}
static bool isForecastFunc(const SNode* pNode) {
return (QUERY_NODE_FUNCTION == nodeType(pNode) && fmIsForecastFunc(((SFunctionNode*)pNode)->funcId));
}
static bool isForecastPseudoColumnFunc(const SNode* pNode) {
return (QUERY_NODE_FUNCTION == nodeType(pNode) && fmIsForecastPseudoColumnFunc(((SFunctionNode*)pNode)->funcId));
}
#ifdef BUILD_NO_CALL
static bool isTimelineFunc(const SNode* pNode) {
return (QUERY_NODE_FUNCTION == nodeType(pNode) && fmIsTimelineFunc(((SFunctionNode*)pNode)->funcId));
@ -1237,7 +1260,7 @@ bool isPrimaryKeyImpl(SNode* pExpr) {
FUNCTION_TYPE_LAST_ROW == pFunc->funcType || FUNCTION_TYPE_TIMETRUNCATE == pFunc->funcType) {
return isPrimaryKeyImpl(nodesListGetNode(pFunc->pParameterList, 0));
} else if (FUNCTION_TYPE_WSTART == pFunc->funcType || FUNCTION_TYPE_WEND == pFunc->funcType ||
FUNCTION_TYPE_IROWTS == pFunc->funcType) {
FUNCTION_TYPE_IROWTS == pFunc->funcType || FUNCTION_TYPE_FORECAST_ROWTS == pFunc->funcType) {
return true;
}
} else if (QUERY_NODE_OPERATOR == nodeType(pExpr)) {
@ -2250,7 +2273,7 @@ static EDealRes translateOperator(STranslateContext* pCxt, SOperatorNode* pOp) {
static EDealRes haveVectorFunction(SNode* pNode, void* pContext) {
if (isAggFunc(pNode) || isIndefiniteRowsFunc(pNode) || isWindowPseudoColumnFunc(pNode) ||
isInterpPseudoColumnFunc(pNode)) {
isInterpPseudoColumnFunc(pNode) || isForecastPseudoColumnFunc(pNode)) {
*((bool*)pContext) = true;
return DEAL_RES_END;
}
@ -2553,6 +2576,72 @@ static int32_t translateInterpPseudoColumnFunc(STranslateContext* pCxt, SNode**
return TSDB_CODE_SUCCESS;
}
static int32_t translateForecastFunc(STranslateContext* pCxt, SFunctionNode* pFunc) {
if (!fmIsForecastFunc(pFunc->funcId)) {
return TSDB_CODE_SUCCESS;
}
if (!isSelectStmt(pCxt->pCurrStmt) || SQL_CLAUSE_SELECT != pCxt->currClause) {
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC);
}
SSelectStmt* pSelect = (SSelectStmt*)pCxt->pCurrStmt;
SNode* pTable = pSelect->pFromTable;
if (pSelect->hasAggFuncs || pSelect->hasMultiRowsFunc || pSelect->hasIndefiniteRowsFunc) {
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC);
}
if (pSelect->hasForecastFunc &&
(FUNC_RETURN_ROWS_INDEFINITE == pSelect->returnRows || pSelect->returnRows != fmGetFuncReturnRows(pFunc))) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC,
"%s ignoring null value options cannot be used when applying to multiple columns",
pFunc->functionName);
}
if (NULL != pSelect->pWindow || NULL != pSelect->pGroupByList) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC,
"%s function is not supported in window query or group query", pFunc->functionName);
}
if (hasInvalidFuncNesting(pFunc->pParameterList)) {
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_AGG_FUNC_NESTING);
}
return TSDB_CODE_SUCCESS;
}
static int32_t translateForecastPseudoColumnFunc(STranslateContext* pCxt, SNode** ppNode, bool* pRewriteToColumn) {
SFunctionNode* pFunc = (SFunctionNode*)(*ppNode);
if (!fmIsForecastPseudoColumnFunc(pFunc->funcId)) {
return TSDB_CODE_SUCCESS;
}
if (!isSelectStmt(pCxt->pCurrStmt)) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC,
"%s must be used in select statements", pFunc->functionName);
}
if (pCxt->currClause == SQL_CLAUSE_WHERE) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_INTERP_CLAUSE,
"%s is not allowed in where clause", pFunc->functionName);
}
SSelectStmt* pSelect = (SSelectStmt*)pCxt->pCurrStmt;
SNode* pNode = NULL;
bool bFound = false;
FOREACH(pNode, pSelect->pProjectionList) {
if (nodeType(pNode) == QUERY_NODE_FUNCTION && strcasecmp(((SFunctionNode*)pNode)->functionName, "forecast") == 0) {
bFound = true;
break;
}
}
if (!bFound) {
*pRewriteToColumn = true;
int32_t code = replacePsedudoColumnFuncWithColumn(pCxt, ppNode);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
(void)translateColumn(pCxt, (SColumnNode**)ppNode);
return pCxt->errCode;
}
return TSDB_CODE_SUCCESS;
}
static int32_t translateTimelineFunc(STranslateContext* pCxt, SFunctionNode* pFunc) {
if (!fmIsTimelineFunc(pFunc->funcId)) {
return TSDB_CODE_SUCCESS;
@ -2738,6 +2827,8 @@ static void setFuncClassification(STranslateContext* pCxt, SFunctionNode* pFunc)
pSelect->returnRows = fmGetFuncReturnRows(pFunc);
} else if (fmIsInterpFunc(pFunc->funcId)) {
pSelect->returnRows = fmGetFuncReturnRows(pFunc);
} else if (fmIsForecastFunc(pFunc->funcId)) {
pSelect->returnRows = fmGetFuncReturnRows(pFunc);
}
if (fmIsProcessByRowFunc(pFunc->funcId)) {
pSelect->lastProcessByRowFuncId = pFunc->funcId;
@ -2755,6 +2846,9 @@ static void setFuncClassification(STranslateContext* pCxt, SFunctionNode* pFunc)
pSelect->hasInterpFunc = pSelect->hasInterpFunc ? true : (FUNCTION_TYPE_INTERP == pFunc->funcType);
pSelect->hasInterpPseudoColFunc =
pSelect->hasInterpPseudoColFunc ? true : fmIsInterpPseudoColumnFunc(pFunc->funcId);
pSelect->hasForecastFunc = pSelect->hasForecastFunc ? true : (FUNCTION_TYPE_FORECAST == pFunc->funcType);
pSelect->hasForecastPseudoColFunc =
pSelect->hasForecastPseudoColFunc ? true : fmIsForecastPseudoColumnFunc(pFunc->funcId);
pSelect->hasLastRowFunc = pSelect->hasLastRowFunc ? true : (FUNCTION_TYPE_LAST_ROW == pFunc->funcType);
pSelect->hasLastFunc = pSelect->hasLastFunc ? true : (FUNCTION_TYPE_LAST == pFunc->funcType);
pSelect->hasTimeLineFunc = pSelect->hasTimeLineFunc ? true : fmIsTimelineFunc(pFunc->funcId);
@ -2946,6 +3040,9 @@ static int32_t translateScanPseudoColumnFunc(STranslateContext* pCxt, SNode** pp
return TSDB_CODE_SUCCESS;
}
if (0 == LIST_LENGTH(pFunc->pParameterList)) {
if (pFunc->funcType == FUNCTION_TYPE_FORECAST_LOW || pFunc->funcType == FUNCTION_TYPE_FORECAST_HIGH) {
return TSDB_CODE_SUCCESS;
}
if (!isSelectStmt(pCxt->pCurrStmt) || NULL == ((SSelectStmt*)pCxt->pCurrStmt)->pFromTable) {
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_TBNAME);
}
@ -3016,6 +3113,16 @@ static int32_t translateNormalFunction(STranslateContext* pCxt, SNode** ppNode)
return code;
}
}
if (TSDB_CODE_SUCCESS == code) {
code = translateForecastFunc(pCxt, pFunc);
}
if (TSDB_CODE_SUCCESS == code) {
bool bRewriteToColumn = false;
code = translateForecastPseudoColumnFunc(pCxt, ppNode, &bRewriteToColumn);
if (bRewriteToColumn) {
return code;
}
}
if (TSDB_CODE_SUCCESS == code) {
code = translateTimelineFunc(pCxt, pFunc);
}
@ -3759,7 +3866,8 @@ static int32_t resetSelectFuncNumWithoutDup(SSelectStmt* pSelect) {
static int32_t checkAggColCoexist(STranslateContext* pCxt, SSelectStmt* pSelect) {
if (NULL != pSelect->pGroupByList || NULL != pSelect->pWindow || isWindowJoinStmt(pSelect) ||
(!pSelect->hasAggFuncs && !pSelect->hasIndefiniteRowsFunc && !pSelect->hasInterpFunc)) {
(!pSelect->hasAggFuncs && !pSelect->hasIndefiniteRowsFunc && !pSelect->hasInterpFunc &&
!pSelect->hasForecastFunc)) {
return TSDB_CODE_SUCCESS;
}
if (!pSelect->onlyHasKeepOrderFunc) {
@ -3781,8 +3889,8 @@ static int32_t checkAggColCoexist(STranslateContext* pCxt, SSelectStmt* pSelect)
}
static int32_t checkWinJoinAggColCoexist(STranslateContext* pCxt, SSelectStmt* pSelect) {
if (!isWindowJoinStmt(pSelect) ||
(!pSelect->hasAggFuncs && !pSelect->hasIndefiniteRowsFunc && !pSelect->hasInterpFunc)) {
if (!isWindowJoinStmt(pSelect) || (!pSelect->hasAggFuncs && !pSelect->hasIndefiniteRowsFunc &&
!pSelect->hasInterpFunc && !pSelect->hasForecastFunc)) {
return TSDB_CODE_SUCCESS;
}
if (!pSelect->onlyHasKeepOrderFunc) {
@ -5795,6 +5903,40 @@ static int32_t translateCountWindow(STranslateContext* pCxt, SSelectStmt* pSelec
return TSDB_CODE_SUCCESS;
}
static int32_t checkAnomalyExpr(STranslateContext* pCxt, SNode* pNode) {
int32_t type = ((SExprNode*)pNode)->resType.type;
if (!IS_MATHABLE_TYPE(type)) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_ANOMALY_WIN_TYPE,
"ANOMALY_WINDOW only support mathable column");
}
if (QUERY_NODE_COLUMN == nodeType(pNode) && COLUMN_TYPE_TAG == ((SColumnNode*)pNode)->colType) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_ANOMALY_WIN_COL,
"ANOMALY_WINDOW not support on tag column");
}
return TSDB_CODE_SUCCESS;
}
static int32_t translateAnomalyWindow(STranslateContext* pCxt, SSelectStmt* pSelect) {
if (QUERY_NODE_TEMP_TABLE == nodeType(pSelect->pFromTable) &&
!isGlobalTimeLineQuery(((STempTableNode*)pSelect->pFromTable)->pSubquery)) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_TIMELINE_QUERY,
"ANOMALY_WINDOW requires valid time series input");
}
SAnomalyWindowNode* pAnomaly = (SAnomalyWindowNode*)pSelect->pWindow;
int32_t code = checkAnomalyExpr(pCxt, pAnomaly->pExpr);
if (TSDB_CODE_SUCCESS == code) {
if (!taosAnalGetOptStr(pAnomaly->anomalyOpt, "algo", NULL, 0) != 0) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_ANOMALY_WIN_OPT,
"ANOMALY_WINDOW option should include algo field");
}
}
return code;
}
static int32_t translateSpecificWindow(STranslateContext* pCxt, SSelectStmt* pSelect) {
switch (nodeType(pSelect->pWindow)) {
case QUERY_NODE_STATE_WINDOW:
@ -5807,6 +5949,8 @@ static int32_t translateSpecificWindow(STranslateContext* pCxt, SSelectStmt* pSe
return translateEventWindow(pCxt, pSelect);
case QUERY_NODE_COUNT_WINDOW:
return translateCountWindow(pCxt, pSelect);
case QUERY_NODE_ANOMALY_WINDOW:
return translateAnomalyWindow(pCxt, pSelect);
default:
break;
}
@ -6043,6 +6187,26 @@ static int32_t translateInterp(STranslateContext* pCxt, SSelectStmt* pSelect) {
return code;
}
static int32_t translateForecast(STranslateContext* pCxt, SSelectStmt* pSelect) {
if (!pSelect->hasForecastFunc) {
if (pSelect->hasForecastPseudoColFunc) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC,
"Has Forecast pseudo column(s) but missing forcast function");
}
return TSDB_CODE_SUCCESS;
}
if ((NULL != pSelect->pFromTable) && (QUERY_NODE_JOIN_TABLE == nodeType(pSelect->pFromTable))) {
SJoinTableNode* pJoinTable = (SJoinTableNode*)pSelect->pFromTable;
if (IS_WINDOW_JOIN(pJoinTable->subType)) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_FORECAST_CLAUSE,
"Forecast not supported to be used in WINDOW join");
}
}
return 0;
}
static int32_t removeConstantValueFromList(SNodeList** pList) {
SNode* pNode = NULL;
WHERE_EACH(pNode, *pList) {
@ -6884,6 +7048,9 @@ static int32_t translateSelectFrom(STranslateContext* pCxt, SSelectStmt* pSelect
if (TSDB_CODE_SUCCESS == code) {
code = translateInterp(pCxt, pSelect);
}
if (TSDB_CODE_SUCCESS == code) {
code = translateForecast(pCxt, pSelect);
}
if (TSDB_CODE_SUCCESS == code) {
code = appendTsForImplicitTsFunc(pCxt, pSelect);
}
@ -7895,6 +8062,19 @@ static int32_t fillCmdSql(STranslateContext* pCxt, int16_t msgType, void* pReq)
break;
}
case TDMT_MND_CREATE_ANODE: {
FILL_CMD_SQL(sql, sqlLen, pCmdReq, SMCreateAnodeReq, pReq);
break;
}
case TDMT_MND_DROP_ANODE: {
FILL_CMD_SQL(sql, sqlLen, pCmdReq, SMDropAnodeReq, pReq);
break;
}
case TDMT_MND_UPDATE_ANODE: {
FILL_CMD_SQL(sql, sqlLen, pCmdReq, SMUpdateAnodeReq, pReq);
break;
}
case TDMT_MND_CREATE_MNODE: {
FILL_CMD_SQL(sql, sqlLen, pCmdReq, SMCreateMnodeReq, pReq);
break;
@ -9398,6 +9578,43 @@ static int32_t translateDropUser(STranslateContext* pCxt, SDropUserStmt* pStmt)
return code;
}
static int32_t translateCreateAnode(STranslateContext* pCxt, SCreateAnodeStmt* pStmt) {
SMCreateAnodeReq createReq = {0};
createReq.urlLen = strlen(pStmt->url) + 1;
if (createReq.urlLen > TSDB_ANAL_ANODE_URL_LEN) {
return TSDB_CODE_MND_ANODE_TOO_LONG_URL;
}
createReq.url = taosMemoryCalloc(createReq.urlLen, 1);
if (createReq.url == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
}
tstrncpy(createReq.url, pStmt->url, createReq.urlLen);
int32_t code = buildCmdMsg(pCxt, TDMT_MND_CREATE_ANODE, (FSerializeFunc)tSerializeSMCreateAnodeReq, &createReq);
tFreeSMCreateAnodeReq(&createReq);
return code;
}
static int32_t translateDropAnode(STranslateContext* pCxt, SDropAnodeStmt* pStmt) {
SMDropAnodeReq dropReq = {0};
dropReq.anodeId = pStmt->anodeId;
int32_t code = buildCmdMsg(pCxt, TDMT_MND_DROP_ANODE, (FSerializeFunc)tSerializeSMDropAnodeReq, &dropReq);
tFreeSMDropAnodeReq(&dropReq);
return code;
}
static int32_t translateUpdateAnode(STranslateContext* pCxt, SUpdateAnodeStmt* pStmt) {
SMUpdateAnodeReq updateReq = {0};
updateReq.anodeId = pStmt->anodeId;
int32_t code = buildCmdMsg(pCxt, TDMT_MND_UPDATE_ANODE, (FSerializeFunc)tSerializeSMUpdateAnodeReq, &updateReq);
tFreeSMUpdateAnodeReq(&updateReq);
return code;
}
static int32_t translateCreateDnode(STranslateContext* pCxt, SCreateDnodeStmt* pStmt) {
SCreateDnodeReq createReq = {0};
strcpy(createReq.fqdn, pStmt->fqdn);
@ -9820,7 +10037,7 @@ static int32_t translateDropComponentNode(STranslateContext* pCxt, SDropComponen
}
static int32_t checkTopicQuery(STranslateContext* pCxt, SSelectStmt* pSelect) {
if (pSelect->hasAggFuncs || pSelect->hasInterpFunc || pSelect->hasIndefiniteRowsFunc) {
if (pSelect->hasAggFuncs || pSelect->hasForecastFunc || pSelect->hasInterpFunc || pSelect->hasIndefiniteRowsFunc) {
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_TOPIC_QUERY);
}
return TSDB_CODE_SUCCESS;
@ -10186,7 +10403,8 @@ static int32_t translateKillTransaction(STranslateContext* pCxt, SKillStmt* pStm
static bool crossTableWithoutAggOper(SSelectStmt* pSelect) {
return NULL == pSelect->pWindow && !pSelect->hasAggFuncs && !pSelect->hasIndefiniteRowsFunc &&
!pSelect->hasInterpFunc && TSDB_SUPER_TABLE == ((SRealTableNode*)pSelect->pFromTable)->pMeta->tableType &&
!pSelect->hasInterpFunc && !pSelect->hasForecastFunc &&
TSDB_SUPER_TABLE == ((SRealTableNode*)pSelect->pFromTable)->pMeta->tableType &&
!hasTbnameFunction(pSelect->pPartitionByList);
}
@ -12389,6 +12607,15 @@ static int32_t translateQuery(STranslateContext* pCxt, SNode* pNode) {
case QUERY_NODE_ALTER_DNODE_STMT:
code = translateAlterDnode(pCxt, (SAlterDnodeStmt*)pNode);
break;
case QUERY_NODE_CREATE_ANODE_STMT:
code = translateCreateAnode(pCxt, (SCreateAnodeStmt*)pNode);
break;
case QUERY_NODE_DROP_ANODE_STMT:
code = translateDropAnode(pCxt, (SDropAnodeStmt*)pNode);
break;
case QUERY_NODE_UPDATE_ANODE_STMT:
code = translateUpdateAnode(pCxt, (SUpdateAnodeStmt*)pNode);
break;
case QUERY_NODE_CREATE_INDEX_STMT:
code = translateCreateIndex(pCxt, (SCreateIndexStmt*)pNode);
break;
@ -15749,6 +15976,8 @@ static int32_t rewriteQuery(STranslateContext* pCxt, SQuery* pQuery) {
case QUERY_NODE_SHOW_MNODES_STMT:
case QUERY_NODE_SHOW_MODULES_STMT:
case QUERY_NODE_SHOW_QNODES_STMT:
case QUERY_NODE_SHOW_ANODES_STMT:
case QUERY_NODE_SHOW_ANODES_FULL_STMT:
case QUERY_NODE_SHOW_FUNCTIONS_STMT:
case QUERY_NODE_SHOW_INDEXES_STMT:
case QUERY_NODE_SHOW_STREAMS_STMT:

2
source/libs/parser/src/parUtil.c
View File

@ -185,6 +185,8 @@ static char* getSyntaxErrFormat(int32_t errCode) {
return "%s is not supported in system table query";
case TSDB_CODE_PAR_INVALID_INTERP_CLAUSE:
return "Invalid usage of RANGE clause, EVERY clause or FILL clause";
case TSDB_CODE_PAR_INVALID_FORECAST_CLAUSE:
return "Invalid usage of forecast clause";
case TSDB_CODE_PAR_NO_VALID_FUNC_IN_WIN:
return "No valid function in window query";
case TSDB_CODE_PAR_INVALID_OPTR_USAGE:

11273
source/libs/parser/src/sql.c
View File

File diff suppressed because it is too large Load Diff

86
source/libs/planner/src/planLogicCreater.c
View File

@ -973,6 +973,45 @@ static int32_t createInterpFuncLogicNode(SLogicPlanContext* pCxt, SSelectStmt* p
return code;
}
static bool isForecastFunc(int32_t funcId) {
return fmIsForecastFunc(funcId) || fmIsForecastPseudoColumnFunc(funcId) || fmIsGroupKeyFunc(funcId) || fmisSelectGroupConstValueFunc(funcId);
}
static int32_t createForecastFuncLogicNode(SLogicPlanContext* pCxt, SSelectStmt* pSelect, SLogicNode** pLogicNode) {
if (!pSelect->hasForecastFunc) {
return TSDB_CODE_SUCCESS;
}
SForecastFuncLogicNode* pForecastFunc = NULL;
int32_t code = nodesMakeNode(QUERY_NODE_LOGIC_PLAN_FORECAST_FUNC, (SNode**)&pForecastFunc);
if (NULL == pForecastFunc) {
return code;
}
pForecastFunc->node.groupAction = getGroupAction(pCxt, pSelect);
pForecastFunc->node.requireDataOrder = getRequireDataOrder(true, pSelect);
pForecastFunc->node.resultDataOrder = pForecastFunc->node.requireDataOrder;
// interp functions and _group_key functions
code = nodesCollectFuncs(pSelect, SQL_CLAUSE_SELECT, NULL, isForecastFunc, &pForecastFunc->pFuncs);
if (TSDB_CODE_SUCCESS == code) {
code = rewriteExprsForSelect(pForecastFunc->pFuncs, pSelect, SQL_CLAUSE_SELECT, NULL);
}
// set the output
if (TSDB_CODE_SUCCESS == code) {
code = createColumnByRewriteExprs(pForecastFunc->pFuncs, &pForecastFunc->node.pTargets);
}
if (TSDB_CODE_SUCCESS == code) {
*pLogicNode = (SLogicNode*)pForecastFunc;
} else {
nodesDestroyNode((SNode*)pForecastFunc);
}
return code;
}
static int32_t createWindowLogicNodeFinalize(SLogicPlanContext* pCxt, SSelectStmt* pSelect, SWindowLogicNode* pWindow,
SLogicNode** pLogicNode) {
if (pCxt->pPlanCxt->streamQuery) {
@ -1174,6 +1213,48 @@ static int32_t createWindowLogicNodeByCount(SLogicPlanContext* pCxt, SCountWindo
return createWindowLogicNodeFinalize(pCxt, pSelect, pWindow, pLogicNode);
}
static int32_t createWindowLogicNodeByAnomaly(SLogicPlanContext* pCxt, SAnomalyWindowNode* pAnomaly,
SSelectStmt* pSelect, SLogicNode** pLogicNode) {
SWindowLogicNode* pWindow = NULL;
int32_t code = nodesMakeNode(QUERY_NODE_LOGIC_PLAN_WINDOW, (SNode**)&pWindow);
if (NULL == pWindow) {
return code;
}
pWindow->winType = WINDOW_TYPE_ANOMALY;
pWindow->node.groupAction = getGroupAction(pCxt, pSelect);
pWindow->node.requireDataOrder =
pCxt->pPlanCxt->streamQuery ? DATA_ORDER_LEVEL_IN_BLOCK : getRequireDataOrder(true, pSelect);
pWindow->node.resultDataOrder =
pCxt->pPlanCxt->streamQuery ? DATA_ORDER_LEVEL_GLOBAL : pWindow->node.requireDataOrder;
pWindow->pAnomalyExpr = NULL;
code = nodesCloneNode(pAnomaly->pExpr, &pWindow->pAnomalyExpr);
if (TSDB_CODE_SUCCESS != code) {
nodesDestroyNode((SNode*)pWindow);
return code;
}
tstrncpy(pWindow->anomalyOpt, pAnomaly->anomalyOpt, sizeof(pWindow->anomalyOpt));
pWindow->pTspk = NULL;
code = nodesCloneNode(pAnomaly->pCol, &pWindow->pTspk);
if (NULL == pWindow->pTspk) {
nodesDestroyNode((SNode*)pWindow);
return code;
}
// rewrite the expression in subsequent clauses
code = rewriteExprForSelect(pWindow->pAnomalyExpr, pSelect, SQL_CLAUSE_WINDOW);
if (TSDB_CODE_SUCCESS == code) {
code = createWindowLogicNodeFinalize(pCxt, pSelect, pWindow, pLogicNode);
} else {
nodesDestroyNode((SNode*)pWindow);
}
return code;
}
static int32_t createWindowLogicNode(SLogicPlanContext* pCxt, SSelectStmt* pSelect, SLogicNode** pLogicNode) {
if (NULL == pSelect->pWindow) {
return TSDB_CODE_SUCCESS;
@ -1189,6 +1270,8 @@ static int32_t createWindowLogicNode(SLogicPlanContext* pCxt, SSelectStmt* pSele
return createWindowLogicNodeByEvent(pCxt, (SEventWindowNode*)pSelect->pWindow, pSelect, pLogicNode);
case QUERY_NODE_COUNT_WINDOW:
return createWindowLogicNodeByCount(pCxt, (SCountWindowNode*)pSelect->pWindow, pSelect, pLogicNode);
case QUERY_NODE_ANOMALY_WINDOW:
return createWindowLogicNodeByAnomaly(pCxt, (SAnomalyWindowNode*)pSelect->pWindow, pSelect, pLogicNode);
default:
break;
}
@ -1600,6 +1683,9 @@ static int32_t createSelectFromLogicNode(SLogicPlanContext* pCxt, SSelectStmt* p
if (TSDB_CODE_SUCCESS == code) {
code = createSelectRootLogicNode(pCxt, pSelect, createInterpFuncLogicNode, &pRoot);
}
if (TSDB_CODE_SUCCESS == code) {
code = createSelectRootLogicNode(pCxt, pSelect, createForecastFuncLogicNode, &pRoot);
}
if (TSDB_CODE_SUCCESS == code) {
code = createSelectRootLogicNode(pCxt, pSelect, createDistinctLogicNode, &pRoot);
}

2
source/libs/planner/src/planOptimizer.c
View File

@ -2380,6 +2380,8 @@ static bool sortPriKeyOptHasUnsupportedPkFunc(SLogicNode* pLogicNode, EOrder sor
case QUERY_NODE_LOGIC_PLAN_INTERP_FUNC:
pFuncList = ((SInterpFuncLogicNode*)pLogicNode)->pFuncs;
break;
case QUERY_NODE_LOGIC_PLAN_FORECAST_FUNC:
pFuncList = ((SForecastFuncLogicNode*)pLogicNode)->pFuncs;
default:
break;
}

95
source/libs/planner/src/planPhysiCreater.c
View File

@ -1990,6 +1990,50 @@ static int32_t createInterpFuncPhysiNode(SPhysiPlanContext* pCxt, SNodeList* pCh
return code;
}
static int32_t createForecastFuncPhysiNode(SPhysiPlanContext* pCxt, SNodeList* pChildren,
SForecastFuncLogicNode* pFuncLogicNode, SPhysiNode** pPhyNode) {
SForecastFuncPhysiNode* pForecastFunc =
(SForecastFuncPhysiNode*)makePhysiNode(pCxt, (SLogicNode*)pFuncLogicNode, QUERY_NODE_PHYSICAL_PLAN_FORECAST_FUNC);
if (NULL == pForecastFunc) {
return terrno;
}
SNodeList* pPrecalcExprs = NULL;
SNodeList* pFuncs = NULL;
int32_t code = rewritePrecalcExprs(pCxt, pFuncLogicNode->pFuncs, &pPrecalcExprs, &pFuncs);
SDataBlockDescNode* pChildTupe = (((SPhysiNode*)nodesListGetNode(pChildren, 0))->pOutputDataBlockDesc);
// push down expression to pOutputDataBlockDesc of child node
if (TSDB_CODE_SUCCESS == code && NULL != pPrecalcExprs) {
code = setListSlotId(pCxt, pChildTupe->dataBlockId, -1, pPrecalcExprs, &pForecastFunc->pExprs);
if (TSDB_CODE_SUCCESS == code) {
code = pushdownDataBlockSlots(pCxt, pForecastFunc->pExprs, pChildTupe);
}
}
if (TSDB_CODE_SUCCESS == code) {
code = setListSlotId(pCxt, pChildTupe->dataBlockId, -1, pFuncs, &pForecastFunc->pFuncs);
if (TSDB_CODE_SUCCESS == code) {
code = addDataBlockSlots(pCxt, pForecastFunc->pFuncs, pForecastFunc->node.pOutputDataBlockDesc);
}
}
if (TSDB_CODE_SUCCESS == code) {
code = setConditionsSlotId(pCxt, (const SLogicNode*)pFuncLogicNode, (SPhysiNode*)pForecastFunc);
}
if (TSDB_CODE_SUCCESS == code) {
*pPhyNode = (SPhysiNode*)pForecastFunc;
} else {
nodesDestroyNode((SNode*)pForecastFunc);
}
nodesDestroyList(pPrecalcExprs);
nodesDestroyList(pFuncs);
return code;
}
static bool projectCanMergeDataBlock(SProjectLogicNode* pProject) {
if (GROUP_ACTION_KEEP == pProject->node.groupAction) {
return false;
@ -2325,6 +2369,53 @@ static int32_t createCountWindowPhysiNode(SPhysiPlanContext* pCxt, SNodeList* pC
return code;
}
static int32_t createAnomalyWindowPhysiNode(SPhysiPlanContext* pCxt, SNodeList* pChildren,
SWindowLogicNode* pWindowLogicNode, SPhysiNode** pPhyNode) {
SAnomalyWindowPhysiNode* pAnomaly = (SAnomalyWindowPhysiNode*)makePhysiNode(
pCxt, (SLogicNode*)pWindowLogicNode,
(pCxt->pPlanCxt->streamQuery ? QUERY_NODE_PHYSICAL_PLAN_STREAM_ANOMALY : QUERY_NODE_PHYSICAL_PLAN_MERGE_ANOMALY));
if (NULL == pAnomaly) {
return terrno;
}
SNodeList* pPrecalcExprs = NULL;
SNode* pAnomalyKey = NULL;
int32_t code = rewritePrecalcExpr(pCxt, pWindowLogicNode->pAnomalyExpr, &pPrecalcExprs, &pAnomalyKey);
SDataBlockDescNode* pChildTupe = (((SPhysiNode*)nodesListGetNode(pChildren, 0))->pOutputDataBlockDesc);
// push down expression to pOutputDataBlockDesc of child node
if (TSDB_CODE_SUCCESS == code && NULL != pPrecalcExprs) {
code = setListSlotId(pCxt, pChildTupe->dataBlockId, -1, pPrecalcExprs, &pAnomaly->window.pExprs);
if (TSDB_CODE_SUCCESS == code) {
code = addDataBlockSlots(pCxt, pAnomaly->window.pExprs, pChildTupe);
}
}
if (TSDB_CODE_SUCCESS == code) {
code = setNodeSlotId(pCxt, pChildTupe->dataBlockId, -1, pAnomalyKey, &pAnomaly->pAnomalyKey);
// if (TSDB_CODE_SUCCESS == code) {
// code = addDataBlockSlot(pCxt, &pAnomaly->pAnomalyKey, pAnomaly->window.node.pOutputDataBlockDesc);
// }
}
tstrncpy(pAnomaly->anomalyOpt, pWindowLogicNode->anomalyOpt, sizeof(pAnomaly->anomalyOpt));
if (TSDB_CODE_SUCCESS == code) {
code = createWindowPhysiNodeFinalize(pCxt, pChildren, &pAnomaly->window, pWindowLogicNode);
}
if (TSDB_CODE_SUCCESS == code) {
*pPhyNode = (SPhysiNode*)pAnomaly;
} else {
nodesDestroyNode((SNode*)pAnomaly);
}
nodesDestroyList(pPrecalcExprs);
nodesDestroyNode(pAnomalyKey);
return code;
}
static int32_t createWindowPhysiNode(SPhysiPlanContext* pCxt, SNodeList* pChildren, SWindowLogicNode* pWindowLogicNode,
SPhysiNode** pPhyNode) {
switch (pWindowLogicNode->winType) {
@ -2338,6 +2429,8 @@ static int32_t createWindowPhysiNode(SPhysiPlanContext* pCxt, SNodeList* pChildr
return createEventWindowPhysiNode(pCxt, pChildren, pWindowLogicNode, pPhyNode);
case WINDOW_TYPE_COUNT:
return createCountWindowPhysiNode(pCxt, pChildren, pWindowLogicNode, pPhyNode);
case WINDOW_TYPE_ANOMALY:
return createAnomalyWindowPhysiNode(pCxt, pChildren, pWindowLogicNode, pPhyNode);
default:
break;
}
@ -2652,6 +2745,8 @@ static int32_t doCreatePhysiNode(SPhysiPlanContext* pCxt, SLogicNode* pLogicNode
return createIndefRowsFuncPhysiNode(pCxt, pChildren, (SIndefRowsFuncLogicNode*)pLogicNode, pPhyNode);
case QUERY_NODE_LOGIC_PLAN_INTERP_FUNC:
return createInterpFuncPhysiNode(pCxt, pChildren, (SInterpFuncLogicNode*)pLogicNode, pPhyNode);
case QUERY_NODE_LOGIC_PLAN_FORECAST_FUNC:
return createForecastFuncPhysiNode(pCxt, pChildren, (SForecastFuncLogicNode*)pLogicNode, pPhyNode);
case QUERY_NODE_LOGIC_PLAN_MERGE:
return createMergePhysiNode(pCxt, pChildren, (SMergeLogicNode*)pLogicNode, pPhyNode);
case QUERY_NODE_LOGIC_PLAN_GROUP_CACHE:

17
source/libs/planner/src/planSpliter.c
View File

@ -939,6 +939,18 @@ static int32_t stbSplSplitCount(SSplitContext* pCxt, SStableSplitInfo* pInfo) {
}
}
static int32_t stbSplSplitAnomalyForStream(SSplitContext* pCxt, SStableSplitInfo* pInfo) {
return TSDB_CODE_PLAN_INTERNAL_ERROR;
}
static int32_t stbSplSplitAnomaly(SSplitContext* pCxt, SStableSplitInfo* pInfo) {
if (pCxt->pPlanCxt->streamQuery) {
return stbSplSplitAnomalyForStream(pCxt, pInfo);
} else {
return stbSplSplitSessionOrStateForBatch(pCxt, pInfo);
}
}
static int32_t stbSplSplitWindowForCrossTable(SSplitContext* pCxt, SStableSplitInfo* pInfo) {
switch (((SWindowLogicNode*)pInfo->pSplitNode)->winType) {
case WINDOW_TYPE_INTERVAL:
@ -951,6 +963,8 @@ static int32_t stbSplSplitWindowForCrossTable(SSplitContext* pCxt, SStableSplitI
return stbSplSplitEvent(pCxt, pInfo);
case WINDOW_TYPE_COUNT:
return stbSplSplitCount(pCxt, pInfo);
case WINDOW_TYPE_ANOMALY:
return stbSplSplitAnomaly(pCxt, pInfo);
default:
break;
}
@ -2000,7 +2014,8 @@ typedef struct SQnodeSplitInfo {
static bool qndSplFindSplitNode(SSplitContext* pCxt, SLogicSubplan* pSubplan, SLogicNode* pNode,
SQnodeSplitInfo* pInfo) {
if (QUERY_NODE_LOGIC_PLAN_SCAN == nodeType(pNode) && NULL != pNode->pParent &&
QUERY_NODE_LOGIC_PLAN_INTERP_FUNC != nodeType(pNode->pParent) && ((SScanLogicNode*)pNode)->scanSeq[0] <= 1 &&
QUERY_NODE_LOGIC_PLAN_INTERP_FUNC != nodeType(pNode->pParent) &&
QUERY_NODE_LOGIC_PLAN_FORECAST_FUNC != nodeType(pNode->pParent) && ((SScanLogicNode*)pNode)->scanSeq[0] <= 1 &&
((SScanLogicNode*)pNode)->scanSeq[1] <= 1) {
pInfo->pSplitNode = pNode;
pInfo->pSubplan = pSubplan;

23
source/libs/planner/src/planUtil.c
View File

@ -256,6 +256,15 @@ static int32_t adjustCountDataRequirement(SWindowLogicNode* pWindow, EDataOrderL
return TSDB_CODE_SUCCESS;
}
static int32_t adjustAnomalyDataRequirement(SWindowLogicNode* pWindow, EDataOrderLevel requirement) {
if (requirement <= pWindow->node.resultDataOrder) {
return TSDB_CODE_SUCCESS;
}
pWindow->node.resultDataOrder = requirement;
pWindow->node.requireDataOrder = requirement;
return TSDB_CODE_SUCCESS;
}
static int32_t adjustWindowDataRequirement(SWindowLogicNode* pWindow, EDataOrderLevel requirement) {
switch (pWindow->winType) {
case WINDOW_TYPE_INTERVAL:
@ -268,6 +277,8 @@ static int32_t adjustWindowDataRequirement(SWindowLogicNode* pWindow, EDataOrder
return adjustEventDataRequirement(pWindow, requirement);
case WINDOW_TYPE_COUNT:
return adjustCountDataRequirement(pWindow, requirement);
case WINDOW_TYPE_ANOMALY:
return adjustAnomalyDataRequirement(pWindow, requirement);
default:
break;
}
@ -318,6 +329,15 @@ static int32_t adjustInterpDataRequirement(SInterpFuncLogicNode* pInterp, EDataO
return TSDB_CODE_SUCCESS;
}
static int32_t adjustForecastDataRequirement(SForecastFuncLogicNode* pForecast, EDataOrderLevel requirement) {
if (requirement <= pForecast->node.requireDataOrder) {
return TSDB_CODE_SUCCESS;
}
pForecast->node.resultDataOrder = requirement;
pForecast->node.requireDataOrder = requirement;
return TSDB_CODE_SUCCESS;
}
int32_t adjustLogicNodeDataRequirement(SLogicNode* pNode, EDataOrderLevel requirement) {
int32_t code = TSDB_CODE_SUCCESS;
switch (nodeType(pNode)) {
@ -355,6 +375,9 @@ int32_t adjustLogicNodeDataRequirement(SLogicNode* pNode, EDataOrderLevel requir
case QUERY_NODE_LOGIC_PLAN_INTERP_FUNC:
code = adjustInterpDataRequirement((SInterpFuncLogicNode*)pNode, requirement);
break;
case QUERY_NODE_LOGIC_PLAN_FORECAST_FUNC:
code = adjustForecastDataRequirement((SForecastFuncLogicNode*)pNode, requirement);
break;
default:
break;
}

2
source/libs/qworker/src/qwUtil.c
View File

@ -539,7 +539,7 @@ int32_t qwSaveTbVersionInfo(qTaskInfo_t pTaskInfo, SQWTaskCtx *ctx) {
}
if (dbFName[0] && tbName[0]) {
(void)sprintf(tbInfo.tbFName, "%s.%s", dbFName, tbName);
(void)snprintf(tbInfo.tbFName, sizeof(tbInfo.tbFName), "%s.%s", dbFName, tbName);
} else {
tbInfo.tbFName[0] = 0;
}

4
source/libs/scalar/src/sclfunc.c
View File

@ -2193,7 +2193,7 @@ int32_t toISO8601Function(SScalarParam *pInput, int32_t inputNum, SScalarParam *
NUM_TO_STRING(type, input, sizeof(fraction), fraction);
int32_t fractionLen;
char buf[64] = {0};
char buf[TD_TIME_STR_LEN] = {0};
int64_t timeVal;
char* format = NULL;
int64_t quot = 0;
@ -2244,7 +2244,7 @@ int32_t toISO8601Function(SScalarParam *pInput, int32_t inputNum, SScalarParam *
struct tm tmInfo;
int32_t len = 0;
if (taosLocalTime((const time_t *)&quot, &tmInfo, buf) == NULL) {
if (taosLocalTime((const time_t *)&quot, &tmInfo, buf, sizeof(buf)) == NULL) {
len = (int32_t)strlen(buf);
goto _end;
}

2
source/libs/scheduler/src/schRemote.c
View File

@ -729,7 +729,7 @@ int32_t schMakeHbCallbackParam(SSchJob *pJob, SSchTask *pTask, void **pParam) {
param->nodeEpId.nodeId = addr->nodeId;
SEp* pEp = SCH_GET_CUR_EP(addr);
strcpy(param->nodeEpId.ep.fqdn, pEp->fqdn);
tstrncpy(param->nodeEpId.ep.fqdn, pEp->fqdn, sizeof(param->nodeEpId.ep.fqdn));
param->nodeEpId.ep.port = pEp->port;
param->pTrans = pJob->pTrans;

4
source/libs/scheduler/src/schTask.c
View File

@ -288,7 +288,7 @@ int32_t schProcessOnTaskSuccess(SSchJob *pJob, SSchTask *pTask) {
/*
if (SCH_IS_DATA_SRC_TASK(task) && job->dataSrcEps.numOfEps < SCH_MAX_CANDIDATE_EP_NUM) {
strncpy(job->dataSrcEps.fqdn[job->dataSrcEps.numOfEps], task->execAddr.fqdn, sizeof(task->execAddr.fqdn));
tstrncpy(job->dataSrcEps.fqdn[job->dataSrcEps.numOfEps], task->execAddr.fqdn, sizeof(task->execAddr.fqdn));
job->dataSrcEps.port[job->dataSrcEps.numOfEps] = task->execAddr.port;
++job->dataSrcEps.numOfEps;
@ -840,7 +840,7 @@ int32_t schSetTaskCandidateAddrs(SSchJob *pJob, SSchTask *pTask) {
/*
for (int32_t i = 0; i < job->dataSrcEps.numOfEps && addNum < SCH_MAX_CANDIDATE_EP_NUM; ++i) {
strncpy(epSet->fqdn[epSet->numOfEps], job->dataSrcEps.fqdn[i], sizeof(job->dataSrcEps.fqdn[i]));
tstrncpy(epSet->fqdn[epSet->numOfEps], job->dataSrcEps.fqdn[i], sizeof(job->dataSrcEps.fqdn[i]));
epSet->port[epSet->numOfEps] = job->dataSrcEps.port[i];
++epSet->numOfEps;

2
source/libs/sync/src/syncUtil.c
View File

@ -60,7 +60,7 @@ bool syncUtilNodeInfo2RaftId(const SNodeInfo* pInfo, SyncGroupId vgId, SRaftId*
return false;
}
char ipbuf[128] = {0};
char ipbuf[TD_IP_LEN] = {0};
tinet_ntoa(ipbuf, ipv4);
raftId->addr = SYNC_ADDR(pInfo);
raftId->vgId = vgId;

2
source/libs/transport/src/thttp.c
View File

@ -223,7 +223,7 @@ static FORCE_INLINE int32_t taosBuildDstAddr(const char* server, uint16_t port,
tError("http-report failed to resolving domain names %s, reason: %s", server, tstrerror(code));
return TSDB_CODE_RPC_FQDN_ERROR;
}
char buf[256] = {0};
char buf[TD_IP_LEN] = {0};
tinet_ntoa(buf, ip);
int ret = uv_ip4_addr(buf, port, dest);
if (ret != 0) {

2
source/libs/transport/src/transCli.c
View File

@ -1790,7 +1790,7 @@ static FORCE_INLINE int32_t cliUpdateFqdnCache(SHashObj* cache, char* fqdn) {
size_t len = strlen(fqdn);
uint32_t* v = taosHashGet(cache, fqdn, len);
if (addr != *v) {
char old[64] = {0}, new[64] = {0};
char old[TD_IP_LEN] = {0}, new[TD_IP_LEN] = {0};
tinet_ntoa(old, *v);
tinet_ntoa(new, addr);
tWarn("update ip of fqdn:%s, old: %s, new: %s", fqdn, old, new);

15
source/os/src/osDir.c
View File

@ -151,7 +151,7 @@ int32_t taosMulMkDir(const char *dirname) {
}
if (temp[1] == ':') pos += 3;
#else
(void)strcpy(temp, dirname);
tstrncpy(temp, dirname, sizeof(temp));
#endif
if (taosDirExist(temp)) return code;
@ -216,7 +216,7 @@ int32_t taosMulModeMkDir(const char *dirname, int mode, bool checkAccess) {
}
if (temp[1] == ':') pos += 3;
#else
(void)strcpy(temp, dirname);
tstrncpy(temp, dirname, sizeof(temp));
#endif
if (taosDirExist(temp)) {
@ -341,7 +341,7 @@ int32_t taosExpandDir(const char *dirname, char *outname, int32_t maxlen) {
}
if (full_path.we_wordv != NULL && full_path.we_wordv[0] != NULL) {
(void)strncpy(outname, full_path.we_wordv[0], maxlen);
tstrncpy(outname, full_path.we_wordv[0], maxlen);
}
wordfree(&full_path);
@ -358,9 +358,9 @@ int32_t taosRealPath(char *dirname, char *realPath, int32_t maxlen) {
#endif
if (strlen(tmp) < maxlen) {
if (realPath == NULL) {
(void)strncpy(dirname, tmp, maxlen);
tstrncpy(dirname, tmp, maxlen);
} else {
(void)strncpy(realPath, tmp, maxlen);
tstrncpy(realPath, tmp, maxlen);
}
return 0;
}
@ -440,8 +440,7 @@ TdDirPtr taosOpenDir(const char *dirname) {
return NULL;
}
strcpy(szFind, dirname);
strcat(szFind, "\\*.*"); //利用通配符找这个目录下的所以文件，包括目录
snprintf(szFind, sizeof(szFind), "%s%s", dirname, "\\*.*"); //利用通配符找这个目录下的所以文件，包括目录
pDir->hFind = FindFirstFile(szFind, &(pDir->dirEntry.findFileData));
if (INVALID_HANDLE_VALUE == pDir->hFind) {
@ -560,6 +559,6 @@ void taosGetCwd(char *buf, int32_t len) {
char *unused __attribute__((unused));
unused = getcwd(buf, len - 1);
#else
strncpy(buf, "not implemented on windows", len - 1);
tstrncpy(buf, "not implemented on windows", len);
#endif
}

18
source/os/src/osEnv.c
View File

@ -77,21 +77,21 @@ int32_t osDefaultInit() {
tmpDir = getenv("temp");
}
if (tmpDir != NULL) {
(void)strcpy(tsTempDir, tmpDir);
tstrncpy(tsTempDir, tmpDir, sizeof(tsTempDir));
}
(void)strcpy(tsOsName, "Windows");
tstrncpy(tsOsName, "Windows", sizeof(tsOsName));
#elif defined(_TD_DARWIN_64)
(void)strcpy(tsOsName, "Darwin");
tstrncpy(tsOsName, "Darwin", sizeof(tsOsName));
#else
(void)strcpy(tsOsName, "Linux");
tstrncpy(tsOsName, "Linux", sizeof(tsOsName));
#endif
if (configDir[0] == 0) {
(void)strcpy(configDir, TD_CFG_DIR_PATH);
tstrncpy(configDir, TD_CFG_DIR_PATH, sizeof(configDir));
}
(void)strcpy(tsDataDir, TD_DATA_DIR_PATH);
(void)strcpy(tsLogDir, TD_LOG_DIR_PATH);
if(strlen(tsTempDir) == 0){
(void)strcpy(tsTempDir, TD_TMP_DIR_PATH);
tstrncpy(tsDataDir, TD_DATA_DIR_PATH, sizeof(tsDataDir));
tstrncpy(tsLogDir, TD_LOG_DIR_PATH, sizeof(tsLogDir));
if (strlen(tsTempDir) == 0){
tstrncpy(tsTempDir, TD_TMP_DIR_PATH, sizeof(tsTempDir));
}
return code;

14
source/os/src/osFile.c
View File

@ -91,11 +91,7 @@ void taosGetTmpfilePath(const char *inputTmpDir, const char *fileNamePrefix, cha
tmpPath[len++] = '\\';
}
strcpy(tmpPath + len, TD_TMP_FILE_PREFIX);
if (strlen(tmpPath) + strlen(fileNamePrefix) + strlen("-%d-%s") < PATH_MAX) {
strcat(tmpPath, fileNamePrefix);
strcat(tmpPath, "-%d-%s");
}
snprintf(tmpPath + len, sizeof(tmpPath) - len, "%s%s%s", TD_TMP_FILE_PREFIX, fileNamePrefix, "-%d-%s");
char rand[8] = {0};
taosRandStr(rand, tListLen(rand) - 1);
@ -112,15 +108,11 @@ void taosGetTmpfilePath(const char *inputTmpDir, const char *fileNamePrefix, cha
tmpPath[len++] = '/';
}
(void)strcpy(tmpPath + len, TD_TMP_FILE_PREFIX);
if (strlen(tmpPath) + strlen(fileNamePrefix) + strlen("-%d-%s") < PATH_MAX) {
(void)strcat(tmpPath, fileNamePrefix);
(void)strcat(tmpPath, "-%d-%s");
}
snprintf(tmpPath + len, sizeof(tmpPath) - len, "%s%s%s", TD_TMP_FILE_PREFIX, fileNamePrefix, "-%d-%s");
char rand[32] = {0};
(void)sprintf(rand, "%" PRIu64, atomic_add_fetch_64(&seqId, 1));
(void)snprintf(rand, sizeof(rand), "%" PRIu64, atomic_add_fetch_64(&seqId, 1));
(void)snprintf(dstPath, PATH_MAX, tmpPath, getpid(), rand);

16
source/os/src/osLocale.c
View File

@ -95,7 +95,7 @@ void taosGetSystemLocale(char *outLocale, char *outCharset) {
if (locale != NULL) {
tstrncpy(outLocale, locale, TD_LOCALE_LEN);
}
strcpy(outCharset, "UTF-8");
tstrncpy(outCharset, "UTF-8", TD_CHARSET_LEN);
#elif defined(_TD_DARWIN_64)
/*
@ -123,7 +123,7 @@ void taosGetSystemLocale(char *outLocale, char *outCharset) {
locale = setlocale(LC_CTYPE, "");
if (locale == NULL) {
// printf("can't get locale from system, set it to en_US.UTF-8 since error:%d:%s", errno, strerror(errno));
strcpy(outLocale, "en_US.UTF-8");
tstrncpy(outLocale, "en_US.UTF-8", TD_LOCALE_LEN);
} else {
tstrncpy(outLocale, locale, TD_LOCALE_LEN);
// printf("locale not configured, set to system default:%s", outLocale);
@ -137,7 +137,7 @@ void taosGetSystemLocale(char *outLocale, char *outCharset) {
char *revisedCharset = taosCharsetReplace(str);
if (NULL == revisedCharset) {
(void)strcpy(outCharset, "UTF-8");
tstrncpy(outCharset, "UTF-8", TD_CHARSET_LEN);
} else {
tstrncpy(outCharset, revisedCharset, TD_CHARSET_LEN);
@ -145,7 +145,7 @@ void taosGetSystemLocale(char *outLocale, char *outCharset) {
}
// printf("charset not configured, set to system default:%s", outCharset);
} else {
strcpy(outCharset, "UTF-8");
tstrncpy(outCharset, "UTF-8", TD_CHARSET_LEN);
// printf("can't get locale and charset from system, set it to UTF-8");
}
@ -173,7 +173,7 @@ void taosGetSystemLocale(char *outLocale, char *outCharset) {
locale = setlocale(LC_CTYPE, "");
if (locale == NULL) {
// printf("can't get locale from system, set it to en_US.UTF-8 since error:%d:%s", errno, strerror(errno));
(void)strcpy(outLocale, "en_US.UTF-8");
tstrncpy(outLocale, "en_US.UTF-8", TD_LOCALE_LEN);
} else {
tstrncpy(outLocale, locale, TD_LOCALE_LEN);
//printf("locale not configured, set to system default:%s\n", outLocale);
@ -186,15 +186,15 @@ void taosGetSystemLocale(char *outLocale, char *outCharset) {
char *revisedCharset = taosCharsetReplace(str);
if (NULL == revisedCharset) {
(void)strcpy(outCharset, "UTF-8");
tstrncpy(outCharset, "UTF-8", TD_CHARSET_LEN);
} else {
tstrncpy(outCharset, revisedCharset, TD_LOCALE_LEN);
tstrncpy(outCharset, revisedCharset, TD_CHARSET_LEN);
taosMemoryFree(revisedCharset);
}
// printf("charset not configured, set to system default:%s", outCharset);
} else {
(void)strcpy(outCharset, "UTF-8");
tstrncpy(outCharset, "UTF-8", TD_CHARSET_LEN);
// printf("can't get locale and charset from system, set it to UTF-8");
}

28
source/os/src/osSemaphore.c
View File

@ -377,20 +377,20 @@ int32_t tsem2_wait(tsem2_t* sem) {
}
int32_t tsem2_timewait(tsem2_t* sem, int64_t ms) {
int ret = 0;
int32_t code = 0;
ret = taosThreadMutexLock(&sem->mutex);
if (ret) {
return ret;
code = taosThreadMutexLock(&sem->mutex);
if (code) {
return code;
}
if (sem->count <= 0) {
struct timespec ts = {0};
if (clock_gettime(CLOCK_MONOTONIC, &ts) == -1) {
ret = TAOS_SYSTEM_ERROR(errno);
code = TAOS_SYSTEM_ERROR(errno);
(void)taosThreadMutexUnlock(&sem->mutex);
terrno = ret;
return ret;
terrno = code;
return code;
}
ts.tv_sec += ms / 1000;
@ -399,22 +399,18 @@ int32_t tsem2_timewait(tsem2_t* sem, int64_t ms) {
ts.tv_nsec %= 1000000000;
while (sem->count <= 0) {
ret = taosThreadCondTimedWait(&sem->cond, &sem->mutex, &ts);
if (ret != 0) {
code = taosThreadCondTimedWait(&sem->cond, &sem->mutex, &ts);
if (code != 0) {
(void)taosThreadMutexUnlock(&sem->mutex);
if (errno == ETIMEDOUT) {
return TSDB_CODE_TIMEOUT_ERROR;
} else {
return TAOS_SYSTEM_ERROR(errno);
}
return code;
}
}
}
sem->count--;
ret = taosThreadMutexUnlock(&sem->mutex);
return ret;
code = taosThreadMutexUnlock(&sem->mutex);
return code;
}
#endif

10
source/os/src/osSocket.c
View File

@ -332,8 +332,8 @@ int32_t taosGetFqdn(char *fqdn) {
// thus, we choose AF_INET (ipv4 for the moment) to make getaddrinfo return
// immediately
// hints.ai_family = AF_INET;
strcpy(fqdn, hostname);
strcpy(fqdn + strlen(hostname), ".local");
tstrncpy(fqdn, hostname, TD_FQDN_LEN);
tstrncpy(fqdn + strlen(hostname), ".local", TD_FQDN_LEN - strlen(hostname));
#else // linux
#endif // linux
@ -361,7 +361,7 @@ int32_t taosGetFqdn(char *fqdn) {
break;
}
(void)strcpy(fqdn, result->ai_canonname);
tstrncpy(fqdn, result->ai_canonname, TD_FQDN_LEN);
freeaddrinfo(result);
@ -375,7 +375,7 @@ int32_t taosGetFqdn(char *fqdn) {
// fprintf(stderr, "failed to get fqdn, code:%d, hostname:%s, reason:%s\n", ret, hostname, gai_strerror(ret));
return TAOS_SYSTEM_WINSOCKET_ERROR(WSAGetLastError());
}
strcpy(fqdn, result->ai_canonname);
tstrncpy(fqdn, result->ai_canonname, TD_FQDN_LEN);
freeaddrinfo(result);
#endif
@ -384,7 +384,7 @@ int32_t taosGetFqdn(char *fqdn) {
}
void tinet_ntoa(char *ipstr, uint32_t ip) {
(void)sprintf(ipstr, "%d.%d.%d.%d", ip & 0xFF, (ip >> 8) & 0xFF, (ip >> 16) & 0xFF, ip >> 24);
(void)snprintf(ipstr, TD_IP_LEN, "%d.%d.%d.%d", ip & 0xFF, (ip >> 8) & 0xFF, (ip >> 16) & 0xFF, ip >> 24);
}
int32_t taosIgnSIGPIPE() {

6
source/os/src/osString.c
View File

@ -71,7 +71,7 @@ char *taosStrndup(const char *s, int size) {
if (l > size) l = size;
s2 = malloc(l + 1);
if (s2) {
strncpy(s2, s, l);
tstrncpy(s2, s, l + 1);
s2[l] = '\0';
} else {
terrno = TSDB_CODE_OUT_OF_MEMORY;
@ -442,14 +442,14 @@ int32_t taosUcs4len(TdUcs4 *ucs4) {
}
// dst buffer size should be at least 2*len + 1
int32_t taosHexEncode(const unsigned char *src, char *dst, int32_t len) {
int32_t taosHexEncode(const unsigned char *src, char *dst, int32_t len, int32_t bufSize) {
if (!dst) {
terrno = TSDB_CODE_INVALID_PARA;
return terrno;
}
for (int32_t i = 0; i < len; ++i) {
(void)sprintf(dst + i * 2, "%02x", src[i]);
(void)snprintf(dst + i * 2, bufSize - i * 2, "%02x", src[i]);
}
return 0;

10
source/os/src/osSysinfo.c
View File

@ -389,10 +389,10 @@ int32_t taosGetOsReleaseName(char *releaseName, char* sName, char* ver, int32_t
}
if (major >= 20) {
major -= 9; // macOS 11 and newer
sprintf(releaseName, "macOS %u.%u", major, minor);
snprintf(releaseName, maxLen, "macOS %u.%u", major, minor);
} else {
major -= 4; // macOS 10.1.1 and newer
sprintf(releaseName, "macOS 10.%d.%d", major, minor);
snprintf(releaseName, maxLen, "macOS 10.%d.%d", major, minor);
}
return 0;
@ -474,7 +474,7 @@ int32_t taosGetCpuInfo(char *cpuModel, int32_t maxLen, float *numOfCores) {
if (taosGetsCmd(pCmd, sizeof(buf) - 1, buf) > 0) {
code = 0;
done |= 2;
*numOfCores = atof(buf);
*numOfCores = taosStr2Float(buf, NULL);
}
taosCloseCmd(&pCmd);
@ -498,7 +498,7 @@ int32_t taosGetCpuInfo(char *cpuModel, int32_t maxLen, float *numOfCores) {
done |= 1;
} else if (((done & 2) == 0) && strncmp(line, "cpu cores", 9) == 0) {
const char *v = strchr(line, ':') + 2;
*numOfCores = atof(v);
*numOfCores = taosStr2Float(v, NULL);
done |= 2;
}
if (strncmp(line, "processor", 9) == 0) coreCount += 1;
@ -1101,7 +1101,7 @@ char *taosGetCmdlineByPID(int pid) {
return cmdline;
#else
static char cmdline[1024];
(void)sprintf(cmdline, "/proc/%d/cmdline", pid);
(void)snprintf(cmdline, sizeof(cmdline), "/proc/%d/cmdline", pid);
// int fd = open(cmdline, O_RDONLY);
TdFilePtr pFile = taosOpenFile(cmdline, TD_FILE_READ);

19
source/os/src/osThread.c
View File

@ -235,19 +235,22 @@ int32_t taosThreadCondWait(TdThreadCond *cond, TdThreadMutex *mutex) {
int32_t taosThreadCondTimedWait(TdThreadCond *cond, TdThreadMutex *mutex, const struct timespec *abstime) {
#ifdef __USE_WIN_THREAD
if (!abstime) return EINVAL;
if (!abstime) return 0;
if (SleepConditionVariableCS(cond, mutex, (DWORD)(abstime->tv_sec * 1e3 + abstime->tv_nsec / 1e6))) return 0;
if (GetLastError() == ERROR_TIMEOUT) {
return ETIMEDOUT;
DWORD error = GetLastError();
if (error == ERROR_TIMEOUT) {
return TSDB_CODE_TIMEOUT_ERROR;
}
return EINVAL;
return TAOS_SYSTEM_WINAPI_ERROR(error);
#else
int32_t code = pthread_cond_timedwait(cond, mutex, abstime);
if (code && code != ETIMEDOUT) {
terrno = TAOS_SYSTEM_ERROR(code);
return terrno;
if(code == ETIMEDOUT) {
return TSDB_CODE_TIMEOUT_ERROR;
} else if (code) {
return TAOS_SYSTEM_ERROR(code);
} else {
return 0;
}
return code;
#endif
}

21
source/os/src/osTime.c
View File

@ -474,22 +474,15 @@ time_t taosMktime(struct tm *timep) {
#endif
}
struct tm *taosLocalTime(const time_t *timep, struct tm *result, char *buf) {
struct tm *taosLocalTime(const time_t *timep, struct tm *result, char *buf, int32_t bufSize) {
struct tm *res = NULL;
if (timep == NULL) {
if (timep == NULL || result == NULL) {
return NULL;
}
if (result == NULL) {
res = localtime(timep);
if (res == NULL && buf != NULL) {
(void)sprintf(buf, "NaN");
}
return res;
}
#ifdef WINDOWS
if (*timep < -2208988800LL) {
if (buf != NULL) {
sprintf(buf, "NaN");
snprintf(buf, bufSize, "NaN");
}
return NULL;
} else if (*timep < 0) {
@ -501,7 +494,7 @@ struct tm *taosLocalTime(const time_t *timep, struct tm *result, char *buf) {
time_t tt = 0;
if (localtime_s(&tm1, &tt) != 0) {
if (buf != NULL) {
sprintf(buf, "NaN");
snprintf(buf, bufSize, "NaN");
}
return NULL;
}
@ -532,7 +525,7 @@ struct tm *taosLocalTime(const time_t *timep, struct tm *result, char *buf) {
} else {
if (localtime_s(result, timep) != 0) {
if (buf != NULL) {
sprintf(buf, "NaN");
snprintf(buf, bufSize, "NaN");
}
return NULL;
}
@ -540,7 +533,7 @@ struct tm *taosLocalTime(const time_t *timep, struct tm *result, char *buf) {
#else
res = localtime_r(timep, result);
if (res == NULL && buf != NULL) {
(void)sprintf(buf, "NaN");
(void)snprintf(buf, bufSize, "NaN");
}
#endif
return result;
@ -559,7 +552,7 @@ static int isLeapYear(time_t year) {
struct tm *taosLocalTimeNolock(struct tm *result, const time_t *timep, int dst) {
if (result == NULL) {
return localtime(timep);
return NULL;
}
#ifdef WINDOWS
if (*timep < 0) {

30
source/os/src/osTimezone.c
View File

@ -783,15 +783,15 @@ int32_t taosSetSystemTimezone(const char *inTimezoneStr, char *outTimezoneStr, i
memset(winStr, 0, sizeof(winStr));
for (size_t i = 0; i < 554; i++) {
if (strcmp(tz_win[i][0], buf) == 0) {
char keyPath[100];
char keyPath[256];
char keyValue[100];
DWORD keyValueSize = sizeof(keyValue);
sprintf(keyPath, "SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Time Zones\\%s", tz_win[i][1]);
snprintf(keyPath, sizeof(keyPath), "SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Time Zones\\%s", tz_win[i][1]);
RegGetValue(HKEY_LOCAL_MACHINE, keyPath, "Display", RRF_RT_ANY, NULL, (PVOID)&keyValue, &keyValueSize);
if (keyValueSize > 0) {
keyValue[4] = (keyValue[4] == '+' ? '-' : '+');
keyValue[10] = 0;
sprintf(winStr, "TZ=%s:00", &(keyValue[1]));
snprintf(winStr, sizeof(winStr), "TZ=%s:00", &(keyValue[1]));
*tsTimezone = -taosStr2Int32(&keyValue[4], NULL, 10);
}
break;
@ -805,7 +805,7 @@ int32_t taosSetSystemTimezone(const char *inTimezoneStr, char *outTimezoneStr, i
char *ppp = strchr(inTimezoneStr, ',');
int indexStr;
if (pp == NULL || ppp == NULL) {
indexStr = sprintf(winStr, "TZ=UTC");
indexStr = snprintf(winStr, sizeof(winStr), "TZ=UTC");
} else {
memcpy(winStr, "TZ=", 3);
pp++;
@ -814,7 +814,7 @@ int32_t taosSetSystemTimezone(const char *inTimezoneStr, char *outTimezoneStr, i
}
char to[5];
parseTimeStr(p, to);
sprintf(&winStr[indexStr], "%c%c%c:%c%c:00", (to[0] == '+' ? '+' : '-'), to[1], to[2], to[3], to[4]);
snprintf(&winStr[indexStr], sizeof(winStr) - indexStr, "%c%c%c:%c%c:00", (to[0] == '+' ? '+' : '-'), to[1], to[2], to[3], to[4]);
*tsTimezone = -taosStr2Int32(p, NULL, 10);
} else {
*tsTimezone = 0;
@ -823,7 +823,7 @@ int32_t taosSetSystemTimezone(const char *inTimezoneStr, char *outTimezoneStr, i
_putenv(winStr);
_tzset();
if (outTimezoneStr != inTimezoneStr) {
strcpy(outTimezoneStr, inTimezoneStr);
tstrncpy(outTimezoneStr, inTimezoneStr, TD_TIMEZONE_LEN);
}
*outDaylight = 0;
@ -839,7 +839,7 @@ int32_t taosSetSystemTimezone(const char *inTimezoneStr, char *outTimezoneStr, i
*tsTimezone = tz;
tz += isdst_now;
sprintf(outTimezoneStr, "%s (%s, %s%02d00)", buf, tzname[isdst_now], tz >= 0 ? "+" : "-", abs(tz));
snprintf(outTimezoneStr, TD_TIMEZONE_LEN, "%s (%s, %s%02d00)", buf, tzname[isdst_now], tz >= 0 ? "+" : "-", abs(tz));
*outDaylight = isdst_now;
#else
@ -853,7 +853,7 @@ int32_t taosSetSystemTimezone(const char *inTimezoneStr, char *outTimezoneStr, i
int32_t tz = (int32_t)((-timezone * MILLISECOND_PER_SECOND) / MILLISECOND_PER_HOUR);
*tsTimezone = tz;
tz += isdst_now;
(void)sprintf(outTimezoneStr, "%s (%s, %s%02d00)", buf, tzname[isdst_now], tz >= 0 ? "+" : "-", abs(tz));
(void)snprintf(outTimezoneStr, TD_TIMEZONE_LEN, "%s (%s, %s%02d00)", buf, tzname[isdst_now], tz >= 0 ? "+" : "-", abs(tz));
*outDaylight = isdst_now;
#endif
@ -872,21 +872,21 @@ int32_t taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone)
if (result != ERROR_SUCCESS) {
return TAOS_SYSTEM_WINAPI_ERROR(result);
}
strcpy(outTimezoneStr, "not configured");
tstrncpy(outTimezoneStr, "not configured", TD_TIMEZONE_LEN);
*tsTimezone = 0;
if (bufferSize > 0) {
for (size_t i = 0; i < 139; i++) {
if (strcmp(win_tz[i][0], value) == 0) {
strcpy(outTimezoneStr, win_tz[i][1]);
tstrncpy(outTimezoneStr, win_tz[i][1], TD_TIMEZONE_LEN);
bufferSize = sizeof(value);
sprintf(keyPath, "SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Time Zones\\%s", value);
snprintf(keyPath, sizeof(keyPath), "SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Time Zones\\%s", value);
result = RegGetValue(HKEY_LOCAL_MACHINE, keyPath, "Display", RRF_RT_ANY, NULL, (PVOID)&value, &bufferSize);
if (result != ERROR_SUCCESS) {
return TAOS_SYSTEM_WINAPI_ERROR(result);
}
if (bufferSize > 0) {
// value[4] = (value[4] == '+' ? '-' : '+');
sprintf(outTimezoneStr, "%s (UTC, %c%c%c%c%c)", outTimezoneStr, value[4], value[5], value[6], value[8],
snprintf(outTimezoneStr, TD_TIMEZONE_LEN, "%s (UTC, %c%c%c%c%c)", outTimezoneStr, value[4], value[5], value[6], value[8],
value[9]);
*tsTimezone = taosStr2Int32(&value[4], NULL, 10);
}
@ -926,7 +926,7 @@ int32_t taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone)
*/
time_t tx1 = taosGetTimestampSec();
struct tm tm1;
if (taosLocalTime(&tx1, &tm1, NULL) == NULL) {
if (taosLocalTime(&tx1, &tm1, NULL, 0) == NULL) {
return TSDB_CODE_TIME_ERROR;
}
daylight = tm1.tm_isdst;
@ -956,7 +956,7 @@ int32_t taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone)
*/
time_t tx1 = taosGetTimestampSec();
struct tm tm1;
if(taosLocalTime(&tx1, &tm1, NULL) == NULL) {
if(taosLocalTime(&tx1, &tm1, NULL, 0) == NULL) {
return TSDB_CODE_TIME_ERROR;
}
/* load time zone string from /etc/timezone */
@ -1037,7 +1037,7 @@ int32_t taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone)
*/
time_t tx1 = taosGetTimestampSec();
struct tm tm1;
if(taosLocalTime(&tx1, &tm1, NULL) == NULL) {
if(taosLocalTime(&tx1, &tm1, NULL, 0) == NULL) {
return TSDB_CODE_TIME_ERROR;
}
isdst_now = tm1.tm_isdst;

2
source/os/test/osTests.cpp
View File

@ -37,7 +37,7 @@
#include <arpa/inet.h>
TEST(osTest, osFQDNSuccess) {
char fqdn[1024];
char fqdn[TD_FQDN_LEN];
char ipString[INET_ADDRSTRLEN];
int code = taosGetFqdn(fqdn);
uint32_t ipv4 = 0;

45
source/os/test/osTimeTests.cpp
View File

@ -31,20 +31,20 @@
TEST(osTimeTests, taosLocalTimeNolock) {
time_t currentTime;
// Test when result is NULL
struct tm* result = taosLocalTimeNolock(NULL, &currentTime, 0);
// Test when result is not NULL
struct tm expectedTime;
result = taosLocalTimeNolock(&expectedTime, &currentTime, 1);
EXPECT_EQ(expectedTime.tm_year, result->tm_year);
EXPECT_EQ(expectedTime.tm_mon, result->tm_mon);
EXPECT_EQ(expectedTime.tm_mday, result->tm_mday);
EXPECT_EQ(expectedTime.tm_hour, result->tm_hour);
EXPECT_EQ(expectedTime.tm_min, result->tm_min);
EXPECT_EQ(expectedTime.tm_sec, result->tm_sec);
EXPECT_EQ(expectedTime.tm_wday, result->tm_wday);
EXPECT_EQ(expectedTime.tm_yday, result->tm_yday);
EXPECT_EQ(expectedTime.tm_isdst, result->tm_isdst);
struct tm* result = taosLocalTimeNolock(&expectedTime, &currentTime, 1);
if (result) {
EXPECT_EQ(expectedTime.tm_year, result->tm_year);
EXPECT_EQ(expectedTime.tm_mon, result->tm_mon);
EXPECT_EQ(expectedTime.tm_mday, result->tm_mday);
EXPECT_EQ(expectedTime.tm_hour, result->tm_hour);
EXPECT_EQ(expectedTime.tm_min, result->tm_min);
EXPECT_EQ(expectedTime.tm_sec, result->tm_sec);
EXPECT_EQ(expectedTime.tm_wday, result->tm_wday);
EXPECT_EQ(expectedTime.tm_yday, result->tm_yday);
EXPECT_EQ(expectedTime.tm_isdst, result->tm_isdst);
}
}
@ -52,7 +52,7 @@ TEST(osTimeTests, taosLocalTime) {
// Test 1: Test when both timep and result are not NULL
time_t timep = 1617531000; // 2021-04-04 18:10:00
struct tm result;
struct tm* local_time = taosLocalTime(&timep, &result, NULL);
struct tm* local_time = taosLocalTime(&timep, &result, NULL, 0);
ASSERT_NE(local_time, nullptr);
ASSERT_EQ(local_time->tm_year, 121);
ASSERT_EQ(local_time->tm_mon, 3);
@ -62,20 +62,13 @@ TEST(osTimeTests, taosLocalTime) {
ASSERT_EQ(local_time->tm_sec, 00);
// Test 2: Test when timep is NULL
local_time = taosLocalTime(NULL, &result, NULL);
local_time = taosLocalTime(NULL, &result, NULL, 0);
ASSERT_EQ(local_time, nullptr);
// Test 3: Test when result is NULL
local_time = taosLocalTime(&timep, NULL, NULL);
ASSERT_NE(local_time, nullptr);
ASSERT_EQ(local_time->tm_year, 121);
ASSERT_EQ(local_time->tm_mon, 3);
ASSERT_EQ(local_time->tm_mday, 4);
// Test 4: Test when timep is negative on Windows
#ifdef WINDOWS
time_t pos_timep = 1609459200; // 2021-01-01 08:00:00
local_time = taosLocalTime(&pos_timep, &result, NULL);
local_time = taosLocalTime(&pos_timep, &result, NULL, 0);
ASSERT_NE(local_time, nullptr);
ASSERT_EQ(local_time->tm_year, 121);
ASSERT_EQ(local_time->tm_mon, 0);
@ -85,7 +78,7 @@ TEST(osTimeTests, taosLocalTime) {
ASSERT_EQ(local_time->tm_sec, 0);
time_t neg_timep = -1617531000; // 1918-09-29 21:50:00
local_time = taosLocalTime(&neg_timep, &result, NULL);
local_time = taosLocalTime(&neg_timep, &result, NULL, 0);
ASSERT_NE(local_time, nullptr);
ASSERT_EQ(local_time->tm_year, 18);
ASSERT_EQ(local_time->tm_mon, 8);
@ -95,7 +88,7 @@ TEST(osTimeTests, taosLocalTime) {
ASSERT_EQ(local_time->tm_sec, 0);
time_t neg_timep2 = -315619200; // 1960-01-01 08:00:00
local_time = taosLocalTime(&neg_timep2, &result, NULL);
local_time = taosLocalTime(&neg_timep2, &result, NULL, 0);
ASSERT_NE(local_time, nullptr);
ASSERT_EQ(local_time->tm_year, 60);
ASSERT_EQ(local_time->tm_mon, 0);
@ -105,7 +98,7 @@ TEST(osTimeTests, taosLocalTime) {
ASSERT_EQ(local_time->tm_sec, 0);
time_t zero_timep = 0; // 1970-01-01 08:00:00
local_time = taosLocalTime(&zero_timep, &result, NULL);
local_time = taosLocalTime(&zero_timep, &result, NULL, 0);
ASSERT_NE(local_time, nullptr);
ASSERT_EQ(local_time->tm_year, 70);
ASSERT_EQ(local_time->tm_mon, 0);
@ -115,7 +108,7 @@ TEST(osTimeTests, taosLocalTime) {
ASSERT_EQ(local_time->tm_sec, 0);
time_t neg_timep3 = -78115158887;
local_time = taosLocalTime(&neg_timep3, &result, NULL);
local_time = taosLocalTime(&neg_timep3, &result, NULL, 0);
ASSERT_EQ(local_time, nullptr);
#endif
}

Some files were not shown because too many files have changed in this diff Show More