xxq250
/
homework-jianmu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Merge branch '3.0' into fix/TD-31320-3.0a

This commit is contained in:
kailixu 2024-08-14 22:06:43 +08:00
parent 92704f8b7a ed7081eaad
commit 462748e715
84 changed files with 10372 additions and 8289 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/en/14-reference/03-taos-sql/03-table.md
View File

@ -25,7 +25,7 @@ create_definition:
col_name column_definition col_name column_definition
column_definition: column_definition:
type_name [comment 'string_value'] [PRIMARY KEY] [ENCODE 'encode_type'] [COMPRESS 'compress_type'] [LEVEL 'level_type'] type_name [PRIMARY KEY] [ENCODE 'encode_type'] [COMPRESS 'compress_type'] [LEVEL 'level_type']
table_options: table_options:
table_option ... table_option ...

2
docs/en/14-reference/03-taos-sql/04-stable.md
View File

@ -13,7 +13,7 @@ create_definition:
col_name column_definition col_name column_definition
column_definition: column_definition:
type_name [comment 'string_value'] [PRIMARY KEY] [ENCODE 'encode_type'] [COMPRESS 'compress_type'] [LEVEL 'level_type'] type_name [PRIMARY KEY] [ENCODE 'encode_type'] [COMPRESS 'compress_type'] [LEVEL 'level_type']
table_options: table_options:
table_option ... table_option ...

149
docs/en/14-reference/03-taos-sql/32-view.md Normal file
View File

@ -0,0 +1,149 @@
---
toc_max_heading_level: 4
title: "View"
sidebar_label: "View"
---
## Introduction
Starting from TDengine 3.2.1.0, TDengine Enterprise Edition provides view function, which is convenient for users to simplify operation and improve sharing ability among users.
A view is essentially a query statement stored in a database. The view (non-materialized view) itself does not contain data, and only dynamically executes the query statement specified by the view when reading data from the view. We specify a name when creating a view, and then we can query it like using a regular table. The use of views should follow the following rules:
- Views can be defined and used nested, and are bound to the specified or current database when created.
- Within the same database, duplicate view names are not allowed, and it is recommended not to have duplicate view names and table names (not mandatory). When the view and table names have the same name, operations such as writing, querying, authorizing, and revoking permissions will prioritize using the same-named table.
## Grammar
### Create (update) view
```sql
CREATE [ OR REPLACE ] VIEW [db_name.]view_name AS query
```
Description:
- When creating a view, you can specify the database name ( db_name ) to which the view is bound. If not explicitly specified, it defaults to the database bound to the current connection.
- It is recommended to specify the database name in the query statement, support cross-database views, and default to the database bound to the view when not specified (it may not be the database specified by the current connection);
### View View
1. View all views under a database
```sql
SHOW [db_name.]VIEWS;
```
2. View the creation statement of the view
```sql
SHOW CREATE VIEW [db_name.]view_name;
```
3. View view column information
```sql
DESCRIBE [db_name.]view_name;
```
4. View all view information
```sql
SELECT ... FROM information_schema.ins_views;
```
### Delete a view
```sql
DROP VIEW [IF EXISTS] [db_name.]view_name;
```
## Permissions
### Description
View permissions are divided into three types: READ, WRITE, and ALTER. Query operations require READ permissions, write operations require WRITE permissions, and delete and modify operations on the view itself require ALTER permissions.
### Rules
- The creator of the view and the root user have all permissions by default.
- Authorization and revocation of permissions for other users can be performed through the GRANT and REVOKE statements, which can only be performed by the root user.
- View permissions need to be authorized and revoked separately. Authorization and revocation through db. * do not include view permissions.
- Views can be defined and used nested, and the verification of view permissions is also performed by recursion.
- In order to facilitate the sharing and use of views, the concept of view effective user (i.e. the user who creates the view) is introduced. Authorized users can use the read and write permissions of the view effective user's library, table, and nested view. Note: After the view is REPLACE, the effective user will also be updated.
The detailed rules for controlling relevant permissions are summarized as follows:
| Serial number | Operation | Permission requirements |
| --- | --- | --- |
| 1 | CREATE OR REPLACE VIEW (Create a new view) | The user has WRITE permission on the database to which the view belongs And Users have query permissions for the target library, table, and view of the view. If the object in the query is a view, it must meet rule 8 in the current table. |
| 2 | CREATE OR REPLACE VIEW (Overwrite old view) | The user has WRITE permission on the database to which the view belongs, and ALTER permission on the old view And Users have query permissions for the target library, table, and view of the view. If the object in the query is a view, it must meet rule 8 in the current table. |
| 3 | DROP VIEW | The user has ALTER permission on the view |
| 4 | SHOW VIEWS | No |
| 5 | SHOW CREATE VIEW | No |
| 6 | DESCRIBE VIEW | No |
| 7 | System table query | No |
| 8 | SELECT FROM VIEW | The operating user has READ permissions for the view And Operating users or view effective users have READ permissions on the target library, table, and view of the view |
| 9 | INSERT INTO VIEW | The operation user has WRITE permission on the view And Operating users or view effective users have WRITE permissions on the target library, table, and view of the view |
| 10 | GRANT/REVOKE | Only the root user has permission |
### Grammar
#### Authorization
```sql
GRANT privileges ON [db_name.]view_name TO user_name
privileges: {
ALL,
| priv_type [, priv_type] ...
}
priv_type: {
READ
| WRITE
| ALTER
}
```
#### Recover permissions
```sql
REVOKE privileges ON [db_name.]view_name FROM user_name
privileges: {
ALL,
| priv_type [, priv_type] ...
}
priv_type: {
READ
| WRITE
| ALTER
}
```
## Usage scenarios
| SQL query | SQL write | STMT query | STMT write | Subscribe | Stream |
| --- | --- | --- | --- | --- | --- |
| Support | Not supported yet | Not supported yet | Not supported yet | Support | Not supported yet |
## Example
- Create a view
```sql
CREATE VIEW view1 AS SELECT _wstart, count(*) FROM table1 INTERVAL(1d);
CREATE VIEW view2 AS SELECT ts, col2 FROM table1;
CREATE VIEW view3 AS SELECT * from view1;
```
- Query data
```sql
SELECT * from view1;
```
- Delete data
```sql
DROP VIEW view1;
```

2
docs/zh/02-concept.md
View File

@ -91,7 +91,7 @@ toc_max_heading_level: 4
5. 对于小数据量场景，私有化部署太重：在物联网、车联网场景中，因为涉及到生产经营数据的安全，很多还是采取私有化部署。而每个私有化部署，处理的数据量有很大的区别，从几百台联网设备到数千万台设备不等。对于数据量小的场景，通用的大数据解决方案就显得过于臃肿，投入产出不成正比。因此有的平台提供商往往有两套方案，一套针对大数据场景，使用通用的大数据平台，一套针对小数据规模场景，就使用 MySQL 或其他数据库来搞定一切，但是随着历史数据的累积，或接入设备量的增长，关系型数据库性能不足、运维复杂、扩展性差等缺点都会逐渐暴露出来，终究不是长久之计。 5. 对于小数据量场景，私有化部署太重：在物联网、车联网场景中，因为涉及到生产经营数据的安全，很多还是采取私有化部署。而每个私有化部署，处理的数据量有很大的区别，从几百台联网设备到数千万台设备不等。对于数据量小的场景，通用的大数据解决方案就显得过于臃肿，投入产出不成正比。因此有的平台提供商往往有两套方案，一套针对大数据场景，使用通用的大数据平台，一套针对小数据规模场景，就使用 MySQL 或其他数据库来搞定一切，但是随着历史数据的累积，或接入设备量的增长，关系型数据库性能不足、运维复杂、扩展性差等缺点都会逐渐暴露出来，终究不是长久之计。
由于存在这些根本性的缺陷，导致高速增长的时序大数据市场一直没有一个简单好用而又高效的工具。于是，近些年一批专注时序数据处理的企业杀入了这个赛道，比如美国的 InfluxData，其产品 InfluxDB 在 IT 运维监测方面有相当的市场占有率。开源社区也十分活跃，比如基于 HBase 开发的 OpenTSDB，中国国内，阿里、百度、华为都有基于 OpenTSDB 的产品，涛思数据不依赖任何第三方，推出了自主研发而且开源的的 TDengine。 由于存在这些根本性的缺陷，导致高速增长的时序大数据市场一直没有一个简单好用而又高效的工具。于是，近些年一批专注时序数据处理的企业杀入了这个赛道，比如美国的 InfluxData，其产品 InfluxDB 在 IT 运维监测方面有相当的市场占有率。开源社区也十分活跃，比如基于 HBase 开发的 OpenTSDB，中国国内，阿里、百度、华为都有基于 OpenTSDB 的产品，涛思数据不依赖任何第三方，推出了自主研发而且开源的 TDengine。
由于数据量巨大且应用方式特殊，对时序数据的处理具有相当大的技术挑战，因此要使用专业的大数据平台。对实时时序数据的科学合理地高效处理能够帮助企业实时监控生产与经营过程，对历史时序数据的分析有助于对资源的使用和生产配置做出科学的决策。 由于数据量巨大且应用方式特殊，对时序数据的处理具有相当大的技术挑战，因此要使用专业的大数据平台。对实时时序数据的科学合理地高效处理能够帮助企业实时监控生产与经营过程，对历史时序数据的分析有助于对资源的使用和生产配置做出科学的决策。

2
docs/zh/08-develop/07-tmq.md
View File

@ -492,7 +492,7 @@ Rust 连接器创建消费者的参数为 DSN， 可以设置的参数列表请
3. 对于每个分配，使用 `tmq_offset_seek` 函数将消费者的偏移量设置到最早的偏移量。 3. 对于每个分配，使用 `tmq_offset_seek` 函数将消费者的偏移量设置到最早的偏移量。
4. 如果设置偏移量失败，则打印错误信息。 4. 如果设置偏移量失败，则打印错误信息。
5. 释放分配信息数组以释放资源。 5. 释放分配信息数组以释放资源。
6. 调用 `basic_consume_loop` 函数开始新的的消费循环，处理消息。 6. 调用 `basic_consume_loop` 函数开始新的消费循环，处理消息。
</TabItem> </TabItem>
<TabItem label="REST API" value="rest"> <TabItem label="REST API" value="rest">

2
docs/zh/14-reference/03-taos-sql/03-table.md
View File

@ -26,7 +26,7 @@ create_definition:
col_name column_definition col_name column_definition
column_definition: column_definition:
type_name [comment 'string_value'] [PRIMARY KEY] [ENCODE 'encode_type'] [COMPRESS 'compress_type'] [LEVEL 'level_type'] type_name [PRIMARY KEY] [ENCODE 'encode_type'] [COMPRESS 'compress_type'] [LEVEL 'level_type']
table_options: table_options:
table_option ... table_option ...

2
docs/zh/14-reference/03-taos-sql/04-stable.md
View File

@ -13,7 +13,7 @@ create_definition:
col_name column_definition col_name column_definition
column_definition: column_definition:
type_name [comment 'string_value'] [PRIMARY KEY] [ENCODE 'encode_type'] [COMPRESS 'compress_type'] [LEVEL 'level_type'] type_name [PRIMARY KEY] [ENCODE 'encode_type'] [COMPRESS 'compress_type'] [LEVEL 'level_type']
table_options: table_options:
table_option ... table_option ...

139
docs/zh/14-reference/03-taos-sql/32-view.md Normal file
View File

@ -0,0 +1,139 @@
---
toc_max_heading_level: 4
title: "视图"
sidebar_label: "视图"
---
## 简介
从 TDengine 3.2.1.0 开始，TDengine 企业版提供视图功能，便于用户简化操作，提升用户间的分享能力。
视图（View）本质上是一个存储在数据库中的查询语句。视图（非物化视图）本身不包含数据，只有在从视图读取数据时才动态执行视图所指定的查询语句。我们在创建视图时指定一个名称，然后可以像使用普通表一样对其进行查询等操作。视图的使用需遵循以下规则：
- 视图可以嵌套定义和使用，视图与创建时指定的或当前数据库绑定使用。
- 在同一个数据库内，视图名称不允许重名，视图名跟表名也推荐不重名（不强制）。当出现视图与表名重名时，写入、查询、授权、回收权限等操作优先使用同名表。
## 语法
### 创建（更新）视图
```sql
CREATE [ OR REPLACE ] VIEW [db_name.]view_name AS query
```
说明：
- 创建视图时可以指定视图绑定的数据库名（db_name），未明确指定时默认为当前连接绑定的数据库；
- 查询语句（query）中推荐指定数据库名，支持跨库视图，未指定时默认为与视图绑定的数据库(有可能非当前连接指定的数据库)；
### 查看视图
1. 查看某个数据库下的所有视图
```sql
SHOW [db_name.]VIEWS;
```
2. 查看视图的创建语句
```sql
SHOW CREATE VIEW [db_name.]view_name;
```
3. 查看视图列信息
```sql
DESCRIBE [db_name.]view_name;
```
4. 查看所有视图信息
```sql
SELECT ... FROM information_schema.ins_views;
```
### 删除视图
```sql
DROP VIEW [IF EXISTS] [db_name.]view_name;
```
## 权限
### 说明
视图的权限分为 READ、WRITE、ALTER 三种，查询操作需要具备 READ 权限，写入操作需要具备 WRITE 权限，对视图本身的删改操作需要具备 ALTER 权限。
### 规则
- 视图的创建者和 root 用户默认具备所有权限。
- 对其他用户进行授权与回收权限可以通过 GRANT 和 REVOKE 语句进行，该操作只能由 root 用户进行。
- 视图权限需单独授权与回收，通过db.*进行的授权与回收不含视图权限。
- 视图可以嵌套定义与使用，同理对视图权限的校验也是递归进行的。
- 为了方便视图的分享与使用，引入视图有效用户（即视图的创建用户）的概念，被授权用户可以使用视图有效用户的库、表及嵌套视图的读写权限。注：视图被 REPLACE 后有效用户也会被更新。
具体相关权限控制细则如下表所示：
| 序号 | 操作 | 权限要求 |
| ---- | --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 1 | CREATE VIEW <br/>(创建新用户) | 用户对视图所属数据库有 WRITE 权限<br/>且<br/> 用户对视图的目标库、表、视图有查询权限，若查询中的对象是视图需满足当前表中第8条规则 |
| 2 | CREATE OR REPLACE VIEW <br/>(覆盖旧视图) | 用户对视图所属数据库有 WRITE 权限 且 对旧有视图有 ALTER 权限 <br/>且<br/> 用户对视图的目标库、表、视图有查询权限，若查询中的对象是视图需满足当前表中第8条规则 |
| 3 | DROP VIEW | 用户对视图有 ALTER 权限 |
| 4 | SHOW VIEWS | 无 |
| 5 | SHOW CREATE VIEW | 无 |
| 6 | DESCRIBE VIEW | 无 |
| 7 | 系统表查询 | 无 |
| 8 | SELECT FROM VIEW | 操作用户对视图有 READ 权限 且 操作用户或视图有效用户对视图的目标库、表、视图有 READ 权限 |
| 9 | INSERT INTO VIEW | 操作用户对视图有 WRITE 权限 且 操作用户或视图有效用户对视图的目标库、表、视图有 WRITE 权限 |
| 10 | GRANT/REVOKE | 只有 root 用户有权限 |
### 语法
#### 授权
```sql
GRANT privileges ON [db_name.]view_name TO user_name
privileges: {
ALL,
| priv_type [, priv_type] ...
}
priv_type: {
READ
| WRITE
| ALTER
}
```
#### 回收权限
```sql
REVOKE privileges ON [db_name.]view_name FROM user_name
privileges: {
ALL,
| priv_type [, priv_type] ...
}
priv_type: {
READ
| WRITE
| ALTER
}
```
## 使用场景
| SQL 查询 | SQL 写入 | STMT 查询 | STMT 写入 | 订阅 | 流计算 |
| -------- | -------- | --------- | --------- | ---- | -------- |
| 支持 | 暂不支持 | 暂不支持 | 暂不支持 | 支持 | 暂不支持 |
## 举例
- 创建视图
```sql
CREATE VIEW view1 AS SELECT _wstart, count(*) FROM table1 INTERVAL(1d);
CREATE VIEW view2 AS SELECT ts, col2 FROM table1;
CREATE VIEW view3 AS SELECT * from view1;
```
- 查询数据
```sql
SELECT * from view1;
```
- 删除视图
```sql
DROP VIEW view1;
```

2
docs/zh/14-reference/05-connector/10-cpp.mdx
View File

@ -783,7 +783,7 @@ TDengine 的异步 API 均采用非阻塞调用模式。应用程序可以用多
- **返回值**：非 `NULL`：成功，返回一个 TAOS * 类型的指针，指向与 TDengine 数据库的连接句柄。`NULL`：失败，非法的输入参数。 - **返回值**：非 `NULL`：成功，返回一个 TAOS * 类型的指针，指向与 TDengine 数据库的连接句柄。`NULL`：失败，非法的输入参数。
- `const char *tmq_get_table_name(TAOS_RES *res)` - `const char *tmq_get_table_name(TAOS_RES *res)`
- **接口说明**：从 TMQ 消费者获取的消息结果中获取所属的的表名。 - **接口说明**：从 TMQ 消费者获取的消息结果中获取所属的表名。
- res：[入参] 指向一个有效的 TAOS_RES 结构体指针，该结构体包含了从 TMQ 消费者轮询得到的消息。 - res：[入参] 指向一个有效的 TAOS_RES 结构体指针，该结构体包含了从 TMQ 消费者轮询得到的消息。
- **返回值**：非 `NULL`：成功，返回一个 const char * 类型的指针，指向表名字符串。`NULL`：失败，非法的输入参数。 - **返回值**：非 `NULL`：成功，返回一个 const char * 类型的指针，指向表名字符串。`NULL`：失败，非法的输入参数。

8
docs/zh/14-reference/05-connector/30-python.mdx
View File

@ -307,7 +307,7 @@ TaosResult 对象可以通过循环遍历获取查询到的数据。
- **参数说明**： - **参数说明**：
- `topic`: 订阅的主题。 - `topic`: 订阅的主题。
- `vg_id`: vgroupid。 - `vg_id`: vgroupid。
- `offset`:需要设置的的偏移量。 - `offset`:需要设置的偏移量。
- **异常**：操作失败抛出 ConsumerException 异常。 - **异常**：操作失败抛出 ConsumerException 异常。
- `fn committed(&mut self, topic: &str, vg_id: i32) -> PyResult<i64>` - `fn committed(&mut self, topic: &str, vg_id: i32) -> PyResult<i64>`
- **接口说明**：获取订阅主题的vgroupid分区最后提交的偏移量。 - **接口说明**：获取订阅主题的vgroupid分区最后提交的偏移量。
@ -489,7 +489,7 @@ TaosResult 对象可以通过循环遍历获取查询到的数据。
- `def seek(self, partition)` - `def seek(self, partition)`
- **接口说明**：将给定分区的偏移量设置到指定的位置。 - **接口说明**：将给定分区的偏移量设置到指定的位置。
- **参数说明**： - **参数说明**：
- `partition`: 需要设置的的偏移量。 - `partition`: 需要设置的偏移量。
- `topic`: 订阅的主题 - `topic`: 订阅的主题
- `partition`: 分区 - `partition`: 分区
- `offset`: 偏移量 - `offset`: 偏移量
@ -497,7 +497,7 @@ TaosResult 对象可以通过循环遍历获取查询到的数据。
- `def committed(self, partitions)` - `def committed(self, partitions)`
- **接口说明**：获取订阅主题的分区最后提交的偏移量。 - **接口说明**：获取订阅主题的分区最后提交的偏移量。
- **参数说明**： - **参数说明**：
- `partition`: 需要设置的的偏移量。 - `partition`: 需要设置的偏移量。
- `topic`: 订阅的主题 - `topic`: 订阅的主题
- `partition`: 分区 - `partition`: 分区
- **返回值**：`partition`，分区最后提交的偏移量。 - **返回值**：`partition`，分区最后提交的偏移量。
@ -505,7 +505,7 @@ TaosResult 对象可以通过循环遍历获取查询到的数据。
- `def position(self, partitions)` - `def position(self, partitions)`
- **接口说明**：获取给定分区当前的偏移量。 - **接口说明**：获取给定分区当前的偏移量。
- **参数说明**： - **参数说明**：
- `partition`: 需要设置的的偏移量。 - `partition`: 需要设置的偏移量。
- `topic`: 订阅的主题 - `topic`: 订阅的主题
- `partition`: 分区 - `partition`: 分区
- **返回值**：`partition`，分区最后提交的偏移量。 - **返回值**：`partition`，分区最后提交的偏移量。

4
docs/zh/14-reference/05-connector/35-node.mdx
View File

@ -8,7 +8,7 @@ import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem"; import TabItem from "@theme/TabItem";
import RequestId from "./_request_id.mdx"; import RequestId from "./_request_id.mdx";
`@tdengine/websocket` 是 TDengine 的官方 Node.js 语言连接器。Node.js 开发人员可以通过它开发存取 TDengine 数据库的的应用软件。 `@tdengine/websocket` 是 TDengine 的官方 Node.js 语言连接器。Node.js 开发人员可以通过它开发存取 TDengine 数据库的应用软件。
Node.js 连接器源码托管在 [GitHub](https://github.com/taosdata/taos-connector-node/tree/main)。 Node.js 连接器源码托管在 [GitHub](https://github.com/taosdata/taos-connector-node/tree/main)。
@ -194,7 +194,7 @@ WSConfig 中的配置如下：
### 数据集 ### 数据集
- `getMeta():Array<TDengineMeta> | null` - `getMeta():Array<TDengineMeta> | null`
- **接口说明**：获取查询结果的的列的数量、类型和长度。 - **接口说明**：获取查询结果的列的数量、类型和长度。
- **返回值**：TDengineMeta 数据对象数组。 - **返回值**：TDengineMeta 数据对象数组。
```js ```js
export interface TDengineMeta { export interface TDengineMeta {

2
docs/zh/14-reference/07-supported.md
View File

@ -14,7 +14,7 @@ description: "TDengine 服务端、客户端和连接器支持的平台列表"
| M1 | | | | | | | | ● | | M1 | | | | | | | | ● |
注：1) ● 表示经过官方测试验证， ○ 表示非官方测试验证，E 表示仅企业版支持。 注：1) ● 表示经过官方测试验证， ○ 表示非官方测试验证，E 表示仅企业版支持。
2) 社区版仅支持主流操作系统的较新版本，包括 Ubuntu 18+/CentOS 7+/RetHat/Debian/CoreOS/FreeBSD/OpenSUSE/SUSE Linux/Fedora/macOS 等。如果有其他操作系统及版本的需求，请联系企业版支持。 2) 社区版仅支持主流操作系统的较新版本，包括 Ubuntu 18+/CentOS 7+/RedHat/Debian/CoreOS/FreeBSD/OpenSUSE/SUSE Linux/Fedora/macOS 等。如果有其他操作系统及版本的需求，请联系企业版支持。
## TDengine 客户端和连接器支持的平台列表 ## TDengine 客户端和连接器支持的平台列表

2
include/common/tglobal.h
View File

@ -86,7 +86,7 @@ extern int32_t tsNumOfQnodeQueryThreads;
extern int32_t tsNumOfQnodeFetchThreads; extern int32_t tsNumOfQnodeFetchThreads;
extern int32_t tsNumOfSnodeStreamThreads; extern int32_t tsNumOfSnodeStreamThreads;
extern int32_t tsNumOfSnodeWriteThreads; extern int32_t tsNumOfSnodeWriteThreads;
extern int64_t tsRpcQueueMemoryAllowed; extern int64_t tsQueueMemoryAllowed;
extern int32_t tsRetentionSpeedLimitMB; extern int32_t tsRetentionSpeedLimitMB;
// sync raft // sync raft

231
include/common/ttokendef.h
View File

@ -281,118 +281,125 @@
#define TK_IROWTS 263 #define TK_IROWTS 263
#define TK_ISFILLED 264 #define TK_ISFILLED 264
#define TK_CAST 265 #define TK_CAST 265
#define TK_NOW 266 #define TK_POSITION 266
#define TK_TODAY 267 #define TK_IN 267
#define TK_TIMEZONE 268 #define TK_FOR 268
#define TK_CLIENT_VERSION 269 #define TK_NOW 269
#define TK_SERVER_VERSION 270 #define TK_TODAY 270
#define TK_SERVER_STATUS 271 #define TK_SUBSTR 271
#define TK_CURRENT_USER 272 #define TK_SUBSTRING 272
#define TK_CASE 273 #define TK_BOTH 273
#define TK_WHEN 274 #define TK_TRAILING 274
#define TK_THEN 275 #define TK_LEADING 275
#define TK_ELSE 276 #define TK_TIMEZONE 276
#define TK_BETWEEN 277 #define TK_CLIENT_VERSION 277
#define TK_IS 278 #define TK_SERVER_VERSION 278
#define TK_NK_LT 279 #define TK_SERVER_STATUS 279
#define TK_NK_GT 280 #define TK_CURRENT_USER 280
#define TK_NK_LE 281 #define TK_PI 281
#define TK_NK_GE 282 #define TK_CASE 282
#define TK_NK_NE 283 #define TK_WHEN 283
#define TK_MATCH 284 #define TK_THEN 284
#define TK_NMATCH 285 #define TK_ELSE 285
#define TK_CONTAINS 286 #define TK_BETWEEN 286
#define TK_IN 287 #define TK_IS 287
#define TK_JOIN 288 #define TK_NK_LT 288
#define TK_INNER 289 #define TK_NK_GT 289
#define TK_LEFT 290 #define TK_NK_LE 290
#define TK_RIGHT 291 #define TK_NK_GE 291
#define TK_OUTER 292 #define TK_NK_NE 292
#define TK_SEMI 293 #define TK_MATCH 293
#define TK_ANTI 294 #define TK_NMATCH 294
#define TK_ASOF 295 #define TK_CONTAINS 295
#define TK_WINDOW 296 #define TK_JOIN 296
#define TK_WINDOW_OFFSET 297 #define TK_INNER 297
#define TK_JLIMIT 298 #define TK_LEFT 298
#define TK_SELECT 299 #define TK_RIGHT 299
#define TK_NK_HINT 300 #define TK_OUTER 300
#define TK_DISTINCT 301 #define TK_SEMI 301
#define TK_WHERE 302 #define TK_ANTI 302
#define TK_PARTITION 303 #define TK_ASOF 303
#define TK_BY 304 #define TK_WINDOW 304
#define TK_SESSION 305 #define TK_WINDOW_OFFSET 305
#define TK_STATE_WINDOW 306 #define TK_JLIMIT 306
#define TK_EVENT_WINDOW 307 #define TK_SELECT 307
#define TK_COUNT_WINDOW 308 #define TK_NK_HINT 308
#define TK_SLIDING 309 #define TK_DISTINCT 309
#define TK_FILL 310 #define TK_WHERE 310
#define TK_VALUE 311 #define TK_PARTITION 311
#define TK_VALUE_F 312 #define TK_BY 312
#define TK_NONE 313 #define TK_SESSION 313
#define TK_PREV 314 #define TK_STATE_WINDOW 314
#define TK_NULL_F 315 #define TK_EVENT_WINDOW 315
#define TK_LINEAR 316 #define TK_COUNT_WINDOW 316
#define TK_NEXT 317 #define TK_SLIDING 317
#define TK_HAVING 318 #define TK_FILL 318
#define TK_RANGE 319 #define TK_VALUE 319
#define TK_EVERY 320 #define TK_VALUE_F 320
#define TK_ORDER 321 #define TK_NONE 321
#define TK_SLIMIT 322 #define TK_PREV 322
#define TK_SOFFSET 323 #define TK_NULL_F 323
#define TK_LIMIT 324 #define TK_LINEAR 324
#define TK_OFFSET 325 #define TK_NEXT 325
#define TK_ASC 326 #define TK_HAVING 326
#define TK_NULLS 327 #define TK_RANGE 327
#define TK_ABORT 328 #define TK_EVERY 328
#define TK_AFTER 329 #define TK_ORDER 329
#define TK_ATTACH 330 #define TK_SLIMIT 330
#define TK_BEFORE 331 #define TK_SOFFSET 331
#define TK_BEGIN 332 #define TK_LIMIT 332
#define TK_BITAND 333 #define TK_OFFSET 333
#define TK_BITNOT 334 #define TK_ASC 334
#define TK_BITOR 335 #define TK_NULLS 335
#define TK_BLOCKS 336 #define TK_ABORT 336
#define TK_CHANGE 337 #define TK_AFTER 337
#define TK_COMMA 338 #define TK_ATTACH 338
#define TK_CONCAT 339 #define TK_BEFORE 339
#define TK_CONFLICT 340 #define TK_BEGIN 340
#define TK_COPY 341 #define TK_BITAND 341
#define TK_DEFERRED 342 #define TK_BITNOT 342
#define TK_DELIMITERS 343 #define TK_BITOR 343
#define TK_DETACH 344 #define TK_BLOCKS 344
#define TK_DIVIDE 345 #define TK_CHANGE 345
#define TK_DOT 346 #define TK_COMMA 346
#define TK_EACH 347 #define TK_CONCAT 347
#define TK_FAIL 348 #define TK_CONFLICT 348
#define TK_FOR 349 #define TK_COPY 349
#define TK_GLOB 350 #define TK_DEFERRED 350
#define TK_ID 351 #define TK_DELIMITERS 351
#define TK_IMMEDIATE 352 #define TK_DETACH 352
#define TK_IMPORT 353 #define TK_DIVIDE 353
#define TK_INITIALLY 354 #define TK_DOT 354
#define TK_INSTEAD 355 #define TK_EACH 355
#define TK_ISNULL 356 #define TK_FAIL 356
#define TK_MODULES 357 #define TK_GLOB 357
#define TK_NK_BITNOT 358 #define TK_ID 358
#define TK_NK_SEMI 359 #define TK_IMMEDIATE 359
#define TK_NOTNULL 360 #define TK_IMPORT 360
#define TK_OF 361 #define TK_INITIALLY 361
#define TK_PLUS 362 #define TK_INSTEAD 362
#define TK_PRIVILEGE 363 #define TK_ISNULL 363
#define TK_RAISE 364 #define TK_MODULES 364
#define TK_RESTRICT 365 #define TK_NK_BITNOT 365
#define TK_ROW 366 #define TK_NK_SEMI 366
#define TK_STAR 367 #define TK_NOTNULL 367
#define TK_STATEMENT 368 #define TK_OF 368
#define TK_STRICT 369 #define TK_PLUS 369
#define TK_STRING 370 #define TK_PRIVILEGE 370
#define TK_TIMES 371 #define TK_RAISE 371
#define TK_VALUES 372 #define TK_RESTRICT 372
#define TK_VARIABLE 373 #define TK_ROW 373
#define TK_WAL 374 #define TK_STAR 374
#define TK_ENCODE 375 #define TK_STATEMENT 375
#define TK_COMPRESS 376 #define TK_STRICT 376
#define TK_LEVEL 377 #define TK_STRING 377
#define TK_TIMES 378
#define TK_VALUES 379
#define TK_VARIABLE 380
#define TK_WAL 381
#define TK_ENCODE 382
#define TK_COMPRESS 383
#define TK_LEVEL 384
#define TK_NK_SPACE 600 #define TK_NK_SPACE 600
#define TK_NK_COMMENT 601 #define TK_NK_COMMENT 601

28
include/libs/function/functionMgt.h
View File

@ -45,6 +45,7 @@ typedef enum EFunctionType {
FUNCTION_TYPE_TWA, FUNCTION_TYPE_TWA,
FUNCTION_TYPE_HISTOGRAM, FUNCTION_TYPE_HISTOGRAM,
FUNCTION_TYPE_HYPERLOGLOG, FUNCTION_TYPE_HYPERLOGLOG,
FUNCTION_TYPE_STDVAR,
// nonstandard SQL function // nonstandard SQL function
FUNCTION_TYPE_BOTTOM = 500, FUNCTION_TYPE_BOTTOM = 500,
@ -77,6 +78,15 @@ typedef enum EFunctionType {
FUNCTION_TYPE_ASIN, FUNCTION_TYPE_ASIN,
FUNCTION_TYPE_ACOS, FUNCTION_TYPE_ACOS,
FUNCTION_TYPE_ATAN, FUNCTION_TYPE_ATAN,
FUNCTION_TYPE_PI,
FUNCTION_TYPE_EXP,
FUNCTION_TYPE_LN,
FUNCTION_TYPE_MOD,
FUNCTION_TYPE_RAND,
FUNCTION_TYPE_SIGN,
FUNCTION_TYPE_DEGREES,
FUNCTION_TYPE_RADIANS,
FUNCTION_TYPE_TRUNCATE,
// string function // string function
FUNCTION_TYPE_LENGTH = 1500, FUNCTION_TYPE_LENGTH = 1500,
@ -89,6 +99,13 @@ typedef enum EFunctionType {
FUNCTION_TYPE_RTRIM, FUNCTION_TYPE_RTRIM,
FUNCTION_TYPE_SUBSTR, FUNCTION_TYPE_SUBSTR,
FUNCTION_TYPE_MD5, FUNCTION_TYPE_MD5,
FUNCTION_TYPE_CHAR,
FUNCTION_TYPE_ASCII,
FUNCTION_TYPE_POSITION,
FUNCTION_TYPE_TRIM,
FUNCTION_TYPE_REPLACE,
FUNCTION_TYPE_REPEAT,
FUNCTION_TYPE_SUBSTR_IDX,
// conversion function // conversion function
FUNCTION_TYPE_CAST = 2000, FUNCTION_TYPE_CAST = 2000,
@ -104,6 +121,10 @@ typedef enum EFunctionType {
FUNCTION_TYPE_TIMETRUNCATE, FUNCTION_TYPE_TIMETRUNCATE,
FUNCTION_TYPE_TIMEZONE, FUNCTION_TYPE_TIMEZONE,
FUNCTION_TYPE_TODAY, FUNCTION_TYPE_TODAY,
FUNCTION_TYPE_WEEK,
FUNCTION_TYPE_WEEKDAY,
FUNCTION_TYPE_WEEKOFYEAR,
FUNCTION_TYPE_DAYOFWEEK,
// system function // system function
FUNCTION_TYPE_DATABASE = 3000, FUNCTION_TYPE_DATABASE = 3000,
@ -162,8 +183,9 @@ typedef enum EFunctionType {
FUNCTION_TYPE_LAST_MERGE, FUNCTION_TYPE_LAST_MERGE,
FUNCTION_TYPE_AVG_PARTIAL, FUNCTION_TYPE_AVG_PARTIAL,
FUNCTION_TYPE_AVG_MERGE, FUNCTION_TYPE_AVG_MERGE,
FUNCTION_TYPE_STDDEV_PARTIAL, FUNCTION_TYPE_STD_PARTIAL,
FUNCTION_TYPE_STDDEV_MERGE, FUNCTION_TYPE_STDDEV_MERGE,
FUNCTION_TYPE_STDVAR_MERGE,
FUNCTION_TYPE_IRATE_PARTIAL, FUNCTION_TYPE_IRATE_PARTIAL,
FUNCTION_TYPE_IRATE_MERGE, FUNCTION_TYPE_IRATE_MERGE,
FUNCTION_TYPE_AVG_STATE, FUNCTION_TYPE_AVG_STATE,
@ -174,8 +196,8 @@ typedef enum EFunctionType {
FUNCTION_TYPE_LAST_STATE_MERGE, FUNCTION_TYPE_LAST_STATE_MERGE,
FUNCTION_TYPE_SPREAD_STATE, FUNCTION_TYPE_SPREAD_STATE,
FUNCTION_TYPE_SPREAD_STATE_MERGE, FUNCTION_TYPE_SPREAD_STATE_MERGE,
FUNCTION_TYPE_STDDEV_STATE, FUNCTION_TYPE_STD_STATE,
FUNCTION_TYPE_STDDEV_STATE_MERGE, FUNCTION_TYPE_STD_STATE_MERGE,
FUNCTION_TYPE_HYPERLOGLOG_STATE, FUNCTION_TYPE_HYPERLOGLOG_STATE,
FUNCTION_TYPE_HYPERLOGLOG_STATE_MERGE, FUNCTION_TYPE_HYPERLOGLOG_STATE_MERGE,

7
include/libs/nodes/querynodes.h
View File

@ -170,6 +170,12 @@ typedef struct SNodeListNode {
SNodeList* pNodeList; SNodeList* pNodeList;
} SNodeListNode; } SNodeListNode;
typedef enum ETrimType {
TRIM_TYPE_LEADING = 1,
TRIM_TYPE_TRAILING,
TRIM_TYPE_BOTH,
} ETrimType;
typedef struct SFunctionNode { typedef struct SFunctionNode {
SExprNode node; // QUERY_NODE_FUNCTION SExprNode node; // QUERY_NODE_FUNCTION
char functionName[TSDB_FUNC_NAME_LEN]; char functionName[TSDB_FUNC_NAME_LEN];
@ -181,6 +187,7 @@ typedef struct SFunctionNode {
int32_t pkBytes; int32_t pkBytes;
bool hasOriginalFunc; bool hasOriginalFunc;
int32_t originalFuncId; int32_t originalFuncId;
ETrimType trimType;
} SFunctionNode; } SFunctionNode;
typedef struct STableNode { typedef struct STableNode {

22
include/libs/scalar/scalar.h
View File

@ -61,6 +61,15 @@ int32_t atanFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutp
int32_t ceilFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput); int32_t ceilFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t floorFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput); int32_t floorFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t roundFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput); int32_t roundFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t truncFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t piFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t expFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t lnFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t modFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t signFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t degreesFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t radiansFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
/* String functions */ /* String functions */
int32_t lengthFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput); int32_t lengthFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
@ -73,6 +82,13 @@ int32_t ltrimFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOut
int32_t rtrimFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput); int32_t rtrimFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t substrFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput); int32_t substrFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t md5Function(SScalarParam* pInput, int32_t inputNum, SScalarParam* pOutput); int32_t md5Function(SScalarParam* pInput, int32_t inputNum, SScalarParam* pOutput);
int32_t charFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t asciiFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t positionFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t trimFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t replaceFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t repeatFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t substrIdxFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
/* Conversion functions */ /* Conversion functions */
int32_t castFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput); int32_t castFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
@ -89,6 +105,10 @@ int32_t timeDiffFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *p
int32_t nowFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput); int32_t nowFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t todayFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput); int32_t todayFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t timezoneFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput); int32_t timezoneFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t weekdayFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t dayofweekFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t weekFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t weekofyearFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
bool getTimePseudoFuncEnv(struct SFunctionNode *pFunc, SFuncExecEnv *pEnv); bool getTimePseudoFuncEnv(struct SFunctionNode *pFunc, SFuncExecEnv *pEnv);
@ -106,7 +126,7 @@ int32_t sumScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *
int32_t minScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput); int32_t minScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t maxScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput); int32_t maxScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t avgScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput); int32_t avgScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t stddevScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput); int32_t stdScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t leastSQRScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput); int32_t leastSQRScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t percentileScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput); int32_t percentileScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t apercentileScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput); int32_t apercentileScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);

1
include/util/taoserror.h
View File

@ -877,6 +877,7 @@ int32_t taosGetErrSize();
#define TSDB_CODE_FUNC_TIME_UNIT_TOO_SMALL TAOS_DEF_ERROR_CODE(0, 0x280B) #define TSDB_CODE_FUNC_TIME_UNIT_TOO_SMALL TAOS_DEF_ERROR_CODE(0, 0x280B)
#define TSDB_CODE_FUNC_INVALID_VALUE_RANGE TAOS_DEF_ERROR_CODE(0, 0x280C) #define TSDB_CODE_FUNC_INVALID_VALUE_RANGE TAOS_DEF_ERROR_CODE(0, 0x280C)
#define TSDB_CODE_FUNC_SETUP_ERROR TAOS_DEF_ERROR_CODE(0, 0x280D) #define TSDB_CODE_FUNC_SETUP_ERROR TAOS_DEF_ERROR_CODE(0, 0x280D)
#define TSDB_CODE_FUNC_INVALID_RES_LENGTH TAOS_DEF_ERROR_CODE(0, 0x280E)
//udf //udf

2
include/util/tchecksum.h
View File

@ -29,7 +29,7 @@ static FORCE_INLINE TSCKSUM taosCalcChecksum(TSCKSUM csi, const uint8_t *stream,
} }
static FORCE_INLINE int32_t taosCalcChecksumAppend(TSCKSUM csi, uint8_t *stream, uint32_t ssize) { static FORCE_INLINE int32_t taosCalcChecksumAppend(TSCKSUM csi, uint8_t *stream, uint32_t ssize) {
if (ssize < sizeof(TSCKSUM)) return -1; if (ssize < sizeof(TSCKSUM)) return TSDB_CODE_INVALID_PARA;
*((TSCKSUM *)(stream + ssize - sizeof(TSCKSUM))) = (*crc32c)(csi, stream, (size_t)(ssize - sizeof(TSCKSUM))); *((TSCKSUM *)(stream + ssize - sizeof(TSCKSUM))) = (*crc32c)(csi, stream, (size_t)(ssize - sizeof(TSCKSUM)));

14
source/common/src/tglobal.c
View File

@ -691,8 +691,8 @@ static int32_t taosAddServerCfg(SConfig *pCfg) {
tsNumOfSnodeWriteThreads = tsNumOfCores / 4; tsNumOfSnodeWriteThreads = tsNumOfCores / 4;
tsNumOfSnodeWriteThreads = TRANGE(tsNumOfSnodeWriteThreads, 2, 4); tsNumOfSnodeWriteThreads = TRANGE(tsNumOfSnodeWriteThreads, 2, 4);
tsRpcQueueMemoryAllowed = tsTotalMemoryKB * 1024 * 0.1; tsQueueMemoryAllowed = tsTotalMemoryKB * 1024 * 0.1;
tsRpcQueueMemoryAllowed = TRANGE(tsRpcQueueMemoryAllowed, TSDB_MAX_MSG_SIZE * 10LL, TSDB_MAX_MSG_SIZE * 10000LL); tsQueueMemoryAllowed = TRANGE(tsQueueMemoryAllowed, TSDB_MAX_MSG_SIZE * 10LL, TSDB_MAX_MSG_SIZE * 10000LL);
// clang-format off // clang-format off
TAOS_CHECK_RETURN(cfgAddDir(pCfg, "dataDir", tsDataDir, CFG_SCOPE_SERVER, CFG_DYN_NONE)); TAOS_CHECK_RETURN(cfgAddDir(pCfg, "dataDir", tsDataDir, CFG_SCOPE_SERVER, CFG_DYN_NONE));
@ -722,7 +722,7 @@ static int32_t taosAddServerCfg(SConfig *pCfg) {
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "numOfSnodeSharedThreads", tsNumOfSnodeStreamThreads, 2, 1024, CFG_SCOPE_SERVER, CFG_DYN_NONE)); TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "numOfSnodeSharedThreads", tsNumOfSnodeStreamThreads, 2, 1024, CFG_SCOPE_SERVER, CFG_DYN_NONE));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "numOfSnodeUniqueThreads", tsNumOfSnodeWriteThreads, 2, 1024, CFG_SCOPE_SERVER, CFG_DYN_NONE)); TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "numOfSnodeUniqueThreads", tsNumOfSnodeWriteThreads, 2, 1024, CFG_SCOPE_SERVER, CFG_DYN_NONE));
TAOS_CHECK_RETURN(cfgAddInt64(pCfg, "rpcQueueMemoryAllowed", tsRpcQueueMemoryAllowed, TSDB_MAX_MSG_SIZE * 10L, INT64_MAX, CFG_SCOPE_BOTH, CFG_DYN_NONE)); TAOS_CHECK_RETURN(cfgAddInt64(pCfg, "rpcQueueMemoryAllowed", tsQueueMemoryAllowed, TSDB_MAX_MSG_SIZE * 10L, INT64_MAX, CFG_SCOPE_BOTH, CFG_DYN_NONE));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "syncElectInterval", tsElectInterval, 10, 1000 * 60 * 24 * 2, CFG_SCOPE_SERVER, CFG_DYN_NONE)); TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "syncElectInterval", tsElectInterval, 10, 1000 * 60 * 24 * 2, CFG_SCOPE_SERVER, CFG_DYN_NONE));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "syncHeartbeatInterval", tsHeartbeatInterval, 10, 1000 * 60 * 24 * 2, CFG_SCOPE_SERVER, CFG_DYN_NONE)); TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "syncHeartbeatInterval", tsHeartbeatInterval, 10, 1000 * 60 * 24 * 2, CFG_SCOPE_SERVER, CFG_DYN_NONE));
@ -958,9 +958,9 @@ static int32_t taosUpdateServerCfg(SConfig *pCfg) {
pItem = cfgGetItem(pCfg, "rpcQueueMemoryAllowed"); pItem = cfgGetItem(pCfg, "rpcQueueMemoryAllowed");
if (pItem != NULL && pItem->stype == CFG_STYPE_DEFAULT) { if (pItem != NULL && pItem->stype == CFG_STYPE_DEFAULT) {
tsRpcQueueMemoryAllowed = totalMemoryKB * 1024 * 0.1; tsQueueMemoryAllowed = totalMemoryKB * 1024 * 0.1;
tsRpcQueueMemoryAllowed = TRANGE(tsRpcQueueMemoryAllowed, TSDB_MAX_MSG_SIZE * 10LL, TSDB_MAX_MSG_SIZE * 10000LL); tsQueueMemoryAllowed = TRANGE(tsQueueMemoryAllowed, TSDB_MAX_MSG_SIZE * 10LL, TSDB_MAX_MSG_SIZE * 10000LL);
pItem->i64 = tsRpcQueueMemoryAllowed; pItem->i64 = tsQueueMemoryAllowed;
pItem->stype = stype; pItem->stype = stype;
} }
@ -1357,7 +1357,7 @@ static int32_t taosSetServerCfg(SConfig *pCfg) {
tsNumOfSnodeWriteThreads = pItem->i32; tsNumOfSnodeWriteThreads = pItem->i32;
TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "rpcQueueMemoryAllowed"); TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "rpcQueueMemoryAllowed");
tsRpcQueueMemoryAllowed = pItem->i64; tsQueueMemoryAllowed = pItem->i64;
TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "simdEnable"); TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "simdEnable");
tsSIMDEnable = (bool)pItem->bval; tsSIMDEnable = (bool)pItem->bval;

2
source/common/src/tname.c
View File

@ -169,7 +169,7 @@ void tNameAssign(SName* dst, const SName* src) { memcpy(dst, src, sizeof(SName))
int32_t tNameSetDbName(SName* dst, int32_t acct, const char* dbName, size_t nameLen) { int32_t tNameSetDbName(SName* dst, int32_t acct, const char* dbName, size_t nameLen) {
// too long account id or too long db name // too long account id or too long db name
if (nameLen <= 0 || nameLen >= tListLen(dst->dbname)) { if (nameLen <= 0 || nameLen >= tListLen(dst->dbname)) {
return -1; return TSDB_CODE_INVALID_PARA;
} }
dst->type = TSDB_DB_NAME_T; dst->type = TSDB_DB_NAME_T;

2
source/dnode/mgmt/mgmt_dnode/src/dmHandle.c
View File

@ -124,7 +124,7 @@ void dmSendStatusReq(SDnodeMgmt *pMgmt) {
req.numOfSupportVnodes = tsNumOfSupportVnodes; req.numOfSupportVnodes = tsNumOfSupportVnodes;
req.numOfDiskCfg = tsDiskCfgNum; req.numOfDiskCfg = tsDiskCfgNum;
req.memTotal = tsTotalMemoryKB * 1024; req.memTotal = tsTotalMemoryKB * 1024;
req.memAvail = req.memTotal - tsRpcQueueMemoryAllowed - 16 * 1024 * 1024; req.memAvail = req.memTotal - tsQueueMemoryAllowed - 16 * 1024 * 1024;
tstrncpy(req.dnodeEp, tsLocalEp, TSDB_EP_LEN); tstrncpy(req.dnodeEp, tsLocalEp, TSDB_EP_LEN);
tstrncpy(req.machineId, pMgmt->pData->machineId, TSDB_MACHINE_ID_LEN + 1); tstrncpy(req.machineId, pMgmt->pData->machineId, TSDB_MACHINE_ID_LEN + 1);

2
source/dnode/mgmt/node_mgmt/src/dmTransport.c
View File

@ -218,7 +218,7 @@ static void dmProcessRpcMsg(SDnode *pDnode, SRpcMsg *pRpc, SEpSet *pEpSet) {
pRpc->info.wrapper = pWrapper; pRpc->info.wrapper = pWrapper;
EQItype itype = IsReq(pRpc) ? RPC_QITEM : DEF_QITEM; // rsp msg is not restricted by tsRpcQueueMemoryUsed EQItype itype = IsReq(pRpc) ? RPC_QITEM : DEF_QITEM; // rsp msg is not restricted by tsQueueMemoryUsed
code = taosAllocateQitem(sizeof(SRpcMsg), itype, pRpc->contLen, (void **)&pMsg); code = taosAllocateQitem(sizeof(SRpcMsg), itype, pRpc->contLen, (void **)&pMsg);
if (code) goto _OVER; if (code) goto _OVER;

2
source/dnode/mgmt/test/sut/src/sut.cpp
View File

@ -36,7 +36,7 @@ void Testbase::InitLog(const char* path) {
tstrncpy(tsLogDir, path, PATH_MAX); tstrncpy(tsLogDir, path, PATH_MAX);
taosGetSystemInfo(); taosGetSystemInfo();
tsRpcQueueMemoryAllowed = tsTotalMemoryKB * 0.1; tsQueueMemoryAllowed = tsTotalMemoryKB * 0.1;
if (taosInitLog("taosdlog", 1, false) != 0) { if (taosInitLog("taosdlog", 1, false) != 0) {
printf("failed to init log file\n"); printf("failed to init log file\n");
} }

1
source/dnode/mnode/impl/inc/mndStream.h
View File

@ -152,6 +152,7 @@ bool streamTaskIterNextTask(SStreamTaskIter *pIter);
int32_t streamTaskIterGetCurrent(SStreamTaskIter *pIter, SStreamTask **pTask); int32_t streamTaskIterGetCurrent(SStreamTaskIter *pIter, SStreamTask **pTask);
int32_t mndInitExecInfo(); int32_t mndInitExecInfo();
void mndInitStreamExecInfo(SMnode *pMnode, SStreamExecInfo *pExecInfo); void mndInitStreamExecInfo(SMnode *pMnode, SStreamExecInfo *pExecInfo);
void mndInitStreamExecInfoForLeader(SMnode *pMnode);
int32_t removeExpiredNodeEntryAndTaskInBuf(SArray *pNodeSnapshot); int32_t removeExpiredNodeEntryAndTaskInBuf(SArray *pNodeSnapshot);
void removeStreamTasksInBuf(SStreamObj *pStream, SStreamExecInfo *pExecNode); void removeStreamTasksInBuf(SStreamObj *pStream, SStreamExecInfo *pExecNode);

23
source/dnode/mnode/impl/src/mndMain.c
View File

@ -96,7 +96,7 @@ static void mndPullupTrans(SMnode *pMnode) {
void *pReq = mndBuildTimerMsg(&contLen); void *pReq = mndBuildTimerMsg(&contLen);
if (pReq != NULL) { if (pReq != NULL) {
SRpcMsg rpcMsg = {.msgType = TDMT_MND_TRANS_TIMER, .pCont = pReq, .contLen = contLen}; SRpcMsg rpcMsg = {.msgType = TDMT_MND_TRANS_TIMER, .pCont = pReq, .contLen = contLen};
//TODO check return value // TODO check return value
(void)tmsgPutToQueue(&pMnode->msgCb, WRITE_QUEUE, &rpcMsg); (void)tmsgPutToQueue(&pMnode->msgCb, WRITE_QUEUE, &rpcMsg);
} }
} }
@ -107,7 +107,7 @@ static void mndPullupCompacts(SMnode *pMnode) {
void *pReq = mndBuildTimerMsg(&contLen); void *pReq = mndBuildTimerMsg(&contLen);
if (pReq != NULL) { if (pReq != NULL) {
SRpcMsg rpcMsg = {.msgType = TDMT_MND_COMPACT_TIMER, .pCont = pReq, .contLen = contLen}; SRpcMsg rpcMsg = {.msgType = TDMT_MND_COMPACT_TIMER, .pCont = pReq, .contLen = contLen};
//TODO check return value // TODO check return value
(void)tmsgPutToQueue(&pMnode->msgCb, WRITE_QUEUE, &rpcMsg); (void)tmsgPutToQueue(&pMnode->msgCb, WRITE_QUEUE, &rpcMsg);
} }
} }
@ -117,7 +117,7 @@ static void mndPullupTtl(SMnode *pMnode) {
int32_t contLen = 0; int32_t contLen = 0;
void *pReq = mndBuildTimerMsg(&contLen); void *pReq = mndBuildTimerMsg(&contLen);
SRpcMsg rpcMsg = {.msgType = TDMT_MND_TTL_TIMER, .pCont = pReq, .contLen = contLen}; SRpcMsg rpcMsg = {.msgType = TDMT_MND_TTL_TIMER, .pCont = pReq, .contLen = contLen};
//TODO check return value // TODO check return value
(void)tmsgPutToQueue(&pMnode->msgCb, WRITE_QUEUE, &rpcMsg); (void)tmsgPutToQueue(&pMnode->msgCb, WRITE_QUEUE, &rpcMsg);
} }
@ -125,7 +125,7 @@ static void mndPullupTrimDb(SMnode *pMnode) {
mTrace("pullup s3migrate"); mTrace("pullup s3migrate");
int32_t contLen = 0; int32_t contLen = 0;
void *pReq = mndBuildTimerMsg(&contLen); void *pReq = mndBuildTimerMsg(&contLen);
SRpcMsg rpcMsg = {.msgType = TDMT_MND_S3MIGRATE_DB_TIMER, .pCont = pReq, .contLen = contLen}; SRpcMsg rpcMsg = {.msgType = TDMT_MND_TRIM_DB_TIMER, .pCont = pReq, .contLen = contLen};
// TODO check return value // TODO check return value
(void)tmsgPutToQueue(&pMnode->msgCb, WRITE_QUEUE, &rpcMsg); (void)tmsgPutToQueue(&pMnode->msgCb, WRITE_QUEUE, &rpcMsg);
} }
@ -134,8 +134,8 @@ static void mndPullupS3MigrateDb(SMnode *pMnode) {
mTrace("pullup trim"); mTrace("pullup trim");
int32_t contLen = 0; int32_t contLen = 0;
void *pReq = mndBuildTimerMsg(&contLen); void *pReq = mndBuildTimerMsg(&contLen);
SRpcMsg rpcMsg = {.msgType = TDMT_MND_TRIM_DB_TIMER, .pCont = pReq, .contLen = contLen};
// TODO check return value // TODO check return value
SRpcMsg rpcMsg = {.msgType = TDMT_MND_S3MIGRATE_DB_TIMER, .pCont = pReq, .contLen = contLen};
(void)tmsgPutToQueue(&pMnode->msgCb, WRITE_QUEUE, &rpcMsg); (void)tmsgPutToQueue(&pMnode->msgCb, WRITE_QUEUE, &rpcMsg);
} }
@ -477,7 +477,7 @@ static int32_t mndCreateDir(SMnode *pMnode, const char *path) {
static int32_t mndInitWal(SMnode *pMnode) { static int32_t mndInitWal(SMnode *pMnode) {
int32_t code = 0; int32_t code = 0;
char path[PATH_MAX + 20] = {0}; char path[PATH_MAX + 20] = {0};
(void)snprintf(path, sizeof(path), "%s%swal", pMnode->path, TD_DIRSEP); (void)snprintf(path, sizeof(path), "%s%swal", pMnode->path, TD_DIRSEP);
SWalCfg cfg = {.vgId = 1, SWalCfg cfg = {.vgId = 1,
.fsyncPeriod = 0, .fsyncPeriod = 0,
@ -490,13 +490,12 @@ static int32_t mndInitWal(SMnode *pMnode) {
.encryptKey = {0}}; .encryptKey = {0}};
#if defined(TD_ENTERPRISE) #if defined(TD_ENTERPRISE)
if(tsiEncryptAlgorithm == DND_CA_SM4 && (tsiEncryptScope & DND_CS_MNODE_WAL) == DND_CS_MNODE_WAL){ if (tsiEncryptAlgorithm == DND_CA_SM4 && (tsiEncryptScope & DND_CS_MNODE_WAL) == DND_CS_MNODE_WAL) {
cfg.encryptAlgorithm = (tsiEncryptScope & DND_CS_MNODE_WAL)? tsiEncryptAlgorithm : 0; cfg.encryptAlgorithm = (tsiEncryptScope & DND_CS_MNODE_WAL) ? tsiEncryptAlgorithm : 0;
if(tsEncryptKey[0] == '\0'){ if (tsEncryptKey[0] == '\0') {
code = TSDB_CODE_DNODE_INVALID_ENCRYPTKEY; code = TSDB_CODE_DNODE_INVALID_ENCRYPTKEY;
TAOS_RETURN(code); TAOS_RETURN(code);
} } else {
else{
(void)strncpy(cfg.encryptKey, tsEncryptKey, ENCRYPT_KEY_LEN); (void)strncpy(cfg.encryptKey, tsEncryptKey, ENCRYPT_KEY_LEN);
} }
} }
@ -911,7 +910,7 @@ int32_t mndProcessRpcMsg(SRpcMsg *pMsg, SQueueInfo *pQueueInfo) {
} else if (code == 0) { } else if (code == 0) {
mGTrace("msg:%p, successfully processed", pMsg); mGTrace("msg:%p, successfully processed", pMsg);
} else { } else {
//TODO removve this wrong set code // TODO removve this wrong set code
if (code == -1) { if (code == -1) {
code = terrno; code = terrno;
} }

10
source/dnode/mnode/impl/src/mndStream.c
View File

@ -2564,6 +2564,10 @@ int32_t mndProcessCheckpointReport(SRpcMsg *pReq) {
} }
tDecoderClear(&decoder); tDecoderClear(&decoder);
streamMutexLock(&execInfo.lock);
mndInitStreamExecInfo(pMnode, &execInfo);
streamMutexUnlock(&execInfo.lock);
mDebug("receive stream task checkpoint-report msg, vgId:%d, s-task:0x%x, checkpointId:%" PRId64 mDebug("receive stream task checkpoint-report msg, vgId:%d, s-task:0x%x, checkpointId:%" PRId64
" checkpointVer:%" PRId64 " transId:%d", " checkpointVer:%" PRId64 " transId:%d",
req.nodeId, req.taskId, req.checkpointId, req.checkpointVer, req.transId); req.nodeId, req.taskId, req.checkpointId, req.checkpointVer, req.transId);
@ -2831,6 +2835,12 @@ void mndInitStreamExecInfo(SMnode *pMnode, SStreamExecInfo *pExecInfo) {
pExecInfo->initTaskList = true; pExecInfo->initTaskList = true;
} }
void mndInitStreamExecInfoForLeader(SMnode* pMnode) {
execInfo.initTaskList = false;
mInfo("init stream execInfo for leader");
mndInitStreamExecInfo(pMnode, &execInfo);
}
void addAllStreamTasksIntoBuf(SMnode *pMnode, SStreamExecInfo *pExecInfo) { void addAllStreamTasksIntoBuf(SMnode *pMnode, SStreamExecInfo *pExecInfo) {
SSdb *pSdb = pMnode->pSdb; SSdb *pSdb = pMnode->pSdb;
SStreamObj *pStream = NULL; SStreamObj *pStream = NULL;

2
source/dnode/mnode/impl/src/mndSync.c
View File

@ -18,6 +18,7 @@
#include "mndCluster.h" #include "mndCluster.h"
#include "mndTrans.h" #include "mndTrans.h"
#include "mndUser.h" #include "mndUser.h"
#include "mndStream.h"
static int32_t mndSyncEqCtrlMsg(const SMsgCb *msgcb, SRpcMsg *pMsg) { static int32_t mndSyncEqCtrlMsg(const SMsgCb *msgcb, SRpcMsg *pMsg) {
if (pMsg == NULL || pMsg->pCont == NULL) { if (pMsg == NULL || pMsg->pCont == NULL) {
@ -381,6 +382,7 @@ static void mndBecomeLearner(const SSyncFSM *pFsm) {
static void mndBecomeLeader(const SSyncFSM *pFsm) { static void mndBecomeLeader(const SSyncFSM *pFsm) {
mInfo("vgId:1, become leader"); mInfo("vgId:1, become leader");
SMnode *pMnode = pFsm->data; SMnode *pMnode = pFsm->data;
mndInitStreamExecInfoForLeader(pMnode);
} }
static bool mndApplyQueueEmpty(const SSyncFSM *pFsm) { static bool mndApplyQueueEmpty(const SSyncFSM *pFsm) {

2
source/dnode/mnode/impl/test/stream/stream.cpp
View File

File diff suppressed because one or more lines are too long

17
source/dnode/vnode/src/inc/tsdb.h
View File

@ -343,7 +343,6 @@ typedef struct {
rocksdb_readoptions_t *readoptions; rocksdb_readoptions_t *readoptions;
rocksdb_writebatch_t *writebatch; rocksdb_writebatch_t *writebatch;
rocksdb_writebatch_t *rwritebatch; rocksdb_writebatch_t *rwritebatch;
TdThreadMutex rMutex;
STSchema *pTSchema; STSchema *pTSchema;
} SRocksCache; } SRocksCache;
@ -915,12 +914,20 @@ typedef enum {
READER_EXEC_ROWS = 0x2, READER_EXEC_ROWS = 0x2,
} EExecMode; } EExecMode;
#define LAST_COL_VERSION (0x1) #define LAST_COL_VERSION_1 (0x1) // add primary key, version
#define LAST_COL_VERSION_2 (0x2) // add cache status
#define LAST_COL_VERSION LAST_COL_VERSION_2
typedef enum {
TSDB_LAST_CACHE_VALID = 0, // last_cache has valid data
TSDB_LAST_CACHE_NO_CACHE, // last_cache has no data, but tsdb may have data
} ELastCacheStatus;
typedef struct { typedef struct {
SRowKey rowKey; SRowKey rowKey;
int8_t dirty; int8_t dirty;
SColVal colVal; SColVal colVal;
ELastCacheStatus cacheStatus;
} SLastCol; } SLastCol;
typedef struct { typedef struct {

6
source/dnode/vnode/src/meta/metaCache.c
View File

@ -271,7 +271,7 @@ int32_t metaCacheUpsert(SMeta* pMeta, SMetaInfo* pInfo) {
if (*ppEntry) { // update if (*ppEntry) { // update
if (pInfo->suid != (*ppEntry)->info.suid) { if (pInfo->suid != (*ppEntry)->info.suid) {
metaError("meta/cache: suid should be same as the one in cache."); metaError("meta/cache: suid should be same as the one in cache.");
return TSDB_CODE_FAILED; return TSDB_CODE_INVALID_PARA;
} }
if (pInfo->version > (*ppEntry)->info.version) { if (pInfo->version > (*ppEntry)->info.version) {
(*ppEntry)->info.version = pInfo->version; (*ppEntry)->info.version = pInfo->version;
@ -543,7 +543,7 @@ int32_t metaGetCachedTableUidList(void* pVnode, tb_uid_t suid, const uint8_t* pK
STagFilterResEntry** pEntry = taosHashGet(pTableMap, &suid, sizeof(uint64_t)); STagFilterResEntry** pEntry = taosHashGet(pTableMap, &suid, sizeof(uint64_t));
if (NULL == pEntry) { if (NULL == pEntry) {
metaError("meta/cache: pEntry should not be NULL."); metaError("meta/cache: pEntry should not be NULL.");
return TSDB_CODE_FAILED; return TSDB_CODE_NOT_FOUND;
} }
*acquireRes = 1; *acquireRes = 1;
@ -750,7 +750,7 @@ int32_t metaGetCachedTbGroup(void* pVnode, tb_uid_t suid, const uint8_t* pKey, i
STagFilterResEntry** pEntry = taosHashGet(pTableMap, &suid, sizeof(uint64_t)); STagFilterResEntry** pEntry = taosHashGet(pTableMap, &suid, sizeof(uint64_t));
if (NULL == pEntry) { if (NULL == pEntry) {
metaDebug("suid %" PRIu64 " not in tb group cache", suid); metaDebug("suid %" PRIu64 " not in tb group cache", suid);
return TSDB_CODE_FAILED; return TSDB_CODE_NOT_FOUND;
} }
*pList = taosArrayDup(taosLRUCacheValue(pCache, pHandle), NULL); *pList = taosArrayDup(taosLRUCacheValue(pCache, pHandle), NULL);

688
source/dnode/vnode/src/tsdb/tsdbCache.c
View File

File diff suppressed because it is too large Load Diff

2
source/dnode/vnode/src/tsdb/tsdbReadUtil.c
View File

@ -139,7 +139,7 @@ int32_t getPosInBlockInfoBuf(SBlockInfoBuf* pBuf, int32_t index, STableBlockScan
int32_t bucketIndex = index / pBuf->numPerBucket; int32_t bucketIndex = index / pBuf->numPerBucket;
char** pBucket = taosArrayGet(pBuf->pData, bucketIndex); char** pBucket = taosArrayGet(pBuf->pData, bucketIndex);
if (pBucket == NULL) { if (pBucket == NULL) {
return TSDB_CODE_FAILED; return TSDB_CODE_NOT_FOUND;
} }
*pInfo = (STableBlockScanInfo*)((*pBucket) + (index % pBuf->numPerBucket) * sizeof(STableBlockScanInfo)); *pInfo = (STableBlockScanInfo*)((*pBucket) + (index % pBuf->numPerBucket) * sizeof(STableBlockScanInfo));

3
source/dnode/vnode/src/vnd/vnodeCfg.c
View File

@ -301,8 +301,7 @@ int vnodeDecodeConfig(const SJson *pJson, void *pObj) {
#if defined(TD_ENTERPRISE) #if defined(TD_ENTERPRISE)
if (pCfg->tdbEncryptAlgorithm == DND_CA_SM4) { if (pCfg->tdbEncryptAlgorithm == DND_CA_SM4) {
if (tsEncryptKey[0] == 0) { if (tsEncryptKey[0] == 0) {
terrno = TSDB_CODE_DNODE_INVALID_ENCRYPTKEY; return terrno = TSDB_CODE_DNODE_INVALID_ENCRYPTKEY;
return -1;
} else { } else {
strncpy(pCfg->tdbEncryptKey, tsEncryptKey, ENCRYPT_KEY_LEN); strncpy(pCfg->tdbEncryptKey, tsEncryptKey, ENCRYPT_KEY_LEN);
} }

7
source/dnode/vnode/src/vnd/vnodeCommit.c
View File

@ -413,9 +413,9 @@ static int vnodeCommitImpl(SCommitInfo *pInfo) {
pInfo->info.state.commitID, pInfo->info.state.committed, pInfo->info.state.commitTerm); pInfo->info.state.commitID, pInfo->info.state.committed, pInfo->info.state.commitTerm);
// persist wal before starting // persist wal before starting
if (walPersist(pVnode->pWal) < 0) { if ((code = walPersist(pVnode->pWal)) < 0) {
vError("vgId:%d, failed to persist wal since %s", TD_VID(pVnode), terrstr()); vError("vgId:%d, failed to persist wal since %s", TD_VID(pVnode), tstrerror(code));
return -1; return code;
} }
(void)vnodeGetPrimaryDir(pVnode->path, pVnode->diskPrimary, pVnode->pTfs, dir, TSDB_FILENAME_LEN); (void)vnodeGetPrimaryDir(pVnode->path, pVnode->diskPrimary, pVnode->pTfs, dir, TSDB_FILENAME_LEN);
@ -556,7 +556,6 @@ int vnodeDecodeInfo(uint8_t *pData, SVnodeInfo *pInfo) {
pJson = tjsonParse(pData); pJson = tjsonParse(pData);
if (pJson == NULL) { if (pJson == NULL) {
TSDB_CHECK_CODE(code = TSDB_CODE_INVALID_DATA_FMT, lino, _exit); TSDB_CHECK_CODE(code = TSDB_CODE_INVALID_DATA_FMT, lino, _exit);
return -1;
} }
code = tjsonToObject(pJson, "config", vnodeDecodeConfig, (void *)&pInfo->config); code = tjsonToObject(pJson, "config", vnodeDecodeConfig, (void *)&pInfo->config);

31
source/dnode/vnode/src/vnd/vnodeOpen.c
View File

@ -39,13 +39,14 @@ static int32_t vnodeMkDir(STfs *pTfs, const char *path) {
} }
int32_t vnodeCreate(const char *path, SVnodeCfg *pCfg, int32_t diskPrimary, STfs *pTfs) { int32_t vnodeCreate(const char *path, SVnodeCfg *pCfg, int32_t diskPrimary, STfs *pTfs) {
int32_t code = 0;
SVnodeInfo info = {0}; SVnodeInfo info = {0};
char dir[TSDB_FILENAME_LEN] = {0}; char dir[TSDB_FILENAME_LEN] = {0};
// check config // check config
if (vnodeCheckCfg(pCfg) < 0) { if ((code = vnodeCheckCfg(pCfg)) < 0) {
vError("vgId:%d, failed to create vnode since:%s", pCfg->vgId, tstrerror(terrno)); vError("vgId:%d, failed to create vnode since:%s", pCfg->vgId, tstrerror(code));
return -1; return code;
} }
// create vnode env // create vnode env
@ -72,9 +73,9 @@ int32_t vnodeCreate(const char *path, SVnodeCfg *pCfg, int32_t diskPrimary, STfs
} }
vInfo("vgId:%d, save config while create", info.config.vgId); vInfo("vgId:%d, save config while create", info.config.vgId);
if (vnodeSaveInfo(dir, &info) < 0 || vnodeCommitInfo(dir) < 0) { if ((code = vnodeSaveInfo(dir, &info)) < 0 || (code = vnodeCommitInfo(dir)) < 0) {
vError("vgId:%d, failed to save vnode config since %s", pCfg ? pCfg->vgId : 0, tstrerror(terrno)); vError("vgId:%d, failed to save vnode config since %s", pCfg ? pCfg->vgId : 0, tstrerror(code));
return -1; return code;
} }
vInfo("vgId:%d, vnode is created", info.config.vgId); vInfo("vgId:%d, vnode is created", info.config.vgId);
@ -93,7 +94,7 @@ int32_t vnodeAlterReplica(const char *path, SAlterVnodeReplicaReq *pReq, int32_t
ret = vnodeLoadInfo(dir, &info); ret = vnodeLoadInfo(dir, &info);
if (ret < 0) { if (ret < 0) {
vError("vgId:%d, failed to read vnode config from %s since %s", pReq->vgId, path, tstrerror(terrno)); vError("vgId:%d, failed to read vnode config from %s since %s", pReq->vgId, path, tstrerror(terrno));
return -1; return ret;
} }
SSyncCfg *pCfg = &info.config.syncCfg; SSyncCfg *pCfg = &info.config.syncCfg;
@ -144,13 +145,13 @@ int32_t vnodeAlterReplica(const char *path, SAlterVnodeReplicaReq *pReq, int32_t
ret = vnodeSaveInfo(dir, &info); ret = vnodeSaveInfo(dir, &info);
if (ret < 0) { if (ret < 0) {
vError("vgId:%d, failed to save vnode config since %s", pReq->vgId, tstrerror(terrno)); vError("vgId:%d, failed to save vnode config since %s", pReq->vgId, tstrerror(terrno));
return -1; return ret;
} }
ret = vnodeCommitInfo(dir); ret = vnodeCommitInfo(dir);
if (ret < 0) { if (ret < 0) {
vError("vgId:%d, failed to commit vnode config since %s", pReq->vgId, tstrerror(terrno)); vError("vgId:%d, failed to commit vnode config since %s", pReq->vgId, tstrerror(terrno));
return -1; return ret;
} }
vInfo("vgId:%d, vnode config is saved", info.config.vgId); vInfo("vgId:%d, vnode config is saved", info.config.vgId);
@ -226,7 +227,7 @@ int32_t vnodeAlterHashRange(const char *srcPath, const char *dstPath, SAlterVnod
ret = vnodeLoadInfo(dir, &info); ret = vnodeLoadInfo(dir, &info);
if (ret < 0) { if (ret < 0) {
vError("vgId:%d, failed to read vnode config from %s since %s", pReq->srcVgId, srcPath, tstrerror(terrno)); vError("vgId:%d, failed to read vnode config from %s since %s", pReq->srcVgId, srcPath, tstrerror(terrno));
return -1; return ret;
} }
vInfo("vgId:%d, alter hashrange from [%u, %u] to [%u, %u]", pReq->srcVgId, info.config.hashBegin, info.config.hashEnd, vInfo("vgId:%d, alter hashrange from [%u, %u] to [%u, %u]", pReq->srcVgId, info.config.hashBegin, info.config.hashEnd,
@ -256,13 +257,13 @@ int32_t vnodeAlterHashRange(const char *srcPath, const char *dstPath, SAlterVnod
ret = vnodeSaveInfo(dir, &info); ret = vnodeSaveInfo(dir, &info);
if (ret < 0) { if (ret < 0) {
vError("vgId:%d, failed to save vnode config since %s", pReq->dstVgId, tstrerror(terrno)); vError("vgId:%d, failed to save vnode config since %s", pReq->dstVgId, tstrerror(terrno));
return -1; return ret;
} }
ret = vnodeCommitInfo(dir); ret = vnodeCommitInfo(dir);
if (ret < 0) { if (ret < 0) {
vError("vgId:%d, failed to commit vnode config since %s", pReq->dstVgId, tstrerror(terrno)); vError("vgId:%d, failed to commit vnode config since %s", pReq->dstVgId, tstrerror(terrno));
return -1; return ret;
} }
vInfo("vgId:%d, rename %s to %s", pReq->dstVgId, srcPath, dstPath); vInfo("vgId:%d, rename %s to %s", pReq->dstVgId, srcPath, dstPath);
@ -270,7 +271,7 @@ int32_t vnodeAlterHashRange(const char *srcPath, const char *dstPath, SAlterVnod
if (ret < 0) { if (ret < 0) {
vError("vgId:%d, failed to rename vnode from %s to %s since %s", pReq->dstVgId, srcPath, dstPath, vError("vgId:%d, failed to rename vnode from %s to %s since %s", pReq->dstVgId, srcPath, dstPath,
tstrerror(terrno)); tstrerror(terrno));
return -1; return ret;
} }
vInfo("vgId:%d, vnode hashrange is altered", info.config.vgId); vInfo("vgId:%d, vnode hashrange is altered", info.config.vgId);
@ -293,9 +294,9 @@ int32_t vnodeRestoreVgroupId(const char *srcPath, const char *dstPath, int32_t s
} }
(void)vnodeGetPrimaryDir(srcPath, diskPrimary, pTfs, dir, TSDB_FILENAME_LEN); (void)vnodeGetPrimaryDir(srcPath, diskPrimary, pTfs, dir, TSDB_FILENAME_LEN);
if (vnodeLoadInfo(dir, &info) < 0) { if ((code = vnodeLoadInfo(dir, &info)) < 0) {
vError("vgId:%d, failed to read vnode config from %s since %s", srcVgId, srcPath, tstrerror(terrno)); vError("vgId:%d, failed to read vnode config from %s since %s", srcVgId, srcPath, tstrerror(terrno));
return -1; return code;
} }
if (info.config.vgId == srcVgId) { if (info.config.vgId == srcVgId) {

15
source/dnode/vnode/src/vnd/vnodeQuery.c
View File

@ -545,7 +545,7 @@ int32_t vnodeGetAllTableList(SVnode *pVnode, uint64_t uid, SArray *list) {
SMCtbCursor *pCur = metaOpenCtbCursor(pVnode, uid, 1); SMCtbCursor *pCur = metaOpenCtbCursor(pVnode, uid, 1);
if (NULL == pCur) { if (NULL == pCur) {
qError("vnode get all table list failed"); qError("vnode get all table list failed");
return TSDB_CODE_FAILED; return terrno;
} }
while (1) { while (1) {
@ -576,7 +576,7 @@ int32_t vnodeGetCtbIdList(void *pVnode, int64_t suid, SArray *list) {
SMCtbCursor *pCur = metaOpenCtbCursor(pVnodeObj, suid, 1); SMCtbCursor *pCur = metaOpenCtbCursor(pVnodeObj, suid, 1);
if (NULL == pCur) { if (NULL == pCur) {
qError("vnode get all table list failed"); qError("vnode get all table list failed");
return TSDB_CODE_FAILED; return terrno;
} }
while (1) { while (1) {
@ -627,7 +627,7 @@ int32_t vnodeGetStbIdListByFilter(SVnode *pVnode, int64_t suid, SArray *list, bo
int32_t code = TSDB_CODE_SUCCESS; int32_t code = TSDB_CODE_SUCCESS;
SMStbCursor *pCur = metaOpenStbCursor(pVnode->pMeta, suid); SMStbCursor *pCur = metaOpenStbCursor(pVnode->pMeta, suid);
if (!pCur) { if (!pCur) {
return TSDB_CODE_FAILED; return terrno;
} }
while (1) { while (1) {
@ -655,7 +655,7 @@ _exit:
int32_t vnodeGetCtbNum(SVnode *pVnode, int64_t suid, int64_t *num) { int32_t vnodeGetCtbNum(SVnode *pVnode, int64_t suid, int64_t *num) {
SMCtbCursor *pCur = metaOpenCtbCursor(pVnode, suid, 0); SMCtbCursor *pCur = metaOpenCtbCursor(pVnode, suid, 0);
if (!pCur) { if (!pCur) {
return TSDB_CODE_FAILED; return terrno;
} }
*num = 0; *num = 0;
@ -757,8 +757,7 @@ int32_t vnodeGetTimeSeriesNum(SVnode *pVnode, int64_t *num) {
SArray *suidList = NULL; SArray *suidList = NULL;
if (!(suidList = taosArrayInit(1, sizeof(tb_uid_t)))) { if (!(suidList = taosArrayInit(1, sizeof(tb_uid_t)))) {
terrno = TSDB_CODE_OUT_OF_MEMORY; return terrno = TSDB_CODE_OUT_OF_MEMORY;
return TSDB_CODE_FAILED;
} }
int32_t tbFilterSize = 0; int32_t tbFilterSize = 0;
@ -774,7 +773,7 @@ int32_t vnodeGetTimeSeriesNum(SVnode *pVnode, int64_t *num) {
(tbFilterSize && vnodeGetStbIdListByFilter(pVnode, 0, suidList, vnodeTimeSeriesFilter, pVnode) < 0)) { (tbFilterSize && vnodeGetStbIdListByFilter(pVnode, 0, suidList, vnodeTimeSeriesFilter, pVnode) < 0)) {
qError("vgId:%d, failed to get stb id list error: %s", TD_VID(pVnode), terrstr()); qError("vgId:%d, failed to get stb id list error: %s", TD_VID(pVnode), terrstr());
taosArrayDestroy(suidList); taosArrayDestroy(suidList);
return TSDB_CODE_FAILED; return terrno;
} }
*num = 0; *num = 0;
@ -799,7 +798,7 @@ _exit:
int32_t vnodeGetAllCtbNum(SVnode *pVnode, int64_t *num) { int32_t vnodeGetAllCtbNum(SVnode *pVnode, int64_t *num) {
SMStbCursor *pCur = metaOpenStbCursor(pVnode->pMeta, 0); SMStbCursor *pCur = metaOpenStbCursor(pVnode->pMeta, 0);
if (!pCur) { if (!pCur) {
return TSDB_CODE_FAILED; return terrno;
} }
*num = 0; *num = 0;

2
source/dnode/vnode/src/vnd/vnodeSvr.c
View File

@ -1980,7 +1980,7 @@ _err:
tDecoderClear(&coder); tDecoderClear(&coder);
vError("vgId:%d, failed to create tsma %s:%" PRIi64 " version %" PRIi64 "for table %" PRIi64 " since %s", vError("vgId:%d, failed to create tsma %s:%" PRIi64 " version %" PRIi64 "for table %" PRIi64 " since %s",
TD_VID(pVnode), req.indexName, req.indexUid, ver, req.tableUid, terrstr()); TD_VID(pVnode), req.indexName, req.indexUid, ver, req.tableUid, terrstr());
return -1; return terrno;
} }
/** /**

2
source/dnode/vnode/src/vnd/vnodeSync.c
View File

@ -381,7 +381,7 @@ int32_t vnodeProcessSyncMsg(SVnode *pVnode, SRpcMsg *pMsg, SRpcMsg **pRsp) {
static int32_t vnodeSyncEqCtrlMsg(const SMsgCb *msgcb, SRpcMsg *pMsg) { static int32_t vnodeSyncEqCtrlMsg(const SMsgCb *msgcb, SRpcMsg *pMsg) {
if (pMsg == NULL || pMsg->pCont == NULL) { if (pMsg == NULL || pMsg->pCont == NULL) {
return -1; return TSDB_CODE_INVALID_PARA;
} }
if (msgcb == NULL || msgcb->putToQueueFp == NULL) { if (msgcb == NULL || msgcb->putToQueueFp == NULL) {

11
source/libs/executor/src/dataDispatcher.c
View File

@ -332,17 +332,12 @@ int32_t createDataDispatcher(SDataSinkManager* pManager, const SDataSinkNode* pD
goto _return; goto _return;
} }
if (NULL == dispatcher->pDataBlocks) {
taosMemoryFree(dispatcher);
terrno = TSDB_CODE_OUT_OF_MEMORY;
goto _return;
}
*pHandle = dispatcher; *pHandle = dispatcher;
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
_return: _return:
if (dispatcher) {
taosMemoryFree(pManager); dsDestroyDataSinker(dispatcher);
}
return terrno; return terrno;
} }

3
source/libs/executor/src/exchangeoperator.c
View File

@ -485,6 +485,9 @@ void freeSourceDataInfo(void* p) {
} }
void doDestroyExchangeOperatorInfo(void* param) { void doDestroyExchangeOperatorInfo(void* param) {
if (param == NULL) {
return;
}
SExchangeInfo* pExInfo = (SExchangeInfo*)param; SExchangeInfo* pExInfo = (SExchangeInfo*)param;
if (pExInfo->pFetchRpcHandles) { if (pExInfo->pFetchRpcHandles) {
for (int32_t i = 0; i < pExInfo->pFetchRpcHandles->size; ++i) { for (int32_t i = 0; i < pExInfo->pFetchRpcHandles->size; ++i) {

3
source/libs/executor/src/executor.c
View File

@ -316,7 +316,6 @@ qTaskInfo_t qCreateQueueExecTaskInfo(void* msg, SReadHandle* pReaderHandle, int3
qTaskInfo_t pTaskInfo = NULL; qTaskInfo_t pTaskInfo = NULL;
code = qCreateExecTask(pReaderHandle, vgId, 0, pPlan, &pTaskInfo, NULL, 0, NULL, OPTR_EXEC_MODEL_QUEUE); code = qCreateExecTask(pReaderHandle, vgId, 0, pPlan, &pTaskInfo, NULL, 0, NULL, OPTR_EXEC_MODEL_QUEUE);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
nodesDestroyNode((SNode*)pPlan);
qDestroyTask(pTaskInfo); qDestroyTask(pTaskInfo);
terrno = code; terrno = code;
return NULL; return NULL;
@ -352,7 +351,6 @@ qTaskInfo_t qCreateStreamExecTaskInfo(void* msg, SReadHandle* readers, int32_t v
qTaskInfo_t pTaskInfo = NULL; qTaskInfo_t pTaskInfo = NULL;
code = qCreateExecTask(readers, vgId, taskId, pPlan, &pTaskInfo, NULL, 0, NULL, OPTR_EXEC_MODEL_STREAM); code = qCreateExecTask(readers, vgId, taskId, pPlan, &pTaskInfo, NULL, 0, NULL, OPTR_EXEC_MODEL_STREAM);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
nodesDestroyNode((SNode*)pPlan);
qDestroyTask(pTaskInfo); qDestroyTask(pTaskInfo);
terrno = code; terrno = code;
return NULL; return NULL;
@ -360,7 +358,6 @@ qTaskInfo_t qCreateStreamExecTaskInfo(void* msg, SReadHandle* readers, int32_t v
code = qStreamInfoResetTimewindowFilter(pTaskInfo); code = qStreamInfoResetTimewindowFilter(pTaskInfo);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
nodesDestroyNode((SNode*)pPlan);
qDestroyTask(pTaskInfo); qDestroyTask(pTaskInfo);
terrno = code; terrno = code;
return NULL; return NULL;

2
source/libs/executor/src/querytask.c
View File

@ -65,6 +65,7 @@ int32_t doCreateTask(uint64_t queryId, uint64_t taskId, int32_t vgId, EOPTR_EXEC
p->id.taskId = taskId; p->id.taskId = taskId;
p->id.str = taosMemoryMalloc(64); p->id.str = taosMemoryMalloc(64);
if (p->id.str == NULL) { if (p->id.str == NULL) {
doDestroyTask(p);
return terrno; return terrno;
} }
@ -100,6 +101,7 @@ int32_t createExecTaskInfo(SSubplan* pPlan, SExecTaskInfo** pTaskInfo, SReadHand
int32_t vgId, char* sql, EOPTR_EXEC_MODEL model) { int32_t vgId, char* sql, EOPTR_EXEC_MODEL model) {
int32_t code = doCreateTask(pPlan->id.queryId, taskId, vgId, model, &pHandle->api, pTaskInfo); int32_t code = doCreateTask(pPlan->id.queryId, taskId, vgId, model, &pHandle->api, pTaskInfo);
if (*pTaskInfo == NULL || code != 0) { if (*pTaskInfo == NULL || code != 0) {
nodesDestroyNode((SNode*)pPlan);
taosMemoryFree(sql); taosMemoryFree(sql);
return code; return code;
} }

47
source/libs/executor/src/scanoperator.c
View File

@ -255,21 +255,28 @@ static int32_t doFilterByBlockSMA(SFilterInfo* pFilterInfo, SColumnDataAgg* pCol
return filterRangeExecute(pFilterInfo, pColsAgg, numOfCols, numOfRows, keep); return filterRangeExecute(pFilterInfo, pColsAgg, numOfCols, numOfRows, keep);
} }
static bool doLoadBlockSMA(STableScanBase* pTableScanInfo, SSDataBlock* pBlock, SExecTaskInfo* pTaskInfo) { static int32_t doLoadBlockSMA(STableScanBase* pTableScanInfo, SSDataBlock* pBlock, SExecTaskInfo* pTaskInfo,
bool* pLoad) {
SStorageAPI* pAPI = &pTaskInfo->storageAPI; SStorageAPI* pAPI = &pTaskInfo->storageAPI;
bool allColumnsHaveAgg = true;
bool hasNullSMA = false;
if (pLoad != NULL) {
*pLoad = false;
}
bool allColumnsHaveAgg = true;
bool hasNullSMA = false;
int32_t code = pAPI->tsdReader.tsdReaderRetrieveBlockSMAInfo(pTableScanInfo->dataReader, pBlock, &allColumnsHaveAgg, int32_t code = pAPI->tsdReader.tsdReaderRetrieveBlockSMAInfo(pTableScanInfo->dataReader, pBlock, &allColumnsHaveAgg,
&hasNullSMA); &hasNullSMA);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
T_LONG_JMP(pTaskInfo->env, code); return code;
} }
if (!allColumnsHaveAgg || hasNullSMA) { if (!allColumnsHaveAgg || hasNullSMA) {
return false; *pLoad = false;
} else {
*pLoad = true;
} }
return true;
return code;
} }
static void doSetTagColumnData(STableScanBase* pTableScanInfo, SSDataBlock* pBlock, SExecTaskInfo* pTaskInfo, static void doSetTagColumnData(STableScanBase* pTableScanInfo, SSDataBlock* pBlock, SExecTaskInfo* pTaskInfo,
@ -330,14 +337,14 @@ static int32_t loadDataBlock(SOperatorInfo* pOperator, STableScanBase* pTableSca
int32_t lino = 0; int32_t lino = 0;
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
SStorageAPI* pAPI = &pTaskInfo->storageAPI; SStorageAPI* pAPI = &pTaskInfo->storageAPI;
bool loadSMA = false;
SFileBlockLoadRecorder* pCost = &pTableScanInfo->readRecorder; SFileBlockLoadRecorder* pCost = &pTableScanInfo->readRecorder;
pCost->totalBlocks += 1; pCost->totalBlocks += 1;
pCost->totalRows += pBlock->info.rows; pCost->totalRows += pBlock->info.rows;
bool loadSMA = false;
*status = pTableScanInfo->dataBlockLoadFlag; *status = pTableScanInfo->dataBlockLoadFlag;
if (pOperator->exprSupp.pFilterInfo != NULL) { if (pOperator->exprSupp.pFilterInfo != NULL) {
(*status) = FUNC_DATA_REQUIRED_DATA_LOAD; (*status) = FUNC_DATA_REQUIRED_DATA_LOAD;
} else { } else {
@ -373,7 +380,14 @@ static int32_t loadDataBlock(SOperatorInfo* pOperator, STableScanBase* pTableSca
} else if (*status == FUNC_DATA_REQUIRED_SMA_LOAD) { } else if (*status == FUNC_DATA_REQUIRED_SMA_LOAD) {
pCost->loadBlockStatis += 1; pCost->loadBlockStatis += 1;
loadSMA = true; // mark the operation of load sma; loadSMA = true; // mark the operation of load sma;
bool success = doLoadBlockSMA(pTableScanInfo, pBlock, pTaskInfo); bool success = true;
code = doLoadBlockSMA(pTableScanInfo, pBlock, pTaskInfo, &success);
if (code) {
pAPI->tsdReader.tsdReaderReleaseDataBlock(pTableScanInfo->dataReader);
qError("%s failed to retrieve sma info", GET_TASKID(pTaskInfo));
QUERY_CHECK_CODE(code, lino, _end);
}
if (success) { // failed to load the block sma data, data block statistics does not exist, load data block instead if (success) { // failed to load the block sma data, data block statistics does not exist, load data block instead
qDebug("%s data block SMA loaded, brange:%" PRId64 "-%" PRId64 ", rows:%" PRId64, GET_TASKID(pTaskInfo), qDebug("%s data block SMA loaded, brange:%" PRId64 "-%" PRId64 ", rows:%" PRId64, GET_TASKID(pTaskInfo),
pBlockInfo->window.skey, pBlockInfo->window.ekey, pBlockInfo->rows); pBlockInfo->window.skey, pBlockInfo->window.ekey, pBlockInfo->rows);
@ -387,13 +401,21 @@ static int32_t loadDataBlock(SOperatorInfo* pOperator, STableScanBase* pTableSca
} }
if(*status != FUNC_DATA_REQUIRED_DATA_LOAD) { if(*status != FUNC_DATA_REQUIRED_DATA_LOAD) {
qError("[loadDataBlock] invalid status:%d", *status); pAPI->tsdReader.tsdReaderReleaseDataBlock(pTableScanInfo->dataReader);
qError("%s loadDataBlock invalid status:%d", GET_TASKID(pTaskInfo), *status);
return TSDB_CODE_QRY_EXECUTOR_INTERNAL_ERROR; return TSDB_CODE_QRY_EXECUTOR_INTERNAL_ERROR;
} }
// try to filter data block according to sma info // try to filter data block according to sma info
if (pOperator->exprSupp.pFilterInfo != NULL && (!loadSMA)) { if (pOperator->exprSupp.pFilterInfo != NULL && (!loadSMA)) {
bool success = doLoadBlockSMA(pTableScanInfo, pBlock, pTaskInfo); bool success = true;
code = doLoadBlockSMA(pTableScanInfo, pBlock, pTaskInfo, &success);
if (code) {
pAPI->tsdReader.tsdReaderReleaseDataBlock(pTableScanInfo->dataReader);
qError("%s failed to retrieve sma info", GET_TASKID(pTaskInfo));
QUERY_CHECK_CODE(code, lino, _end);
}
if (success) { if (success) {
size_t size = taosArrayGetSize(pBlock->pDataBlock); size_t size = taosArrayGetSize(pBlock->pDataBlock);
bool keep = false; bool keep = false;
@ -3806,6 +3828,9 @@ _end:
} }
static void destroyStreamScanOperatorInfo(void* param) { static void destroyStreamScanOperatorInfo(void* param) {
if (param == NULL) {
return;
}
SStreamScanInfo* pStreamScan = (SStreamScanInfo*)param; SStreamScanInfo* pStreamScan = (SStreamScanInfo*)param;
if (pStreamScan->pTableScanOp && pStreamScan->pTableScanOp->info) { if (pStreamScan->pTableScanOp && pStreamScan->pTableScanOp->info) {

3
source/libs/executor/src/streameventwindowoperator.c
View File

@ -43,6 +43,9 @@ typedef struct SEventWindowInfo {
} SEventWindowInfo; } SEventWindowInfo;
void destroyStreamEventOperatorInfo(void* param) { void destroyStreamEventOperatorInfo(void* param) {
if (param == NULL) {
return;
}
SStreamEventAggOperatorInfo* pInfo = (SStreamEventAggOperatorInfo*)param; SStreamEventAggOperatorInfo* pInfo = (SStreamEventAggOperatorInfo*)param;
cleanupBasicInfo(&pInfo->binfo); cleanupBasicInfo(&pInfo->binfo);
destroyStreamAggSupporter(&pInfo->streamAggSup); destroyStreamAggSupporter(&pInfo->streamAggSup);

9
source/libs/executor/src/streamtimewindowoperator.c
View File

@ -454,6 +454,9 @@ void clearGroupResInfo(SGroupResInfo* pGroupResInfo) {
} }
void destroyStreamFinalIntervalOperatorInfo(void* param) { void destroyStreamFinalIntervalOperatorInfo(void* param) {
if (param == NULL) {
return;
}
SStreamIntervalOperatorInfo* pInfo = (SStreamIntervalOperatorInfo*)param; SStreamIntervalOperatorInfo* pInfo = (SStreamIntervalOperatorInfo*)param;
cleanupBasicInfo(&pInfo->binfo); cleanupBasicInfo(&pInfo->binfo);
cleanupAggSup(&pInfo->aggSup); cleanupAggSup(&pInfo->aggSup);
@ -2031,6 +2034,9 @@ void destroyStreamAggSupporter(SStreamAggSupporter* pSup) {
} }
void destroyStreamSessionAggOperatorInfo(void* param) { void destroyStreamSessionAggOperatorInfo(void* param) {
if (param == NULL) {
return;
}
SStreamSessionAggOperatorInfo* pInfo = (SStreamSessionAggOperatorInfo*)param; SStreamSessionAggOperatorInfo* pInfo = (SStreamSessionAggOperatorInfo*)param;
cleanupBasicInfo(&pInfo->binfo); cleanupBasicInfo(&pInfo->binfo);
destroyStreamAggSupporter(&pInfo->streamAggSup); destroyStreamAggSupporter(&pInfo->streamAggSup);
@ -4106,6 +4112,9 @@ _error:
} }
void destroyStreamStateOperatorInfo(void* param) { void destroyStreamStateOperatorInfo(void* param) {
if (param == NULL) {
return;
}
SStreamStateAggOperatorInfo* pInfo = (SStreamStateAggOperatorInfo*)param; SStreamStateAggOperatorInfo* pInfo = (SStreamStateAggOperatorInfo*)param;
cleanupBasicInfo(&pInfo->binfo); cleanupBasicInfo(&pInfo->binfo);
destroyStreamAggSupporter(&pInfo->streamAggSup); destroyStreamAggSupporter(&pInfo->streamAggSup);

8
source/libs/executor/src/sysscanoperator.c
View File

@ -552,6 +552,8 @@ static SSDataBlock* sysTableScanUserCols(SOperatorInfo* pOperator) {
blockDataCleanup(pInfo->pRes); blockDataCleanup(pInfo->pRes);
pDataBlock = buildInfoSchemaTableMetaBlock(TSDB_INS_TABLE_COLS); pDataBlock = buildInfoSchemaTableMetaBlock(TSDB_INS_TABLE_COLS);
QUERY_CHECK_NULL(pDataBlock, code, lino, _end, terrno);
code = blockDataEnsureCapacity(pDataBlock, pOperator->resultInfo.capacity); code = blockDataEnsureCapacity(pDataBlock, pOperator->resultInfo.capacity);
QUERY_CHECK_CODE(code, lino, _end); QUERY_CHECK_CODE(code, lino, _end);
@ -708,6 +710,8 @@ static SSDataBlock* sysTableScanUserTags(SOperatorInfo* pOperator) {
int32_t numOfRows = 0; int32_t numOfRows = 0;
dataBlock = buildInfoSchemaTableMetaBlock(TSDB_INS_TABLE_TAGS); dataBlock = buildInfoSchemaTableMetaBlock(TSDB_INS_TABLE_TAGS);
QUERY_CHECK_NULL(dataBlock, code, lino, _end, terrno);
code = blockDataEnsureCapacity(dataBlock, pOperator->resultInfo.capacity); code = blockDataEnsureCapacity(dataBlock, pOperator->resultInfo.capacity);
QUERY_CHECK_CODE(code, lino, _end); QUERY_CHECK_CODE(code, lino, _end);
@ -1354,6 +1358,8 @@ static SSDataBlock* sysTableBuildUserTablesByUids(SOperatorInfo* pOperator) {
varDataSetLen(dbname, strlen(varDataVal(dbname))); varDataSetLen(dbname, strlen(varDataVal(dbname)));
p = buildInfoSchemaTableMetaBlock(TSDB_INS_TABLE_TABLES); p = buildInfoSchemaTableMetaBlock(TSDB_INS_TABLE_TABLES);
QUERY_CHECK_NULL(p, code, lino, _end, terrno);
code = blockDataEnsureCapacity(p, pOperator->resultInfo.capacity); code = blockDataEnsureCapacity(p, pOperator->resultInfo.capacity);
QUERY_CHECK_CODE(code, lino, _end); QUERY_CHECK_CODE(code, lino, _end);
@ -2075,6 +2081,7 @@ static SSDataBlock* sysTableScanFromMNode(SOperatorInfo* pOperator, SSysTableSca
int32_t tempRes = tSerializeSRetrieveTableReq(buf1, contLen, &pInfo->req); int32_t tempRes = tSerializeSRetrieveTableReq(buf1, contLen, &pInfo->req);
if (tempRes < 0) { if (tempRes < 0) {
code = terrno; code = terrno;
taosMemoryFree(buf1);
return NULL; return NULL;
} }
@ -2083,6 +2090,7 @@ static SSDataBlock* sysTableScanFromMNode(SOperatorInfo* pOperator, SSysTableSca
if (NULL == pMsgSendInfo) { if (NULL == pMsgSendInfo) {
qError("%s prepare message %d failed", GET_TASKID(pTaskInfo), (int32_t)sizeof(SMsgSendInfo)); qError("%s prepare message %d failed", GET_TASKID(pTaskInfo), (int32_t)sizeof(SMsgSendInfo));
pTaskInfo->code = TSDB_CODE_OUT_OF_MEMORY; pTaskInfo->code = TSDB_CODE_OUT_OF_MEMORY;
taosMemoryFree(buf1);
return NULL; return NULL;
} }

3
source/libs/executor/src/timewindowoperator.c
View File

@ -1208,6 +1208,9 @@ static void freeItem(void* param) {
} }
void destroyIntervalOperatorInfo(void* param) { void destroyIntervalOperatorInfo(void* param) {
if (param == NULL) {
return;
}
SIntervalAggOperatorInfo* pInfo = (SIntervalAggOperatorInfo*)param; SIntervalAggOperatorInfo* pInfo = (SIntervalAggOperatorInfo*)param;
cleanupBasicInfo(&pInfo->binfo); cleanupBasicInfo(&pInfo->binfo);
cleanupAggSup(&pInfo->aggSup); cleanupAggSup(&pInfo->aggSup);

17
source/libs/function/inc/builtinsimpl.h
View File

@ -96,19 +96,20 @@ int32_t avgInvertFunction(SqlFunctionCtx* pCtx);
int32_t avgCombine(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx); int32_t avgCombine(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx);
int32_t getAvgInfoSize(); int32_t getAvgInfoSize();
bool getStddevFuncEnv(struct SFunctionNode* pFunc, SFuncExecEnv* pEnv); bool getStdFuncEnv(struct SFunctionNode* pFunc, SFuncExecEnv* pEnv);
int32_t stddevFunctionSetup(SqlFunctionCtx* pCtx, SResultRowEntryInfo* pResultInfo); int32_t stdFunctionSetup(SqlFunctionCtx* pCtx, SResultRowEntryInfo* pResultInfo);
int32_t stddevFunction(SqlFunctionCtx* pCtx); int32_t stdFunction(SqlFunctionCtx* pCtx);
int32_t stddevFunctionMerge(SqlFunctionCtx* pCtx); int32_t stdFunctionMerge(SqlFunctionCtx* pCtx);
int32_t stddevFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock); int32_t stddevFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock);
int32_t stddevPartialFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock); int32_t stdvarFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock);
int32_t stdPartialFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock);
#ifdef BUILD_NO_CALL #ifdef BUILD_NO_CALL
int32_t stddevInvertFunction(SqlFunctionCtx* pCtx); int32_t stdInvertFunction(SqlFunctionCtx* pCtx);
#endif #endif
int32_t stddevCombine(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx); int32_t stdCombine(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx);
int32_t getStddevInfoSize(); int32_t getStdInfoSize();
bool getLeastSQRFuncEnv(struct SFunctionNode* pFunc, SFuncExecEnv* pEnv); bool getLeastSQRFuncEnv(struct SFunctionNode* pFunc, SFuncExecEnv* pEnv);
int32_t leastSQRFunctionSetup(SqlFunctionCtx* pCtx, SResultRowEntryInfo* pResultInfo); int32_t leastSQRFunctionSetup(SqlFunctionCtx* pCtx, SResultRowEntryInfo* pResultInfo);

700
source/libs/function/src/builtins.c
View File

@ -247,7 +247,7 @@ static int32_t addTimezoneParam(SNodeList* pList) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t addDbPrecisonParam(SNodeList** pList, uint8_t precision) { static int32_t addUint8Param(SNodeList** pList, uint8_t param) {
SValueNode* pVal = NULL; SValueNode* pVal = NULL;
int32_t code = nodesMakeNode(QUERY_NODE_VALUE, (SNode**)&pVal); int32_t code = nodesMakeNode(QUERY_NODE_VALUE, (SNode**)&pVal);
if (pVal == NULL) { if (pVal == NULL) {
@ -259,9 +259,9 @@ static int32_t addDbPrecisonParam(SNodeList** pList, uint8_t precision) {
pVal->notReserved = true; pVal->notReserved = true;
pVal->node.resType.type = TSDB_DATA_TYPE_TINYINT; pVal->node.resType.type = TSDB_DATA_TYPE_TINYINT;
pVal->node.resType.bytes = tDataTypes[TSDB_DATA_TYPE_TINYINT].bytes; pVal->node.resType.bytes = tDataTypes[TSDB_DATA_TYPE_TINYINT].bytes;
pVal->node.resType.precision = precision; pVal->node.resType.precision = param;
pVal->datum.i = (int64_t)precision; pVal->datum.i = (int64_t)param;
pVal->typeData = (int64_t)precision; pVal->typeData = (int64_t)param;
code = nodesListMakeAppend(pList, (SNode*)pVal); code = nodesListMakeAppend(pList, (SNode*)pVal);
if (TSDB_CODE_SUCCESS != code) { if (TSDB_CODE_SUCCESS != code) {
@ -493,7 +493,7 @@ static int32_t translateAvgStateMerge(SFunctionNode* pFunc, char* pErrBuf, int32
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t translateStddevPartial(SFunctionNode* pFunc, char* pErrBuf, int32_t len) { static int32_t translateStdPartial(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (1 != LIST_LENGTH(pFunc->pParameterList)) { if (1 != LIST_LENGTH(pFunc->pParameterList)) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName); return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
} }
@ -503,11 +503,11 @@ static int32_t translateStddevPartial(SFunctionNode* pFunc, char* pErrBuf, int32
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
} }
pFunc->node.resType = (SDataType){.bytes = getStddevInfoSize() + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY}; pFunc->node.resType = (SDataType){.bytes = getStdInfoSize() + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY};
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t translateStddevMerge(SFunctionNode* pFunc, char* pErrBuf, int32_t len) { static int32_t translateStdMerge(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (1 != LIST_LENGTH(pFunc->pParameterList)) { if (1 != LIST_LENGTH(pFunc->pParameterList)) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName); return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
} }
@ -522,7 +522,7 @@ static int32_t translateStddevMerge(SFunctionNode* pFunc, char* pErrBuf, int32_t
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t translateStddevState(SFunctionNode* pFunc, char* pErrBuf, int32_t len) { static int32_t translateStdState(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (1 != LIST_LENGTH(pFunc->pParameterList)) { if (1 != LIST_LENGTH(pFunc->pParameterList)) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName); return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
} }
@ -532,11 +532,11 @@ static int32_t translateStddevState(SFunctionNode* pFunc, char* pErrBuf, int32_t
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
} }
pFunc->node.resType = (SDataType){.bytes = getStddevInfoSize() + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY}; pFunc->node.resType = (SDataType){.bytes = getStdInfoSize() + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY};
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t translateStddevStateMerge(SFunctionNode* pFunc, char* pErrBuf, int32_t len) { static int32_t translateStdStateMerge(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (1 != LIST_LENGTH(pFunc->pParameterList)) { if (1 != LIST_LENGTH(pFunc->pParameterList)) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName); return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
} }
@ -546,7 +546,7 @@ static int32_t translateStddevStateMerge(SFunctionNode* pFunc, char* pErrBuf, in
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
} }
pFunc->node.resType = (SDataType){.bytes = getStddevInfoSize() + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY}; pFunc->node.resType = (SDataType){.bytes = getStdInfoSize() + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY};
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
@ -563,7 +563,7 @@ static int32_t translateNowToday(SFunctionNode* pFunc, char* pErrBuf, int32_t le
// add database precision as param // add database precision as param
uint8_t dbPrec = pFunc->node.resType.precision; uint8_t dbPrec = pFunc->node.resType.precision;
int32_t code = addDbPrecisonParam(&pFunc->pParameterList, dbPrec); int32_t code = addUint8Param(&pFunc->pParameterList, dbPrec);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
return code; return code;
} }
@ -573,6 +573,53 @@ static int32_t translateNowToday(SFunctionNode* pFunc, char* pErrBuf, int32_t le
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t translatePi(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
pFunc->node.resType =
(SDataType){.bytes = tDataTypes[TSDB_DATA_TYPE_DOUBLE].bytes, .type = TSDB_DATA_TYPE_DOUBLE};
return TSDB_CODE_SUCCESS;
}
static int32_t translateRound(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (2 != LIST_LENGTH(pFunc->pParameterList) && 1 != LIST_LENGTH(pFunc->pParameterList)) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
}
uint8_t paraType = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->type;
if (!IS_NUMERIC_TYPE(paraType) && !IS_NULL_TYPE(paraType)) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
} else if (IS_NULL_TYPE(paraType)) {
paraType = TSDB_DATA_TYPE_BIGINT;
}
if (2 == LIST_LENGTH(pFunc->pParameterList)) {
uint8_t paraType2 = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 1))->type;
if (!IS_NUMERIC_TYPE(paraType2) && !IS_NULL_TYPE(paraType2)) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
}
pFunc->node.resType = (SDataType){.bytes = tDataTypes[paraType].bytes, .type = paraType};
return TSDB_CODE_SUCCESS;
}
static int32_t translateTrunc(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (2 != LIST_LENGTH(pFunc->pParameterList)) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
}
uint8_t paraType = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->type;
if (!IS_NUMERIC_TYPE(paraType) && !IS_NULL_TYPE(paraType)) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
uint8_t paraType2 = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 1))->type;
if (!IS_NUMERIC_TYPE(paraType2) && !IS_NULL_TYPE(paraType2)) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
pFunc->node.resType = (SDataType){.bytes = tDataTypes[paraType].bytes, .type = paraType};
return TSDB_CODE_SUCCESS;
}
static int32_t translateTimePseudoColumn(SFunctionNode* pFunc, char* pErrBuf, int32_t len) { static int32_t translateTimePseudoColumn(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
// pseudo column do not need to check parameters // pseudo column do not need to check parameters
@ -1745,7 +1792,7 @@ static int32_t translateIrate(SFunctionNode* pFunc, char* pErrBuf, int32_t len)
// add database precision as param // add database precision as param
uint8_t dbPrec = pFunc->node.resType.precision; uint8_t dbPrec = pFunc->node.resType.precision;
int32_t code = addDbPrecisonParam(&pFunc->pParameterList, dbPrec); int32_t code = addUint8Param(&pFunc->pParameterList, dbPrec);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
return code; return code;
} }
@ -1776,7 +1823,7 @@ static int32_t translateIrateImpl(SFunctionNode* pFunc, char* pErrBuf, int32_t l
// add database precision as param // add database precision as param
uint8_t dbPrec = pFunc->node.resType.precision; uint8_t dbPrec = pFunc->node.resType.precision;
int32_t code = addDbPrecisonParam(&pFunc->pParameterList, dbPrec); int32_t code = addUint8Param(&pFunc->pParameterList, dbPrec);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
return code; return code;
} }
@ -2032,7 +2079,22 @@ static int32_t translateLength(SFunctionNode* pFunc, char* pErrBuf, int32_t len)
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName); return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
} }
if (!IS_STR_DATA_TYPE(getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->type)) { uint8_t paraType = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->type;
if (!IS_STR_DATA_TYPE(paraType) && !IS_NULL_TYPE(paraType)) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
pFunc->node.resType = (SDataType){.bytes = tDataTypes[TSDB_DATA_TYPE_BIGINT].bytes, .type = TSDB_DATA_TYPE_BIGINT};
return TSDB_CODE_SUCCESS;
}
static int32_t translateCharLength(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (1 != LIST_LENGTH(pFunc->pParameterList)) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
}
uint8_t paraType = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->type;
if (paraType == TSDB_DATA_TYPE_VARBINARY || (!IS_STR_DATA_TYPE(paraType) && !IS_NULL_TYPE(paraType))) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
} }
@ -2124,31 +2186,182 @@ static int32_t translateSubstr(SFunctionNode* pFunc, char* pErrBuf, int32_t len)
uint8_t para0Type = pPara0->resType.type; uint8_t para0Type = pPara0->resType.type;
uint8_t para1Type = pPara1->resType.type; uint8_t para1Type = pPara1->resType.type;
if (TSDB_DATA_TYPE_VARBINARY == para0Type || !IS_STR_DATA_TYPE(para0Type) || !IS_INTEGER_TYPE(para1Type)) { if (TSDB_DATA_TYPE_VARBINARY == para0Type ||
(!IS_STR_DATA_TYPE(para0Type) && !IS_NULL_TYPE(para0Type)) ||
(!IS_INTEGER_TYPE(para1Type) && !IS_NULL_TYPE(para1Type))) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
} }
if (((SValueNode*)pPara1)->datum.i == 0) {
return invaildFuncParaValueErrMsg(pErrBuf, len, pFunc->functionName);
}
if (3 == numOfParams) { if (3 == numOfParams) {
SExprNode* pPara2 = (SExprNode*)nodesListGetNode(pFunc->pParameterList, 2); SExprNode* pPara2 = (SExprNode*)nodesListGetNode(pFunc->pParameterList, 2);
uint8_t para2Type = pPara2->resType.type; uint8_t para2Type = pPara2->resType.type;
if (!IS_INTEGER_TYPE(para2Type)) { if (!IS_INTEGER_TYPE(para2Type) && !IS_NULL_TYPE(para2Type)) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
} }
int64_t v = ((SValueNode*)pPara2)->datum.i; int64_t v = ((SValueNode*)pPara2)->datum.i;
if (v < 0) {
return invaildFuncParaValueErrMsg(pErrBuf, len, pFunc->functionName);
}
} }
pFunc->node.resType = (SDataType){.bytes = pPara0->resType.bytes, .type = pPara0->resType.type}; pFunc->node.resType = (SDataType){.bytes = pPara0->resType.bytes, .type = pPara0->resType.type};
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t translateSubstrIdx(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (3 != LIST_LENGTH(pFunc->pParameterList)) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
}
SExprNode* pPara0 = (SExprNode*)nodesListGetNode(pFunc->pParameterList, 0);
SExprNode* pPara1 = (SExprNode*)nodesListGetNode(pFunc->pParameterList, 1);
SExprNode* pPara2 = (SExprNode*)nodesListGetNode(pFunc->pParameterList, 2);
uint8_t para0Type = pPara0->resType.type;
uint8_t para1Type = pPara1->resType.type;
uint8_t para2Type = pPara2->resType.type;
if (TSDB_DATA_TYPE_VARBINARY == para0Type || (!IS_STR_DATA_TYPE(para0Type) && !IS_NULL_TYPE(para0Type)) ||
TSDB_DATA_TYPE_VARBINARY == para1Type || (!IS_STR_DATA_TYPE(para1Type) && !IS_NULL_TYPE(para1Type)) ||
(!IS_INTEGER_TYPE(para2Type) && !IS_NULL_TYPE(para2Type))) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
pFunc->node.resType = (SDataType){.bytes = pPara0->resType.bytes, .type = pPara0->resType.type};
return TSDB_CODE_SUCCESS;
}
static int32_t translateChar(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
SNode *node;
FOREACH(node, pFunc->pParameterList) {
uint8_t paraType = getSDataTypeFromNode(node)->type;
if (paraType == TSDB_DATA_TYPE_VARBINARY ||
(!IS_STR_DATA_TYPE(paraType) && !IS_NUMERIC_TYPE(paraType) && !IS_NULL_TYPE(paraType))) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
}
int32_t numOfParams = LIST_LENGTH(pFunc->pParameterList);
pFunc->node.resType = (SDataType){.bytes = 4 * numOfParams + 2, .type = TSDB_DATA_TYPE_VARCHAR};
return TSDB_CODE_SUCCESS;
}
static int32_t translateAscii(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (1 != LIST_LENGTH(pFunc->pParameterList)) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
}
uint8_t paraType = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->type;
if (paraType == TSDB_DATA_TYPE_VARBINARY ||
(!IS_STR_DATA_TYPE(paraType) && !IS_NULL_TYPE(paraType))) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
pFunc->node.resType = (SDataType){.bytes = tDataTypes[TSDB_DATA_TYPE_UTINYINT].bytes, .type = TSDB_DATA_TYPE_UTINYINT};
return TSDB_CODE_SUCCESS;
}
static int32_t translatePosition(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (2 != LIST_LENGTH(pFunc->pParameterList)) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
}
uint8_t para0Type = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->type;
if (para0Type == TSDB_DATA_TYPE_VARBINARY ||
(!IS_STR_DATA_TYPE(para0Type) && !IS_NULL_TYPE(para0Type))) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
uint8_t para1Type = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 1))->type;
if (para1Type == TSDB_DATA_TYPE_VARBINARY ||
(!IS_STR_DATA_TYPE(para1Type) && !IS_NULL_TYPE(para1Type))) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
pFunc->node.resType = (SDataType){.bytes = tDataTypes[TSDB_DATA_TYPE_BIGINT].bytes, .type = TSDB_DATA_TYPE_BIGINT};
return TSDB_CODE_SUCCESS;
}
static int32_t translateTrim(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (2 != LIST_LENGTH(pFunc->pParameterList) && 1 != LIST_LENGTH(pFunc->pParameterList)) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
}
uint8_t para0Type = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->type;
if (para0Type == TSDB_DATA_TYPE_VARBINARY ||
(!IS_STR_DATA_TYPE(para0Type) && !IS_NULL_TYPE(para0Type))) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
int32_t resLen = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->bytes;
uint8_t type = para0Type;
if (2 == LIST_LENGTH(pFunc->pParameterList)) {
uint8_t para1Type = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 1))->type;
if (para1Type == TSDB_DATA_TYPE_VARBINARY ||
(!IS_STR_DATA_TYPE(para1Type) && !IS_NULL_TYPE(para1Type))) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
resLen = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 1))->bytes;
type = para1Type;
}
if (type == TSDB_DATA_TYPE_NCHAR) {
resLen *= TSDB_NCHAR_SIZE;
}
uint8_t trimType = pFunc->trimType;
int32_t code = addUint8Param(&pFunc->pParameterList, trimType);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
pFunc->node.resType = (SDataType){.bytes = resLen, .type = type};
return TSDB_CODE_SUCCESS;
}
static int32_t translateReplace(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (3 != LIST_LENGTH(pFunc->pParameterList)) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
}
for (int32_t i = 0; i < 3; ++i) {
uint8_t paraType = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, i))->type;
if (paraType == TSDB_DATA_TYPE_VARBINARY ||
(!IS_STR_DATA_TYPE(paraType) && !IS_NULL_TYPE(paraType))) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
}
uint8_t type = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->type;
int32_t orgLen = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->bytes;
int32_t fromLen = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 1))->bytes;
int32_t toLen = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 2))->bytes;
int32_t resLen = orgLen + orgLen / fromLen * (toLen - fromLen);
pFunc->node.resType = (SDataType){.bytes = resLen, .type = type};
return TSDB_CODE_SUCCESS;
}
static int32_t translateRepeat(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (2 != LIST_LENGTH(pFunc->pParameterList)) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
}
uint8_t para0Type = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->type;
if (para0Type == TSDB_DATA_TYPE_VARBINARY ||
(!IS_STR_DATA_TYPE(para0Type) && !IS_NULL_TYPE(para0Type))) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
uint8_t para1Type = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 1))->type;
if (!IS_INTEGER_TYPE(para1Type) && !IS_NULL_TYPE(para1Type)) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
uint8_t type = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->type;
int32_t orgLen = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->bytes;
int32_t count = TMAX((int32_t)((SValueNode*)nodesListGetNode(pFunc->pParameterList, 1))->datum.i, 1);
int32_t resLen = orgLen * count;
pFunc->node.resType = (SDataType){.bytes = resLen, .type = type};
return TSDB_CODE_SUCCESS;
}
static int32_t translateCast(SFunctionNode* pFunc, char* pErrBuf, int32_t len) { static int32_t translateCast(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
// The number of parameters has been limited by the syntax definition // The number of parameters has been limited by the syntax definition
@ -2180,7 +2393,7 @@ static int32_t translateCast(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
// add database precision as param // add database precision as param
uint8_t dbPrec = pFunc->node.resType.precision; uint8_t dbPrec = pFunc->node.resType.precision;
int32_t code = addDbPrecisonParam(&pFunc->pParameterList, dbPrec); int32_t code = addUint8Param(&pFunc->pParameterList, dbPrec);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
return code; return code;
} }
@ -2251,7 +2464,7 @@ static int32_t translateToUnixtimestamp(SFunctionNode* pFunc, char* pErrBuf, int
// add database precision as param // add database precision as param
uint8_t dbPrec = pFunc->node.resType.precision; uint8_t dbPrec = pFunc->node.resType.precision;
int32_t code = addDbPrecisonParam(&pFunc->pParameterList, dbPrec); int32_t code = addUint8Param(&pFunc->pParameterList, dbPrec);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
return code; return code;
} }
@ -2325,7 +2538,7 @@ static int32_t translateTimeTruncate(SFunctionNode* pFunc, char* pErrBuf, int32_
// add database precision as param // add database precision as param
code = addDbPrecisonParam(&pFunc->pParameterList, dbPrec); code = addUint8Param(&pFunc->pParameterList, dbPrec);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
return code; return code;
} }
@ -2375,7 +2588,7 @@ static int32_t translateTimeDiff(SFunctionNode* pFunc, char* pErrBuf, int32_t le
} }
} }
int32_t code = addDbPrecisonParam(&pFunc->pParameterList, dbPrec); int32_t code = addUint8Param(&pFunc->pParameterList, dbPrec);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
return code; return code;
} }
@ -2384,6 +2597,91 @@ static int32_t translateTimeDiff(SFunctionNode* pFunc, char* pErrBuf, int32_t le
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t translateWeekday(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (1 != LIST_LENGTH(pFunc->pParameterList)) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
}
uint8_t para1Type = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->type;
if ((!IS_STR_DATA_TYPE(para1Type) && !IS_INTEGER_TYPE(para1Type) &&
!IS_TIMESTAMP_TYPE(para1Type) && !IS_NULL_TYPE(para1Type))) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
// add database precision as param
uint8_t dbPrec = pFunc->node.resType.precision;
int32_t code = addUint8Param(&pFunc->pParameterList, dbPrec);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
pFunc->node.resType =
(SDataType){.bytes = tDataTypes[TSDB_DATA_TYPE_BIGINT].bytes, .type = TSDB_DATA_TYPE_BIGINT};
return TSDB_CODE_SUCCESS;
}
static int32_t translateWeek(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (1 != LIST_LENGTH(pFunc->pParameterList) && 2 != LIST_LENGTH(pFunc->pParameterList)) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
}
uint8_t para1Type = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->type;
if ((!IS_STR_DATA_TYPE(para1Type) && !IS_INTEGER_TYPE(para1Type) &&
!IS_TIMESTAMP_TYPE(para1Type)) && !IS_NULL_TYPE(para1Type)) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
if (2 == LIST_LENGTH(pFunc->pParameterList)) {
uint8_t para2Type = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 1))->type;
if (!IS_INTEGER_TYPE(para2Type) && !IS_NULL_TYPE(para2Type)) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
if (IS_INTEGER_TYPE(para2Type)) {
SValueNode* pValue = (SValueNode*)nodesListGetNode(pFunc->pParameterList, 1);
if (pValue->datum.i < 0 || pValue->datum.i > 7) {
return invaildFuncParaValueErrMsg(pErrBuf, len, pFunc->functionName);
}
}
}
// add database precision as param
uint8_t dbPrec = pFunc->node.resType.precision;
int32_t code = addUint8Param(&pFunc->pParameterList, dbPrec);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
pFunc->node.resType =
(SDataType){.bytes = tDataTypes[TSDB_DATA_TYPE_BIGINT].bytes, .type = TSDB_DATA_TYPE_BIGINT};
return TSDB_CODE_SUCCESS;
}
static int32_t translateWeekofyear(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (1 != LIST_LENGTH(pFunc->pParameterList)) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
}
uint8_t para1Type = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->type;
if ((!IS_STR_DATA_TYPE(para1Type) && !IS_INTEGER_TYPE(para1Type) &&
!IS_TIMESTAMP_TYPE(para1Type)) && !IS_NULL_TYPE(para1Type)) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
// add database precision as param
uint8_t dbPrec = pFunc->node.resType.precision;
int32_t code = addUint8Param(&pFunc->pParameterList, dbPrec);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
pFunc->node.resType =
(SDataType){.bytes = tDataTypes[TSDB_DATA_TYPE_BIGINT].bytes, .type = TSDB_DATA_TYPE_BIGINT};
return TSDB_CODE_SUCCESS;
}
static int32_t translateToJson(SFunctionNode* pFunc, char* pErrBuf, int32_t len) { static int32_t translateToJson(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (1 != LIST_LENGTH(pFunc->pParameterList)) { if (1 != LIST_LENGTH(pFunc->pParameterList)) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName); return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
@ -2623,47 +2921,47 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
.type = FUNCTION_TYPE_STDDEV, .type = FUNCTION_TYPE_STDDEV,
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_TSMA_FUNC, .classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_TSMA_FUNC,
.translateFunc = translateInNumOutDou, .translateFunc = translateInNumOutDou,
.getEnvFunc = getStddevFuncEnv, .getEnvFunc = getStdFuncEnv,
.initFunc = stddevFunctionSetup, .initFunc = stdFunctionSetup,
.processFunc = stddevFunction, .processFunc = stdFunction,
.sprocessFunc = stddevScalarFunction, .sprocessFunc = stdScalarFunction,
.finalizeFunc = stddevFinalize, .finalizeFunc = stddevFinalize,
#ifdef BUILD_NO_CALL #ifdef BUILD_NO_CALL
.invertFunc = stddevInvertFunction, .invertFunc = stdInvertFunction,
#endif #endif
.combineFunc = stddevCombine, .combineFunc = stdCombine,
.pPartialFunc = "_stddev_partial", .pPartialFunc = "_std_partial",
.pStateFunc = "_stddev_state", .pStateFunc = "_std_state",
.pMergeFunc = "_stddev_merge" .pMergeFunc = "_stddev_merge"
}, },
{ {
.name = "_stddev_partial", .name = "_std_partial",
.type = FUNCTION_TYPE_STDDEV_PARTIAL, .type = FUNCTION_TYPE_STD_PARTIAL,
.classification = FUNC_MGT_AGG_FUNC, .classification = FUNC_MGT_AGG_FUNC,
.translateFunc = translateStddevPartial, .translateFunc = translateStdPartial,
.getEnvFunc = getStddevFuncEnv, .getEnvFunc = getStdFuncEnv,
.initFunc = stddevFunctionSetup, .initFunc = stdFunctionSetup,
.processFunc = stddevFunction, .processFunc = stdFunction,
.finalizeFunc = stddevPartialFinalize, .finalizeFunc = stdPartialFinalize,
#ifdef BUILD_NO_CALL #ifdef BUILD_NO_CALL
.invertFunc = stddevInvertFunction, .invertFunc = stdInvertFunction,
#endif #endif
.combineFunc = stddevCombine, .combineFunc = stdCombine,
}, },
{ {
.name = "_stddev_merge", .name = "_stddev_merge",
.type = FUNCTION_TYPE_STDDEV_MERGE, .type = FUNCTION_TYPE_STDDEV_MERGE,
.classification = FUNC_MGT_AGG_FUNC, .classification = FUNC_MGT_AGG_FUNC,
.translateFunc = translateStddevMerge, .translateFunc = translateStdMerge,
.getEnvFunc = getStddevFuncEnv, .getEnvFunc = getStdFuncEnv,
.initFunc = stddevFunctionSetup, .initFunc = stdFunctionSetup,
.processFunc = stddevFunctionMerge, .processFunc = stdFunctionMerge,
.finalizeFunc = stddevFinalize, .finalizeFunc = stddevFinalize,
#ifdef BUILD_NO_CALL #ifdef BUILD_NO_CALL
.invertFunc = stddevInvertFunction, .invertFunc = stdInvertFunction,
#endif #endif
.combineFunc = stddevCombine, .combineFunc = stdCombine,
.pPartialFunc = "_stddev_state_merge", .pPartialFunc = "_std_state_merge",
.pMergeFunc = "_stddev_merge", .pMergeFunc = "_stddev_merge",
}, },
{ {
@ -3409,7 +3707,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
.name = "round", .name = "round",
.type = FUNCTION_TYPE_ROUND, .type = FUNCTION_TYPE_ROUND,
.classification = FUNC_MGT_SCALAR_FUNC, .classification = FUNC_MGT_SCALAR_FUNC,
.translateFunc = translateInOutNum, .translateFunc = translateRound,
.getEnvFunc = NULL, .getEnvFunc = NULL,
.initFunc = NULL, .initFunc = NULL,
.sprocessFunc = roundFunction, .sprocessFunc = roundFunction,
@ -3489,7 +3787,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
.name = "char_length", .name = "char_length",
.type = FUNCTION_TYPE_CHAR_LENGTH, .type = FUNCTION_TYPE_CHAR_LENGTH,
.classification = FUNC_MGT_SCALAR_FUNC | FUNC_MGT_STRING_FUNC, .classification = FUNC_MGT_SCALAR_FUNC | FUNC_MGT_STRING_FUNC,
.translateFunc = translateLength, .translateFunc = translateCharLength,
.getEnvFunc = NULL, .getEnvFunc = NULL,
.initFunc = NULL, .initFunc = NULL,
.sprocessFunc = charLengthFunction, .sprocessFunc = charLengthFunction,
@ -3998,26 +4296,26 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
.finalizeFunc = NULL .finalizeFunc = NULL
}, },
{ {
.name = "_stddev_state", .name = "_std_state",
.type = FUNCTION_TYPE_STDDEV_STATE, .type = FUNCTION_TYPE_STD_STATE,
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_TSMA_FUNC, .classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_TSMA_FUNC,
.translateFunc = translateStddevState, .translateFunc = translateStdState,
.getEnvFunc = getStddevFuncEnv, .getEnvFunc = getStdFuncEnv,
.initFunc = stddevFunctionSetup, .initFunc = stdFunctionSetup,
.processFunc = stddevFunction, .processFunc = stdFunction,
.finalizeFunc = stddevPartialFinalize, .finalizeFunc = stdPartialFinalize,
.pPartialFunc = "_stddev_partial", .pPartialFunc = "_std_partial",
.pMergeFunc = "_stddev_state_merge", .pMergeFunc = "_std_state_merge",
}, },
{ {
.name = "_stddev_state_merge", .name = "_std_state_merge",
.type = FUNCTION_TYPE_STDDEV_STATE_MERGE, .type = FUNCTION_TYPE_STD_STATE_MERGE,
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_TSMA_FUNC, .classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_TSMA_FUNC,
.translateFunc = translateStddevStateMerge, .translateFunc = translateStdStateMerge,
.getEnvFunc = getStddevFuncEnv, .getEnvFunc = getStdFuncEnv,
.initFunc = stddevFunctionSetup, .initFunc = stdFunctionSetup,
.processFunc = stddevFunctionMerge, .processFunc = stdFunctionMerge,
.finalizeFunc = stddevPartialFinalize, .finalizeFunc = stdPartialFinalize,
}, },
{ {
.name = "_avg_state", .name = "_avg_state",
@ -4152,6 +4450,268 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
.processFunc = groupConstValueFunction, .processFunc = groupConstValueFunction,
.finalizeFunc = groupConstValueFinalize, .finalizeFunc = groupConstValueFinalize,
}, },
{
.name = "stddev_pop",
.type = FUNCTION_TYPE_STDDEV,
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_TSMA_FUNC,
.translateFunc = translateInNumOutDou,
.getEnvFunc = getStdFuncEnv,
.initFunc = stdFunctionSetup,
.processFunc = stdFunction,
.sprocessFunc = stdScalarFunction,
.finalizeFunc = stddevFinalize,
#ifdef BUILD_NO_CALL
.invertFunc = stdInvertFunction,
#endif
.combineFunc = stdCombine,
.pPartialFunc = "_std_partial",
.pStateFunc = "_std_state",
.pMergeFunc = "_stddev_merge"
},
{
.name = "var_pop",
.type = FUNCTION_TYPE_STDVAR,
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_TSMA_FUNC,
.translateFunc = translateInNumOutDou,
.getEnvFunc = getStdFuncEnv,
.initFunc = stdFunctionSetup,
.processFunc = stdFunction,
.sprocessFunc = stdScalarFunction,
.finalizeFunc = stdvarFinalize,
#ifdef BUILD_NO_CALL
.invertFunc = stdInvertFunction,
#endif
.combineFunc = stdCombine,
.pPartialFunc = "_std_partial",
.pStateFunc = "_std_state",
.pMergeFunc = "_stdvar_merge"
},
{
.name = "_stdvar_merge",
.type = FUNCTION_TYPE_STDVAR_MERGE,
.classification = FUNC_MGT_AGG_FUNC,
.translateFunc = translateStdMerge,
.getEnvFunc = getStdFuncEnv,
.initFunc = stdFunctionSetup,
.processFunc = stdFunctionMerge,
.finalizeFunc = stdvarFinalize,
#ifdef BUILD_NO_CALL
.invertFunc = stdInvertFunction,
#endif
.combineFunc = stdCombine,
.pPartialFunc = "_std_state_merge",
.pMergeFunc = "_stdvar_merge",
},
{
.name = "pi",
.type = FUNCTION_TYPE_PI,
.classification = FUNC_MGT_SCALAR_FUNC,
.translateFunc = translatePi,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = piFunction,
.finalizeFunc = NULL
},
{
.name = "exp",
.type = FUNCTION_TYPE_EXP,
.classification = FUNC_MGT_SCALAR_FUNC,
.translateFunc = translateInNumOutDou,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = expFunction,
.finalizeFunc = NULL
},
{
.name = "ln",
.type = FUNCTION_TYPE_LN,
.classification = FUNC_MGT_SCALAR_FUNC,
.translateFunc = translateInNumOutDou,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = lnFunction,
.finalizeFunc = NULL
},
{
.name = "mod",
.type = FUNCTION_TYPE_MOD,
.classification = FUNC_MGT_SCALAR_FUNC,
.translateFunc = translateIn2NumOutDou,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = modFunction,
.finalizeFunc = NULL
},
{
.name = "sign",
.type = FUNCTION_TYPE_SIGN,
.classification = FUNC_MGT_SCALAR_FUNC,
.translateFunc = translateInOutNum,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = signFunction,
.finalizeFunc = NULL
},
{
.name = "degrees",
.type = FUNCTION_TYPE_DEGREES,
.classification = FUNC_MGT_SCALAR_FUNC,
.translateFunc = translateInNumOutDou,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = degreesFunction,
.finalizeFunc = NULL
},
{
.name = "radians",
.type = FUNCTION_TYPE_RADIANS,
.classification = FUNC_MGT_SCALAR_FUNC,
.translateFunc = translateInNumOutDou,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = radiansFunction,
.finalizeFunc = NULL
},
{
.name = "truncate",
.type = FUNCTION_TYPE_TRUNCATE,
.classification = FUNC_MGT_SCALAR_FUNC,
.translateFunc = translateTrunc,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = truncFunction,
.finalizeFunc = NULL
},
{
.name = "trunc",
.type = FUNCTION_TYPE_TRUNCATE,
.classification = FUNC_MGT_SCALAR_FUNC,
.translateFunc = translateTrunc,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = truncFunction,
.finalizeFunc = NULL
},
{
.name = "substring",
.type = FUNCTION_TYPE_SUBSTR,
.classification = FUNC_MGT_SCALAR_FUNC | FUNC_MGT_STRING_FUNC,
.translateFunc = translateSubstr,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = substrFunction,
.finalizeFunc = NULL
},
{
.name = "substring_index",
.type = FUNCTION_TYPE_SUBSTR_IDX,
.classification = FUNC_MGT_SCALAR_FUNC | FUNC_MGT_STRING_FUNC,
.translateFunc = translateSubstrIdx,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = substrIdxFunction,
.finalizeFunc = NULL
},
{
.name = "char",
.type = FUNCTION_TYPE_CHAR,
.classification = FUNC_MGT_SCALAR_FUNC | FUNC_MGT_STRING_FUNC,
.translateFunc = translateChar,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = charFunction,
.finalizeFunc = NULL
},
{
.name = "ascii",
.type = FUNCTION_TYPE_ASCII,
.classification = FUNC_MGT_SCALAR_FUNC | FUNC_MGT_STRING_FUNC,
.translateFunc = translateAscii,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = asciiFunction,
.finalizeFunc = NULL
},
{
.name = "position",
.type = FUNCTION_TYPE_POSITION,
.classification = FUNC_MGT_SCALAR_FUNC | FUNC_MGT_STRING_FUNC,
.translateFunc = translatePosition,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = positionFunction,
.finalizeFunc = NULL
},
{
.name = "trim",
.type = FUNCTION_TYPE_TRIM,
.classification = FUNC_MGT_SCALAR_FUNC | FUNC_MGT_STRING_FUNC,
.translateFunc = translateTrim,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = trimFunction,
.finalizeFunc = NULL
},
{
.name = "replace",
.type = FUNCTION_TYPE_REPLACE,
.classification = FUNC_MGT_SCALAR_FUNC | FUNC_MGT_STRING_FUNC,
.translateFunc = translateReplace,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = replaceFunction,
.finalizeFunc = NULL
},
{
.name = "repeat",
.type = FUNCTION_TYPE_REPEAT,
.classification = FUNC_MGT_SCALAR_FUNC | FUNC_MGT_STRING_FUNC,
.translateFunc = translateRepeat,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = repeatFunction,
.finalizeFunc = NULL
},
{
.name = "weekday",
.type = FUNCTION_TYPE_WEEKDAY,
.classification = FUNC_MGT_SCALAR_FUNC,
.translateFunc = translateWeekday,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = weekdayFunction,
.finalizeFunc = NULL
},
{
.name = "dayofweek",
.type = FUNCTION_TYPE_DAYOFWEEK,
.classification = FUNC_MGT_SCALAR_FUNC,
.translateFunc = translateWeekday,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = dayofweekFunction,
.finalizeFunc = NULL
},
{
.name = "week",
.type = FUNCTION_TYPE_WEEK,
.classification = FUNC_MGT_SCALAR_FUNC,
.translateFunc = translateWeek,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = weekFunction,
.finalizeFunc = NULL
},
{
.name = "weekofyear",
.type = FUNCTION_TYPE_WEEKOFYEAR,
.classification = FUNC_MGT_SCALAR_FUNC,
.translateFunc = translateWeekofyear,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = weekofyearFunction,
.finalizeFunc = NULL
},
}; };
// clang-format on // clang-format on

162
source/libs/function/src/builtinsimpl.c
View File

@ -63,7 +63,7 @@ typedef struct STopBotRes {
STopBotResItem* pItems; STopBotResItem* pItems;
} STopBotRes; } STopBotRes;
typedef struct SStddevRes { typedef struct SStdRes {
double result; double result;
int64_t count; int64_t count;
union { union {
@ -77,7 +77,7 @@ typedef struct SStddevRes {
uint64_t usum; uint64_t usum;
}; };
int16_t type; int16_t type;
} SStddevRes; } SStdRes;
typedef struct SLeastSQRInfo { typedef struct SLeastSQRInfo {
double matrix[2][3]; double matrix[2][3];
@ -1300,14 +1300,14 @@ int32_t maxCombine(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx) {
return minMaxCombine(pDestCtx, pSourceCtx, 0); return minMaxCombine(pDestCtx, pSourceCtx, 0);
} }
int32_t getStddevInfoSize() { return (int32_t)sizeof(SStddevRes); } int32_t getStdInfoSize() { return (int32_t)sizeof(SStdRes); }
bool getStddevFuncEnv(SFunctionNode* pFunc, SFuncExecEnv* pEnv) { bool getStdFuncEnv(SFunctionNode* pFunc, SFuncExecEnv* pEnv) {
pEnv->calcMemSize = sizeof(SStddevRes); pEnv->calcMemSize = sizeof(SStdRes);
return true; return true;
} }
int32_t stddevFunctionSetup(SqlFunctionCtx* pCtx, SResultRowEntryInfo* pResultInfo) { int32_t stdFunctionSetup(SqlFunctionCtx* pCtx, SResultRowEntryInfo* pResultInfo) {
if (pResultInfo->initialized) { if (pResultInfo->initialized) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
@ -1315,20 +1315,20 @@ int32_t stddevFunctionSetup(SqlFunctionCtx* pCtx, SResultRowEntryInfo* pResultIn
return TSDB_CODE_FUNC_SETUP_ERROR; return TSDB_CODE_FUNC_SETUP_ERROR;
} }
SStddevRes* pRes = GET_ROWCELL_INTERBUF(pResultInfo); SStdRes* pRes = GET_ROWCELL_INTERBUF(pResultInfo);
(void)memset(pRes, 0, sizeof(SStddevRes)); (void)memset(pRes, 0, sizeof(SStdRes));
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
int32_t stddevFunction(SqlFunctionCtx* pCtx) { int32_t stdFunction(SqlFunctionCtx* pCtx) {
int32_t numOfElem = 0; int32_t numOfElem = 0;
// Only the pre-computing information loaded and actual data does not loaded // Only the pre-computing information loaded and actual data does not loaded
SInputColumnInfoData* pInput = &pCtx->input; SInputColumnInfoData* pInput = &pCtx->input;
int32_t type = pInput->pData[0]->info.type; int32_t type = pInput->pData[0]->info.type;
SStddevRes* pStddevRes = GET_ROWCELL_INTERBUF(GET_RES_INFO(pCtx)); SStdRes* pStdRes = GET_ROWCELL_INTERBUF(GET_RES_INFO(pCtx));
pStddevRes->type = type; pStdRes->type = type;
// computing based on the true data block // computing based on the true data block
SColumnInfoData* pCol = pInput->pData[0]; SColumnInfoData* pCol = pInput->pData[0];
@ -1350,9 +1350,9 @@ int32_t stddevFunction(SqlFunctionCtx* pCtx) {
} }
numOfElem += 1; numOfElem += 1;
pStddevRes->count += 1; pStdRes->count += 1;
pStddevRes->isum += plist[i]; pStdRes->isum += plist[i];
pStddevRes->quadraticISum += plist[i] * plist[i]; pStdRes->quadraticISum += plist[i] * plist[i];
} }
break; break;
@ -1366,9 +1366,9 @@ int32_t stddevFunction(SqlFunctionCtx* pCtx) {
} }
numOfElem += 1; numOfElem += 1;
pStddevRes->count += 1; pStdRes->count += 1;
pStddevRes->isum += plist[i]; pStdRes->isum += plist[i];
pStddevRes->quadraticISum += plist[i] * plist[i]; pStdRes->quadraticISum += plist[i] * plist[i];
} }
break; break;
} }
@ -1381,9 +1381,9 @@ int32_t stddevFunction(SqlFunctionCtx* pCtx) {
} }
numOfElem += 1; numOfElem += 1;
pStddevRes->count += 1; pStdRes->count += 1;
pStddevRes->isum += plist[i]; pStdRes->isum += plist[i];
pStddevRes->quadraticISum += plist[i] * plist[i]; pStdRes->quadraticISum += plist[i] * plist[i];
} }
break; break;
@ -1397,9 +1397,9 @@ int32_t stddevFunction(SqlFunctionCtx* pCtx) {
} }
numOfElem += 1; numOfElem += 1;
pStddevRes->count += 1; pStdRes->count += 1;
pStddevRes->isum += plist[i]; pStdRes->isum += plist[i];
pStddevRes->quadraticISum += plist[i] * plist[i]; pStdRes->quadraticISum += plist[i] * plist[i];
} }
break; break;
} }
@ -1412,9 +1412,9 @@ int32_t stddevFunction(SqlFunctionCtx* pCtx) {
} }
numOfElem += 1; numOfElem += 1;
pStddevRes->count += 1; pStdRes->count += 1;
pStddevRes->usum += plist[i]; pStdRes->usum += plist[i];
pStddevRes->quadraticUSum += plist[i] * plist[i]; pStdRes->quadraticUSum += plist[i] * plist[i];
} }
break; break;
@ -1428,9 +1428,9 @@ int32_t stddevFunction(SqlFunctionCtx* pCtx) {
} }
numOfElem += 1; numOfElem += 1;
pStddevRes->count += 1; pStdRes->count += 1;
pStddevRes->usum += plist[i]; pStdRes->usum += plist[i];
pStddevRes->quadraticUSum += plist[i] * plist[i]; pStdRes->quadraticUSum += plist[i] * plist[i];
} }
break; break;
} }
@ -1443,9 +1443,9 @@ int32_t stddevFunction(SqlFunctionCtx* pCtx) {
} }
numOfElem += 1; numOfElem += 1;
pStddevRes->count += 1; pStdRes->count += 1;
pStddevRes->usum += plist[i]; pStdRes->usum += plist[i];
pStddevRes->quadraticUSum += plist[i] * plist[i]; pStdRes->quadraticUSum += plist[i] * plist[i];
} }
break; break;
@ -1459,9 +1459,9 @@ int32_t stddevFunction(SqlFunctionCtx* pCtx) {
} }
numOfElem += 1; numOfElem += 1;
pStddevRes->count += 1; pStdRes->count += 1;
pStddevRes->usum += plist[i]; pStdRes->usum += plist[i];
pStddevRes->quadraticUSum += plist[i] * plist[i]; pStdRes->quadraticUSum += plist[i] * plist[i];
} }
break; break;
} }
@ -1474,9 +1474,9 @@ int32_t stddevFunction(SqlFunctionCtx* pCtx) {
} }
numOfElem += 1; numOfElem += 1;
pStddevRes->count += 1; pStdRes->count += 1;
pStddevRes->dsum += plist[i]; pStdRes->dsum += plist[i];
pStddevRes->quadraticDSum += plist[i] * plist[i]; pStdRes->quadraticDSum += plist[i] * plist[i];
} }
break; break;
} }
@ -1489,9 +1489,9 @@ int32_t stddevFunction(SqlFunctionCtx* pCtx) {
} }
numOfElem += 1; numOfElem += 1;
pStddevRes->count += 1; pStdRes->count += 1;
pStddevRes->dsum += plist[i]; pStdRes->dsum += plist[i];
pStddevRes->quadraticDSum += plist[i] * plist[i]; pStdRes->quadraticDSum += plist[i] * plist[i];
} }
break; break;
} }
@ -1506,7 +1506,7 @@ _stddev_over:
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static void stddevTransferInfo(SStddevRes* pInput, SStddevRes* pOutput) { static void stdTransferInfo(SStdRes* pInput, SStdRes* pOutput) {
if (IS_NULL_TYPE(pInput->type)) { if (IS_NULL_TYPE(pInput->type)) {
return; return;
} }
@ -1525,7 +1525,7 @@ static void stddevTransferInfo(SStddevRes* pInput, SStddevRes* pOutput) {
pOutput->count += pInput->count; pOutput->count += pInput->count;
} }
int32_t stddevFunctionMerge(SqlFunctionCtx* pCtx) { int32_t stdFunctionMerge(SqlFunctionCtx* pCtx) {
SInputColumnInfoData* pInput = &pCtx->input; SInputColumnInfoData* pInput = &pCtx->input;
SColumnInfoData* pCol = pInput->pData[0]; SColumnInfoData* pCol = pInput->pData[0];
@ -1538,13 +1538,13 @@ int32_t stddevFunctionMerge(SqlFunctionCtx* pCtx) {
return TSDB_CODE_FUNC_FUNTION_PARA_TYPE; return TSDB_CODE_FUNC_FUNTION_PARA_TYPE;
} }
SStddevRes* pInfo = GET_ROWCELL_INTERBUF(GET_RES_INFO(pCtx)); SStdRes* pInfo = GET_ROWCELL_INTERBUF(GET_RES_INFO(pCtx));
for (int32_t i = pInput->startRowIndex; i < pInput->startRowIndex + pInput->numOfRows; ++i) { for (int32_t i = pInput->startRowIndex; i < pInput->startRowIndex + pInput->numOfRows; ++i) {
if (colDataIsNull_s(pCol, i)) continue; if (colDataIsNull_s(pCol, i)) continue;
char* data = colDataGetData(pCol, i); char* data = colDataGetData(pCol, i);
SStddevRes* pInputInfo = (SStddevRes*)varDataVal(data); SStdRes* pInputInfo = (SStdRes*)varDataVal(data);
stddevTransferInfo(pInputInfo, pInfo); stdTransferInfo(pInputInfo, pInfo);
} }
SET_VAL(GET_RES_INFO(pCtx), 1, 1); SET_VAL(GET_RES_INFO(pCtx), 1, 1);
@ -1552,14 +1552,14 @@ int32_t stddevFunctionMerge(SqlFunctionCtx* pCtx) {
} }
#ifdef BUILD_NO_CALL #ifdef BUILD_NO_CALL
int32_t stddevInvertFunction(SqlFunctionCtx* pCtx) { int32_t stdInvertFunction(SqlFunctionCtx* pCtx) {
int32_t numOfElem = 0; int32_t numOfElem = 0;
// Only the pre-computing information loaded and actual data does not loaded // Only the pre-computing information loaded and actual data does not loaded
SInputColumnInfoData* pInput = &pCtx->input; SInputColumnInfoData* pInput = &pCtx->input;
int32_t type = pInput->pData[0]->info.type; int32_t type = pInput->pData[0]->info.type;
SStddevRes* pStddevRes = GET_ROWCELL_INTERBUF(GET_RES_INFO(pCtx)); SStdRes* pStdRes = GET_ROWCELL_INTERBUF(GET_RES_INFO(pCtx));
// computing based on the true data block // computing based on the true data block
SColumnInfoData* pCol = pInput->pData[0]; SColumnInfoData* pCol = pInput->pData[0];
@ -1569,43 +1569,43 @@ int32_t stddevInvertFunction(SqlFunctionCtx* pCtx) {
switch (type) { switch (type) {
case TSDB_DATA_TYPE_TINYINT: { case TSDB_DATA_TYPE_TINYINT: {
LIST_STDDEV_SUB_N(pStddevRes->isum, int8_t); LIST_STDDEV_SUB_N(pStdRes->isum, int8_t);
break; break;
} }
case TSDB_DATA_TYPE_SMALLINT: { case TSDB_DATA_TYPE_SMALLINT: {
LIST_STDDEV_SUB_N(pStddevRes->isum, int16_t); LIST_STDDEV_SUB_N(pStdRes->isum, int16_t);
break; break;
} }
case TSDB_DATA_TYPE_INT: { case TSDB_DATA_TYPE_INT: {
LIST_STDDEV_SUB_N(pStddevRes->isum, int32_t); LIST_STDDEV_SUB_N(pStdRes->isum, int32_t);
break; break;
} }
case TSDB_DATA_TYPE_BIGINT: { case TSDB_DATA_TYPE_BIGINT: {
LIST_STDDEV_SUB_N(pStddevRes->isum, int64_t); LIST_STDDEV_SUB_N(pStdRes->isum, int64_t);
break; break;
} }
case TSDB_DATA_TYPE_UTINYINT: { case TSDB_DATA_TYPE_UTINYINT: {
LIST_STDDEV_SUB_N(pStddevRes->isum, uint8_t); LIST_STDDEV_SUB_N(pStdRes->isum, uint8_t);
break; break;
} }
case TSDB_DATA_TYPE_USMALLINT: { case TSDB_DATA_TYPE_USMALLINT: {
LIST_STDDEV_SUB_N(pStddevRes->isum, uint16_t); LIST_STDDEV_SUB_N(pStdRes->isum, uint16_t);
break; break;
} }
case TSDB_DATA_TYPE_UINT: { case TSDB_DATA_TYPE_UINT: {
LIST_STDDEV_SUB_N(pStddevRes->isum, uint32_t); LIST_STDDEV_SUB_N(pStdRes->isum, uint32_t);
break; break;
} }
case TSDB_DATA_TYPE_UBIGINT: { case TSDB_DATA_TYPE_UBIGINT: {
LIST_STDDEV_SUB_N(pStddevRes->isum, uint64_t); LIST_STDDEV_SUB_N(pStdRes->isum, uint64_t);
break; break;
} }
case TSDB_DATA_TYPE_FLOAT: { case TSDB_DATA_TYPE_FLOAT: {
LIST_STDDEV_SUB_N(pStddevRes->dsum, float); LIST_STDDEV_SUB_N(pStdRes->dsum, float);
break; break;
} }
case TSDB_DATA_TYPE_DOUBLE: { case TSDB_DATA_TYPE_DOUBLE: {
LIST_STDDEV_SUB_N(pStddevRes->dsum, double); LIST_STDDEV_SUB_N(pStdRes->dsum, double);
break; break;
} }
default: default:
@ -1620,7 +1620,7 @@ int32_t stddevInvertFunction(SqlFunctionCtx* pCtx) {
int32_t stddevFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) { int32_t stddevFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) {
SInputColumnInfoData* pInput = &pCtx->input; SInputColumnInfoData* pInput = &pCtx->input;
SStddevRes* pStddevRes = GET_ROWCELL_INTERBUF(GET_RES_INFO(pCtx)); SStdRes* pStddevRes = GET_ROWCELL_INTERBUF(GET_RES_INFO(pCtx));
int32_t type = pStddevRes->type; int32_t type = pStddevRes->type;
double avg; double avg;
@ -1648,10 +1648,40 @@ int32_t stddevFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) {
return functionFinalize(pCtx, pBlock); return functionFinalize(pCtx, pBlock);
} }
int32_t stddevPartialFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) { int32_t stdvarFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) {
SInputColumnInfoData* pInput = &pCtx->input;
SStdRes* pStdvarRes = GET_ROWCELL_INTERBUF(GET_RES_INFO(pCtx));
int32_t type = pStdvarRes->type;
double avg;
if (pStdvarRes->count == 0) {
GET_RES_INFO(pCtx)->numOfRes = 0;
return functionFinalize(pCtx, pBlock);
}
if (IS_SIGNED_NUMERIC_TYPE(type)) {
avg = pStdvarRes->isum / ((double)pStdvarRes->count);
pStdvarRes->result = fabs(pStdvarRes->quadraticISum / ((double)pStdvarRes->count) - avg * avg);
} else if (IS_UNSIGNED_NUMERIC_TYPE(type)) {
avg = pStdvarRes->usum / ((double)pStdvarRes->count);
pStdvarRes->result = fabs(pStdvarRes->quadraticUSum / ((double)pStdvarRes->count) - avg * avg);
} else {
avg = pStdvarRes->dsum / ((double)pStdvarRes->count);
pStdvarRes->result = fabs(pStdvarRes->quadraticDSum / ((double)pStdvarRes->count) - avg * avg);
}
// check for overflow
if (isinf(pStdvarRes->result) || isnan(pStdvarRes->result)) {
GET_RES_INFO(pCtx)->numOfRes = 0;
}
return functionFinalize(pCtx, pBlock);
}
int32_t stdPartialFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) {
SResultRowEntryInfo* pResInfo = GET_RES_INFO(pCtx); SResultRowEntryInfo* pResInfo = GET_RES_INFO(pCtx);
SStddevRes* pInfo = GET_ROWCELL_INTERBUF(GET_RES_INFO(pCtx)); SStdRes* pInfo = GET_ROWCELL_INTERBUF(GET_RES_INFO(pCtx));
int32_t resultBytes = getStddevInfoSize(); int32_t resultBytes = getStdInfoSize();
char* res = taosMemoryCalloc(resultBytes + VARSTR_HEADER_SIZE, sizeof(char)); char* res = taosMemoryCalloc(resultBytes + VARSTR_HEADER_SIZE, sizeof(char));
if (NULL == res) { if (NULL == res) {
@ -1673,15 +1703,15 @@ int32_t stddevPartialFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) {
return code; return code;
} }
int32_t stddevCombine(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx) { int32_t stdCombine(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx) {
SResultRowEntryInfo* pDResInfo = GET_RES_INFO(pDestCtx); SResultRowEntryInfo* pDResInfo = GET_RES_INFO(pDestCtx);
SStddevRes* pDBuf = GET_ROWCELL_INTERBUF(pDResInfo); SStdRes* pDBuf = GET_ROWCELL_INTERBUF(pDResInfo);
SResultRowEntryInfo* pSResInfo = GET_RES_INFO(pSourceCtx); SResultRowEntryInfo* pSResInfo = GET_RES_INFO(pSourceCtx);
SStddevRes* pSBuf = GET_ROWCELL_INTERBUF(pSResInfo); SStdRes* pSBuf = GET_ROWCELL_INTERBUF(pSResInfo);
int16_t type = pDBuf->type == TSDB_DATA_TYPE_NULL ? pSBuf->type : pDBuf->type; int16_t type = pDBuf->type == TSDB_DATA_TYPE_NULL ? pSBuf->type : pDBuf->type;
stddevTransferInfo(pSBuf, pDBuf); stdTransferInfo(pSBuf, pDBuf);
pDResInfo->numOfRes = TMAX(pDResInfo->numOfRes, pSResInfo->numOfRes); pDResInfo->numOfRes = TMAX(pDResInfo->numOfRes, pSResInfo->numOfRes);
pDResInfo->isNullRes &= pSResInfo->isNullRes; pDResInfo->isNullRes &= pSResInfo->isNullRes;

8
source/libs/nodes/src/nodesCodeFuncs.c
View File

@ -4254,6 +4254,7 @@ static const char* jkFunctionHasPk = "HasPk";
static const char* jkFunctionPkBytes = "PkBytes"; static const char* jkFunctionPkBytes = "PkBytes";
static const char* jkFunctionIsMergeFunc = "IsMergeFunc"; static const char* jkFunctionIsMergeFunc = "IsMergeFunc";
static const char* jkFunctionMergeFuncOf = "MergeFuncOf"; static const char* jkFunctionMergeFuncOf = "MergeFuncOf";
static const char* jkFunctionTrimType = "TrimType";
static int32_t functionNodeToJson(const void* pObj, SJson* pJson) { static int32_t functionNodeToJson(const void* pObj, SJson* pJson) {
const SFunctionNode* pNode = (const SFunctionNode*)pObj; const SFunctionNode* pNode = (const SFunctionNode*)pObj;
@ -4286,7 +4287,9 @@ static int32_t functionNodeToJson(const void* pObj, SJson* pJson) {
if (TSDB_CODE_SUCCESS == code) { if (TSDB_CODE_SUCCESS == code) {
code = tjsonAddIntegerToObject(pJson, jkFunctionMergeFuncOf, pNode->originalFuncId); code = tjsonAddIntegerToObject(pJson, jkFunctionMergeFuncOf, pNode->originalFuncId);
} }
if (TSDB_CODE_SUCCESS == code) {
code = tjsonAddIntegerToObject(pJson, jkFunctionTrimType, pNode->trimType);
}
return code; return code;
} }
@ -4321,6 +4324,9 @@ static int32_t jsonToFunctionNode(const SJson* pJson, void* pObj) {
if (TSDB_CODE_SUCCESS == code) { if (TSDB_CODE_SUCCESS == code) {
code = tjsonGetIntValue(pJson, jkFunctionMergeFuncOf, &pNode->originalFuncId); code = tjsonGetIntValue(pJson, jkFunctionMergeFuncOf, &pNode->originalFuncId);
} }
if (TSDB_CODE_SUCCESS == code) {
tjsonGetNumberValue(pJson, jkFunctionTrimType, pNode->trimType, code);
}
return code; return code;
} }

7
source/libs/nodes/src/nodesMsgFuncs.c
View File

@ -1112,6 +1112,7 @@ enum {
FUNCTION_NODE_PK_BYTES, FUNCTION_NODE_PK_BYTES,
FUNCTION_CODE_IS_MERGE_FUNC, FUNCTION_CODE_IS_MERGE_FUNC,
FUNCTION_CODE_MERGE_FUNC_OF, FUNCTION_CODE_MERGE_FUNC_OF,
FUNCTION_CODE_TRIM_TYPE,
}; };
static int32_t functionNodeToMsg(const void* pObj, STlvEncoder* pEncoder) { static int32_t functionNodeToMsg(const void* pObj, STlvEncoder* pEncoder) {
@ -1145,6 +1146,9 @@ static int32_t functionNodeToMsg(const void* pObj, STlvEncoder* pEncoder) {
if (TSDB_CODE_SUCCESS == code) { if (TSDB_CODE_SUCCESS == code) {
code = tlvEncodeI32(pEncoder, FUNCTION_CODE_MERGE_FUNC_OF, pNode->originalFuncId); code = tlvEncodeI32(pEncoder, FUNCTION_CODE_MERGE_FUNC_OF, pNode->originalFuncId);
} }
if (TSDB_CODE_SUCCESS == code) {
code = tlvEncodeEnum(pEncoder, FUNCTION_CODE_TRIM_TYPE, pNode->trimType);
}
return code; return code;
} }
@ -1186,6 +1190,9 @@ static int32_t msgToFunctionNode(STlvDecoder* pDecoder, void* pObj) {
case FUNCTION_CODE_MERGE_FUNC_OF: case FUNCTION_CODE_MERGE_FUNC_OF:
code = tlvDecodeI32(pTlv, &pNode->originalFuncId); code = tlvDecodeI32(pTlv, &pNode->originalFuncId);
break; break;
case FUNCTION_CODE_TRIM_TYPE:
code = tlvDecodeEnum(pTlv, &pNode->trimType, sizeof(pNode->trimType));
break;
default: default:
break; break;
} }

5
source/libs/parser/inc/parAst.h
View File

@ -136,6 +136,11 @@ SNode* createBetweenAnd(SAstCreateContext* pCxt, SNode* pExpr, SNode* pLeft,
SNode* createNotBetweenAnd(SAstCreateContext* pCxt, SNode* pExpr, SNode* pLeft, SNode* pRight); SNode* createNotBetweenAnd(SAstCreateContext* pCxt, SNode* pExpr, SNode* pLeft, SNode* pRight);
SNode* createFunctionNode(SAstCreateContext* pCxt, const SToken* pFuncName, SNodeList* pParameterList); SNode* createFunctionNode(SAstCreateContext* pCxt, const SToken* pFuncName, SNodeList* pParameterList);
SNode* createCastFunctionNode(SAstCreateContext* pCxt, SNode* pExpr, SDataType dt); SNode* createCastFunctionNode(SAstCreateContext* pCxt, SNode* pExpr, SDataType dt);
SNode* createPositionFunctionNode(SAstCreateContext* pCxt, SNode* pExpr, SNode* pExpr2);
SNode* createTrimFunctionNode(SAstCreateContext* pCxt, SNode* pExpr, ETrimType type);
SNode* createTrimFunctionNodeExt(SAstCreateContext* pCxt, SNode* pExpr, SNode* pExpr2, ETrimType type);
SNode* createSubstrFunctionNode(SAstCreateContext* pCxt, SNode* pExpr, SNode* pExpr2);
SNode* createSubstrFunctionNodeExt(SAstCreateContext* pCxt, SNode* pExpr, SNode* pExpr2, SNode* pExpr3);
SNode* createNodeListNode(SAstCreateContext* pCxt, SNodeList* pList); SNode* createNodeListNode(SAstCreateContext* pCxt, SNodeList* pList);
SNode* createNodeListNodeEx(SAstCreateContext* pCxt, SNode* p1, SNode* p2); SNode* createNodeListNodeEx(SAstCreateContext* pCxt, SNode* p1, SNode* p2);
SNode* createRealTableNode(SAstCreateContext* pCxt, SToken* pDbName, SToken* pTableName, SToken* pTableAlias); SNode* createRealTableNode(SAstCreateContext* pCxt, SToken* pDbName, SToken* pTableName, SToken* pTableAlias);

39
source/libs/parser/inc/sql.y
View File

@ -1183,19 +1183,45 @@ pseudo_column(A) ::= IROWTS(B).
pseudo_column(A) ::= ISFILLED(B). { A = createRawExprNode(pCxt, &B, createFunctionNode(pCxt, &B, NULL)); } pseudo_column(A) ::= ISFILLED(B). { A = createRawExprNode(pCxt, &B, createFunctionNode(pCxt, &B, NULL)); }
pseudo_column(A) ::= QTAGS(B). { A = createRawExprNode(pCxt, &B, createFunctionNode(pCxt, &B, NULL)); } pseudo_column(A) ::= QTAGS(B). { A = createRawExprNode(pCxt, &B, createFunctionNode(pCxt, &B, NULL)); }
function_expression(A) ::= function_name(B) NK_LP expression_list(C) NK_RP(D). { A = createRawExprNodeExt(pCxt, &B, &D, createFunctionNode(pCxt, &B, C)); } function_expression(A) ::= function_name(B) NK_LP expression_list(C) NK_RP(D). { A = createRawExprNodeExt(pCxt, &B, &D, createFunctionNode(pCxt, &B, C)); }
function_expression(A) ::= star_func(B) NK_LP star_func_para_list(C) NK_RP(D). { A = createRawExprNodeExt(pCxt, &B, &D, createFunctionNode(pCxt, &B, C)); } function_expression(A) ::= star_func(B) NK_LP star_func_para_list(C) NK_RP(D). { A = createRawExprNodeExt(pCxt, &B, &D, createFunctionNode(pCxt, &B, C)); }
function_expression(A) ::= function_expression(A) ::=
CAST(B) NK_LP expr_or_subquery(C) AS type_name(D) NK_RP(E). { A = createRawExprNodeExt(pCxt, &B, &E, createCastFunctionNode(pCxt, releaseRawExprNode(pCxt, C), D)); } CAST(B) NK_LP expr_or_subquery(C) AS type_name(D) NK_RP(E). { A = createRawExprNodeExt(pCxt, &B, &E, createCastFunctionNode(pCxt, releaseRawExprNode(pCxt, C), D)); }
function_expression(A) ::= function_expression(A) ::=
CAST(B) NK_LP expr_or_subquery(C) AS type_name_default_len(D) NK_RP(E). { A = createRawExprNodeExt(pCxt, &B, &E, createCastFunctionNode(pCxt, releaseRawExprNode(pCxt, C), D)); } CAST(B) NK_LP expr_or_subquery(C) AS type_name_default_len(D) NK_RP(E). { A = createRawExprNodeExt(pCxt, &B, &E, createCastFunctionNode(pCxt, releaseRawExprNode(pCxt, C), D)); }
function_expression(A) ::=
function_expression(A) ::= literal_func(B). { A = B; } POSITION(B) NK_LP expr_or_subquery(C) IN expr_or_subquery(D) NK_RP(E). { A = createRawExprNodeExt(pCxt, &B, &E, createPositionFunctionNode(pCxt, releaseRawExprNode(pCxt, C), releaseRawExprNode(pCxt, D))); }
function_expression(A) ::=
TRIM(B) NK_LP expr_or_subquery(C) NK_RP(D). { A = createRawExprNodeExt(pCxt, &B, &D, createTrimFunctionNode(pCxt, releaseRawExprNode(pCxt, C), TRIM_TYPE_BOTH)); }
function_expression(A) ::=
TRIM(B) NK_LP trim_specification_type(C) FROM expr_or_subquery(D) NK_RP(E). { A = createRawExprNodeExt(pCxt, &B, &E, createTrimFunctionNode(pCxt, releaseRawExprNode(pCxt, D), C)); }
function_expression(A) ::=
TRIM(B) NK_LP trim_specification_type(C) expr_or_subquery(D) FROM expr_or_subquery(E) NK_RP(F). { A = createRawExprNodeExt(pCxt, &B, &F, createTrimFunctionNodeExt(pCxt, releaseRawExprNode(pCxt, D), releaseRawExprNode(pCxt, E), C)); }
function_expression(A) ::=
substr_func(B) NK_LP expression_list(C) NK_RP(D). { A = createRawExprNodeExt(pCxt, &B, &D, createFunctionNode(pCxt, &B, C)); }
function_expression(A) ::=
substr_func(B) NK_LP expr_or_subquery(C) FROM expr_or_subquery(D) NK_RP(E). { A = createRawExprNodeExt(pCxt, &B, &E, createSubstrFunctionNode(pCxt, releaseRawExprNode(pCxt, C), releaseRawExprNode(pCxt, D))); }
function_expression(A) ::=
substr_func(B) NK_LP expr_or_subquery(C) FROM expr_or_subquery(D) FOR expr_or_subquery(E) NK_RP(F). { A = createRawExprNodeExt(pCxt, &B, &F, createSubstrFunctionNodeExt(pCxt, releaseRawExprNode(pCxt, C), releaseRawExprNode(pCxt, D), releaseRawExprNode(pCxt, E))); }
function_expression(A) ::= REPLACE(B) NK_LP expression_list(C) NK_RP(D). { A = createRawExprNodeExt(pCxt, &B, &D, createFunctionNode(pCxt, &B, C)); }
function_expression(A) ::= literal_func(B). { A = B; }
literal_func(A) ::= noarg_func(B) NK_LP NK_RP(C). { A = createRawExprNodeExt(pCxt, &B, &C, createFunctionNode(pCxt, &B, NULL)); } literal_func(A) ::= noarg_func(B) NK_LP NK_RP(C). { A = createRawExprNodeExt(pCxt, &B, &C, createFunctionNode(pCxt, &B, NULL)); }
literal_func(A) ::= NOW(B). { A = createRawExprNode(pCxt, &B, createFunctionNode(pCxt, &B, NULL)); } literal_func(A) ::= NOW(B). { A = createRawExprNode(pCxt, &B, createFunctionNode(pCxt, &B, NULL)); }
literal_func(A) ::= TODAY(B). { A = createRawExprNode(pCxt, &B, createFunctionNode(pCxt, &B, NULL)); } literal_func(A) ::= TODAY(B). { A = createRawExprNode(pCxt, &B, createFunctionNode(pCxt, &B, NULL)); }
%type substr_func { SToken }
%destructor substr_func { }
substr_func(A) ::= SUBSTR(B). { A = B; }
substr_func(A) ::= SUBSTRING(B). { A = B; }
%type trim_specification_type ETrimType
%destructor trim_specification_type { }
trim_specification_type(A) ::= . { A = TRIM_TYPE_BOTH; }
trim_specification_type(A) ::= BOTH. { A = TRIM_TYPE_BOTH; }
trim_specification_type(A) ::= TRAILING. { A = TRIM_TYPE_TRAILING; }
trim_specification_type(A) ::= LEADING. { A = TRIM_TYPE_LEADING; }
%type noarg_func { SToken } %type noarg_func { SToken }
%destructor noarg_func { } %destructor noarg_func { }
noarg_func(A) ::= NOW(B). { A = B; } noarg_func(A) ::= NOW(B). { A = B; }
@ -1207,6 +1233,7 @@ noarg_func(A) ::= SERVER_VERSION(B).
noarg_func(A) ::= SERVER_STATUS(B). { A = B; } noarg_func(A) ::= SERVER_STATUS(B). { A = B; }
noarg_func(A) ::= CURRENT_USER(B). { A = B; } noarg_func(A) ::= CURRENT_USER(B). { A = B; }
noarg_func(A) ::= USER(B). { A = B; } noarg_func(A) ::= USER(B). { A = B; }
noarg_func(A) ::= PI(B). { A = B; }
%type star_func { SToken } %type star_func { SToken }
%destructor star_func { } %destructor star_func { }

67
source/libs/parser/src/parAstCreater.c
View File

@ -997,6 +997,73 @@ SNode* createCastFunctionNode(SAstCreateContext* pCxt, SNode* pExpr, SDataType d
return (SNode*)func; return (SNode*)func;
} }
SNode* createPositionFunctionNode(SAstCreateContext* pCxt, SNode* pExpr, SNode* pExpr2) {
CHECK_PARSER_STATUS(pCxt);
SFunctionNode* func = NULL;
pCxt->errCode = nodesMakeNode(QUERY_NODE_FUNCTION, (SNode**)&func);
CHECK_MAKE_NODE(func);
strcpy(func->functionName, "position");
pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr);
CHECK_PARSER_STATUS(pCxt);
pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr2);
CHECK_PARSER_STATUS(pCxt);
return (SNode*)func;
}
SNode* createTrimFunctionNode(SAstCreateContext* pCxt, SNode* pExpr, ETrimType type) {
CHECK_PARSER_STATUS(pCxt);
SFunctionNode* func = NULL;
pCxt->errCode = nodesMakeNode(QUERY_NODE_FUNCTION, (SNode**)&func);
CHECK_MAKE_NODE(func);
strcpy(func->functionName, "trim");
func->trimType = type;
pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr);
CHECK_PARSER_STATUS(pCxt);
return (SNode*)func;
}
SNode* createTrimFunctionNodeExt(SAstCreateContext* pCxt, SNode* pExpr, SNode* pExpr2, ETrimType type) {
CHECK_PARSER_STATUS(pCxt);
SFunctionNode* func = NULL;
pCxt->errCode = nodesMakeNode(QUERY_NODE_FUNCTION, (SNode**)&func);
CHECK_MAKE_NODE(func);
strcpy(func->functionName, "trim");
func->trimType = type;
pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr);
CHECK_PARSER_STATUS(pCxt);
pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr2);
CHECK_PARSER_STATUS(pCxt);
return (SNode*)func;
}
SNode* createSubstrFunctionNode(SAstCreateContext* pCxt, SNode* pExpr, SNode* pExpr2) {
CHECK_PARSER_STATUS(pCxt);
SFunctionNode* func = NULL;
pCxt->errCode = nodesMakeNode(QUERY_NODE_FUNCTION, (SNode**)&func);
CHECK_MAKE_NODE(func);
strcpy(func->functionName, "substr");
pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr);
CHECK_PARSER_STATUS(pCxt);
pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr2);
CHECK_PARSER_STATUS(pCxt);
return (SNode*)func;
}
SNode* createSubstrFunctionNodeExt(SAstCreateContext* pCxt, SNode* pExpr, SNode* pExpr2, SNode* pExpr3) {
CHECK_PARSER_STATUS(pCxt);
SFunctionNode* func = NULL;
pCxt->errCode = nodesMakeNode(QUERY_NODE_FUNCTION, (SNode**)&func);
CHECK_MAKE_NODE(func);
strcpy(func->functionName, "substr");
pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr);
CHECK_PARSER_STATUS(pCxt);
pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr2);
CHECK_PARSER_STATUS(pCxt);
pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr3);
CHECK_PARSER_STATUS(pCxt);
return (SNode*)func;
}
SNode* createNodeListNode(SAstCreateContext* pCxt, SNodeList* pList) { SNode* createNodeListNode(SAstCreateContext* pCxt, SNodeList* pList) {
CHECK_PARSER_STATUS(pCxt); CHECK_PARSER_STATUS(pCxt);
SNodeListNode* list = NULL; SNodeListNode* list = NULL;

8
source/libs/parser/src/parTokenizer.c
View File

@ -52,6 +52,7 @@ static SKeyword keywordTable[] = {
{"BNODE", TK_BNODE}, {"BNODE", TK_BNODE},
{"BNODES", TK_BNODES}, {"BNODES", TK_BNODES},
{"BOOL", TK_BOOL}, {"BOOL", TK_BOOL},
{"BOTH", TK_BOTH},
{"BUFFER", TK_BUFFER}, {"BUFFER", TK_BUFFER},
{"BUFSIZE", TK_BUFSIZE}, {"BUFSIZE", TK_BUFSIZE},
{"BY", TK_BY}, {"BY", TK_BY},
@ -111,6 +112,7 @@ static SKeyword keywordTable[] = {
{"FLOAT", TK_FLOAT}, {"FLOAT", TK_FLOAT},
{"FLUSH", TK_FLUSH}, {"FLUSH", TK_FLUSH},
{"FROM", TK_FROM}, {"FROM", TK_FROM},
{"FOR", TK_FOR},
{"FORCE", TK_FORCE}, {"FORCE", TK_FORCE},
{"FULL", TK_FULL}, {"FULL", TK_FULL},
{"FUNCTION", TK_FUNCTION}, {"FUNCTION", TK_FUNCTION},
@ -148,6 +150,7 @@ static SKeyword keywordTable[] = {
{"LAST", TK_LAST}, {"LAST", TK_LAST},
{"LAST_ROW", TK_LAST_ROW}, {"LAST_ROW", TK_LAST_ROW},
{"LEADER", TK_LEADER}, {"LEADER", TK_LEADER},
{"LEADING", TK_LEADING},
{"LEFT", TK_LEFT}, {"LEFT", TK_LEFT},
{"LICENCES", TK_LICENCES}, {"LICENCES", TK_LICENCES},
{"LIKE", TK_LIKE}, {"LIKE", TK_LIKE},
@ -191,6 +194,7 @@ static SKeyword keywordTable[] = {
{"PARTITION_FIRST", TK_PARTITION_FIRST}, {"PARTITION_FIRST", TK_PARTITION_FIRST},
{"PASS", TK_PASS}, {"PASS", TK_PASS},
{"PORT", TK_PORT}, {"PORT", TK_PORT},
{"POSITION", TK_POSITION},
{"PPS", TK_PPS}, {"PPS", TK_PPS},
{"PRIMARY", TK_PRIMARY}, {"PRIMARY", TK_PRIMARY},
{"PRECISION", TK_PRECISION}, {"PRECISION", TK_PRECISION},
@ -201,6 +205,7 @@ static SKeyword keywordTable[] = {
{"QTIME", TK_QTIME}, {"QTIME", TK_QTIME},
{"QUERIES", TK_QUERIES}, {"QUERIES", TK_QUERIES},
{"QUERY", TK_QUERY}, {"QUERY", TK_QUERY},
{"PI", TK_PI},
{"RANGE", TK_RANGE}, {"RANGE", TK_RANGE},
{"RATIO", TK_RATIO}, {"RATIO", TK_RATIO},
{"PAUSE", TK_PAUSE}, {"PAUSE", TK_PAUSE},
@ -250,6 +255,8 @@ static SKeyword keywordTable[] = {
{"STT_TRIGGER", TK_STT_TRIGGER}, {"STT_TRIGGER", TK_STT_TRIGGER},
{"SUBSCRIBE", TK_SUBSCRIBE}, {"SUBSCRIBE", TK_SUBSCRIBE},
{"SUBSCRIPTIONS", TK_SUBSCRIPTIONS}, {"SUBSCRIPTIONS", TK_SUBSCRIPTIONS},
{"SUBSTR", TK_SUBSTR},
{"SUBSTRING", TK_SUBSTRING},
{"SUBTABLE", TK_SUBTABLE}, {"SUBTABLE", TK_SUBTABLE},
{"SYSINFO", TK_SYSINFO}, {"SYSINFO", TK_SYSINFO},
{"SYSTEM", TK_SYSTEM}, {"SYSTEM", TK_SYSTEM},
@ -268,6 +275,7 @@ static SKeyword keywordTable[] = {
{"TODAY", TK_TODAY}, {"TODAY", TK_TODAY},
{"TOPIC", TK_TOPIC}, {"TOPIC", TK_TOPIC},
{"TOPICS", TK_TOPICS}, {"TOPICS", TK_TOPICS},
{"TRAILING", TK_TRAILING},
{"TRANSACTION", TK_TRANSACTION}, {"TRANSACTION", TK_TRANSACTION},
{"TRANSACTIONS", TK_TRANSACTIONS}, {"TRANSACTIONS", TK_TRANSACTIONS},
{"TRIGGER", TK_TRIGGER}, {"TRIGGER", TK_TRIGGER},

14159
source/libs/parser/src/sql.c
View File

File diff suppressed because it is too large Load Diff

4
source/libs/qworker/src/qworker.c
View File

@ -778,11 +778,13 @@ int32_t qwProcessQuery(QW_FPARAMS_DEF, SQWMsg *qwMsg, char *sql) {
sql = NULL; sql = NULL;
if (code) { if (code) {
QW_TASK_ELOG("qCreateExecTask failed, code:%x - %s", code, tstrerror(code)); QW_TASK_ELOG("qCreateExecTask failed, code:%x - %s", code, tstrerror(code));
qDestroyTask(pTaskInfo);
QW_ERR_JRET(code); QW_ERR_JRET(code);
} }
if (NULL == sinkHandle || NULL == pTaskInfo) { if (NULL == sinkHandle || NULL == pTaskInfo) {
QW_TASK_ELOG("create task result error, taskHandle:%p, sinkHandle:%p", pTaskInfo, sinkHandle); QW_TASK_ELOG("create task result error, taskHandle:%p, sinkHandle:%p", pTaskInfo, sinkHandle);
qDestroyTask(pTaskInfo);
QW_ERR_JRET(TSDB_CODE_APP_ERROR); QW_ERR_JRET(TSDB_CODE_APP_ERROR);
} }
@ -1277,11 +1279,13 @@ int32_t qwProcessDelete(QW_FPARAMS_DEF, SQWMsg *qwMsg, SDeleteRes *pRes) {
if (code) { if (code) {
QW_TASK_ELOG("qCreateExecTask failed, code:%x - %s", code, tstrerror(code)); QW_TASK_ELOG("qCreateExecTask failed, code:%x - %s", code, tstrerror(code));
qDestroyTask(pTaskInfo);
QW_ERR_JRET(code); QW_ERR_JRET(code);
} }
if (NULL == sinkHandle || NULL == pTaskInfo) { if (NULL == sinkHandle || NULL == pTaskInfo) {
QW_TASK_ELOG("create task result error, taskHandle:%p, sinkHandle:%p", pTaskInfo, sinkHandle); QW_TASK_ELOG("create task result error, taskHandle:%p, sinkHandle:%p", pTaskInfo, sinkHandle);
qDestroyTask(pTaskInfo);
QW_ERR_JRET(TSDB_CODE_APP_ERROR); QW_ERR_JRET(TSDB_CODE_APP_ERROR);
} }

46
source/libs/scalar/inc/sclInt.h
View File

@ -54,6 +54,52 @@ typedef struct SScalarCtx {
#define SCL_DOWNGRADE_DATETYPE(_type) \ #define SCL_DOWNGRADE_DATETYPE(_type) \
((_type) == TSDB_DATA_TYPE_BIGINT || TSDB_DATA_TYPE_DOUBLE == (_type) || (_type) == TSDB_DATA_TYPE_UBIGINT) ((_type) == TSDB_DATA_TYPE_BIGINT || TSDB_DATA_TYPE_DOUBLE == (_type) || (_type) == TSDB_DATA_TYPE_UBIGINT)
/** Flags for calculateWeekNum() function. */
#define WEEK_FLAG_MONDAY_FIRST 1 /* If set this, monday is first day of week, else, sunday is first day.*/
#define WEEK_FLAG_FROM_ONE 2 /* If set this, week start from 1, else, week start from 0. */
#define WEEK_FLAG_INCLUDE_FIRST_DAY 4 /* If set this, week contains 'first day of week' is week one,
* else, weeks are numbered according to ISO 8601:1988. */
#ifndef M_PI
#define M_PI 3.14159265358979323846264338327950288 /* pi */
#endif
#ifndef M_1_PI
#define M_1_PI 0.318309886183790671537767526745028724 /* 1/pi */
#endif
#define INT1TOCHAR(T, A) \
{ \
const unsigned long def_temp = (unsigned long)(A); \
((unsigned char *)(T))[0] = (unsigned char)(def_temp); \
T += 1; \
}
#define INT2TOCHAR(T, A) \
{ \
const unsigned long def_temp = (unsigned long)(A); \
((unsigned char *)(T))[1] = (unsigned char)(def_temp); \
((unsigned char *)(T))[0] = (unsigned char)(def_temp >> 8); \
T += 2; \
}
#define INT3TOCHAR(T, A) \
{ \
const unsigned long def_temp = (unsigned long)(A); \
((unsigned char *)(T))[2] = (unsigned char)(def_temp); \
((unsigned char *)(T))[1] = (unsigned char)(def_temp >> 8); \
((unsigned char *)(T))[0] = (unsigned char)(def_temp >> 16); \
T += 3; \
}
#define INT4TOCHAR(T, A) \
{ \
const unsigned long def_temp = (unsigned long)(A); \
((unsigned char *)(T))[3] = (unsigned char)(def_temp); \
((unsigned char *)(T))[2] = (unsigned char)(def_temp >> 8); \
((unsigned char *)(T))[1] = (unsigned char)(def_temp >> 16); \
((unsigned char *)(T))[0] = (unsigned char)(def_temp >> 24); \
T += 4; \
}
#define sclFatal(...) qFatal(__VA_ARGS__) #define sclFatal(...) qFatal(__VA_ARGS__)
#define sclError(...) qError(__VA_ARGS__) #define sclError(...) qError(__VA_ARGS__)
#define sclWarn(...) qWarn(__VA_ARGS__) #define sclWarn(...) qWarn(__VA_ARGS__)

25
source/libs/scalar/src/filter.c
View File

@ -1066,7 +1066,9 @@ int32_t filterAddField(SFilterInfo *info, void *desc, void **data, int32_t type,
info->fields[type].fields = info->fields[type].fields =
taosMemoryRealloc(info->fields[type].fields, info->fields[type].size * sizeof(SFilterField)); taosMemoryRealloc(info->fields[type].fields, info->fields[type].size * sizeof(SFilterField));
if (info->fields[type].fields == NULL) { if (info->fields[type].fields == NULL) {
return TSDB_CODE_OUT_OF_MEMORY; *num = 0;
fltError("taosMemoryRealloc failed, size:%d", (int32_t)(info->fields[type].size * sizeof(SFilterField)));
FLT_ERR_RET(terrno);
} }
} }
@ -1086,14 +1088,20 @@ int32_t filterAddField(SFilterInfo *info, void *desc, void **data, int32_t type,
taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), false, false); taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), false, false);
if (NULL == info->pctx.valHash) { if (NULL == info->pctx.valHash) {
fltError("taosHashInit failed, size:%d", FILTER_DEFAULT_GROUP_SIZE * FILTER_DEFAULT_VALUE_SIZE); fltError("taosHashInit failed, size:%d", FILTER_DEFAULT_GROUP_SIZE * FILTER_DEFAULT_VALUE_SIZE);
FLT_ERR_RET(TSDB_CODE_OUT_OF_MEMORY); if (srcFlag) {
FILTER_SET_FLAG(*srcFlag, FLD_DATA_NO_FREE);
}
FLT_ERR_RET(terrno);
} }
} }
SFilterDataInfo dInfo = {idx, *data}; SFilterDataInfo dInfo = {idx, *data};
if (taosHashPut(info->pctx.valHash, *data, dataLen, &dInfo, sizeof(dInfo))) { if (taosHashPut(info->pctx.valHash, *data, dataLen, &dInfo, sizeof(dInfo))) {
fltError("taosHashPut to set failed"); fltError("taosHashPut to set failed");
FLT_ERR_RET(TSDB_CODE_OUT_OF_MEMORY); if (srcFlag) {
FILTER_SET_FLAG(*srcFlag, FLD_DATA_NO_FREE);
}
FLT_ERR_RET(terrno);
} }
if (srcFlag) { if (srcFlag) {
FILTER_SET_FLAG(*srcFlag, FLD_DATA_NO_FREE); FILTER_SET_FLAG(*srcFlag, FLD_DATA_NO_FREE);
@ -1102,7 +1110,7 @@ int32_t filterAddField(SFilterInfo *info, void *desc, void **data, int32_t type,
} else if (type != FLD_TYPE_COLUMN && data) { } else if (type != FLD_TYPE_COLUMN && data) {
if (freeIfExists) { if (freeIfExists) {
taosMemoryFreeClear(*data); taosMemoryFreeClear(*data);
} else if (sameBuf) { } else if (sameBuf && srcFlag) {
FILTER_SET_FLAG(*srcFlag, FLD_DATA_NO_FREE); FILTER_SET_FLAG(*srcFlag, FLD_DATA_NO_FREE);
} }
} }
@ -1114,11 +1122,11 @@ int32_t filterAddField(SFilterInfo *info, void *desc, void **data, int32_t type,
} }
static FORCE_INLINE int32_t filterAddColFieldFromField(SFilterInfo *info, SFilterField *field, SFilterFieldId *fid) { static FORCE_INLINE int32_t filterAddColFieldFromField(SFilterInfo *info, SFilterField *field, SFilterFieldId *fid) {
FLT_ERR_RET(filterAddField(info, field->desc, &field->data, FILTER_GET_TYPE(field->flag), fid, 0, false, NULL)); int32_t code = filterAddField(info, field->desc, &field->data, FILTER_GET_TYPE(field->flag), fid, 0, false, NULL);
FILTER_SET_FLAG(field->flag, FLD_DATA_NO_FREE); FILTER_SET_FLAG(field->flag, FLD_DATA_NO_FREE);
return TSDB_CODE_SUCCESS; return code;
} }
int32_t filterAddFieldFromNode(SFilterInfo *info, SNode *node, SFilterFieldId *fid) { int32_t filterAddFieldFromNode(SFilterInfo *info, SNode *node, SFilterFieldId *fid) {
@ -2769,6 +2777,7 @@ int32_t filterConvertGroupFromArray(SFilterInfo *info, SArray *group) {
if (info->groupNum > 0) { if (info->groupNum > 0) {
info->groups = taosMemoryCalloc(info->groupNum, sizeof(*info->groups)); info->groups = taosMemoryCalloc(info->groupNum, sizeof(*info->groups));
if (info->groups == NULL) { if (info->groups == NULL) {
info->groupNum = 0;
FLT_ERR_RET(TSDB_CODE_OUT_OF_MEMORY); FLT_ERR_RET(TSDB_CODE_OUT_OF_MEMORY);
} }
} }
@ -2780,6 +2789,7 @@ int32_t filterConvertGroupFromArray(SFilterInfo *info, SArray *group) {
} }
pg->unitFlags = taosMemoryCalloc(pg->unitNum, sizeof(*pg->unitFlags)); pg->unitFlags = taosMemoryCalloc(pg->unitNum, sizeof(*pg->unitFlags));
if (pg->unitFlags == NULL) { if (pg->unitFlags == NULL) {
pg->unitNum = 0;
FLT_ERR_RET(TSDB_CODE_OUT_OF_MEMORY); FLT_ERR_RET(TSDB_CODE_OUT_OF_MEMORY);
} }
info->groups[i] = *pg; info->groups[i] = *pg;
@ -2855,10 +2865,9 @@ int32_t filterRewrite(SFilterInfo *info, SFilterGroupCtx **gRes, int32_t gResNum
} }
} }
} }
FLT_ERR_JRET(filterConvertGroupFromArray(info, group));
_return: _return:
FLT_ERR_RET(filterConvertGroupFromArray(info, group));
taosArrayDestroy(group); taosArrayDestroy(group);
filterFreeInfo(&oinfo); filterFreeInfo(&oinfo);

1499
source/libs/scalar/src/sclfunc.c
View File

File diff suppressed because it is too large Load Diff

28
source/libs/stream/src/streamMeta.c
View File

@ -1589,11 +1589,33 @@ static void displayStatusInfo(SStreamMeta* pMeta, SHashObj* pTaskSet, bool succ)
} }
} }
// check all existed tasks are received rsp
static bool allCheckDownstreamRsp(SStreamMeta* pMeta, STaskStartInfo* pStartInfo, int32_t numOfTotal) {
for (int32_t i = 0; i < numOfTotal; ++i) {
SStreamTaskId* pTaskId = taosArrayGet(pMeta->pTaskList, i);
if (pTaskId == NULL) {
continue;
}
STaskId idx = {.streamId = pTaskId->streamId, .taskId = pTaskId->taskId};
void* px = taosHashGet(pStartInfo->pReadyTaskSet, &idx, sizeof(idx));
if (px == NULL) {
px = taosHashGet(pStartInfo->pFailedTaskSet, &idx, sizeof(idx));
if (px == NULL) {
return false;
}
}
}
return true;
}
int32_t streamMetaAddTaskLaunchResult(SStreamMeta* pMeta, int64_t streamId, int32_t taskId, int64_t startTs, int32_t streamMetaAddTaskLaunchResult(SStreamMeta* pMeta, int64_t streamId, int32_t taskId, int64_t startTs,
int64_t endTs, bool ready) { int64_t endTs, bool ready) {
STaskStartInfo* pStartInfo = &pMeta->startInfo; STaskStartInfo* pStartInfo = &pMeta->startInfo;
STaskId id = {.streamId = streamId, .taskId = taskId}; STaskId id = {.streamId = streamId, .taskId = taskId};
int32_t vgId = pMeta->vgId; int32_t vgId = pMeta->vgId;
bool allRsp = true;
streamMetaWLock(pMeta); streamMetaWLock(pMeta);
SStreamTask** p = taosHashGet(pMeta->pTasksMap, &id, sizeof(id)); SStreamTask** p = taosHashGet(pMeta->pTasksMap, &id, sizeof(id));
@ -1618,9 +1640,8 @@ int32_t streamMetaAddTaskLaunchResult(SStreamMeta* pMeta, int64_t streamId, int3
return 0; return 0;
} }
SHashObj* pDst = ready ? pStartInfo->pReadyTaskSet : pStartInfo->pFailedTaskSet;
STaskInitTs initTs = {.start = startTs, .end = endTs, .success = ready}; STaskInitTs initTs = {.start = startTs, .end = endTs, .success = ready};
SHashObj* pDst = ready ? pStartInfo->pReadyTaskSet : pStartInfo->pFailedTaskSet;
int32_t code = taosHashPut(pDst, &id, sizeof(id), &initTs, sizeof(STaskInitTs)); int32_t code = taosHashPut(pDst, &id, sizeof(id), &initTs, sizeof(STaskInitTs));
if (code) { if (code) {
if (code == TSDB_CODE_DUP_KEY) { if (code == TSDB_CODE_DUP_KEY) {
@ -1637,7 +1658,8 @@ int32_t streamMetaAddTaskLaunchResult(SStreamMeta* pMeta, int64_t streamId, int3
int32_t numOfTotal = streamMetaGetNumOfTasks(pMeta); int32_t numOfTotal = streamMetaGetNumOfTasks(pMeta);
int32_t numOfRecv = taosHashGetSize(pStartInfo->pReadyTaskSet) + taosHashGetSize(pStartInfo->pFailedTaskSet); int32_t numOfRecv = taosHashGetSize(pStartInfo->pReadyTaskSet) + taosHashGetSize(pStartInfo->pFailedTaskSet);
if (numOfRecv == numOfTotal) { allRsp = allCheckDownstreamRsp(pMeta, pStartInfo, numOfTotal);
if (allRsp) {
pStartInfo->readyTs = taosGetTimestampMs(); pStartInfo->readyTs = taosGetTimestampMs();
pStartInfo->elapsedTime = (pStartInfo->startTs != 0) ? pStartInfo->readyTs - pStartInfo->startTs : 0; pStartInfo->elapsedTime = (pStartInfo->startTs != 0) ? pStartInfo->readyTs - pStartInfo->startTs : 0;

14
source/libs/wal/src/walMeta.c
View File

@ -818,7 +818,10 @@ int32_t walMetaDeserialize(SWal* pWal, const char* bytes) {
int sz = cJSON_GetArraySize(pFiles); int sz = cJSON_GetArraySize(pFiles);
// deserialize // deserialize
SArray* pArray = pWal->fileInfoSet; SArray* pArray = pWal->fileInfoSet;
(void)taosArrayEnsureCap(pArray, sz); if (taosArrayEnsureCap(pArray, sz)) {
cJSON_Delete(pRoot);
return terrno;
}
for (int i = 0; i < sz; i++) { for (int i = 0; i < sz; i++) {
pInfoJson = cJSON_GetArrayItem(pFiles, i); pInfoJson = cJSON_GetArrayItem(pFiles, i);
@ -841,7 +844,10 @@ int32_t walMetaDeserialize(SWal* pWal, const char* bytes) {
pField = cJSON_GetObjectItem(pInfoJson, "fileSize"); pField = cJSON_GetObjectItem(pInfoJson, "fileSize");
if (!pField) goto _err; if (!pField) goto _err;
info.fileSize = atoll(cJSON_GetStringValue(pField)); info.fileSize = atoll(cJSON_GetStringValue(pField));
(void)taosArrayPush(pArray, &info); if (!taosArrayPush(pArray, &info)) {
cJSON_Delete(pRoot);
return terrno;
}
} }
pWal->fileInfoSet = pArray; pWal->fileInfoSet = pArray;
pWal->writeCur = sz - 1; pWal->writeCur = sz - 1;
@ -860,8 +866,8 @@ static int walFindCurMetaVer(SWal* pWal) {
TdDirPtr pDir = taosOpenDir(pWal->path); TdDirPtr pDir = taosOpenDir(pWal->path);
if (pDir == NULL) { if (pDir == NULL) {
wError("vgId:%d, path:%s, failed to open since %s", pWal->cfg.vgId, pWal->path, strerror(errno)); wError("vgId:%d, path:%s, failed to open since %s", pWal->cfg.vgId, pWal->path, tstrerror(terrno));
return -1; return terrno;
} }
TdDirEntryPtr pDirEntry; TdDirEntryPtr pDirEntry;

14
source/util/src/tbloomfilter.c
View File

@ -131,16 +131,16 @@ void tBloomFilterDestroy(SBloomFilter* pBF) {
} }
int32_t tBloomFilterEncode(const SBloomFilter* pBF, SEncoder* pEncoder) { int32_t tBloomFilterEncode(const SBloomFilter* pBF, SEncoder* pEncoder) {
if (tEncodeU32(pEncoder, pBF->hashFunctions) < 0) return -1; TAOS_CHECK_RETURN(tEncodeU32(pEncoder, pBF->hashFunctions));
if (tEncodeU64(pEncoder, pBF->expectedEntries) < 0) return -1; TAOS_CHECK_RETURN(tEncodeU64(pEncoder, pBF->expectedEntries));
if (tEncodeU64(pEncoder, pBF->numUnits) < 0) return -1; TAOS_CHECK_RETURN(tEncodeU64(pEncoder, pBF->numUnits));
if (tEncodeU64(pEncoder, pBF->numBits) < 0) return -1; TAOS_CHECK_RETURN(tEncodeU64(pEncoder, pBF->numBits));
if (tEncodeU64(pEncoder, pBF->size) < 0) return -1; TAOS_CHECK_RETURN(tEncodeU64(pEncoder, pBF->size));
for (uint64_t i = 0; i < pBF->numUnits; i++) { for (uint64_t i = 0; i < pBF->numUnits; i++) {
uint64_t* pUnits = (uint64_t*)pBF->buffer; uint64_t* pUnits = (uint64_t*)pBF->buffer;
if (tEncodeU64(pEncoder, pUnits[i]) < 0) return -1; TAOS_CHECK_RETURN(tEncodeU64(pEncoder, pUnits[i]));
} }
if (tEncodeDouble(pEncoder, pBF->errorRate) < 0) return -1; TAOS_CHECK_RETURN(tEncodeDouble(pEncoder, pBF->errorRate));
return 0; return 0;
} }

3
source/util/src/terror.c
View File

@ -92,7 +92,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_CFG_NOT_FOUND, "Config not found")
TAOS_DEFINE_ERROR(TSDB_CODE_REPEAT_INIT, "Repeat initialization") TAOS_DEFINE_ERROR(TSDB_CODE_REPEAT_INIT, "Repeat initialization")
TAOS_DEFINE_ERROR(TSDB_CODE_DUP_KEY, "Cannot add duplicate keys to hash") TAOS_DEFINE_ERROR(TSDB_CODE_DUP_KEY, "Cannot add duplicate keys to hash")
TAOS_DEFINE_ERROR(TSDB_CODE_NEED_RETRY, "Retry needed") TAOS_DEFINE_ERROR(TSDB_CODE_NEED_RETRY, "Retry needed")
TAOS_DEFINE_ERROR(TSDB_CODE_OUT_OF_RPC_MEMORY_QUEUE, "Out of memory in rpc queue") TAOS_DEFINE_ERROR(TSDB_CODE_OUT_OF_RPC_MEMORY_QUEUE, "Out of memory in queue")
TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_TIMESTAMP, "Invalid timestamp format") TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_TIMESTAMP, "Invalid timestamp format")
TAOS_DEFINE_ERROR(TSDB_CODE_MSG_DECODE_ERROR, "Msg decode error") TAOS_DEFINE_ERROR(TSDB_CODE_MSG_DECODE_ERROR, "Msg decode error")
TAOS_DEFINE_ERROR(TSDB_CODE_MSG_ENCODE_ERROR, "Msg encode error") TAOS_DEFINE_ERROR(TSDB_CODE_MSG_ENCODE_ERROR, "Msg encode error")
@ -723,6 +723,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_FUNC_TIME_UNIT_INVALID, "Invalid function tim
TAOS_DEFINE_ERROR(TSDB_CODE_FUNC_TIME_UNIT_TOO_SMALL, "Function time unit cannot be smaller than db precision") TAOS_DEFINE_ERROR(TSDB_CODE_FUNC_TIME_UNIT_TOO_SMALL, "Function time unit cannot be smaller than db precision")
TAOS_DEFINE_ERROR(TSDB_CODE_FUNC_INVALID_VALUE_RANGE, "Function got invalid value range") TAOS_DEFINE_ERROR(TSDB_CODE_FUNC_INVALID_VALUE_RANGE, "Function got invalid value range")
TAOS_DEFINE_ERROR(TSDB_CODE_FUNC_SETUP_ERROR, "Function set up failed") TAOS_DEFINE_ERROR(TSDB_CODE_FUNC_SETUP_ERROR, "Function set up failed")
TAOS_DEFINE_ERROR(TSDB_CODE_FUNC_INVALID_RES_LENGTH, "Function result exceed max length")
//udf //udf
TAOS_DEFINE_ERROR(TSDB_CODE_UDF_STOPPING, "udf is stopping") TAOS_DEFINE_ERROR(TSDB_CODE_UDF_STOPPING, "udf is stopping")

42
source/util/src/tqueue.c
View File

@ -19,8 +19,8 @@
#include "tlog.h" #include "tlog.h"
#include "tutil.h" #include "tutil.h"
int64_t tsRpcQueueMemoryAllowed = 0; int64_t tsQueueMemoryAllowed = 0;
int64_t tsRpcQueueMemoryUsed = 0; int64_t tsQueueMemoryUsed = 0;
struct STaosQueue { struct STaosQueue {
STaosQnode *head; STaosQnode *head;
@ -148,10 +148,20 @@ int64_t taosQueueMemorySize(STaosQueue *queue) {
} }
int32_t taosAllocateQitem(int32_t size, EQItype itype, int64_t dataSize, void **item) { int32_t taosAllocateQitem(int32_t size, EQItype itype, int64_t dataSize, void **item) {
*item = NULL; int64_t alloced = atomic_add_fetch_64(&tsQueueMemoryUsed, size + dataSize);
if (alloced > tsQueueMemoryAllowed) {
if (itype == RPC_QITEM) {
uError("failed to alloc qitem, size:%" PRId64 " alloc:%" PRId64 " allowed:%" PRId64, size + dataSize, alloced,
tsQueueMemoryAllowed);
(void)atomic_sub_fetch_64(&tsQueueMemoryUsed, size + dataSize);
return (terrno = TSDB_CODE_OUT_OF_RPC_MEMORY_QUEUE);
}
}
*item = NULL;
STaosQnode *pNode = taosMemoryCalloc(1, sizeof(STaosQnode) + size); STaosQnode *pNode = taosMemoryCalloc(1, sizeof(STaosQnode) + size);
if (pNode == NULL) { if (pNode == NULL) {
(void)atomic_sub_fetch_64(&tsQueueMemoryUsed, size + dataSize);
return terrno = TSDB_CODE_OUT_OF_MEMORY; return terrno = TSDB_CODE_OUT_OF_MEMORY;
} }
@ -159,21 +169,7 @@ int32_t taosAllocateQitem(int32_t size, EQItype itype, int64_t dataSize, void **
pNode->size = size; pNode->size = size;
pNode->itype = itype; pNode->itype = itype;
pNode->timestamp = taosGetTimestampUs(); pNode->timestamp = taosGetTimestampUs();
uTrace("item:%p, node:%p is allocated, alloc:%" PRId64, pNode->item, pNode, alloced);
if (itype == RPC_QITEM) {
int64_t alloced = atomic_add_fetch_64(&tsRpcQueueMemoryUsed, size + dataSize);
if (alloced > tsRpcQueueMemoryAllowed) {
uError("failed to alloc qitem, size:%" PRId64 " alloc:%" PRId64 " allowed:%" PRId64, size + dataSize, alloced,
tsRpcQueueMemoryAllowed);
(void)atomic_sub_fetch_64(&tsRpcQueueMemoryUsed, size + dataSize);
taosMemoryFree(pNode);
return (terrno = TSDB_CODE_OUT_OF_RPC_MEMORY_QUEUE);
}
uTrace("item:%p, node:%p is allocated, alloc:%" PRId64, pNode->item, pNode, alloced);
} else {
uTrace("item:%p, node:%p is allocated", pNode->item, pNode);
}
*item = pNode->item; *item = pNode->item;
return 0; return 0;
} }
@ -182,12 +178,8 @@ void taosFreeQitem(void *pItem) {
if (pItem == NULL) return; if (pItem == NULL) return;
STaosQnode *pNode = (STaosQnode *)((char *)pItem - sizeof(STaosQnode)); STaosQnode *pNode = (STaosQnode *)((char *)pItem - sizeof(STaosQnode));
if (pNode->itype == RPC_QITEM) { int64_t alloced = atomic_sub_fetch_64(&tsQueueMemoryUsed, pNode->size + pNode->dataSize);
int64_t alloced = atomic_sub_fetch_64(&tsRpcQueueMemoryUsed, pNode->size + pNode->dataSize); uTrace("item:%p, node:%p is freed, alloc:%" PRId64, pItem, pNode, alloced);
uTrace("item:%p, node:%p is freed, alloc:%" PRId64, pItem, pNode, alloced);
} else {
uTrace("item:%p, node:%p is freed", pItem, pNode);
}
taosMemoryFree(pNode); taosMemoryFree(pNode);
} }
@ -380,7 +372,7 @@ void taosQsetThreadResume(STaosQset *qset) {
} }
int32_t taosAddIntoQset(STaosQset *qset, STaosQueue *queue, void *ahandle) { int32_t taosAddIntoQset(STaosQset *qset, STaosQueue *queue, void *ahandle) {
if (queue->qset) return -1; if (queue->qset) return TSDB_CODE_INVALID_PARA;
(void)taosThreadMutexLock(&qset->mutex); (void)taosThreadMutexLock(&qset->mutex);

14
source/util/src/tscalablebf.c
View File

@ -188,19 +188,19 @@ void tScalableBfDestroy(SScalableBf* pSBf) {
int32_t tScalableBfEncode(const SScalableBf* pSBf, SEncoder* pEncoder) { int32_t tScalableBfEncode(const SScalableBf* pSBf, SEncoder* pEncoder) {
if (!pSBf) { if (!pSBf) {
if (tEncodeI32(pEncoder, 0) < 0) return -1; TAOS_CHECK_RETURN(tEncodeI32(pEncoder, 0));
return 0; return 0;
} }
int32_t size = taosArrayGetSize(pSBf->bfArray); int32_t size = taosArrayGetSize(pSBf->bfArray);
if (tEncodeI32(pEncoder, size) < 0) return -1; TAOS_CHECK_RETURN(tEncodeI32(pEncoder, size));
for (int32_t i = 0; i < size; i++) { for (int32_t i = 0; i < size; i++) {
SBloomFilter* pBF = taosArrayGetP(pSBf->bfArray, i); SBloomFilter* pBF = taosArrayGetP(pSBf->bfArray, i);
if (tBloomFilterEncode(pBF, pEncoder) < 0) return -1; TAOS_CHECK_RETURN(tBloomFilterEncode(pBF, pEncoder));
} }
if (tEncodeU32(pEncoder, pSBf->growth) < 0) return -1; TAOS_CHECK_RETURN(tEncodeU32(pEncoder, pSBf->growth));
if (tEncodeU64(pEncoder, pSBf->numBits) < 0) return -1; TAOS_CHECK_RETURN(tEncodeU64(pEncoder, pSBf->numBits));
if (tEncodeU32(pEncoder, pSBf->maxBloomFilters) < 0) return -1; TAOS_CHECK_RETURN(tEncodeU32(pEncoder, pSBf->maxBloomFilters));
if (tEncodeI8(pEncoder, pSBf->status) < 0) return -1; TAOS_CHECK_RETURN(tEncodeI8(pEncoder, pSBf->status));
return 0; return 0;
} }

4
source/util/src/tsched.c
View File

@ -189,12 +189,12 @@ int taosScheduleTask(void *queueScheduler, SSchedMsg *pMsg) {
if (pSched == NULL) { if (pSched == NULL) {
uError("sched is not ready, msg:%p is dropped", pMsg); uError("sched is not ready, msg:%p is dropped", pMsg);
return -1; return TSDB_CODE_INVALID_PARA;
} }
if (atomic_load_8(&pSched->stop)) { if (atomic_load_8(&pSched->stop)) {
uError("sched is already stopped, msg:%p is dropped", pMsg); uError("sched is already stopped, msg:%p is dropped", pMsg);
return -1; return TSDB_CODE_INVALID_PARA;
} }
if ((ret = tsem_wait(&pSched->emptySem)) != 0) { if ((ret = tsem_wait(&pSched->emptySem)) != 0) {

2
source/util/src/tsimplehash.c
View File

@ -209,7 +209,7 @@ static int32_t tSimpleHashTableResize(SSHashObj *pHashObj) {
int32_t tSimpleHashPut(SSHashObj *pHashObj, const void *key, size_t keyLen, const void *data, size_t dataLen) { int32_t tSimpleHashPut(SSHashObj *pHashObj, const void *key, size_t keyLen, const void *data, size_t dataLen) {
if (!pHashObj || !key) { if (!pHashObj || !key) {
return TSDB_CODE_FAILED; return TSDB_CODE_INVALID_PARA;
} }
uint32_t hashVal = (*pHashObj->hashFp)(key, (uint32_t)keyLen); uint32_t hashVal = (*pHashObj->hashFp)(key, (uint32_t)keyLen);

101
tests/army/query/fill/fill_compare_asc_desc.py Normal file
View File

@ -0,0 +1,101 @@
from frame.log import *
from frame.cases import *
from frame.sql import *
from frame.caseBase import *
from frame import *
class TDTestCase(TBase):
def init(self, conn, logSql, replicaVar=1):
self.replicaVar = int(replicaVar)
tdLog.debug(f"start to excute {__file__}")
tdSql.init(conn.cursor(), logSql)
def prepare_data(self):
tdSql.execute("create database db;")
tdSql.execute("use db;")
# data for fill(prev)
tdSql.execute("create stable st_pre (ts timestamp, c1 int) tags(t1 int);")
tdSql.execute("create table ct1 using st_pre tags(1);")
start_ts = 1705783972000
sql = "insert into ct1 values "
for i in range(100):
sql += f"({start_ts + i * 1000}, {str(i+1)})"
sql += ";"
tdSql.execute(sql)
# data for fill(next)
tdSql.execute("create stable st_next (ts timestamp, c1 int) tags(t1 int);")
tdSql.execute("create table ct2 using st_next tags(1);")
start_ts = 1705783972000
sql = "insert into ct1 values "
for i in range(100):
sql += f"({start_ts + i * 1000}, NULL)"
sql += ";"
tdSql.execute(sql)
# data for fill(linear)
tdSql.execute("create stable st_linear (ts timestamp, c1 int) tags(t1 int);")
tdSql.execute("create table ct3 using st_linear tags(1);")
start_ts = 1705783972000
sql = "insert into ct1 values "
for i in range(100):
if i % 2 == 0:
sql += f"({start_ts + i * 1000}, {str(i+1)})"
else:
sql += f"({start_ts + i * 1000}, NULL)"
sql += ";"
tdSql.execute(sql)
tdLog.info("prepare data done")
def test_fill_pre_compare_asc_desc(self):
tdSql.execute("use db;")
for func in ["avg(c1)", "count(c1)", "first(c1)", "last(c1)", "max(c1)", "min(c1)", "sum(c1)"]:
tdSql.query(f"select _wstart, {func} from st_pre where ts between '2024-01-21 04:52:52.000' and '2024-01-21 04:54:31.000' interval(5s) fill(prev) order by _wstart asc;")
res1 = tdSql.res
tdSql.query(f"select _wstart, {func} from st_pre where ts between '2024-01-21 04:52:52.000' and '2024-01-21 04:54:31.000' interval(5s) fill(prev) order by _wstart desc;")
res2 = tdSql.res
assert len(res1) == len(res2)
for i in range(len(res1)):
assert res1[i] in res2
tdLog.info(f"fill(prev) {func} compare asc and desc done")
tdLog.info("Finish the test case 'test_fill_pre_compare_asc_desc'")
def test_fill_next_compare_asc_desc(self):
tdSql.execute("use db;")
for func in ["avg(c1)", "count(c1)", "first(c1)", "last(c1)", "max(c1)", "min(c1)", "sum(c1)"]:
tdSql.query(f"select _wstart, {func} from st_next where ts between '2024-01-21 04:52:52.000' and '2024-01-21 04:54:31.000' interval(5s) fill(next) order by _wstart asc;")
res1 = tdSql.res
tdSql.query(f"select _wstart, {func} from st_next where ts between '2024-01-21 04:52:52.000' and '2024-01-21 04:54:31.000' interval(5s) fill(next) order by _wstart desc;")
res2 = tdSql.res
assert len(res1) == len(res2)
for i in range(len(res1)):
assert res1[i] in res2
tdLog.info(f"fill(next) {func} compare asc and desc done")
tdLog.info("Finish the test case 'test_fill_next_compare_asc_desc'")
def test_fill_linear_compare_asc_desc(self):
tdSql.execute("use db;")
for func in ["avg(c1)", "count(c1)", "first(c1)", "last(c1)", "max(c1)", "min(c1)", "sum(c1)"]:
tdSql.query(f"select _wstart, {func} from st_linear where ts between '2024-01-21 04:52:52.000' and '2024-01-21 04:54:31.000' interval(5s) fill(linear) order by _wstart asc;")
res1 = tdSql.res
tdSql.query(f"select _wstart, {func} from st_linear where ts between '2024-01-21 04:52:52.000' and '2024-01-21 04:54:31.000' interval(5s) fill(linear) order by _wstart desc;")
res2 = tdSql.res
assert len(res1) == len(res2)
for i in range(len(res1)):
assert res1[i] in res2
tdLog.info(f"fill(linear) {func} compare asc and desc done")
tdLog.info("Finish the test case 'test_fill_linear_compare_asc_desc'")
def run(self):
self.prepare_data()
self.test_fill_pre_compare_asc_desc()
self.test_fill_next_compare_asc_desc()
self.test_fill_linear_compare_asc_desc()
def stop(self):
tdSql.close()
tdLog.success(f"{__file__} successfully executed")
tdCases.addLinux(__file__, TDTestCase())
tdCases.addWindows(__file__, TDTestCase())

4
tests/army/query/query_basic.py
View File

@ -262,9 +262,9 @@ class TDTestCase(TBase):
sql1 = f"select substr(bin,1) from {self.db}.d0 order by ts desc limit 100" sql1 = f"select substr(bin,1) from {self.db}.d0 order by ts desc limit 100"
sql2 = f"select bin from {self.db}.d0 order by ts desc limit 100" sql2 = f"select bin from {self.db}.d0 order by ts desc limit 100"
self.checkSameResult(sql1, sql2) self.checkSameResult(sql1, sql2)
#substr error input pos is zero
sql = f"select substr(bin,0,3) from {self.db}.d0 order by ts desc limit 100" sql = f"select substr(bin,0,3) from {self.db}.d0 order by ts desc limit 100"
tdSql.error(sql) tdSql.query(sql)
tdSql.checkData(0, 0, "")
# cast # cast
nch = 99 nch = 99

24
tests/ci/Dockerfile
View File

@ -1,9 +1,14 @@
FROM python:3.8 FROM python:3.8
RUN pip3 config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple COPY sources.list /etc/apt/
RUN pip3 install pandas psutil fabric2 requests faker simplejson toml pexpect tzlocal distro COPY id_ecdsa /root/.ssh/id_ecdsa
RUN apt-get update RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 3B4FE6ACC0B21F32 && apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 871920D1991BC93C
RUN apt-get install -y psmisc sudo tree libgeos-dev libjansson-dev libsnappy-dev liblzma-dev libz-dev zlib1g pkg-config build-essential valgrind \ RUN apt-get update
vim libjemalloc-dev openssh-server screen sshpass net-tools dirmngr gnupg apt-transport-https ca-certificates software-properties-common r-base iputils-ping RUN apt-get install -y locales psmisc sudo tree libgeos-dev libgflags2.2 libgflags-dev libgoogle-glog-dev libjansson-dev libsnappy-dev liblzma-dev libz-dev zlib1g pkg-config build-essential valgrind rsync vim libjemalloc-dev openssh-server screen sshpass net-tools dirmngr gnupg apt-transport-https ca-certificates software-properties-common r-base iputils-ping clang-tools-16
RUN sed -i 's/# en_US.UTF-8/en_US.UTF-8/' /etc/locale.gen && locale-gen
RUN pip3 config set global.index-url http://admin:123456@192.168.0.212:3141/admin/dev/+simple/
RUN pip3 config set global.trusted-host 192.168.0.212
RUN pip3 install taospy==2.7.15 taos-ws-py==0.3.1 pandas psutil fabric2 requests faker simplejson toml pexpect tzlocal distro decorator loguru hyperloglog
ENV LANG=en_US.UTF-8 LANGUAGE=en_US.UTF-8 LC_ALL=en_US.UTF-8
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9 RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9
RUN add-apt-repository 'deb https://cloud.r-project.org/bin/linux/ubuntu focal-cran40/' RUN add-apt-repository 'deb https://cloud.r-project.org/bin/linux/ubuntu focal-cran40/'
RUN apt install -y r-base RUN apt install -y r-base
@ -17,17 +22,16 @@ RUN apt-get install wget -y \
&& apt-get update && apt-get install -y dotnet-sdk-5.0 && apt-get install -y dotnet-sdk-6.0 && apt-get update && apt-get install -y dotnet-sdk-5.0 && apt-get install -y dotnet-sdk-6.0
ADD node-v12.20.0-linux-x64.tar.gz /usr/local/ ADD node-v12.20.0-linux-x64.tar.gz /usr/local/
RUN sh -c "rm -f /etc/localtime;ln -s /usr/share/zoneinfo/Asia/Shanghai /etc/localtime;echo \"Asia/Shanghai\" >/etc/timezone" RUN sh -c "rm -f /etc/localtime;ln -s /usr/share/zoneinfo/Asia/Shanghai /etc/localtime;echo \"Asia/Shanghai\" >/etc/timezone"
COPY id_rsa /root/.ssh/id_rsa
COPY .m2 /root/.m2 COPY .m2 /root/.m2
COPY .nuget /root/.nuget COPY .nuget /root/.nuget
COPY .dotnet /root/.dotnet COPY .dotnet /root/.dotnet
COPY .cargo /root/.cargo
COPY go /root/go COPY go /root/go
ADD cmake-3.21.5-linux-x86_64.tar.gz /usr/local/ ADD cmake-3.21.5-linux-x86_64.tar.gz /usr/local/
RUN echo " export RUSTUP_DIST_SERVER=\"https://rsproxy.cn\" " >> /root/.bashrc RUN echo " export RUSTUP_DIST_SERVER=\"https://rsproxy.cn\" " >> /root/.bashrc
RUN echo " export RUSTUP_UPDATE_ROOT=\"https://rsproxy.cn/rustup\" " >> /root/.bashrc RUN echo " export RUSTUP_UPDATE_ROOT=\"https://rsproxy.cn/rustup\" " >> /root/.bashrc
RUN curl https://sh.rustup.rs -o /tmp/rustup-init.sh RUN curl https://sh.rustup.rs -o /tmp/rustup-init.sh
RUN sh /tmp/rustup-init.sh -y RUN sh /tmp/rustup-init.sh -y
COPY .cargo/config /root/.cargo/config
ENV PATH /usr/local/go/bin:/usr/local/node-v12.20.0-linux-x64/bin:/usr/local/apache-maven-3.8.4/bin:/usr/local/jdk1.8.0_144/bin:/usr/local/cmake-3.21.5-linux-x86_64/bin:/root/.cargo/bin:$PATH ENV PATH /usr/local/go/bin:/usr/local/node-v12.20.0-linux-x64/bin:/usr/local/apache-maven-3.8.4/bin:/usr/local/jdk1.8.0_144/bin:/usr/local/cmake-3.21.5-linux-x86_64/bin:/root/.cargo/bin:$PATH
ENV JAVA_HOME /usr/local/jdk1.8.0_144 ENV JAVA_HOME /usr/local/jdk1.8.0_144
RUN go env -w GOPROXY=https://goproxy.cn RUN go env -w GOPROXY=https://goproxy.cn
@ -39,10 +43,8 @@ RUN R CMD javareconf JAVA_HOME=${JAVA_HOME} JAVA=${JAVA_HOME}/bin/java JAVAC=${J
RUN echo "install.packages(\"RJDBC\", repos=\"http://cran.us.r-project.org\")"|R --no-save RUN echo "install.packages(\"RJDBC\", repos=\"http://cran.us.r-project.org\")"|R --no-save
COPY .gitconfig /root/.gitconfig COPY .gitconfig /root/.gitconfig
RUN mkdir -p /run/sshd RUN mkdir -p /run/sshd
COPY id_rsa.pub /root/.ssh/id_rsa.pub COPY id_ecdsa.pub /root/.ssh/id_ecdsa.pub
COPY id_rsa.pub /root/.ssh/authorized_keys COPY id_ecdsa.pub /root/.ssh/authorized_keys
RUN pip3 uninstall -y taostest RUN pip3 uninstall -y taostest
COPY repository/TDinternal /home/TDinternal COPY repository/TDinternal /home/TDinternal
COPY repository/taos-connector-python /home/taos-connector-python
RUN sh -c "cd /home/taos-connector-python; pip3 install ."
COPY setup.sh /home/setup.sh COPY setup.sh /home/setup.sh

8
tests/ci/scan_file_path.py
View File

@ -122,7 +122,7 @@ def scan_files_path(source_file_path):
for file in files: for file in files:
if any(item in root for item in scan_dir_list): if any(item in root for item in scan_dir_list):
file_path = os.path.join(root, file) file_path = os.path.join(root, file)
if (file_path.endswith(".c") or file_name.endswith(".h") or file_path.endswith(".cpp")) and all(item not in file_path for item in scan_skip_file_list): if (file_path.endswith(".c") or file_path.endswith(".h") or file_path.endswith(".cpp")) and all(item not in file_path for item in scan_skip_file_list):
all_file_path.append(file_path) all_file_path.append(file_path)
logger.info("Found %s files" % len(all_file_path)) logger.info("Found %s files" % len(all_file_path))
@ -134,7 +134,7 @@ def input_files(change_files):
for line in file: for line in file:
file_name = line.strip() file_name = line.strip()
if any(dir_name in file_name for dir_name in scan_dir_list): if any(dir_name in file_name for dir_name in scan_dir_list):
if (file_name.endswith(".c") or line.endswith(".cpp")) and all(dir_name not in file_name for dir_name in scan_skip_file_list): if (file_name.endswith(".c") or file_name.endswith(".cpp")) and all(dir_name not in file_name for dir_name in scan_skip_file_list):
if "enterprise" in file_name: if "enterprise" in file_name:
file_name = os.path.join(TD_project_path, file_name) file_name = os.path.join(TD_project_path, file_name)
else: else:
@ -184,12 +184,12 @@ if __name__ == "__main__":
# os.makedirs(scan_result_path) # os.makedirs(scan_result_path)
for file in all_file_path: for file in all_file_path:
cmd = f"clang-query-10 -p {compile_commands_path} {file} -f {clang_scan_rules_path}" cmd = f"clang-query-16 -p {compile_commands_path} {file} -f {clang_scan_rules_path}"
logger.debug(f"cmd:{cmd}") logger.debug(f"cmd:{cmd}")
try: try:
stdout, stderr = command_executor.execute(cmd) stdout, stderr = command_executor.execute(cmd)
#if "error" in stderr: #if "error" in stderr:
# print(stderr) print(stderr)
lines = stdout.split("\n") lines = stdout.split("\n")
if lines[-2].endswith("matches.") or lines[-2].endswith("match."): if lines[-2].endswith("matches.") or lines[-2].endswith("match."):
match_num = int(lines[-2].split(" ")[0]) match_num = int(lines[-2].split(" ")[0])

1
tests/parallel_test/cases.task
View File

@ -37,6 +37,7 @@
,,y,army,./pytest.sh python3 ./test.py -f grant/grantBugs.py -N 3 ,,y,army,./pytest.sh python3 ./test.py -f grant/grantBugs.py -N 3
,,y,army,./pytest.sh python3 ./test.py -f query/queryBugs.py -N 3 ,,y,army,./pytest.sh python3 ./test.py -f query/queryBugs.py -N 3
,,y,army,./pytest.sh python3 ./test.py -f tmq/tmqBugs.py -N 3 ,,y,army,./pytest.sh python3 ./test.py -f tmq/tmqBugs.py -N 3
,,y,army,./pytest.sh python3 ./test.py -f query/fill/fill_compare_asc_desc.py
# #
# system test # system test

55
tests/parallel_test/container_build.sh
View File

@ -83,7 +83,7 @@ docker run \
-v ${REP_REAL_PATH}/community/contrib/xml2/:${REP_DIR}/community/contrib/xml2 \ -v ${REP_REAL_PATH}/community/contrib/xml2/:${REP_DIR}/community/contrib/xml2 \
-v ${REP_REAL_PATH}/community/contrib/zlib/:${REP_DIR}/community/contrib/zlib \ -v ${REP_REAL_PATH}/community/contrib/zlib/:${REP_DIR}/community/contrib/zlib \
-v ${REP_REAL_PATH}/community/contrib/zstd/:${REP_DIR}/community/contrib/zstd \ -v ${REP_REAL_PATH}/community/contrib/zstd/:${REP_DIR}/community/contrib/zstd \
--rm --ulimit core=-1 taos_test:v1.0 sh -c "pip uninstall taospy -y;pip3 install taospy==2.7.2;cd $REP_DIR;rm -rf debug;mkdir -p debug;cd debug;cmake .. -DBUILD_HTTP=false -DBUILD_TOOLS=true -DBUILD_TEST=true -DWEBSOCKET=true -DBUILD_TAOSX=false -DJEMALLOC_ENABLED=0 -DCMAKE_EXPORT_COMPILE_COMMANDS=1 ;make -j 10|| exit 1" --rm --ulimit core=-1 taos_test:v1.0 sh -c "pip uninstall taospy -y;pip3 install taospy==2.7.2;cd $REP_DIR;rm -rf debug;mkdir -p debug;cd debug;cmake .. -DBUILD_HTTP=false -DBUILD_TOOLS=true -DBUILD_TEST=true -DWEBSOCKET=true -DBUILD_TAOSX=false -DJEMALLOC_ENABLED=0 -DCMAKE_EXPORT_COMPILE_COMMANDS=1 ;make -j|| exit 1"
# -v ${REP_REAL_PATH}/community/contrib/jemalloc/:${REP_DIR}/community/contrib/jemalloc \ # -v ${REP_REAL_PATH}/community/contrib/jemalloc/:${REP_DIR}/community/contrib/jemalloc \
if [[ -d ${WORKDIR}/debugNoSan ]] ;then if [[ -d ${WORKDIR}/debugNoSan ]] ;then
@ -94,13 +94,17 @@ if [[ -d ${WORKDIR}/debugSan ]] ;then
echo "delete ${WORKDIR}/debugSan" echo "delete ${WORKDIR}/debugSan"
rm -rf ${WORKDIR}/debugSan rm -rf ${WORKDIR}/debugSan
fi fi
if [[ -d ${WORKDIR}/debugRelease ]] ;then
if [ "$(uname -m)" = "aarch64" ] ;then echo "delete ${WORKDIR}/debugRelease"
CMAKE_BUILD_TYPE="-DCMAKE_BUILD_TYPE=Debug" rm -rf ${WORKDIR}/debugRelease
else
CMAKE_BUILD_TYPE="-DCMAKE_BUILD_TYPE=Release"
fi fi
# if [ "$(uname -m)" = "aarch64" ] ;then
# CMAKE_BUILD_TYPE="-DCMAKE_BUILD_TYPE=Debug"
# else
# CMAKE_BUILD_TYPE="-DCMAKE_BUILD_TYPE=Release"
# fi
mv ${REP_REAL_PATH}/debug ${WORKDIR}/debugNoSan mv ${REP_REAL_PATH}/debug ${WORKDIR}/debugNoSan
date date
docker run \ docker run \
@ -133,10 +137,47 @@ docker run \
-v ${REP_REAL_PATH}/community/contrib/xml2/:${REP_DIR}/community/contrib/xml2 \ -v ${REP_REAL_PATH}/community/contrib/xml2/:${REP_DIR}/community/contrib/xml2 \
-v ${REP_REAL_PATH}/community/contrib/zlib/:${REP_DIR}/community/contrib/zlib \ -v ${REP_REAL_PATH}/community/contrib/zlib/:${REP_DIR}/community/contrib/zlib \
-v ${REP_REAL_PATH}/community/contrib/zstd/:${REP_DIR}/community/contrib/zstd \ -v ${REP_REAL_PATH}/community/contrib/zstd/:${REP_DIR}/community/contrib/zstd \
--rm --ulimit core=-1 taos_test:v1.0 sh -c "pip uninstall taospy -y;pip3 install taospy==2.7.2;cd $REP_DIR;rm -rf debug;mkdir -p debug;cd debug;cmake .. -DBUILD_HTTP=false -DBUILD_TOOLS=true -DBUILD_TEST=false -DWEBSOCKET=true -DBUILD_SANITIZER=1 -DTOOLS_SANITIZE=true $CMAKE_BUILD_TYPE -DTOOLS_BUILD_TYPE=Debug -DBUILD_TAOSX=false -DJEMALLOC_ENABLED=0;make -j 10|| exit 1 " --rm --ulimit core=-1 taos_test:v1.0 sh -c "pip uninstall taospy -y;pip3 install taospy==2.7.2;cd $REP_DIR;rm -rf debug;mkdir -p debug;cd debug;cmake .. -DBUILD_HTTP=false -DBUILD_TOOLS=true -DBUILD_TEST=false -DWEBSOCKET=true -DBUILD_SANITIZER=1 -DTOOLS_SANITIZE=true -DCMAKE_BUILD_TYPE=Debug -DTOOLS_BUILD_TYPE=Debug -DBUILD_TAOSX=false -DJEMALLOC_ENABLED=0;make -j|| exit 1 "
mv ${REP_REAL_PATH}/debug ${WORKDIR}/debugSan mv ${REP_REAL_PATH}/debug ${WORKDIR}/debugSan
date
# docker run \
# -v $REP_MOUNT_PARAM \
# -v /root/.cargo/registry:/root/.cargo/registry \
# -v /root/.cargo/git:/root/.cargo/git \
# -v /root/go/pkg/mod:/root/go/pkg/mod \
# -v /root/.cache/go-build:/root/.cache/go-build \
# -v /root/.cos-local.1:/root/.cos-local.2 \
# -v ${REP_REAL_PATH}/enterprise/contrib/grant-lib:${REP_DIR}/enterprise/contrib/grant-lib \
# -v ${REP_REAL_PATH}/community/tools/taosadapter:${REP_DIR}/community/tools/taosadapter \
# -v ${REP_REAL_PATH}/community/tools/taos-tools:${REP_DIR}/community/tools/taos-tools \
# -v ${REP_REAL_PATH}/community/tools/taosws-rs:${REP_DIR}/community/tools/taosws-rs \
# -v ${REP_REAL_PATH}/community/tools/taosws-rs/target:${REP_DIR}/community/tools/taosws-rs/target \
# -v ${REP_REAL_PATH}/community/contrib/apr/:${REP_DIR}/community/contrib/apr \
# -v ${REP_REAL_PATH}/community/contrib/apr-util/:${REP_DIR}/community/contrib/apr-util \
# -v ${REP_REAL_PATH}/community/contrib/cJson/:${REP_DIR}/community/contrib/cJson \
# -v ${REP_REAL_PATH}/community/contrib/cpp-stub/:${REP_DIR}/community/contrib/cpp-stub \
# -v ${REP_REAL_PATH}/community/contrib/curl/:${REP_DIR}/community/contrib/curl \
# -v ${REP_REAL_PATH}/community/contrib/curl2/:${REP_DIR}/community/contrib/curl2 \
# -v ${REP_REAL_PATH}/community/contrib/geos/:${REP_DIR}/community/contrib/geos \
# -v ${REP_REAL_PATH}/community/contrib/googletest/:${REP_DIR}/community/contrib/googletest \
# -v ${REP_REAL_PATH}/community/contrib/libs3/:${REP_DIR}/community/contrib/libs3 \
# -v ${REP_REAL_PATH}/community/contrib/libuv/:${REP_DIR}/community/contrib/libuv \
# -v ${REP_REAL_PATH}/community/contrib/lz4/:${REP_DIR}/community/contrib/lz4 \
# -v ${REP_REAL_PATH}/community/contrib/lzma2/:${REP_DIR}/community/contrib/lzma2 \
# -v ${REP_REAL_PATH}/community/contrib/mxml/:${REP_DIR}/community/contrib/mxml \
# -v ${REP_REAL_PATH}/community/contrib/openssl/:${REP_DIR}/community/contrib/openssl \
# -v ${REP_REAL_PATH}/community/contrib/pcre2/:${REP_DIR}/community/contrib/pcre2 \
# -v ${REP_REAL_PATH}/community/contrib/xml2/:${REP_DIR}/community/contrib/xml2 \
# -v ${REP_REAL_PATH}/community/contrib/zlib/:${REP_DIR}/community/contrib/zlib \
# -v ${REP_REAL_PATH}/community/contrib/zstd/:${REP_DIR}/community/contrib/zstd \
# --rm --ulimit core=-1 taos_test:v1.0 sh -c "pip uninstall taospy -y;pip3 install taospy==2.7.2;cd $REP_DIR;rm -rf debug;mkdir -p debug;cd debug;cmake .. -DBUILD_HTTP=false -DBUILD_TOOLS=true -DBUILD_TEST=false -DWEBSOCKET=true -DCMAKE_BUILD_TYPE=Release -DBUILD_TAOSX=false -DJEMALLOC_ENABLED=0;make -j || exit 1 "
# mv ${REP_REAL_PATH}/debug ${WORKDIR}/debugRelease
ret=$? ret=$?
exit $ret exit $ret

2
tests/parallel_test/run_scan_container.sh
View File

@ -79,6 +79,7 @@ scan_scripts="$CONTAINER_TESTDIR/tests/ci/scan_file_path.py"
ulimit -c unlimited ulimit -c unlimited
cat << EOF cat << EOF
docker run \ docker run \
-v /root/.cos-local.1:/root/.cos-local.2 \
-v $REP_MOUNT_PARAM \ -v $REP_MOUNT_PARAM \
-v $REP_MOUNT_DEBUG \ -v $REP_MOUNT_DEBUG \
-v $scan_changefile_temp_path:$docker_can_changefile_temp_path \ -v $scan_changefile_temp_path:$docker_can_changefile_temp_path \
@ -86,6 +87,7 @@ docker run \
--rm --ulimit core=-1 taos_test:v1.0 python3 $scan_scripts -b "${branch_name_id}" -f "${scan_file_name}" -w ${web_server} --rm --ulimit core=-1 taos_test:v1.0 python3 $scan_scripts -b "${branch_name_id}" -f "${scan_file_name}" -w ${web_server}
EOF EOF
docker run \ docker run \
-v /root/.cos-local.1:/root/.cos-local.2 \
-v $REP_MOUNT_PARAM \ -v $REP_MOUNT_PARAM \
-v $REP_MOUNT_DEBUG \ -v $REP_MOUNT_DEBUG \
-v $scan_changefile_temp_path:$docker_can_changefile_temp_path \ -v $scan_changefile_temp_path:$docker_can_changefile_temp_path \

2
tests/script/tsim/scalar/tsConvert.sim
View File

@ -13,7 +13,7 @@ sql insert into tb1 values ('2022-07-10 16:31:00', 1, '1');
sql insert into tb1 values ('2022-07-10 16:32:00', 2, '2'); sql insert into tb1 values ('2022-07-10 16:32:00', 2, '2');
sql insert into tb1 values ('2022-07-10 16:33:00', 3, '3'); sql insert into tb1 values ('2022-07-10 16:33:00', 3, '3');
sql insert into tb1 values ('2022-07-10 16:34:00', 4, '4'); sql insert into tb1 values ('2022-07-10 16:34:00', 4, '4');
sql select * from (select ts,TIMETRUNCATE(ts,1d),TIMETRUNCATE(ts,1h),timediff(TIMETRUNCATE(ts,1d),TIMETRUNCATE(ts,1h),1h) as td from tb1 where ts >='2022-06-01 00:00:00' and ts <='2022-11-3 23:59:59' ) t where td >12; sql select * from (select ts,TIMETRUNCATE(ts,1d),TIMETRUNCATE(ts,1h), abs(timediff(TIMETRUNCATE(ts,1d),TIMETRUNCATE(ts,1h),1h)) as td from tb1 where ts >='2022-06-01 00:00:00' and ts <='2022-11-3 23:59:59' ) t where td >12;
if $rows != 4 then if $rows != 4 then
return -1 return -1
endi endi

32
tests/system-test/1-insert/precisionNS.py
View File

@ -176,52 +176,52 @@ class TDTestCase:
def checkTimeMacro(self): def checkTimeMacro(self):
# 2 week # 2 week
val = 2 val = 2
nsval = val*7*24*60*60*1000*1000*1000 nsval = -val*7*24*60*60*1000*1000*1000
expectVal = self.childCnt * self.childRow expectVal = self.childCnt * self.childRow
sql = f"select count(ts) from st where timediff(ts - {val}w, ts1) = {nsval} " sql = f"select count(ts) from st where timediff(ts - {val}w, ts1) = {nsval} "
self.checkExpect(sql, expectVal) self.checkExpect(sql, expectVal)
# 20 day # 20 day
val = 20 val = 20
nsval = val*24*60*60*1000*1000*1000 nsval = -val*24*60*60*1000*1000*1000
uint = "d" uint = "d"
sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {nsval} " sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {nsval} "
self.checkExpect(sql, expectVal) self.checkExpect(sql, expectVal)
# 30 hour # 30 hour
val = 30 val = 30
nsval = val*60*60*1000*1000*1000 nsval = -val*60*60*1000*1000*1000
uint = "h" uint = "h"
sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {nsval} " sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {nsval} "
self.checkExpect(sql, expectVal) self.checkExpect(sql, expectVal)
# 90 minutes # 90 minutes
val = 90 val = 90
nsval = val*60*1000*1000*1000 nsval = -val*60*1000*1000*1000
uint = "m" uint = "m"
sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {nsval} " sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {nsval} "
self.checkExpect(sql, expectVal) self.checkExpect(sql, expectVal)
# 2s # 2s
val = 2 val = 2
nsval = val*1000*1000*1000 nsval = -val*1000*1000*1000
uint = "s" uint = "s"
sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {nsval} " sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {nsval} "
self.checkExpect(sql, expectVal) self.checkExpect(sql, expectVal)
# 20a # 20a
val = 5 val = 5
nsval = val*1000*1000 nsval = -val*1000*1000
uint = "a" uint = "a"
sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {nsval} " sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {nsval} "
self.checkExpect(sql, expectVal) self.checkExpect(sql, expectVal)
# 300u # 300u
val = 300 val = 300
nsval = val*1000 nsval = -val*1000
uint = "u" uint = "u"
sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {nsval} " sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {nsval} "
self.checkExpect(sql, expectVal) self.checkExpect(sql, expectVal)
# 8b # 8b
val = 8 val = 8
sql = f"select timediff(ts - {val}b, ts1) from st " sql = f"select timediff(ts1, ts - {val}b) from st "
self.checkExpect(sql, val) self.checkExpect(sql, val)
# timetruncate check # timetruncate check
@ -249,14 +249,14 @@ class TDTestCase:
timediff(ts,ts+1w,1w) timediff(ts,ts+1w,1w)
from t0 order by ts desc limit 1;''' from t0 order by ts desc limit 1;'''
tdSql.query(sql) tdSql.query(sql)
tdSql.checkData(0,1, 1) tdSql.checkData(0,1, -1)
tdSql.checkData(0,2, 1) tdSql.checkData(0,2, -1)
tdSql.checkData(0,3, 1) tdSql.checkData(0,3, -1)
tdSql.checkData(0,4, 1) tdSql.checkData(0,4, -1)
tdSql.checkData(0,5, 1) tdSql.checkData(0,5, -1)
tdSql.checkData(0,6, 1) tdSql.checkData(0,6, -1)
tdSql.checkData(0,7, 1) tdSql.checkData(0,7, -1)
tdSql.checkData(0,8, 1) tdSql.checkData(0,8, -1)
# init # init
def init(self, conn, logSql, replicaVar=1): def init(self, conn, logSql, replicaVar=1):

14
tests/system-test/1-insert/precisionUS.py
View File

@ -174,46 +174,46 @@ class TDTestCase:
def checkTimeMacro(self): def checkTimeMacro(self):
# 2 week # 2 week
val = 2 val = 2
usval = val*7*24*60*60*1000*1000 usval = -val*7*24*60*60*1000*1000
expectVal = self.childCnt * self.childRow expectVal = self.childCnt * self.childRow
sql = f"select count(ts) from st where timediff(ts - {val}w, ts1) = {usval} " sql = f"select count(ts) from st where timediff(ts - {val}w, ts1) = {usval} "
self.checkExpect(sql, expectVal) self.checkExpect(sql, expectVal)
# 20 day # 20 day
val = 20 val = 20
usval = val*24*60*60*1000*1000 usval = -val*24*60*60*1000*1000
uint = "d" uint = "d"
sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {usval} " sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {usval} "
self.checkExpect(sql, expectVal) self.checkExpect(sql, expectVal)
# 30 hour # 30 hour
val = 30 val = 30
usval = val*60*60*1000*1000 usval = -val*60*60*1000*1000
uint = "h" uint = "h"
sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {usval} " sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {usval} "
self.checkExpect(sql, expectVal) self.checkExpect(sql, expectVal)
# 90 minutes # 90 minutes
val = 90 val = 90
usval = val*60*1000*1000 usval = -val*60*1000*1000
uint = "m" uint = "m"
sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {usval} " sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {usval} "
self.checkExpect(sql, expectVal) self.checkExpect(sql, expectVal)
# 2s # 2s
val = 2 val = 2
usval = val*1000*1000 usval = -val*1000*1000
uint = "s" uint = "s"
sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {usval} " sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {usval} "
self.checkExpect(sql, expectVal) self.checkExpect(sql, expectVal)
# 20a # 20a
val = 20 val = 20
usval = val*1000 usval = -val*1000
uint = "a" uint = "a"
sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {usval} " sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {usval} "
self.checkExpect(sql, expectVal) self.checkExpect(sql, expectVal)
# 300u # 300u
val = 300 val = 300
usval = val*1 usval = -val*1
uint = "u" uint = "u"
sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {usval} " sql = f"select count(ts) from st where timediff(ts - {val}{uint}, ts1) = {usval} "
self.checkExpect(sql, expectVal) self.checkExpect(sql, expectVal)

8
tests/system-test/1-insert/test_stmt_set_tbname_tag.py
View File

@ -178,8 +178,8 @@ class TDTestCase:
queryparam[0].binary('中文字符') queryparam[0].binary('中文字符')
rows=self.stmtExe(conn,"select CHAR_LENGTH(?) from log ",queryparam) rows=self.stmtExe(conn,"select CHAR_LENGTH(?) from log ",queryparam)
tdLog.debug("assert 4th case %s"%rows) tdLog.debug("assert 4th case %s"%rows)
assert rows[0][0] == 12, '4th case is failed' assert rows[0][0] == 4, '4th case is failed'
assert rows[1][0] == 12, '4th case is failed' assert rows[1][0] == 4, '4th case is failed'
queryparam=new_bind_params(1) queryparam=new_bind_params(1)
queryparam[0].binary('123') queryparam[0].binary('123')
@ -209,8 +209,8 @@ class TDTestCase:
queryparam[0].timestamp(1626861392591112) queryparam[0].timestamp(1626861392591112)
rows=self.stmtExe(conn,"select timediff('2021-07-21 17:56:32.590111',?,1a) from log",queryparam) rows=self.stmtExe(conn,"select timediff('2021-07-21 17:56:32.590111',?,1a) from log",queryparam)
tdLog.debug("assert 7th case %s"%rows) tdLog.debug("assert 7th case %s"%rows)
assert rows[0][0] == 1, '7th case is failed' assert rows[0][0] == -1, '7th case is failed'
assert rows[1][0] == 1, '7th case is failed' assert rows[1][0] == -1, '7th case is failed'
#query: aggregate Functions #query: aggregate Functions
queryparam=new_bind_params(1) queryparam=new_bind_params(1)

8
tests/system-test/2-query/round.py
View File

@ -99,8 +99,8 @@ class TDTestCase:
# f"select round(tbname+1) from {dbname}.t1 ", # f"select round(tbname+1) from {dbname}.t1 ",
f"select round(123--123)==1 from {dbname}.t1", f"select round(123--123)==1 from {dbname}.t1",
f"select round(c1) as 'd1' from {dbname}.t1", f"select round(c1) as 'd1' from {dbname}.t1",
f"select round(c1 ,c2 ) from {dbname}.t1", # f"select round(c1 ,c2 ) from {dbname}.t1",
f"select round(c1 ,NULL) from {dbname}.t1", # f"select round(c1 ,NULL) from {dbname}.t1",
f"select round(,) from {dbname}.t1;", f"select round(,) from {dbname}.t1;",
f"select round(round(c1) ab from {dbname}.t1)", f"select round(round(c1) ab from {dbname}.t1)",
f"select round(c1) as int from {dbname}.t1", f"select round(c1) as int from {dbname}.t1",
@ -113,8 +113,8 @@ class TDTestCase:
# f"select round(tbname+1) from {dbname}.stb1 ", # f"select round(tbname+1) from {dbname}.stb1 ",
f"select round(123--123)==1 from {dbname}.stb1", f"select round(123--123)==1 from {dbname}.stb1",
f"select round(c1) as 'd1' from {dbname}.stb1", f"select round(c1) as 'd1' from {dbname}.stb1",
f"select round(c1 ,c2 ) from {dbname}.stb1", # f"select round(c1 ,c2 ) from {dbname}.stb1",
f"select round(c1 ,NULL) from {dbname}.stb1", # f"select round(c1 ,NULL) from {dbname}.stb1",
f"select round(,) from {dbname}.stb1;", f"select round(,) from {dbname}.stb1;",
f"select round(round(c1) ab from {dbname}.stb1)", f"select round(round(c1) ab from {dbname}.stb1)",
f"select round(c1) as int from {dbname}.stb1" f"select round(c1) as int from {dbname}.stb1"

2
tests/system-test/2-query/substr.py
View File

@ -79,7 +79,7 @@ class TDTestCase:
groups = ["", substr_group_having, substr_group_no_having] groups = ["", substr_group_having, substr_group_no_having]
if pos < 1: if pos < 1:
tdSql.error(f"select substr( {condition}, {pos}, {lens}) , {condition} from {tbname} ") tdSql.query(f"select substr( {condition}, {pos}, {lens}) , {condition} from {tbname} ")
break break
tdSql.query(f"select substr( {condition}, {pos}, {lens}) , {condition} from {tbname} ") tdSql.query(f"select substr( {condition}, {pos}, {lens}) , {condition} from {tbname} ")

2
tests/unit-test/test.sh
View File

@ -40,7 +40,7 @@ pgrep taosd || taosd >> /dev/null 2>&1 &
sleep 10 sleep 10
ctest -j8 ctest -E "cunit_test" -j8
ret=$? ret=$?
exit $ret exit $ret