diff --git a/docs-cn/07-develop/01-connect/index.md b/docs-cn/07-develop/01-connect/index.md index 3a15d03f93..b1857b9739 100644 --- a/docs-cn/07-develop/01-connect/index.md +++ b/docs-cn/07-develop/01-connect/index.md @@ -212,7 +212,7 @@ curl -L -o php-tdengine.tar.gz https://github.com/Yurunsoft/php-tdengine/archive && tar -xzf php-tdengine.tar.gz -C php-tdengine --strip-components=1 ``` -> 版本 `v1.0.0` 可替换为任意更新的版本，可在 Release 中查看最新版本。 +> 版本 `v1.0.2` 只是示例，可替换为任意更新的版本，可在 [TDengine PHP Connector 发布历史](https://github.com/Yurunsoft/php-tdengine/releases) 中查看可用版本。 **非 Swoole 环境：** diff --git a/docs-cn/14-reference/03-connector/java.mdx b/docs-cn/14-reference/03-connector/java.mdx index 2677571606..ddab9e5f24 100644 --- a/docs-cn/14-reference/03-connector/java.mdx +++ b/docs-cn/14-reference/03-connector/java.mdx @@ -93,8 +93,8 @@ Maven 项目中，在 pom.xml 中添加以下依赖： 可以通过下载 TDengine 的源码，自己编译最新版本的 Java connector ```shell -git clone https://github.com/taosdata/TDengine.git -cd TDengine/src/connector/jdbc +git clone https://github.com/taosdata/taos-connector-jdbc.git +cd taos-connector-jdbc mvn clean install -Dmaven.test.skip=true ``` @@ -199,6 +199,7 @@ url 中的配置参数如下： - user：登录 TDengine 用户名，默认值 'root'。 - password：用户登录密码，默认值 'taosdata'。 - batchfetch: true：在执行查询时批量拉取结果集；false：逐行拉取结果集。默认值为：false。逐行拉取结果集使用 HTTP 方式进行数据传输。从 taos-jdbcdriver-2.0.38 和 TDengine 2.4.0.12 版本开始，JDBC REST 连接增加批量拉取数据功能。taos-jdbcdriver 与 TDengine 之间通过 WebSocket 连接进行数据传输。相较于 HTTP，WebSocket 可以使 JDBC REST 连接支持大数据量查询，并提升查询性能。 +- charset: 当开启批量拉取数据时，指定解析字符串数据的字符集。 - batchErrorIgnore：true：在执行 Statement 的 executeBatch 时，如果中间有一条 SQL 执行失败，继续执行下面的 SQL 了。false：不再执行失败 SQL 后的任何语句。默认值为：false。 **注意**：部分配置项（比如：locale、timezone）在 REST 连接中不生效。 @@ -260,7 +261,7 @@ properties 中的配置参数如下： - TSDBDriver.PROPERTY_KEY_BATCH_LOAD: true：在执行查询时批量拉取结果集；false：逐行拉取结果集。默认值为：false。 - TSDBDriver.PROPERTY_KEY_BATCH_ERROR_IGNORE：true：在执行 Statement 的 executeBatch 时，如果中间有一条 SQL 执行失败，继续执行下面的 sq 了。false：不再执行失败 SQL 后的任何语句。默认值为：false。 - TSDBDriver.PROPERTY_KEY_CONFIG_DIR：仅在使用 JDBC 原生连接时生效。客户端配置文件目录路径，Linux OS 上默认值 `/etc/taos`，Windows OS 上默认值 `C:/TDengine/cfg`。 -- TSDBDriver.PROPERTY_KEY_CHARSET：仅在使用 JDBC 原生连接时生效。 客户端使用的字符集，默认值为系统字符集。 +- TSDBDriver.PROPERTY_KEY_CHARSET：客户端使用的字符集，默认值为系统字符集。 - TSDBDriver.PROPERTY_KEY_LOCALE：仅在使用 JDBC 原生连接时生效。 客户端语言环境，默认值系统当前 locale。 - TSDBDriver.PROPERTY_KEY_TIME_ZONE：仅在使用 JDBC 原生连接时生效。 客户端使用的时区，默认值为系统当前时区。 - 此外对 JDBC 原生连接，通过指定 URL 和 Properties 还可以指定其他参数，比如日志级别、SQL 长度等。更多详细配置请参考[客户端配置](/reference/config/#仅客户端适用)。 @@ -348,7 +349,7 @@ JDBC 连接器可能报错的错误码包括 3 种：JDBC driver 本身的报错 具体的错误码请参考： -- [TDengine Java Connector](https://github.com/taosdata/TDengine/blob/develop/src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBErrorNumbers.java) +- [TDengine Java Connector](https://github.com/taosdata/taos-connector-jdbc/blob/main/src/main/java/com/taosdata/jdbc/TSDBErrorNumbers.java) - [TDengine_ERROR_CODE](https://github.com/taosdata/TDengine/blob/develop/src/inc/taoserror.h) ### 通过参数绑定写入数据 diff --git a/docs-cn/14-reference/03-connector/php.mdx b/docs-cn/14-reference/03-connector/php.mdx new file mode 100644 index 0000000000..f150aed4c8 --- /dev/null +++ b/docs-cn/14-reference/03-connector/php.mdx @@ -0,0 +1,150 @@ +--- +sidebar_position: 1 +sidebar_label: PHP +title: PHP Connector +--- + +`php-tdengine` 是由社区贡献的 PHP 连接器扩展，还特别支持了 Swoole 协程化。 + +PHP 连接器依赖 TDengine 客户端驱动。 + +项目地址： + +TDengine 服务端或客户端安装后，`taos.h` 位于： + +- Linux：`/usr/local/taos/include` +- Windows：`C:\TDengine\include` + +TDengine 客户端驱动的动态库位于： + +- Linux: `/usr/local/taos/driver/libtaos.so` +- Windows: `C:\TDengine\taos.dll` + +## 支持的平台 + +* Windows、Linux、MacOS + +* PHP >= 7.4 + +* TDengine >= 2.0 + +* Swoole >= 4.8 (可选) + +## 支持的版本 + +TDengine 客户端驱动的版本号与 TDengine 服务端的版本号是一一对应的强对应关系，建议使用与 TDengine 服务端完全相同的客户端驱动。虽然低版本的客户端驱动在前三段版本号一致（即仅第四段版本号不同）的情况下也能够与高版本的服务端相兼容，但这并非推荐用法。强烈不建议使用高版本的客户端驱动访问低版本的服务端。 + +## 安装步骤 + +### 安装 TDengine 客户端驱动 + +TDengine 客户端驱动的安装请参考 [安装指南](/reference/connector#安装步骤) + +### 编译安装 php-tdengine + +**下载代码并解压：** + +```shell +curl -L -o php-tdengine.tar.gz https://github.com/Yurunsoft/php-tdengine/archive/refs/tags/v1.0.2.tar.gz \ +&& mkdir php-tdengine \ +&& tar -xzf php-tdengine.tar.gz -C php-tdengine --strip-components=1 +``` + +> 版本 `v1.0.2` 可替换为任意更新的版本，可在 [TDengine PHP Connector 发布历史](https://github.com/Yurunsoft/php-tdengine/releases)。 + +**非 Swoole 环境：** + +```shell +phpize && ./configure && make -j && make install +``` + +**手动指定 tdengine 目录：** + +```shell +phpize && ./configure --with-tdengine-dir=/usr/local/Cellar/tdengine/2.4.0.0 && make -j && make install +``` + +> `--with-tdengine-dir=` 后跟上 tdengine 目录。 +> 适用于默认找不到的情况，或者 MacOS 系统用户。 + +**Swoole 环境：** + +```shell +phpize && ./configure --enable-swoole && make -j && make install +``` + +**启用扩展：** + +方法一：在 `php.ini` 中加入 `extension=tdengine` + +方法二：运行带参数 `php -dextension=tdengine test.php` + +## 示例程序 + +本节展示了使用客户端驱动访问 TDengine 集群的常见访问方式的示例代码。 + +> 所有错误都会抛出异常: `TDengine\Exception\TDengineException` + +### 建立连接 + +
+建立连接 + +```c +{{#include docs-examples/php/connect.php}} +``` + +
+ +### 插入数据 + +
+插入数据 + +```c +{{#include docs-examples/php/insert.php}} +``` + +
+ +### 同步查询 + +
+同步查询 + +```c +{{#include docs-examples/php/query.php}} +``` + +
+ +### 参数绑定 + +
+参数绑定 + +```c +{{#include docs-examples/php/insert_stmt.php}} +``` + +
+ +## 常量 + +| 常量 | 说明 | +| ------------ | ------------ +| `TDengine\TSDB_DATA_TYPE_NULL` | null | +| `TDengine\TSDB_DATA_TYPE_BOOL` | bool | +| `TDengine\TSDB_DATA_TYPE_TINYINT` | tinyint | +| `TDengine\TSDB_DATA_TYPE_SMALLINT` | smallint | +| `TDengine\TSDB_DATA_TYPE_INT` | int | +| `TDengine\TSDB_DATA_TYPE_BIGINT` | bigint | +| `TDengine\TSDB_DATA_TYPE_FLOAT` | float | +| `TDengine\TSDB_DATA_TYPE_DOUBLE` | double | +| `TDengine\TSDB_DATA_TYPE_BINARY` | binary | +| `TDengine\TSDB_DATA_TYPE_TIMESTAMP` | timestamp | +| `TDengine\TSDB_DATA_TYPE_NCHAR` | nchar | +| `TDengine\TSDB_DATA_TYPE_UTINYINT` | utinyint | +| `TDengine\TSDB_DATA_TYPE_USMALLINT` | usmallint | +| `TDengine\TSDB_DATA_TYPE_UINT` | uint | +| `TDengine\TSDB_DATA_TYPE_UBIGINT` | ubigint | diff --git a/docs-cn/14-reference/03-connector/python.mdx b/docs-cn/14-reference/03-connector/python.mdx index 6608fb7bd2..828e0a4abb 100644 --- a/docs-cn/14-reference/03-connector/python.mdx +++ b/docs-cn/14-reference/03-connector/python.mdx @@ -199,10 +199,9 @@ curl -u root:taosdata http://:/rest/sql -d "select server_version()" `connect()` 函数的所有参数都是可选的关键字参数。下面是连接参数的具体说明： -- `host`： 要连接的主机。默认是 localhost。 +- `url`： taosAdapter REST 服务的 URL。默认是 。 - `user`： TDenigne 用户名。默认是 root。 - `password`： TDeingine 用户密码。默认是 taosdata。 -- `port`: taosAdapter REST 服务监听端口。默认是 6041. - `timeout`: HTTP 请求超时时间。单位为秒。默认为 `socket._GLOBAL_DEFAULT_TIMEOUT`。 一般无需配置。 diff --git a/docs-cn/27-train-faq/01-faq.md b/docs-cn/27-train-faq/01-faq.md index f298d7e14d..5b57e345c8 100644 --- a/docs-cn/27-train-faq/01-faq.md +++ b/docs-cn/27-train-faq/01-faq.md @@ -74,7 +74,7 @@ title: 常见问题及反馈 检查服务器侧 TCP 端口连接是否工作：`nc -l {port}` 检查客户端侧 TCP 端口连接是否工作：`nc {hostIP} {port}` - - Windows 系统请使用 PowerShell 命令 Net-TestConnection -ComputerName {fqdn} -Port {port} 检测服务段端口是否访问 + - Windows 系统请使用 PowerShell 命令 Test-NetConnection -ComputerName {fqdn} -Port {port} 检测服务段端口是否访问 10. 也可以使用 taos 程序内嵌的网络连通检测功能，来验证服务器和客户端之间指定的端口连接是否通畅（包括 TCP 和 UDP）：[TDengine 内嵌网络检测工具使用指南](https://www.taosdata.com/blog/2020/09/08/1816.html)。 diff --git a/docs-en/07-develop/01-connect/index.md b/docs-en/07-develop/01-connect/index.md index b9217b828d..720f8e2384 100644 --- a/docs-en/07-develop/01-connect/index.md +++ b/docs-en/07-develop/01-connect/index.md @@ -19,7 +19,7 @@ import InstallOnLinux from "../../14-reference/03-connector/\_windows_install.md import VerifyLinux from "../../14-reference/03-connector/\_verify_linux.mdx"; import VerifyWindows from "../../14-reference/03-connector/\_verify_windows.mdx"; -Any application programs running on any kind of platform can access TDengine through the REST API provided by TDengine. For details, please refer to [REST API](/reference/rest-api/). Additionally, application programs can use the connectors of multiple programming languages including C/C++, Java, Python, Go, Node.js, C#, and Rust to access TDengine. This chapter describes how to establish a connection to TDengine and briefly introduces how to install and use connectors. For details about the connectors, please refer to [Connectors](/reference/connector/) +Any application programs running on any kind of platform can access TDengine through the REST API provided by TDengine. For details, please refer to [REST API](/reference/rest-api/). Additionally, application programs can use the connectors of multiple programming languages including C/C++, Java, Python, Go, Node.js, C#, Rust to access TDengine. This chapter describes how to establish a connection to TDengine and briefly introduces how to install and use connectors. TDengine community also provides connectors in LUA and PHP languages. For details about the connectors, please refer to [Connectors](/reference/connector/). ## Establish Connection @@ -200,6 +200,46 @@ install.packages("RJDBC") If the client driver (taosc) is already installed, then the C connector is already available.
+ + + +**Download Source Code Package and Unzip：** + +```shell +curl -L -o php-tdengine.tar.gz https://github.com/Yurunsoft/php-tdengine/archive/refs/tags/v1.0.2.tar.gz \ +&& mkdir php-tdengine \ +&& tar -xzf php-tdengine.tar.gz -C php-tdengine --strip-components=1 +``` + +> Version number `v1.0.2` is only for example, it can be replaced to any newer version, please check available version from [TDengine PHP Connector Releases](https://github.com/Yurunsoft/php-tdengine/releases). + +**Non-Swoole Environment：** + +```shell +phpize && ./configure && make -j && make install +``` + +**Specify TDengine Location：** + +```shell +phpize && ./configure --with-tdengine-dir=/usr/local/Cellar/tdengine/2.4.0.0 && make -j && make install +``` + +> `--with-tdengine-dir=` is followed by the TDengine installation location. +> This way is useful in case TDengine location can't be found automatically or macOS. + +**Swoole Environment：** + +```shell +phpize && ./configure --enable-swoole && make -j && make install +``` + +**Enable The Extension:** + +Option One: Add `extension=tdengine` in `php.ini` + +Option Two: Specify the extension on CLI `php -d extension=tdengine test.php` + diff --git a/docs-en/14-reference/03-connector/cpp.mdx b/docs-en/14-reference/03-connector/cpp.mdx index d549413012..e0cdf2bf2c 100644 --- a/docs-en/14-reference/03-connector/cpp.mdx +++ b/docs-en/14-reference/03-connector/cpp.mdx @@ -26,7 +26,7 @@ Please refer to [list of supported platforms](/reference/connector#supported-pla ## Supported versions -The version number of the TDengine client driver and the version number of the TDengine server should be the same. A lower version of the client driver is compatible with a higher version of the server, if the first three version numbers are the same (i.e., only the fourth version number is different). For e.g. if the client version is x.y.z.1 and the server version is x.y.z.2 the client and server are compatible. But in general we do not recommend using a lower client version with a newer server version. It is also strongly discouraged to use a higher version of the client driver to access a lower version of the TDengine server. +The version number of the TDengine client driver and the version number of the TDengine server should be same. A lower version of the client driver is compatible with a higher version of the server, if the first three version numbers are the same (i.e., only the fourth version number is different). For e.g. if the client version is x.y.z.1 and the server version is x.y.z.2 the client and server are compatible. But in general we do not recommend using a lower client version with a newer server version. It is also strongly discouraged to use a higher version of the client driver to access a lower version of the TDengine server. ## Installation steps diff --git a/docs-en/14-reference/03-connector/java.mdx b/docs-en/14-reference/03-connector/java.mdx index 33d715c2e2..6c40f753be 100644 --- a/docs-en/14-reference/03-connector/java.mdx +++ b/docs-en/14-reference/03-connector/java.mdx @@ -42,18 +42,18 @@ Please refer to [Version Support List](/reference/connector#version-support). TDengine currently supports timestamp, number, character, Boolean type, and the corresponding type conversion with Java is as follows: | TDengine DataType | JDBCType (driver version < 2.0.24) | JDBCType (driver version > = 2.0.24) | -| ----------------- | --------------------------------- | ---------------------------------- | -| TIMESTAMP | java.lang.Long | java.sql.Timestamp | -| INT | java.lang.Integer | java.lang.Integer | -| BIGINT | java.lang.Long | java.lang.Long | -| FLOAT | java.lang.Float | java.lang.Float | -| DOUBLE | java.lang.Double | java.lang.Double | -| SMALLINT | java.lang.Short | java.lang.Short | -| TINYINT | java.lang.Byte | java.lang.Byte | -| BOOL | java.lang.Boolean | java.lang.Boolean | -| BINARY | java.lang.String | byte array | -| NCHAR | java.lang.String | java.lang.String | -| JSON | - | java.lang.String | +| ----------------- | ---------------------------------- | ------------------------------------ | +| TIMESTAMP | java.lang.Long | java.sql.Timestamp | +| INT | java.lang.Integer | java.lang.Integer | +| BIGINT | java.lang.Long | java.lang.Long | +| FLOAT | java.lang.Float | java.lang.Float | +| DOUBLE | java.lang.Double | java.lang.Double | +| SMALLINT | java.lang.Short | java.lang.Short | +| TINYINT | java.lang.Byte | java.lang.Byte | +| BOOL | java.lang.Boolean | java.lang.Boolean | +| BINARY | java.lang.String | byte array | +| NCHAR | java.lang.String | java.lang.String | +| JSON | - | java.lang.String | **Note**: Only TAG supports JSON types @@ -91,8 +91,8 @@ Add following dependency in the `pom.xml` file of your Maven project: You can build Java connector from source code after cloning the TDengine project: ```shell -git clone https://github.com/taosdata/TDengine.git -cd TDengine/src/connector/jdbc +git clone https://github.com/taosdata/taos-connector-jdbc.git +cd taos-connector-jdbc mvn clean install -Dmaven.test.skip=true ``` @@ -197,6 +197,7 @@ The configuration parameters in the URL are as follows. - user: Login TDengine user name, default value 'root'. - password: user login password, default value 'taosdata'. - batchfetch: true: pull the result set in batch when executing the query; false: pull the result set row by row. The default value is false. batchfetch uses HTTP for data transfer. The JDBC REST connection supports bulk data pulling function in taos-jdbcdriver-2.0.38 and TDengine 2.4.0.12 and later versions. taos-jdbcdriver and TDengine transfer data via WebSocket connection. Compared with HTTP, WebSocket enables JDBC REST connection to support large data volume querying and improve query performance. +- charset: specify the charset to parse the string, this parameter is valid only when set batchfetch to true. - batchErrorIgnore: true: when executing executeBatch of Statement, if one SQL execution fails in the middle, continue to execute the following SQL. false: no longer execute any statement after the failed SQL. The default value is: false. **Note**: Some configuration items (e.g., locale, timezone) do not work in the REST connection. @@ -258,10 +259,10 @@ The configuration parameters in properties are as follows. - TSDBDriver.PROPERTY_KEY_BATCH_LOAD: true: pull the result set in batch when executing query; false: pull the result set row by row. The default value is: false. - TSDBDriver.PROPERTY_KEY_BATCH_ERROR_IGNORE: true: when executing executeBatch of Statement, if there is a SQL execution failure in the middle, continue to execute the following sq. false: no longer execute any statement after the failed SQL. The default value is: false. - TSDBDriver.PROPERTY_KEY_CONFIG_DIR: Only works when using JDBC native connection. Client configuration file directory path, default value `/etc/taos` on Linux OS, default value `C:/TDengine/cfg` on Windows OS. -- TSDBDriver.PROPERTY_KEY_CHARSET: takes effect only when using JDBC native connection. In the character set used by the client, the default value is the system character set. +- TSDBDriver.PROPERTY_KEY_CHARSET: In the character set used by the client, the default value is the system character set. - TSDBDriver.PROPERTY_KEY_LOCALE: this only takes effect when using JDBC native connection. Client language environment, the default value is system current locale. - TSDBDriver.PROPERTY_KEY_TIME_ZONE: only takes effect when using JDBC native connection. In the time zone used by the client, the default value is the system's current time zone. -For JDBC native connections, you can specify other parameters, such as log level, SQL length, etc., by specifying URL and Properties. For more detailed configuration, please refer to [Client Configuration](/reference/config/#Client-Only). + For JDBC native connections, you can specify other parameters, such as log level, SQL length, etc., by specifying URL and Properties. For more detailed configuration, please refer to [Client Configuration](/reference/config/#Client-Only). ### Priority of configuration parameters @@ -350,7 +351,7 @@ There are three types of error codes that the JDBC connector can report: For specific error codes, please refer to. -- [TDengine Java Connector](https://github.com/taosdata/TDengine/blob/develop/src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBErrorNumbers.java) +- [TDengine Java Connector](https://github.com/taosdata/taos-connector-jdbc/blob/main/src/main/java/com/taosdata/jdbc/TSDBErrorNumbers.java) - [TDengine_ERROR_CODE](https://github.com/taosdata/TDengine/blob/develop/src/inc/taoserror.h) ### Writing data via parameter binding @@ -808,11 +809,11 @@ Please refer to: [JDBC example](https://github.com/taosdata/TDengine/tree/develo ## Recent update logs -| taos-jdbcdriver version | major changes | -| :------------------: | :----------------------------: | -| 2.0.38 | JDBC REST connections add bulk pull function | -| 2.0.37 | Added support for json tags | -| 2.0.36 | Add support for schemaless writing | +| taos-jdbcdriver version | major changes | +| :---------------------: | :------------------------------------------: | +| 2.0.38 | JDBC REST connections add bulk pull function | +| 2.0.37 | Added support for json tags | +| 2.0.36 | Add support for schemaless writing | ## Frequently Asked Questions diff --git a/docs-en/14-reference/03-connector/php.mdx b/docs-en/14-reference/03-connector/php.mdx new file mode 100644 index 0000000000..839a5c8c3c --- /dev/null +++ b/docs-en/14-reference/03-connector/php.mdx @@ -0,0 +1,150 @@ +--- +sidebar_position: 1 +sidebar_label: PHP +title: PHP Connector +--- + +`php-tdengine` is the TDengine PHP connector provided by TDengine community. In particular, it supports Swoole coroutine. + +PHP Connector relies on TDengine client driver. + +Project Repository： + +After TDengine client or server is installed, `taos.h` is located at: + +- Linux：`/usr/local/taos/include` +- Windows：`C:\TDengine\include` + +TDengine client driver is located at: + +- Linux: `/usr/local/taos/driver/libtaos.so` +- Windows: `C:\TDengine\taos.dll` + +## Supported Platforms + +- Windows、Linux、MacOS + +- PHP >= 7.4 + +- TDengine >= 2.0 + +- Swoole >= 4.8 (Optional) + +## Supported Versions + +Because the version of TDengine client driver is tightly associated with that of TDengine server, it's strongly suggested to use the client driver of same version as TDengine server, even though the client driver can work with TDengine server if the first 3 sections of the versions are same. + +## Installation + +### Install TDengine Client Driver + +Regarding how to install TDengine client driver please refer to [Install Client Driver](/reference/connector#installation-steps) + +### Install php-tdengine + +**Download Source Code Package and Unzip：** + +```shell +curl -L -o php-tdengine.tar.gz https://github.com/Yurunsoft/php-tdengine/archive/refs/tags/v1.0.2.tar.gz \ +&& mkdir php-tdengine \ +&& tar -xzf php-tdengine.tar.gz -C php-tdengine --strip-components=1 +``` + +> Version number `v1.0.2` is only for example, it can be replaced to any newer version, please find available versions in [TDengine PHP Connector Releases](https://github.com/Yurunsoft/php-tdengine/releases). + +**Non-Swoole Environment：** + +```shell +phpize && ./configure && make -j && make install +``` + +**Specify TDengine location：** + +```shell +phpize && ./configure --with-tdengine-dir=/usr/local/Cellar/tdengine/2.4.0.0 && make -j && make install +``` + +> `--with-tdengine-dir=` is followed by TDengine location. +> It's useful in case TDengine installatio location can't be found automatically or MacOS. + +**Swoole Environment：** + +```shell +phpize && ./configure --enable-swoole && make -j && make install +``` + +**Enable Extension:** + +Option One: Add `extension=tdengine` in `php.ini`. + +Option Two: Use CLI `php -dextension=tdengine test.php`. + +## Sample Programs + +In this section a few sample programs which use TDengine PHP connector to access TDengine cluster are demonstrated. + +> Any error would throw exception: `TDengine\Exception\TDengineException` + +### Establish Conection + +
+Establish Connection + +```c +{{#include docs-examples/php/connect.php}} +``` + +
+ +### Insert Data + +
+Insert Data + +```c +{{#include docs-examples/php/insert.php}} +``` + +
+ +### Synchronous Query + +
+Synchronous Query + +```c +{{#include docs-examples/php/query.php}} +``` + +
+ +### Parameter Binding + +
+Parameter Binding + +```c +{{#include docs-examples/php/insert_stmt.php}} +``` + +
+ +## Constants + +| Constant | Description | +| ----------------------------------- | ----------- | +| `TDengine\TSDB_DATA_TYPE_NULL` | null | +| `TDengine\TSDB_DATA_TYPE_BOOL` | bool | +| `TDengine\TSDB_DATA_TYPE_TINYINT` | tinyint | +| `TDengine\TSDB_DATA_TYPE_SMALLINT` | smallint | +| `TDengine\TSDB_DATA_TYPE_INT` | int | +| `TDengine\TSDB_DATA_TYPE_BIGINT` | bigint | +| `TDengine\TSDB_DATA_TYPE_FLOAT` | float | +| `TDengine\TSDB_DATA_TYPE_DOUBLE` | double | +| `TDengine\TSDB_DATA_TYPE_BINARY` | binary | +| `TDengine\TSDB_DATA_TYPE_TIMESTAMP` | timestamp | +| `TDengine\TSDB_DATA_TYPE_NCHAR` | nchar | +| `TDengine\TSDB_DATA_TYPE_UTINYINT` | utinyint | +| `TDengine\TSDB_DATA_TYPE_USMALLINT` | usmallint | +| `TDengine\TSDB_DATA_TYPE_UINT` | uint | +| `TDengine\TSDB_DATA_TYPE_UBIGINT` | ubigint | diff --git a/docs-en/14-reference/03-connector/python.mdx b/docs-en/14-reference/03-connector/python.mdx index 69eec2388d..58b94f13ae 100644 --- a/docs-en/14-reference/03-connector/python.mdx +++ b/docs-en/14-reference/03-connector/python.mdx @@ -199,10 +199,9 @@ The `connect()` function returns a `taos.TaosConnection` instance. In client-sid All arguments to the `connect()` function are optional keyword arguments. The following are the connection parameters specified. -- `host`: The host to connect to. The default is localhost. +- `url`： The URL of taosAdapter REST service. The default is . - `user`: TDengine user name. The default is `root`. - `password`: TDengine user password. The default is `taosdata`. -- `port`: The port on which the taosAdapter REST service listens. Default is 6041. - `timeout`: HTTP request timeout in seconds. The default is `socket._GLOBAL_DEFAULT_TIMEOUT`. Usually, no configuration is needed. diff --git a/docs-en/27-train-faq/01-faq.md b/docs-en/27-train-faq/01-faq.md index e182e25b9e..c10bca1d05 100644 --- a/docs-en/27-train-faq/01-faq.md +++ b/docs-en/27-train-faq/01-faq.md @@ -45,7 +45,7 @@ When the client is unable to connect to the server, you can try the following wa Check whether a TCP port on server side is open: `nc -l {port}` Check whether a TCP port on client side is open: `nc {hostIP} {port}` - - On Windows system `Net-TestConnection -ComputerName {fqdn} -Port {port}` on PowerShell can be used to check whether the port on server side is open for access. + - On Windows system `Test-NetConnection -ComputerName {fqdn} -Port {port}` on PowerShell can be used to check whether the port on server side is open for access. 7. TDengine CLI `taos` can also be used to check network, please refer to [TDengine CLI](/reference/taos-shell). diff --git a/docs-examples/php/insert_stmt.php b/docs-examples/php/insert_stmt.php index 99a9a6aef3..c927a9b0ce 100644 --- a/docs-examples/php/insert_stmt.php +++ b/docs-examples/php/insert_stmt.php @@ -22,7 +22,7 @@ try { // set table name and tags $stmt->setTableNameTags('d1001', [ - // 支持格式同参数绑定 + // same format as parameter binding [TDengine\TSDB_DATA_TYPE_BINARY, 'California.SanFrancisco'], [TDengine\TSDB_DATA_TYPE_INT, 2], ]); diff --git a/docs-examples/python/connect_rest_examples.py b/docs-examples/python/connect_rest_examples.py index 3303eb0e19..900ec1022e 100644 --- a/docs-examples/python/connect_rest_examples.py +++ b/docs-examples/python/connect_rest_examples.py @@ -1,10 +1,9 @@ # ANCHOR: connect from taosrest import connect, TaosRestConnection, TaosRestCursor -conn: TaosRestConnection = connect(host="localhost", +conn: TaosRestConnection = connect(url="http://localhost:6041", user="root", password="taosdata", - port=6041, timeout=30) # ANCHOR_END: connect diff --git a/docs-examples/python/rest_client_example.py b/docs-examples/python/rest_client_example.py index 46d33a1d79..59c629df95 100644 --- a/docs-examples/python/rest_client_example.py +++ b/docs-examples/python/rest_client_example.py @@ -1,6 +1,6 @@ from taosrest import RestClient -client = RestClient("localhost", 6041, "root", "taosdata") +client = RestClient("http://localhost:6041", user="root", password="taosdata") res: dict = client.sql("SELECT ts, current FROM power.meters LIMIT 1") print(res) diff --git a/include/common/tcommon.h b/include/common/tcommon.h index 3d9228f05c..61286cd675 100644 --- a/include/common/tcommon.h +++ b/include/common/tcommon.h @@ -108,6 +108,7 @@ typedef struct SColumnInfoData { typedef struct SQueryTableDataCond { // STimeWindow twindow; + uint64_t suid; int32_t order; // desc|asc order to iterate the data block int32_t numOfCols; SColumnInfo* colList; diff --git a/include/common/tdatablock.h b/include/common/tdatablock.h index 8d69bb6c70..ab76cb404a 100644 --- a/include/common/tdatablock.h +++ b/include/common/tdatablock.h @@ -71,7 +71,7 @@ SEpSet getEpSet_s(SCorEpSet* pEpSet); #define colDataGetData(p1_, r_) \ ((IS_VAR_DATA_TYPE((p1_)->info.type)) ? colDataGetVarData(p1_, r_) : colDataGetNumData(p1_, r_)) -#define IS_JSON_NULL(type,data) ((type) == TSDB_DATA_TYPE_JSON && *(data) == TSDB_DATA_TYPE_NULL) +#define IS_JSON_NULL(type, data) ((type) == TSDB_DATA_TYPE_JSON && *(data) == TSDB_DATA_TYPE_NULL) static FORCE_INLINE bool colDataIsNull_s(const SColumnInfoData* pColumnInfoData, uint32_t row) { if (!pColumnInfoData->hasNull) { @@ -180,7 +180,7 @@ static FORCE_INLINE void colDataAppendDouble(SColumnInfoData* pColumnInfoData, u *(double*)p = *(double*)v; } -int32_t getJsonValueLen(const char *data); +int32_t getJsonValueLen(const char* data); int32_t colDataAppend(SColumnInfoData* pColumnInfoData, uint32_t currentRow, const char* pData, bool isNull); int32_t colDataMergeCol(SColumnInfoData* pColumnInfoData, uint32_t numOfRow1, int32_t* capacity, @@ -223,7 +223,8 @@ int32_t blockDataTrimFirstNRows(SSDataBlock* pBlock, size_t n); SSDataBlock* createOneDataBlock(const SSDataBlock* pDataBlock, bool copyData); -void blockCompressEncode(const SSDataBlock* pBlock, char* data, int32_t* dataLen, int32_t numOfCols, int8_t needCompress); +void blockCompressEncode(const SSDataBlock* pBlock, char* data, int32_t* dataLen, int32_t numOfCols, + int8_t needCompress); const char* blockCompressDecode(SSDataBlock* pBlock, int32_t numOfCols, int32_t numOfRows, const char* pData); void blockDebugShowData(const SArray* dataBlocks, const char* flag); @@ -231,6 +232,8 @@ void blockDebugShowData(const SArray* dataBlocks, const char* flag); int32_t buildSubmitReqFromDataBlock(SSubmitReq** pReq, const SArray* pDataBlocks, STSchema* pTSchema, int32_t vgId, tb_uid_t suid); +char* buildCtbNameByGroupId(const char* stbName, uint64_t groupId); + SSubmitReq* tdBlockToSubmit(const SArray* pBlocks, const STSchema* pSchema, bool createTb, int64_t suid, const char* stbFullName, int32_t vgId); diff --git a/include/common/tmsg.h b/include/common/tmsg.h index 23945cd500..4a3c4b0c3f 100644 --- a/include/common/tmsg.h +++ b/include/common/tmsg.h @@ -1315,6 +1315,31 @@ typedef struct { int32_t tSerializeSKillTransReq(void* buf, int32_t bufLen, SKillTransReq* pReq); int32_t tDeserializeSKillTransReq(void* buf, int32_t bufLen, SKillTransReq* pReq); +typedef struct { + int32_t useless; // useless +} SBalanceVgroupReq; + +int32_t tSerializeSBalanceVgroupReq(void* buf, int32_t bufLen, SBalanceVgroupReq* pReq); +int32_t tDeserializeSBalanceVgroupReq(void* buf, int32_t bufLen, SBalanceVgroupReq* pReq); + +typedef struct { + int32_t vgId1; + int32_t vgId2; +} SMergeVgroupReq; + +int32_t tSerializeSMergeVgroupReq(void* buf, int32_t bufLen, SMergeVgroupReq* pReq); +int32_t tDeserializeSMergeVgroupReq(void* buf, int32_t bufLen, SMergeVgroupReq* pReq); + +typedef struct { + int32_t vgId; + int32_t dnodeId1; + int32_t dnodeId2; + int32_t dnodeId3; +} SRedistributeVgroupReq; + +int32_t tSerializeSRedistributeVgroupReq(void* buf, int32_t bufLen, SRedistributeVgroupReq* pReq); +int32_t tDeserializeSRedistributeVgroupReq(void* buf, int32_t bufLen, SRedistributeVgroupReq* pReq); + typedef struct { char user[TSDB_USER_LEN]; char spi; @@ -2460,6 +2485,32 @@ typedef struct { int32_t tSerializeSUserIndexRsp(void* buf, int32_t bufLen, const SUserIndexRsp* pRsp); int32_t tDeserializeSUserIndexRsp(void* buf, int32_t bufLen, SUserIndexRsp* pRsp); +typedef struct { + char tbFName[TSDB_TABLE_FNAME_LEN]; +} STableIndexReq; + +int32_t tSerializeSTableIndexReq(void* buf, int32_t bufLen, STableIndexReq* pReq); +int32_t tDeserializeSTableIndexReq(void* buf, int32_t bufLen, STableIndexReq* pReq); + +typedef struct { + int8_t intervalUnit; + int8_t slidingUnit; + int64_t interval; + int64_t offset; + int64_t sliding; + int64_t dstTbUid; + int32_t dstVgId; // for stream + char* expr; +} STableIndexInfo; + +typedef struct { + SArray* pIndex; +} STableIndexRsp; + +int32_t tSerializeSTableIndexRsp(void* buf, int32_t bufLen, const STableIndexRsp* pRsp); +int32_t tDeserializeSTableIndexRsp(void* buf, int32_t bufLen, STableIndexRsp* pRsp); + + typedef struct { int8_t mqMsgType; int32_t code; diff --git a/include/common/tmsgdef.h b/include/common/tmsgdef.h index 8a811774b2..e922df64b3 100644 --- a/include/common/tmsgdef.h +++ b/include/common/tmsgdef.h @@ -130,6 +130,7 @@ enum { TD_DEF_MSG_TYPE(TDMT_MND_CREATE_INDEX, "create-index", NULL, NULL) TD_DEF_MSG_TYPE(TDMT_MND_DROP_INDEX, "drop-index", NULL, NULL) TD_DEF_MSG_TYPE(TDMT_MND_GET_INDEX, "get-index", NULL, NULL) + TD_DEF_MSG_TYPE(TDMT_MND_GET_TABLE_INDEX, "get-table-index", NULL, NULL) TD_DEF_MSG_TYPE(TDMT_MND_CREATE_TOPIC, "create-topic", SMCreateTopicReq, SMCreateTopicRsp) TD_DEF_MSG_TYPE(TDMT_MND_ALTER_TOPIC, "alter-topic", NULL, NULL) TD_DEF_MSG_TYPE(TDMT_MND_DROP_TOPIC, "drop-topic", NULL, NULL) @@ -153,6 +154,9 @@ enum { TD_DEF_MSG_TYPE(TDMT_MND_GRANT, "grant", NULL, NULL) TD_DEF_MSG_TYPE(TDMT_MND_AUTH, "auth", NULL, NULL) TD_DEF_MSG_TYPE(TDMT_MND_APPLY_MSG, "mnode-apply", NULL, NULL) + TD_DEF_MSG_TYPE(TDMT_MND_BALANCE_VGROUP, "balance-vgroup", NULL, NULL) + TD_DEF_MSG_TYPE(TDMT_MND_MERGE_VGROUP, "merge-vgroup", NULL, NULL) + TD_DEF_MSG_TYPE(TDMT_MND_REDISTRIBUTE_VGROUP, "redistribute-vgroup", NULL, NULL) TD_NEW_MSG_SEG(TDMT_VND_MSG) TD_DEF_MSG_TYPE(TDMT_VND_SUBMIT, "submit", SSubmitReq, SSubmitRsp) @@ -169,10 +173,6 @@ enum { TD_DEF_MSG_TYPE(TDMT_VND_CREATE_STB, "vnode-create-stb", SVCreateStbReq, NULL) TD_DEF_MSG_TYPE(TDMT_VND_ALTER_STB, "vnode-alter-stb", NULL, NULL) TD_DEF_MSG_TYPE(TDMT_VND_DROP_STB, "vnode-drop-stb", SVDropStbReq, NULL) - TD_DEF_MSG_TYPE(TDMT_VND_MQ_CONSUME, "vnode-mq-consume", NULL, NULL) - TD_DEF_MSG_TYPE(TDMT_VND_MQ_QUERY, "vnode-mq-query", NULL, NULL) - TD_DEF_MSG_TYPE(TDMT_VND_MQ_CONNECT, "vnode-mq-connect", NULL, NULL) - TD_DEF_MSG_TYPE(TDMT_VND_MQ_DISCONNECT, "vnode-mq-disconnect", NULL, NULL) TD_DEF_MSG_TYPE(TDMT_VND_MQ_VG_CHANGE, "vnode-mq-vg-change", SMqRebVgReq, SMqRebVgRsp) TD_DEF_MSG_TYPE(TDMT_VND_MQ_VG_DELETE, "vnode-mq-vg-delete", SMqVDeleteReq, SMqVDeleteRsp) TD_DEF_MSG_TYPE(TDMT_VND_CANCEL_TASK, "vnode-cancel-task", NULL, NULL) @@ -183,12 +183,8 @@ enum { TD_DEF_MSG_TYPE(TDMT_VND_EXPLAIN, "vnode-explain", NULL, NULL) TD_DEF_MSG_TYPE(TDMT_VND_SUBSCRIBE, "vnode-subscribe", SMVSubscribeReq, SMVSubscribeRsp) TD_DEF_MSG_TYPE(TDMT_VND_CONSUME, "vnode-consume", SMqPollReq, SMqDataBlkRsp) - TD_DEF_MSG_TYPE(TDMT_VND_TASK_DEPLOY, "vnode-task-deploy", SStreamTaskDeployReq, SStreamTaskDeployRsp) TD_DEF_MSG_TYPE(TDMT_VND_STREAM_TRIGGER, "vnode-stream-trigger", NULL, NULL) - TD_DEF_MSG_TYPE(TDMT_VND_TASK_RUN, "vnode-stream-task-run", NULL, NULL) - TD_DEF_MSG_TYPE(TDMT_VND_TASK_DISPATCH, "vnode-stream-task-dispatch", NULL, NULL) - TD_DEF_MSG_TYPE(TDMT_VND_TASK_DISPATCH_WRITE, "vnode-stream-task-dispatch-write", NULL, NULL) - TD_DEF_MSG_TYPE(TDMT_VND_TASK_RECOVER, "vnode-stream-task-recover", NULL, NULL) + TD_DEF_MSG_TYPE(TDMT_VND_STREAM_DISPATCH_WRITE, "vnode-stream-task-dispatch-write", NULL, NULL) TD_DEF_MSG_TYPE(TDMT_VND_CREATE_SMA, "vnode-create-sma", NULL, NULL) TD_DEF_MSG_TYPE(TDMT_VND_CANCEL_SMA, "vnode-cancel-sma", NULL, NULL) TD_DEF_MSG_TYPE(TDMT_VND_DROP_SMA, "vnode-drop-sma", NULL, NULL) @@ -202,11 +198,12 @@ enum { TD_NEW_MSG_SEG(TDMT_QND_MSG) - TD_NEW_MSG_SEG(TDMT_SND_MSG) - TD_DEF_MSG_TYPE(TDMT_SND_TASK_DEPLOY, "snode-task-deploy", SStreamTaskDeployReq, SStreamTaskDeployRsp) - TD_DEF_MSG_TYPE(TDMT_SND_TASK_RUN, "snode-stream-task-run", NULL, NULL) - TD_DEF_MSG_TYPE(TDMT_SND_TASK_DISPATCH, "snode-stream-task-dispatch", NULL, NULL) - TD_DEF_MSG_TYPE(TDMT_SND_TASK_RECOVER, "snode-stream-task-recover", NULL, NULL) + //shared by snode and vnode + TD_NEW_MSG_SEG(TDMT_STREAM_MSG) + TD_DEF_MSG_TYPE(TDMT_STREAM_TASK_DEPLOY, "stream-task-deploy", SStreamTaskDeployReq, SStreamTaskDeployRsp) + TD_DEF_MSG_TYPE(TDMT_STREAM_TASK_RUN, "stream-task-run", NULL, NULL) + TD_DEF_MSG_TYPE(TDMT_STREAM_TASK_DISPATCH, "stream-task-dispatch", NULL, NULL) + TD_DEF_MSG_TYPE(TDMT_STREAM_TASK_RECOVER, "stream-task-recover", NULL, NULL) TD_NEW_MSG_SEG(TDMT_SCH_MSG) TD_DEF_MSG_TYPE(TDMT_SCH_LINK_BROKEN, "link-broken", NULL, NULL) diff --git a/include/common/ttokendef.h b/include/common/ttokendef.h index d28f87cb6a..715da5cc99 100644 --- a/include/common/ttokendef.h +++ b/include/common/ttokendef.h @@ -188,10 +188,10 @@ #define TK_KILL 170 #define TK_CONNECTION 171 #define TK_TRANSACTION 172 -#define TK_MERGE 173 +#define TK_BALANCE 173 #define TK_VGROUP 174 -#define TK_REDISTRIBUTE 175 -#define TK_SPLIT 176 +#define TK_MERGE 175 +#define TK_REDISTRIBUTE 176 #define TK_SYNCDB 177 #define TK_DELETE 178 #define TK_NULL 179 diff --git a/include/libs/catalog/catalog.h b/include/libs/catalog/catalog.h index ee237741c3..21fee3cc3e 100644 --- a/include/libs/catalog/catalog.h +++ b/include/libs/catalog/catalog.h @@ -272,6 +272,8 @@ int32_t catalogGetDBCfg(SCatalog* pCtg, void *pRpc, const SEpSet* pMgmtEps, cons int32_t catalogGetIndexMeta(SCatalog* pCtg, void *pRpc, const SEpSet* pMgmtEps, const char* indexName, SIndexInfo* pInfo); +int32_t catalogGetTableIndex(SCatalog* pCtg, void *pTrans, const SEpSet* pMgmtEps, const char* tbFName, SArray** pRes); + int32_t catalogGetUdfInfo(SCatalog* pCtg, void *pRpc, const SEpSet* pMgmtEps, const char* funcName, SFuncInfo* pInfo); int32_t catalogChkAuth(SCatalog* pCtg, void *pRpc, const SEpSet* pMgmtEps, const char* user, const char* dbFName, AUTH_TYPE type, bool *pass); @@ -280,7 +282,6 @@ int32_t catalogUpdateUserAuthInfo(SCatalog* pCtg, SGetUserAuthRsp* pAuth); int32_t catalogUpdateVgEpSet(SCatalog* pCtg, const char* dbFName, int32_t vgId, SEpSet *epSet); - int32_t ctgdLaunchAsyncCall(SCatalog* pCtg, void *pTrans, const SEpSet* pMgmtEps, uint64_t reqId, bool forceUpdate); diff --git a/include/libs/nodes/cmdnodes.h b/include/libs/nodes/cmdnodes.h index 82924bef3f..c267c89384 100644 --- a/include/libs/nodes/cmdnodes.h +++ b/include/libs/nodes/cmdnodes.h @@ -28,6 +28,14 @@ extern "C" { #define DESCRIBE_RESULT_TYPE_LEN (20 + VARSTR_HEADER_SIZE) #define DESCRIBE_RESULT_NOTE_LEN (8 + VARSTR_HEADER_SIZE) +#define PRIVILEGE_TYPE_MASK(n) (1 << n) + +#define PRIVILEGE_TYPE_ALL PRIVILEGE_TYPE_MASK(0) +#define PRIVILEGE_TYPE_READ PRIVILEGE_TYPE_MASK(1) +#define PRIVILEGE_TYPE_WRITE PRIVILEGE_TYPE_MASK(2) + +#define PRIVILEGE_TYPE_TEST_MASK(val, mask) (((val) & (mask)) != 0) + typedef struct SDatabaseOptions { ENodeType type; int32_t buffer; @@ -316,14 +324,6 @@ typedef struct SDropFunctionStmt { bool ignoreNotExists; } SDropFunctionStmt; -#define PRIVILEGE_TYPE_MASK(n) (1 << n) - -#define PRIVILEGE_TYPE_ALL PRIVILEGE_TYPE_MASK(0) -#define PRIVILEGE_TYPE_READ PRIVILEGE_TYPE_MASK(1) -#define PRIVILEGE_TYPE_WRITE PRIVILEGE_TYPE_MASK(2) - -#define PRIVILEGE_TYPE_TEST_MASK(val, mask) (((val) & (mask)) != 0) - typedef struct SGrantStmt { ENodeType type; char userName[TSDB_USER_LEN]; @@ -333,6 +333,30 @@ typedef struct SGrantStmt { typedef SGrantStmt SRevokeStmt; +typedef struct SBalanceVgroupStmt { + ENodeType type; +} SBalanceVgroupStmt; + +typedef struct SMergeVgroupStmt { + ENodeType type; + int32_t vgId1; + int32_t vgId2; +} SMergeVgroupStmt; + +typedef struct SRedistributeVgroupStmt { + ENodeType type; + int32_t vgId; + int32_t dnodeId1; + int32_t dnodeId2; + int32_t dnodeId3; + SNodeList* pDnodes; +} SRedistributeVgroupStmt; + +typedef struct SSplitVgroupStmt { + ENodeType type; + int32_t vgId; +} SSplitVgroupStmt; + #ifdef __cplusplus } #endif diff --git a/include/libs/nodes/nodes.h b/include/libs/nodes/nodes.h index e3ed8e1d67..36e9443ff9 100644 --- a/include/libs/nodes/nodes.h +++ b/include/libs/nodes/nodes.h @@ -140,6 +140,7 @@ typedef enum ENodeType { QUERY_NODE_DROP_FUNCTION_STMT, QUERY_NODE_CREATE_STREAM_STMT, QUERY_NODE_DROP_STREAM_STMT, + QUERY_NODE_BALANCE_VGROUP_STMT, QUERY_NODE_MERGE_VGROUP_STMT, QUERY_NODE_REDISTRIBUTE_VGROUP_STMT, QUERY_NODE_SPLIT_VGROUP_STMT, diff --git a/include/libs/stream/tstream.h b/include/libs/stream/tstream.h index db8d3ac033..ddd7e1cd02 100644 --- a/include/libs/stream/tstream.h +++ b/include/libs/stream/tstream.h @@ -131,40 +131,13 @@ static FORCE_INLINE void* streamQueueNextItem(SStreamQueue* queue) { SStreamDataSubmit* streamDataSubmitNew(SSubmitReq* pReq); -static FORCE_INLINE void streamDataSubmitRefDec(SStreamDataSubmit* pDataSubmit) { - int32_t ref = atomic_sub_fetch_32(pDataSubmit->dataRef, 1); - ASSERT(ref >= 0); - if (ref == 0) { - taosMemoryFree(pDataSubmit->data); - taosMemoryFree(pDataSubmit->dataRef); - } -} +void streamDataSubmitRefDec(SStreamDataSubmit* pDataSubmit); SStreamDataSubmit* streamSubmitRefClone(SStreamDataSubmit* pSubmit); #if 0 int32_t streamDataBlockEncode(void** buf, const SStreamDataBlock* pOutput); void* streamDataBlockDecode(const void* buf, SStreamDataBlock* pInput); - -static FORCE_INLINE int32_t streamEnqueue1(SStreamQueue* queue, SStreamQueueItem* pItem) { - int8_t inputStatus = atomic_load_8(&queue->enqueueStatus); - if (inputStatus == TASK_INPUT_STATUS__NORMAL) { - if (pItem->type == STREAM_INPUT__DATA_SUBMIT) { - SStreamDataSubmit* pSubmitClone = streamSubmitRefClone((SStreamDataSubmit*)pItem); - if (pSubmitClone == NULL) { - atomic_store_8(&queue->enqueueStatus, TASK_INPUT_STATUS__FAILED); - return -1; - } - taosWriteQitem(queue->queue, pSubmitClone); - } else if (pItem->type == STREAM_INPUT__DATA_BLOCK) { - taosWriteQitem(queue->queue, pItem); - } else if (pItem->type == STREAM_INPUT__CHECKPOINT) { - taosWriteQitem(queue->queue, pItem); - } - return 0; - } - return 0; -} #endif typedef struct { @@ -209,6 +182,7 @@ typedef void FSmaSink(void* vnode, int64_t smaId, const SArray* data); typedef struct { int64_t smaId; // following are not applicable to encoder and decoder + void* vnode; FSmaSink* smaSink; } STaskSinkSma; @@ -290,7 +264,7 @@ struct SStreamTask { SStreamQueue* outputQueue; // application storage - void* ahandle; + // void* ahandle; }; SStreamTask* tNewSStreamTask(int64_t streamId, int32_t childId); @@ -331,7 +305,16 @@ static FORCE_INLINE void streamTaskInputFail(SStreamTask* pTask) { } static FORCE_INLINE int32_t streamTaskOutput(SStreamTask* pTask, SStreamDataBlock* pBlock) { - taosWriteQitem(pTask->outputQueue->queue, pBlock); + if (pTask->sinkType == TASK_SINK__TABLE) { + ASSERT(pTask->dispatchType == TASK_DISPATCH__NONE); + pTask->tbSink.tbSinkFunc(pTask, pTask->tbSink.vnode, 0, pBlock->blocks); + } else if (pTask->sinkType == TASK_SINK__SMA) { + ASSERT(pTask->dispatchType == TASK_DISPATCH__NONE); + pTask->smaSink.smaSink(pTask->smaSink.vnode, pTask->smaSink.smaId, pBlock->blocks); + } else { + ASSERT(pTask->dispatchType != TASK_DISPATCH__NONE); + taosWriteQitem(pTask->outputQueue->queue, pBlock); + } return 0; } @@ -339,26 +322,11 @@ typedef struct { int32_t reserved; } SStreamTaskDeployRsp; -typedef struct { - // SMsgHead head; - int64_t streamId; - int32_t taskId; - SArray* data; // SArray -} SStreamTaskExecReq; - typedef struct { // SMsgHead head; SStreamTask* task; } SStreamTaskDeployReq; -int32_t tEncodeSStreamTaskExecReq(void** buf, const SStreamTaskExecReq* pReq); -void* tDecodeSStreamTaskExecReq(const void* buf, SStreamTaskExecReq* pReq); -void tFreeSStreamTaskExecReq(SStreamTaskExecReq* pReq); - -typedef struct { - int32_t reserved; -} SStreamTaskExecRsp; - typedef struct { SMsgHead head; int64_t streamId; diff --git a/include/util/taoserror.h b/include/util/taoserror.h index 842c8a44e1..5a83d1525e 100644 --- a/include/util/taoserror.h +++ b/include/util/taoserror.h @@ -185,7 +185,7 @@ int32_t* taosGetErrno(); #define TSDB_CODE_MND_BNODE_NOT_EXIST TAOS_DEF_ERROR_CODE(0, 0x0357) #define TSDB_CODE_MND_TOO_FEW_MNODES TAOS_DEF_ERROR_CODE(0, 0x0358) #define TSDB_CODE_MND_TOO_MANY_MNODES TAOS_DEF_ERROR_CODE(0, 0x0359) -#define TSDB_CODE_MND_CANT_DROP_MASTER TAOS_DEF_ERROR_CODE(0, 0x035A) +#define TSDB_CODE_MND_CANT_DROP_LEADER TAOS_DEF_ERROR_CODE(0, 0x035A) // mnode-acct #define TSDB_CODE_MND_ACCT_ALREADY_EXIST TAOS_DEF_ERROR_CODE(0, 0x0360) @@ -653,6 +653,7 @@ int32_t* taosGetErrno(); #define TSDB_CODE_PAR_VALUE_TOO_LONG TAOS_DEF_ERROR_CODE(0, 0x2653) #define TSDB_CODE_PAR_INTERNAL_ERROR TAOS_DEF_ERROR_CODE(0, 0x2654) #define TSDB_CODE_PAR_INVALID_DELETE_WHERE TAOS_DEF_ERROR_CODE(0, 0x2655) +#define TSDB_CODE_PAR_INVALID_REDISTRIBUTE_VG TAOS_DEF_ERROR_CODE(0, 0x2656) //planner #define TSDB_CODE_PLAN_INTERNAL_ERROR TAOS_DEF_ERROR_CODE(0, 0x2700) diff --git a/include/util/tpagedbuf.h b/include/util/tpagedbuf.h index acaff759b7..af82e29ec5 100644 --- a/include/util/tpagedbuf.h +++ b/include/util/tpagedbuf.h @@ -188,6 +188,13 @@ SDiskbasedBufStatis getDBufStatis(const SDiskbasedBuf* pBuf); */ void dBufPrintStatis(const SDiskbasedBuf* pBuf); +/** + * Set all of page buffer are not need + * @param pBuf + * @return + */ +void clearDiskbasedBuf(SDiskbasedBuf* pBuf); + #ifdef __cplusplus } #endif diff --git a/source/client/src/tmq.c b/source/client/src/tmq.c index c2170631c2..ef37eed284 100644 --- a/source/client/src/tmq.c +++ b/source/client/src/tmq.c @@ -195,7 +195,7 @@ typedef struct { tmq_conf_t* tmq_conf_new() { tmq_conf_t* conf = taosMemoryCalloc(1, sizeof(tmq_conf_t)); - conf->withTbName = -1; + conf->withTbName = false; conf->autoCommit = true; conf->autoCommitInterval = 5000; conf->resetOffset = TMQ_CONF__RESET_OFFSET__EARLIEAST; @@ -256,13 +256,10 @@ tmq_conf_res_t tmq_conf_set(tmq_conf_t* conf, const char* key, const char* value if (strcmp(key, "msg.with.table.name") == 0) { if (strcmp(value, "true") == 0) { - conf->withTbName = 1; + conf->withTbName = true; return TMQ_CONF_OK; } else if (strcmp(value, "false") == 0) { - conf->withTbName = 0; - return TMQ_CONF_OK; - } else if (strcmp(value, "none") == 0) { - conf->withTbName = -1; + conf->withTbName = false; return TMQ_CONF_OK; } else { return TMQ_CONF_INVALID; diff --git a/source/common/src/tdatablock.c b/source/common/src/tdatablock.c index d9293433ea..99ff4a4a42 100644 --- a/source/common/src/tdatablock.c +++ b/source/common/src/tdatablock.c @@ -18,6 +18,7 @@ #include "tcompare.h" #include "tglobal.h" #include "tlog.h" +#include "tname.h" int32_t taosGetFqdnPortFromEp(const char* ep, SEp* pEp) { pEp->port = 0; @@ -99,7 +100,7 @@ void colDataTrim(SColumnInfoData* pColumnInfoData) { // TODO } -int32_t getJsonValueLen(const char *data) { +int32_t getJsonValueLen(const char* data) { int32_t dataLen = 0; if (*data == TSDB_DATA_TYPE_NULL) { dataLen = CHAR_BYTES; @@ -109,7 +110,7 @@ int32_t getJsonValueLen(const char *data) { dataLen = DOUBLE_BYTES + CHAR_BYTES; } else if (*data == TSDB_DATA_TYPE_BOOL) { dataLen = CHAR_BYTES + CHAR_BYTES; - } else if (*data & TD_TAG_JSON) { // json string + } else if (*data & TD_TAG_JSON) { // json string dataLen = ((STag*)(data))->len; } else { ASSERT(0); @@ -137,7 +138,7 @@ int32_t colDataAppend(SColumnInfoData* pColumnInfoData, uint32_t currentRow, con int32_t dataLen = 0; if (type == TSDB_DATA_TYPE_JSON) { dataLen = getJsonValueLen(pData); - }else { + } else { dataLen = varDataTLen(pData); } @@ -1283,7 +1284,7 @@ static void doShiftBitmap(char* nullBitmap, size_t n, size_t total) { if (n % 8 == 0) { memmove(nullBitmap, nullBitmap + n / 8, newLen); } else { - int32_t tail = n % 8; + int32_t tail = n % 8; int32_t i = 0; uint8_t* p = (uint8_t*)nullBitmap; @@ -1301,7 +1302,7 @@ static void doShiftBitmap(char* nullBitmap, size_t n, size_t total) { } } else if (n > 8) { int32_t gap = len - newLen; - while(i < newLen) { + while (i < newLen) { uint8_t v = p[i + gap]; p[i] = (v << tail); @@ -1316,7 +1317,6 @@ static void doShiftBitmap(char* nullBitmap, size_t n, size_t total) { } } - static void colDataTrimFirstNRows(SColumnInfoData* pColInfoData, size_t n, size_t total) { if (IS_VAR_DATA_TYPE(pColInfoData->info.type)) { memmove(pColInfoData->varmeta.offset, &pColInfoData->varmeta.offset[n], (total - n) * sizeof(int32_t)); @@ -1500,7 +1500,7 @@ void blockDebugShowData(const SArray* dataBlocks, const char* flag) { for (int32_t k = 0; k < colNum; k++) { SColumnInfoData* pColInfoData = taosArrayGet(pDataBlock->pDataBlock, k); void* var = POINTER_SHIFT(pColInfoData->pData, j * pColInfoData->info.bytes); - if (pColInfoData->hasNull) { + if (colDataIsNull(pColInfoData, rows, j, NULL)) { printf(" %15s |", "NULL"); continue; } @@ -1544,7 +1544,8 @@ void blockDebugShowData(const SArray* dataBlocks, const char* flag) { * * TODO: colId should be set */ -int32_t buildSubmitReqFromDataBlock(SSubmitReq** pReq, const SArray* pDataBlocks, STSchema* pTSchema, int32_t vgId, tb_uid_t suid) { +int32_t buildSubmitReqFromDataBlock(SSubmitReq** pReq, const SArray* pDataBlocks, STSchema* pTSchema, int32_t vgId, + tb_uid_t suid) { int32_t sz = taosArrayGetSize(pDataBlocks); int32_t bufSize = sizeof(SSubmitReq); for (int32_t i = 0; i < sz; ++i) { @@ -1585,12 +1586,12 @@ int32_t buildSubmitReqFromDataBlock(SSubmitReq** pReq, const SArray* pDataBlocks int32_t dataLen = 0; for (int32_t j = 0; j < rows; ++j) { // iterate by row tdSRowResetBuf(&rb, POINTER_SHIFT(pDataBuf, msgLen)); // set row buf - bool isStartKey = false; + bool isStartKey = false; int32_t offset = 0; for (int32_t k = 0; k < colNum; ++k) { // iterate by column SColumnInfoData* pColInfoData = taosArrayGet(pDataBlock->pDataBlock, k); - STColumn* pCol = &pTSchema->columns[k]; - void* var = POINTER_SHIFT(pColInfoData->pData, j * pColInfoData->info.bytes); + STColumn* pCol = &pTSchema->columns[k]; + void* var = POINTER_SHIFT(pColInfoData->pData, j * pColInfoData->info.bytes); switch (pColInfoData->info.type) { case TSDB_DATA_TYPE_TIMESTAMP: if (!isStartKey) { @@ -1599,15 +1600,18 @@ int32_t buildSubmitReqFromDataBlock(SSubmitReq** pReq, const SArray* pDataBlocks offset, k); } else { - tdAppendColValToRow(&rb, PRIMARYKEY_TIMESTAMP_COL_ID + k, TSDB_DATA_TYPE_TIMESTAMP, TD_VTYPE_NORM, var, true, offset, k); + tdAppendColValToRow(&rb, PRIMARYKEY_TIMESTAMP_COL_ID + k, TSDB_DATA_TYPE_TIMESTAMP, TD_VTYPE_NORM, var, + true, offset, k); } break; case TSDB_DATA_TYPE_NCHAR: { - tdAppendColValToRow(&rb, PRIMARYKEY_TIMESTAMP_COL_ID + k, TSDB_DATA_TYPE_NCHAR, TD_VTYPE_NORM, var, true, offset, k); + tdAppendColValToRow(&rb, PRIMARYKEY_TIMESTAMP_COL_ID + k, TSDB_DATA_TYPE_NCHAR, TD_VTYPE_NORM, var, true, + offset, k); break; } case TSDB_DATA_TYPE_VARCHAR: { // TSDB_DATA_TYPE_BINARY - tdAppendColValToRow(&rb, PRIMARYKEY_TIMESTAMP_COL_ID + k, TSDB_DATA_TYPE_VARCHAR, TD_VTYPE_NORM, var, true, offset, k); + tdAppendColValToRow(&rb, PRIMARYKEY_TIMESTAMP_COL_ID + k, TSDB_DATA_TYPE_VARCHAR, TD_VTYPE_NORM, var, true, + offset, k); break; } case TSDB_DATA_TYPE_VARBINARY: @@ -1645,7 +1649,7 @@ int32_t buildSubmitReqFromDataBlock(SSubmitReq** pReq, const SArray* pDataBlocks } break; } - offset += TYPE_BYTES[pCol->type]; // sum/avg would convert to int64_t/uint64_t/double during aggregation + offset += TYPE_BYTES[pCol->type]; // sum/avg would convert to int64_t/uint64_t/double during aggregation } dataLen += TD_ROW_LEN(rb.pBuf); #ifdef TD_DEBUG_PRINT_ROW @@ -1681,11 +1685,38 @@ int32_t buildSubmitReqFromDataBlock(SSubmitReq** pReq, const SArray* pDataBlocks return TSDB_CODE_SUCCESS; } +char* buildCtbNameByGroupId(const char* stbName, uint64_t groupId) { + SArray* tags = taosArrayInit(0, sizeof(void*)); + SSmlKv* pTag = taosMemoryCalloc(1, sizeof(SSmlKv)); + pTag->key = "group_id"; + pTag->keyLen = strlen(pTag->key); + pTag->type = TSDB_DATA_TYPE_UBIGINT; + pTag->u = groupId; + taosArrayPush(tags, &pTag); + + void* cname = taosMemoryCalloc(1, TSDB_TABLE_NAME_LEN + 1); + + RandTableName rname = { + .tags = tags, + .sTableName = stbName, + .sTableNameLen = strlen(stbName), + .childTableName = cname, + }; + + buildChildTableName(&rname); + + taosMemoryFree(pTag); + taosArrayDestroy(tags); + + ASSERT(rname.childTableName && rname.childTableName[0]); + return rname.childTableName; +} + SSubmitReq* tdBlockToSubmit(const SArray* pBlocks, const STSchema* pTSchema, bool createTb, int64_t suid, const char* stbFullName, int32_t vgId) { SSubmitReq* ret = NULL; SArray* tagArray = taosArrayInit(1, sizeof(STagVal)); - if(!tagArray) { + if (!tagArray) { terrno = TSDB_CODE_OUT_OF_MEMORY; return NULL; } @@ -1703,15 +1734,12 @@ SSubmitReq* tdBlockToSubmit(const SArray* pBlocks, const STSchema* pTSchema, boo if (createTb) { SVCreateTbReq createTbReq = {0}; - char* cname = taosMemoryCalloc(1, TSDB_TABLE_FNAME_LEN); - snprintf(cname, TSDB_TABLE_FNAME_LEN, "%s:%ld", stbFullName, pDataBlock->info.groupId); + char* cname = buildCtbNameByGroupId(stbFullName, pDataBlock->info.groupId); createTbReq.name = cname; createTbReq.flags = 0; createTbReq.type = TSDB_CHILD_TABLE; createTbReq.ctb.suid = suid; - - STagVal tagVal = {.cid = 1, .type = TSDB_DATA_TYPE_UBIGINT, .pData = (uint8_t*)&pDataBlock->info.groupId, @@ -1849,11 +1877,12 @@ SSubmitReq* tdBlockToSubmit(const SArray* pBlocks, const STSchema* pTSchema, boo } ret->length = htonl(ret->length); - taosArrayDestroy(tagArray); + taosArrayDestroy(tagArray); return ret; } -void blockCompressEncode(const SSDataBlock* pBlock, char* data, int32_t* dataLen, int32_t numOfCols, int8_t needCompress) { +void blockCompressEncode(const SSDataBlock* pBlock, char* data, int32_t* dataLen, int32_t numOfCols, + int8_t needCompress) { int32_t* actualLen = (int32_t*)data; data += sizeof(int32_t); @@ -1947,4 +1976,4 @@ const char* blockCompressDecode(SSDataBlock* pBlock, int32_t numOfCols, int32_t ASSERT(pStart - pData == dataLen); return pStart; -} \ No newline at end of file +} diff --git a/source/common/src/tmsg.c b/source/common/src/tmsg.c index 773e430b83..51c40827b5 100644 --- a/source/common/src/tmsg.c +++ b/source/common/src/tmsg.c @@ -2394,6 +2394,102 @@ int32_t tDeserializeSUserIndexRsp(void *buf, int32_t bufLen, SUserIndexRsp *pRsp return 0; } +int32_t tSerializeSTableIndexReq(void *buf, int32_t bufLen, STableIndexReq *pReq) { + SEncoder encoder = {0}; + tEncoderInit(&encoder, buf, bufLen); + + if (tStartEncode(&encoder) < 0) return -1; + if (tEncodeCStr(&encoder, pReq->tbFName) < 0) return -1; + tEndEncode(&encoder); + + int32_t tlen = encoder.pos; + tEncoderClear(&encoder); + return tlen; +} + +int32_t tDeserializeSTableIndexReq(void *buf, int32_t bufLen, STableIndexReq *pReq) { + SDecoder decoder = {0}; + tDecoderInit(&decoder, buf, bufLen); + + if (tStartDecode(&decoder) < 0) return -1; + if (tDecodeCStrTo(&decoder, pReq->tbFName) < 0) return -1; + tEndDecode(&decoder); + + tDecoderClear(&decoder); + return 0; +} + +int32_t tSerializeSTableIndexInfo(SEncoder *pEncoder, STableIndexInfo* pInfo) { + if (tEncodeI8(pEncoder, pInfo->intervalUnit) < 0) return -1; + if (tEncodeI8(pEncoder, pInfo->slidingUnit) < 0) return -1; + if (tEncodeI64(pEncoder, pInfo->interval) < 0) return -1; + if (tEncodeI64(pEncoder, pInfo->offset) < 0) return -1; + if (tEncodeI64(pEncoder, pInfo->sliding) < 0) return -1; + if (tEncodeI64(pEncoder, pInfo->dstTbUid) < 0) return -1; + if (tEncodeI32(pEncoder, pInfo->dstVgId) < 0) return -1; + if (tEncodeCStr(pEncoder, pInfo->expr) < 0) return -1; + return 0; +} + +int32_t tSerializeSTableIndexRsp(void *buf, int32_t bufLen, const STableIndexRsp *pRsp) { + SEncoder encoder = {0}; + tEncoderInit(&encoder, buf, bufLen); + + if (tStartEncode(&encoder) < 0) return -1; + int32_t num = taosArrayGetSize(pRsp->pIndex); + if (tEncodeI32(&encoder, num) < 0) return -1; + if (num > 0) { + for (int32_t i = 0; i < num; ++i) { + STableIndexInfo* pInfo = (STableIndexInfo*)taosArrayGet(pRsp->pIndex, i); + if (tSerializeSTableIndexInfo(&encoder, pInfo) < 0) return -1; + } + } + tEndEncode(&encoder); + + int32_t tlen = encoder.pos; + tEncoderClear(&encoder); + return tlen; +} + +int32_t tDeserializeSTableIndexInfo(SDecoder *pDecoder, STableIndexInfo *pInfo) { + if (tDecodeI8(pDecoder, &pInfo->intervalUnit) < 0) return -1; + if (tDecodeI8(pDecoder, &pInfo->slidingUnit) < 0) return -1; + if (tDecodeI64(pDecoder, &pInfo->interval) < 0) return -1; + if (tDecodeI64(pDecoder, &pInfo->offset) < 0) return -1; + if (tDecodeI64(pDecoder, &pInfo->sliding) < 0) return -1; + if (tDecodeI64(pDecoder, &pInfo->dstTbUid) < 0) return -1; + if (tDecodeI32(pDecoder, &pInfo->dstVgId) < 0) return -1; + if (tDecodeCStrAlloc(pDecoder, &pInfo->expr) < 0) return -1; + + return 0; +} + +int32_t tDeserializeSTableIndexRsp(void *buf, int32_t bufLen, STableIndexRsp *pRsp) { + SDecoder decoder = {0}; + tDecoderInit(&decoder, buf, bufLen); + + if (tStartDecode(&decoder) < 0) return -1; + int32_t num = 0; + if (tDecodeI32(&decoder, &num) < 0) return -1; + if (num > 0) { + pRsp->pIndex = taosArrayInit(num, sizeof(STableIndexInfo)); + if (NULL == pRsp->pIndex) return -1; + STableIndexInfo info; + for (int32_t i = 0; i < num; ++i) { + if (tDeserializeSTableIndexInfo(&decoder, &info) < 0) return -1; + if (NULL == taosArrayPush(pRsp->pIndex, &info)) { + taosMemoryFree(info.expr); + return -1; + } + } + } + tEndDecode(&decoder); + + tDecoderClear(&decoder); + return 0; +} + + int32_t tSerializeSShowReq(void *buf, int32_t bufLen, SShowReq *pReq) { SEncoder encoder = {0}; tEncoderInit(&encoder, buf, bufLen); @@ -3258,6 +3354,89 @@ int32_t tDeserializeSKillTransReq(void *buf, int32_t bufLen, SKillTransReq *pReq return 0; } +int32_t tSerializeSBalanceVgroupReq(void *buf, int32_t bufLen, SBalanceVgroupReq *pReq) { + SEncoder encoder = {0}; + tEncoderInit(&encoder, buf, bufLen); + + if (tStartEncode(&encoder) < 0) return -1; + if (tEncodeI32(&encoder, pReq->useless) < 0) return -1; + tEndEncode(&encoder); + + int32_t tlen = encoder.pos; + tEncoderClear(&encoder); + return tlen; +} + +int32_t tDeserializeSBalanceVgroupReq(void *buf, int32_t bufLen, SBalanceVgroupReq *pReq) { + SDecoder decoder = {0}; + tDecoderInit(&decoder, buf, bufLen); + + if (tStartDecode(&decoder) < 0) return -1; + if (tDecodeI32(&decoder, &pReq->useless) < 0) return -1; + tEndDecode(&decoder); + + tDecoderClear(&decoder); + return 0; +} + +int32_t tSerializeSMergeVgroupReq(void *buf, int32_t bufLen, SMergeVgroupReq *pReq) { + SEncoder encoder = {0}; + tEncoderInit(&encoder, buf, bufLen); + + if (tStartEncode(&encoder) < 0) return -1; + if (tEncodeI32(&encoder, pReq->vgId1) < 0) return -1; + if (tEncodeI32(&encoder, pReq->vgId2) < 0) return -1; + tEndEncode(&encoder); + + int32_t tlen = encoder.pos; + tEncoderClear(&encoder); + return tlen; +} + +int32_t tDeserializeSMergeVgroupReq(void *buf, int32_t bufLen, SMergeVgroupReq *pReq) { + SDecoder decoder = {0}; + tDecoderInit(&decoder, buf, bufLen); + + if (tStartDecode(&decoder) < 0) return -1; + if (tDecodeI32(&decoder, &pReq->vgId1) < 0) return -1; + if (tDecodeI32(&decoder, &pReq->vgId2) < 0) return -1; + tEndDecode(&decoder); + + tDecoderClear(&decoder); + return 0; +} + +int32_t tSerializeSRedistributeVgroupReq(void *buf, int32_t bufLen, SRedistributeVgroupReq *pReq) { + SEncoder encoder = {0}; + tEncoderInit(&encoder, buf, bufLen); + + if (tStartEncode(&encoder) < 0) return -1; + if (tEncodeI32(&encoder, pReq->vgId) < 0) return -1; + if (tEncodeI32(&encoder, pReq->dnodeId1) < 0) return -1; + if (tEncodeI32(&encoder, pReq->dnodeId2) < 0) return -1; + if (tEncodeI32(&encoder, pReq->dnodeId3) < 0) return -1; + tEndEncode(&encoder); + + int32_t tlen = encoder.pos; + tEncoderClear(&encoder); + return tlen; +} + +int32_t tDeserializeSRedistributeVgroupReq(void *buf, int32_t bufLen, SRedistributeVgroupReq *pReq) { + SDecoder decoder = {0}; + tDecoderInit(&decoder, buf, bufLen); + + if (tStartDecode(&decoder) < 0) return -1; + if (tDecodeI32(&decoder, &pReq->vgId) < 0) return -1; + if (tDecodeI32(&decoder, &pReq->dnodeId1) < 0) return -1; + if (tDecodeI32(&decoder, &pReq->dnodeId2) < 0) return -1; + if (tDecodeI32(&decoder, &pReq->dnodeId3) < 0) return -1; + tEndDecode(&decoder); + + tDecoderClear(&decoder); + return 0; +} + int32_t tSerializeSDCreateMnodeReq(void *buf, int32_t bufLen, SDCreateMnodeReq *pReq) { SEncoder encoder = {0}; tEncoderInit(&encoder, buf, bufLen); diff --git a/source/dnode/mgmt/mgmt_mnode/src/mmHandle.c b/source/dnode/mgmt/mgmt_mnode/src/mmHandle.c index 27f5e6f7af..6c2783cb5c 100644 --- a/source/dnode/mgmt/mgmt_mnode/src/mmHandle.c +++ b/source/dnode/mgmt/mgmt_mnode/src/mmHandle.c @@ -181,6 +181,7 @@ SArray *mmGetMsgHandles() { if (dmSetMgmtHandle(pArray, TDMT_MND_DROP_SMA, mmPutNodeMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_MND_CREATE_STREAM, mmPutNodeMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_MND_GET_INDEX, mmPutNodeMsgToReadQueue, 0) == NULL) goto _OVER; + if (dmSetMgmtHandle(pArray, TDMT_MND_GET_TABLE_INDEX, mmPutNodeMsgToReadQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_MND_CREATE_TOPIC, mmPutNodeMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_MND_ALTER_TOPIC, mmPutNodeMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_MND_DROP_TOPIC, mmPutNodeMsgToWriteQueue, 0) == NULL) goto _OVER; @@ -210,7 +211,7 @@ SArray *mmGetMsgHandles() { if (dmSetMgmtHandle(pArray, TDMT_VND_MQ_VG_CHANGE_RSP, mmPutNodeMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_MQ_VG_DELETE_RSP, mmPutNodeMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_DROP_TASK, mmPutNodeMsgToQueryQueue, 1) == NULL) goto _OVER; - if (dmSetMgmtHandle(pArray, TDMT_VND_TASK_DEPLOY_RSP, mmPutNodeMsgToWriteQueue, 0) == NULL) goto _OVER; + if (dmSetMgmtHandle(pArray, TDMT_STREAM_TASK_DEPLOY_RSP, mmPutNodeMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_ALTER_CONFIG_RSP, mmPutNodeMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_ALTER_REPLICA_RSP, mmPutNodeMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_ALTER_CONFIRM_RSP, mmPutNodeMsgToWriteQueue, 0) == NULL) goto _OVER; diff --git a/source/dnode/mgmt/mgmt_snode/src/smHandle.c b/source/dnode/mgmt/mgmt_snode/src/smHandle.c index a3aab439de..66ab627e32 100644 --- a/source/dnode/mgmt/mgmt_snode/src/smHandle.c +++ b/source/dnode/mgmt/mgmt_snode/src/smHandle.c @@ -94,9 +94,10 @@ SArray *smGetMsgHandles() { if (dmSetMgmtHandle(pArray, TDMT_MON_SM_INFO, smPutNodeMsgToMonitorQueue, 0) == NULL) goto _OVER; - // Requests handled by SNODE - if (dmSetMgmtHandle(pArray, TDMT_SND_TASK_DEPLOY, smPutNodeMsgToMgmtQueue, 0) == NULL) goto _OVER; - /*if (dmSetMgmtHandle(pArray, TDMT_SND_TASK_EXEC, smPutNodeMsgToExecQueue, 0) == NULL) goto _OVER;*/ + if (dmSetMgmtHandle(pArray, TDMT_STREAM_TASK_DEPLOY, smPutNodeMsgToMgmtQueue, 1) == NULL) goto _OVER; + if (dmSetMgmtHandle(pArray, TDMT_STREAM_TASK_RUN, smPutNodeMsgToMgmtQueue, 1) == NULL) goto _OVER; + if (dmSetMgmtHandle(pArray, TDMT_STREAM_TASK_DISPATCH, smPutNodeMsgToMgmtQueue, 1) == NULL) goto _OVER; + if (dmSetMgmtHandle(pArray, TDMT_STREAM_TASK_RECOVER, smPutNodeMsgToMgmtQueue, 1) == NULL) goto _OVER; code = 0; _OVER: diff --git a/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c b/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c index ccc399605b..4c5a32536f 100644 --- a/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c +++ b/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c @@ -210,8 +210,7 @@ int32_t vmProcessCreateVnodeReq(SVnodeMgmt *pMgmt, SRpcMsg *pMsg) { return -1; } - dDebug("vgId:%d, start to create vnode, tsma:%d standby:%d", createReq.vgId, createReq.isTsma, - createReq.standby); + dDebug("vgId:%d, start to create vnode, tsma:%d standby:%d", createReq.vgId, createReq.isTsma, createReq.standby); vmGenerateVnodeCfg(&createReq, &vnodeCfg); if (vmTsmaAdjustDays(&vnodeCfg, &createReq) < 0) { @@ -333,11 +332,6 @@ SArray *vmGetMsgHandles() { if (dmSetMgmtHandle(pArray, TDMT_VND_UPDATE_TAG_VAL, vmPutMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_TABLE_META, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_TABLES_META, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER; - if (dmSetMgmtHandle(pArray, TDMT_VND_MQ_CONSUME, vmPutMsgToQueryQueue, 0) == NULL) goto _OVER; - if (dmSetMgmtHandle(pArray, TDMT_VND_MQ_QUERY, vmPutMsgToQueryQueue, 0) == NULL) goto _OVER; - if (dmSetMgmtHandle(pArray, TDMT_VND_MQ_CONNECT, vmPutMsgToWriteQueue, 0) == NULL) goto _OVER; - if (dmSetMgmtHandle(pArray, TDMT_VND_MQ_DISCONNECT, vmPutMsgToWriteQueue, 0) == NULL) goto _OVER; - // if (dmSetMgmtHandle(pArray, TDMT_VND_MQ_SET_CUR, vmPutMsgToWriteQueue, 0)== NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_CANCEL_TASK, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_DROP_TASK, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_CREATE_STB, vmPutMsgToWriteQueue, 0) == NULL) goto _OVER; @@ -352,13 +346,14 @@ SArray *vmGetMsgHandles() { if (dmSetMgmtHandle(pArray, TDMT_VND_MQ_VG_CHANGE, vmPutMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_MQ_VG_DELETE, vmPutMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_CONSUME, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER; - if (dmSetMgmtHandle(pArray, TDMT_VND_TASK_DEPLOY, vmPutMsgToWriteQueue, 0) == NULL) goto _OVER; - if (dmSetMgmtHandle(pArray, TDMT_VND_QUERY_HEARTBEAT, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER; - if (dmSetMgmtHandle(pArray, TDMT_VND_STREAM_TRIGGER, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER; - if (dmSetMgmtHandle(pArray, TDMT_VND_TASK_RUN, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER; - if (dmSetMgmtHandle(pArray, TDMT_VND_TASK_DISPATCH, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER; - if (dmSetMgmtHandle(pArray, TDMT_VND_TASK_RECOVER, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_DELETE, vmPutMsgToWriteQueue, 0) == NULL) goto _OVER; + if (dmSetMgmtHandle(pArray, TDMT_VND_QUERY_HEARTBEAT, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER; + + if (dmSetMgmtHandle(pArray, TDMT_VND_STREAM_TRIGGER, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER; + if (dmSetMgmtHandle(pArray, TDMT_STREAM_TASK_DEPLOY, vmPutMsgToWriteQueue, 0) == NULL) goto _OVER; + if (dmSetMgmtHandle(pArray, TDMT_STREAM_TASK_RUN, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER; + if (dmSetMgmtHandle(pArray, TDMT_STREAM_TASK_DISPATCH, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER; + if (dmSetMgmtHandle(pArray, TDMT_STREAM_TASK_RECOVER, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_ALTER_REPLICA, vmPutMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_ALTER_CONFIG, vmPutMsgToWriteQueue, 0) == NULL) goto _OVER; diff --git a/source/dnode/mgmt/node_mgmt/src/dmTransport.c b/source/dnode/mgmt/node_mgmt/src/dmTransport.c index 8b939d15ce..0ad1c508b2 100644 --- a/source/dnode/mgmt/node_mgmt/src/dmTransport.c +++ b/source/dnode/mgmt/node_mgmt/src/dmTransport.c @@ -95,6 +95,8 @@ static void dmProcessRpcMsg(SDnode *pDnode, SRpcMsg *pRpc, SEpSet *pEpSet) { int32_t vgId = ntohl(pHead->vgId); if (vgId == QNODE_HANDLE) { pWrapper = &pDnode->wrappers[QNODE]; + } else if (vgId == SNODE_HANDLE) { + pWrapper = &pDnode->wrappers[SNODE]; } else if (vgId == MNODE_HANDLE) { pWrapper = &pDnode->wrappers[MNODE]; } else { @@ -131,7 +133,8 @@ static void dmProcessRpcMsg(SDnode *pDnode, SRpcMsg *pRpc, SEpSet *pEpSet) { _OVER: if (code != 0) { - dTrace("msg:%p, failed to process since %s, type:%s", pMsg, terrstr(), TMSG_INFO(pRpc->msgType)); + dTrace("failed to process msg:%p since %s, handle:%p", pMsg, terrstr(), pRpc->info.handle); + if (terrno != 0) code = terrno; if (IsReq(pRpc)) { @@ -149,8 +152,10 @@ _OVER: } } - dTrace("msg:%p, is freed", pMsg); - taosFreeQitem(pMsg); + if (pMsg != NULL) { + dTrace("msg:%p, is freed", pMsg); + taosFreeQitem(pMsg); + } rpcFreeCont(pRpc->pCont); } diff --git a/source/dnode/mgmt/node_util/src/dmEps.c b/source/dnode/mgmt/node_util/src/dmEps.c index 937c1ab7fa..6f6896ff9f 100644 --- a/source/dnode/mgmt/node_util/src/dmEps.c +++ b/source/dnode/mgmt/node_util/src/dmEps.c @@ -318,9 +318,9 @@ void dmGetMnodeEpSet(SDnodeData *pData, SEpSet *pEpSet) { void dmGetMnodeEpSetForRedirect(SDnodeData *pData, SRpcMsg *pMsg, SEpSet *pEpSet) { dmGetMnodeEpSet(pData, pEpSet); - dDebug("msg:%p, is redirected, num:%d use:%d", pMsg, pEpSet->numOfEps, pEpSet->inUse); + dTrace("msg is redirected, handle:%p num:%d use:%d", pMsg->info.handle, pEpSet->numOfEps, pEpSet->inUse); for (int32_t i = 0; i < pEpSet->numOfEps; ++i) { - dDebug("mnode index:%d %s:%u", i, pEpSet->eps[i].fqdn, pEpSet->eps[i].port); + dTrace("mnode index:%d %s:%u", i, pEpSet->eps[i].fqdn, pEpSet->eps[i].port); if (strcmp(pEpSet->eps[i].fqdn, tsLocalFqdn) == 0 && pEpSet->eps[i].port == tsServerPort) { pEpSet->inUse = (i + 1) % pEpSet->numOfEps; } diff --git a/source/dnode/mnode/impl/inc/mndDef.h b/source/dnode/mnode/impl/inc/mndDef.h index 8ea172ef94..382f8dd55f 100644 --- a/source/dnode/mnode/impl/inc/mndDef.h +++ b/source/dnode/mnode/impl/inc/mndDef.h @@ -309,6 +309,7 @@ typedef struct { int8_t slidingUnit; int8_t timezone; int32_t dstVgId; // for stream + int64_t dstTbUid; int64_t interval; int64_t offset; int64_t sliding; diff --git a/source/dnode/mnode/impl/inc/mndTrans.h b/source/dnode/mnode/impl/inc/mndTrans.h index 736fa3b361..5ac9d2233f 100644 --- a/source/dnode/mnode/impl/inc/mndTrans.h +++ b/source/dnode/mnode/impl/inc/mndTrans.h @@ -36,19 +36,19 @@ typedef enum { } ETrnAct; typedef struct { - int32_t id; - int32_t errCode; - int32_t acceptableCode; - int8_t stage; - ETrnAct actionType; - int8_t rawWritten; - int8_t msgSent; - int8_t msgReceived; - tmsg_t msgType; - SEpSet epSet; - int32_t contLen; - void *pCont; - SSdbRaw *pRaw; + int32_t id; + int32_t errCode; + int32_t acceptableCode; + ETrnStage stage; + ETrnAct actionType; + int8_t rawWritten; + int8_t msgSent; + int8_t msgReceived; + tmsg_t msgType; + SEpSet epSet; + int32_t contLen; + void *pCont; + SSdbRaw *pRaw; } STransAction; typedef void (*TransCbFp)(SMnode *pMnode, void *param, int32_t paramLen); diff --git a/source/dnode/mnode/impl/src/mndDnode.c b/source/dnode/mnode/impl/src/mndDnode.c index c7e347fb8b..7be4939337 100644 --- a/source/dnode/mnode/impl/src/mndDnode.c +++ b/source/dnode/mnode/impl/src/mndDnode.c @@ -87,6 +87,10 @@ int32_t mndInitDnode(SMnode *pMnode) { void mndCleanupDnode(SMnode *pMnode) {} static int32_t mndCreateDefaultDnode(SMnode *pMnode) { + int32_t code = -1; + SSdbRaw *pRaw = NULL; + STrans *pTrans = NULL; + SDnodeObj dnodeObj = {0}; dnodeObj.id = 1; dnodeObj.createdTime = taosGetTimestampMs(); @@ -95,54 +99,42 @@ static int32_t mndCreateDefaultDnode(SMnode *pMnode) { memcpy(&dnodeObj.fqdn, pMnode->replicas[0].fqdn, TSDB_FQDN_LEN); snprintf(dnodeObj.ep, TSDB_EP_LEN, "%s:%u", dnodeObj.fqdn, dnodeObj.port); - SSdbRaw *pRaw = mndDnodeActionEncode(&dnodeObj); - if (pRaw == NULL) return -1; - if (sdbSetRawStatus(pRaw, SDB_STATUS_READY) != 0) return -1; + pTrans = mndTransCreate(pMnode, TRN_POLICY_RETRY, TRN_CONFLICT_GLOBAL, NULL); + if (pTrans == NULL) goto _OVER; + mDebug("trans:%d, used to create dnode:%s on first deploy", pTrans->id, dnodeObj.ep); - mDebug("dnode:%d, will be created when deploying, raw:%p", dnodeObj.id, pRaw); - - STrans *pTrans = mndTransCreate(pMnode, TRN_POLICY_RETRY, TRN_CONFLICT_GLOBAL, NULL); - if (pTrans == NULL) { - mError("dnode:%s, failed to create since %s", dnodeObj.ep, terrstr()); - return -1; - } - mDebug("trans:%d, used to create dnode:%s", pTrans->id, dnodeObj.ep); - - if (mndTransAppendCommitlog(pTrans, pRaw) != 0) { - mError("trans:%d, failed to append commit log since %s", pTrans->id, terrstr()); - mndTransDrop(pTrans); - return -1; - } + pRaw = mndDnodeActionEncode(&dnodeObj); + if (pRaw == NULL || mndTransAppendCommitlog(pTrans, pRaw) != 0) goto _OVER; sdbSetRawStatus(pRaw, SDB_STATUS_READY); + pRaw = NULL; - if (mndTransPrepare(pMnode, pTrans) != 0) { - mError("trans:%d, failed to prepare since %s", pTrans->id, terrstr()); - mndTransDrop(pTrans); - return -1; - } + if (mndTransPrepare(pMnode, pTrans) != 0) goto _OVER; + code = 0; +_OVER: mndTransDrop(pTrans); - return 0; + sdbFreeRaw(pRaw); + return code; } static SSdbRaw *mndDnodeActionEncode(SDnodeObj *pDnode) { terrno = TSDB_CODE_OUT_OF_MEMORY; SSdbRaw *pRaw = sdbAllocRaw(SDB_DNODE, TSDB_DNODE_VER_NUMBER, sizeof(SDnodeObj) + TSDB_DNODE_RESERVE_SIZE); - if (pRaw == NULL) goto DNODE_ENCODE_OVER; + if (pRaw == NULL) goto _OVER; int32_t dataPos = 0; - SDB_SET_INT32(pRaw, dataPos, pDnode->id, DNODE_ENCODE_OVER) - SDB_SET_INT64(pRaw, dataPos, pDnode->createdTime, DNODE_ENCODE_OVER) - SDB_SET_INT64(pRaw, dataPos, pDnode->updateTime, DNODE_ENCODE_OVER) - SDB_SET_INT16(pRaw, dataPos, pDnode->port, DNODE_ENCODE_OVER) - SDB_SET_BINARY(pRaw, dataPos, pDnode->fqdn, TSDB_FQDN_LEN, DNODE_ENCODE_OVER) - SDB_SET_RESERVE(pRaw, dataPos, TSDB_DNODE_RESERVE_SIZE, DNODE_ENCODE_OVER) - SDB_SET_DATALEN(pRaw, dataPos, DNODE_ENCODE_OVER); + SDB_SET_INT32(pRaw, dataPos, pDnode->id, _OVER) + SDB_SET_INT64(pRaw, dataPos, pDnode->createdTime, _OVER) + SDB_SET_INT64(pRaw, dataPos, pDnode->updateTime, _OVER) + SDB_SET_INT16(pRaw, dataPos, pDnode->port, _OVER) + SDB_SET_BINARY(pRaw, dataPos, pDnode->fqdn, TSDB_FQDN_LEN, _OVER) + SDB_SET_RESERVE(pRaw, dataPos, TSDB_DNODE_RESERVE_SIZE, _OVER) + SDB_SET_DATALEN(pRaw, dataPos, _OVER); terrno = 0; -DNODE_ENCODE_OVER: +_OVER: if (terrno != 0) { mError("dnode:%d, failed to encode to raw:%p since %s", pDnode->id, pRaw, terrstr()); sdbFreeRaw(pRaw); @@ -154,33 +146,32 @@ DNODE_ENCODE_OVER: } static SSdbRow *mndDnodeActionDecode(SSdbRaw *pRaw) { + SSdbRow *pRow = NULL; terrno = TSDB_CODE_OUT_OF_MEMORY; int8_t sver = 0; - if (sdbGetRawSoftVer(pRaw, &sver) != 0) goto DNODE_DECODE_OVER; - + if (sdbGetRawSoftVer(pRaw, &sver) != 0) goto _OVER; if (sver != TSDB_DNODE_VER_NUMBER) { terrno = TSDB_CODE_SDB_INVALID_DATA_VER; - goto DNODE_DECODE_OVER; + goto _OVER; } - SSdbRow *pRow = sdbAllocRow(sizeof(SDnodeObj)); - if (pRow == NULL) goto DNODE_DECODE_OVER; - + pRow = sdbAllocRow(sizeof(SDnodeObj)); + if (pRow == NULL) goto _OVER; SDnodeObj *pDnode = sdbGetRowObj(pRow); - if (pDnode == NULL) goto DNODE_DECODE_OVER; + if (pDnode == NULL) goto _OVER; int32_t dataPos = 0; - SDB_GET_INT32(pRaw, dataPos, &pDnode->id, DNODE_DECODE_OVER) - SDB_GET_INT64(pRaw, dataPos, &pDnode->createdTime, DNODE_DECODE_OVER) - SDB_GET_INT64(pRaw, dataPos, &pDnode->updateTime, DNODE_DECODE_OVER) - SDB_GET_INT16(pRaw, dataPos, &pDnode->port, DNODE_DECODE_OVER) - SDB_GET_BINARY(pRaw, dataPos, pDnode->fqdn, TSDB_FQDN_LEN, DNODE_DECODE_OVER) - SDB_GET_RESERVE(pRaw, dataPos, TSDB_DNODE_RESERVE_SIZE, DNODE_DECODE_OVER) + SDB_GET_INT32(pRaw, dataPos, &pDnode->id, _OVER) + SDB_GET_INT64(pRaw, dataPos, &pDnode->createdTime, _OVER) + SDB_GET_INT64(pRaw, dataPos, &pDnode->updateTime, _OVER) + SDB_GET_INT16(pRaw, dataPos, &pDnode->port, _OVER) + SDB_GET_BINARY(pRaw, dataPos, pDnode->fqdn, TSDB_FQDN_LEN, _OVER) + SDB_GET_RESERVE(pRaw, dataPos, TSDB_DNODE_RESERVE_SIZE, _OVER) terrno = 0; -DNODE_DECODE_OVER: +_OVER: if (terrno != 0) { mError("dnode:%d, failed to decode from raw:%p since %s", pDnode->id, pRaw, terrstr()); taosMemoryFreeClear(pRow); @@ -246,6 +237,7 @@ static SDnodeObj *mndAcquireDnodeByEp(SMnode *pMnode, char *pEpStr) { sdbRelease(pSdb, pDnode); } + terrno = TSDB_CODE_MND_DNODE_NOT_EXIST; return NULL; } @@ -290,25 +282,26 @@ static void mndGetDnodeData(SMnode *pMnode, SArray *pDnodeEps) { } } -static int32_t mndCheckClusterCfgPara(SMnode *pMnode, const SClusterCfg *pCfg) { +static int32_t mndCheckClusterCfgPara(SMnode *pMnode, SDnodeObj *pDnode, const SClusterCfg *pCfg) { if (pCfg->statusInterval != tsStatusInterval) { - mError("statusInterval [%d - %d] cfg inconsistent", pCfg->statusInterval, tsStatusInterval); + mError("dnode:%d, statusInterval:%d inconsistent with cluster:%d", pDnode->id, pCfg->statusInterval, + tsStatusInterval); return DND_REASON_STATUS_INTERVAL_NOT_MATCH; } if ((0 != strcasecmp(pCfg->timezone, tsTimezoneStr)) && (pMnode->checkTime != pCfg->checkTime)) { - mError("timezone [%s - %s] [%" PRId64 " - %" PRId64 "] cfg inconsistent", pCfg->timezone, tsTimezoneStr, - pCfg->checkTime, pMnode->checkTime); + mError("dnode:%d, timezone:%s checkTime:%" PRId64 " inconsistent with cluster %s %" PRId64, pDnode->id, + pCfg->timezone, pCfg->checkTime, tsTimezoneStr, pMnode->checkTime); return DND_REASON_TIME_ZONE_NOT_MATCH; } if (0 != strcasecmp(pCfg->locale, tsLocale)) { - mError("locale [%s - %s] cfg inconsistent", pCfg->locale, tsLocale); + mError("dnode:%d, locale:%s inconsistent with cluster:%s", pDnode->id, pCfg->locale, tsLocale); return DND_REASON_LOCALE_NOT_MATCH; } if (0 != strcasecmp(pCfg->charset, tsCharset)) { - mError("charset [%s - %s] cfg inconsistent.", pCfg->charset, tsCharset); + mError("dnode:%d, charset:%s inconsistent with cluster:%s", pDnode->id, pCfg->charset, tsCharset); return DND_REASON_CHARSET_NOT_MATCH; } @@ -323,15 +316,14 @@ static int32_t mndProcessStatusReq(SRpcMsg *pReq) { if (tDeserializeSStatusReq(pReq->pCont, pReq->contLen, &statusReq) != 0) { terrno = TSDB_CODE_INVALID_MSG; - goto PROCESS_STATUS_MSG_OVER; + goto _OVER; } if (statusReq.dnodeId == 0) { pDnode = mndAcquireDnodeByEp(pMnode, statusReq.dnodeEp); if (pDnode == NULL) { mDebug("dnode:%s, not created yet", statusReq.dnodeEp); - terrno = TSDB_CODE_MND_DNODE_NOT_EXIST; - goto PROCESS_STATUS_MSG_OVER; + goto _OVER; } } else { pDnode = mndAcquireDnode(pMnode, statusReq.dnodeId); @@ -341,13 +333,11 @@ static int32_t mndProcessStatusReq(SRpcMsg *pReq) { pDnode->offlineReason = DND_REASON_DNODE_ID_NOT_MATCH; } mError("dnode:%d, %s not exist", statusReq.dnodeId, statusReq.dnodeEp); - terrno = TSDB_CODE_MND_DNODE_NOT_EXIST; - goto PROCESS_STATUS_MSG_OVER; + goto _OVER; } } - int32_t numOfVloads = (int32_t)taosArrayGetSize(statusReq.pVloads); - for (int32_t v = 0; v < numOfVloads; ++v) { + for (int32_t v = 0; v < taosArrayGetSize(statusReq.pVloads); ++v) { SVnodeLoad *pVload = taosArrayGet(statusReq.pVloads, v); SVgObj *pVgroup = mndAcquireVgroup(pMnode, pVload->vgId); @@ -366,6 +356,7 @@ static int32_t mndProcessStatusReq(SRpcMsg *pReq) { roleChanged = true; } pVgroup->vnodeGid[vg].role = pVload->syncState; + break; } } if (roleChanged) { @@ -405,11 +396,11 @@ static int32_t mndProcessStatusReq(SRpcMsg *pReq) { } mError("dnode:%d, status msg version:%d not match cluster:%d", statusReq.dnodeId, statusReq.sver, tsVersion); terrno = TSDB_CODE_VERSION_NOT_COMPATIBLE; - goto PROCESS_STATUS_MSG_OVER; + goto _OVER; } if (statusReq.dnodeId == 0) { - mDebug("dnode:%d, %s first access, set clusterId %" PRId64, pDnode->id, pDnode->ep, pMnode->clusterId); + mInfo("dnode:%d, %s first access, set clusterId %" PRId64, pDnode->id, pDnode->ep, pMnode->clusterId); } else { if (statusReq.clusterId != pMnode->clusterId) { if (pDnode != NULL) { @@ -418,7 +409,7 @@ static int32_t mndProcessStatusReq(SRpcMsg *pReq) { mError("dnode:%d, clusterId %" PRId64 " not match exist %" PRId64, pDnode->id, statusReq.clusterId, pMnode->clusterId); terrno = TSDB_CODE_MND_INVALID_CLUSTER_ID; - goto PROCESS_STATUS_MSG_OVER; + goto _OVER; } else { pDnode->accessTimes++; mTrace("dnode:%d, status received, access times %d", pDnode->id, pDnode->accessTimes); @@ -426,18 +417,17 @@ static int32_t mndProcessStatusReq(SRpcMsg *pReq) { } // Verify whether the cluster parameters are consistent when status change from offline to ready - int32_t ret = mndCheckClusterCfgPara(pMnode, &statusReq.clusterCfg); - if (0 != ret) { - pDnode->offlineReason = ret; - mError("dnode:%d, cluster cfg inconsistent since:%s", pDnode->id, offlineReason[ret]); + pDnode->offlineReason = mndCheckClusterCfgPara(pMnode, pDnode, &statusReq.clusterCfg); + if (pDnode->offlineReason != 0) { + mError("dnode:%d, cluster cfg inconsistent since:%s", pDnode->id, offlineReason[pDnode->offlineReason]); terrno = TSDB_CODE_MND_INVALID_CLUSTER_CFG; - goto PROCESS_STATUS_MSG_OVER; + goto _OVER; } if (!online) { mInfo("dnode:%d, from offline to online", pDnode->id); } else { - mDebug("dnode:%d, send dnode epset, online:%d ver:% " PRId64 ":%" PRId64 " reboot:%d", pDnode->id, online, + mDebug("dnode:%d, send dnode epset, online:%d dnode_ver:%" PRId64 ":%" PRId64 " reboot:%d", pDnode->id, online, statusReq.dnodeVer, dnodeVer, reboot); } @@ -452,7 +442,7 @@ static int32_t mndProcessStatusReq(SRpcMsg *pReq) { statusRsp.pDnodeEps = taosArrayInit(mndGetDnodeSize(pMnode), sizeof(SDnodeEp)); if (statusRsp.pDnodeEps == NULL) { terrno = TSDB_CODE_OUT_OF_MEMORY; - goto PROCESS_STATUS_MSG_OVER; + goto _OVER; } mndGetDnodeData(pMnode, statusRsp.pDnodeEps); @@ -469,13 +459,17 @@ static int32_t mndProcessStatusReq(SRpcMsg *pReq) { pDnode->lastAccessTime = curMs; code = 0; -PROCESS_STATUS_MSG_OVER: +_OVER: mndReleaseDnode(pMnode, pDnode); taosArrayDestroy(statusReq.pVloads); return code; } static int32_t mndCreateDnode(SMnode *pMnode, SRpcMsg *pReq, SCreateDnodeReq *pCreate) { + int32_t code = -1; + SSdbRaw *pRaw = NULL; + STrans *pTrans = NULL; + SDnodeObj dnodeObj = {0}; dnodeObj.id = sdbGetMaxId(pMnode->pSdb, SDB_DNODE); dnodeObj.createdTime = taosGetTimestampMs(); @@ -484,29 +478,22 @@ static int32_t mndCreateDnode(SMnode *pMnode, SRpcMsg *pReq, SCreateDnodeReq *pC memcpy(dnodeObj.fqdn, pCreate->fqdn, TSDB_FQDN_LEN); snprintf(dnodeObj.ep, TSDB_EP_LEN, "%s:%u", dnodeObj.fqdn, dnodeObj.port); - STrans *pTrans = mndTransCreate(pMnode, TRN_POLICY_ROLLBACK, TRN_CONFLICT_GLOBAL, pReq); - if (pTrans == NULL) { - mError("dnode:%s, failed to create since %s", dnodeObj.ep, terrstr()); - return -1; - } + pTrans = mndTransCreate(pMnode, TRN_POLICY_ROLLBACK, TRN_CONFLICT_GLOBAL, pReq); + if (pTrans == NULL) goto _OVER; mDebug("trans:%d, used to create dnode:%s", pTrans->id, dnodeObj.ep); - SSdbRaw *pCommitRaw = mndDnodeActionEncode(&dnodeObj); - if (pCommitRaw == NULL || mndTransAppendCommitlog(pTrans, pCommitRaw) != 0) { - mError("trans:%d, failed to append commit log since %s", pTrans->id, terrstr()); - mndTransDrop(pTrans); - return -1; - } - sdbSetRawStatus(pCommitRaw, SDB_STATUS_READY); + pRaw = mndDnodeActionEncode(&dnodeObj); + if (pRaw == NULL || mndTransAppendCommitlog(pTrans, pRaw) != 0) goto _OVER; + sdbSetRawStatus(pRaw, SDB_STATUS_READY); + pRaw = NULL; - if (mndTransPrepare(pMnode, pTrans) != 0) { - mError("trans:%d, failed to prepare since %s", pTrans->id, terrstr()); - mndTransDrop(pTrans); - return -1; - } + if (mndTransPrepare(pMnode, pTrans) != 0) goto _OVER; + code = 0; +_OVER: mndTransDrop(pTrans); - return 0; + sdbFreeRaw(pRaw); + return code; } static int32_t mndProcessCreateDnodeReq(SRpcMsg *pReq) { @@ -518,38 +505,37 @@ static int32_t mndProcessCreateDnodeReq(SRpcMsg *pReq) { if (tDeserializeSCreateDnodeReq(pReq->pCont, pReq->contLen, &createReq) != 0) { terrno = TSDB_CODE_INVALID_MSG; - goto CREATE_DNODE_OVER; + goto _OVER; } mDebug("dnode:%s:%d, start to create", createReq.fqdn, createReq.port); if (createReq.fqdn[0] == 0 || createReq.port <= 0 || createReq.port > UINT16_MAX) { terrno = TSDB_CODE_MND_INVALID_DNODE_EP; - goto CREATE_DNODE_OVER; + goto _OVER; } char ep[TSDB_EP_LEN]; snprintf(ep, TSDB_EP_LEN, "%s:%d", createReq.fqdn, createReq.port); pDnode = mndAcquireDnodeByEp(pMnode, ep); if (pDnode != NULL) { - terrno = TSDB_CODE_MND_DNODE_ALREADY_EXIST; - goto CREATE_DNODE_OVER; + goto _OVER; } pUser = mndAcquireUser(pMnode, pReq->conn.user); if (pUser == NULL) { terrno = TSDB_CODE_MND_NO_USER_FROM_CONN; - goto CREATE_DNODE_OVER; + goto _OVER; } if (mndCheckNodeAuth(pUser) != 0) { - goto CREATE_DNODE_OVER; + goto _OVER; } code = mndCreateDnode(pMnode, pReq, &createReq); if (code == 0) code = TSDB_CODE_ACTION_IN_PROGRESS; -CREATE_DNODE_OVER: +_OVER: if (code != 0 && code != TSDB_CODE_ACTION_IN_PROGRESS) { mError("dnode:%s:%d, failed to create since %s", createReq.fqdn, createReq.port, terrstr()); } @@ -559,7 +545,7 @@ CREATE_DNODE_OVER: return code; } -static int32_t mndDropDnode(SMnode *pMnode, SRpcMsg *pReq, SDnodeObj *pDnode, SMnodeObj *pMObj) { +static int32_t mndDropDnode(SMnode *pMnode, SRpcMsg *pReq, SDnodeObj *pDnode, SMnodeObj *pMObj, int32_t numOfVnodes) { int32_t code = -1; SSdbRaw *pRaw = NULL; STrans *pTrans = NULL; @@ -579,8 +565,12 @@ static int32_t mndDropDnode(SMnode *pMnode, SRpcMsg *pReq, SDnodeObj *pDnode, SM sdbSetRawStatus(pRaw, SDB_STATUS_DROPPED); pRaw = NULL; - if (mndSetDropMnodeInfoToTrans(pMnode, pTrans, pMObj) != 0) goto _OVER; - if (mndSetMoveVgroupsInfoToTrans(pMnode, pTrans, pDnode->id) != 0) goto _OVER; + if (pMObj != NULL) { + if (mndSetDropMnodeInfoToTrans(pMnode, pTrans, pMObj) != 0) goto _OVER; + } + if (numOfVnodes > 0) { + if (mndSetMoveVgroupsInfoToTrans(pMnode, pTrans, pDnode->id) != 0) goto _OVER; + } if (mndTransPrepare(pMnode, pTrans) != 0) goto _OVER; code = 0; @@ -617,11 +607,6 @@ static int32_t mndProcessDropDnodeReq(SRpcMsg *pReq) { goto _OVER; } - if (!mndIsDnodeOnline(pDnode, taosGetTimestampMs())) { - terrno = TSDB_CODE_NODE_OFFLINE; - goto _OVER; - } - pMObj = mndAcquireMnode(pMnode, dropReq.dnodeId); if (pMObj != NULL) { if (sdbGetSize(pMnode->pSdb, SDB_MNODE) <= 1) { @@ -629,7 +614,17 @@ static int32_t mndProcessDropDnodeReq(SRpcMsg *pReq) { goto _OVER; } if (pMnode->selfDnodeId == dropReq.dnodeId) { - terrno = TSDB_CODE_MND_CANT_DROP_MASTER; + terrno = TSDB_CODE_MND_CANT_DROP_LEADER; + goto _OVER; + } + } + + int32_t numOfVnodes = mndGetVnodesNum(pMnode, pDnode->id); + if (numOfVnodes > 0 || pMObj != NULL) { + if (!mndIsDnodeOnline(pDnode, taosGetTimestampMs())) { + terrno = TSDB_CODE_NODE_OFFLINE; + mError("dnode:%d, failed to drop since %s, has_mnode:%d numOfVnodes:%d", pDnode->id, terrstr(), pMObj != NULL, + numOfVnodes); goto _OVER; } } @@ -644,7 +639,7 @@ static int32_t mndProcessDropDnodeReq(SRpcMsg *pReq) { goto _OVER; } - code = mndDropDnode(pMnode, pReq, pDnode, pMObj); + code = mndDropDnode(pMnode, pReq, pDnode, pMObj, numOfVnodes); if (code == 0) code = TSDB_CODE_ACTION_IN_PROGRESS; _OVER: @@ -669,7 +664,6 @@ static int32_t mndProcessConfigDnodeReq(SRpcMsg *pReq) { SDnodeObj *pDnode = mndAcquireDnode(pMnode, cfgReq.dnodeId); if (pDnode == NULL) { - terrno = TSDB_CODE_MND_DNODE_NOT_EXIST; mError("dnode:%d, failed to config since %s ", cfgReq.dnodeId, terrstr()); return -1; } @@ -679,17 +673,18 @@ static int32_t mndProcessConfigDnodeReq(SRpcMsg *pReq) { int32_t bufLen = tSerializeSMCfgDnodeReq(NULL, 0, &cfgReq); void *pBuf = rpcMallocCont(bufLen); + + if (pBuf == NULL) return -1; tSerializeSMCfgDnodeReq(pBuf, bufLen, &cfgReq); + mDebug("dnode:%d, send config req to dnode, app:%p", cfgReq.dnodeId, pReq->info.ahandle); SRpcMsg rpcMsg = {.msgType = TDMT_DND_CONFIG_DNODE, .pCont = pBuf, .contLen = bufLen, .info = pReq->info}; - - mDebug("dnode:%d, send config req to dnode, app:%p", cfgReq.dnodeId, rpcMsg.info.ahandle); return tmsgSendReq(&epSet, &rpcMsg); } static int32_t mndProcessConfigDnodeRsp(SRpcMsg *pRsp) { mDebug("config rsp from dnode, app:%p", pRsp->info.ahandle); - return TSDB_CODE_SUCCESS; + return 0; } static int32_t mndRetrieveConfigs(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBlock, int32_t rows) { @@ -698,7 +693,7 @@ static int32_t mndRetrieveConfigs(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *p int32_t numOfRows = 0; char *cfgOpts[TSDB_CONFIG_NUMBER] = {0}; char cfgVals[TSDB_CONFIG_NUMBER][TSDB_CONIIG_VALUE_LEN + 1] = {0}; - char *pWrite; + char *pWrite = NULL; int32_t cols = 0; cfgOpts[totalRows] = "statusInterval"; @@ -724,7 +719,6 @@ static int32_t mndRetrieveConfigs(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *p cols = 0; STR_WITH_MAXSIZE_TO_VARSTR(buf, cfgOpts[i], TSDB_CONFIG_OPTION_LEN); - SColumnInfoData *pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); colDataAppend(pColInfo, numOfRows, (const char *)buf, false); @@ -774,7 +768,6 @@ static int32_t mndRetrieveDnodes(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pB char b1[9] = {0}; STR_TO_VARSTR(b1, online ? "ready" : "offline"); - pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); colDataAppend(pColInfo, numOfRows, b1, false); @@ -792,7 +785,6 @@ static int32_t mndRetrieveDnodes(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pB } pShow->numOfRows += numOfRows; - return numOfRows; } diff --git a/source/dnode/mnode/impl/src/mndMnode.c b/source/dnode/mnode/impl/src/mndMnode.c index 072ad05f5a..027de66b42 100644 --- a/source/dnode/mnode/impl/src/mndMnode.c +++ b/source/dnode/mnode/impl/src/mndMnode.c @@ -588,7 +588,7 @@ static int32_t mndProcessDropMnodeReq(SRpcMsg *pReq) { } if (pMnode->selfDnodeId == dropReq.dnodeId) { - terrno = TSDB_CODE_MND_CANT_DROP_MASTER; + terrno = TSDB_CODE_MND_CANT_DROP_LEADER; goto _OVER; } diff --git a/source/dnode/mnode/impl/src/mndScheduler.c b/source/dnode/mnode/impl/src/mndScheduler.c index 8497a57edd..3d5dab1894 100644 --- a/source/dnode/mnode/impl/src/mndScheduler.c +++ b/source/dnode/mnode/impl/src/mndScheduler.c @@ -132,7 +132,7 @@ int32_t mndAssignTaskToVg(SMnode* pMnode, STrans* pTrans, SStreamTask* pTask, SS terrno = TSDB_CODE_QRY_INVALID_INPUT; return -1; } - mndPersistTaskDeployReq(pTrans, pTask, &plan->execNode.epSet, TDMT_VND_TASK_DEPLOY, pVgroup->vgId); + mndPersistTaskDeployReq(pTrans, pTask, &plan->execNode.epSet, TDMT_STREAM_TASK_DEPLOY, pVgroup->vgId); return 0; } @@ -156,7 +156,7 @@ int32_t mndAssignTaskToSnode(SMnode* pMnode, STrans* pTrans, SStreamTask* pTask, terrno = TSDB_CODE_QRY_INVALID_INPUT; return -1; } - mndPersistTaskDeployReq(pTrans, pTask, &plan->execNode.epSet, TDMT_SND_TASK_DEPLOY, 0); + mndPersistTaskDeployReq(pTrans, pTask, &plan->execNode.epSet, TDMT_STREAM_TASK_DEPLOY, 0); return 0; } @@ -222,7 +222,7 @@ int32_t mndAddShuffledSinkToStream(SMnode* pMnode, STrans* pTrans, SStreamObj* p // dispatch pTask->dispatchType = TASK_DISPATCH__NONE; - mndPersistTaskDeployReq(pTrans, pTask, &pTask->epSet, TDMT_VND_TASK_DEPLOY, pVgroup->vgId); + mndPersistTaskDeployReq(pTrans, pTask, &pTask->epSet, TDMT_STREAM_TASK_DEPLOY, pVgroup->vgId); } return 0; } @@ -267,8 +267,7 @@ int32_t mndAddFixedSinkToStream(SMnode* pMnode, STrans* pTrans, SStreamObj* pStr // dispatch pTask->dispatchType = TASK_DISPATCH__NONE; - /*mndPersistTaskDeployReq(pTrans, pTask, &pTask->epSet, TDMT_VND_TASK_DEPLOY, pVgroup->vgId);*/ - mndPersistTaskDeployReq(pTrans, pTask, &pTask->epSet, TDMT_VND_TASK_DEPLOY, pStream->fixedSinkVg.vgId); + mndPersistTaskDeployReq(pTrans, pTask, &pTask->epSet, TDMT_STREAM_TASK_DEPLOY, pStream->fixedSinkVg.vgId); return 0; } @@ -361,7 +360,7 @@ int32_t mndScheduleStream(SMnode* pMnode, STrans* pTrans, SStreamObj* pStream) { ASSERT(taosArrayGetSize(pArray) == 1); SStreamTask* lastLevelTask = taosArrayGetP(pArray, 0); /*pTask->dispatchMsgType = TDMT_VND_TASK_MERGE_EXEC;*/ - pTask->dispatchMsgType = TDMT_VND_TASK_DISPATCH; + pTask->dispatchMsgType = TDMT_STREAM_TASK_DISPATCH; pTask->dispatchType = TASK_DISPATCH__FIXED; pTask->fixedEpDispatcher.taskId = lastLevelTask->taskId; @@ -407,7 +406,7 @@ int32_t mndScheduleStream(SMnode* pMnode, STrans* pTrans, SStreamObj* pStream) { pTask->dispatchType = TASK_DISPATCH__SHUFFLE; /*pTask->dispatchMsgType = TDMT_VND_TASK_WRITE_EXEC;*/ - pTask->dispatchMsgType = TDMT_VND_TASK_DISPATCH; + pTask->dispatchMsgType = TDMT_STREAM_TASK_DISPATCH; SDbObj* pDb = mndAcquireDb(pMnode, pStream->targetDb); ASSERT(pDb); if (mndExtractDbInfo(pMnode, pDb, &pTask->shuffleDispatcher.dbInfo, NULL) < 0) { @@ -438,7 +437,7 @@ int32_t mndScheduleStream(SMnode* pMnode, STrans* pTrans, SStreamObj* pStream) { } else { pTask->dispatchType = TASK_DISPATCH__FIXED; /*pTask->dispatchMsgType = TDMT_VND_TASK_WRITE_EXEC;*/ - pTask->dispatchMsgType = TDMT_VND_TASK_DISPATCH; + pTask->dispatchMsgType = TDMT_STREAM_TASK_DISPATCH; SArray* pArray = taosArrayGetP(pStream->tasks, 0); // one sink only ASSERT(taosArrayGetSize(pArray) == 1); diff --git a/source/dnode/mnode/impl/src/mndSma.c b/source/dnode/mnode/impl/src/mndSma.c index bcc413c1de..1d47f8fc7a 100644 --- a/source/dnode/mnode/impl/src/mndSma.c +++ b/source/dnode/mnode/impl/src/mndSma.c @@ -40,6 +40,7 @@ static int32_t mndSmaGetVgEpSet(SMnode *pMnode, SDbObj *pDb, SVgEpSet **ppVgEpS static int32_t mndProcessMCreateSmaReq(SRpcMsg *pReq); static int32_t mndProcessMDropSmaReq(SRpcMsg *pReq); static int32_t mndProcessGetSmaReq(SRpcMsg *pReq); +static int32_t mndProcessGetTbSmaReq(SRpcMsg *pReq); static int32_t mndRetrieveSma(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBlock, int32_t rows); static void mndCancelGetNextSma(SMnode *pMnode, void *pIter); @@ -59,6 +60,7 @@ int32_t mndInitSma(SMnode *pMnode) { mndSetMsgHandle(pMnode, TDMT_VND_CREATE_SMA_RSP, mndTransProcessRsp); mndSetMsgHandle(pMnode, TDMT_VND_DROP_SMA_RSP, mndTransProcessRsp); mndSetMsgHandle(pMnode, TDMT_MND_GET_INDEX, mndProcessGetSmaReq); + mndSetMsgHandle(pMnode, TDMT_MND_GET_TABLE_INDEX, mndProcessGetTbSmaReq); mndAddShowRetrieveHandle(pMnode, TSDB_MGMT_TABLE_INDEX, mndRetrieveSma); mndAddShowFreeIterHandle(pMnode, TSDB_MGMT_TABLE_INDEX, mndCancelGetNextSma); @@ -870,6 +872,55 @@ static int32_t mndGetSma(SMnode *pMnode, SUserIndexReq *indexReq, SUserIndexRsp return code; } +static int32_t mndGetTableSma(SMnode *pMnode, STableIndexReq *indexReq, STableIndexRsp *rsp, bool *exist) { + int32_t code = 0; + SSmaObj *pSma = NULL; + SSdb *pSdb = pMnode->pSdb; + void *pIter = NULL; + STableIndexInfo info; + + while (1) { + pIter = sdbFetch(pSdb, SDB_SMA, pIter, (void **)&pSma); + if (pIter == NULL) break; + + if (pSma->stb[0] != indexReq->tbFName[0] || strcmp(pSma->stb, indexReq->tbFName)) { + continue; + } + + info.intervalUnit = pSma->intervalUnit; + info.slidingUnit = pSma->slidingUnit; + info.interval = pSma->interval; + info.offset = pSma->offset; + info.sliding = pSma->sliding; + info.dstTbUid = pSma->dstTbUid; + info.dstVgId = pSma->dstVgId; + info.expr = taosMemoryMalloc(pSma->exprLen + 1); + if (info.expr == NULL) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + code = -1; + sdbRelease(pSdb, pSma); + return code; + } + + memcpy(info.expr, pSma->expr, pSma->exprLen); + info.expr[pSma->exprLen] = 0; + + if (NULL == taosArrayPush(rsp->pIndex, &info)) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + code = -1; + taosMemoryFree(info.expr); + sdbRelease(pSdb, pSma); + return code; + } + + *exist = true; + + sdbRelease(pSdb, pSma); + } + + return code; +} + static int32_t mndProcessGetSmaReq(SRpcMsg *pReq) { SUserIndexReq indexReq = {0}; SMnode *pMnode = pReq->info.node; @@ -916,6 +967,59 @@ _OVER: return code; } +static int32_t mndProcessGetTbSmaReq(SRpcMsg *pReq) { + STableIndexReq indexReq = {0}; + SMnode *pMnode = pReq->info.node; + int32_t code = -1; + STableIndexRsp rsp = {0}; + bool exist = false; + + if (tDeserializeSTableIndexReq(pReq->pCont, pReq->contLen, &indexReq) != 0) { + terrno = TSDB_CODE_INVALID_MSG; + goto _OVER; + } + + rsp.pIndex = taosArrayInit(10, sizeof(STableIndexInfo)); + if (NULL == rsp.pIndex) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + code = -1; + goto _OVER; + } + + code = mndGetTableSma(pMnode, &indexReq, &rsp, &exist); + if (code) { + goto _OVER; + } + + if (!exist) { + code = -1; + terrno = TSDB_CODE_MND_DB_INDEX_NOT_EXIST; + } else { + int32_t contLen = tSerializeSTableIndexRsp(NULL, 0, &rsp); + void *pRsp = rpcMallocCont(contLen); + if (pRsp == NULL) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + code = -1; + goto _OVER; + } + + tSerializeSTableIndexRsp(pRsp, contLen, &rsp); + + pReq->info.rsp = pRsp; + pReq->info.rspLen = contLen; + + code = 0; + } + +_OVER: + if (code != 0) { + mError("failed to get table index %s since %s", indexReq.tbFName, terrstr()); + } + + return code; +} + + static int32_t mndRetrieveSma(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBlock, int32_t rows) { SMnode *pMnode = pReq->info.node; SSdb *pSdb = pMnode->pSdb; diff --git a/source/dnode/mnode/impl/src/mndStream.c b/source/dnode/mnode/impl/src/mndStream.c index a0aa2067d3..7abe9e3c0d 100644 --- a/source/dnode/mnode/impl/src/mndStream.c +++ b/source/dnode/mnode/impl/src/mndStream.c @@ -54,8 +54,8 @@ int32_t mndInitStream(SMnode *pMnode) { }; mndSetMsgHandle(pMnode, TDMT_MND_CREATE_STREAM, mndProcessCreateStreamReq); - mndSetMsgHandle(pMnode, TDMT_VND_TASK_DEPLOY_RSP, mndTransProcessRsp); - mndSetMsgHandle(pMnode, TDMT_SND_TASK_DEPLOY_RSP, mndTransProcessRsp); + mndSetMsgHandle(pMnode, TDMT_STREAM_TASK_DEPLOY_RSP, mndTransProcessRsp); + /*mndSetMsgHandle(pMnode, TDMT_SND_TASK_DEPLOY_RSP, mndTransProcessRsp);*/ /*mndSetMsgHandle(pMnode, TDMT_MND_DROP_STREAM, mndProcessDropStreamReq);*/ /*mndSetMsgHandle(pMnode, TDMT_MND_DROP_STREAM_RSP, mndProcessDropStreamInRsp);*/ diff --git a/source/dnode/mnode/impl/src/mndSync.c b/source/dnode/mnode/impl/src/mndSync.c index 245f0938b9..ce025d5547 100644 --- a/source/dnode/mnode/impl/src/mndSync.c +++ b/source/dnode/mnode/impl/src/mndSync.c @@ -193,7 +193,7 @@ int32_t mndInitSync(SMnode *pMnode) { void mndCleanupSync(SMnode *pMnode) { SSyncMgmt *pMgmt = &pMnode->syncMgmt; syncStop(pMgmt->sync); - mDebug("sync:%" PRId64 " is stopped", pMgmt->sync); + mDebug("mnode sync is stopped, id:%" PRId64, pMgmt->sync); tsem_destroy(&pMgmt->syncSem); if (pMgmt->pWal != NULL) { diff --git a/source/dnode/mnode/impl/src/mndTrans.c b/source/dnode/mnode/impl/src/mndTrans.c index 8c0ba65fb2..c2097c069b 100644 --- a/source/dnode/mnode/impl/src/mndTrans.c +++ b/source/dnode/mnode/impl/src/mndTrans.c @@ -117,10 +117,10 @@ static SSdbRaw *mndTransActionEncode(STrans *pTrans) { int32_t dataPos = 0; SDB_SET_INT32(pRaw, dataPos, pTrans->id, _OVER) - SDB_SET_INT16(pRaw, dataPos, pTrans->stage, _OVER) - SDB_SET_INT16(pRaw, dataPos, pTrans->policy, _OVER) - SDB_SET_INT16(pRaw, dataPos, pTrans->conflict, _OVER) - SDB_SET_INT16(pRaw, dataPos, pTrans->exec, _OVER) + SDB_SET_INT8(pRaw, dataPos, pTrans->stage, _OVER) + SDB_SET_INT8(pRaw, dataPos, pTrans->policy, _OVER) + SDB_SET_INT8(pRaw, dataPos, pTrans->conflict, _OVER) + SDB_SET_INT8(pRaw, dataPos, pTrans->exec, _OVER) SDB_SET_INT64(pRaw, dataPos, pTrans->createdTime, _OVER) SDB_SET_BINARY(pRaw, dataPos, pTrans->dbname, TSDB_DB_FNAME_LEN, _OVER) SDB_SET_INT32(pRaw, dataPos, pTrans->redoActionPos, _OVER) @@ -256,15 +256,15 @@ static SSdbRow *mndTransActionDecode(SSdbRaw *pRaw) { SDB_GET_INT32(pRaw, dataPos, &pTrans->id, _OVER) - int16_t stage = 0; - int16_t policy = 0; - int16_t conflict = 0; - int16_t exec = 0; - int8_t actionType = 0; - SDB_GET_INT16(pRaw, dataPos, &stage, _OVER) - SDB_GET_INT16(pRaw, dataPos, &policy, _OVER) - SDB_GET_INT16(pRaw, dataPos, &conflict, _OVER) - SDB_GET_INT16(pRaw, dataPos, &exec, _OVER) + int8_t stage = 0; + int8_t policy = 0; + int8_t conflict = 0; + int8_t exec = 0; + int8_t actionType = 0; + SDB_GET_INT8(pRaw, dataPos, &stage, _OVER) + SDB_GET_INT8(pRaw, dataPos, &policy, _OVER) + SDB_GET_INT8(pRaw, dataPos, &conflict, _OVER) + SDB_GET_INT8(pRaw, dataPos, &exec, _OVER) pTrans->stage = stage; pTrans->policy = policy; pTrans->conflict = conflict; @@ -290,7 +290,8 @@ static SSdbRow *mndTransActionDecode(SSdbRaw *pRaw) { SDB_GET_INT32(pRaw, dataPos, &action.acceptableCode, _OVER) SDB_GET_INT8(pRaw, dataPos, &actionType, _OVER) action.actionType = actionType; - SDB_GET_INT8(pRaw, dataPos, &action.stage, _OVER) + SDB_GET_INT8(pRaw, dataPos, &stage, _OVER) + action.stage = stage; if (action.actionType == TRANS_ACTION_RAW) { SDB_GET_INT8(pRaw, dataPos, &action.rawWritten, _OVER) SDB_GET_INT32(pRaw, dataPos, &dataLen, _OVER) @@ -322,7 +323,8 @@ static SSdbRow *mndTransActionDecode(SSdbRaw *pRaw) { SDB_GET_INT32(pRaw, dataPos, &action.acceptableCode, _OVER) SDB_GET_INT8(pRaw, dataPos, &actionType, _OVER) action.actionType = actionType; - SDB_GET_INT8(pRaw, dataPos, &action.stage, _OVER) + SDB_GET_INT8(pRaw, dataPos, &stage, _OVER) + action.stage = stage; if (action.actionType == TRANS_ACTION_RAW) { SDB_GET_INT8(pRaw, dataPos, &action.rawWritten, _OVER) SDB_GET_INT32(pRaw, dataPos, &dataLen, _OVER) @@ -354,7 +356,8 @@ static SSdbRow *mndTransActionDecode(SSdbRaw *pRaw) { SDB_GET_INT32(pRaw, dataPos, &action.acceptableCode, _OVER) SDB_GET_INT8(pRaw, dataPos, &actionType, _OVER) action.actionType = actionType; - SDB_GET_INT8(pRaw, dataPos, &action.stage, _OVER) + SDB_GET_INT8(pRaw, dataPos, &stage, _OVER) + action.stage = stage; if (action.actionType) { SDB_GET_INT8(pRaw, dataPos, &action.rawWritten, _OVER) SDB_GET_INT32(pRaw, dataPos, &dataLen, _OVER) @@ -878,7 +881,6 @@ static void mndTransResetActions(SMnode *pMnode, STrans *pTrans, SArray *pArray) mDebug("trans:%d, %s:%d execute status is reset", pTrans->id, mndTransStr(pAction->stage), action); } pAction->errCode = 0; - } } @@ -890,11 +892,12 @@ static int32_t mndTransWriteSingleLog(SMnode *pMnode, STrans *pTrans, STransActi pAction->rawWritten = true; pAction->errCode = 0; code = 0; - mDebug("trans:%d, %s:%d write to sdb", pTrans->id, mndTransStr(pAction->stage), pAction->id); + mDebug("trans:%d, %s:%d write to sdb, type:%s status:%s", pTrans->id, mndTransStr(pAction->stage), pAction->id, + sdbTableName(pAction->pRaw->type), sdbStatusName(pAction->pRaw->status)); } else { pAction->errCode = (terrno != 0) ? terrno : code; - mError("trans:%d, %s:%d failed to write sdb since %s", pTrans->id, mndTransStr(pAction->stage), pAction->id, - terrstr()); + mError("trans:%d, %s:%d failed to write sdb since %s, type:%s status:%s", pTrans->id, mndTransStr(pAction->stage), + pAction->id, terrstr(), sdbTableName(pAction->pRaw->type), sdbStatusName(pAction->pRaw->status)); } return code; @@ -916,18 +919,26 @@ static int32_t mndTransSendSingleMsg(SMnode *pMnode, STrans *pTrans, STransActio } memcpy(rpcMsg.pCont, pAction->pCont, pAction->contLen); + char detail[1024] = {0}; + int32_t len = snprintf(detail, sizeof(detail), "msgType:%s numOfEps:%d inUse:%d", TMSG_INFO(pAction->msgType), + pAction->epSet.numOfEps, pAction->epSet.inUse); + for (int32_t i = 0; i < pTrans->lastErrorEpset.numOfEps; ++i) { + len += snprintf(detail + len, sizeof(detail) - len, " ep:%d-%s:%u", i, pAction->epSet.eps[i].fqdn, + pAction->epSet.eps[i].port); + } + int32_t code = tmsgSendReq(&pAction->epSet, &rpcMsg); if (code == 0) { pAction->msgSent = 1; pAction->msgReceived = 0; pAction->errCode = 0; - mDebug("trans:%d, %s:%d is sent to %s:%u", pTrans->id, mndTransStr(pAction->stage), pAction->id, - pAction->epSet.eps[pAction->epSet.inUse].fqdn, pAction->epSet.eps[pAction->epSet.inUse].port); + mDebug("trans:%d, %s:%d is sent, %s", pTrans->id, mndTransStr(pAction->stage), pAction->id, detail); } else { pAction->msgSent = 0; pAction->msgReceived = 0; pAction->errCode = (terrno != 0) ? terrno : code; - mError("trans:%d, %s:%d not send since %s", pTrans->id, mndTransStr(pAction->stage), pAction->id, terrstr()); + mError("trans:%d, %s:%d not send since %s, %s", pTrans->id, mndTransStr(pAction->stage), pAction->id, terrstr(), + detail); } return code; @@ -1424,9 +1435,9 @@ static int32_t mndRetrieveTrans(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBl if (epset.numOfEps > 0) { len += snprintf(detail + len, sizeof(detail) - len, "msgType:%s numOfEps:%d inUse:%d ", TMSG_INFO(pTrans->lastErrorMsgType), epset.numOfEps, epset.inUse); - } - for (int32_t i = 0; i < pTrans->lastErrorEpset.numOfEps; ++i) { - len += snprintf(detail + len, sizeof(detail) - len, "ep:%d-%s:%u ", i, epset.eps[i].fqdn, epset.eps[i].port); + for (int32_t i = 0; i < pTrans->lastErrorEpset.numOfEps; ++i) { + len += snprintf(detail + len, sizeof(detail) - len, "ep:%d-%s:%u ", i, epset.eps[i].fqdn, epset.eps[i].port); + } } } STR_WITH_MAXSIZE_TO_VARSTR(lastError, detail, pShow->pMeta->pSchemas[cols].bytes); diff --git a/source/dnode/mnode/sdb/inc/sdb.h b/source/dnode/mnode/sdb/inc/sdb.h index 4a00befa1e..9444543804 100644 --- a/source/dnode/mnode/sdb/inc/sdb.h +++ b/source/dnode/mnode/sdb/inc/sdb.h @@ -392,6 +392,7 @@ int32_t sdbStopWrite(SSdb *pSdb, SSdbIter *pIter, bool isApply); int32_t sdbDoWrite(SSdb *pSdb, SSdbIter *pIter, void *pBuf, int32_t len); const char *sdbTableName(ESdbType type); +const char *sdbStatusName(ESdbStatus status); void sdbPrintOper(SSdb *pSdb, SSdbRow *pRow, const char *oper); int32_t sdbGetIdFromRaw(SSdb *pSdb, SSdbRaw *pRaw); diff --git a/source/dnode/mnode/sdb/src/sdbHash.c b/source/dnode/mnode/sdb/src/sdbHash.c index 162da2bd0a..d1b1e31635 100644 --- a/source/dnode/mnode/sdb/src/sdbHash.c +++ b/source/dnode/mnode/sdb/src/sdbHash.c @@ -65,7 +65,7 @@ const char *sdbTableName(ESdbType type) { } } -static const char *sdbStatusName(ESdbStatus status) { +const char *sdbStatusName(ESdbStatus status) { switch (status) { case SDB_STATUS_CREATING: return "creating"; diff --git a/source/dnode/snode/src/snode.c b/source/dnode/snode/src/snode.c index 37b406466d..cbbe071c5f 100644 --- a/source/dnode/snode/src/snode.c +++ b/source/dnode/snode/src/snode.c @@ -90,7 +90,7 @@ void sndProcessUMsg(SSnode *pSnode, SRpcMsg *pMsg) { // stream deploy // stream stop/resume // operator exec - if (pMsg->msgType == TDMT_SND_TASK_DEPLOY) { + if (pMsg->msgType == TDMT_STREAM_TASK_DEPLOY) { void *msg = POINTER_SHIFT(pMsg->pCont, sizeof(SMsgHead)); SStreamTask *pTask = taosMemoryMalloc(sizeof(SStreamTask)); if (pTask == NULL) { diff --git a/source/dnode/vnode/CMakeLists.txt b/source/dnode/vnode/CMakeLists.txt index 05b4a270d6..c92a9d7cb7 100644 --- a/source/dnode/vnode/CMakeLists.txt +++ b/source/dnode/vnode/CMakeLists.txt @@ -36,12 +36,12 @@ target_sources( # tsdb "src/tsdb/tsdbCommit.c" - "src/tsdb/tsdbCommit2.c" + # "src/tsdb/tsdbCommit2.c" "src/tsdb/tsdbFile.c" "src/tsdb/tsdbFS.c" "src/tsdb/tsdbOpen.c" "src/tsdb/tsdbMemTable.c" - "src/tsdb/tsdbMemTable2.c" + # "src/tsdb/tsdbMemTable2.c" "src/tsdb/tsdbRead.c" "src/tsdb/tsdbReadImpl.c" "src/tsdb/tsdbWrite.c" diff --git a/source/dnode/vnode/src/inc/tsdb.h b/source/dnode/vnode/src/inc/tsdb.h index 171d52164c..ea1f38f37e 100644 --- a/source/dnode/vnode/src/inc/tsdb.h +++ b/source/dnode/vnode/src/inc/tsdb.h @@ -39,24 +39,46 @@ typedef struct SDelOp SDelOp; static int tsdbKeyCmprFn(const void *p1, const void *p2); +// tsdbMemTable ============================================================================================== +typedef struct STbData STbData; +typedef struct SMemTable SMemTable; +typedef struct STbDataIter STbDataIter; +typedef struct SMergeInfo SMergeInfo; +typedef struct STable STable; + +// SMemTable +int32_t tsdbMemTableCreate(STsdb *pTsdb, SMemTable **ppMemTable); +void tsdbMemTableDestroy(SMemTable *pMemTable); +void tsdbGetTbDataFromMemTable(SMemTable *pMemTable, tb_uid_t suid, tb_uid_t uid, STbData **ppTbData); + +// STbDataIter +int32_t tsdbTbDataIterCreate(STbData *pTbData, TSDBKEY *pFrom, int8_t backward, STbDataIter **ppIter); +void *tsdbTbDataIterDestroy(STbDataIter *pIter); +void tsdbTbDataIterOpen(STbData *pTbData, TSDBKEY *pFrom, int8_t backward, STbDataIter *pIter); +bool tsdbTbDataIterNext(STbDataIter *pIter); +bool tsdbTbDataIterGet(STbDataIter *pIter, TSDBROW *pRow); + +int tsdbLoadDataFromCache(STsdb *pTsdb, STable *pTable, STbDataIter *pIter, TSKEY maxKey, int maxRowsToRead, + SDataCols *pCols, TKEY *filterKeys, int nFilterKeys, bool keepDup, SMergeInfo *pMergeInfo); + // tsdbMemTable2.c ============================================================================================== -typedef struct SMemTable SMemTable; -typedef struct SMemData SMemData; -typedef struct SMemDataIter SMemDataIter; +// typedef struct SMemTable2 SMemTable2; +// typedef struct SMemData SMemData; +// typedef struct SMemDataIter SMemDataIter; -int32_t tsdbMemTableCreate2(STsdb *pTsdb, SMemTable **ppMemTable); -void tsdbMemTableDestroy2(SMemTable *pMemTable); -int32_t tsdbInsertTableData2(STsdb *pTsdb, int64_t version, SVSubmitBlk *pSubmitBlk); -int32_t tsdbDeleteTableData2(STsdb *pTsdb, int64_t version, tb_uid_t suid, tb_uid_t uid, TSKEY sKey, TSKEY eKey); +// int32_t tsdbMemTableCreate2(STsdb *pTsdb, SMemTable2 **ppMemTable); +// void tsdbMemTableDestroy2(SMemTable2 *pMemTable); +// int32_t tsdbInsertTableData2(STsdb *pTsdb, int64_t version, SVSubmitBlk *pSubmitBlk); +// int32_t tsdbDeleteTableData2(STsdb *pTsdb, int64_t version, tb_uid_t suid, tb_uid_t uid, TSKEY sKey, TSKEY eKey); -/* SMemDataIter */ -void tsdbMemDataIterOpen(SMemData *pMemData, TSDBKEY *pKey, int8_t backward, SMemDataIter *pIter); -bool tsdbMemDataIterNext(SMemDataIter *pIter); -void tsdbMemDataIterGet(SMemDataIter *pIter, TSDBROW **ppRow); +// /* SMemDataIter */ +// void tsdbMemDataIterOpen(SMemData *pMemData, TSDBKEY *pKey, int8_t backward, SMemDataIter *pIter); +// bool tsdbMemDataIterNext(SMemDataIter *pIter); +// void tsdbMemDataIterGet(SMemDataIter *pIter, TSDBROW **ppRow); -// tsdbCommit2.c ============================================================================================== -int32_t tsdbBegin2(STsdb *pTsdb); -int32_t tsdbCommit2(STsdb *pTsdb); +// // tsdbCommit2.c ============================================================================================== +// int32_t tsdbBegin2(STsdb *pTsdb); +// int32_t tsdbCommit2(STsdb *pTsdb); // tsdbFile.c ============================================================================================== typedef int32_t TSDB_FILE_T; @@ -124,17 +146,6 @@ int tsdbRLockFS(STsdbFS *pFs); int tsdbWLockFS(STsdbFS *pFs); int tsdbUnLockFS(STsdbFS *pFs); -// tsdbMemTable ================ -typedef struct STbData STbData; -typedef struct STsdbMemTable STsdbMemTable; -typedef struct SMergeInfo SMergeInfo; -typedef struct STable STable; - -int tsdbMemTableCreate(STsdb *pTsdb, STsdbMemTable **ppMemTable); -void tsdbMemTableDestroy(STsdbMemTable *pMemTable); -int tsdbLoadDataFromCache(STsdb *pTsdb, STable *pTable, SSkipListIterator *pIter, TSKEY maxKey, int maxRowsToRead, - SDataCols *pCols, TKEY *filterKeys, int nFilterKeys, bool keepDup, SMergeInfo *pMergeInfo); - // structs typedef struct { int minFid; @@ -145,30 +156,27 @@ typedef struct { #define TSDB_DATA_DIR_LEN 6 // adapt accordingly struct STsdb { - char *path; - SVnode *pVnode; - TdThreadMutex mutex; - char dir[TSDB_DATA_DIR_LEN]; - bool repoLocked; - STsdbKeepCfg keepCfg; - STsdbMemTable *mem; - STsdbMemTable *imem; - SRtn rtn; - STsdbFS *fs; + char *path; + SVnode *pVnode; + TdThreadMutex mutex; + char dir[TSDB_DATA_DIR_LEN]; + bool repoLocked; + STsdbKeepCfg keepCfg; + SMemTable *mem; + SMemTable *imem; + SRtn rtn; + STsdbFS *fs; }; #if 1 // ====================================== struct STable { - uint64_t tid; + uint64_t suid; uint64_t uid; STSchema *pSchema; // latest schema STSchema *pCacheSchema; // cached cache }; -#define TABLE_TID(t) (t)->tid -#define TABLE_UID(t) (t)->uid - // int tsdbPrepareCommit(STsdb *pTsdb); typedef enum { TSDB_FILE_HEAD = 0, // .head @@ -206,27 +214,44 @@ struct SDFileSet { SDFile files[TSDB_FILE_MAX]; }; -struct STbData { - tb_uid_t uid; - TSKEY keyMin; - TSKEY keyMax; - int64_t minVer; - int64_t maxVer; - int64_t nrows; - SSkipList *pData; +struct TSDBKEY { + int64_t version; + TSKEY ts; }; -struct STsdbMemTable { - SVBufPool *pPool; - T_REF_DECLARE() - SRWLatch latch; - TSKEY keyMin; - TSKEY keyMax; - int64_t minVer; - int64_t maxVer; - int64_t nRow; - SSkipList *pSlIdx; // SSkiplist - SHashObj *pHashIdx; +typedef struct SMemSkipListNode SMemSkipListNode; +struct SMemSkipListNode { + int8_t level; + SMemSkipListNode *forwards[0]; +}; +typedef struct SMemSkipList { + uint32_t seed; + int64_t size; + int8_t maxLevel; + int8_t level; + SMemSkipListNode *pHead; + SMemSkipListNode *pTail; +} SMemSkipList; + +struct STbData { + tb_uid_t suid; + tb_uid_t uid; + TSDBKEY minKey; + TSDBKEY maxKey; + SDelOp *pHead; + SDelOp *pTail; + SMemSkipList sl; +}; + +struct SMemTable { + SRWLatch latch; + STsdb *pTsdb; + int32_t nRef; + TSDBKEY minKey; + TSDBKEY maxKey; + int64_t nRow; + int64_t nDelOp; + SArray *aTbData; // SArray }; struct STsdbFSMeta { @@ -237,9 +262,11 @@ struct STsdbFSMeta { // ================== typedef struct { - STsdbFSMeta meta; // FS meta - SArray *df; // data file array - SArray *sf; // sma data file array v2f1900.index_name_1 + STsdbFSMeta meta; // FS meta + SDFile cacheFile; // cache file + SDFile tombstone; // tomestome file + SArray *df; // data file array + SArray *sf; // sma data file array v2f1900.index_name_1 } SFSStatus; struct STsdbFS { @@ -292,16 +319,24 @@ static void *taosTZfree(void *ptr); static size_t taosTSizeof(void *ptr); static void taosTMemset(void *ptr, int c); -static FORCE_INLINE STSRow *tsdbNextIterRow(SSkipListIterator *pIter) { +struct TSDBROW { + int64_t version; + STSRow *pTSRow; +}; + +static FORCE_INLINE STSRow *tsdbNextIterRow(STbDataIter *pIter) { + TSDBROW row; + if (pIter == NULL) return NULL; - SSkipListNode *node = tSkipListIterGet(pIter); - if (node == NULL) return NULL; + if (tsdbTbDataIterGet(pIter, &row)) { + return row.pTSRow; + } - return (STSRow *)SL_GET_NODE_DATA(node); + return NULL; } -static FORCE_INLINE TSKEY tsdbNextIterKey(SSkipListIterator *pIter) { +static FORCE_INLINE TSKEY tsdbNextIterKey(STbDataIter *pIter) { STSRow *row = tsdbNextIterRow(pIter); if (row == NULL) return TSDB_DATA_TIMESTAMP_NULL; @@ -311,11 +346,6 @@ static FORCE_INLINE TSKEY tsdbNextIterKey(SSkipListIterator *pIter) { // tsdbReadImpl typedef struct SReadH SReadH; -struct TSDBKEY { - int64_t version; - TSKEY ts; -}; - typedef struct { uint64_t suid; uint64_t uid; @@ -354,7 +384,7 @@ typedef struct { typedef struct { int32_t delimiter; // For recovery usage - int32_t tid; + uint64_t suid; uint64_t uid; SBlock blocks[]; } SBlockInfo; @@ -650,11 +680,6 @@ struct SFSIter { #define TSDB_FS_ITER_FORWARD TSDB_ORDER_ASC #define TSDB_FS_ITER_BACKWARD TSDB_ORDER_DESC -struct TSDBROW { - int64_t version; - STSRow2 tsRow; -}; - struct TABLEID { tb_uid_t suid; tb_uid_t uid; @@ -675,7 +700,7 @@ typedef struct { TSKEY eKey; } SDelInfo; -struct SMemTable { +struct SMemTable2 { STsdb *pTsdb; int32_t nRef; TSDBKEY minKey; @@ -705,16 +730,6 @@ static FORCE_INLINE int tsdbKeyCmprFn(const void *p1, const void *p2) { return 0; } -typedef struct SMemSkipListNode SMemSkipListNode; -typedef struct SMemSkipList { - uint32_t seed; - int32_t size; - int8_t maxLevel; - int8_t level; - SMemSkipListNode *pHead; - SMemSkipListNode *pTail; -} SMemSkipList; - struct SMemData { tb_uid_t suid; tb_uid_t uid; @@ -726,13 +741,19 @@ struct SMemData { }; struct SMemDataIter { - SMemData *pMemData; + STbData *pMemData; int8_t backward; TSDBROW *pRow; SMemSkipListNode *pNode; // current node TSDBROW row; }; +struct STbDataIter { + STbData *pTbData; + int8_t backward; + SMemSkipListNode *pNode; +}; + #endif #ifdef __cplusplus diff --git a/source/dnode/vnode/src/inc/vnodeInt.h b/source/dnode/vnode/src/inc/vnodeInt.h index 15469786e0..77005767fe 100644 --- a/source/dnode/vnode/src/inc/vnodeInt.h +++ b/source/dnode/vnode/src/inc/vnodeInt.h @@ -116,7 +116,9 @@ int tsdbBegin(STsdb* pTsdb); int32_t tsdbCommit(STsdb* pTsdb); int tsdbScanAndConvertSubmitMsg(STsdb* pTsdb, SSubmitReq* pMsg); int tsdbInsertData(STsdb* pTsdb, int64_t version, SSubmitReq* pMsg, SSubmitRsp* pRsp); -int tsdbInsertTableData(STsdb* pTsdb, SSubmitMsgIter* pMsgIter, SSubmitBlk* pBlock, SSubmitBlkRsp* pRsp); +int32_t tsdbInsertTableData(STsdb* pTsdb, int64_t version, SSubmitMsgIter* pMsgIter, SSubmitBlk* pBlock, + SSubmitBlkRsp* pRsp); +int32_t tsdbDeleteTableData(STsdb* pTsdb, int64_t version, tb_uid_t suid, tb_uid_t uid, TSKEY sKey, TSKEY eKey); tsdbReaderT* tsdbQueryTables(SVnode* pVnode, SQueryTableDataCond* pCond, STableListInfo* tableList, uint64_t qId, uint64_t taskId); tsdbReaderT tsdbQueryCacheLastT(STsdb* tsdb, SQueryTableDataCond* pCond, STableListInfo* tableList, uint64_t qId, diff --git a/source/dnode/vnode/src/meta/metaQuery.c b/source/dnode/vnode/src/meta/metaQuery.c index 21a55d6463..063a78b0c6 100644 --- a/source/dnode/vnode/src/meta/metaQuery.c +++ b/source/dnode/vnode/src/meta/metaQuery.c @@ -31,7 +31,7 @@ void metaReaderClear(SMetaReader *pReader) { } int metaGetTableEntryByVersion(SMetaReader *pReader, int64_t version, tb_uid_t uid) { - SMeta * pMeta = pReader->pMeta; + SMeta *pMeta = pReader->pMeta; STbDbKey tbDbKey = {.version = version, .uid = uid}; // query table.db @@ -54,7 +54,7 @@ _err: } int metaGetTableEntryByUid(SMetaReader *pReader, tb_uid_t uid) { - SMeta * pMeta = pReader->pMeta; + SMeta *pMeta = pReader->pMeta; int64_t version; // query uid.idx @@ -68,7 +68,7 @@ int metaGetTableEntryByUid(SMetaReader *pReader, tb_uid_t uid) { } int metaGetTableEntryByName(SMetaReader *pReader, const char *name) { - SMeta * pMeta = pReader->pMeta; + SMeta *pMeta = pReader->pMeta; tb_uid_t uid; // query name.idx @@ -82,7 +82,7 @@ int metaGetTableEntryByName(SMetaReader *pReader, const char *name) { } tb_uid_t metaGetTableEntryUidByName(SMeta *pMeta, const char *name) { - void * pData = NULL; + void *pData = NULL; int nData = 0; tb_uid_t uid = 0; @@ -134,7 +134,7 @@ void metaCloseTbCursor(SMTbCursor *pTbCur) { int metaTbCursorNext(SMTbCursor *pTbCur) { int ret; - void * pBuf; + void *pBuf; STbCfg tbCfg; for (;;) { @@ -155,7 +155,7 @@ int metaTbCursorNext(SMTbCursor *pTbCur) { } SSchemaWrapper *metaGetTableSchema(SMeta *pMeta, tb_uid_t uid, int32_t sver, bool isinline) { - void * pData = NULL; + void *pData = NULL; int nData = 0; int64_t version; SSchemaWrapper schema = {0}; @@ -163,37 +163,47 @@ SSchemaWrapper *metaGetTableSchema(SMeta *pMeta, tb_uid_t uid, int32_t sver, boo SDecoder dc = {0}; metaRLock(pMeta); - if (sver < 0) { - if (tdbTbGet(pMeta->pUidIdx, &uid, sizeof(uid), &pData, &nData) < 0) { - goto _err; - } - - version = *(int64_t *)pData; - - tdbTbGet(pMeta->pTbDb, &(STbDbKey){.uid = uid, .version = version}, sizeof(STbDbKey), &pData, &nData); - - SMetaEntry me = {0}; - tDecoderInit(&dc, pData, nData); - metaDecodeEntry(&dc, &me); - if (me.type == TSDB_SUPER_TABLE) { - pSchema = tCloneSSchemaWrapper(&me.stbEntry.schemaRow); - } else if (me.type == TSDB_NORMAL_TABLE) { - pSchema = tCloneSSchemaWrapper(&me.ntbEntry.schemaRow); - } else { - ASSERT(0); - } - tDecoderClear(&dc); - } else { - if (tdbTbGet(pMeta->pSkmDb, &(SSkmDbKey){.uid = uid, .sver = sver}, sizeof(SSkmDbKey), &pData, &nData) < 0) { - goto _err; - } - - tDecoderInit(&dc, pData, nData); - tDecodeSSchemaWrapper(&dc, &schema); - pSchema = tCloneSSchemaWrapper(&schema); - tDecoderClear(&dc); +_query: + if (tdbTbGet(pMeta->pUidIdx, &uid, sizeof(uid), &pData, &nData) < 0) { + goto _err; } + version = *(int64_t *)pData; + + tdbTbGet(pMeta->pTbDb, &(STbDbKey){.uid = uid, .version = version}, sizeof(STbDbKey), &pData, &nData); + SMetaEntry me = {0}; + tDecoderInit(&dc, pData, nData); + metaDecodeEntry(&dc, &me); + if (me.type == TSDB_SUPER_TABLE) { + if (sver == -1 || sver == me.stbEntry.schemaRow.version) { + pSchema = tCloneSSchemaWrapper(&me.stbEntry.schemaRow); + tDecoderClear(&dc); + goto _exit; + } + } else if (me.type == TSDB_CHILD_TABLE) { + uid = me.ctbEntry.suid; + tDecoderClear(&dc); + goto _query; + } else { + if (sver == -1 || sver == me.ntbEntry.schemaRow.version) { + pSchema = tCloneSSchemaWrapper(&me.ntbEntry.schemaRow); + tDecoderClear(&dc); + goto _exit; + } + } + tDecoderClear(&dc); + + // query from skm db + if (tdbTbGet(pMeta->pSkmDb, &(SSkmDbKey){.uid = uid, .sver = sver}, sizeof(SSkmDbKey), &pData, &nData) < 0) { + goto _err; + } + + tDecoderInit(&dc, pData, nData); + tDecodeSSchemaWrapper(&dc, &schema); + pSchema = tCloneSSchemaWrapper(&schema); + tDecoderClear(&dc); + +_exit: metaULock(pMeta); tdbFree(pData); return pSchema; @@ -205,11 +215,11 @@ _err: } struct SMCtbCursor { - SMeta * pMeta; - TBC * pCur; + SMeta *pMeta; + TBC *pCur; tb_uid_t suid; - void * pKey; - void * pVal; + void *pKey; + void *pVal; int kLen; int vLen; }; @@ -279,25 +289,13 @@ tb_uid_t metaCtbCursorNext(SMCtbCursor *pCtbCur) { } STSchema *metaGetTbTSchema(SMeta *pMeta, tb_uid_t uid, int32_t sver) { - tb_uid_t quid; - SMetaReader mr = {0}; - STSchema * pTSchema = NULL; + // SMetaReader mr = {0}; + STSchema *pTSchema = NULL; SSchemaWrapper *pSW = NULL; STSchemaBuilder sb = {0}; - SSchema * pSchema; + SSchema *pSchema; - metaReaderInit(&mr, pMeta, 0); - metaGetTableEntryByUid(&mr, uid); - - if (mr.me.type == TSDB_CHILD_TABLE) { - quid = mr.me.ctbEntry.suid; - } else { - quid = uid; - } - - metaReaderClear(&mr); - - pSW = metaGetTableSchema(pMeta, quid, sver, 0); + pSW = metaGetTableSchema(pMeta, uid, sver, 0); if (!pSW) return NULL; tdInitTSchemaBuilder(&sb, pSW->version); @@ -321,11 +319,11 @@ int metaGetTbNum(SMeta *pMeta) { } typedef struct { - SMeta * pMeta; - TBC * pCur; + SMeta *pMeta; + TBC *pCur; tb_uid_t uid; - void * pKey; - void * pVal; + void *pKey; + void *pVal; int kLen; int vLen; } SMSmaCursor; @@ -397,7 +395,7 @@ tb_uid_t metaSmaCursorNext(SMSmaCursor *pSmaCur) { STSmaWrapper *metaGetSmaInfoByTable(SMeta *pMeta, tb_uid_t uid, bool deepCopy) { STSmaWrapper *pSW = NULL; - SArray * pSmaIds = NULL; + SArray *pSmaIds = NULL; if (!(pSmaIds = metaGetSmaIdsByTable(pMeta, uid))) { return NULL; @@ -421,7 +419,7 @@ STSmaWrapper *metaGetSmaInfoByTable(SMeta *pMeta, tb_uid_t uid, bool deepCopy) { metaReaderInit(&mr, pMeta, 0); int64_t smaId; int smaIdx = 0; - STSma * pTSma = NULL; + STSma *pTSma = NULL; for (int i = 0; i < pSW->number; ++i) { smaId = *(tb_uid_t *)taosArrayGet(pSmaIds, i); if (metaGetTableEntryByUid(&mr, smaId) < 0) { @@ -469,7 +467,7 @@ _err: } STSma *metaGetSmaInfoByIndex(SMeta *pMeta, int64_t indexUid) { - STSma * pTSma = NULL; + STSma *pTSma = NULL; SMetaReader mr = {0}; metaReaderInit(&mr, pMeta, 0); if (metaGetTableEntryByUid(&mr, indexUid) < 0) { @@ -491,7 +489,7 @@ STSma *metaGetSmaInfoByIndex(SMeta *pMeta, int64_t indexUid) { } SArray *metaGetSmaIdsByTable(SMeta *pMeta, tb_uid_t uid) { - SArray * pUids = NULL; + SArray *pUids = NULL; SSmaIdxKey *pSmaIdxKey = NULL; SMSmaCursor *pCur = metaOpenSmaCursor(pMeta, uid); @@ -529,7 +527,7 @@ SArray *metaGetSmaIdsByTable(SMeta *pMeta, tb_uid_t uid) { } SArray *metaGetSmaTbUids(SMeta *pMeta) { - SArray * pUids = NULL; + SArray *pUids = NULL; SSmaIdxKey *pSmaIdxKey = NULL; tb_uid_t lastUid = 0; @@ -591,13 +589,13 @@ const void *metaGetTableTagVal(SMetaEntry *pEntry, int16_t type, STagVal *val) { } typedef struct { - SMeta * pMeta; - TBC * pCur; + SMeta *pMeta; + TBC *pCur; tb_uid_t suid; int16_t cid; int16_t type; - void * pKey; - void * pVal; + void *pKey; + void *pVal; int32_t kLen; int32_t vLen; } SIdxCursor; @@ -621,7 +619,7 @@ int32_t metaFilteTableIds(SMeta *pMeta, SMetaFltParam *param, SArray *pUids) { int32_t nKey = 0; int32_t nTagData = 0; - void * tagData = NULL; + void *tagData = NULL; if (IS_VAR_DATA_TYPE(param->type)) { tagData = varDataVal(param->val); @@ -640,7 +638,7 @@ int32_t metaFilteTableIds(SMeta *pMeta, SMetaFltParam *param, SArray *pUids) { goto END; } - void * entryKey = NULL, *entryVal = NULL; + void *entryKey = NULL, *entryVal = NULL; int32_t nEntryKey, nEntryVal; while (1) { valid = tdbTbcGet(pCursor->pCur, (const void **)&entryKey, &nEntryKey, (const void **)&entryVal, &nEntryVal); diff --git a/source/dnode/vnode/src/sma/sma.c b/source/dnode/vnode/src/sma/sma.c index 04f65275d7..fd5dd080ca 100644 --- a/source/dnode/vnode/src/sma/sma.c +++ b/source/dnode/vnode/src/sma/sma.c @@ -47,15 +47,16 @@ int32_t tdUpdateExpireWindow(SSma* pSma, const SSubmitReq* pMsg, int64_t version int32_t tdGetTSmaData(SSma* pSma, char* pData, int64_t indexUid, TSKEY querySKey, int32_t nMaxResult) { int32_t code = TSDB_CODE_SUCCESS; if ((code = tdGetTSmaDataImpl(pSma, pData, indexUid, querySKey, nMaxResult)) < 0) { - smaWarn("vgId:%d, get tSma data failed since %s", SMA_VID(pSma), tstrerror(terrno)); + smaWarn("vgId:%d, get tsma data failed since %s", SMA_VID(pSma), tstrerror(terrno)); } return code; } -int32_t smaGetTSmaDays(SVnodeCfg* pCfg, void* pCont, uint32_t contLen, int32_t *days) { +int32_t smaGetTSmaDays(SVnodeCfg* pCfg, void* pCont, uint32_t contLen, int32_t* days) { int32_t code = TSDB_CODE_SUCCESS; if ((code = tdGetTSmaDaysImpl(pCfg, pCont, contLen, days)) < 0) { - smaWarn("vgId:%d get tSma days failed since %s", pCfg->vgId, tstrerror(terrno)); + smaWarn("vgId:%d, get tsma days failed since %s", pCfg->vgId, tstrerror(terrno)); } + smaDebug("vgId:%d, get tsma days %d", pCfg->vgId, *days); return code; } diff --git a/source/dnode/vnode/src/sma/smaTimeRange2.c b/source/dnode/vnode/src/sma/smaTimeRange2.c index 5ef171c799..09adc1a6a2 100644 --- a/source/dnode/vnode/src/sma/smaTimeRange2.c +++ b/source/dnode/vnode/src/sma/smaTimeRange2.c @@ -97,12 +97,16 @@ int32_t tdGetTSmaDaysImpl(SVnodeCfg *pCfg, void *pCont, uint32_t contLen, int32_ goto _err; } STsdbCfg *pTsdbCfg = &pCfg->tsdbCfg; - int64_t mInterval = convertTimeFromPrecisionToUnit(tsma.interval, pTsdbCfg->precision, TIME_UNIT_MINUTE); - int64_t records = pTsdbCfg->days / mInterval; - + int64_t sInterval = convertTimeFromPrecisionToUnit(tsma.interval, pTsdbCfg->precision, TIME_UNIT_SECOND); + if (sInterval <= 0) { + *days = pTsdbCfg->days; + return 0; + } + int64_t records = pTsdbCfg->days * 60 / sInterval; if (records >= SMA_STORAGE_SPLIT_FACTOR) { *days = pTsdbCfg->days; } else { + int64_t mInterval = convertTimeFromPrecisionToUnit(tsma.interval, pTsdbCfg->precision, TIME_UNIT_MINUTE); int64_t daysPerFile = mInterval * SMA_STORAGE_MINUTES_DAY * 2; if (daysPerFile > SMA_STORAGE_MINUTES_MAX) { @@ -111,7 +115,7 @@ int32_t tdGetTSmaDaysImpl(SVnodeCfg *pCfg, void *pCont, uint32_t contLen, int32_ *days = (int32_t)daysPerFile; } - if(*days < pTsdbCfg->days) { + if (*days < pTsdbCfg->days) { *days = pTsdbCfg->days; } } diff --git a/source/dnode/vnode/src/tq/tq.c b/source/dnode/vnode/src/tq/tq.c index 2b5e18c1db..181cddee47 100644 --- a/source/dnode/vnode/src/tq/tq.c +++ b/source/dnode/vnode/src/tq/tq.c @@ -349,8 +349,9 @@ int32_t tqProcessTaskDeploy(STQ* pTq, char* msg, int32_t msgLen) { } // sink - pTask->ahandle = pTq->pVnode; + /*pTask->ahandle = pTq->pVnode;*/ if (pTask->sinkType == TASK_SINK__SMA) { + pTask->smaSink.vnode = pTq->pVnode; pTask->smaSink.smaSink = smaHandleRes; } else if (pTask->sinkType == TASK_SINK__TABLE) { pTask->tbSink.vnode = pTq->pVnode; diff --git a/source/dnode/vnode/src/tq/tqRead.c b/source/dnode/vnode/src/tq/tqRead.c index 43f49b4cc2..2ecaeff747 100644 --- a/source/dnode/vnode/src/tq/tqRead.c +++ b/source/dnode/vnode/src/tq/tqRead.c @@ -141,10 +141,10 @@ int32_t tqRetrieveDataBlock(SArray** ppCols, STqReadHandle* pHandle, uint64_t* p } // this interface use suid instead of uid - pHandle->pSchemaWrapper = metaGetTableSchema(pHandle->pVnodeMeta, pHandle->msgIter.suid, sversion, true); + pHandle->pSchemaWrapper = metaGetTableSchema(pHandle->pVnodeMeta, pHandle->msgIter.uid, sversion, true); if (pHandle->pSchemaWrapper == NULL) { tqWarn("cannot found schema wrapper for table: suid: %ld, version %d, possibly dropped table", - pHandle->msgIter.suid, pHandle->cachedSchemaVer); + pHandle->msgIter.uid, pHandle->cachedSchemaVer); /*ASSERT(0);*/ terrno = TSDB_CODE_TQ_TABLE_SCHEMA_NOT_FOUND; return -1; diff --git a/source/dnode/vnode/src/tsdb/tsdbCommit.c b/source/dnode/vnode/src/tsdb/tsdbCommit.c index 59a795b77a..865023e628 100644 --- a/source/dnode/vnode/src/tsdb/tsdbCommit.c +++ b/source/dnode/vnode/src/tsdb/tsdbCommit.c @@ -18,8 +18,8 @@ #define TSDB_MAX_SUBBLOCKS 8 typedef struct { - STable *pTable; - SSkipListIterator *pIter; + STable *pTable; + STbDataIter *pIter; } SCommitIter; typedef struct { @@ -58,26 +58,26 @@ typedef struct { #define TSDB_COMMIT_DEFAULT_ROWS(ch) TSDB_DEFAULT_BLOCK_ROWS(TSDB_COMMIT_REPO(ch)->pVnode->config.tsdbCfg.maxRows) #define TSDB_COMMIT_TXN_VERSION(ch) FS_TXN_VERSION(REPO_FS(TSDB_COMMIT_REPO(ch))) -static void tsdbStartCommit(STsdb *pRepo); -static void tsdbEndCommit(STsdb *pTsdb, int eno); -static int tsdbInitCommitH(SCommitH *pCommith, STsdb *pRepo); -static void tsdbSeekCommitIter(SCommitH *pCommith, TSKEY key); -static int tsdbNextCommitFid(SCommitH *pCommith); -static void tsdbDestroyCommitH(SCommitH *pCommith); -static int tsdbCreateCommitIters(SCommitH *pCommith); -static void tsdbDestroyCommitIters(SCommitH *pCommith); -static int tsdbCommitToFile(SCommitH *pCommith, SDFileSet *pSet, int fid); -static void tsdbResetCommitFile(SCommitH *pCommith); -static int tsdbSetAndOpenCommitFile(SCommitH *pCommith, SDFileSet *pSet, int fid); -static int tsdbCommitToTable(SCommitH *pCommith, int tid); -static bool tsdbCommitIsSameFile(SCommitH *pCommith, int bidx); -static int tsdbMoveBlkIdx(SCommitH *pCommith, SBlockIdx *pIdx); -static int tsdbSetCommitTable(SCommitH *pCommith, STable *pTable); -static int tsdbComparKeyBlock(const void *arg1, const void *arg2); -static int tsdbWriteBlockInfo(SCommitH *pCommih); -static int tsdbCommitMemData(SCommitH *pCommith, SCommitIter *pIter, TSKEY keyLimit, bool toData); -static int tsdbMergeMemData(SCommitH *pCommith, SCommitIter *pIter, int bidx); -static int tsdbMoveBlock(SCommitH *pCommith, int bidx); +static void tsdbStartCommit(STsdb *pRepo); +static void tsdbEndCommit(STsdb *pTsdb, int eno); +static int tsdbInitCommitH(SCommitH *pCommith, STsdb *pRepo); +static void tsdbSeekCommitIter(SCommitH *pCommith, TSKEY key); +static int tsdbNextCommitFid(SCommitH *pCommith); +static void tsdbDestroyCommitH(SCommitH *pCommith); +static int32_t tsdbCreateCommitIters(SCommitH *pCommith); +static void tsdbDestroyCommitIters(SCommitH *pCommith); +static int tsdbCommitToFile(SCommitH *pCommith, SDFileSet *pSet, int fid); +static void tsdbResetCommitFile(SCommitH *pCommith); +static int tsdbSetAndOpenCommitFile(SCommitH *pCommith, SDFileSet *pSet, int fid); +static int tsdbCommitToTable(SCommitH *pCommith, int tid); +static bool tsdbCommitIsSameFile(SCommitH *pCommith, int bidx); +static int tsdbMoveBlkIdx(SCommitH *pCommith, SBlockIdx *pIdx); +static int tsdbSetCommitTable(SCommitH *pCommith, STable *pTable); +static int tsdbComparKeyBlock(const void *arg1, const void *arg2); +static int tsdbWriteBlockInfo(SCommitH *pCommih); +static int tsdbCommitMemData(SCommitH *pCommith, SCommitIter *pIter, TSKEY keyLimit, bool toData); +static int tsdbMergeMemData(SCommitH *pCommith, SCommitIter *pIter, int bidx); +static int tsdbMoveBlock(SCommitH *pCommith, int bidx); static int tsdbCommitAddBlock(SCommitH *pCommith, const SBlock *pSupBlock, const SBlock *pSubBlocks, int nSubBlocks); static int tsdbMergeBlockData(SCommitH *pCommith, SCommitIter *pIter, SDataCols *pDataCols, TSKEY keyLimit, bool isLastOneBlock); @@ -92,7 +92,7 @@ static int tsdbApplyRtnOnFSet(STsdb *pRepo, SDFileSet *pSet, SRtn *pRtn); int tsdbBegin(STsdb *pTsdb) { if (!pTsdb) return 0; - STsdbMemTable *pMem; + SMemTable *pMem; if (tsdbMemTableCreate(pTsdb, &pTsdb->mem) < 0) { return -1; @@ -244,7 +244,7 @@ void tsdbGetRtnSnap(STsdb *pRepo, SRtn *pRtn) { } static void tsdbStartCommit(STsdb *pRepo) { - STsdbMemTable *pMem = pRepo->imem; + SMemTable *pMem = pRepo->imem; tsdbInfo("vgId:%d, start to commit", REPO_ID(pRepo)); @@ -400,7 +400,7 @@ static int tsdbCommitToFile(SCommitH *pCommith, SDFileSet *pSet, int fid) { break; } - if (pIter && pIter->pTable && (!pIdx || (pIter->pTable->uid <= pIdx->uid))) { + if (pIter && pIter->pTable && (!pIdx || (pIter->pTable->suid <= pIdx->suid || pIter->pTable->uid <= pIdx->uid))) { if (tsdbCommitToTable(pCommith, mIter) < 0) { tsdbCloseCommitFile(pCommith, true); // revert the file change @@ -453,57 +453,48 @@ static int tsdbCommitToFile(SCommitH *pCommith, SDFileSet *pSet, int fid) { return 0; } -static int tsdbCreateCommitIters(SCommitH *pCommith) { - STsdb *pRepo = TSDB_COMMIT_REPO(pCommith); - STsdbMemTable *pMem = pRepo->imem; - SSkipListIterator *pSlIter; - SCommitIter *pCommitIter; - SSkipListNode *pNode; - STbData *pTbData; - STSchema *pTSchema = NULL; +static int32_t tsdbCreateCommitIters(SCommitH *pCommith) { + int32_t code = 0; + STsdb *pRepo = TSDB_COMMIT_REPO(pCommith); + SMemTable *pMem = pRepo->imem; + STbData *pTbData; + SCommitIter *pCommitIter; + STSchema *pTSchema = NULL; - pCommith->niters = SL_SIZE(pMem->pSlIdx); + pCommith->niters = taosArrayGetSize(pMem->aTbData); pCommith->iters = (SCommitIter *)taosMemoryCalloc(pCommith->niters, sizeof(SCommitIter)); if (pCommith->iters == NULL) { - terrno = TSDB_CODE_TDB_OUT_OF_MEMORY; - return -1; + code = TSDB_CODE_OUT_OF_MEMORY; + goto _err; } - // Loop to create iters for each skiplist - pSlIter = tSkipListCreateIter(pMem->pSlIdx); - if (pSlIter == NULL) { - terrno = TSDB_CODE_TDB_OUT_OF_MEMORY; - return -1; - } - for (int i = 0; i < pCommith->niters; i++) { - tSkipListIterNext(pSlIter); - pNode = tSkipListIterGet(pSlIter); - pTbData = (STbData *)pNode->pData; + for (int32_t iIter = 0; iIter < pCommith->niters; iIter++) { + pTbData = (STbData *)taosArrayGetP(pMem->aTbData, iIter); + pCommitIter = &pCommith->iters[iIter]; - pCommitIter = pCommith->iters + i; pTSchema = metaGetTbTSchema(REPO_META(pRepo), pTbData->uid, -1); - if (pTSchema) { - pCommitIter->pIter = tSkipListCreateIter(pTbData->pData); - tSkipListIterNext(pCommitIter->pIter); + tsdbTbDataIterCreate(pTbData, NULL, 0, &pCommitIter->pIter); pCommitIter->pTable = (STable *)taosMemoryMalloc(sizeof(STable)); pCommitIter->pTable->uid = pTbData->uid; - pCommitIter->pTable->tid = pTbData->uid; + pCommitIter->pTable->suid = pTbData->suid; pCommitIter->pTable->pSchema = pTSchema; pCommitIter->pTable->pCacheSchema = NULL; } } - tSkipListDestroyIter(pSlIter); - return 0; + return code; + +_err: + return code; } static void tsdbDestroyCommitIters(SCommitH *pCommith) { if (pCommith->iters == NULL) return; for (int i = 1; i < pCommith->niters; i++) { - tSkipListDestroyIter(pCommith->iters[i].pIter); + tsdbTbDataIterDestroy(pCommith->iters[i].pIter); if (pCommith->iters[i].pTable) { tdFreeSchema(pCommith->iters[i].pTable->pSchema); tdFreeSchema(pCommith->iters[i].pTable->pCacheSchema); @@ -743,8 +734,8 @@ static int tsdbWriteBlockInfoImpl(SDFile *pHeadf, STable *pTable, SArray *pSupA, pBlkInfo = *ppBuf; pBlkInfo->delimiter = TSDB_FILE_DELIMITER; - pBlkInfo->tid = TABLE_TID(pTable); - pBlkInfo->uid = TABLE_UID(pTable); + pBlkInfo->suid = pTable->suid; + pBlkInfo->uid = pTable->uid; memcpy((void *)(pBlkInfo->blocks), taosArrayGet(pSupA, 0), nSupBlocks * sizeof(SBlock)); if (nSubBlocks > 0) { @@ -770,7 +761,8 @@ static int tsdbWriteBlockInfoImpl(SDFile *pHeadf, STable *pTable, SArray *pSupA, // Set pIdx pBlock = taosArrayGetLast(pSupA); - pIdx->uid = TABLE_UID(pTable); + pIdx->suid = pTable->suid; + pIdx->uid = pTable->uid; pIdx->hasLast = pBlock->last ? 1 : 0; pIdx->maxKey = pBlock->maxKey; pIdx->numOfBlocks = (uint32_t)nSupBlocks; @@ -925,7 +917,7 @@ static int tsdbMoveBlkIdx(SCommitH *pCommith, SBlockIdx *pIdx) { return -1; } - STable table = {.tid = pIdx->uid, .uid = pIdx->uid, .pSchema = NULL}; + STable table = {.suid = pIdx->suid, .uid = pIdx->uid, .pSchema = NULL}; pCommith->pTable = &table; while (bidx < nBlocks) { @@ -1186,7 +1178,7 @@ static int tsdbWriteBlockImpl(STsdb *pRepo, STable *pTable, SDFile *pDFile, SDFi } pBlockData->delimiter = TSDB_FILE_DELIMITER; - pBlockData->uid = TABLE_UID(pTable); + pBlockData->uid = pTable->uid; pBlockData->numOfCols = nColsNotAllNull; taosCalcChecksumAppend(0, (uint8_t *)pBlockData, tsize); @@ -1226,7 +1218,7 @@ static int tsdbWriteBlockImpl(STsdb *pRepo, STable *pTable, SDFile *pDFile, SDFi tsdbDebug("vgId:%d, uid:%" PRId64 " a block of data is written to file %s, offset %" PRId64 " numOfRows %d len %d numOfCols %" PRId16 " keyFirst %" PRId64 " keyLast %" PRId64, - REPO_ID(pRepo), TABLE_UID(pTable), TSDB_FILE_FULL_NAME(pDFile), offset, rowsToWrite, pBlock->len, + REPO_ID(pRepo), pTable->uid, TSDB_FILE_FULL_NAME(pDFile), offset, rowsToWrite, pBlock->len, pBlock->numOfCols, pBlock->minKey.ts, pBlock->maxKey.ts); return 0; @@ -1313,7 +1305,7 @@ static int tsdbMergeMemData(SCommitH *pCommith, SCommitIter *pIter, int bidx) { keyLimit = pBlock[1].minKey.ts - 1; } - SSkipListIterator titer = *(pIter->pIter); + STbDataIter titer = *(pIter->pIter); if (tsdbLoadBlockDataCols(&(pCommith->readh), pBlock, NULL, &colId, 1, false) < 0) return -1; tsdbLoadDataFromCache(TSDB_COMMIT_REPO(pCommith), pIter->pTable, &titer, keyLimit, INT32_MAX, NULL, @@ -1522,7 +1514,7 @@ static void tsdbLoadAndMergeFromCache(STsdb *pTsdb, SDataCols *pDataCols, int *i lastKey = key2; } - tSkipListIterNext(pCommitIter->pIter); + tsdbTbDataIterNext(pCommitIter->pIter); } else { if (lastKey != key1) { if (lastKey != TSKEY_INITIAL_VAL) { @@ -1554,7 +1546,7 @@ static void tsdbLoadAndMergeFromCache(STsdb *pTsdb, SDataCols *pDataCols, int *i tdAppendSTSRowToDataCol(row, pSchema, pTarget, true); } ++(*iter); - tSkipListIterNext(pCommitIter->pIter); + tsdbTbDataIterNext(pCommitIter->pIter); } if (pTarget->numOfRows >= (maxRows - 1)) break; diff --git a/source/dnode/vnode/src/tsdb/tsdbCommit2.c b/source/dnode/vnode/src/tsdb/tsdbCommit2.c index 321667c55b..07d4ef8656 100644 --- a/source/dnode/vnode/src/tsdb/tsdbCommit2.c +++ b/source/dnode/vnode/src/tsdb/tsdbCommit2.c @@ -16,19 +16,19 @@ #include "tsdb.h" typedef struct { - SMemTable *pMemTable; - int32_t minutes; - int8_t precision; - TSKEY nCommitKey; - int32_t fid; - TSKEY minKey; - TSKEY maxKey; - SReadH readh; - SDFileSet wSet; - SArray *aBlkIdx; - SArray *aSupBlk; - SArray *aSubBlk; - SArray *aDelInfo; + SMemTable2 *pMemTable; + int32_t minutes; + int8_t precision; + TSKEY nCommitKey; + int32_t fid; + TSKEY minKey; + TSKEY maxKey; + SReadH readh; + SDFileSet wSet; + SArray *aBlkIdx; + SArray *aSupBlk; + SArray *aSubBlk; + SArray *aDelInfo; } SCommitH; static int32_t tsdbCommitStart(SCommitH *pCHandle, STsdb *pTsdb); @@ -39,7 +39,7 @@ int32_t tsdbBegin2(STsdb *pTsdb) { int32_t code = 0; ASSERT(pTsdb->mem == NULL); - code = tsdbMemTableCreate2(pTsdb, (SMemTable **)&pTsdb->mem); + code = tsdbMemTableCreate2(pTsdb, (SMemTable2 **)&pTsdb->mem); if (code) { tsdbError("vgId:%d failed to begin TSDB since %s", TD_VID(pTsdb->pVnode), tstrerror(code)); goto _exit; @@ -80,8 +80,8 @@ _err: } static int32_t tsdbCommitStart(SCommitH *pCHandle, STsdb *pTsdb) { - int32_t code = 0; - SMemTable *pMemTable = (SMemTable *)pTsdb->mem; + int32_t code = 0; + SMemTable2 *pMemTable = (SMemTable2 *)pTsdb->mem; tsdbInfo("vgId:%d start to commit", TD_VID(pTsdb->pVnode)); @@ -131,9 +131,9 @@ _err: } static int32_t tsdbCommitEnd(SCommitH *pCHandle) { - int32_t code = 0; - STsdb *pTsdb = pCHandle->pMemTable->pTsdb; - SMemTable *pMemTable = (SMemTable *)pTsdb->imem; + int32_t code = 0; + STsdb *pTsdb = pCHandle->pMemTable->pTsdb; + SMemTable2 *pMemTable = (SMemTable2 *)pTsdb->imem; // end transaction code = tsdbEndFSTxn(pTsdb); diff --git a/source/dnode/vnode/src/tsdb/tsdbFile.c b/source/dnode/vnode/src/tsdb/tsdbFile.c index a88cb8a353..11d206dc35 100644 --- a/source/dnode/vnode/src/tsdb/tsdbFile.c +++ b/source/dnode/vnode/src/tsdb/tsdbFile.c @@ -25,8 +25,7 @@ static const char *TSDB_FNAME_SUFFIX[] = { "meta", // TSDB_FILE_META }; -static void tsdbGetFilename(int vid, int fid, uint32_t ver, TSDB_FILE_T ftype, const char *dname, char *fname); -// static int tsdbRollBackMFile(SMFile *pMFile); +static void tsdbGetFilename(int vid, int fid, uint32_t ver, TSDB_FILE_T ftype, const char *dname, char *fname); static int tsdbEncodeDFInfo(void **buf, SDFInfo *pInfo); static void *tsdbDecodeDFInfo(void *buf, SDFInfo *pInfo); static int tsdbRollBackDFile(SDFile *pDFile); diff --git a/source/dnode/vnode/src/tsdb/tsdbMemTable.c b/source/dnode/vnode/src/tsdb/tsdbMemTable.c index fa392baa16..3a4ada04a8 100644 --- a/source/dnode/vnode/src/tsdb/tsdbMemTable.c +++ b/source/dnode/vnode/src/tsdb/tsdbMemTable.c @@ -15,67 +15,299 @@ #include "tsdb.h" -static STbData *tsdbNewTbData(tb_uid_t uid); -static void tsdbFreeTbData(STbData *pTbData); -static char *tsdbGetTsTupleKey(const void *data); -static int tsdbTbDataComp(const void *arg1, const void *arg2); -static char *tsdbTbDataGetUid(const void *arg); -static int tsdbAppendTableRowToCols(STsdb *pTsdb, STable *pTable, SDataCols *pCols, STSchema **ppSchema, STSRow *row, - bool merge); +#define SL_MAX_LEVEL 5 -int tsdbMemTableCreate(STsdb *pTsdb, STsdbMemTable **ppMemTable) { - STsdbMemTable *pMemTable; - SVnode *pVnode; +#define SL_NODE_SIZE(l) (sizeof(SMemSkipListNode) + sizeof(SMemSkipListNode *) * (l)*2) +#define SL_NODE_FORWARD(n, l) ((n)->forwards[l]) +#define SL_NODE_BACKWARD(n, l) ((n)->forwards[(n)->level + (l)]) +#define SL_NODE_DATA(n) (&SL_NODE_BACKWARD(n, (n)->level)) - *ppMemTable = NULL; - pVnode = pTsdb->pVnode; +#define SL_MOVE_BACKWARD 0x1 +#define SL_MOVE_FROM_POS 0x2 - // alloc handle - pMemTable = (STsdbMemTable *)taosMemoryCalloc(1, sizeof(*pMemTable)); +static int32_t tPutTSDBRow(uint8_t *p, TSDBROW *pRow); +static int32_t tGetTSDBRow(uint8_t *p, TSDBROW *pRow); +static void tbDataMovePosTo(STbData *pTbData, SMemSkipListNode **pos, TSDBKEY *pKey, int32_t flags); +static int32_t tsdbGetOrCreateTbData(SMemTable *pMemTable, tb_uid_t suid, tb_uid_t uid, STbData **ppTbData); +static int32_t tsdbInsertTableDataImpl(SMemTable *pMemTable, STbData *pTbData, int64_t version, + SSubmitMsgIter *pMsgIter, SSubmitBlk *pBlock, SSubmitBlkRsp *pRsp); + +int32_t tsdbMemTableCreate(STsdb *pTsdb, SMemTable **ppMemTable) { + int32_t code = 0; + SMemTable *pMemTable = NULL; + + pMemTable = (SMemTable *)taosMemoryCalloc(1, sizeof(*pMemTable)); if (pMemTable == NULL) { - return -1; + code = TSDB_CODE_OUT_OF_MEMORY; + goto _err; } - - pMemTable->pPool = pTsdb->pVnode->inUse; - T_REF_INIT_VAL(pMemTable, 1); taosInitRWLatch(&pMemTable->latch); - pMemTable->keyMin = TSKEY_MAX; - pMemTable->keyMax = TSKEY_MIN; + pMemTable->pTsdb = pTsdb; + pMemTable->nRef = 1; + pMemTable->minKey = (TSDBKEY){.ts = TSKEY_MAX, .version = INT64_MAX}; + pMemTable->maxKey = (TSDBKEY){.ts = TSKEY_MIN, .version = -1}; pMemTable->nRow = 0; - pMemTable->pSlIdx = tSkipListCreate(pVnode->config.tsdbCfg.slLevel, TSDB_DATA_TYPE_BIGINT, sizeof(tb_uid_t), - tsdbTbDataComp, SL_DISCARD_DUP_KEY, tsdbTbDataGetUid); - if (pMemTable->pSlIdx == NULL) { + pMemTable->nDelOp = 0; + pMemTable->aTbData = taosArrayInit(128, sizeof(STbData *)); + if (pMemTable->aTbData == NULL) { + code = TSDB_CODE_OUT_OF_MEMORY; taosMemoryFree(pMemTable); - return -1; - } - - pMemTable->pHashIdx = taosHashInit(1024, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), false, HASH_NO_LOCK); - if (pMemTable->pHashIdx == NULL) { - tSkipListDestroy(pMemTable->pSlIdx); - taosMemoryFree(pMemTable); - return -1; + goto _err; } *ppMemTable = pMemTable; + return code; + +_err: + *ppMemTable = NULL; + return code; +} + +void tsdbMemTableDestroy(SMemTable *pMemTable) { + if (pMemTable) { + taosArrayDestroy(pMemTable->aTbData); + taosMemoryFree(pMemTable); + } +} + +static int32_t tbDataPCmprFn(const void *p1, const void *p2) { + STbData *pTbData1 = *(STbData **)p1; + STbData *pTbData2 = *(STbData **)p2; + + if (pTbData1->suid < pTbData2->suid) { + return -1; + } else if (pTbData1->suid > pTbData2->suid) { + return 1; + } + + if (pTbData1->uid < pTbData2->uid) { + return -1; + } else if (pTbData1->uid > pTbData2->uid) { + return 1; + } + + return 0; +} +void tsdbGetTbDataFromMemTable(SMemTable *pMemTable, tb_uid_t suid, tb_uid_t uid, STbData **ppTbData) { + STbData *pTbData = &(STbData){.suid = suid, .uid = uid}; + void *p = taosArraySearch(pMemTable->aTbData, &pTbData, tbDataPCmprFn, TD_EQ); + *ppTbData = p ? *(STbData **)p : NULL; +} + +int32_t tsdbInsertTableData(STsdb *pTsdb, int64_t version, SSubmitMsgIter *pMsgIter, SSubmitBlk *pBlock, + SSubmitBlkRsp *pRsp) { + int32_t code = 0; + SMemTable *pMemTable = pTsdb->mem; + STbData *pTbData = NULL; + tb_uid_t suid = pMsgIter->suid; + tb_uid_t uid = pMsgIter->uid; + int32_t sverNew; + + // check if table exists (todo: refact) + SMetaReader mr = {0}; + SMetaEntry me = {0}; + metaReaderInit(&mr, pTsdb->pVnode->pMeta, 0); + if (metaGetTableEntryByUid(&mr, pMsgIter->uid) < 0) { + metaReaderClear(&mr); + code = TSDB_CODE_PAR_TABLE_NOT_EXIST; + goto _err; + } + if (pRsp->tblFName) strcat(pRsp->tblFName, mr.me.name); + + if (mr.me.type == TSDB_NORMAL_TABLE) { + sverNew = mr.me.ntbEntry.schemaRow.version; + } else { + metaGetTableEntryByUid(&mr, mr.me.ctbEntry.suid); + sverNew = mr.me.stbEntry.schemaRow.version; + } + metaReaderClear(&mr); + pRsp->sver = sverNew; + + // create/get STbData to op + code = tsdbGetOrCreateTbData(pMemTable, suid, uid, &pTbData); + if (code) { + goto _err; + } + + // do insert impl + code = tsdbInsertTableDataImpl(pMemTable, pTbData, version, pMsgIter, pBlock, pRsp); + if (code) { + goto _err; + } + + return code; + +_err: + return code; +} + +int32_t tsdbDeleteTableData(STsdb *pTsdb, int64_t version, tb_uid_t suid, tb_uid_t uid, TSKEY sKey, TSKEY eKey) { + int32_t code = 0; + SMemTable *pMemTable = pTsdb->mem; + STbData *pTbData = NULL; + SVBufPool *pPool = pTsdb->pVnode->inUse; + + // check if table exists (todo) + + code = tsdbGetOrCreateTbData(pMemTable, suid, uid, &pTbData); + if (code) { + goto _err; + } + + // do delete + SDelOp *pDelOp = (SDelOp *)vnodeBufPoolMalloc(pPool, sizeof(*pDelOp)); + if (pDelOp == NULL) { + code = TSDB_CODE_OUT_OF_MEMORY; + goto _err; + } + pDelOp->version = version; + pDelOp->sKey = sKey; + pDelOp->eKey = eKey; + pDelOp->pNext = NULL; + if (pTbData->pHead == NULL) { + ASSERT(pTbData->pTail == NULL); + pTbData->pHead = pTbData->pTail = pDelOp; + } else { + pTbData->pTail->pNext = pDelOp; + pTbData->pTail = pDelOp; + } + + // update the state of pMemTable and other (todo) + + pMemTable->nDelOp++; + + tsdbError("vgId:%d delete data from table suid:%" PRId64 " uid:%" PRId64 " skey:%" PRId64 " eKey:%" PRId64 + " since %s", + TD_VID(pTsdb->pVnode), suid, uid, sKey, eKey, tstrerror(code)); + return code; + +_err: + tsdbError("vgId:%d failed to delete data from table suid:%" PRId64 " uid:%" PRId64 " skey:%" PRId64 " eKey:%" PRId64 + " since %s", + TD_VID(pTsdb->pVnode), suid, uid, sKey, eKey, tstrerror(code)); + return code; +} + +static int tsdbAppendTableRowToCols(STsdb *pTsdb, STable *pTable, SDataCols *pCols, STSchema **ppSchema, STSRow *row, + bool merge) { + if (pCols) { + if (*ppSchema == NULL || schemaVersion(*ppSchema) != TD_ROW_SVER(row)) { + *ppSchema = tsdbGetTableSchemaImpl(pTsdb, pTable, false, false, TD_ROW_SVER(row)); + if (*ppSchema == NULL) { + ASSERT(false); + return -1; + } + } + + tdAppendSTSRowToDataCol(row, *ppSchema, pCols, merge); + } + return 0; } -void tsdbMemTableDestroy(STsdbMemTable *pMemTable) { - if (pMemTable) { - taosHashCleanup(pMemTable->pHashIdx); - SSkipListIterator *pIter = tSkipListCreateIter(pMemTable->pSlIdx); - SSkipListNode *pNode = NULL; - STbData *pTbData = NULL; - for (;;) { - if (!tSkipListIterNext(pIter)) break; - pNode = tSkipListIterGet(pIter); - pTbData = (STbData *)pNode->pData; - tsdbFreeTbData(pTbData); - } - tSkipListDestroyIter(pIter); - tSkipListDestroy(pMemTable->pSlIdx); - taosMemoryFree(pMemTable); +int32_t tsdbTbDataIterCreate(STbData *pTbData, TSDBKEY *pFrom, int8_t backward, STbDataIter **ppIter) { + int32_t code = 0; + + (*ppIter) = (STbDataIter *)taosMemoryCalloc(1, sizeof(STbDataIter)); + if ((*ppIter) == NULL) { + code = TSDB_CODE_OUT_OF_MEMORY; + goto _exit; } + + tsdbTbDataIterOpen(pTbData, pFrom, backward, *ppIter); + +_exit: + return code; +} + +void *tsdbTbDataIterDestroy(STbDataIter *pIter) { + if (pIter) { + taosMemoryFree(pIter); + } + + return NULL; +} + +void tsdbTbDataIterOpen(STbData *pTbData, TSDBKEY *pFrom, int8_t backward, STbDataIter *pIter) { + SMemSkipListNode *pos[SL_MAX_LEVEL]; + + pIter->pTbData = pTbData; + pIter->backward = backward; + if (pFrom == NULL) { + // create from head or tail + if (backward) { + pIter->pNode = SL_NODE_BACKWARD(pTbData->sl.pTail, 0); + } else { + pIter->pNode = SL_NODE_FORWARD(pTbData->sl.pHead, 0); + } + } else { + // create from a key + if (backward) { + tbDataMovePosTo(pTbData, pos, pFrom, SL_MOVE_BACKWARD); + pIter->pNode = SL_NODE_BACKWARD(pos[0], 0); + } else { + tbDataMovePosTo(pTbData, pos, pFrom, 0); + pIter->pNode = SL_NODE_FORWARD(pos[0], 0); + } + } +} + +bool tsdbTbDataIterNext(STbDataIter *pIter) { + SMemSkipListNode *pHead = pIter->pTbData->sl.pHead; + SMemSkipListNode *pTail = pIter->pTbData->sl.pTail; + + if (pIter->backward) { + ASSERT(pIter->pNode != pTail); + + if (pIter->pNode == pHead) { + return false; + } + + pIter->pNode = SL_NODE_BACKWARD(pIter->pNode, 0); + if (pIter->pNode == pHead) { + return false; + } + } else { + ASSERT(pIter->pNode != pHead); + + if (pIter->pNode == pTail) { + return false; + } + + pIter->pNode = SL_NODE_FORWARD(pIter->pNode, 0); + if (pIter->pNode == pTail) { + return false; + } + } + + return true; +} + +bool tsdbTbDataIterGet(STbDataIter *pIter, TSDBROW *pRow) { + SMemSkipListNode *pHead = pIter->pTbData->sl.pHead; + SMemSkipListNode *pTail = pIter->pTbData->sl.pTail; + TSDBROW row = {0}; + + if (pRow == NULL) { + pRow = &row; + } + + if (pIter->backward) { + ASSERT(pIter->pNode != pTail); + + if (pIter->pNode == pHead) { + return false; + } + } else { + ASSERT(pIter->pNode != pHead); + + if (pIter->pNode == pTail) { + return false; + } + } + + tGetTSDBRow((uint8_t *)SL_NODE_DATA(pIter->pNode), pRow); + return true; } /** @@ -89,7 +321,7 @@ void tsdbMemTableDestroy(STsdbMemTable *pMemTable) { * * The function tries to procceed AS MUCH AS POSSIBLE. */ -int tsdbLoadDataFromCache(STsdb *pTsdb, STable *pTable, SSkipListIterator *pIter, TSKEY maxKey, int maxRowsToRead, +int tsdbLoadDataFromCache(STsdb *pTsdb, STable *pTable, STbDataIter *pIter, TSKEY maxKey, int maxRowsToRead, SDataCols *pCols, TKEY *filterKeys, int nFilterKeys, bool keepDup, SMergeInfo *pMergeInfo) { ASSERT(maxRowsToRead > 0 && nFilterKeys >= 0); if (pIter == NULL) return 0; @@ -172,7 +404,7 @@ int tsdbLoadDataFromCache(STsdb *pTsdb, STable *pTable, SSkipListIterator *pIter } } - tSkipListIterNext(pIter); + tsdbTbDataIterNext(pIter); row = tsdbNextIterRow(pIter); if (row == NULL || TD_ROW_KEY(row) > maxKey) { rowKey = INT64_MAX; @@ -212,7 +444,7 @@ int tsdbLoadDataFromCache(STsdb *pTsdb, STable *pTable, SSkipListIterator *pIter } } - tSkipListIterNext(pIter); + tsdbTbDataIterNext(pIter); row = tsdbNextIterRow(pIter); if (row == NULL || TD_ROW_KEY(row) > maxKey) { rowKey = INT64_MAX; @@ -238,151 +470,281 @@ int tsdbLoadDataFromCache(STsdb *pTsdb, STable *pTable, SSkipListIterator *pIter return 0; } -int tsdbInsertTableData(STsdb *pTsdb, SSubmitMsgIter *pMsgIter, SSubmitBlk *pBlock, SSubmitBlkRsp *pRsp) { - SSubmitBlkIter blkIter = {0}; - STsdbMemTable *pMemTable = pTsdb->mem; - void *tptr; - STbData *pTbData; - STSRow *row; - TSKEY keyMin; - TSKEY keyMax; - SSubmitBlk *pBlkCopy; - int64_t sverNew; +static int32_t tsdbGetOrCreateTbData(SMemTable *pMemTable, tb_uid_t suid, tb_uid_t uid, STbData **ppTbData) { + int32_t code = 0; + int32_t idx = 0; + STbData *pTbData = NULL; + STbData *pTbDataT = &(STbData){.suid = suid, .uid = uid}; - // check if table exists - SMetaReader mr = {0}; - SMetaEntry me = {0}; - metaReaderInit(&mr, pTsdb->pVnode->pMeta, 0); - if (metaGetTableEntryByUid(&mr, pMsgIter->uid) < 0) { - metaReaderClear(&mr); - terrno = TSDB_CODE_PAR_TABLE_NOT_EXIST; - return -1; + // get + idx = taosArraySearchIdx(pMemTable->aTbData, &pTbDataT, tbDataPCmprFn, TD_GE); + if (idx >= 0) { + pTbData = (STbData *)taosArrayGetP(pMemTable->aTbData, idx); + if (tbDataPCmprFn(&pTbDataT, &pTbData) == 0) goto _exit; } - if (pRsp->tblFName) strcat(pRsp->tblFName, mr.me.name); - if (mr.me.type == TSDB_NORMAL_TABLE) { - sverNew = mr.me.ntbEntry.schemaRow.version; + // create + SVBufPool *pPool = pMemTable->pTsdb->pVnode->inUse; + int8_t maxLevel = pMemTable->pTsdb->pVnode->config.tsdbCfg.slLevel; + + pTbData = vnodeBufPoolMalloc(pPool, sizeof(*pTbData) + SL_NODE_SIZE(maxLevel) * 2); + if (pTbData == NULL) { + code = TSDB_CODE_OUT_OF_MEMORY; + goto _err; + } + pTbData->suid = suid; + pTbData->uid = uid; + pTbData->minKey = (TSDBKEY){.ts = TSKEY_MAX, .version = INT64_MAX}; + pTbData->maxKey = (TSDBKEY){.ts = TSKEY_MIN, .version = -1}; + pTbData->pHead = NULL; + pTbData->pTail = NULL; + pTbData->sl.seed = taosRand(); + pTbData->sl.size = 0; + pTbData->sl.maxLevel = maxLevel; + pTbData->sl.level = 0; + pTbData->sl.pHead = (SMemSkipListNode *)&pTbData[1]; + pTbData->sl.pTail = (SMemSkipListNode *)POINTER_SHIFT(pTbData->sl.pHead, SL_NODE_SIZE(maxLevel)); + pTbData->sl.pHead->level = maxLevel; + pTbData->sl.pTail->level = maxLevel; + for (int8_t iLevel = 0; iLevel < maxLevel; iLevel++) { + SL_NODE_FORWARD(pTbData->sl.pHead, iLevel) = pTbData->sl.pTail; + SL_NODE_BACKWARD(pTbData->sl.pTail, iLevel) = pTbData->sl.pHead; + + SL_NODE_BACKWARD(pTbData->sl.pHead, iLevel) = NULL; + SL_NODE_FORWARD(pTbData->sl.pTail, iLevel) = NULL; + } + + void *p; + if (idx < 0) { + p = taosArrayPush(pMemTable->aTbData, &pTbData); } else { - metaGetTableEntryByUid(&mr, mr.me.ctbEntry.suid); - sverNew = mr.me.stbEntry.schemaRow.version; + p = taosArrayInsert(pMemTable->aTbData, idx, &pTbData); + } + if (p == NULL) { + code = TSDB_CODE_OUT_OF_MEMORY; + goto _err; } - metaReaderClear(&mr); - // create container is nedd - tptr = taosHashGet(pMemTable->pHashIdx, &(pMsgIter->uid), sizeof(pMsgIter->uid)); - if (tptr == NULL) { - pTbData = tsdbNewTbData(pMsgIter->uid); - if (pTbData == NULL) { - return -1; +_exit: + *ppTbData = pTbData; + return code; + +_err: + *ppTbData = NULL; + return code; +} + +static void tbDataMovePosTo(STbData *pTbData, SMemSkipListNode **pos, TSDBKEY *pKey, int32_t flags) { + SMemSkipListNode *px; + SMemSkipListNode *pn; + TSDBKEY *pTKey; + int c; + int backward = flags & SL_MOVE_BACKWARD; + int fromPos = flags & SL_MOVE_FROM_POS; + + if (backward) { + px = pTbData->sl.pTail; + + for (int8_t iLevel = pTbData->sl.maxLevel - 1; iLevel >= pTbData->sl.level; iLevel--) { + pos[iLevel] = px; } - // Put into hash - taosHashPut(pMemTable->pHashIdx, &(pMsgIter->uid), sizeof(pMsgIter->uid), &(pTbData), sizeof(pTbData)); + if (pTbData->sl.level) { + if (fromPos) px = pos[pTbData->sl.level - 1]; - // Put into skiplist - tSkipListPut(pMemTable->pSlIdx, pTbData); - } else { - pTbData = *(STbData **)tptr; - } + for (int8_t iLevel = pTbData->sl.level - 1; iLevel >= 0; iLevel--) { + pn = SL_NODE_BACKWARD(px, iLevel); + while (pn != pTbData->sl.pHead) { + pTKey = (TSDBKEY *)SL_NODE_DATA(pn); - // copy data to buffer pool - int32_t tlen = pMsgIter->dataLen + pMsgIter->schemaLen + sizeof(*pBlock); - pBlkCopy = (SSubmitBlk *)vnodeBufPoolMalloc(pTsdb->mem->pPool, tlen); - memcpy(pBlkCopy, pBlock, tlen); + c = tsdbKeyCmprFn(pTKey, pKey); + if (c <= 0) { + break; + } else { + px = pn; + pn = SL_NODE_BACKWARD(px, iLevel); + } + } - tInitSubmitBlkIter(pMsgIter, pBlkCopy, &blkIter); - if (blkIter.row == NULL) return 0; - keyMin = TD_ROW_KEY(blkIter.row); - - tSkipListPutBatchByIter(pTbData->pData, &blkIter, (iter_next_fn_t)tGetSubmitBlkNext); - -#ifdef TD_DEBUG_PRINT_ROW - printf("!!! %s:%d vgId:%d dir:%s table:%" PRIi64 " has %d rows in skiplist\n\n", __func__, __LINE__, - TD_VID(pTsdb->pVnode), pTsdb->dir, pTbData->uid, SL_SIZE(pTbData->pData)); -#endif - - // Set statistics - keyMax = TD_ROW_KEY(blkIter.row); - - pTbData->nrows += pMsgIter->numOfRows; - if (pTbData->keyMin > keyMin) pTbData->keyMin = keyMin; - if (pTbData->keyMax < keyMax) pTbData->keyMax = keyMax; - - pMemTable->nRow += pMsgIter->numOfRows; - if (pMemTable->keyMin > keyMin) pMemTable->keyMin = keyMin; - if (pMemTable->keyMax < keyMax) pMemTable->keyMax = keyMax; - - pRsp->numOfRows = pMsgIter->numOfRows; - pRsp->affectedRows = pMsgIter->numOfRows; - pRsp->sver = sverNew; - - return 0; -} - -static STbData *tsdbNewTbData(tb_uid_t uid) { - STbData *pTbData = (STbData *)taosMemoryCalloc(1, sizeof(*pTbData)); - if (pTbData == NULL) { - return NULL; - } - - pTbData->uid = uid; - pTbData->keyMin = TSKEY_MAX; - pTbData->keyMax = TSKEY_MIN; - pTbData->nrows = 0; -#if 0 - pTbData->pData = tSkipListCreate(5, TSDB_DATA_TYPE_TIMESTAMP, sizeof(int64_t), tkeyComparFn, SL_DISCARD_DUP_KEY, - tsdbGetTsTupleKey); -#endif - pTbData->pData = - tSkipListCreate(5, TSDB_DATA_TYPE_TIMESTAMP, sizeof(int64_t), tkeyComparFn, SL_ALLOW_DUP_KEY, tsdbGetTsTupleKey); - if (pTbData->pData == NULL) { - taosMemoryFree(pTbData); - return NULL; - } - - return pTbData; -} - -static void tsdbFreeTbData(STbData *pTbData) { - if (pTbData) { - tSkipListDestroy(pTbData->pData); - taosMemoryFree(pTbData); - } -} - -static char *tsdbGetTsTupleKey(const void *data) { return (char *)TD_ROW_KEY_ADDR((STSRow *)data); } - -static int tsdbTbDataComp(const void *arg1, const void *arg2) { - STbData *pTbData1 = (STbData *)arg1; - STbData *pTbData2 = (STbData *)arg2; - - if (pTbData1->uid > pTbData2->uid) { - return 1; - } else if (pTbData1->uid == pTbData2->uid) { - return 0; - } else { - return -1; - } -} - -static char *tsdbTbDataGetUid(const void *arg) { - STbData *pTbData = (STbData *)arg; - return (char *)(&(pTbData->uid)); -} - -static int tsdbAppendTableRowToCols(STsdb *pTsdb, STable *pTable, SDataCols *pCols, STSchema **ppSchema, STSRow *row, - bool merge) { - if (pCols) { - if (*ppSchema == NULL || schemaVersion(*ppSchema) != TD_ROW_SVER(row)) { - *ppSchema = tsdbGetTableSchemaImpl(pTsdb, pTable, false, false, TD_ROW_SVER(row)); - if (*ppSchema == NULL) { - ASSERT(false); - return -1; + pos[iLevel] = px; } } + } else { + px = pTbData->sl.pHead; - tdAppendSTSRowToDataCol(row, *ppSchema, pCols, merge); + for (int8_t iLevel = pTbData->sl.maxLevel - 1; iLevel >= pTbData->sl.level; iLevel--) { + pos[iLevel] = px; + } + + if (pTbData->sl.level) { + if (fromPos) px = pos[pTbData->sl.level - 1]; + + for (int8_t iLevel = pTbData->sl.level - 1; iLevel >= 0; iLevel--) { + pn = SL_NODE_FORWARD(px, iLevel); + while (pn != pTbData->sl.pTail) { + pTKey = (TSDBKEY *)SL_NODE_DATA(pn); + + c = tsdbKeyCmprFn(pTKey, pKey); + if (c >= 0) { + break; + } else { + px = pn; + pn = SL_NODE_FORWARD(px, iLevel); + } + } + + pos[iLevel] = px; + } + } + } +} + +static FORCE_INLINE int8_t tsdbMemSkipListRandLevel(SMemSkipList *pSl) { + int8_t level = 1; + int8_t tlevel = TMIN(pSl->maxLevel, pSl->level + 1); + const uint32_t factor = 4; + + while ((taosRandR(&pSl->seed) % factor) == 0 && level < tlevel) { + level++; } - return 0; + return level; +} +static int32_t tbDataDoPut(SMemTable *pMemTable, STbData *pTbData, SMemSkipListNode **pos, TSDBROW *pRow, + int8_t forward) { + int32_t code = 0; + int8_t level; + SMemSkipListNode *pNode; + SVBufPool *pPool = pMemTable->pTsdb->pVnode->inUse; + + // node + level = tsdbMemSkipListRandLevel(&pTbData->sl); + pNode = (SMemSkipListNode *)vnodeBufPoolMalloc(pPool, SL_NODE_SIZE(level) + tPutTSDBRow(NULL, pRow)); + if (pNode == NULL) { + code = TSDB_CODE_OUT_OF_MEMORY; + goto _exit; + } + pNode->level = level; + for (int8_t iLevel = 0; iLevel < level; iLevel++) { + SL_NODE_FORWARD(pNode, iLevel) = NULL; + SL_NODE_BACKWARD(pNode, iLevel) = NULL; + } + + tPutTSDBRow((uint8_t *)SL_NODE_DATA(pNode), pRow); + + // put + for (int8_t iLevel = 0; iLevel < pNode->level; iLevel++) { + SMemSkipListNode *px = pos[iLevel]; + + if (forward) { + SMemSkipListNode *pNext = SL_NODE_FORWARD(px, iLevel); + + SL_NODE_FORWARD(pNode, iLevel) = pNext; + SL_NODE_BACKWARD(pNode, iLevel) = px; + + SL_NODE_BACKWARD(pNext, iLevel) = pNode; + SL_NODE_FORWARD(px, iLevel) = pNode; + } else { + SMemSkipListNode *pPrev = SL_NODE_BACKWARD(px, iLevel); + + SL_NODE_FORWARD(pNode, iLevel) = px; + SL_NODE_BACKWARD(pNode, iLevel) = pPrev; + + SL_NODE_FORWARD(pPrev, iLevel) = pNode; + SL_NODE_BACKWARD(px, iLevel) = pNode; + } + } + + pTbData->sl.size++; + if (pTbData->sl.level < pNode->level) { + pTbData->sl.level = pNode->level; + } + +_exit: + return code; +} + +static int32_t tsdbInsertTableDataImpl(SMemTable *pMemTable, STbData *pTbData, int64_t version, + SSubmitMsgIter *pMsgIter, SSubmitBlk *pBlock, SSubmitBlkRsp *pRsp) { + int32_t code = 0; + SSubmitBlkIter blkIter = {0}; + TSDBKEY key = {.version = version}; + SMemSkipListNode *pos[SL_MAX_LEVEL]; + TSDBROW row = {.version = version, .pTSRow = NULL}; + int32_t nRow = 0; + + tInitSubmitBlkIter(pMsgIter, pBlock, &blkIter); + + // backward put first data + row.pTSRow = tGetSubmitBlkNext(&blkIter); + key.ts = row.pTSRow->ts; + nRow++; + tbDataMovePosTo(pTbData, pos, &key, SL_MOVE_BACKWARD); + code = tbDataDoPut(pMemTable, pTbData, pos, &row, 0); + if (code) { + goto _err; + } + + if (tsdbKeyCmprFn(&key, &pTbData->minKey) < 0) { + pTbData->minKey = key; + } + + if (tsdbKeyCmprFn(&key, &pMemTable->minKey) < 0) { + pMemTable->minKey = key; + } + + // forward put rest data + row.pTSRow = tGetSubmitBlkNext(&blkIter); + if (row.pTSRow) { + for (int8_t iLevel = 0; iLevel < pTbData->sl.maxLevel; iLevel++) { + pos[iLevel] = SL_NODE_BACKWARD(pos[iLevel], iLevel); + } + do { + key.ts = row.pTSRow->ts; + nRow++; + tbDataMovePosTo(pTbData, pos, &key, SL_MOVE_FROM_POS); + code = tbDataDoPut(pMemTable, pTbData, pos, &row, 1); + if (code) { + goto _err; + } + + row.pTSRow = tGetSubmitBlkNext(&blkIter); + } while (row.pTSRow); + } + + if (tsdbKeyCmprFn(&key, &pTbData->maxKey) > 0) { + pTbData->maxKey = key; + } + + if (tsdbKeyCmprFn(&key, &pMemTable->maxKey) > 0) { + pMemTable->maxKey = key; + } + pMemTable->nRef++; + pRsp->numOfRows = nRow; + pRsp->affectedRows = nRow; + + return code; + +_err: + return code; +} + +static int32_t tPutTSDBRow(uint8_t *p, TSDBROW *pRow) { + int32_t n = 0; + + n += tPutI64(p, pRow->version); + if (p) memcpy(p + n, pRow->pTSRow, pRow->pTSRow->len); + n += pRow->pTSRow->len; + + return n; +} + +static int32_t tGetTSDBRow(uint8_t *p, TSDBROW *pRow) { + int32_t n = 0; + + n += tGetI64(p, &pRow->version); + pRow->pTSRow = (STSRow *)(p + n); + n += pRow->pTSRow->len; + + return n; } \ No newline at end of file diff --git a/source/dnode/vnode/src/tsdb/tsdbMemTable2.c b/source/dnode/vnode/src/tsdb/tsdbMemTable2.c index cf8557bba3..aa58cc92a0 100644 --- a/source/dnode/vnode/src/tsdb/tsdbMemTable2.c +++ b/source/dnode/vnode/src/tsdb/tsdbMemTable2.c @@ -15,11 +15,6 @@ #include "tsdb.h" -struct SMemSkipListNode { - int8_t level; - SMemSkipListNode *forwards[0]; -}; - typedef struct { tb_uid_t uid; STSchema *pTSchema; @@ -35,21 +30,21 @@ typedef struct { #define SL_MOVE_BACKWARD 0x1 #define SL_MOVE_FROM_POS 0x2 -static int32_t tsdbGetOrCreateMemData(SMemTable *pMemTable, tb_uid_t suid, tb_uid_t uid, SMemData **ppMemData); +static int32_t tsdbGetOrCreateMemData(SMemTable2 *pMemTable, tb_uid_t suid, tb_uid_t uid, SMemData **ppMemData); static int memDataPCmprFn(const void *p1, const void *p2); static int32_t tPutTSDBRow(uint8_t *p, TSDBROW *pRow); static int32_t tGetTSDBRow(uint8_t *p, TSDBROW *pRow); static int8_t tsdbMemSkipListRandLevel(SMemSkipList *pSl); -static int32_t tsdbInsertTableDataImpl(SMemTable *pMemTable, SMemData *pMemData, int64_t version, +static int32_t tsdbInsertTableDataImpl(SMemTable2 *pMemTable, SMemData *pMemData, int64_t version, SVSubmitBlk *pSubmitBlk); static void memDataMovePosTo(SMemData *pMemData, SMemSkipListNode **pos, TSDBKEY *pKey, int32_t flags); // SMemTable ============================================== -int32_t tsdbMemTableCreate2(STsdb *pTsdb, SMemTable **ppMemTable) { - int32_t code = 0; - SMemTable *pMemTable = NULL; +int32_t tsdbMemTableCreate2(STsdb *pTsdb, SMemTable2 **ppMemTable) { + int32_t code = 0; + SMemTable2 *pMemTable = NULL; - pMemTable = (SMemTable *)taosMemoryCalloc(1, sizeof(*pMemTable)); + pMemTable = (SMemTable2 *)taosMemoryCalloc(1, sizeof(*pMemTable)); if (pMemTable == NULL) { code = TSDB_CODE_OUT_OF_MEMORY; goto _err; @@ -75,16 +70,16 @@ _err: return code; } -void tsdbMemTableDestroy2(SMemTable *pMemTable) { +void tsdbMemTableDestroy2(SMemTable2 *pMemTable) { taosArrayDestroyEx(pMemTable->aMemData, NULL /*TODO*/); taosMemoryFree(pMemTable); } int32_t tsdbInsertTableData2(STsdb *pTsdb, int64_t version, SVSubmitBlk *pSubmitBlk) { - int32_t code = 0; - SMemTable *pMemTable = (SMemTable *)pTsdb->mem; // TODO - SMemData *pMemData; - TSDBROW row = {.version = version}; + int32_t code = 0; + SMemTable2 *pMemTable = (SMemTable2 *)pTsdb->mem; // TODO + SMemData *pMemData; + TSDBROW row = {.version = version}; ASSERT(pMemTable); ASSERT(pSubmitBlk->nData > 0); @@ -112,10 +107,10 @@ _err: } int32_t tsdbDeleteTableData2(STsdb *pTsdb, int64_t version, tb_uid_t suid, tb_uid_t uid, TSKEY sKey, TSKEY eKey) { - int32_t code = 0; - SMemTable *pMemTable = (SMemTable *)pTsdb->mem; // TODO - SMemData *pMemData; - SVBufPool *pPool = pTsdb->pVnode->inUse; + int32_t code = 0; + SMemTable2 *pMemTable = (SMemTable2 *)pTsdb->mem; // TODO + SMemData *pMemData; + SVBufPool *pPool = pTsdb->pVnode->inUse; ASSERT(pMemTable); @@ -250,7 +245,7 @@ void tsdbMemDataIterGet(SMemDataIter *pIter, TSDBROW **ppRow) { } } -static int32_t tsdbGetOrCreateMemData(SMemTable *pMemTable, tb_uid_t suid, tb_uid_t uid, SMemData **ppMemData) { +static int32_t tsdbGetOrCreateMemData(SMemTable2 *pMemTable, tb_uid_t suid, tb_uid_t uid, SMemData **ppMemData) { int32_t code = 0; int32_t idx = 0; SMemData *pMemDataT = &(SMemData){.suid = suid, .uid = uid}; @@ -421,7 +416,7 @@ static void memDataMovePosTo(SMemData *pMemData, SMemSkipListNode **pos, TSDBKEY } } -static int32_t memDataDoPut(SMemTable *pMemTable, SMemData *pMemData, SMemSkipListNode **pos, TSDBROW *pRow, +static int32_t memDataDoPut(SMemTable2 *pMemTable, SMemData *pMemData, SMemSkipListNode **pos, TSDBROW *pRow, int8_t forward) { int32_t code = 0; int8_t level; @@ -475,7 +470,7 @@ _exit: return code; } -static int32_t tsdbInsertTableDataImpl(SMemTable *pMemTable, SMemData *pMemData, int64_t version, +static int32_t tsdbInsertTableDataImpl(SMemTable2 *pMemTable, SMemData *pMemData, int64_t version, SVSubmitBlk *pSubmitBlk) { int32_t code = 0; int32_t n = 0; diff --git a/source/dnode/vnode/src/tsdb/tsdbRead.c b/source/dnode/vnode/src/tsdb/tsdbRead.c index 5fdb052ee5..146757096f 100644 --- a/source/dnode/vnode/src/tsdb/tsdbRead.c +++ b/source/dnode/vnode/src/tsdb/tsdbRead.c @@ -67,15 +67,16 @@ enum { }; typedef struct STableCheckInfo { - uint64_t tableId; - TSKEY lastKey; - SBlockInfo* pCompInfo; - int32_t compSize; - int32_t numOfBlocks : 29; // number of qualified data blocks not the original blocks - uint8_t chosen : 2; // indicate which iterator should move forward - bool initBuf : 1; // whether to initialize the in-memory skip list iterator or not - SSkipListIterator* iter; // mem buffer skip list iterator - SSkipListIterator* iiter; // imem buffer skip list iterator + uint64_t suid; + uint64_t tableId; + TSKEY lastKey; + SBlockInfo* pCompInfo; + int32_t compSize; + int32_t numOfBlocks : 29; // number of qualified data blocks not the original blocks + uint8_t chosen : 2; // indicate which iterator should move forward + bool initBuf : 1; // whether to initialize the in-memory skip list iterator or not + STbDataIter* iter; // mem buffer skip list iterator + STbDataIter* iiter; // imem buffer skip list iterator } STableCheckInfo; typedef struct STableBlockInfo { @@ -107,6 +108,7 @@ typedef struct SBlockLoadSuppInfo { typedef struct STsdbReadHandle { STsdb* pTsdb; + uint64_t suid; SQueryFilePos cur; // current position int16_t order; STimeWindow window; // the primary query time window that applies to all queries @@ -200,8 +202,8 @@ static SArray* getDefaultLoadColumns(STsdbReadHandle* pTsdbReadHandle, bool load int64_t tsdbGetNumOfRowsInMemTable(tsdbReaderT* pHandle) { STsdbReadHandle* pTsdbReadHandle = (STsdbReadHandle*)pHandle; - int64_t rows = 0; - STsdbMemTable* pMemTable = NULL; // pTsdbReadHandle->pMemTable; + int64_t rows = 0; + SMemTable* pMemTable = NULL; // pTsdbReadHandle->pMemTable; if (pMemTable == NULL) { return rows; } @@ -237,6 +239,7 @@ static SArray* createCheckInfoFromTableGroup(STsdbReadHandle* pTsdbReadHandle, S STableKeyInfo* pKeyInfo = (STableKeyInfo*)taosArrayGet(pTableList->pTableList, j); STableCheckInfo info = {.lastKey = pKeyInfo->lastKey, .tableId = pKeyInfo->uid}; + info.suid = pTsdbReadHandle->suid; if (ASCENDING_TRAVERSE(pTsdbReadHandle->order)) { if (info.lastKey == INT64_MIN || info.lastKey < pTsdbReadHandle->window.skey) { info.lastKey = pTsdbReadHandle->window.skey; @@ -265,8 +268,8 @@ static void resetCheckInfo(STsdbReadHandle* pTsdbReadHandle) { for (int32_t i = 0; i < numOfTables; ++i) { STableCheckInfo* pCheckInfo = (STableCheckInfo*)taosArrayGet(pTsdbReadHandle->pTableCheckInfo, i); pCheckInfo->lastKey = pTsdbReadHandle->window.skey; - pCheckInfo->iter = tSkipListDestroyIter(pCheckInfo->iter); - pCheckInfo->iiter = tSkipListDestroyIter(pCheckInfo->iiter); + pCheckInfo->iter = tsdbTbDataIterDestroy(pCheckInfo->iter); + pCheckInfo->iiter = tsdbTbDataIterDestroy(pCheckInfo->iiter); pCheckInfo->initBuf = false; if (ASCENDING_TRAVERSE(pTsdbReadHandle->order)) { @@ -387,6 +390,7 @@ static STsdbReadHandle* tsdbQueryTablesImpl(SVnode* pVnode, SQueryTableDataCond* pReadHandle->locateStart = false; pReadHandle->loadType = pCond->type; + pReadHandle->suid = pCond->suid; pReadHandle->outputCapacity = 4096; //((STsdb*)tsdb)->config.maxRowsPerFileBlock; pReadHandle->loadExternalRow = pCond->loadExternalRows; pReadHandle->currentLoadExternalRows = pCond->loadExternalRows; @@ -658,7 +662,7 @@ tsdbReaderT tsdbQueryLastRow(SVnode* pVnode, SQueryTableDataCond* pCond, STableL } #if 0 -tsdbReaderT tsdbQueryCacheLastT(STsdb *tsdb, SQueryTableDataCond *pCond, STableGroupInfo *groupList, uint64_t qId, STsdbMemTable* pMemRef) { +tsdbReaderT tsdbQueryCacheLastT(STsdb *tsdb, SQueryTableDataCond *pCond, STableGroupInfo *groupList, uint64_t qId, SMemTable* pMemRef) { STsdbReadHandle *pTsdbReadHandle = (STsdbReadHandle*) tsdbQueryTablesT(tsdb, pCond, groupList, qId, pMemRef); if (pTsdbReadHandle == NULL) { return NULL; @@ -752,23 +756,22 @@ static bool initTableMemIterator(STsdbReadHandle* pHandle, STableCheckInfo* pChe pCheckInfo->initBuf = true; int32_t order = pHandle->order; - STbData** pMem = NULL; - STbData** pIMem = NULL; + STbData* pMem = NULL; + STbData* pIMem = NULL; + int8_t backward = (pHandle->order == TSDB_ORDER_DESC) ? 1 : 0; TSKEY tLastKey = keyToTkey(pCheckInfo->lastKey); if (pHandle->pTsdb->mem != NULL) { - pMem = taosHashGet(pHandle->pTsdb->mem->pHashIdx, &pCheckInfo->tableId, sizeof(pCheckInfo->tableId)); + tsdbGetTbDataFromMemTable(pHandle->pTsdb->mem, pCheckInfo->suid, pCheckInfo->tableId, &pMem); if (pMem != NULL) { - pCheckInfo->iter = - tSkipListCreateIterFromVal((*pMem)->pData, (const char*)&tLastKey, TSDB_DATA_TYPE_TIMESTAMP, order); + tsdbTbDataIterCreate(pMem, &(TSDBKEY){.version = 0, .ts = tLastKey}, backward, &pCheckInfo->iter); } } if (pHandle->pTsdb->imem != NULL) { - pIMem = taosHashGet(pHandle->pTsdb->imem->pHashIdx, &pCheckInfo->tableId, sizeof(pCheckInfo->tableId)); + tsdbGetTbDataFromMemTable(pHandle->pTsdb->mem, pCheckInfo->suid, pCheckInfo->tableId, &pIMem); if (pIMem != NULL) { - pCheckInfo->iiter = - tSkipListCreateIterFromVal((*pIMem)->pData, (const char*)&tLastKey, TSDB_DATA_TYPE_TIMESTAMP, order); + tsdbTbDataIterCreate(pIMem, &(TSDBKEY){.version = 0, .ts = tLastKey}, backward, &pCheckInfo->iiter); } } @@ -777,22 +780,23 @@ static bool initTableMemIterator(STsdbReadHandle* pHandle, STableCheckInfo* pChe return false; } - bool memEmpty = (pCheckInfo->iter == NULL) || (pCheckInfo->iter != NULL && !tSkipListIterNext(pCheckInfo->iter)); - bool imemEmpty = (pCheckInfo->iiter == NULL) || (pCheckInfo->iiter != NULL && !tSkipListIterNext(pCheckInfo->iiter)); + bool memEmpty = + (pCheckInfo->iter == NULL) || (pCheckInfo->iter != NULL && !tsdbTbDataIterGet(pCheckInfo->iter, NULL)); + bool imemEmpty = + (pCheckInfo->iiter == NULL) || (pCheckInfo->iiter != NULL && !tsdbTbDataIterGet(pCheckInfo->iiter, NULL)); if (memEmpty && imemEmpty) { // buffer is empty return false; } if (!memEmpty) { - SSkipListNode* node = tSkipListIterGet(pCheckInfo->iter); - assert(node != NULL); + TSDBROW row; - STSRow* row = (STSRow*)SL_GET_NODE_DATA(node); - TSKEY key = TD_ROW_KEY(row); // first timestamp in buffer + tsdbTbDataIterGet(pCheckInfo->iter, &row); + TSKEY key = row.pTSRow->ts; // first timestamp in buffer tsdbDebug("%p uid:%" PRId64 ", check data in mem from skey:%" PRId64 ", order:%d, ts range in buf:%" PRId64 "-%" PRId64 ", lastKey:%" PRId64 ", numOfRows:%" PRId64 ", %s", - pHandle, pCheckInfo->tableId, key, order, (*pMem)->keyMin, (*pMem)->keyMax, pCheckInfo->lastKey, - (*pMem)->nrows, pHandle->idStr); + pHandle, pCheckInfo->tableId, key, order, pMem->minKey.ts, pMem->maxKey.ts, pCheckInfo->lastKey, + pMem->sl.size, pHandle->idStr); if (ASCENDING_TRAVERSE(order)) { assert(pCheckInfo->lastKey <= key); @@ -805,15 +809,14 @@ static bool initTableMemIterator(STsdbReadHandle* pHandle, STableCheckInfo* pChe } if (!imemEmpty) { - SSkipListNode* node = tSkipListIterGet(pCheckInfo->iiter); - assert(node != NULL); + TSDBROW row; - STSRow* row = (STSRow*)SL_GET_NODE_DATA(node); - TSKEY key = TD_ROW_KEY(row); // first timestamp in buffer + tsdbTbDataIterGet(pCheckInfo->iter, &row); + TSKEY key = row.pTSRow->ts; // first timestamp in buffer tsdbDebug("%p uid:%" PRId64 ", check data in imem from skey:%" PRId64 ", order:%d, ts range in buf:%" PRId64 "-%" PRId64 ", lastKey:%" PRId64 ", numOfRows:%" PRId64 ", %s", - pHandle, pCheckInfo->tableId, key, order, (*pIMem)->keyMin, (*pIMem)->keyMax, pCheckInfo->lastKey, - (*pIMem)->nrows, pHandle->idStr); + pHandle, pCheckInfo->tableId, key, order, pIMem->minKey.ts, pIMem->maxKey.ts, pCheckInfo->lastKey, + pIMem->sl.size, pHandle->idStr); if (ASCENDING_TRAVERSE(order)) { assert(pCheckInfo->lastKey <= key); @@ -828,31 +831,23 @@ static bool initTableMemIterator(STsdbReadHandle* pHandle, STableCheckInfo* pChe } static void destroyTableMemIterator(STableCheckInfo* pCheckInfo) { - tSkipListDestroyIter(pCheckInfo->iter); - tSkipListDestroyIter(pCheckInfo->iiter); + tsdbTbDataIterDestroy(pCheckInfo->iter); + tsdbTbDataIterDestroy(pCheckInfo->iiter); } static TSKEY extractFirstTraverseKey(STableCheckInfo* pCheckInfo, int32_t order, int32_t update, TDRowVerT maxVer) { + TSDBROW row = {0}; STSRow *rmem = NULL, *rimem = NULL; + if (pCheckInfo->iter) { - SSkipListNode* node = tSkipListIterGet(pCheckInfo->iter); - if (node != NULL) { - rmem = (STSRow*)SL_GET_NODE_DATA(node); - // TODO: filter max version - // if (TD_ROW_VER(rmem) > maxVer) { - // rmem = NULL; - // } + if (tsdbTbDataIterGet(pCheckInfo->iter, &row)) { + rmem = row.pTSRow; } } if (pCheckInfo->iiter) { - SSkipListNode* node = tSkipListIterGet(pCheckInfo->iiter); - if (node != NULL) { - rimem = (STSRow*)SL_GET_NODE_DATA(node); - // TODO: filter max version - // if (TD_ROW_VER(rimem) > maxVer) { - // rimem = NULL; - // } + if (tsdbTbDataIterGet(pCheckInfo->iiter, &row)) { + rimem = row.pTSRow; } } @@ -889,7 +884,7 @@ static TSKEY extractFirstTraverseKey(STableCheckInfo* pCheckInfo, int32_t order, pCheckInfo->chosen = CHECKINFO_CHOSEN_BOTH; } else { pCheckInfo->chosen = CHECKINFO_CHOSEN_IMEM; - tSkipListIterNext(pCheckInfo->iter); + tsdbTbDataIterNext(pCheckInfo->iter); } return r1; } else if (r1 < r2 && ASCENDING_TRAVERSE(order)) { @@ -903,28 +898,17 @@ static TSKEY extractFirstTraverseKey(STableCheckInfo* pCheckInfo, int32_t order, static STSRow* getSRowInTableMem(STableCheckInfo* pCheckInfo, int32_t order, int32_t update, STSRow** extraRow, TDRowVerT maxVer) { + TSDBROW row; STSRow *rmem = NULL, *rimem = NULL; if (pCheckInfo->iter) { - SSkipListNode* node = tSkipListIterGet(pCheckInfo->iter); - if (node != NULL) { - rmem = (STSRow*)SL_GET_NODE_DATA(node); -#if 0 // TODO: skiplist refactor - if (TD_ROW_VER(rmem) > maxVer) { - rmem = NULL; - } -#endif + if (tsdbTbDataIterGet(pCheckInfo->iter, &row)) { + rmem = row.pTSRow; } } if (pCheckInfo->iiter) { - SSkipListNode* node = tSkipListIterGet(pCheckInfo->iiter); - if (node != NULL) { - rimem = (STSRow*)SL_GET_NODE_DATA(node); -#if 0 // TODO: skiplist refactor - if (TD_ROW_VER(rimem) > maxVer) { - rimem = NULL; - } -#endif + if (tsdbTbDataIterGet(pCheckInfo->iiter, &row)) { + rimem = row.pTSRow; } } @@ -966,7 +950,7 @@ static STSRow* getSRowInTableMem(STableCheckInfo* pCheckInfo, int32_t order, int *extraRow = rimem; return rmem; } else { - tSkipListIterNext(pCheckInfo->iter); + tsdbTbDataIterNext(pCheckInfo->iter); pCheckInfo->chosen = CHECKINFO_CHOSEN_IMEM; return rimem; } @@ -995,7 +979,7 @@ static bool moveToNextRowInMem(STableCheckInfo* pCheckInfo) { bool hasNext = false; if (pCheckInfo->chosen == CHECKINFO_CHOSEN_MEM) { if (pCheckInfo->iter != NULL) { - hasNext = tSkipListIterNext(pCheckInfo->iter); + hasNext = tsdbTbDataIterNext(pCheckInfo->iter); } if (hasNext) { @@ -1003,11 +987,11 @@ static bool moveToNextRowInMem(STableCheckInfo* pCheckInfo) { } if (pCheckInfo->iiter != NULL) { - return tSkipListIterGet(pCheckInfo->iiter) != NULL; + return tsdbTbDataIterGet(pCheckInfo->iiter, NULL); } } else if (pCheckInfo->chosen == CHECKINFO_CHOSEN_IMEM) { if (pCheckInfo->iiter != NULL) { - hasNext = tSkipListIterNext(pCheckInfo->iiter); + hasNext = tsdbTbDataIterNext(pCheckInfo->iiter); } if (hasNext) { @@ -1015,14 +999,14 @@ static bool moveToNextRowInMem(STableCheckInfo* pCheckInfo) { } if (pCheckInfo->iter != NULL) { - return tSkipListIterGet(pCheckInfo->iter) != NULL; + return tsdbTbDataIterGet(pCheckInfo->iter, NULL); } } else { if (pCheckInfo->iter != NULL) { - hasNext = tSkipListIterNext(pCheckInfo->iter); + hasNext = tsdbTbDataIterNext(pCheckInfo->iter); } if (pCheckInfo->iiter != NULL) { - hasNext = tSkipListIterNext(pCheckInfo->iiter) || hasNext; + hasNext = tsdbTbDataIterNext(pCheckInfo->iiter) || hasNext; } } @@ -1126,7 +1110,7 @@ static int32_t loadBlockInfo(STsdbReadHandle* pTsdbReadHandle, int32_t index, in STableCheckInfo* pCheckInfo = taosArrayGet(pTsdbReadHandle->pTableCheckInfo, index); pCheckInfo->numOfBlocks = 0; - STable table = {.uid = pCheckInfo->tableId, .tid = pCheckInfo->tableId}; + STable table = {.uid = pCheckInfo->tableId, .suid = pCheckInfo->suid}; table.pSchema = pTsdbReadHandle->pSchema; if (tsdbSetReadTable(&pTsdbReadHandle->rhelper, &table) != TSDB_CODE_SUCCESS) { @@ -2924,7 +2908,7 @@ static bool loadBlockOfActiveTable(STsdbReadHandle* pTsdbReadHandle) { // current result is empty if (pTsdbReadHandle->currentLoadExternalRows && pTsdbReadHandle->window.skey == pTsdbReadHandle->window.ekey && pTsdbReadHandle->cur.rows == 0) { - // STsdbMemTable* pMemRef = pTsdbReadHandle->pMemTable; + // SMemTable* pMemRef = pTsdbReadHandle->pMemTable; // doGetExternalRow(pTsdbReadHandle, TSDB_PREV_ROW, pMemRef); // doGetExternalRow(pTsdbReadHandle, TSDB_NEXT_ROW, pMemRef); @@ -3222,7 +3206,7 @@ bool tsdbNextDataBlock(tsdbReaderT pHandle) { } } -// static int32_t doGetExternalRow(STsdbReadHandle* pTsdbReadHandle, int16_t type, STsdbMemTable* pMemRef) { +// static int32_t doGetExternalRow(STsdbReadHandle* pTsdbReadHandle, int16_t type, SMemTable* pMemRef) { // STsdbReadHandle* pSecQueryHandle = NULL; // // if (type == TSDB_PREV_ROW && pTsdbReadHandle->prev) { diff --git a/source/dnode/vnode/src/tsdb/tsdbReadImpl.c b/source/dnode/vnode/src/tsdb/tsdbReadImpl.c index a26436156d..747f1f3a71 100644 --- a/source/dnode/vnode/src/tsdb/tsdbReadImpl.c +++ b/source/dnode/vnode/src/tsdb/tsdbReadImpl.c @@ -156,6 +156,24 @@ int tsdbLoadBlockIdx(SReadH *pReadh) { return 0; } +static int32_t tsdbBlockIdxCmprFn(const void *p1, const void *p2) { + SBlockIdx *pBlockIdx1 = (SBlockIdx *)p1; + SBlockIdx *pBlockIdx2 = (SBlockIdx *)p2; + + if (pBlockIdx1->suid < pBlockIdx2->suid) { + return -1; + } else if (pBlockIdx1->suid > pBlockIdx2->suid) { + return 1; + } + + if (pBlockIdx1->uid < pBlockIdx2->uid) { + return -1; + } else if (pBlockIdx1->uid > pBlockIdx2->uid) { + return 1; + } + + return 0; +} int tsdbSetReadTable(SReadH *pReadh, STable *pTable) { STSchema *pSchema = tsdbGetTableSchemaImpl(TSDB_READ_REPO(pReadh), pTable, false, false, -1); @@ -171,33 +189,40 @@ int tsdbSetReadTable(SReadH *pReadh, STable *pTable) { return -1; } - size_t size = taosArrayGetSize(pReadh->aBlkIdx); - if (size > 0) { - while (true) { - if (pReadh->cidx >= size) { - pReadh->pBlkIdx = NULL; - break; - } - - SBlockIdx *pBlkIdx = taosArrayGet(pReadh->aBlkIdx, pReadh->cidx); - if (pBlkIdx->uid == TABLE_TID(pTable)) { - if (pBlkIdx->uid == TABLE_UID(pTable)) { - pReadh->pBlkIdx = pBlkIdx; - } else { - pReadh->pBlkIdx = NULL; - } - pReadh->cidx++; - break; - } else if (pBlkIdx->uid > TABLE_TID(pTable)) { - pReadh->pBlkIdx = NULL; - break; - } else { - pReadh->cidx++; - } - } - } else { + uint8_t *p = taosArraySearch(pReadh->aBlkIdx, &(SBlockIdx){.suid = pTable->suid, .uid = pTable->uid}, + tsdbBlockIdxCmprFn, TD_EQ); + if (p == NULL) { pReadh->pBlkIdx = NULL; + } else { + pReadh->pBlkIdx = (SBlockIdx *)p; } + // size_t size = taosArrayGetSize(pReadh->aBlkIdx); + // if (size > 0) { + // while (true) { + // if (pReadh->cidx >= size) { + // pReadh->pBlkIdx = NULL; + // break; + // } + + // SBlockIdx *pBlkIdx = taosArrayGet(pReadh->aBlkIdx, pReadh->cidx); + // if (pBlkIdx->uid == TABLE_TID(pTable)) { + // if (pBlkIdx->uid == TABLE_UID(pTable)) { + // pReadh->pBlkIdx = pBlkIdx; + // } else { + // pReadh->pBlkIdx = NULL; + // } + // pReadh->cidx++; + // break; + // } else if (pBlkIdx->uid > TABLE_TID(pTable)) { + // pReadh->pBlkIdx = NULL; + // break; + // } else { + // pReadh->cidx++; + // } + // } + // } else { + // pReadh->pBlkIdx = NULL; + // } return 0; } @@ -553,12 +578,12 @@ static int tsdbLoadBlockOffset(SReadH *pReadh, SBlock *pBlock) { int tsdbEncodeSBlockIdx(void **buf, SBlockIdx *pIdx) { int tlen = 0; - // tlen += taosEncodeVariantI32(buf, pIdx->tid); + tlen += taosEncodeFixedU64(buf, pIdx->suid); + tlen += taosEncodeFixedU64(buf, pIdx->uid); tlen += taosEncodeVariantU32(buf, pIdx->len); tlen += taosEncodeVariantU32(buf, pIdx->offset); tlen += taosEncodeFixedU8(buf, pIdx->hasLast); tlen += taosEncodeVariantU32(buf, pIdx->numOfBlocks); - tlen += taosEncodeFixedU64(buf, pIdx->uid); tlen += taosEncodeFixedU64(buf, pIdx->maxKey.ts); return tlen; @@ -570,6 +595,10 @@ void *tsdbDecodeSBlockIdx(void *buf, SBlockIdx *pIdx) { uint64_t value = 0; // if ((buf = taosDecodeVariantI32(buf, &(pIdx->tid))) == NULL) return NULL; + if ((buf = taosDecodeFixedU64(buf, &value)) == NULL) return NULL; + pIdx->suid = (int64_t)value; + if ((buf = taosDecodeFixedU64(buf, &value)) == NULL) return NULL; + pIdx->uid = (int64_t)value; if ((buf = taosDecodeVariantU32(buf, &(pIdx->len))) == NULL) return NULL; if ((buf = taosDecodeVariantU32(buf, &(pIdx->offset))) == NULL) return NULL; if ((buf = taosDecodeFixedU8(buf, &(hasLast))) == NULL) return NULL; @@ -577,8 +606,6 @@ void *tsdbDecodeSBlockIdx(void *buf, SBlockIdx *pIdx) { if ((buf = taosDecodeVariantU32(buf, &(numOfBlocks))) == NULL) return NULL; pIdx->numOfBlocks = numOfBlocks; if ((buf = taosDecodeFixedU64(buf, &value)) == NULL) return NULL; - pIdx->uid = (int64_t)value; - if ((buf = taosDecodeFixedU64(buf, &value)) == NULL) return NULL; pIdx->maxKey.ts = (TSKEY)value; return buf; diff --git a/source/dnode/vnode/src/tsdb/tsdbWrite.c b/source/dnode/vnode/src/tsdb/tsdbWrite.c index 6faf6bd167..e184763bc8 100644 --- a/source/dnode/vnode/src/tsdb/tsdbWrite.c +++ b/source/dnode/vnode/src/tsdb/tsdbWrite.c @@ -39,7 +39,7 @@ int tsdbInsertData(STsdb *pTsdb, int64_t version, SSubmitReq *pMsg, SSubmitRsp * SSubmitBlkRsp r = {0}; tGetSubmitMsgNext(&msgIter, &pBlock); if (pBlock == NULL) break; - if (tsdbInsertTableData(pTsdb, &msgIter, pBlock, &r) < 0) { + if (tsdbInsertTableData(pTsdb, version, &msgIter, pBlock, &r) < 0) { return -1; } diff --git a/source/dnode/vnode/src/vnd/vnodeSvr.c b/source/dnode/vnode/src/vnd/vnodeSvr.c index 3eee6713d8..7a9c9ef393 100644 --- a/source/dnode/vnode/src/vnd/vnodeSvr.c +++ b/source/dnode/vnode/src/vnd/vnodeSvr.c @@ -158,7 +158,7 @@ int32_t vnodeProcessWriteReq(SVnode *pVnode, SRpcMsg *pMsg, int64_t version, SRp // TODO: handle error } break; - case TDMT_VND_TASK_DEPLOY: { + case TDMT_STREAM_TASK_DEPLOY: { if (tqProcessTaskDeploy(pVnode->pTq, POINTER_SHIFT(pMsg->pCont, sizeof(SMsgHead)), pMsg->contLen - sizeof(SMsgHead)) < 0) { } @@ -238,21 +238,19 @@ int32_t vnodeProcessFetchMsg(SVnode *pVnode, SRpcMsg *pMsg, SQueueInfo *pInfo) { case TDMT_VND_TABLE_META: return vnodeGetTableMeta(pVnode, pMsg); + case TDMT_VND_CONSUME: return tqProcessPollReq(pVnode->pTq, pMsg, pInfo->workerId); - case TDMT_VND_TASK_RUN: { - int32_t code = tqProcessTaskRunReq(pVnode->pTq, pMsg); - pMsg->pCont = NULL; - return code; - } - case TDMT_VND_TASK_DISPATCH: + case TDMT_STREAM_TASK_RUN: + return tqProcessTaskRunReq(pVnode->pTq, pMsg); + case TDMT_STREAM_TASK_DISPATCH: return tqProcessTaskDispatchReq(pVnode->pTq, pMsg); - case TDMT_VND_TASK_RECOVER: + case TDMT_STREAM_TASK_RECOVER: return tqProcessTaskRecoverReq(pVnode->pTq, pMsg); - case TDMT_VND_TASK_DISPATCH_RSP: + case TDMT_STREAM_TASK_DISPATCH_RSP: return tqProcessTaskDispatchRsp(pVnode->pTq, pMsg); - case TDMT_VND_TASK_RECOVER_RSP: + case TDMT_STREAM_TASK_RECOVER_RSP: return tqProcessTaskRecoverRsp(pVnode->pTq, pMsg); default: vError("unknown msg type:%d in fetch queue", pMsg->msgType); @@ -262,11 +260,11 @@ int32_t vnodeProcessFetchMsg(SVnode *pVnode, SRpcMsg *pMsg, SQueueInfo *pInfo) { int32_t vnodeProcessWriteMsg(SVnode *pVnode, int64_t version, SRpcMsg *pMsg, SRpcMsg *pRsp) { vTrace("message in write queue is processing"); - char *msgstr = POINTER_SHIFT(pMsg->pCont, sizeof(SMsgHead)); - int32_t msgLen = pMsg->contLen - sizeof(SMsgHead); - SDeleteRes res = {0}; + char *msgstr = POINTER_SHIFT(pMsg->pCont, sizeof(SMsgHead)); + int32_t msgLen = pMsg->contLen - sizeof(SMsgHead); + SDeleteRes res = {0}; SReadHandle handle = {.meta = pVnode->pMeta, .config = &pVnode->config, .vnode = pVnode, .pMsgCb = &pVnode->msgCb}; - + switch (pMsg->msgType) { case TDMT_VND_DELETE: return qWorkerProcessDeleteMsg(&handle, pVnode->pQuery, pMsg, pRsp, &res); @@ -799,7 +797,7 @@ static int32_t vnodeProcessSubmitReq(SVnode *pVnode, int64_t version, void *pReq sprintf(submitBlkRsp.tblFName, "%s.", pVnode->config.dbname); } - if (tsdbInsertTableData(pVnode->pTsdb, &msgIter, pBlock, &submitBlkRsp) < 0) { + if (tsdbInsertTableData(pVnode->pTsdb, version, &msgIter, pBlock, &submitBlkRsp) < 0) { submitBlkRsp.code = terrno; } diff --git a/source/libs/catalog/inc/catalogInt.h b/source/libs/catalog/inc/catalogInt.h index 1eec92f47e..260ed1977e 100644 --- a/source/libs/catalog/inc/catalogInt.h +++ b/source/libs/catalog/inc/catalogInt.h @@ -310,22 +310,19 @@ typedef struct SCtgCacheOperation { int32_t opId; void *data; bool syncOp; - uint64_t seqId; + tsem_t rspSem; } SCtgCacheOperation; typedef struct SCtgQNode { - SCtgCacheOperation op; + SCtgCacheOperation *op; struct SCtgQNode *next; } SCtgQNode; typedef struct SCtgQueue { SRWLatch qlock; - uint64_t seqId; - uint64_t seqDone; SCtgQNode *head; SCtgQNode *tail; tsem_t reqSem; - tsem_t rspSem; uint64_t qRemainNum; } SCtgQueue; @@ -493,6 +490,7 @@ int32_t ctgGetDBVgInfoFromMnode(CTG_PARAMS, SBuildUseDBInput *input, SUseDbOutpu int32_t ctgGetQnodeListFromMnode(CTG_PARAMS, SArray *out, SCtgTask* pTask); int32_t ctgGetDBCfgFromMnode(CTG_PARAMS, const char *dbFName, SDbCfgInfo *out, SCtgTask* pTask); int32_t ctgGetIndexInfoFromMnode(CTG_PARAMS, const char *indexName, SIndexInfo *out, SCtgTask* pTask); +int32_t ctgGetTbIndexFromMnode(CTG_PARAMS, const char *tbFName, SArray** out, SCtgTask* pTask); int32_t ctgGetUdfInfoFromMnode(CTG_PARAMS, const char *funcName, SFuncInfo *out, SCtgTask* pTask); int32_t ctgGetUserDbAuthFromMnode(CTG_PARAMS, const char *user, SGetUserAuthRsp *out, SCtgTask* pTask); int32_t ctgGetTbMetaFromMnodeImpl(CTG_PARAMS, char *dbFName, char* tbName, STableMetaOutput* out, SCtgTask* pTask); diff --git a/source/libs/catalog/src/catalog.c b/source/libs/catalog/src/catalog.c index 040156eca2..0f6a79c14c 100644 --- a/source/libs/catalog/src/catalog.c +++ b/source/libs/catalog/src/catalog.c @@ -506,11 +506,6 @@ int32_t catalogInit(SCatalogCfg *cfg) { CTG_ERR_RET(TSDB_CODE_CTG_SYS_ERROR); } - if (tsem_init(&gCtgMgmt.queue.rspSem, 0, 0)) { - qError("tsem_init failed, error:%s", tstrerror(TAOS_SYSTEM_ERROR(errno))); - CTG_ERR_RET(TSDB_CODE_CTG_SYS_ERROR); - } - gCtgMgmt.queue.head = taosMemoryCalloc(1, sizeof(SCtgQNode)); if (NULL == gCtgMgmt.queue.head) { qError("calloc %d failed", (int32_t)sizeof(SCtgQNode)); @@ -1141,6 +1136,17 @@ int32_t catalogGetIndexMeta(SCatalog* pCtg, void *pTrans, const SEpSet* pMgmtEps CTG_API_LEAVE(ctgGetIndexInfoFromMnode(CTG_PARAMS_LIST(), indexName, pInfo, NULL)); } +int32_t catalogGetTableIndex(SCatalog* pCtg, void *pTrans, const SEpSet* pMgmtEps, const char* tbFName, SArray** pRes) { + CTG_API_ENTER(); + + if (NULL == pCtg || NULL == pTrans || NULL == pMgmtEps || NULL == tbFName || NULL == pRes) { + CTG_API_LEAVE(TSDB_CODE_CTG_INVALID_INPUT); + } + + CTG_API_LEAVE(ctgGetTbIndexFromMnode(CTG_PARAMS_LIST(), tbFName, pRes, NULL)); +} + + int32_t catalogGetUdfInfo(SCatalog* pCtg, void *pTrans, const SEpSet* pMgmtEps, const char* funcName, SFuncInfo* pInfo) { CTG_API_ENTER(); @@ -1194,10 +1200,6 @@ void catalogDestroy(void) { if (tsem_post(&gCtgMgmt.queue.reqSem)) { qError("tsem_post failed, error:%s", tstrerror(TAOS_SYSTEM_ERROR(errno))); } - - if (tsem_post(&gCtgMgmt.queue.rspSem)) { - qError("tsem_post failed, error:%s", tstrerror(TAOS_SYSTEM_ERROR(errno))); - } while (CTG_IS_LOCKED(&gCtgMgmt.lock)) { taosUsleep(1); diff --git a/source/libs/catalog/src/ctgCache.c b/source/libs/catalog/src/ctgCache.c index 8332c7b068..59aab9fd22 100644 --- a/source/libs/catalog/src/ctgCache.c +++ b/source/libs/catalog/src/ctgCache.c @@ -501,25 +501,6 @@ _return: return TSDB_CODE_SUCCESS; } - -void ctgWaitOpDone(SCtgCacheOperation *action) { - while (true) { - tsem_wait(&gCtgMgmt.queue.rspSem); - - if (atomic_load_8((int8_t*)&gCtgMgmt.exit)) { - tsem_post(&gCtgMgmt.queue.rspSem); - break; - } - - if (gCtgMgmt.queue.seqDone >= action->seqId) { - break; - } - - tsem_post(&gCtgMgmt.queue.rspSem); - sched_yield(); - } -} - void ctgDequeue(SCtgCacheOperation **op) { SCtgQNode *orig = gCtgMgmt.queue.head; @@ -530,7 +511,7 @@ void ctgDequeue(SCtgCacheOperation **op) { taosMemoryFreeClear(orig); - *op = &node->op; + *op = node->op; } @@ -541,9 +522,11 @@ int32_t ctgEnqueue(SCatalog* pCtg, SCtgCacheOperation *operation) { CTG_RET(TSDB_CODE_CTG_MEM_ERROR); } - operation->seqId = atomic_add_fetch_64(&gCtgMgmt.queue.seqId, 1); + if (operation->syncOp) { + tsem_init(&operation->rspSem, 0, 0); + } - node->op = *operation; + node->op = operation; CTG_LOCK(CTG_WRITE, &gCtgMgmt.queue.qlock); gCtgMgmt.queue.tail->next = node; @@ -558,7 +541,8 @@ int32_t ctgEnqueue(SCatalog* pCtg, SCtgCacheOperation *operation) { ctgDebug("action [%s] added into queue", gCtgCacheOperation[operation->opId].name); if (operation->syncOp) { - ctgWaitOpDone(operation); + tsem_wait(&operation->rspSem); + taosMemoryFree(operation); } return TSDB_CODE_SUCCESS; @@ -567,7 +551,9 @@ int32_t ctgEnqueue(SCatalog* pCtg, SCtgCacheOperation *operation) { int32_t ctgDropDbCacheEnqueue(SCatalog* pCtg, const char *dbFName, int64_t dbId) { int32_t code = 0; - SCtgCacheOperation action= {.opId = CTG_OP_DROP_DB_CACHE}; + SCtgCacheOperation *op = taosMemoryCalloc(1, sizeof(SCtgCacheOperation)); + op->opId = CTG_OP_DROP_DB_CACHE; + SCtgDropDBMsg *msg = taosMemoryMalloc(sizeof(SCtgDropDBMsg)); if (NULL == msg) { ctgError("malloc %d failed", (int32_t)sizeof(SCtgDropDBMsg)); @@ -583,21 +569,24 @@ int32_t ctgDropDbCacheEnqueue(SCatalog* pCtg, const char *dbFName, int64_t dbId) strncpy(msg->dbFName, dbFName, sizeof(msg->dbFName)); msg->dbId = dbId; - action.data = msg; + op->data = msg; - CTG_ERR_JRET(ctgEnqueue(pCtg, &action)); + CTG_ERR_JRET(ctgEnqueue(pCtg, op)); return TSDB_CODE_SUCCESS; _return: - taosMemoryFreeClear(action.data); + taosMemoryFreeClear(op->data); CTG_RET(code); } int32_t ctgDropDbVgroupEnqueue(SCatalog* pCtg, const char *dbFName, bool syncOp) { int32_t code = 0; - SCtgCacheOperation action= {.opId = CTG_OP_DROP_DB_VGROUP, .syncOp = syncOp}; + SCtgCacheOperation *op = taosMemoryCalloc(1, sizeof(SCtgCacheOperation)); + op->opId = CTG_OP_DROP_DB_VGROUP; + op->syncOp = syncOp; + SCtgDropDbVgroupMsg *msg = taosMemoryMalloc(sizeof(SCtgDropDbVgroupMsg)); if (NULL == msg) { ctgError("malloc %d failed", (int32_t)sizeof(SCtgDropDbVgroupMsg)); @@ -612,15 +601,15 @@ int32_t ctgDropDbVgroupEnqueue(SCatalog* pCtg, const char *dbFName, bool syncOp) msg->pCtg = pCtg; strncpy(msg->dbFName, dbFName, sizeof(msg->dbFName)); - action.data = msg; + op->data = msg; - CTG_ERR_JRET(ctgEnqueue(pCtg, &action)); + CTG_ERR_JRET(ctgEnqueue(pCtg, op)); return TSDB_CODE_SUCCESS; _return: - taosMemoryFreeClear(action.data); + taosMemoryFreeClear(op->data); CTG_RET(code); } @@ -628,7 +617,10 @@ _return: int32_t ctgDropStbMetaEnqueue(SCatalog* pCtg, const char *dbFName, int64_t dbId, const char *stbName, uint64_t suid, bool syncOp) { int32_t code = 0; - SCtgCacheOperation action= {.opId = CTG_OP_DROP_STB_META, .syncOp = syncOp}; + SCtgCacheOperation *op = taosMemoryCalloc(1, sizeof(SCtgCacheOperation)); + op->opId = CTG_OP_DROP_STB_META; + op->syncOp = syncOp; + SCtgDropStbMetaMsg *msg = taosMemoryMalloc(sizeof(SCtgDropStbMetaMsg)); if (NULL == msg) { ctgError("malloc %d failed", (int32_t)sizeof(SCtgDropStbMetaMsg)); @@ -641,15 +633,15 @@ int32_t ctgDropStbMetaEnqueue(SCatalog* pCtg, const char *dbFName, int64_t dbId, msg->dbId = dbId; msg->suid = suid; - action.data = msg; + op->data = msg; - CTG_ERR_JRET(ctgEnqueue(pCtg, &action)); + CTG_ERR_JRET(ctgEnqueue(pCtg, op)); return TSDB_CODE_SUCCESS; _return: - taosMemoryFreeClear(action.data); + taosMemoryFreeClear(op->data); CTG_RET(code); } @@ -657,7 +649,10 @@ _return: int32_t ctgDropTbMetaEnqueue(SCatalog* pCtg, const char *dbFName, int64_t dbId, const char *tbName, bool syncOp) { int32_t code = 0; - SCtgCacheOperation action= {.opId = CTG_OP_DROP_TB_META, .syncOp = syncOp}; + SCtgCacheOperation *op = taosMemoryCalloc(1, sizeof(SCtgCacheOperation)); + op->opId = CTG_OP_DROP_TB_META; + op->syncOp = syncOp; + SCtgDropTblMetaMsg *msg = taosMemoryMalloc(sizeof(SCtgDropTblMetaMsg)); if (NULL == msg) { ctgError("malloc %d failed", (int32_t)sizeof(SCtgDropTblMetaMsg)); @@ -669,21 +664,24 @@ int32_t ctgDropTbMetaEnqueue(SCatalog* pCtg, const char *dbFName, int64_t dbId, strncpy(msg->tbName, tbName, sizeof(msg->tbName)); msg->dbId = dbId; - action.data = msg; + op->data = msg; - CTG_ERR_JRET(ctgEnqueue(pCtg, &action)); + CTG_ERR_JRET(ctgEnqueue(pCtg, op)); return TSDB_CODE_SUCCESS; _return: - taosMemoryFreeClear(action.data); + taosMemoryFreeClear(op->data); CTG_RET(code); } int32_t ctgUpdateVgroupEnqueue(SCatalog* pCtg, const char *dbFName, int64_t dbId, SDBVgInfo* dbInfo, bool syncOp) { int32_t code = 0; - SCtgCacheOperation action= {.opId = CTG_OP_UPDATE_VGROUP, .syncOp = syncOp}; + SCtgCacheOperation *op = taosMemoryCalloc(1, sizeof(SCtgCacheOperation)); + op->opId = CTG_OP_UPDATE_VGROUP; + op->syncOp = syncOp; + SCtgUpdateVgMsg *msg = taosMemoryMalloc(sizeof(SCtgUpdateVgMsg)); if (NULL == msg) { ctgError("malloc %d failed", (int32_t)sizeof(SCtgUpdateVgMsg)); @@ -701,22 +699,25 @@ int32_t ctgUpdateVgroupEnqueue(SCatalog* pCtg, const char *dbFName, int64_t dbId msg->dbId = dbId; msg->dbInfo = dbInfo; - action.data = msg; + op->data = msg; - CTG_ERR_JRET(ctgEnqueue(pCtg, &action)); + CTG_ERR_JRET(ctgEnqueue(pCtg, op)); return TSDB_CODE_SUCCESS; _return: ctgFreeVgInfo(dbInfo); - taosMemoryFreeClear(action.data); + taosMemoryFreeClear(op->data); CTG_RET(code); } int32_t ctgUpdateTbMetaEnqueue(SCatalog* pCtg, STableMetaOutput *output, bool syncOp) { int32_t code = 0; - SCtgCacheOperation action= {.opId = CTG_OP_UPDATE_TB_META, .syncOp = syncOp}; + SCtgCacheOperation *op = taosMemoryCalloc(1, sizeof(SCtgCacheOperation)); + op->opId = CTG_OP_UPDATE_TB_META; + op->syncOp = syncOp; + SCtgUpdateTblMsg *msg = taosMemoryMalloc(sizeof(SCtgUpdateTblMsg)); if (NULL == msg) { ctgError("malloc %d failed", (int32_t)sizeof(SCtgUpdateTblMsg)); @@ -731,9 +732,9 @@ int32_t ctgUpdateTbMetaEnqueue(SCatalog* pCtg, STableMetaOutput *output, bool sy msg->pCtg = pCtg; msg->output = output; - action.data = msg; + op->data = msg; - CTG_ERR_JRET(ctgEnqueue(pCtg, &action)); + CTG_ERR_JRET(ctgEnqueue(pCtg, op)); return TSDB_CODE_SUCCESS; @@ -746,7 +747,9 @@ _return: int32_t ctgUpdateVgEpsetEnqueue(SCatalog* pCtg, char *dbFName, int32_t vgId, SEpSet* pEpSet) { int32_t code = 0; - SCtgCacheOperation operation= {.opId = CTG_OP_UPDATE_VG_EPSET}; + SCtgCacheOperation *op = taosMemoryCalloc(1, sizeof(SCtgCacheOperation)); + op->opId = CTG_OP_UPDATE_VG_EPSET; + SCtgUpdateEpsetMsg *msg = taosMemoryMalloc(sizeof(SCtgUpdateEpsetMsg)); if (NULL == msg) { ctgError("malloc %d failed", (int32_t)sizeof(SCtgUpdateEpsetMsg)); @@ -758,9 +761,9 @@ int32_t ctgUpdateVgEpsetEnqueue(SCatalog* pCtg, char *dbFName, int32_t vgId, SEp msg->vgId = vgId; msg->epSet = *pEpSet; - operation.data = msg; + op->data = msg; - CTG_ERR_JRET(ctgEnqueue(pCtg, &operation)); + CTG_ERR_JRET(ctgEnqueue(pCtg, op)); return TSDB_CODE_SUCCESS; @@ -775,7 +778,10 @@ _return: int32_t ctgUpdateUserEnqueue(SCatalog* pCtg, SGetUserAuthRsp *pAuth, bool syncOp) { int32_t code = 0; - SCtgCacheOperation action= {.opId = CTG_OP_UPDATE_USER, .syncOp = syncOp}; + SCtgCacheOperation *op = taosMemoryCalloc(1, sizeof(SCtgCacheOperation)); + op->opId = CTG_OP_UPDATE_USER; + op->syncOp = syncOp; + SCtgUpdateUserMsg *msg = taosMemoryMalloc(sizeof(SCtgUpdateUserMsg)); if (NULL == msg) { ctgError("malloc %d failed", (int32_t)sizeof(SCtgUpdateUserMsg)); @@ -785,9 +791,9 @@ int32_t ctgUpdateUserEnqueue(SCatalog* pCtg, SGetUserAuthRsp *pAuth, bool syncOp msg->pCtg = pCtg; msg->userAuth = *pAuth; - action.data = msg; + op->data = msg; - CTG_ERR_JRET(ctgEnqueue(pCtg, &action)); + CTG_ERR_JRET(ctgEnqueue(pCtg, op)); return TSDB_CODE_SUCCESS; @@ -1607,6 +1613,39 @@ void ctgUpdateThreadUnexpectedStopped(void) { if (CTG_IS_LOCKED(&gCtgMgmt.lock) > 0) CTG_UNLOCK(CTG_READ, &gCtgMgmt.lock); } +void ctgCleanupCacheQueue(void) { + SCtgQNode *node = NULL; + SCtgQNode *nodeNext = NULL; + + while (true) { + node = gCtgMgmt.queue.head->next; + while (node) { + if (node->op) { + taosMemoryFree(node->op->data); + if (node->op->syncOp) { + tsem_post(&node->op->rspSem); + } else { + taosMemoryFree(node->op); + } + } + + nodeNext = node->next; + taosMemoryFree(node); + + node = nodeNext; + } + + if (CTG_IS_LOCKED(&gCtgMgmt.lock)) { + taosUsleep(1); + } else { + break; + } + } + + taosMemoryFreeClear(gCtgMgmt.queue.head); + gCtgMgmt.queue.tail = NULL; +} + void* ctgUpdateThreadFunc(void* param) { setThreadName("catalog"); #ifdef WINDOWS @@ -1622,7 +1661,8 @@ void* ctgUpdateThreadFunc(void* param) { } if (atomic_load_8((int8_t*)&gCtgMgmt.exit)) { - tsem_post(&gCtgMgmt.queue.rspSem); + CTG_UNLOCK(CTG_READ, &gCtgMgmt.lock); + ctgCleanupCacheQueue(); break; } @@ -1634,10 +1674,8 @@ void* ctgUpdateThreadFunc(void* param) { (*gCtgCacheOperation[operation->opId].func)(operation); - gCtgMgmt.queue.seqDone = operation->seqId; - if (operation->syncOp) { - tsem_post(&gCtgMgmt.queue.rspSem); + tsem_post(&operation->rspSem); } CTG_RT_STAT_INC(qDoneNum, 1); diff --git a/source/libs/catalog/src/ctgRemote.c b/source/libs/catalog/src/ctgRemote.c index 188f5b44c0..94c59ddc53 100644 --- a/source/libs/catalog/src/ctgRemote.c +++ b/source/libs/catalog/src/ctgRemote.c @@ -85,6 +85,21 @@ int32_t ctgProcessRspMsg(void* out, int32_t reqType, char* msg, int32_t msgSize, qDebug("Got index from mnode, indexName:%s", target); break; } + case TDMT_MND_GET_TABLE_INDEX: { + if (TSDB_CODE_SUCCESS != rspCode) { + qError("error rsp for get table index, error:%s, tbFName:%s", tstrerror(rspCode), target); + CTG_ERR_RET(rspCode); + } + + code = queryProcessMsgRsp[TMSG_INDEX(reqType)](out, msg, msgSize); + if (code) { + qError("Process get table index rsp failed, error:%s, tbFName:%s", tstrerror(code), target); + CTG_ERR_RET(code); + } + + qDebug("Got table index from mnode, tbFName:%s", target); + break; + } case TDMT_MND_RETRIEVE_FUNC: { if (TSDB_CODE_SUCCESS != rspCode) { qError("error rsp for get udf, error:%s, funcName:%s", tstrerror(rspCode), target); @@ -412,6 +427,44 @@ int32_t ctgGetIndexInfoFromMnode(CTG_PARAMS, const char *indexName, SIndexInfo * return TSDB_CODE_SUCCESS; } +int32_t ctgGetTbIndexFromMnode(CTG_PARAMS, const char *tbFName, SArray** out, SCtgTask* pTask) { + char *msg = NULL; + int32_t msgLen = 0; + int32_t reqType = TDMT_MND_GET_TABLE_INDEX; + void*(*mallocFp)(int32_t) = pTask ? taosMemoryMalloc : rpcMallocCont; + + ctgDebug("try to get tb index from mnode, tbFName:%s", tbFName); + + int32_t code = queryBuildMsg[TMSG_INDEX(reqType)]((void *)tbFName, &msg, 0, &msgLen, mallocFp); + if (code) { + ctgError("Build get index msg failed, code:%s, tbFName:%s", tstrerror(code), tbFName); + CTG_ERR_RET(code); + } + + if (pTask) { + void* pOut = taosMemoryCalloc(1, POINTER_BYTES); + if (NULL == pOut) { + CTG_ERR_RET(TSDB_CODE_OUT_OF_MEMORY); + } + CTG_ERR_RET(ctgUpdateMsgCtx(&pTask->msgCtx, reqType, pOut, (char*)tbFName)); + + CTG_RET(ctgAsyncSendMsg(CTG_PARAMS_LIST(), pTask, reqType, msg, msgLen)); + } + + SRpcMsg rpcMsg = { + .msgType = reqType, + .pCont = msg, + .contLen = msgLen, + }; + + SRpcMsg rpcRsp = {0}; + rpcSendRecv(pTrans, (SEpSet*)pMgmtEps, &rpcMsg, &rpcRsp); + + CTG_ERR_RET(ctgProcessRspMsg(out, reqType, rpcRsp.pCont, rpcRsp.contLen, rpcRsp.code, (char*)tbFName)); + + return TSDB_CODE_SUCCESS; +} + int32_t ctgGetUdfInfoFromMnode(CTG_PARAMS, const char *funcName, SFuncInfo *out, SCtgTask* pTask) { char *msg = NULL; int32_t msgLen = 0; diff --git a/source/libs/executor/inc/executorimpl.h b/source/libs/executor/inc/executorimpl.h index 4804984397..9b8b9dca26 100644 --- a/source/libs/executor/inc/executorimpl.h +++ b/source/libs/executor/inc/executorimpl.h @@ -488,6 +488,8 @@ typedef struct SStreamFinalIntervalOperatorInfo { int32_t order; // current SSDataBlock scan order STimeWindowAggSupp twAggSup; SArray* pChildren; + SSDataBlock* pUpdateRes; + SPhysiNode* pPhyNode; // create new child } SStreamFinalIntervalOperatorInfo; typedef struct SAggOperatorInfo { @@ -793,9 +795,8 @@ SOperatorInfo* createSortedMergeOperatorInfo(SOperatorInfo** downstream, int32_t SOperatorInfo* createIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols, SSDataBlock* pResBlock, SInterval* pInterval, int32_t primaryTsSlotId, STimeWindowAggSupp *pTwAggSupp, SExecTaskInfo* pTaskInfo); -SOperatorInfo* createStreamFinalIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols, - SSDataBlock* pResBlock, SInterval* pInterval, int32_t primaryTsSlotId, - STimeWindowAggSupp *pTwAggSupp, SExecTaskInfo* pTaskInfo); +SOperatorInfo* createStreamFinalIntervalOperatorInfo(SOperatorInfo* downstream, + SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo, int32_t numOfChild); SOperatorInfo* createStreamIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols, SSDataBlock* pResBlock, SInterval* pInterval, int32_t primaryTsSlotId, STimeWindowAggSupp *pTwAggSupp, SExecTaskInfo* pTaskInfo); diff --git a/source/libs/executor/src/executorimpl.c b/source/libs/executor/src/executorimpl.c index 9d7f939bb0..e266178068 100644 --- a/source/libs/executor/src/executorimpl.c +++ b/source/libs/executor/src/executorimpl.c @@ -4634,6 +4634,12 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo int32_t tsSlotId = ((SColumnNode*)pIntervalPhyNode->window.pTspk)->slotId; pOptr = createIntervalOperatorInfo(ops[0], pExprInfo, num, pResBlock, &interval, tsSlotId, &as, pTaskInfo); + } else if (QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_INTERVAL == type) { + int32_t children = 8; + pOptr = createStreamFinalIntervalOperatorInfo(ops[0], pPhyNode, pTaskInfo, children); + } else if (QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_INTERVAL == type) { + int32_t children = 0; + pOptr = createStreamFinalIntervalOperatorInfo(ops[0], pPhyNode, pTaskInfo, children); } else if (QUERY_NODE_PHYSICAL_PLAN_SORT == type) { SSortPhysiNode* pSortPhyNode = (SSortPhysiNode*)pPhyNode; @@ -4760,6 +4766,7 @@ int32_t initQueryTableDataCond(SQueryTableDataCond* pCond, const STableScanPhysi pCond->numOfTWindows = 1; pCond->twindows = taosMemoryCalloc(pCond->numOfTWindows, sizeof(STimeWindow)); pCond->twindows[0] = pTableScanNode->scanRange; + pCond->suid = pTableScanNode->scan.suid; #if 1 // todo work around a problem, remove it later diff --git a/source/libs/executor/src/scanoperator.c b/source/libs/executor/src/scanoperator.c index ba0397fd34..686add5508 100644 --- a/source/libs/executor/src/scanoperator.c +++ b/source/libs/executor/src/scanoperator.c @@ -956,16 +956,15 @@ static SSDataBlock* doStreamBlockScan(SOperatorInfo* pOperator) { if (rows == 0) { pOperator->status = OP_EXEC_DONE; } else if (pInfo->pUpdateInfo) { - SSDataBlock* upRes = createOneDataBlock(pInfo->pRes, false); - getUpdateDataBlock(pInfo, true, pInfo->pRes, upRes); - if (upRes) { - pInfo->pUpdateRes = upRes; - if (upRes->info.type == STREAM_REPROCESS) { + blockDataCleanup(pInfo->pUpdateRes); + getUpdateDataBlock(pInfo, true, pInfo->pRes, pInfo->pUpdateRes); + if (pInfo->pUpdateRes->info.rows > 0) { + if (pInfo->pUpdateRes->info.type == STREAM_REPROCESS) { pInfo->updateResIndex = 0; pInfo->scanMode = STREAM_SCAN_FROM_UPDATERES; - } else if (upRes->info.type == STREAM_INVERT) { + } else if (pInfo->pUpdateRes->info.type == STREAM_INVERT) { pInfo->scanMode = STREAM_SCAN_FROM_RES; - return upRes; + return pInfo->pUpdateRes; } } } @@ -1044,6 +1043,7 @@ SOperatorInfo* createStreamScanOperatorInfo(void* pDataReader, SReadHandle* pHan pInfo->tableUid = pScanPhyNode->uid; pInfo->streamBlockReader = pHandle->reader; pInfo->pRes = createResDataBlock(pDescNode); + pInfo->pUpdateRes = createResDataBlock(pDescNode); pInfo->pCondition = pScanPhyNode->node.pConditions; pInfo->pDataReader = pDataReader; pInfo->scanMode = STREAM_SCAN_FROM_READERHANDLE; diff --git a/source/libs/executor/src/timewindowoperator.c b/source/libs/executor/src/timewindowoperator.c index b621d729e0..95b7dc72b6 100644 --- a/source/libs/executor/src/timewindowoperator.c +++ b/source/libs/executor/src/timewindowoperator.c @@ -1,3 +1,17 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ #include "executorimpl.h" #include "function.h" #include "functionMgt.h" @@ -9,7 +23,6 @@ typedef enum SResultTsInterpType { RESULT_ROW_END_INTERP = 2, } SResultTsInterpType; -static SSDataBlock* doStreamFinalIntervalAgg(SOperatorInfo* pOperator); static SSDataBlock* doStreamSessionAgg(SOperatorInfo* pOperator); static int64_t* extractTsCol(SSDataBlock* pBlock, const SIntervalAggOperatorInfo* pInfo); @@ -764,6 +777,22 @@ static int32_t saveResult(SResultRow* result, uint64_t groupId, SArray* pUpdated return TSDB_CODE_SUCCESS; } +static void removeResult(SArray* pUpdated, TSKEY key) { + int32_t size = taosArrayGetSize(pUpdated); + int32_t index = binarySearch(pUpdated, size, key, TSDB_ORDER_DESC, getReskey); + if (index >= 0 && key == getReskey(pUpdated, index)) { + taosArrayRemove(pUpdated, index); + } +} + +static void removeResults(SArray* pWins, SArray* pUpdated) { + int32_t size = taosArrayGetSize(pWins); + for (int32_t i = 0; i < size; i++) { + STimeWindow* pW = taosArrayGet(pWins, i); + removeResult(pUpdated, pW->skey); + } +} + static void hashIntervalAgg(SOperatorInfo* pOperatorInfo, SResultRowInfo* pResultRowInfo, SSDataBlock* pBlock, int32_t scanFlag, SArray* pUpdated) { SIntervalAggOperatorInfo* pInfo = (SIntervalAggOperatorInfo*)pOperatorInfo->info; @@ -1198,6 +1227,7 @@ void doClearWindow(SAggSupporter* pSup, SOptrBasicInfo* pBinfo, char* pData, int SET_RES_WINDOW_KEY(pSup->keyBuf, pData, bytes, groupId); SResultRowPosition* p1 = (SResultRowPosition*)taosHashGet(pSup->pResultRowHashTable, pSup->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes)); + ASSERT(p1); doClearWindowImpl(p1, pSup->pResultBuf, pBinfo, numOfOutput); } @@ -1349,6 +1379,7 @@ void destroyStreamFinalIntervalOperatorInfo(void* param, int32_t numOfOutput) { taosMemoryFreeClear(pChildOp); } } + nodesDestroyNode(pInfo->pPhyNode); } static bool allInvertible(SqlFunctionCtx* pFCtx, int32_t numOfCols) { @@ -1471,69 +1502,6 @@ _error: return NULL; } -SOperatorInfo* createStreamFinalIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols, - SSDataBlock* pResBlock, SInterval* pInterval, - int32_t primaryTsSlotId, STimeWindowAggSupp* pTwAggSupp, - SExecTaskInfo* pTaskInfo) { - SStreamFinalIntervalOperatorInfo* pInfo = taosMemoryCalloc(1, sizeof(SStreamFinalIntervalOperatorInfo)); - SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); - if (pInfo == NULL || pOperator == NULL) { - goto _error; - } - pInfo->order = TSDB_ORDER_ASC; - pInfo->interval = *pInterval; - pInfo->twAggSup = *pTwAggSupp; - pInfo->primaryTsIndex = primaryTsSlotId; - size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; - initResultSizeInfo(pOperator, 4096); - int32_t code = - initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfCols, pResBlock, keyBufSize, pTaskInfo->id.str); - initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pTaskInfo->window); - if (code != TSDB_CODE_SUCCESS) { - goto _error; - } - initResultRowInfo(&pInfo->binfo.resultRowInfo, (int32_t)1); - int32_t numOfChild = 8; // Todo(liuyao) get it from phy plan - pInfo->pChildren = taosArrayInit(numOfChild, sizeof(SOperatorInfo)); - for (int32_t i = 0; i < numOfChild; i++) { - SSDataBlock* chRes = createOneDataBlock(pResBlock, false); - SOperatorInfo* pChildOp = createIntervalOperatorInfo(NULL, pExprInfo, numOfCols, chRes, pInterval, primaryTsSlotId, - pTwAggSupp, pTaskInfo); - if (pChildOp && chRes) { - taosArrayPush(pInfo->pChildren, &pChildOp); - continue; - } - goto _error; - } - - pOperator->name = "StreamFinalIntervalOperator"; - pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_INTERVAL; - pOperator->blocking = true; - pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExprInfo; - pOperator->pTaskInfo = pTaskInfo; - pOperator->numOfExprs = numOfCols; - pOperator->info = pInfo; - - pOperator->fpSet = - createOperatorFpSet(NULL, doStreamFinalIntervalAgg, NULL, NULL, destroyStreamFinalIntervalOperatorInfo, - aggEncodeResultRow, aggDecodeResultRow, NULL); - - code = appendDownstream(pOperator, &downstream, 1); - if (code != TSDB_CODE_SUCCESS) { - goto _error; - } - - return pOperator; - -_error: - destroyStreamFinalIntervalOperatorInfo(pInfo, numOfCols); - taosMemoryFreeClear(pInfo); - taosMemoryFreeClear(pOperator); - pTaskInfo->code = code; - return NULL; -} - SOperatorInfo* createStreamIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols, SSDataBlock* pResBlock, SInterval* pInterval, int32_t primaryTsSlotId, STimeWindowAggSupp* pTwAggSupp, SExecTaskInfo* pTaskInfo) { @@ -1899,12 +1867,12 @@ _error: return NULL; } -static SArray* doHashInterval(SOperatorInfo* pOperatorInfo, SSDataBlock* pSDataBlock, int32_t tableGroupId) { +static void doHashInterval(SOperatorInfo* pOperatorInfo, SSDataBlock* pSDataBlock, + int32_t tableGroupId, SArray* pUpdated) { SStreamFinalIntervalOperatorInfo* pInfo = (SStreamFinalIntervalOperatorInfo*)pOperatorInfo->info; SResultRowInfo* pResultRowInfo = &(pInfo->binfo.resultRowInfo); SExecTaskInfo* pTaskInfo = pOperatorInfo->pTaskInfo; int32_t numOfOutput = pOperatorInfo->numOfExprs; - SArray* pUpdated = taosArrayInit(4, POINTER_BYTES); int32_t step = 1; bool ascScan = true; TSKEY* tsCols = NULL; @@ -1915,7 +1883,7 @@ static SArray* doHashInterval(SOperatorInfo* pOperatorInfo, SSDataBlock* pSDataB SColumnInfoData* pColDataInfo = taosArrayGet(pSDataBlock->pDataBlock, pInfo->primaryTsIndex); tsCols = (int64_t*)pColDataInfo->pData; } else { - return pUpdated; + return ; } int32_t startPos = ascScan ? 0 : (pSDataBlock->info.rows - 1); @@ -1932,13 +1900,13 @@ static SArray* doHashInterval(SOperatorInfo* pOperatorInfo, SSDataBlock* pSDataB pos->groupId = tableGroupId; pos->pos = (SResultRowPosition){.pageId = pResult->pageId, .offset = pResult->offset}; *(int64_t*)pos->key = pResult->win.skey; - taosArrayPush(pUpdated, &pos); - forwardRows = getNumOfRowsInTimeWindow(&pSDataBlock->info, tsCols, startPos, nextWin.ekey, binarySearchForKey, NULL, - TSDB_ORDER_ASC); + forwardRows = getNumOfRowsInTimeWindow(&pSDataBlock->info, tsCols, startPos, + nextWin.ekey, binarySearchForKey, NULL, TSDB_ORDER_ASC); + if (pInfo->twAggSup.calTrigger == STREAM_TRIGGER_AT_ONCE && pUpdated) { + saveResult(pResult, tableGroupId, pUpdated); + } // window start(end) key interpolation - // disable it temporarily - // doWindowBorderInterpolation(pInfo, pSDataBlock, numOfOutput, pInfo->binfo.pCtx, pResult, &nextWin, startPos, - // forwardRows); + // doWindowBorderInterpolation(pInfo, pSDataBlock, numOfOutput, pInfo->binfo.pCtx, pResult, &nextWin, startPos, forwardRows); updateTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &nextWin, true); doApplyFunctions(pTaskInfo, pInfo->binfo.pCtx, &nextWin, &pInfo->twAggSup.timeWindowData, startPos, forwardRows, tsCols, pSDataBlock->info.rows, numOfOutput, TSDB_ORDER_ASC); @@ -1948,7 +1916,6 @@ static SArray* doHashInterval(SOperatorInfo* pOperatorInfo, SSDataBlock* pSDataB break; } } - return pUpdated; } bool isFinalInterval(SStreamFinalIntervalOperatorInfo* pInfo) { return pInfo->pChildren != NULL; } @@ -1992,24 +1959,72 @@ static void rebuildIntervalWindow(SStreamFinalIntervalOperatorInfo* pInfo, SArra } } +static void clearStreamIntervalOperator(SStreamFinalIntervalOperatorInfo* pInfo) { + taosHashClear(pInfo->aggSup.pResultRowHashTable); + clearDiskbasedBuf(pInfo->aggSup.pResultBuf); + cleanupResultRowInfo(&pInfo->binfo.resultRowInfo); + initResultRowInfo(&pInfo->binfo.resultRowInfo, 1); +} + +static void clearUpdateDataBlock(SSDataBlock* pBlock) { + if (pBlock->info.rows <= 0) { + return; + } + blockDataCleanup(pBlock); +} + +static void copyUpdateDataBlock(SSDataBlock* pDest, SSDataBlock* pSource, int32_t tsColIndex) { + ASSERT(pDest->info.capacity >= pSource->info.rows); + clearUpdateDataBlock(pDest); + SColumnInfoData* pDestCol = taosArrayGet(pDest->pDataBlock, 0); + SColumnInfoData* pSourceCol = taosArrayGet(pSource->pDataBlock, tsColIndex); + // copy timestamp column + colDataAssign(pDestCol, pSourceCol, pSource->info.rows); + for (int32_t i = 1; i < pDest->info.numOfCols; i++) { + SColumnInfoData* pCol = taosArrayGet(pDest->pDataBlock, i); + colDataAppendNNULL(pCol, 0, pSource->info.rows); + } + pDest->info.rows = pSource->info.rows; + blockDataUpdateTsWindow(pDest, 0); +} + +static int32_t getChildIndex(SSDataBlock* pBlock) { + // if (pBlock->info.type != STREAM_INVALID && pBlock->info.rows < 4) { // for test + // return pBlock->info.rows - 1; + // } + return 0; +} + static SSDataBlock* doStreamFinalIntervalAgg(SOperatorInfo* pOperator) { SStreamFinalIntervalOperatorInfo* pInfo = pOperator->info; SOperatorInfo* downstream = pOperator->pDownstream[0]; - SArray* pUpdated = NULL; + SArray* pUpdated = taosArrayInit(4, POINTER_BYTES); + SArray* pClosed = taosArrayInit(4, POINTER_BYTES); if (pOperator->status == OP_EXEC_DONE) { return NULL; } else if (pOperator->status == OP_RES_TO_RETURN) { doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); - if (pInfo->binfo.pRes->info.rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (pInfo->binfo.pRes->info.rows == 0) { pOperator->status = OP_EXEC_DONE; + if (isFinalInterval(pInfo) || pInfo->pUpdateRes->info.rows == 0) { + if (!isFinalInterval(pInfo)) { + // semi interval operator clear disk buffer + clearStreamIntervalOperator(pInfo); + } + return NULL; + } + // process the rest of the data + pOperator->status = OP_OPENED; + return pInfo->pUpdateRes; } - return pInfo->binfo.pRes->info.rows == 0 ? NULL : pInfo->binfo.pRes; + return pInfo->binfo.pRes; } while (1) { SSDataBlock* pBlock = downstream->fpSet.getNextFn(downstream); if (pBlock == NULL) { + clearUpdateDataBlock(pInfo->pUpdateRes); break; } @@ -2019,31 +2034,149 @@ static SSDataBlock* doStreamFinalIntervalAgg(SOperatorInfo* pOperator) { doClearWindows(&pInfo->aggSup, &pInfo->binfo, &pInfo->interval, pInfo->primaryTsIndex, pOperator->numOfExprs, pBlock, pUpWins); if (isFinalInterval(pInfo)) { - int32_t childIndex = 0; // Todo(liuyao) get child id from SSDataBlock + int32_t childIndex = getChildIndex(pBlock); SOperatorInfo* pChildOp = taosArrayGetP(pInfo->pChildren, childIndex); SIntervalAggOperatorInfo* pChildInfo = pChildOp->info; - doClearWindows(&pChildInfo->aggSup, &pChildInfo->binfo, &pChildInfo->interval, pChildInfo->primaryTsIndex, - pChildOp->numOfExprs, pBlock, NULL); - rebuildIntervalWindow(pInfo, pUpWins, pInfo->binfo.pRes->info.groupId, pOperator->numOfExprs, - pOperator->pTaskInfo); + doClearWindows(&pChildInfo->aggSup, &pChildInfo->binfo, &pChildInfo->interval, + pChildInfo->primaryTsIndex, pChildOp->numOfExprs, pBlock, NULL); + rebuildIntervalWindow(pInfo, pUpWins, pInfo->binfo.pRes->info.groupId, + pOperator->numOfExprs, pOperator->pTaskInfo); + taosArrayDestroy(pUpWins); + continue; } + removeResults(pUpWins, pUpdated); + copyUpdateDataBlock(pInfo->pUpdateRes, pBlock, pInfo->primaryTsIndex); taosArrayDestroy(pUpWins); - continue; + break; } if (isFinalInterval(pInfo)) { - int32_t chIndex = 1; // Todo(liuyao) get it from SSDataBlock + int32_t chIndex = getChildIndex(pBlock); + int32_t size = taosArrayGetSize(pInfo->pChildren); + // if chIndex + 1 - size > 0, add new child + for (int32_t i = 0; i < chIndex + 1 - size; i++) { + SOperatorInfo* pChildOp = createStreamFinalIntervalOperatorInfo(NULL, pInfo->pPhyNode, pOperator->pTaskInfo, 0); + if (!pChildOp) { + longjmp(pOperator->pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); + } + taosArrayPush(pInfo->pChildren, &pChildOp); + } SOperatorInfo* pChildOp = taosArrayGetP(pInfo->pChildren, chIndex); - doStreamIntervalAgg(pChildOp); + SStreamFinalIntervalOperatorInfo* pChInfo = pChildOp->info; + setInputDataBlock(pChildOp, pChInfo->binfo.pCtx, pBlock, pChInfo->order, MAIN_SCAN, true); + doHashInterval(pChildOp, pBlock, pBlock->info.groupId, NULL); } - pUpdated = doHashInterval(pOperator, pBlock, 0); + doHashInterval(pOperator, pBlock, pBlock->info.groupId, pUpdated); + pInfo->twAggSup.maxTs = TMAX(pInfo->twAggSup.maxTs, pBlock->info.window.ekey); } + + if (isFinalInterval(pInfo)) { + closeIntervalWindow(pInfo->aggSup.pResultRowHashTable, &pInfo->twAggSup, + &pInfo->interval, pClosed); + finalizeUpdatedResult(pOperator->numOfExprs, pInfo->aggSup.pResultBuf, pClosed, + pInfo->binfo.rowCellInfoOffset); + if (pInfo->twAggSup.calTrigger == STREAM_TRIGGER_WINDOW_CLOSE) { + taosArrayAddAll(pUpdated, pClosed); + } + } + taosArrayDestroy(pClosed); finalizeUpdatedResult(pOperator->numOfExprs, pInfo->aggSup.pResultBuf, pUpdated, pInfo->binfo.rowCellInfoOffset); initMultiResInfoFromArrayList(&pInfo->groupResInfo, pUpdated); blockDataEnsureCapacity(pInfo->binfo.pRes, pOperator->resultInfo.capacity); doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); pOperator->status = OP_RES_TO_RETURN; - return pInfo->binfo.pRes->info.rows == 0 ? NULL : pInfo->binfo.pRes; + if (pInfo->binfo.pRes->info.rows == 0) { + pOperator->status = OP_EXEC_DONE; + if (pInfo->pUpdateRes->info.rows == 0) { + return NULL; + } + // process the rest of the data + pOperator->status = OP_OPENED; + return pInfo->pUpdateRes; + } + return pInfo->binfo.pRes; +} + +SOperatorInfo* createStreamFinalIntervalOperatorInfo(SOperatorInfo* downstream, + SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo, int32_t numOfChild) { + SIntervalPhysiNode* pIntervalPhyNode = (SIntervalPhysiNode*)pPhyNode; + SStreamFinalIntervalOperatorInfo* pInfo = taosMemoryCalloc(1, sizeof(SStreamFinalIntervalOperatorInfo)); + SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); + if (pInfo == NULL || pOperator == NULL) { + goto _error; + } + pInfo->order = TSDB_ORDER_ASC; + pInfo->interval = (SInterval) {.interval = pIntervalPhyNode->interval, + .sliding = pIntervalPhyNode->sliding, + .intervalUnit = pIntervalPhyNode->intervalUnit, + .slidingUnit = pIntervalPhyNode->slidingUnit, + .offset = pIntervalPhyNode->offset, + .precision = + ((SColumnNode*)pIntervalPhyNode->window.pTspk)->node.resType.precision}; + pInfo->twAggSup = (STimeWindowAggSupp){.waterMark = pIntervalPhyNode->window.watermark, + .calTrigger = pIntervalPhyNode->window.triggerType, + .maxTs = INT64_MIN, + .winMap = NULL, }; + pInfo->primaryTsIndex = ((SColumnNode*)pIntervalPhyNode->window.pTspk)->slotId; + size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; + initResultSizeInfo(pOperator, 4096); + int32_t numOfCols = 0; + SExprInfo* pExprInfo = createExprInfo(pIntervalPhyNode->window.pFuncs, NULL, &numOfCols); + SSDataBlock* pResBlock = createResDataBlock(pPhyNode->pOutputDataBlockDesc); + int32_t code = initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfCols, + pResBlock, keyBufSize, pTaskInfo->id.str); + initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pTaskInfo->window); + if (code != TSDB_CODE_SUCCESS) { + goto _error; + } + initResultRowInfo(&pInfo->binfo.resultRowInfo, (int32_t)1); + pInfo->pChildren = NULL; + if (numOfChild > 0) { + pInfo->pChildren = taosArrayInit(numOfChild, sizeof(SOperatorInfo)); + for (int32_t i = 0; i < numOfChild; i++) { + SOperatorInfo* pChildOp = createStreamFinalIntervalOperatorInfo(NULL, pPhyNode, pTaskInfo, 0); + if (pChildOp) { + taosArrayPush(pInfo->pChildren, &pChildOp); + continue; + } + goto _error; + } + } + // semi interval operator does not catch result + if (!isFinalInterval(pInfo)) { + pInfo->twAggSup.calTrigger = STREAM_TRIGGER_AT_ONCE; + } + pInfo->pUpdateRes = createResDataBlock(pPhyNode->pOutputDataBlockDesc);\ + pInfo->pUpdateRes->info.type = STREAM_REPROCESS; + blockDataEnsureCapacity(pInfo->pUpdateRes, 128); + pInfo->pPhyNode = nodesCloneNode(pPhyNode); + + pOperator->name = "StreamFinalIntervalOperator"; + pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_INTERVAL; + pOperator->blocking = true; + pOperator->status = OP_NOT_OPENED; + pOperator->pExpr = pExprInfo; + pOperator->pTaskInfo = pTaskInfo; + pOperator->numOfExprs = numOfCols; + pOperator->info = pInfo; + + pOperator->fpSet = createOperatorFpSet(NULL, doStreamFinalIntervalAgg, NULL, NULL, + destroyStreamFinalIntervalOperatorInfo, aggEncodeResultRow, aggDecodeResultRow, + NULL); + + code = appendDownstream(pOperator, &downstream, 1); + if (code != TSDB_CODE_SUCCESS) { + goto _error; + } + + return pOperator; + +_error: + destroyStreamFinalIntervalOperatorInfo(pInfo, numOfCols); + taosMemoryFreeClear(pInfo); + taosMemoryFreeClear(pOperator); + pTaskInfo->code = code; + return NULL; } void destroyStreamAggSupporter(SStreamAggSupporter* pSup) { diff --git a/source/libs/function/inc/builtinsimpl.h b/source/libs/function/inc/builtinsimpl.h index cd4fb6f3b8..2febf86e60 100644 --- a/source/libs/function/inc/builtinsimpl.h +++ b/source/libs/function/inc/builtinsimpl.h @@ -71,6 +71,7 @@ bool leastSQRFunctionSetup(SqlFunctionCtx *pCtx, SResultRowEntryInfo* pResultInf int32_t leastSQRFunction(SqlFunctionCtx* pCtx); int32_t leastSQRFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock); int32_t leastSQRInvertFunction(SqlFunctionCtx* pCtx); +int32_t leastSQRCombine(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx); bool getPercentileFuncEnv(struct SFunctionNode* pFunc, SFuncExecEnv* pEnv); bool percentileFunctionSetup(SqlFunctionCtx *pCtx, SResultRowEntryInfo* pResultInfo); diff --git a/source/libs/function/src/builtins.c b/source/libs/function/src/builtins.c index 41f3cc776e..88259e2cbd 100644 --- a/source/libs/function/src/builtins.c +++ b/source/libs/function/src/builtins.c @@ -1191,7 +1191,8 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = { .initFunc = leastSQRFunctionSetup, .processFunc = leastSQRFunction, .finalizeFunc = leastSQRFinalize, - .invertFunc = leastSQRInvertFunction, + .invertFunc = NULL, + .combineFunc = leastSQRCombine, }, { .name = "avg", diff --git a/source/libs/function/src/builtinsimpl.c b/source/libs/function/src/builtinsimpl.c index ef4c00503f..e1a43e70f7 100644 --- a/source/libs/function/src/builtinsimpl.c +++ b/source/libs/function/src/builtinsimpl.c @@ -1812,17 +1812,17 @@ int32_t leastSQRFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) { param[1][1] = (double)pInfo->num; param[1][0] = param[0][1]; - param[0][0] -= param[1][0] * (param[0][1] / param[1][1]); - param[0][2] -= param[1][2] * (param[0][1] / param[1][1]); - param[0][1] = 0; - param[1][2] -= param[0][2] * (param[1][0] / param[0][0]); - param[1][0] = 0; - param[0][2] /= param[0][0]; + double param00 = param[0][0] - param[1][0] * (param[0][1] / param[1][1]); + double param02 = param[0][2] - param[1][2] * (param[0][1] / param[1][1]); + // param[0][1] = 0; + double param12 = param[1][2] - param02 * (param[1][0] / param00); + // param[1][0] = 0; + param02 /= param00; - param[1][2] /= param[1][1]; + param12 /= param[1][1]; char buf[64] = {0}; - size_t len = snprintf(varDataVal(buf), sizeof(buf) - VARSTR_HEADER_SIZE, "{slop:%.6lf, intercept:%.6lf}", param[0][2], param[1][2]); + size_t len = snprintf(varDataVal(buf), sizeof(buf) - VARSTR_HEADER_SIZE, "{slop:%.6lf, intercept:%.6lf}", param02, param12); varDataSetLen(buf, len); colDataAppend(pCol, currentRow, buf, pResInfo->isNullRes); @@ -1835,6 +1835,27 @@ int32_t leastSQRInvertFunction(SqlFunctionCtx* pCtx) { return TSDB_CODE_SUCCESS; } +int32_t leastSQRCombine(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx) { + SResultRowEntryInfo* pDResInfo = GET_RES_INFO(pDestCtx); + SLeastSQRInfo* pDBuf = GET_ROWCELL_INTERBUF(pDResInfo); + int32_t type = pDestCtx->input.pData[0]->info.type; + double (*pDparam)[3] = pDBuf->matrix; + + SResultRowEntryInfo* pSResInfo = GET_RES_INFO(pSourceCtx); + SLeastSQRInfo* pSBuf = GET_ROWCELL_INTERBUF(pSResInfo); + double (*pSparam)[3] = pSBuf->matrix; + for (int32_t i = 0; i < pSBuf->num; i++) { + pDparam[0][0] += pDBuf->startVal * pDBuf->startVal; + pDparam[0][1] += pDBuf->startVal; + pDBuf->startVal += pDBuf->stepVal; + } + pDparam[0][2] += pSparam[0][2] + pDBuf->num * pDBuf->stepVal * pSparam[1][2]; + pDparam[1][2] += pSparam[1][2]; + pDBuf->num += pSBuf->num; + pDResInfo->numOfRes = TMAX(pDResInfo->numOfRes, pSResInfo->numOfRes); + return TSDB_CODE_SUCCESS; +} + bool getPercentileFuncEnv(SFunctionNode* pFunc, SFuncExecEnv* pEnv) { pEnv->calcMemSize = sizeof(SPercentileInfo); return true; diff --git a/source/libs/nodes/src/nodesCloneFuncs.c b/source/libs/nodes/src/nodesCloneFuncs.c index 41ba7a7459..b354470629 100644 --- a/source/libs/nodes/src/nodesCloneFuncs.c +++ b/source/libs/nodes/src/nodesCloneFuncs.c @@ -449,6 +449,34 @@ static SNode* logicSubplanCopy(const SLogicSubplan* pSrc, SLogicSubplan* pDst) { return (SNode*)pDst; } +static SNode* physiNodeCopy(const SPhysiNode* pSrc, SPhysiNode* pDst) { + CLONE_NODE_FIELD(pOutputDataBlockDesc); + CLONE_NODE_FIELD(pConditions); + CLONE_NODE_LIST_FIELD(pChildren); + return (SNode*)pDst; +} + +static SNode* physiWindowCopy(const SWinodwPhysiNode* pSrc, SWinodwPhysiNode* pDst) { + COPY_BASE_OBJECT_FIELD(node, physiNodeCopy); + CLONE_NODE_LIST_FIELD(pExprs); + CLONE_NODE_LIST_FIELD(pFuncs); + CLONE_NODE_FIELD(pTspk); + COPY_SCALAR_FIELD(triggerType); + COPY_SCALAR_FIELD(watermark); + COPY_SCALAR_FIELD(filesFactor); + return (SNode*)pDst; +} + +static SNode* physiIntervalCopy(const SIntervalPhysiNode* pSrc, SIntervalPhysiNode* pDst) { + COPY_BASE_OBJECT_FIELD(window, physiWindowCopy); + COPY_SCALAR_FIELD(interval); + COPY_SCALAR_FIELD(offset); + COPY_SCALAR_FIELD(sliding); + COPY_SCALAR_FIELD(intervalUnit); + COPY_SCALAR_FIELD(slidingUnit); + return (SNode*)pDst; +} + static SNode* dataBlockDescCopy(const SDataBlockDescNode* pSrc, SDataBlockDescNode* pDst) { COPY_SCALAR_FIELD(dataBlockId); CLONE_NODE_LIST_FIELD(pSlots); @@ -575,6 +603,12 @@ SNodeptr nodesCloneNode(const SNodeptr pNode) { return logicIndefRowsFuncCopy((const SIndefRowsFuncLogicNode*)pNode, (SIndefRowsFuncLogicNode*)pDst); case QUERY_NODE_LOGIC_SUBPLAN: return logicSubplanCopy((const SLogicSubplan*)pNode, (SLogicSubplan*)pDst); + case QUERY_NODE_PHYSICAL_PLAN_HASH_INTERVAL: + case QUERY_NODE_PHYSICAL_PLAN_MERGE_INTERVAL: + case QUERY_NODE_PHYSICAL_PLAN_STREAM_INTERVAL: + case QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_INTERVAL: + case QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_INTERVAL: + return physiIntervalCopy((const SIntervalPhysiNode*)pNode, (SIntervalPhysiNode*)pDst); default: break; } diff --git a/source/libs/nodes/src/nodesCodeFuncs.c b/source/libs/nodes/src/nodesCodeFuncs.c index 2feb25d2bb..fa4d1ba171 100644 --- a/source/libs/nodes/src/nodesCodeFuncs.c +++ b/source/libs/nodes/src/nodesCodeFuncs.c @@ -2441,6 +2441,7 @@ static const char* jkValueLiteralSize = "LiteralSize"; static const char* jkValueLiteral = "Literal"; static const char* jkValueDuration = "Duration"; static const char* jkValueTranslate = "Translate"; +static const char* jkValueNotReserved = "NotReserved"; static const char* jkValueDatum = "Datum"; static int32_t datumToJson(const void* pObj, SJson* pJson) { @@ -2513,6 +2514,9 @@ static int32_t valueNodeToJson(const void* pObj, SJson* pJson) { if (TSDB_CODE_SUCCESS == code) { code = tjsonAddBoolToObject(pJson, jkValueTranslate, pNode->translate); } + if (TSDB_CODE_SUCCESS == code) { + code = tjsonAddBoolToObject(pJson, jkValueNotReserved, pNode->notReserved); + } if (TSDB_CODE_SUCCESS == code && pNode->translate) { code = datumToJson(pNode, pJson); } @@ -2634,6 +2638,9 @@ static int32_t jsonToValueNode(const SJson* pJson, void* pObj) { if (TSDB_CODE_SUCCESS == code) { code = tjsonGetBoolValue(pJson, jkValueTranslate, &pNode->translate); } + if (TSDB_CODE_SUCCESS == code) { + code = tjsonGetBoolValue(pJson, jkValueNotReserved, &pNode->notReserved); + } if (TSDB_CODE_SUCCESS == code && pNode->translate) { code = jsonToDatum(pJson, pNode); } diff --git a/source/libs/nodes/src/nodesUtilFuncs.c b/source/libs/nodes/src/nodesUtilFuncs.c index d8914fc7e6..a753871834 100644 --- a/source/libs/nodes/src/nodesUtilFuncs.c +++ b/source/libs/nodes/src/nodesUtilFuncs.c @@ -164,9 +164,14 @@ SNodeptr nodesMakeNode(ENodeType type) { return makeNode(type, sizeof(SCreateStreamStmt)); case QUERY_NODE_DROP_STREAM_STMT: return makeNode(type, sizeof(SDropStreamStmt)); + case QUERY_NODE_BALANCE_VGROUP_STMT: + return makeNode(type, sizeof(SBalanceVgroupStmt)); case QUERY_NODE_MERGE_VGROUP_STMT: + return makeNode(type, sizeof(SMergeVgroupStmt)); case QUERY_NODE_REDISTRIBUTE_VGROUP_STMT: + return makeNode(type, sizeof(SRedistributeVgroupStmt)); case QUERY_NODE_SPLIT_VGROUP_STMT: + return makeNode(type, sizeof(SSplitVgroupStmt)); case QUERY_NODE_SYNCDB_STMT: break; case QUERY_NODE_GRANT_STMT: diff --git a/source/libs/parser/inc/parAst.h b/source/libs/parser/inc/parAst.h index ca7fa7a4c6..0c6663d29a 100644 --- a/source/libs/parser/inc/parAst.h +++ b/source/libs/parser/inc/parAst.h @@ -187,6 +187,7 @@ SNode* createCreateStreamStmt(SAstCreateContext* pCxt, bool ignoreExists, const SNode* pOptions, SNode* pQuery); SNode* createDropStreamStmt(SAstCreateContext* pCxt, bool ignoreNotExists, const SToken* pStreamName); SNode* createKillStmt(SAstCreateContext* pCxt, ENodeType type, const SToken* pId); +SNode* createBalanceVgroupStmt(SAstCreateContext* pCxt); SNode* createMergeVgroupStmt(SAstCreateContext* pCxt, const SToken* pVgId1, const SToken* pVgId2); SNode* createRedistributeVgroupStmt(SAstCreateContext* pCxt, const SToken* pVgId, SNodeList* pDnodes); SNode* createSplitVgroupStmt(SAstCreateContext* pCxt, const SToken* pVgId); diff --git a/source/libs/parser/inc/parInsertData.h b/source/libs/parser/inc/parInsertData.h index aeebf51c96..167970838b 100644 --- a/source/libs/parser/inc/parInsertData.h +++ b/source/libs/parser/inc/parInsertData.h @@ -116,8 +116,7 @@ static FORCE_INLINE void getSTSRowAppendInfo(uint8_t rowType, SParsedDataColInfo } static FORCE_INLINE int32_t setBlockInfo(SSubmitBlk *pBlocks, STableDataBlocks *dataBuf, int32_t numOfRows) { - pBlocks->suid = - (TSDB_NORMAL_TABLE == dataBuf->pTableMeta->tableType ? dataBuf->pTableMeta->uid : dataBuf->pTableMeta->suid); + pBlocks->suid = (TSDB_NORMAL_TABLE == dataBuf->pTableMeta->tableType ? 0 : dataBuf->pTableMeta->suid); pBlocks->uid = dataBuf->pTableMeta->uid; pBlocks->sversion = dataBuf->pTableMeta->sversion; pBlocks->schemaLen = dataBuf->createTbReqLen; diff --git a/source/libs/parser/inc/sql.y b/source/libs/parser/inc/sql.y index d63ce74fa7..bf23ff08b2 100644 --- a/source/libs/parser/inc/sql.y +++ b/source/libs/parser/inc/sql.y @@ -469,9 +469,10 @@ cmd ::= KILL QUERY NK_INTEGER(A). cmd ::= KILL TRANSACTION NK_INTEGER(A). { pCxt->pRootNode = createKillStmt(pCxt, QUERY_NODE_KILL_TRANSACTION_STMT, &A); } /************************************************ merge/redistribute/ vgroup ******************************************/ +cmd ::= BALANCE VGROUP. { pCxt->pRootNode = createBalanceVgroupStmt(pCxt); } cmd ::= MERGE VGROUP NK_INTEGER(A) NK_INTEGER(B). { pCxt->pRootNode = createMergeVgroupStmt(pCxt, &A, &B); } cmd ::= REDISTRIBUTE VGROUP NK_INTEGER(A) dnode_list(B). { pCxt->pRootNode = createRedistributeVgroupStmt(pCxt, &A, B); } -cmd ::= SPLIT VGROUP NK_INTEGER(A). { pCxt->pRootNode = createSplitVgroupStmt(pCxt, &A); } +//cmd ::= SPLIT VGROUP NK_INTEGER(A). { pCxt->pRootNode = createSplitVgroupStmt(pCxt, &A); } %type dnode_list { SNodeList* } %destructor dnode_list { nodesDestroyList($$); } diff --git a/source/libs/parser/src/parAstCreater.c b/source/libs/parser/src/parAstCreater.c index 24db0671ac..613a2d867d 100644 --- a/source/libs/parser/src/parAstCreater.c +++ b/source/libs/parser/src/parAstCreater.c @@ -1435,25 +1435,37 @@ SNode* createKillStmt(SAstCreateContext* pCxt, ENodeType type, const SToken* pId return (SNode*)pStmt; } +SNode* createBalanceVgroupStmt(SAstCreateContext* pCxt) { + CHECK_PARSER_STATUS(pCxt); + SBalanceVgroupStmt* pStmt = nodesMakeNode(QUERY_NODE_BALANCE_VGROUP_STMT); + CHECK_OUT_OF_MEM(pStmt); + return (SNode*)pStmt; +} + SNode* createMergeVgroupStmt(SAstCreateContext* pCxt, const SToken* pVgId1, const SToken* pVgId2) { CHECK_PARSER_STATUS(pCxt); - SNode* pStmt = nodesMakeNode(QUERY_NODE_MERGE_VGROUP_STMT); + SMergeVgroupStmt* pStmt = nodesMakeNode(QUERY_NODE_MERGE_VGROUP_STMT); CHECK_OUT_OF_MEM(pStmt); - return pStmt; + pStmt->vgId1 = taosStr2Int32(pVgId1->z, NULL, 10); + pStmt->vgId2 = taosStr2Int32(pVgId2->z, NULL, 10); + return (SNode*)pStmt; } SNode* createRedistributeVgroupStmt(SAstCreateContext* pCxt, const SToken* pVgId, SNodeList* pDnodes) { CHECK_PARSER_STATUS(pCxt); - SNode* pStmt = nodesMakeNode(QUERY_NODE_REDISTRIBUTE_VGROUP_STMT); + SRedistributeVgroupStmt* pStmt = nodesMakeNode(QUERY_NODE_REDISTRIBUTE_VGROUP_STMT); CHECK_OUT_OF_MEM(pStmt); - return pStmt; + pStmt->vgId = taosStr2Int32(pVgId->z, NULL, 10); + pStmt->pDnodes = pDnodes; + return (SNode*)pStmt; } SNode* createSplitVgroupStmt(SAstCreateContext* pCxt, const SToken* pVgId) { CHECK_PARSER_STATUS(pCxt); - SNode* pStmt = nodesMakeNode(QUERY_NODE_SPLIT_VGROUP_STMT); + SSplitVgroupStmt* pStmt = nodesMakeNode(QUERY_NODE_SPLIT_VGROUP_STMT); CHECK_OUT_OF_MEM(pStmt); - return pStmt; + pStmt->vgId = taosStr2Int32(pVgId->z, NULL, 10); + return (SNode*)pStmt; } SNode* createSyncdbStmt(SAstCreateContext* pCxt, const SToken* pDbName) { diff --git a/source/libs/parser/src/parTokenizer.c b/source/libs/parser/src/parTokenizer.c index d13f17f895..284a3a620b 100644 --- a/source/libs/parser/src/parTokenizer.c +++ b/source/libs/parser/src/parTokenizer.c @@ -41,6 +41,7 @@ static SKeyword keywordTable[] = { {"AS", TK_AS}, {"ASC", TK_ASC}, {"AT_ONCE", TK_AT_ONCE}, + {"BALANCE", TK_BALANCE}, {"BETWEEN", TK_BETWEEN}, {"BINARY", TK_BINARY}, {"BIGINT", TK_BIGINT}, @@ -116,6 +117,7 @@ static SKeyword keywordTable[] = { {"LOCAL", TK_LOCAL}, {"MATCH", TK_MATCH}, {"MAXROWS", TK_MAXROWS}, + {"MERGE", TK_MERGE}, {"MINROWS", TK_MINROWS}, {"MINUS", TK_MINUS}, {"MNODE", TK_MNODE}, @@ -151,6 +153,7 @@ static SKeyword keywordTable[] = { {"QUERY", TK_QUERY}, {"RATIO", TK_RATIO}, {"READ", TK_READ}, + {"REDISTRIBUTE", TK_REDISTRIBUTE}, {"RENAME", TK_RENAME}, {"REPLICA", TK_REPLICA}, {"RESET", TK_RESET}, @@ -171,6 +174,7 @@ static SKeyword keywordTable[] = { {"SNODE", TK_SNODE}, {"SNODES", TK_SNODES}, {"SOFFSET", TK_SOFFSET}, + // {"SPLIT", TK_SPLIT}, {"STABLE", TK_STABLE}, {"STABLES", TK_STABLES}, {"STATE", TK_STATE}, @@ -208,6 +212,7 @@ static SKeyword keywordTable[] = { {"VARCHAR", TK_VARCHAR}, {"VARIABLES", TK_VARIABLES}, {"VERBOSE", TK_VERBOSE}, + {"VGROUP", TK_VGROUP}, {"VGROUPS", TK_VGROUPS}, {"VNODES", TK_VNODES}, {"WAL", TK_WAL}, diff --git a/source/libs/parser/src/parTranslater.c b/source/libs/parser/src/parTranslater.c index d81c6760e8..f77bbb34f3 100644 --- a/source/libs/parser/src/parTranslater.c +++ b/source/libs/parser/src/parTranslater.c @@ -847,9 +847,9 @@ static EDealRes translateJsonOperator(STranslateContext* pCxt, SOperatorNode* pO if (TSDB_DATA_TYPE_JSON != ldt.type || TSDB_DATA_TYPE_BINARY != rdt.type) { return generateDealNodeErrMsg(pCxt, TSDB_CODE_PAR_WRONG_VALUE_TYPE, ((SExprNode*)(pOp->pRight))->aliasName); } - if(pOp->opType == OP_TYPE_JSON_GET_VALUE){ + if (pOp->opType == OP_TYPE_JSON_GET_VALUE) { pOp->node.resType.type = TSDB_DATA_TYPE_JSON; - }else if(pOp->opType == OP_TYPE_JSON_CONTAINS){ + } else if (pOp->opType == OP_TYPE_JSON_CONTAINS) { pOp->node.resType.type = TSDB_DATA_TYPE_BOOL; } pOp->node.resType.bytes = tDataTypes[pOp->node.resType.type].bytes; @@ -3611,6 +3611,55 @@ static int32_t translateRevoke(STranslateContext* pCxt, SRevokeStmt* pStmt) { return buildCmdMsg(pCxt, TDMT_MND_ALTER_USER, (FSerializeFunc)tSerializeSAlterUserReq, &req); } +static int32_t translateBalanceVgroup(STranslateContext* pCxt, SBalanceVgroupStmt* pStmt) { + SBalanceVgroupReq req = {0}; + return buildCmdMsg(pCxt, TDMT_MND_BALANCE_VGROUP, (FSerializeFunc)tSerializeSBalanceVgroupReq, &req); +} + +static int32_t translateMergeVgroup(STranslateContext* pCxt, SMergeVgroupStmt* pStmt) { + SMergeVgroupReq req = {.vgId1 = pStmt->vgId1, .vgId2 = pStmt->vgId2}; + return buildCmdMsg(pCxt, TDMT_MND_MERGE_VGROUP, (FSerializeFunc)tSerializeSMergeVgroupReq, &req); +} + +static int32_t checkDnodeIds(STranslateContext* pCxt, SRedistributeVgroupStmt* pStmt) { + int32_t numOfDnodes = LIST_LENGTH(pStmt->pDnodes); + if (numOfDnodes > 3 || numOfDnodes < 1) { + return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_REDISTRIBUTE_VG); + } + + SNode* pNode = NULL; + FOREACH(pNode, pStmt->pDnodes) { + SValueNode* pVal = (SValueNode*)pNode; + if (DEAL_RES_ERROR == translateValue(pCxt, pVal)) { + return pCxt->errCode; + } + } + + pStmt->dnodeId1 = getBigintFromValueNode((SValueNode*)nodesListGetNode(pStmt->pDnodes, 0)); + pStmt->dnodeId2 = -1; + pStmt->dnodeId3 = -1; + if (numOfDnodes > 1) { + pStmt->dnodeId2 = getBigintFromValueNode((SValueNode*)nodesListGetNode(pStmt->pDnodes, 1)); + } + if (numOfDnodes > 2) { + pStmt->dnodeId3 = getBigintFromValueNode((SValueNode*)nodesListGetNode(pStmt->pDnodes, 2)); + } + + return TSDB_CODE_SUCCESS; +} + +static int32_t translateRedistributeVgroup(STranslateContext* pCxt, SRedistributeVgroupStmt* pStmt) { + SRedistributeVgroupReq req = {.vgId = pStmt->vgId}; + int32_t code = checkDnodeIds(pCxt, pStmt); + if (TSDB_CODE_SUCCESS == code) { + req.dnodeId1 = pStmt->dnodeId1; + req.dnodeId2 = pStmt->dnodeId2; + req.dnodeId3 = pStmt->dnodeId3; + code = buildCmdMsg(pCxt, TDMT_MND_REDISTRIBUTE_VGROUP, (FSerializeFunc)tSerializeSRedistributeVgroupReq, &req); + } + return code; +} + static int32_t translateQuery(STranslateContext* pCxt, SNode* pNode) { int32_t code = TSDB_CODE_SUCCESS; switch (nodeType(pNode)) { @@ -3733,6 +3782,15 @@ static int32_t translateQuery(STranslateContext* pCxt, SNode* pNode) { case QUERY_NODE_REVOKE_STMT: code = translateRevoke(pCxt, (SRevokeStmt*)pNode); break; + case QUERY_NODE_BALANCE_VGROUP_STMT: + code = translateBalanceVgroup(pCxt, (SBalanceVgroupStmt*)pNode); + break; + case QUERY_NODE_MERGE_VGROUP_STMT: + code = translateMergeVgroup(pCxt, (SMergeVgroupStmt*)pNode); + break; + case QUERY_NODE_REDISTRIBUTE_VGROUP_STMT: + code = translateRedistributeVgroup(pCxt, (SRedistributeVgroupStmt*)pNode); + break; default: break; } diff --git a/source/libs/parser/src/parUtil.c b/source/libs/parser/src/parUtil.c index f30cc1111f..147ff933fb 100644 --- a/source/libs/parser/src/parUtil.c +++ b/source/libs/parser/src/parUtil.c @@ -180,6 +180,8 @@ static char* getSyntaxErrFormat(int32_t errCode) { return "Value too long for column/tag: %s"; case TSDB_CODE_PAR_INVALID_DELETE_WHERE: return "The DELETE statement must have a definite time window range"; + case TSDB_CODE_PAR_INVALID_REDISTRIBUTE_VG: + return "The REDISTRIBUTE VGROUP statement only support 1 to 3 dnodes"; case TSDB_CODE_OUT_OF_MEMORY: return "Out of memory"; default: diff --git a/source/libs/parser/src/sql.c b/source/libs/parser/src/sql.c index ed837ae20f..1626c05454 100644 --- a/source/libs/parser/src/sql.c +++ b/source/libs/parser/src/sql.c @@ -138,17 +138,17 @@ typedef union { #define ParseCTX_FETCH #define ParseCTX_STORE #define YYFALLBACK 1 -#define YYNSTATE 616 +#define YYNSTATE 615 #define YYNRULE 452 #define YYNTOKEN 238 -#define YY_MAX_SHIFT 615 -#define YY_MIN_SHIFTREDUCE 902 -#define YY_MAX_SHIFTREDUCE 1353 -#define YY_ERROR_ACTION 1354 -#define YY_ACCEPT_ACTION 1355 -#define YY_NO_ACTION 1356 -#define YY_MIN_REDUCE 1357 -#define YY_MAX_REDUCE 1808 +#define YY_MAX_SHIFT 614 +#define YY_MIN_SHIFTREDUCE 901 +#define YY_MAX_SHIFTREDUCE 1352 +#define YY_ERROR_ACTION 1353 +#define YY_ACCEPT_ACTION 1354 +#define YY_NO_ACTION 1355 +#define YY_MIN_REDUCE 1356 +#define YY_MAX_REDUCE 1807 /************* End control #defines *******************************************/ #define YY_NLOOKAHEAD ((int)(sizeof(yy_lookahead)/sizeof(yy_lookahead[0]))) @@ -217,223 +217,223 @@ typedef union { *********** Begin parsing tables **********************************************/ #define YY_ACTTAB_COUNT (2165) static const YYACTIONTYPE yy_action[] = { - /* 0 */ 72, 355, 1655, 1642, 28, 230, 296, 1465, 1642, 608, - /* 10 */ 607, 313, 35, 33, 530, 36, 34, 32, 31, 30, - /* 20 */ 305, 1481, 1167, 1639, 530, 490, 106, 1477, 1639, 389, - /* 30 */ 1671, 390, 1389, 428, 490, 277, 152, 349, 514, 1635, - /* 40 */ 1641, 529, 133, 1488, 1635, 1641, 1445, 1165, 513, 132, - /* 50 */ 533, 1369, 1625, 1488, 114, 533, 529, 494, 14, 530, - /* 60 */ 35, 33, 1294, 114, 1173, 1416, 932, 63, 305, 1684, - /* 70 */ 1167, 353, 82, 1656, 516, 1658, 1659, 512, 1786, 533, - /* 80 */ 110, 1, 1724, 56, 1532, 1308, 278, 1720, 1488, 1483, - /* 90 */ 147, 295, 112, 517, 1783, 1165, 1530, 529, 1786, 1578, - /* 100 */ 564, 112, 39, 612, 936, 937, 14, 145, 1731, 1732, - /* 110 */ 149, 1736, 1173, 1166, 1783, 492, 144, 1731, 1732, 563, - /* 120 */ 1736, 562, 561, 560, 407, 588, 587, 586, 320, 2, - /* 130 */ 585, 584, 583, 116, 578, 577, 576, 575, 574, 573, - /* 140 */ 572, 571, 123, 567, 1358, 36, 34, 32, 31, 30, - /* 150 */ 397, 612, 390, 1389, 142, 317, 1168, 1191, 38, 1671, - /* 160 */ 55, 1166, 1568, 1570, 1466, 96, 1526, 483, 95, 94, - /* 170 */ 93, 92, 91, 90, 89, 88, 87, 479, 388, 1171, - /* 180 */ 1172, 392, 1218, 1219, 1221, 1222, 1223, 1224, 1225, 509, - /* 190 */ 531, 1233, 1234, 1235, 1236, 1237, 1238, 36, 34, 32, - /* 200 */ 31, 30, 517, 482, 1168, 308, 581, 96, 1577, 150, + /* 0 */ 72, 354, 1654, 1641, 28, 230, 296, 1464, 1641, 607, + /* 10 */ 606, 313, 35, 33, 529, 36, 34, 32, 31, 30, + /* 20 */ 305, 1480, 1166, 1638, 529, 489, 106, 1476, 1638, 388, + /* 30 */ 1670, 389, 1388, 427, 489, 277, 152, 348, 513, 1634, + /* 40 */ 1640, 528, 133, 1487, 1634, 1640, 1444, 1164, 512, 132, + /* 50 */ 532, 1368, 1624, 1487, 114, 532, 528, 493, 14, 529, + /* 60 */ 35, 33, 1293, 114, 1172, 1415, 931, 63, 305, 1683, + /* 70 */ 1166, 352, 82, 1655, 515, 1657, 1658, 511, 1785, 532, + /* 80 */ 110, 1, 1723, 56, 1531, 1307, 278, 1719, 1487, 1482, + /* 90 */ 147, 295, 112, 516, 1782, 1164, 1529, 528, 1785, 1577, + /* 100 */ 563, 112, 39, 611, 935, 936, 14, 145, 1730, 1731, + /* 110 */ 149, 1735, 1172, 1165, 1782, 491, 144, 1730, 1731, 562, + /* 120 */ 1735, 561, 560, 559, 406, 587, 586, 585, 320, 2, + /* 130 */ 584, 583, 582, 116, 577, 576, 575, 574, 573, 572, + /* 140 */ 571, 570, 123, 566, 1357, 36, 34, 32, 31, 30, + /* 150 */ 396, 611, 389, 1388, 142, 317, 1167, 1190, 38, 1670, + /* 160 */ 55, 1165, 1567, 1569, 1465, 96, 1525, 482, 95, 94, + /* 170 */ 93, 92, 91, 90, 89, 88, 87, 478, 387, 1170, + /* 180 */ 1171, 391, 1217, 1218, 1220, 1221, 1222, 1223, 1224, 508, + /* 190 */ 530, 1232, 1233, 1234, 1235, 1236, 1237, 36, 34, 32, + /* 200 */ 31, 30, 516, 481, 1167, 308, 580, 96, 1576, 150, /* 210 */ 95, 94, 93, 92, 91, 90, 89, 88, 87, 288, - /* 220 */ 130, 407, 36, 34, 32, 31, 30, 1171, 1172, 1491, - /* 230 */ 1218, 1219, 1221, 1222, 1223, 1224, 1225, 509, 531, 1233, - /* 240 */ 1234, 1235, 1236, 1237, 1238, 35, 33, 343, 485, 480, - /* 250 */ 1355, 442, 441, 305, 1655, 1167, 440, 200, 252, 111, - /* 260 */ 437, 1518, 379, 436, 435, 434, 35, 33, 289, 447, - /* 270 */ 287, 286, 1001, 430, 305, 1479, 1167, 432, 129, 309, - /* 280 */ 1165, 79, 1671, 55, 455, 66, 150, 130, 150, 1003, - /* 290 */ 514, 14, 1205, 484, 115, 1639, 1490, 1173, 193, 431, - /* 300 */ 513, 1165, 1480, 949, 1625, 948, 161, 160, 321, 494, - /* 310 */ 450, 1635, 1641, 394, 2, 444, 1192, 1643, 1173, 1189, - /* 320 */ 192, 1684, 533, 490, 82, 1656, 516, 1658, 1659, 512, - /* 330 */ 24, 533, 950, 1350, 1724, 8, 612, 1639, 278, 1720, - /* 340 */ 36, 34, 32, 31, 30, 51, 1166, 1464, 50, 1786, - /* 350 */ 1786, 566, 114, 1635, 1641, 65, 294, 612, 1380, 194, - /* 360 */ 559, 147, 147, 1463, 533, 1783, 1783, 1166, 1357, 1039, - /* 370 */ 556, 555, 554, 1043, 553, 1045, 1046, 552, 1048, 549, - /* 380 */ 1379, 1054, 546, 1056, 1057, 543, 540, 7, 55, 1168, - /* 390 */ 112, 1190, 105, 104, 103, 102, 101, 100, 99, 98, - /* 400 */ 97, 32, 31, 30, 566, 146, 1731, 1732, 1625, 1736, - /* 410 */ 1168, 150, 1171, 1172, 1349, 1218, 1219, 1221, 1222, 1223, - /* 420 */ 1224, 1225, 509, 531, 1233, 1234, 1235, 1236, 1237, 1238, - /* 430 */ 1625, 569, 157, 1171, 1172, 463, 1218, 1219, 1221, 1222, - /* 440 */ 1223, 1224, 1225, 509, 531, 1233, 1234, 1235, 1236, 1237, - /* 450 */ 1238, 35, 33, 1239, 26, 1378, 564, 61, 466, 305, - /* 460 */ 60, 1167, 564, 150, 36, 34, 32, 31, 30, 63, - /* 470 */ 1406, 396, 35, 33, 392, 563, 1655, 562, 561, 560, - /* 480 */ 305, 563, 1167, 562, 561, 560, 1165, 1193, 530, 1318, - /* 490 */ 530, 1484, 443, 36, 34, 32, 31, 30, 1786, 1786, - /* 500 */ 354, 1645, 106, 1173, 1671, 1625, 1473, 1165, 55, 433, - /* 510 */ 1785, 147, 493, 456, 1783, 1783, 150, 1488, 1377, 1488, - /* 520 */ 9, 570, 513, 1460, 1173, 1419, 1625, 454, 476, 1316, - /* 530 */ 1317, 1319, 1320, 36, 34, 32, 31, 30, 1647, 279, - /* 540 */ 452, 9, 612, 1684, 439, 438, 83, 1656, 516, 1658, - /* 550 */ 1659, 512, 1166, 533, 1786, 1475, 1724, 1532, 1289, 315, - /* 560 */ 298, 1720, 143, 612, 310, 1786, 147, 130, 1625, 1530, - /* 570 */ 1783, 1256, 1301, 1166, 222, 1786, 1490, 148, 1191, 472, - /* 580 */ 1751, 1783, 348, 1738, 347, 442, 441, 1784, 530, 340, - /* 590 */ 440, 1783, 1532, 111, 437, 1168, 1532, 436, 435, 434, - /* 600 */ 364, 582, 580, 316, 1531, 1569, 1570, 1735, 1530, 342, - /* 610 */ 338, 1293, 221, 1376, 936, 937, 1168, 1488, 1171, 1172, - /* 620 */ 1257, 1218, 1219, 1221, 1222, 1223, 1224, 1225, 509, 531, - /* 630 */ 1233, 1234, 1235, 1236, 1237, 1238, 150, 11, 10, 1171, - /* 640 */ 1172, 1262, 1218, 1219, 1221, 1222, 1223, 1224, 1225, 509, - /* 650 */ 531, 1233, 1234, 1235, 1236, 1237, 1238, 35, 33, 276, - /* 660 */ 1655, 1189, 530, 1625, 530, 305, 1375, 1167, 372, 54, - /* 670 */ 1374, 384, 1373, 1471, 365, 197, 406, 508, 27, 303, - /* 680 */ 1251, 1252, 1253, 1254, 1255, 1259, 1260, 1261, 1671, 385, - /* 690 */ 530, 1488, 1165, 1488, 1244, 530, 514, 1372, 1565, 530, - /* 700 */ 1191, 432, 1485, 1371, 318, 159, 513, 1604, 1191, 1173, - /* 710 */ 1625, 464, 130, 530, 1151, 1152, 1625, 1194, 1270, 1488, - /* 720 */ 1625, 1490, 1625, 431, 1488, 527, 2, 1684, 1488, 1258, - /* 730 */ 83, 1656, 516, 1658, 1659, 512, 948, 533, 1368, 1367, - /* 740 */ 1724, 1220, 1488, 498, 298, 1720, 1799, 1625, 612, 120, - /* 750 */ 1263, 1743, 1289, 1625, 1366, 1758, 1365, 1364, 1166, 383, - /* 760 */ 1363, 426, 378, 377, 376, 375, 374, 371, 370, 369, - /* 770 */ 368, 367, 363, 362, 361, 360, 359, 358, 357, 356, - /* 780 */ 530, 131, 530, 530, 1738, 1370, 258, 25, 1625, 1625, - /* 790 */ 468, 1738, 528, 1362, 243, 319, 1361, 501, 256, 53, - /* 800 */ 1613, 1168, 52, 1360, 1625, 506, 1625, 1625, 1734, 1488, - /* 810 */ 1625, 1488, 1488, 185, 1446, 1733, 183, 1292, 187, 162, - /* 820 */ 189, 186, 1176, 188, 1171, 1172, 46, 1218, 1219, 1221, - /* 830 */ 1222, 1223, 1224, 1225, 509, 531, 1233, 1234, 1235, 1236, - /* 840 */ 1237, 1238, 206, 1625, 55, 330, 1625, 36, 34, 32, - /* 850 */ 31, 30, 191, 1625, 1401, 190, 1167, 1399, 209, 11, - /* 860 */ 10, 1352, 1353, 1220, 46, 37, 37, 1315, 37, 232, - /* 870 */ 558, 1220, 457, 118, 119, 78, 445, 120, 1175, 448, - /* 880 */ 81, 1165, 538, 225, 973, 74, 119, 477, 1179, 120, - /* 890 */ 496, 216, 1672, 279, 121, 1390, 1655, 1527, 1173, 211, - /* 900 */ 425, 974, 119, 1754, 491, 251, 1264, 1226, 322, 1124, - /* 910 */ 234, 59, 58, 352, 522, 240, 156, 1205, 1032, 1189, - /* 920 */ 224, 346, 3, 1060, 1671, 1256, 1248, 1064, 499, 227, - /* 930 */ 1070, 229, 493, 275, 5, 1068, 336, 612, 332, 328, - /* 940 */ 153, 323, 513, 122, 1178, 329, 1625, 1166, 325, 284, - /* 950 */ 1001, 285, 312, 311, 248, 1135, 158, 366, 1567, 373, - /* 960 */ 381, 380, 1181, 1684, 382, 386, 83, 1656, 516, 1658, - /* 970 */ 1659, 512, 150, 533, 1257, 1655, 1724, 1195, 1198, 387, - /* 980 */ 298, 1720, 143, 395, 502, 398, 399, 1174, 1197, 400, - /* 990 */ 1168, 165, 167, 1199, 170, 1262, 172, 404, 401, 403, - /* 1000 */ 1752, 1196, 175, 1671, 1173, 62, 405, 408, 178, 427, - /* 1010 */ 1173, 514, 86, 1171, 1172, 429, 293, 1478, 1608, 182, - /* 1020 */ 1474, 513, 184, 124, 249, 1625, 125, 1476, 1655, 1472, - /* 1030 */ 126, 127, 27, 303, 1251, 1252, 1253, 1254, 1255, 1259, - /* 1040 */ 1260, 1261, 1684, 534, 195, 83, 1656, 516, 1658, 1659, - /* 1050 */ 512, 458, 533, 1177, 198, 1724, 1671, 462, 465, 298, - /* 1060 */ 1720, 1799, 201, 467, 514, 469, 459, 470, 478, 204, - /* 1070 */ 1781, 1194, 1765, 207, 513, 520, 1764, 475, 1625, 1655, - /* 1080 */ 297, 1755, 210, 481, 215, 6, 487, 1745, 474, 217, - /* 1090 */ 1655, 1289, 113, 1193, 40, 1684, 1182, 503, 83, 1656, - /* 1100 */ 516, 1658, 1659, 512, 1739, 533, 218, 1671, 1724, 137, - /* 1110 */ 500, 299, 298, 1720, 1799, 514, 18, 518, 1671, 1185, - /* 1120 */ 519, 1705, 1576, 1742, 1782, 513, 514, 223, 226, 1625, - /* 1130 */ 531, 1233, 1234, 228, 494, 1575, 513, 1802, 1655, 523, - /* 1140 */ 1625, 307, 236, 497, 524, 494, 1684, 238, 504, 263, - /* 1150 */ 1656, 516, 1658, 1659, 512, 525, 533, 1684, 250, 71, - /* 1160 */ 263, 1656, 516, 1658, 1659, 512, 1671, 533, 1489, 73, - /* 1170 */ 1461, 536, 253, 245, 514, 1786, 611, 47, 1655, 255, - /* 1180 */ 136, 264, 274, 265, 513, 1619, 1786, 149, 1625, 257, - /* 1190 */ 1618, 1783, 57, 1617, 324, 1614, 326, 327, 147, 331, - /* 1200 */ 1160, 1161, 1783, 154, 1612, 1684, 1671, 333, 84, 1656, - /* 1210 */ 516, 1658, 1659, 512, 514, 533, 334, 335, 1724, 1611, - /* 1220 */ 180, 337, 1723, 1720, 513, 1610, 339, 1609, 1625, 341, - /* 1230 */ 1594, 155, 141, 1138, 344, 345, 1137, 1655, 424, 420, - /* 1240 */ 416, 412, 179, 1588, 1587, 1684, 350, 351, 84, 1656, - /* 1250 */ 516, 1658, 1659, 512, 1586, 533, 1585, 1107, 1724, 1109, - /* 1260 */ 117, 1542, 505, 1720, 1560, 1671, 1559, 64, 1558, 1557, - /* 1270 */ 177, 1556, 1555, 511, 1554, 1553, 1552, 1551, 1550, 1549, - /* 1280 */ 1548, 1547, 1546, 513, 1545, 1544, 1543, 1625, 1541, 1655, - /* 1290 */ 1540, 1539, 1538, 1537, 1536, 1535, 1534, 1533, 1655, 1418, - /* 1300 */ 1386, 140, 163, 939, 1684, 1385, 1602, 272, 1656, 516, - /* 1310 */ 1658, 1659, 512, 510, 533, 507, 1696, 1671, 938, 1596, - /* 1320 */ 108, 1584, 171, 1583, 109, 514, 1671, 176, 1573, 168, - /* 1330 */ 1467, 173, 967, 402, 514, 513, 1417, 1415, 1413, 1625, - /* 1340 */ 164, 1411, 391, 1409, 513, 393, 169, 174, 1625, 411, - /* 1350 */ 409, 166, 410, 415, 1655, 413, 1684, 1398, 414, 134, - /* 1360 */ 1656, 516, 1658, 1659, 512, 1684, 533, 417, 84, 1656, - /* 1370 */ 516, 1658, 1659, 512, 418, 533, 419, 422, 1724, 615, - /* 1380 */ 421, 423, 1671, 1721, 1397, 1384, 1469, 1074, 1073, 1468, - /* 1390 */ 514, 579, 1000, 247, 999, 998, 997, 581, 994, 993, - /* 1400 */ 513, 992, 495, 1800, 1625, 107, 181, 473, 1655, 1407, - /* 1410 */ 290, 604, 600, 596, 592, 246, 45, 1655, 1402, 291, - /* 1420 */ 446, 1684, 1400, 292, 273, 1656, 516, 1658, 1659, 512, - /* 1430 */ 449, 533, 1383, 451, 1382, 453, 1671, 85, 1601, 1145, - /* 1440 */ 80, 1595, 460, 241, 514, 1671, 1582, 1581, 1580, 1572, - /* 1450 */ 67, 203, 15, 514, 513, 208, 4, 37, 1625, 49, - /* 1460 */ 461, 135, 48, 513, 199, 43, 213, 1625, 16, 1314, - /* 1470 */ 1655, 205, 212, 214, 22, 1684, 526, 1307, 268, 1656, - /* 1480 */ 516, 1658, 1659, 512, 1684, 533, 128, 134, 1656, 516, - /* 1490 */ 1658, 1659, 512, 68, 533, 1645, 23, 220, 1671, 42, - /* 1500 */ 1655, 471, 17, 1286, 202, 1285, 514, 138, 1343, 1338, - /* 1510 */ 1332, 1337, 300, 1342, 1341, 486, 513, 301, 10, 19, - /* 1520 */ 1625, 1228, 1143, 302, 196, 1249, 29, 139, 1671, 151, - /* 1530 */ 1213, 1801, 1227, 12, 515, 20, 511, 1684, 21, 521, - /* 1540 */ 273, 1656, 516, 1658, 1659, 512, 513, 533, 13, 41, - /* 1550 */ 1625, 231, 1655, 1312, 233, 235, 69, 1571, 70, 237, - /* 1560 */ 1687, 1644, 242, 74, 1230, 532, 1183, 1684, 1655, 44, - /* 1570 */ 272, 1656, 516, 1658, 1659, 512, 239, 533, 535, 1697, - /* 1580 */ 1671, 1061, 537, 314, 539, 541, 1058, 542, 514, 1055, - /* 1590 */ 544, 545, 547, 1049, 548, 550, 1671, 1047, 513, 1053, - /* 1600 */ 551, 1052, 1625, 1051, 514, 304, 1050, 1038, 1655, 75, - /* 1610 */ 76, 1069, 77, 557, 513, 1067, 1066, 965, 1625, 1684, - /* 1620 */ 1655, 306, 273, 1656, 516, 1658, 1659, 512, 565, 533, - /* 1630 */ 989, 1007, 244, 568, 987, 1684, 1671, 986, 273, 1656, - /* 1640 */ 516, 1658, 1659, 512, 514, 533, 985, 984, 1671, 983, - /* 1650 */ 982, 981, 980, 1004, 513, 1002, 514, 977, 1625, 976, - /* 1660 */ 975, 972, 971, 970, 1414, 589, 513, 590, 1655, 591, - /* 1670 */ 1625, 1412, 593, 595, 594, 1684, 1410, 597, 259, 1656, - /* 1680 */ 516, 1658, 1659, 512, 598, 533, 599, 1684, 1408, 601, - /* 1690 */ 267, 1656, 516, 1658, 1659, 512, 1671, 533, 602, 603, - /* 1700 */ 1655, 605, 606, 1395, 514, 1396, 1381, 609, 610, 1356, - /* 1710 */ 1356, 1169, 254, 613, 513, 614, 1356, 1356, 1625, 1356, - /* 1720 */ 1356, 1356, 1356, 1356, 1356, 1356, 1356, 1356, 1671, 1356, - /* 1730 */ 1655, 1356, 1356, 1356, 1356, 1684, 514, 1356, 269, 1656, - /* 1740 */ 516, 1658, 1659, 512, 1356, 533, 513, 1356, 1356, 1356, - /* 1750 */ 1625, 1356, 1356, 1356, 1356, 1356, 1356, 1356, 1671, 1356, - /* 1760 */ 1356, 1356, 1655, 1356, 1356, 1356, 514, 1684, 1356, 1356, - /* 1770 */ 260, 1656, 516, 1658, 1659, 512, 513, 533, 1655, 1356, - /* 1780 */ 1625, 1356, 1356, 1356, 1356, 1356, 1356, 1356, 1356, 1356, - /* 1790 */ 1671, 1356, 1356, 1356, 1655, 1356, 1356, 1684, 514, 1356, - /* 1800 */ 270, 1656, 516, 1658, 1659, 512, 1671, 533, 513, 1356, - /* 1810 */ 1655, 1356, 1625, 1356, 514, 1356, 1356, 1356, 1356, 1356, - /* 1820 */ 1356, 1356, 1671, 1356, 513, 1356, 1356, 1356, 1625, 1684, - /* 1830 */ 514, 1356, 261, 1656, 516, 1658, 1659, 512, 1671, 533, - /* 1840 */ 513, 1356, 1356, 1356, 1625, 1684, 514, 1356, 271, 1656, - /* 1850 */ 516, 1658, 1659, 512, 1356, 533, 513, 1356, 1356, 1356, - /* 1860 */ 1625, 1684, 1655, 1356, 262, 1656, 516, 1658, 1659, 512, - /* 1870 */ 1356, 533, 1356, 1356, 1356, 1356, 1356, 1684, 1655, 1356, - /* 1880 */ 1667, 1656, 516, 1658, 1659, 512, 1356, 533, 1356, 1356, - /* 1890 */ 1671, 1356, 1356, 1356, 1356, 1356, 1356, 1356, 514, 1356, - /* 1900 */ 1356, 1356, 1356, 1356, 1356, 1356, 1671, 1356, 513, 1356, - /* 1910 */ 1655, 1356, 1625, 1356, 514, 1356, 1356, 1356, 1356, 1356, - /* 1920 */ 1356, 1356, 1356, 1356, 513, 1356, 1356, 1356, 1625, 1684, - /* 1930 */ 1356, 1356, 1666, 1656, 516, 1658, 1659, 512, 1671, 533, - /* 1940 */ 1356, 1356, 1655, 1356, 1356, 1684, 514, 1356, 1665, 1656, - /* 1950 */ 516, 1658, 1659, 512, 1356, 533, 513, 1356, 1356, 1356, - /* 1960 */ 1625, 1356, 1356, 1356, 1356, 1356, 1356, 1356, 1356, 1356, - /* 1970 */ 1671, 1356, 1655, 1356, 1356, 1356, 1356, 1684, 514, 1356, - /* 1980 */ 282, 1656, 516, 1658, 1659, 512, 1356, 533, 513, 1356, - /* 1990 */ 1356, 1356, 1625, 1356, 1356, 1356, 1356, 1356, 1356, 1356, - /* 2000 */ 1671, 1356, 1356, 1356, 1655, 1356, 1356, 1356, 514, 1684, - /* 2010 */ 1356, 1356, 281, 1656, 516, 1658, 1659, 512, 513, 533, - /* 2020 */ 1655, 1356, 1625, 1356, 1356, 1356, 1356, 1356, 1356, 1356, - /* 2030 */ 1356, 1356, 1671, 1356, 1356, 1356, 1356, 1356, 1356, 1684, - /* 2040 */ 514, 1356, 283, 1656, 516, 1658, 1659, 512, 1671, 533, - /* 2050 */ 513, 490, 1356, 1356, 1625, 1356, 514, 1356, 1356, 490, - /* 2060 */ 1356, 1356, 1356, 1356, 1356, 1356, 513, 1356, 1356, 1356, - /* 2070 */ 1625, 1684, 1356, 1356, 280, 1656, 516, 1658, 1659, 512, - /* 2080 */ 114, 533, 1356, 1356, 1356, 1356, 1356, 1684, 114, 1356, - /* 2090 */ 266, 1656, 516, 1658, 1659, 512, 1356, 533, 1356, 494, - /* 2100 */ 1356, 1356, 1356, 1356, 1356, 1356, 1356, 494, 1356, 1356, - /* 2110 */ 1356, 1356, 1356, 1356, 1356, 1356, 1356, 1356, 112, 1356, - /* 2120 */ 1356, 1356, 1356, 1356, 1356, 1356, 112, 1356, 1356, 1356, - /* 2130 */ 1356, 1356, 1356, 219, 1731, 489, 1356, 488, 1356, 1356, - /* 2140 */ 1786, 219, 1731, 489, 1356, 488, 1356, 1356, 1786, 1356, - /* 2150 */ 1356, 1356, 149, 1356, 1356, 1356, 1783, 1356, 1356, 1356, - /* 2160 */ 147, 1356, 1356, 1356, 1783, + /* 220 */ 130, 406, 36, 34, 32, 31, 30, 1170, 1171, 1490, + /* 230 */ 1217, 1218, 1220, 1221, 1222, 1223, 1224, 508, 530, 1232, + /* 240 */ 1233, 1234, 1235, 1236, 1237, 35, 33, 342, 484, 479, + /* 250 */ 1354, 441, 440, 305, 1654, 1166, 439, 200, 252, 111, + /* 260 */ 436, 1517, 378, 435, 434, 433, 35, 33, 289, 446, + /* 270 */ 287, 286, 1000, 429, 305, 1478, 1166, 431, 129, 309, + /* 280 */ 1164, 79, 1670, 55, 454, 66, 150, 130, 150, 1002, + /* 290 */ 513, 14, 1204, 483, 115, 1638, 1489, 1172, 193, 430, + /* 300 */ 512, 1164, 1479, 948, 1624, 947, 161, 160, 321, 493, + /* 310 */ 449, 1634, 1640, 393, 2, 443, 1191, 1642, 1172, 1188, + /* 320 */ 192, 1683, 532, 489, 82, 1655, 515, 1657, 1658, 511, + /* 330 */ 24, 532, 949, 1349, 1723, 8, 611, 1638, 278, 1719, + /* 340 */ 36, 34, 32, 31, 30, 51, 1165, 1463, 50, 1785, + /* 350 */ 1785, 565, 114, 1634, 1640, 65, 294, 611, 1379, 194, + /* 360 */ 558, 147, 147, 1462, 532, 1782, 1782, 1165, 1356, 1038, + /* 370 */ 555, 554, 553, 1042, 552, 1044, 1045, 551, 1047, 548, + /* 380 */ 1378, 1053, 545, 1055, 1056, 542, 539, 7, 55, 1167, + /* 390 */ 112, 1189, 105, 104, 103, 102, 101, 100, 99, 98, + /* 400 */ 97, 32, 31, 30, 565, 146, 1730, 1731, 1624, 1735, + /* 410 */ 1167, 150, 1170, 1171, 1348, 1217, 1218, 1220, 1221, 1222, + /* 420 */ 1223, 1224, 508, 530, 1232, 1233, 1234, 1235, 1236, 1237, + /* 430 */ 1624, 568, 157, 1170, 1171, 462, 1217, 1218, 1220, 1221, + /* 440 */ 1222, 1223, 1224, 508, 530, 1232, 1233, 1234, 1235, 1236, + /* 450 */ 1237, 35, 33, 1238, 26, 1377, 563, 61, 465, 305, + /* 460 */ 60, 1166, 563, 150, 36, 34, 32, 31, 30, 63, + /* 470 */ 1405, 395, 35, 33, 391, 562, 1654, 561, 560, 559, + /* 480 */ 305, 562, 1166, 561, 560, 559, 1164, 1192, 529, 1317, + /* 490 */ 529, 1483, 442, 36, 34, 32, 31, 30, 1785, 1785, + /* 500 */ 353, 1644, 106, 1172, 1670, 1624, 1472, 1164, 55, 432, + /* 510 */ 1784, 147, 492, 455, 1782, 1782, 150, 1487, 1376, 1487, + /* 520 */ 9, 569, 512, 1459, 1172, 1418, 1624, 453, 475, 1315, + /* 530 */ 1316, 1318, 1319, 36, 34, 32, 31, 30, 1646, 279, + /* 540 */ 451, 9, 611, 1683, 438, 437, 83, 1655, 515, 1657, + /* 550 */ 1658, 511, 1165, 532, 1785, 1474, 1723, 1531, 1288, 315, + /* 560 */ 298, 1719, 143, 611, 310, 1785, 147, 130, 1624, 1529, + /* 570 */ 1782, 1255, 1300, 1165, 222, 1785, 1489, 148, 1190, 471, + /* 580 */ 1750, 1782, 347, 1737, 346, 441, 440, 1783, 529, 339, + /* 590 */ 439, 1782, 1531, 111, 436, 1167, 1531, 435, 434, 433, + /* 600 */ 363, 581, 579, 316, 1530, 1568, 1569, 1734, 1529, 341, + /* 610 */ 337, 1292, 221, 1375, 935, 936, 1167, 1487, 1170, 1171, + /* 620 */ 1256, 1217, 1218, 1220, 1221, 1222, 1223, 1224, 508, 530, + /* 630 */ 1232, 1233, 1234, 1235, 1236, 1237, 150, 11, 10, 1170, + /* 640 */ 1171, 1261, 1217, 1218, 1220, 1221, 1222, 1223, 1224, 508, + /* 650 */ 530, 1232, 1233, 1234, 1235, 1236, 1237, 35, 33, 276, + /* 660 */ 1654, 1188, 529, 1624, 529, 305, 1374, 1166, 371, 54, + /* 670 */ 1373, 383, 1372, 1470, 364, 197, 405, 507, 27, 303, + /* 680 */ 1250, 1251, 1252, 1253, 1254, 1258, 1259, 1260, 1670, 384, + /* 690 */ 529, 1487, 1164, 1487, 1243, 529, 513, 1371, 1564, 529, + /* 700 */ 1190, 431, 1484, 1370, 318, 159, 512, 1603, 1190, 1172, + /* 710 */ 1624, 463, 130, 529, 1150, 1151, 1624, 1193, 1269, 1487, + /* 720 */ 1624, 1489, 1624, 430, 1487, 526, 2, 1683, 1487, 1257, + /* 730 */ 83, 1655, 515, 1657, 1658, 511, 947, 532, 1367, 1366, + /* 740 */ 1723, 1219, 1487, 497, 298, 1719, 1798, 1624, 611, 120, + /* 750 */ 1262, 1742, 1288, 1624, 1365, 1757, 1364, 1363, 1165, 382, + /* 760 */ 1362, 425, 377, 376, 375, 374, 373, 370, 369, 368, + /* 770 */ 367, 366, 362, 361, 360, 359, 358, 357, 356, 355, + /* 780 */ 529, 131, 529, 529, 1737, 1369, 258, 25, 1624, 1624, + /* 790 */ 467, 1737, 527, 1361, 243, 319, 1360, 500, 256, 53, + /* 800 */ 1613, 1167, 52, 1359, 1624, 505, 1624, 1624, 1733, 1487, + /* 810 */ 1624, 1487, 1487, 185, 1445, 1732, 183, 1291, 187, 162, + /* 820 */ 189, 186, 1175, 188, 1170, 1171, 46, 1217, 1218, 1220, + /* 830 */ 1221, 1222, 1223, 1224, 508, 530, 1232, 1233, 1234, 1235, + /* 840 */ 1236, 1237, 206, 1624, 55, 327, 1624, 36, 34, 32, + /* 850 */ 31, 30, 191, 1624, 1400, 190, 1166, 1398, 209, 11, + /* 860 */ 10, 1351, 1352, 1219, 46, 37, 37, 1314, 37, 232, + /* 870 */ 557, 1219, 456, 118, 119, 78, 444, 120, 1174, 447, + /* 880 */ 81, 1164, 537, 225, 972, 74, 119, 476, 1178, 120, + /* 890 */ 495, 216, 1671, 279, 121, 1389, 1654, 1526, 1172, 211, + /* 900 */ 424, 973, 119, 1753, 490, 251, 1263, 1225, 322, 1123, + /* 910 */ 234, 59, 58, 351, 521, 240, 156, 1204, 1031, 1188, + /* 920 */ 224, 345, 3, 1059, 1670, 1255, 1247, 1063, 498, 227, + /* 930 */ 1069, 229, 492, 275, 5, 1067, 335, 611, 333, 329, + /* 940 */ 153, 323, 512, 122, 1177, 326, 1624, 1165, 325, 284, + /* 950 */ 1000, 285, 312, 311, 248, 1134, 158, 365, 1566, 372, + /* 960 */ 380, 379, 1180, 1683, 381, 385, 83, 1655, 515, 1657, + /* 970 */ 1658, 511, 150, 532, 1256, 1654, 1723, 1194, 1197, 386, + /* 980 */ 298, 1719, 143, 394, 501, 397, 398, 1173, 1196, 399, + /* 990 */ 1167, 165, 167, 1198, 170, 1261, 172, 403, 400, 402, + /* 1000 */ 1751, 1195, 175, 1670, 1172, 62, 404, 407, 178, 426, + /* 1010 */ 1172, 513, 86, 1170, 1171, 428, 293, 1477, 1607, 182, + /* 1020 */ 1473, 512, 184, 124, 249, 1624, 125, 1475, 1654, 1471, + /* 1030 */ 126, 127, 27, 303, 1250, 1251, 1252, 1253, 1254, 1258, + /* 1040 */ 1259, 1260, 1683, 533, 195, 83, 1655, 515, 1657, 1658, + /* 1050 */ 511, 457, 532, 1176, 198, 1723, 1670, 461, 464, 298, + /* 1060 */ 1719, 1798, 201, 466, 513, 468, 458, 469, 477, 204, + /* 1070 */ 1780, 1193, 1764, 207, 512, 519, 1763, 474, 1624, 1654, + /* 1080 */ 297, 1754, 210, 480, 215, 6, 486, 1744, 473, 217, + /* 1090 */ 1654, 1288, 113, 1192, 40, 1683, 1181, 502, 83, 1655, + /* 1100 */ 515, 1657, 1658, 511, 1738, 532, 218, 1670, 1723, 137, + /* 1110 */ 499, 299, 298, 1719, 1798, 513, 18, 517, 1670, 1184, + /* 1120 */ 518, 1704, 1575, 1741, 1781, 512, 513, 223, 226, 1624, + /* 1130 */ 530, 1232, 1233, 228, 493, 1574, 512, 1801, 1654, 522, + /* 1140 */ 1624, 307, 236, 496, 523, 493, 1683, 238, 503, 263, + /* 1150 */ 1655, 515, 1657, 1658, 511, 524, 532, 1683, 250, 71, + /* 1160 */ 263, 1655, 515, 1657, 1658, 511, 1670, 532, 1488, 73, + /* 1170 */ 1460, 535, 253, 245, 513, 1785, 610, 47, 1654, 255, + /* 1180 */ 136, 264, 274, 265, 512, 1618, 1785, 149, 1624, 257, + /* 1190 */ 1617, 1782, 57, 1616, 324, 1159, 1160, 154, 147, 328, + /* 1200 */ 1612, 330, 1782, 331, 332, 1683, 1670, 1611, 84, 1655, + /* 1210 */ 515, 1657, 1658, 511, 513, 532, 334, 1610, 1723, 336, + /* 1220 */ 180, 1609, 1722, 1719, 512, 338, 1608, 340, 1624, 343, + /* 1230 */ 344, 1137, 141, 1593, 155, 1136, 1587, 1654, 423, 419, + /* 1240 */ 415, 411, 179, 1586, 349, 1683, 350, 1585, 84, 1655, + /* 1250 */ 515, 1657, 1658, 511, 1584, 532, 1106, 1559, 1723, 117, + /* 1260 */ 1541, 1540, 504, 1719, 1558, 1670, 1557, 64, 1556, 1555, + /* 1270 */ 177, 1554, 1553, 510, 1552, 1551, 1550, 1549, 1548, 1547, + /* 1280 */ 1546, 1545, 1544, 512, 1543, 1542, 1539, 1624, 1538, 1654, + /* 1290 */ 1537, 1536, 1108, 1535, 1534, 1533, 1532, 1417, 1654, 1385, + /* 1300 */ 108, 938, 163, 937, 1683, 1384, 140, 272, 1655, 515, + /* 1310 */ 1657, 1658, 511, 509, 532, 506, 1695, 1670, 1601, 1595, + /* 1320 */ 1583, 164, 171, 1582, 1572, 513, 1670, 176, 1466, 168, + /* 1330 */ 1416, 173, 966, 401, 513, 512, 390, 109, 169, 1624, + /* 1340 */ 174, 1414, 392, 410, 512, 1412, 414, 1410, 1624, 408, + /* 1350 */ 1408, 166, 409, 418, 1654, 412, 1683, 413, 417, 134, + /* 1360 */ 1655, 515, 1657, 1658, 511, 1683, 532, 421, 84, 1655, + /* 1370 */ 515, 1657, 1658, 511, 422, 532, 416, 1397, 1723, 614, + /* 1380 */ 420, 1396, 1670, 1720, 1383, 1468, 1073, 1072, 1467, 999, + /* 1390 */ 513, 998, 578, 247, 45, 997, 996, 580, 181, 993, + /* 1400 */ 512, 992, 494, 1799, 1624, 107, 991, 472, 1654, 1406, + /* 1410 */ 1401, 603, 599, 595, 591, 246, 290, 1654, 291, 445, + /* 1420 */ 1399, 1683, 292, 448, 273, 1655, 515, 1657, 1658, 511, + /* 1430 */ 1382, 532, 1381, 450, 452, 85, 1670, 1600, 1144, 1594, + /* 1440 */ 80, 459, 1581, 241, 513, 1670, 1580, 1579, 1571, 67, + /* 1450 */ 203, 15, 208, 513, 512, 4, 1313, 37, 1624, 49, + /* 1460 */ 460, 205, 48, 512, 43, 1306, 213, 1624, 199, 16, + /* 1470 */ 1654, 135, 214, 212, 22, 1683, 525, 1644, 268, 1655, + /* 1480 */ 515, 1657, 1658, 511, 1683, 532, 128, 134, 1655, 515, + /* 1490 */ 1657, 1658, 511, 23, 532, 220, 42, 138, 1670, 68, + /* 1500 */ 1654, 470, 17, 1285, 202, 1337, 513, 1336, 1284, 300, + /* 1510 */ 1341, 1342, 1340, 1331, 301, 485, 512, 10, 19, 1248, + /* 1520 */ 1624, 29, 1142, 302, 196, 1227, 1226, 139, 1670, 151, + /* 1530 */ 12, 1800, 1212, 1570, 514, 20, 510, 1683, 41, 231, + /* 1540 */ 273, 1655, 515, 1657, 1658, 511, 512, 532, 13, 21, + /* 1550 */ 1624, 1311, 1654, 233, 235, 69, 237, 70, 1643, 1686, + /* 1560 */ 520, 74, 239, 1229, 242, 531, 1182, 1683, 1654, 44, + /* 1570 */ 272, 1655, 515, 1657, 1658, 511, 1060, 532, 534, 1696, + /* 1580 */ 1670, 536, 314, 538, 1052, 1057, 540, 541, 513, 1054, + /* 1590 */ 543, 1048, 544, 546, 547, 549, 1670, 550, 512, 1046, + /* 1600 */ 1051, 1050, 1624, 1049, 513, 304, 1037, 75, 1654, 76, + /* 1610 */ 1068, 556, 77, 1066, 512, 1065, 964, 564, 1624, 1683, + /* 1620 */ 1654, 306, 273, 1655, 515, 1657, 1658, 511, 988, 532, + /* 1630 */ 1006, 567, 244, 986, 985, 1683, 1670, 984, 273, 1655, + /* 1640 */ 515, 1657, 1658, 511, 513, 532, 983, 982, 1670, 981, + /* 1650 */ 980, 979, 1003, 1001, 512, 976, 513, 975, 1624, 974, + /* 1660 */ 971, 970, 969, 1413, 588, 589, 512, 590, 1654, 592, + /* 1670 */ 1624, 1411, 594, 593, 1409, 1683, 596, 597, 259, 1655, + /* 1680 */ 515, 1657, 1658, 511, 598, 532, 1407, 1683, 600, 601, + /* 1690 */ 267, 1655, 515, 1657, 1658, 511, 1670, 532, 602, 1395, + /* 1700 */ 1654, 604, 605, 1394, 513, 1380, 608, 1168, 609, 613, + /* 1710 */ 254, 612, 1355, 1355, 512, 1355, 1355, 1355, 1624, 1355, + /* 1720 */ 1355, 1355, 1355, 1355, 1355, 1355, 1355, 1355, 1670, 1355, + /* 1730 */ 1654, 1355, 1355, 1355, 1355, 1683, 513, 1355, 269, 1655, + /* 1740 */ 515, 1657, 1658, 511, 1355, 532, 512, 1355, 1355, 1355, + /* 1750 */ 1624, 1355, 1355, 1355, 1355, 1355, 1355, 1355, 1670, 1355, + /* 1760 */ 1355, 1355, 1654, 1355, 1355, 1355, 513, 1683, 1355, 1355, + /* 1770 */ 260, 1655, 515, 1657, 1658, 511, 512, 532, 1654, 1355, + /* 1780 */ 1624, 1355, 1355, 1355, 1355, 1355, 1355, 1355, 1355, 1355, + /* 1790 */ 1670, 1355, 1355, 1355, 1654, 1355, 1355, 1683, 513, 1355, + /* 1800 */ 270, 1655, 515, 1657, 1658, 511, 1670, 532, 512, 1355, + /* 1810 */ 1654, 1355, 1624, 1355, 513, 1355, 1355, 1355, 1355, 1355, + /* 1820 */ 1355, 1355, 1670, 1355, 512, 1355, 1355, 1355, 1624, 1683, + /* 1830 */ 513, 1355, 261, 1655, 515, 1657, 1658, 511, 1670, 532, + /* 1840 */ 512, 1355, 1355, 1355, 1624, 1683, 513, 1355, 271, 1655, + /* 1850 */ 515, 1657, 1658, 511, 1355, 532, 512, 1355, 1355, 1355, + /* 1860 */ 1624, 1683, 1654, 1355, 262, 1655, 515, 1657, 1658, 511, + /* 1870 */ 1355, 532, 1355, 1355, 1355, 1355, 1355, 1683, 1654, 1355, + /* 1880 */ 1666, 1655, 515, 1657, 1658, 511, 1355, 532, 1355, 1355, + /* 1890 */ 1670, 1355, 1355, 1355, 1355, 1355, 1355, 1355, 513, 1355, + /* 1900 */ 1355, 1355, 1355, 1355, 1355, 1355, 1670, 1355, 512, 1355, + /* 1910 */ 1654, 1355, 1624, 1355, 513, 1355, 1355, 1355, 1355, 1355, + /* 1920 */ 1355, 1355, 1355, 1355, 512, 1355, 1355, 1355, 1624, 1683, + /* 1930 */ 1355, 1355, 1665, 1655, 515, 1657, 1658, 511, 1670, 532, + /* 1940 */ 1355, 1355, 1654, 1355, 1355, 1683, 513, 1355, 1664, 1655, + /* 1950 */ 515, 1657, 1658, 511, 1355, 532, 512, 1355, 1355, 1355, + /* 1960 */ 1624, 1355, 1355, 1355, 1355, 1355, 1355, 1355, 1355, 1355, + /* 1970 */ 1670, 1355, 1654, 1355, 1355, 1355, 1355, 1683, 513, 1355, + /* 1980 */ 282, 1655, 515, 1657, 1658, 511, 1355, 532, 512, 1355, + /* 1990 */ 1355, 1355, 1624, 1355, 1355, 1355, 1355, 1355, 1355, 1355, + /* 2000 */ 1670, 1355, 1355, 1355, 1654, 1355, 1355, 1355, 513, 1683, + /* 2010 */ 1355, 1355, 281, 1655, 515, 1657, 1658, 511, 512, 532, + /* 2020 */ 1654, 1355, 1624, 1355, 1355, 1355, 1355, 1355, 1355, 1355, + /* 2030 */ 1355, 1355, 1670, 1355, 1355, 1355, 1355, 1355, 1355, 1683, + /* 2040 */ 513, 1355, 283, 1655, 515, 1657, 1658, 511, 1670, 532, + /* 2050 */ 512, 489, 1355, 1355, 1624, 1355, 513, 1355, 1355, 489, + /* 2060 */ 1355, 1355, 1355, 1355, 1355, 1355, 512, 1355, 1355, 1355, + /* 2070 */ 1624, 1683, 1355, 1355, 280, 1655, 515, 1657, 1658, 511, + /* 2080 */ 114, 532, 1355, 1355, 1355, 1355, 1355, 1683, 114, 1355, + /* 2090 */ 266, 1655, 515, 1657, 1658, 511, 1355, 532, 1355, 493, + /* 2100 */ 1355, 1355, 1355, 1355, 1355, 1355, 1355, 493, 1355, 1355, + /* 2110 */ 1355, 1355, 1355, 1355, 1355, 1355, 1355, 1355, 112, 1355, + /* 2120 */ 1355, 1355, 1355, 1355, 1355, 1355, 112, 1355, 1355, 1355, + /* 2130 */ 1355, 1355, 1355, 219, 1730, 488, 1355, 487, 1355, 1355, + /* 2140 */ 1785, 219, 1730, 488, 1355, 487, 1355, 1355, 1785, 1355, + /* 2150 */ 1355, 1355, 149, 1355, 1355, 1355, 1782, 1355, 1355, 1355, + /* 2160 */ 147, 1355, 1355, 1355, 1782, }; static const YYCODETYPE yy_lookahead[] = { /* 0 */ 251, 248, 241, 271, 322, 323, 274, 0, 271, 249, @@ -555,59 +555,59 @@ static const YYCODETYPE yy_lookahead[] = { /* 1160 */ 311, 312, 313, 314, 315, 316, 269, 318, 277, 81, /* 1170 */ 259, 273, 248, 251, 277, 337, 243, 299, 241, 252, /* 1180 */ 303, 263, 263, 263, 287, 0, 337, 349, 291, 239, - /* 1190 */ 0, 353, 40, 0, 72, 0, 47, 174, 349, 174, - /* 1200 */ 47, 47, 353, 47, 0, 308, 269, 47, 311, 312, - /* 1210 */ 313, 314, 315, 316, 277, 318, 47, 174, 321, 0, - /* 1220 */ 33, 47, 325, 326, 287, 0, 47, 0, 291, 47, - /* 1230 */ 0, 81, 45, 113, 160, 159, 156, 241, 51, 52, - /* 1240 */ 53, 54, 55, 0, 0, 308, 152, 151, 311, 312, - /* 1250 */ 313, 314, 315, 316, 0, 318, 0, 44, 321, 22, - /* 1260 */ 40, 0, 325, 326, 0, 269, 0, 80, 0, 0, + /* 1190 */ 0, 353, 40, 0, 72, 47, 47, 47, 349, 174, + /* 1200 */ 0, 47, 353, 47, 174, 308, 269, 0, 311, 312, + /* 1210 */ 313, 314, 315, 316, 277, 318, 174, 0, 321, 47, + /* 1220 */ 33, 0, 325, 326, 287, 47, 0, 47, 291, 160, + /* 1230 */ 159, 113, 45, 0, 81, 156, 0, 241, 51, 52, + /* 1240 */ 53, 54, 55, 0, 152, 308, 151, 0, 311, 312, + /* 1250 */ 313, 314, 315, 316, 0, 318, 44, 0, 321, 40, + /* 1260 */ 0, 0, 325, 326, 0, 269, 0, 80, 0, 0, /* 1270 */ 83, 0, 0, 277, 0, 0, 0, 0, 0, 0, /* 1280 */ 0, 0, 0, 287, 0, 0, 0, 291, 0, 241, - /* 1290 */ 0, 0, 0, 0, 0, 0, 0, 0, 241, 0, - /* 1300 */ 0, 41, 40, 14, 308, 0, 0, 311, 312, 313, - /* 1310 */ 314, 315, 316, 317, 318, 319, 320, 269, 14, 0, - /* 1320 */ 37, 0, 147, 0, 37, 277, 269, 140, 0, 142, - /* 1330 */ 0, 144, 59, 146, 277, 287, 0, 0, 0, 291, - /* 1340 */ 38, 0, 44, 0, 287, 44, 37, 37, 291, 37, - /* 1350 */ 47, 164, 45, 37, 241, 47, 308, 0, 45, 311, - /* 1360 */ 312, 313, 314, 315, 316, 308, 318, 47, 311, 312, - /* 1370 */ 313, 314, 315, 316, 45, 318, 37, 45, 321, 19, - /* 1380 */ 47, 37, 269, 326, 0, 0, 0, 47, 22, 0, - /* 1390 */ 277, 41, 47, 33, 47, 47, 47, 41, 47, 47, - /* 1400 */ 287, 47, 354, 355, 291, 45, 88, 294, 241, 0, - /* 1410 */ 22, 51, 52, 53, 54, 55, 90, 241, 0, 22, - /* 1420 */ 48, 308, 0, 22, 311, 312, 313, 314, 315, 316, - /* 1430 */ 47, 318, 0, 22, 0, 22, 269, 20, 0, 47, - /* 1440 */ 80, 0, 22, 83, 277, 269, 0, 0, 0, 0, - /* 1450 */ 81, 37, 213, 277, 287, 82, 41, 41, 291, 145, - /* 1460 */ 145, 81, 145, 287, 142, 41, 41, 291, 213, 82, - /* 1470 */ 241, 140, 81, 44, 81, 308, 116, 82, 311, 312, + /* 1290 */ 0, 0, 22, 0, 0, 0, 0, 0, 241, 0, + /* 1300 */ 37, 14, 40, 14, 308, 0, 41, 311, 312, 313, + /* 1310 */ 314, 315, 316, 317, 318, 319, 320, 269, 0, 0, + /* 1320 */ 0, 38, 147, 0, 0, 277, 269, 140, 0, 142, + /* 1330 */ 0, 144, 59, 146, 277, 287, 44, 37, 37, 291, + /* 1340 */ 37, 0, 44, 37, 287, 0, 37, 0, 291, 47, + /* 1350 */ 0, 164, 45, 37, 241, 47, 308, 45, 45, 311, + /* 1360 */ 312, 313, 314, 315, 316, 308, 318, 45, 311, 312, + /* 1370 */ 313, 314, 315, 316, 37, 318, 47, 0, 321, 19, + /* 1380 */ 47, 0, 269, 326, 0, 0, 47, 22, 0, 47, + /* 1390 */ 277, 47, 41, 33, 90, 47, 47, 41, 88, 47, + /* 1400 */ 287, 47, 354, 355, 291, 45, 47, 294, 241, 0, + /* 1410 */ 0, 51, 52, 53, 54, 55, 22, 241, 22, 48, + /* 1420 */ 0, 308, 22, 47, 311, 312, 313, 314, 315, 316, + /* 1430 */ 0, 318, 0, 22, 22, 20, 269, 0, 47, 0, + /* 1440 */ 80, 22, 0, 83, 277, 269, 0, 0, 0, 81, + /* 1450 */ 37, 213, 82, 277, 287, 41, 82, 41, 291, 145, + /* 1460 */ 145, 140, 145, 287, 41, 82, 41, 291, 142, 213, + /* 1470 */ 241, 81, 44, 81, 81, 308, 116, 44, 311, 312, /* 1480 */ 313, 314, 315, 316, 308, 318, 161, 311, 312, 313, - /* 1490 */ 314, 315, 316, 81, 318, 44, 41, 44, 269, 41, - /* 1500 */ 241, 141, 41, 82, 144, 82, 277, 44, 82, 47, - /* 1510 */ 82, 47, 47, 47, 47, 348, 287, 47, 2, 41, - /* 1520 */ 291, 82, 162, 294, 164, 179, 81, 44, 269, 44, - /* 1530 */ 22, 355, 82, 81, 181, 81, 277, 308, 81, 143, - /* 1540 */ 311, 312, 313, 314, 315, 316, 287, 318, 213, 207, - /* 1550 */ 291, 82, 241, 82, 81, 81, 81, 0, 81, 37, - /* 1560 */ 81, 44, 44, 91, 82, 81, 22, 308, 241, 81, - /* 1570 */ 311, 312, 313, 314, 315, 316, 140, 318, 92, 320, - /* 1580 */ 269, 82, 47, 47, 81, 47, 82, 81, 277, 82, - /* 1590 */ 47, 81, 47, 82, 81, 47, 269, 82, 287, 105, - /* 1600 */ 81, 105, 291, 105, 277, 294, 105, 22, 241, 81, - /* 1610 */ 81, 47, 81, 93, 287, 113, 22, 59, 291, 308, - /* 1620 */ 241, 294, 311, 312, 313, 314, 315, 316, 58, 318, - /* 1630 */ 47, 64, 41, 79, 47, 308, 269, 47, 311, 312, - /* 1640 */ 313, 314, 315, 316, 277, 318, 47, 47, 269, 47, - /* 1650 */ 22, 47, 47, 64, 287, 47, 277, 47, 291, 47, - /* 1660 */ 47, 47, 47, 47, 0, 47, 287, 45, 241, 37, - /* 1670 */ 291, 0, 47, 37, 45, 308, 0, 47, 311, 312, - /* 1680 */ 313, 314, 315, 316, 45, 318, 37, 308, 0, 47, - /* 1690 */ 311, 312, 313, 314, 315, 316, 269, 318, 45, 37, - /* 1700 */ 241, 47, 46, 0, 277, 0, 0, 22, 21, 358, - /* 1710 */ 358, 22, 22, 21, 287, 20, 358, 358, 291, 358, + /* 1490 */ 314, 315, 316, 41, 318, 44, 41, 44, 269, 81, + /* 1500 */ 241, 141, 41, 82, 144, 47, 277, 47, 82, 47, + /* 1510 */ 47, 82, 47, 82, 47, 348, 287, 2, 41, 179, + /* 1520 */ 291, 81, 162, 294, 164, 82, 82, 44, 269, 44, + /* 1530 */ 81, 355, 22, 0, 181, 81, 277, 308, 207, 82, + /* 1540 */ 311, 312, 313, 314, 315, 316, 287, 318, 213, 81, + /* 1550 */ 291, 82, 241, 81, 81, 81, 37, 81, 44, 81, + /* 1560 */ 143, 91, 140, 82, 44, 81, 22, 308, 241, 81, + /* 1570 */ 311, 312, 313, 314, 315, 316, 82, 318, 92, 320, + /* 1580 */ 269, 47, 47, 81, 105, 82, 47, 81, 277, 82, + /* 1590 */ 47, 82, 81, 47, 81, 47, 269, 81, 287, 82, + /* 1600 */ 105, 105, 291, 105, 277, 294, 22, 81, 241, 81, + /* 1610 */ 47, 93, 81, 113, 287, 22, 59, 58, 291, 308, + /* 1620 */ 241, 294, 311, 312, 313, 314, 315, 316, 47, 318, + /* 1630 */ 64, 79, 41, 47, 47, 308, 269, 47, 311, 312, + /* 1640 */ 313, 314, 315, 316, 277, 318, 47, 47, 269, 22, + /* 1650 */ 47, 47, 64, 47, 287, 47, 277, 47, 291, 47, + /* 1660 */ 47, 47, 47, 0, 47, 45, 287, 37, 241, 47, + /* 1670 */ 291, 0, 37, 45, 0, 308, 47, 45, 311, 312, + /* 1680 */ 313, 314, 315, 316, 37, 318, 0, 308, 47, 45, + /* 1690 */ 311, 312, 313, 314, 315, 316, 269, 318, 37, 0, + /* 1700 */ 241, 47, 46, 0, 277, 0, 22, 22, 21, 20, + /* 1710 */ 22, 21, 358, 358, 287, 358, 358, 358, 291, 358, /* 1720 */ 358, 358, 358, 358, 358, 358, 358, 358, 269, 358, /* 1730 */ 241, 358, 358, 358, 358, 308, 277, 358, 311, 312, /* 1740 */ 313, 314, 315, 316, 358, 318, 287, 358, 358, 358, @@ -654,9 +654,9 @@ static const YYCODETYPE yy_lookahead[] = { /* 2150 */ 358, 358, 349, 358, 358, 358, 353, 358, 358, 358, /* 2160 */ 349, 358, 358, 358, 353, }; -#define YY_SHIFT_COUNT (615) +#define YY_SHIFT_COUNT (614) #define YY_SHIFT_MIN (0) -#define YY_SHIFT_MAX (1706) +#define YY_SHIFT_MAX (1705) static const unsigned short int yy_shift_ofst[] = { /* 0 */ 763, 0, 0, 48, 233, 233, 233, 233, 254, 254, /* 10 */ 233, 233, 439, 460, 645, 460, 460, 460, 460, 460, @@ -690,36 +690,36 @@ static const unsigned short int yy_shift_ofst[] = { /* 290 */ 470, 854, 857, 506, 547, 708, 785, 817, 858, 667, /* 300 */ 702, 756, 824, 747, 825, 457, 827, 828, 832, 833, /* 310 */ 836, 775, 831, 823, 841, 845, 848, 853, 861, 794, - /* 320 */ 837, 1185, 1190, 1152, 1193, 1122, 1195, 1149, 1023, 1153, - /* 330 */ 1154, 1156, 1025, 1204, 1160, 1169, 1043, 1219, 1174, 1225, - /* 340 */ 1179, 1227, 1182, 1230, 1150, 1074, 1076, 1120, 1080, 1243, - /* 350 */ 1244, 1094, 1096, 1254, 1256, 1213, 1264, 1266, 1268, 1269, + /* 320 */ 837, 1185, 1190, 1152, 1193, 1122, 1148, 1149, 1150, 1025, + /* 330 */ 1200, 1154, 1156, 1030, 1207, 1042, 1217, 1172, 1221, 1178, + /* 340 */ 1226, 1180, 1233, 1153, 1069, 1071, 1118, 1079, 1236, 1243, + /* 350 */ 1092, 1095, 1247, 1254, 1212, 1257, 1264, 1266, 1268, 1269, /* 360 */ 1271, 1272, 1274, 1275, 1276, 1277, 1278, 1279, 1280, 1281, - /* 370 */ 1282, 1284, 1285, 1286, 1220, 1261, 1288, 1290, 1291, 1292, - /* 380 */ 1293, 1237, 1294, 1295, 1296, 1297, 1299, 1300, 1262, 1283, - /* 390 */ 1260, 1289, 1298, 1304, 1301, 1305, 1302, 1287, 1306, 1319, - /* 400 */ 1321, 1309, 1175, 1323, 1328, 1310, 1330, 1273, 1336, 1337, - /* 410 */ 1303, 1307, 1312, 1338, 1308, 1313, 1316, 1341, 1320, 1329, - /* 420 */ 1339, 1343, 1333, 1332, 1344, 1357, 1384, 1385, 1386, 1326, - /* 430 */ 1318, 1340, 1366, 1389, 1345, 1347, 1348, 1349, 1350, 1356, - /* 440 */ 1351, 1352, 1354, 1409, 1388, 1418, 1397, 1372, 1422, 1401, - /* 450 */ 1383, 1432, 1411, 1434, 1413, 1417, 1438, 1314, 1392, 1441, - /* 460 */ 1325, 1420, 1315, 1322, 1446, 1447, 1448, 1317, 1449, 1369, - /* 470 */ 1414, 1331, 1415, 1416, 1239, 1373, 1424, 1387, 1380, 1391, - /* 480 */ 1393, 1395, 1425, 1429, 1451, 1412, 1455, 1255, 1421, 1423, - /* 490 */ 1453, 1342, 1458, 1463, 1426, 1461, 1335, 1428, 1462, 1464, - /* 500 */ 1465, 1466, 1467, 1470, 1428, 1516, 1346, 1478, 1439, 1445, - /* 510 */ 1450, 1483, 1452, 1454, 1485, 1508, 1353, 1457, 1469, 1471, - /* 520 */ 1473, 1474, 1396, 1475, 1557, 1522, 1436, 1477, 1472, 1517, - /* 530 */ 1518, 1479, 1482, 1484, 1544, 1488, 1486, 1499, 1535, 1536, - /* 540 */ 1503, 1504, 1538, 1506, 1507, 1543, 1510, 1511, 1545, 1513, - /* 550 */ 1515, 1548, 1519, 1494, 1496, 1498, 1501, 1585, 1520, 1528, - /* 560 */ 1529, 1564, 1531, 1502, 1594, 1558, 1570, 1583, 1567, 1554, - /* 570 */ 1591, 1587, 1590, 1599, 1600, 1602, 1628, 1604, 1605, 1589, - /* 580 */ 1350, 1608, 1356, 1610, 1612, 1613, 1614, 1615, 1616, 1664, - /* 590 */ 1618, 1622, 1632, 1671, 1625, 1629, 1636, 1676, 1630, 1639, - /* 600 */ 1649, 1688, 1642, 1653, 1662, 1705, 1654, 1656, 1703, 1706, - /* 610 */ 1685, 1687, 1689, 1690, 1692, 1695, + /* 370 */ 1282, 1284, 1285, 1219, 1260, 1261, 1286, 1288, 1290, 1291, + /* 380 */ 1270, 1293, 1294, 1295, 1296, 1297, 1299, 1262, 1263, 1265, + /* 390 */ 1287, 1292, 1289, 1298, 1305, 1283, 1300, 1318, 1319, 1320, + /* 400 */ 1301, 1175, 1323, 1324, 1303, 1328, 1273, 1330, 1341, 1302, + /* 410 */ 1307, 1306, 1345, 1308, 1312, 1309, 1347, 1329, 1313, 1316, + /* 420 */ 1350, 1333, 1322, 1337, 1377, 1381, 1384, 1385, 1304, 1310, + /* 430 */ 1339, 1365, 1388, 1342, 1344, 1348, 1349, 1351, 1356, 1352, + /* 440 */ 1354, 1359, 1409, 1394, 1410, 1396, 1371, 1420, 1400, 1376, + /* 450 */ 1430, 1411, 1432, 1412, 1415, 1437, 1314, 1391, 1439, 1325, + /* 460 */ 1419, 1315, 1326, 1442, 1446, 1447, 1317, 1448, 1368, 1413, + /* 470 */ 1321, 1414, 1416, 1238, 1370, 1423, 1374, 1390, 1392, 1393, + /* 480 */ 1383, 1425, 1428, 1433, 1418, 1452, 1256, 1421, 1426, 1451, + /* 490 */ 1331, 1455, 1453, 1429, 1461, 1335, 1431, 1458, 1460, 1462, + /* 500 */ 1463, 1465, 1467, 1431, 1515, 1340, 1477, 1443, 1440, 1444, + /* 510 */ 1483, 1449, 1454, 1485, 1510, 1353, 1468, 1457, 1469, 1472, + /* 520 */ 1473, 1417, 1474, 1533, 1519, 1422, 1476, 1470, 1514, 1520, + /* 530 */ 1478, 1481, 1484, 1544, 1488, 1486, 1494, 1534, 1535, 1502, + /* 540 */ 1503, 1539, 1506, 1507, 1543, 1511, 1509, 1546, 1513, 1517, + /* 550 */ 1548, 1516, 1479, 1495, 1496, 1498, 1584, 1518, 1526, 1528, + /* 560 */ 1563, 1531, 1500, 1593, 1557, 1559, 1581, 1566, 1552, 1591, + /* 570 */ 1586, 1587, 1590, 1599, 1600, 1627, 1603, 1604, 1588, 1351, + /* 580 */ 1606, 1356, 1608, 1610, 1612, 1613, 1614, 1615, 1663, 1617, + /* 590 */ 1620, 1630, 1671, 1622, 1628, 1635, 1674, 1629, 1632, 1647, + /* 600 */ 1686, 1641, 1644, 1661, 1699, 1654, 1656, 1703, 1705, 1684, + /* 610 */ 1687, 1685, 1688, 1690, 1689, }; #define YY_REDUCE_COUNT (254) #define YY_REDUCE_MIN (-318) @@ -753,68 +753,68 @@ static const short yy_reduce_ofst[] = { /* 250 */ 918, 919, 920, 927, 950, }; static const YYACTIONTYPE yy_default[] = { - /* 0 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 10 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 20 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 30 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 40 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 50 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 60 */ 1354, 1354, 1354, 1354, 1423, 1354, 1354, 1354, 1354, 1354, - /* 70 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 80 */ 1421, 1561, 1354, 1726, 1354, 1354, 1354, 1354, 1354, 1354, - /* 90 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 100 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 110 */ 1423, 1354, 1737, 1737, 1737, 1421, 1354, 1354, 1354, 1354, - /* 120 */ 1354, 1354, 1517, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 130 */ 1354, 1597, 1354, 1354, 1803, 1354, 1603, 1761, 1354, 1354, - /* 140 */ 1354, 1354, 1470, 1753, 1729, 1743, 1730, 1788, 1788, 1788, - /* 150 */ 1746, 1354, 1757, 1354, 1354, 1354, 1589, 1354, 1354, 1566, - /* 160 */ 1563, 1563, 1354, 1354, 1354, 1354, 1423, 1354, 1423, 1354, - /* 170 */ 1354, 1423, 1354, 1423, 1354, 1354, 1423, 1423, 1354, 1423, - /* 180 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 190 */ 1354, 1354, 1354, 1354, 1354, 1354, 1421, 1599, 1354, 1421, - /* 200 */ 1354, 1354, 1421, 1354, 1354, 1421, 1354, 1768, 1766, 1354, - /* 210 */ 1768, 1766, 1354, 1354, 1354, 1780, 1776, 1759, 1757, 1743, - /* 220 */ 1354, 1354, 1354, 1794, 1790, 1806, 1794, 1790, 1794, 1790, - /* 230 */ 1354, 1766, 1354, 1354, 1766, 1354, 1574, 1354, 1354, 1421, - /* 240 */ 1354, 1421, 1354, 1486, 1354, 1354, 1421, 1354, 1591, 1605, - /* 250 */ 1520, 1520, 1520, 1424, 1359, 1354, 1354, 1354, 1354, 1354, - /* 260 */ 1354, 1354, 1354, 1354, 1354, 1354, 1670, 1779, 1778, 1702, - /* 270 */ 1701, 1700, 1698, 1669, 1482, 1354, 1354, 1354, 1354, 1354, - /* 280 */ 1663, 1664, 1662, 1661, 1354, 1354, 1354, 1354, 1354, 1354, - /* 290 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1727, 1354, - /* 300 */ 1791, 1795, 1354, 1354, 1354, 1646, 1354, 1354, 1354, 1354, - /* 310 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 320 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 330 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 340 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 350 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 360 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 370 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 380 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 390 */ 1388, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 400 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 410 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 420 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 430 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1451, 1450, - /* 440 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 450 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 460 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 470 */ 1354, 1354, 1750, 1760, 1354, 1354, 1354, 1354, 1354, 1354, - /* 480 */ 1354, 1354, 1354, 1354, 1646, 1354, 1777, 1354, 1736, 1732, - /* 490 */ 1354, 1354, 1728, 1354, 1354, 1789, 1354, 1354, 1354, 1354, - /* 500 */ 1354, 1354, 1354, 1354, 1354, 1722, 1354, 1695, 1354, 1354, - /* 510 */ 1354, 1354, 1354, 1354, 1354, 1354, 1657, 1354, 1354, 1354, - /* 520 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1645, - /* 530 */ 1354, 1686, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 540 */ 1514, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 550 */ 1354, 1354, 1354, 1499, 1497, 1496, 1495, 1354, 1492, 1354, - /* 560 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 570 */ 1443, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 580 */ 1434, 1354, 1433, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 590 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 600 */ 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, 1354, - /* 610 */ 1354, 1354, 1354, 1354, 1354, 1354, + /* 0 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 10 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 20 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 30 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 40 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 50 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 60 */ 1353, 1353, 1353, 1353, 1422, 1353, 1353, 1353, 1353, 1353, + /* 70 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 80 */ 1420, 1560, 1353, 1725, 1353, 1353, 1353, 1353, 1353, 1353, + /* 90 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 100 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 110 */ 1422, 1353, 1736, 1736, 1736, 1420, 1353, 1353, 1353, 1353, + /* 120 */ 1353, 1353, 1516, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 130 */ 1353, 1596, 1353, 1353, 1802, 1353, 1602, 1760, 1353, 1353, + /* 140 */ 1353, 1353, 1469, 1752, 1728, 1742, 1729, 1787, 1787, 1787, + /* 150 */ 1745, 1353, 1756, 1353, 1353, 1353, 1588, 1353, 1353, 1565, + /* 160 */ 1562, 1562, 1353, 1353, 1353, 1353, 1422, 1353, 1422, 1353, + /* 170 */ 1353, 1422, 1353, 1422, 1353, 1353, 1422, 1422, 1353, 1422, + /* 180 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 190 */ 1353, 1353, 1353, 1353, 1353, 1353, 1420, 1598, 1353, 1420, + /* 200 */ 1353, 1353, 1420, 1353, 1353, 1420, 1353, 1767, 1765, 1353, + /* 210 */ 1767, 1765, 1353, 1353, 1353, 1779, 1775, 1758, 1756, 1742, + /* 220 */ 1353, 1353, 1353, 1793, 1789, 1805, 1793, 1789, 1793, 1789, + /* 230 */ 1353, 1765, 1353, 1353, 1765, 1353, 1573, 1353, 1353, 1420, + /* 240 */ 1353, 1420, 1353, 1485, 1353, 1353, 1420, 1353, 1590, 1604, + /* 250 */ 1519, 1519, 1519, 1423, 1358, 1353, 1353, 1353, 1353, 1353, + /* 260 */ 1353, 1353, 1353, 1353, 1353, 1353, 1669, 1778, 1777, 1701, + /* 270 */ 1700, 1699, 1697, 1668, 1481, 1353, 1353, 1353, 1353, 1353, + /* 280 */ 1662, 1663, 1661, 1660, 1353, 1353, 1353, 1353, 1353, 1353, + /* 290 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1726, 1353, + /* 300 */ 1790, 1794, 1353, 1353, 1353, 1645, 1353, 1353, 1353, 1353, + /* 310 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 320 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 330 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 340 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 350 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 360 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 370 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 380 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1387, + /* 390 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 400 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 410 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 420 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 430 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1450, 1449, 1353, + /* 440 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 450 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 460 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 470 */ 1353, 1749, 1759, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 480 */ 1353, 1353, 1353, 1645, 1353, 1776, 1353, 1735, 1731, 1353, + /* 490 */ 1353, 1727, 1353, 1353, 1788, 1353, 1353, 1353, 1353, 1353, + /* 500 */ 1353, 1353, 1353, 1353, 1721, 1353, 1694, 1353, 1353, 1353, + /* 510 */ 1353, 1353, 1353, 1353, 1353, 1656, 1353, 1353, 1353, 1353, + /* 520 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1644, 1353, + /* 530 */ 1685, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1513, + /* 540 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 550 */ 1353, 1353, 1498, 1496, 1495, 1494, 1353, 1491, 1353, 1353, + /* 560 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1442, + /* 570 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1433, + /* 580 */ 1353, 1432, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 590 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 600 */ 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, 1353, + /* 610 */ 1353, 1353, 1353, 1353, 1353, }; /********** End of lemon-generated parsing tables *****************************/ @@ -1007,10 +1007,10 @@ static const YYCODETYPE yyFallback[] = { 0, /* KILL => nothing */ 0, /* CONNECTION => nothing */ 0, /* TRANSACTION => nothing */ - 0, /* MERGE => nothing */ + 0, /* BALANCE => nothing */ 0, /* VGROUP => nothing */ + 0, /* MERGE => nothing */ 0, /* REDISTRIBUTE => nothing */ - 0, /* SPLIT => nothing */ 0, /* SYNCDB => nothing */ 0, /* DELETE => nothing */ 0, /* NULL => nothing */ @@ -1332,10 +1332,10 @@ static const char *const yyTokenName[] = { /* 170 */ "KILL", /* 171 */ "CONNECTION", /* 172 */ "TRANSACTION", - /* 173 */ "MERGE", + /* 173 */ "BALANCE", /* 174 */ "VGROUP", - /* 175 */ "REDISTRIBUTE", - /* 176 */ "SPLIT", + /* 175 */ "MERGE", + /* 176 */ "REDISTRIBUTE", /* 177 */ "SYNCDB", /* 178 */ "DELETE", /* 179 */ "NULL", @@ -1779,9 +1779,9 @@ static const char *const yyRuleName[] = { /* 252 */ "cmd ::= KILL CONNECTION NK_INTEGER", /* 253 */ "cmd ::= KILL QUERY NK_INTEGER", /* 254 */ "cmd ::= KILL TRANSACTION NK_INTEGER", - /* 255 */ "cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER", - /* 256 */ "cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list", - /* 257 */ "cmd ::= SPLIT VGROUP NK_INTEGER", + /* 255 */ "cmd ::= BALANCE VGROUP", + /* 256 */ "cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER", + /* 257 */ "cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list", /* 258 */ "dnode_list ::= DNODE NK_INTEGER", /* 259 */ "dnode_list ::= dnode_list DNODE NK_INTEGER", /* 260 */ "cmd ::= SYNCDB db_name REPLICA", @@ -2823,9 +2823,9 @@ static const struct { { 238, -3 }, /* (252) cmd ::= KILL CONNECTION NK_INTEGER */ { 238, -3 }, /* (253) cmd ::= KILL QUERY NK_INTEGER */ { 238, -3 }, /* (254) cmd ::= KILL TRANSACTION NK_INTEGER */ - { 238, -4 }, /* (255) cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER */ - { 238, -4 }, /* (256) cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list */ - { 238, -3 }, /* (257) cmd ::= SPLIT VGROUP NK_INTEGER */ + { 238, -2 }, /* (255) cmd ::= BALANCE VGROUP */ + { 238, -4 }, /* (256) cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER */ + { 238, -4 }, /* (257) cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list */ { 305, -2 }, /* (258) dnode_list ::= DNODE NK_INTEGER */ { 305, -3 }, /* (259) dnode_list ::= dnode_list DNODE NK_INTEGER */ { 238, -3 }, /* (260) cmd ::= SYNCDB db_name REPLICA */ @@ -3900,15 +3900,15 @@ static YYACTIONTYPE yy_reduce( case 254: /* cmd ::= KILL TRANSACTION NK_INTEGER */ { pCxt->pRootNode = createKillStmt(pCxt, QUERY_NODE_KILL_TRANSACTION_STMT, &yymsp[0].minor.yy0); } break; - case 255: /* cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER */ + case 255: /* cmd ::= BALANCE VGROUP */ +{ pCxt->pRootNode = createBalanceVgroupStmt(pCxt); } + break; + case 256: /* cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER */ { pCxt->pRootNode = createMergeVgroupStmt(pCxt, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy0); } break; - case 256: /* cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list */ + case 257: /* cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list */ { pCxt->pRootNode = createRedistributeVgroupStmt(pCxt, &yymsp[-1].minor.yy0, yymsp[0].minor.yy60); } break; - case 257: /* cmd ::= SPLIT VGROUP NK_INTEGER */ -{ pCxt->pRootNode = createSplitVgroupStmt(pCxt, &yymsp[0].minor.yy0); } - break; case 258: /* dnode_list ::= DNODE NK_INTEGER */ { yymsp[-1].minor.yy60 = createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } break; diff --git a/source/libs/parser/test/parExplainToSyncdbTest.cpp b/source/libs/parser/test/parExplainToSyncdbTest.cpp index a040721031..4a5a92e621 100644 --- a/source/libs/parser/test/parExplainToSyncdbTest.cpp +++ b/source/libs/parser/test/parExplainToSyncdbTest.cpp @@ -19,7 +19,7 @@ using namespace std; namespace ParserTest { -class ParserExplainToSyncdbTest : public ParserTestBase {}; +class ParserExplainToSyncdbTest : public ParserDdlTest {}; TEST_F(ParserExplainToSyncdbTest, explain) { useDb("root", "test"); @@ -43,8 +43,62 @@ TEST_F(ParserExplainToSyncdbTest, grant) { // todo kill connection // todo kill query // todo kill stream -// todo merge vgroup -// todo redistribute vgroup + +TEST_F(ParserExplainToSyncdbTest, mergeVgroup) { + useDb("root", "test"); + + SMergeVgroupReq expect = {0}; + + auto setMergeVgroupReqFunc = [&](int32_t vgId1, int32_t vgId2) { + expect.vgId1 = vgId1; + expect.vgId2 = vgId2; + }; + + setCheckDdlFunc([&](const SQuery* pQuery, ParserStage stage) { + ASSERT_EQ(nodeType(pQuery->pRoot), QUERY_NODE_MERGE_VGROUP_STMT); + ASSERT_EQ(pQuery->pCmdMsg->msgType, TDMT_MND_MERGE_VGROUP); + SMergeVgroupReq req = {0}; + ASSERT_EQ(tDeserializeSMergeVgroupReq(pQuery->pCmdMsg->pMsg, pQuery->pCmdMsg->msgLen, &req), TSDB_CODE_SUCCESS); + ASSERT_EQ(req.vgId1, expect.vgId1); + ASSERT_EQ(req.vgId2, expect.vgId2); + }); + + setMergeVgroupReqFunc(1, 2); + run("MERGE VGROUP 1 2"); +} + +TEST_F(ParserExplainToSyncdbTest, redistributeVgroup) { + useDb("root", "test"); + + SRedistributeVgroupReq expect = {0}; + + auto setRedistributeVgroupReqFunc = [&](int32_t vgId, int32_t dnodeId1, int32_t dnodeId2 = -1, + int32_t dnodeId3 = -1) { + expect.vgId = vgId; + expect.dnodeId1 = dnodeId1; + expect.dnodeId2 = dnodeId2; + expect.dnodeId3 = dnodeId3; + }; + + setCheckDdlFunc([&](const SQuery* pQuery, ParserStage stage) { + ASSERT_EQ(nodeType(pQuery->pRoot), QUERY_NODE_REDISTRIBUTE_VGROUP_STMT); + ASSERT_EQ(pQuery->pCmdMsg->msgType, TDMT_MND_REDISTRIBUTE_VGROUP); + SRedistributeVgroupReq req = {0}; + ASSERT_EQ(tDeserializeSRedistributeVgroupReq(pQuery->pCmdMsg->pMsg, pQuery->pCmdMsg->msgLen, &req), + TSDB_CODE_SUCCESS); + ASSERT_EQ(req.vgId, expect.vgId); + ASSERT_EQ(req.dnodeId1, expect.dnodeId1); + ASSERT_EQ(req.dnodeId2, expect.dnodeId2); + ASSERT_EQ(req.dnodeId3, expect.dnodeId3); + }); + + setRedistributeVgroupReqFunc(3, 1); + run("REDISTRIBUTE VGROUP 3 DNODE 1"); + + setRedistributeVgroupReqFunc(5, 10, 20, 30); + run("REDISTRIBUTE VGROUP 5 DNODE 10 DNODE 20 DNODE 30"); +} + // todo reset query cache TEST_F(ParserExplainToSyncdbTest, revoke) { diff --git a/source/libs/parser/test/parInitialATest.cpp b/source/libs/parser/test/parInitialATest.cpp index 784586dfb2..6ac793cb0d 100644 --- a/source/libs/parser/test/parInitialATest.cpp +++ b/source/libs/parser/test/parInitialATest.cpp @@ -305,6 +305,19 @@ TEST_F(ParserInitialATest, alterUser) { run("ALTER user wxy privilege 'write'"); } +TEST_F(ParserInitialATest, balanceVgroup) { + useDb("root", "test"); + + setCheckDdlFunc([&](const SQuery* pQuery, ParserStage stage) { + ASSERT_EQ(nodeType(pQuery->pRoot), QUERY_NODE_BALANCE_VGROUP_STMT); + ASSERT_EQ(pQuery->pCmdMsg->msgType, TDMT_MND_BALANCE_VGROUP); + SBalanceVgroupReq req = {0}; + ASSERT_EQ(tDeserializeSBalanceVgroupReq(pQuery->pCmdMsg->pMsg, pQuery->pCmdMsg->msgLen, &req), TSDB_CODE_SUCCESS); + }); + + run("BALANCE VGROUP"); +} + TEST_F(ParserInitialATest, bug001) { useDb("root", "test"); diff --git a/source/libs/qcom/src/querymsg.c b/source/libs/qcom/src/querymsg.c index 949a58c2fa..6de1620de1 100644 --- a/source/libs/qcom/src/querymsg.c +++ b/source/libs/qcom/src/querymsg.c @@ -203,6 +203,24 @@ int32_t queryBuildGetUserAuthMsg(void *input, char **msg, int32_t msgSize, int32 return TSDB_CODE_SUCCESS; } +int32_t queryBuildGetTbIndexMsg(void *input, char **msg, int32_t msgSize, int32_t *msgLen, void*(*mallcFp)(int32_t)) { + if (NULL == msg || NULL == msgLen) { + return TSDB_CODE_TSC_INVALID_INPUT; + } + + STableIndexReq indexReq = {0}; + strcpy(indexReq.tbFName, input); + + int32_t bufLen = tSerializeSTableIndexReq(NULL, 0, &indexReq); + void *pBuf = (*mallcFp)(bufLen); + tSerializeSTableIndexReq(pBuf, bufLen, &indexReq); + + *msg = pBuf; + *msgLen = bufLen; + + return TSDB_CODE_SUCCESS; +} + int32_t queryProcessUseDBRsp(void *output, char *msg, int32_t msgSize) { SUseDbOutput *pOut = output; @@ -459,26 +477,43 @@ int32_t queryProcessGetUserAuthRsp(void *output, char *msg, int32_t msgSize) { return TSDB_CODE_SUCCESS; } +int32_t queryProcessGetTbIndexRsp(void *output, char *msg, int32_t msgSize) { + if (NULL == output || NULL == msg || msgSize <= 0) { + return TSDB_CODE_TSC_INVALID_INPUT; + } + + STableIndexRsp out = {0}; + if (tDeserializeSTableIndexRsp(msg, msgSize, &out) != 0) { + qError("tDeserializeSTableIndexRsp failed, msgSize:%d", msgSize); + return TSDB_CODE_INVALID_MSG; + } + + *(void **)output = out.pIndex; + + return TSDB_CODE_SUCCESS; +} + void initQueryModuleMsgHandle() { - queryBuildMsg[TMSG_INDEX(TDMT_VND_TABLE_META)] = queryBuildTableMetaReqMsg; - queryBuildMsg[TMSG_INDEX(TDMT_MND_TABLE_META)] = queryBuildTableMetaReqMsg; - queryBuildMsg[TMSG_INDEX(TDMT_MND_USE_DB)] = queryBuildUseDbMsg; - queryBuildMsg[TMSG_INDEX(TDMT_MND_QNODE_LIST)] = queryBuildQnodeListMsg; - queryBuildMsg[TMSG_INDEX(TDMT_MND_GET_DB_CFG)] = queryBuildGetDBCfgMsg; - queryBuildMsg[TMSG_INDEX(TDMT_MND_GET_INDEX)] = queryBuildGetIndexMsg; - queryBuildMsg[TMSG_INDEX(TDMT_MND_RETRIEVE_FUNC)] = queryBuildRetrieveFuncMsg; - queryBuildMsg[TMSG_INDEX(TDMT_MND_GET_USER_AUTH)] = queryBuildGetUserAuthMsg; - + queryBuildMsg[TMSG_INDEX(TDMT_VND_TABLE_META)] = queryBuildTableMetaReqMsg; + queryBuildMsg[TMSG_INDEX(TDMT_MND_TABLE_META)] = queryBuildTableMetaReqMsg; + queryBuildMsg[TMSG_INDEX(TDMT_MND_USE_DB)] = queryBuildUseDbMsg; + queryBuildMsg[TMSG_INDEX(TDMT_MND_QNODE_LIST)] = queryBuildQnodeListMsg; + queryBuildMsg[TMSG_INDEX(TDMT_MND_GET_DB_CFG)] = queryBuildGetDBCfgMsg; + queryBuildMsg[TMSG_INDEX(TDMT_MND_GET_INDEX)] = queryBuildGetIndexMsg; + queryBuildMsg[TMSG_INDEX(TDMT_MND_RETRIEVE_FUNC)] = queryBuildRetrieveFuncMsg; + queryBuildMsg[TMSG_INDEX(TDMT_MND_GET_USER_AUTH)] = queryBuildGetUserAuthMsg; + queryBuildMsg[TMSG_INDEX(TDMT_MND_GET_TABLE_INDEX)] = queryBuildGetTbIndexMsg; - queryProcessMsgRsp[TMSG_INDEX(TDMT_VND_TABLE_META)] = queryProcessTableMetaRsp; - queryProcessMsgRsp[TMSG_INDEX(TDMT_MND_TABLE_META)] = queryProcessTableMetaRsp; - queryProcessMsgRsp[TMSG_INDEX(TDMT_MND_USE_DB)] = queryProcessUseDBRsp; - queryProcessMsgRsp[TMSG_INDEX(TDMT_MND_QNODE_LIST)] = queryProcessQnodeListRsp; - queryProcessMsgRsp[TMSG_INDEX(TDMT_MND_GET_DB_CFG)] = queryProcessGetDbCfgRsp; - queryProcessMsgRsp[TMSG_INDEX(TDMT_MND_GET_INDEX)] = queryProcessGetIndexRsp; - queryProcessMsgRsp[TMSG_INDEX(TDMT_MND_RETRIEVE_FUNC)] = queryProcessRetrieveFuncRsp; - queryProcessMsgRsp[TMSG_INDEX(TDMT_MND_GET_USER_AUTH)] = queryProcessGetUserAuthRsp; + queryProcessMsgRsp[TMSG_INDEX(TDMT_VND_TABLE_META)] = queryProcessTableMetaRsp; + queryProcessMsgRsp[TMSG_INDEX(TDMT_MND_TABLE_META)] = queryProcessTableMetaRsp; + queryProcessMsgRsp[TMSG_INDEX(TDMT_MND_USE_DB)] = queryProcessUseDBRsp; + queryProcessMsgRsp[TMSG_INDEX(TDMT_MND_QNODE_LIST)] = queryProcessQnodeListRsp; + queryProcessMsgRsp[TMSG_INDEX(TDMT_MND_GET_DB_CFG)] = queryProcessGetDbCfgRsp; + queryProcessMsgRsp[TMSG_INDEX(TDMT_MND_GET_INDEX)] = queryProcessGetIndexRsp; + queryProcessMsgRsp[TMSG_INDEX(TDMT_MND_RETRIEVE_FUNC)] = queryProcessRetrieveFuncRsp; + queryProcessMsgRsp[TMSG_INDEX(TDMT_MND_GET_USER_AUTH)] = queryProcessGetUserAuthRsp; + queryProcessMsgRsp[TMSG_INDEX(TDMT_MND_GET_TABLE_INDEX)] = queryProcessGetTbIndexRsp; } #pragma GCC diagnostic pop diff --git a/source/libs/qworker/inc/qwInt.h b/source/libs/qworker/inc/qwInt.h index 1d31c86308..2368b13dd6 100644 --- a/source/libs/qworker/inc/qwInt.h +++ b/source/libs/qworker/inc/qwInt.h @@ -356,7 +356,7 @@ int32_t qwOpenRef(void); void qwSetHbParam(int64_t refId, SQWHbParam **pParam); int32_t qwUpdateTimeInQueue(SQWorker *mgmt, int64_t ts, EQueueType type); int64_t qwGetTimeInQueue(SQWorker *mgmt, EQueueType type); -void qwClearExpiredSch(SArray* pExpiredSch); +void qwClearExpiredSch(SQWorker *mgmt, SArray* pExpiredSch); int32_t qwAcquireScheduler(SQWorker *mgmt, uint64_t sId, int32_t rwType, SQWSchStatus **sch); void qwFreeTaskCtx(QW_FPARAMS_DEF, SQWTaskCtx *ctx); diff --git a/source/libs/qworker/src/qwUtil.c b/source/libs/qworker/src/qwUtil.c index 667008e68e..e5dd9e71df 100644 --- a/source/libs/qworker/src/qwUtil.c +++ b/source/libs/qworker/src/qwUtil.c @@ -539,8 +539,23 @@ int64_t qwGetTimeInQueue(SQWorker *mgmt, EQueueType type) { } -void qwClearExpiredSch(SArray* pExpiredSch) { +void qwClearExpiredSch(SQWorker *mgmt, SArray* pExpiredSch) { + int32_t num = taosArrayGetSize(pExpiredSch); + for (int32_t i = 0; i < num; ++i) { + uint64_t *sId = taosArrayGet(pExpiredSch, i); + SQWSchStatus *pSch = NULL; + if (qwAcquireScheduler(mgmt, *sId, QW_WRITE, &pSch)) { + continue; + } + if (taosHashGetSize(pSch->tasksHash) <= 0) { + qwDestroySchStatus(pSch); + taosHashRemove(mgmt->schHash, sId, sizeof(*sId)); + qError("sch %" PRIx64 "destroyed", *sId); + } + + qwReleaseScheduler(QW_WRITE, mgmt); + } } diff --git a/source/libs/qworker/src/qworker.c b/source/libs/qworker/src/qworker.c index 333884f883..ae0eda5449 100644 --- a/source/libs/qworker/src/qworker.c +++ b/source/libs/qworker/src/qworker.c @@ -790,9 +790,10 @@ int32_t qwProcessHb(SQWorker *mgmt, SQWMsg *qwMsg, SSchedulerHbReq *req) { } QW_ERR_JRET(qwAcquireAddScheduler(mgmt, req->sId, QW_READ, &sch)); - QW_ERR_JRET(qwRegisterHbBrokenLinkArg(mgmt, req->sId, &qwMsg->connInfo)); + sch->hbBrokenTs = 0; + QW_LOCK(QW_WRITE, &sch->hbConnLock); if (sch->hbConnInfo.handle) { @@ -912,7 +913,7 @@ _return: } if (taosArrayGetSize(pExpiredSch) > 0) { - qwClearExpiredSch(pExpiredSch); + qwClearExpiredSch(mgmt, pExpiredSch); } taosMemoryFreeClear(rspList); diff --git a/source/libs/scalar/src/sclvector.c b/source/libs/scalar/src/sclvector.c index 747da673d1..59208de3c4 100644 --- a/source/libs/scalar/src/sclvector.c +++ b/source/libs/scalar/src/sclvector.c @@ -373,9 +373,6 @@ static FORCE_INLINE void ncharToVar(char* buf, SScalarParam* pOut, int32_t rowIn //TODO opt performance, tmp is not needed. int32_t vectorConvertFromVarData(const SScalarParam* pIn, SScalarParam* pOut, int32_t inType, int32_t outType) { - int32_t bufSize = pIn->columnData->info.bytes; - char *tmp = taosMemoryMalloc(bufSize + VARSTR_HEADER_SIZE); - bool vton = false; _bufConverteFunc func = NULL; @@ -423,6 +420,12 @@ int32_t vectorConvertFromVarData(const SScalarParam* pIn, SScalarParam* pOut, in continue; } } + int32_t bufSize = pIn->columnData->info.bytes; + char *tmp = taosMemoryMalloc(varDataTLen(data)); + if(!tmp){ + sclError("out of memory in vectorConvertFromVarData"); + return TSDB_CODE_OUT_OF_MEMORY; + } if (vton) { memcpy(tmp, data, varDataTLen(data)); } else { @@ -444,9 +447,9 @@ int32_t vectorConvertFromVarData(const SScalarParam* pIn, SScalarParam* pOut, in } (*func)(tmp, pOut, i); + taosMemoryFreeClear(tmp); } - taosMemoryFreeClear(tmp); return TSDB_CODE_SUCCESS; } @@ -1674,6 +1677,14 @@ void vectorNotNull(SScalarParam* pLeft, SScalarParam* pRight, SScalarParam *pOut void vectorIsTrue(SScalarParam* pLeft, SScalarParam* pRight, SScalarParam *pOut, int32_t _ord) { vectorConvertImpl(pLeft, pOut); + for(int32_t i = 0; i < pOut->numOfRows; ++i) { + if(colDataIsNull_s(pOut->columnData, i)) { + int8_t v = 0; + colDataAppendInt8(pOut->columnData, i, &v); + colDataSetNotNull_f(pOut->columnData->nullbitmap, i); + } + } + pOut->columnData->hasNull = false; } STagVal getJsonValue(char *json, char *key, bool *isExist) { diff --git a/source/libs/scalar/test/scalar/scalarTests.cpp b/source/libs/scalar/test/scalar/scalarTests.cpp index 284ce4b95c..d879150200 100644 --- a/source/libs/scalar/test/scalar/scalarTests.cpp +++ b/source/libs/scalar/test/scalar/scalarTests.cpp @@ -1,17 +1,17 @@ /* - * Copyright (c) 2019 TAOS Data, Inc. - * - * This program is free software: you can use, redistribute, and/or modify - * it under the terms of the GNU Affero General Public License, version 3 - * or later ("AGPL"), as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, but WITHOUT - * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or - * FITNESS FOR A PARTICULAR PURPOSE. - * - * You should have received a copy of the GNU Affero General Public License - * along with this program. If not, see . - */ +* Copyright (c) 2019 TAOS Data, Inc. +* +* This program is free software: you can use, redistribute, and/or modify +* it under the terms of the GNU Affero General Public License, version 3 +* or later ("AGPL"), as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, but WITHOUT +* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see . +*/ #include #include @@ -55,229 +55,229 @@ namespace { SColumnInfo createColumnInfo(int32_t colId, int32_t type, int32_t bytes) { - SColumnInfo info = {0}; - info.colId = colId; - info.type = type; - info.bytes = bytes; - return info; + SColumnInfo info = {0}; + info.colId = colId; + info.type = type; + info.bytes = bytes; + return info; } int64_t scltLeftV = 21, scltRightV = 10; double scltLeftVd = 21.0, scltRightVd = 10.0; void scltFreeDataBlock(void *block) { - blockDataDestroy(*(SSDataBlock **)block); + blockDataDestroy(*(SSDataBlock **)block); } void scltInitLogFile() { - const char *defaultLogFileNamePrefix = "taoslog"; - const int32_t maxLogFileNum = 10; + const char *defaultLogFileNamePrefix = "taoslog"; + const int32_t maxLogFileNum = 10; - tsAsyncLog = 0; - qDebugFlag = 159; - strcpy(tsLogDir, TD_LOG_DIR_PATH); + tsAsyncLog = 0; + qDebugFlag = 159; + strcpy(tsLogDir, TD_LOG_DIR_PATH); - if (taosInitLog(defaultLogFileNamePrefix, maxLogFileNum) < 0) { - printf("failed to open log file in directory:%s\n", tsLogDir); - } + if (taosInitLog(defaultLogFileNamePrefix, maxLogFileNum) < 0) { + printf("failed to open log file in directory:%s\n", tsLogDir); + } } void scltAppendReservedSlot(SArray *pBlockList, int16_t *dataBlockId, int16_t *slotId, bool newBlock, int32_t rows, SColumnInfo *colInfo) { - if (newBlock) { - SSDataBlock *res = (SSDataBlock *)taosMemoryCalloc(1, sizeof(SSDataBlock)); - res->info.numOfCols = 1; - res->info.rows = rows; - res->pDataBlock = taosArrayInit(1, sizeof(SColumnInfoData)); - SColumnInfoData idata = {0}; - idata.info = *colInfo; + if (newBlock) { + SSDataBlock *res = (SSDataBlock *)taosMemoryCalloc(1, sizeof(SSDataBlock)); + res->info.numOfCols = 1; + res->info.rows = rows; + res->pDataBlock = taosArrayInit(1, sizeof(SColumnInfoData)); + SColumnInfoData idata = {0}; + idata.info = *colInfo; - taosArrayPush(res->pDataBlock, &idata); - taosArrayPush(pBlockList, &res); - - blockDataEnsureCapacity(res, rows); + taosArrayPush(res->pDataBlock, &idata); + taosArrayPush(pBlockList, &res); - *dataBlockId = taosArrayGetSize(pBlockList) - 1; - res->info.blockId = *dataBlockId; - *slotId = 0; - } else { - SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(pBlockList); - res->info.numOfCols++; - SColumnInfoData idata = {0}; - idata.info = *colInfo; + blockDataEnsureCapacity(res, rows); - colInfoDataEnsureCapacity(&idata, 0, rows); + *dataBlockId = taosArrayGetSize(pBlockList) - 1; + res->info.blockId = *dataBlockId; + *slotId = 0; + } else { + SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(pBlockList); + res->info.numOfCols++; + SColumnInfoData idata = {0}; + idata.info = *colInfo; - taosArrayPush(res->pDataBlock, &idata); - - *dataBlockId = taosArrayGetSize(pBlockList) - 1; - *slotId = taosArrayGetSize(res->pDataBlock) - 1; - } + colInfoDataEnsureCapacity(&idata, 0, rows); + + taosArrayPush(res->pDataBlock, &idata); + + *dataBlockId = taosArrayGetSize(pBlockList) - 1; + *slotId = taosArrayGetSize(res->pDataBlock) - 1; + } } void scltMakeValueNode(SNode **pNode, int32_t dataType, void *value) { - SNode *node = (SNode*)nodesMakeNode(QUERY_NODE_VALUE); - SValueNode *vnode = (SValueNode *)node; - vnode->node.resType.type = dataType; + SNode *node = (SNode*)nodesMakeNode(QUERY_NODE_VALUE); + SValueNode *vnode = (SValueNode *)node; + vnode->node.resType.type = dataType; - if (IS_VAR_DATA_TYPE(dataType)) { - vnode->datum.p = (char *)taosMemoryMalloc(varDataTLen(value)); - varDataCopy(vnode->datum.p, value); - vnode->node.resType.bytes = varDataTLen(value); - } else { - vnode->node.resType.bytes = tDataTypes[dataType].bytes; - assignVal((char *)nodesGetValueFromNode(vnode), (const char *)value, 0, dataType); - } - - *pNode = (SNode *)vnode; + if (IS_VAR_DATA_TYPE(dataType)) { + vnode->datum.p = (char *)taosMemoryMalloc(varDataTLen(value)); + varDataCopy(vnode->datum.p, value); + vnode->node.resType.bytes = varDataTLen(value); + } else { + vnode->node.resType.bytes = tDataTypes[dataType].bytes; + assignVal((char *)nodesGetValueFromNode(vnode), (const char *)value, 0, dataType); + } + + *pNode = (SNode *)vnode; } void scltMakeColumnNode(SNode **pNode, SSDataBlock **block, int32_t dataType, int32_t dataBytes, int32_t rowNum, void *value) { - SNode *node = (SNode*)nodesMakeNode(QUERY_NODE_COLUMN); - SColumnNode *rnode = (SColumnNode *)node; - rnode->node.resType.type = dataType; - rnode->node.resType.bytes = dataBytes; - rnode->dataBlockId = 0; + SNode *node = (SNode*)nodesMakeNode(QUERY_NODE_COLUMN); + SColumnNode *rnode = (SColumnNode *)node; + rnode->node.resType.type = dataType; + rnode->node.resType.bytes = dataBytes; + rnode->dataBlockId = 0; - if (NULL == block) { - *pNode = (SNode *)rnode; - return; - } + if (NULL == block) { + *pNode = (SNode *)rnode; + return; + } - if (NULL == *block) { - SSDataBlock *res = (SSDataBlock *)taosMemoryCalloc(1, sizeof(SSDataBlock)); - res->info.numOfCols = 3; - res->info.rows = rowNum; - res->pDataBlock = taosArrayInit(3, sizeof(SColumnInfoData)); - for (int32_t i = 0; i < 2; ++i) { - SColumnInfoData idata = {{0}}; - idata.info.type = TSDB_DATA_TYPE_NULL; - idata.info.bytes = 10; - idata.info.colId = i + 1; + if (NULL == *block) { + SSDataBlock *res = (SSDataBlock *)taosMemoryCalloc(1, sizeof(SSDataBlock)); + res->info.numOfCols = 3; + res->info.rows = rowNum; + res->pDataBlock = taosArrayInit(3, sizeof(SColumnInfoData)); + for (int32_t i = 0; i < 2; ++i) { + SColumnInfoData idata = {{0}}; + idata.info.type = TSDB_DATA_TYPE_NULL; + idata.info.bytes = 10; + idata.info.colId = i + 1; - int32_t size = idata.info.bytes * rowNum; - idata.pData = (char *)taosMemoryCalloc(1, size); - taosArrayPush(res->pDataBlock, &idata); - } + int32_t size = idata.info.bytes * rowNum; + idata.pData = (char *)taosMemoryCalloc(1, size); + taosArrayPush(res->pDataBlock, &idata); + } - SColumnInfoData idata = {{0}}; - idata.info.type = dataType; - idata.info.bytes = dataBytes; - idata.info.colId = 3; - int32_t size = idata.info.bytes * rowNum; - idata.pData = (char *)taosMemoryCalloc(1, size); - colInfoDataEnsureCapacity(&idata, 0, rowNum); - taosArrayPush(res->pDataBlock, &idata); - SColumnInfoData *pColumn = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); - for (int32_t i = 0; i < rowNum; ++i) { - colDataAppend(pColumn, i, (const char *)value, false); - if (IS_VAR_DATA_TYPE(dataType)) { - value = (char *)value + varDataTLen(value); - } else { - value = (char *)value + dataBytes; - } - } + SColumnInfoData idata = {{0}}; + idata.info.type = dataType; + idata.info.bytes = dataBytes; + idata.info.colId = 3; + int32_t size = idata.info.bytes * rowNum; + idata.pData = (char *)taosMemoryCalloc(1, size); + colInfoDataEnsureCapacity(&idata, 0, rowNum); + taosArrayPush(res->pDataBlock, &idata); + SColumnInfoData *pColumn = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); + for (int32_t i = 0; i < rowNum; ++i) { + colDataAppend(pColumn, i, (const char *)value, false); + if (IS_VAR_DATA_TYPE(dataType)) { + value = (char *)value + varDataTLen(value); + } else { + value = (char *)value + dataBytes; + } + } - rnode->slotId = 2; - rnode->colId = 3; + rnode->slotId = 2; + rnode->colId = 3; - *block = res; - } else { - SSDataBlock *res = *block; - - int32_t idx = taosArrayGetSize(res->pDataBlock); - SColumnInfoData idata = {{0}}; - idata.info.type = dataType; - idata.info.bytes = dataBytes; - idata.info.colId = 1 + idx; - int32_t size = idata.info.bytes * rowNum; - idata.pData = (char *)taosMemoryCalloc(1, size); - taosArrayPush(res->pDataBlock, &idata); - res->info.numOfCols++; - SColumnInfoData *pColumn = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); - - colInfoDataEnsureCapacity(pColumn, 0, rowNum); + *block = res; + } else { + SSDataBlock *res = *block; - for (int32_t i = 0; i < rowNum; ++i) { - colDataAppend(pColumn, i, (const char *)value, false); - if (IS_VAR_DATA_TYPE(dataType)) { - value = (char *)value + varDataTLen(value); - } else { - value = (char *)value + dataBytes; - } - } - - rnode->slotId = idx; - rnode->colId = 1 + idx; - } + int32_t idx = taosArrayGetSize(res->pDataBlock); + SColumnInfoData idata = {{0}}; + idata.info.type = dataType; + idata.info.bytes = dataBytes; + idata.info.colId = 1 + idx; + int32_t size = idata.info.bytes * rowNum; + idata.pData = (char *)taosMemoryCalloc(1, size); + taosArrayPush(res->pDataBlock, &idata); + res->info.numOfCols++; + SColumnInfoData *pColumn = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); - *pNode = (SNode *)rnode; + colInfoDataEnsureCapacity(pColumn, 0, rowNum); + + for (int32_t i = 0; i < rowNum; ++i) { + colDataAppend(pColumn, i, (const char *)value, false); + if (IS_VAR_DATA_TYPE(dataType)) { + value = (char *)value + varDataTLen(value); + } else { + value = (char *)value + dataBytes; + } + } + + rnode->slotId = idx; + rnode->colId = 1 + idx; + } + + *pNode = (SNode *)rnode; } void scltMakeOpNode2(SNode **pNode, EOperatorType opType, int32_t resType, SNode *pLeft, SNode *pRight, bool isReverse) { - SNode *node = (SNode*)nodesMakeNode(QUERY_NODE_OPERATOR); - SOperatorNode *onode = (SOperatorNode *)node; - onode->node.resType.type = resType; - onode->node.resType.bytes = tDataTypes[resType].bytes; + SNode *node = (SNode*)nodesMakeNode(QUERY_NODE_OPERATOR); + SOperatorNode *onode = (SOperatorNode *)node; + onode->node.resType.type = resType; + onode->node.resType.bytes = tDataTypes[resType].bytes; - onode->opType = opType; - if(isReverse){ - onode->pLeft = pRight; - onode->pRight = pLeft; - }else{ - onode->pLeft = pLeft; - onode->pRight = pRight; - } + onode->opType = opType; + if(isReverse){ + onode->pLeft = pRight; + onode->pRight = pLeft; + }else{ + onode->pLeft = pLeft; + onode->pRight = pRight; + } - *pNode = (SNode *)onode; + *pNode = (SNode *)onode; } void scltMakeOpNode(SNode **pNode, EOperatorType opType, int32_t resType, SNode *pLeft, SNode *pRight) { - SNode *node = (SNode*)nodesMakeNode(QUERY_NODE_OPERATOR); - SOperatorNode *onode = (SOperatorNode *)node; - onode->node.resType.type = resType; - onode->node.resType.bytes = tDataTypes[resType].bytes; + SNode *node = (SNode*)nodesMakeNode(QUERY_NODE_OPERATOR); + SOperatorNode *onode = (SOperatorNode *)node; + onode->node.resType.type = resType; + onode->node.resType.bytes = tDataTypes[resType].bytes; - onode->opType = opType; - onode->pLeft = pLeft; - onode->pRight = pRight; + onode->opType = opType; + onode->pLeft = pLeft; + onode->pRight = pRight; - *pNode = (SNode *)onode; + *pNode = (SNode *)onode; } void scltMakeListNode(SNode **pNode, SNodeList *list, int32_t resType) { - SNode *node = (SNode*)nodesMakeNode(QUERY_NODE_NODE_LIST); - SNodeListNode *lnode = (SNodeListNode *)node; - lnode->dataType.type = resType; - lnode->pNodeList = list; + SNode *node = (SNode*)nodesMakeNode(QUERY_NODE_NODE_LIST); + SNodeListNode *lnode = (SNodeListNode *)node; + lnode->dataType.type = resType; + lnode->pNodeList = list; - *pNode = (SNode *)lnode; + *pNode = (SNode *)lnode; } void scltMakeLogicNode(SNode **pNode, ELogicConditionType opType, SNode **nodeList, int32_t nodeNum) { - SNode *node = (SNode*)nodesMakeNode(QUERY_NODE_LOGIC_CONDITION); - SLogicConditionNode *onode = (SLogicConditionNode *)node; - onode->condType = opType; - onode->node.resType.type = TSDB_DATA_TYPE_BOOL; - onode->node.resType.bytes = sizeof(bool); + SNode *node = (SNode*)nodesMakeNode(QUERY_NODE_LOGIC_CONDITION); + SLogicConditionNode *onode = (SLogicConditionNode *)node; + onode->condType = opType; + onode->node.resType.type = TSDB_DATA_TYPE_BOOL; + onode->node.resType.bytes = sizeof(bool); - onode->pParameterList = nodesMakeList(); - for (int32_t i = 0; i < nodeNum; ++i) { - nodesListAppend(onode->pParameterList, nodeList[i]); - } - - *pNode = (SNode *)onode; + onode->pParameterList = nodesMakeList(); + for (int32_t i = 0; i < nodeNum; ++i) { + nodesListAppend(onode->pParameterList, nodeList[i]); + } + + *pNode = (SNode *)onode; } void scltMakeTargetNode(SNode **pNode, int16_t dataBlockId, int16_t slotId, SNode *snode) { - SNode *node = (SNode*)nodesMakeNode(QUERY_NODE_TARGET); - STargetNode *onode = (STargetNode *)node; - onode->pExpr = snode; - onode->dataBlockId = dataBlockId; - onode->slotId = slotId; - - *pNode = (SNode *)onode; + SNode *node = (SNode*)nodesMakeNode(QUERY_NODE_TARGET); + STargetNode *onode = (STargetNode *)node; + onode->pExpr = snode; + onode->dataBlockId = dataBlockId; + onode->slotId = slotId; + + *pNode = (SNode *)onode; } @@ -285,3381 +285,3381 @@ void scltMakeTargetNode(SNode **pNode, int16_t dataBlockId, int16_t slotId, SNod } TEST(constantTest, bigint_add_bigint) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BIGINT, &scltLeftV); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BIGINT, &scltRightV); - scltMakeOpNode(&opNode, OP_TYPE_ADD, TSDB_DATA_TYPE_DOUBLE, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_DOUBLE); - ASSERT_FLOAT_EQ(v->datum.d, (scltLeftV + scltRightV)); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BIGINT, &scltLeftV); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BIGINT, &scltRightV); + scltMakeOpNode(&opNode, OP_TYPE_ADD, TSDB_DATA_TYPE_DOUBLE, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_DOUBLE); + ASSERT_FLOAT_EQ(v->datum.d, (scltLeftV + scltRightV)); + nodesDestroyNode(res); } TEST(constantTest, double_sub_bigint) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_DOUBLE, &scltLeftVd); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BIGINT, &scltRightV); - scltMakeOpNode(&opNode, OP_TYPE_SUB, TSDB_DATA_TYPE_DOUBLE, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_DOUBLE); - ASSERT_FLOAT_EQ(v->datum.d, (scltLeftVd - scltRightV)); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_DOUBLE, &scltLeftVd); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BIGINT, &scltRightV); + scltMakeOpNode(&opNode, OP_TYPE_SUB, TSDB_DATA_TYPE_DOUBLE, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_DOUBLE); + ASSERT_FLOAT_EQ(v->datum.d, (scltLeftVd - scltRightV)); + nodesDestroyNode(res); } TEST(constantTest, tinyint_and_smallint) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_TINYINT, &scltLeftV); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &scltRightV); - scltMakeOpNode(&opNode, OP_TYPE_BIT_AND, TSDB_DATA_TYPE_BIGINT, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BIGINT); - ASSERT_EQ(v->datum.i, (int64_t)scltLeftV & (int64_t)scltRightV); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_TINYINT, &scltLeftV); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &scltRightV); + scltMakeOpNode(&opNode, OP_TYPE_BIT_AND, TSDB_DATA_TYPE_BIGINT, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BIGINT); + ASSERT_EQ(v->datum.i, (int64_t)scltLeftV & (int64_t)scltRightV); + nodesDestroyNode(res); } TEST(constantTest, bigint_or_double) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BIGINT, &scltLeftV); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_DOUBLE, &scltRightVd); - scltMakeOpNode(&opNode, OP_TYPE_BIT_OR, TSDB_DATA_TYPE_BIGINT, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BIGINT); - ASSERT_EQ(v->datum.i, (int64_t)scltLeftV | (int64_t)scltRightVd); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BIGINT, &scltLeftV); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_DOUBLE, &scltRightVd); + scltMakeOpNode(&opNode, OP_TYPE_BIT_OR, TSDB_DATA_TYPE_BIGINT, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BIGINT); + ASSERT_EQ(v->datum.i, (int64_t)scltLeftV | (int64_t)scltRightVd); + nodesDestroyNode(res); } TEST(constantTest, int_or_binary) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - char binaryStr[64] = {0}; - sprintf(&binaryStr[2], "%d", scltRightV); - varDataSetLen(binaryStr, strlen(&binaryStr[2])); - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &scltLeftV); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, binaryStr); - scltMakeOpNode(&opNode, OP_TYPE_BIT_OR, TSDB_DATA_TYPE_BIGINT, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BIGINT); - ASSERT_EQ(v->datum.b, scltLeftV | scltRightV); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + char binaryStr[64] = {0}; + sprintf(&binaryStr[2], "%d", scltRightV); + varDataSetLen(binaryStr, strlen(&binaryStr[2])); + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &scltLeftV); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, binaryStr); + scltMakeOpNode(&opNode, OP_TYPE_BIT_OR, TSDB_DATA_TYPE_BIGINT, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BIGINT); + ASSERT_EQ(v->datum.b, scltLeftV | scltRightV); + nodesDestroyNode(res); } TEST(constantTest, int_greater_double) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &scltLeftV); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_DOUBLE, &scltRightVd); - scltMakeOpNode(&opNode, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, scltLeftV > scltRightVd); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &scltLeftV); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_DOUBLE, &scltRightVd); + scltMakeOpNode(&opNode, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, scltLeftV > scltRightVd); + nodesDestroyNode(res); } TEST(constantTest, int_greater_equal_binary) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - char binaryStr[64] = {0}; - sprintf(&binaryStr[2], "%d", scltRightV); - varDataSetLen(binaryStr, strlen(&binaryStr[2])); - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &scltLeftV); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, binaryStr); - scltMakeOpNode(&opNode, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, scltLeftV > scltRightVd); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + char binaryStr[64] = {0}; + sprintf(&binaryStr[2], "%d", scltRightV); + varDataSetLen(binaryStr, strlen(&binaryStr[2])); + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &scltLeftV); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, binaryStr); + scltMakeOpNode(&opNode, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, scltLeftV > scltRightVd); + nodesDestroyNode(res); } TEST(constantTest, tinyint_lower_ubigint) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_TINYINT, &scltLeftV); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_UBIGINT, &scltRightV); - scltMakeOpNode(&opNode, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, scltLeftV < scltRightV); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_TINYINT, &scltLeftV); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_UBIGINT, &scltRightV); + scltMakeOpNode(&opNode, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, scltLeftV < scltRightV); + nodesDestroyNode(res); } TEST(constantTest, usmallint_lower_equal_ubigint) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - int32_t leftv = 1; - int64_t rightv = 1; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_USMALLINT, &leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_UBIGINT, &rightv); - scltMakeOpNode(&opNode, OP_TYPE_LOWER_EQUAL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, leftv <= rightv); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + int32_t leftv = 1; + int64_t rightv = 1; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_USMALLINT, &leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_UBIGINT, &rightv); + scltMakeOpNode(&opNode, OP_TYPE_LOWER_EQUAL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, leftv <= rightv); + nodesDestroyNode(res); } TEST(constantTest, int_equal_smallint1) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - int32_t leftv = 1; - int16_t rightv = 1; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv); - scltMakeOpNode(&opNode, OP_TYPE_EQUAL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, leftv == rightv); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + int32_t leftv = 1; + int16_t rightv = 1; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv); + scltMakeOpNode(&opNode, OP_TYPE_EQUAL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, leftv == rightv); + nodesDestroyNode(res); } TEST(constantTest, int_equal_smallint2) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - int32_t leftv = 0, rightv = 1; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv); - scltMakeOpNode(&opNode, OP_TYPE_EQUAL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, leftv == rightv); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + int32_t leftv = 0, rightv = 1; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv); + scltMakeOpNode(&opNode, OP_TYPE_EQUAL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, leftv == rightv); + nodesDestroyNode(res); } TEST(constantTest, int_not_equal_smallint1) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - int32_t leftv = 1, rightv = 1; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv); - scltMakeOpNode(&opNode, OP_TYPE_NOT_EQUAL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, leftv != rightv); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + int32_t leftv = 1, rightv = 1; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv); + scltMakeOpNode(&opNode, OP_TYPE_NOT_EQUAL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, leftv != rightv); + nodesDestroyNode(res); } TEST(constantTest, int_not_equal_smallint2) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - int32_t leftv = 0, rightv = 1; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv); - scltMakeOpNode(&opNode, OP_TYPE_NOT_EQUAL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, leftv != rightv); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + int32_t leftv = 0, rightv = 1; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv); + scltMakeOpNode(&opNode, OP_TYPE_NOT_EQUAL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, leftv != rightv); + nodesDestroyNode(res); } TEST(constantTest, int_in_smallint1) { - scltInitLogFile(); - - SNode *pLeft = NULL, *pRight = NULL, *listNode = NULL, *res = NULL, *opNode = NULL; - int32_t leftv = 1, rightv1 = 1,rightv2 = 2,rightv3 = 3; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); - SNodeList* list = nodesMakeList(); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv1); - nodesListAppend(list, pRight); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv2); - nodesListAppend(list, pRight); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv3); - nodesListAppend(list, pRight); - scltMakeListNode(&listNode,list, TSDB_DATA_TYPE_INT); - scltMakeOpNode(&opNode, OP_TYPE_IN, TSDB_DATA_TYPE_BOOL, pLeft, listNode); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, true); - nodesDestroyNode(res); + scltInitLogFile(); + + SNode *pLeft = NULL, *pRight = NULL, *listNode = NULL, *res = NULL, *opNode = NULL; + int32_t leftv = 1, rightv1 = 1,rightv2 = 2,rightv3 = 3; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); + SNodeList* list = nodesMakeList(); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv1); + nodesListAppend(list, pRight); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv2); + nodesListAppend(list, pRight); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv3); + nodesListAppend(list, pRight); + scltMakeListNode(&listNode,list, TSDB_DATA_TYPE_INT); + scltMakeOpNode(&opNode, OP_TYPE_IN, TSDB_DATA_TYPE_BOOL, pLeft, listNode); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, true); + nodesDestroyNode(res); } TEST(constantTest, int_in_smallint2) { - scltInitLogFile(); - - SNode *pLeft = NULL, *pRight = NULL, *listNode = NULL, *res = NULL, *opNode = NULL; - int32_t leftv = 4, rightv1 = 1,rightv2 = 2,rightv3 = 3; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); - SNodeList* list = nodesMakeList(); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv1); - nodesListAppend(list, pRight); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv2); - nodesListAppend(list, pRight); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv3); - nodesListAppend(list, pRight); - scltMakeListNode(&listNode,list, TSDB_DATA_TYPE_INT); - scltMakeOpNode(&opNode, OP_TYPE_IN, TSDB_DATA_TYPE_BOOL, pLeft, listNode); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, false); - nodesDestroyNode(res); + scltInitLogFile(); + + SNode *pLeft = NULL, *pRight = NULL, *listNode = NULL, *res = NULL, *opNode = NULL; + int32_t leftv = 4, rightv1 = 1,rightv2 = 2,rightv3 = 3; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); + SNodeList* list = nodesMakeList(); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv1); + nodesListAppend(list, pRight); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv2); + nodesListAppend(list, pRight); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv3); + nodesListAppend(list, pRight); + scltMakeListNode(&listNode,list, TSDB_DATA_TYPE_INT); + scltMakeOpNode(&opNode, OP_TYPE_IN, TSDB_DATA_TYPE_BOOL, pLeft, listNode); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, false); + nodesDestroyNode(res); } TEST(constantTest, int_not_in_smallint1) { - SNode *pLeft = NULL, *pRight = NULL, *listNode = NULL, *res = NULL, *opNode = NULL; - int32_t leftv = 1, rightv1 = 1,rightv2 = 2,rightv3 = 3; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); - SNodeList* list = nodesMakeList(); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv1); - nodesListAppend(list, pRight); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv2); - nodesListAppend(list, pRight); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv3); - nodesListAppend(list, pRight); - scltMakeListNode(&listNode,list, TSDB_DATA_TYPE_INT); - scltMakeOpNode(&opNode, OP_TYPE_NOT_IN, TSDB_DATA_TYPE_BOOL, pLeft, listNode); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, false); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *listNode = NULL, *res = NULL, *opNode = NULL; + int32_t leftv = 1, rightv1 = 1,rightv2 = 2,rightv3 = 3; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); + SNodeList* list = nodesMakeList(); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv1); + nodesListAppend(list, pRight); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv2); + nodesListAppend(list, pRight); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv3); + nodesListAppend(list, pRight); + scltMakeListNode(&listNode,list, TSDB_DATA_TYPE_INT); + scltMakeOpNode(&opNode, OP_TYPE_NOT_IN, TSDB_DATA_TYPE_BOOL, pLeft, listNode); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, false); + nodesDestroyNode(res); } TEST(constantTest, int_not_in_smallint2) { - scltInitLogFile(); - - SNode *pLeft = NULL, *pRight = NULL, *listNode = NULL, *res = NULL, *opNode = NULL; - int32_t leftv = 4, rightv1 = 1,rightv2 = 2,rightv3 = 3; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); - SNodeList* list = nodesMakeList(); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv1); - nodesListAppend(list, pRight); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv2); - nodesListAppend(list, pRight); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv3); - nodesListAppend(list, pRight); - scltMakeListNode(&listNode,list, TSDB_DATA_TYPE_INT); - scltMakeOpNode(&opNode, OP_TYPE_NOT_IN, TSDB_DATA_TYPE_BOOL, pLeft, listNode); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, true); - nodesDestroyNode(res); + scltInitLogFile(); + + SNode *pLeft = NULL, *pRight = NULL, *listNode = NULL, *res = NULL, *opNode = NULL; + int32_t leftv = 4, rightv1 = 1,rightv2 = 2,rightv3 = 3; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); + SNodeList* list = nodesMakeList(); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv1); + nodesListAppend(list, pRight); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv2); + nodesListAppend(list, pRight); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_SMALLINT, &rightv3); + nodesListAppend(list, pRight); + scltMakeListNode(&listNode,list, TSDB_DATA_TYPE_INT); + scltMakeOpNode(&opNode, OP_TYPE_NOT_IN, TSDB_DATA_TYPE_BOOL, pLeft, listNode); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, true); + nodesDestroyNode(res); } TEST(constantTest, binary_like_binary1) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - char leftv[64] = {0}, rightv[64] = {0}; - sprintf(&leftv[2], "%s", "abc"); - varDataSetLen(leftv, strlen(&leftv[2])); - sprintf(&rightv[2], "%s", "a_c"); - varDataSetLen(rightv, strlen(&rightv[2])); - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BINARY, leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); - scltMakeOpNode(&opNode, OP_TYPE_LIKE, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, true); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + char leftv[64] = {0}, rightv[64] = {0}; + sprintf(&leftv[2], "%s", "abc"); + varDataSetLen(leftv, strlen(&leftv[2])); + sprintf(&rightv[2], "%s", "a_c"); + varDataSetLen(rightv, strlen(&rightv[2])); + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BINARY, leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); + scltMakeOpNode(&opNode, OP_TYPE_LIKE, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, true); + nodesDestroyNode(res); } TEST(constantTest, binary_like_binary2) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - char leftv[64] = {0}, rightv[64] = {0}; - sprintf(&leftv[2], "%s", "abc"); - varDataSetLen(leftv, strlen(&leftv[2])); - sprintf(&rightv[2], "%s", "ac"); - varDataSetLen(rightv, strlen(&rightv[2])); - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BINARY, leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); - scltMakeOpNode(&opNode, OP_TYPE_LIKE, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, false); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + char leftv[64] = {0}, rightv[64] = {0}; + sprintf(&leftv[2], "%s", "abc"); + varDataSetLen(leftv, strlen(&leftv[2])); + sprintf(&rightv[2], "%s", "ac"); + varDataSetLen(rightv, strlen(&rightv[2])); + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BINARY, leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); + scltMakeOpNode(&opNode, OP_TYPE_LIKE, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, false); + nodesDestroyNode(res); } TEST(constantTest, binary_not_like_binary1) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - char leftv[64] = {0}, rightv[64] = {0}; - sprintf(&leftv[2], "%s", "abc"); - varDataSetLen(leftv, strlen(&leftv[2])); - sprintf(&rightv[2], "%s", "a%c"); - varDataSetLen(rightv, strlen(&rightv[2])); - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BINARY, leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); - scltMakeOpNode(&opNode, OP_TYPE_NOT_LIKE, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, false); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + char leftv[64] = {0}, rightv[64] = {0}; + sprintf(&leftv[2], "%s", "abc"); + varDataSetLen(leftv, strlen(&leftv[2])); + sprintf(&rightv[2], "%s", "a%c"); + varDataSetLen(rightv, strlen(&rightv[2])); + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BINARY, leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); + scltMakeOpNode(&opNode, OP_TYPE_NOT_LIKE, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, false); + nodesDestroyNode(res); } TEST(constantTest, binary_not_like_binary2) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - char leftv[64] = {0}, rightv[64] = {0}; - sprintf(&leftv[2], "%s", "abc"); - varDataSetLen(leftv, strlen(&leftv[2])); - sprintf(&rightv[2], "%s", "ac"); - varDataSetLen(rightv, strlen(&rightv[2])); - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BINARY, leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); - scltMakeOpNode(&opNode, OP_TYPE_NOT_LIKE, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, true); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + char leftv[64] = {0}, rightv[64] = {0}; + sprintf(&leftv[2], "%s", "abc"); + varDataSetLen(leftv, strlen(&leftv[2])); + sprintf(&rightv[2], "%s", "ac"); + varDataSetLen(rightv, strlen(&rightv[2])); + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BINARY, leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); + scltMakeOpNode(&opNode, OP_TYPE_NOT_LIKE, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, true); + nodesDestroyNode(res); } TEST(constantTest, binary_match_binary1) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - char leftv[64] = {0}, rightv[64] = {0}; - sprintf(&leftv[2], "%s", "abc"); - varDataSetLen(leftv, strlen(&leftv[2])); - sprintf(&rightv[2], "%s", ".*"); - varDataSetLen(rightv, strlen(&rightv[2])); - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BINARY, leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); - scltMakeOpNode(&opNode, OP_TYPE_MATCH, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, true); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + char leftv[64] = {0}, rightv[64] = {0}; + sprintf(&leftv[2], "%s", "abc"); + varDataSetLen(leftv, strlen(&leftv[2])); + sprintf(&rightv[2], "%s", ".*"); + varDataSetLen(rightv, strlen(&rightv[2])); + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BINARY, leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); + scltMakeOpNode(&opNode, OP_TYPE_MATCH, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, true); + nodesDestroyNode(res); } TEST(constantTest, binary_match_binary2) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - char leftv[64] = {0}, rightv[64] = {0}; - sprintf(&leftv[2], "%s", "abc"); - varDataSetLen(leftv, strlen(&leftv[2])); - sprintf(&rightv[2], "%s", "abc.+"); - varDataSetLen(rightv, strlen(&rightv[2])); - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BINARY, leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); - scltMakeOpNode(&opNode, OP_TYPE_MATCH, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, false); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + char leftv[64] = {0}, rightv[64] = {0}; + sprintf(&leftv[2], "%s", "abc"); + varDataSetLen(leftv, strlen(&leftv[2])); + sprintf(&rightv[2], "%s", "abc.+"); + varDataSetLen(rightv, strlen(&rightv[2])); + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BINARY, leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); + scltMakeOpNode(&opNode, OP_TYPE_MATCH, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, false); + nodesDestroyNode(res); } TEST(constantTest, binary_not_match_binary1) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - char leftv[64] = {0}, rightv[64] = {0}; - sprintf(&leftv[2], "%s", "abc"); - varDataSetLen(leftv, strlen(&leftv[2])); - sprintf(&rightv[2], "%s", "a[1-9]c"); - varDataSetLen(rightv, strlen(&rightv[2])); - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BINARY, leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); - scltMakeOpNode(&opNode, OP_TYPE_NMATCH, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, true); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + char leftv[64] = {0}, rightv[64] = {0}; + sprintf(&leftv[2], "%s", "abc"); + varDataSetLen(leftv, strlen(&leftv[2])); + sprintf(&rightv[2], "%s", "a[1-9]c"); + varDataSetLen(rightv, strlen(&rightv[2])); + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BINARY, leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); + scltMakeOpNode(&opNode, OP_TYPE_NMATCH, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, true); + nodesDestroyNode(res); } TEST(constantTest, binary_not_match_binary2) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - char leftv[64] = {0}, rightv[64] = {0}; - sprintf(&leftv[2], "%s", "abc"); - varDataSetLen(leftv, strlen(&leftv[2])); - sprintf(&rightv[2], "%s", "a[ab]c"); - varDataSetLen(rightv, strlen(&rightv[2])); - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BINARY, leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); - scltMakeOpNode(&opNode, OP_TYPE_NMATCH, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, false); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + char leftv[64] = {0}, rightv[64] = {0}; + sprintf(&leftv[2], "%s", "abc"); + varDataSetLen(leftv, strlen(&leftv[2])); + sprintf(&rightv[2], "%s", "a[ab]c"); + varDataSetLen(rightv, strlen(&rightv[2])); + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_BINARY, leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); + scltMakeOpNode(&opNode, OP_TYPE_NMATCH, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, false); + nodesDestroyNode(res); } TEST(constantTest, int_is_null1) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - int32_t leftv = 1, rightv = 1; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); - scltMakeOpNode(&opNode, OP_TYPE_IS_NULL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, false); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + int32_t leftv = 1, rightv = 1; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); + scltMakeOpNode(&opNode, OP_TYPE_IS_NULL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, false); + nodesDestroyNode(res); } TEST(constantTest, int_is_null2) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - int32_t leftv = TSDB_DATA_INT_NULL, rightv = 1; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_NULL, &leftv); - scltMakeOpNode(&opNode, OP_TYPE_IS_NULL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, true); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + int32_t leftv = TSDB_DATA_INT_NULL, rightv = 1; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_NULL, &leftv); + scltMakeOpNode(&opNode, OP_TYPE_IS_NULL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, true); + nodesDestroyNode(res); } TEST(constantTest, int_is_not_null1) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - int32_t leftv = 1, rightv = 1; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); - scltMakeOpNode(&opNode, OP_TYPE_IS_NOT_NULL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, true); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + int32_t leftv = 1, rightv = 1; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); + scltMakeOpNode(&opNode, OP_TYPE_IS_NOT_NULL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, true); + nodesDestroyNode(res); } TEST(constantTest, int_is_not_null2) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - int32_t leftv = 1, rightv = 1; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_NULL, &leftv); - scltMakeOpNode(&opNode, OP_TYPE_IS_NOT_NULL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, false); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + int32_t leftv = 1, rightv = 1; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_NULL, &leftv); + scltMakeOpNode(&opNode, OP_TYPE_IS_NOT_NULL, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, false); + nodesDestroyNode(res); } TEST(constantTest, int_add_int_is_true1) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - int32_t leftv = 1, rightv = 1; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_INT, &rightv); - scltMakeOpNode(&opNode, OP_TYPE_ADD, TSDB_DATA_TYPE_DOUBLE, pLeft, pRight); - scltMakeOpNode(&opNode, OP_TYPE_IS_TRUE, TSDB_DATA_TYPE_BOOL, opNode, NULL); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, true); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + int32_t leftv = 1, rightv = 1; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_INT, &rightv); + scltMakeOpNode(&opNode, OP_TYPE_ADD, TSDB_DATA_TYPE_DOUBLE, pLeft, pRight); + scltMakeOpNode(&opNode, OP_TYPE_IS_TRUE, TSDB_DATA_TYPE_BOOL, opNode, NULL); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, true); + nodesDestroyNode(res); } TEST(constantTest, int_add_int_is_true2) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - int32_t leftv = 1, rightv = -1; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_INT, &rightv); - scltMakeOpNode(&opNode, OP_TYPE_ADD, TSDB_DATA_TYPE_DOUBLE, pLeft, pRight); - scltMakeOpNode(&opNode, OP_TYPE_IS_TRUE, TSDB_DATA_TYPE_BOOL, opNode, NULL); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, false); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + int32_t leftv = 1, rightv = -1; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_INT, &rightv); + scltMakeOpNode(&opNode, OP_TYPE_ADD, TSDB_DATA_TYPE_DOUBLE, pLeft, pRight); + scltMakeOpNode(&opNode, OP_TYPE_IS_TRUE, TSDB_DATA_TYPE_BOOL, opNode, NULL); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, false); + nodesDestroyNode(res); } TEST(constantTest, int_greater_int_is_true1) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - int32_t leftv = 1, rightv = 1; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_INT, &rightv); - scltMakeOpNode(&opNode, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - scltMakeOpNode(&opNode, OP_TYPE_IS_TRUE, TSDB_DATA_TYPE_BOOL, opNode, NULL); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, false); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + int32_t leftv = 1, rightv = 1; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_INT, &rightv); + scltMakeOpNode(&opNode, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + scltMakeOpNode(&opNode, OP_TYPE_IS_TRUE, TSDB_DATA_TYPE_BOOL, opNode, NULL); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, false); + nodesDestroyNode(res); } TEST(constantTest, int_greater_int_is_true2) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; - int32_t leftv = 1, rightv = 0; - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_INT, &rightv); - scltMakeOpNode(&opNode, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - scltMakeOpNode(&opNode, OP_TYPE_IS_TRUE, TSDB_DATA_TYPE_BOOL, opNode, NULL); - - int32_t code = scalarCalculateConstants(opNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, true); - nodesDestroyNode(res); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL, *res = NULL; + int32_t leftv = 1, rightv = 0; + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_INT, &rightv); + scltMakeOpNode(&opNode, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + scltMakeOpNode(&opNode, OP_TYPE_IS_TRUE, TSDB_DATA_TYPE_BOOL, opNode, NULL); + + int32_t code = scalarCalculateConstants(opNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, true); + nodesDestroyNode(res); } TEST(constantTest, greater_and_lower) { - scltInitLogFile(); + scltInitLogFile(); - SNode *pval1 = NULL, *pval2 = NULL, *opNode1 = NULL, *opNode2 = NULL, *logicNode = NULL, *res = NULL; - bool eRes[5] = {false, false, true, true, true}; - int64_t v1 = 333, v2 = 222, v3 = -10, v4 = 20; - SNode *list[2] = {0}; - scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v1); - scltMakeValueNode(&pval2, TSDB_DATA_TYPE_BIGINT, &v2); - scltMakeOpNode(&opNode1, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); - scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v3); - scltMakeValueNode(&pval2, TSDB_DATA_TYPE_BIGINT, &v4); - scltMakeOpNode(&opNode2, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); - list[0] = opNode1; - list[1] = opNode2; - scltMakeLogicNode(&logicNode, LOGIC_COND_TYPE_AND, list, 2); - - int32_t code = scalarCalculateConstants(logicNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, true); - nodesDestroyNode(res); + SNode *pval1 = NULL, *pval2 = NULL, *opNode1 = NULL, *opNode2 = NULL, *logicNode = NULL, *res = NULL; + bool eRes[5] = {false, false, true, true, true}; + int64_t v1 = 333, v2 = 222, v3 = -10, v4 = 20; + SNode *list[2] = {0}; + scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v1); + scltMakeValueNode(&pval2, TSDB_DATA_TYPE_BIGINT, &v2); + scltMakeOpNode(&opNode1, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); + scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v3); + scltMakeValueNode(&pval2, TSDB_DATA_TYPE_BIGINT, &v4); + scltMakeOpNode(&opNode2, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); + list[0] = opNode1; + list[1] = opNode2; + scltMakeLogicNode(&logicNode, LOGIC_COND_TYPE_AND, list, 2); + + int32_t code = scalarCalculateConstants(logicNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, true); + nodesDestroyNode(res); } TEST(constantTest, column_and_value1) { - scltInitLogFile(); + scltInitLogFile(); - SNode *pval1 = NULL, *pval2 = NULL, *opNode1 = NULL, *opNode2 = NULL, *logicNode = NULL, *res = NULL; - bool eRes[5] = {false, false, true, true, true}; - int64_t v1 = 333, v2 = 222, v3 = -10, v4 = 20; - SNode *list[2] = {0}; - scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v1); - scltMakeValueNode(&pval2, TSDB_DATA_TYPE_BIGINT, &v2); - scltMakeOpNode(&opNode1, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); - scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v3); - scltMakeColumnNode(&pval2, NULL, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t), 0, NULL); - scltMakeOpNode(&opNode2, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); - list[0] = opNode1; - list[1] = opNode2; - scltMakeLogicNode(&logicNode, LOGIC_COND_TYPE_AND, list, 2); - - int32_t code = scalarCalculateConstants(logicNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_LOGIC_CONDITION); - SLogicConditionNode *v = (SLogicConditionNode *)res; - ASSERT_EQ(v->condType, LOGIC_COND_TYPE_AND); - ASSERT_EQ(v->pParameterList->length, 1); - nodesDestroyNode(res); + SNode *pval1 = NULL, *pval2 = NULL, *opNode1 = NULL, *opNode2 = NULL, *logicNode = NULL, *res = NULL; + bool eRes[5] = {false, false, true, true, true}; + int64_t v1 = 333, v2 = 222, v3 = -10, v4 = 20; + SNode *list[2] = {0}; + scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v1); + scltMakeValueNode(&pval2, TSDB_DATA_TYPE_BIGINT, &v2); + scltMakeOpNode(&opNode1, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); + scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v3); + scltMakeColumnNode(&pval2, NULL, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t), 0, NULL); + scltMakeOpNode(&opNode2, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); + list[0] = opNode1; + list[1] = opNode2; + scltMakeLogicNode(&logicNode, LOGIC_COND_TYPE_AND, list, 2); + + int32_t code = scalarCalculateConstants(logicNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_LOGIC_CONDITION); + SLogicConditionNode *v = (SLogicConditionNode *)res; + ASSERT_EQ(v->condType, LOGIC_COND_TYPE_AND); + ASSERT_EQ(v->pParameterList->length, 1); + nodesDestroyNode(res); } TEST(constantTest, column_and_value2) { - scltInitLogFile(); + scltInitLogFile(); - SNode *pval1 = NULL, *pval2 = NULL, *opNode1 = NULL, *opNode2 = NULL, *logicNode = NULL, *res = NULL; - bool eRes[5] = {false, false, true, true, true}; - int64_t v1 = 333, v2 = 222, v3 = -10, v4 = 20; - SNode *list[2] = {0}; - scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v1); - scltMakeValueNode(&pval2, TSDB_DATA_TYPE_BIGINT, &v2); - scltMakeOpNode(&opNode1, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); - scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v3); - scltMakeColumnNode(&pval2, NULL, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t), 0, NULL); - scltMakeOpNode(&opNode2, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); - list[0] = opNode1; - list[1] = opNode2; - scltMakeLogicNode(&logicNode, LOGIC_COND_TYPE_AND, list, 2); - - int32_t code = scalarCalculateConstants(logicNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, false); - nodesDestroyNode(res); + SNode *pval1 = NULL, *pval2 = NULL, *opNode1 = NULL, *opNode2 = NULL, *logicNode = NULL, *res = NULL; + bool eRes[5] = {false, false, true, true, true}; + int64_t v1 = 333, v2 = 222, v3 = -10, v4 = 20; + SNode *list[2] = {0}; + scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v1); + scltMakeValueNode(&pval2, TSDB_DATA_TYPE_BIGINT, &v2); + scltMakeOpNode(&opNode1, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); + scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v3); + scltMakeColumnNode(&pval2, NULL, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t), 0, NULL); + scltMakeOpNode(&opNode2, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); + list[0] = opNode1; + list[1] = opNode2; + scltMakeLogicNode(&logicNode, LOGIC_COND_TYPE_AND, list, 2); + + int32_t code = scalarCalculateConstants(logicNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, false); + nodesDestroyNode(res); } TEST(constantTest, column_and_value3) { - scltInitLogFile(); + scltInitLogFile(); - SNode *pval1 = NULL, *pval2 = NULL, *opNode1 = NULL, *opNode2 = NULL, *logicNode = NULL, *res = NULL; - bool eRes[5] = {false, false, true, true, true}; - int64_t v1 = 333, v2 = 222, v3 = -10, v4 = 20; - SNode *list[2] = {0}; - scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v1); - scltMakeValueNode(&pval2, TSDB_DATA_TYPE_BIGINT, &v2); - scltMakeOpNode(&opNode1, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); - scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v3); - scltMakeColumnNode(&pval2, NULL, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t), 0, NULL); - scltMakeOpNode(&opNode2, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); - list[0] = opNode1; - list[1] = opNode2; - scltMakeLogicNode(&logicNode, LOGIC_COND_TYPE_OR, list, 2); - - int32_t code = scalarCalculateConstants(logicNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); - SValueNode *v = (SValueNode *)res; - ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(v->datum.b, true); - nodesDestroyNode(res); + SNode *pval1 = NULL, *pval2 = NULL, *opNode1 = NULL, *opNode2 = NULL, *logicNode = NULL, *res = NULL; + bool eRes[5] = {false, false, true, true, true}; + int64_t v1 = 333, v2 = 222, v3 = -10, v4 = 20; + SNode *list[2] = {0}; + scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v1); + scltMakeValueNode(&pval2, TSDB_DATA_TYPE_BIGINT, &v2); + scltMakeOpNode(&opNode1, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); + scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v3); + scltMakeColumnNode(&pval2, NULL, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t), 0, NULL); + scltMakeOpNode(&opNode2, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); + list[0] = opNode1; + list[1] = opNode2; + scltMakeLogicNode(&logicNode, LOGIC_COND_TYPE_OR, list, 2); + + int32_t code = scalarCalculateConstants(logicNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_VALUE); + SValueNode *v = (SValueNode *)res; + ASSERT_EQ(v->node.resType.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(v->datum.b, true); + nodesDestroyNode(res); } TEST(constantTest, column_and_value4) { - scltInitLogFile(); + scltInitLogFile(); - SNode *pval1 = NULL, *pval2 = NULL, *opNode1 = NULL, *opNode2 = NULL, *logicNode = NULL, *res = NULL; - bool eRes[5] = {false, false, true, true, true}; - int64_t v1 = 333, v2 = 222, v3 = -10, v4 = 20; - SNode *list[2] = {0}; - scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v1); - scltMakeValueNode(&pval2, TSDB_DATA_TYPE_BIGINT, &v2); - scltMakeOpNode(&opNode1, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); - scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v3); - scltMakeColumnNode(&pval2, NULL, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t), 0, NULL); - scltMakeOpNode(&opNode2, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); - list[0] = opNode1; - list[1] = opNode2; - scltMakeLogicNode(&logicNode, LOGIC_COND_TYPE_OR, list, 2); - - int32_t code = scalarCalculateConstants(logicNode, &res); - ASSERT_EQ(code, 0); - ASSERT_TRUE(res); - ASSERT_EQ(nodeType(res), QUERY_NODE_LOGIC_CONDITION); - SLogicConditionNode *v = (SLogicConditionNode *)res; - ASSERT_EQ(v->condType, LOGIC_COND_TYPE_OR); - ASSERT_EQ(v->pParameterList->length, 1); - nodesDestroyNode(res); + SNode *pval1 = NULL, *pval2 = NULL, *opNode1 = NULL, *opNode2 = NULL, *logicNode = NULL, *res = NULL; + bool eRes[5] = {false, false, true, true, true}; + int64_t v1 = 333, v2 = 222, v3 = -10, v4 = 20; + SNode *list[2] = {0}; + scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v1); + scltMakeValueNode(&pval2, TSDB_DATA_TYPE_BIGINT, &v2); + scltMakeOpNode(&opNode1, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); + scltMakeValueNode(&pval1, TSDB_DATA_TYPE_BIGINT, &v3); + scltMakeColumnNode(&pval2, NULL, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t), 0, NULL); + scltMakeOpNode(&opNode2, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pval1, pval2); + list[0] = opNode1; + list[1] = opNode2; + scltMakeLogicNode(&logicNode, LOGIC_COND_TYPE_OR, list, 2); + + int32_t code = scalarCalculateConstants(logicNode, &res); + ASSERT_EQ(code, 0); + ASSERT_TRUE(res); + ASSERT_EQ(nodeType(res), QUERY_NODE_LOGIC_CONDITION); + SLogicConditionNode *v = (SLogicConditionNode *)res; + ASSERT_EQ(v->condType, LOGIC_COND_TYPE_OR); + ASSERT_EQ(v->pParameterList->length, 1); + nodesDestroyNode(res); } void makeJsonArrow(SSDataBlock **src, SNode **opNode, void *json, char *key){ - char keyVar[32] = {0}; - memcpy(varDataVal(keyVar), key, strlen(key)); - varDataLen(keyVar) = strlen(key); + char keyVar[32] = {0}; + memcpy(varDataVal(keyVar), key, strlen(key)); + varDataLen(keyVar) = strlen(key); - SNode *pLeft = NULL, *pRight = NULL; - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, keyVar); - scltMakeColumnNode(&pLeft, src, TSDB_DATA_TYPE_JSON, ((STag*)json)->len, 1, json); - scltMakeOpNode(opNode, OP_TYPE_JSON_GET_VALUE, TSDB_DATA_TYPE_JSON, pLeft, pRight); + SNode *pLeft = NULL, *pRight = NULL; + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, keyVar); + scltMakeColumnNode(&pLeft, src, TSDB_DATA_TYPE_JSON, ((STag*)json)->len, 1, json); + scltMakeOpNode(opNode, OP_TYPE_JSON_GET_VALUE, TSDB_DATA_TYPE_JSON, pLeft, pRight); } void makeOperator(SNode **opNode, SArray *blockList, EOperatorType opType, int32_t rightType, void *rightData, bool isReverse){ - int32_t resType = TSDB_DATA_TYPE_NULL; - if(opType == OP_TYPE_ADD || opType == OP_TYPE_SUB || opType == OP_TYPE_MULTI || - opType == OP_TYPE_DIV || opType == OP_TYPE_REM || opType == OP_TYPE_MINUS){ - resType = TSDB_DATA_TYPE_DOUBLE; - }else if(opType == OP_TYPE_BIT_AND || opType == OP_TYPE_BIT_OR){ - resType = TSDB_DATA_TYPE_BIGINT; - }else if(opType == OP_TYPE_GREATER_THAN || opType == OP_TYPE_GREATER_EQUAL || - opType == OP_TYPE_LOWER_THAN || opType == OP_TYPE_LOWER_EQUAL || - opType == OP_TYPE_EQUAL || opType == OP_TYPE_NOT_EQUAL || - opType == OP_TYPE_IS_NULL || opType == OP_TYPE_IS_NOT_NULL || opType == OP_TYPE_IS_TRUE || - opType == OP_TYPE_LIKE || opType == OP_TYPE_NOT_LIKE || opType == OP_TYPE_MATCH || - opType == OP_TYPE_NMATCH){ - resType = TSDB_DATA_TYPE_BOOL; - } + int32_t resType = TSDB_DATA_TYPE_NULL; + if(opType == OP_TYPE_ADD || opType == OP_TYPE_SUB || opType == OP_TYPE_MULTI || + opType == OP_TYPE_DIV || opType == OP_TYPE_REM || opType == OP_TYPE_MINUS){ + resType = TSDB_DATA_TYPE_DOUBLE; + }else if(opType == OP_TYPE_BIT_AND || opType == OP_TYPE_BIT_OR){ + resType = TSDB_DATA_TYPE_BIGINT; + }else if(opType == OP_TYPE_GREATER_THAN || opType == OP_TYPE_GREATER_EQUAL || + opType == OP_TYPE_LOWER_THAN || opType == OP_TYPE_LOWER_EQUAL || + opType == OP_TYPE_EQUAL || opType == OP_TYPE_NOT_EQUAL || + opType == OP_TYPE_IS_NULL || opType == OP_TYPE_IS_NOT_NULL || opType == OP_TYPE_IS_TRUE || + opType == OP_TYPE_LIKE || opType == OP_TYPE_NOT_LIKE || opType == OP_TYPE_MATCH || + opType == OP_TYPE_NMATCH){ + resType = TSDB_DATA_TYPE_BOOL; + } - SNode *right = NULL; - scltMakeValueNode(&right, rightType, rightData); - scltMakeOpNode2(opNode, opType, resType, *opNode, right, isReverse); + SNode *right = NULL; + scltMakeValueNode(&right, rightType, rightData); + scltMakeOpNode2(opNode, opType, resType, *opNode, right, isReverse); - SColumnInfo colInfo = createColumnInfo(1, resType, tDataTypes[resType].bytes); - int16_t dataBlockId = 0, slotId = 0; - scltAppendReservedSlot(blockList, &dataBlockId, &slotId, true, 1, &colInfo); - scltMakeTargetNode(opNode, dataBlockId, slotId, *opNode); + SColumnInfo colInfo = createColumnInfo(1, resType, tDataTypes[resType].bytes); + int16_t dataBlockId = 0, slotId = 0; + scltAppendReservedSlot(blockList, &dataBlockId, &slotId, true, 1, &colInfo); + scltMakeTargetNode(opNode, dataBlockId, slotId, *opNode); } void makeCalculate(void *json, void *key, int32_t rightType, void *rightData, double exceptValue, EOperatorType opType, bool isReverse){ - SArray *blockList = taosArrayInit(2, POINTER_BYTES); - SSDataBlock *src = NULL; - SNode *opNode = NULL; + SArray *blockList = taosArrayInit(2, POINTER_BYTES); + SSDataBlock *src = NULL; + SNode *opNode = NULL; - makeJsonArrow(&src, &opNode, json, (char*)key); - taosArrayPush(blockList, &src); + makeJsonArrow(&src, &opNode, json, (char*)key); + taosArrayPush(blockList, &src); - makeOperator(&opNode, blockList, opType, rightType, rightData, isReverse); + makeOperator(&opNode, blockList, opType, rightType, rightData, isReverse); - int32_t code = scalarCalculate(opNode, blockList, NULL); - ASSERT_EQ(code, 0); + int32_t code = scalarCalculate(opNode, blockList, NULL); + ASSERT_EQ(code, 0); - SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); - ASSERT_EQ(res->info.rows, 1); - SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); + SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); + ASSERT_EQ(res->info.rows, 1); + SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); - if(colDataIsNull_f(column->nullbitmap, 0)){ - ASSERT_EQ(DBL_MAX, exceptValue); - printf("result:NULL\n"); + if(colDataIsNull_f(column->nullbitmap, 0)){ + ASSERT_EQ(DBL_MAX, exceptValue); + printf("result:NULL\n"); - }else if(opType == OP_TYPE_ADD || opType == OP_TYPE_SUB || opType == OP_TYPE_MULTI || opType == OP_TYPE_DIV || - opType == OP_TYPE_REM || opType == OP_TYPE_MINUS){ - printf("op:%s,1result:%f,except:%f\n", gOptrStr[opType].str, *((double *)colDataGetData(column, 0)), exceptValue); - ASSERT_TRUE(fabs(*((double *)colDataGetData(column, 0)) - exceptValue) < 0.0001); - }else if(opType == OP_TYPE_BIT_AND || opType == OP_TYPE_BIT_OR){ - printf("op:%s,2result:%ld,except:%f\n", gOptrStr[opType].str, *((int64_t *)colDataGetData(column, 0)), exceptValue); - ASSERT_EQ(*((int64_t *)colDataGetData(column, 0)), exceptValue); - }else if(opType == OP_TYPE_GREATER_THAN || opType == OP_TYPE_GREATER_EQUAL || opType == OP_TYPE_LOWER_THAN || - opType == OP_TYPE_LOWER_EQUAL || opType == OP_TYPE_EQUAL || opType == OP_TYPE_NOT_EQUAL || - opType == OP_TYPE_IS_NULL || opType == OP_TYPE_IS_NOT_NULL || opType == OP_TYPE_IS_TRUE || - opType == OP_TYPE_LIKE || opType == OP_TYPE_NOT_LIKE || opType == OP_TYPE_MATCH || opType == OP_TYPE_NMATCH){ - printf("op:%s,3result:%d,except:%f\n", gOptrStr[opType].str, *((bool *)colDataGetData(column, 0)), exceptValue); - ASSERT_EQ(*((bool *)colDataGetData(column, 0)), exceptValue); - } + }else if(opType == OP_TYPE_ADD || opType == OP_TYPE_SUB || opType == OP_TYPE_MULTI || opType == OP_TYPE_DIV || + opType == OP_TYPE_REM || opType == OP_TYPE_MINUS){ + printf("op:%s,1result:%f,except:%f\n", gOptrStr[opType].str, *((double *)colDataGetData(column, 0)), exceptValue); + ASSERT_TRUE(fabs(*((double *)colDataGetData(column, 0)) - exceptValue) < 0.0001); + }else if(opType == OP_TYPE_BIT_AND || opType == OP_TYPE_BIT_OR){ + printf("op:%s,2result:%ld,except:%f\n", gOptrStr[opType].str, *((int64_t *)colDataGetData(column, 0)), exceptValue); + ASSERT_EQ(*((int64_t *)colDataGetData(column, 0)), exceptValue); + }else if(opType == OP_TYPE_GREATER_THAN || opType == OP_TYPE_GREATER_EQUAL || opType == OP_TYPE_LOWER_THAN || + opType == OP_TYPE_LOWER_EQUAL || opType == OP_TYPE_EQUAL || opType == OP_TYPE_NOT_EQUAL || + opType == OP_TYPE_IS_NULL || opType == OP_TYPE_IS_NOT_NULL || opType == OP_TYPE_IS_TRUE || + opType == OP_TYPE_LIKE || opType == OP_TYPE_NOT_LIKE || opType == OP_TYPE_MATCH || opType == OP_TYPE_NMATCH){ + printf("op:%s,3result:%d,except:%f\n", gOptrStr[opType].str, *((bool *)colDataGetData(column, 0)), exceptValue); + ASSERT_EQ(*((bool *)colDataGetData(column, 0)), exceptValue); + } - taosArrayDestroyEx(blockList, scltFreeDataBlock); - nodesDestroyNode(opNode); + taosArrayDestroyEx(blockList, scltFreeDataBlock); + nodesDestroyNode(opNode); } TEST(columnTest, json_column_arith_op) { - scltInitLogFile(); - char *rightvTmp= "{\"k1\":4,\"k2\":\"hello\",\"k3\":null,\"k4\":true,\"k5\":5.44,\"k6\":-10,\"k7\":-9.8,\"k8\":false,\"k9\":\"8hel\"}"; + scltInitLogFile(); + char *rightvTmp= "{\"k1\":4,\"k2\":\"hello\",\"k3\":null,\"k4\":true,\"k5\":5.44,\"k6\":-10,\"k7\":-9.8,\"k8\":false,\"k9\":\"8hel\"}"; - char rightv[256] = {0}; - memcpy(rightv, rightvTmp, strlen(rightvTmp)); - SArray *tags = taosArrayInit(1, sizeof(STagVal)); - STag* row = NULL; - parseJsontoTagData(rightv, tags, &row, NULL); + char rightv[256] = {0}; + memcpy(rightv, rightvTmp, strlen(rightvTmp)); + SArray *tags = taosArrayInit(1, sizeof(STagVal)); + STag* row = NULL; + parseJsontoTagData(rightv, tags, &row, NULL); - const int32_t len = 8; - EOperatorType op[len] = {OP_TYPE_ADD, OP_TYPE_SUB, OP_TYPE_MULTI, OP_TYPE_DIV, - OP_TYPE_REM, OP_TYPE_MINUS, OP_TYPE_BIT_AND, OP_TYPE_BIT_OR}; - int32_t input[len] = {1, 8, 2, 2, 3, 0, -4, 9}; + const int32_t len = 8; + EOperatorType op[len] = {OP_TYPE_ADD, OP_TYPE_SUB, OP_TYPE_MULTI, OP_TYPE_DIV, + OP_TYPE_REM, OP_TYPE_MINUS, OP_TYPE_BIT_AND, OP_TYPE_BIT_OR}; + int32_t input[len] = {1, 8, 2, 2, 3, 0, -4, 9}; - printf("--------------------json int-4 op {1, 8, 2, 2, 3, 0, -4, 9}--------------------\n"); - char *key = "k1"; - double eRes00[len] = {5.0, -4, 8.0, 2.0, 1.0, -4, 4&-4, 4|9}; - double eRes01[len] = {5.0, 4, 8.0, 0.5, 3, 0, 4&-4, 4|9}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes00[i], op[i], false); - } - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes01[i], op[i], true); - } + printf("--------------------json int-4 op {1, 8, 2, 2, 3, 0, -4, 9}--------------------\n"); + char *key = "k1"; + double eRes00[len] = {5.0, -4, 8.0, 2.0, 1.0, -4, 4&-4, 4|9}; + double eRes01[len] = {5.0, 4, 8.0, 0.5, 3, 0, 4&-4, 4|9}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes00[i], op[i], false); + } + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes01[i], op[i], true); + } - printf("--------------------json string- 0 op {1, 8, 2, 2, 3, 0, -4, 9}--------------------\n"); + printf("--------------------json string- 0 op {1, 8, 2, 2, 3, 0, -4, 9}--------------------\n"); - key = "k2"; - double eRes10[len] = {1.0, -8, 0, 0, 0, 0, 0, 9}; - double eRes11[len] = {1.0, 8, 0, DBL_MAX, DBL_MAX, 0, 0, 9}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes10[i], op[i], false); - } - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes11[i], op[i], true); - } + key = "k2"; + double eRes10[len] = {1.0, -8, 0, 0, 0, 0, 0, 9}; + double eRes11[len] = {1.0, 8, 0, DBL_MAX, DBL_MAX, 0, 0, 9}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes10[i], op[i], false); + } + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes11[i], op[i], true); + } - printf("---------------------json null- null op {1, 8, 2, 2, 3, 0, -4, 9}-------------------\n"); + printf("---------------------json null- null op {1, 8, 2, 2, 3, 0, -4, 9}-------------------\n"); - key = "k3"; - double eRes20[len] = {DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX}; - double eRes21[len] = {DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, 0, DBL_MAX, DBL_MAX}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes20[i], op[i], false); - } - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes21[i], op[i], true); - } + key = "k3"; + double eRes20[len] = {DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX}; + double eRes21[len] = {DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, 0, DBL_MAX, DBL_MAX}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes20[i], op[i], false); + } + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes21[i], op[i], true); + } - printf("---------------------json bool- true op {1, 8, 2, 2, 3, 0, -4, 9}-------------------\n"); + printf("---------------------json bool- true op {1, 8, 2, 2, 3, 0, -4, 9}-------------------\n"); - key = "k4"; - double eRes30[len] = {2.0, -7, 2, 0.5, 1, -1, 1&-4, 1|9}; - double eRes31[len] = {2.0, 7, 2, 2, 0, 0, 1&-4, 1|9}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes30[i], op[i], false); - } - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes31[i], op[i], true); - } + key = "k4"; + double eRes30[len] = {2.0, -7, 2, 0.5, 1, -1, 1&-4, 1|9}; + double eRes31[len] = {2.0, 7, 2, 2, 0, 0, 1&-4, 1|9}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes30[i], op[i], false); + } + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes31[i], op[i], true); + } - printf("----------------------json double-- 5.44 op {1, 8, 2, 2, 3, 0, -4, 9}------------------\n"); + printf("----------------------json double-- 5.44 op {1, 8, 2, 2, 3, 0, -4, 9}------------------\n"); - key = "k5"; - double eRes40[len] = {6.44, -2.56, 10.88, 2.72, 2.44, -5.44, 5&-4, 5|9}; - double eRes41[len] = {6.44, 2.56, 10.88, 0.3676470588235294, 3, 0, 5&-4, 5|9}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes40[i], op[i], false); - } - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes41[i], op[i], true); - } + key = "k5"; + double eRes40[len] = {6.44, -2.56, 10.88, 2.72, 2.44, -5.44, 5&-4, 5|9}; + double eRes41[len] = {6.44, 2.56, 10.88, 0.3676470588235294, 3, 0, 5&-4, 5|9}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes40[i], op[i], false); + } + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes41[i], op[i], true); + } - printf("----------------------json int-- -10 op {1, 8, 2, 2, 3, 0, -4, 9}------------------\n"); + printf("----------------------json int-- -10 op {1, 8, 2, 2, 3, 0, -4, 9}------------------\n"); - key = "k6"; - double eRes50[len] = {-9, -18, -20, -5, -10%3, 10, -10&-4, -10|9}; - double eRes51[len] = {-9, 18, -20, -0.2, 3%-10, 0, -10&-4, -10|9}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes50[i], op[i], false); - } - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes51[i], op[i], true); - } + key = "k6"; + double eRes50[len] = {-9, -18, -20, -5, -10%3, 10, -10&-4, -10|9}; + double eRes51[len] = {-9, 18, -20, -0.2, 3%-10, 0, -10&-4, -10|9}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes50[i], op[i], false); + } + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes51[i], op[i], true); + } - printf("----------------------json double-- -9.8 op {1, 8, 2, 2, 3, 0, -4, 9}------------------\n"); + printf("----------------------json double-- -9.8 op {1, 8, 2, 2, 3, 0, -4, 9}------------------\n"); - key = "k7"; - double eRes60[len] = {-8.8, -17.8, -19.6, -4.9, -0.8, 9.8, -9&-4, -9|9}; - double eRes61[len] = {-8.8, 17.8, -19.6, -0.2040816326530612, 3, 0, -9&-4, -9|9}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes60[i], op[i], false); - } - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes61[i], op[i], true); - } + key = "k7"; + double eRes60[len] = {-8.8, -17.8, -19.6, -4.9, -0.8, 9.8, -9&-4, -9|9}; + double eRes61[len] = {-8.8, 17.8, -19.6, -0.2040816326530612, 3, 0, -9&-4, -9|9}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes60[i], op[i], false); + } + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes61[i], op[i], true); + } - printf("----------------------json bool-- 0 op {1, 8, 2, 2, 3, 0, -4, 9}------------------\n"); + printf("----------------------json bool-- 0 op {1, 8, 2, 2, 3, 0, -4, 9}------------------\n"); - key = "k8"; - double eRes70[len] = {1.0, -8, 0, 0, 0, 0, 0, 9}; - double eRes71[len] = {1.0, 8, 0, DBL_MAX, DBL_MAX, 0, 0, 9}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes70[i], op[i], false); - } - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes71[i], op[i], true); - } + key = "k8"; + double eRes70[len] = {1.0, -8, 0, 0, 0, 0, 0, 9}; + double eRes71[len] = {1.0, 8, 0, DBL_MAX, DBL_MAX, 0, 0, 9}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes70[i], op[i], false); + } + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes71[i], op[i], true); + } - printf("----------------------json string-- 8 op {1, 8, 2, 2, 3, 0, -4, 9}------------------\n"); + printf("----------------------json string-- 8 op {1, 8, 2, 2, 3, 0, -4, 9}------------------\n"); - key = "k9"; - double eRes80[len] = {9, 0, 16, 4, 8%3, -8, 8&-4, 8|9}; - double eRes81[len] = {9, 0, 16, 0.25, 3%8, 0, 8&-4, 8|9}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes80[i], op[i], false); - } - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes81[i], op[i], true); - } + key = "k9"; + double eRes80[len] = {9, 0, 16, 4, 8%3, -8, 8&-4, 8|9}; + double eRes81[len] = {9, 0, 16, 0.25, 3%8, 0, 8&-4, 8|9}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes80[i], op[i], false); + } + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes81[i], op[i], true); + } - printf("---------------------json not exist-- NULL op {1, 8, 2, 2, 3, 0, -4, 9}------------------\n"); + printf("---------------------json not exist-- NULL op {1, 8, 2, 2, 3, 0, -4, 9}------------------\n"); - key = "k10"; - double eRes90[len] = {DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX}; - double eRes91[len] = {DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, 0, DBL_MAX, DBL_MAX}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes90[i], op[i], false); - } - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes91[i], op[i], true); - } + key = "k10"; + double eRes90[len] = {DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX}; + double eRes91[len] = {DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, DBL_MAX, 0, DBL_MAX, DBL_MAX}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes90[i], op[i], false); + } + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes91[i], op[i], true); + } - taosArrayDestroy(tags); - taosMemoryFree(row); + taosArrayDestroy(tags); + taosMemoryFree(row); } void *prepareNchar(char* rightData){ - int32_t len = 0; - int32_t inputLen = strlen(rightData); + int32_t len = 0; + int32_t inputLen = strlen(rightData); - char* t = (char*)taosMemoryCalloc(1,(inputLen + 1) * TSDB_NCHAR_SIZE + VARSTR_HEADER_SIZE); - taosMbsToUcs4(rightData, inputLen, (TdUcs4*) varDataVal(t), inputLen * TSDB_NCHAR_SIZE, &len); - varDataSetLen(t, len); - return t; + char* t = (char*)taosMemoryCalloc(1,(inputLen + 1) * TSDB_NCHAR_SIZE + VARSTR_HEADER_SIZE); + taosMbsToUcs4(rightData, inputLen, (TdUcs4*) varDataVal(t), inputLen * TSDB_NCHAR_SIZE, &len); + varDataSetLen(t, len); + return t; } TEST(columnTest, json_column_logic_op) { - scltInitLogFile(); - char *rightvTmp= "{\"k1\":4,\"k2\":\"hello\",\"k3\":null,\"k4\":true,\"k5\":5.44,\"k6\":-10,\"k7\":-9.8,\"k8\":false,\"k9\":\"6.6hello\"}"; + scltInitLogFile(); + char *rightvTmp= "{\"k1\":4,\"k2\":\"hello\",\"k3\":null,\"k4\":true,\"k5\":5.44,\"k6\":-10,\"k7\":-9.8,\"k8\":false,\"k9\":\"6.6hello\"}"; - char rightv[256] = {0}; - memcpy(rightv, rightvTmp, strlen(rightvTmp)); - SArray *tags = taosArrayInit(1, sizeof(STagVal)); - STag* row = NULL; - parseJsontoTagData(rightv, tags, &row, NULL); + char rightv[256] = {0}; + memcpy(rightv, rightvTmp, strlen(rightvTmp)); + SArray *tags = taosArrayInit(1, sizeof(STagVal)); + STag* row = NULL; + parseJsontoTagData(rightv, tags, &row, NULL); - const int32_t len0 = 6; - const int32_t len = 9; - const int32_t len1 = 4; - EOperatorType op[len+len1] = {OP_TYPE_GREATER_THAN, OP_TYPE_GREATER_EQUAL, OP_TYPE_LOWER_THAN, OP_TYPE_LOWER_EQUAL, OP_TYPE_EQUAL, OP_TYPE_NOT_EQUAL, - OP_TYPE_IS_NULL, OP_TYPE_IS_NOT_NULL, OP_TYPE_IS_TRUE, OP_TYPE_LIKE, OP_TYPE_NOT_LIKE, OP_TYPE_MATCH, OP_TYPE_NMATCH}; + const int32_t len0 = 6; + const int32_t len = 9; + const int32_t len1 = 4; + EOperatorType op[len+len1] = {OP_TYPE_GREATER_THAN, OP_TYPE_GREATER_EQUAL, OP_TYPE_LOWER_THAN, OP_TYPE_LOWER_EQUAL, OP_TYPE_EQUAL, OP_TYPE_NOT_EQUAL, + OP_TYPE_IS_NULL, OP_TYPE_IS_NOT_NULL, OP_TYPE_IS_TRUE, OP_TYPE_LIKE, OP_TYPE_NOT_LIKE, OP_TYPE_MATCH, OP_TYPE_NMATCH}; - int32_t input[len] = {1, 8, 2, 2, 3, 0, 0, 0, 0}; - char *inputNchar[len1] = {"hell_", "hel%", "hell", "llll"}; + int32_t input[len] = {1, 8, 2, 2, 3, 0, 0, 0, 0}; + char *inputNchar[len1] = {"hell_", "hel%", "hell", "llll"}; - printf("--------------------json int---4 {1, 8, 2, 2, 3, 0, 0, 0, 0}------------------\n"); - char *key = "k1"; - bool eRes[len+len1] = {true, false, false, false, false, true, false, true, true, false, false, false, false}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes[i], op[i], false); - } - bool eRes_0[len0] = {false, true, true, true, false, true}; - for(int i = 0; i < len0; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_0[i], op[i], true); - } - for(int i = len; i < len + len1; i++){ - void* rightData = prepareNchar(inputNchar[i-len]); - makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes[i], op[i], false); - taosMemoryFree(rightData); - } + printf("--------------------json int---4 {1, 8, 2, 2, 3, 0, 0, 0, 0}------------------\n"); + char *key = "k1"; + bool eRes[len+len1] = {true, false, false, false, false, true, false, true, true, false, false, false, false}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes[i], op[i], false); + } + bool eRes_0[len0] = {false, true, true, true, false, true}; + for(int i = 0; i < len0; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_0[i], op[i], true); + } + for(int i = len; i < len + len1; i++){ + void* rightData = prepareNchar(inputNchar[i-len]); + makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes[i], op[i], false); + taosMemoryFree(rightData); + } - printf("--------------------json string--0 {1, 8, 2, 2, 3, 0, 0, 0, 0}-------------------\n"); + printf("--------------------json string--0 {1, 8, 2, 2, 3, 0, 0, 0, 0}-------------------\n"); - key = "k2"; - bool eRes1[len+len1] = {false, false, true, true, false, false, false, true, false, true, false, true, true}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes1[i], op[i], false); - } - bool eRes_1[len0] = {true, true, false, false, false, false}; - for(int i = 0; i < len0; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_1[i], op[i], true); - } + key = "k2"; + bool eRes1[len+len1] = {false, false, true, true, false, false, false, true, false, true, false, true, true}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes1[i], op[i], false); + } + bool eRes_1[len0] = {true, true, false, false, false, false}; + for(int i = 0; i < len0; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_1[i], op[i], true); + } - for(int i = len; i < len + len1; i++){ - void* rightData = prepareNchar(inputNchar[i-len]); - makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes1[i], op[i], false); - taosMemoryFree(rightData); - } + for(int i = len; i < len + len1; i++){ + void* rightData = prepareNchar(inputNchar[i-len]); + makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes1[i], op[i], false); + taosMemoryFree(rightData); + } - printf("--------------------json null---null {1, 8, 2, 2, 3, 0, 0, 0, 0}------------------\n"); + printf("--------------------json null---null {1, 8, 2, 2, 3, 0, 0, 0, 0}------------------\n"); - key = "k3"; // (null is true) return NULL, so use DBL_MAX represent NULL - double eRes2[len+len1] = {false, false, false, false, false, false, true, false, DBL_MAX, false, false, false, false}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes2[i], op[i], false); - } - bool eRes_2[len0] = {false, false, false, false, false, false}; - for(int i = 0; i < len0; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_2[i], op[i], true); - } + key = "k3"; // (null is true) return NULL, so use DBL_MAX represent NULL + bool eRes2[len+len1] = {false, false, false, false, false, false, true, false, false, false, false, false, false}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes2[i], op[i], false); + } + bool eRes_2[len0] = {false, false, false, false, false, false}; + for(int i = 0; i < len0; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_2[i], op[i], true); + } - for(int i = len; i < len + len1; i++){ - void* rightData = prepareNchar(inputNchar[i-len]); - makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes2[i], op[i], false); - taosMemoryFree(rightData); - } + for(int i = len; i < len + len1; i++){ + void* rightData = prepareNchar(inputNchar[i-len]); + makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes2[i], op[i], false); + taosMemoryFree(rightData); + } - printf("--------------------json bool--1 {1, 8, 2, 2, 3, 0, 0, 0, 0}-------------------\n"); + printf("--------------------json bool--1 {1, 8, 2, 2, 3, 0, 0, 0, 0}-------------------\n"); - key = "k4"; - bool eRes3[len+len1] = {false, false, true, true, false, true, false, true, true, false, false, false, false}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes3[i], op[i], false); - } - bool eRes_3[len0] = {false, true, false, false, false, true}; - for(int i = 0; i < len0; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_3[i], op[i], true); - } + key = "k4"; + bool eRes3[len+len1] = {false, false, true, true, false, true, false, true, true, false, false, false, false}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes3[i], op[i], false); + } + bool eRes_3[len0] = {false, true, false, false, false, true}; + for(int i = 0; i < len0; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_3[i], op[i], true); + } - for(int i = len; i < len + len1; i++){ - void* rightData = prepareNchar(inputNchar[i-len]); - makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes3[i], op[i], false); - taosMemoryFree(rightData); - } + for(int i = len; i < len + len1; i++){ + void* rightData = prepareNchar(inputNchar[i-len]); + makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes3[i], op[i], false); + taosMemoryFree(rightData); + } - printf("--------------------json double--5.44 {1, 8, 2, 2, 3, 0, 0, 0, 0}-------------------\n"); + printf("--------------------json double--5.44 {1, 8, 2, 2, 3, 0, 0, 0, 0}-------------------\n"); - key = "k5"; - bool eRes4[len+len1] = {true, false, false, false, false, true, false, true, true, false, false, false, false}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes4[i], op[i], false); - } - bool eRes_4[len0] = {false, true, true, true, false, true}; - for(int i = 0; i < len0; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_4[i], op[i], true); - } + key = "k5"; + bool eRes4[len+len1] = {true, false, false, false, false, true, false, true, true, false, false, false, false}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes4[i], op[i], false); + } + bool eRes_4[len0] = {false, true, true, true, false, true}; + for(int i = 0; i < len0; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_4[i], op[i], true); + } - for(int i = len; i < len + len1; i++){ - void* rightData = prepareNchar(inputNchar[i-len]); - makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes4[i], op[i], false); - taosMemoryFree(rightData); - } + for(int i = len; i < len + len1; i++){ + void* rightData = prepareNchar(inputNchar[i-len]); + makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes4[i], op[i], false); + taosMemoryFree(rightData); + } - printf("--------------------json int-- -10 {1, 8, 2, 2, 3, 0, 0, 0, 0}-------------------\n"); + printf("--------------------json int-- -10 {1, 8, 2, 2, 3, 0, 0, 0, 0}-------------------\n"); - key = "k6"; - bool eRes5[len+len1] = {false, false, true, true, false, true, false, true, true, false, false, false, false}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes5[i], op[i], false); - } - bool eRes_5[len0] = {true, true, false, false, false, true}; - for(int i = 0; i < len0; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_5[i], op[i], true); - } + key = "k6"; + bool eRes5[len+len1] = {false, false, true, true, false, true, false, true, true, false, false, false, false}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes5[i], op[i], false); + } + bool eRes_5[len0] = {true, true, false, false, false, true}; + for(int i = 0; i < len0; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_5[i], op[i], true); + } - for(int i = len; i < len + len1; i++){ - void* rightData = prepareNchar(inputNchar[i-len]); - makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes5[i], op[i], false); - taosMemoryFree(rightData); - } + for(int i = len; i < len + len1; i++){ + void* rightData = prepareNchar(inputNchar[i-len]); + makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes5[i], op[i], false); + taosMemoryFree(rightData); + } - printf("--------------------json double-- -9.8 {1, 8, 2, 2, 3, 0, 0, 0, 0}-------------------\n"); + printf("--------------------json double-- -9.8 {1, 8, 2, 2, 3, 0, 0, 0, 0}-------------------\n"); - key = "k7"; - bool eRes6[len+len1] = {false, false, true, true, false, true, false, true, true, false, false, false, false}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes6[i], op[i], false); - } - bool eRes_6[len0] = {true, true, false, false, false, true}; - for(int i = 0; i < len0; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_6[i], op[i], true); - } + key = "k7"; + bool eRes6[len+len1] = {false, false, true, true, false, true, false, true, true, false, false, false, false}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes6[i], op[i], false); + } + bool eRes_6[len0] = {true, true, false, false, false, true}; + for(int i = 0; i < len0; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_6[i], op[i], true); + } - for(int i = len; i < len + len1; i++){ - void* rightData = prepareNchar(inputNchar[i-len]); - makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes6[i], op[i], false); - taosMemoryFree(rightData); - } + for(int i = len; i < len + len1; i++){ + void* rightData = prepareNchar(inputNchar[i-len]); + makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes6[i], op[i], false); + taosMemoryFree(rightData); + } - printf("--------------------json bool-- 0 {1, 8, 2, 2, 3, 0, 0, 0, 0}-------------------\n"); + printf("--------------------json bool-- 0 {1, 8, 2, 2, 3, 0, 0, 0, 0}-------------------\n"); - key = "k8"; - bool eRes7[len+len1] = {false, false, true, true, false, false, false, true, false, false, false, false, false}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes7[i], op[i], false); - } - bool eRes_7[len0] = {true, true, false, false, false, false}; - for(int i = 0; i < len0; i++) { - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_7[i], op[i], true); - } + key = "k8"; + bool eRes7[len+len1] = {false, false, true, true, false, false, false, true, false, false, false, false, false}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes7[i], op[i], false); + } + bool eRes_7[len0] = {true, true, false, false, false, false}; + for(int i = 0; i < len0; i++) { + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_7[i], op[i], true); + } - for(int i = len; i < len + len1; i++){ - void* rightData = prepareNchar(inputNchar[i-len]); - makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes7[i], op[i], false); - taosMemoryFree(rightData); - } + for(int i = len; i < len + len1; i++){ + void* rightData = prepareNchar(inputNchar[i-len]); + makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes7[i], op[i], false); + taosMemoryFree(rightData); + } - printf("--------------------json string-- 6.6hello {1, 8, 2, 2, 3, 0, 0, 0, 0}-------------------\n"); + printf("--------------------json string-- 6.6hello {1, 8, 2, 2, 3, 0, 0, 0, 0}-------------------\n"); - key = "k9"; - bool eRes8[len+len1] = {true, false, false, false, false, true, false, true, true, false, true, false, true}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes8[i], op[i], false); - } - bool eRes_8[len0] = {false, true, true, true, false, true}; - for(int i = 0; i < len0; i++) { - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_8[i], op[i], true); - } + key = "k9"; + bool eRes8[len+len1] = {true, false, false, false, false, true, false, true, true, false, true, false, true}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes8[i], op[i], false); + } + bool eRes_8[len0] = {false, true, true, true, false, true}; + for(int i = 0; i < len0; i++) { + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_8[i], op[i], true); + } - for(int i = len; i < len + len1; i++){ - void* rightData = prepareNchar(inputNchar[i-len]); - makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes8[i], op[i], false); - taosMemoryFree(rightData); - } + for(int i = len; i < len + len1; i++){ + void* rightData = prepareNchar(inputNchar[i-len]); + makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes8[i], op[i], false); + taosMemoryFree(rightData); + } - printf("---------------------json not exist-- NULL {1, 8, 2, 2, 3, 0, 0, 0, 0}------------------\n"); + printf("---------------------json not exist-- NULL {1, 8, 2, 2, 3, 0, 0, 0, 0}------------------\n"); - key = "k10"; // (NULL is true) return NULL, so use DBL_MAX represent NULL - double eRes9[len+len1] = {false, false, false, false, false, false, true, false, DBL_MAX, false, false, false, false}; - for(int i = 0; i < len; i++){ - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes9[i], op[i], false); - } - bool eRes_9[len0] = {false, false, false, false, false, false}; - for(int i = 0; i < len0; i++) { - makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_9[i], op[i], true); - } + key = "k10"; // (NULL is true) return NULL, so use DBL_MAX represent NULL + bool eRes9[len+len1] = {false, false, false, false, false, false, true, false, false, false, false, false, false}; + for(int i = 0; i < len; i++){ + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes9[i], op[i], false); + } + bool eRes_9[len0] = {false, false, false, false, false, false}; + for(int i = 0; i < len0; i++) { + makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes_9[i], op[i], true); + } - for(int i = len; i < len + len1; i++){ - void* rightData = prepareNchar(inputNchar[i-len]); - makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes9[i], op[i], false); - taosMemoryFree(rightData); - } + for(int i = len; i < len + len1; i++){ + void* rightData = prepareNchar(inputNchar[i-len]); + makeCalculate(row, key, TSDB_DATA_TYPE_NCHAR, rightData, eRes9[i], op[i], false); + taosMemoryFree(rightData); + } - taosArrayDestroy(tags); - taosMemoryFree(row); + taosArrayDestroy(tags); + taosMemoryFree(row); } TEST(columnTest, smallint_value_add_int_column) { - scltInitLogFile(); - - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL; - int32_t leftv = 1; - int16_t rightv[5]= {0, -5, -4, 23, 100}; - double eRes[5] = {1.0, -4, -3, 24, 101}; - SSDataBlock *src = NULL; - int32_t rowNum = sizeof(rightv)/sizeof(rightv[0]); - scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); - scltMakeColumnNode(&pRight, &src, TSDB_DATA_TYPE_SMALLINT, sizeof(int16_t), rowNum, rightv); - scltMakeOpNode(&opNode, OP_TYPE_ADD, TSDB_DATA_TYPE_DOUBLE, pLeft, pRight); + scltInitLogFile(); - SArray *blockList = taosArrayInit(2, POINTER_BYTES); - taosArrayPush(blockList, &src); - SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_DOUBLE, sizeof(double)); - int16_t dataBlockId = 0, slotId = 0; - scltAppendReservedSlot(blockList, &dataBlockId, &slotId, true, rowNum, &colInfo); - scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL; + int32_t leftv = 1; + int16_t rightv[5]= {0, -5, -4, 23, 100}; + double eRes[5] = {1.0, -4, -3, 24, 101}; + SSDataBlock *src = NULL; + int32_t rowNum = sizeof(rightv)/sizeof(rightv[0]); + scltMakeValueNode(&pLeft, TSDB_DATA_TYPE_INT, &leftv); + scltMakeColumnNode(&pRight, &src, TSDB_DATA_TYPE_SMALLINT, sizeof(int16_t), rowNum, rightv); + scltMakeOpNode(&opNode, OP_TYPE_ADD, TSDB_DATA_TYPE_DOUBLE, pLeft, pRight); - int32_t code = scalarCalculate(opNode, blockList, NULL); - ASSERT_EQ(code, 0); + SArray *blockList = taosArrayInit(2, POINTER_BYTES); + taosArrayPush(blockList, &src); + SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_DOUBLE, sizeof(double)); + int16_t dataBlockId = 0, slotId = 0; + scltAppendReservedSlot(blockList, &dataBlockId, &slotId, true, rowNum, &colInfo); + scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); - SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); - ASSERT_EQ(res->info.rows, rowNum); - SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); - ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_DOUBLE); - ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_DOUBLE].bytes); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(column, i)), eRes[i]); - } + int32_t code = scalarCalculate(opNode, blockList, NULL); + ASSERT_EQ(code, 0); - taosArrayDestroyEx(blockList, scltFreeDataBlock); - nodesDestroyNode(opNode); + SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); + ASSERT_EQ(res->info.rows, rowNum); + SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); + ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_DOUBLE); + ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_DOUBLE].bytes); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(column, i)), eRes[i]); + } + + taosArrayDestroyEx(blockList, scltFreeDataBlock); + nodesDestroyNode(opNode); } TEST(columnTest, bigint_column_multi_binary_column) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL; - int64_t leftv[5]= {1, 2, 3, 4, 5}; - char rightv[5][5]= {0}; - for (int32_t i = 0; i < 5; ++i) { - rightv[i][2] = rightv[i][3] = '0'; - rightv[i][4] = '0' + i; - varDataSetLen(rightv[i], 3); - } - double eRes[5] = {0, 2, 6, 12, 20}; - SSDataBlock *src = NULL; - int32_t rowNum = sizeof(rightv)/sizeof(rightv[0]); - scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t), rowNum, leftv); - scltMakeColumnNode(&pRight, &src, TSDB_DATA_TYPE_BINARY, 5, rowNum, rightv); - scltMakeOpNode(&opNode, OP_TYPE_MULTI, TSDB_DATA_TYPE_DOUBLE, pLeft, pRight); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL; + int64_t leftv[5]= {1, 2, 3, 4, 5}; + char rightv[5][5]= {0}; + for (int32_t i = 0; i < 5; ++i) { + rightv[i][2] = rightv[i][3] = '0'; + rightv[i][4] = '0' + i; + varDataSetLen(rightv[i], 3); + } + double eRes[5] = {0, 2, 6, 12, 20}; + SSDataBlock *src = NULL; + int32_t rowNum = sizeof(rightv)/sizeof(rightv[0]); + scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t), rowNum, leftv); + scltMakeColumnNode(&pRight, &src, TSDB_DATA_TYPE_BINARY, 5, rowNum, rightv); + scltMakeOpNode(&opNode, OP_TYPE_MULTI, TSDB_DATA_TYPE_DOUBLE, pLeft, pRight); - SArray *blockList = taosArrayInit(1, POINTER_BYTES); - taosArrayPush(blockList, &src); - SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_DOUBLE, sizeof(double)); - int16_t dataBlockId = 0, slotId = 0; - scltAppendReservedSlot(blockList, &dataBlockId, &slotId, false, rowNum, &colInfo); - scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); - - int32_t code = scalarCalculate(opNode, blockList, NULL); - ASSERT_EQ(code, 0); + SArray *blockList = taosArrayInit(1, POINTER_BYTES); + taosArrayPush(blockList, &src); + SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_DOUBLE, sizeof(double)); + int16_t dataBlockId = 0, slotId = 0; + scltAppendReservedSlot(blockList, &dataBlockId, &slotId, false, rowNum, &colInfo); + scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); - SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); - ASSERT_EQ(res->info.rows, rowNum); - SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); - ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_DOUBLE); - ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_DOUBLE].bytes); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(column, i)), eRes[i]); - } - taosArrayDestroyEx(blockList, scltFreeDataBlock); - nodesDestroyNode(opNode); + int32_t code = scalarCalculate(opNode, blockList, NULL); + ASSERT_EQ(code, 0); + + SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); + ASSERT_EQ(res->info.rows, rowNum); + SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); + ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_DOUBLE); + ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_DOUBLE].bytes); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(column, i)), eRes[i]); + } + taosArrayDestroyEx(blockList, scltFreeDataBlock); + nodesDestroyNode(opNode); } TEST(columnTest, smallint_column_and_binary_column) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL; - int16_t leftv[5]= {1, 2, 3, 4, 5}; - char rightv[5][5]= {0}; - for (int32_t i = 0; i < 5; ++i) { - rightv[i][2] = rightv[i][3] = '0'; - rightv[i][4] = '0' + i; - varDataSetLen(rightv[i], 3); - } - int64_t eRes[5] = {0, 0, 2, 0, 4}; - SSDataBlock *src = NULL; - int32_t rowNum = sizeof(rightv)/sizeof(rightv[0]); - scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_SMALLINT, sizeof(int16_t), rowNum, leftv); - scltMakeColumnNode(&pRight, &src, TSDB_DATA_TYPE_BINARY, 5, rowNum, rightv); - scltMakeOpNode(&opNode, OP_TYPE_BIT_AND, TSDB_DATA_TYPE_BIGINT, pLeft, pRight); - - SArray *blockList = taosArrayInit(1, POINTER_BYTES); - taosArrayPush(blockList, &src); - SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t)); - int16_t dataBlockId = 0, slotId = 0; - scltAppendReservedSlot(blockList, &dataBlockId, &slotId, false, rowNum, &colInfo); - scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); - - int32_t code = scalarCalculate(opNode, blockList, NULL); - ASSERT_EQ(code, 0); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL; + int16_t leftv[5]= {1, 2, 3, 4, 5}; + char rightv[5][5]= {0}; + for (int32_t i = 0; i < 5; ++i) { + rightv[i][2] = rightv[i][3] = '0'; + rightv[i][4] = '0' + i; + varDataSetLen(rightv[i], 3); + } + int64_t eRes[5] = {0, 0, 2, 0, 4}; + SSDataBlock *src = NULL; + int32_t rowNum = sizeof(rightv)/sizeof(rightv[0]); + scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_SMALLINT, sizeof(int16_t), rowNum, leftv); + scltMakeColumnNode(&pRight, &src, TSDB_DATA_TYPE_BINARY, 5, rowNum, rightv); + scltMakeOpNode(&opNode, OP_TYPE_BIT_AND, TSDB_DATA_TYPE_BIGINT, pLeft, pRight); - SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); - ASSERT_EQ(res->info.rows, rowNum); - SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); - ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BIGINT); - ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BIGINT].bytes); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int64_t *)colDataGetData(column, i)), eRes[i]); - } - taosArrayDestroyEx(blockList, scltFreeDataBlock); - nodesDestroyNode(opNode); + SArray *blockList = taosArrayInit(1, POINTER_BYTES); + taosArrayPush(blockList, &src); + SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t)); + int16_t dataBlockId = 0, slotId = 0; + scltAppendReservedSlot(blockList, &dataBlockId, &slotId, false, rowNum, &colInfo); + scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); + + int32_t code = scalarCalculate(opNode, blockList, NULL); + ASSERT_EQ(code, 0); + + SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); + ASSERT_EQ(res->info.rows, rowNum); + SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); + ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BIGINT); + ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BIGINT].bytes); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int64_t *)colDataGetData(column, i)), eRes[i]); + } + taosArrayDestroyEx(blockList, scltFreeDataBlock); + nodesDestroyNode(opNode); } TEST(columnTest, smallint_column_or_float_column) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL; - int16_t leftv[5]= {1, 2, 3, 4, 5}; - float rightv[5]= {2.0, 3.0, 4.1, 5.2, 6.0}; - int64_t eRes[5] = {3, 3, 7, 5, 7}; - SSDataBlock *src = NULL; - int32_t rowNum = sizeof(rightv)/sizeof(rightv[0]); - scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_SMALLINT, sizeof(int16_t), rowNum, leftv); - scltMakeColumnNode(&pRight, &src, TSDB_DATA_TYPE_FLOAT, sizeof(float), rowNum, rightv); - scltMakeOpNode(&opNode, OP_TYPE_BIT_OR, TSDB_DATA_TYPE_BIGINT, pLeft, pRight); - - SArray *blockList = taosArrayInit(1, POINTER_BYTES); - taosArrayPush(blockList, &src); - SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t)); - int16_t dataBlockId = 0, slotId = 0; - scltAppendReservedSlot(blockList, &dataBlockId, &slotId, true, rowNum, &colInfo); - scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); - - int32_t code = scalarCalculate(opNode, blockList, NULL); - ASSERT_EQ(code, 0); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL; + int16_t leftv[5]= {1, 2, 3, 4, 5}; + float rightv[5]= {2.0, 3.0, 4.1, 5.2, 6.0}; + int64_t eRes[5] = {3, 3, 7, 5, 7}; + SSDataBlock *src = NULL; + int32_t rowNum = sizeof(rightv)/sizeof(rightv[0]); + scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_SMALLINT, sizeof(int16_t), rowNum, leftv); + scltMakeColumnNode(&pRight, &src, TSDB_DATA_TYPE_FLOAT, sizeof(float), rowNum, rightv); + scltMakeOpNode(&opNode, OP_TYPE_BIT_OR, TSDB_DATA_TYPE_BIGINT, pLeft, pRight); - SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); - ASSERT_EQ(res->info.rows, rowNum); - SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); - ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BIGINT); - ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BIGINT].bytes); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int64_t *)colDataGetData(column, i)), eRes[i]); - } - taosArrayDestroyEx(blockList, scltFreeDataBlock); - nodesDestroyNode(opNode); + SArray *blockList = taosArrayInit(1, POINTER_BYTES); + taosArrayPush(blockList, &src); + SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t)); + int16_t dataBlockId = 0, slotId = 0; + scltAppendReservedSlot(blockList, &dataBlockId, &slotId, true, rowNum, &colInfo); + scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); + + int32_t code = scalarCalculate(opNode, blockList, NULL); + ASSERT_EQ(code, 0); + + SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); + ASSERT_EQ(res->info.rows, rowNum); + SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); + ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BIGINT); + ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BIGINT].bytes); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int64_t *)colDataGetData(column, i)), eRes[i]); + } + taosArrayDestroyEx(blockList, scltFreeDataBlock); + nodesDestroyNode(opNode); } TEST(columnTest, smallint_column_or_double_value) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL; - int16_t leftv[5]= {1, 2, 3, 4, 5}; - double rightv= 10.2; - int64_t eRes[5] = {11, 10, 11, 14, 15}; - SSDataBlock *src = NULL; - int32_t rowNum = sizeof(leftv)/sizeof(leftv[0]); - scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_SMALLINT, sizeof(int16_t), rowNum, leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_DOUBLE, &rightv); - scltMakeOpNode(&opNode, OP_TYPE_BIT_OR, TSDB_DATA_TYPE_BIGINT, pLeft, pRight); - - SArray *blockList = taosArrayInit(1, POINTER_BYTES); - taosArrayPush(blockList, &src); - SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t)); - int16_t dataBlockId = 0, slotId = 0; - scltAppendReservedSlot(blockList, &dataBlockId, &slotId, true, rowNum, &colInfo); - scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); - - int32_t code = scalarCalculate(opNode, blockList, NULL); - ASSERT_EQ(code, 0); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL; + int16_t leftv[5]= {1, 2, 3, 4, 5}; + double rightv= 10.2; + int64_t eRes[5] = {11, 10, 11, 14, 15}; + SSDataBlock *src = NULL; + int32_t rowNum = sizeof(leftv)/sizeof(leftv[0]); + scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_SMALLINT, sizeof(int16_t), rowNum, leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_DOUBLE, &rightv); + scltMakeOpNode(&opNode, OP_TYPE_BIT_OR, TSDB_DATA_TYPE_BIGINT, pLeft, pRight); - SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); - ASSERT_EQ(res->info.rows, rowNum); - SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); - ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BIGINT); - ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BIGINT].bytes); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int64_t *)colDataGetData(column, i)), eRes[i]); - } - taosArrayDestroyEx(blockList, scltFreeDataBlock); - nodesDestroyNode(opNode); + SArray *blockList = taosArrayInit(1, POINTER_BYTES); + taosArrayPush(blockList, &src); + SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t)); + int16_t dataBlockId = 0, slotId = 0; + scltAppendReservedSlot(blockList, &dataBlockId, &slotId, true, rowNum, &colInfo); + scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); + + int32_t code = scalarCalculate(opNode, blockList, NULL); + ASSERT_EQ(code, 0); + + SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); + ASSERT_EQ(res->info.rows, rowNum); + SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); + ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BIGINT); + ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BIGINT].bytes); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int64_t *)colDataGetData(column, i)), eRes[i]); + } + taosArrayDestroyEx(blockList, scltFreeDataBlock); + nodesDestroyNode(opNode); } TEST(columnTest, smallint_column_greater_double_value) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL; - int16_t leftv[5]= {1, 2, 3, 4, 5}; - double rightv= 2.5; - bool eRes[5] = {false, false, true, true, true}; - SSDataBlock *src = NULL; - int32_t rowNum = sizeof(leftv)/sizeof(leftv[0]); - scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_SMALLINT, sizeof(int16_t), rowNum, leftv); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_DOUBLE, &rightv); - scltMakeOpNode(&opNode, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - SArray *blockList = taosArrayInit(1, POINTER_BYTES); - taosArrayPush(blockList, &src); - SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BOOL, sizeof(bool)); - int16_t dataBlockId = 0, slotId = 0; - scltAppendReservedSlot(blockList, &dataBlockId, &slotId, true, rowNum, &colInfo); - scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); - - int32_t code = scalarCalculate(opNode, blockList, NULL); - ASSERT_EQ(code, 0); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL; + int16_t leftv[5]= {1, 2, 3, 4, 5}; + double rightv= 2.5; + bool eRes[5] = {false, false, true, true, true}; + SSDataBlock *src = NULL; + int32_t rowNum = sizeof(leftv)/sizeof(leftv[0]); + scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_SMALLINT, sizeof(int16_t), rowNum, leftv); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_DOUBLE, &rightv); + scltMakeOpNode(&opNode, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); - ASSERT_EQ(res->info.rows, rowNum); - SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); - ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BOOL].bytes); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((bool *)colDataGetData(column, i)), eRes[i]); - } - taosArrayDestroyEx(blockList, scltFreeDataBlock); - nodesDestroyNode(opNode); + SArray *blockList = taosArrayInit(1, POINTER_BYTES); + taosArrayPush(blockList, &src); + SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BOOL, sizeof(bool)); + int16_t dataBlockId = 0, slotId = 0; + scltAppendReservedSlot(blockList, &dataBlockId, &slotId, true, rowNum, &colInfo); + scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); + + int32_t code = scalarCalculate(opNode, blockList, NULL); + ASSERT_EQ(code, 0); + + SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); + ASSERT_EQ(res->info.rows, rowNum); + SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); + ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BOOL].bytes); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((bool *)colDataGetData(column, i)), eRes[i]); + } + taosArrayDestroyEx(blockList, scltFreeDataBlock); + nodesDestroyNode(opNode); } TEST(columnTest, int_column_in_double_list) { - SNode *pLeft = NULL, *pRight = NULL, *listNode = NULL, *opNode = NULL; - int32_t leftv[5] = {1, 2, 3, 4, 5}; - double rightv1 = 1.1,rightv2 = 2.2,rightv3 = 3.3; - bool eRes[5] = {true, true, true, false, false}; - SSDataBlock *src = NULL; - int32_t rowNum = sizeof(leftv)/sizeof(leftv[0]); - scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_INT, sizeof(int32_t), rowNum, leftv); - SNodeList* list = nodesMakeList(); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_DOUBLE, &rightv1); - nodesListAppend(list, pRight); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_DOUBLE, &rightv2); - nodesListAppend(list, pRight); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_DOUBLE, &rightv3); - nodesListAppend(list, pRight); - scltMakeListNode(&listNode,list, TSDB_DATA_TYPE_INT); - scltMakeOpNode(&opNode, OP_TYPE_IN, TSDB_DATA_TYPE_BOOL, pLeft, listNode); + SNode *pLeft = NULL, *pRight = NULL, *listNode = NULL, *opNode = NULL; + int32_t leftv[5] = {1, 2, 3, 4, 5}; + double rightv1 = 1.1,rightv2 = 2.2,rightv3 = 3.3; + bool eRes[5] = {true, true, true, false, false}; + SSDataBlock *src = NULL; + int32_t rowNum = sizeof(leftv)/sizeof(leftv[0]); + scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_INT, sizeof(int32_t), rowNum, leftv); + SNodeList* list = nodesMakeList(); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_DOUBLE, &rightv1); + nodesListAppend(list, pRight); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_DOUBLE, &rightv2); + nodesListAppend(list, pRight); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_DOUBLE, &rightv3); + nodesListAppend(list, pRight); + scltMakeListNode(&listNode,list, TSDB_DATA_TYPE_INT); + scltMakeOpNode(&opNode, OP_TYPE_IN, TSDB_DATA_TYPE_BOOL, pLeft, listNode); - SArray *blockList = taosArrayInit(1, POINTER_BYTES); - taosArrayPush(blockList, &src); - SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BOOL, sizeof(bool)); - int16_t dataBlockId = 0, slotId = 0; - scltAppendReservedSlot(blockList, &dataBlockId, &slotId, true, rowNum, &colInfo); - scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); - - int32_t code = scalarCalculate(opNode, blockList, NULL); - ASSERT_EQ(code, 0); + SArray *blockList = taosArrayInit(1, POINTER_BYTES); + taosArrayPush(blockList, &src); + SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BOOL, sizeof(bool)); + int16_t dataBlockId = 0, slotId = 0; + scltAppendReservedSlot(blockList, &dataBlockId, &slotId, true, rowNum, &colInfo); + scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); - SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); - ASSERT_EQ(res->info.rows, rowNum); - SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); - ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BOOL].bytes); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((bool *)colDataGetData(column, i)), eRes[i]); - } - taosArrayDestroyEx(blockList, scltFreeDataBlock); - nodesDestroyNode(opNode); + int32_t code = scalarCalculate(opNode, blockList, NULL); + ASSERT_EQ(code, 0); + + SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); + ASSERT_EQ(res->info.rows, rowNum); + SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); + ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BOOL].bytes); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((bool *)colDataGetData(column, i)), eRes[i]); + } + taosArrayDestroyEx(blockList, scltFreeDataBlock); + nodesDestroyNode(opNode); } TEST(columnTest, binary_column_in_binary_list) { - SNode *pLeft = NULL, *pRight = NULL, *listNode = NULL, *opNode = NULL; - bool eRes[5] = {true, true, false, false, false}; - SSDataBlock *src = NULL; - char leftv[5][5]= {0}; - char rightv[3][5]= {0}; - for (int32_t i = 0; i < 5; ++i) { - leftv[i][2] = 'a' + i; - leftv[i][3] = 'b' + i; - leftv[i][4] = '0' + i; - varDataSetLen(leftv[i], 3); - } - for (int32_t i = 0; i < 2; ++i) { - rightv[i][2] = 'a' + i; - rightv[i][3] = 'b' + i; - rightv[i][4] = '0' + i; - varDataSetLen(rightv[i], 3); - } - for (int32_t i = 2; i < 3; ++i) { - rightv[i][2] = 'a' + i; - rightv[i][3] = 'a' + i; - rightv[i][4] = 'a' + i; - varDataSetLen(rightv[i], 3); - } - - int32_t rowNum = sizeof(leftv)/sizeof(leftv[0]); - scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_BINARY, 3, rowNum, leftv); - SNodeList* list = nodesMakeList(); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv[0]); - nodesListAppend(list, pRight); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv[1]); - nodesListAppend(list, pRight); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv[2]); - nodesListAppend(list, pRight); - scltMakeListNode(&listNode,list, TSDB_DATA_TYPE_BINARY); - scltMakeOpNode(&opNode, OP_TYPE_IN, TSDB_DATA_TYPE_BOOL, pLeft, listNode); - - SArray *blockList = taosArrayInit(1, POINTER_BYTES); - taosArrayPush(blockList, &src); - SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BOOL, sizeof(bool)); - int16_t dataBlockId = 0, slotId = 0; - scltAppendReservedSlot(blockList, &dataBlockId, &slotId, false, rowNum, &colInfo); - scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); - - int32_t code = scalarCalculate(opNode, blockList, NULL); - ASSERT_EQ(code, 0); + SNode *pLeft = NULL, *pRight = NULL, *listNode = NULL, *opNode = NULL; + bool eRes[5] = {true, true, false, false, false}; + SSDataBlock *src = NULL; + char leftv[5][5]= {0}; + char rightv[3][5]= {0}; + for (int32_t i = 0; i < 5; ++i) { + leftv[i][2] = 'a' + i; + leftv[i][3] = 'b' + i; + leftv[i][4] = '0' + i; + varDataSetLen(leftv[i], 3); + } + for (int32_t i = 0; i < 2; ++i) { + rightv[i][2] = 'a' + i; + rightv[i][3] = 'b' + i; + rightv[i][4] = '0' + i; + varDataSetLen(rightv[i], 3); + } + for (int32_t i = 2; i < 3; ++i) { + rightv[i][2] = 'a' + i; + rightv[i][3] = 'a' + i; + rightv[i][4] = 'a' + i; + varDataSetLen(rightv[i], 3); + } - SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); - ASSERT_EQ(res->info.rows, rowNum); - SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); - ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BOOL].bytes); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((bool *)colDataGetData(column, i)), eRes[i]); - } - taosArrayDestroyEx(blockList, scltFreeDataBlock); - nodesDestroyNode(opNode); + int32_t rowNum = sizeof(leftv)/sizeof(leftv[0]); + scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_BINARY, 3, rowNum, leftv); + SNodeList* list = nodesMakeList(); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv[0]); + nodesListAppend(list, pRight); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv[1]); + nodesListAppend(list, pRight); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv[2]); + nodesListAppend(list, pRight); + scltMakeListNode(&listNode,list, TSDB_DATA_TYPE_BINARY); + scltMakeOpNode(&opNode, OP_TYPE_IN, TSDB_DATA_TYPE_BOOL, pLeft, listNode); + + SArray *blockList = taosArrayInit(1, POINTER_BYTES); + taosArrayPush(blockList, &src); + SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BOOL, sizeof(bool)); + int16_t dataBlockId = 0, slotId = 0; + scltAppendReservedSlot(blockList, &dataBlockId, &slotId, false, rowNum, &colInfo); + scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); + + int32_t code = scalarCalculate(opNode, blockList, NULL); + ASSERT_EQ(code, 0); + + SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); + ASSERT_EQ(res->info.rows, rowNum); + SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); + ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BOOL].bytes); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((bool *)colDataGetData(column, i)), eRes[i]); + } + taosArrayDestroyEx(blockList, scltFreeDataBlock); + nodesDestroyNode(opNode); } TEST(columnTest, binary_column_like_binary) { - SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL; - char rightv[64] = {0}; - char leftv[5][5]= {0}; - SSDataBlock *src = NULL; - bool eRes[5] = {true, false, true, false, true}; - - for (int32_t i = 0; i < 5; ++i) { - leftv[i][2] = 'a'; - leftv[i][3] = 'a'; - leftv[i][4] = '0' + i % 2; - varDataSetLen(leftv[i], 3); - } - - int32_t rowNum = sizeof(leftv)/sizeof(leftv[0]); - scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_BINARY, 3, rowNum, leftv); + SNode *pLeft = NULL, *pRight = NULL, *opNode = NULL; + char rightv[64] = {0}; + char leftv[5][5]= {0}; + SSDataBlock *src = NULL; + bool eRes[5] = {true, false, true, false, true}; - sprintf(&rightv[2], "%s", "__0"); - varDataSetLen(rightv, strlen(&rightv[2])); - scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); - scltMakeOpNode(&opNode, OP_TYPE_LIKE, TSDB_DATA_TYPE_BOOL, pLeft, pRight); - - SArray *blockList = taosArrayInit(1, POINTER_BYTES); - taosArrayPush(blockList, &src); - SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BOOL, sizeof(bool)); - int16_t dataBlockId = 0, slotId = 0; - scltAppendReservedSlot(blockList, &dataBlockId, &slotId, false, rowNum, &colInfo); - scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); - - int32_t code = scalarCalculate(opNode, blockList, NULL); - ASSERT_EQ(code, 0); + for (int32_t i = 0; i < 5; ++i) { + leftv[i][2] = 'a'; + leftv[i][3] = 'a'; + leftv[i][4] = '0' + i % 2; + varDataSetLen(leftv[i], 3); + } - SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); - ASSERT_EQ(res->info.rows, rowNum); - SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); - ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BOOL].bytes); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((bool *)colDataGetData(column, i)), eRes[i]); - } + int32_t rowNum = sizeof(leftv)/sizeof(leftv[0]); + scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_BINARY, 3, rowNum, leftv); - taosArrayDestroyEx(blockList, scltFreeDataBlock); - nodesDestroyNode(opNode); + sprintf(&rightv[2], "%s", "__0"); + varDataSetLen(rightv, strlen(&rightv[2])); + scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, rightv); + scltMakeOpNode(&opNode, OP_TYPE_LIKE, TSDB_DATA_TYPE_BOOL, pLeft, pRight); + + SArray *blockList = taosArrayInit(1, POINTER_BYTES); + taosArrayPush(blockList, &src); + SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BOOL, sizeof(bool)); + int16_t dataBlockId = 0, slotId = 0; + scltAppendReservedSlot(blockList, &dataBlockId, &slotId, false, rowNum, &colInfo); + scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); + + int32_t code = scalarCalculate(opNode, blockList, NULL); + ASSERT_EQ(code, 0); + + SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); + ASSERT_EQ(res->info.rows, rowNum); + SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); + ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BOOL].bytes); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((bool *)colDataGetData(column, i)), eRes[i]); + } + + taosArrayDestroyEx(blockList, scltFreeDataBlock); + nodesDestroyNode(opNode); } TEST(columnTest, binary_column_is_true) { - SNode *pLeft = NULL, *opNode = NULL; - char leftv[5][5]= {0}; - SSDataBlock *src = NULL; - bool eRes[5] = {false, true, false, true, false}; - - for (int32_t i = 0; i < 5; ++i) { - leftv[i][2] = '0' + i % 2; - leftv[i][3] = 'a'; - leftv[i][4] = '0' + i % 2; - varDataSetLen(leftv[i], 3); - } - - int32_t rowNum = sizeof(leftv)/sizeof(leftv[0]); - scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_BINARY, 5, rowNum, leftv); + SNode *pLeft = NULL, *opNode = NULL; + char leftv[5][5]= {0}; + SSDataBlock *src = NULL; + bool eRes[5] = {false, true, false, true, false}; - scltMakeOpNode(&opNode, OP_TYPE_IS_TRUE, TSDB_DATA_TYPE_BOOL, pLeft, NULL); - - SArray *blockList = taosArrayInit(1, POINTER_BYTES); - taosArrayPush(blockList, &src); - SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BOOL, sizeof(bool)); - int16_t dataBlockId = 0, slotId = 0; - scltAppendReservedSlot(blockList, &dataBlockId, &slotId, false, rowNum, &colInfo); - scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); - - int32_t code = scalarCalculate(opNode, blockList, NULL); - ASSERT_EQ(code, 0); + for (int32_t i = 0; i < 5; ++i) { + leftv[i][2] = '0' + i % 2; + leftv[i][3] = 'a'; + leftv[i][4] = '0' + i % 2; + varDataSetLen(leftv[i], 3); + } - SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); - ASSERT_EQ(res->info.rows, rowNum); - SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); - ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BOOL].bytes); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((bool *)colDataGetData(column, i)), eRes[i]); - } - taosArrayDestroyEx(blockList, scltFreeDataBlock); - nodesDestroyNode(opNode); + int32_t rowNum = sizeof(leftv)/sizeof(leftv[0]); + scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_BINARY, 5, rowNum, leftv); + + scltMakeOpNode(&opNode, OP_TYPE_IS_TRUE, TSDB_DATA_TYPE_BOOL, pLeft, NULL); + + SArray *blockList = taosArrayInit(1, POINTER_BYTES); + taosArrayPush(blockList, &src); + SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BOOL, sizeof(bool)); + int16_t dataBlockId = 0, slotId = 0; + scltAppendReservedSlot(blockList, &dataBlockId, &slotId, false, rowNum, &colInfo); + scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); + + int32_t code = scalarCalculate(opNode, blockList, NULL); + ASSERT_EQ(code, 0); + + SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); + ASSERT_EQ(res->info.rows, rowNum); + SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); + ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BOOL].bytes); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((bool *)colDataGetData(column, i)), eRes[i]); + } + taosArrayDestroyEx(blockList, scltFreeDataBlock); + nodesDestroyNode(opNode); } TEST(columnTest, binary_column_is_null) { - SNode *pLeft = NULL, *opNode = NULL; - char leftv[5][5]= {0}; - SSDataBlock *src = NULL; - bool eRes[5] = {false, false, true, false, true}; - - for (int32_t i = 0; i < 5; ++i) { - leftv[i][2] = '0' + i % 2; - leftv[i][3] = 'a'; - leftv[i][4] = '0' + i % 2; - varDataSetLen(leftv[i], 3); - } - - int32_t rowNum = sizeof(leftv)/sizeof(leftv[0]); - scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_BINARY, 3, rowNum, leftv); + SNode *pLeft = NULL, *opNode = NULL; + char leftv[5][5]= {0}; + SSDataBlock *src = NULL; + bool eRes[5] = {false, false, true, false, true}; - SColumnInfoData *pcolumn = (SColumnInfoData *)taosArrayGetLast(src->pDataBlock); - colDataAppend(pcolumn, 2, NULL, true); - colDataAppend(pcolumn, 4, NULL, true); + for (int32_t i = 0; i < 5; ++i) { + leftv[i][2] = '0' + i % 2; + leftv[i][3] = 'a'; + leftv[i][4] = '0' + i % 2; + varDataSetLen(leftv[i], 3); + } - scltMakeOpNode(&opNode, OP_TYPE_IS_NULL, TSDB_DATA_TYPE_BOOL, pLeft, NULL); - - SArray *blockList = taosArrayInit(1, POINTER_BYTES); - taosArrayPush(blockList, &src); - SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BOOL, sizeof(bool)); - int16_t dataBlockId = 0, slotId = 0; - scltAppendReservedSlot(blockList, &dataBlockId, &slotId, false, rowNum, &colInfo); - scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); - - int32_t code = scalarCalculate(opNode, blockList, NULL); - ASSERT_EQ(code, 0); + int32_t rowNum = sizeof(leftv)/sizeof(leftv[0]); + scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_BINARY, 3, rowNum, leftv); - SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); - ASSERT_EQ(res->info.rows, rowNum); - SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); - ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BOOL].bytes); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((bool *)colDataGetData(column, i)), eRes[i]); - } - taosArrayDestroyEx(blockList, scltFreeDataBlock); - nodesDestroyNode(opNode); + SColumnInfoData *pcolumn = (SColumnInfoData *)taosArrayGetLast(src->pDataBlock); + colDataAppend(pcolumn, 2, NULL, true); + colDataAppend(pcolumn, 4, NULL, true); + + scltMakeOpNode(&opNode, OP_TYPE_IS_NULL, TSDB_DATA_TYPE_BOOL, pLeft, NULL); + + SArray *blockList = taosArrayInit(1, POINTER_BYTES); + taosArrayPush(blockList, &src); + SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BOOL, sizeof(bool)); + int16_t dataBlockId = 0, slotId = 0; + scltAppendReservedSlot(blockList, &dataBlockId, &slotId, false, rowNum, &colInfo); + scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); + + int32_t code = scalarCalculate(opNode, blockList, NULL); + ASSERT_EQ(code, 0); + + SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); + ASSERT_EQ(res->info.rows, rowNum); + SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); + ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BOOL].bytes); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((bool *)colDataGetData(column, i)), eRes[i]); + } + taosArrayDestroyEx(blockList, scltFreeDataBlock); + nodesDestroyNode(opNode); } TEST(columnTest, binary_column_is_not_null) { - SNode *pLeft = NULL, *opNode = NULL; - char leftv[5][5]= {0}; - SSDataBlock *src = NULL; - bool eRes[5] = {true, true, true, true, false}; - - for (int32_t i = 0; i < 5; ++i) { - leftv[i][2] = '0' + i % 2; - leftv[i][3] = 'a'; - leftv[i][4] = '0' + i % 2; - varDataSetLen(leftv[i], 3); - } - - int32_t rowNum = sizeof(leftv)/sizeof(leftv[0]); - scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_BINARY, 3, rowNum, leftv); - - SColumnInfoData *pcolumn = (SColumnInfoData *)taosArrayGetLast(src->pDataBlock); - colDataAppend(pcolumn, 4, NULL, true); + SNode *pLeft = NULL, *opNode = NULL; + char leftv[5][5]= {0}; + SSDataBlock *src = NULL; + bool eRes[5] = {true, true, true, true, false}; - scltMakeOpNode(&opNode, OP_TYPE_IS_NOT_NULL, TSDB_DATA_TYPE_BOOL, pLeft, NULL); - - SArray *blockList = taosArrayInit(1, POINTER_BYTES); - taosArrayPush(blockList, &src); - SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BOOL, sizeof(bool)); - int16_t dataBlockId = 0, slotId = 0; - scltAppendReservedSlot(blockList, &dataBlockId, &slotId, false, rowNum, &colInfo); - scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); - - int32_t code = scalarCalculate(opNode, blockList, NULL); - ASSERT_EQ(code, 0); + for (int32_t i = 0; i < 5; ++i) { + leftv[i][2] = '0' + i % 2; + leftv[i][3] = 'a'; + leftv[i][4] = '0' + i % 2; + varDataSetLen(leftv[i], 3); + } - SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); - ASSERT_EQ(res->info.rows, rowNum); - SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); - ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BOOL].bytes); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((bool *)colDataGetData(column, i)), eRes[i]); - } - taosArrayDestroyEx(blockList, scltFreeDataBlock); - nodesDestroyNode(opNode); + int32_t rowNum = sizeof(leftv)/sizeof(leftv[0]); + scltMakeColumnNode(&pLeft, &src, TSDB_DATA_TYPE_BINARY, 3, rowNum, leftv); + + SColumnInfoData *pcolumn = (SColumnInfoData *)taosArrayGetLast(src->pDataBlock); + colDataAppend(pcolumn, 4, NULL, true); + + scltMakeOpNode(&opNode, OP_TYPE_IS_NOT_NULL, TSDB_DATA_TYPE_BOOL, pLeft, NULL); + + SArray *blockList = taosArrayInit(1, POINTER_BYTES); + taosArrayPush(blockList, &src); + SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BOOL, sizeof(bool)); + int16_t dataBlockId = 0, slotId = 0; + scltAppendReservedSlot(blockList, &dataBlockId, &slotId, false, rowNum, &colInfo); + scltMakeTargetNode(&opNode, dataBlockId, slotId, opNode); + + int32_t code = scalarCalculate(opNode, blockList, NULL); + ASSERT_EQ(code, 0); + + SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); + ASSERT_EQ(res->info.rows, rowNum); + SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); + ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BOOL].bytes); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((bool *)colDataGetData(column, i)), eRes[i]); + } + taosArrayDestroyEx(blockList, scltFreeDataBlock); + nodesDestroyNode(opNode); } TEST(columnTest, greater_and_lower) { - SNode *pcol1 = NULL, *pcol2 = NULL, *opNode1 = NULL, *opNode2 = NULL, *logicNode = NULL; - SNode *list[2] = {0}; - int16_t v1[5]= {1, 2, 3, 4, 5}; - int32_t v2[5]= {5, 1, 4, 2, 6}; - int64_t v3[5]= {1, 2, 3, 4, 5}; - int32_t v4[5]= {5, 3, 4, 2, 6}; - bool eRes[5] = {false, true, false, false, false}; - SSDataBlock *src = NULL; - int32_t rowNum = sizeof(v1)/sizeof(v1[0]); - scltMakeColumnNode(&pcol1, &src, TSDB_DATA_TYPE_SMALLINT, sizeof(int16_t), rowNum, v1); - scltMakeColumnNode(&pcol2, &src, TSDB_DATA_TYPE_INT, sizeof(int32_t), rowNum, v2); - scltMakeOpNode(&opNode1, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pcol1, pcol2); - scltMakeColumnNode(&pcol1, &src, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t), rowNum, v3); - scltMakeColumnNode(&pcol2, &src, TSDB_DATA_TYPE_INT, sizeof(int32_t), rowNum, v4); - scltMakeOpNode(&opNode2, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pcol1, pcol2); - list[0] = opNode1; - list[1] = opNode2; - scltMakeLogicNode(&logicNode, LOGIC_COND_TYPE_AND, list, 2); - - SArray *blockList = taosArrayInit(1, POINTER_BYTES); - taosArrayPush(blockList, &src); - SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BOOL, sizeof(bool)); - int16_t dataBlockId = 0, slotId = 0; - scltAppendReservedSlot(blockList, &dataBlockId, &slotId, false, rowNum, &colInfo); - scltMakeTargetNode(&logicNode, dataBlockId, slotId, logicNode); - - int32_t code = scalarCalculate(logicNode, blockList, NULL); - ASSERT_EQ(code, 0); + SNode *pcol1 = NULL, *pcol2 = NULL, *opNode1 = NULL, *opNode2 = NULL, *logicNode = NULL; + SNode *list[2] = {0}; + int16_t v1[5]= {1, 2, 3, 4, 5}; + int32_t v2[5]= {5, 1, 4, 2, 6}; + int64_t v3[5]= {1, 2, 3, 4, 5}; + int32_t v4[5]= {5, 3, 4, 2, 6}; + bool eRes[5] = {false, true, false, false, false}; + SSDataBlock *src = NULL; + int32_t rowNum = sizeof(v1)/sizeof(v1[0]); + scltMakeColumnNode(&pcol1, &src, TSDB_DATA_TYPE_SMALLINT, sizeof(int16_t), rowNum, v1); + scltMakeColumnNode(&pcol2, &src, TSDB_DATA_TYPE_INT, sizeof(int32_t), rowNum, v2); + scltMakeOpNode(&opNode1, OP_TYPE_GREATER_THAN, TSDB_DATA_TYPE_BOOL, pcol1, pcol2); + scltMakeColumnNode(&pcol1, &src, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t), rowNum, v3); + scltMakeColumnNode(&pcol2, &src, TSDB_DATA_TYPE_INT, sizeof(int32_t), rowNum, v4); + scltMakeOpNode(&opNode2, OP_TYPE_LOWER_THAN, TSDB_DATA_TYPE_BOOL, pcol1, pcol2); + list[0] = opNode1; + list[1] = opNode2; + scltMakeLogicNode(&logicNode, LOGIC_COND_TYPE_AND, list, 2); - SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); - ASSERT_EQ(res->info.rows, rowNum); - SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); - ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BOOL); - ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BOOL].bytes); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((bool *)colDataGetData(column, i)), eRes[i]); - } - taosArrayDestroyEx(blockList, scltFreeDataBlock); - nodesDestroyNode(logicNode); + SArray *blockList = taosArrayInit(1, POINTER_BYTES); + taosArrayPush(blockList, &src); + SColumnInfo colInfo = createColumnInfo(1, TSDB_DATA_TYPE_BOOL, sizeof(bool)); + int16_t dataBlockId = 0, slotId = 0; + scltAppendReservedSlot(blockList, &dataBlockId, &slotId, false, rowNum, &colInfo); + scltMakeTargetNode(&logicNode, dataBlockId, slotId, logicNode); + + int32_t code = scalarCalculate(logicNode, blockList, NULL); + ASSERT_EQ(code, 0); + + SSDataBlock *res = *(SSDataBlock **)taosArrayGetLast(blockList); + ASSERT_EQ(res->info.rows, rowNum); + SColumnInfoData *column = (SColumnInfoData *)taosArrayGetLast(res->pDataBlock); + ASSERT_EQ(column->info.type, TSDB_DATA_TYPE_BOOL); + ASSERT_EQ(column->info.bytes, tDataTypes[TSDB_DATA_TYPE_BOOL].bytes); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((bool *)colDataGetData(column, i)), eRes[i]); + } + taosArrayDestroyEx(blockList, scltFreeDataBlock); + nodesDestroyNode(logicNode); } void scltMakeDataBlock(SScalarParam **pInput, int32_t type, void *pVal, int32_t num, bool setVal) { - SScalarParam *input = (SScalarParam *)taosMemoryCalloc(1, sizeof(SScalarParam)); - int32_t bytes; - switch (type) { - case TSDB_DATA_TYPE_TINYINT: { - bytes = sizeof(int8_t); - break; - } - case TSDB_DATA_TYPE_SMALLINT: { - bytes = sizeof(int16_t); - break; - } - case TSDB_DATA_TYPE_INT: { - bytes = sizeof(int32_t); - break; - } - case TSDB_DATA_TYPE_BIGINT: { - bytes = sizeof(int64_t); - break; - } - case TSDB_DATA_TYPE_FLOAT: { - bytes = sizeof(float); - break; - } - case TSDB_DATA_TYPE_DOUBLE: { - bytes = sizeof(double); - break; - } - } + SScalarParam *input = (SScalarParam *)taosMemoryCalloc(1, sizeof(SScalarParam)); + int32_t bytes; + switch (type) { + case TSDB_DATA_TYPE_TINYINT: { + bytes = sizeof(int8_t); + break; + } + case TSDB_DATA_TYPE_SMALLINT: { + bytes = sizeof(int16_t); + break; + } + case TSDB_DATA_TYPE_INT: { + bytes = sizeof(int32_t); + break; + } + case TSDB_DATA_TYPE_BIGINT: { + bytes = sizeof(int64_t); + break; + } + case TSDB_DATA_TYPE_FLOAT: { + bytes = sizeof(float); + break; + } + case TSDB_DATA_TYPE_DOUBLE: { + bytes = sizeof(double); + break; + } + } - input->columnData = (SColumnInfoData*) taosMemoryCalloc(1, sizeof(SColumnInfoData)); - input->numOfRows = num; + input->columnData = (SColumnInfoData*) taosMemoryCalloc(1, sizeof(SColumnInfoData)); + input->numOfRows = num; - input->columnData->info = createColumnInfo(0, type, bytes); - colInfoDataEnsureCapacity(input->columnData, 0, num); + input->columnData->info = createColumnInfo(0, type, bytes); + colInfoDataEnsureCapacity(input->columnData, 0, num); - if (setVal) { - for (int32_t i = 0; i < num; ++i) { - colDataAppend(input->columnData, i, (const char*) pVal, false); - } - } else { -// memset(input->data, 0, num * bytes); - } + if (setVal) { + for (int32_t i = 0; i < num; ++i) { + colDataAppend(input->columnData, i, (const char*) pVal, false); + } + } else { + // memset(input->data, 0, num * bytes); + } - *pInput = input; + *pInput = input; } void scltDestroyDataBlock(SScalarParam *pInput) { - colDataDestroy(pInput->columnData); - taosMemoryFree(pInput->columnData); - taosMemoryFree(pInput); + colDataDestroy(pInput->columnData); + taosMemoryFree(pInput->columnData); + taosMemoryFree(pInput); } TEST(ScalarFunctionTest, absFunction_constant) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; - //TINYINT - int8_t val_tinyint = 10; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + //TINYINT + int8_t val_tinyint = 10; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), val_tinyint); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), val_tinyint); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - val_tinyint = -10; - scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + val_tinyint = -10; + scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), -val_tinyint); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), -val_tinyint); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //SMALLINT - int16_t val_smallint = 10; - type = TSDB_DATA_TYPE_SMALLINT; - scltMakeDataBlock(&pInput, type, &val_smallint, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + //SMALLINT + int16_t val_smallint = 10; + type = TSDB_DATA_TYPE_SMALLINT; + scltMakeDataBlock(&pInput, type, &val_smallint, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int16_t *)colDataGetData(pOutput->columnData, i)), val_smallint); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int16_t *)colDataGetData(pOutput->columnData, i)), val_smallint); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - val_smallint = -10; - scltMakeDataBlock(&pInput, type, &val_smallint, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + val_smallint = -10; + scltMakeDataBlock(&pInput, type, &val_smallint, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int16_t *)colDataGetData(pOutput->columnData, i)), -val_smallint); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int16_t *)colDataGetData(pOutput->columnData, i)), -val_smallint); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //INT - int32_t val_int = 10; - type = TSDB_DATA_TYPE_INT; - scltMakeDataBlock(&pInput, type, &val_int, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + //INT + int32_t val_int = 10; + type = TSDB_DATA_TYPE_INT; + scltMakeDataBlock(&pInput, type, &val_int, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int32_t *)colDataGetData(pOutput->columnData, i)), val_int); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int32_t *)colDataGetData(pOutput->columnData, i)), val_int); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - val_int = -10; - scltMakeDataBlock(&pInput, type, &val_int, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + val_int = -10; + scltMakeDataBlock(&pInput, type, &val_int, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int32_t *)colDataGetData(pOutput->columnData, i)), -val_int); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int32_t *)colDataGetData(pOutput->columnData, i)), -val_int); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //BIGINT - int64_t val_bigint = 10; - type = TSDB_DATA_TYPE_BIGINT; - scltMakeDataBlock(&pInput, type, &val_bigint, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + //BIGINT + int64_t val_bigint = 10; + type = TSDB_DATA_TYPE_BIGINT; + scltMakeDataBlock(&pInput, type, &val_bigint, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int64_t *)colDataGetData(pOutput->columnData, i)), val_bigint); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int64_t *)colDataGetData(pOutput->columnData, i)), val_bigint); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - val_bigint = -10; - scltMakeDataBlock(&pInput, type, &val_bigint, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + val_bigint = -10; + scltMakeDataBlock(&pInput, type, &val_bigint, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int64_t *)colDataGetData(pOutput->columnData, i)), -val_bigint); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int64_t *)colDataGetData(pOutput->columnData, i)), -val_bigint); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float = 10.15; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - PRINTF("float before ABS:%f\n", *(float *)pInput->data); + //FLOAT + float val_float = 10.15; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + PRINTF("float before ABS:%f\n", *(float *)pInput->data); - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), val_float); - PRINTF("float after ABS:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), val_float); + PRINTF("float after ABS:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - val_float = -10.15; - scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - PRINTF("float before ABS:%f\n", *(float *)pInput->data); + val_float = -10.15; + scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + PRINTF("float before ABS:%f\n", *(float *)pInput->data); - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), -val_float); - PRINTF("float after ABS:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), -val_float); + PRINTF("float after ABS:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //DOUBLE - double val_double = 10.15; - type = TSDB_DATA_TYPE_DOUBLE; - scltMakeDataBlock(&pInput, type, &val_double, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + //DOUBLE + double val_double = 10.15; + type = TSDB_DATA_TYPE_DOUBLE; + scltMakeDataBlock(&pInput, type, &val_double, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), val_double); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), val_double); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - val_double = -10.15; - scltMakeDataBlock(&pInput, type, &val_double, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + val_double = -10.15; + scltMakeDataBlock(&pInput, type, &val_double, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), -val_double); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), -val_double); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, absFunction_column) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 5; - int32_t type; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 5; + int32_t type; - //TINYINT - int8_t val_tinyint = 10; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - int8_t v = val_tinyint + i; - colDataAppend(pInput->columnData, i, (const char*) &v, false); - PRINTF("tiny_int before ABS:%d\n", v); - } + //TINYINT + int8_t val_tinyint = 10; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + int8_t v = val_tinyint + i; + colDataAppend(pInput->columnData, i, (const char*) &v, false); + PRINTF("tiny_int before ABS:%d\n", v); + } - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), val_tinyint + i); - PRINTF("tiny_int after ABS:%d\n", *((int8_t *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), val_tinyint + i); + PRINTF("tiny_int after ABS:%d\n", *((int8_t *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - val_tinyint = -10; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - int8_t v = val_tinyint + i; - colDataAppend(pInput->columnData, i, (const char*) &v, false); - PRINTF("tiny_int before ABS:%d\n", v); - } + val_tinyint = -10; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + int8_t v = val_tinyint + i; + colDataAppend(pInput->columnData, i, (const char*) &v, false); + PRINTF("tiny_int before ABS:%d\n", v); + } - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), -(val_tinyint + i)); - PRINTF("tiny_int after ABS:%d\n", *((int8_t *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), -(val_tinyint + i)); + PRINTF("tiny_int after ABS:%d\n", *((int8_t *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //SMALLINT - int16_t val_smallint = 10; - type = TSDB_DATA_TYPE_SMALLINT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - int16_t v = val_smallint + i; - colDataAppend(pInput->columnData, i, (const char*) &v, false); - PRINTF("small_int before ABS:%d\n", v); - } + //SMALLINT + int16_t val_smallint = 10; + type = TSDB_DATA_TYPE_SMALLINT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + int16_t v = val_smallint + i; + colDataAppend(pInput->columnData, i, (const char*) &v, false); + PRINTF("small_int before ABS:%d\n", v); + } - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int16_t *)colDataGetData(pOutput->columnData, i)), val_smallint + i); - PRINTF("small_int after ABS:%d\n", *((int16_t *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int16_t *)colDataGetData(pOutput->columnData, i)), val_smallint + i); + PRINTF("small_int after ABS:%d\n", *((int16_t *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - val_smallint = -10; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - int16_t v = val_smallint + i; - colDataAppend(pInput->columnData, i, (const char*) &v, false); - PRINTF("small_int before ABS:%d\n", v); - } + val_smallint = -10; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + int16_t v = val_smallint + i; + colDataAppend(pInput->columnData, i, (const char*) &v, false); + PRINTF("small_int before ABS:%d\n", v); + } - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int16_t *)colDataGetData(pOutput->columnData, i)), -(val_smallint + i)); - PRINTF("small_int after ABS:%d\n", *((int16_t *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int16_t *)colDataGetData(pOutput->columnData, i)), -(val_smallint + i)); + PRINTF("small_int after ABS:%d\n", *((int16_t *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //INT - int32_t val_int = 10; - type = TSDB_DATA_TYPE_INT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - int32_t v = val_int + i; - colDataAppend(pInput->columnData, i, (const char*) &v, false); - PRINTF("int before ABS:%d\n", v); - } + //INT + int32_t val_int = 10; + type = TSDB_DATA_TYPE_INT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + int32_t v = val_int + i; + colDataAppend(pInput->columnData, i, (const char*) &v, false); + PRINTF("int before ABS:%d\n", v); + } - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int32_t *)colDataGetData(pOutput->columnData, i)), val_int + i); - PRINTF("int after ABS:%d\n", *((int32_t *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int32_t *)colDataGetData(pOutput->columnData, i)), val_int + i); + PRINTF("int after ABS:%d\n", *((int32_t *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - val_int = -10; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - int32_t v = val_int + i; - colDataAppend(pInput->columnData, i, (const char*) &v, false); - PRINTF("int before ABS:%d\n", v); - } + val_int = -10; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + int32_t v = val_int + i; + colDataAppend(pInput->columnData, i, (const char*) &v, false); + PRINTF("int before ABS:%d\n", v); + } - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int32_t *)colDataGetData(pOutput->columnData, i)), -(val_int + i)); - PRINTF("int after ABS:%d\n", *((int32_t *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int32_t *)colDataGetData(pOutput->columnData, i)), -(val_int + i)); + PRINTF("int after ABS:%d\n", *((int32_t *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float = 10.15; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - float v = val_float + i; - colDataAppend(pInput->columnData, i, (const char*) &v, false); - PRINTF("float before ABS:%f\n", v); - } + //FLOAT + float val_float = 10.15; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + float v = val_float + i; + colDataAppend(pInput->columnData, i, (const char*) &v, false); + PRINTF("float before ABS:%f\n", v); + } - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), val_float + i); - PRINTF("float after ABS:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), val_float + i); + PRINTF("float after ABS:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - val_float = -10.15; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - float v = val_float + i; - colDataAppend(pInput->columnData, i, (const char*) &v, false); - PRINTF("float before ABS:%f\n", v); - } + val_float = -10.15; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + float v = val_float + i; + colDataAppend(pInput->columnData, i, (const char*) &v, false); + PRINTF("float before ABS:%f\n", v); + } - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), -(val_float + i)); - PRINTF("float after ABS:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), -(val_float + i)); + PRINTF("float after ABS:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //DOUBLE - double val_double = 10.15; - type = TSDB_DATA_TYPE_DOUBLE; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - double v = val_double + i; - colDataAppend(pInput->columnData, i, (const char*) &v, false); - PRINTF("double before ABS:%f\n", v); - } + //DOUBLE + double val_double = 10.15; + type = TSDB_DATA_TYPE_DOUBLE; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + double v = val_double + i; + colDataAppend(pInput->columnData, i, (const char*) &v, false); + PRINTF("double before ABS:%f\n", v); + } - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), val_double + i); - PRINTF("double after ABS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), val_double + i); + PRINTF("double after ABS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - val_double = -10.15; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - double v = val_double + i; - colDataAppend(pInput->columnData, i, (const char*) &v, false); - PRINTF("double before ABS:%f\n", v); - } + val_double = -10.15; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + double v = val_double + i; + colDataAppend(pInput->columnData, i, (const char*) &v, false); + PRINTF("double before ABS:%f\n", v); + } - code = absFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), -(val_double + i)); - PRINTF("double after ABS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = absFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), -(val_double + i)); + PRINTF("double after ABS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, sinFunction_constant) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result = 0.42016703682664092; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result = 0.42016703682664092; - //TINYINT - int8_t val_tinyint = 13; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("tiny_int before SIN:%d\n", *((int8_t *)pInput->data)); + //TINYINT + int8_t val_tinyint = 13; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("tiny_int before SIN:%d\n", *((int8_t *)pInput->data)); - code = sinFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("tiny_int after SIN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = sinFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("tiny_int after SIN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float = 13.00; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("float before SIN:%f\n", *((float *)pInput->data)); + //FLOAT + float val_float = 13.00; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("float before SIN:%f\n", *((float *)pInput->data)); - code = sinFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("float after SIN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = sinFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("float after SIN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, sinFunction_column) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result[] = {0.42016703682664092, 0.99060735569487035, 0.65028784015711683}; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result[] = {0.42016703682664092, 0.99060735569487035, 0.65028784015711683}; - //TINYINT - int8_t val_tinyint[] = {13, 14, 15}; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - colDataAppend(pInput->columnData, i, (const char*) &val_tinyint[i], false); - PRINTF("tiny_int before SIN:%d\n", *(int8_t *)colDataGetData(pInput->columnData, i)); - } + //TINYINT + int8_t val_tinyint[] = {13, 14, 15}; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + colDataAppend(pInput->columnData, i, (const char*) &val_tinyint[i], false); + PRINTF("tiny_int before SIN:%d\n", *(int8_t *)colDataGetData(pInput->columnData, i)); + } - code = sinFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("tiny_int after SIN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = sinFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("tiny_int after SIN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float[] = {13.00, 14.00, 15.00}; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - colDataAppend(pInput->columnData, i, (const char*) &val_float[i], false); - PRINTF("float before SIN:%f\n", *((float *)colDataGetData(pInput->columnData, i))); - } + //FLOAT + float val_float[] = {13.00, 14.00, 15.00}; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + colDataAppend(pInput->columnData, i, (const char*) &val_float[i], false); + PRINTF("float before SIN:%f\n", *((float *)colDataGetData(pInput->columnData, i))); + } - code = sinFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("float after SIN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = sinFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("float after SIN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, cosFunction_constant) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result = 0.90744678145019619; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result = 0.90744678145019619; - //TINYINT - int8_t val_tinyint = 13; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("tiny_int before COS:%d\n", *((int8_t *)pInput->data)); + //TINYINT + int8_t val_tinyint = 13; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("tiny_int before COS:%d\n", *((int8_t *)pInput->data)); - code = cosFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("tiny_int after COS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = cosFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("tiny_int after COS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float = 13.00; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("float before COS:%f\n", *((float *)pInput->data)); + //FLOAT + float val_float = 13.00; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("float before COS:%f\n", *((float *)pInput->data)); - code = cosFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("float after COS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = cosFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("float after COS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, cosFunction_column) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result[] = {0.90744678145019619, 0.13673721820783361, -0.75968791285882131}; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result[] = {0.90744678145019619, 0.13673721820783361, -0.75968791285882131}; - //TINYINT - int8_t val_tinyint[] = {13, 14, 15}; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - colDataAppend(pInput->columnData, i, (const char*) &val_tinyint[i], false); - PRINTF("tiny_int before COS:%d\n", *(int8_t *)colDataGetData(pInput->columnData, i)); - } + //TINYINT + int8_t val_tinyint[] = {13, 14, 15}; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + colDataAppend(pInput->columnData, i, (const char*) &val_tinyint[i], false); + PRINTF("tiny_int before COS:%d\n", *(int8_t *)colDataGetData(pInput->columnData, i)); + } - code = cosFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("tiny_int after COS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = cosFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("tiny_int after COS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float[] = {13.00, 14.00, 15.00}; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - colDataAppend(pInput->columnData, i, (const char*) &val_float[i], false); - PRINTF("float before COS:%f\n", *(float *)colDataGetData(pInput->columnData, i)); - } + //FLOAT + float val_float[] = {13.00, 14.00, 15.00}; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + colDataAppend(pInput->columnData, i, (const char*) &val_float[i], false); + PRINTF("float before COS:%f\n", *(float *)colDataGetData(pInput->columnData, i)); + } - code = cosFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("float after COS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = cosFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("float after COS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, tanFunction_constant) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result = 0.46302113293648961; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result = 0.46302113293648961; - //TINYINT - int8_t val_tinyint = 13; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("tiny_int before TAN:%d\n", *((int8_t *)pInput->data)); + //TINYINT + int8_t val_tinyint = 13; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("tiny_int before TAN:%d\n", *((int8_t *)pInput->data)); - code = tanFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("tiny_int after TAN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = tanFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("tiny_int after TAN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float = 13.00; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("float before TAN:%f\n", *((float *)pInput->data)); + //FLOAT + float val_float = 13.00; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("float before TAN:%f\n", *((float *)pInput->data)); - code = tanFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("float after TAN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = tanFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("float after TAN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, tanFunction_column) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result[] = {0.46302113293648961, 7.24460661609480550, -0.85599340090851872}; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result[] = {0.46302113293648961, 7.24460661609480550, -0.85599340090851872}; - //TINYINT - int8_t val_tinyint[] = {13, 14, 15}; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - colDataAppend(pInput->columnData, i, (const char*) &val_tinyint[i], false); - PRINTF("tiny_int before TAN:%d\n", *((int8_t *)colDataGetData(pInput->columnData, i))); - } + //TINYINT + int8_t val_tinyint[] = {13, 14, 15}; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + colDataAppend(pInput->columnData, i, (const char*) &val_tinyint[i], false); + PRINTF("tiny_int before TAN:%d\n", *((int8_t *)colDataGetData(pInput->columnData, i))); + } - code = tanFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_NEAR(*((double *)colDataGetData(pOutput->columnData, i)), result[i], 1e-15); - PRINTF("tiny_int after TAN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = tanFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_NEAR(*((double *)colDataGetData(pOutput->columnData, i)), result[i], 1e-15); + PRINTF("tiny_int after TAN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float[] = {13.00, 14.00, 15.00}; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - colDataAppend(pInput->columnData, i, (const char*) &val_float[i], false); - PRINTF("float before TAN:%f\n", *((float *)colDataGetData(pInput->columnData, i))); - } + //FLOAT + float val_float[] = {13.00, 14.00, 15.00}; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + colDataAppend(pInput->columnData, i, (const char*) &val_float[i], false); + PRINTF("float before TAN:%f\n", *((float *)colDataGetData(pInput->columnData, i))); + } - code = tanFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_NEAR(*((double *)colDataGetData(pOutput->columnData, i)), result[i], 1e-15); - PRINTF("float after TAN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = tanFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_NEAR(*((double *)colDataGetData(pOutput->columnData, i)), result[i], 1e-15); + PRINTF("float after TAN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, asinFunction_constant) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result = 1.57079632679489656; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result = 1.57079632679489656; - //TINYINT - int8_t val_tinyint = 1; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("tiny_int before ASIN:%d\n", *((int8_t *)pInput->data)); + //TINYINT + int8_t val_tinyint = 1; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("tiny_int before ASIN:%d\n", *((int8_t *)pInput->data)); - code = asinFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("tiny_int after ASIN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = asinFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("tiny_int after ASIN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float = 1.00; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("float before ASIN:%f\n", *((float *)pInput->data)); + //FLOAT + float val_float = 1.00; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("float before ASIN:%f\n", *((float *)pInput->data)); - code = asinFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("float after ASIN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = asinFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("float after ASIN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, asinFunction_column) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result[] = {-1.57079632679489656, 0.0, 1.57079632679489656}; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result[] = {-1.57079632679489656, 0.0, 1.57079632679489656}; - //TINYINT - int8_t val_tinyint[] = {-1, 0, 1}; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - colDataAppend(pInput->columnData, i, (const char*) &val_tinyint[i], false); - PRINTF("tiny_int before ASIN:%d\n", *((int8_t *)colDataGetData(pInput->columnData, i))); - } + //TINYINT + int8_t val_tinyint[] = {-1, 0, 1}; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + colDataAppend(pInput->columnData, i, (const char*) &val_tinyint[i], false); + PRINTF("tiny_int before ASIN:%d\n", *((int8_t *)colDataGetData(pInput->columnData, i))); + } - code = asinFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("tiny_int after ASIN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = asinFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("tiny_int after ASIN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float[] = {-1.0, 0.0, 1.0}; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - *((float *)colDataGetData(pInput->columnData, i)) = val_float[i]; - PRINTF("float before ASIN:%f\n", *((float *)colDataGetData(pInput->columnData, i))); - } + //FLOAT + float val_float[] = {-1.0, 0.0, 1.0}; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + *((float *)colDataGetData(pInput->columnData, i)) = val_float[i]; + PRINTF("float before ASIN:%f\n", *((float *)colDataGetData(pInput->columnData, i))); + } - code = asinFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("float after ASIN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = asinFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("float after ASIN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, acosFunction_constant) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result = 0.0; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result = 0.0; - //TINYINT - int8_t val_tinyint = 1; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("tiny_int before ACOS:%d\n", *((int8_t *)pInput->data)); + //TINYINT + int8_t val_tinyint = 1; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("tiny_int before ACOS:%d\n", *((int8_t *)pInput->data)); - code = acosFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("tiny_int after ACOS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = acosFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("tiny_int after ACOS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float = 1.00; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("float before ACOS:%f\n", *((float *)pInput->data)); + //FLOAT + float val_float = 1.00; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("float before ACOS:%f\n", *((float *)pInput->data)); - code = acosFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("float after ACOS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = acosFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("float after ACOS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, acosFunction_column) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result[] = {3.14159265358979312, 1.57079632679489656, 0.0}; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result[] = {3.14159265358979312, 1.57079632679489656, 0.0}; - //TINYINT - int8_t val_tinyint[] = {-1, 0, 1}; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - *((int8_t *)colDataGetData(pInput->columnData, i)) = val_tinyint[i]; - PRINTF("tiny_int before ACOS:%d\n", *((int8_t *)colDataGetData(pInput->columnData, i))); - } + //TINYINT + int8_t val_tinyint[] = {-1, 0, 1}; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + *((int8_t *)colDataGetData(pInput->columnData, i)) = val_tinyint[i]; + PRINTF("tiny_int before ACOS:%d\n", *((int8_t *)colDataGetData(pInput->columnData, i))); + } - code = acosFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("tiny_int after ACOS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = acosFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("tiny_int after ACOS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float[] = {-1.0, 0.0, 1.0}; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - *((float *)colDataGetData(pInput->columnData, i)) = val_float[i]; - PRINTF("float before ACOS:%f\n", *((float *)colDataGetData(pInput->columnData, i))); - } + //FLOAT + float val_float[] = {-1.0, 0.0, 1.0}; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + *((float *)colDataGetData(pInput->columnData, i)) = val_float[i]; + PRINTF("float before ACOS:%f\n", *((float *)colDataGetData(pInput->columnData, i))); + } - code = acosFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("float after ACOS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = acosFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("float after ACOS:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, atanFunction_constant) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result = 0.78539816339744828; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result = 0.78539816339744828; - //TINYINT - int8_t val_tinyint = 1; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("tiny_int before ATAN:%d\n", *((int8_t *)pInput->data)); + //TINYINT + int8_t val_tinyint = 1; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("tiny_int before ATAN:%d\n", *((int8_t *)pInput->data)); - code = atanFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("tiny_int after ATAN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = atanFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("tiny_int after ATAN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float = 1.00; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("float before ATAN:%f\n", *((float *)pInput->data)); + //FLOAT + float val_float = 1.00; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("float before ATAN:%f\n", *((float *)pInput->data)); - code = atanFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("float after ATAN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = atanFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("float after ATAN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, atanFunction_column) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result[] = {-0.78539816339744828, 0.0, 0.78539816339744828}; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result[] = {-0.78539816339744828, 0.0, 0.78539816339744828}; - //TINYINT - int8_t val_tinyint[] = {-1, 0, 1}; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - *((int8_t *)colDataGetData(pInput->columnData, i)) = val_tinyint[i]; - PRINTF("tiny_int before ATAN:%d\n", *((int8_t *)colDataGetData(pInput->columnData, i))); - } + //TINYINT + int8_t val_tinyint[] = {-1, 0, 1}; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + *((int8_t *)colDataGetData(pInput->columnData, i)) = val_tinyint[i]; + PRINTF("tiny_int before ATAN:%d\n", *((int8_t *)colDataGetData(pInput->columnData, i))); + } - code = atanFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("tiny_int after ATAN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = atanFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("tiny_int after ATAN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float[] = {-1.0, 0.0, 1.0}; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - *((float *)colDataGetData(pInput->columnData, i)) = val_float[i]; - PRINTF("float before ATAN:%f\n", *((float *)colDataGetData(pInput->columnData, i))); - } + //FLOAT + float val_float[] = {-1.0, 0.0, 1.0}; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + *((float *)colDataGetData(pInput->columnData, i)) = val_float[i]; + PRINTF("float before ATAN:%f\n", *((float *)colDataGetData(pInput->columnData, i))); + } - code = atanFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("float after ATAN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = atanFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("float after ATAN:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, ceilFunction_constant) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - double result = 10.0; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + double result = 10.0; - //TINYINT - int8_t val_tinyint = 10; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - PRINTF("tiny_int before CEIL:%d\n", *((int8_t *)pInput->data)); + //TINYINT + int8_t val_tinyint = 10; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + PRINTF("tiny_int before CEIL:%d\n", *((int8_t *)pInput->data)); - code = ceilFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), (int8_t)result); - PRINTF("tiny_int after CEIL:%d\n", *((int8_t *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = ceilFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), (int8_t)result); + PRINTF("tiny_int after CEIL:%d\n", *((int8_t *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float = 9.5; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - PRINTF("float before CEIL:%f\n", *((float *)pInput->data)); + //FLOAT + float val_float = 9.5; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + PRINTF("float before CEIL:%f\n", *((float *)pInput->data)); - code = ceilFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), (float)result); - PRINTF("float after CEIL:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); - } + code = ceilFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), (float)result); + PRINTF("float after CEIL:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, ceilFunction_column) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - double result[] = {-10.0, 0.0, 10.0}; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + double result[] = {-10.0, 0.0, 10.0}; - //TINYINT - int8_t val_tinyint[] = {-10, 0, 10}; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - *((int8_t *)colDataGetData(pInput->columnData, i)) = val_tinyint[i]; - PRINTF("tiny_int before CEIL:%d\n", *((int8_t *)colDataGetData(pInput->columnData, i))); - } + //TINYINT + int8_t val_tinyint[] = {-10, 0, 10}; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + *((int8_t *)colDataGetData(pInput->columnData, i)) = val_tinyint[i]; + PRINTF("tiny_int before CEIL:%d\n", *((int8_t *)colDataGetData(pInput->columnData, i))); + } - code = ceilFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("tiny_int after CEIL:%d\n", *((int8_t *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = ceilFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("tiny_int after CEIL:%d\n", *((int8_t *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float[] = {-10.5, 0.0, 9.5}; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - *((float *)colDataGetData(pInput->columnData, i)) = val_float[i]; - PRINTF("float before CEIL:%f\n", *((float *)colDataGetData(pInput->columnData, i))); - } + //FLOAT + float val_float[] = {-10.5, 0.0, 9.5}; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + *((float *)colDataGetData(pInput->columnData, i)) = val_float[i]; + PRINTF("float before CEIL:%f\n", *((float *)colDataGetData(pInput->columnData, i))); + } - code = ceilFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("float after CEIL:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); - } + code = ceilFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("float after CEIL:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, floorFunction_constant) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - double result = 10.0; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + double result = 10.0; - //TINYINT - int8_t val_tinyint = 10; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - PRINTF("tiny_int before FLOOR:%d\n", *((int8_t *)pInput->data)); + //TINYINT + int8_t val_tinyint = 10; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + PRINTF("tiny_int before FLOOR:%d\n", *((int8_t *)pInput->data)); - code = floorFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), (int8_t)result); - PRINTF("tiny_int after FLOOR:%d\n", *((int8_t *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = floorFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), (int8_t)result); + PRINTF("tiny_int after FLOOR:%d\n", *((int8_t *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float = 10.5; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - PRINTF("float before FLOOR:%f\n", *((float *)pInput->data)); + //FLOAT + float val_float = 10.5; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + PRINTF("float before FLOOR:%f\n", *((float *)pInput->data)); - code = floorFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), (float)result); - PRINTF("float after FLOOR:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); - } + code = floorFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), (float)result); + PRINTF("float after FLOOR:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, floorFunction_column) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - double result[] = {-10.0, 0.0, 10.0}; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + double result[] = {-10.0, 0.0, 10.0}; - //TINYINT - int8_t val_tinyint[] = {-10, 0, 10}; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - *((int8_t *)colDataGetData(pInput->columnData, i)) = val_tinyint[i]; - PRINTF("tiny_int before FLOOR:%d\n", *((int8_t *)colDataGetData(pInput->columnData, i))); - } + //TINYINT + int8_t val_tinyint[] = {-10, 0, 10}; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + *((int8_t *)colDataGetData(pInput->columnData, i)) = val_tinyint[i]; + PRINTF("tiny_int before FLOOR:%d\n", *((int8_t *)colDataGetData(pInput->columnData, i))); + } - code = floorFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("tiny_int after FLOOR:%d\n", *((int8_t *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = floorFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("tiny_int after FLOOR:%d\n", *((int8_t *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float[] = {-9.5, 0.0, 10.5}; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - *((float *)colDataGetData(pInput->columnData, i)) = val_float[i]; - PRINTF("float before FLOOR:%f\n", *((float *)colDataGetData(pInput->columnData, i))); - } + //FLOAT + float val_float[] = {-9.5, 0.0, 10.5}; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + *((float *)colDataGetData(pInput->columnData, i)) = val_float[i]; + PRINTF("float before FLOOR:%f\n", *((float *)colDataGetData(pInput->columnData, i))); + } - code = floorFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("float after FLOOR:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); - } + code = floorFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("float after FLOOR:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, roundFunction_constant) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - double result = 10.0; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + double result = 10.0; - //TINYINT - int8_t val_tinyint = 10; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - PRINTF("tiny_int before ROUND:%d\n", *((int8_t *)pInput->data)); + //TINYINT + int8_t val_tinyint = 10; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + PRINTF("tiny_int before ROUND:%d\n", *((int8_t *)pInput->data)); - code = roundFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), (int8_t)result); - PRINTF("tiny_int after ROUND:%d\n", *((int8_t *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = roundFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), (int8_t)result); + PRINTF("tiny_int after ROUND:%d\n", *((int8_t *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float = 9.5; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - PRINTF("float before ROUND:%f\n", *((float *)pInput->data)); + //FLOAT + float val_float = 9.5; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + PRINTF("float before ROUND:%f\n", *((float *)pInput->data)); - code = roundFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), (float)result); - PRINTF("float after ROUND:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); - } + code = roundFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), (float)result); + PRINTF("float after ROUND:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, roundFunction_column) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - double result[] = {-10.0, 0.0, 10.0}; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + double result[] = {-10.0, 0.0, 10.0}; - //TINYINT - int8_t val_tinyint[] = {-10, 0, 10}; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - *((int8_t *)colDataGetData(pInput->columnData, i)) = val_tinyint[i]; - PRINTF("tiny_int before ROUND:%d\n", *((int8_t *)colDataGetData(pInput->columnData, i))); - } + //TINYINT + int8_t val_tinyint[] = {-10, 0, 10}; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + *((int8_t *)colDataGetData(pInput->columnData, i)) = val_tinyint[i]; + PRINTF("tiny_int before ROUND:%d\n", *((int8_t *)colDataGetData(pInput->columnData, i))); + } - code = roundFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("tiny_int after ROUND:%d\n", *((int8_t *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = roundFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((int8_t *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("tiny_int after ROUND:%d\n", *((int8_t *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float[] = {-9.5, 0.0, 9.5}; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, type, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - *((float *)colDataGetData(pInput->columnData, i)) = val_float[i]; - PRINTF("float before ROUND:%f\n", *((float *)colDataGetData(pInput->columnData, i))); - } + //FLOAT + float val_float[] = {-9.5, 0.0, 9.5}; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, type, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + *((float *)colDataGetData(pInput->columnData, i)) = val_float[i]; + PRINTF("float before ROUND:%f\n", *((float *)colDataGetData(pInput->columnData, i))); + } - code = roundFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("float after ROUND:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); - } + code = roundFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((float *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("float after ROUND:%f\n", *((float *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, sqrtFunction_constant) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result = 5.0; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result = 5.0; - //TINYINT - int8_t val_tinyint = 25; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("tiny_int before SQRT:%d\n", *((int8_t *)pInput->data)); + //TINYINT + int8_t val_tinyint = 25; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, &val_tinyint, rowNum, true); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("tiny_int before SQRT:%d\n", *((int8_t *)pInput->data)); - code = sqrtFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("tiny_int after SQRT:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = sqrtFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("tiny_int after SQRT:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float = 25.0; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("float before SQRT:%f\n", *((float *)pInput->data)); + //FLOAT + float val_float = 25.0; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, &val_float, rowNum, true); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("float before SQRT:%f\n", *((float *)pInput->data)); - code = sqrtFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("float after SQRT:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = sqrtFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("float after SQRT:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, sqrtFunction_column) { - SScalarParam *pInput, *pOutput; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result[] = {5.0, 9.0, 10.0}; + SScalarParam *pInput, *pOutput; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result[] = {5.0, 9.0, 10.0}; - //TINYINT - int8_t val_tinyint[] = {25, 81, 100}; - type = TSDB_DATA_TYPE_TINYINT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - *((int8_t *)colDataGetData(pInput->columnData, i)) = val_tinyint[i]; - PRINTF("tiny_int before SQRT:%d\n", *((int8_t *)colDataGetData(pInput->columnData, i))); - } + //TINYINT + int8_t val_tinyint[] = {25, 81, 100}; + type = TSDB_DATA_TYPE_TINYINT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + *((int8_t *)colDataGetData(pInput->columnData, i)) = val_tinyint[i]; + PRINTF("tiny_int before SQRT:%d\n", *((int8_t *)colDataGetData(pInput->columnData, i))); + } - code = sqrtFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("tiny_int after SQRT:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + code = sqrtFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("tiny_int after SQRT:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float[] = {25.0, 81.0, 100.0}; - type = TSDB_DATA_TYPE_FLOAT; - scltMakeDataBlock(&pInput, type, 0, rowNum, false); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - for (int32_t i = 0; i < rowNum; ++i) { - *((float *)colDataGetData(pInput->columnData, i)) = val_float[i]; - PRINTF("float before SQRT:%f\n", *((float *)colDataGetData(pInput->columnData, i))); - } + //FLOAT + float val_float[] = {25.0, 81.0, 100.0}; + type = TSDB_DATA_TYPE_FLOAT; + scltMakeDataBlock(&pInput, type, 0, rowNum, false); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + for (int32_t i = 0; i < rowNum; ++i) { + *((float *)colDataGetData(pInput->columnData, i)) = val_float[i]; + PRINTF("float before SQRT:%f\n", *((float *)colDataGetData(pInput->columnData, i))); + } - code = sqrtFunction(pInput, 1, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("float after SQRT:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = sqrtFunction(pInput, 1, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("float after SQRT:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(pInput); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(pInput); + scltDestroyDataBlock(pOutput); } TEST(ScalarFunctionTest, logFunction_constant) { - SScalarParam *pInput, *pOutput; - SScalarParam *input[2]; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result = 3.0; - pInput = (SScalarParam *)taosMemoryCalloc(2, sizeof(SScalarParam)); + SScalarParam *pInput, *pOutput; + SScalarParam *input[2]; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result = 3.0; + pInput = (SScalarParam *)taosMemoryCalloc(2, sizeof(SScalarParam)); - //TINYINT - int8_t val_tinyint[] = {27, 3}; - type = TSDB_DATA_TYPE_TINYINT; - for (int32_t i = 0; i < 2; ++i) { - scltMakeDataBlock(&input[i], type, &val_tinyint[i], rowNum, true); - pInput[i] = *input[i]; - } - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("tiny_int before LOG: %d,%d\n", *((int8_t *)pInput[0].data), - *((int8_t *)pInput[1].data)); + //TINYINT + int8_t val_tinyint[] = {27, 3}; + type = TSDB_DATA_TYPE_TINYINT; + for (int32_t i = 0; i < 2; ++i) { + scltMakeDataBlock(&input[i], type, &val_tinyint[i], rowNum, true); + pInput[i] = *input[i]; + } + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("tiny_int before LOG: %d,%d\n", *((int8_t *)pInput[0].data), + *((int8_t *)pInput[1].data)); - code = logFunction(pInput, 2, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("tiny_int after LOG:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(input[0]); - scltDestroyDataBlock(input[1]); - scltDestroyDataBlock(pOutput); + code = logFunction(pInput, 2, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("tiny_int after LOG:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(input[0]); + scltDestroyDataBlock(input[1]); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float[] = {64.0, 4.0}; - type = TSDB_DATA_TYPE_FLOAT; - for (int32_t i = 0; i < 2; ++i) { - scltMakeDataBlock(&input[i], type, &val_float[i], rowNum, true); - pInput[i] = *input[i]; - } - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("float before LOG: %f,%f\n", *((float *)pInput[0].data), - *((float *)pInput[1].data)); + //FLOAT + float val_float[] = {64.0, 4.0}; + type = TSDB_DATA_TYPE_FLOAT; + for (int32_t i = 0; i < 2; ++i) { + scltMakeDataBlock(&input[i], type, &val_float[i], rowNum, true); + pInput[i] = *input[i]; + } + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("float before LOG: %f,%f\n", *((float *)pInput[0].data), + *((float *)pInput[1].data)); - code = logFunction(pInput, 2, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("float after LOG:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(input[0]); - scltDestroyDataBlock(input[1]); - scltDestroyDataBlock(pOutput); + code = logFunction(pInput, 2, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("float after LOG:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(input[0]); + scltDestroyDataBlock(input[1]); + scltDestroyDataBlock(pOutput); - //TINYINT AND FLOAT - int8_t param0 = 64; - float param1 = 4.0; - scltMakeDataBlock(&input[0], TSDB_DATA_TYPE_TINYINT, ¶m0, rowNum, true); - pInput[0] = *input[0]; - scltMakeDataBlock(&input[1], TSDB_DATA_TYPE_FLOAT, ¶m1, rowNum, true); - pInput[1] = *input[1]; - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + //TINYINT AND FLOAT + int8_t param0 = 64; + float param1 = 4.0; + scltMakeDataBlock(&input[0], TSDB_DATA_TYPE_TINYINT, ¶m0, rowNum, true); + pInput[0] = *input[0]; + scltMakeDataBlock(&input[1], TSDB_DATA_TYPE_FLOAT, ¶m1, rowNum, true); + pInput[1] = *input[1]; + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("tiny_int,float before LOG: %d,%f\n", *((int8_t *)pInput[0].data), *((float *)pInput[1].data)); + PRINTF("tiny_int,float before LOG: %d,%f\n", *((int8_t *)pInput[0].data), *((float *)pInput[1].data)); - code = logFunction(pInput, 2, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("tiny_int,float after LOG:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = logFunction(pInput, 2, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("tiny_int,float after LOG:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(input[0]); - scltDestroyDataBlock(input[1]); - scltDestroyDataBlock(pOutput); - taosMemoryFree(pInput); + scltDestroyDataBlock(input[0]); + scltDestroyDataBlock(input[1]); + scltDestroyDataBlock(pOutput); + taosMemoryFree(pInput); } TEST(ScalarFunctionTest, logFunction_column) { - SScalarParam *pInput, *pOutput; - SScalarParam *input[2]; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result[] = {2.0, 3.0, 4.0}; - pInput = (SScalarParam *)taosMemoryCalloc(2, sizeof(SScalarParam)); + SScalarParam *pInput, *pOutput; + SScalarParam *input[2]; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result[] = {2.0, 3.0, 4.0}; + pInput = (SScalarParam *)taosMemoryCalloc(2, sizeof(SScalarParam)); - //TINYINT - int8_t val_tinyint[2][3] = {{9, 27, 81}, {3, 3, 3}}; - type = TSDB_DATA_TYPE_TINYINT; - for (int32_t i = 0; i < 2; ++i) { - scltMakeDataBlock(&input[i], type, 0, rowNum, false); - pInput[i] = *input[i]; - for (int32_t j = 0; j < rowNum; ++j) { - colDataAppend(pInput[i].columnData, j, (const char*) &val_tinyint[i][j], false); + //TINYINT + int8_t val_tinyint[2][3] = {{9, 27, 81}, {3, 3, 3}}; + type = TSDB_DATA_TYPE_TINYINT; + for (int32_t i = 0; i < 2; ++i) { + scltMakeDataBlock(&input[i], type, 0, rowNum, false); + pInput[i] = *input[i]; + for (int32_t j = 0; j < rowNum; ++j) { + colDataAppend(pInput[i].columnData, j, (const char*) &val_tinyint[i][j], false); - } - PRINTF("tiny_int before LOG:%d,%d,%d\n", *((int8_t *)pInput[i].data + 0), - *((int8_t *)pInput[i].data + 1), - *((int8_t *)pInput[i].data + 2)); - } - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + } + PRINTF("tiny_int before LOG:%d,%d,%d\n", *((int8_t *)pInput[i].data + 0), + *((int8_t *)pInput[i].data + 1), + *((int8_t *)pInput[i].data + 2)); + } + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - code = logFunction(pInput, 2, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("tiny_int after LOG:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(input[0]); - scltDestroyDataBlock(input[1]); - scltDestroyDataBlock(pOutput); + code = logFunction(pInput, 2, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("tiny_int after LOG:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(input[0]); + scltDestroyDataBlock(input[1]); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float[2][3] = {{9.0, 27.0, 81.0}, {3.0, 3.0, 3.0}}; - type = TSDB_DATA_TYPE_FLOAT; - for (int32_t i = 0; i < 2; ++i) { - scltMakeDataBlock(&input[i], type, 0, rowNum, false); - pInput[i] = *input[i]; - for (int32_t j = 0; j < rowNum; ++j) { - colDataAppend(pInput[i].columnData, j, (const char*) &val_float[i][j], false); - } - PRINTF("float before LOG:%f,%f,%f\n", *((float *)colDataGetData(pInput[i], 0)), - *((float *)colDataGetData(pInput[i], 1)), - *((float *)colDataGetData(pInput[i], 2))); - } - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + //FLOAT + float val_float[2][3] = {{9.0, 27.0, 81.0}, {3.0, 3.0, 3.0}}; + type = TSDB_DATA_TYPE_FLOAT; + for (int32_t i = 0; i < 2; ++i) { + scltMakeDataBlock(&input[i], type, 0, rowNum, false); + pInput[i] = *input[i]; + for (int32_t j = 0; j < rowNum; ++j) { + colDataAppend(pInput[i].columnData, j, (const char*) &val_float[i][j], false); + } + PRINTF("float before LOG:%f,%f,%f\n", *((float *)colDataGetData(pInput[i], 0)), + *((float *)colDataGetData(pInput[i], 1)), + *((float *)colDataGetData(pInput[i], 2))); + } + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - code = logFunction(pInput, 2, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("float after LOG:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(input[0]); - scltDestroyDataBlock(input[1]); - scltDestroyDataBlock(pOutput); + code = logFunction(pInput, 2, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("float after LOG:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(input[0]); + scltDestroyDataBlock(input[1]); + scltDestroyDataBlock(pOutput); - //TINYINT AND FLOAT - int8_t param0[] = {9, 27, 81}; - float param1[] = {3.0, 3.0, 3.0}; - scltMakeDataBlock(&input[0], TSDB_DATA_TYPE_TINYINT, 0, rowNum, false); - pInput[0] = *input[0]; - for (int32_t i = 0; i < rowNum; ++i) { - colDataAppend(pInput[0].columnData, i, (const char*) ¶m0[i], false); + //TINYINT AND FLOAT + int8_t param0[] = {9, 27, 81}; + float param1[] = {3.0, 3.0, 3.0}; + scltMakeDataBlock(&input[0], TSDB_DATA_TYPE_TINYINT, 0, rowNum, false); + pInput[0] = *input[0]; + for (int32_t i = 0; i < rowNum; ++i) { + colDataAppend(pInput[0].columnData, i, (const char*) ¶m0[i], false); - } - scltMakeDataBlock(&input[1], TSDB_DATA_TYPE_FLOAT, 0, rowNum, false); - pInput[1] = *input[1]; - for (int32_t i = 0; i < rowNum; ++i) { - colDataAppend(pInput[1].columnData, i, (const char*) ¶m1[i], false); + } + scltMakeDataBlock(&input[1], TSDB_DATA_TYPE_FLOAT, 0, rowNum, false); + pInput[1] = *input[1]; + for (int32_t i = 0; i < rowNum; ++i) { + colDataAppend(pInput[1].columnData, i, (const char*) ¶m1[i], false); - } - PRINTF("tiny_int, float before LOG:{%d,%f}, {%d,%f}, {%d,%f}\n", *((int8_t *)pInput[0].data + 0), *((float *)pInput[1].data + 0), - *((int8_t *)pInput[0].data + 1), *((float *)pInput[1].data + 1), - *((int8_t *)pInput[0].data + 2), *((float *)pInput[1].data + 2)); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + } + PRINTF("tiny_int, float before LOG:{%d,%f}, {%d,%f}, {%d,%f}\n", *((int8_t *)pInput[0].data + 0), *((float *)pInput[1].data + 0), + *((int8_t *)pInput[0].data + 1), *((float *)pInput[1].data + 1), + *((int8_t *)pInput[0].data + 2), *((float *)pInput[1].data + 2)); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - code = logFunction(pInput, 2, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("tiny_int,float after LOG:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = logFunction(pInput, 2, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("tiny_int,float after LOG:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(input[0]); - scltDestroyDataBlock(input[1]); - scltDestroyDataBlock(pOutput); - taosMemoryFree(pInput); + scltDestroyDataBlock(input[0]); + scltDestroyDataBlock(input[1]); + scltDestroyDataBlock(pOutput); + taosMemoryFree(pInput); } TEST(ScalarFunctionTest, powFunction_constant) { - SScalarParam *pInput, *pOutput; - SScalarParam *input[2]; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result = 16.0; - pInput = (SScalarParam *)taosMemoryCalloc(2, sizeof(SScalarParam)); + SScalarParam *pInput, *pOutput; + SScalarParam *input[2]; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result = 16.0; + pInput = (SScalarParam *)taosMemoryCalloc(2, sizeof(SScalarParam)); - //TINYINT - int8_t val_tinyint[] = {2, 4}; - type = TSDB_DATA_TYPE_TINYINT; - for (int32_t i = 0; i < 2; ++i) { - scltMakeDataBlock(&input[i], type, &val_tinyint[i], rowNum, true); - pInput[i] = *input[i]; - } - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("tiny_int before POW: %d,%d\n", *((int8_t *)pInput[0].data), - *((int8_t *)pInput[1].data)); + //TINYINT + int8_t val_tinyint[] = {2, 4}; + type = TSDB_DATA_TYPE_TINYINT; + for (int32_t i = 0; i < 2; ++i) { + scltMakeDataBlock(&input[i], type, &val_tinyint[i], rowNum, true); + pInput[i] = *input[i]; + } + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("tiny_int before POW: %d,%d\n", *((int8_t *)pInput[0].data), + *((int8_t *)pInput[1].data)); - code = powFunction(pInput, 2, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("tiny_int after POW:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(input[0]); - scltDestroyDataBlock(input[1]); - scltDestroyDataBlock(pOutput); + code = powFunction(pInput, 2, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("tiny_int after POW:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(input[0]); + scltDestroyDataBlock(input[1]); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float[] = {2.0, 4.0}; - type = TSDB_DATA_TYPE_FLOAT; - for (int32_t i = 0; i < 2; ++i) { - scltMakeDataBlock(&input[i], type, &val_float[i], rowNum, true); - pInput[i] = *input[i]; - } - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("float before POW: %f,%f\n", *((float *)pInput[0].data), - *((float *)pInput[1].data)); + //FLOAT + float val_float[] = {2.0, 4.0}; + type = TSDB_DATA_TYPE_FLOAT; + for (int32_t i = 0; i < 2; ++i) { + scltMakeDataBlock(&input[i], type, &val_float[i], rowNum, true); + pInput[i] = *input[i]; + } + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + PRINTF("float before POW: %f,%f\n", *((float *)pInput[0].data), + *((float *)pInput[1].data)); - code = powFunction(pInput, 2, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("float after POW:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(input[0]); - scltDestroyDataBlock(input[1]); - scltDestroyDataBlock(pOutput); + code = powFunction(pInput, 2, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("float after POW:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(input[0]); + scltDestroyDataBlock(input[1]); + scltDestroyDataBlock(pOutput); - //TINYINT AND FLOAT - int8_t param0 = 2; - float param1 = 4.0; - scltMakeDataBlock(&input[0], TSDB_DATA_TYPE_TINYINT, ¶m0, rowNum, true); - pInput[0] = *input[0]; - scltMakeDataBlock(&input[1], TSDB_DATA_TYPE_FLOAT, ¶m1, rowNum, true); - pInput[1] = *input[1]; - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + //TINYINT AND FLOAT + int8_t param0 = 2; + float param1 = 4.0; + scltMakeDataBlock(&input[0], TSDB_DATA_TYPE_TINYINT, ¶m0, rowNum, true); + pInput[0] = *input[0]; + scltMakeDataBlock(&input[1], TSDB_DATA_TYPE_FLOAT, ¶m1, rowNum, true); + pInput[1] = *input[1]; + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - PRINTF("tiny_int,float before POW: %d,%f\n", *((int8_t *)pInput[0].data), *((float *)pInput[1].data)); + PRINTF("tiny_int,float before POW: %d,%f\n", *((int8_t *)pInput[0].data), *((float *)pInput[1].data)); - code = powFunction(pInput, 2, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); - PRINTF("tiny_int,float after POW:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = powFunction(pInput, 2, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result); + PRINTF("tiny_int,float after POW:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(input[0]); - scltDestroyDataBlock(input[1]); - scltDestroyDataBlock(pOutput); - taosMemoryFree(pInput); + scltDestroyDataBlock(input[0]); + scltDestroyDataBlock(input[1]); + scltDestroyDataBlock(pOutput); + taosMemoryFree(pInput); } TEST(ScalarFunctionTest, powFunction_column) { - SScalarParam *pInput, *pOutput; - SScalarParam *input[2]; - int32_t code = TSDB_CODE_SUCCESS; - int32_t rowNum = 3; - int32_t type; - int32_t otype = TSDB_DATA_TYPE_DOUBLE; - double result[] = {8.0, 27.0, 64.0}; - pInput = (SScalarParam *)taosMemoryCalloc(2, sizeof(SScalarParam)); + SScalarParam *pInput, *pOutput; + SScalarParam *input[2]; + int32_t code = TSDB_CODE_SUCCESS; + int32_t rowNum = 3; + int32_t type; + int32_t otype = TSDB_DATA_TYPE_DOUBLE; + double result[] = {8.0, 27.0, 64.0}; + pInput = (SScalarParam *)taosMemoryCalloc(2, sizeof(SScalarParam)); - //TINYINT - int8_t val_tinyint[2][3] = {{2, 3, 4}, {3, 3, 3}}; - type = TSDB_DATA_TYPE_TINYINT; - for (int32_t i = 0; i < 2; ++i) { - scltMakeDataBlock(&input[i], type, 0, rowNum, false); - pInput[i] = *input[i]; - for (int32_t j = 0; j < rowNum; ++j) { - colDataAppend(pInput[i].columnData, j, (const char*) &val_tinyint[i][j], false); + //TINYINT + int8_t val_tinyint[2][3] = {{2, 3, 4}, {3, 3, 3}}; + type = TSDB_DATA_TYPE_TINYINT; + for (int32_t i = 0; i < 2; ++i) { + scltMakeDataBlock(&input[i], type, 0, rowNum, false); + pInput[i] = *input[i]; + for (int32_t j = 0; j < rowNum; ++j) { + colDataAppend(pInput[i].columnData, j, (const char*) &val_tinyint[i][j], false); - } - PRINTF("tiny_int before POW:%d,%d,%d\n", *((int8_t *)pInput[i].data + 0), - *((int8_t *)pInput[i].data + 1), - *((int8_t *)pInput[i].data + 2)); - } - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + } + PRINTF("tiny_int before POW:%d,%d,%d\n", *((int8_t *)pInput[i].data + 0), + *((int8_t *)pInput[i].data + 1), + *((int8_t *)pInput[i].data + 2)); + } + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - code = powFunction(pInput, 2, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("tiny_int after POW:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = powFunction(pInput, 2, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("tiny_int after POW:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(input[0]); - scltDestroyDataBlock(input[1]); - scltDestroyDataBlock(pOutput); + scltDestroyDataBlock(input[0]); + scltDestroyDataBlock(input[1]); + scltDestroyDataBlock(pOutput); - //FLOAT - float val_float[2][3] = {{2.0, 3.0, 4.0}, {3.0, 3.0, 3.0}}; - type = TSDB_DATA_TYPE_FLOAT; - for (int32_t i = 0; i < 2; ++i) { - scltMakeDataBlock(&input[i], type, 0, rowNum, false); - pInput[i] = *input[i]; - for (int32_t j = 0; j < rowNum; ++j) { - colDataAppend(pInput[i].columnData, j, (const char*) &val_float[i][j], false); - } - PRINTF("float before POW:%f,%f,%f\n", *((float *)pInput[i].data + 0), - *((float *)pInput[i].data + 1), - *((float *)pInput[i].data + 2)); - } - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + //FLOAT + float val_float[2][3] = {{2.0, 3.0, 4.0}, {3.0, 3.0, 3.0}}; + type = TSDB_DATA_TYPE_FLOAT; + for (int32_t i = 0; i < 2; ++i) { + scltMakeDataBlock(&input[i], type, 0, rowNum, false); + pInput[i] = *input[i]; + for (int32_t j = 0; j < rowNum; ++j) { + colDataAppend(pInput[i].columnData, j, (const char*) &val_float[i][j], false); + } + PRINTF("float before POW:%f,%f,%f\n", *((float *)pInput[i].data + 0), + *((float *)pInput[i].data + 1), + *((float *)pInput[i].data + 2)); + } + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - code = powFunction(pInput, 2, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("float after POW:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } - scltDestroyDataBlock(input[0]); - scltDestroyDataBlock(input[1]); - scltDestroyDataBlock(pOutput); + code = powFunction(pInput, 2, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("float after POW:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } + scltDestroyDataBlock(input[0]); + scltDestroyDataBlock(input[1]); + scltDestroyDataBlock(pOutput); - //TINYINT AND FLOAT - int8_t param0[] = {2, 3, 4}; - float param1[] = {3.0, 3.0, 3.0}; - scltMakeDataBlock(&input[0], TSDB_DATA_TYPE_TINYINT, 0, rowNum, false); - pInput[0] = *input[0]; - for (int32_t i = 0; i < rowNum; ++i) { - colDataAppend(pInput[0].columnData, i, (const char*) ¶m0[i], false); + //TINYINT AND FLOAT + int8_t param0[] = {2, 3, 4}; + float param1[] = {3.0, 3.0, 3.0}; + scltMakeDataBlock(&input[0], TSDB_DATA_TYPE_TINYINT, 0, rowNum, false); + pInput[0] = *input[0]; + for (int32_t i = 0; i < rowNum; ++i) { + colDataAppend(pInput[0].columnData, i, (const char*) ¶m0[i], false); - } - scltMakeDataBlock(&input[1], TSDB_DATA_TYPE_FLOAT, 0, rowNum, false); - pInput[1] = *input[1]; - for (int32_t i = 0; i < rowNum; ++i) { - colDataAppend(pInput[1].columnData, i, (const char*) ¶m1[i], false); - } - PRINTF("tiny_int, float before POW:{%d,%f}, {%d,%f}, {%d,%f}\n", *((int8_t *)pInput[0].data + 0), *((float *)pInput[1].data + 0), - *((int8_t *)pInput[0].data + 1), *((float *)pInput[1].data + 1), - *((int8_t *)pInput[0].data + 2), *((float *)pInput[1].data + 2)); - scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); + } + scltMakeDataBlock(&input[1], TSDB_DATA_TYPE_FLOAT, 0, rowNum, false); + pInput[1] = *input[1]; + for (int32_t i = 0; i < rowNum; ++i) { + colDataAppend(pInput[1].columnData, i, (const char*) ¶m1[i], false); + } + PRINTF("tiny_int, float before POW:{%d,%f}, {%d,%f}, {%d,%f}\n", *((int8_t *)pInput[0].data + 0), *((float *)pInput[1].data + 0), + *((int8_t *)pInput[0].data + 1), *((float *)pInput[1].data + 1), + *((int8_t *)pInput[0].data + 2), *((float *)pInput[1].data + 2)); + scltMakeDataBlock(&pOutput, otype, 0, rowNum, false); - code = powFunction(pInput, 2, pOutput); - ASSERT_EQ(code, TSDB_CODE_SUCCESS); - for (int32_t i = 0; i < rowNum; ++i) { - ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); - PRINTF("tiny_int,float after POW:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); - } + code = powFunction(pInput, 2, pOutput); + ASSERT_EQ(code, TSDB_CODE_SUCCESS); + for (int32_t i = 0; i < rowNum; ++i) { + ASSERT_EQ(*((double *)colDataGetData(pOutput->columnData, i)), result[i]); + PRINTF("tiny_int,float after POW:%f\n", *((double *)colDataGetData(pOutput->columnData, i))); + } - scltDestroyDataBlock(input[0]); - scltDestroyDataBlock(input[1]); - scltDestroyDataBlock(pOutput); - taosMemoryFree(pInput); + scltDestroyDataBlock(input[0]); + scltDestroyDataBlock(input[1]); + scltDestroyDataBlock(pOutput); + taosMemoryFree(pInput); } int main(int argc, char** argv) { - taosSeedRand(taosGetTimestampSec()); - testing::InitGoogleTest(&argc, argv); - return RUN_ALL_TESTS(); + taosSeedRand(taosGetTimestampSec()); + testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); } #pragma GCC diagnostic pop diff --git a/source/libs/scheduler/inc/schedulerInt.h b/source/libs/scheduler/inc/schedulerInt.h index d961780ae5..9e302e569a 100644 --- a/source/libs/scheduler/inc/schedulerInt.h +++ b/source/libs/scheduler/inc/schedulerInt.h @@ -207,6 +207,7 @@ typedef struct SSchJob { SArray *dataSrcTasks; // SArray int32_t levelIdx; SEpSet dataSrcEps; + SHashObj *taskList; SHashObj *execTasks; // executing tasks, key:taskid, value:SQueryTask* SHashObj *succTasks; // succeed tasks, key:taskid, value:SQueryTask* SHashObj *failTasks; // failed tasks, key:taskid, value:SQueryTask* diff --git a/source/libs/scheduler/src/schJob.c b/source/libs/scheduler/src/schJob.c index 09f2efc7e2..d799169ee4 100644 --- a/source/libs/scheduler/src/schJob.c +++ b/source/libs/scheduler/src/schJob.c @@ -63,6 +63,13 @@ int32_t schInitJob(SSchJob **pSchJob, SQueryPlan *pDag, void *pTrans, SArray *pN if (pNodeList != NULL) { pJob->nodeList = taosArrayDup(pNodeList); } + + pJob->taskList = + taosHashInit(pDag->numOfSubplans, taosGetDefaultHashFunction(TSDB_DATA_TYPE_UBIGINT), false, HASH_ENTRY_LOCK); + if (NULL == pJob->taskList) { + SCH_JOB_ELOG("taosHashInit %d taskList failed", pDag->numOfSubplans); + SCH_ERR_JRET(TSDB_CODE_QRY_OUT_OF_MEMORY); + } SCH_ERR_JRET(schValidateAndBuildJob(pDag, pJob)); @@ -486,23 +493,26 @@ int32_t schValidateAndBuildJob(SQueryPlan *pDag, SSchJob *pJob) { SCH_SET_JOB_TYPE(pJob, plan->subplanType); SSchTask task = {0}; - SSchTask *pTask = &task; - SCH_ERR_JRET(schInitTask(pJob, &task, plan, pLevel)); - void *p = taosArrayPush(pLevel->subTasks, &task); - if (NULL == p) { + SSchTask *pTask = taosArrayPush(pLevel->subTasks, &task); + if (NULL == pTask) { SCH_TASK_ELOG("taosArrayPush task to level failed, level:%d, taskIdx:%d", pLevel->level, n); SCH_ERR_JRET(TSDB_CODE_QRY_OUT_OF_MEMORY); } - SCH_ERR_JRET(schRecordQueryDataSrc(pJob, p)); + SCH_ERR_JRET(schRecordQueryDataSrc(pJob, pTask)); - if (0 != taosHashPut(planToTask, &plan, POINTER_BYTES, &p, POINTER_BYTES)) { + if (0 != taosHashPut(planToTask, &plan, POINTER_BYTES, &pTask, POINTER_BYTES)) { SCH_TASK_ELOG("taosHashPut to planToTaks failed, taskIdx:%d", n); SCH_ERR_JRET(TSDB_CODE_QRY_OUT_OF_MEMORY); } + if (0 != taosHashPut(pJob->taskList, &pTask->taskId, sizeof(pTask->taskId), &pTask, POINTER_BYTES)) { + SCH_TASK_ELOG("taosHashPut to taskList failed, taskIdx:%d", n); + SCH_ERR_JRET(TSDB_CODE_QRY_OUT_OF_MEMORY); + } + ++pJob->taskNum; } @@ -1276,14 +1286,10 @@ int32_t schGetTaskFromList(SHashObj *pTaskList, uint64_t taskId, SSchTask **pTas } int32_t schGetTaskInJob(SSchJob *pJob, uint64_t taskId, SSchTask **pTask) { - schGetTaskFromList(pJob->execTasks, taskId, pTask); + schGetTaskFromList(pJob->taskList, taskId, pTask); if (NULL == *pTask) { - schGetTaskFromList(pJob->succTasks, taskId, pTask); - - if (NULL == *pTask) { - SCH_JOB_ELOG("task not found in execList & succList, taskId:%" PRIx64, taskId); - SCH_ERR_RET(TSDB_CODE_SCH_INTERNAL_ERROR); - } + SCH_JOB_ELOG("task not found in job task list, taskId:%" PRIx64, taskId); + SCH_ERR_RET(TSDB_CODE_SCH_INTERNAL_ERROR); } return TSDB_CODE_SUCCESS; @@ -1445,7 +1451,8 @@ void schFreeJobImpl(void *job) { taosHashCleanup(pJob->execTasks); taosHashCleanup(pJob->failTasks); taosHashCleanup(pJob->succTasks); - + taosHashCleanup(pJob->taskList); + taosArrayDestroy(pJob->levels); taosArrayDestroy(pJob->nodeList); taosArrayDestroy(pJob->dataSrcTasks); diff --git a/source/libs/stream/inc/streamInc.h b/source/libs/stream/inc/streamInc.h index b5f7362689..27746fff78 100644 --- a/source/libs/stream/inc/streamInc.h +++ b/source/libs/stream/inc/streamInc.h @@ -24,8 +24,9 @@ extern "C" { #endif int32_t streamExec(SStreamTask* pTask, SMsgCb* pMsgCb); -int32_t streamSink1(SStreamTask* pTask, SMsgCb* pMsgCb); -int32_t streamDispatch(SStreamTask* pTask, SMsgCb* pMsgCb, SStreamDataBlock* data); +int32_t streamDispatch(SStreamTask* pTask, SMsgCb* pMsgCb); +int32_t streamDispatchReqToData(const SStreamDispatchReq* pReq, SStreamDataBlock* pData); +int32_t streamBuildDispatchMsg(SStreamTask* pTask, SStreamDataBlock* data, SRpcMsg* pMsg, SEpSet** ppEpSet); #ifdef __cplusplus } diff --git a/source/libs/stream/src/stream.c b/source/libs/stream/src/stream.c index 99c61e1479..823ce6d2fe 100644 --- a/source/libs/stream/src/stream.c +++ b/source/libs/stream/src/stream.c @@ -26,7 +26,7 @@ int32_t streamTriggerByWrite(SStreamTask* pTask, int32_t vgId, SMsgCb* pMsgCb) { pRunReq->streamId = pTask->streamId; pRunReq->taskId = pTask->taskId; SRpcMsg msg = { - .msgType = TDMT_VND_TASK_RUN, + .msgType = TDMT_STREAM_TASK_RUN, .pCont = pRunReq, .contLen = sizeof(SStreamTaskRunReq), }; @@ -35,18 +35,19 @@ int32_t streamTriggerByWrite(SStreamTask* pTask, int32_t vgId, SMsgCb* pMsgCb) { return 0; } -#if 1 int32_t streamTaskEnqueue(SStreamTask* pTask, SStreamDispatchReq* pReq, SRpcMsg* pRsp) { - SStreamDataBlock* pBlock = taosAllocateQitem(sizeof(SStreamDataBlock), DEF_QITEM); + SStreamDataBlock* pData = taosAllocateQitem(sizeof(SStreamDataBlock), DEF_QITEM); int8_t status; // enqueue - if (pBlock != NULL) { - pBlock->type = STREAM_DATA_TYPE_SSDATA_BLOCK; - pBlock->sourceVg = pReq->sourceVg; - pBlock->blocks = pReq->data; + if (pData != NULL) { + pData->type = STREAM_DATA_TYPE_SSDATA_BLOCK; + pData->sourceVg = pReq->sourceVg; + // decode + /*pData->blocks = pReq->data;*/ /*pBlock->sourceVer = pReq->sourceVer;*/ - if (streamTaskInput(pTask, (SStreamQueueItem*)pBlock) == 0) { + streamDispatchReqToData(pReq, pData); + if (streamTaskInput(pTask, (SStreamQueueItem*)pData) == 0) { status = TASK_INPUT_STATUS__NORMAL; } else { status = TASK_INPUT_STATUS__FAILED; @@ -68,7 +69,6 @@ int32_t streamTaskEnqueue(SStreamTask* pTask, SStreamDispatchReq* pReq, SRpcMsg* tmsgSendRsp(pRsp); return status == TASK_INPUT_STATUS__NORMAL ? 0 : -1; } -#endif int32_t streamProcessDispatchReq(SStreamTask* pTask, SMsgCb* pMsgCb, SStreamDispatchReq* pReq, SRpcMsg* pRsp) { // 1. handle input @@ -83,7 +83,9 @@ int32_t streamProcessDispatchReq(SStreamTask* pTask, SMsgCb* pMsgCb, SStreamDisp // 3. handle output // 3.1 check and set status // 3.2 dispatch / sink - streamSink1(pTask, pMsgCb); + if (pTask->dispatchType != TASK_DISPATCH__NONE) { + streamDispatch(pTask, pMsgCb); + } return 0; } @@ -97,13 +99,17 @@ int32_t streamProcessDispatchRsp(SStreamTask* pTask, SMsgCb* pMsgCb, SStreamDisp return 0; } // continue dispatch - streamSink1(pTask, pMsgCb); + if (pTask->dispatchType != TASK_DISPATCH__NONE) { + streamDispatch(pTask, pMsgCb); + } return 0; } int32_t streamTaskProcessRunReq(SStreamTask* pTask, SMsgCb* pMsgCb) { streamExec(pTask, pMsgCb); - streamSink1(pTask, pMsgCb); + if (pTask->dispatchType != TASK_DISPATCH__NONE) { + streamDispatch(pTask, pMsgCb); + } return 0; } diff --git a/source/libs/stream/src/streamData.c b/source/libs/stream/src/streamData.c index 95c0290058..473ad32752 100644 --- a/source/libs/stream/src/streamData.c +++ b/source/libs/stream/src/streamData.c @@ -13,7 +13,7 @@ * along with this program. If not, see . */ -#include "tstream.h" +#include "streamInc.h" #if 0 int32_t streamDataBlockEncode(void** buf, const SStreamDataBlock* pOutput) { @@ -36,6 +36,29 @@ void* streamDataBlockDecode(const void* buf, SStreamDataBlock* pInput) { } #endif +int32_t streamDispatchReqToData(const SStreamDispatchReq* pReq, SStreamDataBlock* pData) { + int32_t blockNum = pReq->blockNum; + SArray* pArray = taosArrayInit(blockNum, sizeof(SSDataBlock)); + if (pArray == NULL) { + return -1; + } + taosArraySetSize(pArray, blockNum); + + ASSERT(pReq->blockNum == taosArrayGetSize(pReq->data)); + ASSERT(pReq->blockNum == taosArrayGetSize(pReq->dataLen)); + + for (int32_t i = 0; i < blockNum; i++) { + int32_t len = *(int32_t*)taosArrayGet(pReq->dataLen, i); + SRetrieveTableRsp* pRetrieve = taosArrayGetP(pReq->data, i); + SSDataBlock* pDataBlock = taosArrayGet(pArray, i); + blockCompressDecode(pDataBlock, htonl(pRetrieve->numOfCols), htonl(pRetrieve->numOfRows), pRetrieve->data); + // TODO: refactor + pDataBlock->info.childId = pReq->sourceChildId; + } + pData->blocks = pArray; + return 0; +} + SStreamDataSubmit* streamDataSubmitNew(SSubmitReq* pReq) { SStreamDataSubmit* pDataSubmit = (SStreamDataSubmit*)taosAllocateQitem(sizeof(SStreamDataSubmit), DEF_QITEM); if (pDataSubmit == NULL) return NULL; @@ -51,7 +74,6 @@ FAIL: } static FORCE_INLINE void streamDataSubmitRefInc(SStreamDataSubmit* pDataSubmit) { - // atomic_add_fetch_32(pDataSubmit->dataRef, 1); } @@ -64,22 +86,3 @@ SStreamDataSubmit* streamSubmitRefClone(SStreamDataSubmit* pSubmit) { memcpy(pSubmitClone, pSubmit, sizeof(SStreamDataSubmit)); return pSubmitClone; } - -#if 0 -int32_t tEncodeSStreamTaskExecReq(void** buf, const SStreamTaskExecReq* pReq) { - int32_t tlen = 0; - tlen += taosEncodeFixedI64(buf, pReq->streamId); - tlen += taosEncodeFixedI32(buf, pReq->taskId); - tlen += tEncodeDataBlocks(buf, pReq->data); - return tlen; -} - -void* tDecodeSStreamTaskExecReq(const void* buf, SStreamTaskExecReq* pReq) { - buf = taosDecodeFixedI64(buf, &pReq->streamId); - buf = taosDecodeFixedI32(buf, &pReq->taskId); - buf = tDecodeDataBlocks(buf, &pReq->data); - return (void*)buf; -} - -void tFreeSStreamTaskExecReq(SStreamTaskExecReq* pReq) { taosArrayDestroy(pReq->data); } -#endif diff --git a/source/libs/stream/src/streamMsg.c b/source/libs/stream/src/streamDispatch.c similarity index 62% rename from source/libs/stream/src/streamMsg.c rename to source/libs/stream/src/streamDispatch.c index 769d672042..d1e3fa0799 100644 --- a/source/libs/stream/src/streamMsg.c +++ b/source/libs/stream/src/streamDispatch.c @@ -71,6 +71,7 @@ static int32_t streamAddBlockToDispatchMsg(const SSDataBlock* pBlock, SStreamDis pRetrieve->compressed = 0; pRetrieve->completed = 1; pRetrieve->numOfRows = htonl(pBlock->info.rows); + pRetrieve->numOfCols = htonl(pBlock->info.numOfCols); int32_t actualLen = 0; blockCompressEncode(pBlock, pRetrieve->data, &actualLen, pBlock->info.numOfCols, false); @@ -115,11 +116,9 @@ int32_t streamBuildDispatchMsg(SStreamTask* pTask, SStreamDataBlock* data, SRpcM *ppEpSet = &pTask->fixedEpDispatcher.epSet; downstreamTaskId = pTask->fixedEpDispatcher.taskId; } else if (pTask->dispatchType == TASK_DISPATCH__SHUFFLE) { - // TODO get ctbName - char ctbName[TSDB_TABLE_FNAME_LEN + 22] = {0}; + // TODO get ctbName for each block SSDataBlock* pBlock = taosArrayGet(data->blocks, 0); - sprintf(ctbName, "%s:%ld", pTask->shuffleDispatcher.stbFullName, pBlock->info.groupId); - // get vg and ep + char* ctbName = buildCtbNameByGroupId(pTask->shuffleDispatcher.stbFullName, pBlock->info.groupId); // TODO: get hash function by hashMethod // get groupId, compute hash value @@ -175,131 +174,26 @@ FAIL: return code; } -int32_t streamDispatch(SStreamTask* pTask, SMsgCb* pMsgCb, SStreamDataBlock* data) { -#if 0 +int32_t streamDispatch(SStreamTask* pTask, SMsgCb* pMsgCb) { +#if 1 int8_t old = atomic_val_compare_exchange_8(&pTask->outputStatus, TASK_OUTPUT_STATUS__NORMAL, TASK_OUTPUT_STATUS__WAIT); if (old != TASK_OUTPUT_STATUS__NORMAL) { return 0; } #endif - if (pTask->dispatchType == TASK_DISPATCH__INPLACE) { - SRpcMsg dispatchMsg = {0}; - if (streamBuildDispatchMsg(pTask, data, &dispatchMsg, NULL) < 0) { - ASSERT(0); - return -1; - } - int32_t qType; - if (pTask->dispatchMsgType == TDMT_VND_TASK_DISPATCH || pTask->dispatchMsgType == TDMT_SND_TASK_DISPATCH) { - qType = FETCH_QUEUE; - } else if (pTask->dispatchMsgType == TDMT_VND_TASK_DISPATCH_WRITE) { - qType = WRITE_QUEUE; - } else { - ASSERT(0); - } - tmsgPutToQueue(pMsgCb, qType, &dispatchMsg); - } else if (pTask->dispatchType == TASK_DISPATCH__FIXED) { - SRpcMsg dispatchMsg = {0}; - SEpSet* pEpSet = NULL; - if (streamBuildDispatchMsg(pTask, data, &dispatchMsg, &pEpSet) < 0) { - ASSERT(0); - return -1; - } + SStreamDataBlock* pBlock = streamQueueNextItem(pTask->outputQueue); + if (pBlock == NULL) return 0; + ASSERT(pBlock->type == STREAM_DATA_TYPE_SSDATA_BLOCK); - tmsgSendReq(pEpSet, &dispatchMsg); - } else if (pTask->dispatchType == TASK_DISPATCH__SHUFFLE) { - SRpcMsg dispatchMsg = {0}; - SEpSet* pEpSet = NULL; - if (streamBuildDispatchMsg(pTask, data, &dispatchMsg, &pEpSet) < 0) { - ASSERT(0); - return -1; - } - - tmsgSendReq(pEpSet, &dispatchMsg); - } - return 0; -} - -#if 0 -static int32_t streamBuildExecMsg(SStreamTask* pTask, SArray* data, SRpcMsg* pMsg, SEpSet** ppEpSet) { - SStreamTaskExecReq req = { - .streamId = pTask->streamId, - .data = data, - }; - - int32_t tlen = sizeof(SMsgHead) + tEncodeSStreamTaskExecReq(NULL, &req); - void* buf = rpcMallocCont(tlen); - - if (buf == NULL) { + SRpcMsg dispatchMsg = {0}; + SEpSet* pEpSet = NULL; + if (streamBuildDispatchMsg(pTask, pBlock, &dispatchMsg, &pEpSet) < 0) { + ASSERT(0); return -1; } - if (pTask->dispatchType == TASK_DISPATCH__INPLACE) { - ((SMsgHead*)buf)->vgId = 0; - req.taskId = pTask->inplaceDispatcher.taskId; - - } else if (pTask->dispatchType == TASK_DISPATCH__FIXED) { - ((SMsgHead*)buf)->vgId = htonl(pTask->fixedEpDispatcher.nodeId); - *ppEpSet = &pTask->fixedEpDispatcher.epSet; - req.taskId = pTask->fixedEpDispatcher.taskId; - - } else if (pTask->dispatchType == TASK_DISPATCH__SHUFFLE) { - // TODO use general name rule of schemaless - char ctbName[TSDB_TABLE_FNAME_LEN + 22] = {0}; - // all groupId must be the same in an array - SSDataBlock* pBlock = taosArrayGet(data, 0); - sprintf(ctbName, "%s:%ld", pTask->shuffleDispatcher.stbFullName, pBlock->info.groupId); - - // TODO: get hash function by hashMethod - - // get groupId, compute hash value - uint32_t hashValue = MurmurHash3_32(ctbName, strlen(ctbName)); - - // get node - // TODO: optimize search process - SArray* vgInfo = pTask->shuffleDispatcher.dbInfo.pVgroupInfos; - int32_t sz = taosArrayGetSize(vgInfo); - int32_t nodeId = 0; - for (int32_t i = 0; i < sz; i++) { - SVgroupInfo* pVgInfo = taosArrayGet(vgInfo, i); - if (hashValue >= pVgInfo->hashBegin && hashValue <= pVgInfo->hashEnd) { - nodeId = pVgInfo->vgId; - req.taskId = pVgInfo->taskId; - *ppEpSet = &pVgInfo->epSet; - break; - } - } - ASSERT(nodeId != 0); - ((SMsgHead*)buf)->vgId = htonl(nodeId); - } - - void* abuf = POINTER_SHIFT(buf, sizeof(SMsgHead)); - tEncodeSStreamTaskExecReq(&abuf, &req); - - pMsg->pCont = buf; - pMsg->contLen = tlen; - pMsg->code = 0; - pMsg->msgType = pTask->dispatchMsgType; - pMsg->info.noResp = 1; - + tmsgSendReq(pEpSet, &dispatchMsg); return 0; } - -static int32_t streamShuffleDispatch(SStreamTask* pTask, SMsgCb* pMsgCb, SHashObj* data) { - void* pIter = NULL; - while (1) { - pIter = taosHashIterate(data, pIter); - if (pIter == NULL) return 0; - SArray* pData = *(SArray**)pIter; - SRpcMsg dispatchMsg = {0}; - SEpSet* pEpSet; - if (streamBuildExecMsg(pTask, pData, &dispatchMsg, &pEpSet) < 0) { - ASSERT(0); - return -1; - } - tmsgSendReq(pEpSet, &dispatchMsg); - } - return 0; -} -#endif diff --git a/source/libs/stream/src/streamQueue.c b/source/libs/stream/src/streamQueue.c index f82ef1b42f..d5a4da60f5 100644 --- a/source/libs/stream/src/streamQueue.c +++ b/source/libs/stream/src/streamQueue.c @@ -41,3 +41,12 @@ void streamQueueClose(SStreamQueue* queue) { return; } } + +void streamDataSubmitRefDec(SStreamDataSubmit* pDataSubmit) { + int32_t ref = atomic_sub_fetch_32(pDataSubmit->dataRef, 1); + ASSERT(ref >= 0); + if (ref == 0) { + taosMemoryFree(pDataSubmit->data); + taosMemoryFree(pDataSubmit->dataRef); + } +} diff --git a/source/libs/stream/src/streamSink.c b/source/libs/stream/src/streamSink.c deleted file mode 100644 index 35bebe0e63..0000000000 --- a/source/libs/stream/src/streamSink.c +++ /dev/null @@ -1,159 +0,0 @@ -/* - * Copyright (c) 2019 TAOS Data, Inc. - * - * This program is free software: you can use, redistribute, and/or modify - * it under the terms of the GNU Affero General Public License, version 3 - * or later ("AGPL"), as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, but WITHOUT - * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or - * FITNESS FOR A PARTICULAR PURPOSE. - * - * You should have received a copy of the GNU Affero General Public License - * along with this program. If not, see . - */ - -#include "streamInc.h" - -int32_t streamSink1(SStreamTask* pTask, SMsgCb* pMsgCb) { - SStreamQueue* queue; - if (pTask->execType == TASK_EXEC__NONE) { - queue = pTask->inputQueue; - } else { - queue = pTask->outputQueue; - } - - /*if (streamDequeueBegin(queue) == true) {*/ - /*return -1;*/ - /*}*/ - - if (pTask->sinkType == TASK_SINK__TABLE || pTask->sinkType == TASK_SINK__SMA || - pTask->dispatchType != TASK_DISPATCH__NONE) { - while (1) { - SStreamDataBlock* pBlock = streamQueueNextItem(queue); - if (pBlock == NULL) break; - ASSERT(pBlock->type == STREAM_DATA_TYPE_SSDATA_BLOCK); - - // local sink - if (pTask->sinkType == TASK_SINK__TABLE) { - ASSERT(pTask->dispatchType == TASK_DISPATCH__NONE); - pTask->tbSink.tbSinkFunc(pTask, pTask->tbSink.vnode, 0, pBlock->blocks); - } else if (pTask->sinkType == TASK_SINK__SMA) { - ASSERT(pTask->dispatchType == TASK_DISPATCH__NONE); - pTask->smaSink.smaSink(pTask->ahandle, pTask->smaSink.smaId, pBlock->blocks); - } - - // TODO: sink and dispatch should be only one - if (pTask->dispatchType != TASK_DISPATCH__NONE) { - ASSERT(queue == pTask->outputQueue); - ASSERT(pTask->sinkType == TASK_SINK__NONE); - - streamDispatch(pTask, pMsgCb, pBlock); - } - - streamQueueProcessSuccess(queue); - } - } - - return 0; -} - -#if 0 -int32_t streamSink(SStreamTask* pTask, SMsgCb* pMsgCb) { - bool firstRun = 1; - while (1) { - SStreamDataBlock* pBlock = NULL; - if (!firstRun) { - taosReadAllQitems(pTask->outputQ, pTask->outputQAll); - } - taosGetQitem(pTask->outputQAll, (void**)&pBlock); - if (pBlock == NULL) { - if (firstRun) { - firstRun = 0; - continue; - } else { - break; - } - } - - SArray* pRes = pBlock->blocks; - - // sink - if (pTask->sinkType == TASK_SINK__TABLE) { - // blockDebugShowData(pRes); - pTask->tbSink.tbSinkFunc(pTask, pTask->tbSink.vnode, 0, pRes); - } else if (pTask->sinkType == TASK_SINK__SMA) { - pTask->smaSink.smaSink(pTask->ahandle, pTask->smaSink.smaId, pRes); - // - } else if (pTask->sinkType == TASK_SINK__FETCH) { - // - } else { - ASSERT(pTask->sinkType == TASK_SINK__NONE); - } - - // dispatch - // TODO dispatch guard - int8_t outputStatus = atomic_load_8(&pTask->outputStatus); - if (outputStatus == TASK_OUTPUT_STATUS__NORMAL) { - if (pTask->dispatchType == TASK_DISPATCH__INPLACE) { - SRpcMsg dispatchMsg = {0}; - if (streamBuildExecMsg(pTask, pRes, &dispatchMsg, NULL) < 0) { - ASSERT(0); - return -1; - } - - int32_t qType; - if (pTask->dispatchMsgType == TDMT_VND_TASK_DISPATCH || pTask->dispatchMsgType == TDMT_SND_TASK_DISPATCH) { - qType = FETCH_QUEUE; - /*} else if (pTask->dispatchMsgType == TDMT_VND_TASK_MERGE_EXEC ||*/ - /*pTask->dispatchMsgType == TDMT_SND_TASK_MERGE_EXEC) {*/ - /*qType = MERGE_QUEUE;*/ - /*} else if (pTask->dispatchMsgType == TDMT_VND_TASK_WRITE_EXEC) {*/ - /*qType = WRITE_QUEUE;*/ - } else { - ASSERT(0); - } - tmsgPutToQueue(pMsgCb, qType, &dispatchMsg); - - } else if (pTask->dispatchType == TASK_DISPATCH__FIXED) { - SRpcMsg dispatchMsg = {0}; - SEpSet* pEpSet = NULL; - if (streamBuildExecMsg(pTask, pRes, &dispatchMsg, &pEpSet) < 0) { - ASSERT(0); - return -1; - } - - tmsgSendReq(pEpSet, &dispatchMsg); - - } else if (pTask->dispatchType == TASK_DISPATCH__SHUFFLE) { - SHashObj* pShuffleRes = taosHashInit(64, taosGetDefaultHashFunction(TSDB_DATA_TYPE_INT), false, HASH_NO_LOCK); - if (pShuffleRes == NULL) { - return -1; - } - - int32_t sz = taosArrayGetSize(pRes); - for (int32_t i = 0; i < sz; i++) { - SSDataBlock* pDataBlock = taosArrayGet(pRes, i); - SArray* pArray = taosHashGet(pShuffleRes, &pDataBlock->info.groupId, sizeof(int64_t)); - if (pArray == NULL) { - pArray = taosArrayInit(0, sizeof(SSDataBlock)); - if (pArray == NULL) { - return -1; - } - taosHashPut(pShuffleRes, &pDataBlock->info.groupId, sizeof(int64_t), &pArray, sizeof(void*)); - } - taosArrayPush(pArray, pDataBlock); - } - - if (streamShuffleDispatch(pTask, pMsgCb, pShuffleRes) < 0) { - return -1; - } - - } else { - ASSERT(pTask->dispatchType == TASK_DISPATCH__NONE); - } - } - } - return 0; -} -#endif diff --git a/source/util/src/terror.c b/source/util/src/terror.c index c14c279d81..71c348f810 100644 --- a/source/util/src/terror.c +++ b/source/util/src/terror.c @@ -188,9 +188,9 @@ TAOS_DEFINE_ERROR(TSDB_CODE_MND_SNODE_ALREADY_EXIST, "Snode already exists" TAOS_DEFINE_ERROR(TSDB_CODE_MND_SNODE_NOT_EXIST, "Snode not there") TAOS_DEFINE_ERROR(TSDB_CODE_MND_BNODE_ALREADY_EXIST, "Bnode already exists") TAOS_DEFINE_ERROR(TSDB_CODE_MND_BNODE_NOT_EXIST, "Bnode not there") -TAOS_DEFINE_ERROR(TSDB_CODE_MND_TOO_FEW_MNODES, "The replicas of mnode cannot less than 1") -TAOS_DEFINE_ERROR(TSDB_CODE_MND_TOO_MANY_MNODES, "The replicas of mnode cannot exceed 3") -TAOS_DEFINE_ERROR(TSDB_CODE_MND_CANT_DROP_MASTER, "Can't drop mnode which is leader") +TAOS_DEFINE_ERROR(TSDB_CODE_MND_TOO_FEW_MNODES, "The replica of mnode cannot less than 1") +TAOS_DEFINE_ERROR(TSDB_CODE_MND_TOO_MANY_MNODES, "The replica of mnode cannot exceed 3") +TAOS_DEFINE_ERROR(TSDB_CODE_MND_CANT_DROP_LEADER, "Cannot drop mnode which is leader") // mnode-acct TAOS_DEFINE_ERROR(TSDB_CODE_MND_ACCT_ALREADY_EXIST, "Account already exists") @@ -451,9 +451,89 @@ TAOS_DEFINE_ERROR(TSDB_CODE_SCH_TIMEOUT_ERROR, "Task timeout") TAOS_DEFINE_ERROR(TSDB_CODE_QW_MSG_ERROR, "Invalid msg order") // parser -TAOS_DEFINE_ERROR(TSDB_CODE_PAR_TABLE_NOT_EXIST, "Table does not exist") -TAOS_DEFINE_ERROR(TSDB_CODE_PAR_PERMISSION_DENIED, "Permission denied") -TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INTERNAL_ERROR, "Parser internal error") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_PERMISSION_DENIED, "Permission denied") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INTERNAL_ERROR, "Parser internal error") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_SYNTAX_ERROR, "syntax error near") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INCOMPLETE_SQL, "Incomplete SQL statement") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_COLUMN, "Invalid column name") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_TABLE_NOT_EXIST, "Table does not exist") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_AMBIGUOUS_COLUMN, "Column ambiguously defined") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_WRONG_VALUE_TYPE, "Invalid value type") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_ILLEGAL_USE_AGG_FUNCTION, "There mustn't be aggregation") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_WRONG_NUMBER_OF_SELECT, "ORDER BY item must be the number of a SELECT-list expression") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_GROUPBY_LACK_EXPRESSION, "Not a GROUP BY expression") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_NOT_SELECTED_EXPRESSION, "Not SELECTed expression") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_NOT_SINGLE_GROUP, "Not a single-group group function") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_TAGS_NOT_MATCHED, "Tags number not matched") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_TAG_NAME, "Invalid tag name") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_NAME_OR_PASSWD_TOO_LONG, "Name or password too long") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_PASSWD_EMPTY, "Password can not be empty") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_PORT, "Port should be an integer that is less than 65535 and greater than 0") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_ENDPOINT, "Endpoint should be in the format of 'fqdn:port'") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_EXPRIE_STATEMENT, "This statement is no longer supported") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INTER_VALUE_TOO_SMALL, "Interval too small") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_DB_NOT_SPECIFIED, "Database not specified") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_IDENTIFIER_NAME, "Invalid identifier name") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_CORRESPONDING_STABLE_ERR, "Corresponding super table not in this db") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_RANGE_OPTION, "Invalid option") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_STR_OPTION, "Invalid option") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_ENUM_OPTION, "Invalid option") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_KEEP_NUM, "Invalid number of keep options") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_KEEP_ORDER, "Invalid keep value, should be keep0 <= keep1 <= keep2") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_KEEP_VALUE, "Invalid option keep") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_COMMENT_OPTION, "Invalid option comment") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_F_RANGE_OPTION, "Invalid option") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_ROLLUP_OPTION, "Invalid option rollup: only one function is allowed") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_RETENTIONS_OPTION, "Invalid option retentions") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_GROUPBY_WINDOW_COEXIST, "GROUP BY and WINDOW-clause can't be used together") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_OPTION_UNIT, "Invalid option unit: only m, h, d allowed") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_KEEP_UNIT, "Invalid option keep unit: only m, h, d allowed") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_AGG_FUNC_NESTING, "Aggregate functions do not support nesting") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_STATE_WIN_TYPE, "Only support STATE_WINDOW on integer/bool/varchar column") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_STATE_WIN_COL, "Not support STATE_WINDOW on tag column") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_STATE_WIN_TABLE, "STATE_WINDOW not support for super table query") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INTER_SESSION_GAP, "SESSION gap should be fixed time window, and greater than 0") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INTER_SESSION_COL, "Only support SESSION on primary timestamp column") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INTER_OFFSET_NEGATIVE, "Interval offset cannot be negative") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INTER_OFFSET_UNIT, "Cannot use 'year' as offset when interval is 'month'") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INTER_OFFSET_TOO_BIG, "Interval offset should be shorter than interval") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INTER_SLIDING_UNIT, "Does not support sliding when interval is natural month/year") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INTER_SLIDING_TOO_BIG, "sliding value no larger than the interval value") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INTER_SLIDING_TOO_SMALL, "sliding value can not less than 1% of interval value") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_ONLY_ONE_JSON_TAG, "Only one tag if there is a json tag") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INCORRECT_NUM_OF_COL, "Query block has incorrect number of result columns") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INCORRECT_TIMESTAMP_VAL, "Incorrect TIMESTAMP value") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_DAYS_VALUE, "Invalid days value, should be keep2 >= keep1 >= keep0 >= days") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_OFFSET_LESS_ZERO, "soffset/offset can not be less than 0") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_SLIMIT_LEAK_PARTITION_BY, "slimit/soffset only available for PARTITION BY query") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_TOPIC_QUERY, "Invalid topic query") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_DROP_STABLE, "Cannot drop super table in batch") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_FILL_TIME_RANGE, "Start(end) time of query range required or time range too large") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_DUPLICATED_COLUMN, "Duplicated column names") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_TAGS_LENGTH, "Tags length exceeds max length") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_ROW_LENGTH, "Row length exceeds max length") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_COLUMNS_NUM, "Illegal number of columns") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_TOO_MANY_COLUMNS, "Too many columns") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_FIRST_COLUMN, "First column must be timestamp") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_VAR_COLUMN_LEN, "Invalid binary/nchar column length") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_TAGS_NUM, "Invalid number of tag columns") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_INTERNAL_PK, "Invalid _c0 or _rowts expression") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_TIMELINE_FUNC, "Invalid timeline function") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_PASSWD, "Invalid password") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_ALTER_TABLE, "Invalid alter table statement") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_CANNOT_DROP_PRIMARY_KEY, "Primary timestamp column cannot be dropped") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_MODIFY_COL, "Only binary/nchar column length could be modified") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_TBNAME, "Invalid tbname pseudo column") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_FUNCTION_NAME, "Invalid function name") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_COMMENT_TOO_LONG, "Comment too long") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_NOT_ALLOWED_FUNC, "Some functions are allowed only in the SELECT list of a query. " + "And, cannot be mixed with other non scalar functions or columns.") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_NOT_ALLOWED_WIN_QUERY, "Window query not supported, since the result of subquery not include valid timestamp column") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_DROP_COL, "No columns can be dropped") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_COL_JSON, "Only tag can be json type") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_VALUE_TOO_LONG, "Value too long for column/tag") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_DELETE_WHERE, "The DELETE statement must have a definite time window range") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_REDISTRIBUTE_VG, "The REDISTRIBUTE VGROUP statement only support 1 to 3 dnodes") //planner TAOS_DEFINE_ERROR(TSDB_CODE_PLAN_INTERNAL_ERROR, "Planner internal error") diff --git a/source/util/src/tpagedbuf.c b/source/util/src/tpagedbuf.c index 101ac78e18..cdf2629671 100644 --- a/source/util/src/tpagedbuf.c +++ b/source/util/src/tpagedbuf.c @@ -651,3 +651,31 @@ void dBufPrintStatis(const SDiskbasedBuf* pBuf) { ps->getPages, ps->releasePages, ps->flushBytes / 1024.0f, ps->flushPages, ps->loadBytes / 1024.0f, ps->loadPages, ps->loadBytes / (1024.0 * ps->loadPages)); } + +void clearDiskbasedBuf(SDiskbasedBuf* pBuf) { + SArray** p = taosHashIterate(pBuf->groupSet, NULL); + while (p) { + size_t n = taosArrayGetSize(*p); + for (int32_t i = 0; i < n; ++i) { + SPageInfo* pi = taosArrayGetP(*p, i); + taosMemoryFreeClear(pi->pData); + taosMemoryFreeClear(pi); + } + taosArrayDestroy(*p); + p = taosHashIterate(pBuf->groupSet, p); + } + + tdListEmpty(pBuf->lruList); + tdListEmpty(pBuf->freePgList); + + taosArrayClear(pBuf->emptyDummyIdList); + taosArrayClear(pBuf->pFree); + + taosHashClear(pBuf->groupSet); + taosHashClear(pBuf->all); + + pBuf->numOfPages = 0; // all pages are in buffer in the first place + pBuf->totalBufSize = 0; + pBuf->allocateId = -1; + pBuf->fileSize = 0; +} \ No newline at end of file diff --git a/tests/pytest/test.py b/tests/pytest/test.py index 30ab6ae3cc..1b185ef189 100644 --- a/tests/pytest/test.py +++ b/tests/pytest/test.py @@ -131,7 +131,7 @@ if __name__ == "__main__": is_test_framework = 0 key_word = 'tdCases.addWindows' try: - if key_word in open(fileName).read(): + if key_word in open(fileName, encoding='UTF-8').read(): is_test_framework = 1 except: pass diff --git a/tests/pytest/util/dnodes.py b/tests/pytest/util/dnodes.py index 63b841cfc5..c2aed6555d 100644 --- a/tests/pytest/util/dnodes.py +++ b/tests/pytest/util/dnodes.py @@ -416,7 +416,7 @@ class TDDnode: psCmd, shell=True).decode("utf-8") if not platform.system().lower() == 'windows': for port in range(6030, 6041): - fuserCmd = "fuser -k -n tcp %d" % port + fuserCmd = "fuser -k -n tcp %d > /dev/null" % port os.system(fuserCmd) if self.valgrind: time.sleep(2) diff --git a/tests/script/jenkins/basic.txt b/tests/script/jenkins/basic.txt index 32056e8d0c..28bc98a972 100644 --- a/tests/script/jenkins/basic.txt +++ b/tests/script/jenkins/basic.txt @@ -21,7 +21,7 @@ ./test.sh -f tsim/db/taosdlog.sim # ---- dnode -./test.sh -f tsim/dnode/basic1.sim +./test.sh -f tsim/dnode/create_dnode.sim # ---- insert ./test.sh -f tsim/insert/basic0.sim @@ -72,6 +72,10 @@ ./test.sh -f tsim/stream/basic2.sim # ./test.sh -f tsim/stream/session0.sim # ./test.sh -f tsim/stream/session1.sim +# ./test.sh -f tsim/stream/state0.sim +# ./test.sh -f tsim/stream/triggerInterval0.sim +# ./test.sh -f tsim/stream/triggerSession0.sim + # ---- transaction ./test.sh -f tsim/trans/lossdata1.sim diff --git a/tests/script/tsim/dnode/basic1.sim b/tests/script/tsim/dnode/create_dnode.sim similarity index 100% rename from tests/script/tsim/dnode/basic1.sim rename to tests/script/tsim/dnode/create_dnode.sim diff --git a/tests/script/tsim/stream/session0.sim b/tests/script/tsim/stream/session0.sim index a5cd73d17e..a2fe773edb 100644 --- a/tests/script/tsim/stream/session0.sim +++ b/tests/script/tsim/stream/session0.sim @@ -23,89 +23,98 @@ sql insert into t1 values(1648791223001,10,2,3,1.1,2); sql insert into t1 values(1648791233002,3,2,3,2.1,3); sql insert into t1 values(1648791243003,NULL,NULL,NULL,NULL,4); sql insert into t1 values(1648791213002,NULL,NULL,NULL,NULL,5) (1648791233012,NULL,NULL,NULL,NULL,6); + +$loop_count = 0 +loop0: + sleep 300 sql select * from streamt order by s desc; +$loop_count = $loop_count + 1 +if $loop_count == 10 then + return -1 +endi + # row 0 if $data01 != 3 then - print ======$data01 - return -1 + print ======data01=$data01 + goto loop0 endi if $data02 != 3 then - print ======$data02 - return -1 + print ======data02=$data02 + goto loop0 endi if $data03 != 3 then - print ======$data03 - return -1 + print ======data03=$data03 + goto loop0 endi if $data04 != 2.100000000 then - print ======$data04 + print ======data04=$data04 return -1 endi if $data05 != 0.000000000 then - print ======$data05 + print ======data05=$data05 return -1 endi if $data06 != 3 then - print ======$data05 + print ======data06=$data06 return -1 endi if $data07 != 2.100000000 then - print ======$data05 + print ======data07=$data07 return -1 endi if $data08 != 6 then - print ======$data05 + print ======data08=$data08 return -1 endi # row 1 if $data11 != 3 then - print ======$data11 - return -1 + print ======data11=$data11 + goto loop0 endi if $data12 != 10 then - print ======$data12 - return -1 + print ======data12=$data12 + goto loop0 endi if $data13 != 10 then - print ======$data13 - return -1 + print ======data13=$data13 + goto loop0 endi if $data14 != 1.100000000 then - print ======$data14 + print ======data14=$data14 return -1 endi if $data15 != 0.000000000 then - print ======$data15 + print ======data15=$data15 return -1 endi if $data16 != 10 then - print ======$data15 + print ======data16=$data16 return -1 endi if $data17 != 1.100000000 then - print ======$data17 + print ======data17=$data17 return -1 endi if $data18 != 5 then - print ======$data18 + print ======data18=$data18 return -1 endi @@ -115,23 +124,31 @@ sql insert into t1 values(1648791233002,3,2,3,2.1,9); sql insert into t1 values(1648791243003,4,2,3,3.1,10); sql insert into t1 values(1648791213002,4,2,3,4.1,11) ; sql insert into t1 values(1648791213002,4,2,3,4.1,12) (1648791223009,4,2,3,4.1,13); + +$loop_count = 0 +loop1: sleep 300 sql select * from streamt order by s desc ; +$loop_count = $loop_count + 1 +if $loop_count == 10 then + return -1 +endi + # row 0 if $data01 != 7 then - print ======$data01 - return -1 + print =====data01=$data01 + goto loop1 endi if $data02 != 9 then - print ======$data02 - return -1 + print =====data02=$data02 + goto loop1 endi if $data03 != 4 then - print ======$data03 - return -1 + print =====data03=$data03 + goto loop1 endi if $data04 != 1.100000000 then diff --git a/tests/script/tsim/stream/state0.sim b/tests/script/tsim/stream/state0.sim index 3529f836f4..2f2038b914 100644 --- a/tests/script/tsim/stream/state0.sim +++ b/tests/script/tsim/stream/state0.sim @@ -20,21 +20,33 @@ sql create stream streams1 trigger at_once into streamt1 as select _wstartts, sql insert into t1 values(1648791213000,1,2,3,1.0,1); sql insert into t1 values(1648791213000,1,2,3,1.0,2); +$loop_count = 0 +loop0: sql select * from streamt1 order by c desc; sleep 300 +$loop_count = $loop_count + 1 +if $loop_count == 10 then + return -1 +endi if $rows != 1 then - print ======$rows - return -1; + print =====rows=$rows + goto loop0 endi sql insert into t1 values(1648791214000,1,2,3,1.0,3); +$loop_count = 0 +loop00: sql select * from streamt1 order by c desc; sleep 300 +$loop_count = $loop_count + 1 +if $loop_count == 10 then + return -1 +endi if $rows != 1 then - print ======$rows - return -1; + print =====rows=$rows + goto loop00 endi sql insert into t1 values(1648791213010,2,2,3,1.0,4); @@ -44,27 +56,25 @@ $loop_count = 0 loop1: sql select * from streamt1 where c >=4 order by `_wstartts`; sleep 300 - $loop_count = $loop_count + 1 if $loop_count == 10 then return -1 endi if $rows != 3 then - print ======$rows + print =====rows=$rows goto loop1 - return -1 endi # row 0 if $data01 != 1 then - print ======$data01 - return -1 + print =====data01=$data01 + goto loop1 endi if $data02 != 1 then - print ======$data02 - return -1 + print =====data02=$data02 + goto loop1 endi if $data03 != 1 then @@ -151,11 +161,10 @@ endi sql insert into t1 values(1648791213011,1,2,3,1.0,7); -loop2: $loop_count = 0 +loop2: sql select * from streamt1 where c in (5,4,7) order by `_wstartts`; sleep 300 - $loop_count = $loop_count + 1 if $loop_count == 10 then return -1 @@ -163,57 +172,51 @@ endi # row 2 if $data21 != 2 then - print ======$data21 + print =====data21=$data21 goto loop2 return -1 endi if $data22 != 2 then - print ======$data22 + print =====data22=$data22 goto loop2 return -1 endi if $data23 != 2 then print ======$data23 - goto loop2 return -1 endi if $data24 != 1 then print ======$data24 - goto loop2 return -1 endi if $data25 != 3 then print ======$data25 - goto loop2 return -1 endi if $data26 != 7 then print ======$data26 - goto loop2 return -1 endi sql insert into t1 values(1648791213011,1,2,3,1.0,8); -loop21: $loop_count = 0 +loop21: sql select * from streamt1 where c in (5,4,8) order by `_wstartts`; sleep 300 - $loop_count = $loop_count + 1 if $loop_count == 10 then return -1 endi if $data26 != 8 then - print ======$data26 + print =====data26=$data26 goto loop21 - return -1 endi @@ -222,11 +225,10 @@ sql insert into t1 values(1648791213020,3,2,3,1.0,10); sql insert into t1 values(1648791214000,1,2,3,1.0,11); sql insert into t1 values(1648791213011,10,20,10,10.0,12); -loop3: $loop_count = 0 +loop3: sql select * from streamt1 where c in (5,4,10,11,12) order by `_wstartts`; sleep 300 - $loop_count = $loop_count + 1 if $loop_count == 10 then return -1 @@ -234,112 +236,100 @@ endi # row 2 if $data21 != 1 then - print ======$data21 + print =====data21=$data21 goto loop3 return -1 endi if $data22 != 1 then - print ======$data22 + print =====data22=$data22 goto loop3 return -1 endi if $data23 != 10 then print ======$data23 - goto loop3 return -1 endi if $data24 != 10 then print ======$data24 - goto loop3 return -1 endi if $data25 != 10 then print ======$data25 - goto loop3 return -1 endi if $data26 != 12 then print ======$data26 - goto loop3 return -1 endi # row 3 if $data31 != 1 then - print ======$data31 + print =====data31=$data31 goto loop3 return -1 endi if $data32 != 1 then - print ======$data32 + print =====data32=$data32 goto loop3 return -1 endi if $data33 != 3 then print ======$data33 - goto loop3 return -1 endi if $data34 != 3 then print ======$data34 - goto loop3 return -1 endi if $data35 != 3 then print ======$data35 - goto loop3 return -1 endi if $data36 != 10 then print ======$data36 - goto loop3 return -1 endi # row 4 if $data41 != 1 then - print ======$data41 + print =====data41=$data41 goto loop3 return -1 endi if $data42 != 1 then - print ======$data42 + print =====data42=$data42 goto loop3 return -1 endi if $data43 != 1 then print ======$data43 - goto loop3 return -1 endi if $data44 != 1 then print ======$data44 - goto loop3 return -1 endi if $data45 != 3 then print ======$data45 - goto loop3 return -1 endi if $data46 != 11 then print ======$data46 - goto loop3 return -1 endi @@ -347,8 +337,8 @@ sql insert into t1 values(1648791213030,3,12,12,12.0,13); sql insert into t1 values(1648791214040,1,13,13,13.0,14); sql insert into t1 values(1648791213030,3,14,14,14.0,15) (1648791214020,15,15,15,15.0,16); -loop4: $loop_count = 0 +loop4: sql select * from streamt1 where c in (14,15,16) order by `_wstartts`; sleep 300 @@ -358,119 +348,104 @@ if $loop_count == 10 then endi if $rows != 3 then - print ======$rows - goto loop4 - return -1; + print ====loop4=rows=$rows +# goto loop4 endi # row 0 if $data01 != 2 then - print ======$data01 + print =====data01=$data01 goto loop4 - return -1 endi if $data02 != 2 then print ======$data02 - goto loop4 return -1 endi if $data03 != 6 then print ======$data03 - goto loop4 return -1 endi if $data04 != 3 then print ======$data04 - goto loop4 return -1 endi if $data05 != 3 then print ======$data05 - goto loop4 return -1 endi if $data06 != 15 then print ======$data06 - goto loop4 return -1 endi # row 1 if $data11 != 1 then - print ======$data11 + print =====data11=$data11 goto loop4 return -1 endi if $data12 != 1 then - print ======$data12 + print =====data12=$data12 goto loop4 return -1 endi if $data13 != 15 then print ======$data13 - goto loop4 return -1 endi if $data14 != 15 then print ======$data14 - goto loop4 return -1 endi if $data15 != 15 then print ======$data15 - goto loop4 return -1 endi if $data16 != 16 then print ======$data16 - goto loop4 return -1 endi # row 2 if $data21 != 1 then - print ======$data21 + print =====data21=$data21 goto loop4 return -1 endi if $data22 != 1 then - print ======$data22 + print =====data22=$data22 goto loop4 return -1 endi if $data23 != 1 then print ======$data23 - goto loop4 return -1 endi if $data24 != 1 then print ======$data24 - goto loop4 return -1 endi if $data25 != 13 then print ======$data25 - goto loop4 return -1 endi if $data26 != 14 then print ======$data26 - goto loop4 return -1 endi diff --git a/tests/script/tsim/sync/oneReplica1VgElectWithInsert.sim b/tests/script/tsim/sync/oneReplica1VgElectWithInsert.sim index 06a67b3c1b..b816226cff 100644 --- a/tests/script/tsim/sync/oneReplica1VgElectWithInsert.sim +++ b/tests/script/tsim/sync/oneReplica1VgElectWithInsert.sim @@ -31,7 +31,7 @@ if $data[0][4] != ready then goto check_dnode_ready endi -#sql connect +sql connect sql create dnode $hostname port 7200 sql create dnode $hostname port 7300 sql create dnode $hostname port 7400 @@ -71,7 +71,7 @@ sql create database db replica $replica vgroups $vgroups $loop_cnt = 0 check_db_ready: $loop_cnt = $loop_cnt + 1 -sleep 200 +sleep 20 if $loop_cnt == 10 then print ====> db not ready! return -1 @@ -93,13 +93,12 @@ $loop_cnt = 0 check_vg_ready: $loop_cnt = $loop_cnt + 1 sleep 200 -if $loop_cnt == 10 then +if $loop_cnt == 300 then print ====> vgroups not ready! return -1 endi sql show vgroups print ===> rows: $rows -print $data(2)[0] $data(2)[1] $data(2)[2] $data(2)[3] $data(2)[4] $data(2)[5] $data(2)[6] $data(2)[7] $data(2)[8] $data(2)[9] $data(2)[10] $data(2)[11] $data(2)[12] $data(2)[13] print $data[0][0] $data[0][1] $data[0][2] $data[0][3] $data[0][4] $data[0][5] $data[0][6] $data[0][7] $data[0][8] $data[0][9] $data[10][6] $data[0][11] $data[0][12] $data[0][13] if $rows != $vgroups then return -1 @@ -132,10 +131,12 @@ if $data[0][8] == leader then goto check_vg_ready endi print ---- vgroup $data[0][0] leader locate on dnode $data[0][7] - goto vg_ready + goto vg_ready +else + goto check_vg_ready endi -vg_ready: +vg_ready: print ====> create stable/child table sql create table stb (ts timestamp, c1 int, c2 float, c3 binary(10)) tags (t1 int) @@ -156,27 +157,13 @@ while $i < $tbNum sql create table $ctb using stb tags( $i ) $ntb = $ntbPrefix . $i sql create table $ntb (ts timestamp, c1 int, c2 float, c3 binary(10)) - -# $x = 0 -# while $x < $rowNum -# $binary = ' . binary -# $binary = $binary . $i -# $binary = $binary . ' -# -# sql insert into $ctb values ($tstart , $i , $x , $binary ) -# sql insert into $ntb values ($tstart , 999 , 999 , 'binary-ntb' ) -# $tstart = $tstart + 1 -# $x = $x + 1 -# endw - -# print ====> insert rows: $rowNum into $ctb and $ntb - $i = $i + 1 -# $tstart = 1640966400000 endw $totalTblNum = $tbNum * 2 +sleep 1000 sql show tables +print ====> expect $totalTblNum and infinsert $rows in fact if $rows != $totalTblNum then return -1 endi @@ -185,7 +172,7 @@ print ====> create a normal table for interaction between main and back threads sql create table interaction (ts timestamp, flag binary(10), childrows int, stbrows int) print ====> start to run_back to insert data -run_back tsim/tmq/insertDataByRunBack.sim +run_back tsim/sync/insertDataByRunBack.sim print ====> waiting insert thread starting insert data @@ -222,12 +209,15 @@ endi $dnodeId = dnode . $dnodeId print ====> stop $dnodeId system sh/exec.sh -n $dnodeId -s stop -x SIGINT +sleep 1000 +print ====> start $dnodeId +system sh/exec.sh -n $dnodeId -s start $loop_cnt = 0 check_vg_ready_2: $loop_cnt = $loop_cnt + 1 sleep 200 -if $loop_cnt == 10 then +if $loop_cnt == 300 then print ====> vgroups switch fail!!! return -1 endi @@ -242,38 +232,38 @@ if $data[0][4] == leader then if $data[0][6] != NULL then goto check_vg_ready_2 endi - if $data[0][8] != NULL then + if $data[0][8] != FOLLOWER then goto check_vg_ready_2 endi - print ---- vgroup $data[0][0] leader switch to dnode $data[0][3] + print ---- vgroup $dnodeId leader switch to dnode $data[0][3] goto vg_ready_2 endi if $data[0][6] == leader then if $data[0][4] != NULL then goto check_vg_ready_2 endi - if $data[0][8] != NULL then + if $data[0][8] != FOLLOWER then goto check_vg_ready_2 endi - print ---- vgroup $data[0][0] leader switch to dnode $data[0][5] + print ---- vgroup $dnodeId leader switch to dnode $data[0][5] goto vg_ready_2 endi if $data[0][8] == leader then if $data[0][4] != NULL then goto check_vg_ready_2 endi - if $data[0][6] != NULL then + if $data[0][6] != FOLLOWER then goto check_vg_ready_2 endi - print ---- vgroup $data[0][0] leader switch to dnode $data[0][7] + print ---- vgroup $dnodeId leader switch to dnode $data[0][7] goto vg_ready_2 +else + goto check_vg_ready_2 endi vg_ready_2: $switch_loop_cnt = $switch_loop_cnt + 1 if $switch_loop_cnt < 3 then - print ====> start $dnodeId - system sh/exec.sh -n $dnodeId -s start goto switch_leader_loop endi @@ -347,7 +337,7 @@ if $data[0][4] == leader then if $data[0][6] != NULL then goto check_vg_ready_1 endi - if $data[0][8] != NULL then + if $data[0][8] != FOLLOWER then goto check_vg_ready_1 endi goto vg_ready_1 @@ -356,7 +346,7 @@ if $data[0][6] == leader then if $data[0][4] != NULL then goto check_vg_ready_1 endi - if $data[0][8] != NULL then + if $data[0][8] != FOLLOWER then goto check_vg_ready_1 endi goto vg_ready_1 @@ -365,7 +355,7 @@ if $data[0][8] == leader then if $data[0][4] != NULL then goto check_vg_ready_1 endi - if $data[0][6] != NULL then + if $data[0][6] != FOLLOWER then goto check_vg_ready_1 endi goto vg_ready_1 @@ -394,6 +384,73 @@ if $data[0][0] != $totalRowsOfStb then return -1 endi +print ====> once stop one dnode by loop, and do query every time +$i = 2 +loop_stop_dnode: +$dnodeId = dnode . $i +print ====> stop $dnodeId +system sh/exec.sh -n $dnodeId -s stop -x SIGINT + +check_vg_ready_3: +sql show vgroups +print $data(2)[0] $data(2)[1] $data(2)[2] $data(2)[3] $data(2)[4] $data(2)[5] $data(2)[6] $data(2)[7] $data(2)[8] $data(2)[9] $data(2)[10] $data(2)[11] $data(2)[12] $data(2)[13] +if $data[0][4] == LEADER then + if $data[0][6] == LEADER then + goto check_vg_ready_3 + endi + if $data[0][8] == LEADER then + goto check_vg_ready_3 + endi + print ---- vgroup $data[0][0] leader locating dnode $data[0][5] +elif $data[0][6] == LEADER then + if $data[0][4] == LEADER then + goto check_vg_ready_3 + endi + if $data[0][8] == LEADER then + goto check_vg_ready_3 + endi + print ---- vgroup $data[0][0] leader locating dnode $data[0][7] +elif $data[0][8] == LEADER then + if $data[0][4] == LEADER then + goto check_vg_ready_3 + endi + if $data[0][6] == LEADER then + goto check_vg_ready_3 + endi + print ---- vgroup $data[0][0] leader locating dnode $data[0][9] +else + print ====> no leader vnode!!! + return -1 +endi + +sql select count(*) from ntb0 +print rows: $rows +print $data[0][0] $data[0][1] +if $data[0][0] != $totalRowsOfCtb then + return -1 +endi + +sql select count(*) from ctb0 +print rows: $rows +print $data[0][0] $data[0][1] +if $data[0][0] != $totalRowsOfCtb then + return -1 +endi + +sql select count(*) from stb +print rows: $rows +print $data[0][0] $data[0][1] +if $data[0][0] != $totalRowsOfStb then + return -1 +endi + +$i = $i + 1 +if $i <= 4 then + print ====> start $dnodeId + system sh/exec.sh -n $dnodeId -s start + goto loop_stop_dnode +endi + system sh/exec.sh -n dnode1 -s stop -x SIGINT system sh/exec.sh -n dnode2 -s stop -x SIGINT system sh/exec.sh -n dnode3 -s stop -x SIGINT diff --git a/tests/system-test/1-insert/test_stmt_insert_query_ex.py b/tests/system-test/1-insert/test_stmt_insert_query_ex.py index 376b60d615..c0836b79c8 100644 --- a/tests/system-test/1-insert/test_stmt_insert_query_ex.py +++ b/tests/system-test/1-insert/test_stmt_insert_query_ex.py @@ -80,93 +80,9 @@ class TDTestCase: con=taos.connect(host=host, user=user, password=password, config=cfg ,port=port) print(con) return con - - def test_stmt_insert_multi(self,conn): - # type: (TaosConnection) -> None - - dbname = "pytest_taos_stmt_multi" - try: - conn.execute("drop database if exists %s" % dbname) - conn.execute("create database if not exists %s" % dbname) - conn.select_db(dbname) - - conn.execute( - "create table if not exists log(ts timestamp, bo bool, nil tinyint, ti tinyint, si smallint, ii int,\ - bi bigint, tu tinyint unsigned, su smallint unsigned, iu int unsigned, bu bigint unsigned, \ - ff float, dd double, bb binary(100), nn nchar(100), tt timestamp)", - ) - # conn.load_table_info("log") - - start = datetime.now() - stmt = conn.statement("insert into log values(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)") - - params = new_multi_binds(16) - params[0].timestamp((1626861392589, 1626861392590, 1626861392591)) - params[1].bool((True, None, False)) - params[2].tinyint([-128, -128, None]) # -128 is tinyint null - params[3].tinyint([0, 127, None]) - params[4].smallint([3, None, 2]) - params[5].int([3, 4, None]) - params[6].bigint([3, 4, None]) - params[7].tinyint_unsigned([3, 4, None]) - params[8].smallint_unsigned([3, 4, None]) - params[9].int_unsigned([3, 4, None]) - params[10].bigint_unsigned([3, 4, None]) - params[11].float([3, None, 1]) - params[12].double([3, None, 1.2]) - params[13].binary(["abc", "dddafadfadfadfadfa", None]) - params[14].nchar(["涛思数据", None, "a long string with 中文字符"]) - params[15].timestamp([None, None, 1626861392591]) - # print(type(stmt)) - stmt.bind_param_batch(params) - stmt.execute() - end = datetime.now() - print("elapsed time: ", end - start) - assert stmt.affected_rows == 3 - - #query - querystmt=conn.statement("select ?,bu from log") - queryparam=new_bind_params(1) - print(type(queryparam)) - queryparam[0].binary("ts") - querystmt.bind_param(queryparam) - querystmt.execute() - result=querystmt.use_result() - # rows=result.fetch_all() - # print( querystmt.use_result()) - - # result = conn.query("select * from log") - rows=result.fetch_all() - # rows=result.fetch_all() - print(rows) - assert rows[1][0] == "ts" - assert rows[0][1] == 3 - - #query - querystmt1=conn.statement("select * from log where bu < ?") - queryparam1=new_bind_params(1) - print(type(queryparam1)) - queryparam1[0].int(4) - querystmt1.bind_param(queryparam1) - querystmt1.execute() - result1=querystmt1.use_result() - rows1=result1.fetch_all() - assert str(rows1[0][0]) == "2021-07-21 17:56:32.589000" - assert rows1[0][10] == 3 - - - stmt.close() - - # conn.execute("drop database if exists %s" % dbname) - conn.close() - - except Exception as err: - # conn.execute("drop database if exists %s" % dbname) - conn.close() - raise err def test_stmt_set_tbname_tag(self,conn): - dbname = "pytest_taos_stmt_set_tbname_tag" + dbname = "stmt_set_tbname_tag" try: conn.execute("drop database if exists %s" % dbname) @@ -174,16 +90,16 @@ class TDTestCase: conn.select_db(dbname) conn.execute("create table if not exists log(ts timestamp, bo bool, nil tinyint, ti tinyint, si smallint, ii int,\ bi bigint, tu tinyint unsigned, su smallint unsigned, iu int unsigned, bu bigint unsigned, \ - ff float, dd double, bb binary(100), nn nchar(100), tt timestamp) tags (t1 timestamp, t2 bool,\ + ff float, dd double, bb binary(100), nn nchar(100), tt timestamp , vc varchar(100)) tags (t1 timestamp, t2 bool,\ t3 tinyint, t4 tinyint, t5 smallint, t6 int, t7 bigint, t8 tinyint unsigned, t9 smallint unsigned, \ t10 int unsigned, t11 bigint unsigned, t12 float, t13 double, t14 binary(100), t15 nchar(100), t16 timestamp)") stmt = conn.statement("insert into ? using log tags (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) \ - values (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)") + values (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)") tags = new_bind_params(16) tags[0].timestamp(1626861392589123, PrecisionEnum.Microseconds) tags[1].bool(True) - tags[2].null() + tags[2].bool(False) tags[3].tinyint(2) tags[4].smallint(3) tags[5].int(4) @@ -198,7 +114,7 @@ class TDTestCase: tags[14].nchar("stmt") tags[15].timestamp(1626861392589, PrecisionEnum.Milliseconds) stmt.set_tbname_tags("tb1", tags) - params = new_multi_binds(16) + params = new_multi_binds(17) params[0].timestamp((1626861392589111, 1626861392590111, 1626861392591111)) params[1].bool((True, None, False)) params[2].tinyint([-128, -128, None]) # -128 is tinyint null @@ -213,25 +129,32 @@ class TDTestCase: params[11].float([3, None, 1]) params[12].double([3, None, 1.2]) params[13].binary(["abc", "dddafadfadfadfadfa", None]) - params[14].nchar(["涛思数据", None, "a? long string with 中文字符"]) + params[14].nchar(["涛思数据", None, "a long string with 中文字符"]) params[15].timestamp([None, None, 1626861392591]) - + params[16].binary(["涛思数据16", None, "a long string with 中文-字符"]) + stmt.bind_param_batch(params) stmt.execute() assert stmt.affected_rows == 3 - #query + #query all querystmt1=conn.statement("select * from log where bu < ?") queryparam1=new_bind_params(1) print(type(queryparam1)) - queryparam1[0].int(5) + queryparam1[0].int(10) querystmt1.bind_param(queryparam1) querystmt1.execute() result1=querystmt1.use_result() rows1=result1.fetch_all() - print("1",rows1) + print(rows1[0]) + print(rows1[1]) + print(rows1[2]) + assert str(rows1[0][0]) == "2021-07-21 17:56:32.589111" + assert rows1[0][10] == 3 + assert rows1[1][10] == 4 + #query: Numeric Functions querystmt2=conn.statement("select abs(?) from log where bu < ?") queryparam2=new_bind_params(2) print(type(queryparam2)) @@ -242,6 +165,11 @@ class TDTestCase: result2=querystmt2.use_result() rows2=result2.fetch_all() print("2",rows2) + assert rows2[0][0] == 5 + assert rows2[1][0] == 5 + + + #query: Numeric Functions and escapes querystmt3=conn.statement("select abs(?) from log where nn= 'a? long string with 中文字符' ") queryparam3=new_bind_params(1) @@ -252,9 +180,63 @@ class TDTestCase: result3=querystmt3.use_result() rows3=result3.fetch_all() print("3",rows3) - # assert str(rows1[0][0]) == "2021-07-21 17:56:32.589111" - # assert rows1[0][10] == 3 - # assert rows1[1][10] == 4 + assert rows3 == [] + + #query: string Functions + + querystmt3=conn.statement("select CHAR_LENGTH(?) from log ") + queryparam3=new_bind_params(1) + print(type(queryparam3)) + queryparam3[0].binary('中文字符') + querystmt3.bind_param(queryparam3) + querystmt3.execute() + result3=querystmt3.use_result() + rows3=result3.fetch_all() + print("4",rows3) + assert rows3[0][0] == 12, 'fourth case is failed' + assert rows3[1][0] == 12, 'fourth case is failed' + + # #query: conversion Functions + + # querystmt4=conn.statement("select cast( ? as bigint) from log ") + # queryparam4=new_bind_params(1) + # print(type(queryparam4)) + # queryparam4[0].binary('1232a') + # querystmt4.bind_param(queryparam4) + # querystmt4.execute() + # result4=querystmt4.use_result() + # rows4=result4.fetch_all() + # print("5",rows4) + # assert rows4[0][0] == 1232 + # assert rows4[1][0] == 1232 + + # querystmt4=conn.statement("select cast( ? as binary(10)) from log ") + # queryparam4=new_bind_params(1) + # print(type(queryparam4)) + # queryparam4[0].int(123) + # querystmt4.bind_param(queryparam4) + # querystmt4.execute() + # result4=querystmt4.use_result() + # rows4=result4.fetch_all() + # print("6",rows4) + # assert rows4[0][0] == '123' + # assert rows4[1][0] == '123' + + # #query: datatime Functions + + # querystmt4=conn.statement(" select timediff('2021-07-21 17:56:32.590111',?,1s) from log ") + # queryparam4=new_bind_params(1) + # print(type(queryparam4)) + # queryparam4[0].timestamp(1626861392591111) + # querystmt4.bind_param(queryparam4) + # querystmt4.execute() + # result4=querystmt4.use_result() + # rows4=result4.fetch_all() + # print("7",rows4) + # assert rows4[0][0] == 1, 'seventh case is failed' + # assert rows4[1][0] == 1, 'seventh case is failed' + + # conn.execute("drop database if exists %s" % dbname) conn.close() @@ -269,8 +251,6 @@ class TDTestCase: config = buildPath+ "../sim/dnode1/cfg/" host="localhost" connectstmt=self.newcon(host,config) - self.test_stmt_insert_multi(connectstmt) - connectstmt=self.newcon(host,config) self.test_stmt_set_tbname_tag(connectstmt) return diff --git a/tests/system-test/1-insert/test_stmt_muti_insert_query.py b/tests/system-test/1-insert/test_stmt_muti_insert_query.py index 486bcd8062..f838fd90f5 100644 --- a/tests/system-test/1-insert/test_stmt_muti_insert_query.py +++ b/tests/system-test/1-insert/test_stmt_muti_insert_query.py @@ -124,7 +124,7 @@ class TDTestCase: print("elapsed time: ", end - start) assert stmt.affected_rows == 3 - #query + #query 1 querystmt=conn.statement("select ?,bu from log") queryparam=new_bind_params(1) print(type(queryparam)) @@ -141,8 +141,9 @@ class TDTestCase: print(rows) assert rows[1][0] == "ts" assert rows[0][1] == 3 + assert rows[2][1] == None - #query + #query 2 querystmt1=conn.statement("select * from log where bu < ?") queryparam1=new_bind_params(1) print(type(queryparam1)) diff --git a/tests/system-test/1-insert/test_stmt_set_tbname_tag.py b/tests/system-test/1-insert/test_stmt_set_tbname_tag.py index 54d5cfbafb..fb9f455d4d 100644 --- a/tests/system-test/1-insert/test_stmt_set_tbname_tag.py +++ b/tests/system-test/1-insert/test_stmt_set_tbname_tag.py @@ -82,7 +82,7 @@ class TDTestCase: return con def test_stmt_set_tbname_tag(self,conn): - dbname = "pytest_taos_stmt_set_tbname_tag" + dbname = "stmt_set_tbname_tag" try: conn.execute("drop database if exists %s" % dbname) @@ -99,7 +99,7 @@ class TDTestCase: tags = new_bind_params(16) tags[0].timestamp(1626861392589123, PrecisionEnum.Microseconds) tags[1].bool(True) - tags[2].null() + tags[2].bool(False) tags[3].tinyint(2) tags[4].smallint(3) tags[5].int(4) @@ -114,7 +114,7 @@ class TDTestCase: tags[14].nchar("stmt") tags[15].timestamp(1626861392589, PrecisionEnum.Milliseconds) stmt.set_tbname_tags("tb1", tags) - params = new_multi_binds(16) + params = new_multi_binds(17) params[0].timestamp((1626861392589111, 1626861392590111, 1626861392591111)) params[1].bool((True, None, False)) params[2].tinyint([-128, -128, None]) # -128 is tinyint null @@ -129,28 +129,153 @@ class TDTestCase: params[11].float([3, None, 1]) params[12].double([3, None, 1.2]) params[13].binary(["abc", "dddafadfadfadfadfa", None]) - params[14].nchar(["涛思数据", None, "a long string with 中文字符"]) + params[14].nchar(["涛思数据", None, "a long string with 中文?字符"]) params[15].timestamp([None, None, 1626861392591]) - params[16].binary(["涛思数据16", None, "a long string with 中文-字符"]) + params[16].binary(["涛思数据16", None, None]) stmt.bind_param_batch(params) stmt.execute() assert stmt.affected_rows == 3 - #query + #query all querystmt1=conn.statement("select * from log where bu < ?") queryparam1=new_bind_params(1) print(type(queryparam1)) - queryparam1[0].int(5) + queryparam1[0].int(10) querystmt1.bind_param(queryparam1) querystmt1.execute() result1=querystmt1.use_result() rows1=result1.fetch_all() - print(rows1) - # assert str(rows1[0][0]) == "2021-07-21 17:56:32.589111" - # assert rows1[0][10] == 3 - # assert rows1[1][10] == 4 + print(rows1[0]) + print(rows1[1]) + print(rows1[2]) + assert str(rows1[0][0]) == "2021-07-21 17:56:32.589111" + assert rows1[0][10] == 3 + assert rows1[1][10] == 4 + + #query: Numeric Functions + querystmt2=conn.statement("select abs(?) from log where bu < ?") + queryparam2=new_bind_params(2) + print(type(queryparam2)) + queryparam2[0].int(5) + queryparam2[1].int(5) + querystmt2.bind_param(queryparam2) + querystmt2.execute() + result2=querystmt2.use_result() + rows2=result2.fetch_all() + print("2",rows2) + assert rows2[0][0] == 5 + assert rows2[1][0] == 5 + + + #query: Numeric Functions and escapes + + querystmt3=conn.statement("select abs(?) from log where nn= 'a? long string with 中文字符' ") + queryparam3=new_bind_params(1) + print(type(queryparam3)) + queryparam3[0].int(5) + querystmt3.bind_param(queryparam3) + querystmt3.execute() + result3=querystmt3.use_result() + rows3=result3.fetch_all() + print("3",rows3) + assert rows3 == [] + + # #query: string Functions + + # querystmt3=conn.statement("select CHAR_LENGTH(?) from log ") + # queryparam3=new_bind_params(1) + # print(type(queryparam3)) + # queryparam3[0].binary('中文字符') + # querystmt3.bind_param(queryparam3) + # querystmt3.execute() + # result3=querystmt3.use_result() + # rows3=result3.fetch_all() + # print("4",rows3) + # assert rows3[0][0] == 12, 'fourth case is failed' + # assert rows3[1][0] == 12, 'fourth case is failed' + + # #query: conversion Functions + + # querystmt4=conn.statement("select cast( ? as bigint) from log ") + # queryparam4=new_bind_params(1) + # print(type(queryparam4)) + # queryparam4[0].binary('1232a') + # querystmt4.bind_param(queryparam4) + # querystmt4.execute() + # result4=querystmt4.use_result() + # rows4=result4.fetch_all() + # print("5",rows4) + # assert rows4[0][0] == 1232 + # assert rows4[1][0] == 1232 + + # querystmt4=conn.statement("select cast( ? as binary(10)) from log ") + # queryparam4=new_bind_params(1) + # print(type(queryparam4)) + # queryparam4[0].int(123) + # querystmt4.bind_param(queryparam4) + # querystmt4.execute() + # result4=querystmt4.use_result() + # rows4=result4.fetch_all() + # print("6",rows4) + # assert rows4[0][0] == '123' + # assert rows4[1][0] == '123' + + # #query: datatime Functions + + # querystmt4=conn.statement(" select timediff('2021-07-21 17:56:32.590111',?,1s) from log ") + # queryparam4=new_bind_params(1) + # print(type(queryparam4)) + # queryparam4[0].timestamp(1626861392591111) + # querystmt4.bind_param(queryparam4) + # querystmt4.execute() + # result4=querystmt4.use_result() + # rows4=result4.fetch_all() + # print("7",rows4) + # assert rows4[0][0] == 1, 'seventh case is failed' + # assert rows4[1][0] == 1, 'seventh case is failed' + + #query: aggregate Functions + + querystmt4=conn.statement(" select count(?) from log ") + queryparam4=new_bind_params(1) + print(type(queryparam4)) + queryparam4[0].int(123) + querystmt4.bind_param(queryparam4) + querystmt4.execute() + result4=querystmt4.use_result() + rows4=result4.fetch_all() + print("8",rows4) + assert rows4[0][0] == 3, ' 8 case is failed' + + #query: selector Functions 9 + + querystmt4=conn.statement(" select bottom(bu,?) from log group by bu ; ") + queryparam4=new_bind_params(1) + print(type(queryparam4)) + queryparam4[0].int(2) + querystmt4.bind_param(queryparam4) + querystmt4.execute() + result4=querystmt4.use_result() + rows4=result4.fetch_all() + print("9",rows4) + assert rows4[0][0] == 4, ' 9 case is failed' + assert rows4[1][0] == 3, ' 9 case is failed' + + # #query: time-series specific Functions 10 + + querystmt4=conn.statement(" select twa(?) from log; ") + queryparam4=new_bind_params(1) + print(type(queryparam4)) + queryparam4[0].int(15) + querystmt4.bind_param(queryparam4) + querystmt4.execute() + result4=querystmt4.use_result() + rows4=result4.fetch_all() + print("10",rows4) + assert rows4[0][0] == 15, ' 10 case is failed' + # conn.execute("drop database if exists %s" % dbname) conn.close() diff --git a/tests/system-test/2-query/between.py b/tests/system-test/2-query/between.py index 44750abd46..7e2ac1c8b9 100644 --- a/tests/system-test/2-query/between.py +++ b/tests/system-test/2-query/between.py @@ -143,7 +143,7 @@ class TDTestCase: tdSql.checkRows(9) tdSql.query("select * from t1 where c5 between 136 and 127") tdSql.checkRows(0) - tdSql.query("select * from t1 where c5 between '~' and 'ˆ'") + tdSql.query("select * from t1 where c5 between '~' and '^'") tdSql.checkRows(0) tdSql.query("select * from t1 where c5 not between 1 and 2") # tdSql.checkRows(0) diff --git a/tests/system-test/2-query/cast.py b/tests/system-test/2-query/cast.py index e07ab95f45..387170991f 100644 --- a/tests/system-test/2-query/cast.py +++ b/tests/system-test/2-query/cast.py @@ -565,7 +565,7 @@ class TDTestCase: if data_ct4_c10[i] is None: tdSql.checkData( i, 0, None ) else: - time2str = str(int(datetime.datetime.timestamp(data_ct4_c10[i])*1000)) + time2str = str(int((data_ct4_c10[i]-datetime.datetime.fromtimestamp(0)).total_seconds()*1000)) tdSql.checkData( i, 0, time2str ) tdSql.query("select cast(c10 as nchar(32)) as b from t1") for i in range(len(data_t1_c10)): @@ -574,7 +574,7 @@ class TDTestCase: elif i == 10: continue else: - time2str = str(int(datetime.datetime.timestamp(data_t1_c10[i])*1000)) + time2str = str(int((data_t1_c10[i]-datetime.datetime.fromtimestamp(0)).total_seconds()*1000)) tdSql.checkData( i, 0, time2str ) tdLog.printNoPrefix("==========step38: cast timestamp to binary, expect no changes ") @@ -583,7 +583,7 @@ class TDTestCase: if data_ct4_c10[i] is None: tdSql.checkData( i, 0, None ) else: - time2str = str(int(datetime.datetime.timestamp(data_ct4_c10[i])*1000)) + time2str = str(int((data_ct4_c10[i]-datetime.datetime.fromtimestamp(0)).total_seconds()*1000)) tdSql.checkData( i, 0, time2str ) tdSql.query("select cast(c10 as binary(32)) as b from t1") for i in range(len(data_t1_c10)): @@ -592,7 +592,7 @@ class TDTestCase: elif i == 10: continue else: - time2str = str(int(datetime.datetime.timestamp(data_t1_c10[i])*1000)) + time2str = str(int((data_t1_c10[i]-datetime.datetime.fromtimestamp(0)).total_seconds()*1000)) tdSql.checkData( i, 0, time2str ) tdLog.printNoPrefix("==========step39: cast constant operation to bigint, expect change to int ") diff --git a/tests/system-test/2-query/json_tag.py b/tests/system-test/2-query/json_tag.py index 7838b5c3e4..4d7aa2fd7e 100644 --- a/tests/system-test/2-query/json_tag.py +++ b/tests/system-test/2-query/json_tag.py @@ -109,9 +109,6 @@ class TDTestCase: # print("============== STEP 3 ===== query table") # # test error syntax tdSql.error("select * from jsons1 where jtag->tag1='beijing'") - #tdSql.error("select * from jsons1 where jtag->'location'") - #tdSql.error("select * from jsons1 where jtag->''") - #tdSql.error("select * from jsons1 where jtag->''=9") tdSql.error("select -> from jsons1") tdSql.error("select * from jsons1 where contains") tdSql.error("select * from jsons1 where jtag->") @@ -341,6 +338,13 @@ class TDTestCase: # tdSql.checkRows(3) # tdSql.query("select * from jsons1 where jtag->'tag1' between 'femail' and 'beijing'") # tdSql.checkRows(2) + + # test is true + tdSql.query("select * from jsons1 where jtag->'location'") + tdSql.checkRows(0) + tdSql.query("select * from jsons1 where jtag->'tag1'") + tdSql.checkRows(3) + # # # test with tbname/normal column tdSql.query("select * from jsons1 where tbname = 'jsons1_1'") diff --git a/tests/system-test/test-all.bat b/tests/system-test/test-all.bat index baafbc9e08..9c2e58ea3f 100644 --- a/tests/system-test/test-all.bat +++ b/tests/system-test/test-all.bat @@ -13,6 +13,9 @@ echo Linux Taosd Test for /F "usebackq tokens=*" %%i in (fulltest.bat) do ( for /f "tokens=1* delims= " %%a in ("%%i") do if not "%%a" == "@REM" ( echo Processing %%i + call :GetTimeSeconds %time% + set time1=!_timeTemp! + echo Start at %time% set /a a+=1 call %%i ARG1 -m %1 > result_!a!.txt 2>error_!a!.txt if errorlevel 1 ( call :colorEcho 0c "failed" &echo. && echo result: && cat result_!a!.txt && echo error: && cat error_!a!.txt && exit 8 ) else ( call :colorEcho 0a "Success" &echo. ) @@ -21,7 +24,33 @@ for /F "usebackq tokens=*" %%i in (fulltest.bat) do ( exit :colorEcho +call :GetTimeSeconds %time% +set time2=%_timeTemp% +set /a interTime=%time2% - %time1% +echo End at %time% , cast %interTime%s echo off "%~2" findstr /v /a:%1 /R "^$" "%~2" nul -del "%~2" > nul 2>&1i \ No newline at end of file +del "%~2" > nul 2>&1i +goto :eof + +:GetTimeSeconds +set tt=%1 +set tt=%tt:.= % +set tt=%tt::= % +set index=1 +for %%a in (%tt%) do ( + if !index! EQU 1 ( + set hh=%%a + )^ + else if !index! EQU 2 ( + set mm=%%a + + )^ + else if !index! EQU 3 ( + set ss=%%a + ) + set /a index=index+1 +) +set /a _timeTemp=(%hh%*60+%mm%)*60+%ss% +goto :eof \ No newline at end of file diff --git a/tests/system-test/test.py b/tests/system-test/test.py index d9ae792901..8350f6752f 100644 --- a/tests/system-test/test.py +++ b/tests/system-test/test.py @@ -168,7 +168,7 @@ if __name__ == "__main__": key_word = 'tdCases.addWindows' is_test_framework = 0 try: - if key_word in open(fileName).read(): + if key_word in open(fileName, encoding='UTF-8').read(): is_test_framework = 1 except: pass