xxq250
/
homework-jianmu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

other: merge 3.0

This commit is contained in:
Haojun Liao 2022-11-07 14:31:01 +08:00
parent 2a49fed4ad b8dd38430f
commit fea40da5dc
37 changed files with 1619 additions and 876 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
cmake/taosws_CMakeLists.txt.in
View File

@ -2,7 +2,7 @@
# taosws-rs # taosws-rs
ExternalProject_Add(taosws-rs ExternalProject_Add(taosws-rs
GIT_REPOSITORY https://github.com/taosdata/taos-connector-rust.git GIT_REPOSITORY https://github.com/taosdata/taos-connector-rust.git
GIT_TAG 0373a70 GIT_TAG 38c4599
SOURCE_DIR "${TD_SOURCE_DIR}/tools/taosws-rs" SOURCE_DIR "${TD_SOURCE_DIR}/tools/taosws-rs"
BINARY_DIR "" BINARY_DIR ""
#BUILD_IN_SOURCE TRUE #BUILD_IN_SOURCE TRUE

4
docs/en/07-develop/03-insert-data/02-influxdb-line.mdx
View File

@ -37,9 +37,9 @@ meters,location=California.LosAngeles,groupid=2 current=13.4,voltage=223,phase=0
- All the data in `tag_set` will be converted to NCHAR type automatically . - All the data in `tag_set` will be converted to NCHAR type automatically .
- Each data in `field_set` must be self-descriptive for its data type. For example 1.2f32 means a value 1.2 of float type. Without the "f" type suffix, it will be treated as type double. - Each data in `field_set` must be self-descriptive for its data type. For example 1.2f32 means a value 1.2 of float type. Without the "f" type suffix, it will be treated as type double.
- Multiple kinds of precision can be used for the `timestamp` field. Time precision can be from nanosecond (ns) to hour (h). - Multiple kinds of precision can be used for the `timestamp` field. Time precision can be from nanosecond (ns) to hour (h).
- You can configure smlChildTableName in taos.cfg to specify table names, for example, `smlChildTableName=tname`. You can insert `st,tname=cpul,t1=4 c1=3 1626006833639000000` and the cpu1 table will be automatically created. Note that if multiple rows have the same tname but different tag_set values, the tag_set of the first row is used to create the table and the others are ignored. - The child table name is created automatically in a rule to guarantee its uniqueness. But you can configure `smlChildTableName` in taos.cfg to specify a tag value as the table names if the tag value is unique globally. For example, if a tag is called `tname` and you set `smlChildTableName=tname` in taos.cfg, when you insert `st,tname=cpu1,t1=4 c1=3 1626006833639000000`, the child table `cpu1` will be created automatically. Note that if multiple rows have the same tname but different tag_set values, the tag_set of the first row is used to create the table and the others are ignored.
- It is assumed that the order of field_set in a supertable is consistent, meaning that the first record contains all fields and subsequent records store fields in the same order. If the order is not consistent, set smlDataFormat in taos.cfg to false. Otherwise, data will be written out of order and a database error will occur.(smlDataFormat in taos.cfg default to false after version of 3.0.1.3) - It is assumed that the order of field_set in a supertable is consistent, meaning that the first record contains all fields and subsequent records store fields in the same order. If the order is not consistent, set smlDataFormat in taos.cfg to false. Otherwise, data will be written out of order and a database error will occur.(smlDataFormat in taos.cfg default to false after version of 3.0.1.3)
::: :::
For more details please refer to [InfluxDB Line Protocol](https://docs.influxdata.com/influxdb/v2.0/reference/syntax/line-protocol/) and [TDengine Schemaless](/reference/schemaless/#Schemaless-Line-Protocol) For more details please refer to [InfluxDB Line Protocol](https://docs.influxdata.com/influxdb/v2.0/reference/syntax/line-protocol/) and [TDengine Schemaless](/reference/schemaless/#Schemaless-Line-Protocol)

2
docs/en/07-develop/03-insert-data/03-opentsdb-telnet.mdx
View File

@ -32,7 +32,7 @@ For example:
meters.current 1648432611250 11.3 location=California.LosAngeles groupid=3 meters.current 1648432611250 11.3 location=California.LosAngeles groupid=3
``` ```
- The defult child table name is generated by rules.You can configure smlChildTableName in taos.cfg to specify child table names, for example, `smlChildTableName=tname`. You can insert `meters.current 1648432611250 11.3 tname=cpu1 location=California.LosAngeles groupid=3` and the cpu1 table will be automatically created. Note that if multiple rows have the same tname but different tag_set values, the tag_set of the first row is used to create the table and the others are ignored. - The child table name is created automatically in a rule to guarantee its uniqueness. But you can configure `smlChildTableName` in taos.cfg to specify a tag value as the table names if the tag value is unique globally. For example, if a tag is called `tname` and you set `smlChildTableName=tname` in taos.cfg, when you insert `st,tname=cpu1,t1=4 c1=3 1626006833639000000`, the child table `cpu1` will be automatically created. Note that if multiple rows have the same tname but different tag_set values, the tag_set of the first row is used to create the table and the others are ignored.
Please refer to [OpenTSDB Telnet API](http://opentsdb.net/docs/build/html/api_telnet/put.html) for more details. Please refer to [OpenTSDB Telnet API](http://opentsdb.net/docs/build/html/api_telnet/put.html) for more details.
## Examples ## Examples

4
docs/en/07-develop/03-insert-data/04-opentsdb-json.mdx
View File

@ -48,8 +48,8 @@ Please refer to [OpenTSDB HTTP API](http://opentsdb.net/docs/build/html/api_http
- In JSON protocol, strings will be converted to NCHAR type and numeric values will be converted to double type. - In JSON protocol, strings will be converted to NCHAR type and numeric values will be converted to double type.
- Only data in array format is accepted and so an array must be used even if there is only one row. - Only data in array format is accepted and so an array must be used even if there is only one row.
- The defult child table name is generated by rules.You can configure smlChildTableName in taos.cfg to specify child table names, for example, `smlChildTableName=tname`. You can insert `"tags": { "host": "web02","dc": "lga","tname":"cpu1"}` and the cpu1 table will be automatically created. Note that if multiple rows have the same tname but different tag_set values, the tag_set of the first row is used to create the table and the others are ignored. - The child table name is created automatically in a rule to guarantee its uniqueness. But you can configure `smlChildTableName` in taos.cfg to specify a tag value as the table names if the tag value is unique globally. For example, if a tag is called `tname` and you set `smlChildTableName=tname` in taos.cfg, when you insert `st,tname=cpu1,t1=4 c1=3 1626006833639000000`, the child table `cpu1` will be automatically created. Note that if multiple rows have the same tname but different tag_set values, the tag_set of the first row is used to create the table and the others are ignored.
::: :::
## Examples ## Examples

5
docs/zh/07-develop/03-insert-data/02-influxdb-line.mdx
View File

@ -38,9 +38,8 @@ meters,location=California.LosAngeles,groupid=2 current=13.4,voltage=223,phase=0
- field_set 中的每个数据项都需要对自身的数据类型进行描述, 比如 1.2f32 代表 FLOAT 类型的数值 1.2, 如果不带类型后缀会被当作 DOUBLE 处理; - field_set 中的每个数据项都需要对自身的数据类型进行描述, 比如 1.2f32 代表 FLOAT 类型的数值 1.2, 如果不带类型后缀会被当作 DOUBLE 处理;
- timestamp 支持多种时间精度。写入数据的时候需要用参数指定时间精度，支持从小时到纳秒的 6 种时间精度。 - timestamp 支持多种时间精度。写入数据的时候需要用参数指定时间精度，支持从小时到纳秒的 6 种时间精度。
- 为了提高写入的效率，默认假设同一个超级表中 field_set 的顺序是一样的（第一条数据包含所有的 field，后面的数据按照这个顺序），如果顺序不一样，需要配置参数 smlDataFormat 为 false，否则，数据写入按照相同顺序写入，库中数据会异常。（3.0.1.3 之后的版本 smlDataFormat 默认为 false） [TDengine 无模式写入参考指南](/reference/schemaless/#无模式写入行协议) - 为了提高写入的效率，默认假设同一个超级表中 field_set 的顺序是一样的（第一条数据包含所有的 field，后面的数据按照这个顺序），如果顺序不一样，需要配置参数 smlDataFormat 为 false，否则，数据写入按照相同顺序写入，库中数据会异常。（3.0.1.3 之后的版本 smlDataFormat 默认为 false） [TDengine 无模式写入参考指南](/reference/schemaless/#无模式写入行协议)
- 默认生产的子表名是根据规则生成的唯一 ID 值。为了让用户可以指定生成的表名，可以通过在 taos.cfg 里配置 smlChildTableName 参数来指定。 - 默认产生的子表名是根据规则生成的唯一 ID 值。用户也可以通过在 taos.cfg 里配置 smlChildTableName 参数来指定某个标签值作为子表名。该标签值应该具有全局唯一性。举例如下：假设有个标签名为tname, 配置 smlChildTableName=tname, 插入数据为 st,tname=cpu1,t1=4 c1=3 1626006833639000000 则创建的子表名为 cpu1。注意如果多行数据 tname 相同，但是后面的 tag_set 不同，则使用第一行自动建表时指定的 tag_set，其他的行会忽略）。[TDengine 无模式写入参考指南](/reference/schemaless/#无模式写入行协议)
举例如下：配置 smlChildTableName=tname 插入数据为 st,tname=cpu1,t1=4 c1=3 1626006833639000000 则创建的表名为 cpu1，注意如果多行数据 tname 相同，但是后面的 tag_set 不同，则使用第一行自动建表时指定的 tag_set，其他的行会忽略）。[TDengine 无模式写入参考指南](/reference/schemaless/#无模式写入行协议) :::
:::
要了解更多可参考：[InfluxDB Line 协议官方文档](https://docs.influxdata.com/influxdb/v2.0/reference/syntax/line-protocol/) 和 [TDengine 无模式写入参考指南](/reference/schemaless/#无模式写入行协议) 要了解更多可参考：[InfluxDB Line 协议官方文档](https://docs.influxdata.com/influxdb/v2.0/reference/syntax/line-protocol/) 和 [TDengine 无模式写入参考指南](/reference/schemaless/#无模式写入行协议)

3
docs/zh/07-develop/03-insert-data/03-opentsdb-telnet.mdx
View File

@ -32,8 +32,7 @@ OpenTSDB 行协议同样采用一行字符串来表示一行数据。OpenTSDB
meters.current 1648432611250 11.3 location=California.LosAngeles groupid=3 meters.current 1648432611250 11.3 location=California.LosAngeles groupid=3
``` ```
- 默认生产的子表名是根据规则生成的唯一 ID 值。为了让用户可以指定生成的表名，可以通过在 taos.cfg 里配置 smlChildTableName 参数来指定。 - 默认生产的子表名是根据规则生成的唯一 ID 值。用户也可以通过在 taos.cfg 里配置 smlChildTableName 参数来指定某个标签值作为子表名。该标签值应该具有全局唯一性。举例如下：假设有个标签名为tname, 配置 smlChildTableName=tname, 插入数据为 meters.current 1648432611250 11.3 tname=cpu1 location=California.LosAngeles groupid=3 则创建的表名为 cpu1，注意如果多行数据 tname 相同，但是后面的 tag_set 不同，则使用第一行自动建表时指定的 tag_set，其他的行会忽略）。
举例如下：配置 smlChildTableName=tname 插入数据为 meters.current 1648432611250 11.3 tname=cpu1 location=California.LosAngeles groupid=3 则创建的表名为 cpu1，注意如果多行数据 tname 相同，但是后面的 tag_set 不同，则使用第一行自动建表时指定的 tag_set，其他的行会忽略）。
参考 [OpenTSDB Telnet API 文档](http://opentsdb.net/docs/build/html/api_telnet/put.html)。 参考 [OpenTSDB Telnet API 文档](http://opentsdb.net/docs/build/html/api_telnet/put.html)。
## 示例代码 ## 示例代码

5
docs/zh/07-develop/03-insert-data/04-opentsdb-json.mdx
View File

@ -48,9 +48,8 @@ OpenTSDB JSON 格式协议采用一个 JSON 字符串表示一行或多行数据
- 对于 JSON 格式协议，TDengine 并不会自动把所有标签转成 NCHAR 类型， 字符串将将转为 NCHAR 类型， 数值将同样转换为 DOUBLE 类型。 - 对于 JSON 格式协议，TDengine 并不会自动把所有标签转成 NCHAR 类型， 字符串将将转为 NCHAR 类型， 数值将同样转换为 DOUBLE 类型。
- TDengine 只接收 JSON **数组格式**的字符串，即使一行数据也需要转换成数组形式。 - TDengine 只接收 JSON **数组格式**的字符串，即使一行数据也需要转换成数组形式。
- 默认生产的子表名是根据规则生成的唯一 ID 值。为了让用户可以指定生成的表名，可以通过在 taos.cfg 里配置 smlChildTableName 参数来指定。 - 默认生成的子表名是根据规则生成的唯一 ID 值。用户也可以通过在 taos.cfg 里配置 smlChildTableName 参数来指定某个标签值作为子表名。该标签值应该具有全局唯一性。举例如下：假设有个标签名为tname, 配置 smlChildTableName=tname, 插入数据为 `"tags": { "host": "web02","dc": "lga","tname":"cpu1"}` 则创建的子表名为 cpu1。注意如果多行数据 tname 相同，但是后面的 tag_set 不同，则使用第一行自动建表时指定的 tag_set，其他的行会忽略）。
举例如下：配置 smlChildTableName=tname 插入数据为 `"tags": { "host": "web02","dc": "lga","tname":"cpu1"}` 则创建的表名为 cpu1，注意如果多行数据 tname 相同，但是后面的 tag_set 不同，则使用第一行自动建表时指定的 tag_set，其他的行会忽略）。 :::
:::
## 示例代码 ## 示例代码

19
examples/c/CMakeLists.txt
View File

@ -15,6 +15,7 @@ IF (TD_LINUX)
add_executable(tmq "") add_executable(tmq "")
add_executable(stream_demo "") add_executable(stream_demo "")
add_executable(demoapi "") add_executable(demoapi "")
add_executable(api_reqid "")
target_sources(tmq target_sources(tmq
PRIVATE PRIVATE
@ -31,6 +32,12 @@ IF (TD_LINUX)
"demoapi.c" "demoapi.c"
) )
target_sources(api_reqid
PRIVATE
"api_with_reqid_test.c"
)
target_link_libraries(tmq target_link_libraries(tmq
taos_static taos_static
) )
@ -43,6 +50,11 @@ IF (TD_LINUX)
taos_static taos_static
) )
target_link_libraries(api_reqid
taos_static
)
target_include_directories(tmq target_include_directories(tmq
PUBLIC "${TD_SOURCE_DIR}/include/os" PUBLIC "${TD_SOURCE_DIR}/include/os"
PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc" PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
@ -58,9 +70,16 @@ IF (TD_LINUX)
PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc" PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
) )
target_include_directories(api_reqid
PUBLIC "${TD_SOURCE_DIR}/include/client"
PUBLIC "${TD_SOURCE_DIR}/include/os"
PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
)
SET_TARGET_PROPERTIES(tmq PROPERTIES OUTPUT_NAME tmq) SET_TARGET_PROPERTIES(tmq PROPERTIES OUTPUT_NAME tmq)
SET_TARGET_PROPERTIES(stream_demo PROPERTIES OUTPUT_NAME stream_demo) SET_TARGET_PROPERTIES(stream_demo PROPERTIES OUTPUT_NAME stream_demo)
SET_TARGET_PROPERTIES(demoapi PROPERTIES OUTPUT_NAME demoapi) SET_TARGET_PROPERTIES(demoapi PROPERTIES OUTPUT_NAME demoapi)
SET_TARGET_PROPERTIES(api_reqid PROPERTIES OUTPUT_NAME api_reqid)
ENDIF () ENDIF ()
IF (TD_DARWIN) IF (TD_DARWIN)
INCLUDE_DIRECTORIES(. ${TD_SOURCE_DIR}/src/inc ${TD_SOURCE_DIR}/src/client/inc ${TD_SOURCE_DIR}/inc) INCLUDE_DIRECTORIES(. ${TD_SOURCE_DIR}/src/inc ${TD_SOURCE_DIR}/src/client/inc ${TD_SOURCE_DIR}/inc)

449
examples/c/api_with_reqid_test.c Normal file
View File

@ -0,0 +1,449 @@
// sample code to verify all TDengine API
// to compile: gcc -o apitest apitest.c -ltaos
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include "taos.h"
static int64_t count = 10000;
int64_t genReqid() {
count += 100;
return count;
}
static void prepare_data(TAOS* taos) {
TAOS_RES* result;
result = taos_query_with_reqid(taos, "drop database if exists test;", genReqid());
taos_free_result(result);
usleep(100000);
result = taos_query_with_reqid(taos, "create database test precision 'us';", genReqid());
taos_free_result(result);
usleep(100000);
taos_select_db(taos, "test");
result = taos_query_with_reqid(taos, "create table meters(ts timestamp, a int) tags(area int);", genReqid());
taos_free_result(result);
result = taos_query_with_reqid(taos, "create table t0 using meters tags(0);", genReqid());
taos_free_result(result);
result = taos_query_with_reqid(taos, "create table t1 using meters tags(1);", genReqid());
taos_free_result(result);
result = taos_query_with_reqid(taos, "create table t2 using meters tags(2);", genReqid());
taos_free_result(result);
result = taos_query_with_reqid(taos, "create table t3 using meters tags(3);", genReqid());
taos_free_result(result);
result = taos_query_with_reqid(taos, "create table t4 using meters tags(4);", genReqid());
taos_free_result(result);
result = taos_query_with_reqid(taos, "create table t5 using meters tags(5);", genReqid());
taos_free_result(result);
result = taos_query_with_reqid(taos, "create table t6 using meters tags(6);", genReqid());
taos_free_result(result);
result = taos_query_with_reqid(taos, "create table t7 using meters tags(7);", genReqid());
taos_free_result(result);
result = taos_query_with_reqid(taos, "create table t8 using meters tags(8);", genReqid());
taos_free_result(result);
result = taos_query(taos, "create table t9 using meters tags(9);");
taos_free_result(result);
result = taos_query_with_reqid(taos,
"insert into t0 values('2020-01-01 00:00:00.000', 0)"
" ('2020-01-01 00:01:00.000', 0)"
" ('2020-01-01 00:02:00.000', 0)"
" t1 values('2020-01-01 00:00:00.000', 0)"
" ('2020-01-01 00:01:00.000', 0)"
" ('2020-01-01 00:02:00.000', 0)"
" ('2020-01-01 00:03:00.000', 0)"
" t2 values('2020-01-01 00:00:00.000', 0)"
" ('2020-01-01 00:01:00.000', 0)"
" ('2020-01-01 00:01:01.000', 0)"
" ('2020-01-01 00:01:02.000', 0)"
" t3 values('2020-01-01 00:01:02.000', 0)"
" t4 values('2020-01-01 00:01:02.000', 0)"
" t5 values('2020-01-01 00:01:02.000', 0)"
" t6 values('2020-01-01 00:01:02.000', 0)"
" t7 values('2020-01-01 00:01:02.000', 0)"
" t8 values('2020-01-01 00:01:02.000', 0)"
" t9 values('2020-01-01 00:01:02.000', 0)",
genReqid());
int affected = taos_affected_rows(result);
if (affected != 18) {
printf("\033[31m%d rows affected by last insert statement, but it should be 18\033[0m\n", affected);
}
taos_free_result(result);
// super tables subscription
usleep(1000000);
}
static int print_result(TAOS_RES* res, int blockFetch) {
TAOS_ROW row = NULL;
int num_fields = taos_num_fields(res);
TAOS_FIELD* fields = taos_fetch_fields(res);
int nRows = 0;
if (blockFetch) {
int rows = 0;
while ((rows = taos_fetch_block(res, &row))) {
// for (int i = 0; i < rows; i++) {
// char temp[256];
// taos_print_row(temp, row + i, fields, num_fields);
// puts(temp);
// }
nRows += rows;
}
} else {
while ((row = taos_fetch_row(res))) {
char temp[256] = {0};
taos_print_row(temp, row, fields, num_fields);
puts(temp);
nRows++;
}
}
printf("%d rows consumed.\n", nRows);
return nRows;
}
static void check_row_count(int line, TAOS_RES* res, int expected) {
int actual = print_result(res, expected % 2);
if (actual != expected) {
printf("\033[31mline %d: row count mismatch, expected: %d, actual: %d\033[0m\n", line, expected, actual);
} else {
printf("line %d: %d rows consumed as expected\n", line, actual);
}
}
static void verify_query(TAOS* taos) {
prepare_data(taos);
int code = taos_load_table_info(taos, "t0,t1,t2,t3,t4,t5,t6,t7,t8,t9");
if (code != 0) {
printf("\033[31mfailed to load table info: 0x%08x\033[0m\n", code);
}
code = taos_validate_sql(taos, "select * from nonexisttable");
if (code == 0) {
printf("\033[31mimpossible, the table does not exists\033[0m\n");
}
code = taos_validate_sql(taos, "select * from meters");
if (code != 0) {
printf("\033[31mimpossible, the table does exists: 0x%08x\033[0m\n", code);
}
TAOS_RES* res = taos_query_with_reqid(taos, "select * from meters", genReqid());
check_row_count(__LINE__, res, 18);
printf("result precision is: %d\n", taos_result_precision(res));
int c = taos_field_count(res);
printf("field count is: %d\n", c);
int* lengths = taos_fetch_lengths(res);
for (int i = 0; i < c; i++) {
printf("length of column %d is %d\n", i, lengths[i]);
}
taos_free_result(res);
res = taos_query_with_reqid(taos, "select * from t0", genReqid());
check_row_count(__LINE__, res, 3);
taos_free_result(res);
res = taos_query_with_reqid(taos, "select * from nonexisttable", genReqid());
code = taos_errno(res);
printf("code=%d, error msg=%s\n", code, taos_errstr(res));
taos_free_result(res);
res = taos_query_with_reqid(taos, "select * from meters", genReqid());
taos_stop_query(res);
taos_free_result(res);
}
void retrieve_callback(void* param, TAOS_RES* tres, int numOfRows) {
if (numOfRows > 0) {
printf("%d rows async retrieved\n", numOfRows);
taos_fetch_rows_a(tres, retrieve_callback, param);
} else {
if (numOfRows < 0) {
printf("\033[31masync retrieve failed, code: %d\033[0m\n", numOfRows);
} else {
printf("async retrieve completed\n");
}
taos_free_result(tres);
}
}
void select_callback(void* param, TAOS_RES* tres, int code) {
if (code == 0 && tres) {
taos_fetch_rows_a(tres, retrieve_callback, param);
} else {
printf("\033[31masync select failed, code: %d\033[0m\n", code);
}
}
void verify_async(TAOS* taos) {
prepare_data(taos);
taos_query_a_with_reqid(taos, "select * from meters", select_callback, NULL, genReqid());
usleep(1000000);
}
int32_t verify_schema_less(TAOS* taos) {
TAOS_RES* result;
result = taos_query_with_reqid(taos, "drop database if exists test;", genReqid());
taos_free_result(result);
usleep(100000);
result = taos_query_with_reqid(taos, "create database test precision 'us' update 1;", genReqid());
taos_free_result(result);
usleep(100000);
taos_select_db(taos, "test");
result = taos_query_with_reqid(taos, "create stable ste(ts timestamp, f int) tags(t1 bigint)", genReqid());
taos_free_result(result);
usleep(100000);
int code = 0;
char* lines[] = {
"st,t1=3i64,t2=4f64,t3=\"t3\" c1=3i64,c3=L\"passit\",c2=false,c4=4f64 1626006833639000000ns",
"st,t1=4i64,t3=\"t4\",t2=5f64,t4=5f64 c1=3i64,c3=L\"passitagin\",c2=true,c4=5f64,c5=5f64 1626006833640000000ns",
"ste,t2=5f64,t3=L\"ste\" c1=true,c2=4i64,c3=\"iam\" 1626056811823316532ns",
"st,t1=4i64,t2=5f64,t3=\"t4\" c1=3i64,c3=L\"passitagain\",c2=true,c4=5f64 1626006833642000000ns",
"ste,t2=5f64,t3=L\"ste2\" c3=\"iamszhou\",c4=false 1626056811843316532ns",
"ste,t2=5f64,t3=L\"ste2\" c3=\"iamszhou\",c4=false,c5=32i8,c6=64i16,c7=32i32,c8=88.88f32 1626056812843316532ns",
"st,t1=4i64,t3=\"t4\",t2=5f64,t4=5f64 c1=3i64,c3=L\"passitagin\",c2=true,c4=5f64,c5=5f64,c6=7u64 "
"1626006933640000000ns",
"stf,t1=4i64,t3=\"t4\",t2=5f64,t4=5f64 c1=3i64,c3=L\"passitagin\",c2=true,c4=5f64,c5=5f64,c6=7u64 "
"1626006933640000000ns",
"stf,t1=4i64,t3=\"t4\",t2=5f64,t4=5f64 c1=3i64,c3=L\"passitagin_stf\",c2=false,c5=5f64,c6=7u64 "
"1626006933641000000ns"};
taos_select_db(taos, "test");
TAOS_RES* res = taos_schemaless_insert_with_reqid(taos, lines, sizeof(lines) / sizeof(char*), TSDB_SML_LINE_PROTOCOL,
TSDB_SML_TIMESTAMP_NOT_CONFIGURED, genReqid());
if (taos_errno(res) != 0) {
printf("failed to insert schema-less data, reason: %s\n", taos_errstr(res));
} else {
int affectedRow = taos_affected_rows(res);
printf("successfully inserted %d rows\n", affectedRow);
}
taos_free_result(res);
return (code);
}
void veriry_stmt(TAOS* taos) {
TAOS_RES* result = taos_query(taos, "drop database if exists test;");
taos_free_result(result);
usleep(100000);
result = taos_query(taos, "create database test;");
int code = taos_errno(result);
if (code != 0) {
printf("\033[31mfailed to create database, reason:%s\033[0m\n", taos_errstr(result));
taos_free_result(result);
return;
}
taos_free_result(result);
usleep(100000);
taos_select_db(taos, "test");
// create table
const char* sql =
"create table m1 (ts timestamp, b bool, v1 tinyint, v2 smallint, v4 int, v8 bigint, f4 float, f8 double, bin "
"binary(40), blob nchar(10))";
result = taos_query(taos, sql);
code = taos_errno(result);
if (code != 0) {
printf("\033[31mfailed to create table, reason:%s\033[0m\n", taos_errstr(result));
taos_free_result(result);
return;
}
taos_free_result(result);
// insert 10 records
struct {
int64_t ts[10];
int8_t b[10];
int8_t v1[10];
int16_t v2[10];
int32_t v4[10];
int64_t v8[10];
float f4[10];
double f8[10];
char bin[10][40];
char blob[10][80];
} v;
int32_t* t8_len = malloc(sizeof(int32_t) * 10);
int32_t* t16_len = malloc(sizeof(int32_t) * 10);
int32_t* t32_len = malloc(sizeof(int32_t) * 10);
int32_t* t64_len = malloc(sizeof(int32_t) * 10);
int32_t* float_len = malloc(sizeof(int32_t) * 10);
int32_t* double_len = malloc(sizeof(int32_t) * 10);
int32_t* bin_len = malloc(sizeof(int32_t) * 10);
int32_t* blob_len = malloc(sizeof(int32_t) * 10);
TAOS_STMT* stmt = taos_stmt_init_with_reqid(taos, genReqid());
TAOS_MULTI_BIND params[10];
char is_null[10] = {0};
params[0].buffer_type = TSDB_DATA_TYPE_TIMESTAMP;
params[0].buffer_length = sizeof(v.ts[0]);
params[0].buffer = v.ts;
params[0].length = t64_len;
params[0].is_null = is_null;
params[0].num = 10;
params[1].buffer_type = TSDB_DATA_TYPE_BOOL;
params[1].buffer_length = sizeof(v.b[0]);
params[1].buffer = v.b;
params[1].length = t8_len;
params[1].is_null = is_null;
params[1].num = 10;
params[2].buffer_type = TSDB_DATA_TYPE_TINYINT;
params[2].buffer_length = sizeof(v.v1[0]);
params[2].buffer = v.v1;
params[2].length = t8_len;
params[2].is_null = is_null;
params[2].num = 10;
params[3].buffer_type = TSDB_DATA_TYPE_SMALLINT;
params[3].buffer_length = sizeof(v.v2[0]);
params[3].buffer = v.v2;
params[3].length = t16_len;
params[3].is_null = is_null;
params[3].num = 10;
params[4].buffer_type = TSDB_DATA_TYPE_INT;
params[4].buffer_length = sizeof(v.v4[0]);
params[4].buffer = v.v4;
params[4].length = t32_len;
params[4].is_null = is_null;
params[4].num = 10;
params[5].buffer_type = TSDB_DATA_TYPE_BIGINT;
params[5].buffer_length = sizeof(v.v8[0]);
params[5].buffer = v.v8;
params[5].length = t64_len;
params[5].is_null = is_null;
params[5].num = 10;
params[6].buffer_type = TSDB_DATA_TYPE_FLOAT;
params[6].buffer_length = sizeof(v.f4[0]);
params[6].buffer = v.f4;
params[6].length = float_len;
params[6].is_null = is_null;
params[6].num = 10;
params[7].buffer_type = TSDB_DATA_TYPE_DOUBLE;
params[7].buffer_length = sizeof(v.f8[0]);
params[7].buffer = v.f8;
params[7].length = double_len;
params[7].is_null = is_null;
params[7].num = 10;
params[8].buffer_type = TSDB_DATA_TYPE_BINARY;
params[8].buffer_length = sizeof(v.bin[0]);
params[8].buffer = v.bin;
params[8].length = bin_len;
params[8].is_null = is_null;
params[8].num = 10;
params[9].buffer_type = TSDB_DATA_TYPE_NCHAR;
params[9].buffer_length = sizeof(v.blob[0]);
params[9].buffer = v.blob;
params[9].length = blob_len;
params[9].is_null = is_null;
params[9].num = 10;
sql = "insert into ? (ts, b, v1, v2, v4, v8, f4, f8, bin, blob) values(?,?,?,?,?,?,?,?,?,?)";
code = taos_stmt_prepare(stmt, sql, 0);
if (code != 0) {
printf("\033[31mfailed to execute taos_stmt_prepare. error:%s\033[0m\n", taos_stmt_errstr(stmt));
taos_stmt_close(stmt);
return;
}
code = taos_stmt_set_tbname(stmt, "m1");
if (code != 0) {
printf("\033[31mfailed to execute taos_stmt_prepare. error:%s\033[0m\n", taos_stmt_errstr(stmt));
taos_stmt_close(stmt);
return;
}
int64_t ts = 1591060628000;
for (int i = 0; i < 10; ++i) {
v.ts[i] = ts++;
is_null[i] = 0;
v.b[i] = (int8_t)i % 2;
v.v1[i] = (int8_t)i;
v.v2[i] = (int16_t)(i * 2);
v.v4[i] = (int32_t)(i * 4);
v.v8[i] = (int64_t)(i * 8);
v.f4[i] = (float)(i * 40);
v.f8[i] = (double)(i * 80);
for (int j = 0; j < sizeof(v.bin[0]); ++j) {
v.bin[i][j] = (char)(i + '0');
}
strcpy(v.blob[i], "一二三四五六七八九十");
t8_len[i] = sizeof(int8_t);
t16_len[i] = sizeof(int16_t);
t32_len[i] = sizeof(int32_t);
t64_len[i] = sizeof(int64_t);
float_len[i] = sizeof(float);
double_len[i] = sizeof(double);
bin_len[i] = sizeof(v.bin[0]);
blob_len[i] = (int32_t)strlen(v.blob[i]);
}
taos_stmt_bind_param_batch(stmt, params);
taos_stmt_add_batch(stmt);
if (taos_stmt_execute(stmt) != 0) {
printf("\033[31mfailed to execute insert statement.error:%s\033[0m\n", taos_stmt_errstr(stmt));
taos_stmt_close(stmt);
return;
}
taos_stmt_close(stmt);
free(t8_len);
free(t16_len);
free(t32_len);
free(t64_len);
free(float_len);
free(double_len);
free(bin_len);
free(blob_len);
}
int main(int argc, char* argv[]) {
const char* host = "127.0.0.1";
const char* user = "root";
const char* passwd = "taosdata";
taos_options(TSDB_OPTION_TIMEZONE, "GMT-8");
TAOS* taos = taos_connect(host, user, passwd, "", 0);
if (taos == NULL) {
printf("\033[31mfailed to connect to db, reason:%s\033[0m\n", taos_errstr(taos));
exit(1);
}
printf("************ verify schema-less *************\n");
verify_schema_less(taos);
printf("************ verify query *************\n");
verify_query(taos);
printf("********* verify async query **********\n");
verify_async(taos);
printf("********* verify stmt query **********\n");
veriry_stmt(taos);
printf("done\n");
taos_close(taos);
taos_cleanup();
}

551
examples/c/apitest.c
View File

File diff suppressed because it is too large Load Diff

4
examples/go/taosdemo.go
View File

@ -26,7 +26,7 @@ import (
"sync" "sync"
"time" "time"
_ "github.com/taosdata/driver-go/taosSql" _ "github.com/taosdata/driver-go/v3/taosSql"
) )
const ( const (
@ -173,7 +173,7 @@ func createDatabase(dbName string, supTblName string) {
time.Sleep(time.Second) time.Sleep(time.Second)
// create database // create database
sqlStr = "create database " + dbName + " keep " + strconv.Itoa(configPara.keep) + " days " + strconv.Itoa(configPara.days) sqlStr = "create database " + dbName
_, err = db.Exec(sqlStr) _, err = db.Exec(sqlStr)
checkErr(err, sqlStr) checkErr(err, sqlStr)

35
include/client/taos.h
View File

@ -130,17 +130,16 @@ typedef struct TAOS_VGROUP_HASH_INFO {
int32_t vgId; int32_t vgId;
uint32_t hashBegin; uint32_t hashBegin;
uint32_t hashEnd; uint32_t hashEnd;
} TAOS_VGROUP_HASH_INFO ; } TAOS_VGROUP_HASH_INFO;
typedef struct TAOS_DB_ROUTE_INFO { typedef struct TAOS_DB_ROUTE_INFO {
int32_t routeVersion; int32_t routeVersion;
int16_t hashPrefix; int16_t hashPrefix;
int16_t hashSuffix; int16_t hashSuffix;
int8_t hashMethod; int8_t hashMethod;
int32_t vgNum; int32_t vgNum;
TAOS_VGROUP_HASH_INFO *vgHash; TAOS_VGROUP_HASH_INFO *vgHash;
} TAOS_DB_ROUTE_INFO ; } TAOS_DB_ROUTE_INFO;
DLL_EXPORT void taos_cleanup(void); DLL_EXPORT void taos_cleanup(void);
DLL_EXPORT int taos_options(TSDB_OPTION option, const void *arg, ...); DLL_EXPORT int taos_options(TSDB_OPTION option, const void *arg, ...);
@ -153,6 +152,7 @@ DLL_EXPORT void taos_close(TAOS *taos);
const char *taos_data_type(int type); const char *taos_data_type(int type);
DLL_EXPORT TAOS_STMT *taos_stmt_init(TAOS *taos); DLL_EXPORT TAOS_STMT *taos_stmt_init(TAOS *taos);
DLL_EXPORT TAOS_STMT *taos_stmt_init_with_reqid(TAOS *taos, int64_t reqid);
DLL_EXPORT int taos_stmt_prepare(TAOS_STMT *stmt, const char *sql, unsigned long length); DLL_EXPORT int taos_stmt_prepare(TAOS_STMT *stmt, const char *sql, unsigned long length);
DLL_EXPORT int taos_stmt_set_tbname_tags(TAOS_STMT *stmt, const char *name, TAOS_MULTI_BIND *tags); DLL_EXPORT int taos_stmt_set_tbname_tags(TAOS_STMT *stmt, const char *name, TAOS_MULTI_BIND *tags);
DLL_EXPORT int taos_stmt_set_tbname(TAOS_STMT *stmt, const char *name); DLL_EXPORT int taos_stmt_set_tbname(TAOS_STMT *stmt, const char *name);
@ -176,6 +176,7 @@ DLL_EXPORT int taos_stmt_affected_rows(TAOS_STMT *stmt);
DLL_EXPORT int taos_stmt_affected_rows_once(TAOS_STMT *stmt); DLL_EXPORT int taos_stmt_affected_rows_once(TAOS_STMT *stmt);
DLL_EXPORT TAOS_RES *taos_query(TAOS *taos, const char *sql); DLL_EXPORT TAOS_RES *taos_query(TAOS *taos, const char *sql);
DLL_EXPORT TAOS_RES *taos_query_with_reqid(TAOS *taos, const char *sql, int64_t reqId);
DLL_EXPORT TAOS_ROW taos_fetch_row(TAOS_RES *res); DLL_EXPORT TAOS_ROW taos_fetch_row(TAOS_RES *res);
DLL_EXPORT int taos_result_precision(TAOS_RES *res); // get the time precision of result DLL_EXPORT int taos_result_precision(TAOS_RES *res); // get the time precision of result
@ -207,17 +208,23 @@ DLL_EXPORT const char *taos_get_client_info();
DLL_EXPORT const char *taos_errstr(TAOS_RES *res); DLL_EXPORT const char *taos_errstr(TAOS_RES *res);
DLL_EXPORT int taos_errno(TAOS_RES *res); DLL_EXPORT int taos_errno(TAOS_RES *res);
DLL_EXPORT void taos_query_a(TAOS *taos, const char *sql, __taos_async_fn_t fp, void *param); DLL_EXPORT void taos_query_a(TAOS *taos, const char *sql, __taos_async_fn_t fp, void *param);
DLL_EXPORT void taos_fetch_rows_a(TAOS_RES *res, __taos_async_fn_t fp, void *param); DLL_EXPORT void taos_query_a_with_reqid(TAOS *taos, const char *sql, __taos_async_fn_t fp, void *param, int64_t reqid);
DLL_EXPORT void taos_fetch_raw_block_a(TAOS_RES *res, __taos_async_fn_t fp, void *param); DLL_EXPORT void taos_fetch_rows_a(TAOS_RES *res, __taos_async_fn_t fp, void *param);
DLL_EXPORT void taos_fetch_raw_block_a(TAOS_RES *res, __taos_async_fn_t fp, void *param);
DLL_EXPORT const void *taos_get_raw_block(TAOS_RES *res); DLL_EXPORT const void *taos_get_raw_block(TAOS_RES *res);
DLL_EXPORT int taos_get_db_route_info(TAOS* taos, const char* db, TAOS_DB_ROUTE_INFO* dbInfo); DLL_EXPORT int taos_get_db_route_info(TAOS *taos, const char *db, TAOS_DB_ROUTE_INFO *dbInfo);
DLL_EXPORT int taos_get_table_vgId(TAOS* taos, const char* db, const char* table, int* vgId); DLL_EXPORT int taos_get_table_vgId(TAOS *taos, const char *db, const char *table, int *vgId);
DLL_EXPORT int taos_load_table_info(TAOS *taos, const char *tableNameList); DLL_EXPORT int taos_load_table_info(TAOS *taos, const char *tableNameList);
DLL_EXPORT TAOS_RES *taos_schemaless_insert(TAOS *taos, char *lines[], int numLines, int protocol, int precision); DLL_EXPORT TAOS_RES *taos_schemaless_insert(TAOS *taos, char *lines[], int numLines, int protocol, int precision);
DLL_EXPORT TAOS_RES *taos_schemaless_insert_raw(TAOS* taos, char* lines, int len, int32_t *totalRows, int protocol, int precision); DLL_EXPORT TAOS_RES *taos_schemaless_insert_with_reqid(TAOS *taos, char *lines[], int numLines, int protocol,
int precision, int64_t reqid);
DLL_EXPORT TAOS_RES *taos_schemaless_insert_raw(TAOS *taos, char *lines, int len, int32_t *totalRows, int protocol,
int precision);
DLL_EXPORT TAOS_RES *taos_schemaless_insert_raw_with_reqid(TAOS *taos, char *lines, int len, int32_t *totalRows,
int protocol, int precision, int64_t reqid);
/* --------------------------TMQ INTERFACE------------------------------- */ /* --------------------------TMQ INTERFACE------------------------------- */

12
source/client/inc/clientInt.h
View File

@ -271,7 +271,11 @@ int32_t transferTableNameList(const char* tbList, int32_t acctId, char* dbName,
void syncCatalogFn(SMetaData* pResult, void* param, int32_t code); void syncCatalogFn(SMetaData* pResult, void* param, int32_t code);
TAOS_RES* taosQueryImpl(TAOS* taos, const char* sql, bool validateOnly); TAOS_RES* taosQueryImpl(TAOS* taos, const char* sql, bool validateOnly);
void taosAsyncQueryImpl(uint64_t connId, const char* sql, __taos_async_fn_t fp, void* param, bool validateOnly); TAOS_RES* taosQueryImplWithReqid(TAOS* taos, const char* sql, bool validateOnly, int64_t reqid);
void taosAsyncQueryImpl(uint64_t connId, const char* sql, __taos_async_fn_t fp, void* param, bool validateOnly);
void taosAsyncQueryImplWithReqid(uint64_t connId, const char* sql, __taos_async_fn_t fp, void* param, bool validateOnly,
int64_t reqid);
int32_t getVersion1BlockMetaSize(const char* p, int32_t numOfCols); int32_t getVersion1BlockMetaSize(const char* p, int32_t numOfCols);
@ -318,11 +322,11 @@ void* createTscObj(const char* user, const char* auth, const char* db, int32_
void destroyTscObj(void* pObj); void destroyTscObj(void* pObj);
STscObj* acquireTscObj(int64_t rid); STscObj* acquireTscObj(int64_t rid);
int32_t releaseTscObj(int64_t rid); int32_t releaseTscObj(int64_t rid);
void destroyAppInst(SAppInstInfo *pAppInfo); void destroyAppInst(SAppInstInfo* pAppInfo);
uint64_t generateRequestId(); uint64_t generateRequestId();
void* createRequest(uint64_t connId, int32_t type); void* createRequest(uint64_t connId, int32_t type, int64_t reqid);
void destroyRequest(SRequestObj* pRequest); void destroyRequest(SRequestObj* pRequest);
SRequestObj* acquireRequest(int64_t rid); SRequestObj* acquireRequest(int64_t rid);
int32_t releaseRequest(int64_t rid); int32_t releaseRequest(int64_t rid);
@ -353,7 +357,7 @@ int32_t parseSql(SRequestObj* pRequest, bool topicQuery, SQuery** pQuery, SStmtC
int32_t getPlan(SRequestObj* pRequest, SQuery* pQuery, SQueryPlan** pPlan, SArray* pNodeList); int32_t getPlan(SRequestObj* pRequest, SQuery* pQuery, SQueryPlan** pPlan, SArray* pNodeList);
int32_t buildRequest(uint64_t connId, const char* sql, int sqlLen, void* param, bool validateSql, int32_t buildRequest(uint64_t connId, const char* sql, int sqlLen, void* param, bool validateSql,
SRequestObj** pRequest); SRequestObj** pRequest, int64_t reqid);
void taos_close_internal(void* taos); void taos_close_internal(void* taos);

11
source/client/inc/clientStmt.h
View File

@ -101,11 +101,18 @@ typedef struct STscStmt {
SStmtSQLInfo sql; SStmtSQLInfo sql;
SStmtExecInfo exec; SStmtExecInfo exec;
SStmtBindInfo bInfo; SStmtBindInfo bInfo;
int64_t reqid;
} STscStmt; } STscStmt;
extern char *gStmtStatusStr[]; extern char *gStmtStatusStr[];
#define STMT_LOG_SEQ(n) do { (pStmt)->seqId++; (pStmt)->seqIds[n]++; STMT_DLOG("the %dth:%d %s", (pStmt)->seqIds[n], (pStmt)->seqId, gStmtStatusStr[n]); } while (0) #define STMT_LOG_SEQ(n) \
do { \
(pStmt)->seqId++; \
(pStmt)->seqIds[n]++; \
STMT_DLOG("the %dth:%d %s", (pStmt)->seqIds[n], (pStmt)->seqId, gStmtStatusStr[n]); \
} while (0)
#define STMT_STATUS_NE(S) (pStmt->sql.status != STMT_##S) #define STMT_STATUS_NE(S) (pStmt->sql.status != STMT_##S)
#define STMT_STATUS_EQ(S) (pStmt->sql.status == STMT_##S) #define STMT_STATUS_EQ(S) (pStmt->sql.status == STMT_##S)
@ -141,7 +148,7 @@ extern char *gStmtStatusStr[];
#define STMT_ELOG_E(param) qError("stmt:%p " param, pStmt) #define STMT_ELOG_E(param) qError("stmt:%p " param, pStmt)
#define STMT_DLOG_E(param) qDebug("stmt:%p " param, pStmt) #define STMT_DLOG_E(param) qDebug("stmt:%p " param, pStmt)
TAOS_STMT *stmtInit(STscObj *taos); TAOS_STMT *stmtInit(STscObj *taos, int64_t reqid);
int stmtClose(TAOS_STMT *stmt); int stmtClose(TAOS_STMT *stmt);
int stmtExec(TAOS_STMT *stmt); int stmtExec(TAOS_STMT *stmt);
const char *stmtErrstr(TAOS_STMT *stmt); const char *stmtErrstr(TAOS_STMT *stmt);

26
source/client/src/clientEnv.c
View File

@ -77,19 +77,19 @@ static void deregisterRequest(SRequestObj *pRequest) {
pRequest->self, pTscObj->id, pRequest->requestId, duration / 1000.0, num, currentInst); pRequest->self, pTscObj->id, pRequest->requestId, duration / 1000.0, num, currentInst);
if (QUERY_NODE_VNODE_MODIF_STMT == pRequest->stmtType) { if (QUERY_NODE_VNODE_MODIF_STMT == pRequest->stmtType) {
// tscPerf("insert duration %" PRId64 "us: syntax:%" PRId64 "us, ctg:%" PRId64 "us, semantic:%" PRId64 // tscPerf("insert duration %" PRId64 "us: syntax:%" PRId64 "us, ctg:%" PRId64 "us, semantic:%" PRId64
// "us, exec:%" PRId64 "us", // "us, exec:%" PRId64 "us",
// duration, pRequest->metric.syntaxEnd - pRequest->metric.syntaxStart, // duration, pRequest->metric.syntaxEnd - pRequest->metric.syntaxStart,
// pRequest->metric.ctgEnd - pRequest->metric.ctgStart, pRequest->metric.semanticEnd - pRequest->metric.ctgEnd, // pRequest->metric.ctgEnd - pRequest->metric.ctgStart, pRequest->metric.semanticEnd -
// pRequest->metric.execEnd - pRequest->metric.semanticEnd); // pRequest->metric.ctgEnd, pRequest->metric.execEnd - pRequest->metric.semanticEnd);
atomic_add_fetch_64((int64_t *)&pActivity->insertElapsedTime, duration); atomic_add_fetch_64((int64_t *)&pActivity->insertElapsedTime, duration);
} else if (QUERY_NODE_SELECT_STMT == pRequest->stmtType) { } else if (QUERY_NODE_SELECT_STMT == pRequest->stmtType) {
// tscPerf("select duration %" PRId64 "us: syntax:%" PRId64 "us, ctg:%" PRId64 "us, semantic:%" PRId64 // tscPerf("select duration %" PRId64 "us: syntax:%" PRId64 "us, ctg:%" PRId64 "us, semantic:%" PRId64
// "us, planner:%" PRId64 "us, exec:%" PRId64 "us, reqId:0x%" PRIx64, // "us, planner:%" PRId64 "us, exec:%" PRId64 "us, reqId:0x%" PRIx64,
// duration, pRequest->metric.syntaxEnd - pRequest->metric.syntaxStart, // duration, pRequest->metric.syntaxEnd - pRequest->metric.syntaxStart,
// pRequest->metric.ctgEnd - pRequest->metric.ctgStart, pRequest->metric.semanticEnd - pRequest->metric.ctgEnd, // pRequest->metric.ctgEnd - pRequest->metric.ctgStart, pRequest->metric.semanticEnd -
// pRequest->metric.planEnd - pRequest->metric.semanticEnd, // pRequest->metric.ctgEnd, pRequest->metric.planEnd - pRequest->metric.semanticEnd,
// pRequest->metric.resultReady - pRequest->metric.planEnd, pRequest->requestId); // pRequest->metric.resultReady - pRequest->metric.planEnd, pRequest->requestId);
atomic_add_fetch_64((int64_t *)&pActivity->queryElapsedTime, duration); atomic_add_fetch_64((int64_t *)&pActivity->queryElapsedTime, duration);
} }
@ -271,7 +271,7 @@ STscObj *acquireTscObj(int64_t rid) { return (STscObj *)taosAcquireRef(clientCon
int32_t releaseTscObj(int64_t rid) { return taosReleaseRef(clientConnRefPool, rid); } int32_t releaseTscObj(int64_t rid) { return taosReleaseRef(clientConnRefPool, rid); }
void *createRequest(uint64_t connId, int32_t type) { void *createRequest(uint64_t connId, int32_t type, int64_t reqid) {
SRequestObj *pRequest = (SRequestObj *)taosMemoryCalloc(1, sizeof(SRequestObj)); SRequestObj *pRequest = (SRequestObj *)taosMemoryCalloc(1, sizeof(SRequestObj));
if (NULL == pRequest) { if (NULL == pRequest) {
terrno = TSDB_CODE_TSC_OUT_OF_MEMORY; terrno = TSDB_CODE_TSC_OUT_OF_MEMORY;
@ -286,7 +286,7 @@ void *createRequest(uint64_t connId, int32_t type) {
} }
pRequest->resType = RES_TYPE__QUERY; pRequest->resType = RES_TYPE__QUERY;
pRequest->requestId = generateRequestId(); pRequest->requestId = reqid == 0 ? generateRequestId() : reqid;
pRequest->metric.start = taosGetTimestampUs(); pRequest->metric.start = taosGetTimestampUs();
pRequest->body.resInfo.convertUcs4 = true; // convert ucs4 by default pRequest->body.resInfo.convertUcs4 = true; // convert ucs4 by default

55
source/client/src/clientImpl.c
View File

@ -154,8 +154,8 @@ STscObj* taos_connect_internal(const char* ip, const char* user, const char* pas
} }
int32_t buildRequest(uint64_t connId, const char* sql, int sqlLen, void* param, bool validateSql, int32_t buildRequest(uint64_t connId, const char* sql, int sqlLen, void* param, bool validateSql,
SRequestObj** pRequest) { SRequestObj** pRequest, int64_t reqid) {
*pRequest = createRequest(connId, TSDB_SQL_SELECT); *pRequest = createRequest(connId, TSDB_SQL_SELECT, reqid);
if (*pRequest == NULL) { if (*pRequest == NULL) {
tscError("failed to malloc sqlObj, %s", sql); tscError("failed to malloc sqlObj, %s", sql);
return terrno; return terrno;
@ -1230,7 +1230,7 @@ STscObj* taosConnectImpl(const char* user, const char* auth, const char* db, __t
return pTscObj; return pTscObj;
} }
SRequestObj* pRequest = createRequest(pTscObj->id, TDMT_MND_CONNECT); SRequestObj* pRequest = createRequest(pTscObj->id, TDMT_MND_CONNECT, 0);
if (pRequest == NULL) { if (pRequest == NULL) {
destroyTscObj(pTscObj); destroyTscObj(pTscObj);
return NULL; return NULL;
@ -2234,7 +2234,37 @@ void taosAsyncQueryImpl(uint64_t connId, const char* sql, __taos_async_fn_t fp,
} }
SRequestObj* pRequest = NULL; SRequestObj* pRequest = NULL;
int32_t code = buildRequest(connId, sql, sqlLen, param, validateOnly, &pRequest); int32_t code = buildRequest(connId, sql, sqlLen, param, validateOnly, &pRequest, 0);
if (code != TSDB_CODE_SUCCESS) {
terrno = code;
fp(param, NULL, terrno);
return;
}
pRequest->body.queryFp = fp;
doAsyncQuery(pRequest, false);
}
void taosAsyncQueryImplWithReqid(uint64_t connId, const char* sql, __taos_async_fn_t fp, void* param, bool validateOnly,
int64_t reqid) {
if (sql == NULL || NULL == fp) {
terrno = TSDB_CODE_INVALID_PARA;
if (fp) {
fp(param, NULL, terrno);
}
return;
}
size_t sqlLen = strlen(sql);
if (sqlLen > (size_t)TSDB_MAX_ALLOWED_SQL_LEN) {
tscError("sql string exceeds max length:%d", TSDB_MAX_ALLOWED_SQL_LEN);
terrno = TSDB_CODE_TSC_EXCEED_SQL_LIMIT;
fp(param, NULL, terrno);
return;
}
SRequestObj* pRequest = NULL;
int32_t code = buildRequest(connId, sql, sqlLen, param, validateOnly, &pRequest, reqid);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
terrno = code; terrno = code;
fp(param, NULL, terrno); fp(param, NULL, terrno);
@ -2261,3 +2291,20 @@ TAOS_RES* taosQueryImpl(TAOS* taos, const char* sql, bool validateOnly) {
} }
return param->pRequest; return param->pRequest;
} }
TAOS_RES* taosQueryImplWithReqid(TAOS* taos, const char* sql, bool validateOnly, int64_t reqid) {
if (NULL == taos) {
terrno = TSDB_CODE_TSC_DISCONNECTED;
return NULL;
}
SSyncQueryParam* param = taosMemoryCalloc(1, sizeof(SSyncQueryParam));
tsem_init(&param->sem, 0, 0);
taosAsyncQueryImplWithReqid(*(int64_t*)taos, sql, syncQueryFn, param, validateOnly, reqid);
tsem_wait(&param->sem);
if (param->pRequest != NULL) {
param->pRequest->syncQuery = true;
}
return param->pRequest;
}

31
source/client/src/clientMain.c
View File

@ -248,6 +248,9 @@ TAOS_FIELD *taos_fetch_fields(TAOS_RES *res) {
} }
TAOS_RES *taos_query(TAOS *taos, const char *sql) { return taosQueryImpl(taos, sql, false); } TAOS_RES *taos_query(TAOS *taos, const char *sql) { return taosQueryImpl(taos, sql, false); }
TAOS_RES *taos_query_with_reqid(TAOS *taos, const char *sql, int64_t reqid) {
return taosQueryImplWithReqid(taos, sql, false, reqid);
}
TAOS_ROW taos_fetch_row(TAOS_RES *res) { TAOS_ROW taos_fetch_row(TAOS_RES *res) {
if (res == NULL) { if (res == NULL) {
@ -763,6 +766,11 @@ void taos_query_a(TAOS *taos, const char *sql, __taos_async_fn_t fp, void *param
taosAsyncQueryImpl(connId, sql, fp, param, false); taosAsyncQueryImpl(connId, sql, fp, param, false);
} }
void taos_query_a_with_reqid(TAOS *taos, const char *sql, __taos_async_fn_t fp, void *param, int64_t reqid) {
int64_t connId = *(int64_t *)taos;
taosAsyncQueryImplWithReqid(connId, sql, fp, param, false, reqid);
}
int32_t createParseContext(const SRequestObj *pRequest, SParseContext **pCxt) { int32_t createParseContext(const SRequestObj *pRequest, SParseContext **pCxt) {
const STscObj *pTscObj = pRequest->pTscObj; const STscObj *pTscObj = pRequest->pTscObj;
@ -992,7 +1000,7 @@ int taos_get_db_route_info(TAOS *taos, const char *db, TAOS_DB_ROUTE_INFO *dbInf
int64_t connId = *(int64_t *)taos; int64_t connId = *(int64_t *)taos;
SRequestObj *pRequest = NULL; SRequestObj *pRequest = NULL;
char *sql = "taos_get_db_route_info"; char *sql = "taos_get_db_route_info";
int32_t code = buildRequest(connId, sql, strlen(sql), NULL, false, &pRequest); int32_t code = buildRequest(connId, sql, strlen(sql), NULL, false, &pRequest, 0);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
terrno = code; terrno = code;
return terrno; return terrno;
@ -1041,7 +1049,7 @@ int taos_get_table_vgId(TAOS *taos, const char *db, const char *table, int *vgId
int64_t connId = *(int64_t *)taos; int64_t connId = *(int64_t *)taos;
SRequestObj *pRequest = NULL; SRequestObj *pRequest = NULL;
char *sql = "taos_get_table_vgId"; char *sql = "taos_get_table_vgId";
int32_t code = buildRequest(connId, sql, strlen(sql), NULL, false, &pRequest); int32_t code = buildRequest(connId, sql, strlen(sql), NULL, false, &pRequest, 0);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
return terrno; return terrno;
} }
@ -1102,7 +1110,7 @@ int taos_load_table_info(TAOS *taos, const char *tableNameList) {
} }
char *sql = "taos_load_table_info"; char *sql = "taos_load_table_info";
code = buildRequest(connId, sql, strlen(sql), NULL, false, &pRequest); code = buildRequest(connId, sql, strlen(sql), NULL, false, &pRequest, 0);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
terrno = code; terrno = code;
goto _return; goto _return;
@ -1147,7 +1155,22 @@ TAOS_STMT *taos_stmt_init(TAOS *taos) {
return NULL; return NULL;
} }
TAOS_STMT *pStmt = stmtInit(pObj); TAOS_STMT *pStmt = stmtInit(pObj, 0);
releaseTscObj(*(int64_t *)taos);
return pStmt;
}
TAOS_STMT *taos_stmt_init_with_reqid(TAOS *taos, int64_t reqid) {
STscObj *pObj = acquireTscObj(*(int64_t *)taos);
if (NULL == pObj) {
tscError("invalid parameter for %s", __FUNCTION__);
terrno = TSDB_CODE_TSC_DISCONNECTED;
return NULL;
}
TAOS_STMT *pStmt = stmtInit(pObj, reqid);
releaseTscObj(*(int64_t *)taos); releaseTscObj(*(int64_t *)taos);

24
source/client/src/clientRawBlockWrite.c
View File

@ -410,7 +410,7 @@ static char* processAlterTable(SMqMetaRsp* metaRsp) {
SDecoder decoder = {0}; SDecoder decoder = {0};
SVAlterTbReq vAlterTbReq = {0}; SVAlterTbReq vAlterTbReq = {0};
char* string = NULL; char* string = NULL;
cJSON* json = NULL; cJSON* json = NULL;
// decode // decode
void* data = POINTER_SHIFT(metaRsp->metaRsp, sizeof(SMsgHead)); void* data = POINTER_SHIFT(metaRsp->metaRsp, sizeof(SMsgHead));
@ -525,7 +525,7 @@ static char* processDropSTable(SMqMetaRsp* metaRsp) {
SDecoder decoder = {0}; SDecoder decoder = {0};
SVDropStbReq req = {0}; SVDropStbReq req = {0};
char* string = NULL; char* string = NULL;
cJSON* json = NULL; cJSON* json = NULL;
// decode // decode
void* data = POINTER_SHIFT(metaRsp->metaRsp, sizeof(SMsgHead)); void* data = POINTER_SHIFT(metaRsp->metaRsp, sizeof(SMsgHead));
@ -558,7 +558,7 @@ static char* processDropTable(SMqMetaRsp* metaRsp) {
SDecoder decoder = {0}; SDecoder decoder = {0};
SVDropTbBatchReq req = {0}; SVDropTbBatchReq req = {0};
char* string = NULL; char* string = NULL;
cJSON* json = NULL; cJSON* json = NULL;
// decode // decode
void* data = POINTER_SHIFT(metaRsp->metaRsp, sizeof(SMsgHead)); void* data = POINTER_SHIFT(metaRsp->metaRsp, sizeof(SMsgHead));
@ -603,7 +603,7 @@ static int32_t taosCreateStb(TAOS* taos, void* meta, int32_t metaLen) {
int32_t code = TSDB_CODE_SUCCESS; int32_t code = TSDB_CODE_SUCCESS;
SRequestObj* pRequest = NULL; SRequestObj* pRequest = NULL;
code = buildRequest(*(int64_t*)taos, "", 0, NULL, false, &pRequest); code = buildRequest(*(int64_t*)taos, "", 0, NULL, false, &pRequest, 0);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
goto end; goto end;
} }
@ -692,7 +692,7 @@ static int32_t taosDropStb(TAOS* taos, void* meta, int32_t metaLen) {
int32_t code = TSDB_CODE_SUCCESS; int32_t code = TSDB_CODE_SUCCESS;
SRequestObj* pRequest = NULL; SRequestObj* pRequest = NULL;
code = buildRequest(*(int64_t*)taos, "", 0, NULL, false, &pRequest); code = buildRequest(*(int64_t*)taos, "", 0, NULL, false, &pRequest, 0);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
goto end; goto end;
} }
@ -773,7 +773,7 @@ static int32_t taosCreateTable(TAOS* taos, void* meta, int32_t metaLen) {
SQuery* pQuery = NULL; SQuery* pQuery = NULL;
SHashObj* pVgroupHashmap = NULL; SHashObj* pVgroupHashmap = NULL;
code = buildRequest(*(int64_t*)taos, "", 0, NULL, false, &pRequest); code = buildRequest(*(int64_t*)taos, "", 0, NULL, false, &pRequest, 0);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
goto end; goto end;
} }
@ -926,7 +926,7 @@ static int32_t taosDropTable(TAOS* taos, void* meta, int32_t metaLen) {
SQuery* pQuery = NULL; SQuery* pQuery = NULL;
SHashObj* pVgroupHashmap = NULL; SHashObj* pVgroupHashmap = NULL;
code = buildRequest(*(int64_t*)taos, "", 0, NULL, false, &pRequest); code = buildRequest(*(int64_t*)taos, "", 0, NULL, false, &pRequest, 0);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
goto end; goto end;
} }
@ -1097,7 +1097,7 @@ static int32_t taosAlterTable(TAOS* taos, void* meta, int32_t metaLen) {
SArray* pArray = NULL; SArray* pArray = NULL;
SVgDataBlocks* pVgData = NULL; SVgDataBlocks* pVgData = NULL;
code = buildRequest(*(int64_t*)taos, "", 0, NULL, false, &pRequest); code = buildRequest(*(int64_t*)taos, "", 0, NULL, false, &pRequest, 0);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
goto end; goto end;
@ -1217,7 +1217,7 @@ int taos_write_raw_block(TAOS* taos, int rows, char* pData, const char* tbname)
SQuery* pQuery = NULL; SQuery* pQuery = NULL;
SSubmitReq* subReq = NULL; SSubmitReq* subReq = NULL;
SRequestObj* pRequest = (SRequestObj*)createRequest(*(int64_t*)taos, TSDB_SQL_INSERT); SRequestObj* pRequest = (SRequestObj*)createRequest(*(int64_t*)taos, TSDB_SQL_INSERT, 0);
if (!pRequest) { if (!pRequest) {
uError("WriteRaw:createRequest error request is null"); uError("WriteRaw:createRequest error request is null");
code = terrno; code = terrno;
@ -1281,7 +1281,7 @@ int taos_write_raw_block(TAOS* taos, int rows, char* pData, const char* tbname)
int32_t schemaLen = 0; int32_t schemaLen = 0;
int32_t submitLen = sizeof(SSubmitBlk) + schemaLen + rows * extendedRowSize; int32_t submitLen = sizeof(SSubmitBlk) + schemaLen + rows * extendedRowSize;
int32_t totalLen = sizeof(SSubmitReq) + submitLen; int32_t totalLen = sizeof(SSubmitReq) + submitLen;
subReq = taosMemoryCalloc(1, totalLen); subReq = taosMemoryCalloc(1, totalLen);
SSubmitBlk* blk = POINTER_SHIFT(subReq, sizeof(SSubmitReq)); SSubmitBlk* blk = POINTER_SHIFT(subReq, sizeof(SSubmitReq));
void* blkSchema = POINTER_SHIFT(blk, sizeof(SSubmitBlk)); void* blkSchema = POINTER_SHIFT(blk, sizeof(SSubmitBlk));
@ -1407,7 +1407,7 @@ static int32_t tmqWriteRawDataImpl(TAOS* taos, void* data, int32_t dataLen) {
STableMeta* pTableMeta = NULL; STableMeta* pTableMeta = NULL;
terrno = TSDB_CODE_SUCCESS; terrno = TSDB_CODE_SUCCESS;
SRequestObj* pRequest = (SRequestObj*)createRequest(*(int64_t*)taos, TSDB_SQL_INSERT); SRequestObj* pRequest = (SRequestObj*)createRequest(*(int64_t*)taos, TSDB_SQL_INSERT, 0);
if (!pRequest) { if (!pRequest) {
uError("WriteRaw:createRequest error request is null"); uError("WriteRaw:createRequest error request is null");
return terrno; return terrno;
@ -1674,7 +1674,7 @@ static int32_t tmqWriteRawMetaDataImpl(TAOS* taos, void* data, int32_t dataLen)
STableMeta* pTableMeta = NULL; STableMeta* pTableMeta = NULL;
terrno = TSDB_CODE_SUCCESS; terrno = TSDB_CODE_SUCCESS;
SRequestObj* pRequest = (SRequestObj*)createRequest(*(int64_t*)taos, TSDB_SQL_INSERT); SRequestObj* pRequest = (SRequestObj*)createRequest(*(int64_t*)taos, TSDB_SQL_INSERT, 0);
if (!pRequest) { if (!pRequest) {
uError("WriteRaw:createRequest error request is null"); uError("WriteRaw:createRequest error request is null");
return terrno; return terrno;

158
source/client/src/clientSml.c
View File

@ -28,12 +28,12 @@
#define QUOTE '"' #define QUOTE '"'
#define SLASH '\\' #define SLASH '\\'
#define JUMP_SPACE(sql, sqlEnd) \ #define JUMP_SPACE(sql, sqlEnd) \
while (sql < sqlEnd) { \ while (sql < sqlEnd) { \
if (*sql == SPACE) \ if (*sql == SPACE) \
sql++; \ sql++; \
else \ else \
break; \ break; \
} }
// comma , // comma ,
#define IS_SLASH_COMMA(sql) (*(sql) == COMMA && *((sql)-1) == SLASH) #define IS_SLASH_COMMA(sql) (*(sql) == COMMA && *((sql)-1) == SLASH)
@ -353,7 +353,7 @@ static int32_t smlSendMetaMsg(SSmlHandle *info, SName *pName, SArray *pColumns,
pReq.numOfTags = taosArrayGetSize(pTags); pReq.numOfTags = taosArrayGetSize(pTags);
pReq.pTags = pTags; pReq.pTags = pTags;
code = buildRequest(info->taos->id, "", 0, NULL, false, &pRequest); code = buildRequest(info->taos->id, "", 0, NULL, false, &pRequest, 0);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
goto end; goto end;
} }
@ -1010,9 +1010,9 @@ static void smlParseTelnetElement(const char **sql, const char *sqlEnd, const ch
} }
} }
static int32_t smlParseTelnetTags(const char *data, const char *sqlEnd, SArray *cols, char *childTableName, SHashObj *dumplicateKey, static int32_t smlParseTelnetTags(const char *data, const char *sqlEnd, SArray *cols, char *childTableName,
SSmlMsgBuf *msg) { SHashObj *dumplicateKey, SSmlMsgBuf *msg) {
if(!cols) return TSDB_CODE_OUT_OF_MEMORY; if (!cols) return TSDB_CODE_OUT_OF_MEMORY;
const char *sql = data; const char *sql = data;
size_t childTableNameLen = strlen(tsSmlChildTableName); size_t childTableNameLen = strlen(tsSmlChildTableName);
while (sql < sqlEnd) { while (sql < sqlEnd) {
@ -1093,7 +1093,8 @@ static int32_t smlParseTelnetTags(const char *data, const char *sqlEnd, SArray *
} }
// format: <metric> <timestamp> <value> <tagk_1>=<tagv_1>[ <tagk_n>=<tagv_n>] // format: <metric> <timestamp> <value> <tagk_1>=<tagv_1>[ <tagk_n>=<tagv_n>]
static int32_t smlParseTelnetString(SSmlHandle *info, const char *sql, const char *sqlEnd, SSmlTableInfo *tinfo, SArray *cols) { static int32_t smlParseTelnetString(SSmlHandle *info, const char *sql, const char *sqlEnd, SSmlTableInfo *tinfo,
SArray *cols) {
if (!sql) return TSDB_CODE_SML_INVALID_DATA; if (!sql) return TSDB_CODE_SML_INVALID_DATA;
// parse metric // parse metric
@ -1372,19 +1373,19 @@ static int32_t smlKvTimeArrayCompare(const void *key1, const void *key2) {
static int32_t smlKvTimeHashCompare(const void *key1, const void *key2) { static int32_t smlKvTimeHashCompare(const void *key1, const void *key2) {
SHashObj *s1 = *(SHashObj **)key1; SHashObj *s1 = *(SHashObj **)key1;
SHashObj *s2 = *(SHashObj **)key2; SHashObj *s2 = *(SHashObj **)key2;
SSmlKv **kv1pp = (SSmlKv **)taosHashGet(s1, TS, TS_LEN); SSmlKv **kv1pp = (SSmlKv **)taosHashGet(s1, TS, TS_LEN);
SSmlKv **kv2pp = (SSmlKv **)taosHashGet(s2, TS, TS_LEN); SSmlKv **kv2pp = (SSmlKv **)taosHashGet(s2, TS, TS_LEN);
if(!kv1pp || !kv2pp){ if (!kv1pp || !kv2pp) {
uError("smlKvTimeHashCompare kv is null"); uError("smlKvTimeHashCompare kv is null");
return -1; return -1;
} }
SSmlKv *kv1 = *kv1pp; SSmlKv *kv1 = *kv1pp;
SSmlKv *kv2 = *kv2pp; SSmlKv *kv2 = *kv2pp;
if(!kv1 || kv1->type != TSDB_DATA_TYPE_TIMESTAMP){ if (!kv1 || kv1->type != TSDB_DATA_TYPE_TIMESTAMP) {
uError("smlKvTimeHashCompare kv1"); uError("smlKvTimeHashCompare kv1");
return -1; return -1;
} }
if(!kv2 || kv2->type != TSDB_DATA_TYPE_TIMESTAMP){ if (!kv2 || kv2->type != TSDB_DATA_TYPE_TIMESTAMP) {
uError("smlKvTimeHashCompare kv2"); uError("smlKvTimeHashCompare kv2");
return -1; return -1;
} }
@ -1947,7 +1948,7 @@ static int32_t smlParseValueFromJSON(cJSON *root, SSmlKv *kv) {
} }
static int32_t smlParseColsFromJSON(cJSON *root, SArray *cols) { static int32_t smlParseColsFromJSON(cJSON *root, SArray *cols) {
if(!cols) return TSDB_CODE_OUT_OF_MEMORY; if (!cols) return TSDB_CODE_OUT_OF_MEMORY;
cJSON *metricVal = cJSON_GetObjectItem(root, "value"); cJSON *metricVal = cJSON_GetObjectItem(root, "value");
if (metricVal == NULL) { if (metricVal == NULL) {
return TSDB_CODE_TSC_INVALID_JSON; return TSDB_CODE_TSC_INVALID_JSON;
@ -1971,7 +1972,7 @@ static int32_t smlParseColsFromJSON(cJSON *root, SArray *cols) {
static int32_t smlParseTagsFromJSON(cJSON *root, SArray *pKVs, char *childTableName, SHashObj *dumplicateKey, static int32_t smlParseTagsFromJSON(cJSON *root, SArray *pKVs, char *childTableName, SHashObj *dumplicateKey,
SSmlMsgBuf *msg) { SSmlMsgBuf *msg) {
int32_t ret = TSDB_CODE_SUCCESS; int32_t ret = TSDB_CODE_SUCCESS;
if (!pKVs){ if (!pKVs) {
return TSDB_CODE_OUT_OF_MEMORY; return TSDB_CODE_OUT_OF_MEMORY;
} }
cJSON *tags = cJSON_GetObjectItem(root, "tags"); cJSON *tags = cJSON_GetObjectItem(root, "tags");
@ -2204,7 +2205,7 @@ static int32_t smlParseTelnetLine(SSmlHandle *info, void *data, const int len) {
} }
if (info->protocol == TSDB_SML_TELNET_PROTOCOL) { if (info->protocol == TSDB_SML_TELNET_PROTOCOL) {
ret = smlParseTelnetString(info, (const char *)data, (char*)data + len, tinfo, cols); ret = smlParseTelnetString(info, (const char *)data, (char *)data + len, tinfo, cols);
} else if (info->protocol == TSDB_SML_JSON_PROTOCOL) { } else if (info->protocol == TSDB_SML_JSON_PROTOCOL) {
ret = smlParseJSONString(info, (cJSON *)data, tinfo, cols); ret = smlParseJSONString(info, (cJSON *)data, tinfo, cols);
} else { } else {
@ -2384,16 +2385,16 @@ static void smlPrintStatisticInfo(SSmlHandle *info) {
info->cost.endTime - info->cost.insertRpcTime, info->cost.endTime - info->cost.parseTime); info->cost.endTime - info->cost.insertRpcTime, info->cost.endTime - info->cost.parseTime);
} }
static int32_t smlParseLine(SSmlHandle *info, char *lines[], char* rawLine, char* rawLineEnd, int numLines) { static int32_t smlParseLine(SSmlHandle *info, char *lines[], char *rawLine, char *rawLineEnd, int numLines) {
int32_t code = TSDB_CODE_SUCCESS; int32_t code = TSDB_CODE_SUCCESS;
if (info->protocol == TSDB_SML_JSON_PROTOCOL) { if (info->protocol == TSDB_SML_JSON_PROTOCOL) {
if(lines){ if (lines) {
code = smlParseJSON(info, *lines); code = smlParseJSON(info, *lines);
}else if(rawLine){ } else if (rawLine) {
code = smlParseJSON(info, rawLine); code = smlParseJSON(info, rawLine);
} }
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
uError("SML:0x%" PRIx64 " smlParseJSON failed:%s", info->id, lines?*lines:rawLine); uError("SML:0x%" PRIx64 " smlParseJSON failed:%s", info->id, lines ? *lines : rawLine);
return code; return code;
} }
return code; return code;
@ -2401,19 +2402,19 @@ static int32_t smlParseLine(SSmlHandle *info, char *lines[], char* rawLine, char
for (int32_t i = 0; i < numLines; ++i) { for (int32_t i = 0; i < numLines; ++i) {
char *tmp = NULL; char *tmp = NULL;
int len = 0; int len = 0;
if(lines){ if (lines) {
tmp = lines[i]; tmp = lines[i];
len = strlen(tmp); len = strlen(tmp);
}else if(rawLine){ } else if (rawLine) {
tmp = rawLine; tmp = rawLine;
while(rawLine < rawLineEnd){ while (rawLine < rawLineEnd) {
if(*(rawLine++) == '\n'){ if (*(rawLine++) == '\n') {
break; break;
} }
len++; len++;
} }
if(info->protocol == TSDB_SML_LINE_PROTOCOL && tmp[0] == '#'){ // this line is comment if (info->protocol == TSDB_SML_LINE_PROTOCOL && tmp[0] == '#') { // this line is comment
continue; continue;
} }
} }
@ -2433,7 +2434,7 @@ static int32_t smlParseLine(SSmlHandle *info, char *lines[], char* rawLine, char
return code; return code;
} }
static int smlProcess(SSmlHandle *info, char *lines[], char* rawLine, char* rawLineEnd, int numLines) { static int smlProcess(SSmlHandle *info, char *lines[], char *rawLine, char *rawLineEnd, int numLines) {
int32_t code = TSDB_CODE_SUCCESS; int32_t code = TSDB_CODE_SUCCESS;
int32_t retryNum = 0; int32_t retryNum = 0;
@ -2532,8 +2533,8 @@ static void smlInsertCallback(void *param, void *res, int32_t code) {
smlDestroyInfo(info); smlDestroyInfo(info);
} }
TAOS_RES *taos_schemaless_insert_inner(SRequestObj *request, char *lines[], char *rawLine, char *rawLineEnd,
TAOS_RES *taos_schemaless_insert_inner(SRequestObj *request, char *lines[], char *rawLine, char *rawLineEnd, int numLines, int protocol, int precision) { int numLines, int protocol, int precision) {
int batchs = 0; int batchs = 0;
STscObj *pTscObj = request->pTscObj; STscObj *pTscObj = request->pTscObj;
@ -2581,7 +2582,7 @@ TAOS_RES *taos_schemaless_insert_inner(SRequestObj *request, char *lines[], char
batchs = ceil(((double)numLines) / LINE_BATCH); batchs = ceil(((double)numLines) / LINE_BATCH);
params.total = batchs; params.total = batchs;
for (int i = 0; i < batchs; ++i) { for (int i = 0; i < batchs; ++i) {
SRequestObj *req = (SRequestObj *)createRequest(pTscObj->id, TSDB_SQL_INSERT); SRequestObj *req = (SRequestObj *)createRequest(pTscObj->id, TSDB_SQL_INSERT, 0);
if (!req) { if (!req) {
request->code = TSDB_CODE_OUT_OF_MEMORY; request->code = TSDB_CODE_OUT_OF_MEMORY;
uError("SML:taos_schemaless_insert error request is null"); uError("SML:taos_schemaless_insert error request is null");
@ -2608,16 +2609,16 @@ TAOS_RES *taos_schemaless_insert_inner(SRequestObj *request, char *lines[], char
info->pRequest->body.queryFp = smlInsertCallback; info->pRequest->body.queryFp = smlInsertCallback;
info->pRequest->body.param = info; info->pRequest->body.param = info;
int32_t code = smlProcess(info, lines, rawLine, rawLineEnd, perBatch); int32_t code = smlProcess(info, lines, rawLine, rawLineEnd, perBatch);
if(lines){ if (lines) {
lines += perBatch; lines += perBatch;
} }
if(rawLine){ if (rawLine) {
int num = 0; int num = 0;
while(rawLine < rawLineEnd){ while (rawLine < rawLineEnd) {
if(*(rawLine++) == '\n'){ if (*(rawLine++) == '\n') {
num++; num++;
} }
if(num == perBatch){ if (num == perBatch) {
break; break;
} }
} }
@ -2628,7 +2629,7 @@ TAOS_RES *taos_schemaless_insert_inner(SRequestObj *request, char *lines[], char
} }
tsem_wait(&params.sem); tsem_wait(&params.sem);
end: end:
taosThreadSpinDestroy(&params.lock); taosThreadSpinDestroy(&params.lock);
tsem_destroy(&params.sem); tsem_destroy(&params.sem);
// ((STscObj *)taos)->schemalessType = 0; // ((STscObj *)taos)->schemalessType = 0;
@ -2653,9 +2654,7 @@ TAOS_RES *taos_schemaless_insert_inner(SRequestObj *request, char *lines[], char
* 0 - influxDB line protocol * 0 - influxDB line protocol
* 1 - OpenTSDB telnet line protocol * 1 - OpenTSDB telnet line protocol
* 2 - OpenTSDB JSON format protocol * 2 - OpenTSDB JSON format protocol
* @return return zero for successful insertion. Otherwise return none-zero error code of * @return TAOS_RES
* failure reason.
*
*/ */
TAOS_RES *taos_schemaless_insert(TAOS *taos, char *lines[], int numLines, int protocol, int precision) { TAOS_RES *taos_schemaless_insert(TAOS *taos, char *lines[], int numLines, int protocol, int precision) {
@ -2664,7 +2663,7 @@ TAOS_RES *taos_schemaless_insert(TAOS *taos, char *lines[], int numLines, int pr
return NULL; return NULL;
} }
SRequestObj *request = (SRequestObj *)createRequest(*(int64_t *)taos, TSDB_SQL_INSERT); SRequestObj *request = (SRequestObj *)createRequest(*(int64_t *)taos, TSDB_SQL_INSERT, 0);
if (!request) { if (!request) {
uError("SML:taos_schemaless_insert error request is null"); uError("SML:taos_schemaless_insert error request is null");
return NULL; return NULL;
@ -2680,13 +2679,37 @@ TAOS_RES *taos_schemaless_insert(TAOS *taos, char *lines[], int numLines, int pr
return taos_schemaless_insert_inner(request, lines, NULL, NULL, numLines, protocol, precision); return taos_schemaless_insert_inner(request, lines, NULL, NULL, numLines, protocol, precision);
} }
TAOS_RES *taos_schemaless_insert_raw(TAOS* taos, char* lines, int len, int32_t *totalRows, int protocol, int precision){ TAOS_RES *taos_schemaless_insert_with_reqid(TAOS *taos, char *lines[], int numLines, int protocol, int precision,
int64_t reqid) {
if (NULL == taos) { if (NULL == taos) {
terrno = TSDB_CODE_TSC_DISCONNECTED; terrno = TSDB_CODE_TSC_DISCONNECTED;
return NULL; return NULL;
} }
SRequestObj *request = (SRequestObj *)createRequest(*(int64_t *)taos, TSDB_SQL_INSERT); SRequestObj *request = (SRequestObj *)createRequest(*(int64_t *)taos, TSDB_SQL_INSERT, reqid);
if (!request) {
uError("SML:taos_schemaless_insert error request is null");
return NULL;
}
if (!lines) {
SSmlMsgBuf msg = {ERROR_MSG_BUF_DEFAULT_SIZE, request->msgBuf};
request->code = TSDB_CODE_SML_INVALID_DATA;
smlBuildInvalidDataMsg(&msg, "lines is null", NULL);
return (TAOS_RES *)request;
}
return taos_schemaless_insert_inner(request, lines, NULL, NULL, numLines, protocol, precision);
}
TAOS_RES *taos_schemaless_insert_raw(TAOS *taos, char *lines, int len, int32_t *totalRows, int protocol,
int precision) {
if (NULL == taos) {
terrno = TSDB_CODE_TSC_DISCONNECTED;
return NULL;
}
SRequestObj *request = (SRequestObj *)createRequest(*(int64_t *)taos, TSDB_SQL_INSERT, 0);
if (!request) { if (!request) {
uError("SML:taos_schemaless_insert error request is null"); uError("SML:taos_schemaless_insert error request is null");
return NULL; return NULL;
@ -2702,10 +2725,45 @@ TAOS_RES *taos_schemaless_insert_raw(TAOS* taos, char* lines, int len, int32_t *
int numLines = 0; int numLines = 0;
*totalRows = 0; *totalRows = 0;
char *tmp = lines; char *tmp = lines;
for(int i = 0; i < len; i++){ for (int i = 0; i < len; i++) {
if(lines[i] == '\n' || i == len - 1){ if (lines[i] == '\n' || i == len - 1) {
numLines++; numLines++;
if(tmp[0] != '#' || protocol != TSDB_SML_LINE_PROTOCOL){ //ignore comment if (tmp[0] != '#' || protocol != TSDB_SML_LINE_PROTOCOL) { // ignore comment
(*totalRows)++;
}
tmp = lines + i + 1;
}
}
return taos_schemaless_insert_inner(request, NULL, lines, lines + len, numLines, protocol, precision);
}
TAOS_RES *taos_schemaless_insert_raw_with_reqid(TAOS *taos, char *lines, int len, int32_t *totalRows, int protocol,
int precision, int64_t reqid) {
if (NULL == taos) {
terrno = TSDB_CODE_TSC_DISCONNECTED;
return NULL;
}
SRequestObj *request = (SRequestObj *)createRequest(*(int64_t *)taos, TSDB_SQL_INSERT, reqid);
if (!request) {
uError("SML:taos_schemaless_insert error request is null");
return NULL;
}
if (!lines || len <= 0) {
SSmlMsgBuf msg = {ERROR_MSG_BUF_DEFAULT_SIZE, request->msgBuf};
request->code = TSDB_CODE_SML_INVALID_DATA;
smlBuildInvalidDataMsg(&msg, "lines is null", NULL);
return (TAOS_RES *)request;
}
int numLines = 0;
*totalRows = 0;
char *tmp = lines;
for (int i = 0; i < len; i++) {
if (lines[i] == '\n' || i == len - 1) {
numLines++;
if (tmp[0] != '#' || protocol != TSDB_SML_LINE_PROTOCOL) { // ignore comment
(*totalRows)++; (*totalRows)++;
} }
tmp = lines + i + 1; tmp = lines + i + 1;

18
source/client/src/clientStmt.c
View File

@ -5,13 +5,18 @@
#include "clientStmt.h" #include "clientStmt.h"
char *gStmtStatusStr[] = {"unknown", "init", "prepare", "settbname", "settags", "fetchFields", "bind", "bindCol", "addBatch", "exec"}; char* gStmtStatusStr[] = {"unknown", "init", "prepare", "settbname", "settags",
"fetchFields", "bind", "bindCol", "addBatch", "exec"};
static int32_t stmtCreateRequest(STscStmt* pStmt) { static int32_t stmtCreateRequest(STscStmt* pStmt) {
int32_t code = 0; int32_t code = 0;
if (pStmt->exec.pRequest == NULL) { if (pStmt->exec.pRequest == NULL) {
code = buildRequest(pStmt->taos->id, pStmt->sql.sqlStr, pStmt->sql.sqlLen, NULL, false, &pStmt->exec.pRequest); code = buildRequest(pStmt->taos->id, pStmt->sql.sqlStr, pStmt->sql.sqlLen, NULL, false, &pStmt->exec.pRequest,
pStmt->reqid);
if (pStmt->reqid != 0) {
pStmt->reqid++;
}
if (TSDB_CODE_SUCCESS == code) { if (TSDB_CODE_SUCCESS == code) {
pStmt->exec.pRequest->syncQuery = true; pStmt->exec.pRequest->syncQuery = true;
} }
@ -207,10 +212,10 @@ int32_t stmtCacheBlock(STscStmt* pStmt) {
} }
STableDataBlocks** pSrc = taosHashGet(pStmt->exec.pBlockHash, pStmt->bInfo.tbFName, strlen(pStmt->bInfo.tbFName)); STableDataBlocks** pSrc = taosHashGet(pStmt->exec.pBlockHash, pStmt->bInfo.tbFName, strlen(pStmt->bInfo.tbFName));
if(!pSrc){ if (!pSrc) {
return TSDB_CODE_OUT_OF_MEMORY; return TSDB_CODE_OUT_OF_MEMORY;
} }
STableDataBlocks* pDst = NULL; STableDataBlocks* pDst = NULL;
STMT_ERR_RET(qCloneStmtDataBlock(&pDst, *pSrc)); STMT_ERR_RET(qCloneStmtDataBlock(&pDst, *pSrc));
@ -513,7 +518,7 @@ int32_t stmtResetStmt(STscStmt* pStmt) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
TAOS_STMT* stmtInit(STscObj* taos) { TAOS_STMT* stmtInit(STscObj* taos, int64_t reqid) {
STscObj* pObj = (STscObj*)taos; STscObj* pObj = (STscObj*)taos;
STscStmt* pStmt = NULL; STscStmt* pStmt = NULL;
@ -533,9 +538,10 @@ TAOS_STMT* stmtInit(STscObj* taos) {
pStmt->taos = pObj; pStmt->taos = pObj;
pStmt->bInfo.needParse = true; pStmt->bInfo.needParse = true;
pStmt->sql.status = STMT_INIT; pStmt->sql.status = STMT_INIT;
pStmt->reqid = reqid;
STMT_LOG_SEQ(STMT_INIT); STMT_LOG_SEQ(STMT_INIT);
tscDebug("stmt:%p initialized", pStmt); tscDebug("stmt:%p initialized", pStmt);
return pStmt; return pStmt;

6
source/common/src/tdatablock.c
View File

@ -1172,11 +1172,7 @@ static int32_t doEnsureCapacity(SColumnInfoData* pColumn, const SDataBlockInfo*
pColumn->nullbitmap = tmp; pColumn->nullbitmap = tmp;
memset(&pColumn->nullbitmap[oldLen], 0, BitmapLen(numOfRows) - oldLen); memset(&pColumn->nullbitmap[oldLen], 0, BitmapLen(numOfRows) - oldLen);
if (pColumn->info.type == TSDB_DATA_TYPE_NULL) { ASSERT(pColumn->info.bytes);
return TSDB_CODE_SUCCESS;
}
assert(pColumn->info.bytes);
tmp = taosMemoryRealloc(pColumn->pData, numOfRows * pColumn->info.bytes); tmp = taosMemoryRealloc(pColumn->pData, numOfRows * pColumn->info.bytes);
if (tmp == NULL) { if (tmp == NULL) {
return TSDB_CODE_OUT_OF_MEMORY; return TSDB_CODE_OUT_OF_MEMORY;

27
source/libs/executor/inc/executorimpl.h
View File

@ -311,7 +311,6 @@ typedef struct STableScanInfo {
SFileBlockLoadRecorder readRecorder; SFileBlockLoadRecorder readRecorder;
SScanInfo scanInfo; SScanInfo scanInfo;
int32_t scanTimes; int32_t scanTimes;
SNode* pFilterNode; // filter info, which is push down by optimizer
SSDataBlock* pResBlock; SSDataBlock* pResBlock;
SColMatchInfo matchInfo; SColMatchInfo matchInfo;
SExprSupp pseudoSup; SExprSupp pseudoSup;
@ -348,7 +347,6 @@ typedef struct STableMergeScanInfo {
int64_t numOfRows; int64_t numOfRows;
SScanInfo scanInfo; SScanInfo scanInfo;
int32_t scanTimes; int32_t scanTimes;
SNode* pFilterNode; // filter info, which is push down by optimizer
SqlFunctionCtx* pCtx; // which belongs to the direct upstream operator operator query context SqlFunctionCtx* pCtx; // which belongs to the direct upstream operator operator query context
SResultRowInfo* pResultRowInfo; SResultRowInfo* pResultRowInfo;
int32_t* rowEntryInfoOffset; int32_t* rowEntryInfoOffset;
@ -466,7 +464,6 @@ typedef struct SStreamScanInfo {
SReadHandle readHandle; SReadHandle readHandle;
SInterval interval; // if the upstream is an interval operator, the interval info is also kept here. SInterval interval; // if the upstream is an interval operator, the interval info is also kept here.
SColMatchInfo matchInfo; SColMatchInfo matchInfo;
SNode* pCondition;
SArray* pBlockLists; // multiple SSDatablock. SArray* pBlockLists; // multiple SSDatablock.
SSDataBlock* pRes; // result SSDataBlock SSDataBlock* pRes; // result SSDataBlock
@ -570,7 +567,6 @@ typedef struct SIntervalAggOperatorInfo {
EOPTR_EXEC_MODEL execModel; // operator execution model [batch model|stream model] EOPTR_EXEC_MODEL execModel; // operator execution model [batch model|stream model]
STimeWindowAggSupp twAggSup; STimeWindowAggSupp twAggSup;
SArray* pPrevValues; // SArray<SGroupKeys> used to keep the previous not null value for interpolation. SArray* pPrevValues; // SArray<SGroupKeys> used to keep the previous not null value for interpolation.
SNode* pCondition;
} SIntervalAggOperatorInfo; } SIntervalAggOperatorInfo;
typedef struct SMergeAlignedIntervalAggOperatorInfo { typedef struct SMergeAlignedIntervalAggOperatorInfo {
@ -579,12 +575,10 @@ typedef struct SMergeAlignedIntervalAggOperatorInfo {
uint64_t groupId; // current groupId uint64_t groupId; // current groupId
int64_t curTs; // current ts int64_t curTs; // current ts
SSDataBlock* prefetchedBlock; SSDataBlock* prefetchedBlock;
SNode* pCondition;
SResultRow* pResultRow; SResultRow* pResultRow;
} SMergeAlignedIntervalAggOperatorInfo; } SMergeAlignedIntervalAggOperatorInfo;
typedef struct SStreamIntervalOperatorInfo { typedef struct SStreamIntervalOperatorInfo {
// SOptrBasicInfo should be first, SAggSupporter should be second for stream encode
SOptrBasicInfo binfo; // basic info SOptrBasicInfo binfo; // basic info
SAggSupporter aggSup; // aggregate supporter SAggSupporter aggSup; // aggregate supporter
SExprSupp scalarSupp; // supporter for perform scalar function SExprSupp scalarSupp; // supporter for perform scalar function
@ -610,26 +604,21 @@ typedef struct SStreamIntervalOperatorInfo {
} SStreamIntervalOperatorInfo; } SStreamIntervalOperatorInfo;
typedef struct SAggOperatorInfo { typedef struct SAggOperatorInfo {
// SOptrBasicInfo should be first, SAggSupporter should be second for stream encode SOptrBasicInfo binfo;
SOptrBasicInfo binfo; SAggSupporter aggSup;
SAggSupporter aggSup;
STableQueryInfo* current; STableQueryInfo* current;
uint64_t groupId; uint64_t groupId;
SGroupResInfo groupResInfo; SGroupResInfo groupResInfo;
SExprSupp scalarExprSup; SExprSupp scalarExprSup;
SNode* pCondition;
} SAggOperatorInfo; } SAggOperatorInfo;
typedef struct SProjectOperatorInfo { typedef struct SProjectOperatorInfo {
SOptrBasicInfo binfo; SOptrBasicInfo binfo;
SAggSupporter aggSup; SAggSupporter aggSup;
SNode* pFilterNode; // filter info, which is push down by optimizer
SArray* pPseudoColInfo; SArray* pPseudoColInfo;
SLimitInfo limitInfo; SLimitInfo limitInfo;
bool mergeDataBlocks; bool mergeDataBlocks;
SSDataBlock* pFinalRes; SSDataBlock* pFinalRes;
SNode* pCondition;
} SProjectOperatorInfo; } SProjectOperatorInfo;
typedef struct SIndefOperatorInfo { typedef struct SIndefOperatorInfo {
@ -637,10 +626,8 @@ typedef struct SIndefOperatorInfo {
SAggSupporter aggSup; SAggSupporter aggSup;
SArray* pPseudoColInfo; SArray* pPseudoColInfo;
SExprSupp scalarSup; SExprSupp scalarSup;
SNode* pCondition;
uint64_t groupId; uint64_t groupId;
SSDataBlock* pNextGroupRes;
SSDataBlock* pNextGroupRes;
} SIndefOperatorInfo; } SIndefOperatorInfo;
typedef struct SFillOperatorInfo { typedef struct SFillOperatorInfo {
@ -651,7 +638,6 @@ typedef struct SFillOperatorInfo {
void** p; void** p;
SSDataBlock* existNewGroupBlock; SSDataBlock* existNewGroupBlock;
STimeWindow win; STimeWindow win;
SNode* pCondition;
SColMatchInfo matchInfo; SColMatchInfo matchInfo;
int32_t primaryTsCol; int32_t primaryTsCol;
int32_t primarySrcSlotId; int32_t primarySrcSlotId;
@ -666,7 +652,6 @@ typedef struct SGroupbyOperatorInfo {
SAggSupporter aggSup; SAggSupporter aggSup;
SArray* pGroupCols; // group by columns, SArray<SColumn> SArray* pGroupCols; // group by columns, SArray<SColumn>
SArray* pGroupColVals; // current group column values, SArray<SGroupKeys> SArray* pGroupColVals; // current group column values, SArray<SGroupKeys>
SNode* pCondition;
bool isInit; // denote if current val is initialized or not bool isInit; // denote if current val is initialized or not
char* keyBuf; // group by keys for hash char* keyBuf; // group by keys for hash
int32_t groupKeyLen; // total group by column width int32_t groupKeyLen; // total group by column width
@ -716,7 +701,6 @@ typedef struct SSessionAggOperatorInfo {
int64_t gap; // session window gap int64_t gap; // session window gap
int32_t tsSlotId; // primary timestamp slot id int32_t tsSlotId; // primary timestamp slot id
STimeWindowAggSupp twAggSup; STimeWindowAggSupp twAggSup;
const SNode* pCondition;
} SSessionAggOperatorInfo; } SSessionAggOperatorInfo;
typedef struct SResultWindowInfo { typedef struct SResultWindowInfo {
@ -790,7 +774,6 @@ typedef struct SStreamFillOperatorInfo {
SSDataBlock* pSrcDelBlock; SSDataBlock* pSrcDelBlock;
int32_t srcDelRowIndex; int32_t srcDelRowIndex;
SSDataBlock* pDelRes; SSDataBlock* pDelRes;
SNode* pCondition;
SColMatchInfo matchInfo; SColMatchInfo matchInfo;
int32_t primaryTsCol; int32_t primaryTsCol;
int32_t primarySrcSlotId; int32_t primarySrcSlotId;
@ -827,7 +810,6 @@ typedef struct SStateWindowOperatorInfo {
SStateKeys stateKey; SStateKeys stateKey;
int32_t tsSlotId; // primary timestamp column slot id int32_t tsSlotId; // primary timestamp column slot id
STimeWindowAggSupp twAggSup; STimeWindowAggSupp twAggSup;
const SNode* pCondition;
} SStateWindowOperatorInfo; } SStateWindowOperatorInfo;
typedef struct SSortOperatorInfo { typedef struct SSortOperatorInfo {
@ -840,7 +822,6 @@ typedef struct SSortOperatorInfo {
int64_t startTs; // sort start time int64_t startTs; // sort start time
uint64_t sortElapsed; // sort elapsed time, time to flush to disk not included. uint64_t sortElapsed; // sort elapsed time, time to flush to disk not included.
SLimitInfo limitInfo; SLimitInfo limitInfo;
SNode* pCondition;
} SSortOperatorInfo; } SSortOperatorInfo;
typedef struct SJoinOperatorInfo { typedef struct SJoinOperatorInfo {
@ -901,7 +882,7 @@ int32_t getTableScanInfo(SOperatorInfo* pOperator, int32_t* order, int32_t* scan
int32_t getBufferPgSize(int32_t rowSize, uint32_t* defaultPgsz, uint32_t* defaultBufsz); int32_t getBufferPgSize(int32_t rowSize, uint32_t* defaultPgsz, uint32_t* defaultBufsz);
void doSetOperatorCompleted(SOperatorInfo* pOperator); void doSetOperatorCompleted(SOperatorInfo* pOperator);
void doFilter(const SNode* pFilterNode, SSDataBlock* pBlock, SColMatchInfo* pColMatchInfo, SFilterInfo* pFilterInfo); void doFilter(SSDataBlock* pBlock, SFilterInfo* pFilterInfo, SColMatchInfo* pColMatchInfo);
int32_t addTagPseudoColumnData(SReadHandle* pHandle, const SExprInfo* pExpr, int32_t numOfExpr, int32_t addTagPseudoColumnData(SReadHandle* pHandle, const SExprInfo* pExpr, int32_t numOfExpr,
SSDataBlock* pBlock, int32_t rows, const char* idStr, STableMetaCacheInfo * pCache); SSDataBlock* pBlock, int32_t rows, const char* idStr, STableMetaCacheInfo * pCache);

43
source/libs/executor/src/executorimpl.c
View File

@ -1039,39 +1039,24 @@ void setResultRowInitCtx(SResultRow* pResult, SqlFunctionCtx* pCtx, int32_t numO
static void extractQualifiedTupleByFilterResult(SSDataBlock* pBlock, const SColumnInfoData* p, bool keep, static void extractQualifiedTupleByFilterResult(SSDataBlock* pBlock, const SColumnInfoData* p, bool keep,
int32_t status); int32_t status);
void doFilter(const SNode* pFilterNode, SSDataBlock* pBlock, SColMatchInfo* pColMatchInfo, SFilterInfo* pFilterInfo) { void doFilter(SSDataBlock* pBlock, SFilterInfo* pFilterInfo, SColMatchInfo* pColMatchInfo) {
if (pFilterNode == NULL || pBlock->info.rows == 0) { if (pFilterInfo == NULL || pBlock->info.rows == 0) {
return; return;
} }
SFilterInfo* filter = pFilterInfo;
int64_t st = taosGetTimestampUs();
// todo move to the initialization function
int32_t code = 0;
bool needFree = false;
if (filter == NULL) {
needFree = true;
code = filterInitFromNode((SNode*)pFilterNode, &filter, 0);
}
SFilterColumnParam param1 = {.numOfCols = taosArrayGetSize(pBlock->pDataBlock), .pDataBlock = pBlock->pDataBlock}; SFilterColumnParam param1 = {.numOfCols = taosArrayGetSize(pBlock->pDataBlock), .pDataBlock = pBlock->pDataBlock};
code = filterSetDataFromSlotId(filter, &param1); int32_t code = filterSetDataFromSlotId(pFilterInfo, &param1);
SColumnInfoData* p = NULL; SColumnInfoData* p = NULL;
int32_t status = 0; int32_t status = 0;
// todo the keep seems never to be True?? // todo the keep seems never to be True??
bool keep = filterExecute(filter, pBlock, &p, NULL, param1.numOfCols, &status); bool keep = filterExecute(pFilterInfo, pBlock, &p, NULL, param1.numOfCols, &status);
if (needFree) {
filterFreeInfo(filter);
}
extractQualifiedTupleByFilterResult(pBlock, p, keep, status); extractQualifiedTupleByFilterResult(pBlock, p, keep, status);
if (pColMatchInfo != NULL) { if (pColMatchInfo != NULL) {
for (int32_t i = 0; i < taosArrayGetSize(pColMatchInfo->pList); ++i) { size_t size = taosArrayGetSize(pColMatchInfo->pList);
for (int32_t i = 0; i < size; ++i) {
SColMatchItem* pInfo = taosArrayGet(pColMatchInfo->pList, i); SColMatchItem* pInfo = taosArrayGet(pColMatchInfo->pList, i);
if (pInfo->colId == PRIMARYKEY_TIMESTAMP_COL_ID) { if (pInfo->colId == PRIMARYKEY_TIMESTAMP_COL_ID) {
SColumnInfoData* pColData = taosArrayGet(pBlock->pDataBlock, pInfo->dstSlotId); SColumnInfoData* pColData = taosArrayGet(pBlock->pDataBlock, pInfo->dstSlotId);
@ -2393,7 +2378,7 @@ static SSDataBlock* getAggregateResult(SOperatorInfo* pOperator) {
blockDataEnsureCapacity(pInfo->pRes, pOperator->resultInfo.capacity); blockDataEnsureCapacity(pInfo->pRes, pOperator->resultInfo.capacity);
while (1) { while (1) {
doBuildResultDatablock(pOperator, pInfo, &pAggInfo->groupResInfo, pAggInfo->aggSup.pResultBuf); doBuildResultDatablock(pOperator, pInfo, &pAggInfo->groupResInfo, pAggInfo->aggSup.pResultBuf);
doFilter(pAggInfo->pCondition, pInfo->pRes, NULL, NULL); doFilter(pInfo->pRes, pOperator->exprSupp.pFilterInfo, NULL);
if (!hasRemainResults(&pAggInfo->groupResInfo)) { if (!hasRemainResults(&pAggInfo->groupResInfo)) {
doSetOperatorCompleted(pOperator); doSetOperatorCompleted(pOperator);
@ -2726,7 +2711,7 @@ static SSDataBlock* doFill(SOperatorInfo* pOperator) {
break; break;
} }
doFilter(pInfo->pCondition, fillResult, &pInfo->matchInfo, NULL); doFilter(fillResult, pOperator->exprSupp.pFilterInfo, &pInfo->matchInfo );
if (fillResult->info.rows > 0) { if (fillResult->info.rows > 0) {
break; break;
} }
@ -2945,9 +2930,13 @@ SOperatorInfo* createAggregateOperatorInfo(SOperatorInfo* downstream, SAggPhysiN
goto _error; goto _error;
} }
code = filterInitFromNode((SNode*)pAggNode->node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
if (code != TSDB_CODE_SUCCESS) {
goto _error;
}
pInfo->binfo.mergeResultBlock = pAggNode->mergeDataBlock; pInfo->binfo.mergeResultBlock = pAggNode->mergeDataBlock;
pInfo->groupId = UINT64_MAX; pInfo->groupId = UINT64_MAX;
pInfo->pCondition = pAggNode->node.pConditions;
pOperator->name = "TableAggregate"; pOperator->name = "TableAggregate";
pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_HASH_AGG; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_HASH_AGG;
pOperator->blocking = true; pOperator->blocking = true;
@ -3173,7 +3162,11 @@ SOperatorInfo* createFillOperatorInfo(SOperatorInfo* downstream, SFillPhysiNode*
pInfo->pFinalRes = createOneDataBlock(pInfo->pRes, false); pInfo->pFinalRes = createOneDataBlock(pInfo->pRes, false);
blockDataEnsureCapacity(pInfo->pFinalRes, pOperator->resultInfo.capacity); blockDataEnsureCapacity(pInfo->pFinalRes, pOperator->resultInfo.capacity);
pInfo->pCondition = pPhyFillNode->node.pConditions; code = filterInitFromNode((SNode*)pPhyFillNode->node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
if (code != TSDB_CODE_SUCCESS) {
goto _error;
}
pOperator->name = "FillOperator"; pOperator->name = "FillOperator";
pOperator->blocking = false; pOperator->blocking = false;
pOperator->status = OP_NOT_OPENED; pOperator->status = OP_NOT_OPENED;

10
source/libs/executor/src/groupoperator.c
View File

@ -13,6 +13,7 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#include "filter.h"
#include "function.h" #include "function.h"
#include "os.h" #include "os.h"
#include "tname.h" #include "tname.h"
@ -312,7 +313,7 @@ static SSDataBlock* buildGroupResultDataBlock(SOperatorInfo* pOperator) {
SSDataBlock* pRes = pInfo->binfo.pRes; SSDataBlock* pRes = pInfo->binfo.pRes;
while (1) { while (1) {
doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf);
doFilter(pInfo->pCondition, pRes, NULL, NULL); doFilter(pRes, pOperator->exprSupp.pFilterInfo, NULL);
if (!hasRemainResults(&pInfo->groupResInfo)) { if (!hasRemainResults(&pInfo->groupResInfo)) {
doSetOperatorCompleted(pOperator); doSetOperatorCompleted(pOperator);
@ -411,8 +412,6 @@ SOperatorInfo* createGroupOperatorInfo(SOperatorInfo* downstream, SAggPhysiNode*
} }
pInfo->pGroupCols = extractColumnInfo(pAggNode->pGroupKeys); pInfo->pGroupCols = extractColumnInfo(pAggNode->pGroupKeys);
pInfo->pCondition = pAggNode->node.pConditions;
int32_t code = initExprSupp(&pInfo->scalarSup, pScalarExprInfo, numOfScalarExpr); int32_t code = initExprSupp(&pInfo->scalarSup, pScalarExprInfo, numOfScalarExpr);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
goto _error; goto _error;
@ -433,6 +432,11 @@ SOperatorInfo* createGroupOperatorInfo(SOperatorInfo* downstream, SAggPhysiNode*
goto _error; goto _error;
} }
code = filterInitFromNode((SNode*)pAggNode->node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
if (code != TSDB_CODE_SUCCESS) {
goto _error;
}
initResultRowInfo(&pInfo->binfo.resultRowInfo); initResultRowInfo(&pInfo->binfo.resultRowInfo);
pOperator->name = "GroupbyAggOperator"; pOperator->name = "GroupbyAggOperator";

10
source/libs/executor/src/joinoperator.c
View File

@ -13,6 +13,7 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#include "filter.h"
#include "executorimpl.h" #include "executorimpl.h"
#include "function.h" #include "function.h"
#include "os.h" #include "os.h"
@ -108,6 +109,11 @@ SOperatorInfo* createMergeJoinOperatorInfo(SOperatorInfo** pDownstream, int32_t
pInfo->pCondAfterMerge = NULL; pInfo->pCondAfterMerge = NULL;
} }
code = filterInitFromNode(pInfo->pCondAfterMerge, &pOperator->exprSupp.pFilterInfo, 0);
if (code != TSDB_CODE_SUCCESS) {
goto _error;
}
pInfo->inputOrder = TSDB_ORDER_ASC; pInfo->inputOrder = TSDB_ORDER_ASC;
if (pJoinNode->inputTsOrder == ORDER_ASC) { if (pJoinNode->inputTsOrder == ORDER_ASC) {
pInfo->inputOrder = TSDB_ORDER_ASC; pInfo->inputOrder = TSDB_ORDER_ASC;
@ -400,8 +406,8 @@ SSDataBlock* doMergeJoin(struct SOperatorInfo* pOperator) {
if (numOfNewRows == 0) { if (numOfNewRows == 0) {
break; break;
} }
if (pJoinInfo->pCondAfterMerge != NULL) { if (pOperator->exprSupp.pFilterInfo != NULL) {
doFilter(pJoinInfo->pCondAfterMerge, pRes, NULL, NULL); doFilter(pRes, pOperator->exprSupp.pFilterInfo, NULL);
} }
if (pRes->info.rows >= pOperator->resultInfo.threshold) { if (pRes->info.rows >= pOperator->resultInfo.threshold) {
break; break;

29
source/libs/executor/src/projectoperator.c
View File

@ -13,6 +13,7 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#include "filter.h"
#include "executorimpl.h" #include "executorimpl.h"
#include "functionMgt.h" #include "functionMgt.h"
@ -71,13 +72,7 @@ SOperatorInfo* createProjectOperatorInfo(SOperatorInfo* downstream, SProjectPhys
pInfo->binfo.pRes = pResBlock; pInfo->binfo.pRes = pResBlock;
pInfo->pFinalRes = createOneDataBlock(pResBlock, false); pInfo->pFinalRes = createOneDataBlock(pResBlock, false);
pInfo->pFilterNode = pProjPhyNode->node.pConditions; pInfo->mergeDataBlocks = (pTaskInfo->execModel == OPTR_EXEC_MODEL_STREAM)? false:pProjPhyNode->mergeDataBlock;
if (pTaskInfo->execModel == OPTR_EXEC_MODEL_STREAM) {
pInfo->mergeDataBlocks = false;
} else {
pInfo->mergeDataBlocks = pProjPhyNode->mergeDataBlock;
}
int32_t numOfRows = 4096; int32_t numOfRows = 4096;
size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES;
@ -97,6 +92,11 @@ SOperatorInfo* createProjectOperatorInfo(SOperatorInfo* downstream, SProjectPhys
initBasicInfo(&pInfo->binfo, pResBlock); initBasicInfo(&pInfo->binfo, pResBlock);
setFunctionResultOutput(pOperator, &pInfo->binfo, &pInfo->aggSup, MAIN_SCAN, numOfCols); setFunctionResultOutput(pOperator, &pInfo->binfo, &pInfo->aggSup, MAIN_SCAN, numOfCols);
code = filterInitFromNode((SNode*)pProjPhyNode->node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
if (code != TSDB_CODE_SUCCESS) {
goto _error;
}
pInfo->pPseudoColInfo = setRowTsColumnOutputInfo(pOperator->exprSupp.pCtx, numOfCols); pInfo->pPseudoColInfo = setRowTsColumnOutputInfo(pOperator->exprSupp.pCtx, numOfCols);
pOperator->name = "ProjectOperator"; pOperator->name = "ProjectOperator";
pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_PROJECT; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_PROJECT;
@ -313,7 +313,7 @@ SSDataBlock* doProjectOperation(SOperatorInfo* pOperator) {
} }
// do apply filter // do apply filter
doFilter(pProjectInfo->pFilterNode, pFinalRes, NULL, NULL); doFilter(pFinalRes, pOperator->exprSupp.pFilterInfo, NULL);
// when apply the limit/offset for each group, pRes->info.rows may be 0, due to limit constraint. // when apply the limit/offset for each group, pRes->info.rows may be 0, due to limit constraint.
if (pFinalRes->info.rows > 0 || (pOperator->status == OP_EXEC_DONE)) { if (pFinalRes->info.rows > 0 || (pOperator->status == OP_EXEC_DONE)) {
@ -323,7 +323,7 @@ SSDataBlock* doProjectOperation(SOperatorInfo* pOperator) {
} else { } else {
// do apply filter // do apply filter
if (pRes->info.rows > 0) { if (pRes->info.rows > 0) {
doFilter(pProjectInfo->pFilterNode, pRes, NULL, NULL); doFilter(pRes, pOperator->exprSupp.pFilterInfo, NULL);
if (pRes->info.rows == 0) { if (pRes->info.rows == 0) {
continue; continue;
} }
@ -392,7 +392,12 @@ SOperatorInfo* createIndefinitOutputOperatorInfo(SOperatorInfo* downstream, SPhy
} }
setFunctionResultOutput(pOperator, &pInfo->binfo, &pInfo->aggSup, MAIN_SCAN, numOfExpr); setFunctionResultOutput(pOperator, &pInfo->binfo, &pInfo->aggSup, MAIN_SCAN, numOfExpr);
pInfo->pCondition = pPhyNode->node.pConditions; code = filterInitFromNode((SNode*)pPhyNode->node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
if (code != TSDB_CODE_SUCCESS) {
goto _error;
}
pInfo->binfo.pRes = pResBlock;
pInfo->pPseudoColInfo = setRowTsColumnOutputInfo(pSup->pCtx, numOfExpr); pInfo->pPseudoColInfo = setRowTsColumnOutputInfo(pSup->pCtx, numOfExpr);
pOperator->name = "IndefinitOperator"; pOperator->name = "IndefinitOperator";
@ -515,7 +520,7 @@ SSDataBlock* doApplyIndefinitFunction(SOperatorInfo* pOperator) {
} }
} }
doFilter(pIndefInfo->pCondition, pInfo->pRes, NULL, NULL); doFilter(pInfo->pRes, pOperator->exprSupp.pFilterInfo, NULL);
size_t rows = pInfo->pRes->info.rows; size_t rows = pInfo->pRes->info.rows;
if (rows > 0 || pOperator->status == OP_EXEC_DONE) { if (rows > 0 || pOperator->status == OP_EXEC_DONE) {
break; break;
@ -617,7 +622,7 @@ SSDataBlock* doGenerateSourceData(SOperatorInfo* pOperator) {
} }
pRes->info.rows = 1; pRes->info.rows = 1;
doFilter(pProjectInfo->pFilterNode, pRes, NULL, NULL); doFilter(pRes, pOperator->exprSupp.pFilterInfo, NULL);
/*int32_t status = */ doIngroupLimitOffset(&pProjectInfo->limitInfo, 0, pRes, pOperator); /*int32_t status = */ doIngroupLimitOffset(&pProjectInfo->limitInfo, 0, pRes, pOperator);

109
source/libs/executor/src/scanoperator.c
View File

@ -116,8 +116,10 @@ static int32_t sysTableUserTagsFillOneTableTags(const SSysTableScanInfo* pInfo,
SMetaReader* smrChildTable, const char* dbname, const char* tableName, SMetaReader* smrChildTable, const char* dbname, const char* tableName,
int32_t* pNumOfRows, const SSDataBlock* dataBlock); int32_t* pNumOfRows, const SSDataBlock* dataBlock);
static void relocateAndFilterSysTagsScanResult(SSysTableScanInfo* pInfo, int32_t numOfRows, SSDataBlock* dataBlock); static void relocateAndFilterSysTagsScanResult(SSysTableScanInfo* pInfo, int32_t numOfRows, SSDataBlock* dataBlock,
bool processBlockWithProbability(const SSampleExecInfo* pInfo) { SFilterInfo* pFilterInfo);
bool processBlockWithProbability(const SSampleExecInfo* pInfo) {
#if 0 #if 0
if (pInfo->sampleRatio == 1) { if (pInfo->sampleRatio == 1) {
return true; return true;
@ -281,19 +283,13 @@ static int32_t doDynamicPruneDataBlock(SOperatorInfo* pOperator, SDataBlockInfo*
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static bool doFilterByBlockSMA(const SNode* pFilterNode, SColumnDataAgg** pColsAgg, int32_t numOfCols, static bool doFilterByBlockSMA(SFilterInfo* pFilterInfo, SColumnDataAgg** pColsAgg, int32_t numOfCols,
int32_t numOfRows) { int32_t numOfRows) {
if (pColsAgg == NULL || pFilterNode == NULL) { if (pColsAgg == NULL || pFilterInfo == NULL) {
return true; return true;
} }
SFilterInfo* filter = NULL; bool keep = filterRangeExecute(pFilterInfo, pColsAgg, numOfCols, numOfRows);
// todo move to the initialization function
int32_t code = filterInitFromNode((SNode*)pFilterNode, &filter, 0);
bool keep = filterRangeExecute(filter, pColsAgg, numOfCols, numOfRows);
filterFreeInfo(filter);
return keep; return keep;
} }
@ -387,7 +383,7 @@ static int32_t loadDataBlock(SOperatorInfo* pOperator, STableScanInfo* pTableSca
bool loadSMA = false; bool loadSMA = false;
*status = pInfo->dataBlockLoadFlag; *status = pInfo->dataBlockLoadFlag;
if (pTableScanInfo->pFilterNode != NULL || if (pOperator->exprSupp.pFilterInfo != NULL ||
overlapWithTimeWindow(&pTableScanInfo->pdInfo.interval, &pBlock->info, pTableScanInfo->cond.order)) { overlapWithTimeWindow(&pTableScanInfo->pdInfo.interval, &pBlock->info, pTableScanInfo->cond.order)) {
(*status) = FUNC_DATA_REQUIRED_DATA_LOAD; (*status) = FUNC_DATA_REQUIRED_DATA_LOAD;
} }
@ -425,11 +421,11 @@ static int32_t loadDataBlock(SOperatorInfo* pOperator, STableScanInfo* pTableSca
ASSERT(*status == FUNC_DATA_REQUIRED_DATA_LOAD); ASSERT(*status == FUNC_DATA_REQUIRED_DATA_LOAD);
// try to filter data block according to sma info // try to filter data block according to sma info
if (pTableScanInfo->pFilterNode != NULL && (!loadSMA)) { if (pOperator->exprSupp.pFilterInfo != NULL && (!loadSMA)) {
bool success = doLoadBlockSMA(pTableScanInfo, pBlock, pTaskInfo); bool success = doLoadBlockSMA(pTableScanInfo, pBlock, pTaskInfo);
if (success) { if (success) {
size_t size = taosArrayGetSize(pBlock->pDataBlock); size_t size = taosArrayGetSize(pBlock->pDataBlock);
bool keep = doFilterByBlockSMA(pTableScanInfo->pFilterNode, pBlock->pBlockAgg, size, pBlockInfo->rows); bool keep = doFilterByBlockSMA(pOperator->exprSupp.pFilterInfo, pBlock->pBlockAgg, size, pBlockInfo->rows);
if (!keep) { if (!keep) {
qDebug("%s data block filter out by block SMA, brange:%" PRId64 "-%" PRId64 ", rows:%d", GET_TASKID(pTaskInfo), qDebug("%s data block filter out by block SMA, brange:%" PRId64 "-%" PRId64 ", rows:%d", GET_TASKID(pTaskInfo),
pBlockInfo->window.skey, pBlockInfo->window.ekey, pBlockInfo->rows); pBlockInfo->window.skey, pBlockInfo->window.ekey, pBlockInfo->rows);
@ -469,9 +465,9 @@ static int32_t loadDataBlock(SOperatorInfo* pOperator, STableScanInfo* pTableSca
// restore the previous value // restore the previous value
pCost->totalRows -= pBlock->info.rows; pCost->totalRows -= pBlock->info.rows;
if (pTableScanInfo->pFilterNode != NULL) { if (pOperator->exprSupp.pFilterInfo != NULL) {
int64_t st = taosGetTimestampUs(); int64_t st = taosGetTimestampUs();
doFilter(pTableScanInfo->pFilterNode, pBlock, &pTableScanInfo->matchInfo, pOperator->exprSupp.pFilterInfo); doFilter(pBlock, pOperator->exprSupp.pFilterInfo, &pTableScanInfo->matchInfo);
double el = (taosGetTimestampUs() - st) / 1000.0; double el = (taosGetTimestampUs() - st) / 1000.0;
pTableScanInfo->readRecorder.filterTime += el; pTableScanInfo->readRecorder.filterTime += el;
@ -936,12 +932,9 @@ SOperatorInfo* createTableScanOperatorInfo(STableScanPhysiNode* pTableScanNode,
pInfo->pResBlock = createResDataBlock(pDescNode); pInfo->pResBlock = createResDataBlock(pDescNode);
blockDataEnsureCapacity(pInfo->pResBlock, pOperator->resultInfo.capacity); blockDataEnsureCapacity(pInfo->pResBlock, pOperator->resultInfo.capacity);
pInfo->pFilterNode = pScanNode->node.pConditions; code = filterInitFromNode((SNode*)pTableScanNode->scan.node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
if (pInfo->pFilterNode != NULL) { if (code != TSDB_CODE_SUCCESS) {
code = filterInitFromNode((SNode*)pInfo->pFilterNode, &pOperator->exprSupp.pFilterInfo, 0); goto _error;
if (code != TSDB_CODE_SUCCESS) {
goto _error;
}
} }
pInfo->scanFlag = MAIN_SCAN; pInfo->scanFlag = MAIN_SCAN;
@ -1372,7 +1365,7 @@ static SSDataBlock* doRangeScan(SStreamScanInfo* pInfo, SSDataBlock* pSDB, int32
return NULL; return NULL;
} }
doFilter(pInfo->pCondition, pResult, NULL, NULL); doFilter(pResult, pInfo->pTableScanOp->exprSupp.pFilterInfo, NULL);
if (pResult->info.rows == 0) { if (pResult->info.rows == 0) {
continue; continue;
} }
@ -1716,7 +1709,7 @@ static int32_t setBlockIntoRes(SStreamScanInfo* pInfo, const SSDataBlock* pBlock
} }
if (filter) { if (filter) {
doFilter(pInfo->pCondition, pInfo->pRes, NULL, NULL); doFilter(pInfo->pRes, pOperator->exprSupp.pFilterInfo, NULL);
} }
blockDataUpdateTsWindow(pInfo->pRes, pInfo->primaryTsIndex); blockDataUpdateTsWindow(pInfo->pRes, pInfo->primaryTsIndex);
@ -2168,7 +2161,7 @@ FETCH_NEXT_BLOCK:
} }
} }
doFilter(pInfo->pCondition, pInfo->pRes, NULL, NULL); doFilter(pInfo->pRes, pOperator->exprSupp.pFilterInfo, NULL);
blockDataUpdateTsWindow(pInfo->pRes, pInfo->primaryTsIndex); blockDataUpdateTsWindow(pInfo->pRes, pInfo->primaryTsIndex);
if (pBlockInfo->rows > 0 || pInfo->pUpdateDataRes->info.rows > 0) { if (pBlockInfo->rows > 0 || pInfo->pUpdateDataRes->info.rows > 0) {
@ -2533,9 +2526,13 @@ SOperatorInfo* createStreamScanOperatorInfo(SReadHandle* pHandle, STableScanPhys
pInfo->pPseudoExpr = createExprInfo(pTableScanNode->scan.pScanPseudoCols, NULL, &pInfo->numOfPseudoExpr); pInfo->pPseudoExpr = createExprInfo(pTableScanNode->scan.pScanPseudoCols, NULL, &pInfo->numOfPseudoExpr);
} }
code = filterInitFromNode((SNode*)pScanPhyNode->node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
if (code != TSDB_CODE_SUCCESS) {
goto _error;
}
pInfo->pRes = createResDataBlock(pDescNode); pInfo->pRes = createResDataBlock(pDescNode);
pInfo->pUpdateRes = createSpecialDataBlock(STREAM_CLEAR); pInfo->pUpdateRes = createSpecialDataBlock(STREAM_CLEAR);
pInfo->pCondition = pScanPhyNode->node.pConditions;
pInfo->scanMode = STREAM_SCAN_FROM_READERHANDLE; pInfo->scanMode = STREAM_SCAN_FROM_READERHANDLE;
pInfo->windowSup = (SWindowSupporter){.pStreamAggSup = NULL, .gap = -1, .parentType = QUERY_NODE_PHYSICAL_PLAN}; pInfo->windowSup = (SWindowSupporter){.pStreamAggSup = NULL, .gap = -1, .parentType = QUERY_NODE_PHYSICAL_PLAN};
pInfo->groupId = 0; pInfo->groupId = 0;
@ -2676,13 +2673,13 @@ static int32_t loadSysTableCallback(void* param, SDataBuf* pMsg, int32_t code) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static SSDataBlock* doFilterResult(SSysTableScanInfo* pInfo) { static SSDataBlock* doFilterResult(SSDataBlock* pDataBlock, SFilterInfo* pFilterInfo) {
if (pInfo->pCondition == NULL) { if (pFilterInfo == NULL) {
return pInfo->pRes->info.rows == 0 ? NULL : pInfo->pRes; return pDataBlock->info.rows == 0 ? NULL : pDataBlock;
} }
doFilter(pInfo->pCondition, pInfo->pRes, NULL, NULL); doFilter(pDataBlock, pFilterInfo, NULL);
return pInfo->pRes->info.rows == 0 ? NULL : pInfo->pRes; return pDataBlock->info.rows == 0 ? NULL : pDataBlock;
} }
static SSDataBlock* buildInfoSchemaTableMetaBlock(char* tableName) { static SSDataBlock* buildInfoSchemaTableMetaBlock(char* tableName) {
@ -2887,7 +2884,7 @@ static SSDataBlock* sysTableScanUserTags(SOperatorInfo* pOperator) {
metaReaderClear(&smrSuperTable); metaReaderClear(&smrSuperTable);
metaReaderClear(&smrChildTable); metaReaderClear(&smrChildTable);
if (numOfRows > 0) { if (numOfRows > 0) {
relocateAndFilterSysTagsScanResult(pInfo, numOfRows, dataBlock); relocateAndFilterSysTagsScanResult(pInfo, numOfRows, dataBlock, pOperator->exprSupp.pFilterInfo);
numOfRows = 0; numOfRows = 0;
} }
blockDataDestroy(dataBlock); blockDataDestroy(dataBlock);
@ -2927,7 +2924,7 @@ static SSDataBlock* sysTableScanUserTags(SOperatorInfo* pOperator) {
metaReaderClear(&smrSuperTable); metaReaderClear(&smrSuperTable);
if (numOfRows >= pOperator->resultInfo.capacity) { if (numOfRows >= pOperator->resultInfo.capacity) {
relocateAndFilterSysTagsScanResult(pInfo, numOfRows, dataBlock); relocateAndFilterSysTagsScanResult(pInfo, numOfRows, dataBlock, pOperator->exprSupp.pFilterInfo);
numOfRows = 0; numOfRows = 0;
if (pInfo->pRes->info.rows > 0) { if (pInfo->pRes->info.rows > 0) {
@ -2937,7 +2934,7 @@ static SSDataBlock* sysTableScanUserTags(SOperatorInfo* pOperator) {
} }
if (numOfRows > 0) { if (numOfRows > 0) {
relocateAndFilterSysTagsScanResult(pInfo, numOfRows, dataBlock); relocateAndFilterSysTagsScanResult(pInfo, numOfRows, dataBlock, pOperator->exprSupp.pFilterInfo);
numOfRows = 0; numOfRows = 0;
} }
@ -2952,13 +2949,13 @@ static SSDataBlock* sysTableScanUserTags(SOperatorInfo* pOperator) {
return (pInfo->pRes->info.rows == 0) ? NULL : pInfo->pRes; return (pInfo->pRes->info.rows == 0) ? NULL : pInfo->pRes;
} }
static void relocateAndFilterSysTagsScanResult(SSysTableScanInfo* pInfo, int32_t numOfRows, SSDataBlock* dataBlock) { void relocateAndFilterSysTagsScanResult(SSysTableScanInfo* pInfo, int32_t numOfRows, SSDataBlock* dataBlock,
SFilterInfo* pFilterInfo) {
dataBlock->info.rows = numOfRows; dataBlock->info.rows = numOfRows;
pInfo->pRes->info.rows = numOfRows; pInfo->pRes->info.rows = numOfRows;
relocateColumnData(pInfo->pRes, pInfo->matchInfo.pList, dataBlock->pDataBlock, false); relocateColumnData(pInfo->pRes, pInfo->matchInfo.pList, dataBlock->pDataBlock, false);
doFilterResult(pInfo); doFilterResult(pInfo->pRes, pFilterInfo);
blockDataCleanup(dataBlock); blockDataCleanup(dataBlock);
} }
@ -3712,7 +3709,7 @@ static SSDataBlock* sysTableBuildUserTablesByUids(SOperatorInfo* pOperator) {
pInfo->pRes->info.rows = numOfRows; pInfo->pRes->info.rows = numOfRows;
relocateColumnData(pInfo->pRes, pInfo->matchInfo.pList, p->pDataBlock, false); relocateColumnData(pInfo->pRes, pInfo->matchInfo.pList, p->pDataBlock, false);
doFilterResult(pInfo); doFilterResult(pInfo->pRes, pOperator->exprSupp.pFilterInfo);
blockDataCleanup(p); blockDataCleanup(p);
numOfRows = 0; numOfRows = 0;
@ -3728,7 +3725,7 @@ static SSDataBlock* sysTableBuildUserTablesByUids(SOperatorInfo* pOperator) {
pInfo->pRes->info.rows = numOfRows; pInfo->pRes->info.rows = numOfRows;
relocateColumnData(pInfo->pRes, pInfo->matchInfo.pList, p->pDataBlock, false); relocateColumnData(pInfo->pRes, pInfo->matchInfo.pList, p->pDataBlock, false);
doFilterResult(pInfo); doFilterResult(pInfo->pRes, pOperator->exprSupp.pFilterInfo);
blockDataCleanup(p); blockDataCleanup(p);
numOfRows = 0; numOfRows = 0;
@ -3745,6 +3742,7 @@ static SSDataBlock* sysTableBuildUserTablesByUids(SOperatorInfo* pOperator) {
pInfo->loadInfo.totalRows += pInfo->pRes->info.rows; pInfo->loadInfo.totalRows += pInfo->pRes->info.rows;
return (pInfo->pRes->info.rows == 0) ? NULL : pInfo->pRes; return (pInfo->pRes->info.rows == 0) ? NULL : pInfo->pRes;
} }
static SSDataBlock* sysTableBuildUserTables(SOperatorInfo* pOperator) { static SSDataBlock* sysTableBuildUserTables(SOperatorInfo* pOperator) {
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
@ -3888,7 +3886,7 @@ static SSDataBlock* sysTableBuildUserTables(SOperatorInfo* pOperator) {
pInfo->pRes->info.rows = numOfRows; pInfo->pRes->info.rows = numOfRows;
relocateColumnData(pInfo->pRes, pInfo->matchInfo.pList, p->pDataBlock, false); relocateColumnData(pInfo->pRes, pInfo->matchInfo.pList, p->pDataBlock, false);
doFilterResult(pInfo); doFilterResult(pInfo->pRes, pOperator->exprSupp.pFilterInfo);
blockDataCleanup(p); blockDataCleanup(p);
numOfRows = 0; numOfRows = 0;
@ -3904,7 +3902,7 @@ static SSDataBlock* sysTableBuildUserTables(SOperatorInfo* pOperator) {
pInfo->pRes->info.rows = numOfRows; pInfo->pRes->info.rows = numOfRows;
relocateColumnData(pInfo->pRes, pInfo->matchInfo.pList, p->pDataBlock, false); relocateColumnData(pInfo->pRes, pInfo->matchInfo.pList, p->pDataBlock, false);
doFilterResult(pInfo); doFilterResult(pInfo->pRes, pOperator->exprSupp.pFilterInfo);
blockDataCleanup(p); blockDataCleanup(p);
numOfRows = 0; numOfRows = 0;
@ -3935,8 +3933,7 @@ static SSDataBlock* sysTableScanUserTables(SOperatorInfo* pOperator) {
// the retrieve is executed on the mnode, so return tables that belongs to the information schema database. // the retrieve is executed on the mnode, so return tables that belongs to the information schema database.
if (pInfo->readHandle.mnd != NULL) { if (pInfo->readHandle.mnd != NULL) {
buildSysDbTableInfo(pInfo, pOperator->resultInfo.capacity); buildSysDbTableInfo(pInfo, pOperator->resultInfo.capacity);
doFilterResult(pInfo->pRes, pOperator->exprSupp.pFilterInfo);
doFilterResult(pInfo);
pInfo->loadInfo.totalRows += pInfo->pRes->info.rows; pInfo->loadInfo.totalRows += pInfo->pRes->info.rows;
doSetOperatorCompleted(pOperator); doSetOperatorCompleted(pOperator);
@ -4071,7 +4068,7 @@ static SSDataBlock* doSysTableScan(SOperatorInfo* pOperator) {
updateLoadRemoteInfo(&pInfo->loadInfo, pRsp->numOfRows, pRsp->compLen, startTs, pOperator); updateLoadRemoteInfo(&pInfo->loadInfo, pRsp->numOfRows, pRsp->compLen, startTs, pOperator);
// todo log the filter info // todo log the filter info
doFilterResult(pInfo); doFilterResult(pInfo->pRes, pOperator->exprSupp.pFilterInfo);
taosMemoryFree(pRsp); taosMemoryFree(pRsp);
if (pInfo->pRes->info.rows > 0) { if (pInfo->pRes->info.rows > 0) {
return pInfo->pRes; return pInfo->pRes;
@ -4169,9 +4166,15 @@ SOperatorInfo* createSysTableScanOperatorInfo(void* readHandle, SSystemTableScan
pInfo->sysInfo = pScanPhyNode->sysInfo; pInfo->sysInfo = pScanPhyNode->sysInfo;
pInfo->showRewrite = pScanPhyNode->showRewrite; pInfo->showRewrite = pScanPhyNode->showRewrite;
pInfo->pRes = createResDataBlock(pDescNode); pInfo->pRes = createResDataBlock(pDescNode);
pInfo->pCondition = pScanNode->node.pConditions; pInfo->pCondition = pScanNode->node.pConditions;
code = filterInitFromNode(pScanNode->node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
if (code != TSDB_CODE_SUCCESS) {
goto _error;
}
initResultSizeInfo(&pOperator->resultInfo, 4096); initResultSizeInfo(&pOperator->resultInfo, 4096);
blockDataEnsureCapacity(pInfo->pRes, pOperator->resultInfo.capacity);
tNameAssign(&pInfo->name, &pScanNode->tableName); tNameAssign(&pInfo->name, &pScanNode->tableName);
const char* name = tNameGetTableName(&pInfo->name); const char* name = tNameGetTableName(&pInfo->name);
@ -4179,7 +4182,6 @@ SOperatorInfo* createSysTableScanOperatorInfo(void* readHandle, SSystemTableScan
if (strncasecmp(name, TSDB_INS_TABLE_TABLES, TSDB_TABLE_FNAME_LEN) == 0 || if (strncasecmp(name, TSDB_INS_TABLE_TABLES, TSDB_TABLE_FNAME_LEN) == 0 ||
strncasecmp(name, TSDB_INS_TABLE_TAGS, TSDB_TABLE_FNAME_LEN) == 0) { strncasecmp(name, TSDB_INS_TABLE_TAGS, TSDB_TABLE_FNAME_LEN) == 0) {
pInfo->readHandle = *(SReadHandle*)readHandle; pInfo->readHandle = *(SReadHandle*)readHandle;
blockDataEnsureCapacity(pInfo->pRes, pOperator->resultInfo.capacity);
} else { } else {
tsem_init(&pInfo->ready, 0, 0); tsem_init(&pInfo->ready, 0, 0);
pInfo->epSet = pScanPhyNode->mgmtEpSet; pInfo->epSet = pScanPhyNode->mgmtEpSet;
@ -4195,7 +4197,6 @@ SOperatorInfo* createSysTableScanOperatorInfo(void* readHandle, SSystemTableScan
pOperator->pTaskInfo = pTaskInfo; pOperator->pTaskInfo = pTaskInfo;
pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doSysTableScan, NULL, NULL, destroySysScanOperator, NULL); pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doSysTableScan, NULL, NULL, destroySysScanOperator, NULL);
return pOperator; return pOperator;
_error: _error:
@ -4203,7 +4204,7 @@ _error:
destroySysScanOperator(pInfo); destroySysScanOperator(pInfo);
} }
taosMemoryFreeClear(pOperator); taosMemoryFreeClear(pOperator);
terrno = TSDB_CODE_QRY_OUT_OF_MEMORY; pTaskInfo->code = code;
return NULL; return NULL;
} }
@ -4357,7 +4358,7 @@ static int32_t loadDataBlockFromOneTable(SOperatorInfo* pOperator, STableMergeSc
pCost->totalRows += pBlock->info.rows; pCost->totalRows += pBlock->info.rows;
*status = pInfo->dataBlockLoadFlag; *status = pInfo->dataBlockLoadFlag;
if (pTableScanInfo->pFilterNode != NULL || if (pOperator->exprSupp.pFilterInfo != NULL ||
overlapWithTimeWindow(&pTableScanInfo->interval, &pBlock->info, pTableScanInfo->cond.order)) { overlapWithTimeWindow(&pTableScanInfo->interval, &pBlock->info, pTableScanInfo->cond.order)) {
(*status) = FUNC_DATA_REQUIRED_DATA_LOAD; (*status) = FUNC_DATA_REQUIRED_DATA_LOAD;
} }
@ -4443,9 +4444,9 @@ static int32_t loadDataBlockFromOneTable(SOperatorInfo* pOperator, STableMergeSc
T_LONG_JMP(pTaskInfo->env, code); T_LONG_JMP(pTaskInfo->env, code);
} }
if (pTableScanInfo->pFilterNode != NULL) { if (pOperator->exprSupp.pFilterInfo!= NULL) {
int64_t st = taosGetTimestampMs(); int64_t st = taosGetTimestampMs();
doFilter(pTableScanInfo->pFilterNode, pBlock, &pTableScanInfo->matchInfo, NULL); doFilter(pBlock, pOperator->exprSupp.pFilterInfo, &pTableScanInfo->matchInfo);
double el = (taosGetTimestampUs() - st) / 1000.0; double el = (taosGetTimestampUs() - st) / 1000.0;
pTableScanInfo->readRecorder.filterTime += el; pTableScanInfo->readRecorder.filterTime += el;
@ -4818,7 +4819,13 @@ SOperatorInfo* createTableMergeScanOperatorInfo(STableScanPhysiNode* pTableScanN
pInfo->sample.sampleRatio = pTableScanNode->ratio; pInfo->sample.sampleRatio = pTableScanNode->ratio;
pInfo->sample.seed = taosGetTimestampSec(); pInfo->sample.seed = taosGetTimestampSec();
pInfo->dataBlockLoadFlag = pTableScanNode->dataRequired; pInfo->dataBlockLoadFlag = pTableScanNode->dataRequired;
pInfo->pFilterNode = pTableScanNode->scan.node.pConditions;
code = filterInitFromNode((SNode*)pTableScanNode->scan.node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
if (code != TSDB_CODE_SUCCESS) {
goto _error;
}
pInfo->tableListInfo = pTableListInfo; pInfo->tableListInfo = pTableListInfo;
pInfo->scanFlag = MAIN_SCAN; pInfo->scanFlag = MAIN_SCAN;

9
source/libs/executor/src/sortoperator.c
View File

@ -13,6 +13,7 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#include "filter.h"
#include "executorimpl.h" #include "executorimpl.h"
#include "tdatablock.h" #include "tdatablock.h"
@ -42,12 +43,14 @@ SOperatorInfo* createSortOperatorInfo(SOperatorInfo* downstream, SSortPhysiNode*
extractColMatchInfo(pSortNode->pTargets, pDescNode, &numOfOutputCols, COL_MATCH_FROM_SLOT_ID, &pInfo->matchInfo); extractColMatchInfo(pSortNode->pTargets, pDescNode, &numOfOutputCols, COL_MATCH_FROM_SLOT_ID, &pInfo->matchInfo);
pOperator->exprSupp.pCtx = createSqlFunctionCtx(pExprInfo, numOfCols, &pOperator->exprSupp.rowEntryInfoOffset); pOperator->exprSupp.pCtx = createSqlFunctionCtx(pExprInfo, numOfCols, &pOperator->exprSupp.rowEntryInfoOffset);
initResultSizeInfo(&pOperator->resultInfo, 1024); initResultSizeInfo(&pOperator->resultInfo, 1024);
code = filterInitFromNode((SNode*)pSortNode->node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
if (code != TSDB_CODE_SUCCESS) {
goto _error;
}
pInfo->binfo.pRes = pResBlock; pInfo->binfo.pRes = pResBlock;
pInfo->pSortInfo = createSortInfo(pSortNode->pSortKeys); pInfo->pSortInfo = createSortInfo(pSortNode->pSortKeys);
pInfo->pCondition = pSortNode->node.pConditions;
initLimitInfo(pSortNode->node.pLimit, pSortNode->node.pSlimit, &pInfo->limitInfo); initLimitInfo(pSortNode->node.pLimit, pSortNode->node.pSlimit, &pInfo->limitInfo);
pOperator->name = "SortOperator"; pOperator->name = "SortOperator";
@ -215,7 +218,7 @@ SSDataBlock* doSort(SOperatorInfo* pOperator) {
return NULL; return NULL;
} }
doFilter(pInfo->pCondition, pBlock, &pInfo->matchInfo, NULL); doFilter(pBlock, pOperator->exprSupp.pFilterInfo, &pInfo->matchInfo);
if (blockDataGetNumOfRows(pBlock) == 0) { if (blockDataGetNumOfRows(pBlock) == 0) {
continue; continue;
} }

10
source/libs/executor/src/tfill.c
View File

@ -13,6 +13,7 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#include "filter.h"
#include "os.h" #include "os.h"
#include "query.h" #include "query.h"
#include "taosdef.h" #include "taosdef.h"
@ -1499,7 +1500,7 @@ static SSDataBlock* doStreamFill(SOperatorInfo* pOperator) {
} }
doStreamFillImpl(pOperator); doStreamFillImpl(pOperator);
doFilter(pInfo->pCondition, pInfo->pRes, &pInfo->matchInfo, NULL); doFilter(pInfo->pRes, pOperator->exprSupp.pFilterInfo, &pInfo->matchInfo);
memcpy(pInfo->pRes->info.parTbName, pInfo->pSrcBlock->info.parTbName, TSDB_TABLE_NAME_LEN); memcpy(pInfo->pRes->info.parTbName, pInfo->pSrcBlock->info.parTbName, TSDB_TABLE_NAME_LEN);
pOperator->resultInfo.totalRows += pInfo->pRes->info.rows; pOperator->resultInfo.totalRows += pInfo->pRes->info.rows;
if (pInfo->pRes->info.rows > 0) { if (pInfo->pRes->info.rows > 0) {
@ -1677,7 +1678,12 @@ SOperatorInfo* createStreamFillOperatorInfo(SOperatorInfo* downstream, SStreamFi
int32_t numOfOutputCols = 0; int32_t numOfOutputCols = 0;
int32_t code = extractColMatchInfo(pPhyFillNode->pFillExprs, pPhyFillNode->node.pOutputDataBlockDesc, int32_t code = extractColMatchInfo(pPhyFillNode->pFillExprs, pPhyFillNode->node.pOutputDataBlockDesc,
&numOfOutputCols, COL_MATCH_FROM_SLOT_ID, &pInfo->matchInfo); &numOfOutputCols, COL_MATCH_FROM_SLOT_ID, &pInfo->matchInfo);
pInfo->pCondition = pPhyFillNode->node.pConditions;
code = filterInitFromNode((SNode*)pPhyFillNode->node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
if (code != TSDB_CODE_SUCCESS) {
goto _error;
}
code = initExprSupp(&pOperator->exprSupp, pFillExprInfo, numOfFillCols); code = initExprSupp(&pOperator->exprSupp, pFillExprInfo, numOfFillCols);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
goto _error; goto _error;

40
source/libs/executor/src/timewindowoperator.c
View File

@ -12,6 +12,7 @@
* You should have received a copy of the GNU Affero General Public License * You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#include "filter.h"
#include "executorimpl.h" #include "executorimpl.h"
#include "function.h" #include "function.h"
#include "functionMgt.h" #include "functionMgt.h"
@ -1227,7 +1228,7 @@ static SSDataBlock* doStateWindowAgg(SOperatorInfo* pOperator) {
blockDataEnsureCapacity(pBInfo->pRes, pOperator->resultInfo.capacity); blockDataEnsureCapacity(pBInfo->pRes, pOperator->resultInfo.capacity);
while (1) { while (1) {
doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf);
doFilter(pInfo->pCondition, pBInfo->pRes, NULL, NULL); doFilter(pBInfo->pRes, pOperator->exprSupp.pFilterInfo, NULL);
bool hasRemain = hasRemainResults(&pInfo->groupResInfo); bool hasRemain = hasRemainResults(&pInfo->groupResInfo);
if (!hasRemain) { if (!hasRemain) {
@ -1265,7 +1266,7 @@ static SSDataBlock* doBuildIntervalResult(SOperatorInfo* pOperator) {
blockDataEnsureCapacity(pBlock, pOperator->resultInfo.capacity); blockDataEnsureCapacity(pBlock, pOperator->resultInfo.capacity);
while (1) { while (1) {
doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf);
doFilter(pInfo->pCondition, pBlock, NULL, NULL); doFilter(pBlock, pOperator->exprSupp.pFilterInfo, NULL);
bool hasRemain = hasRemainResults(&pInfo->groupResInfo); bool hasRemain = hasRemainResults(&pInfo->groupResInfo);
if (!hasRemain) { if (!hasRemain) {
@ -1747,7 +1748,6 @@ SOperatorInfo* createIntervalOperatorInfo(SOperatorInfo* downstream, SIntervalPh
pInfo->interval = interval; pInfo->interval = interval;
pInfo->execModel = pTaskInfo->execModel; pInfo->execModel = pTaskInfo->execModel;
pInfo->twAggSup = as; pInfo->twAggSup = as;
pInfo->pCondition = pPhyNode->window.node.pConditions;
pInfo->binfo.mergeResultBlock = pPhyNode->window.mergeDataBlock; pInfo->binfo.mergeResultBlock = pPhyNode->window.mergeDataBlock;
if (pPhyNode->window.pExprs != NULL) { if (pPhyNode->window.pExprs != NULL) {
@ -1759,6 +1759,11 @@ SOperatorInfo* createIntervalOperatorInfo(SOperatorInfo* downstream, SIntervalPh
} }
} }
code = filterInitFromNode((SNode*)pPhyNode->window.node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
if (code != TSDB_CODE_SUCCESS) {
goto _error;
}
if (isStream) { if (isStream) {
ASSERT(num > 0); ASSERT(num > 0);
initStreamFunciton(pSup->pCtx, pSup->numOfExprs); initStreamFunciton(pSup->pCtx, pSup->numOfExprs);
@ -1883,7 +1888,7 @@ static SSDataBlock* doSessionWindowAgg(SOperatorInfo* pOperator) {
if (pOperator->status == OP_RES_TO_RETURN) { if (pOperator->status == OP_RES_TO_RETURN) {
while (1) { while (1) {
doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf);
doFilter(pInfo->pCondition, pBInfo->pRes, NULL, NULL); doFilter(pBInfo->pRes, pOperator->exprSupp.pFilterInfo, NULL);
bool hasRemain = hasRemainResults(&pInfo->groupResInfo); bool hasRemain = hasRemainResults(&pInfo->groupResInfo);
if (!hasRemain) { if (!hasRemain) {
@ -1926,7 +1931,7 @@ static SSDataBlock* doSessionWindowAgg(SOperatorInfo* pOperator) {
blockDataEnsureCapacity(pBInfo->pRes, pOperator->resultInfo.capacity); blockDataEnsureCapacity(pBInfo->pRes, pOperator->resultInfo.capacity);
while (1) { while (1) {
doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf);
doFilter(pInfo->pCondition, pBInfo->pRes, NULL, NULL); doFilter(pBInfo->pRes, pOperator->exprSupp.pFilterInfo, NULL);
bool hasRemain = hasRemainResults(&pInfo->groupResInfo); bool hasRemain = hasRemainResults(&pInfo->groupResInfo);
if (!hasRemain) { if (!hasRemain) {
@ -2600,17 +2605,22 @@ SOperatorInfo* createStatewindowOperatorInfo(SOperatorInfo* downstream, SStateWi
pInfo->stateKey.type = pInfo->stateCol.type; pInfo->stateKey.type = pInfo->stateCol.type;
pInfo->stateKey.bytes = pInfo->stateCol.bytes; pInfo->stateKey.bytes = pInfo->stateCol.bytes;
pInfo->stateKey.pData = taosMemoryCalloc(1, pInfo->stateCol.bytes); pInfo->stateKey.pData = taosMemoryCalloc(1, pInfo->stateCol.bytes);
pInfo->pCondition = pStateNode->window.node.pConditions;
if (pInfo->stateKey.pData == NULL) { if (pInfo->stateKey.pData == NULL) {
goto _error; goto _error;
} }
int32_t code = filterInitFromNode((SNode*)pStateNode->window.node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
if (code != TSDB_CODE_SUCCESS) {
goto _error;
}
size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES;
int32_t num = 0; int32_t num = 0;
SExprInfo* pExprInfo = createExprInfo(pStateNode->window.pFuncs, NULL, &num); SExprInfo* pExprInfo = createExprInfo(pStateNode->window.pFuncs, NULL, &num);
initResultSizeInfo(&pOperator->resultInfo, 4096); initResultSizeInfo(&pOperator->resultInfo, 4096);
int32_t code = initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, num, keyBufSize, pTaskInfo->id.str);
code = initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, num, keyBufSize, pTaskInfo->id.str);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
goto _error; goto _error;
} }
@ -2698,7 +2708,10 @@ SOperatorInfo* createSessionAggOperatorInfo(SOperatorInfo* downstream, SSessionW
pInfo->binfo.pRes = pResBlock; pInfo->binfo.pRes = pResBlock;
pInfo->winSup.prevTs = INT64_MIN; pInfo->winSup.prevTs = INT64_MIN;
pInfo->reptScan = false; pInfo->reptScan = false;
pInfo->pCondition = pSessionNode->window.node.pConditions; code = filterInitFromNode((SNode*)pSessionNode->window.node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
if (code != TSDB_CODE_SUCCESS) {
goto _error;
}
pOperator->name = "SessionWindowAggOperator"; pOperator->name = "SessionWindowAggOperator";
pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION;
@ -4877,7 +4890,7 @@ static void doMergeAlignedIntervalAgg(SOperatorInfo* pOperator) {
setInputDataBlock(pSup, pBlock, pIaInfo->inputOrder, scanFlag, true); setInputDataBlock(pSup, pBlock, pIaInfo->inputOrder, scanFlag, true);
doMergeAlignedIntervalAggImpl(pOperator, &pIaInfo->binfo.resultRowInfo, pBlock, pRes); doMergeAlignedIntervalAggImpl(pOperator, &pIaInfo->binfo.resultRowInfo, pBlock, pRes);
doFilter(pMiaInfo->pCondition, pRes, NULL, NULL); doFilter(pRes, pOperator->exprSupp.pFilterInfo, NULL);
if (pRes->info.rows >= pOperator->resultInfo.capacity) { if (pRes->info.rows >= pOperator->resultInfo.capacity) {
break; break;
} }
@ -4940,7 +4953,11 @@ SOperatorInfo* createMergeAlignedIntervalOperatorInfo(SOperatorInfo* downstream,
SIntervalAggOperatorInfo* iaInfo = miaInfo->intervalAggOperatorInfo; SIntervalAggOperatorInfo* iaInfo = miaInfo->intervalAggOperatorInfo;
SExprSupp* pSup = &pOperator->exprSupp; SExprSupp* pSup = &pOperator->exprSupp;
miaInfo->pCondition = pNode->window.node.pConditions; int32_t code = filterInitFromNode((SNode*)pNode->window.node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
if (code != TSDB_CODE_SUCCESS) {
goto _error;
}
miaInfo->curTs = INT64_MIN; miaInfo->curTs = INT64_MIN;
iaInfo->win = pTaskInfo->window; iaInfo->win = pTaskInfo->window;
iaInfo->inputOrder = TSDB_ORDER_ASC; iaInfo->inputOrder = TSDB_ORDER_ASC;
@ -4954,7 +4971,8 @@ SOperatorInfo* createMergeAlignedIntervalOperatorInfo(SOperatorInfo* downstream,
int32_t num = 0; int32_t num = 0;
SExprInfo* pExprInfo = createExprInfo(pNode->window.pFuncs, NULL, &num); SExprInfo* pExprInfo = createExprInfo(pNode->window.pFuncs, NULL, &num);
int32_t code = initAggInfo(&pOperator->exprSupp, &iaInfo->aggSup, pExprInfo, num, keyBufSize, pTaskInfo->id.str);
code = initAggInfo(&pOperator->exprSupp, &iaInfo->aggSup, pExprInfo, num, keyBufSize, pTaskInfo->id.str);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
goto _error; goto _error;
} }

20
source/libs/function/src/builtinsimpl.c
View File

@ -181,6 +181,7 @@ typedef enum { UNKNOWN_BIN = 0, USER_INPUT_BIN, LINEAR_BIN, LOG_BIN } EHistoBinT
typedef struct SHLLFuncInfo { typedef struct SHLLFuncInfo {
uint64_t result; uint64_t result;
uint64_t totalCount;
uint8_t buckets[HLL_BUCKETS]; uint8_t buckets[HLL_BUCKETS];
} SHLLInfo; } SHLLInfo;
@ -551,7 +552,7 @@ int32_t countFunction(SqlFunctionCtx* pCtx) {
if (tsCountAlwaysReturnValue) { if (tsCountAlwaysReturnValue) {
pResInfo->numOfRes = 1; pResInfo->numOfRes = 1;
} else { } else {
SET_VAL(pResInfo, 1, 1); SET_VAL(pResInfo, *((int64_t*)buf), 1);
} }
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
@ -4605,7 +4606,14 @@ int32_t hllFunction(SqlFunctionCtx* pCtx) {
} }
} }
SET_VAL(GET_RES_INFO(pCtx), numOfElems, 1); pInfo->totalCount += numOfElems;
if (pInfo->totalCount == 0 && !tsCountAlwaysReturnValue) {
SET_VAL(GET_RES_INFO(pCtx), 0, 1);
} else {
SET_VAL(GET_RES_INFO(pCtx), 1, 1);
}
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
@ -4615,6 +4623,7 @@ static void hllTransferInfo(SHLLInfo* pInput, SHLLInfo* pOutput) {
pOutput->buckets[k] = pInput->buckets[k]; pOutput->buckets[k] = pInput->buckets[k];
} }
} }
pOutput->totalCount += pInput->totalCount;
} }
int32_t hllFunctionMerge(SqlFunctionCtx* pCtx) { int32_t hllFunctionMerge(SqlFunctionCtx* pCtx) {
@ -4632,7 +4641,12 @@ int32_t hllFunctionMerge(SqlFunctionCtx* pCtx) {
hllTransferInfo(pInputInfo, pInfo); hllTransferInfo(pInputInfo, pInfo);
} }
SET_VAL(GET_RES_INFO(pCtx), 1, 1); if (pInfo->totalCount == 0 && !tsCountAlwaysReturnValue) {
SET_VAL(GET_RES_INFO(pCtx), 0, 1);
} else {
SET_VAL(GET_RES_INFO(pCtx), 1, 1);
}
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }

6
source/libs/function/src/functionMgt.c
View File

@ -241,7 +241,11 @@ bool fmIsNotNullOutputFunc(int32_t funcId) {
FUNCTION_TYPE_LAST_MERGE == funcMgtBuiltins[funcId].type || FUNCTION_TYPE_LAST_MERGE == funcMgtBuiltins[funcId].type ||
FUNCTION_TYPE_FIRST == funcMgtBuiltins[funcId].type || FUNCTION_TYPE_FIRST == funcMgtBuiltins[funcId].type ||
FUNCTION_TYPE_FIRST_PARTIAL == funcMgtBuiltins[funcId].type || FUNCTION_TYPE_FIRST_PARTIAL == funcMgtBuiltins[funcId].type ||
FUNCTION_TYPE_FIRST_MERGE == funcMgtBuiltins[funcId].type; FUNCTION_TYPE_FIRST_MERGE == funcMgtBuiltins[funcId].type ||
FUNCTION_TYPE_COUNT == funcMgtBuiltins[funcId].type ||
FUNCTION_TYPE_HYPERLOGLOG == funcMgtBuiltins[funcId].type ||
FUNCTION_TYPE_HYPERLOGLOG_PARTIAL == funcMgtBuiltins[funcId].type ||
FUNCTION_TYPE_HYPERLOGLOG_MERGE == funcMgtBuiltins[funcId].type;
} }
bool fmIsSelectValueFunc(int32_t funcId) { bool fmIsSelectValueFunc(int32_t funcId) {

7
source/libs/scalar/src/filter.c
View File

@ -3994,9 +3994,12 @@ int32_t filterSetDataFromColId(SFilterInfo *info, void *param) {
} }
int32_t filterInitFromNode(SNode *pNode, SFilterInfo **pInfo, uint32_t options) { int32_t filterInitFromNode(SNode *pNode, SFilterInfo **pInfo, uint32_t options) {
int32_t code = 0;
SFilterInfo *info = NULL; SFilterInfo *info = NULL;
if (pNode == NULL) {
return TSDB_CODE_SUCCESS;
}
int32_t code = 0;
if (pNode == NULL || pInfo == NULL) { if (pNode == NULL || pInfo == NULL) {
fltError("invalid param"); fltError("invalid param");
FLT_ERR_RET(TSDB_CODE_QRY_APP_ERROR); FLT_ERR_RET(TSDB_CODE_QRY_APP_ERROR);
@ -4034,9 +4037,7 @@ int32_t filterInitFromNode(SNode *pNode, SFilterInfo **pInfo, uint32_t options)
_return: _return:
filterFreeInfo(*pInfo); filterFreeInfo(*pInfo);
*pInfo = NULL; *pInfo = NULL;
FLT_RET(code); FLT_RET(code);
} }

625
tests/system-test/1-insert/delete_data.py
View File

@ -1,312 +1,313 @@
###################################################################
# Copyright (c) 2016 by TAOS Technologies, Inc. ###################################################################
# All rights reserved. # Copyright (c) 2016 by TAOS Technologies, Inc.
# # All rights reserved.
# This file is proprietary and confidential to TAOS Technologies. #
# No part of this file may be reproduced, stored, transmitted, # This file is proprietary and confidential to TAOS Technologies.
# disclosed or used in any form or by any means other than as # No part of this file may be reproduced, stored, transmitted,
# expressly provided by the written permission from Jianhui Tao # disclosed or used in any form or by any means other than as
# # expressly provided by the written permission from Jianhui Tao
################################################################### #
###################################################################
# -*- coding: utf-8 -*-
# -*- coding: utf-8 -*-
import random
import string import random
import string
from numpy import logspace
from util import constant from numpy import logspace
from util.log import * from util import constant
from util.cases import * from util.log import *
from util.sql import * from util.cases import *
from util.common import * from util.sql import *
from util.sqlset import TDSetSql from util.common import *
from util.sqlset import TDSetSql
class TDTestCase:
def init(self, conn, logSql, replicaVar=1): class TDTestCase:
self.replicaVar = int(replicaVar) def init(self, conn, logSql, replicaVar=1):
tdLog.debug("start to execute %s" % __file__) self.replicaVar = int(replicaVar)
tdSql.init(conn.cursor(), True) tdLog.debug("start to execute %s" % __file__)
self.dbname = 'db_test' tdSql.init(conn.cursor(), True)
self.setsql = TDSetSql() self.dbname = 'db_test'
self.stbname = 'stb' self.setsql = TDSetSql()
self.ntbname = 'ntb' self.stbname = 'stb'
self.rowNum = 10 self.ntbname = 'ntb'
self.tbnum = 3 self.rowNum = 10
self.ts = 1537146000000 self.tbnum = 3
self.binary_str = 'taosdata' self.ts = 1537146000000
self.nchar_str = '涛思数据' self.binary_str = 'taosdata'
self.str_length = 20 self.nchar_str = '涛思数据'
self.column_dict = { self.str_length = 20
'col1': 'tinyint', self.column_dict = {
'col2': 'smallint', 'col1': 'tinyint',
'col3': 'int', 'col2': 'smallint',
'col4': 'bigint', 'col3': 'int',
'col5': 'tinyint unsigned', 'col4': 'bigint',
'col6': 'smallint unsigned', 'col5': 'tinyint unsigned',
'col7': 'int unsigned', 'col6': 'smallint unsigned',
'col8': 'bigint unsigned', 'col7': 'int unsigned',
'col9': 'float', 'col8': 'bigint unsigned',
'col10': 'double', 'col9': 'float',
'col11': 'bool', 'col10': 'double',
'col12': f'binary({self.str_length})', 'col11': 'bool',
'col13': f'nchar({self.str_length})', 'col12': f'binary({self.str_length})',
'col13': f'nchar({self.str_length})',
}
}
self.tinyint_val = random.randint(constant.TINYINT_MIN,constant.TINYINT_MAX)
self.smallint_val = random.randint(constant.SMALLINT_MIN,constant.SMALLINT_MAX) self.tinyint_val = random.randint(constant.TINYINT_MIN,constant.TINYINT_MAX)
self.int_val = random.randint(constant.INT_MIN,constant.INT_MAX) self.smallint_val = random.randint(constant.SMALLINT_MIN,constant.SMALLINT_MAX)
self.bigint_val = random.randint(constant.BIGINT_MIN,constant.BIGINT_MAX) self.int_val = random.randint(constant.INT_MIN,constant.INT_MAX)
self.untingint_val = random.randint(constant.TINYINT_UN_MIN,constant.TINYINT_UN_MAX) self.bigint_val = random.randint(constant.BIGINT_MIN,constant.BIGINT_MAX)
self.unsmallint_val = random.randint(constant.SMALLINT_UN_MIN,constant.SMALLINT_UN_MAX) self.untingint_val = random.randint(constant.TINYINT_UN_MIN,constant.TINYINT_UN_MAX)
self.unint_val = random.randint(constant.INT_UN_MIN,constant.INT_MAX) self.unsmallint_val = random.randint(constant.SMALLINT_UN_MIN,constant.SMALLINT_UN_MAX)
self.unbigint_val = random.randint(constant.BIGINT_UN_MIN,constant.BIGINT_UN_MAX) self.unint_val = random.randint(constant.INT_UN_MIN,constant.INT_MAX)
self.float_val = random.uniform(constant.FLOAT_MIN,constant.FLOAT_MAX) self.unbigint_val = random.randint(constant.BIGINT_UN_MIN,constant.BIGINT_UN_MAX)
self.double_val = random.uniform(constant.DOUBLE_MIN*(1E-300),constant.DOUBLE_MAX*(1E-300)) self.float_val = random.uniform(constant.FLOAT_MIN,constant.FLOAT_MAX)
self.bool_val = random.randint(0,100)%2 self.double_val = random.uniform(constant.DOUBLE_MIN*(1E-300),constant.DOUBLE_MAX*(1E-300))
self.binary_val = tdCom.getLongName(random.randint(0,self.str_length)) self.bool_val = random.randint(0,100)%2
self.nchar_val = tdCom.getLongName(random.randint(0,self.str_length)) self.binary_val = tdCom.getLongName(random.randint(0,self.str_length))
self.base_data = { self.nchar_val = tdCom.getLongName(random.randint(0,self.str_length))
'tinyint':self.tinyint_val, self.base_data = {
'smallint':self.smallint_val, 'tinyint':self.tinyint_val,
'int':self.int_val, 'smallint':self.smallint_val,
'bigint':self.bigint_val, 'int':self.int_val,
'tinyint unsigned':self.untingint_val, 'bigint':self.bigint_val,
'smallint unsigned':self.unsmallint_val, 'tinyint unsigned':self.untingint_val,
'int unsigned':self.unint_val, 'smallint unsigned':self.unsmallint_val,
'bigint unsigned':self.unbigint_val, 'int unsigned':self.unint_val,
'bool':self.bool_val, 'bigint unsigned':self.unbigint_val,
'float':self.float_val, 'bool':self.bool_val,
'double':self.double_val, 'float':self.float_val,
'binary':self.binary_val, 'double':self.double_val,
'nchar':self.nchar_val 'binary':self.binary_val,
} 'nchar':self.nchar_val
}
def insert_base_data(self,col_type,tbname,rows,base_data):
for i in range(rows): def insert_base_data(self,col_type,tbname,rows,base_data):
if col_type.lower() == 'tinyint': for i in range(rows):
tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["tinyint"]})') if col_type.lower() == 'tinyint':
elif col_type.lower() == 'smallint': tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["tinyint"]})')
tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["smallint"]})') elif col_type.lower() == 'smallint':
elif col_type.lower() == 'int': tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["smallint"]})')
tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["int"]})') elif col_type.lower() == 'int':
elif col_type.lower() == 'bigint': tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["int"]})')
tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["bigint"]})') elif col_type.lower() == 'bigint':
elif col_type.lower() == 'tinyint unsigned': tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["bigint"]})')
tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["tinyint unsigned"]})') elif col_type.lower() == 'tinyint unsigned':
elif col_type.lower() == 'smallint unsigned': tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["tinyint unsigned"]})')
tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["smallint unsigned"]})') elif col_type.lower() == 'smallint unsigned':
elif col_type.lower() == 'int unsigned': tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["smallint unsigned"]})')
tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["int unsigned"]})') elif col_type.lower() == 'int unsigned':
elif col_type.lower() == 'bigint unsigned': tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["int unsigned"]})')
tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["bigint unsigned"]})') elif col_type.lower() == 'bigint unsigned':
elif col_type.lower() == 'bool': tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["bigint unsigned"]})')
tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["bool"]})') elif col_type.lower() == 'bool':
elif col_type.lower() == 'float': tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["bool"]})')
tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["float"]})') elif col_type.lower() == 'float':
elif col_type.lower() == 'double': tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["float"]})')
tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["double"]})') elif col_type.lower() == 'double':
elif 'binary' in col_type.lower(): tdSql.execute(f'insert into {tbname} values({self.ts+i},{base_data["double"]})')
tdSql.execute(f'''insert into {tbname} values({self.ts+i},"{base_data['binary']}")''') elif 'binary' in col_type.lower():
elif 'nchar' in col_type.lower(): tdSql.execute(f'''insert into {tbname} values({self.ts+i},"{base_data['binary']}")''')
tdSql.execute(f'''insert into {tbname} values({self.ts+i},"{base_data['nchar']}")''') elif 'nchar' in col_type.lower():
def delete_all_data(self,tbname,col_type,row_num,base_data,dbname,tb_type,tb_num=1,stbname=''): tdSql.execute(f'''insert into {tbname} values({self.ts+i},"{base_data['nchar']}")''')
tdSql.query(f'select count(*) from {tbname}') def delete_all_data(self,tbname,col_type,row_num,base_data,dbname,tb_type,tb_num=1,stbname=''):
tdSql.execute(f'delete from {tbname}') tdSql.query(f'select count(*) from {tbname}')
tdSql.execute(f'flush database {dbname}') tdSql.execute(f'delete from {tbname}')
tdSql.execute('reset query cache') tdSql.execute(f'flush database {dbname}')
tdSql.query(f'select * from {tbname}') tdSql.execute('reset query cache')
tdSql.checkRows(0) tdSql.query(f'select * from {tbname}')
if tb_type == 'ntb' or tb_type == 'ctb': tdSql.checkRows(0)
if tb_type == 'ctb': if tb_type == 'ntb' or tb_type == 'ctb':
tdSql.query(f'select count(*) from {stbname}') if tb_type == 'ctb':
if tb_num <= 1: tdSql.query(f'select count(*) from {stbname}')
if len(tdSql.queryResult) != 0: if tb_num <= 1:
tdLog.exit('delete case failure!') if len(tdSql.queryResult) != 0:
else: tdLog.exit('delete case failure!')
tdSql.checkEqual(tdSql.queryResult[0][0],(tb_num-1)*row_num) else:
tdSql.checkEqual(tdSql.queryResult[0][0],(tb_num-1)*row_num)
self.insert_base_data(col_type,tbname,row_num,base_data)
elif tb_type == 'stb': self.insert_base_data(col_type,tbname,row_num,base_data)
for i in range(tb_num): elif tb_type == 'stb':
self.insert_base_data(col_type,f'{tbname}_{i}',row_num,base_data) for i in range(tb_num):
tdSql.execute(f'flush database {dbname}') self.insert_base_data(col_type,f'{tbname}_{i}',row_num,base_data)
tdSql.execute('reset query cache') tdSql.execute(f'flush database {dbname}')
tdSql.query(f'select * from {tbname}') tdSql.execute('reset query cache')
if tb_type == 'ntb' or tb_type == 'ctb': tdSql.query(f'select * from {tbname}')
tdSql.checkRows(row_num) if tb_type == 'ntb' or tb_type == 'ctb':
elif tb_type =='stb': tdSql.checkRows(row_num)
tdSql.checkRows(row_num*tb_num) elif tb_type =='stb':
def delete_one_row(self,tbname,column_type,column_name,base_data,row_num,dbname,tb_type,tb_num=1): tdSql.checkRows(row_num*tb_num)
tdSql.execute(f'delete from {tbname} where ts={self.ts}') def delete_one_row(self,tbname,column_type,column_name,base_data,row_num,dbname,tb_type,tb_num=1):
tdSql.execute(f'flush database {dbname}') tdSql.execute(f'delete from {tbname} where ts={self.ts}')
tdSql.execute('reset query cache') tdSql.execute(f'flush database {dbname}')
tdSql.query(f'select {column_name} from {tbname}') tdSql.execute('reset query cache')
if tb_type == 'ntb' or tb_type == 'ctb': tdSql.query(f'select {column_name} from {tbname}')
tdSql.checkRows(row_num-1) if tb_type == 'ntb' or tb_type == 'ctb':
elif tb_type == 'stb': tdSql.checkRows(row_num-1)
tdSql.checkRows((row_num-1)*tb_num) elif tb_type == 'stb':
tdSql.query(f'select {column_name} from {tbname} where ts={self.ts}') tdSql.checkRows((row_num-1)*tb_num)
tdSql.checkRows(0) tdSql.query(f'select {column_name} from {tbname} where ts={self.ts}')
if tb_type == 'ntb' or tb_type == 'ctb': tdSql.checkRows(0)
if 'binary' in column_type.lower(): if tb_type == 'ntb' or tb_type == 'ctb':
tdSql.execute(f'''insert into {tbname} values({self.ts},"{base_data['binary']}")''') if 'binary' in column_type.lower():
elif 'nchar' in column_type.lower(): tdSql.execute(f'''insert into {tbname} values({self.ts},"{base_data['binary']}")''')
tdSql.execute(f'''insert into {tbname} values({self.ts},"{base_data['nchar']}")''') elif 'nchar' in column_type.lower():
else: tdSql.execute(f'''insert into {tbname} values({self.ts},"{base_data['nchar']}")''')
tdSql.execute(f'insert into {tbname} values({self.ts},{base_data[column_type]})') else:
elif tb_type == 'stb': tdSql.execute(f'insert into {tbname} values({self.ts},{base_data[column_type]})')
for i in range(tb_num): elif tb_type == 'stb':
if 'binary' in column_type.lower(): for i in range(tb_num):
tdSql.execute(f'''insert into {tbname}_{i} values({self.ts},"{base_data['binary']}")''') if 'binary' in column_type.lower():
elif 'nchar' in column_type.lower(): tdSql.execute(f'''insert into {tbname}_{i} values({self.ts},"{base_data['binary']}")''')
tdSql.execute(f'''insert into {tbname}_{i} values({self.ts},"{base_data['nchar']}")''') elif 'nchar' in column_type.lower():
else: tdSql.execute(f'''insert into {tbname}_{i} values({self.ts},"{base_data['nchar']}")''')
tdSql.execute(f'insert into {tbname}_{i} values({self.ts},{base_data[column_type]})') else:
tdSql.query(f'select {column_name} from {tbname} where ts={self.ts}') tdSql.execute(f'insert into {tbname}_{i} values({self.ts},{base_data[column_type]})')
if column_type.lower() == 'float' or column_type.lower() == 'double': tdSql.query(f'select {column_name} from {tbname} where ts={self.ts}')
if abs(tdSql.queryResult[0][0] - base_data[column_type]) / base_data[column_type] <= 0.0001: if column_type.lower() == 'float' or column_type.lower() == 'double':
tdSql.checkEqual(tdSql.queryResult[0][0],tdSql.queryResult[0][0]) if abs(tdSql.queryResult[0][0] - base_data[column_type]) / base_data[column_type] <= 0.0001:
else: tdSql.checkEqual(tdSql.queryResult[0][0],tdSql.queryResult[0][0])
tdLog.exit(f'{column_type} data check failure') else:
elif 'binary' in column_type.lower(): tdLog.exit(f'{column_type} data check failure')
tdSql.checkEqual(tdSql.queryResult[0][0],base_data['binary']) elif 'binary' in column_type.lower():
elif 'nchar' in column_type.lower(): tdSql.checkEqual(tdSql.queryResult[0][0],base_data['binary'])
tdSql.checkEqual(tdSql.queryResult[0][0],base_data['nchar']) elif 'nchar' in column_type.lower():
else: tdSql.checkEqual(tdSql.queryResult[0][0],base_data['nchar'])
tdSql.checkEqual(tdSql.queryResult[0][0],base_data[column_type]) else:
def delete_rows(self,dbname,tbname,col_name,col_type,base_data,row_num,tb_type,tb_num=1): tdSql.checkEqual(tdSql.queryResult[0][0],base_data[column_type])
for i in range(row_num): def delete_rows(self,dbname,tbname,col_name,col_type,base_data,row_num,tb_type,tb_num=1):
tdSql.execute(f'delete from {tbname} where ts>{self.ts+i}') for i in range(row_num):
tdSql.execute(f'flush database {dbname}') tdSql.execute(f'delete from {tbname} where ts>{self.ts+i}')
tdSql.execute('reset query cache') tdSql.execute(f'flush database {dbname}')
tdSql.query(f'select {col_name} from {tbname}') tdSql.execute('reset query cache')
if tb_type == 'ntb' or tb_type == 'ctb': tdSql.query(f'select {col_name} from {tbname}')
tdSql.checkRows(i+1) if tb_type == 'ntb' or tb_type == 'ctb':
self.insert_base_data(col_type,tbname,row_num,base_data) tdSql.checkRows(i+1)
elif tb_type == 'stb': self.insert_base_data(col_type,tbname,row_num,base_data)
tdSql.checkRows((i+1)*tb_num) elif tb_type == 'stb':
for j in range(tb_num): tdSql.checkRows((i+1)*tb_num)
self.insert_base_data(col_type,f'{tbname}_{j}',row_num,base_data) for j in range(tb_num):
for i in range(row_num): self.insert_base_data(col_type,f'{tbname}_{j}',row_num,base_data)
tdSql.execute(f'delete from {tbname} where ts>={self.ts+i}') for i in range(row_num):
tdSql.execute(f'flush database {dbname}') tdSql.execute(f'delete from {tbname} where ts>={self.ts+i}')
tdSql.execute('reset query cache') tdSql.execute(f'flush database {dbname}')
tdSql.query(f'select {col_name} from {tbname}') tdSql.execute('reset query cache')
if tb_type == 'ntb' or tb_type == 'ctb': tdSql.query(f'select {col_name} from {tbname}')
tdSql.checkRows(i) if tb_type == 'ntb' or tb_type == 'ctb':
self.insert_base_data(col_type,tbname,row_num,base_data) tdSql.checkRows(i)
elif tb_type == 'stb': self.insert_base_data(col_type,tbname,row_num,base_data)
tdSql.checkRows(i*tb_num) elif tb_type == 'stb':
for j in range(tb_num): tdSql.checkRows(i*tb_num)
self.insert_base_data(col_type,f'{tbname}_{j}',row_num,base_data) for j in range(tb_num):
for i in range(row_num): self.insert_base_data(col_type,f'{tbname}_{j}',row_num,base_data)
tdSql.execute(f'delete from {tbname} where ts<={self.ts+i}') for i in range(row_num):
tdSql.execute(f'flush database {dbname}') tdSql.execute(f'delete from {tbname} where ts<={self.ts+i}')
tdSql.execute('reset query cache') tdSql.execute(f'flush database {dbname}')
tdSql.query(f'select {col_name} from {tbname}') tdSql.execute('reset query cache')
if tb_type == 'ntb' or tb_type == 'ctb': tdSql.query(f'select {col_name} from {tbname}')
tdSql.checkRows(row_num-i-1) if tb_type == 'ntb' or tb_type == 'ctb':
self.insert_base_data(col_type,tbname,row_num,base_data) tdSql.checkRows(row_num-i-1)
elif tb_type == 'stb': self.insert_base_data(col_type,tbname,row_num,base_data)
tdSql.checkRows((row_num-i-1)*tb_num) elif tb_type == 'stb':
for j in range(tb_num): tdSql.checkRows((row_num-i-1)*tb_num)
self.insert_base_data(col_type,f'{tbname}_{j}',row_num,base_data) for j in range(tb_num):
for i in range(row_num): self.insert_base_data(col_type,f'{tbname}_{j}',row_num,base_data)
tdSql.execute(f'delete from {tbname} where ts<{self.ts+i}') for i in range(row_num):
tdSql.execute(f'flush database {dbname}') tdSql.execute(f'delete from {tbname} where ts<{self.ts+i}')
tdSql.execute('reset query cache') tdSql.execute(f'flush database {dbname}')
tdSql.query(f'select {col_name} from {tbname}') tdSql.execute('reset query cache')
if tb_type == 'ntb' or tb_type == 'ctb': tdSql.query(f'select {col_name} from {tbname}')
tdSql.checkRows(row_num-i) if tb_type == 'ntb' or tb_type == 'ctb':
self.insert_base_data(col_type,tbname,row_num,base_data) tdSql.checkRows(row_num-i)
elif tb_type == 'stb': self.insert_base_data(col_type,tbname,row_num,base_data)
tdSql.checkRows((row_num-i)*tb_num) elif tb_type == 'stb':
for j in range(tb_num): tdSql.checkRows((row_num-i)*tb_num)
self.insert_base_data(col_type,f'{tbname}_{j}',row_num,base_data) for j in range(tb_num):
for i in range(row_num): self.insert_base_data(col_type,f'{tbname}_{j}',row_num,base_data)
tdSql.execute(f'delete from {tbname} where ts between {self.ts} and {self.ts+i}') for i in range(row_num):
tdSql.execute(f'flush database {dbname}') tdSql.execute(f'delete from {tbname} where ts between {self.ts} and {self.ts+i}')
tdSql.execute('reset query cache') tdSql.execute(f'flush database {dbname}')
tdSql.query(f'select {col_name} from {tbname}') tdSql.execute('reset query cache')
if tb_type == 'ntb' or tb_type == 'ctb': tdSql.query(f'select {col_name} from {tbname}')
tdSql.checkRows(row_num - i-1) if tb_type == 'ntb' or tb_type == 'ctb':
self.insert_base_data(col_type,tbname,row_num,base_data) tdSql.checkRows(row_num - i-1)
elif tb_type == 'stb': self.insert_base_data(col_type,tbname,row_num,base_data)
tdSql.checkRows(tb_num*(row_num - i-1)) elif tb_type == 'stb':
for j in range(tb_num): tdSql.checkRows(tb_num*(row_num - i-1))
self.insert_base_data(col_type,f'{tbname}_{j}',row_num,base_data) for j in range(tb_num):
tdSql.execute(f'delete from {tbname} where ts between {self.ts+i+1} and {self.ts}') self.insert_base_data(col_type,f'{tbname}_{j}',row_num,base_data)
tdSql.query(f'select {col_name} from {tbname}') tdSql.execute(f'delete from {tbname} where ts between {self.ts+i+1} and {self.ts}')
if tb_type == 'ntb' or tb_type == 'ctb': tdSql.query(f'select {col_name} from {tbname}')
tdSql.checkRows(row_num) if tb_type == 'ntb' or tb_type == 'ctb':
elif tb_type == 'stb': tdSql.checkRows(row_num)
tdSql.checkRows(tb_num*row_num) elif tb_type == 'stb':
def delete_error(self,tbname,column_name,column_type,base_data): tdSql.checkRows(tb_num*row_num)
for error_list in ['',f'ts = {self.ts} and',f'ts = {self.ts} or']: def delete_error(self,tbname,column_name,column_type,base_data):
if 'binary' in column_type.lower(): for error_list in ['',f'ts = {self.ts} and',f'ts = {self.ts} or']:
tdSql.error(f'''delete from {tbname} where {error_list} {column_name} ="{base_data['binary']}"''') if 'binary' in column_type.lower():
elif 'nchar' in column_type.lower(): tdSql.error(f'''delete from {tbname} where {error_list} {column_name} ="{base_data['binary']}"''')
tdSql.error(f'''delete from {tbname} where {error_list} {column_name} ="{base_data['nchar']}"''') elif 'nchar' in column_type.lower():
else: tdSql.error(f'''delete from {tbname} where {error_list} {column_name} ="{base_data['nchar']}"''')
tdSql.error(f'delete from {tbname} where {error_list} {column_name} = {base_data[column_type]}') else:
tdSql.error(f'delete from {tbname} where {error_list} {column_name} = {base_data[column_type]}')
def delete_data_ntb(self):
tdSql.execute(f'create database if not exists {self.dbname}') def delete_data_ntb(self):
tdSql.execute(f'use {self.dbname}') tdSql.execute(f'create database if not exists {self.dbname}')
for col_name,col_type in self.column_dict.items(): tdSql.execute(f'use {self.dbname}')
tdSql.execute(f'create table {self.ntbname} (ts timestamp,{col_name} {col_type})') for col_name,col_type in self.column_dict.items():
self.insert_base_data(col_type,self.ntbname,self.rowNum,self.base_data) tdSql.execute(f'create table {self.ntbname} (ts timestamp,{col_name} {col_type})')
self.delete_one_row(self.ntbname,col_type,col_name,self.base_data,self.rowNum,self.dbname,'ntb') self.insert_base_data(col_type,self.ntbname,self.rowNum,self.base_data)
self.delete_all_data(self.ntbname,col_type,self.rowNum,self.base_data,self.dbname,'ntb') self.delete_one_row(self.ntbname,col_type,col_name,self.base_data,self.rowNum,self.dbname,'ntb')
self.delete_error(self.ntbname,col_name,col_type,self.base_data) self.delete_all_data(self.ntbname,col_type,self.rowNum,self.base_data,self.dbname,'ntb')
self.delete_rows(self.dbname,self.ntbname,col_name,col_type,self.base_data,self.rowNum,'ntb') self.delete_error(self.ntbname,col_name,col_type,self.base_data)
for func in ['first','last']: self.delete_rows(self.dbname,self.ntbname,col_name,col_type,self.base_data,self.rowNum,'ntb')
tdSql.query(f'select {func}(*) from {self.ntbname}') for func in ['first','last']:
tdSql.execute(f'drop table {self.ntbname}') tdSql.query(f'select {func}(*) from {self.ntbname}')
tdSql.execute(f'drop database {self.dbname}') tdSql.execute(f'drop table {self.ntbname}')
def delete_data_ctb(self): tdSql.execute(f'drop database {self.dbname}')
tdSql.execute(f'create database if not exists {self.dbname}') def delete_data_ctb(self):
tdSql.execute(f'use {self.dbname}') tdSql.execute(f'create database if not exists {self.dbname}')
for col_name,col_type in self.column_dict.items(): tdSql.execute(f'use {self.dbname}')
tdSql.execute(f'create table {self.stbname} (ts timestamp,{col_name} {col_type}) tags(t1 int)') for col_name,col_type in self.column_dict.items():
for i in range(self.tbnum): tdSql.execute(f'create table {self.stbname} (ts timestamp,{col_name} {col_type}) tags(t1 int)')
tdSql.execute(f'create table {self.stbname}_{i} using {self.stbname} tags(1)') for i in range(self.tbnum):
self.insert_base_data(col_type,f'{self.stbname}_{i}',self.rowNum,self.base_data) tdSql.execute(f'create table {self.stbname}_{i} using {self.stbname} tags(1)')
self.delete_one_row(f'{self.stbname}_{i}',col_type,col_name,self.base_data,self.rowNum,self.dbname,'ctb') self.insert_base_data(col_type,f'{self.stbname}_{i}',self.rowNum,self.base_data)
self.delete_all_data(f'{self.stbname}_{i}',col_type,self.rowNum,self.base_data,self.dbname,'ctb',i+1,self.stbname) self.delete_one_row(f'{self.stbname}_{i}',col_type,col_name,self.base_data,self.rowNum,self.dbname,'ctb')
self.delete_error(f'{self.stbname}_{i}',col_name,col_type,self.base_data) self.delete_all_data(f'{self.stbname}_{i}',col_type,self.rowNum,self.base_data,self.dbname,'ctb',i+1,self.stbname)
self.delete_rows(self.dbname,f'{self.stbname}_{i}',col_name,col_type,self.base_data,self.rowNum,'ctb') self.delete_error(f'{self.stbname}_{i}',col_name,col_type,self.base_data)
for func in ['first','last']: self.delete_rows(self.dbname,f'{self.stbname}_{i}',col_name,col_type,self.base_data,self.rowNum,'ctb')
tdSql.query(f'select {func}(*) from {self.stbname}_{i}') for func in ['first','last']:
tdSql.execute(f'drop table {self.stbname}') tdSql.query(f'select {func}(*) from {self.stbname}_{i}')
def delete_data_stb(self): tdSql.execute(f'drop table {self.stbname}')
tdSql.execute(f'create database if not exists {self.dbname}') def delete_data_stb(self):
tdSql.execute(f'use {self.dbname}') tdSql.execute(f'create database if not exists {self.dbname}')
for col_name,col_type in self.column_dict.items(): tdSql.execute(f'use {self.dbname}')
tdSql.execute(f'create table {self.stbname} (ts timestamp,{col_name} {col_type}) tags(t1 int)') for col_name,col_type in self.column_dict.items():
for i in range(self.tbnum): tdSql.execute(f'create table {self.stbname} (ts timestamp,{col_name} {col_type}) tags(t1 int)')
tdSql.execute(f'create table {self.stbname}_{i} using {self.stbname} tags(1)') for i in range(self.tbnum):
self.insert_base_data(col_type,f'{self.stbname}_{i}',self.rowNum,self.base_data) tdSql.execute(f'create table {self.stbname}_{i} using {self.stbname} tags(1)')
self.delete_error(self.stbname,col_name,col_type,self.base_data) self.insert_base_data(col_type,f'{self.stbname}_{i}',self.rowNum,self.base_data)
self.delete_one_row(self.stbname,col_type,col_name,self.base_data,self.rowNum,self.dbname,'stb',self.tbnum) self.delete_error(self.stbname,col_name,col_type,self.base_data)
self.delete_all_data(self.stbname,col_type,self.rowNum,self.base_data,self.dbname,'stb',self.tbnum) self.delete_one_row(self.stbname,col_type,col_name,self.base_data,self.rowNum,self.dbname,'stb',self.tbnum)
self.delete_rows(self.dbname,self.stbname,col_name,col_type,self.base_data,self.rowNum,'stb',self.tbnum) self.delete_all_data(self.stbname,col_type,self.rowNum,self.base_data,self.dbname,'stb',self.tbnum)
for func in ['first','last']: self.delete_rows(self.dbname,self.stbname,col_name,col_type,self.base_data,self.rowNum,'stb',self.tbnum)
tdSql.query(f'select {func}(*) from {self.stbname}') for func in ['first','last']:
tdSql.execute(f'drop table {self.stbname}') tdSql.query(f'select {func}(*) from {self.stbname}')
tdSql.execute(f'drop database {self.dbname}') tdSql.execute(f'drop table {self.stbname}')
def run(self): tdSql.execute(f'drop database {self.dbname}')
self.delete_data_ntb() def run(self):
self.delete_data_ctb() self.delete_data_ntb()
self.delete_data_stb() self.delete_data_ctb()
tdDnodes.stoptaosd(1) self.delete_data_stb()
tdDnodes.starttaosd(1) tdDnodes.stoptaosd(1)
self.delete_data_ntb() tdDnodes.starttaosd(1)
def stop(self): self.delete_data_ntb()
tdSql.close() def stop(self):
tdLog.success("%s successfully executed" % __file__) tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase()) tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())

115
tests/system-test/2-query/countAlwaysReturnValue.py Normal file
View File

@ -0,0 +1,115 @@
import taos
import sys
import datetime
import inspect
from util.log import *
from util.sql import *
from util.cases import *
import random
class TDTestCase:
updatecfgDict = {"countAlwaysReturnValue":0}
def init(self, conn, logSql, replicaVar=1):
tdLog.debug(f"start to excute {__file__}")
tdSql.init(conn.cursor(), False)
def prepare_data(self, dbname="db"):
tdSql.execute(
f"create database if not exists {dbname} keep 3650 duration 1000")
tdSql.execute(f"use {dbname} ")
tdSql.execute(
f"create table {dbname}.tb (ts timestamp, c0 int)"
)
tdSql.execute(
f"create table {dbname}.stb (ts timestamp, c0 int) tags (t0 int)"
)
tdSql.execute(
f"create table {dbname}.ctb1 using {dbname}.stb tags (1)"
)
tdSql.execute(
f"create table {dbname}.ctb2 using {dbname}.stb tags (2)"
)
tdSql.execute(
f"insert into {dbname}.tb values (now(), NULL)")
tdSql.execute(
f"insert into {dbname}.ctb1 values (now(), NULL)")
tdSql.execute(
f"insert into {dbname}.ctb2 values (now() + 1s, NULL)")
def test_results(self, dbname="db"):
# count
tdSql.query(f"select count(c0) from {dbname}.tb")
tdSql.checkRows(0)
tdSql.query(f"select count(NULL) from {dbname}.tb")
tdSql.checkRows(0)
tdSql.query(f"select c0,count(c0) from {dbname}.tb group by c0")
tdSql.checkRows(1)
tdSql.checkData(0, 0, None)
tdSql.query(f"select count(c0) from {dbname}.stb")
tdSql.checkRows(0)
tdSql.query(f"select count(NULL) from {dbname}.stb")
tdSql.checkRows(0)
tdSql.query(f"select c0,count(c0) from {dbname}.stb group by c0")
tdSql.checkRows(1)
tdSql.checkData(0, 0, None)
tdSql.query(f"select count(NULL)")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 0)
# hyperloglog
tdSql.query(f"select hyperloglog(c0) from {dbname}.tb")
tdSql.checkRows(0)
tdSql.query(f"select hyperloglog(NULL) from {dbname}.tb")
tdSql.checkRows(0)
tdSql.query(f"select c0,hyperloglog(c0) from {dbname}.tb group by c0")
tdSql.checkRows(1)
tdSql.checkData(0, 0, None)
tdSql.query(f"select hyperloglog(c0) from {dbname}.stb")
tdSql.checkRows(0)
tdSql.query(f"select hyperloglog(NULL) from {dbname}.stb")
tdSql.checkRows(0)
tdSql.query(f"select c0,hyperloglog(c0) from {dbname}.stb group by c0")
tdSql.checkRows(1)
tdSql.checkData(0, 0, None)
tdSql.query(f"select hyperloglog(NULL)")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 0)
def run(self):
tdSql.prepare()
tdLog.printNoPrefix("==========step1:prepare data ==============")
self.prepare_data()
tdLog.printNoPrefix("==========step2:test results ==============")
self.test_results()
def stop(self):
tdSql.close()
tdLog.success(f"{__file__} successfully executed")
tdCases.addLinux(__file__, TDTestCase())
tdCases.addWindows(__file__, TDTestCase())

2
tests/system-test/2-query/hyperloglog.py
View File

@ -28,7 +28,7 @@ ALL_COL = [ INT_COL, BINT_COL, SINT_COL, TINT_COL, FLOAT_COL, DOUBLE_COL, BOOL_C
DBNAME = "db" DBNAME = "db"
class TDTestCase: class TDTestCase:
updatecfgDict = {"maxTablesPerVnode":2 ,"minTablesPerVnode":2,"tableIncStepPerVnode":2 } updatecfgDict = {"maxTablesPerVnode":2 ,"minTablesPerVnode":2,"tableIncStepPerVnode":2 }
def init(self, conn, logSql, replicaVar=1): def init(self, conn, logSql, replicaVar=1):

9
tests/system-test/fulltest.sh
View File

@ -17,7 +17,7 @@ python3 ./test.py -f 0-others/taosdShell.py -N 5 -M 3 -Q 3
python3 ./test.py -f 0-others/sysinfo.py python3 ./test.py -f 0-others/sysinfo.py
python3 ./test.py -f 0-others/user_control.py python3 ./test.py -f 0-others/user_control.py
python3 ./test.py -f 0-others/fsync.py python3 ./test.py -f 0-others/fsync.py
python3 ./test.py -f 0-others/compatibility.py python3 ./test.py -f 0-others/compatibility.py
python3 ./test.py -f 1-insert/alter_database.py python3 ./test.py -f 1-insert/alter_database.py
python3 ./test.py -f 1-insert/influxdb_line_taosc_insert.py python3 ./test.py -f 1-insert/influxdb_line_taosc_insert.py
python3 ./test.py -f 1-insert/opentsdb_telnet_line_taosc_insert.py python3 ./test.py -f 1-insert/opentsdb_telnet_line_taosc_insert.py
@ -76,6 +76,8 @@ python3 ./test.py -f 2-query/count_partition.py
python3 ./test.py -f 2-query/count_partition.py -R python3 ./test.py -f 2-query/count_partition.py -R
python3 ./test.py -f 2-query/count.py python3 ./test.py -f 2-query/count.py
python3 ./test.py -f 2-query/count.py -R python3 ./test.py -f 2-query/count.py -R
# python3 ./test.py -f 2-query/countAlwaysReturnValue.py
# python3 ./test.py -f 2-query/countAlwaysReturnValue.py -R
python3 ./test.py -f 2-query/db.py python3 ./test.py -f 2-query/db.py
python3 ./test.py -f 2-query/db.py -R python3 ./test.py -f 2-query/db.py -R
python3 ./test.py -f 2-query/diff.py python3 ./test.py -f 2-query/diff.py
@ -385,6 +387,7 @@ python3 ./test.py -f 2-query/Today.py -Q 2
python3 ./test.py -f 2-query/max.py -Q 2 python3 ./test.py -f 2-query/max.py -Q 2
python3 ./test.py -f 2-query/min.py -Q 2 python3 ./test.py -f 2-query/min.py -Q 2
python3 ./test.py -f 2-query/count.py -Q 2 python3 ./test.py -f 2-query/count.py -Q 2
# python3 ./test.py -f 2-query/countAlwaysReturnValue.py -Q 2
python3 ./test.py -f 2-query/last.py -Q 2 python3 ./test.py -f 2-query/last.py -Q 2
python3 ./test.py -f 2-query/first.py -Q 2 python3 ./test.py -f 2-query/first.py -Q 2
python3 ./test.py -f 2-query/To_iso8601.py -Q 2 python3 ./test.py -f 2-query/To_iso8601.py -Q 2
@ -480,6 +483,7 @@ python3 ./test.py -f 2-query/Today.py -Q 3
python3 ./test.py -f 2-query/max.py -Q 3 python3 ./test.py -f 2-query/max.py -Q 3
python3 ./test.py -f 2-query/min.py -Q 3 python3 ./test.py -f 2-query/min.py -Q 3
python3 ./test.py -f 2-query/count.py -Q 3 python3 ./test.py -f 2-query/count.py -Q 3
# python3 ./test.py -f 2-query/countAlwaysReturnValue.py -Q 3
python3 ./test.py -f 2-query/last.py -Q 3 python3 ./test.py -f 2-query/last.py -Q 3
python3 ./test.py -f 2-query/first.py -Q 3 python3 ./test.py -f 2-query/first.py -Q 3
python3 ./test.py -f 2-query/To_iso8601.py -Q 3 python3 ./test.py -f 2-query/To_iso8601.py -Q 3
@ -577,6 +581,7 @@ python3 ./test.py -f 2-query/Today.py -Q 4
python3 ./test.py -f 2-query/max.py -Q 4 python3 ./test.py -f 2-query/max.py -Q 4
python3 ./test.py -f 2-query/min.py -Q 4 python3 ./test.py -f 2-query/min.py -Q 4
python3 ./test.py -f 2-query/count.py -Q 4 python3 ./test.py -f 2-query/count.py -Q 4
# python3 ./test.py -f 2-query/countAlwaysReturnValue.py -Q 4
python3 ./test.py -f 2-query/last.py -Q 4 python3 ./test.py -f 2-query/last.py -Q 4
python3 ./test.py -f 2-query/first.py -Q 4 python3 ./test.py -f 2-query/first.py -Q 4
python3 ./test.py -f 2-query/To_iso8601.py -Q 4 python3 ./test.py -f 2-query/To_iso8601.py -Q 4
@ -592,7 +597,7 @@ python3 ./test.py -f 2-query/apercentile.py -Q 4
python3 ./test.py -f 2-query/abs.py -Q 4 python3 ./test.py -f 2-query/abs.py -Q 4
python3 ./test.py -f 2-query/ceil.py -Q 4 python3 ./test.py -f 2-query/ceil.py -Q 4
python3 ./test.py -f 2-query/floor.py -Q 4 python3 ./test.py -f 2-query/floor.py -Q 4
python3 ./test.py -f 2-query/round.py -Q 4 python3 ./test.py -f 2-query/round.py -Q 4
python3 ./test.py -f 2-query/log.py -Q 4 python3 ./test.py -f 2-query/log.py -Q 4
python3 ./test.py -f 2-query/pow.py -Q 4 python3 ./test.py -f 2-query/pow.py -Q 4
python3 ./test.py -f 2-query/sqrt.py -Q 4 python3 ./test.py -f 2-query/sqrt.py -Q 4