diff --git a/docs/en/14-reference/05-taosbenchmark.md b/docs/en/14-reference/05-taosbenchmark.md index 7cf1f95eb1..5d47cc06e8 100644 --- a/docs/en/14-reference/05-taosbenchmark.md +++ b/docs/en/14-reference/05-taosbenchmark.md @@ -7,7 +7,7 @@ description: "taosBenchmark (once called taosdemo ) is a tool for testing the pe ## Introduction -taosBenchmark (formerly taosdemo ) is a tool for testing the performance of TDengine products. taosBenchmark can test the performance of TDengine's insert, query, and subscription functions and simulate large amounts of data generated by many devices. taosBenchmark can flexibly control the number and type of databases, supertables, tag columns, number and type of data columns, and sub-tables, and types of databases, super tables, the number and types of data columns, the number of sub-tables, the amount of data per sub-table, the time interval for inserting data, the number of working threads, whether and how to insert disordered data, and so on. The installer provides taosdemo as a soft link to taosBenchmark for compatibility and for the convenience of past users. +taosBenchmark (formerly taosdemo ) is a tool for testing the performance of TDengine products. taosBenchmark can test the performance of TDengine's insert, query, and subscription functions and simulate large amounts of data generated by many devices. taosBenchmark can be configured to generate user defined databases, supertables, subtables, and the time series data to populate these for performance benchmarking. taosBenchmark is highly configurable and some of the configurations include the time interval for inserting data, the number of working threads and the capability to insert disordered data. The installer provides taosdemo as a soft link to taosBenchmark for compatibility with past users. ## Installation @@ -21,9 +21,13 @@ There are two ways to install taosBenchmark: ### Configuration and running methods -TaosBenchmark needs to be executed on the terminal of the operating system, it supports two configuration methods: [Command-line arguments](#Command-line arguments in detailed) and [JSON configuration file](#Configuration file arguments in detailed). These two methods are mutually exclusive. Users can use `-f ` to specify a configuration file. When running taosBenchmark with command-line arguments to control its behavior, users should use other parameters for configuration, but not the `-f` parameter. In addition, taosBenchmark offers a special way of running without parameters. +TaosBenchmark needs to be executed on the terminal of the operating system, it supports two configuration methods: [Command-line arguments](#command-line-arguments-in-detail) and [JSON configuration file](#configuration-file-parameters-in-detail). These two methods are mutually exclusive. Users can use `-f ` to specify a configuration file. When running taosBenchmark with command-line arguments to control its behavior, users should use other parameters for configuration, but not the `-f` parameter. In addition, taosBenchmark offers a special way of running without parameters. +<<<<<<< HEAD taosBenchmark supports complete performance testing of TDengine. taosBenchmark supports the TDengine functions in three categories: write, query, and subscribe. These three functions are mutually exclusive, and users can select only one of them each time taosBenchmark runs. It is important to note that the type of functionality to be tested is not configurable when using the command-line configuration method, which can only test writing performance. To test the query and subscription performance of the TDengine, you must use the configuration file method and specify the function type to test via the parameter `filetype` in the configuration file. +======= +taosBenchmark supports the complete performance testing of TDengine by providing functionally to write, query, and subscribe. These three functions are mutually exclusive, users can only select one of them each time taosBenchmark runs. The query and subscribe functionalities are only configurable using a json configuration file by specifying the parameter `filetype`, while write can be performed through both the command-line and a configuration file. +>>>>>>> 108548b4d6 (docs: typo) **Make sure that the TDengine cluster is running correctly before running taosBenchmark. ** @@ -57,9 +61,8 @@ Use the following command-line to run taosBenchmark and control its behavior via taosBenchmark -f ``` -**Here are a few examples of configuration files:** - -#### Example of inserting a scenario JSON configuration file +#### Configuration file examples +##### Example of inserting a scenario JSON configuration file
insert.json @@ -70,7 +73,7 @@ taosBenchmark -f
-#### Query Scenario JSON Profile Example +##### Query Scenario JSON Profile Example
query.json @@ -81,7 +84,7 @@ taosBenchmark -f
-#### Subscription JSON configuration example +##### Subscription JSON configuration example
subscribe.json @@ -172,7 +175,7 @@ taosBenchmark -A INT,DOUBLE,NCHAR,BINARY\(16\) Switch parameter specifying whether to use escape characters in the super table and sub-table names. By default is not used. - **-C/--chinese** : - Switch specifying whether to use Unicode Chinese characters in nchar and binary. By default is not used. + specify whether to use Unicode Chinese characters in nchar and binary, the default is no. - **-N/--normal-table** : This parameter indicates that taosBenchmark will create only normal tables instead of super tables. The default value is false. It can be used if the insert mode is taosc, stmt, and rest. @@ -373,7 +376,7 @@ The configuration parameters for querying the sub-tables or the normal tables ar - **sqls**. - **sql**: the SQL command to be executed. - - **result**: the file to save the query result. If it is unspecified, taosBenchark will not save the result. + - **result**: the file to save the query result. If it is unspecified, taosBenchmark will not save the result. #### Configuration parameters of query super table diff --git a/docs/en/20-third-party/01-grafana.mdx b/docs/en/20-third-party/01-grafana.mdx index 9076f163be..5dbeb31a23 100644 --- a/docs/en/20-third-party/01-grafana.mdx +++ b/docs/en/20-third-party/01-grafana.mdx @@ -31,38 +31,41 @@ TDengine currently supports Grafana versions 7.5 and above. Users can go to the ### Install Grafana Plugin and Configure Data Source - + -Set the url and authorization environment variables by `export` or a [`.env`(dotenv) file](https://hexdocs.pm/dotenvy/dotenv-file-format.html): +Under Grafana 8, plugin catalog allows you to [browse and manage plugins within Grafana](https://grafana.com/docs/grafana/next/administration/plugin-management/#plugin-catalog) (but for Grafana 7.x, use **With Script** or **Install & Configure Manually**). Find the page at **Configurations > Plugins**, search **TDengine** and click it to install. -```sh -export TDENGINE_API=http://tdengine.local:6041 -# user + password -export TDENGINE_USER=user -export TDENGINE_PASSWORD=password +![Search tdengine in grafana plugins](./grafana/grafana-plugin-search-tdengine.png) -# Other useful variables -# - If to install TDengine data source, default is true -export TDENGINE_DS_ENABLED=false -# - Data source name to be created, default is TDengine -export TDENGINE_DS_NAME=TDengine -# - Data source organization id, default is 1 -export GF_ORG_ID=1 -# - Data source is editable in admin ui or not, default is 0 (false) -export TDENGINE_EDITABLE=1 -``` +Installation may cost some minutes, then you can **Create a TDengine data source**: -Run `install.sh`: +![Install and configure Grafana data source](./grafana/grafana-install-and-config.png) -```sh -bash -c "$(curl -fsSL https://raw.githubusercontent.com/taosdata/grafanaplugin/master/install.sh)" -``` +Then you can add a TDengine data source by filling up the configuration options. -With this script, TDengine data source plugin and the Grafana data source will be installed and created automatically with Grafana provisioning configurations. Save the script and type `./install.sh --help` for the full usage of the script. +![TDengine Database Grafana plugin add data source](./grafana/grafana-data-source.png) -And then, restart Grafana service and open Grafana in web-browser, usually . +You can create dashboards with TDengine now. + + +On a server with Grafana installed, run `install.sh` with TDengine url and username/passwords will install TDengine data source plugin and add a data source named TDengine. This is the recommended way for Grafana 7.x or [Grafana provisioning](https://grafana.com/docs/grafana/latest/administration/provisioning/) users. + +```sh +bash -c "$(curl -fsSL \ + https://raw.githubusercontent.com/taosdata/grafanaplugin/master/install.sh)" -- \ + -a http://localhost:6041 \ + -u root \ + -p taosdata +``` + +Restart Grafana service and open Grafana in web-browser, usually . + +Save the script and type `./install.sh --help` for the full usage of the script. + + + Follow the installation steps in [Grafana](https://grafana.com/grafana/plugins/tdengine-datasource/?tab=installation) with the [``grafana-cli`` command-line tool](https://grafana.com/docs/grafana/latest/administration/cli/) for plugin installation. @@ -115,6 +118,73 @@ Click `Save & Test` to test. You should see a success message if the test worked ![TDengine Database TDinsight plugin add database 4](./grafana/add_datasource4.webp) + + + +Please refer to [Install plugins in the Docker container](https://grafana.com/docs/grafana/next/setup-grafana/installation/docker/#install-plugins-in-the-docker-container). This will install `tdengine-datasource` plugin when Grafana container starts: + +```bash +docker run -d \ + -p 3000:3000 \ + --name=grafana \ + -e "GF_INSTALL_PLUGINS=tdengine-datasource" \ + grafana/grafana +``` + +You can setup a zero-configuration stack for TDengine + Grafana by [docker-compose](https://docs.docker.com/compose/) and [Grafana provisioning](https://grafana.com/docs/grafana/latest/administration/provisioning/) file： + +1. Save the provisioning configuration file to `tdengine.yml`. + + ```yml + apiVersion: 1 + datasources: + - name: TDengine + type: tdengine-datasource + orgId: 1 + url: "$TDENGINE_API" + isDefault: true + secureJsonData: + url: "$TDENGINE_URL" + basicAuth: "$TDENGINE_BASIC_AUTH" + token: "$TDENGINE_CLOUD_TOKEN" + version: 1 + editable: true + ``` + +2. Write `docker-compose.yml` with [TDengine](https://hub.docker.com/r/tdengine/tdengine) and [Grafana](https://hub.docker.com/r/grafana/grafana) image. + + ```yml + version: "3.7" + + services: + tdengine: + image: tdengine/tdengine:2.6.0.2 + environment: + TAOS_FQDN: tdengine + volumes: + - tdengine-data:/var/lib/taos/ + grafana: + image: grafana/grafana:8.5.6 + volumes: + - ./tdengine.yml/:/etc/grafana/provisioning/tdengine.yml + - grafana-data:/var/lib/grafana + environment: + # install tdengine plugin at start + GF_INSTALL_PLUGINS: "tdengine-datasource" + TDENGINE_URL: "http://tdengine:6041" + #printf "$TDENGINE_USER:$TDENGINE_PASSWORD" | base64 + TDENGINE_BASIC_AUTH: "cm9vdDp0YmFzZTEyNQ==" + ports: + - 3000:3000 + volumes: + grafana-data: + tdengine-data: + ``` + +3. Start TDengine and Grafana services: `docker-compose up -d`. + +Open Grafana , and you can add dashboard with TDengine now. + diff --git a/docs/en/20-third-party/grafana/grafana-data-source.png b/docs/en/20-third-party/grafana/grafana-data-source.png new file mode 100644 index 0000000000..989ffcca0b Binary files /dev/null and b/docs/en/20-third-party/grafana/grafana-data-source.png differ diff --git a/docs/en/20-third-party/grafana/grafana-install-and-config.png b/docs/en/20-third-party/grafana/grafana-install-and-config.png new file mode 100644 index 0000000000..b918da8b2d Binary files /dev/null and b/docs/en/20-third-party/grafana/grafana-install-and-config.png differ diff --git a/docs/en/20-third-party/grafana/grafana-plugin-search-tdengine.png b/docs/en/20-third-party/grafana/grafana-plugin-search-tdengine.png new file mode 100644 index 0000000000..cf3b66977b Binary files /dev/null and b/docs/en/20-third-party/grafana/grafana-plugin-search-tdengine.png differ diff --git a/docs/zh/20-third-party/01-grafana.mdx b/docs/zh/20-third-party/01-grafana.mdx index 09c0d786cf..93090ffd38 100644 --- a/docs/zh/20-third-party/01-grafana.mdx +++ b/docs/zh/20-third-party/01-grafana.mdx @@ -29,39 +29,41 @@ TDengine 能够与开源数据可视化系统 [Grafana](https://www.grafana.com/ ### 安装 Grafana Plugin 并配置数据源 + + +使用 Grafana 最新版本（8.5+），您可以在 Grafana 中[浏览和管理插件](https://grafana.com/docs/grafana/next/administration/plugin-management/#plugin-catalog)（对于 7.x 版本，请使用 **安装脚本** 或 **手动安装并配置** 方式）。在 Grafana 管理界面中的 **Configurations > Plugins** 页面直接搜索并按照提示安装 TDengine。 + +![Search tdengine in grafana plugins](grafana-plugin-search-tdengine.png) + +如图示即安装完毕，按照指示 **Create a TDengine data source** 添加数据源。 + +![Install and configure Grafana data source](grafana-install-and-config.png) + +输入 TDengine 相关配置，完成数据源配置。 + +![TDengine Database Grafana plugin add data source](./grafana-data-source.png) + +配置完毕，现在可以使用 TDengine 创建 Dashboard 了。 + + -将集群信息设置为环境变量；也可以使用 `.env` 文件，请参考 [dotenv](https://hexdocs.pm/dotenvy/dotenv-file-format.html)： +对于使用 Grafana 7.x 版本或使用 [Grafana Provisioning](https://grafana.com/docs/grafana/latest/administration/provisioning/) 配置的用户，可以在 Grafana 服务器上使用安装脚本自动安装插件即添加数据源 Provisioning 配置文件。 ```sh -export TDENGINE_API=http://tdengine.local:6041 -# user + password -export TDENGINE_USER=user -export TDENGINE_PASSWORD=password - -# 其他环境变量： -# - 是否安装数据源，默认为 true，表示安装 -export TDENGINE_DS_ENABLED=false -# - 数据源名称，默认为 TDengine -export TDENGINE_DS_NAME=TDengine -# - 数据源所属组织 ID，默认为 1 -export GF_ORG_ID=1 -# - 数据源是否可通过管理面板编辑，默认为 0，表示不可编辑 -export TDENGINE_EDITABLE=1 +bash -c "$(curl -fsSL \ + https://raw.githubusercontent.com/taosdata/grafanaplugin/master/install.sh)" -- \ + -a http://localhost:6041 \ + -u root \ + -p taosdata ``` -运行安装脚本： - -```sh -bash -c "$(curl -fsSL https://raw.githubusercontent.com/taosdata/grafanaplugin/master/install.sh)" -``` - -该脚本将自动安装 Grafana 插件并配置数据源。安装完毕后，需要重启 Grafana 服务后生效。 +安装完毕后，需要重启 Grafana 服务后方可生效。 保存该脚本并执行 `./install.sh --help` 可查看详细帮助文档。 - + 使用 [`grafana-cli` 命令行工具](https://grafana.com/docs/grafana/latest/administration/cli/) 进行插件[安装](https://grafana.com/grafana/plugins/tdengine-datasource/?tab=installation)。 @@ -113,6 +115,73 @@ GF_INSTALL_PLUGINS=tdengine-datasource ![TDengine Database Grafana plugin add data source](./add_datasource4.webp) + + + +参考 [Grafana 容器化安装说明](https://grafana.com/docs/grafana/next/setup-grafana/installation/docker/#install-plugins-in-the-docker-container)。使用如下命令启动一个容器，并自动安装 TDengine 插件： + +```bash +docker run -d \ + -p 3000:3000 \ + --name=grafana \ + -e "GF_INSTALL_PLUGINS=tdengine-datasource" \ + grafana/grafana +``` + +使用 docker-compose，配置 Grafana Provisioning 自动化配置，体验 TDengine + Grafana 组合的零配置启动： + +1. 保存该文件为 `tdengine.yml`。 + + ```yml + apiVersion: 1 + datasources: + - name: TDengine + type: tdengine-datasource + orgId: 1 + url: "$TDENGINE_API" + isDefault: true + secureJsonData: + url: "$TDENGINE_URL" + basicAuth: "$TDENGINE_BASIC_AUTH" + token: "$TDENGINE_CLOUD_TOKEN" + version: 1 + editable: true + ``` + +2. 保存该文件为 `docker-compose.yml`。 + + ```yml + version: "3.7" + + services: + tdengine: + image: tdengine/tdengine:2.6.0.2 + environment: + TAOS_FQDN: tdengine + volumes: + - tdengine-data:/var/lib/taos/ + grafana: + image: grafana/grafana:8.5.6 + volumes: + - ./tdengine.yml/:/etc/grafana/provisioning/tdengine.yml + - grafana-data:/var/lib/grafana + environment: + # install tdengine plugin at start + GF_INSTALL_PLUGINS: "tdengine-datasource" + TDENGINE_URL: "http://tdengine:6041" + #printf "$TDENGINE_USER:$TDENGINE_PASSWORD" | base64 + TDENGINE_BASIC_AUTH: "cm9vdDp0YmFzZTEyNQ==" + ports: + - 3000:3000 + volumes: + grafana-data: + tdengine-data: + ``` + +3. 使用 docker-compose 命令启动 TDengine + Grafana ：`docker-compose up -d`。 + +打开 Grafana ，现在可以添加 Dashboard 了。 + diff --git a/docs/zh/20-third-party/grafana-data-source.png b/docs/zh/20-third-party/grafana-data-source.png new file mode 100644 index 0000000000..989ffcca0b Binary files /dev/null and b/docs/zh/20-third-party/grafana-data-source.png differ diff --git a/docs/zh/20-third-party/grafana-install-and-config.png b/docs/zh/20-third-party/grafana-install-and-config.png new file mode 100644 index 0000000000..b918da8b2d Binary files /dev/null and b/docs/zh/20-third-party/grafana-install-and-config.png differ diff --git a/docs/zh/20-third-party/grafana-plugin-search-tdengine.png b/docs/zh/20-third-party/grafana-plugin-search-tdengine.png new file mode 100644 index 0000000000..cf3b66977b Binary files /dev/null and b/docs/zh/20-third-party/grafana-plugin-search-tdengine.png differ diff --git a/examples/c/stream_demo.c b/examples/c/stream_demo.c index ab59fa5e47..5a141867e7 100644 --- a/examples/c/stream_demo.c +++ b/examples/c/stream_demo.c @@ -32,6 +32,13 @@ int32_t init_env() { } taos_free_result(pRes); + pRes = taos_query(pConn, "create database if not exists abc2 vgroups 20"); + if (taos_errno(pRes) != 0) { + printf("error in create db, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + pRes = taos_query(pConn, "use abc1"); if (taos_errno(pRes) != 0) { printf("error in use db, reason:%s\n", taos_errstr(pRes)); @@ -81,9 +88,9 @@ int32_t create_stream() { /*const char* sql = "select min(k), max(k), sum(k) as sum_of_k from st1";*/ /*const char* sql = "select sum(k) from tu1 interval(10m)";*/ /*pRes = tmq_create_stream(pConn, "stream1", "out1", sql);*/ - pRes = taos_query( - pConn, - "create stream stream1 trigger max_delay 10s into outstb as select _wstartts, sum(k) from st1 interval(10m)"); + pRes = taos_query(pConn, + "create stream stream1 trigger at_once into abc2.outstb as select _wstartts, sum(k) from st1 " + "partition by tbname interval(10m) "); if (taos_errno(pRes) != 0) { printf("failed to create stream stream1, reason:%s\n", taos_errstr(pRes)); return -1; diff --git a/examples/c/tmq.c b/examples/c/tmq.c index 2e8aa21da7..7870d7d9a1 100644 --- a/examples/c/tmq.c +++ b/examples/c/tmq.c @@ -47,7 +47,7 @@ int32_t init_env() { return -1; } - TAOS_RES* pRes = taos_query(pConn, "create database if not exists abc1 vgroups 2"); + TAOS_RES* pRes = taos_query(pConn, "create database if not exists abc1 vgroups 1"); if (taos_errno(pRes) != 0) { printf("error in create db, reason:%s\n", taos_errstr(pRes)); return -1; @@ -146,8 +146,8 @@ int32_t create_topic() { return 0; } -void tmq_commit_cb_print(tmq_t* tmq, tmq_resp_err_t resp, tmq_topic_vgroup_list_t* offsets, void* param) { - printf("commit %d tmq %p offsets %p param %p\n", resp, tmq, offsets, param); +void tmq_commit_cb_print(tmq_t* tmq, int32_t code, void* param) { + printf("commit %d tmq %p param %p\n", code, tmq, param); } tmq_t* build_consumer() { @@ -167,7 +167,7 @@ tmq_t* build_consumer() { tmq_conf_set(conf, "td.connect.user", "root"); tmq_conf_set(conf, "td.connect.pass", "taosdata"); tmq_conf_set(conf, "msg.with.table.name", "true"); - tmq_conf_set(conf, "enable.auto.commit", "false"); + tmq_conf_set(conf, "enable.auto.commit", "true"); tmq_conf_set_auto_commit_cb(conf, tmq_commit_cb_print, NULL); tmq_t* tmq = tmq_consumer_new(conf, NULL, 0); assert(tmq); @@ -183,10 +183,10 @@ tmq_list_t* build_topic_list() { } void basic_consume_loop(tmq_t* tmq, tmq_list_t* topics) { - tmq_resp_err_t err; + int32_t code; - if ((err = tmq_subscribe(tmq, topics))) { - fprintf(stderr, "%% Failed to start consuming topics: %s\n", tmq_err2str(err)); + if ((code = tmq_subscribe(tmq, topics))) { + fprintf(stderr, "%% Failed to start consuming topics: %s\n", tmq_err2str(code)); printf("subscribe err\n"); return; } @@ -201,12 +201,13 @@ void basic_consume_loop(tmq_t* tmq, tmq_list_t* topics) { taos_free_result(tmqmessage); /*} else {*/ /*break;*/ + /*tmq_commit_sync(tmq, NULL);*/ } } - err = tmq_consumer_close(tmq); - if (err) - fprintf(stderr, "%% Failed to close consumer: %s\n", tmq_err2str(err)); + code = tmq_consumer_close(tmq); + if (code) + fprintf(stderr, "%% Failed to close consumer: %s\n", tmq_err2str(code)); else fprintf(stderr, "%% Consumer closed\n"); } @@ -214,11 +215,11 @@ void basic_consume_loop(tmq_t* tmq, tmq_list_t* topics) { void sync_consume_loop(tmq_t* tmq, tmq_list_t* topics) { static const int MIN_COMMIT_COUNT = 1; - int msg_count = 0; - tmq_resp_err_t err; + int msg_count = 0; + int32_t code; - if ((err = tmq_subscribe(tmq, topics))) { - fprintf(stderr, "%% Failed to start consuming topics: %s\n", tmq_err2str(err)); + if ((code = tmq_subscribe(tmq, topics))) { + fprintf(stderr, "%% Failed to start consuming topics: %s\n", tmq_err2str(code)); return; } @@ -239,14 +240,14 @@ void sync_consume_loop(tmq_t* tmq, tmq_list_t* topics) { msg_process(tmqmessage); taos_free_result(tmqmessage); - /*tmq_commit_async(tmq, NULL, tmq_commit_cb_print, NULL);*/ + /*tmq_commit_sync(tmq, NULL);*/ /*if ((++msg_count % MIN_COMMIT_COUNT) == 0) tmq_commit(tmq, NULL, 0);*/ } } - err = tmq_consumer_close(tmq); - if (err) - fprintf(stderr, "%% Failed to close consumer: %s\n", tmq_err2str(err)); + code = tmq_consumer_close(tmq); + if (code) + fprintf(stderr, "%% Failed to close consumer: %s\n", tmq_err2str(code)); else fprintf(stderr, "%% Consumer closed\n"); } diff --git a/include/client/taos.h b/include/client/taos.h index deb4276b54..61538e392a 100644 --- a/include/client/taos.h +++ b/include/client/taos.h @@ -126,47 +126,47 @@ typedef struct setConfRet { char retMsg[RET_MSG_LENGTH]; } setConfRet; -DLL_EXPORT void taos_cleanup(void); -DLL_EXPORT int taos_options(TSDB_OPTION option, const void *arg, ...); -DLL_EXPORT setConfRet taos_set_config(const char *config); -DLL_EXPORT int taos_init(void); -DLL_EXPORT TAOS *taos_connect(const char *ip, const char *user, const char *pass, const char *db, uint16_t port); -DLL_EXPORT TAOS *taos_connect_auth(const char *ip, const char *user, const char *auth, const char *db, uint16_t port); -DLL_EXPORT void taos_close(TAOS *taos); +DLL_EXPORT void taos_cleanup(void); +DLL_EXPORT int taos_options(TSDB_OPTION option, const void *arg, ...); +DLL_EXPORT setConfRet taos_set_config(const char *config); +DLL_EXPORT int taos_init(void); +DLL_EXPORT TAOS *taos_connect(const char *ip, const char *user, const char *pass, const char *db, uint16_t port); +DLL_EXPORT TAOS *taos_connect_auth(const char *ip, const char *user, const char *auth, const char *db, uint16_t port); +DLL_EXPORT void taos_close(TAOS *taos); -const char *taos_data_type(int type); +const char *taos_data_type(int type); -DLL_EXPORT TAOS_STMT *taos_stmt_init(TAOS *taos); -DLL_EXPORT int taos_stmt_prepare(TAOS_STMT *stmt, const char *sql, unsigned long length); -DLL_EXPORT int taos_stmt_set_tbname_tags(TAOS_STMT *stmt, const char *name, TAOS_MULTI_BIND *tags); -DLL_EXPORT int taos_stmt_set_tbname(TAOS_STMT *stmt, const char *name); -DLL_EXPORT int taos_stmt_set_tags(TAOS_STMT *stmt, TAOS_MULTI_BIND *tags); -DLL_EXPORT int taos_stmt_set_sub_tbname(TAOS_STMT *stmt, const char *name); -DLL_EXPORT int taos_stmt_get_tag_fields(TAOS_STMT *stmt, int *fieldNum, TAOS_FIELD_E **fields); -DLL_EXPORT int taos_stmt_get_col_fields(TAOS_STMT *stmt, int *fieldNum, TAOS_FIELD_E **fields); +DLL_EXPORT TAOS_STMT *taos_stmt_init(TAOS *taos); +DLL_EXPORT int taos_stmt_prepare(TAOS_STMT *stmt, const char *sql, unsigned long length); +DLL_EXPORT int taos_stmt_set_tbname_tags(TAOS_STMT *stmt, const char *name, TAOS_MULTI_BIND *tags); +DLL_EXPORT int taos_stmt_set_tbname(TAOS_STMT *stmt, const char *name); +DLL_EXPORT int taos_stmt_set_tags(TAOS_STMT *stmt, TAOS_MULTI_BIND *tags); +DLL_EXPORT int taos_stmt_set_sub_tbname(TAOS_STMT *stmt, const char *name); +DLL_EXPORT int taos_stmt_get_tag_fields(TAOS_STMT *stmt, int *fieldNum, TAOS_FIELD_E **fields); +DLL_EXPORT int taos_stmt_get_col_fields(TAOS_STMT *stmt, int *fieldNum, TAOS_FIELD_E **fields); -DLL_EXPORT int taos_stmt_is_insert(TAOS_STMT *stmt, int *insert); -DLL_EXPORT int taos_stmt_num_params(TAOS_STMT *stmt, int *nums); -DLL_EXPORT int taos_stmt_get_param(TAOS_STMT *stmt, int idx, int *type, int *bytes); -DLL_EXPORT int taos_stmt_bind_param(TAOS_STMT *stmt, TAOS_MULTI_BIND *bind); -DLL_EXPORT int taos_stmt_bind_param_batch(TAOS_STMT *stmt, TAOS_MULTI_BIND *bind); -DLL_EXPORT int taos_stmt_bind_single_param_batch(TAOS_STMT *stmt, TAOS_MULTI_BIND *bind, int colIdx); -DLL_EXPORT int taos_stmt_add_batch(TAOS_STMT *stmt); -DLL_EXPORT int taos_stmt_execute(TAOS_STMT *stmt); -DLL_EXPORT TAOS_RES *taos_stmt_use_result(TAOS_STMT *stmt); -DLL_EXPORT int taos_stmt_close(TAOS_STMT *stmt); -DLL_EXPORT char *taos_stmt_errstr(TAOS_STMT *stmt); -DLL_EXPORT int taos_stmt_affected_rows(TAOS_STMT *stmt); -DLL_EXPORT int taos_stmt_affected_rows_once(TAOS_STMT *stmt); +DLL_EXPORT int taos_stmt_is_insert(TAOS_STMT *stmt, int *insert); +DLL_EXPORT int taos_stmt_num_params(TAOS_STMT *stmt, int *nums); +DLL_EXPORT int taos_stmt_get_param(TAOS_STMT *stmt, int idx, int *type, int *bytes); +DLL_EXPORT int taos_stmt_bind_param(TAOS_STMT *stmt, TAOS_MULTI_BIND *bind); +DLL_EXPORT int taos_stmt_bind_param_batch(TAOS_STMT *stmt, TAOS_MULTI_BIND *bind); +DLL_EXPORT int taos_stmt_bind_single_param_batch(TAOS_STMT *stmt, TAOS_MULTI_BIND *bind, int colIdx); +DLL_EXPORT int taos_stmt_add_batch(TAOS_STMT *stmt); +DLL_EXPORT int taos_stmt_execute(TAOS_STMT *stmt); +DLL_EXPORT TAOS_RES *taos_stmt_use_result(TAOS_STMT *stmt); +DLL_EXPORT int taos_stmt_close(TAOS_STMT *stmt); +DLL_EXPORT char *taos_stmt_errstr(TAOS_STMT *stmt); +DLL_EXPORT int taos_stmt_affected_rows(TAOS_STMT *stmt); +DLL_EXPORT int taos_stmt_affected_rows_once(TAOS_STMT *stmt); -DLL_EXPORT TAOS_RES *taos_query(TAOS *taos, const char *sql); +DLL_EXPORT TAOS_RES *taos_query(TAOS *taos, const char *sql); -DLL_EXPORT TAOS_ROW taos_fetch_row(TAOS_RES *res); -DLL_EXPORT int taos_result_precision(TAOS_RES *res); // get the time precision of result -DLL_EXPORT void taos_free_result(TAOS_RES *res); -DLL_EXPORT int taos_field_count(TAOS_RES *res); -DLL_EXPORT int taos_num_fields(TAOS_RES *res); -DLL_EXPORT int taos_affected_rows(TAOS_RES *res); +DLL_EXPORT TAOS_ROW taos_fetch_row(TAOS_RES *res); +DLL_EXPORT int taos_result_precision(TAOS_RES *res); // get the time precision of result +DLL_EXPORT void taos_free_result(TAOS_RES *res); +DLL_EXPORT int taos_field_count(TAOS_RES *res); +DLL_EXPORT int taos_num_fields(TAOS_RES *res); +DLL_EXPORT int taos_affected_rows(TAOS_RES *res); DLL_EXPORT TAOS_FIELD *taos_fetch_fields(TAOS_RES *res); DLL_EXPORT int taos_select_db(TAOS *taos, const char *db); @@ -181,8 +181,8 @@ DLL_EXPORT int *taos_get_column_data_offset(TAOS_RES *res, int columnInde DLL_EXPORT int taos_validate_sql(TAOS *taos, const char *sql); DLL_EXPORT void taos_reset_current_db(TAOS *taos); -DLL_EXPORT int *taos_fetch_lengths(TAOS_RES *res); -DLL_EXPORT TAOS_ROW *taos_result_block(TAOS_RES *res); +DLL_EXPORT int *taos_fetch_lengths(TAOS_RES *res); +DLL_EXPORT TAOS_ROW *taos_result_block(TAOS_RES *res); DLL_EXPORT const char *taos_get_server_info(TAOS *taos); DLL_EXPORT const char *taos_get_client_info(); @@ -192,8 +192,8 @@ DLL_EXPORT int taos_errno(TAOS_RES *tres); DLL_EXPORT void taos_query_a(TAOS *taos, const char *sql, __taos_async_fn_t fp, void *param); DLL_EXPORT void taos_fetch_rows_a(TAOS_RES *res, __taos_async_fn_t fp, void *param); -DLL_EXPORT void taos_fetch_raw_block_a(TAOS_RES* res, __taos_async_fn_t fp, void* param); -DLL_EXPORT const void *taos_get_raw_block(TAOS_RES* res); +DLL_EXPORT void taos_fetch_raw_block_a(TAOS_RES *res, __taos_async_fn_t fp, void *param); +DLL_EXPORT const void *taos_get_raw_block(TAOS_RES *res); // Shuduo: temporary enable for app build #if 1 @@ -209,21 +209,11 @@ DLL_EXPORT TAOS_RES *taos_schemaless_insert(TAOS *taos, char *lines[], int numLi /* --------------------------TMQ INTERFACE------------------------------- */ -enum { - TMQ_RESP_ERR__FAIL = -1, - TMQ_RESP_ERR__SUCCESS = 0, -}; - -typedef int32_t tmq_resp_err_t; - -typedef struct tmq_t tmq_t; -typedef struct tmq_topic_vgroup_t tmq_topic_vgroup_t; -typedef struct tmq_topic_vgroup_list_t tmq_topic_vgroup_list_t; - +typedef struct tmq_t tmq_t; typedef struct tmq_conf_t tmq_conf_t; typedef struct tmq_list_t tmq_list_t; -typedef void(tmq_commit_cb(tmq_t *, tmq_resp_err_t, tmq_topic_vgroup_list_t *, void *param)); +typedef void(tmq_commit_cb(tmq_t *, int32_t code, void *param)); DLL_EXPORT tmq_list_t *tmq_list_new(); DLL_EXPORT int32_t tmq_list_append(tmq_list_t *, const char *); @@ -233,25 +223,17 @@ DLL_EXPORT char **tmq_list_to_c_array(const tmq_list_t *); DLL_EXPORT tmq_t *tmq_consumer_new(tmq_conf_t *conf, char *errstr, int32_t errstrLen); -DLL_EXPORT const char *tmq_err2str(tmq_resp_err_t); +DLL_EXPORT const char *tmq_err2str(int32_t code); /* ------------------------TMQ CONSUMER INTERFACE------------------------ */ -DLL_EXPORT tmq_resp_err_t tmq_subscribe(tmq_t *tmq, const tmq_list_t *topic_list); -DLL_EXPORT tmq_resp_err_t tmq_unsubscribe(tmq_t *tmq); -DLL_EXPORT tmq_resp_err_t tmq_subscription(tmq_t *tmq, tmq_list_t **topics); - -// timeout: -1 means infinitely waiting +DLL_EXPORT int32_t tmq_subscribe(tmq_t *tmq, const tmq_list_t *topic_list); +DLL_EXPORT int32_t tmq_unsubscribe(tmq_t *tmq); +DLL_EXPORT int32_t tmq_subscription(tmq_t *tmq, tmq_list_t **topics); DLL_EXPORT TAOS_RES *tmq_consumer_poll(tmq_t *tmq, int64_t timeout); - -DLL_EXPORT tmq_resp_err_t tmq_consumer_close(tmq_t *tmq); -DLL_EXPORT tmq_resp_err_t tmq_commit_sync(tmq_t *tmq, const tmq_topic_vgroup_list_t *offsets); -DLL_EXPORT void tmq_commit_async(tmq_t *tmq, const tmq_topic_vgroup_list_t *offsets, tmq_commit_cb *cb, void *param); - -#if 0 -DLL_EXPORT tmq_resp_err_t tmq_commit(tmq_t *tmq, const tmq_topic_vgroup_list_t *offsets, int32_t async); -DLL_EXPORT tmq_resp_err_t tmq_seek(tmq_t *tmq, const tmq_topic_vgroup_t *offset); -#endif +DLL_EXPORT int32_t tmq_consumer_close(tmq_t *tmq); +DLL_EXPORT int32_t tmq_commit_sync(tmq_t *tmq, const TAOS_RES *msg); +DLL_EXPORT void tmq_commit_async(tmq_t *tmq, const TAOS_RES *msg, tmq_commit_cb *cb, void *param); /* ----------------------TMQ CONFIGURATION INTERFACE---------------------- */ @@ -275,11 +257,6 @@ DLL_EXPORT const char *tmq_get_db_name(TAOS_RES *res); DLL_EXPORT int32_t tmq_get_vgroup_id(TAOS_RES *res); DLL_EXPORT const char *tmq_get_table_name(TAOS_RES *res); -#if 0 -DLL_EXPORT int64_t tmq_get_request_offset(tmq_message_t *message); -DLL_EXPORT int64_t tmq_get_response_offset(tmq_message_t *message); -#endif - /* ------------------------------ TMQ END -------------------------------- */ #if 1 // Shuduo: temporary enable for app build diff --git a/include/common/tcommon.h b/include/common/tcommon.h index 2e646f4769..a05287761e 100644 --- a/include/common/tcommon.h +++ b/include/common/tcommon.h @@ -35,6 +35,7 @@ enum { TMQ_MSG_TYPE__DUMMY = 0, TMQ_MSG_TYPE__POLL_RSP, TMQ_MSG_TYPE__EP_RSP, + TMQ_MSG_TYPE__END_RSP, }; typedef enum EStreamType { diff --git a/include/common/tmsg.h b/include/common/tmsg.h index ada11d90b2..857a554f62 100644 --- a/include/common/tmsg.h +++ b/include/common/tmsg.h @@ -429,8 +429,10 @@ STSchema* tdGetSTSChemaFromSSChema(SSchema** pSchema, int32_t nCols); typedef struct { char name[TSDB_TABLE_FNAME_LEN]; int8_t igExists; - float xFilesFactor; - int32_t delay; + int64_t delay1; + int64_t delay2; + int64_t watermark1; + int64_t watermark2; int32_t ttl; int32_t numOfColumns; int32_t numOfTags; diff --git a/include/common/ttokendef.h b/include/common/ttokendef.h index 061c734484..1edfcbabad 100644 --- a/include/common/ttokendef.h +++ b/include/common/ttokendef.h @@ -127,134 +127,133 @@ #define TK_BLOB 109 #define TK_VARBINARY 110 #define TK_DECIMAL 111 -#define TK_FILE_FACTOR 112 -#define TK_NK_FLOAT 113 +#define TK_MAX_DELAY 112 +#define TK_WATERMARK 113 #define TK_ROLLUP 114 #define TK_TTL 115 #define TK_SMA 116 -#define TK_SHOW 117 -#define TK_DATABASES 118 -#define TK_TABLES 119 -#define TK_STABLES 120 -#define TK_MNODES 121 -#define TK_MODULES 122 -#define TK_QNODES 123 -#define TK_FUNCTIONS 124 -#define TK_INDEXES 125 -#define TK_ACCOUNTS 126 -#define TK_APPS 127 -#define TK_CONNECTIONS 128 -#define TK_LICENCE 129 -#define TK_GRANTS 130 -#define TK_QUERIES 131 -#define TK_SCORES 132 -#define TK_TOPICS 133 -#define TK_VARIABLES 134 -#define TK_BNODES 135 -#define TK_SNODES 136 -#define TK_CLUSTER 137 -#define TK_TRANSACTIONS 138 -#define TK_LIKE 139 -#define TK_INDEX 140 -#define TK_FULLTEXT 141 -#define TK_FUNCTION 142 -#define TK_INTERVAL 143 -#define TK_TOPIC 144 -#define TK_AS 145 -#define TK_CONSUMER 146 -#define TK_GROUP 147 -#define TK_DESC 148 -#define TK_DESCRIBE 149 -#define TK_RESET 150 -#define TK_QUERY 151 -#define TK_CACHE 152 -#define TK_EXPLAIN 153 -#define TK_ANALYZE 154 -#define TK_VERBOSE 155 -#define TK_NK_BOOL 156 -#define TK_RATIO 157 -#define TK_COMPACT 158 -#define TK_VNODES 159 -#define TK_IN 160 -#define TK_OUTPUTTYPE 161 -#define TK_AGGREGATE 162 -#define TK_BUFSIZE 163 -#define TK_STREAM 164 -#define TK_INTO 165 -#define TK_TRIGGER 166 -#define TK_AT_ONCE 167 -#define TK_WINDOW_CLOSE 168 -#define TK_MAX_DELAY 169 -#define TK_WATERMARK 170 -#define TK_KILL 171 -#define TK_CONNECTION 172 -#define TK_TRANSACTION 173 -#define TK_BALANCE 174 -#define TK_VGROUP 175 -#define TK_MERGE 176 -#define TK_REDISTRIBUTE 177 -#define TK_SPLIT 178 -#define TK_SYNCDB 179 -#define TK_DELETE 180 -#define TK_NULL 181 -#define TK_NK_QUESTION 182 -#define TK_NK_ARROW 183 -#define TK_ROWTS 184 -#define TK_TBNAME 185 -#define TK_QSTARTTS 186 -#define TK_QENDTS 187 -#define TK_WSTARTTS 188 -#define TK_WENDTS 189 -#define TK_WDURATION 190 -#define TK_CAST 191 -#define TK_NOW 192 -#define TK_TODAY 193 -#define TK_TIMEZONE 194 -#define TK_COUNT 195 -#define TK_FIRST 196 -#define TK_LAST 197 -#define TK_LAST_ROW 198 -#define TK_BETWEEN 199 -#define TK_IS 200 -#define TK_NK_LT 201 -#define TK_NK_GT 202 -#define TK_NK_LE 203 -#define TK_NK_GE 204 -#define TK_NK_NE 205 -#define TK_MATCH 206 -#define TK_NMATCH 207 -#define TK_CONTAINS 208 -#define TK_JOIN 209 -#define TK_INNER 210 -#define TK_SELECT 211 -#define TK_DISTINCT 212 -#define TK_WHERE 213 -#define TK_PARTITION 214 -#define TK_BY 215 -#define TK_SESSION 216 -#define TK_STATE_WINDOW 217 -#define TK_SLIDING 218 -#define TK_FILL 219 -#define TK_VALUE 220 -#define TK_NONE 221 -#define TK_PREV 222 -#define TK_LINEAR 223 -#define TK_NEXT 224 -#define TK_HAVING 225 -#define TK_ORDER 226 -#define TK_SLIMIT 227 -#define TK_SOFFSET 228 -#define TK_LIMIT 229 -#define TK_OFFSET 230 -#define TK_ASC 231 -#define TK_NULLS 232 -#define TK_ID 233 -#define TK_NK_BITNOT 234 -#define TK_INSERT 235 -#define TK_VALUES 236 -#define TK_IMPORT 237 -#define TK_NK_SEMI 238 -#define TK_FILE 239 +#define TK_FIRST 117 +#define TK_LAST 118 +#define TK_SHOW 119 +#define TK_DATABASES 120 +#define TK_TABLES 121 +#define TK_STABLES 122 +#define TK_MNODES 123 +#define TK_MODULES 124 +#define TK_QNODES 125 +#define TK_FUNCTIONS 126 +#define TK_INDEXES 127 +#define TK_ACCOUNTS 128 +#define TK_APPS 129 +#define TK_CONNECTIONS 130 +#define TK_LICENCE 131 +#define TK_GRANTS 132 +#define TK_QUERIES 133 +#define TK_SCORES 134 +#define TK_TOPICS 135 +#define TK_VARIABLES 136 +#define TK_BNODES 137 +#define TK_SNODES 138 +#define TK_CLUSTER 139 +#define TK_TRANSACTIONS 140 +#define TK_LIKE 141 +#define TK_INDEX 142 +#define TK_FULLTEXT 143 +#define TK_FUNCTION 144 +#define TK_INTERVAL 145 +#define TK_TOPIC 146 +#define TK_AS 147 +#define TK_CONSUMER 148 +#define TK_GROUP 149 +#define TK_DESC 150 +#define TK_DESCRIBE 151 +#define TK_RESET 152 +#define TK_QUERY 153 +#define TK_CACHE 154 +#define TK_EXPLAIN 155 +#define TK_ANALYZE 156 +#define TK_VERBOSE 157 +#define TK_NK_BOOL 158 +#define TK_RATIO 159 +#define TK_NK_FLOAT 160 +#define TK_COMPACT 161 +#define TK_VNODES 162 +#define TK_IN 163 +#define TK_OUTPUTTYPE 164 +#define TK_AGGREGATE 165 +#define TK_BUFSIZE 166 +#define TK_STREAM 167 +#define TK_INTO 168 +#define TK_TRIGGER 169 +#define TK_AT_ONCE 170 +#define TK_WINDOW_CLOSE 171 +#define TK_KILL 172 +#define TK_CONNECTION 173 +#define TK_TRANSACTION 174 +#define TK_BALANCE 175 +#define TK_VGROUP 176 +#define TK_MERGE 177 +#define TK_REDISTRIBUTE 178 +#define TK_SPLIT 179 +#define TK_SYNCDB 180 +#define TK_DELETE 181 +#define TK_NULL 182 +#define TK_NK_QUESTION 183 +#define TK_NK_ARROW 184 +#define TK_ROWTS 185 +#define TK_TBNAME 186 +#define TK_QSTARTTS 187 +#define TK_QENDTS 188 +#define TK_WSTARTTS 189 +#define TK_WENDTS 190 +#define TK_WDURATION 191 +#define TK_CAST 192 +#define TK_NOW 193 +#define TK_TODAY 194 +#define TK_TIMEZONE 195 +#define TK_COUNT 196 +#define TK_LAST_ROW 197 +#define TK_BETWEEN 198 +#define TK_IS 199 +#define TK_NK_LT 200 +#define TK_NK_GT 201 +#define TK_NK_LE 202 +#define TK_NK_GE 203 +#define TK_NK_NE 204 +#define TK_MATCH 205 +#define TK_NMATCH 206 +#define TK_CONTAINS 207 +#define TK_JOIN 208 +#define TK_INNER 209 +#define TK_SELECT 210 +#define TK_DISTINCT 211 +#define TK_WHERE 212 +#define TK_PARTITION 213 +#define TK_BY 214 +#define TK_SESSION 215 +#define TK_STATE_WINDOW 216 +#define TK_SLIDING 217 +#define TK_FILL 218 +#define TK_VALUE 219 +#define TK_NONE 220 +#define TK_PREV 221 +#define TK_LINEAR 222 +#define TK_NEXT 223 +#define TK_HAVING 224 +#define TK_ORDER 225 +#define TK_SLIMIT 226 +#define TK_SOFFSET 227 +#define TK_LIMIT 228 +#define TK_OFFSET 229 +#define TK_ASC 230 +#define TK_NULLS 231 +#define TK_ID 232 +#define TK_NK_BITNOT 233 +#define TK_INSERT 234 +#define TK_VALUES 235 +#define TK_IMPORT 236 +#define TK_NK_SEMI 237 +#define TK_FILE 238 #define TK_NK_SPACE 300 #define TK_NK_COMMENT 301 diff --git a/include/libs/executor/executor.h b/include/libs/executor/executor.h index fdedb947d7..083f6ae1b0 100644 --- a/include/libs/executor/executor.h +++ b/include/libs/executor/executor.h @@ -159,11 +159,14 @@ int64_t qGetQueriedTableUid(qTaskInfo_t tinfo); */ int32_t qGetQualifiedTableIdList(void* pTableList, const char* tagCond, int32_t tagCondLen, SArray* pTableIdList); - void qProcessFetchRsp(void* parent, struct SRpcMsg* pMsg, struct SEpSet* pEpSet); int32_t qGetExplainExecInfo(qTaskInfo_t tinfo, int32_t* resNum, SExplainExecInfo** pRes); +int32_t qSerializeTaskStatus(qTaskInfo_t tinfo, char** pOutput, int32_t* len); + +int32_t qDeserializeTaskStatus(qTaskInfo_t tinfo, const char* pInput, int32_t len); + #ifdef __cplusplus } #endif diff --git a/include/libs/function/function.h b/include/libs/function/function.h index 54762a0f17..0d8ae289cc 100644 --- a/include/libs/function/function.h +++ b/include/libs/function/function.h @@ -158,10 +158,10 @@ typedef struct tExprNode { int32_t nodeType; union { struct {// function node - char functionName[FUNCTIONS_NAME_MAX_LENGTH]; // todo refactor - int32_t functionId; - int32_t num; - struct SFunctionNode *pFunctNode; + char functionName[FUNCTIONS_NAME_MAX_LENGTH]; // todo refactor + int32_t functionId; + int32_t num; + struct SFunctionNode *pFunctNode; } _function; struct { diff --git a/include/libs/nodes/cmdnodes.h b/include/libs/nodes/cmdnodes.h index 1282836425..087d252ed7 100644 --- a/include/libs/nodes/cmdnodes.h +++ b/include/libs/nodes/cmdnodes.h @@ -88,8 +88,12 @@ typedef struct SAlterDatabaseStmt { typedef struct STableOptions { ENodeType type; char comment[TSDB_TB_COMMENT_LEN]; - double filesFactor; - int32_t delay; + SNodeList* pMaxDelay; + int64_t maxDelay1; + int64_t maxDelay2; + SNodeList* pWatermark; + int64_t watermark1; + int64_t watermark2; SNodeList* pRollupFuncs; int32_t ttl; SNodeList* pSma; @@ -204,11 +208,18 @@ typedef struct SShowStmt { SNode* pTbNamePattern; // SValueNode } SShowStmt; -typedef struct SShowCreatStmt { +typedef struct SShowCreateDatabaseStmt { ENodeType type; char dbName[TSDB_DB_NAME_LEN]; - char tableName[TSDB_TABLE_NAME_LEN]; -} SShowCreatStmt; + void* pCfg; // SDbCfgInfo +} SShowCreateDatabaseStmt; + +typedef struct SShowCreateTableStmt { + ENodeType type; + char dbName[TSDB_DB_NAME_LEN]; + char tableName[TSDB_TABLE_NAME_LEN]; + STableMeta* pMeta; +} SShowCreateTableStmt; typedef enum EIndexType { INDEX_TYPE_SMA = 1, INDEX_TYPE_FULLTEXT } EIndexType; diff --git a/include/libs/nodes/nodes.h b/include/libs/nodes/nodes.h index 6f239cc892..08190d978e 100644 --- a/include/libs/nodes/nodes.h +++ b/include/libs/nodes/nodes.h @@ -228,6 +228,7 @@ typedef enum ENodeType { QUERY_NODE_PHYSICAL_PLAN_FILL, QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION, QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION, + QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION, QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION, QUERY_NODE_PHYSICAL_PLAN_MERGE_STATE, QUERY_NODE_PHYSICAL_PLAN_STREAM_STATE, diff --git a/include/libs/nodes/plannodes.h b/include/libs/nodes/plannodes.h index 749d58b224..a9002b5d19 100644 --- a/include/libs/nodes/plannodes.h +++ b/include/libs/nodes/plannodes.h @@ -130,13 +130,17 @@ typedef struct SMergeLogicNode { typedef enum EWindowType { WINDOW_TYPE_INTERVAL = 1, WINDOW_TYPE_SESSION, WINDOW_TYPE_STATE } EWindowType; -typedef enum EIntervalAlgorithm { +typedef enum EWindowAlgorithm { INTERVAL_ALGO_HASH = 1, INTERVAL_ALGO_MERGE, INTERVAL_ALGO_STREAM_FINAL, INTERVAL_ALGO_STREAM_SEMI, INTERVAL_ALGO_STREAM_SINGLE, -} EIntervalAlgorithm; + SESSION_ALGO_STREAM_SEMI, + SESSION_ALGO_STREAM_FINAL, + SESSION_ALGO_STREAM_SINGLE, + SESSION_ALGO_MERGE, +} EWindowAlgorithm; typedef struct SWindowLogicNode { SLogicNode node; @@ -153,7 +157,7 @@ typedef struct SWindowLogicNode { int8_t triggerType; int64_t watermark; double filesFactor; - EIntervalAlgorithm intervalAlgo; + EWindowAlgorithm windowAlgo; } SWindowLogicNode; typedef struct SFillLogicNode { @@ -371,6 +375,8 @@ typedef struct SSessionWinodwPhysiNode { } SSessionWinodwPhysiNode; typedef SSessionWinodwPhysiNode SStreamSessionWinodwPhysiNode; +typedef SSessionWinodwPhysiNode SStreamSemiSessionWinodwPhysiNode; +typedef SSessionWinodwPhysiNode SStreamFinalSessionWinodwPhysiNode; typedef struct SStateWinodwPhysiNode { SWinodwPhysiNode window; diff --git a/include/libs/sync/sync.h b/include/libs/sync/sync.h index f49030466e..e8c4faa240 100644 --- a/include/libs/sync/sync.h +++ b/include/libs/sync/sync.h @@ -62,6 +62,7 @@ typedef struct SSyncCfg { typedef struct SFsmCbMeta { SyncIndex index; + SyncIndex lastConfigIndex; bool isWeak; int32_t code; ESyncState state; @@ -75,6 +76,7 @@ typedef struct SReConfigCbMeta { int32_t code; SyncIndex index; SyncTerm term; + SyncIndex lastConfigIndex; SyncTerm currentTerm; SSyncCfg oldCfg; SSyncCfg newCfg; diff --git a/include/libs/transport/trpc.h b/include/libs/transport/trpc.h index e3db1550a0..77de5f4756 100644 --- a/include/libs/transport/trpc.h +++ b/include/libs/transport/trpc.h @@ -23,6 +23,7 @@ extern "C" { #include #include "taosdef.h" #include "tmsg.h" +#include "ttrace.h" #define TAOS_CONN_SERVER 0 #define TAOS_CONN_CLIENT 1 @@ -41,10 +42,12 @@ typedef struct { typedef struct SRpcHandleInfo { // rpc info - void * handle; // rpc handle returned to app - int64_t refId; // refid, used by server - int32_t noResp; // has response or not(default 0, 0: resp, 1: no resp); - int32_t persistHandle; // persist handle or not + void * handle; // rpc handle returned to app + int64_t refId; // refid, used by server + int32_t noResp; // has response or not(default 0, 0: resp, 1: no resp); + int32_t persistHandle; // persist handle or not + STraceId traceId; + // int64_t traceId; // app info void *ahandle; // app handle set by client diff --git a/include/os/osTime.h b/include/os/osTime.h index b9e407cbcf..949c15ed0d 100644 --- a/include/os/osTime.h +++ b/include/os/osTime.h @@ -23,22 +23,22 @@ extern "C" { // If the error is in a third-party library, place this header file under the third-party library header file. // When you want to use this feature, you should find or add the same function in the following section. #ifndef ALLOW_FORBID_FUNC - #define strptime STRPTIME_FUNC_TAOS_FORBID - #define gettimeofday GETTIMEOFDAY_FUNC_TAOS_FORBID - #define localtime LOCALTIME_FUNC_TAOS_FORBID - #define localtime_s LOCALTIMES_FUNC_TAOS_FORBID - #define localtime_r LOCALTIMER_FUNC_TAOS_FORBID - #define time TIME_FUNC_TAOS_FORBID - #define mktime MKTIME_FUNC_TAOS_FORBID +#define strptime STRPTIME_FUNC_TAOS_FORBID +#define gettimeofday GETTIMEOFDAY_FUNC_TAOS_FORBID +#define localtime LOCALTIME_FUNC_TAOS_FORBID +#define localtime_s LOCALTIMES_FUNC_TAOS_FORBID +#define localtime_r LOCALTIMER_FUNC_TAOS_FORBID +#define time TIME_FUNC_TAOS_FORBID +#define mktime MKTIME_FUNC_TAOS_FORBID #endif #ifdef WINDOWS - #define CLOCK_REALTIME 0 +#define CLOCK_REALTIME 0 - #define MILLISECOND_PER_SECOND (1000i64) +#define MILLISECOND_PER_SECOND (1000i64) #else - #define MILLISECOND_PER_SECOND ((int64_t)1000L) +#define MILLISECOND_PER_SECOND ((int64_t)1000L) #endif #define MILLISECOND_PER_MINUTE (MILLISECOND_PER_SECOND * 60) @@ -82,13 +82,13 @@ static FORCE_INLINE int64_t taosGetTimestampNs() { return (int64_t)systemTime.tv_sec * 1000000000L + (int64_t)systemTime.tv_nsec; } -char *taosStrpTime(const char *buf, const char *fmt, struct tm *tm); +char * taosStrpTime(const char *buf, const char *fmt, struct tm *tm); struct tm *taosLocalTime(const time_t *timep, struct tm *result); -time_t taosTime(time_t *t); -time_t taosMktime(struct tm *timep); +time_t taosTime(time_t *t); +time_t taosMktime(struct tm *timep); #ifdef __cplusplus } #endif -#endif /*_TD_OS_TIME_H_*/ +#endif /*_TD_OS_TIME_H_*/ diff --git a/include/util/taoserror.h b/include/util/taoserror.h index e5f4b24614..fde016375b 100644 --- a/include/util/taoserror.h +++ b/include/util/taoserror.h @@ -431,7 +431,8 @@ int32_t* taosGetErrno(); #define TSDB_CODE_TQ_META_KEY_NOT_IN_TXN TAOS_DEF_ERROR_CODE(0, 0x0A09) #define TSDB_CODE_TQ_META_KEY_DUP_IN_TXN TAOS_DEF_ERROR_CODE(0, 0x0A0A) #define TSDB_CODE_TQ_GROUP_NOT_SET TAOS_DEF_ERROR_CODE(0, 0x0A0B) -#define TSDB_CODE_TQ_TABLE_SCHEMA_NOT_FOUND TAOS_DEF_ERROR_CODE(0, 0x0A0B) +#define TSDB_CODE_TQ_TABLE_SCHEMA_NOT_FOUND TAOS_DEF_ERROR_CODE(0, 0x0A0C) +#define TSDB_CODE_TQ_NO_COMMITTED_OFFSET TAOS_DEF_ERROR_CODE(0, 0x0A0D) // wal #define TSDB_CODE_WAL_APP_ERROR TAOS_DEF_ERROR_CODE(0, 0x1000) @@ -441,108 +442,6 @@ int32_t* taosGetErrno(); #define TSDB_CODE_WAL_OUT_OF_MEMORY TAOS_DEF_ERROR_CODE(0, 0x1004) #define TSDB_CODE_WAL_LOG_NOT_EXIST TAOS_DEF_ERROR_CODE(0, 0x1005) -// http -#define TSDB_CODE_HTTP_SERVER_OFFLINE TAOS_DEF_ERROR_CODE(0, 0x1100) //"http server is not online" -#define TSDB_CODE_HTTP_UNSUPPORT_URL TAOS_DEF_ERROR_CODE(0, 0x1101) //"url is not support" -#define TSDB_CODE_HTTP_INVALID_URL TAOS_DEF_ERROR_CODE(0, 0x1102) //invalid url format" -#define TSDB_CODE_HTTP_NO_ENOUGH_MEMORY TAOS_DEF_ERROR_CODE(0, 0x1103) //"no enough memory" -#define TSDB_CODE_HTTP_REQUSET_TOO_BIG TAOS_DEF_ERROR_CODE(0, 0x1104) //"request size is too big" -#define TSDB_CODE_HTTP_NO_AUTH_INFO TAOS_DEF_ERROR_CODE(0, 0x1105) //"no auth info input" -#define TSDB_CODE_HTTP_NO_MSG_INPUT TAOS_DEF_ERROR_CODE(0, 0x1106) //"request is empty" -#define TSDB_CODE_HTTP_NO_SQL_INPUT TAOS_DEF_ERROR_CODE(0, 0x1107) //"no sql input" -#define TSDB_CODE_HTTP_NO_EXEC_USEDB TAOS_DEF_ERROR_CODE(0, 0x1108) //"no need to execute use db cmd" -#define TSDB_CODE_HTTP_SESSION_FULL TAOS_DEF_ERROR_CODE(0, 0x1109) //"session list was full" -#define TSDB_CODE_HTTP_GEN_TAOSD_TOKEN_ERR TAOS_DEF_ERROR_CODE(0, 0x110A) //"generate taosd token error" -#define TSDB_CODE_HTTP_INVALID_MULTI_REQUEST TAOS_DEF_ERROR_CODE(0, 0x110B) //"size of multi request is 0" -#define TSDB_CODE_HTTP_CREATE_GZIP_FAILED TAOS_DEF_ERROR_CODE(0, 0x110C) //"failed to create gzip" -#define TSDB_CODE_HTTP_FINISH_GZIP_FAILED TAOS_DEF_ERROR_CODE(0, 0x110D) //"failed to finish gzip" -#define TSDB_CODE_HTTP_LOGIN_FAILED TAOS_DEF_ERROR_CODE(0, 0x110E) //"failed to login" - -#define TSDB_CODE_HTTP_INVALID_VERSION TAOS_DEF_ERROR_CODE(0, 0x1120) //"invalid http version" -#define TSDB_CODE_HTTP_INVALID_CONTENT_LENGTH TAOS_DEF_ERROR_CODE(0, 0x1121) //"invalid content length" -#define TSDB_CODE_HTTP_INVALID_AUTH_TYPE TAOS_DEF_ERROR_CODE(0, 0x1122) //"invalid type of Authorization" -#define TSDB_CODE_HTTP_INVALID_AUTH_FORMAT TAOS_DEF_ERROR_CODE(0, 0x1123) //"invalid format of Authorization" -#define TSDB_CODE_HTTP_INVALID_BASIC_AUTH TAOS_DEF_ERROR_CODE(0, 0x1124) //"invalid basic Authorization" -#define TSDB_CODE_HTTP_INVALID_TAOSD_AUTH TAOS_DEF_ERROR_CODE(0, 0x1125) //"invalid taosd Authorization" -#define TSDB_CODE_HTTP_PARSE_METHOD_FAILED TAOS_DEF_ERROR_CODE(0, 0x1126) //"failed to parse method" -#define TSDB_CODE_HTTP_PARSE_TARGET_FAILED TAOS_DEF_ERROR_CODE(0, 0x1127) //"failed to parse target" -#define TSDB_CODE_HTTP_PARSE_VERSION_FAILED TAOS_DEF_ERROR_CODE(0, 0x1128) //"failed to parse http version" -#define TSDB_CODE_HTTP_PARSE_SP_FAILED TAOS_DEF_ERROR_CODE(0, 0x1129) //"failed to parse sp" -#define TSDB_CODE_HTTP_PARSE_STATUS_FAILED TAOS_DEF_ERROR_CODE(0, 0x112A) //"failed to parse status" -#define TSDB_CODE_HTTP_PARSE_PHRASE_FAILED TAOS_DEF_ERROR_CODE(0, 0x112B) //"failed to parse phrase" -#define TSDB_CODE_HTTP_PARSE_CRLF_FAILED TAOS_DEF_ERROR_CODE(0, 0x112C) //"failed to parse crlf" -#define TSDB_CODE_HTTP_PARSE_HEADER_FAILED TAOS_DEF_ERROR_CODE(0, 0x112D) //"failed to parse header" -#define TSDB_CODE_HTTP_PARSE_HEADER_KEY_FAILED TAOS_DEF_ERROR_CODE(0, 0x112E) //"failed to parse header key" -#define TSDB_CODE_HTTP_PARSE_HEADER_VAL_FAILED TAOS_DEF_ERROR_CODE(0, 0x112F) //"failed to parse header val" -#define TSDB_CODE_HTTP_PARSE_CHUNK_SIZE_FAILED TAOS_DEF_ERROR_CODE(0, 0x1130) //"failed to parse chunk size" -#define TSDB_CODE_HTTP_PARSE_CHUNK_FAILED TAOS_DEF_ERROR_CODE(0, 0x1131) //"failed to parse chunk" -#define TSDB_CODE_HTTP_PARSE_END_FAILED TAOS_DEF_ERROR_CODE(0, 0x1132) //"failed to parse end section" -#define TSDB_CODE_HTTP_PARSE_INVALID_STATE TAOS_DEF_ERROR_CODE(0, 0x1134) //"invalid parse state" -#define TSDB_CODE_HTTP_PARSE_ERROR_STATE TAOS_DEF_ERROR_CODE(0, 0x1135) //"failed to parse error section" - -#define TSDB_CODE_HTTP_GC_QUERY_NULL TAOS_DEF_ERROR_CODE(0, 0x1150) //"query size is 0" -#define TSDB_CODE_HTTP_GC_QUERY_SIZE TAOS_DEF_ERROR_CODE(0, 0x1151) //"query size can not more than 100" -#define TSDB_CODE_HTTP_GC_REQ_PARSE_ERROR TAOS_DEF_ERROR_CODE(0, 0x1152) //"parse grafana json error" - -#define TSDB_CODE_HTTP_TG_DB_NOT_INPUT TAOS_DEF_ERROR_CODE(0, 0x1160) //"database name can not be null" -#define TSDB_CODE_HTTP_TG_DB_TOO_LONG TAOS_DEF_ERROR_CODE(0, 0x1161) //"database name too long" -#define TSDB_CODE_HTTP_TG_INVALID_JSON TAOS_DEF_ERROR_CODE(0, 0x1162) //"invalid telegraf json fromat" -#define TSDB_CODE_HTTP_TG_METRICS_NULL TAOS_DEF_ERROR_CODE(0, 0x1163) //"metrics size is 0" -#define TSDB_CODE_HTTP_TG_METRICS_SIZE TAOS_DEF_ERROR_CODE(0, 0x1164) //"metrics size can not more than 1K" -#define TSDB_CODE_HTTP_TG_METRIC_NULL TAOS_DEF_ERROR_CODE(0, 0x1165) //"metric name not find" -#define TSDB_CODE_HTTP_TG_METRIC_TYPE TAOS_DEF_ERROR_CODE(0, 0x1166) //"metric name type should be string" -#define TSDB_CODE_HTTP_TG_METRIC_NAME_NULL TAOS_DEF_ERROR_CODE(0, 0x1167) //"metric name length is 0" -#define TSDB_CODE_HTTP_TG_METRIC_NAME_LONG TAOS_DEF_ERROR_CODE(0, 0x1168) //"metric name length too long" -#define TSDB_CODE_HTTP_TG_TIMESTAMP_NULL TAOS_DEF_ERROR_CODE(0, 0x1169) //"timestamp not find" -#define TSDB_CODE_HTTP_TG_TIMESTAMP_TYPE TAOS_DEF_ERROR_CODE(0, 0x116A) //"timestamp type should be integer" -#define TSDB_CODE_HTTP_TG_TIMESTAMP_VAL_NULL TAOS_DEF_ERROR_CODE(0, 0x116B) //"timestamp value smaller than 0" -#define TSDB_CODE_HTTP_TG_TAGS_NULL TAOS_DEF_ERROR_CODE(0, 0x116C) //"tags not find" -#define TSDB_CODE_HTTP_TG_TAGS_SIZE_0 TAOS_DEF_ERROR_CODE(0, 0x116D) //"tags size is 0" -#define TSDB_CODE_HTTP_TG_TAGS_SIZE_LONG TAOS_DEF_ERROR_CODE(0, 0x116E) //"tags size too long" -#define TSDB_CODE_HTTP_TG_TAG_NULL TAOS_DEF_ERROR_CODE(0, 0x116F) //"tag is null" -#define TSDB_CODE_HTTP_TG_TAG_NAME_NULL TAOS_DEF_ERROR_CODE(0, 0x1170) //"tag name is null" -#define TSDB_CODE_HTTP_TG_TAG_NAME_SIZE TAOS_DEF_ERROR_CODE(0, 0x1171) //"tag name length too long" -#define TSDB_CODE_HTTP_TG_TAG_VALUE_TYPE TAOS_DEF_ERROR_CODE(0, 0x1172) //"tag value type should be number or string" -#define TSDB_CODE_HTTP_TG_TAG_VALUE_NULL TAOS_DEF_ERROR_CODE(0, 0x1173) //"tag value is null" -#define TSDB_CODE_HTTP_TG_TABLE_NULL TAOS_DEF_ERROR_CODE(0, 0x1174) //"table is null" -#define TSDB_CODE_HTTP_TG_TABLE_SIZE TAOS_DEF_ERROR_CODE(0, 0x1175) //"table name length too long" -#define TSDB_CODE_HTTP_TG_FIELDS_NULL TAOS_DEF_ERROR_CODE(0, 0x1176) //"fields not find" -#define TSDB_CODE_HTTP_TG_FIELDS_SIZE_0 TAOS_DEF_ERROR_CODE(0, 0x1177) //"fields size is 0" -#define TSDB_CODE_HTTP_TG_FIELDS_SIZE_LONG TAOS_DEF_ERROR_CODE(0, 0x1178) //"fields size too long" -#define TSDB_CODE_HTTP_TG_FIELD_NULL TAOS_DEF_ERROR_CODE(0, 0x1179) //"field is null" -#define TSDB_CODE_HTTP_TG_FIELD_NAME_NULL TAOS_DEF_ERROR_CODE(0, 0x117A) //"field name is null" -#define TSDB_CODE_HTTP_TG_FIELD_NAME_SIZE TAOS_DEF_ERROR_CODE(0, 0x117B) //"field name length too long" -#define TSDB_CODE_HTTP_TG_FIELD_VALUE_TYPE TAOS_DEF_ERROR_CODE(0, 0x117C) //"field value type should be number or string" -#define TSDB_CODE_HTTP_TG_FIELD_VALUE_NULL TAOS_DEF_ERROR_CODE(0, 0x117D) //"field value is null" -#define TSDB_CODE_HTTP_TG_HOST_NOT_STRING TAOS_DEF_ERROR_CODE(0, 0x117E) //"host type should be string" -#define TSDB_CODE_HTTP_TG_STABLE_NOT_EXIST TAOS_DEF_ERROR_CODE(0, 0x117F) //"stable not exist" - -#define TSDB_CODE_HTTP_OP_DB_NOT_INPUT TAOS_DEF_ERROR_CODE(0, 0x1190) //"database name can not be null" -#define TSDB_CODE_HTTP_OP_DB_TOO_LONG TAOS_DEF_ERROR_CODE(0, 0x1191) //"database name too long" -#define TSDB_CODE_HTTP_OP_INVALID_JSON TAOS_DEF_ERROR_CODE(0, 0x1192) //"invalid opentsdb json fromat" -#define TSDB_CODE_HTTP_OP_METRICS_NULL TAOS_DEF_ERROR_CODE(0, 0x1193) //"metrics size is 0" -#define TSDB_CODE_HTTP_OP_METRICS_SIZE TAOS_DEF_ERROR_CODE(0, 0x1194) //"metrics size can not more than 10K" -#define TSDB_CODE_HTTP_OP_METRIC_NULL TAOS_DEF_ERROR_CODE(0, 0x1195) //"metric name not find" -#define TSDB_CODE_HTTP_OP_METRIC_TYPE TAOS_DEF_ERROR_CODE(0, 0x1196) //"metric name type should be string" -#define TSDB_CODE_HTTP_OP_METRIC_NAME_NULL TAOS_DEF_ERROR_CODE(0, 0x1197) //"metric name length is 0" -#define TSDB_CODE_HTTP_OP_METRIC_NAME_LONG TAOS_DEF_ERROR_CODE(0, 0x1198) //"metric name length can not more than 22" -#define TSDB_CODE_HTTP_OP_TIMESTAMP_NULL TAOS_DEF_ERROR_CODE(0, 0x1199) //"timestamp not find" -#define TSDB_CODE_HTTP_OP_TIMESTAMP_TYPE TAOS_DEF_ERROR_CODE(0, 0x119A) //"timestamp type should be integer" -#define TSDB_CODE_HTTP_OP_TIMESTAMP_VAL_NULL TAOS_DEF_ERROR_CODE(0, 0x119B) //"timestamp value smaller than 0" -#define TSDB_CODE_HTTP_OP_TAGS_NULL TAOS_DEF_ERROR_CODE(0, 0x119C) //"tags not find" -#define TSDB_CODE_HTTP_OP_TAGS_SIZE_0 TAOS_DEF_ERROR_CODE(0, 0x119D) //"tags size is 0" -#define TSDB_CODE_HTTP_OP_TAGS_SIZE_LONG TAOS_DEF_ERROR_CODE(0, 0x119E) //"tags size too long" -#define TSDB_CODE_HTTP_OP_TAG_NULL TAOS_DEF_ERROR_CODE(0, 0x119F) //"tag is null" -#define TSDB_CODE_HTTP_OP_TAG_NAME_NULL TAOS_DEF_ERROR_CODE(0, 0x11A0) //"tag name is null" -#define TSDB_CODE_HTTP_OP_TAG_NAME_SIZE TAOS_DEF_ERROR_CODE(0, 0x11A1) //"tag name length too long" -#define TSDB_CODE_HTTP_OP_TAG_VALUE_TYPE TAOS_DEF_ERROR_CODE(0, 0x11A2) //"tag value type should be boolean number or string" -#define TSDB_CODE_HTTP_OP_TAG_VALUE_NULL TAOS_DEF_ERROR_CODE(0, 0x11A3) //"tag value is null" -#define TSDB_CODE_HTTP_OP_TAG_VALUE_TOO_LONG TAOS_DEF_ERROR_CODE(0, 0x11A4) //"tag value can not more than 64" -#define TSDB_CODE_HTTP_OP_VALUE_NULL TAOS_DEF_ERROR_CODE(0, 0x11A5) //"value not find" -#define TSDB_CODE_HTTP_OP_VALUE_TYPE TAOS_DEF_ERROR_CODE(0, 0x11A6) //"value type should be boolean number or string" - -#define TSDB_CODE_HTTP_REQUEST_JSON_ERROR TAOS_DEF_ERROR_CODE(0, 0x1F00) //"http request json error" - // tfs #define TSDB_CODE_FS_APP_ERROR TAOS_DEF_ERROR_CODE(0, 0x2200) #define TSDB_CODE_FS_INVLD_CFG TAOS_DEF_ERROR_CODE(0, 0x2201) @@ -664,6 +563,7 @@ int32_t* taosGetErrno(); #define TSDB_CODE_PAR_WINDOW_NOT_ALLOWED_FUNC TAOS_DEF_ERROR_CODE(0, 0x2659) #define TSDB_CODE_PAR_STREAM_NOT_ALLOWED_FUNC TAOS_DEF_ERROR_CODE(0, 0x265A) #define TSDB_CODE_PAR_GROUP_BY_NOT_ALLOWED_FUNC TAOS_DEF_ERROR_CODE(0, 0x265B) +#define TSDB_CODE_PAR_INVALID_TABLE_OPTION TAOS_DEF_ERROR_CODE(0, 0x265C) //planner #define TSDB_CODE_PLAN_INTERNAL_ERROR TAOS_DEF_ERROR_CODE(0, 0x2700) @@ -712,6 +612,8 @@ int32_t* taosGetErrno(); //index #define TSDB_CODE_INDEX_REBUILDING TAOS_DEF_ERROR_CODE(0, 0x3200) +//tmq +#define TSDB_CODE_TMQ_INVALID_MSG TAOS_DEF_ERROR_CODE(0, 0x4000) #ifdef __cplusplus } diff --git a/include/util/tdef.h b/include/util/tdef.h index a48e823bad..c5a8b95a08 100644 --- a/include/util/tdef.h +++ b/include/util/tdef.h @@ -344,14 +344,13 @@ typedef enum ELogicConditionType { #define TSDB_DB_SCHEMALESS_OFF 0 #define TSDB_DEFAULT_DB_SCHEMALESS TSDB_DB_SCHEMALESS_OFF -// #define TSDB_MIN_ROLLUP_DELAY 1 -// #define TSDB_MAX_ROLLUP_DELAY 10 -// #define TSDB_DEFAULT_ROLLUP_DELAY 1 -#define TSDB_MIN_ROLLUP_FILE_FACTOR 0 -#define TSDB_MAX_ROLLUP_FILE_FACTOR 10 -#define TSDB_DEFAULT_ROLLUP_FILE_FACTOR 0.1 -#define TSDB_MIN_TABLE_TTL 0 -#define TSDB_DEFAULT_TABLE_TTL 0 +#define TSDB_MIN_ROLLUP_MAX_DELAY 1 // unit millisecond +#define TSDB_MAX_ROLLUP_MAX_DELAY (15 * 60 * 1000) +#define TSDB_MIN_ROLLUP_WATERMARK 0 // unit millisecond +#define TSDB_MAX_ROLLUP_WATERMARK (15 * 60 * 1000) +#define TSDB_DEFAULT_ROLLUP_WATERMARK 5000 +#define TSDB_MIN_TABLE_TTL 0 +#define TSDB_DEFAULT_TABLE_TTL 0 #define TSDB_MIN_EXPLAIN_RATIO 0 #define TSDB_MAX_EXPLAIN_RATIO 1 diff --git a/include/util/ttrace.h b/include/util/ttrace.h new file mode 100644 index 0000000000..206cbbf28d --- /dev/null +++ b/include/util/ttrace.h @@ -0,0 +1,57 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ +#ifndef _TD_TRACE_H_ +#define _TD_TRACE_H_ + +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +#pragma(push, 1) + +typedef struct STraceId { + int64_t rootId; + int64_t msgId; +} STraceId; + +#pragma(pop) + +#define TRACE_SET_ROOTID(traceId, root) \ + do { \ + (traceId)->rootId = root; \ + } while (0); + +#define TRACE_GET_ROOTID(traceId) (traceId)->rootId + +#define TRACE_SET_MSGID(traceId, mId) \ + do { \ + (traceId)->msgId = mId; \ + } while (0) + +#define TRACE_GET_MSGID(traceId) (traceId)->msgId + +#define TRACE_TO_STR(traceId, buf) \ + do { \ + sprintf(buf, "0x%" PRIx64 ":0x%" PRIx64 "", traceId->rootId, traceId->msgId); \ + } while (0) + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/source/client/src/tmq.c b/source/client/src/tmq.c index dd900e6045..638b4f1ea5 100644 --- a/source/client/src/tmq.c +++ b/source/client/src/tmq.c @@ -48,14 +48,6 @@ struct tmq_list_t { SArray container; }; -struct tmq_topic_vgroup_t { - SMqOffset offset; -}; - -struct tmq_topic_vgroup_list_t { - SArray container; // SArray -}; - struct tmq_conf_t { char clientId[256]; char groupId[TSDB_CGROUP_LEN]; @@ -161,9 +153,9 @@ typedef struct { } SMqPollRspWrapper; typedef struct { - tmq_t* tmq; - tsem_t rspSem; - tmq_resp_err_t rspErr; + tmq_t* tmq; + tsem_t rspSem; + int32_t rspErr; } SMqSubscribeCbParam; typedef struct { @@ -189,7 +181,7 @@ typedef struct { int8_t freeOffsets; tmq_commit_cb* userCb; tsem_t rspSem; - tmq_resp_err_t rspErr; + int32_t rspErr; SArray* offsets; void* userParam; } SMqCommitCbParam; @@ -201,12 +193,12 @@ typedef struct { int8_t freeOffsets; int32_t waitingRspNum; int32_t totalRspNum; - tmq_resp_err_t rspErr; + int32_t rspErr; tmq_commit_cb* userCb; - SArray* successfulOffsets; - SArray* failedOffsets; - void* userParam; - tsem_t rspSem; + /*SArray* successfulOffsets;*/ + /*SArray* failedOffsets;*/ + void* userParam; + tsem_t rspSem; } SMqCommitCbParamSet; typedef struct { @@ -347,10 +339,9 @@ int32_t tmqCommitCb(void* param, const SDataBuf* pMsg, int32_t code) { pParam->rspErr = code; if (pParam->async) { if (pParam->automatic && pParam->tmq->commitCb) { - pParam->tmq->commitCb(pParam->tmq, pParam->rspErr, (tmq_topic_vgroup_list_t*)pParam->offsets, - pParam->tmq->commitCbUserParam); + pParam->tmq->commitCb(pParam->tmq, pParam->rspErr, pParam->tmq->commitCbUserParam); } else if (!pParam->automatic && pParam->userCb) { - pParam->userCb(pParam->tmq, pParam->rspErr, (tmq_topic_vgroup_list_t*)pParam->offsets, pParam->userParam); + pParam->userCb(pParam->tmq, pParam->rspErr, pParam->userParam); } if (pParam->freeOffsets) { @@ -368,11 +359,16 @@ int32_t tmqCommitCb2(void* param, const SDataBuf* pBuf, int32_t code) { SMqCommitCbParam2* pParam = (SMqCommitCbParam2*)param; SMqCommitCbParamSet* pParamSet = (SMqCommitCbParamSet*)pParam->params; // push into array +#if 0 if (code == 0) { taosArrayPush(pParamSet->failedOffsets, &pParam->pOffset); } else { taosArrayPush(pParamSet->successfulOffsets, &pParam->pOffset); } +#endif + + /*tscDebug("receive offset commit cb of %s on vg %d, offset is %ld", pParam->pOffset->subKey, pParam->->vgId, + * pOffset->version);*/ // count down waiting rsp int32_t waitingRspNum = atomic_sub_fetch_32(&pParamSet->waitingRspNum, 1); @@ -383,23 +379,149 @@ int32_t tmqCommitCb2(void* param, const SDataBuf* pBuf, int32_t code) { if (pParamSet->async) { // call async cb func if (pParamSet->automatic && pParamSet->tmq->commitCb) { - pParamSet->tmq->commitCb(pParamSet->tmq, pParamSet->rspErr, NULL, pParamSet->tmq->commitCbUserParam); + pParamSet->tmq->commitCb(pParamSet->tmq, pParamSet->rspErr, pParamSet->tmq->commitCbUserParam); } else if (!pParamSet->automatic && pParamSet->userCb) { // sem post - pParamSet->userCb(pParamSet->tmq, pParamSet->rspErr, NULL, pParamSet->userParam); + pParamSet->userCb(pParamSet->tmq, pParamSet->rspErr, pParamSet->userParam); } + } else { + tsem_post(&pParamSet->rspSem); } +#if 0 taosArrayDestroyP(pParamSet->successfulOffsets, taosMemoryFree); taosArrayDestroyP(pParamSet->failedOffsets, taosMemoryFree); +#endif } return 0; } -int32_t tmqCommitInner2(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int8_t automatic, int8_t async, - tmq_commit_cb* userCb, void* userParam) { +int32_t tmqCommitInner2(tmq_t* tmq, const TAOS_RES* msg, int8_t automatic, int8_t async, tmq_commit_cb* userCb, + void* userParam) { int32_t code = -1; + if (msg != NULL) { + SMqRspObj* pRspObj = (SMqRspObj*)msg; + if (!TD_RES_TMQ(pRspObj)) { + return TSDB_CODE_TMQ_INVALID_MSG; + } + + SMqCommitCbParamSet* pParamSet = taosMemoryCalloc(1, sizeof(SMqCommitCbParamSet)); + if (pParamSet == NULL) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + return -1; + } + pParamSet->tmq = tmq; + pParamSet->automatic = automatic; + pParamSet->async = async; + pParamSet->freeOffsets = 1; + pParamSet->userCb = userCb; + pParamSet->userParam = userParam; + tsem_init(&pParamSet->rspSem, 0, 0); + + for (int32_t i = 0; i < taosArrayGetSize(tmq->clientTopics); i++) { + SMqClientTopic* pTopic = taosArrayGet(tmq->clientTopics, i); + if (strcmp(pTopic->topicName, pRspObj->topic) == 0) { + for (int32_t j = 0; j < taosArrayGetSize(pTopic->vgs); j++) { + SMqClientVg* pVg = taosArrayGet(pTopic->vgs, j); + if (pVg->vgId == pRspObj->vgId) { + if (pVg->currentOffset < 0 || pVg->committedOffset == pVg->currentOffset) { + tscDebug("consumer %ld skip commit for topic %s vg %d, current offset is %ld, committed offset is %ld", + tmq->consumerId, pTopic->topicName, pVg->vgId, pVg->currentOffset, pVg->committedOffset); + + return 0; + } + + STqOffset* pOffset = taosMemoryCalloc(1, sizeof(STqOffset)); + if (pOffset == NULL) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + return -1; + } + pOffset->type = TMQ_OFFSET__LOG; + pOffset->version = pVg->currentOffset; + + int32_t groupLen = strlen(tmq->groupId); + memcpy(pOffset->subKey, tmq->groupId, groupLen); + pOffset->subKey[groupLen] = TMQ_SEPARATOR; + strcpy(pOffset->subKey + groupLen + 1, pTopic->topicName); + + int32_t len; + int32_t code; + tEncodeSize(tEncodeSTqOffset, pOffset, len, code); + if (code < 0) { + ASSERT(0); + } + void* buf = taosMemoryCalloc(1, sizeof(SMsgHead) + len); + ((SMsgHead*)buf)->vgId = htonl(pVg->vgId); + + void* abuf = POINTER_SHIFT(buf, sizeof(SMsgHead)); + + SEncoder encoder; + tEncoderInit(&encoder, abuf, len); + tEncodeSTqOffset(&encoder, pOffset); + + // build param + SMqCommitCbParam2* pParam = taosMemoryCalloc(1, sizeof(SMqCommitCbParam2)); + pParam->params = pParamSet; + pParam->pOffset = pOffset; + + // build send info + SMsgSendInfo* pMsgSendInfo = taosMemoryCalloc(1, sizeof(SMsgSendInfo)); + if (pMsgSendInfo == NULL) { + // TODO + continue; + } + pMsgSendInfo->msgInfo = (SDataBuf){ + .pData = buf, + .len = sizeof(SMsgHead) + len, + .handle = NULL, + }; + + tscDebug("consumer %ld commit offset of %s on vg %d, offset is %ld", tmq->consumerId, pOffset->subKey, + pVg->vgId, pOffset->version); + + // TODO: put into cb + pVg->committedOffset = pVg->currentOffset; + + pMsgSendInfo->requestId = generateRequestId(); + pMsgSendInfo->requestObjRefId = 0; + pMsgSendInfo->param = pParam; + pMsgSendInfo->fp = tmqCommitCb2; + pMsgSendInfo->msgType = TDMT_VND_MQ_COMMIT_OFFSET; + // send msg + + int64_t transporterId = 0; + asyncSendMsgToServer(tmq->pTscObj->pAppInfo->pTransporter, &pVg->epSet, &transporterId, pMsgSendInfo); + pParamSet->waitingRspNum++; + pParamSet->totalRspNum++; + } + } + } + } + if (pParamSet->totalRspNum == 0) { + tsem_destroy(&pParamSet->rspSem); + taosMemoryFree(pParamSet); + return 0; + } + + if (!async) { + tsem_wait(&pParamSet->rspSem); + code = pParamSet->rspErr; + tsem_destroy(&pParamSet->rspSem); + } else { + code = 0; + } + + if (code != 0 && async) { + if (automatic) { + tmq->commitCb(tmq, code, tmq->commitCbUserParam); + } else { + userCb(tmq, code, userParam); + } + } + return 0; + } + SMqCommitCbParamSet* pParamSet = taosMemoryCalloc(1, sizeof(SMqCommitCbParamSet)); if (pParamSet == NULL) { terrno = TSDB_CODE_OUT_OF_MEMORY; @@ -415,17 +537,35 @@ int32_t tmqCommitInner2(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int8 for (int32_t i = 0; i < taosArrayGetSize(tmq->clientTopics); i++) { SMqClientTopic* pTopic = taosArrayGet(tmq->clientTopics, i); + + tscDebug("consumer %ld begin commit for topic %s, vgNum %d", tmq->consumerId, pTopic->topicName, + (int32_t)taosArrayGetSize(pTopic->vgs)); + for (int32_t j = 0; j < taosArrayGetSize(pTopic->vgs); j++) { - SMqClientVg* pVg = taosArrayGet(pTopic->vgs, i); - STqOffset* pOffset = taosMemoryCalloc(1, sizeof(STqOffset)); + SMqClientVg* pVg = taosArrayGet(pTopic->vgs, j); + + tscDebug("consumer %ld begin commit for topic %s, vgId %d", tmq->consumerId, pTopic->topicName, pVg->vgId); + + /*if (pVg->currentOffset < 0) {*/ + if (pVg->currentOffset < 0 || pVg->committedOffset == pVg->currentOffset) { + tscDebug("consumer %ld skip commit for topic %s vg %d, current offset is %ld, committed offset is %ld", + tmq->consumerId, pTopic->topicName, pVg->vgId, pVg->currentOffset, pVg->committedOffset); + + continue; + } + STqOffset* pOffset = taosMemoryCalloc(1, sizeof(STqOffset)); if (pOffset == NULL) { terrno = TSDB_CODE_OUT_OF_MEMORY; return -1; } - int32_t tlen = strlen(tmq->groupId); - memcpy(pOffset->subKey, tmq->groupId, tlen); - pOffset->subKey[tlen] = TMQ_SEPARATOR; - strcpy(pOffset->subKey + tlen + 1, pTopic->topicName); + pOffset->type = TMQ_OFFSET__LOG; + pOffset->version = pVg->currentOffset; + + int32_t groupLen = strlen(tmq->groupId); + memcpy(pOffset->subKey, tmq->groupId, groupLen); + pOffset->subKey[groupLen] = TMQ_SEPARATOR; + strcpy(pOffset->subKey + groupLen + 1, pTopic->topicName); + int32_t len; int32_t code; tEncodeSize(tEncodeSTqOffset, pOffset, len, code); @@ -454,25 +594,36 @@ int32_t tmqCommitInner2(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int8 } pMsgSendInfo->msgInfo = (SDataBuf){ .pData = buf, - .len = len, + .len = sizeof(SMsgHead) + len, .handle = NULL, }; + tscDebug("consumer %ld commit offset of %s on vg %d, offset is %ld", tmq->consumerId, pOffset->subKey, pVg->vgId, + pOffset->version); + + // TODO: put into cb + pVg->committedOffset = pVg->currentOffset; + pMsgSendInfo->requestId = generateRequestId(); pMsgSendInfo->requestObjRefId = 0; pMsgSendInfo->param = pParam; pMsgSendInfo->fp = tmqCommitCb2; - pMsgSendInfo->msgType = TDMT_MND_MQ_COMMIT_OFFSET; + pMsgSendInfo->msgType = TDMT_VND_MQ_COMMIT_OFFSET; // send msg - SEpSet epSet = getEpSet_s(&tmq->pTscObj->pAppInfo->mgmtEp); int64_t transporterId = 0; - asyncSendMsgToServer(tmq->pTscObj->pAppInfo->pTransporter, &epSet, &transporterId, pMsgSendInfo); + asyncSendMsgToServer(tmq->pTscObj->pAppInfo->pTransporter, &pVg->epSet, &transporterId, pMsgSendInfo); pParamSet->waitingRspNum++; pParamSet->totalRspNum++; } } + if (pParamSet->totalRspNum == 0) { + tsem_destroy(&pParamSet->rspSem); + taosMemoryFree(pParamSet); + return 0; + } + if (!async) { tsem_wait(&pParamSet->rspSem); code = pParamSet->rspErr; @@ -483,21 +634,24 @@ int32_t tmqCommitInner2(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int8 if (code != 0 && async) { if (automatic) { - tmq->commitCb(tmq, code, NULL, tmq->commitCbUserParam); + tmq->commitCb(tmq, code, tmq->commitCbUserParam); } else { - userCb(tmq, code, NULL, userParam); + userCb(tmq, code, userParam); } } +#if 0 if (!async) { taosArrayDestroyP(pParamSet->successfulOffsets, taosMemoryFree); taosArrayDestroyP(pParamSet->failedOffsets, taosMemoryFree); } +#endif return 0; } -int32_t tmqCommitInner(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int8_t automatic, int8_t async, +#if 0 +int32_t tmqCommitInner(tmq_t* tmq, const TAOS_RES* msg, int8_t automatic, int8_t async, tmq_commit_cb* userCb, void* userParam) { SMqCMCommitOffsetReq req; SArray* pOffsets = NULL; @@ -507,7 +661,7 @@ int32_t tmqCommitInner(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int8_ int8_t freeOffsets; int32_t code = -1; - if (offsets == NULL) { + if (msg == NULL) { freeOffsets = 1; pOffsets = taosArrayInit(0, sizeof(SMqOffset)); for (int32_t i = 0; i < taosArrayGetSize(tmq->clientTopics); i++) { @@ -524,7 +678,7 @@ int32_t tmqCommitInner(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int8_ } } else { freeOffsets = 0; - pOffsets = (SArray*)&offsets->container; + pOffsets = (SArray*)&msg->container; } req.num = (int32_t)pOffsets->size; @@ -611,6 +765,7 @@ END: } return code; } +#endif void tmqAssignDelayedHbTask(void* param, void* tmrId) { tmq_t* tmq = (tmq_t*)param; @@ -648,7 +803,7 @@ int32_t tmqHandleAllDelayedTask(tmq_t* tmq) { tmqAskEp(tmq, true); taosTmrReset(tmqAssignDelayedHbTask, 1000, tmq, tmqMgmt.timer, &tmq->hbTimer); } else if (*pTaskType == TMQ_DELAYED_TASK__COMMIT) { - tmqCommitInner(tmq, NULL, 1, 1, tmq->commitCb, tmq->commitCbUserParam); + tmqCommitInner2(tmq, NULL, 1, 1, tmq->commitCb, tmq->commitCbUserParam); taosTmrReset(tmqAssignDelayedCommitTask, tmq->autoCommitInterval, tmq, tmqMgmt.timer, &tmq->commitTimer); } else if (*pTaskType == TMQ_DELAYED_TASK__REPORT) { } else { @@ -689,7 +844,7 @@ int32_t tmqSubscribeCb(void* param, const SDataBuf* pMsg, int32_t code) { return 0; } -tmq_resp_err_t tmq_subscription(tmq_t* tmq, tmq_list_t** topics) { +int32_t tmq_subscription(tmq_t* tmq, tmq_list_t** topics) { if (*topics == NULL) { *topics = tmq_list_new(); } @@ -697,12 +852,12 @@ tmq_resp_err_t tmq_subscription(tmq_t* tmq, tmq_list_t** topics) { SMqClientTopic* topic = taosArrayGet(tmq->clientTopics, i); tmq_list_append(*topics, strchr(topic->topicName, '.') + 1); } - return TMQ_RESP_ERR__SUCCESS; + return 0; } -tmq_resp_err_t tmq_unsubscribe(tmq_t* tmq) { - tmq_list_t* lst = tmq_list_new(); - tmq_resp_err_t rsp = tmq_subscribe(tmq, lst); +int32_t tmq_unsubscribe(tmq_t* tmq) { + tmq_list_t* lst = tmq_list_new(); + int32_t rsp = tmq_subscribe(tmq, lst); tmq_list_destroy(lst); return rsp; } @@ -818,11 +973,13 @@ FAIL: return NULL; } -tmq_resp_err_t tmq_commit(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int32_t async) { - return tmqCommitInner(tmq, offsets, 0, async, tmq->commitCb, tmq->commitCbUserParam); +#if 0 +int32_t tmq_commit(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int32_t async) { + return tmqCommitInner2(tmq, offsets, 0, async, tmq->commitCb, tmq->commitCbUserParam); } +#endif -tmq_resp_err_t tmq_subscribe(tmq_t* tmq, const tmq_list_t* topic_list) { +int32_t tmq_subscribe(tmq_t* tmq, const tmq_list_t* topic_list) { const SArray* container = &topic_list->container; int32_t sz = taosArrayGetSize(container); void* buf = NULL; @@ -862,7 +1019,7 @@ tmq_resp_err_t tmq_subscribe(tmq_t* tmq, const tmq_list_t* topic_list) { if (sendInfo == NULL) goto FAIL; SMqSubscribeCbParam param = { - .rspErr = TMQ_RESP_ERR__SUCCESS, + .rspErr = 0, .tmq = tmq, }; @@ -943,6 +1100,19 @@ int32_t tmqPollCb(void* param, const SDataBuf* pMsg, int32_t code) { if (code != 0) { tscWarn("msg discard from vg %d, epoch %d, code:%x", vgId, epoch, code); if (pMsg->pData) taosMemoryFree(pMsg->pData); + if (code == TSDB_CODE_TQ_NO_COMMITTED_OFFSET) { + SMqPollRspWrapper* pRspWrapper = taosAllocateQitem(sizeof(SMqPollRspWrapper), DEF_QITEM); + if (pRspWrapper == NULL) { + taosMemoryFree(pMsg->pData); + tscWarn("msg discard from vg %d, epoch %d since out of memory", vgId, epoch); + goto CREATE_MSG_FAIL; + } + pRspWrapper->tmqRspType = TMQ_MSG_TYPE__END_RSP; + /*pRspWrapper->vgHandle = pVg;*/ + /*pRspWrapper->topicHandle = pTopic;*/ + taosWriteQitem(tmq->mqueue, pRspWrapper); + tsem_post(&tmq->rspSem); + } goto CREATE_MSG_FAIL; } @@ -1017,14 +1187,13 @@ bool tmqUpdateEp2(tmq_t* tmq, int32_t epoch, SMqAskEpRsp* pRsp) { if (pTopicCur->vgs) { int32_t vgNumCur = taosArrayGetSize(pTopicCur->vgs); tscDebug("consumer %ld new vg num: %d", tmq->consumerId, vgNumCur); - if (vgNumCur == 0) break; for (int32_t j = 0; j < vgNumCur; j++) { SMqClientVg* pVgCur = taosArrayGet(pTopicCur->vgs, j); sprintf(vgKey, "%s:%d", pTopicCur->topicName, pVgCur->vgId); - tscDebug("consumer %ld epoch %d vg %d build %s", tmq->consumerId, epoch, pVgCur->vgId, vgKey); + tscDebug("consumer %ld epoch %d vg %d vgKey is %s, offset is %ld", tmq->consumerId, epoch, pVgCur->vgId, vgKey, + pVgCur->currentOffset); taosHashPut(pHash, vgKey, strlen(vgKey), &pVgCur->currentOffset, sizeof(int64_t)); } - break; } } @@ -1032,7 +1201,6 @@ bool tmqUpdateEp2(tmq_t* tmq, int32_t epoch, SMqAskEpRsp* pRsp) { SMqClientTopic topic = {0}; SMqSubTopicEp* pTopicEp = taosArrayGet(pRsp->topics, i); topic.schema = pTopicEp->schema; - taosHashClear(pHash); topic.topicName = strdup(pTopicEp->topic); tstrncpy(topic.db, pTopicEp->db, TSDB_DB_FNAME_LEN); @@ -1049,7 +1217,8 @@ bool tmqUpdateEp2(tmq_t* tmq, int32_t epoch, SMqAskEpRsp* pRsp) { offset = *pOffset; } - tscDebug("consumer %ld(epoch %d) offset of vg %d updated to %ld", tmq->consumerId, epoch, pVgEp->vgId, offset); + tscDebug("consumer %ld(epoch %d) offset of vg %d updated to %ld, vgKey is %s", tmq->consumerId, epoch, + pVgEp->vgId, offset, vgKey); SMqClientVg clientVg = { .pollCnt = 0, .currentOffset = offset, @@ -1076,6 +1245,7 @@ bool tmqUpdateEp2(tmq_t* tmq, int32_t epoch, SMqAskEpRsp* pRsp) { return set; } +#if 1 bool tmqUpdateEp(tmq_t* tmq, int32_t epoch, SMqAskEpRsp* pRsp) { /*printf("call update ep %d\n", epoch);*/ bool set = false; @@ -1160,6 +1330,7 @@ bool tmqUpdateEp(tmq_t* tmq, int32_t epoch, SMqAskEpRsp* pRsp) { atomic_store_32(&tmq->epoch, epoch); return set; } +#endif int32_t tmqAskEpCb(void* param, const SDataBuf* pMsg, int32_t code) { SMqAskEpCbParam* pParam = (SMqAskEpCbParam*)param; @@ -1186,7 +1357,7 @@ int32_t tmqAskEpCb(void* param, const SDataBuf* pMsg, int32_t code) { tDecodeSMqAskEpRsp(POINTER_SHIFT(pMsg->pData, sizeof(SMqRspHead)), &rsp); /*printf("rsp epoch %ld sz %ld\n", rsp.epoch, rsp.topics->size);*/ /*printf("tmq epoch %ld sz %ld\n", tmq->epoch, tmq->clientTopics->size);*/ - tmqUpdateEp(tmq, head->epoch, &rsp); + tmqUpdateEp2(tmq, head->epoch, &rsp); tDeleteSMqAskEpRsp(&rsp); } else { SMqAskEpRspWrapper* pWrapper = taosAllocateQitem(sizeof(SMqAskEpRspWrapper), DEF_QITEM); @@ -1283,7 +1454,8 @@ int32_t tmqAskEp(tmq_t* tmq, bool async) { return code; } -tmq_resp_err_t tmq_seek(tmq_t* tmq, const tmq_topic_vgroup_t* offset) { +#if 0 +int32_t tmq_seek(tmq_t* tmq, const tmq_topic_vgroup_t* offset) { const SMqOffset* pOffset = &offset->offset; if (strcmp(pOffset->cgroup, tmq->groupId) != 0) { return TMQ_RESP_ERR__FAIL; @@ -1305,16 +1477,17 @@ tmq_resp_err_t tmq_seek(tmq_t* tmq, const tmq_topic_vgroup_t* offset) { } return TMQ_RESP_ERR__FAIL; } +#endif SMqPollReq* tmqBuildConsumeReqImpl(tmq_t* tmq, int64_t timeout, SMqClientTopic* pTopic, SMqClientVg* pVg) { int64_t reqOffset; if (pVg->currentOffset >= 0) { reqOffset = pVg->currentOffset; } else { - if (tmq->resetOffsetCfg == TMQ_CONF__RESET_OFFSET__NONE) { - tscError("unable to poll since no committed offset but reset offset is set to none"); - return NULL; - } + /*if (tmq->resetOffsetCfg == TMQ_CONF__RESET_OFFSET__NONE) {*/ + /*tscError("unable to poll since no committed offset but reset offset is set to none");*/ + /*return NULL;*/ + /*}*/ reqOffset = tmq->resetOffsetCfg; } @@ -1326,10 +1499,10 @@ SMqPollReq* tmqBuildConsumeReqImpl(tmq_t* tmq, int64_t timeout, SMqClientTopic* /*strcpy(pReq->topic, pTopic->topicName);*/ /*strcpy(pReq->cgroup, tmq->groupId);*/ - int32_t tlen = strlen(tmq->groupId); - memcpy(pReq->subKey, tmq->groupId, tlen); - pReq->subKey[tlen] = TMQ_SEPARATOR; - strcpy(pReq->subKey + tlen + 1, pTopic->topicName); + int32_t groupLen = strlen(tmq->groupId); + memcpy(pReq->subKey, tmq->groupId, groupLen); + pReq->subKey[groupLen] = TMQ_SEPARATOR; + strcpy(pReq->subKey + groupLen + 1, pTopic->topicName); pReq->withTbName = tmq->withTbName; pReq->timeout = timeout; @@ -1440,7 +1613,7 @@ int32_t tmqHandleNoPollRsp(tmq_t* tmq, SMqRspWrapper* rspWrapper, bool* pReset) if (rspWrapper->epoch > atomic_load_32(&tmq->epoch)) { SMqAskEpRspWrapper* pEpRspWrapper = (SMqAskEpRspWrapper*)rspWrapper; SMqAskEpRsp* rspMsg = &pEpRspWrapper->msg; - tmqUpdateEp(tmq, rspWrapper->epoch, rspMsg); + tmqUpdateEp2(tmq, rspWrapper->epoch, rspMsg); /*tmqClearUnhandleMsg(tmq);*/ *pReset = true; } else { @@ -1462,7 +1635,11 @@ SMqRspObj* tmqHandleAllRsp(tmq_t* tmq, int64_t timeout, bool pollIfReset) { if (rspWrapper == NULL) return NULL; } - if (rspWrapper->tmqRspType == TMQ_MSG_TYPE__POLL_RSP) { + if (rspWrapper->tmqRspType == TMQ_MSG_TYPE__END_RSP) { + taosFreeQitem(rspWrapper); + terrno = TSDB_CODE_TQ_NO_COMMITTED_OFFSET; + return NULL; + } else if (rspWrapper->tmqRspType == TMQ_MSG_TYPE__POLL_RSP) { SMqPollRspWrapper* pollRspWrapper = (SMqPollRspWrapper*)rspWrapper; /*atomic_sub_fetch_32(&tmq->readyRequest, 1);*/ int32_t consumerEpoch = atomic_load_32(&tmq->epoch); @@ -1523,6 +1700,8 @@ TAOS_RES* tmq_consumer_poll(tmq_t* tmq, int64_t timeout) { rspObj = tmqHandleAllRsp(tmq, timeout, false); if (rspObj) { return (TAOS_RES*)rspObj; + } else if (terrno == TSDB_CODE_TQ_NO_COMMITTED_OFFSET) { + return NULL; } if (timeout != -1) { int64_t endTime = taosGetTimestampMs(); @@ -1539,10 +1718,10 @@ TAOS_RES* tmq_consumer_poll(tmq_t* tmq, int64_t timeout) { } } -tmq_resp_err_t tmq_consumer_close(tmq_t* tmq) { +int32_t tmq_consumer_close(tmq_t* tmq) { if (tmq->status == TMQ_CONSUMER_STATUS__READY) { - tmq_resp_err_t rsp = tmq_commit_sync(tmq, NULL); - if (rsp != TMQ_RESP_ERR__SUCCESS) { + int32_t rsp = tmq_commit_sync(tmq, NULL); + if (rsp != 0) { return rsp; } @@ -1550,18 +1729,18 @@ tmq_resp_err_t tmq_consumer_close(tmq_t* tmq) { rsp = tmq_subscribe(tmq, lst); tmq_list_destroy(lst); - if (rsp != TMQ_RESP_ERR__SUCCESS) { + if (rsp != 0) { return rsp; } } // TODO: free resources - return TMQ_RESP_ERR__SUCCESS; + return 0; } -const char* tmq_err2str(tmq_resp_err_t err) { - if (err == TMQ_RESP_ERR__SUCCESS) { +const char* tmq_err2str(int32_t err) { + if (err == 0) { return "success"; - } else if (err == TMQ_RESP_ERR__FAIL) { + } else if (err == -1) { return "fail"; } else { return tstrerror(err); @@ -1607,10 +1786,8 @@ const char* tmq_get_table_name(TAOS_RES* res) { return NULL; } -void tmq_commit_async(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, tmq_commit_cb* cb, void* param) { - tmqCommitInner(tmq, offsets, 0, 1, cb, param); +void tmq_commit_async(tmq_t* tmq, const TAOS_RES* msg, tmq_commit_cb* cb, void* param) { + tmqCommitInner2(tmq, msg, 0, 1, cb, param); } -tmq_resp_err_t tmq_commit_sync(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets) { - return tmqCommitInner(tmq, offsets, 0, 0, NULL, NULL); -} +int32_t tmq_commit_sync(tmq_t* tmq, const TAOS_RES* msg) { return tmqCommitInner2(tmq, msg, 0, 0, NULL, NULL); } diff --git a/source/common/src/systable.c b/source/common/src/systable.c index 192d1d9922..a1b64b363c 100644 --- a/source/common/src/systable.c +++ b/source/common/src/systable.c @@ -220,7 +220,8 @@ static const SSysDbTableSchema transSchema[] = { {.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, {.name = "stage", .bytes = TSDB_TRANS_STAGE_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "db", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, + {.name = "db1", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, + {.name = "db2", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, {.name = "failed_times", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, {.name = "last_exec_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, {.name = "last_action_info", diff --git a/source/common/src/tdatablock.c b/source/common/src/tdatablock.c index 712b4fcf42..3c3d3e953d 100644 --- a/source/common/src/tdatablock.c +++ b/source/common/src/tdatablock.c @@ -1713,6 +1713,7 @@ int32_t buildSubmitReqFromDataBlock(SSubmitReq** pReq, const SArray* pDataBlocks } char* buildCtbNameByGroupId(const char* stbName, uint64_t groupId) { + ASSERT(stbName[0] != 0); SArray* tags = taosArrayInit(0, sizeof(void*)); SSmlKv* pTag = taosMemoryCalloc(1, sizeof(SSmlKv)); pTag->key = "group_id"; diff --git a/source/common/src/tmsg.c b/source/common/src/tmsg.c index d5dab72ef2..2443ffbb7f 100644 --- a/source/common/src/tmsg.c +++ b/source/common/src/tmsg.c @@ -204,7 +204,7 @@ static int32_t tSerializeSClientHbReq(SEncoder *pEncoder, const SClientHbReq *pR if (tEncodeU64(pEncoder, pReq->app.summary.numOfSlowQueries) < 0) return -1; if (tEncodeU64(pEncoder, pReq->app.summary.totalRequests) < 0) return -1; if (tEncodeU64(pEncoder, pReq->app.summary.currentRequests) < 0) return -1; - + int32_t queryNum = 0; if (pReq->query) { queryNum = 1; @@ -288,7 +288,7 @@ static int32_t tDeserializeSClientHbReq(SDecoder *pDecoder, SClientHbReq *pReq) if (tDecodeI64(pDecoder, &desc.useconds) < 0) return -1; if (tDecodeI64(pDecoder, &desc.stime) < 0) return -1; if (tDecodeI64(pDecoder, &desc.reqRid) < 0) return -1; - if (tDecodeI8(pDecoder, (int8_t*)&desc.stableQuery) < 0) return -1; + if (tDecodeI8(pDecoder, (int8_t *)&desc.stableQuery) < 0) return -1; if (tDecodeCStrTo(pDecoder, desc.fqdn) < 0) return -1; if (tDecodeI32(pDecoder, &desc.subPlanNum) < 0) return -1; @@ -496,8 +496,10 @@ int32_t tSerializeSMCreateStbReq(void *buf, int32_t bufLen, SMCreateStbReq *pReq if (tStartEncode(&encoder) < 0) return -1; if (tEncodeCStr(&encoder, pReq->name) < 0) return -1; if (tEncodeI8(&encoder, pReq->igExists) < 0) return -1; - if (tEncodeFloat(&encoder, pReq->xFilesFactor) < 0) return -1; - if (tEncodeI32(&encoder, pReq->delay) < 0) return -1; + if (tEncodeI64(&encoder, pReq->delay1) < 0) return -1; + if (tEncodeI64(&encoder, pReq->delay2) < 0) return -1; + if (tEncodeI64(&encoder, pReq->watermark1) < 0) return -1; + if (tEncodeI64(&encoder, pReq->watermark2) < 0) return -1; if (tEncodeI32(&encoder, pReq->ttl) < 0) return -1; if (tEncodeI32(&encoder, pReq->numOfColumns) < 0) return -1; if (tEncodeI32(&encoder, pReq->numOfTags) < 0) return -1; @@ -544,8 +546,10 @@ int32_t tDeserializeSMCreateStbReq(void *buf, int32_t bufLen, SMCreateStbReq *pR if (tStartDecode(&decoder) < 0) return -1; if (tDecodeCStrTo(&decoder, pReq->name) < 0) return -1; if (tDecodeI8(&decoder, &pReq->igExists) < 0) return -1; - if (tDecodeFloat(&decoder, &pReq->xFilesFactor) < 0) return -1; - if (tDecodeI32(&decoder, &pReq->delay) < 0) return -1; + if (tDecodeI64(&decoder, &pReq->delay1) < 0) return -1; + if (tDecodeI64(&decoder, &pReq->delay2) < 0) return -1; + if (tDecodeI64(&decoder, &pReq->watermark1) < 0) return -1; + if (tDecodeI64(&decoder, &pReq->watermark2) < 0) return -1; if (tDecodeI32(&decoder, &pReq->ttl) < 0) return -1; if (tDecodeI32(&decoder, &pReq->numOfColumns) < 0) return -1; if (tDecodeI32(&decoder, &pReq->numOfTags) < 0) return -1; diff --git a/source/dnode/mgmt/mgmt_dnode/src/dmWorker.c b/source/dnode/mgmt/mgmt_dnode/src/dmWorker.c index bd78fe46ef..9ffa0e606a 100644 --- a/source/dnode/mgmt/mgmt_dnode/src/dmWorker.c +++ b/source/dnode/mgmt/mgmt_dnode/src/dmWorker.c @@ -101,7 +101,8 @@ void dmStopMonitorThread(SDnodeMgmt *pMgmt) { static void dmProcessMgmtQueue(SQueueInfo *pInfo, SRpcMsg *pMsg) { SDnodeMgmt *pMgmt = pInfo->ahandle; int32_t code = -1; - dTrace("msg:%p, will be processed in dnode queue, type:%s", pMsg, TMSG_INFO(pMsg->msgType)); + STraceId * trace = &pMsg->info.traceId; + dGTrace("msg:%p, will be processed in dnode queue, type:%s", pMsg, TMSG_INFO(pMsg->msgType)); switch (pMsg->msgType) { case TDMT_DND_CONFIG_DNODE: diff --git a/source/dnode/mgmt/mgmt_mnode/src/mmWorker.c b/source/dnode/mgmt/mgmt_mnode/src/mmWorker.c index 88f667ce9e..60c42d31f5 100644 --- a/source/dnode/mgmt/mgmt_mnode/src/mmWorker.c +++ b/source/dnode/mgmt/mgmt_mnode/src/mmWorker.c @@ -48,7 +48,8 @@ static inline void mmSendRsp(SRpcMsg *pMsg, int32_t code) { static void mmProcessRpcMsg(SQueueInfo *pInfo, SRpcMsg *pMsg) { SMnodeMgmt *pMgmt = pInfo->ahandle; int32_t code = -1; - dTrace("msg:%p, get from mnode queue", pMsg); + STraceId * trace = &pMsg->info.traceId; + dGTrace("msg:%p, get from mnode queue", pMsg); switch (pMsg->msgType) { case TDMT_MON_MM_INFO: diff --git a/source/dnode/mgmt/mgmt_vnode/src/vmWorker.c b/source/dnode/mgmt/mgmt_vnode/src/vmWorker.c index 88831384d4..09fdd4b087 100644 --- a/source/dnode/mgmt/mgmt_vnode/src/vmWorker.c +++ b/source/dnode/mgmt/mgmt_vnode/src/vmWorker.c @@ -31,7 +31,8 @@ static void vmProcessMgmtQueue(SQueueInfo *pInfo, SRpcMsg *pMsg) { SVnodeMgmt *pMgmt = pInfo->ahandle; int32_t code = -1; - dTrace("msg:%p, get from vnode-mgmt queue", pMsg); + STraceId *trace = &pMsg->info.traceId; + dGTrace("msg:%p, get from vnode-mgmt queue", pMsg); switch (pMsg->msgType) { case TDMT_MON_VM_INFO: code = vmProcessGetMonitorInfoReq(pMgmt, pMsg); @@ -97,7 +98,7 @@ static void vmProcessFetchQueue(SQueueInfo *pInfo, SRpcMsg *pMsg) { static void vmProcessSyncQueue(SQueueInfo *pInfo, STaosQall *qall, int32_t numOfMsgs) { SVnodeObj *pVnode = pInfo->ahandle; - SRpcMsg *pMsg = NULL; + SRpcMsg * pMsg = NULL; for (int32_t i = 0; i < numOfMsgs; ++i) { if (taosGetQitem(qall, (void **)&pMsg) == 0) continue; @@ -118,7 +119,7 @@ static void vmProcessSyncQueue(SQueueInfo *pInfo, STaosQall *qall, int32_t numOf static void vmProcessMergeQueue(SQueueInfo *pInfo, STaosQall *qall, int32_t numOfMsgs) { SVnodeObj *pVnode = pInfo->ahandle; - SRpcMsg *pMsg = NULL; + SRpcMsg * pMsg = NULL; for (int32_t i = 0; i < numOfMsgs; ++i) { if (taosGetQitem(qall, (void **)&pMsg) == 0) continue; diff --git a/source/dnode/mgmt/node_mgmt/src/dmTransport.c b/source/dnode/mgmt/node_mgmt/src/dmTransport.c index 13f2452e66..660f512fc5 100644 --- a/source/dnode/mgmt/node_mgmt/src/dmTransport.c +++ b/source/dnode/mgmt/node_mgmt/src/dmTransport.c @@ -40,8 +40,9 @@ static void dmProcessRpcMsg(SDnode *pDnode, SRpcMsg *pRpc, SEpSet *pEpSet) { SMgmtWrapper *pWrapper = NULL; SDnodeHandle *pHandle = &pTrans->msgHandles[TMSG_INDEX(pRpc->msgType)]; - dTrace("msg:%s is received, handle:%p len:%d code:0x%x app:%p refId:%" PRId64, TMSG_INFO(pRpc->msgType), - pRpc->info.handle, pRpc->contLen, pRpc->code, pRpc->info.ahandle, pRpc->info.refId); + STraceId *trace = &pRpc->info.traceId; + dGTrace("msg:%s is received, handle:%p len:%d code:0x%x app:%p refId:%" PRId64, TMSG_INFO(pRpc->msgType), + pRpc->info.handle, pRpc->contLen, pRpc->code, pRpc->info.ahandle, pRpc->info.refId); if (pRpc->msgType == TDMT_DND_NET_TEST) { dmProcessNetTestReq(pDnode, pRpc); diff --git a/source/dnode/mgmt/node_util/inc/dmUtil.h b/source/dnode/mgmt/node_util/inc/dmUtil.h index c142a6cfd8..7897f62f62 100644 --- a/source/dnode/mgmt/node_util/inc/dmUtil.h +++ b/source/dnode/mgmt/node_util/inc/dmUtil.h @@ -34,13 +34,13 @@ #include "dnode.h" #include "mnode.h" -#include "qnode.h" #include "monitor.h" +#include "qnode.h" #include "sync.h" #include "wal.h" #include "libs/function/function.h" - +// clang-format off #ifdef __cplusplus extern "C" { #endif @@ -51,6 +51,7 @@ extern "C" { #define dInfo(...) { if (dDebugFlag & DEBUG_INFO) { taosPrintLog("DND ", DEBUG_INFO, 255, __VA_ARGS__); }} #define dDebug(...) { if (dDebugFlag & DEBUG_DEBUG) { taosPrintLog("DND ", DEBUG_DEBUG, dDebugFlag, __VA_ARGS__); }} #define dTrace(...) { if (dDebugFlag & DEBUG_TRACE) { taosPrintLog("DND ", DEBUG_TRACE, dDebugFlag, __VA_ARGS__); }} +#define dGTrace(param, ...) do { char buf[40] = {0}; TRACE_TO_STR(trace, buf); dTrace(param ",GTID: %s", __VA_ARGS__, buf);} while(0) typedef enum { DNODE = 0, @@ -184,3 +185,4 @@ void dmSetMnodeEpSet(SDnodeData *pData, SEpSet *pEpSet); #endif #endif /*_TD_DM_INT_H_*/ +// clang-format on diff --git a/source/dnode/mnode/impl/inc/mndDef.h b/source/dnode/mnode/impl/inc/mndDef.h index 4daeeaa9bf..8963f6be39 100644 --- a/source/dnode/mnode/impl/inc/mndDef.h +++ b/source/dnode/mnode/impl/inc/mndDef.h @@ -124,7 +124,8 @@ typedef struct { int32_t lastErrorNo; tmsg_t lastMsgType; SEpSet lastEpset; - char dbname[TSDB_DB_FNAME_LEN]; + char dbname1[TSDB_DB_FNAME_LEN]; + char dbname2[TSDB_DB_FNAME_LEN]; int32_t startFunc; int32_t stopFunc; int32_t paramLen; diff --git a/source/dnode/mnode/impl/inc/mndInt.h b/source/dnode/mnode/impl/inc/mndInt.h index 37bae1d5c0..c810a0cbc7 100644 --- a/source/dnode/mnode/impl/inc/mndInt.h +++ b/source/dnode/mnode/impl/inc/mndInt.h @@ -40,6 +40,8 @@ extern "C" { #define mInfo(...) { if (mDebugFlag & DEBUG_INFO) { taosPrintLog("MND ", DEBUG_INFO, 255, __VA_ARGS__); }} #define mDebug(...) { if (mDebugFlag & DEBUG_DEBUG) { taosPrintLog("MND ", DEBUG_DEBUG, mDebugFlag, __VA_ARGS__); }} #define mTrace(...) { if (mDebugFlag & DEBUG_TRACE) { taosPrintLog("MND ", DEBUG_TRACE, mDebugFlag, __VA_ARGS__); }} +#define mGTrace(param, ...) do { char buf[40] = {0}; TRACE_TO_STR(trace, buf); mTrace(param ", GTID: %s", __VA_ARGS__, buf);} while(0) + // clang-format on #define SYSTABLE_SCH_TABLE_NAME_LEN ((TSDB_TABLE_NAME_LEN - 1) + VARSTR_HEADER_SIZE) @@ -54,7 +56,7 @@ typedef void (*ShowFreeIterFp)(SMnode *pMnode, void *pIter); typedef struct SQWorker SQHandle; typedef struct { - const char *name; + const char * name; MndInitFp initFp; MndCleanupFp cleanupFp; } SMnodeStep; @@ -63,7 +65,7 @@ typedef struct { int64_t showId; ShowRetrieveFp retrieveFps[TSDB_MGMT_TABLE_MAX]; ShowFreeIterFp freeIterFps[TSDB_MGMT_TABLE_MAX]; - SCacheObj *cache; + SCacheObj * cache; } SShowMgmt; typedef struct { @@ -100,14 +102,14 @@ typedef struct SMnode { bool stopped; bool restored; bool deploy; - char *path; + char * path; int64_t checkTime; - SSdb *pSdb; - SArray *pSteps; - SQHandle *pQuery; - SHashObj *infosMeta; - SHashObj *perfsMeta; - SWal *pWal; + SSdb * pSdb; + SArray * pSteps; + SQHandle * pQuery; + SHashObj * infosMeta; + SHashObj * perfsMeta; + SWal * pWal; SShowMgmt showMgmt; SProfileMgmt profileMgmt; STelemMgmt telemMgmt; diff --git a/source/dnode/mnode/impl/inc/mndTrans.h b/source/dnode/mnode/impl/inc/mndTrans.h index 0175e29a77..bc2d5c82b1 100644 --- a/source/dnode/mnode/impl/inc/mndTrans.h +++ b/source/dnode/mnode/impl/inc/mndTrans.h @@ -68,7 +68,7 @@ int32_t mndTransAppendRedoAction(STrans *pTrans, STransAction *pAction); int32_t mndTransAppendUndoAction(STrans *pTrans, STransAction *pAction); void mndTransSetRpcRsp(STrans *pTrans, void *pCont, int32_t contLen); void mndTransSetCb(STrans *pTrans, ETrnFunc startFunc, ETrnFunc stopFunc, void *param, int32_t paramLen); -void mndTransSetDbName(STrans *pTrans, const char *dbname); +void mndTransSetDbName(STrans *pTrans, const char *dbname1, const char *dbname2); void mndTransSetSerial(STrans *pTrans); int32_t mndTransPrepare(SMnode *pMnode, STrans *pTrans); diff --git a/source/dnode/mnode/impl/src/mndDb.c b/source/dnode/mnode/impl/src/mndDb.c index 49f6aa1edf..345464399e 100644 --- a/source/dnode/mnode/impl/src/mndDb.c +++ b/source/dnode/mnode/impl/src/mndDb.c @@ -477,7 +477,7 @@ static int32_t mndCreateDb(SMnode *pMnode, SRpcMsg *pReq, SCreateDbReq *pCreate, mDebug("trans:%d, used to create db:%s", pTrans->id, pCreate->db); - mndTransSetDbName(pTrans, dbObj.name); + mndTransSetDbName(pTrans, dbObj.name, NULL); if (mndSetCreateDbRedoLogs(pMnode, pTrans, &dbObj, pVgroups) != 0) goto _OVER; if (mndSetCreateDbUndoLogs(pMnode, pTrans, &dbObj, pVgroups) != 0) goto _OVER; if (mndSetCreateDbCommitLogs(pMnode, pTrans, &dbObj, pVgroups) != 0) goto _OVER; @@ -668,7 +668,7 @@ static int32_t mndAlterDb(SMnode *pMnode, SRpcMsg *pReq, SDbObj *pOld, SDbObj *p mDebug("trans:%d, used to alter db:%s", pTrans->id, pOld->name); int32_t code = -1; - mndTransSetDbName(pTrans, pOld->name); + mndTransSetDbName(pTrans, pOld->name, NULL); if (mndSetAlterDbRedoLogs(pMnode, pTrans, pOld, pNew) != 0) goto _OVER; if (mndSetAlterDbCommitLogs(pMnode, pTrans, pOld, pNew) != 0) goto _OVER; if (mndSetAlterDbRedoActions(pMnode, pTrans, pOld, pNew) != 0) goto _OVER; @@ -921,7 +921,7 @@ static int32_t mndDropDb(SMnode *pMnode, SRpcMsg *pReq, SDbObj *pDb) { if (pTrans == NULL) goto _OVER; mDebug("trans:%d, used to drop db:%s", pTrans->id, pDb->name); - mndTransSetDbName(pTrans, pDb->name); + mndTransSetDbName(pTrans, pDb->name, NULL); if (mndSetDropDbRedoLogs(pMnode, pTrans, pDb) != 0) goto _OVER; if (mndSetDropDbCommitLogs(pMnode, pTrans, pDb) != 0) goto _OVER; diff --git a/source/dnode/mnode/impl/src/mndDnode.c b/source/dnode/mnode/impl/src/mndDnode.c index d1bed90175..58c3570c36 100644 --- a/source/dnode/mnode/impl/src/mndDnode.c +++ b/source/dnode/mnode/impl/src/mndDnode.c @@ -761,11 +761,12 @@ static int32_t mndRetrieveDnodes(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pB pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); colDataAppend(pColInfo, numOfRows, (const char *)&pDnode->createdTime, false); - char b[tListLen(offlineReason) + VARSTR_HEADER_SIZE] = {0}; + char *b = taosMemoryCalloc(VARSTR_HEADER_SIZE + strlen(offlineReason[pDnode->offlineReason]) + 1, 1); STR_TO_VARSTR(b, online ? "" : offlineReason[pDnode->offlineReason]); pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); colDataAppend(pColInfo, numOfRows, b, false); + taosMemoryFreeClear(b); numOfRows++; sdbRelease(pSdb, pDnode); diff --git a/source/dnode/mnode/impl/src/mndMain.c b/source/dnode/mnode/impl/src/mndMain.c index 7b5025a986..fa6f45bdad 100644 --- a/source/dnode/mnode/impl/src/mndMain.c +++ b/source/dnode/mnode/impl/src/mndMain.c @@ -58,21 +58,21 @@ static void *mndBuildTimerMsg(int32_t *pContLen) { static void mndPullupTrans(SMnode *pMnode) { int32_t contLen = 0; - void *pReq = mndBuildTimerMsg(&contLen); + void * pReq = mndBuildTimerMsg(&contLen); SRpcMsg rpcMsg = {.msgType = TDMT_MND_TRANS_TIMER, .pCont = pReq, .contLen = contLen}; tmsgPutToQueue(&pMnode->msgCb, WRITE_QUEUE, &rpcMsg); } static void mndCalMqRebalance(SMnode *pMnode) { int32_t contLen = 0; - void *pReq = mndBuildTimerMsg(&contLen); + void * pReq = mndBuildTimerMsg(&contLen); SRpcMsg rpcMsg = {.msgType = TDMT_MND_MQ_TIMER, .pCont = pReq, .contLen = contLen}; tmsgPutToQueue(&pMnode->msgCb, READ_QUEUE, &rpcMsg); } static void mndPullupTelem(SMnode *pMnode) { int32_t contLen = 0; - void *pReq = mndBuildTimerMsg(&contLen); + void * pReq = mndBuildTimerMsg(&contLen); SRpcMsg rpcMsg = {.msgType = TDMT_MND_TELEM_TIMER, .pCont = pReq, .contLen = contLen}; tmsgPutToQueue(&pMnode->msgCb, READ_QUEUE, &rpcMsg); } @@ -378,7 +378,7 @@ void mndStop(SMnode *pMnode) { } int32_t mndProcessSyncMsg(SRpcMsg *pMsg) { - SMnode *pMnode = pMsg->info.node; + SMnode * pMnode = pMsg->info.node; SSyncMgmt *pMgmt = &pMnode->syncMgmt; int32_t code = 0; @@ -510,8 +510,7 @@ static int32_t mndCheckMnodeState(SRpcMsg *pMsg) { if (IsReq(pMsg) && pMsg->msgType != TDMT_MND_MQ_TIMER && pMsg->msgType != TDMT_MND_TELEM_TIMER && pMsg->msgType != TDMT_MND_TRANS_TIMER) { - mError("msg:%p, failed to check mnode state since %s, app:%p type:%s", pMsg, terrstr(), pMsg->info.ahandle, - TMSG_INFO(pMsg->msgType)); + mError("msg:%p, failed to check mnode state since %s, type:%s", pMsg, terrstr(), TMSG_INFO(pMsg->msgType)); SEpSet epSet = {0}; mndGetMnodeEpSet(pMsg->info.node, &epSet); @@ -534,13 +533,14 @@ static int32_t mndCheckMsgContent(SRpcMsg *pMsg) { if (!IsReq(pMsg)) return 0; if (pMsg->contLen != 0 && pMsg->pCont != NULL) return 0; - mError("msg:%p, failed to check msg content, app:%p type:%s", pMsg, pMsg->info.ahandle, TMSG_INFO(pMsg->msgType)); + mError("msg:%p, failed to check msg, cont:%p contLen:%d, app:%p type:%s", pMsg, pMsg->pCont, pMsg->contLen, + pMsg->info.ahandle, TMSG_INFO(pMsg->msgType)); terrno = TSDB_CODE_INVALID_MSG_LEN; return -1; } int32_t mndProcessRpcMsg(SRpcMsg *pMsg) { - SMnode *pMnode = pMsg->info.node; + SMnode * pMnode = pMsg->info.node; MndMsgFp fp = pMnode->msgFp[TMSG_INDEX(pMsg->msgType)]; if (fp == NULL) { mError("msg:%p, failed to get msg handle, app:%p type:%s", pMsg, pMsg->info.ahandle, TMSG_INFO(pMsg->msgType)); @@ -551,14 +551,15 @@ int32_t mndProcessRpcMsg(SRpcMsg *pMsg) { if (mndCheckMsgContent(pMsg) != 0) return -1; if (mndCheckMnodeState(pMsg) != 0) return -1; - mTrace("msg:%p, start to process in mnode, app:%p type:%s", pMsg, pMsg->info.ahandle, TMSG_INFO(pMsg->msgType)); + STraceId *trace = &pMsg->info.traceId; + mGTrace("msg:%p, start to process in mnode, app:%p type:%s", pMsg, pMsg->info.ahandle, TMSG_INFO(pMsg->msgType)); int32_t code = (*fp)(pMsg); mndReleaseRpcRef(pMnode); if (code == TSDB_CODE_ACTION_IN_PROGRESS) { mTrace("msg:%p, won't response immediately since in progress", pMsg); } else if (code == 0) { - mTrace("msg:%p, successfully processed and response", pMsg); + mTrace("msg:%p, successfully processed", pMsg); } else { mError("msg:%p, failed to process since %s, app:%p type:%s", pMsg, terrstr(), pMsg->info.ahandle, TMSG_INFO(pMsg->msgType)); @@ -592,7 +593,7 @@ int32_t mndGetMonitorInfo(SMnode *pMnode, SMonClusterInfo *pClusterInfo, SMonVgr SMonGrantInfo *pGrantInfo) { if (mndAcquireRpcRef(pMnode) != 0) return -1; - SSdb *pSdb = pMnode->pSdb; + SSdb * pSdb = pMnode->pSdb; int64_t ms = taosGetTimestampMs(); pClusterInfo->dnodes = taosArrayInit(sdbGetSize(pSdb, SDB_DNODE), sizeof(SMonDnodeDesc)); @@ -668,7 +669,7 @@ int32_t mndGetMonitorInfo(SMnode *pMnode, SMonClusterInfo *pClusterInfo, SMonVgr pGrantInfo->timeseries_used += pVgroup->numOfTimeSeries; tstrncpy(desc.status, "unsynced", sizeof(desc.status)); for (int32_t i = 0; i < pVgroup->replica; ++i) { - SVnodeGid *pVgid = &pVgroup->vnodeGid[i]; + SVnodeGid * pVgid = &pVgroup->vnodeGid[i]; SMonVnodeDesc *pVnDesc = &desc.vnodes[i]; pVnDesc->dnode_id = pVgid->dnodeId; tstrncpy(pVnDesc->vnode_role, syncStr(pVgid->role), sizeof(pVnDesc->vnode_role)); diff --git a/source/dnode/mnode/impl/src/mndScheduler.c b/source/dnode/mnode/impl/src/mndScheduler.c index 6f8fc748c2..39bb6798aa 100644 --- a/source/dnode/mnode/impl/src/mndScheduler.c +++ b/source/dnode/mnode/impl/src/mndScheduler.c @@ -105,7 +105,7 @@ int32_t mndPersistTaskDeployReq(STrans* pTrans, SStreamTask* pTask, const SEpSet int32_t size = encoder.pos; int32_t tlen = sizeof(SMsgHead) + size; tEncoderClear(&encoder); - void* buf = taosMemoryMalloc(tlen); + void* buf = taosMemoryCalloc(1, tlen); if (buf == NULL) { terrno = TSDB_CODE_OUT_OF_MEMORY; return -1; @@ -157,6 +157,7 @@ int32_t mndAddDispatcherToInnerTask(SMnode* pMnode, STrans* pTrans, SStreamObj* } sdbRelease(pMnode->pSdb, pDb); + memcpy(pTask->shuffleDispatcher.stbFullName, pStream->targetSTbName, TSDB_TABLE_FNAME_LEN); SArray* pVgs = pTask->shuffleDispatcher.dbInfo.pVgroupInfos; int32_t sz = taosArrayGetSize(pVgs); SArray* sinkLv = taosArrayGetP(pStream->tasks, 0); @@ -166,6 +167,7 @@ int32_t mndAddDispatcherToInnerTask(SMnode* pMnode, STrans* pTrans, SStreamObj* for (int32_t j = 0; j < sinkLvSize; j++) { SStreamTask* pLastLevelTask = taosArrayGetP(sinkLv, j); if (pLastLevelTask->nodeId == pVgInfo->vgId) { + ASSERT(pVgInfo->vgId > 0); pVgInfo->taskId = pLastLevelTask->taskId; ASSERT(pVgInfo->taskId != 0); break; diff --git a/source/dnode/mnode/impl/src/mndSma.c b/source/dnode/mnode/impl/src/mndSma.c index 023a28ce35..c44fb03be2 100644 --- a/source/dnode/mnode/impl/src/mndSma.c +++ b/source/dnode/mnode/impl/src/mndSma.c @@ -530,7 +530,10 @@ static int32_t mndCreateSma(SMnode *pMnode, SRpcMsg *pReq, SMCreateSmaReq *pCrea smaObj.dbUid = pStb->dbUid; smaObj.intervalUnit = pCreate->intervalUnit; smaObj.slidingUnit = pCreate->slidingUnit; +#if 0 smaObj.timezone = pCreate->timezone; +#endif + smaObj.timezone = tsTimezone; // use timezone of server smaObj.interval = pCreate->interval; smaObj.offset = pCreate->offset; smaObj.sliding = pCreate->sliding; @@ -609,7 +612,7 @@ static int32_t mndCreateSma(SMnode *pMnode, SRpcMsg *pReq, SMCreateSmaReq *pCrea int32_t code = -1; STrans *pTrans = mndTransCreate(pMnode, TRN_POLICY_RETRY, TRN_CONFLICT_DB, pReq); if (pTrans == NULL) goto _OVER; - mndTransSetDbName(pTrans, pDb->name); + mndTransSetDbName(pTrans, pDb->name, NULL); mndTransSetSerial(pTrans); mDebug("trans:%d, used to create sma:%s", pTrans->id, pCreate->name); @@ -852,7 +855,7 @@ static int32_t mndDropSma(SMnode *pMnode, SRpcMsg *pReq, SDbObj *pDb, SSmaObj *p if (pTrans == NULL) goto _OVER; mDebug("trans:%d, used to drop sma:%s", pTrans->id, pSma->name); - mndTransSetDbName(pTrans, pDb->name); + mndTransSetDbName(pTrans, pDb->name, NULL); if (mndSetDropSmaRedoLogs(pMnode, pTrans, pSma) != 0) goto _OVER; if (mndSetDropSmaVgroupRedoLogs(pMnode, pTrans, pVgroup) != 0) goto _OVER; diff --git a/source/dnode/mnode/impl/src/mndStb.c b/source/dnode/mnode/impl/src/mndStb.c index b8b22cee85..c37cdd1888 100644 --- a/source/dnode/mnode/impl/src/mndStb.c +++ b/source/dnode/mnode/impl/src/mndStb.c @@ -669,8 +669,8 @@ int32_t mndBuildStbFromReq(SMnode *pMnode, SStbObj *pDst, SMCreateStbReq *pCreat pDst->tagVer = 1; pDst->colVer = 1; pDst->nextColId = 1; - pDst->xFilesFactor = pCreate->xFilesFactor; - pDst->delay = pCreate->delay; + // pDst->xFilesFactor = pCreate->xFilesFactor; + // pDst->delay = pCreate->delay; pDst->ttl = pCreate->ttl; pDst->numOfColumns = pCreate->numOfColumns; pDst->numOfTags = pCreate->numOfTags; @@ -758,7 +758,7 @@ _OVER: } int32_t mndAddStbToTrans(SMnode *pMnode, STrans *pTrans, SDbObj *pDb, SStbObj *pStb) { - mndTransSetDbName(pTrans, pDb->name); + mndTransSetDbName(pTrans, pDb->name, NULL); if (mndSetCreateStbRedoLogs(pMnode, pTrans, pDb, pStb) != 0) return -1; if (mndSetCreateStbUndoLogs(pMnode, pTrans, pDb, pStb) != 0) return -1; if (mndSetCreateStbCommitLogs(pMnode, pTrans, pDb, pStb) != 0) return -1; @@ -1396,7 +1396,7 @@ static int32_t mndAlterStb(SMnode *pMnode, SRpcMsg *pReq, const SMAlterStbReq *p if (pTrans == NULL) goto _OVER; mDebug("trans:%d, used to alter stb:%s", pTrans->id, pAlter->name); - mndTransSetDbName(pTrans, pDb->name); + mndTransSetDbName(pTrans, pDb->name, NULL); if (needRsp) { void *pCont = NULL; @@ -1537,7 +1537,7 @@ static int32_t mndDropStb(SMnode *pMnode, SRpcMsg *pReq, SDbObj *pDb, SStbObj *p if (pTrans == NULL) goto _OVER; mDebug("trans:%d, used to drop stb:%s", pTrans->id, pStb->name); - mndTransSetDbName(pTrans, pDb->name); + mndTransSetDbName(pTrans, pDb->name, NULL); if (mndSetDropStbRedoLogs(pMnode, pTrans, pStb) != 0) goto _OVER; if (mndSetDropStbCommitLogs(pMnode, pTrans, pStb) != 0) goto _OVER; diff --git a/source/dnode/mnode/impl/src/mndStream.c b/source/dnode/mnode/impl/src/mndStream.c index 8e82946d68..d432256f15 100644 --- a/source/dnode/mnode/impl/src/mndStream.c +++ b/source/dnode/mnode/impl/src/mndStream.c @@ -613,9 +613,9 @@ static int32_t mndProcessCreateStreamReq(SRpcMsg *pReq) { goto _OVER; } - mndTransSetDbName(pTrans, createStreamReq.sourceDB); + mndTransSetDbName(pTrans, createStreamReq.sourceDB, NULL); // TODO - /*mndTransSetDbName(pTrans, streamObj.targetDb);*/ + /*mndTransSetDbName(pTrans, streamObj.targetDb, NULL);*/ mDebug("trans:%d, used to create stream:%s", pTrans->id, createStreamReq.name); // build stream obj from request diff --git a/source/dnode/mnode/impl/src/mndSubscribe.c b/source/dnode/mnode/impl/src/mndSubscribe.c index 65a5d22bec..d2b7a61e83 100644 --- a/source/dnode/mnode/impl/src/mndSubscribe.c +++ b/source/dnode/mnode/impl/src/mndSubscribe.c @@ -403,7 +403,7 @@ static int32_t mndDoRebalance(SMnode *pMnode, const SMqRebInputObj *pInput, SMqR static int32_t mndPersistRebResult(SMnode *pMnode, SRpcMsg *pMsg, const SMqRebOutputObj *pOutput) { STrans *pTrans = mndTransCreate(pMnode, TRN_POLICY_ROLLBACK, TRN_CONFLICT_DB_INSIDE, pMsg); - mndTransSetDbName(pTrans, pOutput->pSub->dbName); + mndTransSetDbName(pTrans, pOutput->pSub->dbName, NULL); if (pTrans == NULL) return -1; // make txn: diff --git a/source/dnode/mnode/impl/src/mndSync.c b/source/dnode/mnode/impl/src/mndSync.c index 8883431ca8..3e3850de1a 100644 --- a/source/dnode/mnode/impl/src/mndSync.c +++ b/source/dnode/mnode/impl/src/mndSync.c @@ -46,13 +46,14 @@ void mndSyncCommitMsg(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta cbM int32_t transId = sdbGetIdFromRaw(pMnode->pSdb, pRaw); pMgmt->errCode = cbMeta.code; - mDebug("trans:%d, is proposed, saved:%d code:0x%x, index:%" PRId64 " term:%" PRId64 " role:%s raw:%p", transId, - pMgmt->transId, cbMeta.code, cbMeta.index, cbMeta.term, syncStr(cbMeta.state), pRaw); + mDebug("trans:%d, is proposed, saved:%d code:0x%x, apply index:%" PRId64 " term:%" PRIu64 " config:%" PRId64 + " role:%s raw:%p", + transId, pMgmt->transId, cbMeta.code, cbMeta.index, cbMeta.term, cbMeta.lastConfigIndex, syncStr(cbMeta.state), + pRaw); if (pMgmt->errCode == 0) { sdbWriteWithoutFree(pMnode->pSdb, pRaw); - sdbSetApplyIndex(pMnode->pSdb, cbMeta.index); - sdbSetApplyTerm(pMnode->pSdb, cbMeta.term); + sdbSetApplyInfo(pMnode->pSdb, cbMeta.index, cbMeta.term, cbMeta.lastConfigIndex); } if (pMgmt->transId == transId) { @@ -68,36 +69,19 @@ void mndSyncCommitMsg(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta cbM mndReleaseTrans(pMnode, pTrans); } - if (cbMeta.index - sdbGetApplyIndex(pMnode->pSdb) > 100) { - SSnapshotMeta sMeta = {0}; - // if (syncGetSnapshotMeta(pMnode->syncMgmt.sync, &sMeta) == 0) { - if (syncGetSnapshotMetaByIndex(pMnode->syncMgmt.sync, cbMeta.index, &sMeta) == 0) { - sdbSetCurConfig(pMnode->pSdb, sMeta.lastConfigIndex); - } - sdbWriteFile(pMnode->pSdb); - } + sdbWriteFile(pMnode->pSdb, SDB_WRITE_DELTA); } } int32_t mndSyncGetSnapshot(struct SSyncFSM *pFsm, SSnapshot *pSnapshot) { SMnode *pMnode = pFsm->data; - pSnapshot->lastApplyIndex = sdbGetCommitIndex(pMnode->pSdb); - pSnapshot->lastApplyTerm = sdbGetCommitTerm(pMnode->pSdb); - pSnapshot->lastConfigIndex = sdbGetCurConfig(pMnode->pSdb); + sdbGetCommitInfo(pMnode->pSdb, &pSnapshot->lastApplyIndex, &pSnapshot->lastApplyTerm, &pSnapshot->lastConfigIndex); return 0; } void mndRestoreFinish(struct SSyncFSM *pFsm) { SMnode *pMnode = pFsm->data; - SSnapshotMeta sMeta = {0}; - // if (syncGetSnapshotMeta(pMnode->syncMgmt.sync, &sMeta) == 0) { - - SyncIndex snapshotIndex = sdbGetApplyIndex(pMnode->pSdb); - if (syncGetSnapshotMetaByIndex(pMnode->syncMgmt.sync, snapshotIndex, &sMeta) == 0) { - sdbSetCurConfig(pMnode->pSdb, sMeta.lastConfigIndex); - } - if (!pMnode->deploy) { mInfo("mnode sync restore finished, and will handle outstanding transactions"); mndTransPullup(pMnode); diff --git a/source/dnode/mnode/impl/src/mndTopic.c b/source/dnode/mnode/impl/src/mndTopic.c index 8afb7ab354..9632c04f4c 100644 --- a/source/dnode/mnode/impl/src/mndTopic.c +++ b/source/dnode/mnode/impl/src/mndTopic.c @@ -566,7 +566,7 @@ static int32_t mndProcessDropTopicReq(SRpcMsg *pReq) { #endif STrans *pTrans = mndTransCreate(pMnode, TRN_POLICY_ROLLBACK, TRN_CONFLICT_DB_INSIDE, pReq); - mndTransSetDbName(pTrans, pTopic->db); + mndTransSetDbName(pTrans, pTopic->db, NULL); if (pTrans == NULL) { mError("topic:%s, failed to drop since %s", pTopic->name, terrstr()); return -1; diff --git a/source/dnode/mnode/impl/src/mndTrans.c b/source/dnode/mnode/impl/src/mndTrans.c index 858c6f956f..31a955b030 100644 --- a/source/dnode/mnode/impl/src/mndTrans.c +++ b/source/dnode/mnode/impl/src/mndTrans.c @@ -22,8 +22,8 @@ #include "mndSync.h" #include "mndUser.h" -#define TRANS_VER_NUMBER 1 -#define TRANS_ARRAY_SIZE 8 +#define TRANS_VER_NUMBER 1 +#define TRANS_ARRAY_SIZE 8 #define TRANS_RESERVE_SIZE 64 static SSdbRaw *mndTransActionEncode(STrans *pTrans); @@ -52,7 +52,7 @@ static bool mndTransPerformCommitActionStage(SMnode *pMnode, STrans *pTrans); static bool mndTransPerformCommitStage(SMnode *pMnode, STrans *pTrans); static bool mndTransPerformRollbackStage(SMnode *pMnode, STrans *pTrans); static bool mndTransPerfromFinishedStage(SMnode *pMnode, STrans *pTrans); -static bool mndCantExecuteTransAction(SMnode *pMnode) { return !pMnode->deploy && !mndIsMaster(pMnode); } +static bool mndCannotExecuteTransAction(SMnode *pMnode) { return !pMnode->deploy && !mndIsMaster(pMnode); } static void mndTransSendRpcRsp(SMnode *pMnode, STrans *pTrans); static int32_t mndProcessTransReq(SRpcMsg *pReq); @@ -122,7 +122,8 @@ static SSdbRaw *mndTransActionEncode(STrans *pTrans) { SDB_SET_INT8(pRaw, dataPos, pTrans->conflict, _OVER) SDB_SET_INT8(pRaw, dataPos, pTrans->exec, _OVER) SDB_SET_INT64(pRaw, dataPos, pTrans->createdTime, _OVER) - SDB_SET_BINARY(pRaw, dataPos, pTrans->dbname, TSDB_DB_FNAME_LEN, _OVER) + SDB_SET_BINARY(pRaw, dataPos, pTrans->dbname1, TSDB_DB_FNAME_LEN, _OVER) + SDB_SET_BINARY(pRaw, dataPos, pTrans->dbname2, TSDB_DB_FNAME_LEN, _OVER) SDB_SET_INT32(pRaw, dataPos, pTrans->redoActionPos, _OVER) int32_t redoActionNum = taosArrayGetSize(pTrans->redoActions); @@ -270,7 +271,8 @@ static SSdbRow *mndTransActionDecode(SSdbRaw *pRaw) { pTrans->conflict = conflict; pTrans->exec = exec; SDB_GET_INT64(pRaw, dataPos, &pTrans->createdTime, _OVER) - SDB_GET_BINARY(pRaw, dataPos, pTrans->dbname, TSDB_DB_FNAME_LEN, _OVER) + SDB_GET_BINARY(pRaw, dataPos, pTrans->dbname1, TSDB_DB_FNAME_LEN, _OVER) + SDB_GET_BINARY(pRaw, dataPos, pTrans->dbname2, TSDB_DB_FNAME_LEN, _OVER) SDB_GET_INT32(pRaw, dataPos, &pTrans->redoActionPos, _OVER) SDB_GET_INT32(pRaw, dataPos, &redoActionNum, _OVER) SDB_GET_INT32(pRaw, dataPos, &undoActionNum, _OVER) @@ -521,9 +523,10 @@ static int32_t mndTransActionUpdate(SSdb *pSdb, STrans *pOld, STrans *pNew) { } if (pOld->stage == TRN_STAGE_ROLLBACK) { - pOld->stage = TRN_STAGE_FINISHED; - mTrace("trans:%d, stage from rollback to finished since perform update action", pNew->id); + pOld->stage = TRN_STAGE_REDO_ACTION; + mTrace("trans:%d, stage from rollback to undoAction since perform update action", pNew->id); } + return 0; } @@ -649,7 +652,14 @@ void mndTransSetCb(STrans *pTrans, ETrnFunc startFunc, ETrnFunc stopFunc, void * pTrans->paramLen = paramLen; } -void mndTransSetDbName(STrans *pTrans, const char *dbname) { memcpy(pTrans->dbname, dbname, TSDB_DB_FNAME_LEN); } +void mndTransSetDbName(STrans *pTrans, const char *dbname1, const char *dbname2) { + if (dbname1 != NULL) { + memcpy(pTrans->dbname1, dbname1, TSDB_DB_FNAME_LEN); + } + if (dbname2 != NULL) { + memcpy(pTrans->dbname2, dbname2, TSDB_DB_FNAME_LEN); + } +} void mndTransSetSerial(STrans *pTrans) { pTrans->exec = TRN_EXEC_SERIAL; } @@ -661,7 +671,7 @@ static int32_t mndTransSync(SMnode *pMnode, STrans *pTrans) { } sdbSetRawStatus(pRaw, SDB_STATUS_READY); - mDebug("trans:%d, sync to other mnodes", pTrans->id); + mDebug("trans:%d, sync to other mnodes, stage:%s", pTrans->id, mndTransStr(pTrans->stage)); int32_t code = mndSyncPropose(pMnode, pRaw, pTrans->id); if (code != 0) { mError("trans:%d, failed to sync since %s", pTrans->id, terrstr()); @@ -674,6 +684,12 @@ static int32_t mndTransSync(SMnode *pMnode, STrans *pTrans) { return 0; } +static bool mndCheckDbConflict(const char *db, STrans *pTrans) { + if (db[0] == 0) return false; + if (strcmp(db, pTrans->dbname1) == 0 || strcmp(db, pTrans->dbname2) == 0) return true; + return false; +} + static bool mndCheckTransConflict(SMnode *pMnode, STrans *pNew) { STrans *pTrans = NULL; void *pIter = NULL; @@ -688,14 +704,21 @@ static bool mndCheckTransConflict(SMnode *pMnode, STrans *pNew) { if (pNew->conflict == TRN_CONFLICT_GLOBAL) conflict = true; if (pNew->conflict == TRN_CONFLICT_DB) { if (pTrans->conflict == TRN_CONFLICT_GLOBAL) conflict = true; - if (pTrans->conflict == TRN_CONFLICT_DB && strcmp(pNew->dbname, pTrans->dbname) == 0) conflict = true; - if (pTrans->conflict == TRN_CONFLICT_DB_INSIDE && strcmp(pNew->dbname, pTrans->dbname) == 0) conflict = true; + if (pTrans->conflict == TRN_CONFLICT_DB || pTrans->conflict == TRN_CONFLICT_DB_INSIDE) { + if (mndCheckDbConflict(pNew->dbname1, pTrans)) conflict = true; + if (mndCheckDbConflict(pNew->dbname2, pTrans)) conflict = true; + } } if (pNew->conflict == TRN_CONFLICT_DB_INSIDE) { if (pTrans->conflict == TRN_CONFLICT_GLOBAL) conflict = true; - if (pTrans->conflict == TRN_CONFLICT_DB && strcmp(pNew->dbname, pTrans->dbname) == 0) conflict = true; + if (pTrans->conflict == TRN_CONFLICT_DB) { + if (mndCheckDbConflict(pNew->dbname1, pTrans)) conflict = true; + if (mndCheckDbConflict(pNew->dbname2, pTrans)) conflict = true; + } } - mError("trans:%d, can't execute since conflict with trans:%d, db:%s", pNew->id, pTrans->id, pTrans->dbname); + + mError("trans:%d, can't execute since conflict with trans:%d, db1:%s db2:%s", pNew->id, pTrans->id, pTrans->dbname1, + pTrans->dbname2); sdbRelease(pMnode->pSdb, pTrans); } @@ -704,7 +727,7 @@ static bool mndCheckTransConflict(SMnode *pMnode, STrans *pNew) { int32_t mndTransPrepare(SMnode *pMnode, STrans *pTrans) { if (pTrans->conflict == TRN_CONFLICT_DB || pTrans->conflict == TRN_CONFLICT_DB_INSIDE) { - if (strlen(pTrans->dbname) == 0) { + if (strlen(pTrans->dbname1) == 0 && strlen(pTrans->dbname2) == 0) { terrno = TSDB_CODE_MND_TRANS_CONFLICT; mError("trans:%d, failed to prepare conflict db not set", pTrans->id); return -1; @@ -915,7 +938,7 @@ static int32_t mndTransWriteSingleLog(SMnode *pMnode, STrans *pTrans, STransActi static int32_t mndTransSendSingleMsg(SMnode *pMnode, STrans *pTrans, STransAction *pAction) { if (pAction->msgSent) return 0; - if (mndCantExecuteTransAction(pMnode)) return -1; + if (mndCannotExecuteTransAction(pMnode)) return -1; int64_t signature = pTrans->id; signature = (signature << 32); @@ -1115,7 +1138,7 @@ static int32_t mndTransExecuteRedoActionsSerial(SMnode *pMnode, STrans *pTrans) pTrans->lastEpset = pAction->epSet; } - if (mndCantExecuteTransAction(pMnode)) break; + if (mndCannotExecuteTransAction(pMnode)) break; if (code == 0) { pTrans->code = 0; @@ -1158,7 +1181,7 @@ static bool mndTransPerformRedoActionStage(SMnode *pMnode, STrans *pTrans) { code = mndTransExecuteRedoActions(pMnode, pTrans); } - if (mndCantExecuteTransAction(pMnode)) return false; + if (mndCannotExecuteTransAction(pMnode)) return false; if (code == 0) { pTrans->code = 0; @@ -1171,8 +1194,8 @@ static bool mndTransPerformRedoActionStage(SMnode *pMnode, STrans *pTrans) { } else { pTrans->code = terrno; if (pTrans->policy == TRN_POLICY_ROLLBACK) { - pTrans->stage = TRN_STAGE_UNDO_ACTION; - mError("trans:%d, stage from redoAction to undoAction since %s", pTrans->id, terrstr()); + pTrans->stage = TRN_STAGE_ROLLBACK; + mError("trans:%d, stage from redoAction to rollback since %s", pTrans->id, terrstr()); continueExec = true; } else { pTrans->failedTimes++; @@ -1185,7 +1208,7 @@ static bool mndTransPerformRedoActionStage(SMnode *pMnode, STrans *pTrans) { } static bool mndTransPerformCommitStage(SMnode *pMnode, STrans *pTrans) { - if (mndCantExecuteTransAction(pMnode)) return false; + if (mndCannotExecuteTransAction(pMnode)) return false; bool continueExec = true; int32_t code = mndTransCommit(pMnode, pTrans); @@ -1197,16 +1220,9 @@ static bool mndTransPerformCommitStage(SMnode *pMnode, STrans *pTrans) { continueExec = true; } else { pTrans->code = terrno; - if (pTrans->policy == TRN_POLICY_ROLLBACK) { - pTrans->stage = TRN_STAGE_UNDO_ACTION; - mError("trans:%d, stage from commit to undoAction since %s, failedTimes:%d", pTrans->id, terrstr(), - pTrans->failedTimes); - continueExec = true; - } else { - pTrans->failedTimes++; - mError("trans:%d, stage keep on commit since %s, failedTimes:%d", pTrans->id, terrstr(), pTrans->failedTimes); - continueExec = false; - } + pTrans->failedTimes++; + mError("trans:%d, stage keep on commit since %s, failedTimes:%d", pTrans->id, terrstr(), pTrans->failedTimes); + continueExec = false; } return continueExec; @@ -1235,11 +1251,9 @@ static bool mndTransPerformUndoActionStage(SMnode *pMnode, STrans *pTrans) { bool continueExec = true; int32_t code = mndTransExecuteUndoActions(pMnode, pTrans); - if (mndCantExecuteTransAction(pMnode)) return false; - if (code == 0) { - pTrans->stage = TRN_STAGE_ROLLBACK; - mDebug("trans:%d, stage from undoAction to rollback", pTrans->id); + pTrans->stage = TRN_STAGE_FINISHED; + mDebug("trans:%d, stage from undoAction to finished", pTrans->id); continueExec = true; } else if (code == TSDB_CODE_ACTION_IN_PROGRESS) { mDebug("trans:%d, stage keep on undoAction since %s", pTrans->id, tstrerror(code)); @@ -1254,14 +1268,14 @@ static bool mndTransPerformUndoActionStage(SMnode *pMnode, STrans *pTrans) { } static bool mndTransPerformRollbackStage(SMnode *pMnode, STrans *pTrans) { - if (mndCantExecuteTransAction(pMnode)) return false; + if (mndCannotExecuteTransAction(pMnode)) return false; bool continueExec = true; int32_t code = mndTransRollback(pMnode, pTrans); if (code == 0) { - pTrans->stage = TRN_STAGE_FINISHED; - mDebug("trans:%d, stage from rollback to finished", pTrans->id); + pTrans->stage = TRN_STAGE_UNDO_ACTION; + mDebug("trans:%d, stage from rollback to undoAction", pTrans->id); continueExec = true; } else { pTrans->failedTimes++; @@ -1309,12 +1323,12 @@ void mndTransExecute(SMnode *pMnode, STrans *pTrans) { case TRN_STAGE_COMMIT_ACTION: continueExec = mndTransPerformCommitActionStage(pMnode, pTrans); break; - case TRN_STAGE_UNDO_ACTION: - continueExec = mndTransPerformUndoActionStage(pMnode, pTrans); - break; case TRN_STAGE_ROLLBACK: continueExec = mndTransPerformRollbackStage(pMnode, pTrans); break; + case TRN_STAGE_UNDO_ACTION: + continueExec = mndTransPerformUndoActionStage(pMnode, pTrans); + break; case TRN_STAGE_FINISHED: continueExec = mndTransPerfromFinishedStage(pMnode, pTrans); break; @@ -1416,13 +1430,8 @@ void mndTransPullup(SMnode *pMnode) { mndReleaseTrans(pMnode, pTrans); } - SSnapshotMeta sMeta = {0}; - // if (syncGetSnapshotMeta(pMnode->syncMgmt.sync, &sMeta) == 0) { - SyncIndex snapshotIndex = sdbGetApplyIndex(pMnode->pSdb); - if (syncGetSnapshotMetaByIndex(pMnode->syncMgmt.sync, snapshotIndex, &sMeta) == 0) { - sdbSetCurConfig(pMnode->pSdb, sMeta.lastConfigIndex); - } - sdbWriteFile(pMnode->pSdb); + // todo, set to SDB_WRITE_DELTA + sdbWriteFile(pMnode->pSdb, 0); taosArrayDestroy(pArray); } @@ -1451,10 +1460,15 @@ static int32_t mndRetrieveTrans(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBl pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); colDataAppend(pColInfo, numOfRows, (const char *)stage, false); - char dbname[TSDB_DB_NAME_LEN + VARSTR_HEADER_SIZE] = {0}; - STR_WITH_MAXSIZE_TO_VARSTR(dbname, mndGetDbStr(pTrans->dbname), pShow->pMeta->pSchemas[cols].bytes); + char dbname1[TSDB_DB_NAME_LEN + VARSTR_HEADER_SIZE] = {0}; + STR_WITH_MAXSIZE_TO_VARSTR(dbname1, mndGetDbStr(pTrans->dbname1), pShow->pMeta->pSchemas[cols].bytes); pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); - colDataAppend(pColInfo, numOfRows, (const char *)dbname, false); + colDataAppend(pColInfo, numOfRows, (const char *)dbname1, false); + + char dbname2[TSDB_DB_NAME_LEN + VARSTR_HEADER_SIZE] = {0}; + STR_WITH_MAXSIZE_TO_VARSTR(dbname2, mndGetDbStr(pTrans->dbname2), pShow->pMeta->pSchemas[cols].bytes); + pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, numOfRows, (const char *)dbname2, false); pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); colDataAppend(pColInfo, numOfRows, (const char *)&pTrans->failedTimes, false); diff --git a/source/dnode/mnode/impl/src/mndVgroup.c b/source/dnode/mnode/impl/src/mndVgroup.c index cec83d1af5..94ddbcd409 100644 --- a/source/dnode/mnode/impl/src/mndVgroup.c +++ b/source/dnode/mnode/impl/src/mndVgroup.c @@ -1058,7 +1058,7 @@ int32_t mndSetMoveVgroupsInfoToTrans(SMnode *pMnode, STrans *pTrans, int32_t del static int32_t mndAddIncVgroupReplicaToTrans(SMnode *pMnode, STrans *pTrans, SDbObj *pDb, SVgObj *pVgroup, int32_t newDnodeId) { - mDebug("vgId:%d, will add 1 vnode, replica:%d, dnode:%d", pVgroup->vgId, pVgroup->replica, newDnodeId); + mDebug("vgId:%d, will add 1 vnode, replica:%d dnode:%d", pVgroup->vgId, pVgroup->replica, newDnodeId); SVnodeGid *pGid = &pVgroup->vnodeGid[pVgroup->replica]; pVgroup->replica++; @@ -1074,7 +1074,7 @@ static int32_t mndAddIncVgroupReplicaToTrans(SMnode *pMnode, STrans *pTrans, SDb static int32_t mndAddDecVgroupReplicaFromTrans(SMnode *pMnode, STrans *pTrans, SDbObj *pDb, SVgObj *pVgroup, int32_t delDnodeId) { - mDebug("vgId:%d, will remove 1 vnode, replica:%d, dnode:%d", pVgroup->vgId, pVgroup->replica, delDnodeId); + mDebug("vgId:%d, will remove 1 vnode, replica:%d dnode:%d", pVgroup->vgId, pVgroup->replica, delDnodeId); SVnodeGid *pGid = NULL; SVnodeGid delGid = {0}; @@ -1116,7 +1116,8 @@ static int32_t mndRedistributeVgroup(SMnode *pMnode, SRpcMsg *pReq, SDbObj *pDb, memcpy(&newVg, pVgroup, sizeof(SVgObj)); mInfo("vgId:%d, vgroup info before redistribute, replica:%d", newVg.vgId, newVg.replica); for (int32_t i = 0; i < newVg.replica; ++i) { - mInfo("vgId:%d, vnode:%d dnode:%d", newVg.vgId, i, newVg.vnodeGid[i].dnodeId); + mInfo("vgId:%d, vnode:%d dnode:%d role:%s", newVg.vgId, i, newVg.vnodeGid[i].dnodeId, + syncStr(newVg.vnodeGid[i].role)); } if (pNew1 != NULL && pOld1 != NULL) { @@ -1198,7 +1199,7 @@ static int32_t mndProcessRedistributeVgroupMsg(SRpcMsg *pReq) { goto _OVER; } - mInfo("vgId:%d, start to redistribute to dnode %d:%d:%d", req.vgId, req.dnodeId1, req.dnodeId2, req.dnodeId3); + mInfo("vgId:%d, start to redistribute vgroup to dnode %d:%d:%d", req.vgId, req.dnodeId1, req.dnodeId2, req.dnodeId3); if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_REDISTRIBUTE_VGROUP) != 0) goto _OVER; diff --git a/source/dnode/mnode/impl/test/sdb/sdbTest.cpp b/source/dnode/mnode/impl/test/sdb/sdbTest.cpp index 43be55dd1d..bc118ee26e 100644 --- a/source/dnode/mnode/impl/test/sdb/sdbTest.cpp +++ b/source/dnode/mnode/impl/test/sdb/sdbTest.cpp @@ -493,8 +493,11 @@ TEST_F(MndTestSdb, 01_Write_Str) { ASSERT_EQ(sdbGetSize(pSdb, SDB_USER), 2); ASSERT_EQ(sdbGetMaxId(pSdb, SDB_USER), -1); ASSERT_EQ(sdbGetTableVer(pSdb, SDB_USER), 2); - sdbSetApplyIndex(pSdb, -1); - ASSERT_EQ(sdbGetApplyIndex(pSdb), -1); + sdbSetApplyInfo(pSdb, -1, -1, -1); + // int64_t index, config; + // int64_t term; + // sdbGetCommitInfo(pSdb, &index, &term, &config); + // ASSERT_EQ(index, -1); ASSERT_EQ(mnode.insertTimes, 2); ASSERT_EQ(mnode.deleteTimes, 0); @@ -700,11 +703,12 @@ TEST_F(MndTestSdb, 01_Write_Str) { } // write version - sdbSetApplyIndex(pSdb, 0); - sdbSetApplyIndex(pSdb, 1); - ASSERT_EQ(sdbGetApplyIndex(pSdb), 1); - ASSERT_EQ(sdbWriteFile(pSdb), 0); - ASSERT_EQ(sdbWriteFile(pSdb), 0); + sdbSetApplyInfo(pSdb, 0, 0, 0); + sdbSetApplyInfo(pSdb, 1, 0, 0); + // sdbGetApplyInfo(pSdb, &index, &term, &config); + // ASSERT_EQ(index, 1); + ASSERT_EQ(sdbWriteFile(pSdb, 0), 0); + ASSERT_EQ(sdbWriteFile(pSdb, 0), 0); sdbCleanup(pSdb); ASSERT_EQ(mnode.insertTimes, 7); @@ -772,7 +776,11 @@ TEST_F(MndTestSdb, 01_Read_Str) { ASSERT_EQ(sdbGetSize(pSdb, SDB_USER), 2); ASSERT_EQ(sdbGetMaxId(pSdb, SDB_USER), -1); ASSERT_EQ(sdbGetTableVer(pSdb, SDB_USER), 5); - ASSERT_EQ(sdbGetApplyIndex(pSdb), 1); + + int64_t index, config; + int64_t term; + sdbGetCommitInfo(pSdb, &index, &term, &config); + ASSERT_EQ(index, 1); ASSERT_EQ(mnode.insertTimes, 4); ASSERT_EQ(mnode.deleteTimes, 0); diff --git a/source/dnode/mnode/impl/test/trans/trans2.cpp b/source/dnode/mnode/impl/test/trans/trans2.cpp index 022c82c73d..aee8aa2748 100644 --- a/source/dnode/mnode/impl/test/trans/trans2.cpp +++ b/source/dnode/mnode/impl/test/trans/trans2.cpp @@ -128,7 +128,7 @@ class MndTestTrans2 : public ::testing::Test { mndTransSetCb(pTrans, TRANS_START_FUNC_TEST, TRANS_STOP_FUNC_TEST, param, strlen(param) + 1); if (pDb != NULL) { - mndTransSetDbName(pTrans, pDb->name); + mndTransSetDbName(pTrans, pDb->name, NULL); } int32_t code = mndTransPrepare(pMnode, pTrans); @@ -201,7 +201,7 @@ class MndTestTrans2 : public ::testing::Test { } if (pDb != NULL) { - mndTransSetDbName(pTrans, pDb->name); + mndTransSetDbName(pTrans, pDb->name, NULL); } int32_t code = mndTransPrepare(pMnode, pTrans); diff --git a/source/dnode/mnode/sdb/inc/sdb.h b/source/dnode/mnode/sdb/inc/sdb.h index 8536c451b7..1bd09aef63 100644 --- a/source/dnode/mnode/sdb/inc/sdb.h +++ b/source/dnode/mnode/sdb/inc/sdb.h @@ -37,6 +37,8 @@ extern "C" { #define mTrace(...) { if (mDebugFlag & DEBUG_TRACE) { taosPrintLog("MND ", DEBUG_TRACE, mDebugFlag, __VA_ARGS__); }} // clang-format on +#define SDB_WRITE_DELTA 100 + #define SDB_GET_VAL(pData, dataPos, val, pos, func, type) \ { \ if (func(pRaw, dataPos, val) != 0) { \ @@ -169,11 +171,12 @@ typedef struct SSdb { SWal *pWal; char *currDir; char *tmpDir; - int64_t lastCommitVer; - int64_t lastCommitTerm; - int64_t curVer; - int64_t curTerm; - int64_t curConfig; + int64_t commitIndex; + int64_t commitTerm; + int64_t commitConfig; + int64_t applyIndex; + int64_t applyTerm; + int64_t applyConfig; int64_t tableVer[SDB_MAX]; int64_t maxId[SDB_MAX]; EKeyType keyTypes[SDB_MAX]; @@ -257,7 +260,7 @@ int32_t sdbReadFile(SSdb *pSdb); * @param pSdb The sdb object. * @return int32_t 0 for success, -1 for failure. */ -int32_t sdbWriteFile(SSdb *pSdb); +int32_t sdbWriteFile(SSdb *pSdb, int32_t delta); /** * @brief Parse and write raw data to sdb, then free the pRaw object @@ -361,14 +364,8 @@ int64_t sdbGetTableVer(SSdb *pSdb, ESdbType type); * @param index The update value of the apply index. * @return int32_t The current index of sdb */ -void sdbSetApplyIndex(SSdb *pSdb, int64_t index); -void sdbSetApplyTerm(SSdb *pSdb, int64_t term); -void sdbSetCurConfig(SSdb *pSdb, int64_t config); -int64_t sdbGetApplyIndex(SSdb *pSdb); -int64_t sdbGetApplyTerm(SSdb *pSdb); -int64_t sdbGetCommitIndex(SSdb *pSdb); -int64_t sdbGetCommitTerm(SSdb *pSdb); -int64_t sdbGetCurConfig(SSdb *pSdb); +void sdbSetApplyInfo(SSdb *pSdb, int64_t index, int64_t term, int64_t config); +void sdbGetCommitInfo(SSdb *pSdb, int64_t *index, int64_t *term, int64_t *config); SSdbRaw *sdbAllocRaw(ESdbType type, int8_t sver, int32_t dataLen); void sdbFreeRaw(SSdbRaw *pRaw); diff --git a/source/dnode/mnode/sdb/src/sdb.c b/source/dnode/mnode/sdb/src/sdb.c index 39e9c75888..c44f1670c3 100644 --- a/source/dnode/mnode/sdb/src/sdb.c +++ b/source/dnode/mnode/sdb/src/sdb.c @@ -53,11 +53,12 @@ SSdb *sdbInit(SSdbOpt *pOption) { } pSdb->pWal = pOption->pWal; - pSdb->curVer = -1; - pSdb->curTerm = -1; - pSdb->lastCommitVer = -1; - pSdb->lastCommitTerm = -1; - pSdb->curConfig = -1; + pSdb->applyIndex = -1; + pSdb->applyTerm = -1; + pSdb->applyConfig = -1; + pSdb->commitIndex = -1; + pSdb->commitTerm = -1; + pSdb->commitConfig = -1; pSdb->pMnode = pOption->pMnode; taosThreadMutexInit(&pSdb->filelock, NULL); mDebug("sdb init successfully"); @@ -67,7 +68,7 @@ SSdb *sdbInit(SSdbOpt *pOption) { void sdbCleanup(SSdb *pSdb) { mDebug("start to cleanup sdb"); - sdbWriteFile(pSdb); + sdbWriteFile(pSdb, 0); if (pSdb->currDir != NULL) { taosMemoryFreeClear(pSdb->currDir); @@ -159,23 +160,20 @@ static int32_t sdbCreateDir(SSdb *pSdb) { return 0; } -void sdbSetApplyIndex(SSdb *pSdb, int64_t index) { pSdb->curVer = index; } - -void sdbSetApplyTerm(SSdb *pSdb, int64_t term) { pSdb->curTerm = term; } - -void sdbSetCurConfig(SSdb *pSdb, int64_t config) { - if (pSdb->curConfig != config) { - mDebug("mnode sync config set from %" PRId64 " to %" PRId64, pSdb->curConfig, config); - pSdb->curConfig = config; - } +void sdbSetApplyInfo(SSdb *pSdb, int64_t index, int64_t term, int64_t config) { + mTrace("mnode apply info changed, from index:%" PRId64 " term:%" PRId64 " config:%" PRId64 ", to index:%" PRId64 + " term:%" PRId64 " config:%" PRId64, + pSdb->applyIndex, pSdb->applyTerm, pSdb->applyConfig, index, term, config); + pSdb->applyIndex = index; + pSdb->applyTerm = term; + pSdb->applyConfig = config; } -int64_t sdbGetApplyIndex(SSdb *pSdb) { return pSdb->curVer; } - -int64_t sdbGetApplyTerm(SSdb *pSdb) { return pSdb->curTerm; } - -int64_t sdbGetCommitIndex(SSdb *pSdb) { return pSdb->lastCommitVer; } - -int64_t sdbGetCommitTerm(SSdb *pSdb) { return pSdb->lastCommitTerm; } - -int64_t sdbGetCurConfig(SSdb *pSdb) { return pSdb->curConfig; } \ No newline at end of file +void sdbGetCommitInfo(SSdb *pSdb, int64_t *index, int64_t *term, int64_t *config) { + *index = pSdb->commitIndex; + *term = pSdb->commitTerm; + *config = pSdb->commitConfig; + mTrace("mnode current info, apply index:%" PRId64 " term:%" PRId64 " config:%" PRId64 ", commit index:%" PRId64 + " term:%" PRId64 " config:%" PRId64, + pSdb->applyIndex, pSdb->applyTerm, pSdb->applyConfig, *index, *term, *config); +} diff --git a/source/dnode/mnode/sdb/src/sdbFile.c b/source/dnode/mnode/sdb/src/sdbFile.c index 34f5d6f23d..0f4e1276c1 100644 --- a/source/dnode/mnode/sdb/src/sdbFile.c +++ b/source/dnode/mnode/sdb/src/sdbFile.c @@ -67,10 +67,12 @@ static void sdbResetData(SSdb *pSdb) { mDebug("sdb:%s is reset", sdbTableName(i)); } - pSdb->curVer = -1; - pSdb->curTerm = -1; - pSdb->lastCommitVer = -1; - pSdb->lastCommitTerm = -1; + pSdb->applyIndex = -1; + pSdb->applyTerm = -1; + pSdb->applyConfig = -1; + pSdb->commitIndex = -1; + pSdb->commitTerm = -1; + pSdb->commitConfig = -1; mDebug("sdb reset successfully"); } @@ -90,7 +92,7 @@ static int32_t sdbReadFileHead(SSdb *pSdb, TdFilePtr pFile) { return -1; } - ret = taosReadFile(pFile, &pSdb->curVer, sizeof(int64_t)); + ret = taosReadFile(pFile, &pSdb->applyIndex, sizeof(int64_t)); if (ret < 0) { terrno = TAOS_SYSTEM_ERROR(errno); return -1; @@ -100,7 +102,7 @@ static int32_t sdbReadFileHead(SSdb *pSdb, TdFilePtr pFile) { return -1; } - ret = taosReadFile(pFile, &pSdb->curTerm, sizeof(int64_t)); + ret = taosReadFile(pFile, &pSdb->applyTerm, sizeof(int64_t)); if (ret < 0) { terrno = TAOS_SYSTEM_ERROR(errno); return -1; @@ -110,7 +112,7 @@ static int32_t sdbReadFileHead(SSdb *pSdb, TdFilePtr pFile) { return -1; } - ret = taosReadFile(pFile, &pSdb->curConfig, sizeof(int64_t)); + ret = taosReadFile(pFile, &pSdb->applyConfig, sizeof(int64_t)); if (ret < 0) { terrno = TAOS_SYSTEM_ERROR(errno); return -1; @@ -173,17 +175,17 @@ static int32_t sdbWriteFileHead(SSdb *pSdb, TdFilePtr pFile) { return -1; } - if (taosWriteFile(pFile, &pSdb->curVer, sizeof(int64_t)) != sizeof(int64_t)) { + if (taosWriteFile(pFile, &pSdb->applyIndex, sizeof(int64_t)) != sizeof(int64_t)) { terrno = TAOS_SYSTEM_ERROR(errno); return -1; } - if (taosWriteFile(pFile, &pSdb->curTerm, sizeof(int64_t)) != sizeof(int64_t)) { + if (taosWriteFile(pFile, &pSdb->applyTerm, sizeof(int64_t)) != sizeof(int64_t)) { terrno = TAOS_SYSTEM_ERROR(errno); return -1; } - if (taosWriteFile(pFile, &pSdb->curConfig, sizeof(int64_t)) != sizeof(int64_t)) { + if (taosWriteFile(pFile, &pSdb->applyConfig, sizeof(int64_t)) != sizeof(int64_t)) { terrno = TAOS_SYSTEM_ERROR(errno); return -1; } @@ -300,11 +302,12 @@ static int32_t sdbReadFileImp(SSdb *pSdb) { } code = 0; - pSdb->lastCommitVer = pSdb->curVer; - pSdb->lastCommitTerm = pSdb->curTerm; + pSdb->commitIndex = pSdb->applyIndex; + pSdb->commitTerm = pSdb->applyTerm; + pSdb->commitConfig = pSdb->applyConfig; memcpy(pSdb->tableVer, tableVer, sizeof(tableVer)); - mDebug("read sdb file:%s successfully, index:%" PRId64 " term:%" PRId64 " config:%" PRId64, file, pSdb->lastCommitVer, - pSdb->lastCommitTerm, pSdb->curConfig); + mDebug("read sdb file:%s successfully, commit index:%" PRId64 " term:%" PRId64 " config:%" PRId64, file, + pSdb->commitIndex, pSdb->commitTerm, pSdb->commitConfig); _OVER: taosCloseFile(&pFile); @@ -336,9 +339,10 @@ static int32_t sdbWriteFileImp(SSdb *pSdb) { char curfile[PATH_MAX] = {0}; snprintf(curfile, sizeof(curfile), "%s%ssdb.data", pSdb->currDir, TD_DIRSEP); - mDebug("start to write sdb file, current ver:%" PRId64 " term:%" PRId64 ", commit ver:%" PRId64 " term:%" PRId64 - " file:%s", - pSdb->curVer, pSdb->curTerm, pSdb->lastCommitVer, pSdb->lastCommitTerm, curfile); + mDebug("start to write sdb file, apply index:%" PRId64 " term:%" PRId64 " config:%" PRId64 ", commit index:%" PRId64 + " term:%" PRId64 " config:%" PRId64 ", file:%s", + pSdb->applyIndex, pSdb->applyTerm, pSdb->applyConfig, pSdb->commitIndex, pSdb->commitTerm, pSdb->commitConfig, + curfile); TdFilePtr pFile = taosOpenFile(tmpfile, TD_FILE_CREATE | TD_FILE_WRITE | TD_FILE_TRUNC); if (pFile == NULL) { @@ -430,25 +434,30 @@ static int32_t sdbWriteFileImp(SSdb *pSdb) { if (code != 0) { mError("failed to write sdb file:%s since %s", curfile, tstrerror(code)); } else { - pSdb->lastCommitVer = pSdb->curVer; - pSdb->lastCommitTerm = pSdb->curTerm; - mDebug("write sdb file successfully, index:%" PRId64 " term:%" PRId64 " config:%" PRId64 " file:%s", - pSdb->lastCommitVer, pSdb->lastCommitTerm, pSdb->curConfig, curfile); + pSdb->commitIndex = pSdb->applyIndex; + pSdb->commitTerm = pSdb->applyTerm; + pSdb->commitConfig = pSdb->applyConfig; + mDebug("write sdb file successfully, commit index:%" PRId64 " term:%" PRId64 " config:%" PRId64 " file:%s", + pSdb->commitIndex, pSdb->commitTerm, pSdb->commitConfig, curfile); } terrno = code; return code; } -int32_t sdbWriteFile(SSdb *pSdb) { +int32_t sdbWriteFile(SSdb *pSdb, int32_t delta) { int32_t code = 0; - if (pSdb->curVer == pSdb->lastCommitVer) { + if (pSdb->applyIndex == pSdb->commitIndex) { + return 0; + } + + if (pSdb->applyIndex - pSdb->commitIndex < delta) { return 0; } taosThreadMutexLock(&pSdb->filelock); if (pSdb->pWal != NULL) { - code = walBeginSnapshot(pSdb->pWal, pSdb->curVer); + code = walBeginSnapshot(pSdb->pWal, pSdb->applyIndex); } if (code == 0) { code = sdbWriteFileImp(pSdb); @@ -470,7 +479,7 @@ int32_t sdbDeploy(SSdb *pSdb) { return -1; } - if (sdbWriteFile(pSdb) != 0) { + if (sdbWriteFile(pSdb, 0) != 0) { return -1; } @@ -522,9 +531,9 @@ int32_t sdbStartRead(SSdb *pSdb, SSdbIter **ppIter) { snprintf(datafile, sizeof(datafile), "%s%ssdb.data", pSdb->currDir, TD_DIRSEP); taosThreadMutexLock(&pSdb->filelock); - int64_t commitIndex = pSdb->lastCommitVer; - int64_t commitTerm = pSdb->lastCommitTerm; - int64_t curConfig = pSdb->curConfig; + int64_t commitIndex = pSdb->commitIndex; + int64_t commitTerm = pSdb->commitTerm; + int64_t commitConfig = pSdb->commitConfig; if (taosCopyFile(datafile, pIter->name) < 0) { taosThreadMutexUnlock(&pSdb->filelock); terrno = TAOS_SYSTEM_ERROR(errno); @@ -543,8 +552,8 @@ int32_t sdbStartRead(SSdb *pSdb, SSdbIter **ppIter) { } *ppIter = pIter; - mInfo("sdbiter:%p, is created to read snapshot, index:%" PRId64 " term:%" PRId64 " config:%" PRId64 " file:%s", pIter, - commitIndex, commitTerm, curConfig, pIter->name); + mInfo("sdbiter:%p, is created to read snapshot, commit index:%" PRId64 " term:%" PRId64 " config:%" PRId64 " file:%s", + pIter, commitIndex, commitTerm, commitConfig, pIter->name); return 0; } diff --git a/source/dnode/vnode/src/inc/vnd.h b/source/dnode/vnode/src/inc/vnd.h index b0599b82ef..9339402d43 100644 --- a/source/dnode/vnode/src/inc/vnd.h +++ b/source/dnode/vnode/src/inc/vnd.h @@ -18,6 +18,7 @@ #include "sync.h" #include "syncTools.h" +#include "ttrace.h" #include "vnodeInt.h" #ifdef __cplusplus @@ -31,6 +32,7 @@ extern "C" { #define vInfo(...) do { if (vDebugFlag & DEBUG_INFO) { taosPrintLog("VND ", DEBUG_INFO, 255, __VA_ARGS__); }} while(0) #define vDebug(...) do { if (vDebugFlag & DEBUG_DEBUG) { taosPrintLog("VND ", DEBUG_DEBUG, vDebugFlag, __VA_ARGS__); }} while(0) #define vTrace(...) do { if (vDebugFlag & DEBUG_TRACE) { taosPrintLog("VND ", DEBUG_TRACE, vDebugFlag, __VA_ARGS__); }} while(0) +#define vGTrace(param, ...) do { char buf[40] = {0}; TRACE_TO_STR(trace, buf); vTrace(param " GTID: %s", __VA_ARGS__, buf);} while(0)//#define vDye(...) do // clang-format on // vnodeCfg.c @@ -89,4 +91,4 @@ void vnodeSyncClose(SVnode* pVnode); } #endif -#endif /*_TD_VND_H_*/ \ No newline at end of file +#endif /*_TD_VND_H_*/ diff --git a/source/dnode/vnode/src/meta/metaQuery.c b/source/dnode/vnode/src/meta/metaQuery.c index a6339125c4..8cfebe1b98 100644 --- a/source/dnode/vnode/src/meta/metaQuery.c +++ b/source/dnode/vnode/src/meta/metaQuery.c @@ -628,7 +628,7 @@ int32_t metaFilteTableIds(SMeta *pMeta, SMetaFltParam *param, SArray *pUids) { void * tagData = NULL; if (param->val == NULL) { - metaError("vgId:%d failed to filter NULL data", TD_VID(pMeta->pVnode)); + metaError("vgId:%d, failed to filter NULL data", TD_VID(pMeta->pVnode)); return -1; } else { if (IS_VAR_DATA_TYPE(param->type)) { diff --git a/source/dnode/vnode/src/tq/tq.c b/source/dnode/vnode/src/tq/tq.c index c9d3267adb..70b09ec701 100644 --- a/source/dnode/vnode/src/tq/tq.c +++ b/source/dnode/vnode/src/tq/tq.c @@ -124,17 +124,20 @@ int32_t tqProcessOffsetCommitReq(STQ* pTq, char* msg, int32_t msgLen) { tDecoderClear(&decoder); if (offset.type == TMQ_OFFSET__SNAPSHOT) { - tqDebug("receive offset commit msg to %s, offset(type:snapshot) uid: %ld, ts: %ld", offset.subKey, offset.uid, - offset.ts); + tqDebug("receive offset commit msg to %s on vg %d, offset(type:snapshot) uid: %ld, ts: %ld", offset.subKey, + pTq->pVnode->config.vgId, offset.uid, offset.ts); } else if (offset.type == TMQ_OFFSET__LOG) { - tqDebug("receive offset commit msg to %s, offset(type:log) version: %ld", offset.subKey, offset.version); + tqDebug("receive offset commit msg to %s on vg %d, offset(type:log) version: %ld", offset.subKey, + pTq->pVnode->config.vgId, offset.version); } else { ASSERT(0); } - - if (tqOffsetWrite(pTq->pOffsetStore, &offset) < 0) { - ASSERT(0); - return -1; + STqOffset* pOffset = tqOffsetRead(pTq->pOffsetStore, offset.subKey); + if (pOffset == NULL || pOffset->version < offset.version) { + if (tqOffsetWrite(pTq->pOffsetStore, &offset) < 0) { + ASSERT(0); + return -1; + } } return 0; @@ -149,16 +152,33 @@ int32_t tqProcessPollReq(STQ* pTq, SRpcMsg* pMsg, int32_t workerId) { int32_t code = 0; // get offset to fetch message - if (pReq->currentOffset == TMQ_CONF__RESET_OFFSET__EARLIEAST) { - fetchOffset = walGetFirstVer(pTq->pWal); - } else if (pReq->currentOffset == TMQ_CONF__RESET_OFFSET__LATEST) { - fetchOffset = walGetCommittedVer(pTq->pWal); - } else { + if (pReq->currentOffset >= 0) { fetchOffset = pReq->currentOffset + 1; + } else { + STqOffset* pOffset = tqOffsetRead(pTq->pOffsetStore, pReq->subKey); + if (pOffset != NULL) { + ASSERT(pOffset->type == TMQ_OFFSET__LOG); + tqDebug("consumer %ld, restore offset of %s on vg %d, offset(type:log) version: %ld", consumerId, pReq->subKey, + pTq->pVnode->config.vgId, pOffset->version); + fetchOffset = pOffset->version + 1; + } else { + if (pReq->currentOffset == TMQ_CONF__RESET_OFFSET__EARLIEAST) { + fetchOffset = walGetFirstVer(pTq->pWal); + } else if (pReq->currentOffset == TMQ_CONF__RESET_OFFSET__LATEST) { + fetchOffset = walGetCommittedVer(pTq->pWal); + } else if (pReq->currentOffset == TMQ_CONF__RESET_OFFSET__NONE) { + tqError("tmq poll: no offset committed for consumer %ld in vg %d, subkey %s", consumerId, + pTq->pVnode->config.vgId, pReq->subKey); + terrno = TSDB_CODE_TQ_NO_COMMITTED_OFFSET; + return -1; + } + tqDebug("consumer %ld, restore offset of %s on vg %d failed, config is %ld, set to %ld", consumerId, pReq->subKey, + pTq->pVnode->config.vgId, pReq->currentOffset, fetchOffset); + } } - tqDebug("tmq poll: consumer %ld (epoch %d) recv poll req in vg %d, req %ld %ld", consumerId, pReq->epoch, - TD_VID(pTq->pVnode), pReq->currentOffset, fetchOffset); + tqDebug("tmq poll: consumer %ld (epoch %d) recv poll req in vg %d, req offset %ld fetch offset %ld", consumerId, + pReq->epoch, TD_VID(pTq->pVnode), pReq->currentOffset, fetchOffset); STqHandle* pHandle = taosHashGet(pTq->handles, pReq->subKey, strlen(pReq->subKey)); /*ASSERT(pHandle);*/ diff --git a/source/dnode/vnode/src/tq/tqCommit.c b/source/dnode/vnode/src/tq/tqCommit.c index 7b116bff2e..639da22b1c 100644 --- a/source/dnode/vnode/src/tq/tqCommit.c +++ b/source/dnode/vnode/src/tq/tqCommit.c @@ -15,7 +15,4 @@ #include "tq.h" -int tqCommit(STQ* pTq) { - // do nothing - return 0; -} +int tqCommit(STQ* pTq) { return tqOffsetSnapshot(pTq->pOffsetStore); } diff --git a/source/dnode/vnode/src/tq/tqOffset.c b/source/dnode/vnode/src/tq/tqOffset.c index 8d6cb28065..e5475d7c30 100644 --- a/source/dnode/vnode/src/tq/tqOffset.c +++ b/source/dnode/vnode/src/tq/tqOffset.c @@ -92,6 +92,8 @@ STqOffset* tqOffsetRead(STqOffsetStore* pStore, const char* subscribeKey) { } int32_t tqOffsetWrite(STqOffsetStore* pStore, const STqOffset* pOffset) { + ASSERT(pOffset->type == TMQ_OFFSET__LOG); + ASSERT(pOffset->version >= 0); return taosHashPut(pStore->pHash, pOffset->subKey, strlen(pOffset->subKey), pOffset, sizeof(STqOffset)); } @@ -129,7 +131,7 @@ int32_t tqOffsetSnapshot(STqOffsetStore* pStore) { tEncodeSTqOffset(&encoder, pOffset); // write file int64_t writeLen; - if ((writeLen = taosWriteFile(pFile, buf, totLen)) != bodyLen) { + if ((writeLen = taosWriteFile(pFile, buf, totLen)) != totLen) { ASSERT(0); tqError("write offset incomplete, len %d, write len %ld", bodyLen, writeLen); taosHashCancelIterate(pStore->pHash, pIter); diff --git a/source/dnode/vnode/src/tsdb/tsdbCommit.c b/source/dnode/vnode/src/tsdb/tsdbCommit.c index fe89321ae9..e98fd8ae1f 100644 --- a/source/dnode/vnode/src/tsdb/tsdbCommit.c +++ b/source/dnode/vnode/src/tsdb/tsdbCommit.c @@ -111,7 +111,7 @@ int32_t tsdbBegin(STsdb *pTsdb) { int32_t tsdbCommit(STsdb *pTsdb) { if (!pTsdb) return 0; - + int32_t code = 0; SCommitH commith = {0}; SDFileSet *pSet = NULL; @@ -495,7 +495,9 @@ static int32_t tsdbCommitToFile(SCommitH *pCommith, SDFileSet *pSet, int fid) { break; } - if (pIter && pIter->pTable && (!pIdx || (pIter->pTable->suid <= pIdx->suid || pIter->pTable->uid <= pIdx->uid))) { + if (pIter && pIter->pTable && + (!pIdx || ((pIter->pTable->suid < pIdx->suid) || + ((pIter->pTable->suid == pIdx->suid) && (pIter->pTable->uid <= pIdx->uid))))) { if (tsdbCommitToTable(pCommith, mIter) < 0) { tsdbCloseCommitFile(pCommith, true); // revert the file change @@ -503,7 +505,7 @@ static int32_t tsdbCommitToFile(SCommitH *pCommith, SDFileSet *pSet, int fid) { return -1; } - if (pIdx && (pIter->pTable->uid == pIdx->uid)) { + if (pIdx && ((pIter->pTable->uid == pIdx->uid) && (pIter->pTable->suid == pIdx->suid))) { ++fIter; } ++mIter; @@ -518,6 +520,8 @@ static int32_t tsdbCommitToFile(SCommitH *pCommith, SDFileSet *pSet, int fid) { return -1; } ++fIter; + } else { + ASSERT(0); } } diff --git a/source/dnode/vnode/src/vnd/vnodeSvr.c b/source/dnode/vnode/src/vnd/vnodeSvr.c index da1a126a76..feb8419bdf 100644 --- a/source/dnode/vnode/src/vnd/vnodeSvr.c +++ b/source/dnode/vnode/src/vnd/vnodeSvr.c @@ -774,9 +774,6 @@ static int32_t vnodeProcessCreateTSmaReq(SVnode *pVnode, int64_t version, void * goto _err; } - // record current timezone of server side - req.timezoneInt = tsTimezone; - if (tdProcessTSmaCreate(pVnode->pSma, version, (const char *)&req) < 0) { if (pRsp) pRsp->code = terrno; goto _err; diff --git a/source/dnode/vnode/src/vnd/vnodeSync.c b/source/dnode/vnode/src/vnd/vnodeSync.c index f46114746e..ae61aa454e 100644 --- a/source/dnode/vnode/src/vnd/vnodeSync.c +++ b/source/dnode/vnode/src/vnd/vnodeSync.c @@ -61,6 +61,7 @@ static int32_t vnodeSetStandBy(SVnode *pVnode) { return -1; } + vInfo("vgId:%d, start to transfer leader", TD_VID(pVnode)); if (syncLeaderTransfer(pVnode->sync) != 0) { vError("vgId:%d, failed to transfer leader since:%s", TD_VID(pVnode), terrstr()); return -1; @@ -83,8 +84,8 @@ static int32_t vnodeProcessAlterReplicaReq(SVnode *pVnode, SRpcMsg *pMsg) { terrno = TSDB_CODE_INVALID_MSG; return TSDB_CODE_INVALID_MSG; } - - vInfo("vgId:%d, start to alter vnode replica to %d, handle:%p", TD_VID(pVnode), req.replica, pMsg->info.handle); + STraceId *trace = &pMsg->info.traceId; + vGTrace("vgId:%d, start to alter vnode replica to %d, handle:%p", TD_VID(pVnode), req.replica, pMsg->info.handle); SSyncCfg cfg = {.replicaNum = req.replica, .myIndex = req.selfIndex}; for (int32_t r = 0; r < req.replica; ++r) { SNodeInfo *pNode = &cfg.nodeInfo[r]; @@ -118,14 +119,15 @@ static int32_t vnodeProcessAlterReplicaReq(SVnode *pVnode, SRpcMsg *pMsg) { } void vnodeProposeMsg(SQueueInfo *pInfo, STaosQall *qall, int32_t numOfMsgs) { - SVnode *pVnode = pInfo->ahandle; + SVnode * pVnode = pInfo->ahandle; int32_t vgId = pVnode->config.vgId; int32_t code = 0; SRpcMsg *pMsg = NULL; for (int32_t m = 0; m < numOfMsgs; m++) { if (taosGetQitem(qall, (void **)&pMsg) == 0) continue; - vTrace("vgId:%d, msg:%p get from vnode-write queue handle:%p", vgId, pMsg, pMsg->info.handle); + STraceId *trace = &pMsg->info.traceId; + vGTrace("vgId:%d, msg:%p get from vnode-write queue handle:%p", vgId, pMsg, pMsg->info.handle); if (pMsg->msgType == TDMT_VND_ALTER_REPLICA) { code = vnodeProcessAlterReplicaReq(pVnode, pMsg); @@ -148,10 +150,10 @@ void vnodeProposeMsg(SQueueInfo *pInfo, STaosQall *qall, int32_t numOfMsgs) { newEpSet.inUse = (newEpSet.inUse + 1) % newEpSet.numOfEps; } - vTrace("vgId:%d, msg:%p is redirect since not leader, numOfEps:%d inUse:%d", vgId, pMsg, newEpSet.numOfEps, - newEpSet.inUse); + vGTrace("vgId:%d, msg:%p is redirect since not leader, numOfEps:%d inUse:%d", vgId, pMsg, newEpSet.numOfEps, + newEpSet.inUse); for (int32_t i = 0; i < newEpSet.numOfEps; ++i) { - vTrace("vgId:%d, msg:%p redirect:%d ep:%s:%u", vgId, pMsg, i, newEpSet.eps[i].fqdn, newEpSet.eps[i].port); + vGTrace("vgId:%d, msg:%p redirect:%d ep:%s:%u", vgId, pMsg, i, newEpSet.eps[i].fqdn, newEpSet.eps[i].port); } SRpcMsg rsp = {.code = TSDB_CODE_RPC_REDIRECT, .info = pMsg->info}; @@ -163,7 +165,7 @@ void vnodeProposeMsg(SQueueInfo *pInfo, STaosQall *qall, int32_t numOfMsgs) { tmsgSendRsp(&rsp); } - vTrace("vgId:%d, msg:%p is freed, code:0x%x", vgId, pMsg, code); + vGTrace("vgId:%d, msg:%p is freed, code:0x%x", vgId, pMsg, code); rpcFreeCont(pMsg->pCont); taosFreeQitem(pMsg); } @@ -172,15 +174,16 @@ void vnodeProposeMsg(SQueueInfo *pInfo, STaosQall *qall, int32_t numOfMsgs) { } void vnodeApplyMsg(SQueueInfo *pInfo, STaosQall *qall, int32_t numOfMsgs) { - SVnode *pVnode = pInfo->ahandle; + SVnode * pVnode = pInfo->ahandle; int32_t vgId = pVnode->config.vgId; int32_t code = 0; SRpcMsg *pMsg = NULL; for (int32_t i = 0; i < numOfMsgs; ++i) { if (taosGetQitem(qall, (void **)&pMsg) == 0) continue; - vTrace("vgId:%d, msg:%p get from vnode-apply queue, type:%s handle:%p", vgId, pMsg, TMSG_INFO(pMsg->msgType), - pMsg->info.handle); + STraceId *trace = &pMsg->info.traceId; + vGTrace("vgId:%d, msg:%p get from vnode-apply queue, type:%s handle:%p", vgId, pMsg, TMSG_INFO(pMsg->msgType), + pMsg->info.handle); SRpcMsg rsp = {.code = pMsg->code, .info = pMsg->info}; if (rsp.code == 0) { @@ -195,7 +198,7 @@ void vnodeApplyMsg(SQueueInfo *pInfo, STaosQall *qall, int32_t numOfMsgs) { tmsgSendRsp(&rsp); } - vTrace("vgId:%d, msg:%p is freed, code:0x%x", vgId, pMsg, rsp.code); + vGTrace("vgId:%d, msg:%p is freed, code:0x%x", vgId, pMsg, rsp.code); rpcFreeCont(pMsg->pCont); taosFreeQitem(pMsg); } @@ -217,8 +220,9 @@ int32_t vnodeProcessSyncReq(SVnode *pVnode, SRpcMsg *pMsg, SRpcMsg **pRsp) { char *syncNodeStr = sync2SimpleStr(pVnode->sync); snprintf(logBuf, sizeof(logBuf), "==vnodeProcessSyncReq== msgType:%d, syncNode: %s", pMsg->msgType, syncNodeStr); static int64_t vndTick = 0; + STraceId * trace = &pMsg->info.traceId; if (++vndTick % 10 == 1) { - vTrace("sync trace msg:%s, %s", TMSG_INFO(pMsg->msgType), syncNodeStr); + vGTrace("sync trace msg:%s, %s", TMSG_INFO(pMsg->msgType), syncNodeStr); } syncRpcMsgLog2(logBuf, pMsg); taosMemoryFree(syncNodeStr); @@ -297,7 +301,7 @@ int32_t vnodeProcessSyncReq(SVnode *pVnode, SRpcMsg *pMsg, SRpcMsg **pRsp) { ret = -1; } - if (ret != 0) { + if (ret != 0 && terrno == 0) { terrno = TSDB_CODE_SYN_INTERNAL_ERROR; } return ret; @@ -333,8 +337,9 @@ static void vnodeSyncReconfig(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SReCon syncGetAndDelRespRpc(pVnode->sync, cbMeta.seqNum, &rpcMsg.info); rpcMsg.info.conn.applyIndex = cbMeta.index; - vInfo("vgId:%d, alter vnode replica is confirmed, type:%s contLen:%d seq:%" PRIu64 " handle:%p", TD_VID(pVnode), - TMSG_INFO(pMsg->msgType), pMsg->contLen, cbMeta.seqNum, rpcMsg.info.handle); + STraceId *trace = (STraceId *)&pMsg->info.traceId; + vGTrace("vgId:%d, alter vnode replica is confirmed, type:%s contLen:%d seq:%" PRIu64 " handle:%p", TD_VID(pVnode), + TMSG_INFO(pMsg->msgType), pMsg->contLen, cbMeta.seqNum, rpcMsg.info.handle); if (rpcMsg.info.handle != NULL) { tmsgSendRsp(&rpcMsg); } @@ -343,7 +348,7 @@ static void vnodeSyncReconfig(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SReCon } static void vnodeSyncCommitMsg(SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta cbMeta) { - SVnode *pVnode = pFsm->data; + SVnode * pVnode = pFsm->data; SSnapshot snapshot = {0}; SyncIndex beginIndex = SYNC_INDEX_INVALID; char logBuf[256] = {0}; diff --git a/source/libs/command/src/command.c b/source/libs/command/src/command.c index f06664b60b..c6a0e52a42 100644 --- a/source/libs/command/src/command.c +++ b/source/libs/command/src/command.c @@ -47,7 +47,8 @@ static SSDataBlock* buildDescResultDataBlock() { taosArrayPush(pBlock->pDataBlock, &infoData); infoData.info.type = TSDB_DATA_TYPE_INT; - infoData.info.bytes = tDataTypes[TSDB_DATA_TYPE_INT].bytes;; + infoData.info.bytes = tDataTypes[TSDB_DATA_TYPE_INT].bytes; + taosArrayPush(pBlock->pDataBlock, &infoData); infoData.info.type = TSDB_DATA_TYPE_VARCHAR; @@ -63,7 +64,7 @@ static void setDescResultIntoDataBlock(SSDataBlock* pBlock, int32_t numOfRows, S // field SColumnInfoData* pCol1 = taosArrayGet(pBlock->pDataBlock, 0); - char buf[DESCRIBE_RESULT_FIELD_LEN] = {0}; + char buf[DESCRIBE_RESULT_FIELD_LEN] = {0}; for (int32_t i = 0; i < numOfRows; ++i) { STR_TO_VARSTR(buf, pMeta->schema[i].name); colDataAppend(pCol1, i, buf, false); @@ -92,8 +93,8 @@ static void setDescResultIntoDataBlock(SSDataBlock* pBlock, int32_t numOfRows, S } static int32_t execDescribe(SNode* pStmt, SRetrieveTableRsp** pRsp) { - SDescribeStmt* pDesc = (SDescribeStmt*) pStmt; - int32_t numOfRows = TABLE_TOTAL_COL_NUM(pDesc->pMeta); + SDescribeStmt* pDesc = (SDescribeStmt*)pStmt; + int32_t numOfRows = TABLE_TOTAL_COL_NUM(pDesc->pMeta); SSDataBlock* pBlock = buildDescResultDataBlock(); setDescResultIntoDataBlock(pBlock, numOfRows, pDesc->pMeta); @@ -120,9 +121,15 @@ static int32_t execDescribe(SNode* pStmt, SRetrieveTableRsp** pRsp) { return TSDB_CODE_SUCCESS; } -static int32_t execResetQueryCache() { - return catalogClearCache(); -} +static int32_t execResetQueryCache() { return catalogClearCache(); } + +static int32_t execShowCreateDatabase(SShowCreateDatabaseStmt* pStmt) { return TSDB_CODE_FAILED; } + +static int32_t execShowCreateTable(SShowCreateTableStmt* pStmt) { return TSDB_CODE_FAILED; } + +static int32_t execShowCreateSTable(SShowCreateTableStmt* pStmt) { return TSDB_CODE_FAILED; } + +static int32_t execAlterLocal(SAlterLocalStmt* pStmt) { return TSDB_CODE_FAILED; } int32_t qExecCommand(SNode* pStmt, SRetrieveTableRsp** pRsp) { switch (nodeType(pStmt)) { @@ -130,6 +137,14 @@ int32_t qExecCommand(SNode* pStmt, SRetrieveTableRsp** pRsp) { return execDescribe(pStmt, pRsp); case QUERY_NODE_RESET_QUERY_CACHE_STMT: return execResetQueryCache(); + case QUERY_NODE_SHOW_CREATE_DATABASE_STMT: + return execShowCreateDatabase((SShowCreateDatabaseStmt*)pStmt); + case QUERY_NODE_SHOW_CREATE_TABLE_STMT: + return execShowCreateTable((SShowCreateTableStmt*)pStmt); + case QUERY_NODE_SHOW_CREATE_STABLE_STMT: + return execShowCreateSTable((SShowCreateTableStmt*)pStmt); + case QUERY_NODE_ALTER_LOCAL_STMT: + return execAlterLocal((SAlterLocalStmt*)pStmt); default: break; } diff --git a/source/libs/executor/inc/executil.h b/source/libs/executor/inc/executil.h index a70cff5552..07686893db 100644 --- a/source/libs/executor/inc/executil.h +++ b/source/libs/executor/inc/executil.h @@ -15,7 +15,9 @@ #ifndef TDENGINE_QUERYUTIL_H #define TDENGINE_QUERYUTIL_H -#include +#include "function.h" +#include "nodes.h" +#include "plannodes.h" #include "tbuffer.h" #include "tcommon.h" #include "tpagedbuf.h" @@ -77,7 +79,7 @@ typedef struct SResultRowInfo { struct SqlFunctionCtx; size_t getResultRowSize(struct SqlFunctionCtx* pCtx, int32_t numOfOutput); -int32_t initResultRowInfo(SResultRowInfo* pResultRowInfo, int32_t size); +void initResultRowInfo(SResultRowInfo* pResultRowInfo); void cleanupResultRowInfo(SResultRowInfo* pResultRowInfo); void closeAllResultRows(SResultRowInfo* pResultRowInfo); @@ -86,7 +88,7 @@ void initResultRow(SResultRow *pResultRow); void closeResultRow(SResultRow* pResultRow); bool isResultRowClosed(SResultRow* pResultRow); -struct SResultRowEntryInfo* getResultCell(const SResultRow* pRow, int32_t index, const int32_t* offset); +struct SResultRowEntryInfo* getResultEntryInfo(const SResultRow* pRow, int32_t index, const int32_t* offset); static FORCE_INLINE SResultRow *getResultRowByPos(SDiskbasedBuf* pBuf, SResultRowPosition* pos) { SFilePage* bufPage = (SFilePage*) getBufPage(pBuf, pos->pageId); @@ -98,9 +100,27 @@ void initGroupedResultInfo(SGroupResInfo* pGroupResInfo, SHashObj* pHashmap, void initMultiResInfoFromArrayList(SGroupResInfo* pGroupResInfo, SArray* pArrayList); void cleanupGroupResInfo(SGroupResInfo* pGroupResInfo); -bool hashRemainDataInGroupInfo(SGroupResInfo* pGroupResInfo); +bool hasDataInGroupInfo(SGroupResInfo* pGroupResInfo); -bool incNextGroup(SGroupResInfo* pGroupResInfo); int32_t getNumOfTotalRes(SGroupResInfo* pGroupResInfo); +SSDataBlock* createResDataBlock(SDataBlockDescNode* pNode); + +int32_t getTableList(void* metaHandle, SScanPhysiNode* pScanNode, STableListInfo* pListInfo, SNode* pTagCond); +SArray* createSortInfo(SNodeList* pNodeList); +SArray* extractPartitionColInfo(SNodeList* pNodeList); +SArray* extractColMatchInfo(SNodeList* pNodeList, SDataBlockDescNode* pOutputNodeList, int32_t* numOfOutputCols, int32_t type); + +SExprInfo* createExprInfo(SNodeList* pNodeList, SNodeList* pGroupKeys, int32_t* numOfExprs); + +SqlFunctionCtx* createSqlFunctionCtx(SExprInfo* pExprInfo, int32_t numOfOutput, int32_t** rowEntryInfoOffset); +void relocateColumnData(SSDataBlock* pBlock, const SArray* pColMatchInfo, SArray* pCols); +void initExecTimeWindowInfo(SColumnInfoData* pColData, STimeWindow* pQueryWindow); + +SInterval extractIntervalInfo(const STableScanPhysiNode* pTableScanNode); +SColumn extractColumnFromColumnNode(SColumnNode* pColNode); + +int32_t initQueryTableDataCond(SQueryTableDataCond* pCond, const STableScanPhysiNode* pTableScanNode); +void cleanupQueryTableDataCond(SQueryTableDataCond* pCond); + #endif // TDENGINE_QUERYUTIL_H diff --git a/source/libs/executor/inc/executorimpl.h b/source/libs/executor/inc/executorimpl.h index 034e2893df..28241d778c 100644 --- a/source/libs/executor/inc/executorimpl.h +++ b/source/libs/executor/inc/executorimpl.h @@ -49,8 +49,6 @@ typedef int32_t (*__block_search_fn_t)(char* data, int32_t num, int64_t key, int #define Q_STATUS_EQUAL(p, s) (((p) & (s)) != 0u) #define QUERY_IS_ASC_QUERY(q) (GET_FORWARD_DIRECTION_FACTOR((q)->order.order) == QUERY_ASC_FORWARD_STEP) -//#define GET_TABLEGROUP(q, _index) ((SArray*)taosArrayGetP((q)->tableqinfoGroupInfo.pGroupList, (_index))) - #define NEEDTO_COMPRESS_QUERY(size) ((size) > tsCompressColData ? 1 : 0) enum { @@ -64,11 +62,6 @@ enum { TASK_COMPLETED = 0x2u, }; -typedef struct SResultRowCell { - uint64_t groupId; - SResultRowPosition pos; -} SResultRowCell; - /** * If the number of generated results is greater than this value, * query query will be halt and return results to client immediate. @@ -83,7 +76,6 @@ typedef struct SResultInfo { // TODO refactor typedef struct STableQueryInfo { TSKEY lastKey; // last check ts, todo remove it later SResultRowPosition pos; // current active time window -// SVariant tag; } STableQueryInfo; typedef struct SLimit { @@ -123,41 +115,7 @@ typedef struct SOperatorCostInfo { double totalCost; } SOperatorCostInfo; -// The basic query information extracted from the SQueryInfo tree to support the -// execution of query in a data node. -typedef struct STaskAttr { - SLimit limit; - SLimit slimit; - bool stableQuery; // super table query or not - bool topBotQuery; // TODO used bitwise flag - bool groupbyColumn; // denote if this is a groupby normal column query - bool timeWindowInterpo; // if the time window start/end required interpolation - bool tsCompQuery; // is tscomp query - bool diffQuery; // is diff query - bool pointInterpQuery; // point interpolation query - int32_t havingNum; // having expr number - int16_t numOfCols; - int16_t numOfTags; - STimeWindow window; - SInterval interval; - int16_t precision; - int16_t numOfOutput; - int16_t fillType; - int32_t resultRowSize; - int32_t tagLen; // tag value length of current query - - SExprInfo* pExpr1; - SColumnInfo* tagColList; - int32_t numOfFilterCols; - int64_t* fillVal; - void* tsdb; -// STableListInfo tableGroupInfo; // table list - int32_t vgId; -} STaskAttr; - struct SOperatorInfo; -//struct SAggSupporter; -//struct SOptrBasicInfo; typedef int32_t (*__optr_encode_fn_t)(struct SOperatorInfo* pOperator, char** result, int32_t* length); typedef int32_t (*__optr_decode_fn_t)(struct SOperatorInfo* pOperator, char* result); @@ -195,31 +153,6 @@ typedef struct SExecTaskInfo { struct SOperatorInfo* pRoot; } SExecTaskInfo; -typedef struct STaskRuntimeEnv { - STaskAttr* pQueryAttr; - uint32_t status; // query status - uint8_t scanFlag; // denotes reversed scan of data or not - SDiskbasedBuf* pResultBuf; // query result buffer based on blocked-wised disk file - SHashObj* pResultRowHashTable; // quick locate the window object for each result - SHashObj* pResultRowListSet; // used to check if current ResultRowInfo has ResultRow object or not - SArray* pResultRowArrayList; // The array list that contains the Result rows - char* keyBuf; // window key buffer - // The window result objects pool, all the resultRow Objects are allocated and managed by this object. - char** prevRow; - STSBuf* pTsBuf; // timestamp filter list - STSCursor cur; - - char* tagVal; // tag value of current data block -// STableGroupInfo tableqinfoGroupInfo; // this is a table list - struct SOperatorInfo* proot; - SGroupResInfo groupResInfo; - int64_t currentOffset; // dynamic offset value - - STableQueryInfo* current; - SResultInfo resultInfo; - struct SUdfInfo* pUdfInfo; -} STaskRuntimeEnv; - enum { OP_NOT_OPENED = 0x0, OP_OPENED = 0x1, @@ -238,14 +171,20 @@ typedef struct SOperatorFpSet { __optr_explain_fn_t getExplainFn; } SOperatorFpSet; +typedef struct SExprSupp { + SExprInfo* pExprInfo; + int32_t numOfExprs; // the number of scalar expression in group operator + SqlFunctionCtx* pCtx; + int32_t* rowEntryInfoOffset; // offset value for each row result cell info +} SExprSupp; + typedef struct SOperatorInfo { uint8_t operatorType; bool blocking; // block operator or not uint8_t status; // denote if current operator is completed - int32_t numOfExprs; // number of columns of the current operator results - char* name; // name, used to show the query execution plan + char* name; // name, for debug purpose void* info; // extension attribution - SExprInfo* pExpr; + SExprSupp exprSupp; SExecTaskInfo* pTaskInfo; SOperatorCostInfo cost; SResultInfo resultInfo; @@ -260,6 +199,9 @@ typedef enum { EX_SOURCE_DATA_EXHAUSTED = 0x3, } EX_SOURCE_STATUS; +#define COL_MATCH_FROM_COL_ID 0x1 +#define COL_MATCH_FROM_SLOT_ID 0x2 + typedef struct SSourceDataInfo { int32_t index; SRetrieveTableRsp* pRsp; @@ -287,9 +229,6 @@ typedef struct SExchangeInfo { uint64_t self; } SExchangeInfo; -#define COL_MATCH_FROM_COL_ID 0x1 -#define COL_MATCH_FROM_SLOT_ID 0x2 - typedef struct SColMatchInfo { int32_t srcSlotId; // source slot id int32_t colId; @@ -299,8 +238,8 @@ typedef struct SColMatchInfo { } SColMatchInfo; typedef struct SScanInfo { - int32_t numOfAsc; - int32_t numOfDesc; + int32_t numOfAsc; + int32_t numOfDesc; } SScanInfo; typedef struct SSampleExecInfo { @@ -320,17 +259,13 @@ typedef struct STableScanInfo { SNode* pFilterNode; // filter info, which is push down by optimizer SqlFunctionCtx* pCtx; // which belongs to the direct upstream operator operator query context SResultRowInfo* pResultRowInfo; - int32_t* rowCellInfoOffset; + int32_t* rowEntryInfoOffset; SExprInfo* pExpr; SSDataBlock* pResBlock; SArray* pColMatchInfo; int32_t numOfOutput; - SExprInfo* pPseudoExpr; - int32_t numOfPseudoExpr; - SqlFunctionCtx* pPseudoCtx; -// int32_t* rowCellInfoOffset; - + SExprSupp pseudoSup; SQueryTableDataCond cond; int32_t scanFlag; // table scan flag to denote if it is a repeat/reverse/main scan int32_t dataBlockLoadFlag; @@ -436,14 +371,12 @@ typedef struct SBlockDistInfo { void* pHandle; } SBlockDistInfo; +// todo remove this typedef struct SOptrBasicInfo { SResultRowInfo resultRowInfo; - int32_t* rowCellInfoOffset; // offset value for each row result cell info - SqlFunctionCtx* pCtx; SSDataBlock* pRes; } SOptrBasicInfo; -// TODO move the resultrowsiz together with SOptrBasicInfo:rowCellInfoOffset typedef struct SAggSupporter { SHashObj* pResultRowHashTable; // quick locate the window object for each result char* keyBuf; // window key buffer @@ -500,10 +433,7 @@ typedef struct SAggOperatorInfo { STableQueryInfo *current; uint64_t groupId; SGroupResInfo groupResInfo; - SExprInfo *pScalarExprInfo; - int32_t numOfScalarExpr; // the number of scalar expression before the aggregate function can be applied - SqlFunctionCtx *pScalarCtx; // scalar function requried sql function struct. - int32_t *rowCellInfoOffset; // offset value for each row result cell info + SExprSupp scalarExprSup; } SAggOperatorInfo; typedef struct SProjectOperatorInfo { @@ -528,11 +458,7 @@ typedef struct SIndefOperatorInfo { SOptrBasicInfo binfo; SAggSupporter aggSup; SArray* pPseudoColInfo; - - SExprInfo* pScalarExpr; - int32_t numOfScalarExpr; - SqlFunctionCtx* pScalarCtx; - int32_t* rowCellInfoOffset; + SExprSupp scalarSup; } SIndefOperatorInfo; typedef struct SFillOperatorInfo { @@ -544,13 +470,6 @@ typedef struct SFillOperatorInfo { bool multigroupResult; } SFillOperatorInfo; -typedef struct SScalarSupp { - SExprInfo* pScalarExprInfo; - int32_t numOfScalarExpr; // the number of scalar expression in group operator - SqlFunctionCtx* pScalarFuncCtx; - int32_t* rowCellInfoOffset; // offset value for each row result cell info -} SScalarSupp; - typedef struct SGroupbyOperatorInfo { // SOptrBasicInfo should be first, SAggSupporter should be second for stream encode SOptrBasicInfo binfo; @@ -563,7 +482,7 @@ typedef struct SGroupbyOperatorInfo { char* keyBuf; // group by keys for hash int32_t groupKeyLen; // total group by column width SGroupResInfo groupResInfo; - SScalarSupp scalarSup; + SExprSupp scalarSup; } SGroupbyOperatorInfo; typedef struct SDataGroupInfo { @@ -587,7 +506,7 @@ typedef struct SPartitionOperatorInfo { void* pGroupIter; // group iterator int32_t pageIndex; // page index of current group SSDataBlock* pUpdateRes; - SScalarSupp scalarSupp; + SExprSupp scalarSup; } SPartitionOperatorInfo; typedef struct SWindowRowsSup { @@ -740,49 +659,39 @@ SOperatorFpSet createOperatorFpSet(__optr_open_fn_t openFn, __optr_fn_t nextFn, int32_t operatorDummyOpenFn(SOperatorInfo* pOperator); void operatorDummyCloseFn(void* param, int32_t numOfCols); int32_t appendDownstream(SOperatorInfo* p, SOperatorInfo** pDownstream, int32_t num); -int32_t initAggInfo(SOptrBasicInfo* pBasicInfo, SAggSupporter* pAggSup, SExprInfo* pExprInfo, int32_t numOfCols, - SSDataBlock* pResultBlock, size_t keyBufSize, const char* pkey); + +void initBasicInfo(SOptrBasicInfo* pInfo, SSDataBlock* pBlock); +void cleanupBasicInfo(SOptrBasicInfo* pInfo); +void initExprSupp(SExprSupp* pSup, SExprInfo* pExprInfo, int32_t numOfExpr); +void cleanupExprSup(SExprSupp* pSup); +int32_t initAggInfo(SExprSupp *pSup, SAggSupporter* pAggSup, SExprInfo* pExprInfo, int32_t numOfCols, size_t keyBufSize, + const char* pkey); void initResultSizeInfo(SOperatorInfo* pOperator, int32_t numOfRows); void doBuildResultDatablock(SOperatorInfo* pOperator, SOptrBasicInfo* pbInfo, SGroupResInfo* pGroupResInfo, SDiskbasedBuf* pBuf); void doApplyFunctions(SExecTaskInfo* taskInfo, SqlFunctionCtx* pCtx, STimeWindow* pWin, SColumnInfoData* pTimeWindowData, int32_t offset, int32_t forwardStep, TSKEY* tsCol, int32_t numOfTotal, int32_t numOfOutput, int32_t order); -int32_t setGroupResultOutputBuf(SOptrBasicInfo* binfo, int32_t numOfCols, char* pData, int16_t type, int16_t bytes, - int32_t groupId, SDiskbasedBuf* pBuf, SExecTaskInfo* pTaskInfo, SAggSupporter* pAggSup); -void doDestroyBasicInfo(SOptrBasicInfo* pInfo, int32_t numOfOutput); -int32_t setDataBlockFromFetchRsp(SSDataBlock* pRes, SLoadRemoteDataInfo* pLoadInfo, int32_t numOfRows, char* pData, + +int32_t extractDataBlockFromFetchRsp(SSDataBlock* pRes, SLoadRemoteDataInfo* pLoadInfo, int32_t numOfRows, char* pData, int32_t compLen, int32_t numOfOutput, int64_t startTs, uint64_t* total, SArray* pColList); void getAlignQueryTimeWindow(SInterval* pInterval, int32_t precision, int64_t key, STimeWindow* win); int32_t getTableScanInfo(SOperatorInfo* pOperator, int32_t *order, int32_t* scanFlag); int32_t getBufferPgSize(int32_t rowSize, uint32_t* defaultPgsz, uint32_t* defaultBufsz); -SArray* extractPartitionColInfo(SNodeList* pNodeList); - void doSetOperatorCompleted(SOperatorInfo* pOperator); void doFilter(const SNode* pFilterNode, SSDataBlock* pBlock); -SqlFunctionCtx* createSqlFunctionCtx(SExprInfo* pExprInfo, int32_t numOfOutput, int32_t** rowCellInfoOffset); -void relocateColumnData(SSDataBlock* pBlock, const SArray* pColMatchInfo, SArray* pCols); -void initExecTimeWindowInfo(SColumnInfoData* pColData, STimeWindow* pQueryWindow); + void cleanupAggSup(SAggSupporter* pAggSup); void destroyBasicOperatorInfo(void* param, int32_t numOfOutput); void appendOneRowToDataBlock(SSDataBlock* pBlock, STupleHandle* pTupleHandle); void setTbNameColData(void* pMeta, const SSDataBlock* pBlock, SColumnInfoData* pColInfoData, int32_t functionId); -SInterval extractIntervalInfo(const STableScanPhysiNode* pTableScanNode); -SColumn extractColumnFromColumnNode(SColumnNode* pColNode); -SSDataBlock* getSortedBlockData(SSortHandle* pHandle, SSDataBlock* pDataBlock, int32_t capacity, SArray* pColMatchInfo, SSortOperatorInfo* pInfo); +void cleanupExecSupp(SExprSupp* pSupp); + SSDataBlock* loadNextDataBlock(void* param); -void setResultRowInitCtx(SResultRow* pResult, SqlFunctionCtx* pCtx, int32_t numOfOutput, int32_t* rowCellInfoOffset); - -SArray* extractColMatchInfo(SNodeList* pNodeList, SDataBlockDescNode* pOutputNodeList, int32_t* numOfOutputCols, - SExecTaskInfo* pTaskInfo, int32_t type); - -SExprInfo* createExprInfo(SNodeList* pNodeList, SNodeList* pGroupKeys, int32_t* numOfExprs); -SSDataBlock* createResDataBlock(SDataBlockDescNode* pNode); -int32_t initQueryTableDataCond(SQueryTableDataCond* pCond, const STableScanPhysiNode* pTableScanNode); -void clearupQueryTableDataCond(SQueryTableDataCond* pCond); +void setResultRowInitCtx(SResultRow* pResult, SqlFunctionCtx* pCtx, int32_t numOfOutput, int32_t* rowEntryInfoOffset); SResultRow* doSetResultOutBufByKey(SDiskbasedBuf* pResultBuf, SResultRowInfo* pResultRowInfo, char* pData, int16_t bytes, bool masterscan, uint64_t groupId, @@ -799,9 +708,9 @@ SOperatorInfo* createAggregateOperatorInfo(SOperatorInfo* downstream, SExprInfo* int32_t numOfScalarExpr, SExecTaskInfo* pTaskInfo); SOperatorInfo* createIndefinitOutputOperatorInfo(SOperatorInfo* downstream, SPhysiNode *pNode, SExecTaskInfo* pTaskInfo); -SOperatorInfo* createProjectOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t num, SSDataBlock* pResBlock, SLimit* pLimit, SLimit* pSlimit, SNode* pCondition, SExecTaskInfo* pTaskInfo); -SOperatorInfo *createSortOperatorInfo(SOperatorInfo* downstream, SSDataBlock* pResBlock, SArray* pSortInfo, SExprInfo* pExprInfo, int32_t numOfCols, - SArray* pIndexMap, SExecTaskInfo* pTaskInfo); +SOperatorInfo* createProjectOperatorInfo(SOperatorInfo* downstream, SProjectPhysiNode* pProjPhyNode, SExecTaskInfo* pTaskInfo); +SOperatorInfo* createSortOperatorInfo(SOperatorInfo* downstream, SSortPhysiNode* pSortPhyNode, SExecTaskInfo* pTaskInfo); + SOperatorInfo* createMultiwaySortMergeOperatorInfo(SOperatorInfo** downStreams, int32_t numStreams, SSDataBlock* pInputBlock, SSDataBlock* pResBlock, SArray* pSortInfo, SArray* pColMatchColInfo, SExecTaskInfo* pTaskInfo); @@ -831,10 +740,9 @@ SOperatorInfo* createDataBlockInfoScanOperator(void* dataReader, SExecTaskInfo* SOperatorInfo* createStreamScanOperatorInfo(void* pDataReader, SReadHandle* pHandle, STableScanPhysiNode* pTableScanNode, SExecTaskInfo* pTaskInfo, STimeWindowAggSupp* pTwSup); +SOperatorInfo* createFillOperatorInfo(SOperatorInfo* downstream, SFillPhysiNode* pPhyFillNode, bool multigroupResult, + SExecTaskInfo* pTaskInfo); -SOperatorInfo* createFillOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExpr, int32_t numOfCols, - SInterval* pInterval, STimeWindow* pWindow, SSDataBlock* pResBlock, int32_t fillType, SNodeListNode* fillVal, - bool multigroupResult, SExecTaskInfo* pTaskInfo); SOperatorInfo* createStatewindowOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExpr, int32_t numOfCols, SSDataBlock* pResBlock, STimeWindowAggSupp *pTwAggSupp, int32_t tsSlotId, SColumn* pStateKeyCol, SExecTaskInfo* pTaskInfo); @@ -843,10 +751,13 @@ SOperatorInfo* createPartitionOperatorInfo(SOperatorInfo* downstream, SPartition SOperatorInfo* createTimeSliceOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols, SSDataBlock* pResultBlock, const SNodeListNode* pValNode, SExecTaskInfo* pTaskInfo); -SOperatorInfo* createMergeJoinOperatorInfo(SOperatorInfo** pDownstream, int32_t numOfDownstream, SExprInfo* pExprInfo, int32_t numOfCols, SSDataBlock* pResBlock, SNode* pOnCondition, SExecTaskInfo* pTaskInfo); +SOperatorInfo* createMergeJoinOperatorInfo(SOperatorInfo** pDownstream, int32_t numOfDownstream, SJoinPhysiNode* pJoinNode, + SExecTaskInfo* pTaskInfo); -SOperatorInfo* createStreamSessionAggOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols, - SSDataBlock* pResBlock, int64_t gap, int32_t tsSlotId, STimeWindowAggSupp* pTwAggSupp, SExecTaskInfo* pTaskInfo); +SOperatorInfo* createStreamSessionAggOperatorInfo(SOperatorInfo* downstream, + SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo); +SOperatorInfo* createStreamFinalSessionAggOperatorInfo(SOperatorInfo* downstream, + SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo, int32_t numOfChild); SOperatorInfo* createStreamStateAggOperatorInfo(SOperatorInfo* downstream, SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo); @@ -862,8 +773,8 @@ void setInputDataBlock(SOperatorInfo* pOperator, SqlFunctionCtx* pCtx, SSDataBlo bool isTaskKilled(SExecTaskInfo* pTaskInfo); int32_t checkForQueryBuf(size_t numOfTables); -void setTaskKilled(SExecTaskInfo* pTaskInfo); -void queryCostStatis(SExecTaskInfo* pTaskInfo); +void setTaskKilled(SExecTaskInfo* pTaskInfo); +void queryCostStatis(SExecTaskInfo* pTaskInfo); void doDestroyTask(SExecTaskInfo* pTaskInfo); int32_t getMaximumIdleDurationSec(); @@ -882,7 +793,7 @@ int32_t encodeOperator(SOperatorInfo* ops, char** data, int32_t *length); * length: the length of data * return: result code, 0 means success */ -int32_t decodeOperator(SOperatorInfo* ops, char* data, int32_t length); +int32_t decodeOperator(SOperatorInfo* ops, const char* data, int32_t length); void setTaskStatus(SExecTaskInfo* pTaskInfo, int8_t status); int32_t createExecTaskInfoImpl(SSubplan* pPlan, SExecTaskInfo** pTaskInfo, SReadHandle* pHandle, uint64_t taskId, @@ -899,12 +810,13 @@ STimeWindow getActiveTimeWindow(SDiskbasedBuf* pBuf, SResultRowInfo* pResultRowI int32_t getNumOfRowsInTimeWindow(SDataBlockInfo* pDataBlockInfo, TSKEY* pPrimaryColumn, int32_t startPos, TSKEY ekey, __block_search_fn_t searchFn, STableQueryInfo* item, int32_t order); int32_t binarySearchForKey(char* pValue, int num, TSKEY key, int order); -int32_t initStreamAggSupporter(SStreamAggSupporter* pSup, const char* pKey, - SqlFunctionCtx* pCtx, int32_t numOfOutput, int32_t size); +int32_t initStreamAggSupporter(SStreamAggSupporter* pSup, const char* pKey, SqlFunctionCtx* pCtx, int32_t numOfOutput, + int32_t size); SResultRow* getNewResultRow(SDiskbasedBuf* pResultBuf, int64_t tableGroupId, int32_t interBufSize); SResultWindowInfo* getSessionTimeWindow(SStreamAggSupporter* pAggSup, TSKEY ts, uint64_t groupId, int64_t gap, int32_t* pIndex); -int32_t updateSessionWindowInfo(SResultWindowInfo* pWinInfo, TSKEY* pTs, int32_t rows, - int32_t start, int64_t gap, SHashObj* pStDeleted); +int32_t updateSessionWindowInfo(SResultWindowInfo* pWinInfo, TSKEY* pTs, int32_t rows, int32_t start, int64_t gap, + SHashObj* pStDeleted); + bool functionNeedToExecute(SqlFunctionCtx* pCtx); int32_t compareTimeWindow(const void* p1, const void* p2, const void* param); @@ -912,8 +824,6 @@ int32_t finalizeResultRowIntoResultDataBlock(SDiskbasedBuf* pBuf, SResultRowPosi SqlFunctionCtx* pCtx, SExprInfo* pExprInfo, int32_t numOfExprs, const int32_t* rowCellOffset, SSDataBlock* pBlock, SExecTaskInfo* pTaskInfo); -int32_t getTableList(void* metaHandle, int32_t tableType, uint64_t tableUid, STableListInfo* pListInfo, - SNode* pTagCond); int32_t createMultipleDataReaders(STableScanPhysiNode* pTableScanNode, SReadHandle* pHandle, STableListInfo* pTableListInfo, SArray* arrayReader, uint64_t queryId, uint64_t taskId, SNode* pTagCond); diff --git a/source/libs/executor/inc/tsort.h b/source/libs/executor/inc/tsort.h index 86ee841cc2..363f379ee4 100644 --- a/source/libs/executor/inc/tsort.h +++ b/source/libs/executor/inc/tsort.h @@ -63,7 +63,7 @@ typedef int32_t (*_sort_merge_compar_fn_t)(const void* p1, const void* p2, void* * @param type * @return */ -SSortHandle* tsortCreateSortHandle(SArray* pOrderInfo, SArray* pIndexMap, int32_t type, int32_t pageSize, int32_t numOfPages, SSDataBlock* pBlock, const char* idstr); +SSortHandle* tsortCreateSortHandle(SArray* pOrderInfo, int32_t type, int32_t pageSize, int32_t numOfPages, SSDataBlock* pBlock, const char* idstr); /** * diff --git a/source/libs/executor/src/executil.c b/source/libs/executor/src/executil.c index 01ed30c189..99139be409 100644 --- a/source/libs/executor/src/executil.c +++ b/source/libs/executor/src/executil.c @@ -14,24 +14,20 @@ */ #include "os.h" -#include "tmsg.h" +#include "index.h" +#include "function.h" +#include "functionMgt.h" +#include "tdatablock.h" #include "thash.h" +#include "tmsg.h" #include "executil.h" #include "executorimpl.h" #include "tcompression.h" -#include "tlosertree.h" -typedef struct SCompSupporter { - STableQueryInfo **pTableQueryInfo; - int32_t *rowIndex; - int32_t order; -} SCompSupporter; - -int32_t initResultRowInfo(SResultRowInfo *pResultRowInfo, int32_t size) { +void initResultRowInfo(SResultRowInfo *pResultRowInfo) { pResultRowInfo->size = 0; pResultRowInfo->cur.pageId = -1; - return TSDB_CODE_SUCCESS; } void cleanupResultRowInfo(SResultRowInfo *pResultRowInfo) { @@ -46,21 +42,6 @@ void cleanupResultRowInfo(SResultRowInfo *pResultRowInfo) { } } -void resetResultRowInfo(STaskRuntimeEnv *pRuntimeEnv, SResultRowInfo *pResultRowInfo) { - for (int32_t i = 0; i < pResultRowInfo->size; ++i) { -// SResultRow *pWindowRes = pResultRowInfo->pResult[i]; -// clearResultRow(pRuntimeEnv, pWindowRes); - - int32_t groupIndex = 0; - int64_t uid = 0; - - SET_RES_WINDOW_KEY(pRuntimeEnv->keyBuf, &groupIndex, sizeof(groupIndex), uid); - taosHashRemove(pRuntimeEnv->pResultRowHashTable, (const char *)pRuntimeEnv->keyBuf, GET_RES_WINDOW_KEY_LEN(sizeof(groupIndex))); - } - - pResultRowInfo->size = 0; -} - void closeAllResultRows(SResultRowInfo *pResultRowInfo) { // do nothing } @@ -74,7 +55,7 @@ void closeResultRow(SResultRow* pResultRow) { } // TODO refactor: use macro -SResultRowEntryInfo* getResultCell(const SResultRow* pRow, int32_t index, const int32_t* offset) { +SResultRowEntryInfo* getResultEntryInfo(const SResultRow* pRow, int32_t index, const int32_t* offset) { assert(index >= 0 && offset != NULL); return (SResultRowEntryInfo*)((char*) pRow->pEntryInfo + offset[index]); } @@ -160,7 +141,7 @@ void initMultiResInfoFromArrayList(SGroupResInfo* pGroupResInfo, SArray* pArrayL ASSERT(pGroupResInfo->index <= getNumOfTotalRes(pGroupResInfo)); } -bool hashRemainDataInGroupInfo(SGroupResInfo* pGroupResInfo) { +bool hasDataInGroupInfo(SGroupResInfo* pGroupResInfo) { if (pGroupResInfo->pRows == NULL) { return false; } @@ -177,86 +158,532 @@ int32_t getNumOfTotalRes(SGroupResInfo* pGroupResInfo) { return (int32_t) taosArrayGetSize(pGroupResInfo->pRows); } -static int32_t tableResultComparFn(const void *pLeft, const void *pRight, void *param) { - int32_t left = *(int32_t *)pLeft; - int32_t right = *(int32_t *)pRight; - - SCompSupporter * supporter = (SCompSupporter *)param; - - int32_t leftPos = supporter->rowIndex[left]; - int32_t rightPos = supporter->rowIndex[right]; - - /* left source is exhausted */ - if (leftPos == -1) { - return 1; +SArray* createSortInfo(SNodeList* pNodeList) { + size_t numOfCols = LIST_LENGTH(pNodeList); + SArray* pList = taosArrayInit(numOfCols, sizeof(SBlockOrderInfo)); + if (pList == NULL) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + return pList; } - /* right source is exhausted*/ - if (rightPos == -1) { - return -1; + for (int32_t i = 0; i < numOfCols; ++i) { + SOrderByExprNode* pSortKey = (SOrderByExprNode*)nodesListGetNode(pNodeList, i); + SBlockOrderInfo bi = {0}; + bi.order = (pSortKey->order == ORDER_ASC) ? TSDB_ORDER_ASC : TSDB_ORDER_DESC; + bi.nullFirst = (pSortKey->nullOrder == NULL_ORDER_FIRST); + + SColumnNode* pColNode = (SColumnNode*)pSortKey->pExpr; + bi.slotId = pColNode->slotId; + taosArrayPush(pList, &bi); } - ASSERT(0); - STableQueryInfo** pList = supporter->pTableQueryInfo; -// SResultRow* pWindowRes1 = pList[left]->resInfo.pResult[leftPos]; -// SResultRow * pWindowRes1 = getResultRow(&(pList[left]->resInfo), leftPos); -// TSKEY leftTimestamp = pWindowRes1->win.skey; - -// SResultRowInfo *pWindowResInfo2 = &(pList[right]->resInfo); -// SResultRow * pWindowRes2 = getResultRow(pWindowResInfo2, rightPos); -// SResultRow* pWindowRes2 = pList[right]->resInfo.pResult[rightPos]; -// TSKEY rightTimestamp = pWindowRes2->win.skey; - -// if (leftTimestamp == rightTimestamp) { - return 0; -// } - -// if (supporter->order == TSDB_ORDER_ASC) { -// return (leftTimestamp > rightTimestamp)? 1:-1; -// } else { -// return (leftTimestamp < rightTimestamp)? 1:-1; -// } + return pList; } -int32_t tsAscOrder(const void* p1, const void* p2) { - SResultRowCell* pc1 = (SResultRowCell*) p1; - SResultRowCell* pc2 = (SResultRowCell*) p2; +SSDataBlock* createResDataBlock(SDataBlockDescNode* pNode) { + int32_t numOfCols = LIST_LENGTH(pNode->pSlots); - if (pc1->groupId == pc2->groupId) { - ASSERT(0); -// if (pc1->pRow->win.skey == pc2->pRow->win.skey) { -// return 0; -// } else { -// return (pc1->pRow->win.skey < pc2->pRow->win.skey)? -1:1; -// } + SSDataBlock* pBlock = taosMemoryCalloc(1, sizeof(SSDataBlock)); + pBlock->pDataBlock = taosArrayInit(numOfCols, sizeof(SColumnInfoData)); + + pBlock->info.blockId = pNode->dataBlockId; + pBlock->info.rowSize = pNode->totalRowSize; // todo ?? + pBlock->info.type = STREAM_INVALID; + + for (int32_t i = 0; i < numOfCols; ++i) { + SColumnInfoData idata = {{0}}; + SSlotDescNode* pDescNode = (SSlotDescNode*)nodesListGetNode(pNode->pSlots, i); + // if (!pDescNode->output) { // todo disable it temporarily + // continue; + // } + + idata.info.type = pDescNode->dataType.type; + idata.info.bytes = pDescNode->dataType.bytes; + idata.info.scale = pDescNode->dataType.scale; + idata.info.slotId = pDescNode->slotId; + idata.info.precision = pDescNode->dataType.precision; + + if (IS_VAR_DATA_TYPE(idata.info.type)) { + pBlock->info.hasVarCol = true; + } + + taosArrayPush(pBlock->pDataBlock, &idata); + } + + pBlock->info.numOfCols = taosArrayGetSize(pBlock->pDataBlock); + return pBlock; +} + +int32_t getTableList(void* metaHandle, SScanPhysiNode* pScanNode, STableListInfo* pListInfo, SNode* pTagCond) { + int32_t code = TSDB_CODE_SUCCESS; + pListInfo->pTableList = taosArrayInit(8, sizeof(STableKeyInfo)); + + uint64_t tableUid = pScanNode->uid; + + if (pScanNode->tableType == TSDB_SUPER_TABLE) { + if (pTagCond) { + SIndexMetaArg metaArg = { + .metaEx = metaHandle, .idx = tsdbGetIdx(metaHandle), .ivtIdx = tsdbGetIvtIdx(metaHandle), .suid = tableUid}; + + SArray* res = taosArrayInit(8, sizeof(uint64_t)); + code = doFilterTag(pTagCond, &metaArg, res); + if (code == TSDB_CODE_INDEX_REBUILDING) { // todo + // doFilter(); + } else if (code != TSDB_CODE_SUCCESS) { + qError("failed to get tableIds, reason: %s, suid: %" PRIu64 "", tstrerror(code), tableUid); + taosArrayDestroy(res); + terrno = code; + return code; + } else { + qDebug("sucess to get tableIds, size: %d, suid: %" PRIu64 "", (int)taosArrayGetSize(res), tableUid); + } + + for (int i = 0; i < taosArrayGetSize(res); i++) { + STableKeyInfo info = {.lastKey = TSKEY_INITIAL_VAL, .uid = *(uint64_t*)taosArrayGet(res, i)}; + taosArrayPush(pListInfo->pTableList, &info); + } + taosArrayDestroy(res); + } else { + code = tsdbGetAllTableList(metaHandle, tableUid, pListInfo->pTableList); + } + } else { // Create one table group. + STableKeyInfo info = {.lastKey = 0, .uid = tableUid}; + taosArrayPush(pListInfo->pTableList, &info); + } + + return code; +} + +SArray* extractPartitionColInfo(SNodeList* pNodeList) { + if(!pNodeList) { + return NULL; + } + + size_t numOfCols = LIST_LENGTH(pNodeList); + SArray* pList = taosArrayInit(numOfCols, sizeof(SColumn)); + if (pList == NULL) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + return NULL; + } + + for (int32_t i = 0; i < numOfCols; ++i) { + SColumnNode* pColNode = (SColumnNode*)nodesListGetNode(pNodeList, i); + + // todo extract method + SColumn c = {0}; + c.slotId = pColNode->slotId; + c.colId = pColNode->colId; + c.type = pColNode->node.resType.type; + c.bytes = pColNode->node.resType.bytes; + c.precision = pColNode->node.resType.precision; + c.scale = pColNode->node.resType.scale; + + taosArrayPush(pList, &c); + } + + return pList; +} + + +SArray* extractColMatchInfo(SNodeList* pNodeList, SDataBlockDescNode* pOutputNodeList, int32_t* numOfOutputCols, + int32_t type) { + size_t numOfCols = LIST_LENGTH(pNodeList); + SArray* pList = taosArrayInit(numOfCols, sizeof(SColMatchInfo)); + if (pList == NULL) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + return NULL; + } + + for (int32_t i = 0; i < numOfCols; ++i) { + STargetNode* pNode = (STargetNode*)nodesListGetNode(pNodeList, i); + SColumnNode* pColNode = (SColumnNode*)pNode->pExpr; + + SColMatchInfo c = {0}; + c.output = true; + c.colId = pColNode->colId; + c.srcSlotId = pColNode->slotId; + c.matchType = type; + c.targetSlotId = pNode->slotId; + taosArrayPush(pList, &c); + } + + *numOfOutputCols = 0; + int32_t num = LIST_LENGTH(pOutputNodeList->pSlots); + for (int32_t i = 0; i < num; ++i) { + SSlotDescNode* pNode = (SSlotDescNode*)nodesListGetNode(pOutputNodeList->pSlots, i); + + // todo: add reserve flag check + // it is a column reserved for the arithmetic expression calculation + if (pNode->slotId >= numOfCols) { + (*numOfOutputCols) += 1; + continue; + } + + SColMatchInfo* info = taosArrayGet(pList, pNode->slotId); + if (pNode->output) { + (*numOfOutputCols) += 1; + } else { + info->output = false; + } + } + + return pList; +} + +static SResSchema createResSchema(int32_t type, int32_t bytes, int32_t slotId, int32_t scale, int32_t precision, + const char* name) { + SResSchema s = {0}; + s.scale = scale; + s.type = type; + s.bytes = bytes; + s.slotId = slotId; + s.precision = precision; + strncpy(s.name, name, tListLen(s.name)); + + return s; +} + +static SColumn* createColumn(int32_t blockId, int32_t slotId, int32_t colId, SDataType* pType) { + SColumn* pCol = taosMemoryCalloc(1, sizeof(SColumn)); + if (pCol == NULL) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + return NULL; + } + + pCol->slotId = slotId; + pCol->colId = colId; + pCol->bytes = pType->bytes; + pCol->type = pType->type; + pCol->scale = pType->scale; + pCol->precision = pType->precision; + pCol->dataBlockId = blockId; + + return pCol; +} + +SExprInfo* createExprInfo(SNodeList* pNodeList, SNodeList* pGroupKeys, int32_t* numOfExprs) { + int32_t numOfFuncs = LIST_LENGTH(pNodeList); + int32_t numOfGroupKeys = 0; + if (pGroupKeys != NULL) { + numOfGroupKeys = LIST_LENGTH(pGroupKeys); + } + + *numOfExprs = numOfFuncs + numOfGroupKeys; + SExprInfo* pExprs = taosMemoryCalloc(*numOfExprs, sizeof(SExprInfo)); + + for (int32_t i = 0; i < (*numOfExprs); ++i) { + STargetNode* pTargetNode = NULL; + if (i < numOfFuncs) { + pTargetNode = (STargetNode*)nodesListGetNode(pNodeList, i); + } else { + pTargetNode = (STargetNode*)nodesListGetNode(pGroupKeys, i - numOfFuncs); + } + + SExprInfo* pExp = &pExprs[i]; + + pExp->pExpr = taosMemoryCalloc(1, sizeof(tExprNode)); + pExp->pExpr->_function.num = 1; + pExp->pExpr->_function.functionId = -1; + + int32_t type = nodeType(pTargetNode->pExpr); + // it is a project query, or group by column + if (type == QUERY_NODE_COLUMN) { + pExp->pExpr->nodeType = QUERY_NODE_COLUMN; + SColumnNode* pColNode = (SColumnNode*)pTargetNode->pExpr; + + pExp->base.pParam = taosMemoryCalloc(1, sizeof(SFunctParam)); + pExp->base.numOfParams = 1; + + SDataType* pType = &pColNode->node.resType; + pExp->base.resSchema = createResSchema(pType->type, pType->bytes, pTargetNode->slotId, pType->scale, + pType->precision, pColNode->colName); + pExp->base.pParam[0].pCol = createColumn(pColNode->dataBlockId, pColNode->slotId, pColNode->colId, pType); + pExp->base.pParam[0].type = FUNC_PARAM_TYPE_COLUMN; + } else if (type == QUERY_NODE_VALUE) { + pExp->pExpr->nodeType = QUERY_NODE_VALUE; + SValueNode* pValNode = (SValueNode*)pTargetNode->pExpr; + + pExp->base.pParam = taosMemoryCalloc(1, sizeof(SFunctParam)); + pExp->base.numOfParams = 1; + + SDataType* pType = &pValNode->node.resType; + pExp->base.resSchema = createResSchema(pType->type, pType->bytes, pTargetNode->slotId, pType->scale, + pType->precision, pValNode->node.aliasName); + pExp->base.pParam[0].type = FUNC_PARAM_TYPE_VALUE; + nodesValueNodeToVariant(pValNode, &pExp->base.pParam[0].param); + } else if (type == QUERY_NODE_FUNCTION) { + pExp->pExpr->nodeType = QUERY_NODE_FUNCTION; + SFunctionNode* pFuncNode = (SFunctionNode*)pTargetNode->pExpr; + + SDataType* pType = &pFuncNode->node.resType; + pExp->base.resSchema = createResSchema(pType->type, pType->bytes, pTargetNode->slotId, pType->scale, + pType->precision, pFuncNode->node.aliasName); + + pExp->pExpr->_function.functionId = pFuncNode->funcId; + pExp->pExpr->_function.pFunctNode = pFuncNode; + + strncpy(pExp->pExpr->_function.functionName, pFuncNode->functionName, + tListLen(pExp->pExpr->_function.functionName)); +#if 1 + // todo refactor: add the parameter for tbname function + if (strcmp(pExp->pExpr->_function.functionName, "tbname") == 0) { + pFuncNode->pParameterList = nodesMakeList(); + ASSERT(LIST_LENGTH(pFuncNode->pParameterList) == 0); + SValueNode* res = (SValueNode*)nodesMakeNode(QUERY_NODE_VALUE); + if (NULL == res) { // todo handle error + } else { + res->node.resType = (SDataType){.bytes = sizeof(int64_t), .type = TSDB_DATA_TYPE_BIGINT}; + nodesListAppend(pFuncNode->pParameterList, (SNode*)res); + } + } +#endif + + int32_t numOfParam = LIST_LENGTH(pFuncNode->pParameterList); + + pExp->base.pParam = taosMemoryCalloc(numOfParam, sizeof(SFunctParam)); + pExp->base.numOfParams = numOfParam; + + for (int32_t j = 0; j < numOfParam; ++j) { + SNode* p1 = nodesListGetNode(pFuncNode->pParameterList, j); + if (p1->type == QUERY_NODE_COLUMN) { + SColumnNode* pcn = (SColumnNode*)p1; + + pExp->base.pParam[j].type = FUNC_PARAM_TYPE_COLUMN; + pExp->base.pParam[j].pCol = createColumn(pcn->dataBlockId, pcn->slotId, pcn->colId, &pcn->node.resType); + } else if (p1->type == QUERY_NODE_VALUE) { + SValueNode* pvn = (SValueNode*)p1; + pExp->base.pParam[j].type = FUNC_PARAM_TYPE_VALUE; + nodesValueNodeToVariant(pvn, &pExp->base.pParam[j].param); + } + } + } else if (type == QUERY_NODE_OPERATOR) { + pExp->pExpr->nodeType = QUERY_NODE_OPERATOR; + SOperatorNode* pNode = (SOperatorNode*)pTargetNode->pExpr; + + pExp->base.pParam = taosMemoryCalloc(1, sizeof(SFunctParam)); + pExp->base.numOfParams = 1; + + SDataType* pType = &pNode->node.resType; + pExp->base.resSchema = createResSchema(pType->type, pType->bytes, pTargetNode->slotId, pType->scale, + pType->precision, pNode->node.aliasName); + pExp->pExpr->_optrRoot.pRootNode = pTargetNode->pExpr; + } else { + ASSERT(0); + } + } + + return pExprs; +} + +// set the output buffer for the selectivity + tag query +static int32_t setSelectValueColumnInfo(SqlFunctionCtx* pCtx, int32_t numOfOutput) { + int32_t num = 0; + + SqlFunctionCtx* p = NULL; + SqlFunctionCtx** pValCtx = taosMemoryCalloc(numOfOutput, POINTER_BYTES); + if (pValCtx == NULL) { + return TSDB_CODE_QRY_OUT_OF_MEMORY; + } + + for (int32_t i = 0; i < numOfOutput; ++i) { + if (strcmp(pCtx[i].pExpr->pExpr->_function.functionName, "_select_value") == 0) { + pValCtx[num++] = &pCtx[i]; + } else if (fmIsSelectFunc(pCtx[i].functionId)) { + p = &pCtx[i]; + } + } + + if (p != NULL) { + p->subsidiaries.pCtx = pValCtx; + p->subsidiaries.num = num; } else { - return (pc1->groupId < pc2->groupId)? -1:1; + taosMemoryFreeClear(pValCtx); + } + + return TSDB_CODE_SUCCESS; +} + +SqlFunctionCtx* createSqlFunctionCtx(SExprInfo* pExprInfo, int32_t numOfOutput, int32_t** rowEntryInfoOffset) { + SqlFunctionCtx* pFuncCtx = (SqlFunctionCtx*)taosMemoryCalloc(numOfOutput, sizeof(SqlFunctionCtx)); + if (pFuncCtx == NULL) { + return NULL; + } + + *rowEntryInfoOffset = taosMemoryCalloc(numOfOutput, sizeof(int32_t)); + if (*rowEntryInfoOffset == 0) { + taosMemoryFreeClear(pFuncCtx); + return NULL; + } + + for (int32_t i = 0; i < numOfOutput; ++i) { + SExprInfo* pExpr = &pExprInfo[i]; + + SExprBasicInfo* pFunct = &pExpr->base; + SqlFunctionCtx* pCtx = &pFuncCtx[i]; + + pCtx->functionId = -1; + pCtx->curBufPage = -1; + pCtx->pExpr = pExpr; + + if (pExpr->pExpr->nodeType == QUERY_NODE_FUNCTION) { + SFuncExecEnv env = {0}; + pCtx->functionId = pExpr->pExpr->_function.pFunctNode->funcId; + + if (fmIsAggFunc(pCtx->functionId) || fmIsIndefiniteRowsFunc(pCtx->functionId)) { + bool isUdaf = fmIsUserDefinedFunc(pCtx->functionId); + if (!isUdaf) { + fmGetFuncExecFuncs(pCtx->functionId, &pCtx->fpSet); + } else { + char* udfName = pExpr->pExpr->_function.pFunctNode->functionName; + strncpy(pCtx->udfName, udfName, strlen(udfName)); + fmGetUdafExecFuncs(pCtx->functionId, &pCtx->fpSet); + } + pCtx->fpSet.getEnv(pExpr->pExpr->_function.pFunctNode, &env); + } else { + fmGetScalarFuncExecFuncs(pCtx->functionId, &pCtx->sfp); + if (pCtx->sfp.getEnv != NULL) { + pCtx->sfp.getEnv(pExpr->pExpr->_function.pFunctNode, &env); + } + } + pCtx->resDataInfo.interBufSize = env.calcMemSize; + } else if (pExpr->pExpr->nodeType == QUERY_NODE_COLUMN || pExpr->pExpr->nodeType == QUERY_NODE_OPERATOR || + pExpr->pExpr->nodeType == QUERY_NODE_VALUE) { + // for simple column, the result buffer needs to hold at least one element. + pCtx->resDataInfo.interBufSize = pFunct->resSchema.bytes; + } + + pCtx->input.numOfInputCols = pFunct->numOfParams; + pCtx->input.pData = taosMemoryCalloc(pFunct->numOfParams, POINTER_BYTES); + pCtx->input.pColumnDataAgg = taosMemoryCalloc(pFunct->numOfParams, POINTER_BYTES); + + pCtx->pTsOutput = NULL; + pCtx->resDataInfo.bytes = pFunct->resSchema.bytes; + pCtx->resDataInfo.type = pFunct->resSchema.type; + pCtx->order = TSDB_ORDER_ASC; + pCtx->start.key = INT64_MIN; + pCtx->end.key = INT64_MIN; + pCtx->numOfParams = pExpr->base.numOfParams; + pCtx->increase = false; + + pCtx->param = pFunct->pParam; + } + + for (int32_t i = 1; i < numOfOutput; ++i) { + (*rowEntryInfoOffset)[i] = + (int32_t)((*rowEntryInfoOffset)[i - 1] + sizeof(SResultRowEntryInfo) + pFuncCtx[i - 1].resDataInfo.interBufSize); + } + + setSelectValueColumnInfo(pFuncCtx, numOfOutput); + return pFuncCtx; +} + +// NOTE: sources columns are more than the destination SSDatablock columns. +void relocateColumnData(SSDataBlock* pBlock, const SArray* pColMatchInfo, SArray* pCols) { + size_t numOfSrcCols = taosArrayGetSize(pCols); + + int32_t i = 0, j = 0; + while (i < numOfSrcCols && j < taosArrayGetSize(pColMatchInfo)) { + SColumnInfoData* p = taosArrayGet(pCols, i); + SColMatchInfo* pmInfo = taosArrayGet(pColMatchInfo, j); + if (!pmInfo->output) { + j++; + continue; + } + + if (p->info.colId == pmInfo->colId) { + SColumnInfoData* pDst = taosArrayGet(pBlock->pDataBlock, pmInfo->targetSlotId); + colDataAssign(pDst, p, pBlock->info.rows); + i++; + j++; + } else if (p->info.colId < pmInfo->colId) { + i++; + } else { + ASSERT(0); + } } } -int32_t tsDescOrder(const void* p1, const void* p2) { - SResultRowCell* pc1 = (SResultRowCell*) p1; - SResultRowCell* pc2 = (SResultRowCell*) p2; +SInterval extractIntervalInfo(const STableScanPhysiNode* pTableScanNode) { + SInterval interval = { + .interval = pTableScanNode->interval, + .sliding = pTableScanNode->sliding, + .intervalUnit = pTableScanNode->intervalUnit, + .slidingUnit = pTableScanNode->slidingUnit, + .offset = pTableScanNode->offset, + }; - if (pc1->groupId == pc2->groupId) { - ASSERT(0); -// if (pc1->pRow->win.skey == pc2->pRow->win.skey) { -// return 0; -// } else { -// return (pc1->pRow->win.skey < pc2->pRow->win.skey)? 1:-1; -// } - } else { - return (pc1->groupId < pc2->groupId)? -1:1; + return interval; +} + +SColumn extractColumnFromColumnNode(SColumnNode* pColNode) { + SColumn c = {0}; + c.slotId = pColNode->slotId; + c.colId = pColNode->colId; + c.type = pColNode->node.resType.type; + c.bytes = pColNode->node.resType.bytes; + c.scale = pColNode->node.resType.scale; + c.precision = pColNode->node.resType.precision; + return c; +} + +int32_t initQueryTableDataCond(SQueryTableDataCond* pCond, const STableScanPhysiNode* pTableScanNode) { + pCond->loadExternalRows = false; + + pCond->order = pTableScanNode->scanSeq[0] > 0 ? TSDB_ORDER_ASC : TSDB_ORDER_DESC; + pCond->numOfCols = LIST_LENGTH(pTableScanNode->scan.pScanCols); + pCond->colList = taosMemoryCalloc(pCond->numOfCols, sizeof(SColumnInfo)); + if (pCond->colList == NULL) { + terrno = TSDB_CODE_QRY_OUT_OF_MEMORY; + return terrno; } + + // pCond->twindow = pTableScanNode->scanRange; + // TODO: get it from stable scan node + pCond->numOfTWindows = 1; + pCond->twindows = taosMemoryCalloc(pCond->numOfTWindows, sizeof(STimeWindow)); + pCond->twindows[0] = pTableScanNode->scanRange; + pCond->suid = pTableScanNode->scan.suid; + +#if 1 + // todo work around a problem, remove it later + for (int32_t i = 0; i < pCond->numOfTWindows; ++i) { + if ((pCond->order == TSDB_ORDER_ASC && pCond->twindows[i].skey > pCond->twindows[i].ekey) || + (pCond->order == TSDB_ORDER_DESC && pCond->twindows[i].skey < pCond->twindows[i].ekey)) { + TSWAP(pCond->twindows[i].skey, pCond->twindows[i].ekey); + } + } +#endif + + for (int32_t i = 0; i < pCond->numOfTWindows; ++i) { + if ((pCond->order == TSDB_ORDER_ASC && pCond->twindows[i].skey > pCond->twindows[i].ekey) || + (pCond->order == TSDB_ORDER_DESC && pCond->twindows[i].skey < pCond->twindows[i].ekey)) { + TSWAP(pCond->twindows[i].skey, pCond->twindows[i].ekey); + } + } + taosqsort(pCond->twindows, pCond->numOfTWindows, sizeof(STimeWindow), pCond, compareTimeWindow); + + pCond->type = BLOCK_LOAD_OFFSET_SEQ_ORDER; + // pCond->type = pTableScanNode->scanFlag; + + int32_t j = 0; + for (int32_t i = 0; i < pCond->numOfCols; ++i) { + STargetNode* pNode = (STargetNode*)nodesListGetNode(pTableScanNode->scan.pScanCols, i); + SColumnNode* pColNode = (SColumnNode*)pNode->pExpr; + if (pColNode->colType == COLUMN_TYPE_TAG) { + continue; + } + + pCond->colList[j].type = pColNode->node.resType.type; + pCond->colList[j].bytes = pColNode->node.resType.bytes; + pCond->colList[j].colId = pColNode->colId; + j += 1; + } + + pCond->numOfCols = j; + return TSDB_CODE_SUCCESS; } -void orderTheResultRows(STaskRuntimeEnv* pRuntimeEnv) { - __compar_fn_t fn = NULL; -// if (pRuntimeEnv->pQueryAttr->order.order == TSDB_ORDER_ASC) { -// fn = tsAscOrder; -// } else { -// fn = tsDescOrder; -// } - - taosArraySort(pRuntimeEnv->pResultRowArrayList, fn); -} +void cleanupQueryTableDataCond(SQueryTableDataCond* pCond) { + taosMemoryFree(pCond->twindows); + taosMemoryFree(pCond->colList); +} \ No newline at end of file diff --git a/source/libs/executor/src/executorMain.c b/source/libs/executor/src/executorMain.c index 00158d7024..ff281dacbd 100644 --- a/source/libs/executor/src/executorMain.c +++ b/source/libs/executor/src/executorMain.c @@ -219,4 +219,23 @@ int32_t qGetExplainExecInfo(qTaskInfo_t tinfo, int32_t *resNum, SExplainExecInfo return getOperatorExplainExecInfo(pTaskInfo->pRoot, pRes, &capacity, resNum); } +int32_t qSerializeTaskStatus(qTaskInfo_t tinfo, char** pOutput, int32_t* len) { + SExecTaskInfo* pTaskInfo = (struct SExecTaskInfo*)tinfo; + if (pTaskInfo->pRoot == NULL) { + return TSDB_CODE_INVALID_PARA; + } + + return encodeOperator(pTaskInfo->pRoot, pOutput, len); +} + +int32_t qDeserializeTaskStatus(qTaskInfo_t tinfo, const char* pInput, int32_t len) { + SExecTaskInfo* pTaskInfo = (struct SExecTaskInfo*) tinfo; + + if (pTaskInfo == NULL || pInput == NULL || len == 0) { + return TSDB_CODE_INVALID_PARA; + } + + return decodeOperator(pTaskInfo->pRoot, pInput, len); +} + diff --git a/source/libs/executor/src/executorimpl.c b/source/libs/executor/src/executorimpl.c index 40b019eb5d..421fd56e1f 100644 --- a/source/libs/executor/src/executorimpl.c +++ b/source/libs/executor/src/executorimpl.c @@ -90,8 +90,6 @@ static void doSetTagValueToResultBuf(char* output, const char* val, int16_t type static void setBlockStatisInfo(SqlFunctionCtx* pCtx, SExprInfo* pExpr, SSDataBlock* pSDataBlock); -static void destroyTableQueryInfoImpl(STableQueryInfo* pTableQueryInfo); - static void releaseQueryBuf(size_t numOfTables); static void destroySFillOperatorInfo(void* param, int32_t numOfOutput); @@ -143,71 +141,10 @@ static int32_t doCopyToSDataBlock(SExecTaskInfo* taskInfo, SSDataBlock* pBlock, SqlFunctionCtx* pCtx, int32_t numOfExprs); static void initCtxOutputBuffer(SqlFunctionCtx* pCtx, int32_t size); -static void setResultBufSize(STaskAttr* pQueryAttr, SResultInfo* pResultInfo); -static void doSetTableGroupOutputBuf(SAggOperatorInfo* pAggInfo, int32_t numOfOutput, uint64_t groupId, - SExecTaskInfo* pTaskInfo); +static void doSetTableGroupOutputBuf(SOperatorInfo* pOperator, SAggOperatorInfo* pAggInfo, int32_t numOfOutput, uint64_t groupId); -SArray* getOrderCheckColumns(STaskAttr* pQuery); - -typedef struct SRowCompSupporter { - STaskRuntimeEnv* pRuntimeEnv; - int16_t dataOffset; - __compar_fn_t comFunc; -} SRowCompSupporter; - -static int compareRowData(const void* a, const void* b, const void* userData) { - const SResultRow* pRow1 = (const SResultRow*)a; - const SResultRow* pRow2 = (const SResultRow*)b; - - SRowCompSupporter* supporter = (SRowCompSupporter*)userData; - STaskRuntimeEnv* pRuntimeEnv = supporter->pRuntimeEnv; - - SFilePage* page1 = getBufPage(pRuntimeEnv->pResultBuf, pRow1->pageId); - SFilePage* page2 = getBufPage(pRuntimeEnv->pResultBuf, pRow2->pageId); - - int16_t offset = supporter->dataOffset; - return 0; - // char* in1 = getPosInResultPage(pRuntimeEnv->pQueryAttr, page1, pRow1->offset, offset); - // char* in2 = getPosInResultPage(pRuntimeEnv->pQueryAttr, page2, pRow2->offset, offset); - - // return (in1 != NULL && in2 != NULL) ? supporter->comFunc(in1, in2) : 0; -} // setup the output buffer for each operator -SSDataBlock* createResDataBlock(SDataBlockDescNode* pNode) { - int32_t numOfCols = LIST_LENGTH(pNode->pSlots); - - SSDataBlock* pBlock = taosMemoryCalloc(1, sizeof(SSDataBlock)); - pBlock->pDataBlock = taosArrayInit(numOfCols, sizeof(SColumnInfoData)); - - pBlock->info.blockId = pNode->dataBlockId; - pBlock->info.rowSize = pNode->totalRowSize; // todo ?? - pBlock->info.type = STREAM_INVALID; - - for (int32_t i = 0; i < numOfCols; ++i) { - SColumnInfoData idata = {{0}}; - SSlotDescNode* pDescNode = (SSlotDescNode*)nodesListGetNode(pNode->pSlots, i); - // if (!pDescNode->output) { // todo disable it temporarily - // continue; - // } - - idata.info.type = pDescNode->dataType.type; - idata.info.bytes = pDescNode->dataType.bytes; - idata.info.scale = pDescNode->dataType.scale; - idata.info.slotId = pDescNode->slotId; - idata.info.precision = pDescNode->dataType.precision; - - if (IS_VAR_DATA_TYPE(idata.info.type)) { - pBlock->info.hasVarCol = true; - } - - taosArrayPush(pBlock->pDataBlock, &idata); - } - - pBlock->info.numOfCols = taosArrayGetSize(pBlock->pDataBlock); - return pBlock; -} - static bool hasNull(SColumn* pColumn, SColumnDataAgg* pStatis) { if (TSDB_COL_IS_TAG(pColumn->flag) || TSDB_COL_IS_UD_COL(pColumn->flag) || pColumn->colId == PRIMARYKEY_TIMESTAMP_COL_ID) { @@ -221,6 +158,7 @@ static bool hasNull(SColumn* pColumn, SColumnDataAgg* pStatis) { return true; } +#if 0 static bool chkResultRowFromKey(STaskRuntimeEnv* pRuntimeEnv, SResultRowInfo* pResultRowInfo, char* pData, int16_t bytes, bool masterscan, uint64_t uid) { bool existed = false; @@ -257,6 +195,7 @@ static bool chkResultRowFromKey(STaskRuntimeEnv* pRuntimeEnv, SResultRowInfo* pR return p1 != NULL; } +#endif SResultRow* getNewResultRow(SDiskbasedBuf* pResultBuf, int64_t tableGroupId, int32_t interBufSize) { SFilePage* pData = NULL; @@ -365,36 +304,6 @@ SResultRow* doSetResultOutBufByKey(SDiskbasedBuf* pResultBuf, SResultRowInfo* pR return pResult; } -// get the correct time window according to the handled timestamp -static STimeWindow getCurrentActiveTimeWindow(SResultRowInfo* pResultRowInfo, int64_t ts, STaskAttr* pQueryAttr) { - STimeWindow w = {0}; -#if 0 - if (pResultRowInfo->curPos == -1) { // the first window, from the previous stored value - // getInitialStartTimeWindow(pQueryAttr, ts, &w); - - if (pQueryAttr->interval.intervalUnit == 'n' || pQueryAttr->interval.intervalUnit == 'y') { - w.ekey = - taosTimeAdd(w.skey, pQueryAttr->interval.interval, pQueryAttr->interval.intervalUnit, pQueryAttr->precision) - - 1; - } else { - w.ekey = w.skey + pQueryAttr->interval.interval - 1; - } - } else { - w = pRow->win; - } - - /* - * query border check, skey should not be bounded by the query time range, since the value skey will - * be used as the time window index value. So we only change ekey of time window accordingly. - */ - if (w.ekey > pQueryAttr->window.ekey && QUERY_IS_ASC_QUERY(pQueryAttr)) { - w.ekey = pQueryAttr->window.ekey; - } -#endif - - return w; -} - // a new buffer page for each table. Needs to opt this design static int32_t addNewWindowResultBuf(SResultRow* pWindowRes, SDiskbasedBuf* pResultBuf, int32_t tid, uint32_t size) { if (pWindowRes->pageId != -1) { @@ -442,13 +351,6 @@ static int32_t addNewWindowResultBuf(SResultRow* pWindowRes, SDiskbasedBuf* pRes return 0; } -static bool chkWindowOutputBufByKey(STaskRuntimeEnv* pRuntimeEnv, SResultRowInfo* pResultRowInfo, STimeWindow* win, - bool masterscan, SResultRow** pResult, int64_t groupId, SqlFunctionCtx* pCtx, - int32_t numOfOutput, int32_t* rowCellInfoOffset) { - assert(win->skey <= win->ekey); - return chkResultRowFromKey(pRuntimeEnv, pResultRowInfo, (char*)&win->skey, TSDB_KEYSIZE, masterscan, groupId); -} - // query_range_start, query_range_end, window_duration, window_start, window_end void initExecTimeWindowInfo(SColumnInfoData* pColData, STimeWindow* pQueryWindow) { pColData->info.type = TSDB_DATA_TYPE_TIMESTAMP; @@ -517,33 +419,15 @@ void doApplyFunctions(SExecTaskInfo* taskInfo, SqlFunctionCtx* pCtx, STimeWindow } } -static FORCE_INLINE TSKEY reviseWindowEkey(STaskAttr* pQueryAttr, STimeWindow* pWindow) { - TSKEY ekey = -1; - int32_t order = TSDB_ORDER_ASC; - if (order == TSDB_ORDER_ASC) { - ekey = pWindow->ekey; - if (ekey > pQueryAttr->window.ekey) { - ekey = pQueryAttr->window.ekey; - } - } else { - ekey = pWindow->skey; - if (ekey < pQueryAttr->window.ekey) { - ekey = pQueryAttr->window.ekey; - } - } - - return ekey; -} - static int32_t doSetInputDataBlock(SOperatorInfo* pOperator, SqlFunctionCtx* pCtx, SSDataBlock* pBlock, int32_t order, int32_t scanFlag, bool createDummyCol); static void doSetInputDataBlockInfo(SOperatorInfo* pOperator, SqlFunctionCtx* pCtx, SSDataBlock* pBlock, int32_t order) { - for (int32_t i = 0; i < pOperator->numOfExprs; ++i) { + for (int32_t i = 0; i < pOperator->exprSupp.numOfExprs; ++i) { pCtx[i].order = order; pCtx[i].input.numOfRows = pBlock->info.rows; - setBlockStatisInfo(&pCtx[i], &pOperator->pExpr[i], pBlock); + setBlockStatisInfo(&pCtx[i], &pOperator->exprSupp.pExprInfo[i], pBlock); } } @@ -602,7 +486,7 @@ static int32_t doSetInputDataBlock(SOperatorInfo* pOperator, SqlFunctionCtx* pCt int32_t scanFlag, bool createDummyCol) { int32_t code = TSDB_CODE_SUCCESS; - for (int32_t i = 0; i < pOperator->numOfExprs; ++i) { + for (int32_t i = 0; i < pOperator->exprSupp.numOfExprs; ++i) { pCtx[i].order = order; pCtx[i].input.numOfRows = pBlock->info.rows; @@ -613,7 +497,7 @@ static int32_t doSetInputDataBlock(SOperatorInfo* pOperator, SqlFunctionCtx* pCt pInput->uid = pBlock->info.uid; pInput->colDataAggIsSet = false; - SExprInfo* pOneExpr = &pOperator->pExpr[i]; + SExprInfo* pOneExpr = &pOperator->exprSupp.pExprInfo[i]; for (int32_t j = 0; j < pOneExpr->base.numOfParams; ++j) { SFunctParam* pFuncParam = &pOneExpr->base.pParam[j]; if (pFuncParam->type == FUNC_PARAM_TYPE_COLUMN) { @@ -649,7 +533,7 @@ static int32_t doSetInputDataBlock(SOperatorInfo* pOperator, SqlFunctionCtx* pCt } static int32_t doAggregateImpl(SOperatorInfo* pOperator, TSKEY startTs, SqlFunctionCtx* pCtx) { - for (int32_t k = 0; k < pOperator->numOfExprs; ++k) { + for (int32_t k = 0; k < pOperator->exprSupp.numOfExprs; ++k) { if (functionNeedToExecute(&pCtx[k])) { // todo add a dummy funtion to avoid process check if (pCtx[k].fpSet.process == NULL) { @@ -802,20 +686,6 @@ static void setResultRowKey(SResultRow* pResultRow, char* pData, int16_t type) { } } -int32_t setGroupResultOutputBuf(SOptrBasicInfo* binfo, int32_t numOfCols, char* pData, int16_t type, int16_t bytes, - int32_t groupId, SDiskbasedBuf* pBuf, SExecTaskInfo* pTaskInfo, - SAggSupporter* pAggSup) { - SResultRowInfo* pResultRowInfo = &binfo->resultRowInfo; - SqlFunctionCtx* pCtx = binfo->pCtx; - - SResultRow* pResultRow = - doSetResultOutBufByKey(pBuf, pResultRowInfo, (char*)pData, bytes, true, groupId, pTaskInfo, false, pAggSup); - assert(pResultRow != NULL); - - setResultRowInitCtx(pResultRow, pCtx, numOfCols, binfo->rowCellInfoOffset); - return TSDB_CODE_SUCCESS; -} - bool functionNeedToExecute(SqlFunctionCtx* pCtx) { struct SResultRowEntryInfo* pResInfo = GET_RES_INFO(pCtx); @@ -927,137 +797,6 @@ void setBlockStatisInfo(SqlFunctionCtx* pCtx, SExprInfo* pExprInfo, SSDataBlock* // } } -// set the output buffer for the selectivity + tag query -static int32_t setSelectValueColumnInfo(SqlFunctionCtx* pCtx, int32_t numOfOutput) { - int32_t num = 0; - - SqlFunctionCtx* p = NULL; - SqlFunctionCtx** pValCtx = taosMemoryCalloc(numOfOutput, POINTER_BYTES); - if (pValCtx == NULL) { - return TSDB_CODE_QRY_OUT_OF_MEMORY; - } - - for (int32_t i = 0; i < numOfOutput; ++i) { - if (strcmp(pCtx[i].pExpr->pExpr->_function.functionName, "_select_value") == 0) { - pValCtx[num++] = &pCtx[i]; - } else if (fmIsSelectFunc(pCtx[i].functionId)) { - p = &pCtx[i]; - } - // if (functionId == FUNCTION_TAG_DUMMY || functionId == FUNCTION_TS_DUMMY) { - // tagLen += pCtx[i].resDataInfo.bytes; - // pTagCtx[num++] = &pCtx[i]; - // } else if (functionId == FUNCTION_TS || functionId == FUNCTION_TAG) { - // // tag function may be the group by tag column - // // ts may be the required primary timestamp column - // continue; - // } else { - // // the column may be the normal column, group by normal_column, the functionId is FUNCTION_PRJ - // } - } - - if (p != NULL) { - p->subsidiaries.pCtx = pValCtx; - p->subsidiaries.num = num; - } else { - taosMemoryFreeClear(pValCtx); - } - - return TSDB_CODE_SUCCESS; -} - -SqlFunctionCtx* createSqlFunctionCtx(SExprInfo* pExprInfo, int32_t numOfOutput, int32_t** rowCellInfoOffset) { - SqlFunctionCtx* pFuncCtx = (SqlFunctionCtx*)taosMemoryCalloc(numOfOutput, sizeof(SqlFunctionCtx)); - if (pFuncCtx == NULL) { - return NULL; - } - - *rowCellInfoOffset = taosMemoryCalloc(numOfOutput, sizeof(int32_t)); - if (*rowCellInfoOffset == 0) { - taosMemoryFreeClear(pFuncCtx); - return NULL; - } - - for (int32_t i = 0; i < numOfOutput; ++i) { - SExprInfo* pExpr = &pExprInfo[i]; - - SExprBasicInfo* pFunct = &pExpr->base; - SqlFunctionCtx* pCtx = &pFuncCtx[i]; - - pCtx->functionId = -1; - pCtx->curBufPage = -1; - pCtx->pExpr = pExpr; - - if (pExpr->pExpr->nodeType == QUERY_NODE_FUNCTION) { - SFuncExecEnv env = {0}; - pCtx->functionId = pExpr->pExpr->_function.pFunctNode->funcId; - - if (fmIsAggFunc(pCtx->functionId) || fmIsIndefiniteRowsFunc(pCtx->functionId)) { - bool isUdaf = fmIsUserDefinedFunc(pCtx->functionId); - if (!isUdaf) { - fmGetFuncExecFuncs(pCtx->functionId, &pCtx->fpSet); - } else { - char* udfName = pExpr->pExpr->_function.pFunctNode->functionName; - strncpy(pCtx->udfName, udfName, strlen(udfName)); - fmGetUdafExecFuncs(pCtx->functionId, &pCtx->fpSet); - } - pCtx->fpSet.getEnv(pExpr->pExpr->_function.pFunctNode, &env); - } else { - fmGetScalarFuncExecFuncs(pCtx->functionId, &pCtx->sfp); - if (pCtx->sfp.getEnv != NULL) { - pCtx->sfp.getEnv(pExpr->pExpr->_function.pFunctNode, &env); - } - } - pCtx->resDataInfo.interBufSize = env.calcMemSize; - } else if (pExpr->pExpr->nodeType == QUERY_NODE_COLUMN || pExpr->pExpr->nodeType == QUERY_NODE_OPERATOR || - pExpr->pExpr->nodeType == QUERY_NODE_VALUE) { - // for simple column, the result buffer needs to hold at least one element. - pCtx->resDataInfo.interBufSize = pFunct->resSchema.bytes; - } - - pCtx->input.numOfInputCols = pFunct->numOfParams; - pCtx->input.pData = taosMemoryCalloc(pFunct->numOfParams, POINTER_BYTES); - pCtx->input.pColumnDataAgg = taosMemoryCalloc(pFunct->numOfParams, POINTER_BYTES); - - pCtx->pTsOutput = NULL; - pCtx->resDataInfo.bytes = pFunct->resSchema.bytes; - pCtx->resDataInfo.type = pFunct->resSchema.type; - pCtx->order = TSDB_ORDER_ASC; - pCtx->start.key = INT64_MIN; - pCtx->end.key = INT64_MIN; - pCtx->numOfParams = pExpr->base.numOfParams; - pCtx->increase = false; - - pCtx->param = pFunct->pParam; - } - - for (int32_t i = 1; i < numOfOutput; ++i) { - (*rowCellInfoOffset)[i] = - (int32_t)((*rowCellInfoOffset)[i - 1] + sizeof(SResultRowEntryInfo) + pFuncCtx[i - 1].resDataInfo.interBufSize); - } - - setSelectValueColumnInfo(pFuncCtx, numOfOutput); - return pFuncCtx; -} - -static void* destroySqlFunctionCtx(SqlFunctionCtx* pCtx, int32_t numOfOutput) { - if (pCtx == NULL) { - return NULL; - } - - for (int32_t i = 0; i < numOfOutput; ++i) { - for (int32_t j = 0; j < pCtx[i].numOfParams; ++j) { - taosVariantDestroy(&pCtx[i].param[j].param); - } - - taosMemoryFreeClear(pCtx[i].subsidiaries.pCtx); - taosMemoryFree(pCtx[i].input.pData); - taosMemoryFree(pCtx[i].input.pColumnDataAgg); - } - - taosMemoryFreeClear(pCtx); - return NULL; -} - bool isTaskKilled(SExecTaskInfo* pTaskInfo) { // query has been executed more than tsShellActivityTimer, and the retrieve has not arrived // abort current query execution. @@ -1075,36 +814,6 @@ bool isTaskKilled(SExecTaskInfo* pTaskInfo) { void setTaskKilled(SExecTaskInfo* pTaskInfo) { pTaskInfo->code = TSDB_CODE_TSC_QUERY_CANCELLED; } -static bool isCachedLastQuery(STaskAttr* pQueryAttr) { - for (int32_t i = 0; i < pQueryAttr->numOfOutput; ++i) { - int32_t functionId = getExprFunctionId(&pQueryAttr->pExpr1[i]); - // if (functionId == FUNCTION_LAST || functionId == FUNCTION_LAST_DST) { - // continue; - // } - - return false; - } - - int32_t order = TSDB_ORDER_ASC; - if (order != TSDB_ORDER_DESC || !TSWINDOW_IS_EQUAL(pQueryAttr->window, TSWINDOW_DESC_INITIALIZER)) { - return false; - } - - if (pQueryAttr->groupbyColumn) { - return false; - } - - if (pQueryAttr->interval.interval > 0) { - return false; - } - - if (pQueryAttr->numOfFilterCols > 0 || pQueryAttr->havingNum > 0) { - return false; - } - - return true; -} - ///////////////////////////////////////////////////////////////////////////////////////////// // todo refactor : return window void getAlignQueryTimeWindow(SInterval* pInterval, int32_t precision, int64_t key, STimeWindow* win) { @@ -1120,7 +829,9 @@ void getAlignQueryTimeWindow(SInterval* pInterval, int32_t precision, int64_t ke } } +#if 0 static int32_t updateBlockLoadStatus(STaskAttr* pQuery, int32_t status) { + bool hasFirstLastFunc = false; bool hasOtherFunc = false; @@ -1130,7 +841,7 @@ static int32_t updateBlockLoadStatus(STaskAttr* pQuery, int32_t status) { for (int32_t i = 0; i < pQuery->numOfOutput; ++i) { int32_t functionId = getExprFunctionId(&pQuery->pExpr1[i]); -#if 0 + if (functionId == FUNCTION_TS || functionId == FUNCTION_TS_DUMMY || functionId == FUNCTION_TAG || functionId == FUNCTION_TAG_DUMMY) { continue; @@ -1141,7 +852,7 @@ static int32_t updateBlockLoadStatus(STaskAttr* pQuery, int32_t status) { } else { hasOtherFunc = true; } -#endif + } if (hasFirstLastFunc && status == BLK_DATA_NOT_LOAD) { @@ -1155,6 +866,8 @@ static int32_t updateBlockLoadStatus(STaskAttr* pQuery, int32_t status) { return status; } +#endif + // static void updateDataCheckOrder(SQInfo *pQInfo, SQueryTableReq* pQueryMsg, bool stableQuery) { // STaskAttr* pQueryAttr = pQInfo->runtimeEnv.pQueryAttr; // @@ -1266,7 +979,7 @@ static int32_t updateBlockLoadStatus(STaskAttr* pQuery, int32_t status) { // // return filterRangeExecute(pQueryAttr->pFilters, pDataStatis, pQueryAttr->numOfCols, numOfRows); // } - +#if 0 static bool overlapWithTimeWindow(STaskAttr* pQueryAttr, SDataBlockInfo* pBlockInfo) { STimeWindow w = {0}; @@ -1315,55 +1028,7 @@ static bool overlapWithTimeWindow(STaskAttr* pQueryAttr, SDataBlockInfo* pBlockI return false; } - -void doCompactSDataBlock(SSDataBlock* pBlock, int32_t numOfRows, int8_t* p) { - int32_t len = 0; - int32_t start = 0; - for (int32_t j = 0; j < numOfRows; ++j) { - if (p[j] == 1) { - len++; - } else { - if (len > 0) { - int32_t cstart = j - len; - for (int32_t i = 0; i < pBlock->info.numOfCols; ++i) { - SColumnInfoData* pColumnInfoData = taosArrayGet(pBlock->pDataBlock, i); - - int16_t bytes = pColumnInfoData->info.bytes; - memmove(((char*)pColumnInfoData->pData) + start * bytes, pColumnInfoData->pData + cstart * bytes, - len * bytes); - } - - start += len; - len = 0; - } - } - } - - if (len > 0) { - int32_t cstart = numOfRows - len; - for (int32_t i = 0; i < pBlock->info.numOfCols; ++i) { - SColumnInfoData* pColumnInfoData = taosArrayGet(pBlock->pDataBlock, i); - - int16_t bytes = pColumnInfoData->info.bytes; - memmove(pColumnInfoData->pData + start * bytes, pColumnInfoData->pData + cstart * bytes, len * bytes); - } - - start += len; - len = 0; - } - - pBlock->info.rows = start; - pBlock->pBlockAgg = NULL; // clean the block statistics info - - if (start > 0) { - SColumnInfoData* pColumnInfoData = taosArrayGet(pBlock->pDataBlock, 0); - if (pColumnInfoData->info.type == TSDB_DATA_TYPE_TIMESTAMP && - pColumnInfoData->info.colId == PRIMARYKEY_TIMESTAMP_COL_ID) { - pBlock->info.window.skey = *(int64_t*)pColumnInfoData->pData; - pBlock->info.window.ekey = *(int64_t*)(pColumnInfoData->pData + TSDB_KEYSIZE * (start - 1)); - } - } -} +#endif static uint32_t doFilterByBlockTimeWindow(STableScanInfo* pTableScanInfo, SSDataBlock* pBlock) { SqlFunctionCtx* pCtx = pTableScanInfo->pCtx; @@ -1427,17 +1092,17 @@ int32_t loadDataBlockOnDemand(SExecTaskInfo* pTaskInfo, STableScanInfo* pTableSc if (pQueryAttr->pointInterpQuery) { needFilter = chkWindowOutputBufByKey(pRuntimeEnv, pTableScanInfo->pResultRowInfo, &win, masterScan, &pResult, groupId, pTableScanInfo->pCtx, pTableScanInfo->numOfOutput, - pTableScanInfo->rowCellInfoOffset); + pTableScanInfo->rowEntryInfoOffset); } else { if (setResultOutputBufByKey(pRuntimeEnv, pTableScanInfo->pResultRowInfo, pBlock->info.uid, &win, masterScan, &pResult, groupId, pTableScanInfo->pCtx, pTableScanInfo->numOfOutput, - pTableScanInfo->rowCellInfoOffset) != TSDB_CODE_SUCCESS) { + pTableScanInfo->rowEntryInfoOffset) != TSDB_CODE_SUCCESS) { longjmp(pRuntimeEnv->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } } } else if (pQueryAttr->stableQuery && (!pQueryAttr->tsCompQuery) && (!pQueryAttr->diffQuery)) { // stable aggregate, not interval aggregate or normal column aggregate doSetTableGroupOutputBuf(pRuntimeEnv, pTableScanInfo->pResultRowInfo, pTableScanInfo->pCtx, - pTableScanInfo->rowCellInfoOffset, pTableScanInfo->numOfOutput, + pTableScanInfo->rowEntryInfoOffset, pTableScanInfo->numOfOutput, pRuntimeEnv->current->groupIndex); } @@ -1482,7 +1147,7 @@ int32_t loadDataBlockOnDemand(SExecTaskInfo* pTaskInfo, STableScanInfo* pTableSc STimeWindow win = getActiveTimeWindow(pTableScanInfo->pResultRowInfo, k, pQueryAttr); if (setResultOutputBufByKey(pRuntimeEnv, pTableScanInfo->pResultRowInfo, pBlock->info.uid, &win, masterScan, &pResult, groupId, pTableScanInfo->pCtx, pTableScanInfo->numOfOutput, - pTableScanInfo->rowCellInfoOffset) != TSDB_CODE_SUCCESS) { + pTableScanInfo->rowEntryInfoOffset) != TSDB_CODE_SUCCESS) { longjmp(pRuntimeEnv->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } } @@ -1565,14 +1230,13 @@ void initResultRow(SResultRow* pResultRow) { * offset[0] offset[1] offset[2] */ // TODO refactor: some function move away -void setFunctionResultOutput(SOptrBasicInfo* pInfo, SAggSupporter* pSup, int32_t stage, int32_t numOfExprs, - SExecTaskInfo* pTaskInfo) { - SqlFunctionCtx* pCtx = pInfo->pCtx; - SSDataBlock* pDataBlock = pInfo->pRes; - int32_t* rowCellInfoOffset = pInfo->rowCellInfoOffset; +void setFunctionResultOutput(SOperatorInfo *pOperator, SOptrBasicInfo *pInfo, SAggSupporter* pSup, int32_t stage, int32_t numOfExprs) { + SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; + SqlFunctionCtx* pCtx = pOperator->exprSupp.pCtx; + int32_t* rowEntryInfoOffset = pOperator->exprSupp.rowEntryInfoOffset; SResultRowInfo* pResultRowInfo = &pInfo->resultRowInfo; - initResultRowInfo(pResultRowInfo, 16); + initResultRowInfo(pResultRowInfo); int64_t tid = 0; int64_t groupId = 0; @@ -1580,7 +1244,7 @@ void setFunctionResultOutput(SOptrBasicInfo* pInfo, SAggSupporter* pSup, int32_t pTaskInfo, false, pSup); for (int32_t i = 0; i < numOfExprs; ++i) { - struct SResultRowEntryInfo* pEntry = getResultCell(pRow, i, rowCellInfoOffset); + struct SResultRowEntryInfo* pEntry = getResultEntryInfo(pRow, i, rowEntryInfoOffset); cleanupResultRowEntry(pEntry); pCtx[i].resultInfo = pEntry; @@ -1590,42 +1254,6 @@ void setFunctionResultOutput(SOptrBasicInfo* pInfo, SAggSupporter* pSup, int32_t initCtxOutputBuffer(pCtx, numOfExprs); } -void updateOutputBuf(SOptrBasicInfo* pBInfo, int32_t* bufCapacity, int32_t numOfInputRows) { - SSDataBlock* pDataBlock = pBInfo->pRes; - - int32_t newSize = pDataBlock->info.rows + numOfInputRows + 5; // extra output buffer - if ((*bufCapacity) < newSize) { - for (int32_t i = 0; i < pDataBlock->info.numOfCols; ++i) { - SColumnInfoData* pColInfo = taosArrayGet(pDataBlock->pDataBlock, i); - - char* p = taosMemoryRealloc(pColInfo->pData, newSize * pColInfo->info.bytes); - if (p != NULL) { - pColInfo->pData = p; - - // it starts from the tail of the previously generated results. - pBInfo->pCtx[i].pOutput = pColInfo->pData; - (*bufCapacity) = newSize; - } else { - // longjmp - } - } - } - - for (int32_t i = 0; i < pDataBlock->info.numOfCols; ++i) { - SColumnInfoData* pColInfo = taosArrayGet(pDataBlock->pDataBlock, i); - pBInfo->pCtx[i].pOutput = pColInfo->pData + pColInfo->info.bytes * pDataBlock->info.rows; - - // set the correct pointer after the memory buffer reallocated. - int32_t functionId = pBInfo->pCtx[i].functionId; -#if 0 - if (functionId == FUNCTION_TOP || functionId == FUNCTION_BOTTOM || functionId == FUNCTION_DIFF || - functionId == FUNCTION_DERIVATIVE) { - // if (i > 0) pBInfo->pCtx[i].pTsOutput = pBInfo->pCtx[i - 1].pOutput; - } -#endif - } -} - void initCtxOutputBuffer(SqlFunctionCtx* pCtx, int32_t size) { for (int32_t j = 0; j < size; ++j) { struct SResultRowEntryInfo* pResInfo = GET_RES_INFO(&pCtx[j]); @@ -1657,9 +1285,9 @@ void destroyTableQueryInfoImpl(STableQueryInfo* pTableQueryInfo) { // cleanupResultRowInfo(&pTableQueryInfo->resInfo); } -void setResultRowInitCtx(SResultRow* pResult, SqlFunctionCtx* pCtx, int32_t numOfOutput, int32_t* rowCellInfoOffset) { +void setResultRowInitCtx(SResultRow* pResult, SqlFunctionCtx* pCtx, int32_t numOfOutput, int32_t* rowEntryInfoOffset) { for (int32_t i = 0; i < numOfOutput; ++i) { - pCtx[i].resultInfo = getResultCell(pResult, i, rowCellInfoOffset); + pCtx[i].resultInfo = getResultEntryInfo(pResult, i, rowEntryInfoOffset); struct SResultRowEntryInfo* pResInfo = pCtx[i].resultInfo; if (isRowEntryCompleted(pResInfo) && isRowEntryInitialized(pResInfo)) { @@ -1752,14 +1380,12 @@ void extractQualifiedTupleByFilterResult(SSDataBlock* pBlock, const int8_t* rowR } } -void doSetTableGroupOutputBuf(SAggOperatorInfo* pAggInfo, int32_t numOfOutput, uint64_t groupId, - SExecTaskInfo* pTaskInfo) { +void doSetTableGroupOutputBuf(SOperatorInfo* pOperator, SAggOperatorInfo* pAggInfo, int32_t numOfOutput, uint64_t groupId) { // for simple group by query without interval, all the tables belong to one group result. - int64_t uid = 0; - + SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; SResultRowInfo* pResultRowInfo = &pAggInfo->binfo.resultRowInfo; - SqlFunctionCtx* pCtx = pAggInfo->binfo.pCtx; - int32_t* rowCellInfoOffset = pAggInfo->binfo.rowCellInfoOffset; + SqlFunctionCtx* pCtx = pOperator->exprSupp.pCtx; + int32_t* rowEntryInfoOffset = pOperator->exprSupp.rowEntryInfoOffset; SResultRow* pResultRow = doSetResultOutBufByKey(pAggInfo->aggSup.pResultBuf, pResultRowInfo, (char*)&groupId, sizeof(groupId), true, groupId, pTaskInfo, false, &pAggInfo->aggSup); @@ -1777,15 +1403,15 @@ void doSetTableGroupOutputBuf(SAggOperatorInfo* pAggInfo, int32_t numOfOutput, u } } - setResultRowInitCtx(pResultRow, pCtx, numOfOutput, rowCellInfoOffset); + setResultRowInitCtx(pResultRow, pCtx, numOfOutput, rowEntryInfoOffset); } -void setExecutionContext(int32_t numOfOutput, uint64_t groupId, SExecTaskInfo* pTaskInfo, SAggOperatorInfo* pAggInfo) { +void setExecutionContext(SOperatorInfo* pOperator, int32_t numOfOutput, uint64_t groupId, SAggOperatorInfo* pAggInfo) { if (pAggInfo->groupId != INT32_MIN && pAggInfo->groupId == groupId) { return; } - doSetTableGroupOutputBuf(pAggInfo, numOfOutput, groupId, pTaskInfo); + doSetTableGroupOutputBuf(pOperator, pAggInfo, numOfOutput, groupId); // record the current active group id pAggInfo->groupId = groupId; @@ -1793,7 +1419,7 @@ void setExecutionContext(int32_t numOfOutput, uint64_t groupId, SExecTaskInfo* p static void doUpdateNumOfRows(SResultRow* pRow, int32_t numOfExprs, const int32_t* rowCellOffset) { for (int32_t j = 0; j < numOfExprs; ++j) { - struct SResultRowEntryInfo* pResInfo = getResultCell(pRow, j, rowCellOffset); + struct SResultRowEntryInfo* pResInfo = getResultEntryInfo(pRow, j, rowCellOffset); if (!isRowEntryInitialized(pResInfo)) { continue; } @@ -1829,7 +1455,7 @@ int32_t finalizeResultRowIntoResultDataBlock(SDiskbasedBuf* pBuf, SResultRowPosi for (int32_t j = 0; j < numOfExprs; ++j) { int32_t slotId = pExprInfo[j].base.resSchema.slotId; - pCtx[j].resultInfo = getResultCell(pRow, j, rowCellOffset); + pCtx[j].resultInfo = getResultEntryInfo(pRow, j, rowCellOffset); if (pCtx[j].fpSet.finalize) { int32_t code = pCtx[j].fpSet.finalize(&pCtx[j], pBlock); if (TAOS_FAILED(code)) { @@ -1894,7 +1520,7 @@ int32_t doCopyToSDataBlock(SExecTaskInfo* pTaskInfo, SSDataBlock* pBlock, SExprI for (int32_t j = 0; j < numOfExprs; ++j) { int32_t slotId = pExprInfo[j].base.resSchema.slotId; - pCtx[j].resultInfo = getResultCell(pRow, j, rowCellOffset); + pCtx[j].resultInfo = getResultEntryInfo(pRow, j, rowCellOffset); if (pCtx[j].fpSet.finalize) { int32_t code = pCtx[j].fpSet.finalize(&pCtx[j], pBlock); if (TAOS_FAILED(code)) { @@ -1937,16 +1563,16 @@ int32_t doCopyToSDataBlock(SExecTaskInfo* pTaskInfo, SSDataBlock* pBlock, SExprI void doBuildResultDatablock(SOperatorInfo* pOperator, SOptrBasicInfo* pbInfo, SGroupResInfo* pGroupResInfo, SDiskbasedBuf* pBuf) { - SExprInfo* pExprInfo = pOperator->pExpr; - int32_t numOfExprs = pOperator->numOfExprs; + SExprInfo* pExprInfo = pOperator->exprSupp.pExprInfo; + int32_t numOfExprs = pOperator->exprSupp.numOfExprs; SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; - int32_t* rowCellOffset = pbInfo->rowCellInfoOffset; + int32_t* rowCellOffset = pOperator->exprSupp.rowEntryInfoOffset; SSDataBlock* pBlock = pbInfo->pRes; - SqlFunctionCtx* pCtx = pbInfo->pCtx; + SqlFunctionCtx* pCtx = pOperator->exprSupp.pCtx; blockDataCleanup(pBlock); - if (!hashRemainDataInGroupInfo(pGroupResInfo)) { + if (!hasDataInGroupInfo(pGroupResInfo)) { return; } @@ -1956,7 +1582,7 @@ void doBuildResultDatablock(SOperatorInfo* pOperator, SOptrBasicInfo* pbInfo, SG } static void updateNumOfRowsInResultRows(SqlFunctionCtx* pCtx, int32_t numOfOutput, SResultRowInfo* pResultRowInfo, - int32_t* rowCellInfoOffset) { + int32_t* rowEntryInfoOffset) { // update the number of result for each, only update the number of rows for the corresponding window result. // if (QUERY_IS_INTERVAL_QUERY(pQueryAttr)) { // return; @@ -1971,7 +1597,7 @@ static void updateNumOfRowsInResultRows(SqlFunctionCtx* pCtx, int32_t numOfOutpu continue; } - SResultRowEntryInfo* pCell = getResultCell(pResult, j, rowCellInfoOffset); + SResultRowEntryInfo* pCell = getResultEntryInfo(pResult, j, rowEntryInfoOffset); pResult->numOfRows = (uint16_t)(TMAX(pResult->numOfRows, pCell->numOfRes)); } } @@ -2413,33 +2039,7 @@ static int32_t doSendFetchDataRequest(SExchangeInfo* pExchangeInfo, SExecTaskInf return TSDB_CODE_SUCCESS; } -// NOTE: sources columns are more than the destination SSDatablock columns. -void relocateColumnData(SSDataBlock* pBlock, const SArray* pColMatchInfo, SArray* pCols) { - size_t numOfSrcCols = taosArrayGetSize(pCols); - - int32_t i = 0, j = 0; - while (i < numOfSrcCols && j < taosArrayGetSize(pColMatchInfo)) { - SColumnInfoData* p = taosArrayGet(pCols, i); - SColMatchInfo* pmInfo = taosArrayGet(pColMatchInfo, j); - if (!pmInfo->output) { - j++; - continue; - } - - if (p->info.colId == pmInfo->colId) { - SColumnInfoData* pDst = taosArrayGet(pBlock->pDataBlock, pmInfo->targetSlotId); - colDataAssign(pDst, p, pBlock->info.rows); - i++; - j++; - } else if (p->info.colId < pmInfo->colId) { - i++; - } else { - ASSERT(0); - } - } -} - -int32_t setDataBlockFromFetchRsp(SSDataBlock* pRes, SLoadRemoteDataInfo* pLoadInfo, int32_t numOfRows, char* pData, +int32_t extractDataBlockFromFetchRsp(SSDataBlock* pRes, SLoadRemoteDataInfo* pLoadInfo, int32_t numOfRows, char* pData, int32_t compLen, int32_t numOfOutput, int64_t startTs, uint64_t* total, SArray* pColList) { if (pColList == NULL) { // data from other sources @@ -2565,7 +2165,7 @@ static SSDataBlock* concurrentlyLoadRemoteDataImpl(SOperatorInfo* pOperator, SEx } SRetrieveTableRsp* pTableRsp = pDataInfo->pRsp; - code = setDataBlockFromFetchRsp(pExchangeInfo->pResult, pLoadInfo, pTableRsp->numOfRows, pTableRsp->data, + code = extractDataBlockFromFetchRsp(pExchangeInfo->pResult, pLoadInfo, pTableRsp->numOfRows, pTableRsp->data, pTableRsp->compLen, pTableRsp->numOfCols, startTs, &pDataInfo->totalRows, NULL); if (code != 0) { taosMemoryFreeClear(pDataInfo->pRsp); @@ -2680,7 +2280,7 @@ static SSDataBlock* seqLoadRemoteData(SOperatorInfo* pOperator) { SSDataBlock* pRes = pExchangeInfo->pResult; SRetrieveTableRsp* pTableRsp = pDataInfo->pRsp; int32_t code = - setDataBlockFromFetchRsp(pExchangeInfo->pResult, pLoadInfo, pTableRsp->numOfRows, pTableRsp->data, + extractDataBlockFromFetchRsp(pExchangeInfo->pResult, pLoadInfo, pTableRsp->numOfRows, pTableRsp->data, pTableRsp->compLen, pTableRsp->numOfCols, startTs, &pDataInfo->totalRows, NULL); if (pRsp->completed == 1) { @@ -2817,7 +2417,7 @@ SOperatorInfo* createExchangeOperatorInfo(void* pTransporter, SExchangePhysiNode pOperator->blocking = false; pOperator->status = OP_NOT_OPENED; pOperator->info = pInfo; - pOperator->numOfExprs = pInfo->pResult->info.numOfCols; + pOperator->exprSupp.numOfExprs = pInfo->pResult->info.numOfCols; pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = createOperatorFpSet(prepareLoadRemoteData, doLoadRemoteData, NULL, NULL, @@ -2938,7 +2538,7 @@ static bool saveCurrentTuple(char** rowColData, SArray* pColumnList, SSDataBlock static void doMergeImpl(SOperatorInfo* pOperator, int32_t numOfExpr, SSDataBlock* pBlock) { SSortedMergeOperatorInfo* pInfo = pOperator->info; - SqlFunctionCtx* pCtx = pInfo->binfo.pCtx; + SqlFunctionCtx* pCtx = pOperator->exprSupp.pCtx; for (int32_t i = 0; i < pBlock->info.numOfCols; ++i) { // pCtx[i].size = 1; } @@ -2953,8 +2553,8 @@ static void doMergeImpl(SOperatorInfo* pOperator, int32_t numOfExpr, SSDataBlock doMergeResultImpl(pInfo, pCtx, numOfExpr, i); } else { doFinalizeResultImpl(pCtx, numOfExpr); - int32_t numOfRows = getNumOfResult(pInfo->binfo.pCtx, pOperator->numOfExprs, NULL); - // setTagValueForMultipleRows(pCtx, pOperator->numOfExprs, numOfRows); + int32_t numOfRows = getNumOfResult(pOperator->exprSupp.pCtx, pOperator->exprSupp.numOfExprs, NULL); + // setTagValueForMultipleRows(pCtx, pOperator->exprSupp.numOfExprs, numOfRows); // TODO check for available buffer; @@ -3001,16 +2601,16 @@ static SSDataBlock* doMerge(SOperatorInfo* pOperator) { break; } - setInputDataBlock(pOperator, pInfo->binfo.pCtx, pDataBlock, TSDB_ORDER_ASC, MAIN_SCAN, true); + setInputDataBlock(pOperator, pOperator->exprSupp.pCtx, pDataBlock, TSDB_ORDER_ASC, MAIN_SCAN, true); // updateOutputBuf(&pInfo->binfo, &pAggInfo->bufCapacity, pBlock->info.rows * pAggInfo->resultRowFactor, // pOperator->pRuntimeEnv, true); - doMergeImpl(pOperator, pOperator->numOfExprs, pDataBlock); + doMergeImpl(pOperator, pOperator->exprSupp.numOfExprs, pDataBlock); // flush to tuple store, and after all data have been handled, return to upstream node or sink node } - doFinalizeResultImpl(pInfo->binfo.pCtx, pOperator->numOfExprs); - int32_t numOfRows = getNumOfResult(pInfo->binfo.pCtx, pOperator->numOfExprs, NULL); - // setTagValueForMultipleRows(pCtx, pOperator->numOfExprs, numOfRows); + doFinalizeResultImpl(pOperator->exprSupp.pCtx, pOperator->exprSupp.numOfExprs); + int32_t numOfRows = getNumOfResult(pOperator->exprSupp.pCtx, pOperator->exprSupp.numOfExprs, NULL); + // setTagValueForMultipleRows(pCtx, pOperator->exprSupp.numOfExprs, numOfRows); // TODO check for available buffer; @@ -3073,7 +2673,7 @@ static SSDataBlock* doSortedMerge(SOperatorInfo* pOperator) { } int32_t numOfBufPage = pInfo->sortBufSize / pInfo->bufPageSize; - pInfo->pSortHandle = tsortCreateSortHandle(pInfo->pSortInfo, NULL, SORT_MULTISOURCE_MERGE, pInfo->bufPageSize, + pInfo->pSortHandle = tsortCreateSortHandle(pInfo->pSortInfo, SORT_MULTISOURCE_MERGE, pInfo->bufPageSize, numOfBufPage, pInfo->binfo.pRes, "GET_TASKID(pTaskInfo)"); tsortSetFetchRawDataFp(pInfo->pSortHandle, loadNextDataBlock, NULL, NULL); @@ -3151,20 +2751,20 @@ SOperatorInfo* createSortedMergeOperatorInfo(SOperatorInfo** downstream, int32_t goto _error; } - pInfo->binfo.pCtx = createSqlFunctionCtx(pExprInfo, num, &pInfo->binfo.rowCellInfoOffset); - initResultRowInfo(&pInfo->binfo.resultRowInfo, (int32_t)1); + pOperator->exprSupp.pCtx = createSqlFunctionCtx(pExprInfo, num, &pOperator->exprSupp.rowEntryInfoOffset); + initResultRowInfo(&pInfo->binfo.resultRowInfo); - if (pInfo->binfo.pCtx == NULL || pInfo->binfo.pRes == NULL) { + if (pOperator->exprSupp.pCtx == NULL || pInfo->binfo.pRes == NULL) { goto _error; } size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; - int32_t code = doInitAggInfoSup(&pInfo->aggSup, pInfo->binfo.pCtx, num, keyBufSize, pTaskInfo->id.str); + int32_t code = doInitAggInfoSup(&pInfo->aggSup, pOperator->exprSupp.pCtx, num, keyBufSize, pTaskInfo->id.str); if (code != TSDB_CODE_SUCCESS) { goto _error; } - setFunctionResultOutput(&pInfo->binfo, &pInfo->aggSup, MAIN_SCAN, num, pTaskInfo); + setFunctionResultOutput(pOperator, &pInfo->binfo, &pInfo->aggSup, MAIN_SCAN, num); code = initGroupCol(pExprInfo, num, pGroupInfo, pInfo); if (code != TSDB_CODE_SUCCESS) { goto _error; @@ -3183,8 +2783,8 @@ SOperatorInfo* createSortedMergeOperatorInfo(SOperatorInfo** downstream, int32_t pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; pOperator->info = pInfo; - pOperator->numOfExprs = num; - pOperator->pExpr = pExprInfo; + pOperator->exprSupp.numOfExprs = num; + pOperator->exprSupp.pExprInfo = pExprInfo; pOperator->pTaskInfo = pTaskInfo; @@ -3239,6 +2839,7 @@ static int32_t doOpenAggregateOptr(SOperatorInfo* pOperator) { SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; SAggOperatorInfo* pAggInfo = pOperator->info; + SExprSupp* pSup = &pOperator->exprSupp; SOptrBasicInfo* pInfo = &pAggInfo->binfo; SOperatorInfo* downstream = pOperator->pDownstream[0]; @@ -3259,18 +2860,18 @@ static int32_t doOpenAggregateOptr(SOperatorInfo* pOperator) { } // there is an scalar expression that needs to be calculated before apply the group aggregation. - if (pAggInfo->pScalarExprInfo != NULL) { - code = projectApplyFunctions(pAggInfo->pScalarExprInfo, pBlock, pBlock, pAggInfo->pScalarCtx, - pAggInfo->numOfScalarExpr, NULL); + if (pAggInfo->scalarExprSup.pExprInfo != NULL) { + SExprSupp* pSup1 = &pAggInfo->scalarExprSup; + code = projectApplyFunctions(pSup1->pExprInfo, pBlock, pBlock, pSup1->pCtx, pSup1->numOfExprs, NULL); if (code != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, code); } } // the pDataBlock are always the same one, no need to call this again - setExecutionContext(pOperator->numOfExprs, pBlock->info.groupId, pTaskInfo, pAggInfo); - setInputDataBlock(pOperator, pInfo->pCtx, pBlock, order, scanFlag, true); - code = doAggregateImpl(pOperator, 0, pInfo->pCtx); + setExecutionContext(pOperator, pOperator->exprSupp.numOfExprs, pBlock->info.groupId, pAggInfo); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, order, scanFlag, true); + code = doAggregateImpl(pOperator, 0, pSup->pCtx); if (code != 0) { longjmp(pTaskInfo->env, code); } @@ -3316,7 +2917,7 @@ static SSDataBlock* getAggregateResult(SOperatorInfo* pOperator) { blockDataEnsureCapacity(pInfo->pRes, pOperator->resultInfo.capacity); doBuildResultDatablock(pOperator, pInfo, &pAggInfo->groupResInfo, pAggInfo->aggSup.pResultBuf); - if (pInfo->pRes->info.rows == 0 || !hashRemainDataInGroupInfo(&pAggInfo->groupResInfo)) { + if (pInfo->pRes->info.rows == 0 || !hasDataInGroupInfo(&pAggInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); } @@ -3529,6 +3130,7 @@ static SSDataBlock* doProjectOperation(SOperatorInfo* pOperator) { SProjectOperatorInfo* pProjectInfo = pOperator->info; SOptrBasicInfo* pInfo = &pProjectInfo->binfo; + SExprSupp* pSup = &pOperator->exprSupp; SSDataBlock* pRes = pInfo->pRes; blockDataCleanup(pRes); @@ -3546,10 +3148,10 @@ static SSDataBlock* doProjectOperation(SOperatorInfo* pOperator) { setInputDataBlock(pOperator, pInfo->pCtx, pBlock, TSDB_ORDER_ASC); blockDataEnsureCapacity(pInfo->pRes, pBlock->info.rows); - projectApplyFunctions(pOperator->pExpr, pInfo->pRes, pBlock, pInfo->pCtx, pOperator->numOfExprs); + projectApplyFunctions(pOperator->exprSupp.pExprInfo, pInfo->pRes, pBlock, pInfo->pCtx, pOperator->exprSupp.numOfExprs); if (pRes->info.rows >= pProjectInfo->binfo.capacity * 0.8) { - copyTsColoum(pRes, pInfo->pCtx, pOperator->numOfExprs); - resetResultRowEntryResult(pInfo->pCtx, pOperator->numOfExprs); + copyTsColoum(pRes, pInfo->pCtx, pOperator->exprSupp.numOfExprs); + resetResultRowEntryResult(pInfo->pCtx, pOperator->exprSupp.numOfExprs); return pRes; } } @@ -3581,10 +3183,10 @@ static SSDataBlock* doProjectOperation(SOperatorInfo* pOperator) { doFilter(pProjectInfo->pFilterNode, pBlock); - setInputDataBlock(pOperator, pInfo->pCtx, pBlock, order, scanFlag, false); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, order, scanFlag, false); blockDataEnsureCapacity(pInfo->pRes, pInfo->pRes->info.rows + pBlock->info.rows); - code = projectApplyFunctions(pOperator->pExpr, pInfo->pRes, pBlock, pInfo->pCtx, pOperator->numOfExprs, + code = projectApplyFunctions(pSup->pExprInfo, pInfo->pRes, pBlock, pSup->pCtx, pSup->numOfExprs, pProjectInfo->pPseudoColInfo); if (code != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, code); @@ -3735,7 +3337,7 @@ static void destroyOperatorInfo(SOperatorInfo* pOperator) { } if (pOperator->fpSet.closeFn != NULL) { - pOperator->fpSet.closeFn(pOperator->info, pOperator->numOfExprs); + pOperator->fpSet.closeFn(pOperator->info, pOperator->exprSupp.numOfExprs); } if (pOperator->pDownstream != NULL) { @@ -3747,11 +3349,11 @@ static void destroyOperatorInfo(SOperatorInfo* pOperator) { pOperator->numOfDownstream = 0; } - if (pOperator->pExpr != NULL) { - destroyExprInfo(pOperator->pExpr, pOperator->numOfExprs); + if (pOperator->exprSupp.pExprInfo != NULL) { + destroyExprInfo(pOperator->exprSupp.pExprInfo, pOperator->exprSupp.numOfExprs); } - taosMemoryFreeClear(pOperator->pExpr); + taosMemoryFreeClear(pOperator->exprSupp.pExprInfo); taosMemoryFreeClear(pOperator->info); taosMemoryFreeClear(pOperator); } @@ -3801,15 +3403,12 @@ void cleanupAggSup(SAggSupporter* pAggSup) { destroyDiskbasedBuf(pAggSup->pResultBuf); } -int32_t initAggInfo(SOptrBasicInfo* pBasicInfo, SAggSupporter* pAggSup, SExprInfo* pExprInfo, int32_t numOfCols, - SSDataBlock* pResultBlock, size_t keyBufSize, const char* pkey) { - pBasicInfo->pCtx = createSqlFunctionCtx(pExprInfo, numOfCols, &pBasicInfo->rowCellInfoOffset); - pBasicInfo->pRes = pResultBlock; - - doInitAggInfoSup(pAggSup, pBasicInfo->pCtx, numOfCols, keyBufSize, pkey); - +int32_t initAggInfo(SExprSupp* pSup, SAggSupporter* pAggSup, SExprInfo* pExprInfo, int32_t numOfCols, + size_t keyBufSize, const char* pkey) { + initExprSupp(pSup, pExprInfo, numOfCols); + doInitAggInfoSup(pAggSup, pSup->pCtx, numOfCols, keyBufSize, pkey); for (int32_t i = 0; i < numOfCols; ++i) { - pBasicInfo->pCtx[i].pBuf = pAggSup->pResultBuf; + pSup->pCtx[i].pBuf = pAggSup->pResultBuf; } return TSDB_CODE_SUCCESS; @@ -3824,6 +3423,19 @@ void initResultSizeInfo(SOperatorInfo* pOperator, int32_t numOfRows) { } } +void initBasicInfo(SOptrBasicInfo* pInfo, SSDataBlock* pBlock) { + pInfo->pRes = pBlock; + initResultRowInfo(&pInfo->resultRowInfo); +} + +void initExprSupp(SExprSupp* pSup, SExprInfo* pExprInfo, int32_t numOfExpr) { + pSup->pExprInfo = pExprInfo; + pSup->numOfExprs = numOfExpr; + if (pSup->pExprInfo != NULL) { + pSup->pCtx = createSqlFunctionCtx(pExprInfo, numOfExpr, &pSup->rowEntryInfoOffset); + } +} + SOperatorInfo* createAggregateOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols, SSDataBlock* pResultBlock, SExprInfo* pScalarExprInfo, int32_t numOfScalarExpr, SExecTaskInfo* pTaskInfo) { @@ -3837,29 +3449,20 @@ SOperatorInfo* createAggregateOperatorInfo(SOperatorInfo* downstream, SExprInfo* size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; initResultSizeInfo(pOperator, numOfRows); - int32_t code = - initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfCols, pResultBlock, keyBufSize, pTaskInfo->id.str); + int32_t code = initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str); if (code != TSDB_CODE_SUCCESS) { goto _error; } - int32_t numOfGroup = 10; // todo replaced with true value - pInfo->groupId = INT32_MIN; - initResultRowInfo(&pInfo->binfo.resultRowInfo, numOfGroup); + initBasicInfo(&pInfo->binfo, pResultBlock); + initExprSupp(&pInfo->scalarExprSup, pScalarExprInfo, numOfScalarExpr); - pInfo->pScalarExprInfo = pScalarExprInfo; - pInfo->numOfScalarExpr = numOfScalarExpr; - if (pInfo->pScalarExprInfo != NULL) { - pInfo->pScalarCtx = createSqlFunctionCtx(pScalarExprInfo, numOfScalarExpr, &pInfo->rowCellInfoOffset); - } - - pOperator->name = "TableAggregate"; + pInfo->groupId = INT32_MIN; + pOperator->name = "TableAggregate"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_HASH_AGG; - pOperator->blocking = true; - pOperator->status = OP_NOT_OPENED; - pOperator->info = pInfo; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfCols; + pOperator->blocking = true; + pOperator->status = OP_NOT_OPENED; + pOperator->info = pInfo; pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = createOperatorFpSet(doOpenAggregateOptr, getAggregateResult, NULL, NULL, destroyAggOperatorInfo, @@ -3879,24 +3482,39 @@ _error: return NULL; } -void doDestroyBasicInfo(SOptrBasicInfo* pInfo, int32_t numOfOutput) { +static void* destroySqlFunctionCtx(SqlFunctionCtx* pCtx, int32_t numOfOutput) { + if (pCtx == NULL) { + return NULL; + } + + for (int32_t i = 0; i < numOfOutput; ++i) { + for (int32_t j = 0; j < pCtx[i].numOfParams; ++j) { + taosVariantDestroy(&pCtx[i].param[j].param); + } + + taosMemoryFreeClear(pCtx[i].subsidiaries.pCtx); + taosMemoryFree(pCtx[i].input.pData); + taosMemoryFree(pCtx[i].input.pColumnDataAgg); + } + + taosMemoryFreeClear(pCtx); + return NULL; +} + +void cleanupBasicInfo(SOptrBasicInfo* pInfo) { assert(pInfo != NULL); - - destroySqlFunctionCtx(pInfo->pCtx, numOfOutput); - taosMemoryFreeClear(pInfo->rowCellInfoOffset); - cleanupResultRowInfo(&pInfo->resultRowInfo); pInfo->pRes = blockDataDestroy(pInfo->pRes); } void destroyBasicOperatorInfo(void* param, int32_t numOfOutput) { SOptrBasicInfo* pInfo = (SOptrBasicInfo*)param; - doDestroyBasicInfo(pInfo, numOfOutput); + cleanupBasicInfo(pInfo); } void destroyAggOperatorInfo(void* param, int32_t numOfOutput) { SAggOperatorInfo* pInfo = (SAggOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); } void destroySFillOperatorInfo(void* param, int32_t numOfOutput) { @@ -3911,22 +3529,25 @@ static void destroyProjectOperatorInfo(void* param, int32_t numOfOutput) { return; } SProjectOperatorInfo* pInfo = (SProjectOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); cleanupAggSup(&pInfo->aggSup); taosArrayDestroy(pInfo->pPseudoColInfo); } +void cleanupExecSupp(SExprSupp* pSupp) { + destroySqlFunctionCtx(pSupp->pCtx, pSupp->numOfExprs); + destroyExprInfo(pSupp->pExprInfo, pSupp->numOfExprs); + + taosMemoryFree(pSupp->rowEntryInfoOffset); +} + static void destroyIndefinitOperatorInfo(void* param, int32_t numOfOutput) { SIndefOperatorInfo* pInfo = (SIndefOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); taosArrayDestroy(pInfo->pPseudoColInfo); cleanupAggSup(&pInfo->aggSup); - - destroySqlFunctionCtx(pInfo->pScalarCtx, numOfOutput); - destroyExprInfo(pInfo->pScalarExpr, pInfo->numOfScalarExpr); - - taosMemoryFree(pInfo->rowCellInfoOffset); + cleanupExecSupp(&pInfo->scalarSup); } void destroyExchangeOperatorInfo(void* param, int32_t numOfOutput) { @@ -3957,23 +3578,27 @@ static SArray* setRowTsColumnOutputInfo(SqlFunctionCtx* pCtx, int32_t numOfCols) return pList; } -SOperatorInfo* createProjectOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t num, - SSDataBlock* pResBlock, SLimit* pLimit, SLimit* pSlimit, SNode* pCondition, - SExecTaskInfo* pTaskInfo) { +SOperatorInfo* createProjectOperatorInfo(SOperatorInfo* downstream, SProjectPhysiNode* pProjPhyNode, SExecTaskInfo* pTaskInfo) { SProjectOperatorInfo* pInfo = taosMemoryCalloc(1, sizeof(SProjectOperatorInfo)); SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); if (pInfo == NULL || pOperator == NULL) { goto _error; } - pInfo->limit = *pLimit; - pInfo->slimit = *pSlimit; - pInfo->curOffset = pLimit->offset; - pInfo->curSOffset = pSlimit->offset; - pInfo->binfo.pRes = pResBlock; - pInfo->pFilterNode = pCondition; + int32_t numOfCols = 0; + SExprInfo* pExprInfo = createExprInfo(pProjPhyNode->pProjections, NULL, &numOfCols); + + SSDataBlock* pResBlock = createResDataBlock(pProjPhyNode->node.pOutputDataBlockDesc); + SLimit limit = {.limit = pProjPhyNode->limit, .offset = pProjPhyNode->offset}; + SLimit slimit = {.limit = pProjPhyNode->slimit, .offset = pProjPhyNode->soffset}; + + pInfo->limit = limit; + pInfo->slimit = slimit; + pInfo->curOffset = limit.offset; + pInfo->curSOffset = slimit.offset; + pInfo->binfo.pRes = pResBlock; + pInfo->pFilterNode = pProjPhyNode->node.pConditions; - int32_t numOfCols = num; int32_t numOfRows = 4096; size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; @@ -3984,18 +3609,17 @@ SOperatorInfo* createProjectOperatorInfo(SOperatorInfo* downstream, SExprInfo* p } initResultSizeInfo(pOperator, numOfRows); - initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfCols, pResBlock, keyBufSize, pTaskInfo->id.str); - setFunctionResultOutput(&pInfo->binfo, &pInfo->aggSup, MAIN_SCAN, numOfCols, pTaskInfo); + initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str); + initBasicInfo(&pInfo->binfo, pResBlock); + setFunctionResultOutput(pOperator, &pInfo->binfo, &pInfo->aggSup, MAIN_SCAN, numOfCols); - pInfo->pPseudoColInfo = setRowTsColumnOutputInfo(pInfo->binfo.pCtx, numOfCols); - pOperator->name = "ProjectOperator"; + pInfo->pPseudoColInfo = setRowTsColumnOutputInfo(pOperator->exprSupp.pCtx, numOfCols); + pOperator->name = "ProjectOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_PROJECT; - pOperator->blocking = false; - pOperator->status = OP_NOT_OPENED; - pOperator->info = pInfo; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = num; - pOperator->pTaskInfo = pTaskInfo; + pOperator->blocking = false; + pOperator->status = OP_NOT_OPENED; + pOperator->info = pInfo; + pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doProjectOperation, NULL, NULL, destroyProjectOperatorInfo, NULL, NULL, NULL); @@ -4015,6 +3639,7 @@ _error: static SSDataBlock* doApplyIndefinitFunction(SOperatorInfo* pOperator) { SIndefOperatorInfo* pIndefInfo = pOperator->info; SOptrBasicInfo* pInfo = &pIndefInfo->binfo; + SExprSupp* pSup = &pOperator->exprSupp; SSDataBlock* pRes = pInfo->pRes; blockDataCleanup(pRes); @@ -4049,18 +3674,19 @@ static SSDataBlock* doApplyIndefinitFunction(SOperatorInfo* pOperator) { } // there is an scalar expression that needs to be calculated before apply the group aggregation. - if (pIndefInfo->pScalarExpr != NULL) { - code = projectApplyFunctions(pIndefInfo->pScalarExpr, pBlock, pBlock, pIndefInfo->pScalarCtx, - pIndefInfo->numOfScalarExpr, pIndefInfo->pPseudoColInfo); + SExprSupp* pScalarSup = &pIndefInfo->scalarSup; + if (pScalarSup->pExprInfo != NULL) { + code = projectApplyFunctions(pScalarSup->pExprInfo, pBlock, pBlock, pScalarSup->pCtx, pScalarSup->numOfExprs, + pIndefInfo->pPseudoColInfo); if (code != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, code); } } - setInputDataBlock(pOperator, pInfo->pCtx, pBlock, order, scanFlag, false); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, order, scanFlag, false); blockDataEnsureCapacity(pInfo->pRes, pInfo->pRes->info.rows + pBlock->info.rows); - code = projectApplyFunctions(pOperator->pExpr, pInfo->pRes, pBlock, pInfo->pCtx, pOperator->numOfExprs, + code = projectApplyFunctions(pOperator->exprSupp.pExprInfo, pInfo->pRes, pBlock, pSup->pCtx, pOperator->exprSupp.numOfExprs, pIndefInfo->pPseudoColInfo); if (code != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, code); @@ -4085,15 +3711,17 @@ SOperatorInfo* createIndefinitOutputOperatorInfo(SOperatorInfo* downstream, SPhy goto _error; } + SExprSupp* pSup = &pOperator->exprSupp; + SIndefRowsFuncPhysiNode* pPhyNode = (SIndefRowsFuncPhysiNode*)pNode; int32_t numOfExpr = 0; SExprInfo* pExprInfo = createExprInfo(pPhyNode->pVectorFuncs, NULL, &numOfExpr); - int32_t numOfScalarExpr = 0; if (pPhyNode->pExprs != NULL) { - pInfo->pScalarExpr = createExprInfo(pPhyNode->pExprs, NULL, &numOfScalarExpr); - pInfo->pScalarCtx = createSqlFunctionCtx(pInfo->pScalarExpr, numOfScalarExpr, &pInfo->rowCellInfoOffset); + SExprSupp* pSup1 = &pInfo->scalarSup; + pSup1->pExprInfo = createExprInfo(pPhyNode->pExprs, NULL, &pSup1->numOfExprs); + pSup1->pCtx = createSqlFunctionCtx(pSup1->pExprInfo, pSup1->numOfExprs, &pSup1->rowEntryInfoOffset); } SSDataBlock* pResBlock = createResDataBlock(pPhyNode->node.pOutputDataBlockDesc); @@ -4109,21 +3737,22 @@ SOperatorInfo* createIndefinitOutputOperatorInfo(SOperatorInfo* downstream, SPhy } initResultSizeInfo(pOperator, numOfRows); - initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfExpr, pResBlock, keyBufSize, pTaskInfo->id.str); - setFunctionResultOutput(&pInfo->binfo, &pInfo->aggSup, MAIN_SCAN, numOfExpr, pTaskInfo); + initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfExpr, keyBufSize, pTaskInfo->id.str); + initBasicInfo(&pInfo->binfo, pResBlock); + + setFunctionResultOutput(pOperator, &pInfo->binfo, &pInfo->aggSup, MAIN_SCAN, numOfExpr); pInfo->binfo.pRes = pResBlock; - pInfo->numOfScalarExpr = numOfScalarExpr; - pInfo->pPseudoColInfo = setRowTsColumnOutputInfo(pInfo->binfo.pCtx, numOfExpr); + pInfo->pPseudoColInfo = setRowTsColumnOutputInfo(pSup->pCtx, numOfExpr); - pOperator->name = "IndefinitOperator"; + pOperator->name = "IndefinitOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_PROJECT; - pOperator->blocking = false; - pOperator->status = OP_NOT_OPENED; - pOperator->info = pInfo; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfExpr; - pOperator->pTaskInfo = pTaskInfo; + pOperator->blocking = false; + pOperator->status = OP_NOT_OPENED; + pOperator->info = pInfo; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfExpr; + pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doApplyIndefinitFunction, NULL, NULL, destroyIndefinitOperatorInfo, NULL, NULL, NULL); @@ -4162,18 +3791,9 @@ static int32_t initFillInfo(SFillOperatorInfo* pInfo, SExprInfo* pExpr, int32_t } } -SOperatorInfo* createFillOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExpr, int32_t numOfCols, - SInterval* pInterval, STimeWindow* pWindow, SSDataBlock* pResBlock, - int32_t fillType, SNodeListNode* pValueNode, bool multigroupResult, - SExecTaskInfo* pTaskInfo) { - SFillOperatorInfo* pInfo = taosMemoryCalloc(1, sizeof(SFillOperatorInfo)); - SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); - - pInfo->pRes = pResBlock; - pInfo->multigroupResult = multigroupResult; - +static int32_t convertFillType(int32_t mode) { int32_t type = TSDB_FILL_NONE; - switch (fillType) { + switch (mode) { case FILL_MODE_PREV: type = TSDB_FILL_PREV; break; @@ -4196,26 +3816,46 @@ SOperatorInfo* createFillOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExp type = TSDB_FILL_NONE; } + return type; +} + +SOperatorInfo* createFillOperatorInfo(SOperatorInfo* downstream, SFillPhysiNode* pPhyFillNode, bool multigroupResult, + SExecTaskInfo* pTaskInfo) { + SFillOperatorInfo* pInfo = taosMemoryCalloc(1, sizeof(SFillOperatorInfo)); + SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); + if (pInfo == NULL || pOperator == NULL) { + goto _error; + } + + int32_t num = 0; + SSDataBlock* pResBlock = createResDataBlock(pPhyFillNode->node.pOutputDataBlockDesc); + SExprInfo* pExprInfo = createExprInfo(pPhyFillNode->pTargets, NULL, &num); + SInterval* pInterval = &((SIntervalAggOperatorInfo*)downstream->info)->interval; + int32_t type = convertFillType(pPhyFillNode->mode); + SResultInfo* pResultInfo = &pOperator->resultInfo; initResultSizeInfo(pOperator, 4096); - int32_t code = initFillInfo(pInfo, pExpr, numOfCols, pValueNode, *pWindow, pResultInfo->capacity, pTaskInfo->id.str, - pInterval, type); + int32_t code = initFillInfo(pInfo, pExprInfo, num, (SNodeListNode*)pPhyFillNode->pValues, pPhyFillNode->timeRange, + pResultInfo->capacity, pTaskInfo->id.str, pInterval, type); if (code != TSDB_CODE_SUCCESS) { goto _error; } - pOperator->name = "FillOperator"; - pOperator->blocking = false; - pOperator->status = OP_NOT_OPENED; + pInfo->pRes = pResBlock; + pInfo->multigroupResult = multigroupResult; + pOperator->name = "FillOperator"; + pOperator->blocking = false; + pOperator->status = OP_NOT_OPENED; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_FILL; - pOperator->pExpr = pExpr; - pOperator->numOfExprs = numOfCols; - pOperator->info = pInfo; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = num; + pOperator->info = pInfo; + pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doFill, NULL, NULL, destroySFillOperatorInfo, NULL, NULL, NULL); - pOperator->pTaskInfo = pTaskInfo; + code = appendDownstream(pOperator, &downstream, 1); return pOperator; @@ -4225,151 +3865,6 @@ _error: return NULL; } -static SResSchema createResSchema(int32_t type, int32_t bytes, int32_t slotId, int32_t scale, int32_t precision, - const char* name) { - SResSchema s = {0}; - s.scale = scale; - s.type = type; - s.bytes = bytes; - s.slotId = slotId; - s.precision = precision; - strncpy(s.name, name, tListLen(s.name)); - - return s; -} - -static SColumn* createColumn(int32_t blockId, int32_t slotId, int32_t colId, SDataType* pType) { - SColumn* pCol = taosMemoryCalloc(1, sizeof(SColumn)); - if (pCol == NULL) { - terrno = TSDB_CODE_OUT_OF_MEMORY; - return NULL; - } - - pCol->slotId = slotId; - pCol->colId = colId; - pCol->bytes = pType->bytes; - pCol->type = pType->type; - pCol->scale = pType->scale; - pCol->precision = pType->precision; - pCol->dataBlockId = blockId; - - return pCol; -} - -SExprInfo* createExprInfo(SNodeList* pNodeList, SNodeList* pGroupKeys, int32_t* numOfExprs) { - int32_t numOfFuncs = LIST_LENGTH(pNodeList); - int32_t numOfGroupKeys = 0; - if (pGroupKeys != NULL) { - numOfGroupKeys = LIST_LENGTH(pGroupKeys); - } - - *numOfExprs = numOfFuncs + numOfGroupKeys; - SExprInfo* pExprs = taosMemoryCalloc(*numOfExprs, sizeof(SExprInfo)); - - for (int32_t i = 0; i < (*numOfExprs); ++i) { - STargetNode* pTargetNode = NULL; - if (i < numOfFuncs) { - pTargetNode = (STargetNode*)nodesListGetNode(pNodeList, i); - } else { - pTargetNode = (STargetNode*)nodesListGetNode(pGroupKeys, i - numOfFuncs); - } - - SExprInfo* pExp = &pExprs[i]; - - pExp->pExpr = taosMemoryCalloc(1, sizeof(tExprNode)); - pExp->pExpr->_function.num = 1; - pExp->pExpr->_function.functionId = -1; - - int32_t type = nodeType(pTargetNode->pExpr); - // it is a project query, or group by column - if (type == QUERY_NODE_COLUMN) { - pExp->pExpr->nodeType = QUERY_NODE_COLUMN; - SColumnNode* pColNode = (SColumnNode*)pTargetNode->pExpr; - - pExp->base.pParam = taosMemoryCalloc(1, sizeof(SFunctParam)); - pExp->base.numOfParams = 1; - - SDataType* pType = &pColNode->node.resType; - pExp->base.resSchema = createResSchema(pType->type, pType->bytes, pTargetNode->slotId, pType->scale, - pType->precision, pColNode->colName); - pExp->base.pParam[0].pCol = createColumn(pColNode->dataBlockId, pColNode->slotId, pColNode->colId, pType); - pExp->base.pParam[0].type = FUNC_PARAM_TYPE_COLUMN; - } else if (type == QUERY_NODE_VALUE) { - pExp->pExpr->nodeType = QUERY_NODE_VALUE; - SValueNode* pValNode = (SValueNode*)pTargetNode->pExpr; - - pExp->base.pParam = taosMemoryCalloc(1, sizeof(SFunctParam)); - pExp->base.numOfParams = 1; - - SDataType* pType = &pValNode->node.resType; - pExp->base.resSchema = createResSchema(pType->type, pType->bytes, pTargetNode->slotId, pType->scale, - pType->precision, pValNode->node.aliasName); - pExp->base.pParam[0].type = FUNC_PARAM_TYPE_VALUE; - nodesValueNodeToVariant(pValNode, &pExp->base.pParam[0].param); - } else if (type == QUERY_NODE_FUNCTION) { - pExp->pExpr->nodeType = QUERY_NODE_FUNCTION; - SFunctionNode* pFuncNode = (SFunctionNode*)pTargetNode->pExpr; - - SDataType* pType = &pFuncNode->node.resType; - pExp->base.resSchema = createResSchema(pType->type, pType->bytes, pTargetNode->slotId, pType->scale, - pType->precision, pFuncNode->node.aliasName); - - pExp->pExpr->_function.functionId = pFuncNode->funcId; - pExp->pExpr->_function.pFunctNode = pFuncNode; - - strncpy(pExp->pExpr->_function.functionName, pFuncNode->functionName, - tListLen(pExp->pExpr->_function.functionName)); -#if 1 - // todo refactor: add the parameter for tbname function - if (strcmp(pExp->pExpr->_function.functionName, "tbname") == 0) { - pFuncNode->pParameterList = nodesMakeList(); - ASSERT(LIST_LENGTH(pFuncNode->pParameterList) == 0); - SValueNode* res = (SValueNode*)nodesMakeNode(QUERY_NODE_VALUE); - if (NULL == res) { // todo handle error - } else { - res->node.resType = (SDataType){.bytes = sizeof(int64_t), .type = TSDB_DATA_TYPE_BIGINT}; - nodesListAppend(pFuncNode->pParameterList, (SNode*)res); - } - } -#endif - - int32_t numOfParam = LIST_LENGTH(pFuncNode->pParameterList); - - pExp->base.pParam = taosMemoryCalloc(numOfParam, sizeof(SFunctParam)); - pExp->base.numOfParams = numOfParam; - - for (int32_t j = 0; j < numOfParam; ++j) { - SNode* p1 = nodesListGetNode(pFuncNode->pParameterList, j); - if (p1->type == QUERY_NODE_COLUMN) { - SColumnNode* pcn = (SColumnNode*)p1; - - pExp->base.pParam[j].type = FUNC_PARAM_TYPE_COLUMN; - pExp->base.pParam[j].pCol = createColumn(pcn->dataBlockId, pcn->slotId, pcn->colId, &pcn->node.resType); - } else if (p1->type == QUERY_NODE_VALUE) { - SValueNode* pvn = (SValueNode*)p1; - pExp->base.pParam[j].type = FUNC_PARAM_TYPE_VALUE; - nodesValueNodeToVariant(pvn, &pExp->base.pParam[j].param); - } - } - } else if (type == QUERY_NODE_OPERATOR) { - pExp->pExpr->nodeType = QUERY_NODE_OPERATOR; - SOperatorNode* pNode = (SOperatorNode*)pTargetNode->pExpr; - - pExp->base.pParam = taosMemoryCalloc(1, sizeof(SFunctParam)); - pExp->base.numOfParams = 1; - - SDataType* pType = &pNode->node.resType; - pExp->base.resSchema = createResSchema(pType->type, pType->bytes, pTargetNode->slotId, pType->scale, - pType->precision, pNode->node.aliasName); - pExp->pExpr->_optrRoot.pRootNode = pTargetNode->pExpr; - } else { - ASSERT(0); - } - } - - return pExprs; -} - static SExecTaskInfo* createExecTaskInfo(uint64_t queryId, uint64_t taskId, EOPTR_EXEC_MODEL model, char* dbFName) { SExecTaskInfo* pTaskInfo = taosMemoryCalloc(1, sizeof(SExecTaskInfo)); setTaskStatus(pTaskInfo, TASK_NOT_COMPLETED); @@ -4392,8 +3887,6 @@ static tsdbReaderT doCreateDataReader(STableScanPhysiNode* pTableScanNode, SRead static SArray* extractColumnInfo(SNodeList* pNodeList); -static SArray* createSortInfo(SNodeList* pNodeList); - int32_t extractTableSchemaVersion(SReadHandle* pHandle, uint64_t uid, SExecTaskInfo* pTaskInfo) { SMetaReader mr = {0}; metaReaderInit(&mr, pHandle->meta, 0); @@ -4560,7 +4053,7 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo if (pHandle->vnode) { pDataReader = doCreateDataReader(pTableScanNode, pHandle, pTableListInfo, (uint64_t)queryId, taskId, pTagCond); } else { - getTableList(pHandle->meta, pScanPhyNode->tableType, pScanPhyNode->uid, pTableListInfo, pTagCond); + getTableList(pHandle->meta, pScanPhyNode, pTableListInfo, pTagCond); } if (pDataReader == NULL && terrno != 0) { @@ -4587,9 +4080,9 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo } else if (QUERY_NODE_PHYSICAL_PLAN_TAG_SCAN == type) { STagScanPhysiNode* pScanPhyNode = (STagScanPhysiNode*)pPhyNode; - int32_t code = getTableList(pHandle->meta, pScanPhyNode->tableType, pScanPhyNode->uid, pTableListInfo, - pScanPhyNode->node.pConditions); + int32_t code = getTableList(pHandle->meta, pScanPhyNode, pTableListInfo, pScanPhyNode->node.pConditions); if (code != TSDB_CODE_SUCCESS) { + pTaskInfo->code = terrno; return NULL; } @@ -4613,14 +4106,7 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo SOperatorInfo* pOptr = NULL; if (QUERY_NODE_PHYSICAL_PLAN_PROJECT == type) { - SProjectPhysiNode* pProjPhyNode = (SProjectPhysiNode*)pPhyNode; - SExprInfo* pExprInfo = createExprInfo(pProjPhyNode->pProjections, NULL, &num); - - SSDataBlock* pResBlock = createResDataBlock(pPhyNode->pOutputDataBlockDesc); - SLimit limit = {.limit = pProjPhyNode->limit, .offset = pProjPhyNode->offset}; - SLimit slimit = {.limit = pProjPhyNode->slimit, .offset = pProjPhyNode->soffset}; - pOptr = createProjectOperatorInfo(ops[0], pExprInfo, num, pResBlock, &limit, &slimit, - pProjPhyNode->node.pConditions, pTaskInfo); + pOptr = createProjectOperatorInfo(ops[0], (SProjectPhysiNode*)pPhyNode, pTaskInfo); } else if (QUERY_NODE_PHYSICAL_PLAN_HASH_AGG == type) { SAggPhysiNode* pAggNode = (SAggPhysiNode*)pPhyNode; SExprInfo* pExprInfo = createExprInfo(pAggNode->pAggFuncs, pAggNode->pGroupKeys, &num); @@ -4687,21 +4173,7 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo int32_t children = 1; pOptr = createStreamFinalIntervalOperatorInfo(ops[0], pPhyNode, pTaskInfo, children); } else if (QUERY_NODE_PHYSICAL_PLAN_SORT == type) { - SSortPhysiNode* pSortPhyNode = (SSortPhysiNode*)pPhyNode; - - SDataBlockDescNode* pDescNode = pPhyNode->pOutputDataBlockDesc; - - SSDataBlock* pResBlock = createResDataBlock(pDescNode); - SArray* info = createSortInfo(pSortPhyNode->pSortKeys); - - int32_t numOfCols = 0; - SExprInfo* pExprInfo = createExprInfo(pSortPhyNode->pExprs, NULL, &numOfCols); - - int32_t numOfOutputCols = 0; - SArray* pColList = - extractColMatchInfo(pSortPhyNode->pTargets, pDescNode, &numOfOutputCols, pTaskInfo, COL_MATCH_FROM_SLOT_ID); - - pOptr = createSortOperatorInfo(ops[0], pResBlock, info, pExprInfo, numOfCols, pColList, pTaskInfo); + pOptr = createSortOperatorInfo(ops[0], (SSortPhysiNode*)pPhyNode, pTaskInfo); } else if (QUERY_NODE_PHYSICAL_PLAN_MERGE == type) { SMergePhysiNode* pMergePhyNode = (SMergePhysiNode*)pPhyNode; @@ -4711,7 +4183,7 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo SArray* sortInfo = createSortInfo(pMergePhyNode->pMergeKeys); int32_t numOfOutputCols = 0; SArray* pColList = - extractColMatchInfo(pMergePhyNode->pTargets, pDescNode, &numOfOutputCols, pTaskInfo, COL_MATCH_FROM_SLOT_ID); + extractColMatchInfo(pMergePhyNode->pTargets, pDescNode, &numOfOutputCols, COL_MATCH_FROM_SLOT_ID); SPhysiNode* pChildNode = (SPhysiNode*)nodesListGetNode(pPhyNode->pChildren, 0); SSDataBlock* pInputDataBlock = createResDataBlock(pChildNode->pOutputDataBlockDesc); pOptr = createMultiwaySortMergeOperatorInfo(ops, size, pInputDataBlock, pResBlock, sortInfo, pColList, pTaskInfo); @@ -4728,18 +4200,13 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo pOptr = createSessionAggOperatorInfo(ops[0], pExprInfo, num, pResBlock, pSessionNode->gap, tsSlotId, &as, pTaskInfo); } else if (QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION == type) { - SSessionWinodwPhysiNode* pSessionNode = (SSessionWinodwPhysiNode*)pPhyNode; - - STimeWindowAggSupp as = {.waterMark = pSessionNode->window.watermark, - .calTrigger = pSessionNode->window.triggerType}; - - SExprInfo* pExprInfo = createExprInfo(pSessionNode->window.pFuncs, NULL, &num); - SSDataBlock* pResBlock = createResDataBlock(pPhyNode->pOutputDataBlockDesc); - int32_t tsSlotId = ((SColumnNode*)pSessionNode->window.pTspk)->slotId; - - pOptr = createStreamSessionAggOperatorInfo(ops[0], pExprInfo, num, pResBlock, pSessionNode->gap, tsSlotId, &as, - pTaskInfo); - + pOptr = createStreamSessionAggOperatorInfo(ops[0], pPhyNode, pTaskInfo); + } else if (QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION == type) { + int32_t children = 0; + pOptr = createStreamFinalSessionAggOperatorInfo(ops[0], pPhyNode, pTaskInfo, children); + } else if (QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION == type) { + int32_t children = 1; + pOptr = createStreamFinalSessionAggOperatorInfo(ops[0], pPhyNode, pTaskInfo, children); } else if (QUERY_NODE_PHYSICAL_PLAN_PARTITION == type) { pOptr = createPartitionOperatorInfo(ops[0], (SPartitionPhysiNode*)pPhyNode, pTaskInfo); } else if (QUERY_NODE_PHYSICAL_PLAN_MERGE_STATE == type) { @@ -4757,19 +4224,9 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo } else if (QUERY_NODE_PHYSICAL_PLAN_STREAM_STATE == type) { pOptr = createStreamStateAggOperatorInfo(ops[0], pPhyNode, pTaskInfo); } else if (QUERY_NODE_PHYSICAL_PLAN_MERGE_JOIN == type) { - SJoinPhysiNode* pJoinNode = (SJoinPhysiNode*)pPhyNode; - SSDataBlock* pResBlock = createResDataBlock(pPhyNode->pOutputDataBlockDesc); - - SExprInfo* pExprInfo = createExprInfo(pJoinNode->pTargets, NULL, &num); - pOptr = createMergeJoinOperatorInfo(ops, size, pExprInfo, num, pResBlock, pJoinNode->pOnConditions, pTaskInfo); + pOptr = createMergeJoinOperatorInfo(ops, size, (SJoinPhysiNode*)pPhyNode, pTaskInfo); } else if (QUERY_NODE_PHYSICAL_PLAN_FILL == type) { - SFillPhysiNode* pFillNode = (SFillPhysiNode*)pPhyNode; - SSDataBlock* pResBlock = createResDataBlock(pPhyNode->pOutputDataBlockDesc); - SExprInfo* pExprInfo = createExprInfo(pFillNode->pTargets, NULL, &num); - - SInterval* pInterval = &((SIntervalAggOperatorInfo*)ops[0]->info)->interval; - pOptr = createFillOperatorInfo(ops[0], pExprInfo, num, pInterval, &pFillNode->timeRange, pResBlock, pFillNode->mode, - (SNodeListNode*)pFillNode->pValues, false, pTaskInfo); + pOptr = createFillOperatorInfo(ops[0], (SFillPhysiNode*)pPhyNode, false, pTaskInfo); } else if (QUERY_NODE_PHYSICAL_PLAN_INDEF_ROWS_FUNC == type) { pOptr = createIndefinitOutputOperatorInfo(ops[0], pPhyNode, pTaskInfo); } else { @@ -4792,79 +4249,6 @@ int32_t compareTimeWindow(const void* p1, const void* p2, const void* param) { return 0; } -int32_t initQueryTableDataCond(SQueryTableDataCond* pCond, const STableScanPhysiNode* pTableScanNode) { - pCond->loadExternalRows = false; - - pCond->order = pTableScanNode->scanSeq[0] > 0 ? TSDB_ORDER_ASC : TSDB_ORDER_DESC; - pCond->numOfCols = LIST_LENGTH(pTableScanNode->scan.pScanCols); - pCond->colList = taosMemoryCalloc(pCond->numOfCols, sizeof(SColumnInfo)); - if (pCond->colList == NULL) { - terrno = TSDB_CODE_QRY_OUT_OF_MEMORY; - return terrno; - } - - // pCond->twindow = pTableScanNode->scanRange; - // TODO: get it from stable scan node - pCond->numOfTWindows = 1; - pCond->twindows = taosMemoryCalloc(pCond->numOfTWindows, sizeof(STimeWindow)); - pCond->twindows[0] = pTableScanNode->scanRange; - pCond->suid = pTableScanNode->scan.suid; - -#if 1 - // todo work around a problem, remove it later - for (int32_t i = 0; i < pCond->numOfTWindows; ++i) { - if ((pCond->order == TSDB_ORDER_ASC && pCond->twindows[i].skey > pCond->twindows[i].ekey) || - (pCond->order == TSDB_ORDER_DESC && pCond->twindows[i].skey < pCond->twindows[i].ekey)) { - TSWAP(pCond->twindows[i].skey, pCond->twindows[i].ekey); - } - } -#endif - - for (int32_t i = 0; i < pCond->numOfTWindows; ++i) { - if ((pCond->order == TSDB_ORDER_ASC && pCond->twindows[i].skey > pCond->twindows[i].ekey) || - (pCond->order == TSDB_ORDER_DESC && pCond->twindows[i].skey < pCond->twindows[i].ekey)) { - TSWAP(pCond->twindows[i].skey, pCond->twindows[i].ekey); - } - } - taosqsort(pCond->twindows, pCond->numOfTWindows, sizeof(STimeWindow), pCond, compareTimeWindow); - - pCond->type = BLOCK_LOAD_OFFSET_SEQ_ORDER; - // pCond->type = pTableScanNode->scanFlag; - - int32_t j = 0; - for (int32_t i = 0; i < pCond->numOfCols; ++i) { - STargetNode* pNode = (STargetNode*)nodesListGetNode(pTableScanNode->scan.pScanCols, i); - SColumnNode* pColNode = (SColumnNode*)pNode->pExpr; - if (pColNode->colType == COLUMN_TYPE_TAG) { - continue; - } - - pCond->colList[j].type = pColNode->node.resType.type; - pCond->colList[j].bytes = pColNode->node.resType.bytes; - pCond->colList[j].colId = pColNode->colId; - j += 1; - } - - pCond->numOfCols = j; - return TSDB_CODE_SUCCESS; -} - -void clearupQueryTableDataCond(SQueryTableDataCond* pCond) { - taosMemoryFree(pCond->twindows); - taosMemoryFree(pCond->colList); -} - -SColumn extractColumnFromColumnNode(SColumnNode* pColNode) { - SColumn c = {0}; - c.slotId = pColNode->slotId; - c.colId = pColNode->colId; - c.type = pColNode->node.resType.type; - c.bytes = pColNode->node.resType.bytes; - c.scale = pColNode->node.resType.scale; - c.precision = pColNode->node.resType.precision; - return c; -} - SArray* extractColumnInfo(SNodeList* pNodeList) { size_t numOfCols = LIST_LENGTH(pNodeList); SArray* pList = taosArrayInit(numOfCols, sizeof(SColumn)); @@ -4886,7 +4270,7 @@ SArray* extractColumnInfo(SNodeList* pNodeList) { SColumn c = {0}; c.slotId = pNode->slotId; c.colId = pNode->slotId; - c.type = pValNode->node.type; + c.type = pValNode->node.type; c.bytes = pValNode->node.resType.bytes; c.scale = pValNode->node.resType.scale; c.precision = pValNode->node.resType.precision; @@ -4898,146 +4282,10 @@ SArray* extractColumnInfo(SNodeList* pNodeList) { return pList; } -SArray* extractPartitionColInfo(SNodeList* pNodeList) { - if(!pNodeList) { - return NULL; - } - - size_t numOfCols = LIST_LENGTH(pNodeList); - SArray* pList = taosArrayInit(numOfCols, sizeof(SColumn)); - if (pList == NULL) { - terrno = TSDB_CODE_OUT_OF_MEMORY; - return NULL; - } - - for (int32_t i = 0; i < numOfCols; ++i) { - SColumnNode* pColNode = (SColumnNode*)nodesListGetNode(pNodeList, i); - - // todo extract method - SColumn c = {0}; - c.slotId = pColNode->slotId; - c.colId = pColNode->colId; - c.type = pColNode->node.resType.type; - c.bytes = pColNode->node.resType.bytes; - c.precision = pColNode->node.resType.precision; - c.scale = pColNode->node.resType.scale; - - taosArrayPush(pList, &c); - } - - return pList; -} - -SArray* createSortInfo(SNodeList* pNodeList) { - size_t numOfCols = LIST_LENGTH(pNodeList); - SArray* pList = taosArrayInit(numOfCols, sizeof(SBlockOrderInfo)); - if (pList == NULL) { - terrno = TSDB_CODE_OUT_OF_MEMORY; - return pList; - } - - for (int32_t i = 0; i < numOfCols; ++i) { - SOrderByExprNode* pSortKey = (SOrderByExprNode*)nodesListGetNode(pNodeList, i); - SBlockOrderInfo bi = {0}; - bi.order = (pSortKey->order == ORDER_ASC) ? TSDB_ORDER_ASC : TSDB_ORDER_DESC; - bi.nullFirst = (pSortKey->nullOrder == NULL_ORDER_FIRST); - - SColumnNode* pColNode = (SColumnNode*)pSortKey->pExpr; - bi.slotId = pColNode->slotId; - taosArrayPush(pList, &bi); - } - - return pList; -} - -SArray* extractColMatchInfo(SNodeList* pNodeList, SDataBlockDescNode* pOutputNodeList, int32_t* numOfOutputCols, - SExecTaskInfo* pTaskInfo, int32_t type) { - size_t numOfCols = LIST_LENGTH(pNodeList); - SArray* pList = taosArrayInit(numOfCols, sizeof(SColMatchInfo)); - if (pList == NULL) { - terrno = TSDB_CODE_OUT_OF_MEMORY; - return NULL; - } - - for (int32_t i = 0; i < numOfCols; ++i) { - STargetNode* pNode = (STargetNode*)nodesListGetNode(pNodeList, i); - SColumnNode* pColNode = (SColumnNode*)pNode->pExpr; - - SColMatchInfo c = {0}; - c.output = true; - c.colId = pColNode->colId; - c.srcSlotId = pColNode->slotId; - c.matchType = type; - c.targetSlotId = pNode->slotId; - taosArrayPush(pList, &c); - } - - *numOfOutputCols = 0; - int32_t num = LIST_LENGTH(pOutputNodeList->pSlots); - for (int32_t i = 0; i < num; ++i) { - SSlotDescNode* pNode = (SSlotDescNode*)nodesListGetNode(pOutputNodeList->pSlots, i); - - // todo: add reserve flag check - // it is a column reserved for the arithmetic expression calculation - if (pNode->slotId >= numOfCols) { - (*numOfOutputCols) += 1; - continue; - } - - SColMatchInfo* info = taosArrayGet(pList, pNode->slotId); - if (pNode->output) { - (*numOfOutputCols) += 1; - } else { - info->output = false; - } - } - - return pList; -} - -int32_t getTableList(void* metaHandle, int32_t tableType, uint64_t tableUid, STableListInfo* pListInfo, - SNode* pTagCond) { - int32_t code = TSDB_CODE_SUCCESS; - pListInfo->pTableList = taosArrayInit(8, sizeof(STableKeyInfo)); - - if (tableType == TSDB_SUPER_TABLE) { - if (pTagCond) { - SIndexMetaArg metaArg = { - .metaEx = metaHandle, .idx = tsdbGetIdx(metaHandle), .ivtIdx = tsdbGetIvtIdx(metaHandle), .suid = tableUid}; - - SArray* res = taosArrayInit(8, sizeof(uint64_t)); - code = doFilterTag(pTagCond, &metaArg, res); - if (code == TSDB_CODE_INDEX_REBUILDING) { // todo - // doFilter(); - } else if (code != TSDB_CODE_SUCCESS) { - qError("failed to get tableIds, reason: %s, suid: %" PRIu64 "", tstrerror(code), tableUid); - taosArrayDestroy(res); - terrno = code; - return code; - } else { - qDebug("sucess to get tableIds, size: %d, suid: %" PRIu64 "", (int)taosArrayGetSize(res), tableUid); - } - - for (int i = 0; i < taosArrayGetSize(res); i++) { - STableKeyInfo info = {.lastKey = TSKEY_INITIAL_VAL, .uid = *(uint64_t*)taosArrayGet(res, i)}; - taosArrayPush(pListInfo->pTableList, &info); - } - taosArrayDestroy(res); - } else { - code = tsdbGetAllTableList(metaHandle, tableUid, pListInfo->pTableList); - } - } else { // Create one table group. - STableKeyInfo info = {.lastKey = 0, .uid = tableUid}; - taosArrayPush(pListInfo->pTableList, &info); - } - - return code; -} - tsdbReaderT doCreateDataReader(STableScanPhysiNode* pTableScanNode, SReadHandle* pHandle, STableListInfo* pTableListInfo, uint64_t queryId, uint64_t taskId, SNode* pTagCond) { int32_t code = - getTableList(pHandle->meta, pTableScanNode->scan.tableType, pTableScanNode->scan.uid, pTableListInfo, pTagCond); + getTableList(pHandle->meta, &pTableScanNode->scan, pTableListInfo, pTagCond); if (code != TSDB_CODE_SUCCESS) { goto _error; } @@ -5055,7 +4303,7 @@ tsdbReaderT doCreateDataReader(STableScanPhysiNode* pTableScanNode, SReadHandle* } tsdbReaderT* pReader = tsdbReaderOpen(pHandle->vnode, &cond, pTableListInfo, queryId, taskId); - clearupQueryTableDataCond(&cond); + cleanupQueryTableDataCond(&cond); return pReader; @@ -5116,15 +4364,17 @@ int32_t encodeOperator(SOperatorInfo* ops, char** result, int32_t* length) { return TDB_CODE_SUCCESS; } -int32_t decodeOperator(SOperatorInfo* ops, char* result, int32_t length) { +int32_t decodeOperator(SOperatorInfo* ops, const char* result, int32_t length) { int32_t code = TDB_CODE_SUCCESS; if (ops->fpSet.decodeResultRow) { if (result == NULL) { return TSDB_CODE_TSC_INVALID_INPUT; } + ASSERT(length == *(int32_t*)result); - char* data = result + sizeof(int32_t); - code = ops->fpSet.decodeResultRow(ops, data); + + const char* data = result + sizeof(int32_t); + code = ops->fpSet.decodeResultRow(ops, (char*) data); if (code != TDB_CODE_SUCCESS) { return code; } diff --git a/source/libs/executor/src/groupoperator.c b/source/libs/executor/src/groupoperator.c index 4fc25688c4..527f4520bf 100644 --- a/source/libs/executor/src/groupoperator.c +++ b/source/libs/executor/src/groupoperator.c @@ -26,15 +26,18 @@ #include "ttypes.h" #include "executorInt.h" +static void* getCurrentDataGroupInfo(const SPartitionOperatorInfo* pInfo, SDataGroupInfo** pGroupInfo, int32_t len); static int32_t* setupColumnOffset(const SSDataBlock* pBlock, int32_t rowCapacity); -static void* getCurrentDataGroupInfo(const SPartitionOperatorInfo* pInfo, SDataGroupInfo** pGroupInfo, int32_t len); +static int32_t setGroupResultOutputBuf(SOperatorInfo* pOperator, SOptrBasicInfo* binfo, int32_t numOfCols, char* pData, int16_t bytes, + int32_t groupId, SDiskbasedBuf* pBuf, SAggSupporter* pAggSup); static void destroyGroupOperatorInfo(void* param, int32_t numOfOutput) { SGroupbyOperatorInfo* pInfo = (SGroupbyOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); taosMemoryFreeClear(pInfo->keyBuf); taosArrayDestroy(pInfo->pGroupCols); taosArrayDestroy(pInfo->pGroupColVals); + cleanupExecSupp(&pInfo->scalarSup); } static int32_t initGroupOptrInfo(SArray** pGroupColVals, int32_t* keyLen, char** keyBuf, const SArray* pGroupColList) { @@ -214,7 +217,7 @@ static void doHashGroupbyAgg(SOperatorInfo* pOperator, SSDataBlock* pBlock) { SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; SGroupbyOperatorInfo* pInfo = pOperator->info; - SqlFunctionCtx* pCtx = pInfo->binfo.pCtx; + SqlFunctionCtx* pCtx = pOperator->exprSupp.pCtx; int32_t numOfGroupCols = taosArrayGetSize(pInfo->pGroupCols); // if (type == TSDB_DATA_TYPE_FLOAT || type == TSDB_DATA_TYPE_DOUBLE) { // qError("QInfo:0x%"PRIx64" group by not supported on double/float columns, abort", GET_TASKID(pRuntimeEnv)); @@ -248,16 +251,16 @@ static void doHashGroupbyAgg(SOperatorInfo* pOperator, SSDataBlock* pBlock) { } len = buildGroupKeys(pInfo->keyBuf, pInfo->pGroupColVals); - int32_t ret = setGroupResultOutputBuf(&(pInfo->binfo), pOperator->numOfExprs, pInfo->keyBuf, TSDB_DATA_TYPE_VARCHAR, len, 0, pInfo->aggSup.pResultBuf, pTaskInfo, &pInfo->aggSup); + int32_t ret = setGroupResultOutputBuf(pOperator, &(pInfo->binfo), pOperator->exprSupp.numOfExprs, pInfo->keyBuf, len, 0, pInfo->aggSup.pResultBuf, &pInfo->aggSup); if (ret != TSDB_CODE_SUCCESS) { // null data, too many state code longjmp(pTaskInfo->env, TSDB_CODE_QRY_APP_ERROR); } int32_t rowIndex = j - num; - doApplyFunctions(pTaskInfo, pCtx, &w, NULL, rowIndex, num, NULL, pBlock->info.rows, pOperator->numOfExprs, TSDB_ORDER_ASC); + doApplyFunctions(pTaskInfo, pCtx, &w, NULL, rowIndex, num, NULL, pBlock->info.rows, pOperator->exprSupp.numOfExprs, TSDB_ORDER_ASC); // assign the group keys or user input constant values if required - doAssignGroupKeys(pCtx, pOperator->numOfExprs, pBlock->info.rows, rowIndex); + doAssignGroupKeys(pCtx, pOperator->exprSupp.numOfExprs, pBlock->info.rows, rowIndex); recordNewGroupKeys(pInfo->pGroupCols, pInfo->pGroupColVals, pBlock, j); num = 1; } @@ -265,15 +268,15 @@ static void doHashGroupbyAgg(SOperatorInfo* pOperator, SSDataBlock* pBlock) { if (num > 0) { len = buildGroupKeys(pInfo->keyBuf, pInfo->pGroupColVals); int32_t ret = - setGroupResultOutputBuf(&(pInfo->binfo), pOperator->numOfExprs, pInfo->keyBuf, TSDB_DATA_TYPE_VARCHAR, len, - 0, pInfo->aggSup.pResultBuf, pTaskInfo, &pInfo->aggSup); + setGroupResultOutputBuf(pOperator, &(pInfo->binfo), pOperator->exprSupp.numOfExprs, pInfo->keyBuf, len, + 0, pInfo->aggSup.pResultBuf, &pInfo->aggSup); if (ret != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_APP_ERROR); } int32_t rowIndex = pBlock->info.rows - num; - doApplyFunctions(pTaskInfo, pCtx, &w, NULL, rowIndex, num, NULL, pBlock->info.rows, pOperator->numOfExprs, TSDB_ORDER_ASC); - doAssignGroupKeys(pCtx, pOperator->numOfExprs, pBlock->info.rows, rowIndex); + doApplyFunctions(pTaskInfo, pCtx, &w, NULL, rowIndex, num, NULL, pBlock->info.rows, pOperator->exprSupp.numOfExprs, TSDB_ORDER_ASC); + doAssignGroupKeys(pCtx, pOperator->exprSupp.numOfExprs, pBlock->info.rows, rowIndex); } } @@ -291,7 +294,7 @@ static SSDataBlock* hashGroupbyAggregate(SOperatorInfo* pOperator) { doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); size_t rows = pRes->info.rows; - if (rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); } @@ -317,11 +320,11 @@ static SSDataBlock* hashGroupbyAggregate(SOperatorInfo* pOperator) { } // the pDataBlock are always the same one, no need to call this again - setInputDataBlock(pOperator, pInfo->binfo.pCtx, pBlock, order, scanFlag, true); + setInputDataBlock(pOperator, pOperator->exprSupp.pCtx, pBlock, order, scanFlag, true); // there is an scalar expression that needs to be calculated right before apply the group aggregation. - if (pInfo->scalarSup.pScalarExprInfo != NULL) { - pTaskInfo->code = projectApplyFunctions(pInfo->scalarSup.pScalarExprInfo, pBlock, pBlock, pInfo->scalarSup.pScalarFuncCtx, pInfo->scalarSup.numOfScalarExpr, NULL); + if (pInfo->scalarSup.pExprInfo != NULL) { + pTaskInfo->code = projectApplyFunctions(pInfo->scalarSup.pExprInfo, pBlock, pBlock, pInfo->scalarSup.pCtx, pInfo->scalarSup.numOfExprs, NULL); if (pTaskInfo->code != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, pTaskInfo->code); } @@ -355,7 +358,7 @@ static SSDataBlock* hashGroupbyAggregate(SOperatorInfo* pOperator) { doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); doFilter(pInfo->pCondition, pRes); - bool hasRemain = hashRemainDataInGroupInfo(&pInfo->groupResInfo); + bool hasRemain = hasDataInGroupInfo(&pInfo->groupResInfo); if (!hasRemain) { doSetOperatorCompleted(pOperator); break; @@ -384,9 +387,9 @@ SOperatorInfo* createGroupOperatorInfo(SOperatorInfo* downstream, SExprInfo* pEx pInfo->pGroupCols = pGroupColList; pInfo->pCondition = pCondition; - pInfo->scalarSup.pScalarExprInfo = pScalarExprInfo; - pInfo->scalarSup.numOfScalarExpr = numOfScalarExpr; - pInfo->scalarSup.pScalarFuncCtx = createSqlFunctionCtx(pScalarExprInfo, numOfScalarExpr, &pInfo->scalarSup.rowCellInfoOffset); + pInfo->scalarSup.pExprInfo = pScalarExprInfo; + pInfo->scalarSup.numOfExprs = numOfScalarExpr; + pInfo->scalarSup.pCtx = createSqlFunctionCtx(pScalarExprInfo, numOfScalarExpr, &pInfo->scalarSup.rowEntryInfoOffset); int32_t code = initGroupOptrInfo(&pInfo->pGroupColVals, &pInfo->groupKeyLen, &pInfo->keyBuf, pGroupColList); if (code != TSDB_CODE_SUCCESS) { @@ -394,15 +397,16 @@ SOperatorInfo* createGroupOperatorInfo(SOperatorInfo* downstream, SExprInfo* pEx } initResultSizeInfo(pOperator, 4096); - initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfCols, pResultBlock, pInfo->groupKeyLen, pTaskInfo->id.str); - initResultRowInfo(&pInfo->binfo.resultRowInfo, 8); + initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfCols, pInfo->groupKeyLen, pTaskInfo->id.str); + initBasicInfo(&pInfo->binfo, pResultBlock); + initResultRowInfo(&pInfo->binfo.resultRowInfo); pOperator->name = "GroupbyAggOperator"; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; // pOperator->operatorType = OP_Groupby; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->info = pInfo; pOperator->pTaskInfo = pTaskInfo; @@ -439,9 +443,9 @@ static void doHashPartition(SOperatorInfo* pOperator, SSDataBlock* pBlock) { // group id - size_t numOfCols = pOperator->numOfExprs; + size_t numOfCols = pOperator->exprSupp.numOfExprs; for(int32_t i = 0; i < numOfCols; ++i) { - SExprInfo* pExpr = &pOperator->pExpr[i]; + SExprInfo* pExpr = &pOperator->exprSupp.pExprInfo[i]; int32_t slotId = pExpr->base.pParam[0].pCol->slotId; SColumnInfoData* pColInfoData = taosArrayGet(pBlock->pDataBlock, slotId); @@ -643,8 +647,8 @@ static SSDataBlock* hashPartition(SOperatorInfo* pOperator) { } // there is an scalar expression that needs to be calculated right before apply the group aggregation. - if (pInfo->scalarSupp.pScalarExprInfo != NULL) { - pTaskInfo->code = projectApplyFunctions(pInfo->scalarSupp.pScalarExprInfo, pBlock, pBlock, pInfo->scalarSupp.pScalarFuncCtx, pInfo->scalarSupp.numOfScalarExpr, NULL); + if (pInfo->scalarSup.pExprInfo != NULL) { + pTaskInfo->code = projectApplyFunctions(pInfo->scalarSup.pExprInfo, pBlock, pBlock, pInfo->scalarSup.pCtx, pInfo->scalarSup.numOfExprs, NULL); if (pTaskInfo->code != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, pTaskInfo->code); } @@ -662,16 +666,20 @@ static SSDataBlock* hashPartition(SOperatorInfo* pOperator) { static void destroyPartitionOperatorInfo(void* param, int32_t numOfOutput) { SPartitionOperatorInfo* pInfo = (SPartitionOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); taosArrayDestroy(pInfo->pGroupCols); + for(int i = 0; i < taosArrayGetSize(pInfo->pGroupColVals); i++){ SGroupKeys key = *(SGroupKeys*)taosArrayGet(pInfo->pGroupColVals, i); taosMemoryFree(key.pData); } + taosArrayDestroy(pInfo->pGroupColVals); taosMemoryFree(pInfo->keyBuf); taosHashCleanup(pInfo->pGroupSet); taosMemoryFree(pInfo->columnOffset); + + cleanupExecSupp(&pInfo->scalarSup); } SOperatorInfo* createPartitionOperatorInfo(SOperatorInfo* downstream, SPartitionPhysiNode* pPartNode, SExecTaskInfo* pTaskInfo) { @@ -689,10 +697,10 @@ SOperatorInfo* createPartitionOperatorInfo(SOperatorInfo* downstream, SPartition pInfo->pGroupCols = extractPartitionColInfo(pPartNode->pPartitionKeys); if (pPartNode->pExprs != NULL) { - pInfo->scalarSupp.numOfScalarExpr = 0; - pInfo->scalarSupp.pScalarExprInfo = createExprInfo(pPartNode->pExprs, NULL, &pInfo->scalarSupp.numOfScalarExpr); - pInfo->scalarSupp.pScalarFuncCtx = createSqlFunctionCtx( - pInfo->scalarSupp.pScalarExprInfo, pInfo->scalarSupp.numOfScalarExpr, &pInfo->scalarSupp.rowCellInfoOffset); + pInfo->scalarSup.numOfExprs = 0; + pInfo->scalarSup.pExprInfo = createExprInfo(pPartNode->pExprs, NULL, &pInfo->scalarSup.numOfExprs); + pInfo->scalarSup.pCtx = createSqlFunctionCtx( + pInfo->scalarSup.pExprInfo, pInfo->scalarSup.numOfExprs, &pInfo->scalarSup.rowEntryInfoOffset); } _hash_fn_t hashFn = taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY); @@ -722,8 +730,8 @@ SOperatorInfo* createPartitionOperatorInfo(SOperatorInfo* downstream, SPartition pOperator->status = OP_NOT_OPENED; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_PARTITION; pInfo->binfo.pRes = pResBlock; - pOperator->numOfExprs = numOfCols; - pOperator->pExpr = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfCols; + pOperator->exprSupp.pExprInfo = pExprInfo; pOperator->info = pInfo; pOperator->pTaskInfo = pTaskInfo; @@ -738,4 +746,18 @@ SOperatorInfo* createPartitionOperatorInfo(SOperatorInfo* downstream, SPartition taosMemoryFreeClear(pInfo); taosMemoryFreeClear(pOperator); return NULL; +} + +int32_t setGroupResultOutputBuf(SOperatorInfo* pOperator, SOptrBasicInfo* binfo, int32_t numOfCols, char* pData, int16_t bytes, + int32_t groupId, SDiskbasedBuf* pBuf, SAggSupporter* pAggSup) { + SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; + SResultRowInfo* pResultRowInfo = &binfo->resultRowInfo; + SqlFunctionCtx* pCtx = pOperator->exprSupp.pCtx; + + SResultRow* pResultRow = + doSetResultOutBufByKey(pBuf, pResultRowInfo, (char*)pData, bytes, true, groupId, pTaskInfo, false, pAggSup); + assert(pResultRow != NULL); + + setResultRowInitCtx(pResultRow, pCtx, numOfCols, pOperator->exprSupp.rowEntryInfoOffset); + return TSDB_CODE_SUCCESS; } \ No newline at end of file diff --git a/source/libs/executor/src/joinoperator.c b/source/libs/executor/src/joinoperator.c index 7c8ab244a1..6fbda77808 100644 --- a/source/libs/executor/src/joinoperator.c +++ b/source/libs/executor/src/joinoperator.c @@ -28,27 +28,32 @@ static SSDataBlock* doMergeJoin(struct SOperatorInfo* pOperator); static void destroyMergeJoinOperator(void* param, int32_t numOfOutput); static void extractTimeCondition(SJoinOperatorInfo* Info, SLogicConditionNode* pLogicConditionNode); -SOperatorInfo* createMergeJoinOperatorInfo(SOperatorInfo** pDownstream, int32_t numOfDownstream, SExprInfo* pExprInfo, - int32_t numOfCols, SSDataBlock* pResBlock, SNode* pOnCondition, - SExecTaskInfo* pTaskInfo) { +SOperatorInfo* createMergeJoinOperatorInfo(SOperatorInfo** pDownstream, int32_t numOfDownstream, SJoinPhysiNode* pJoinNode, + SExecTaskInfo* pTaskInfo) { SJoinOperatorInfo* pInfo = taosMemoryCalloc(1, sizeof(SJoinOperatorInfo)); SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); if (pOperator == NULL || pInfo == NULL) { goto _error; } + SSDataBlock* pResBlock = createResDataBlock(pJoinNode->node.pOutputDataBlockDesc); + + int32_t numOfCols = 0; + SExprInfo* pExprInfo = createExprInfo(pJoinNode->pTargets, NULL, &numOfCols); + initResultSizeInfo(pOperator, 4096); - pInfo->pRes = pResBlock; - pOperator->name = "MergeJoinOperator"; + pInfo->pRes = pResBlock; + pOperator->name = "MergeJoinOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_MERGE_JOIN; - pOperator->blocking = false; - pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfCols; - pOperator->info = pInfo; - pOperator->pTaskInfo = pTaskInfo; + pOperator->blocking = false; + pOperator->status = OP_NOT_OPENED; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfCols; + pOperator->info = pInfo; + pOperator->pTaskInfo = pTaskInfo; + SNode* pOnCondition = pJoinNode->pOnConditions; if (nodeType(pOnCondition) == QUERY_NODE_OPERATOR) { SOperatorNode* pNode = (SOperatorNode*)pOnCondition; setJoinColumnInfo(&pInfo->leftCol, (SColumnNode*)pNode->pLeft); @@ -127,10 +132,10 @@ SSDataBlock* doMergeJoin(struct SOperatorInfo* pOperator) { // only the timestamp match support for ordinary table ASSERT(pLeftCol->info.type == TSDB_DATA_TYPE_TIMESTAMP); if (*(int64_t*)pLeftVal == *(int64_t*)pRightVal) { - for (int32_t i = 0; i < pOperator->numOfExprs; ++i) { + for (int32_t i = 0; i < pOperator->exprSupp.numOfExprs; ++i) { SColumnInfoData* pDst = taosArrayGet(pRes->pDataBlock, i); - SExprInfo* pExprInfo = &pOperator->pExpr[i]; + SExprInfo* pExprInfo = &pOperator->exprSupp.pExprInfo[i]; int32_t blockId = pExprInfo->base.pParam[0].pCol->dataBlockId; int32_t slotId = pExprInfo->base.pParam[0].pCol->slotId; diff --git a/source/libs/executor/src/scanoperator.c b/source/libs/executor/src/scanoperator.c index 2871145dcc..812272c559 100644 --- a/source/libs/executor/src/scanoperator.c +++ b/source/libs/executor/src/scanoperator.c @@ -261,9 +261,9 @@ static int32_t loadDataBlock(SOperatorInfo* pOperator, STableScanInfo* pTableSca relocateColumnData(pBlock, pTableScanInfo->pColMatchInfo, pCols); // currently only the tbname pseudo column - if (pTableScanInfo->numOfPseudoExpr > 0) { - addTagPseudoColumnData(&pTableScanInfo->readHandle, pTableScanInfo->pPseudoExpr, pTableScanInfo->numOfPseudoExpr, - pBlock); + if (pTableScanInfo->pseudoSup.numOfExprs > 0) { + SExprSupp* pSup = &pTableScanInfo->pseudoSup; + addTagPseudoColumnData(&pTableScanInfo->readHandle, pSup->pExprInfo, pSup->numOfExprs, pBlock); } int64_t st = taosGetTimestampMs(); @@ -496,18 +496,6 @@ static SSDataBlock* doTableScan(SOperatorInfo* pOperator) { return NULL; } -SInterval extractIntervalInfo(const STableScanPhysiNode* pTableScanNode) { - SInterval interval = { - .interval = pTableScanNode->interval, - .sliding = pTableScanNode->sliding, - .intervalUnit = pTableScanNode->intervalUnit, - .slidingUnit = pTableScanNode->slidingUnit, - .offset = pTableScanNode->offset, - }; - - return interval; -} - static int32_t getTableScannerExecInfo(struct SOperatorInfo* pOptr, void** pOptrExplain, uint32_t* len) { SFileBlockLoadRecorder* pRecorder = taosMemoryCalloc(1, sizeof(SFileBlockLoadRecorder)); STableScanInfo* pTableScanInfo = pOptr->info; @@ -520,7 +508,7 @@ static int32_t getTableScannerExecInfo(struct SOperatorInfo* pOptr, void** pOptr static void destroyTableScanOperatorInfo(void* param, int32_t numOfOutput) { STableScanInfo* pTableScanInfo = (STableScanInfo*)param; blockDataDestroy(pTableScanInfo->pResBlock); - clearupQueryTableDataCond(&pTableScanInfo->cond); + cleanupQueryTableDataCond(&pTableScanInfo->cond); tsdbCleanupReadHandle(pTableScanInfo->dataReader); @@ -542,8 +530,7 @@ SOperatorInfo* createTableScanOperatorInfo(STableScanPhysiNode* pTableScanNode, SDataBlockDescNode* pDescNode = pTableScanNode->scan.node.pOutputDataBlockDesc; int32_t numOfCols = 0; - SArray* pColList = - extractColMatchInfo(pTableScanNode->scan.pScanCols, pDescNode, &numOfCols, pTaskInfo, COL_MATCH_FROM_COL_ID); + SArray* pColList = extractColMatchInfo(pTableScanNode->scan.pScanCols, pDescNode, &numOfCols, COL_MATCH_FROM_COL_ID); int32_t code = initQueryTableDataCond(&pInfo->cond, pTableScanNode); if (code != TSDB_CODE_SUCCESS) { @@ -551,8 +538,9 @@ SOperatorInfo* createTableScanOperatorInfo(STableScanPhysiNode* pTableScanNode, } if (pTableScanNode->scan.pScanPseudoCols != NULL) { - pInfo->pPseudoExpr = createExprInfo(pTableScanNode->scan.pScanPseudoCols, NULL, &pInfo->numOfPseudoExpr); - pInfo->pPseudoCtx = createSqlFunctionCtx(pInfo->pPseudoExpr, pInfo->numOfPseudoExpr, &pInfo->rowCellInfoOffset); + SExprSupp* pSup = &pInfo->pseudoSup; + pSup->pExprInfo = createExprInfo(pTableScanNode->scan.pScanPseudoCols, NULL, &pSup->numOfExprs); + pSup->pCtx = createSqlFunctionCtx(pSup->pExprInfo, pSup->numOfExprs, &pSup->rowEntryInfoOffset); } pInfo->scanInfo = (SScanInfo){.numOfAsc = pTableScanNode->scanSeq[0], .numOfDesc = pTableScanNode->scanSeq[1]}; @@ -576,7 +564,7 @@ SOperatorInfo* createTableScanOperatorInfo(STableScanPhysiNode* pTableScanNode, pOperator->blocking = false; pOperator->status = OP_NOT_OPENED; pOperator->info = pInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doTableScan, NULL, NULL, destroyTableScanOperatorInfo, @@ -1066,8 +1054,7 @@ SOperatorInfo* createStreamScanOperatorInfo(void* pDataReader, SReadHandle* pHan STableScanInfo* pSTInfo = (STableScanInfo*)pTableScanDummy->info; int32_t numOfCols = 0; - pInfo->pColMatchInfo = - extractColMatchInfo(pScanPhyNode->pScanCols, pDescNode, &numOfCols, pTaskInfo, COL_MATCH_FROM_COL_ID); + pInfo->pColMatchInfo = extractColMatchInfo(pScanPhyNode->pScanCols, pDescNode, &numOfCols, COL_MATCH_FROM_COL_ID); int32_t numOfOutput = taosArrayGetSize(pInfo->pColMatchInfo); SArray* pColIds = taosArrayInit(numOfOutput, sizeof(int16_t)); @@ -1131,7 +1118,7 @@ SOperatorInfo* createStreamScanOperatorInfo(void* pDataReader, SReadHandle* pHan pOperator->blocking = false; pOperator->status = OP_NOT_OPENED; pOperator->info = pInfo; - pOperator->numOfExprs = pInfo->pRes->info.numOfCols; + pOperator->exprSupp.numOfExprs = pInfo->pRes->info.numOfCols; pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = @@ -1525,8 +1512,8 @@ static SSDataBlock* doSysTableScan(SOperatorInfo* pOperator) { } } - setDataBlockFromFetchRsp(pInfo->pRes, &pInfo->loadInfo, pRsp->numOfRows, pRsp->data, pRsp->compLen, - pOperator->numOfExprs, startTs, NULL, pInfo->scanCols); + extractDataBlockFromFetchRsp(pInfo->pRes, &pInfo->loadInfo, pRsp->numOfRows, pRsp->data, pRsp->compLen, + pOperator->exprSupp.numOfExprs, startTs, NULL, pInfo->scanCols); // todo log the filter info doFilterResult(pInfo); @@ -1614,7 +1601,7 @@ SOperatorInfo* createSysTableScanOperatorInfo(void* readHandle, SSystemTableScan SSDataBlock* pResBlock = createResDataBlock(pDescNode); int32_t num = 0; - SArray* colList = extractColMatchInfo(pScanNode->pScanCols, pDescNode, &num, pTaskInfo, COL_MATCH_FROM_COL_ID); + SArray* colList = extractColMatchInfo(pScanNode->pScanCols, pDescNode, &num, COL_MATCH_FROM_COL_ID); pInfo->accountId = pScanPhyNode->accountId; pInfo->showRewrite = pScanPhyNode->showRewrite; @@ -1641,7 +1628,7 @@ SOperatorInfo* createSysTableScanOperatorInfo(void* readHandle, SSystemTableScan pOperator->blocking = false; pOperator->status = OP_NOT_OPENED; pOperator->info = pInfo; - pOperator->numOfExprs = pResBlock->info.numOfCols; + pOperator->exprSupp.numOfExprs = pResBlock->info.numOfCols; pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = @@ -1672,11 +1659,11 @@ static SSDataBlock* doTagScan(SOperatorInfo* pOperator) { int32_t count = 0; SArray* pa = GET_TABLEGROUP(pRuntimeEnv, 0); - int32_t functionId = getExprFunctionId(&pOperator->pExpr[0]); + int32_t functionId = getExprFunctionId(&pOperator->exprSupp.pExprInfo[0]); if (functionId == FUNCTION_TID_TAG) { // return the tags & table Id assert(pQueryAttr->numOfOutput == 1); - SExprInfo* pExprInfo = &pOperator->pExpr[0]; + SExprInfo* pExprInfo = &pOperator->exprSupp.pExprInfo[0]; int32_t rsize = pExprInfo->base.resSchema.bytes; count = 0; @@ -1739,7 +1726,7 @@ static SSDataBlock* doTagScan(SOperatorInfo* pOperator) { #endif STagScanInfo* pInfo = pOperator->info; - SExprInfo* pExprInfo = &pOperator->pExpr[0]; + SExprInfo* pExprInfo = &pOperator->exprSupp.pExprInfo[0]; SSDataBlock* pRes = pInfo->pRes; int32_t size = taosArrayGetSize(pInfo->pTableList->pTableList); @@ -1757,7 +1744,7 @@ static SSDataBlock* doTagScan(SOperatorInfo* pOperator) { STableKeyInfo* item = taosArrayGet(pInfo->pTableList->pTableList, pInfo->curPos); metaGetTableEntryByUid(&mr, item->uid); - for (int32_t j = 0; j < pOperator->numOfExprs; ++j) { + for (int32_t j = 0; j < pOperator->exprSupp.numOfExprs; ++j) { SColumnInfoData* pDst = taosArrayGet(pRes->pDataBlock, pExprInfo[j].base.resSchema.slotId); // refactor later @@ -1820,27 +1807,26 @@ SOperatorInfo* createTagScanOperatorInfo(SReadHandle* pReadHandle, STagScanPhysi SDataBlockDescNode* pDescNode = pPhyNode->node.pOutputDataBlockDesc; + int32_t num = 0; int32_t numOfExprs = 0; SExprInfo* pExprInfo = createExprInfo(pPhyNode->pScanPseudoCols, NULL, &numOfExprs); + SArray* colList = extractColMatchInfo(pPhyNode->pScanPseudoCols, pDescNode, &num, COL_MATCH_FROM_COL_ID); - int32_t num = 0; - SArray* colList = extractColMatchInfo(pPhyNode->pScanPseudoCols, pDescNode, &num, pTaskInfo, COL_MATCH_FROM_COL_ID); + pInfo->pTableList = pTableListInfo; + pInfo->pColMatchInfo = colList; + pInfo->pRes = createResDataBlock(pDescNode); + pInfo->readHandle = *pReadHandle; + pInfo->curPos = 0; + pInfo->pFilterNode = pPhyNode->node.pConditions; - pInfo->pTableList = pTableListInfo; - pInfo->pColMatchInfo = colList; - pInfo->pRes = createResDataBlock(pDescNode); - ; - pInfo->readHandle = *pReadHandle; - pInfo->curPos = 0; - pInfo->pFilterNode = pPhyNode->node.pConditions; - pOperator->name = "TagScanOperator"; + pOperator->name = "TagScanOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_TAG_SCAN; - pOperator->blocking = false; - pOperator->status = OP_NOT_OPENED; - pOperator->info = pInfo; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfExprs; - pOperator->pTaskInfo = pTaskInfo; + pOperator->blocking = false; + pOperator->status = OP_NOT_OPENED; + pOperator->info = pInfo; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfExprs; + pOperator->pTaskInfo = pTaskInfo; initResultSizeInfo(pOperator, 4096); blockDataEnsureCapacity(pInfo->pRes, pOperator->resultInfo.capacity); @@ -1884,7 +1870,7 @@ typedef struct STableMergeScanInfo { SNode* pFilterNode; // filter info, which is push down by optimizer SqlFunctionCtx* pCtx; // which belongs to the direct upstream operator operator query context SResultRowInfo* pResultRowInfo; - int32_t* rowCellInfoOffset; + int32_t* rowEntryInfoOffset; SExprInfo* pExpr; SSDataBlock* pResBlock; SArray* pColMatchInfo; @@ -1893,7 +1879,7 @@ typedef struct STableMergeScanInfo { SExprInfo* pPseudoExpr; int32_t numOfPseudoExpr; SqlFunctionCtx* pPseudoCtx; - // int32_t* rowCellInfoOffset; + // int32_t* rowEntryInfoOffset; SQueryTableDataCond cond; int32_t scanFlag; // table scan flag to denote if it is a repeat/reverse/main scan @@ -1910,7 +1896,7 @@ int32_t createMultipleDataReaders(STableScanPhysiNode* pTableScanNode, SReadHand STableListInfo* pTableListInfo, SArray* arrayReader, uint64_t queryId, uint64_t taskId, SNode* pTagCond) { int32_t code = - getTableList(pHandle->meta, pTableScanNode->scan.tableType, pTableScanNode->scan.uid, pTableListInfo, pTagCond); + getTableList(pHandle->meta, &pTableScanNode->scan, pTableListInfo, pTagCond); if (code != TSDB_CODE_SUCCESS) { goto _error; } @@ -1937,7 +1923,7 @@ int32_t createMultipleDataReaders(STableScanPhysiNode* pTableScanNode, SReadHand taosArrayDestroy(subListInfo->pTableList); taosMemoryFree(subListInfo); } - clearupQueryTableDataCond(&cond); + cleanupQueryTableDataCond(&cond); return 0; @@ -2135,7 +2121,7 @@ int32_t doOpenTableMergeScanOperator(SOperatorInfo* pOperator) { int32_t numOfBufPage = pInfo->sortBufSize / pInfo->bufPageSize; pInfo->pSortHandle = - tsortCreateSortHandle(pInfo->pSortInfo, pInfo->pColMatchInfo, SORT_MULTISOURCE_MERGE, pInfo->bufPageSize, + tsortCreateSortHandle(pInfo->pSortInfo, SORT_MULTISOURCE_MERGE, pInfo->bufPageSize, numOfBufPage, pInfo->pSortInputBlock, pTaskInfo->id.str); tsortSetFetchRawDataFp(pInfo->pSortHandle, getTableDataBlock, NULL, NULL); @@ -2213,7 +2199,7 @@ SSDataBlock* doTableMergeScan(SOperatorInfo* pOperator) { void destroyTableMergeScanOperatorInfo(void* param, int32_t numOfOutput) { STableMergeScanInfo* pTableScanInfo = (STableMergeScanInfo*)param; - clearupQueryTableDataCond(&pTableScanInfo->cond); + cleanupQueryTableDataCond(&pTableScanInfo->cond); for (int32_t i = 0; i < taosArrayGetSize(pTableScanInfo->dataReaders); ++i) { tsdbReaderT* reader = taosArrayGetP(pTableScanInfo->dataReaders, i); @@ -2263,7 +2249,7 @@ SOperatorInfo* createTableMergeScanOperatorInfo(STableScanPhysiNode* pTableScanN int32_t numOfCols = 0; SArray* pColList = - extractColMatchInfo(pTableScanNode->scan.pScanCols, pDescNode, &numOfCols, pTaskInfo, COL_MATCH_FROM_COL_ID); + extractColMatchInfo(pTableScanNode->scan.pScanCols, pDescNode, &numOfCols, COL_MATCH_FROM_COL_ID); int32_t code = initQueryTableDataCond(&pInfo->cond, pTableScanNode); if (code != TSDB_CODE_SUCCESS) { @@ -2272,7 +2258,7 @@ SOperatorInfo* createTableMergeScanOperatorInfo(STableScanPhysiNode* pTableScanN if (pTableScanNode->scan.pScanPseudoCols != NULL) { pInfo->pPseudoExpr = createExprInfo(pTableScanNode->scan.pScanPseudoCols, NULL, &pInfo->numOfPseudoExpr); - pInfo->pPseudoCtx = createSqlFunctionCtx(pInfo->pPseudoExpr, pInfo->numOfPseudoExpr, &pInfo->rowCellInfoOffset); + pInfo->pPseudoCtx = createSqlFunctionCtx(pInfo->pPseudoExpr, pInfo->numOfPseudoExpr, &pInfo->rowEntryInfoOffset); } pInfo->scanInfo = (SScanInfo){.numOfAsc = pTableScanNode->scanSeq[0], .numOfDesc = pTableScanNode->scanSeq[1]}; @@ -2318,7 +2304,7 @@ SOperatorInfo* createTableMergeScanOperatorInfo(STableScanPhysiNode* pTableScanN pOperator->blocking = false; pOperator->status = OP_NOT_OPENED; pOperator->info = pInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->pTaskInfo = pTaskInfo; initResultSizeInfo(pOperator, 1024); diff --git a/source/libs/executor/src/sortoperator.c b/source/libs/executor/src/sortoperator.c index 81899b68cd..9d13276e6d 100644 --- a/source/libs/executor/src/sortoperator.c +++ b/source/libs/executor/src/sortoperator.c @@ -22,41 +22,52 @@ static int32_t getExplainExecInfo(SOperatorInfo* pOptr, void** pOptrExplain static void destroyOrderOperatorInfo(void* param, int32_t numOfOutput); -SOperatorInfo* createSortOperatorInfo(SOperatorInfo* downstream, SSDataBlock* pResBlock, SArray* pSortInfo, - SExprInfo* pExprInfo, int32_t numOfCols, SArray* pColMatchColInfo, - SExecTaskInfo* pTaskInfo) { +SOperatorInfo* createSortOperatorInfo(SOperatorInfo* downstream, SSortPhysiNode* pSortPhyNode, SExecTaskInfo* pTaskInfo) { SSortOperatorInfo* pInfo = taosMemoryCalloc(1, sizeof(SSortOperatorInfo)); SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); - int32_t rowSize = pResBlock->info.rowSize; - - if (pInfo == NULL || pOperator == NULL || rowSize > 100 * 1024 * 1024) { + if (pInfo == NULL || pOperator == NULL/* || rowSize > 100 * 1024 * 1024*/) { goto _error; } - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfCols; - pInfo->binfo.pCtx = createSqlFunctionCtx(pExprInfo, numOfCols, &pInfo->binfo.rowCellInfoOffset); + SDataBlockDescNode* pDescNode = pSortPhyNode->node.pOutputDataBlockDesc; + + int32_t numOfCols = 0; + SSDataBlock* pResBlock = createResDataBlock(pDescNode); + SExprInfo* pExprInfo = createExprInfo(pSortPhyNode->pExprs, NULL, &numOfCols); + + int32_t numOfOutputCols = 0; + SArray* pColMatchColInfo = + extractColMatchInfo(pSortPhyNode->pTargets, pDescNode, &numOfOutputCols, COL_MATCH_FROM_SLOT_ID); + + pOperator->exprSupp.pCtx = createSqlFunctionCtx(pExprInfo, numOfCols, &pOperator->exprSupp.rowEntryInfoOffset); pInfo->binfo.pRes = pResBlock; initResultSizeInfo(pOperator, 1024); - pInfo->pSortInfo = pSortInfo; + pInfo->pSortInfo = createSortInfo(pSortPhyNode->pSortKeys);; pInfo->pColMatchInfo = pColMatchColInfo; pOperator->name = "SortOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_SORT; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; pOperator->info = pInfo; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfCols; + pOperator->pTaskInfo = pTaskInfo; // lazy evaluation for the following parameter since the input datablock is not known till now. - // pInfo->bufPageSize = rowSize < 1024 ? 1024 * 2 : rowSize * 2; // there are headers, so pageSize = rowSize + - // header pInfo->sortBufSize = pInfo->bufPageSize * 16; // TODO dynamic set the available sort buffer + // pInfo->bufPageSize = rowSize < 1024 ? 1024 * 2 : rowSize * 2; + // there are headers, so pageSize = rowSize + header pInfo->sortBufSize = pInfo->bufPageSize * 16; + // TODO dynamic set the available sort buffer - pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = createOperatorFpSet(doOpenSortOperator, doSort, NULL, NULL, destroyOrderOperatorInfo, NULL, NULL, getExplainExecInfo); int32_t code = appendDownstream(pOperator, &downstream, 1); + if (code != TSDB_CODE_SUCCESS) { + goto _error; + } + return pOperator; _error: @@ -134,9 +145,9 @@ SSDataBlock* loadNextDataBlock(void* param) { void applyScalarFunction(SSDataBlock* pBlock, void* param) { SOperatorInfo* pOperator = param; SSortOperatorInfo* pSort = pOperator->info; - if (pOperator->pExpr != NULL) { + if (pOperator->exprSupp.pExprInfo != NULL) { int32_t code = - projectApplyFunctions(pOperator->pExpr, pBlock, pBlock, pSort->binfo.pCtx, pOperator->numOfExprs, NULL); + projectApplyFunctions(pOperator->exprSupp.pExprInfo, pBlock, pBlock, pOperator->exprSupp.pCtx, pOperator->exprSupp.numOfExprs, NULL); if (code != TSDB_CODE_SUCCESS) { longjmp(pOperator->pTaskInfo->env, code); } @@ -154,7 +165,7 @@ int32_t doOpenSortOperator(SOperatorInfo* pOperator) { pInfo->startTs = taosGetTimestampUs(); // pInfo->binfo.pRes is not equalled to the input datablock. - pInfo->pSortHandle = tsortCreateSortHandle(pInfo->pSortInfo, pInfo->pColMatchInfo, SORT_SINGLESOURCE_SORT, -1, -1, + pInfo->pSortHandle = tsortCreateSortHandle(pInfo->pSortInfo, SORT_SINGLESOURCE_SORT, -1, -1, NULL, pTaskInfo->id.str); tsortSetFetchRawDataFp(pInfo->pSortHandle, loadNextDataBlock, applyScalarFunction, pOperator); @@ -248,7 +259,7 @@ int32_t doOpenMultiwaySortMergeOperator(SOperatorInfo* pOperator) { int32_t numOfBufPage = pInfo->sortBufSize / pInfo->bufPageSize; - pInfo->pSortHandle = tsortCreateSortHandle(pInfo->pSortInfo, pInfo->pColMatchInfo, SORT_MULTISOURCE_MERGE, + pInfo->pSortHandle = tsortCreateSortHandle(pInfo->pSortInfo, SORT_MULTISOURCE_MERGE, pInfo->bufPageSize, numOfBufPage, pInfo->pInputBlock, pTaskInfo->id.str); tsortSetFetchRawDataFp(pInfo->pSortHandle, loadNextDataBlock, NULL, NULL); @@ -392,10 +403,8 @@ SOperatorInfo* createMultiwaySortMergeOperatorInfo(SOperatorInfo** downStreams, pInfo->bufPageSize = getProperSortPageSize(rowSize); - uint32_t numOfSources = taosArrayGetSize(pSortInfo); - numOfSources = TMAX(4, numOfSources); - - pInfo->sortBufSize = numOfSources * pInfo->bufPageSize; + // one additional is reserved for merged result. + pInfo->sortBufSize = pInfo->bufPageSize * (numStreams + 1); pOperator->fpSet = createOperatorFpSet(doOpenMultiwaySortMergeOperator, doMultiwaySortMerge, NULL, NULL, diff --git a/source/libs/executor/src/timewindowoperator.c b/source/libs/executor/src/timewindowoperator.c index d7ae823522..a31bfab62d 100644 --- a/source/libs/executor/src/timewindowoperator.c +++ b/source/libs/executor/src/timewindowoperator.c @@ -103,7 +103,7 @@ STimeWindow getActiveTimeWindow(SDiskbasedBuf* pBuf, SResultRowInfo* pResultRowI static int32_t setTimeWindowOutputBuf(SResultRowInfo* pResultRowInfo, STimeWindow* win, bool masterscan, SResultRow** pResult, int64_t tableGroupId, SqlFunctionCtx* pCtx, - int32_t numOfOutput, int32_t* rowCellInfoOffset, SAggSupporter* pAggSup, + int32_t numOfOutput, int32_t* rowEntryInfoOffset, SAggSupporter* pAggSup, SExecTaskInfo* pTaskInfo) { assert(win->skey <= win->ekey); SResultRow* pResultRow = doSetResultOutBufByKey(pAggSup->pResultBuf, pResultRowInfo, (char*)&win->skey, TSDB_KEYSIZE, @@ -118,7 +118,7 @@ static int32_t setTimeWindowOutputBuf(SResultRowInfo* pResultRowInfo, STimeWindo pResultRow->win = (*win); *pResult = pResultRow; - setResultRowInitCtx(pResultRow, pCtx, numOfOutput, rowCellInfoOffset); + setResultRowInitCtx(pResultRow, pCtx, numOfOutput, rowEntryInfoOffset); return TSDB_CODE_SUCCESS; } @@ -322,12 +322,12 @@ static void getNextTimeWindow(SInterval* pInterval, int32_t precision, int32_t o tw->ekey -= 1; } -void doTimeWindowInterpolation(SIntervalAggOperatorInfo* pInfo, int32_t numOfExprs, SArray* pDataBlock, TSKEY prevTs, - int32_t prevRowIndex, TSKEY curTs, int32_t curRowIndex, TSKEY windowKey, int32_t type) { - SqlFunctionCtx* pCtx = pInfo->binfo.pCtx; +void doTimeWindowInterpolation(SArray* pPrevValues, SArray* pDataBlock, TSKEY prevTs, int32_t prevRowIndex, + TSKEY curTs, int32_t curRowIndex, TSKEY windowKey, int32_t type, SExprSupp* pSup) { + SqlFunctionCtx* pCtx = pSup->pCtx; int32_t index = 1; - for (int32_t k = 0; k < numOfExprs; ++k) { + for (int32_t k = 0; k < pSup->numOfExprs; ++k) { if (!fmIsIntervalInterpoFunc(pCtx[k].functionId)) { pCtx[k].start.key = INT64_MIN; continue; @@ -340,7 +340,7 @@ void doTimeWindowInterpolation(SIntervalAggOperatorInfo* pInfo, int32_t numOfExp double v1 = 0, v2 = 0, v = 0; if (prevRowIndex == -1) { - SGroupKeys* p = taosArrayGet(pInfo->pPrevValues, index); + SGroupKeys* p = taosArrayGet(pPrevValues, index); GET_TYPED_DATA(v1, double, pColInfo->info.type, p->pData); } else { GET_TYPED_DATA(v1, double, pColInfo->info.type, colDataGetData(pColInfo, prevRowIndex)); @@ -403,8 +403,8 @@ static void setNotInterpoWindowKey(SqlFunctionCtx* pCtx, int32_t numOfOutput, in } } -static bool setTimeWindowInterpolationStartTs(SIntervalAggOperatorInfo* pInfo, SqlFunctionCtx* pCtx, int32_t numOfExprs, - int32_t pos, SSDataBlock* pBlock, const TSKEY* tsCols, STimeWindow* win) { +static bool setTimeWindowInterpolationStartTs(SIntervalAggOperatorInfo* pInfo, int32_t pos, SSDataBlock* pBlock, const TSKEY* tsCols, + STimeWindow* win, SExprSupp* pSup) { bool ascQuery = (pInfo->order == TSDB_ORDER_ASC); TSKEY curTs = tsCols[pos]; @@ -416,23 +416,23 @@ static bool setTimeWindowInterpolationStartTs(SIntervalAggOperatorInfo* pInfo, S // start exactly from this point, no need to do interpolation TSKEY key = ascQuery ? win->skey : win->ekey; if (key == curTs) { - setNotInterpoWindowKey(pCtx, numOfExprs, RESULT_ROW_START_INTERP); + setNotInterpoWindowKey(pSup->pCtx, pSup->numOfExprs, RESULT_ROW_START_INTERP); return true; } // it is the first time window, no need to do interpolation if (pTsKey->isNull && pos == 0) { - setNotInterpoWindowKey(pCtx, numOfExprs, RESULT_ROW_START_INTERP); + setNotInterpoWindowKey(pSup->pCtx, pSup->numOfExprs, RESULT_ROW_START_INTERP); } else { TSKEY prevTs = ((pos == 0) ? lastTs : tsCols[pos - 1]); - doTimeWindowInterpolation(pInfo, numOfExprs, pBlock->pDataBlock, prevTs, pos - 1, curTs, pos, key, - RESULT_ROW_START_INTERP); + doTimeWindowInterpolation(pInfo->pPrevValues, pBlock->pDataBlock, prevTs, pos - 1, curTs, pos, key, + RESULT_ROW_START_INTERP, pSup); } return true; } -static bool setTimeWindowInterpolationEndTs(SIntervalAggOperatorInfo* pInfo, SqlFunctionCtx* pCtx, int32_t numOfExprs, +static bool setTimeWindowInterpolationEndTs(SIntervalAggOperatorInfo* pInfo, SExprSupp* pSup, int32_t endRowIndex, SArray* pDataBlock, const TSKEY* tsCols, TSKEY blockEkey, STimeWindow* win) { int32_t order = pInfo->order; @@ -442,13 +442,13 @@ static bool setTimeWindowInterpolationEndTs(SIntervalAggOperatorInfo* pInfo, Sql // not ended in current data block, do not invoke interpolation if ((key > blockEkey && (order == TSDB_ORDER_ASC)) || (key < blockEkey && (order == TSDB_ORDER_DESC))) { - setNotInterpoWindowKey(pCtx, numOfExprs, RESULT_ROW_END_INTERP); + setNotInterpoWindowKey(pSup->pCtx, pSup->numOfExprs, RESULT_ROW_END_INTERP); return false; } // there is actual end point of current time window, no interpolation needs if (key == actualEndKey) { - setNotInterpoWindowKey(pCtx, numOfExprs, RESULT_ROW_END_INTERP); + setNotInterpoWindowKey(pSup->pCtx, pSup->numOfExprs, RESULT_ROW_END_INTERP); return true; } @@ -456,8 +456,8 @@ static bool setTimeWindowInterpolationEndTs(SIntervalAggOperatorInfo* pInfo, Sql assert(nextRowIndex >= 0); TSKEY nextKey = tsCols[nextRowIndex]; - doTimeWindowInterpolation(pInfo, numOfExprs, pDataBlock, actualEndKey, endRowIndex, nextKey, nextRowIndex, key, - RESULT_ROW_END_INTERP); + doTimeWindowInterpolation(pInfo->pPrevValues, pDataBlock, actualEndKey, endRowIndex, nextKey, nextRowIndex, key, + RESULT_ROW_END_INTERP, pSup); return true; } @@ -546,9 +546,8 @@ static void setResultRowInterpo(SResultRow* pResult, SResultTsInterpType type) { } } -static void doWindowBorderInterpolation(SIntervalAggOperatorInfo* pInfo, SSDataBlock* pBlock, int32_t numOfExprs, - SqlFunctionCtx* pCtx, SResultRow* pResult, STimeWindow* win, int32_t startPos, - int32_t forwardRows) { +static void doWindowBorderInterpolation(SIntervalAggOperatorInfo* pInfo, SSDataBlock* pBlock, SResultRow* pResult, STimeWindow* win, int32_t startPos, + int32_t forwardRows, SExprSupp* pSup) { if (!pInfo->timeWindowInterpo) { return; } @@ -564,12 +563,12 @@ static void doWindowBorderInterpolation(SIntervalAggOperatorInfo* pInfo, SSDataB TSKEY* tsCols = (TSKEY*)(pColInfo->pData); bool done = isResultRowInterpolated(pResult, RESULT_ROW_START_INTERP); if (!done) { // it is not interpolated, now start to generated the interpolated value - bool interp = setTimeWindowInterpolationStartTs(pInfo, pCtx, numOfExprs, startPos, pBlock, tsCols, win); + bool interp = setTimeWindowInterpolationStartTs(pInfo, startPos, pBlock, tsCols, win, pSup); if (interp) { setResultRowInterpo(pResult, RESULT_ROW_START_INTERP); } } else { - setNotInterpoWindowKey(pCtx, numOfExprs, RESULT_ROW_START_INTERP); + setNotInterpoWindowKey(pSup->pCtx, pSup->numOfExprs, RESULT_ROW_START_INTERP); } // point interpolation does not require the end key time window interpolation. @@ -583,13 +582,12 @@ static void doWindowBorderInterpolation(SIntervalAggOperatorInfo* pInfo, SSDataB int32_t endRowIndex = startPos + forwardRows - 1; TSKEY endKey = (pInfo->order == TSDB_ORDER_ASC) ? pBlock->info.window.ekey : pBlock->info.window.skey; - bool interp = - setTimeWindowInterpolationEndTs(pInfo, pCtx, numOfExprs, endRowIndex, pBlock->pDataBlock, tsCols, endKey, win); + bool interp = setTimeWindowInterpolationEndTs(pInfo, pSup, endRowIndex, pBlock->pDataBlock, tsCols, endKey, win); if (interp) { setResultRowInterpo(pResult, RESULT_ROW_END_INTERP); } } else { - setNotInterpoWindowKey(pCtx, numOfExprs, RESULT_ROW_END_INTERP); + setNotInterpoWindowKey(pSup->pCtx, pSup->numOfExprs, RESULT_ROW_END_INTERP); } } @@ -628,9 +626,10 @@ static void doInterpUnclosedTimeWindow(SOperatorInfo* pOperatorInfo, int32_t num SExecTaskInfo* pTaskInfo = pOperatorInfo->pTaskInfo; SIntervalAggOperatorInfo* pInfo = (SIntervalAggOperatorInfo*)pOperatorInfo->info; + SExprSupp* pSup = &pOperatorInfo->exprSupp; int32_t startPos = 0; - int32_t numOfOutput = pOperatorInfo->numOfExprs; + int32_t numOfOutput = pSup->numOfExprs; uint64_t groupId = pBlock->info.groupId; SResultRow* pResult = NULL; @@ -655,8 +654,8 @@ static void doInterpUnclosedTimeWindow(SOperatorInfo* pOperatorInfo, int32_t num STimeWindow w = pr->win; int32_t ret = - setTimeWindowOutputBuf(pResultRowInfo, &w, (scanFlag == MAIN_SCAN), &pResult, groupId, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(pResultRowInfo, &w, (scanFlag == MAIN_SCAN), &pResult, groupId, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } @@ -665,13 +664,13 @@ static void doInterpUnclosedTimeWindow(SOperatorInfo* pOperatorInfo, int32_t num SGroupKeys* pTsKey = taosArrayGet(pInfo->pPrevValues, 0); int64_t prevTs = *(int64_t*)pTsKey->pData; - doTimeWindowInterpolation(pInfo, numOfOutput, pBlock->pDataBlock, prevTs, -1, tsCols[startPos], startPos, w.ekey, - RESULT_ROW_END_INTERP); + doTimeWindowInterpolation(pInfo->pPrevValues, pBlock->pDataBlock, prevTs, -1, tsCols[startPos], startPos, w.ekey, + RESULT_ROW_END_INTERP, pSup); setResultRowInterpo(pResult, RESULT_ROW_END_INTERP); - setNotInterpoWindowKey(pInfo->binfo.pCtx, numOfExprs, RESULT_ROW_START_INTERP); + setNotInterpoWindowKey(pSup->pCtx, numOfExprs, RESULT_ROW_START_INTERP); - doApplyFunctions(pTaskInfo, pInfo->binfo.pCtx, &w, &pInfo->twAggSup.timeWindowData, startPos, 0, tsCols, + doApplyFunctions(pTaskInfo, pSup->pCtx, &w, &pInfo->twAggSup.timeWindowData, startPos, 0, tsCols, pBlock->info.rows, numOfExprs, pInfo->order); if (isResultRowInterpolated(pResult, RESULT_ROW_END_INTERP)) { @@ -798,9 +797,10 @@ static void hashIntervalAgg(SOperatorInfo* pOperatorInfo, SResultRowInfo* pResul SIntervalAggOperatorInfo* pInfo = (SIntervalAggOperatorInfo*)pOperatorInfo->info; SExecTaskInfo* pTaskInfo = pOperatorInfo->pTaskInfo; + SExprSupp* pSup = &pOperatorInfo->exprSupp; int32_t startPos = 0; - int32_t numOfOutput = pOperatorInfo->numOfExprs; + int32_t numOfOutput = pSup->numOfExprs; int64_t* tsCols = extractTsCol(pBlock, pInfo); uint64_t tableGroupId = pBlock->info.groupId; bool ascScan = (pInfo->order == TSDB_ORDER_ASC); @@ -811,8 +811,8 @@ static void hashIntervalAgg(SOperatorInfo* pOperatorInfo, SResultRowInfo* pResul pInfo->interval.precision, &pInfo->win); int32_t ret = - setTimeWindowOutputBuf(pResultRowInfo, &win, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(pResultRowInfo, &win, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS || pResult == NULL) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } @@ -835,18 +835,18 @@ static void hashIntervalAgg(SOperatorInfo* pOperatorInfo, SResultRowInfo* pResul // restore current time window ret = - setTimeWindowOutputBuf(pResultRowInfo, &win, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(pResultRowInfo, &win, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } // window start key interpolation - doWindowBorderInterpolation(pInfo, pBlock, numOfOutput, pInfo->binfo.pCtx, pResult, &win, startPos, forwardRows); + doWindowBorderInterpolation(pInfo, pBlock, pResult, &win, startPos, forwardRows, pSup); } updateTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &win, true); - doApplyFunctions(pTaskInfo, pInfo->binfo.pCtx, &win, &pInfo->twAggSup.timeWindowData, startPos, forwardRows, tsCols, + doApplyFunctions(pTaskInfo, pSup->pCtx, &win, &pInfo->twAggSup.timeWindowData, startPos, forwardRows, tsCols, pBlock->info.rows, numOfOutput, pInfo->order); doCloseWindow(pResultRowInfo, pInfo, pResult); @@ -861,7 +861,7 @@ static void hashIntervalAgg(SOperatorInfo* pOperatorInfo, SResultRowInfo* pResul // null data, failed to allocate more memory buffer int32_t code = setTimeWindowOutputBuf(pResultRowInfo, &nextWin, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, - pInfo->binfo.pCtx, numOfOutput, pInfo->binfo.rowCellInfoOffset, + pSup->pCtx, numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (code != TSDB_CODE_SUCCESS || pResult == NULL) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); @@ -878,11 +878,10 @@ static void hashIntervalAgg(SOperatorInfo* pOperatorInfo, SResultRowInfo* pResul getNumOfRowsInTimeWindow(&pBlock->info, tsCols, startPos, ekey, binarySearchForKey, NULL, pInfo->order); // window start(end) key interpolation - doWindowBorderInterpolation(pInfo, pBlock, numOfOutput, pInfo->binfo.pCtx, pResult, &nextWin, startPos, - forwardRows); + doWindowBorderInterpolation(pInfo, pBlock, pResult, &nextWin, startPos, forwardRows, pSup); updateTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &nextWin, true); - doApplyFunctions(pTaskInfo, pInfo->binfo.pCtx, &nextWin, &pInfo->twAggSup.timeWindowData, startPos, forwardRows, + doApplyFunctions(pTaskInfo, pSup->pCtx, &nextWin, &pInfo->twAggSup.timeWindowData, startPos, forwardRows, tsCols, pBlock->info.rows, numOfOutput, pInfo->order); doCloseWindow(pResultRowInfo, pInfo, pResult); } @@ -937,6 +936,7 @@ static int32_t doOpenIntervalAgg(SOperatorInfo* pOperator) { SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; SIntervalAggOperatorInfo* pInfo = pOperator->info; + SExprSupp* pSup = &pOperator->exprSupp; int32_t scanFlag = MAIN_SCAN; @@ -952,7 +952,7 @@ static int32_t doOpenIntervalAgg(SOperatorInfo* pOperator) { getTableScanInfo(pOperator, &pInfo->order, &scanFlag); // the pDataBlock are always the same one, no need to call this again - setInputDataBlock(pOperator, pInfo->binfo.pCtx, pBlock, pInfo->order, scanFlag, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, pInfo->order, scanFlag, true); hashIntervalAgg(pOperator, &pInfo->binfo.resultRowInfo, pBlock, scanFlag, NULL); #if 0 // test for encode/decode result info @@ -993,12 +993,13 @@ static bool compareVal(const char* v, const SStateKeys* pKey) { static void doStateWindowAggImpl(SOperatorInfo* pOperator, SStateWindowOperatorInfo* pInfo, SSDataBlock* pBlock) { SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; + SExprSupp* pSup = &pOperator->exprSupp; SColumnInfoData* pStateColInfoData = taosArrayGet(pBlock->pDataBlock, pInfo->stateCol.slotId); int64_t gid = pBlock->info.groupId; bool masterScan = true; - int32_t numOfOutput = pOperator->numOfExprs; + int32_t numOfOutput = pOperator->exprSupp.numOfExprs; int16_t bytes = pStateColInfoData->info.bytes; SColumnInfoData* pColInfoData = taosArrayGet(pBlock->pDataBlock, pInfo->tsSlotId); @@ -1041,14 +1042,14 @@ static void doStateWindowAggImpl(SOperatorInfo* pOperator, SStateWindowOperatorI pRowSup->win.ekey = pRowSup->win.skey; int32_t ret = - setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, &window, masterScan, &pResult, gid, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, &window, masterScan, &pResult, gid, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS) { // null data, too many state code longjmp(pTaskInfo->env, TSDB_CODE_QRY_APP_ERROR); } updateTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &window, false); - doApplyFunctions(pTaskInfo, pInfo->binfo.pCtx, &window, &pInfo->twAggSup.timeWindowData, pRowSup->startRowIndex, + doApplyFunctions(pTaskInfo, pSup->pCtx, &window, &pInfo->twAggSup.timeWindowData, pRowSup->startRowIndex, pRowSup->numOfRows, NULL, pBlock->info.rows, numOfOutput, TSDB_ORDER_ASC); // here we start a new session window @@ -1067,14 +1068,14 @@ static void doStateWindowAggImpl(SOperatorInfo* pOperator, SStateWindowOperatorI SResultRow* pResult = NULL; pRowSup->win.ekey = tsList[pBlock->info.rows - 1]; int32_t ret = - setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, &pRowSup->win, masterScan, &pResult, gid, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, &pRowSup->win, masterScan, &pResult, gid, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS) { // null data, too many state code longjmp(pTaskInfo->env, TSDB_CODE_QRY_APP_ERROR); } updateTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pRowSup->win, false); - doApplyFunctions(pTaskInfo, pInfo->binfo.pCtx, &pRowSup->win, &pInfo->twAggSup.timeWindowData, pRowSup->startRowIndex, + doApplyFunctions(pTaskInfo, pSup->pCtx, &pRowSup->win, &pInfo->twAggSup.timeWindowData, pRowSup->startRowIndex, pRowSup->numOfRows, NULL, pBlock->info.rows, numOfOutput, TSDB_ORDER_ASC); } @@ -1086,11 +1087,13 @@ static SSDataBlock* doStateWindowAgg(SOperatorInfo* pOperator) { SStateWindowOperatorInfo* pInfo = pOperator->info; SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; + SExprSupp* pSup = &pOperator->exprSupp; + SOptrBasicInfo* pBInfo = &pInfo->binfo; if (pOperator->status == OP_RES_TO_RETURN) { doBuildResultDatablock(pOperator, pBInfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); - if (pBInfo->pRes->info.rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (pBInfo->pRes->info.rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); return NULL; } @@ -1108,7 +1111,7 @@ static SSDataBlock* doStateWindowAgg(SOperatorInfo* pOperator) { break; } - setInputDataBlock(pOperator, pBInfo->pCtx, pBlock, order, MAIN_SCAN, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, order, MAIN_SCAN, true); blockDataUpdateTsWindow(pBlock, pInfo->tsSlotId); doStateWindowAggImpl(pOperator, pInfo, pBlock); @@ -1122,7 +1125,7 @@ static SSDataBlock* doStateWindowAgg(SOperatorInfo* pOperator) { initGroupedResultInfo(&pInfo->groupResInfo, pInfo->aggSup.pResultRowHashTable, TSDB_ORDER_ASC); blockDataEnsureCapacity(pBInfo->pRes, pOperator->resultInfo.capacity); doBuildResultDatablock(pOperator, pBInfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); - if (pBInfo->pRes->info.rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (pBInfo->pRes->info.rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); } @@ -1153,7 +1156,7 @@ static SSDataBlock* doBuildIntervalResult(SOperatorInfo* pOperator) { blockDataEnsureCapacity(pBlock, pOperator->resultInfo.capacity); doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); - if (pBlock->info.rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (pBlock->info.rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); } @@ -1166,7 +1169,7 @@ static SSDataBlock* doBuildIntervalResult(SOperatorInfo* pOperator) { // todo merged with the build group result. static void finalizeUpdatedResult(int32_t numOfOutput, SDiskbasedBuf* pBuf, SArray* pUpdateList, - int32_t* rowCellInfoOffset) { + int32_t* rowEntryInfoOffset) { size_t num = taosArrayGetSize(pUpdateList); for (int32_t i = 0; i < num; ++i) { @@ -1176,7 +1179,7 @@ static void finalizeUpdatedResult(int32_t numOfOutput, SDiskbasedBuf* pBuf, SArr SResultRow* pRow = (SResultRow*)((char*)bufPage + pPos->pos.offset); for (int32_t j = 0; j < numOfOutput; ++j) { - SResultRowEntryInfo* pEntry = getResultCell(pRow, j, rowCellInfoOffset); + SResultRowEntryInfo* pEntry = getResultEntryInfo(pRow, j, rowEntryInfoOffset); if (pRow->numOfRows < pEntry->numOfRes) { pRow->numOfRows = pEntry->numOfRes; } @@ -1195,11 +1198,11 @@ static void setInverFunction(SqlFunctionCtx* pCtx, int32_t num, EStreamType type } } -void doClearWindowImpl(SResultRowPosition* p1, SDiskbasedBuf* pResultBuf, SOptrBasicInfo* pBinfo, int32_t numOfOutput) { +void doClearWindowImpl(SResultRowPosition* p1, SDiskbasedBuf* pResultBuf, SExprSupp *pSup, int32_t numOfOutput) { SResultRow* pResult = getResultRowByPos(pResultBuf, p1); - SqlFunctionCtx* pCtx = pBinfo->pCtx; + SqlFunctionCtx* pCtx = pSup->pCtx; for (int32_t i = 0; i < numOfOutput; ++i) { - pCtx[i].resultInfo = getResultCell(pResult, i, pBinfo->rowCellInfoOffset); + pCtx[i].resultInfo = getResultEntryInfo(pResult, i, pSup->rowEntryInfoOffset); struct SResultRowEntryInfo* pResInfo = pCtx[i].resultInfo; if (fmIsWindowPseudoColumnFunc(pCtx[i].functionId)) { continue; @@ -1211,19 +1214,19 @@ void doClearWindowImpl(SResultRowPosition* p1, SDiskbasedBuf* pResultBuf, SOptrB } } -void doClearWindow(SAggSupporter* pSup, SOptrBasicInfo* pBinfo, char* pData, int16_t bytes, uint64_t groupId, +void doClearWindow(SAggSupporter* pAggSup, SExprSupp *pSup, char* pData, int16_t bytes, uint64_t groupId, int32_t numOfOutput) { - SET_RES_WINDOW_KEY(pSup->keyBuf, pData, bytes, groupId); + SET_RES_WINDOW_KEY(pAggSup->keyBuf, pData, bytes, groupId); SResultRowPosition* p1 = - (SResultRowPosition*)taosHashGet(pSup->pResultRowHashTable, pSup->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes)); + (SResultRowPosition*)taosHashGet(pAggSup->pResultRowHashTable, pAggSup->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes)); if (!p1) { // window has been closed return; } - doClearWindowImpl(p1, pSup->pResultBuf, pBinfo, numOfOutput); + doClearWindowImpl(p1, pAggSup->pResultBuf, pSup, numOfOutput); } -static void doClearWindows(SAggSupporter* pSup, SOptrBasicInfo* pBinfo, SInterval* pInterval, int32_t tsIndex, +static void doClearWindows(SAggSupporter* pAggSup, SExprSupp* pSup1, SInterval* pInterval, int32_t tsIndex, int32_t numOfOutput, SSDataBlock* pBlock, SArray* pUpWins) { SColumnInfoData* pColDataInfo = taosArrayGet(pBlock->pDataBlock, tsIndex); TSKEY* tsCols = (TSKEY*)pColDataInfo->pData; @@ -1233,7 +1236,7 @@ static void doClearWindows(SAggSupporter* pSup, SOptrBasicInfo* pBinfo, SInterva dumyInfo.cur.pageId = -1; STimeWindow win = getActiveTimeWindow(NULL, &dumyInfo, tsCols[i], pInterval, pInterval->precision, NULL); step = getNumOfRowsInTimeWindow(&pBlock->info, tsCols, i, win.ekey, binarySearchForKey, NULL, TSDB_ORDER_ASC); - doClearWindow(pSup, pBinfo, (char*)&win.skey, sizeof(TKEY), pBlock->info.groupId, numOfOutput); + doClearWindow(pAggSup, pSup1, (char*)&win.skey, sizeof(TKEY), pBlock->info.groupId, numOfOutput); if (pUpWins) { taosArrayPush(pUpWins, &win); } @@ -1294,6 +1297,7 @@ static SSDataBlock* doStreamIntervalAgg(SOperatorInfo* pOperator) { SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; pInfo->order = TSDB_ORDER_ASC; + SExprSupp* pSup = &pOperator->exprSupp; if (pOperator->status == OP_EXEC_DONE) { return NULL; @@ -1301,7 +1305,7 @@ static SSDataBlock* doStreamIntervalAgg(SOperatorInfo* pOperator) { if (pOperator->status == OP_RES_TO_RETURN) { doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); - if (pInfo->binfo.pRes->info.rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (pInfo->binfo.pRes->info.rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { pOperator->status = OP_EXEC_DONE; } return pInfo->binfo.pRes->info.rows == 0 ? NULL : pInfo->binfo.pRes; @@ -1317,7 +1321,7 @@ static SSDataBlock* doStreamIntervalAgg(SOperatorInfo* pOperator) { } if (pBlock->info.type == STREAM_REPROCESS) { - doClearWindows(&pInfo->aggSup, &pInfo->binfo, &pInfo->interval, 0, pOperator->numOfExprs, pBlock, NULL); + doClearWindows(&pInfo->aggSup, &pOperator->exprSupp, &pInfo->interval, 0, pOperator->exprSupp.numOfExprs, pBlock, NULL); qDebug("%s clear existed time window results for updates checked", GET_TASKID(pTaskInfo)); continue; } else if (pBlock->info.type == STREAM_GET_ALL) { @@ -1328,9 +1332,9 @@ static SSDataBlock* doStreamIntervalAgg(SOperatorInfo* pOperator) { // The timewindow that overlaps the timestamps of the input pBlock need to be recalculated and return to the // caller. Note that all the time window are not close till now. // the pDataBlock are always the same one, no need to call this again - setInputDataBlock(pOperator, pInfo->binfo.pCtx, pBlock, pInfo->order, MAIN_SCAN, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, pInfo->order, MAIN_SCAN, true); if (pInfo->invertible) { - setInverFunction(pInfo->binfo.pCtx, pOperator->numOfExprs, pBlock->info.type); + setInverFunction(pSup->pCtx, pOperator->exprSupp.numOfExprs, pBlock->info.type); } pInfo->twAggSup.maxTs = TMAX(pInfo->twAggSup.maxTs, pBlock->info.window.ekey); @@ -1338,7 +1342,7 @@ static SSDataBlock* doStreamIntervalAgg(SOperatorInfo* pOperator) { } closeIntervalWindow(pInfo->aggSup.pResultRowHashTable, &pInfo->twAggSup, &pInfo->interval, pUpdated); - finalizeUpdatedResult(pOperator->numOfExprs, pInfo->aggSup.pResultBuf, pUpdated, pInfo->binfo.rowCellInfoOffset); + finalizeUpdatedResult(pOperator->exprSupp.numOfExprs, pInfo->aggSup.pResultBuf, pUpdated, pSup->rowEntryInfoOffset); initMultiResInfoFromArrayList(&pInfo->groupResInfo, pUpdated); blockDataEnsureCapacity(pInfo->binfo.pRes, pOperator->resultInfo.capacity); doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); @@ -1350,19 +1354,19 @@ static SSDataBlock* doStreamIntervalAgg(SOperatorInfo* pOperator) { static void destroyStateWindowOperatorInfo(void* param, int32_t numOfOutput) { SStateWindowOperatorInfo* pInfo = (SStateWindowOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); taosMemoryFreeClear(pInfo->stateKey.pData); } void destroyIntervalOperatorInfo(void* param, int32_t numOfOutput) { SIntervalAggOperatorInfo* pInfo = (SIntervalAggOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); cleanupAggSup(&pInfo->aggSup); } void destroyStreamFinalIntervalOperatorInfo(void* param, int32_t numOfOutput) { SStreamFinalIntervalOperatorInfo* pInfo = (SStreamFinalIntervalOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); cleanupAggSup(&pInfo->aggSup); if (pInfo->pChildren) { int32_t size = taosArrayGetSize(pInfo->pChildren); @@ -1452,23 +1456,25 @@ SOperatorInfo* createIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo* pInfo->primaryTsIndex = primaryTsSlotId; + SExprSupp* pSup = &pOperator->exprSupp; + size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; initResultSizeInfo(pOperator, 4096); - int32_t code = - initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfCols, pResBlock, keyBufSize, pTaskInfo->id.str); - + int32_t code = initAggInfo(pSup, &pInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str); + initBasicInfo(&pInfo->binfo, pResBlock); + if (isStream) { ASSERT(numOfCols > 0); - increaseTs(pInfo->binfo.pCtx); + increaseTs(pSup->pCtx); } initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pInfo->win); - pInfo->invertible = allInvertible(pInfo->binfo.pCtx, numOfCols); + pInfo->invertible = allInvertible(pSup->pCtx, numOfCols); pInfo->invertible = false; // Todo(liuyao): Dependent TSDB API - pInfo->timeWindowInterpo = timeWindowinterpNeeded(pInfo->binfo.pCtx, numOfCols, pInfo); + pInfo->timeWindowInterpo = timeWindowinterpNeeded(pSup->pCtx, numOfCols, pInfo); if (pInfo->timeWindowInterpo) { pInfo->binfo.resultRowInfo.openWindow = tdListNew(sizeof(SResultRowPosition)); if (pInfo->binfo.resultRowInfo.openWindow == NULL) { @@ -1476,15 +1482,15 @@ SOperatorInfo* createIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo* } } - initResultRowInfo(&pInfo->binfo.resultRowInfo, (int32_t)1); + initResultRowInfo(&pInfo->binfo.resultRowInfo); pOperator->name = "TimeIntervalAggOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_HASH_INTERVAL; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExprInfo; + pOperator->exprSupp.pExprInfo = pExprInfo; pOperator->pTaskInfo = pTaskInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->info = pInfo; pOperator->fpSet = createOperatorFpSet(doOpenIntervalAgg, doBuildIntervalResult, doStreamIntervalAgg, NULL, @@ -1525,23 +1531,23 @@ SOperatorInfo* createStreamIntervalOperatorInfo(SOperatorInfo* downstream, SExpr size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; initResultSizeInfo(pOperator, numOfRows); - int32_t code = - initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfCols, pResBlock, keyBufSize, pTaskInfo->id.str); + int32_t code = initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str); + initBasicInfo(&pInfo->binfo, pResBlock); initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pInfo->win); if (code != TSDB_CODE_SUCCESS) { goto _error; } - initResultRowInfo(&pInfo->binfo.resultRowInfo, (int32_t)1); + initResultRowInfo(&pInfo->binfo.resultRowInfo); pOperator->name = "StreamTimeIntervalAggOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_HASH_INTERVAL; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExprInfo; + pOperator->exprSupp.pExprInfo = pExprInfo; pOperator->pTaskInfo = pTaskInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->info = pInfo; pOperator->fpSet = createOperatorFpSet(doOpenIntervalAgg, doStreamIntervalAgg, doStreamIntervalAgg, NULL, @@ -1565,11 +1571,12 @@ _error: // todo handle multiple tables cases. static void doSessionWindowAggImpl(SOperatorInfo* pOperator, SSessionAggOperatorInfo* pInfo, SSDataBlock* pBlock) { SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; + SExprSupp* pSup = &pOperator->exprSupp; SColumnInfoData* pColInfoData = taosArrayGet(pBlock->pDataBlock, pInfo->tsSlotId); bool masterScan = true; - int32_t numOfOutput = pOperator->numOfExprs; + int32_t numOfOutput = pOperator->exprSupp.numOfExprs; int64_t gid = pBlock->info.groupId; int64_t gap = pInfo->gap; @@ -1602,15 +1609,15 @@ static void doSessionWindowAggImpl(SOperatorInfo* pOperator, SSessionAggOperator pRowSup->win.ekey = pRowSup->win.skey; int32_t ret = - setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, &window, masterScan, &pResult, gid, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, &window, masterScan, &pResult, gid, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS) { // null data, too many state code longjmp(pTaskInfo->env, TSDB_CODE_QRY_APP_ERROR); } // pInfo->numOfRows data belong to the current session window updateTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &window, false); - doApplyFunctions(pTaskInfo, pInfo->binfo.pCtx, &window, &pInfo->twAggSup.timeWindowData, pRowSup->startRowIndex, + doApplyFunctions(pTaskInfo, pSup->pCtx, &window, &pInfo->twAggSup.timeWindowData, pRowSup->startRowIndex, pRowSup->numOfRows, NULL, pBlock->info.rows, numOfOutput, TSDB_ORDER_ASC); // here we start a new session window @@ -1622,14 +1629,14 @@ static void doSessionWindowAggImpl(SOperatorInfo* pOperator, SSessionAggOperator SResultRow* pResult = NULL; pRowSup->win.ekey = tsList[pBlock->info.rows - 1]; int32_t ret = - setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, &pRowSup->win, masterScan, &pResult, gid, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, &pRowSup->win, masterScan, &pResult, gid, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS) { // null data, too many state code longjmp(pTaskInfo->env, TSDB_CODE_QRY_APP_ERROR); } updateTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pRowSup->win, false); - doApplyFunctions(pTaskInfo, pInfo->binfo.pCtx, &pRowSup->win, &pInfo->twAggSup.timeWindowData, pRowSup->startRowIndex, + doApplyFunctions(pTaskInfo, pSup->pCtx, &pRowSup->win, &pInfo->twAggSup.timeWindowData, pRowSup->startRowIndex, pRowSup->numOfRows, NULL, pBlock->info.rows, numOfOutput, TSDB_ORDER_ASC); } @@ -1640,10 +1647,11 @@ static SSDataBlock* doSessionWindowAgg(SOperatorInfo* pOperator) { SSessionAggOperatorInfo* pInfo = pOperator->info; SOptrBasicInfo* pBInfo = &pInfo->binfo; + SExprSupp* pSup = &pOperator->exprSupp; if (pOperator->status == OP_RES_TO_RETURN) { doBuildResultDatablock(pOperator, pBInfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); - if (pBInfo->pRes->info.rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (pBInfo->pRes->info.rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); return NULL; } @@ -1663,7 +1671,7 @@ static SSDataBlock* doSessionWindowAgg(SOperatorInfo* pOperator) { } // the pDataBlock are always the same one, no need to call this again - setInputDataBlock(pOperator, pBInfo->pCtx, pBlock, order, MAIN_SCAN, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, order, MAIN_SCAN, true); blockDataUpdateTsWindow(pBlock, pInfo->tsSlotId); doSessionWindowAggImpl(pOperator, pInfo, pBlock); @@ -1678,7 +1686,7 @@ static SSDataBlock* doSessionWindowAgg(SOperatorInfo* pOperator) { initGroupedResultInfo(&pInfo->groupResInfo, pInfo->aggSup.pResultRowHashTable, TSDB_ORDER_ASC); blockDataEnsureCapacity(pBInfo->pRes, pOperator->resultInfo.capacity); doBuildResultDatablock(pOperator, pBInfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); - if (pBInfo->pRes->info.rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (pBInfo->pRes->info.rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); } @@ -1711,10 +1719,11 @@ static SSDataBlock* doTimeslice(SOperatorInfo* pOperator) { STimeSliceOperatorInfo* pSliceInfo = pOperator->info; SSDataBlock* pResBlock = pSliceInfo->binfo.pRes; + SExprSupp* pSup = &pOperator->exprSupp; // if (pOperator->status == OP_RES_TO_RETURN) { // // doBuildResultDatablock(&pRuntimeEnv->groupResInfo, pRuntimeEnv, pIntervalInfo->pRes); -// if (pResBlock->info.rows == 0 || !hashRemainDataInGroupInfo(&pSliceInfo->groupResInfo)) { +// if (pResBlock->info.rows == 0 || !hasDataInGroupInfo(&pSliceInfo->groupResInfo)) { // doSetOperatorCompleted(pOperator); // } // @@ -1733,15 +1742,15 @@ static SSDataBlock* doTimeslice(SOperatorInfo* pOperator) { } // the pDataBlock are always the same one, no need to call this again - setInputDataBlock(pOperator, pSliceInfo->binfo.pCtx, pBlock, order, MAIN_SCAN, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, order, MAIN_SCAN, true); SColumnInfoData* pTsCol = taosArrayGet(pBlock->pDataBlock, 0); for(int32_t i = 0; i < pBlock->info.rows; ++i) { int64_t ts = *(int64_t*) colDataGetData(pTsCol, i); if (ts == pSliceInfo->current) { - for(int32_t j = 0; j < pOperator->numOfExprs; ++j) { - SExprInfo* pExprInfo = &pOperator->pExpr[j]; + for(int32_t j = 0; j < pOperator->exprSupp.numOfExprs; ++j) { + SExprInfo* pExprInfo = &pOperator->exprSupp.pExprInfo[j]; int32_t dstSlot = pExprInfo->base.resSchema.slotId; int32_t srcSlot = pExprInfo->base.pParam[0].pCol->slotId; @@ -1764,8 +1773,8 @@ static SSDataBlock* doTimeslice(SOperatorInfo* pOperator) { int64_t nextTs = *(int64_t*) colDataGetData(pTsCol, i + 1); if (nextTs > pSliceInfo->current) { // output the result - for (int32_t j = 0; j < pOperator->numOfExprs; ++j) { - SExprInfo* pExprInfo = &pOperator->pExpr[j]; + for (int32_t j = 0; j < pOperator->exprSupp.numOfExprs; ++j) { + SExprInfo* pExprInfo = &pOperator->exprSupp.pExprInfo[j]; int32_t dstSlot = pExprInfo->base.resSchema.slotId; int32_t srcSlot = pExprInfo->base.pParam[0].pCol->slotId; @@ -1903,12 +1912,14 @@ SOperatorInfo* createTimeSliceOperatorInfo(SOperatorInfo* downstream, SExprInfo* goto _error; } - int32_t code = initTimesliceInfo(pInfo, pInfo->binfo.pCtx, numOfCols); + SExprSupp* pSup = &pOperator->exprSupp; + + int32_t code = initTimesliceInfo(pInfo, pSup->pCtx, numOfCols); if (code != TSDB_CODE_SUCCESS) { goto _error; } - initResultRowInfo(&pInfo->binfo.resultRowInfo, 8); + initResultRowInfo(&pInfo->binfo.resultRowInfo); pInfo->pFillColInfo = createFillColInfo(pExprInfo, numOfCols, pValNode); pInfo->binfo.pRes = pResultBlock; @@ -1917,8 +1928,8 @@ SOperatorInfo* createTimeSliceOperatorInfo(SOperatorInfo* downstream, SExprInfo* // pOperator->operatorType = OP_AllTimeWindow; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->info = pInfo; pOperator->pTaskInfo = pTaskInfo; @@ -1955,8 +1966,10 @@ SOperatorInfo* createStatewindowOperatorInfo(SOperatorInfo* downstream, SExprInf size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; initResultSizeInfo(pOperator, 4096); - initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExpr, numOfCols, pResBlock, keyBufSize, pTaskInfo->id.str); - initResultRowInfo(&pInfo->binfo.resultRowInfo, 8); + initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExpr, numOfCols, keyBufSize, pTaskInfo->id.str); + initBasicInfo(&pInfo->binfo, pResBlock); + + initResultRowInfo(&pInfo->binfo.resultRowInfo); pInfo->twAggSup = *pTwAggSup; initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pTaskInfo->window); @@ -1966,8 +1979,8 @@ SOperatorInfo* createStatewindowOperatorInfo(SOperatorInfo* downstream, SExprInf pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_MERGE_STATE; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExpr; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.pExprInfo = pExpr; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->pTaskInfo = pTaskInfo; pOperator->info = pInfo; @@ -1984,7 +1997,7 @@ _error: void destroySWindowOperatorInfo(void* param, int32_t numOfOutput) { SSessionAggOperatorInfo* pInfo = (SSessionAggOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); } SOperatorInfo* createSessionAggOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols, @@ -1999,14 +2012,15 @@ SOperatorInfo* createSessionAggOperatorInfo(SOperatorInfo* downstream, SExprInfo size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; initResultSizeInfo(pOperator, 4096); - int32_t code = - initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfCols, pResBlock, keyBufSize, pTaskInfo->id.str); + int32_t code = initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str); if (code != TSDB_CODE_SUCCESS) { goto _error; } + initBasicInfo(&pInfo->binfo, pResBlock); + pInfo->twAggSup = *pTwAggSupp; - initResultRowInfo(&pInfo->binfo.resultRowInfo, 8); + initResultRowInfo(&pInfo->binfo.resultRowInfo); initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pTaskInfo->window); pInfo->tsSlotId = tsSlotId; @@ -2018,8 +2032,8 @@ SOperatorInfo* createSessionAggOperatorInfo(SOperatorInfo* downstream, SExprInfo pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->info = pInfo; pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doSessionWindowAgg, NULL, NULL, @@ -2060,23 +2074,25 @@ void compactFunctions(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx, int3 } } -static void rebuildIntervalWindow(SStreamFinalIntervalOperatorInfo* pInfo, SArray* pWinArray, int32_t groupId, +static void rebuildIntervalWindow(SStreamFinalIntervalOperatorInfo* pInfo, SExprSupp* pSup, SArray* pWinArray, int32_t groupId, int32_t numOfOutput, SExecTaskInfo* pTaskInfo) { int32_t size = taosArrayGetSize(pWinArray); ASSERT(pInfo->pChildren); for (int32_t i = 0; i < size; i++) { STimeWindow* pParentWin = taosArrayGet(pWinArray, i); SResultRow* pCurResult = NULL; - setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, pParentWin, true, &pCurResult, 0, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, pParentWin, true, &pCurResult, 0, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); int32_t numOfChildren = taosArrayGetSize(pInfo->pChildren); for (int32_t j = 0; j < numOfChildren; j++) { SOperatorInfo* pChildOp = taosArrayGetP(pInfo->pChildren, j); SIntervalAggOperatorInfo* pChInfo = pChildOp->info; + SExprSupp* pChildSup = &pChildOp->exprSupp; + SResultRow* pChResult = NULL; - setTimeWindowOutputBuf(&pChInfo->binfo.resultRowInfo, pParentWin, true, &pChResult, 0, pChInfo->binfo.pCtx, - pChildOp->numOfExprs, pChInfo->binfo.rowCellInfoOffset, &pChInfo->aggSup, pTaskInfo); - compactFunctions(pInfo->binfo.pCtx, pChInfo->binfo.pCtx, numOfOutput, pTaskInfo); + setTimeWindowOutputBuf(&pChInfo->binfo.resultRowInfo, pParentWin, true, &pChResult, 0, pChildSup->pCtx, + pChildSup->numOfExprs, pChildSup->rowEntryInfoOffset, &pChInfo->aggSup, pTaskInfo); + compactFunctions(pSup->pCtx, pChildSup->pCtx, numOfOutput, pTaskInfo); } } } @@ -2093,7 +2109,8 @@ static void doHashInterval(SOperatorInfo* pOperatorInfo, SSDataBlock* pSDataBloc SStreamFinalIntervalOperatorInfo* pInfo = (SStreamFinalIntervalOperatorInfo*)pOperatorInfo->info; SResultRowInfo* pResultRowInfo = &(pInfo->binfo.resultRowInfo); SExecTaskInfo* pTaskInfo = pOperatorInfo->pTaskInfo; - int32_t numOfOutput = pOperatorInfo->numOfExprs; + SExprSupp* pSup = &pOperatorInfo->exprSupp; + int32_t numOfOutput = pSup->numOfExprs; int32_t step = 1; bool ascScan = true; TSKEY* tsCols = NULL; @@ -2116,12 +2133,11 @@ static void doHashInterval(SOperatorInfo* pOperatorInfo, SSDataBlock* pSDataBloc isDeletedWindow(&nextWin, tableGroupId, &pInfo->aggSup)) { SArray* pUpWins = taosArrayInit(8, sizeof(STimeWindow)); taosArrayPush(pUpWins, &nextWin); - rebuildIntervalWindow(pInfo, pUpWins, pInfo->binfo.pRes->info.groupId, - pOperatorInfo->numOfExprs, pOperatorInfo->pTaskInfo); + rebuildIntervalWindow(pInfo, pSup, pUpWins, pInfo->binfo.pRes->info.groupId, pSup->numOfExprs, pOperatorInfo->pTaskInfo); taosArrayDestroy(pUpWins); } - int32_t code = setTimeWindowOutputBuf(pResultRowInfo, &nextWin, true, &pResult, tableGroupId, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + int32_t code = setTimeWindowOutputBuf(pResultRowInfo, &nextWin, true, &pResult, tableGroupId, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (code != TSDB_CODE_SUCCESS || pResult == NULL) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } @@ -2135,10 +2151,10 @@ static void doHashInterval(SOperatorInfo* pOperatorInfo, SSDataBlock* pSDataBloc saveResultRow(pResult, tableGroupId, pUpdated); } // window start(end) key interpolation - // doWindowBorderInterpolation(pInfo, pSDataBlock, numOfOutput, pInfo->binfo.pCtx, pResult, &nextWin, startPos, + // doWindowBorderInterpolation(pInfo, pSDataBlock, numOfOutput, pSup->pCtx, pResult, &nextWin, startPos, // forwardRows); updateTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &nextWin, true); - doApplyFunctions(pTaskInfo, pInfo->binfo.pCtx, &nextWin, &pInfo->twAggSup.timeWindowData, startPos, forwardRows, + doApplyFunctions(pTaskInfo, pSup->pCtx, &nextWin, &pInfo->twAggSup.timeWindowData, startPos, forwardRows, tsCols, pSDataBlock->info.rows, numOfOutput, TSDB_ORDER_ASC); int32_t prevEndPos = (forwardRows - 1) * step + startPos; ASSERT(pSDataBlock->info.window.skey > 0 && pSDataBlock->info.window.ekey > 0); @@ -2153,7 +2169,7 @@ static void clearStreamIntervalOperator(SStreamFinalIntervalOperatorInfo* pInfo) taosHashClear(pInfo->aggSup.pResultRowHashTable); clearDiskbasedBuf(pInfo->aggSup.pResultBuf); cleanupResultRowInfo(&pInfo->binfo.resultRowInfo); - initResultRowInfo(&pInfo->binfo.resultRowInfo, 1); + initResultRowInfo(&pInfo->binfo.resultRowInfo); } static void clearUpdateDataBlock(SSDataBlock* pBlock) { @@ -2190,6 +2206,8 @@ static SSDataBlock* doStreamFinalIntervalAgg(SOperatorInfo* pOperator) { SArray* pUpdated = taosArrayInit(4, POINTER_BYTES); TSKEY maxTs = INT64_MIN; + SExprSupp* pSup = &pOperator->exprSupp; + if (pOperator->status == OP_EXEC_DONE) { return NULL; } else if (pOperator->status == OP_RES_TO_RETURN) { @@ -2219,15 +2237,17 @@ static SSDataBlock* doStreamFinalIntervalAgg(SOperatorInfo* pOperator) { if (pBlock->info.type == STREAM_REPROCESS) { SArray* pUpWins = taosArrayInit(8, sizeof(STimeWindow)); - doClearWindows(&pInfo->aggSup, &pInfo->binfo, &pInfo->interval, pInfo->primaryTsIndex, pOperator->numOfExprs, + doClearWindows(&pInfo->aggSup, pSup, &pInfo->interval, pInfo->primaryTsIndex, pOperator->exprSupp.numOfExprs, pBlock, pUpWins); if (isFinalInterval(pInfo)) { int32_t childIndex = getChildIndex(pBlock); SOperatorInfo* pChildOp = taosArrayGetP(pInfo->pChildren, childIndex); SIntervalAggOperatorInfo* pChildInfo = pChildOp->info; - doClearWindows(&pChildInfo->aggSup, &pChildInfo->binfo, &pChildInfo->interval, pChildInfo->primaryTsIndex, - pChildOp->numOfExprs, pBlock, NULL); - rebuildIntervalWindow(pInfo, pUpWins, pInfo->binfo.pRes->info.groupId, pOperator->numOfExprs, + SExprSupp* pChildSup = &pChildOp->exprSupp; + + doClearWindows(&pChildInfo->aggSup, pChildSup, &pChildInfo->interval, pChildInfo->primaryTsIndex, + pChildSup->numOfExprs, pBlock, NULL); + rebuildIntervalWindow(pInfo, pSup, pUpWins, pInfo->binfo.pRes->info.groupId, pOperator->exprSupp.numOfExprs, pOperator->pTaskInfo); taosArrayDestroy(pUpWins); continue; @@ -2241,7 +2261,7 @@ static SSDataBlock* doStreamFinalIntervalAgg(SOperatorInfo* pOperator) { continue; } - setInputDataBlock(pOperator, pInfo->binfo.pCtx, pBlock, pInfo->order, MAIN_SCAN, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, pInfo->order, MAIN_SCAN, true); doHashInterval(pOperator, pBlock, pBlock->info.groupId, pUpdated); if (isFinalInterval(pInfo)) { int32_t chIndex = getChildIndex(pBlock); @@ -2256,7 +2276,7 @@ static SSDataBlock* doStreamFinalIntervalAgg(SOperatorInfo* pOperator) { } SOperatorInfo* pChildOp = taosArrayGetP(pInfo->pChildren, chIndex); SStreamFinalIntervalOperatorInfo* pChInfo = pChildOp->info; - setInputDataBlock(pChildOp, pChInfo->binfo.pCtx, pBlock, pChInfo->order, MAIN_SCAN, true); + setInputDataBlock(pChildOp, pChildOp->exprSupp.pCtx, pBlock, pChInfo->order, MAIN_SCAN, true); doHashInterval(pChildOp, pBlock, pBlock->info.groupId, NULL); } maxTs = TMAX(maxTs, pBlock->info.window.ekey); @@ -2267,7 +2287,7 @@ static SSDataBlock* doStreamFinalIntervalAgg(SOperatorInfo* pOperator) { closeIntervalWindow(pInfo->aggSup.pResultRowHashTable, &pInfo->twAggSup, &pInfo->interval, pUpdated); } - finalizeUpdatedResult(pOperator->numOfExprs, pInfo->aggSup.pResultBuf, pUpdated, pInfo->binfo.rowCellInfoOffset); + finalizeUpdatedResult(pOperator->exprSupp.numOfExprs, pInfo->aggSup.pResultBuf, pUpdated, pSup->rowEntryInfoOffset); initMultiResInfoFromArrayList(&pInfo->groupResInfo, pUpdated); blockDataEnsureCapacity(pInfo->binfo.pRes, pOperator->resultInfo.capacity); doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); @@ -2292,6 +2312,7 @@ SOperatorInfo* createStreamFinalIntervalOperatorInfo(SOperatorInfo* downstream, if (pInfo == NULL || pOperator == NULL) { goto _error; } + pInfo->order = TSDB_ORDER_ASC; pInfo->interval = (SInterval){.interval = pIntervalPhyNode->interval, .sliding = pIntervalPhyNode->sliding, @@ -2311,15 +2332,17 @@ SOperatorInfo* createStreamFinalIntervalOperatorInfo(SOperatorInfo* downstream, int32_t numOfCols = 0; SExprInfo* pExprInfo = createExprInfo(pIntervalPhyNode->window.pFuncs, NULL, &numOfCols); SSDataBlock* pResBlock = createResDataBlock(pPhyNode->pOutputDataBlockDesc); - int32_t code = initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfCols, - pResBlock, keyBufSize, pTaskInfo->id.str); + + int32_t code = initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str); + initBasicInfo(&pInfo->binfo, pResBlock); + ASSERT(numOfCols > 0); - increaseTs(pInfo->binfo.pCtx); + increaseTs(pOperator->exprSupp.pCtx); initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pTaskInfo->window); if (code != TSDB_CODE_SUCCESS) { goto _error; } - initResultRowInfo(&pInfo->binfo.resultRowInfo, (int32_t)1); + initResultRowInfo(&pInfo->binfo.resultRowInfo); pInfo->pChildren = NULL; if (numOfChild > 0) { pInfo->pChildren = taosArrayInit(numOfChild, sizeof(SOperatorInfo)); @@ -2345,9 +2368,9 @@ SOperatorInfo* createStreamFinalIntervalOperatorInfo(SOperatorInfo* downstream, pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_INTERVAL; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExprInfo; + pOperator->exprSupp.pExprInfo = pExprInfo; pOperator->pTaskInfo = pTaskInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->info = pInfo; pOperator->fpSet = @@ -2382,7 +2405,7 @@ void destroyStreamAggSupporter(SStreamAggSupporter* pSup) { void destroyStreamSessionAggOperatorInfo(void* param, int32_t numOfOutput) { SStreamSessionAggOperatorInfo* pInfo = (SStreamSessionAggOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); destroyStreamAggSupporter(&pInfo->streamAggSup); cleanupGroupResInfo(&pInfo->groupResInfo); if (pInfo->pChildren != NULL) { @@ -2397,14 +2420,16 @@ void destroyStreamSessionAggOperatorInfo(void* param, int32_t numOfOutput) { } } -int32_t initBiasicInfo(SOptrBasicInfo* pBasicInfo, SExprInfo* pExprInfo, int32_t numOfCols, SSDataBlock* pResultBlock) { - pBasicInfo->pCtx = createSqlFunctionCtx(pExprInfo, numOfCols, &pBasicInfo->rowCellInfoOffset); - pBasicInfo->pRes = pResultBlock; +int32_t initBasicInfoEx(SOptrBasicInfo* pBasicInfo, SExprSupp* pSup, SExprInfo* pExprInfo, int32_t numOfCols, SSDataBlock* pResultBlock) { + initExprSupp(pSup, pExprInfo, numOfCols); + initBasicInfo(pBasicInfo, pResultBlock); + for (int32_t i = 0; i < numOfCols; ++i) { - pBasicInfo->pCtx[i].pBuf = NULL; + pSup->pCtx[i].pBuf = NULL; } + ASSERT(numOfCols > 0); - increaseTs(pBasicInfo->pCtx); + increaseTs(pSup->pCtx); return TSDB_CODE_SUCCESS; } @@ -2425,24 +2450,29 @@ int32_t initSessionAggSupporter(SStreamAggSupporter* pSup, const char* pKey, Sql return initStreamAggSupporter(pSup, pKey, pCtx, numOfOutput, sizeof(SResultWindowInfo)); } -SOperatorInfo* createStreamSessionAggOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols, - SSDataBlock* pResBlock, int64_t gap, int32_t tsSlotId, - STimeWindowAggSupp* pTwAggSupp, SExecTaskInfo* pTaskInfo) { - int32_t code = TSDB_CODE_OUT_OF_MEMORY; +SOperatorInfo* createStreamSessionAggOperatorInfo(SOperatorInfo* downstream, SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo) { + SSessionWinodwPhysiNode* pSessionNode = (SSessionWinodwPhysiNode*)pPhyNode; + int32_t numOfCols = 0; + SExprInfo* pExprInfo = createExprInfo(pSessionNode->window.pFuncs, NULL, &numOfCols); + SSDataBlock* pResBlock = createResDataBlock(pPhyNode->pOutputDataBlockDesc); + int32_t tsSlotId = ((SColumnNode*)pSessionNode->window.pTspk)->slotId; + int32_t code = TSDB_CODE_OUT_OF_MEMORY; SStreamSessionAggOperatorInfo* pInfo = taosMemoryCalloc(1, sizeof(SStreamSessionAggOperatorInfo)); - SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); + SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); if (pInfo == NULL || pOperator == NULL) { goto _error; } initResultSizeInfo(pOperator, 4096); + SExprSupp* pSup = &pOperator->exprSupp; - code = initBiasicInfo(&pInfo->binfo, pExprInfo, numOfCols, pResBlock); + code = initBasicInfoEx(&pInfo->binfo, pSup, pExprInfo, numOfCols, pResBlock); if (code != TSDB_CODE_SUCCESS) { goto _error; } - code = initSessionAggSupporter(&pInfo->streamAggSup, "StreamSessionAggOperatorInfo", pInfo->binfo.pCtx, numOfCols); + + code = initSessionAggSupporter(&pInfo->streamAggSup, "StreamSessionAggOperatorInfo", pSup->pCtx, numOfCols); if (code != TSDB_CODE_SUCCESS) { goto _error; } @@ -2451,14 +2481,18 @@ SOperatorInfo* createStreamSessionAggOperatorInfo(SOperatorInfo* downstream, SEx if (pInfo->pDummyCtx == NULL) { goto _error; } - initDummyFunction(pInfo->pDummyCtx, pInfo->binfo.pCtx, numOfCols); + initDummyFunction(pInfo->pDummyCtx, pSup->pCtx, numOfCols); - pInfo->twAggSup = *pTwAggSupp; - initResultRowInfo(&pInfo->binfo.resultRowInfo, 8); + pInfo->twAggSup = (STimeWindowAggSupp) { + .waterMark = pSessionNode->window.watermark, + .calTrigger = pSessionNode->window.triggerType, + .maxTs = INT64_MIN}; + + initResultRowInfo(&pInfo->binfo.resultRowInfo); initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pTaskInfo->window); pInfo->primaryTsIndex = tsSlotId; - pInfo->gap = gap; + pInfo->gap = pSessionNode->gap; pInfo->binfo.pRes = pResBlock; pInfo->order = TSDB_ORDER_ASC; _hash_fn_t hashFn = taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY); @@ -2472,8 +2506,8 @@ SOperatorInfo* createStreamSessionAggOperatorInfo(SOperatorInfo* downstream, SEx pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->info = pInfo; pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doStreamSessionAgg, NULL, NULL, destroyStreamSessionAggOperatorInfo, @@ -2585,7 +2619,7 @@ int32_t updateSessionWindowInfo(SResultWindowInfo* pWinInfo, TSKEY* pTs, int32_t } static int32_t setWindowOutputBuf(SResultWindowInfo* pWinInfo, SResultRow** pResult, SqlFunctionCtx* pCtx, - uint64_t groupId, int32_t numOfOutput, int32_t* rowCellInfoOffset, + uint64_t groupId, int32_t numOfOutput, int32_t* rowEntryInfoOffset, SStreamAggSupporter* pAggSup, SExecTaskInfo* pTaskInfo) { assert(pWinInfo->win.skey <= pWinInfo->win.ekey); // too many time window in query @@ -2614,39 +2648,42 @@ static int32_t setWindowOutputBuf(SResultWindowInfo* pWinInfo, SResultRow** pRes // set time window for current result (*pResult)->win = pWinInfo->win; - setResultRowInitCtx(*pResult, pCtx, numOfOutput, rowCellInfoOffset); + setResultRowInitCtx(*pResult, pCtx, numOfOutput, rowEntryInfoOffset); return TSDB_CODE_SUCCESS; } static int32_t doOneWindowAggImpl(int32_t tsColId, SOptrBasicInfo* pBinfo, SStreamAggSupporter* pAggSup, SColumnInfoData* pTimeWindowData, SSDataBlock* pSDataBlock, SResultWindowInfo* pCurWin, SResultRow** pResult, int32_t startIndex, int32_t winRows, - int32_t numOutput, SExecTaskInfo* pTaskInfo) { + int32_t numOutput, SOperatorInfo* pOperator) { + SExprSupp* pSup = &pOperator->exprSupp; + SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; + SColumnInfoData* pColDataInfo = taosArrayGet(pSDataBlock->pDataBlock, tsColId); TSKEY* tsCols = (int64_t*)pColDataInfo->pData; - int32_t code = setWindowOutputBuf(pCurWin, pResult, pBinfo->pCtx, pSDataBlock->info.groupId, numOutput, - pBinfo->rowCellInfoOffset, pAggSup, pTaskInfo); + int32_t code = setWindowOutputBuf(pCurWin, pResult, pSup->pCtx, pSDataBlock->info.groupId, numOutput, + pSup->rowEntryInfoOffset, pAggSup, pTaskInfo); if (code != TSDB_CODE_SUCCESS || (*pResult) == NULL) { return TSDB_CODE_QRY_OUT_OF_MEMORY; } updateTimeWindowInfo(pTimeWindowData, &pCurWin->win, true); - doApplyFunctions(pTaskInfo, pBinfo->pCtx, &pCurWin->win, pTimeWindowData, startIndex, winRows, tsCols, + doApplyFunctions(pTaskInfo, pSup->pCtx, &pCurWin->win, pTimeWindowData, startIndex, winRows, tsCols, pSDataBlock->info.rows, numOutput, TSDB_ORDER_ASC); return TSDB_CODE_SUCCESS; } static int32_t doOneWindowAgg(SStreamSessionAggOperatorInfo* pInfo, SSDataBlock* pSDataBlock, SResultWindowInfo* pCurWin, SResultRow** pResult, int32_t startIndex, int32_t winRows, - int32_t numOutput, SExecTaskInfo* pTaskInfo) { + int32_t numOutput, SOperatorInfo * pOperator) { return doOneWindowAggImpl(pInfo->primaryTsIndex, &pInfo->binfo, &pInfo->streamAggSup, &pInfo->twAggSup.timeWindowData, - pSDataBlock, pCurWin, pResult, startIndex, winRows, numOutput, pTaskInfo); + pSDataBlock, pCurWin, pResult, startIndex, winRows, numOutput, pOperator); } static int32_t doOneStateWindowAgg(SStreamStateAggOperatorInfo* pInfo, SSDataBlock* pSDataBlock, SResultWindowInfo* pCurWin, SResultRow** pResult, int32_t startIndex, - int32_t winRows, int32_t numOutput, SExecTaskInfo* pTaskInfo) { + int32_t winRows, int32_t numOutput, SOperatorInfo * pOperator) { return doOneWindowAggImpl(pInfo->primaryTsIndex, &pInfo->binfo, &pInfo->streamAggSup, &pInfo->twAggSup.timeWindowData, - pSDataBlock, pCurWin, pResult, startIndex, winRows, numOutput, pTaskInfo); + pSDataBlock, pCurWin, pResult, startIndex, winRows, numOutput, pOperator); } int32_t getNumCompactWindow(SArray* pWinInfos, int32_t startIndex, int64_t gap) { @@ -2664,10 +2701,13 @@ int32_t getNumCompactWindow(SArray* pWinInfos, int32_t startIndex, int64_t gap) } void compactTimeWindow(SStreamSessionAggOperatorInfo* pInfo, int32_t startIndex, int32_t num, uint64_t groupId, - int32_t numOfOutput, SExecTaskInfo* pTaskInfo, SHashObj* pStUpdated, SHashObj* pStDeleted) { + int32_t numOfOutput, SHashObj* pStUpdated, SHashObj* pStDeleted, SOperatorInfo* pOperator) { + SExprSupp* pSup = &pOperator->exprSupp; + SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; + SResultWindowInfo* pCurWin = taosArrayGet(pInfo->streamAggSup.pCurWins, startIndex); SResultRow* pCurResult = NULL; - setWindowOutputBuf(pCurWin, &pCurResult, pInfo->binfo.pCtx, groupId, numOfOutput, pInfo->binfo.rowCellInfoOffset, + setWindowOutputBuf(pCurWin, &pCurResult, pSup->pCtx, groupId, numOfOutput, pSup->rowEntryInfoOffset, &pInfo->streamAggSup, pTaskInfo); num += startIndex + 1; ASSERT(num <= taosArrayGetSize(pInfo->streamAggSup.pCurWins)); @@ -2675,10 +2715,10 @@ void compactTimeWindow(SStreamSessionAggOperatorInfo* pInfo, int32_t startIndex, for (int32_t i = startIndex + 1; i < num; i++) { SResultWindowInfo* pWinInfo = taosArrayGet(pInfo->streamAggSup.pCurWins, i); SResultRow* pWinResult = NULL; - setWindowOutputBuf(pWinInfo, &pWinResult, pInfo->pDummyCtx, groupId, numOfOutput, pInfo->binfo.rowCellInfoOffset, + setWindowOutputBuf(pWinInfo, &pWinResult, pInfo->pDummyCtx, groupId, numOfOutput, pSup->rowEntryInfoOffset, &pInfo->streamAggSup, pTaskInfo); pCurWin->win.ekey = TMAX(pCurWin->win.ekey, pWinInfo->win.ekey); - compactFunctions(pInfo->binfo.pCtx, pInfo->pDummyCtx, numOfOutput, pTaskInfo); + compactFunctions(pSup->pCtx, pInfo->pDummyCtx, numOfOutput, pTaskInfo); taosHashRemove(pStUpdated, &pWinInfo->pos, sizeof(SResultRowPosition)); if (pWinInfo->isOutput) { taosHashPut(pStDeleted, &pWinInfo->pos, sizeof(SResultRowPosition), &pWinInfo->win.skey, sizeof(TSKEY)); @@ -2698,7 +2738,7 @@ static void doStreamSessionAggImpl(SOperatorInfo* pOperator, SSDataBlock* pSData SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; SStreamSessionAggOperatorInfo* pInfo = pOperator->info; bool masterScan = true; - int32_t numOfOutput = pOperator->numOfExprs; + int32_t numOfOutput = pOperator->exprSupp.numOfExprs; uint64_t groupId = pSDataBlock->info.groupId; int64_t gap = pInfo->gap; int64_t code = TSDB_CODE_SUCCESS; @@ -2721,17 +2761,17 @@ static void doStreamSessionAggImpl(SOperatorInfo* pOperator, SSDataBlock* pSData int32_t winIndex = 0; SResultWindowInfo* pCurWin = getSessionTimeWindow(pAggSup, tsCols[i], groupId, gap, &winIndex); winRows = updateSessionWindowInfo(pCurWin, tsCols, pSDataBlock->info.rows, i, pInfo->gap, pStDeleted); - code = doOneWindowAgg(pInfo, pSDataBlock, pCurWin, &pResult, i, winRows, numOfOutput, pTaskInfo); + code = doOneWindowAgg(pInfo, pSDataBlock, pCurWin, &pResult, i, winRows, numOfOutput, pOperator); if (code != TSDB_CODE_SUCCESS || pResult == NULL) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } // window start(end) key interpolation - // doWindowBorderInterpolation(pOperatorInfo, pSDataBlock, pInfo->binfo.pCtx, pResult, &nextWin, startPos, + // doWindowBorderInterpolation(pOperatorInfo, pSDataBlock, pSup->pCtx, pResult, &nextWin, startPos, // forwardRows, // pInfo->order, false); int32_t winNum = getNumCompactWindow(pAggSup->pCurWins, winIndex, gap); if (winNum > 0) { - compactTimeWindow(pInfo, winIndex, winNum, groupId, numOfOutput, pTaskInfo, pStUpdated, pStDeleted); + compactTimeWindow(pInfo, winIndex, winNum, groupId, numOfOutput, pStUpdated, pStDeleted, pOperator); } pCurWin->isClosed = false; if (pInfo->twAggSup.calTrigger == STREAM_TRIGGER_AT_ONCE) { @@ -2746,7 +2786,7 @@ static void doStreamSessionAggImpl(SOperatorInfo* pOperator, SSDataBlock* pSData } } -static void doClearSessionWindows(SStreamAggSupporter* pAggSup, SOptrBasicInfo* pBinfo, SSDataBlock* pBlock, +static void doClearSessionWindows(SStreamAggSupporter* pAggSup, SExprSupp* pSup, SSDataBlock* pBlock, int32_t tsIndex, int32_t numOfOutput, int64_t gap, SArray* result) { SColumnInfoData* pColDataInfo = taosArrayGet(pBlock->pDataBlock, tsIndex); TSKEY* tsCols = (TSKEY*)pColDataInfo->pData; @@ -2756,7 +2796,7 @@ static void doClearSessionWindows(SStreamAggSupporter* pAggSup, SOptrBasicInfo* SResultWindowInfo* pCurWin = getSessionTimeWindow(pAggSup, tsCols[i], pBlock->info.groupId, gap, &winIndex); step = updateSessionWindowInfo(pCurWin, tsCols, pBlock->info.rows, i, gap, NULL); ASSERT(isInWindow(pCurWin, tsCols[i], gap)); - doClearWindowImpl(&pCurWin->pos, pAggSup->pResultBuf, pBinfo, numOfOutput); + doClearWindowImpl(&pCurWin->pos, pAggSup->pResultBuf, pSup, numOfOutput); if (result) { taosArrayPush(result, pCurWin); } @@ -2802,13 +2842,17 @@ void doBuildDeleteDataBlock(SHashObj* pStDeleted, SSDataBlock* pBlock, void** It } static void rebuildTimeWindow(SStreamSessionAggOperatorInfo* pInfo, SArray* pWinArray, int32_t groupId, - int32_t numOfOutput, SExecTaskInfo* pTaskInfo) { + int32_t numOfOutput, SOperatorInfo* pOperator) { + SExprSupp* pSup = &pOperator->exprSupp; + SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; + int32_t size = taosArrayGetSize(pWinArray); ASSERT(pInfo->pChildren); + for (int32_t i = 0; i < size; i++) { SResultWindowInfo* pParentWin = taosArrayGet(pWinArray, i); SResultRow* pCurResult = NULL; - setWindowOutputBuf(pParentWin, &pCurResult, pInfo->binfo.pCtx, groupId, numOfOutput, pInfo->binfo.rowCellInfoOffset, + setWindowOutputBuf(pParentWin, &pCurResult, pSup->pCtx, groupId, numOfOutput, pSup->rowEntryInfoOffset, &pInfo->streamAggSup, pTaskInfo); int32_t numOfChildren = taosArrayGetSize(pInfo->pChildren); for (int32_t j = 0; j < numOfChildren; j++) { @@ -2821,9 +2865,9 @@ static void rebuildTimeWindow(SStreamSessionAggOperatorInfo* pInfo, SArray* pWin SResultWindowInfo* pcw = taosArrayGet(pChWins, k); if (pParentWin->win.skey <= pcw->win.skey && pcw->win.ekey <= pParentWin->win.ekey) { SResultRow* pChResult = NULL; - setWindowOutputBuf(pcw, &pChResult, pChInfo->binfo.pCtx, groupId, numOfOutput, - pChInfo->binfo.rowCellInfoOffset, &pChInfo->streamAggSup, pTaskInfo); - compactFunctions(pInfo->binfo.pCtx, pChInfo->binfo.pCtx, numOfOutput, pTaskInfo); + setWindowOutputBuf(pcw, &pChResult, pChild->exprSupp.pCtx, groupId, numOfOutput, + pChild->exprSupp.rowEntryInfoOffset, &pChInfo->streamAggSup, pTaskInfo); + compactFunctions(pSup->pCtx, pChild->exprSupp.pCtx, numOfOutput, pTaskInfo); continue; } break; @@ -2888,6 +2932,7 @@ static SSDataBlock* doStreamSessionAgg(SOperatorInfo* pOperator) { return NULL; } + SExprSupp* pSup = &pOperator->exprSupp; SStreamSessionAggOperatorInfo* pInfo = pOperator->info; SOptrBasicInfo* pBInfo = &pInfo->binfo; if (pOperator->status == OP_RES_TO_RETURN) { @@ -2896,7 +2941,7 @@ static SSDataBlock* doStreamSessionAgg(SOperatorInfo* pOperator) { return pInfo->pDelRes; } doBuildResultDatablock(pOperator, pBInfo, &pInfo->groupResInfo, pInfo->streamAggSup.pResultBuf); - if (pBInfo->pRes->info.rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (pBInfo->pRes->info.rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); } return pBInfo->pRes->info.rows == 0 ? NULL : pBInfo->pRes; @@ -2914,14 +2959,14 @@ static SSDataBlock* doStreamSessionAgg(SOperatorInfo* pOperator) { if (pBlock->info.type == STREAM_REPROCESS) { SArray* pWins = taosArrayInit(16, sizeof(SResultWindowInfo)); - doClearSessionWindows(&pInfo->streamAggSup, &pInfo->binfo, pBlock, 0, pOperator->numOfExprs, pInfo->gap, pWins); + doClearSessionWindows(&pInfo->streamAggSup, &pOperator->exprSupp, pBlock, 0, pOperator->exprSupp.numOfExprs, pInfo->gap, pWins); if (isFinalSession(pInfo)) { int32_t childIndex = 0; // Todo(liuyao) get child id from SSDataBlock SOperatorInfo* pChildOp = taosArrayGetP(pInfo->pChildren, childIndex); SStreamSessionAggOperatorInfo* pChildInfo = pChildOp->info; - doClearSessionWindows(&pChildInfo->streamAggSup, &pChildInfo->binfo, pBlock, 0, pChildOp->numOfExprs, + doClearSessionWindows(&pChildInfo->streamAggSup, &pChildOp->exprSupp, pBlock, 0, pChildOp->exprSupp.numOfExprs, pChildInfo->gap, NULL); - rebuildTimeWindow(pInfo, pWins, pBlock->info.groupId, pOperator->numOfExprs, pOperator->pTaskInfo); + rebuildTimeWindow(pInfo, pWins, pBlock->info.groupId, pOperator->exprSupp.numOfExprs, pOperator); } taosArrayDestroy(pWins); continue; @@ -2931,7 +2976,7 @@ static SSDataBlock* doStreamSessionAgg(SOperatorInfo* pOperator) { } // the pDataBlock are always the same one, no need to call this again - setInputDataBlock(pOperator, pBInfo->pCtx, pBlock, TSDB_ORDER_ASC, MAIN_SCAN, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, TSDB_ORDER_ASC, MAIN_SCAN, true); if (isFinalSession(pInfo)) { int32_t childIndex = 0; // Todo(liuyao) get child id from SSDataBlock SOptrBasicInfo* pChildOp = taosArrayGetP(pInfo->pChildren, childIndex); @@ -2948,8 +2993,8 @@ static SSDataBlock* doStreamSessionAgg(SOperatorInfo* pOperator) { copyUpdateResult(pStUpdated, pUpdated); taosHashCleanup(pStUpdated); - finalizeUpdatedResult(pOperator->numOfExprs, pInfo->streamAggSup.pResultBuf, pUpdated, - pInfo->binfo.rowCellInfoOffset); + finalizeUpdatedResult(pOperator->exprSupp.numOfExprs, pInfo->streamAggSup.pResultBuf, pUpdated, + pSup->rowEntryInfoOffset); initMultiResInfoFromArrayList(&pInfo->groupResInfo, pUpdated); blockDataEnsureCapacity(pInfo->binfo.pRes, pOperator->resultInfo.capacity); doBuildDeleteDataBlock(pInfo->pStDeleted, pInfo->pDelRes, &pInfo->pDelIterator); @@ -2960,25 +3005,20 @@ static SSDataBlock* doStreamSessionAgg(SOperatorInfo* pOperator) { return pBInfo->pRes->info.rows == 0 ? NULL : pBInfo->pRes; } -SOperatorInfo* createStreamFinalSessionAggOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, - int32_t numOfCols, SSDataBlock* pResBlock, int64_t gap, - int32_t tsSlotId, STimeWindowAggSupp* pTwAggSupp, - SExecTaskInfo* pTaskInfo) { - int32_t code = TSDB_CODE_OUT_OF_MEMORY; - SStreamSessionAggOperatorInfo* pInfo = NULL; - SOperatorInfo* pOperator = createStreamSessionAggOperatorInfo(downstream, pExprInfo, numOfCols, pResBlock, gap, - tsSlotId, pTwAggSupp, pTaskInfo); +SOperatorInfo* createStreamFinalSessionAggOperatorInfo(SOperatorInfo* downstream, + SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo, int32_t numOfChild) { + int32_t code = TSDB_CODE_OUT_OF_MEMORY; + SOperatorInfo* pOperator = createStreamSessionAggOperatorInfo(downstream, pPhyNode, pTaskInfo); if (pOperator == NULL) { goto _error; } pOperator->name = "StreamFinalSessionWindowAggOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION; - int32_t numOfChild = 1; // Todo(liuyao) get it from phy plan - pInfo = pOperator->info; + SStreamSessionAggOperatorInfo* pInfo = pOperator->info; pInfo->pChildren = taosArrayInit(8, sizeof(void*)); for (int32_t i = 0; i < numOfChild; i++) { SOperatorInfo* pChild = - createStreamSessionAggOperatorInfo(NULL, pExprInfo, numOfCols, NULL, gap, tsSlotId, pTwAggSupp, pTaskInfo); + createStreamSessionAggOperatorInfo(NULL, pPhyNode, pTaskInfo); if (pChild == NULL) { goto _error; } @@ -2988,7 +3028,7 @@ SOperatorInfo* createStreamFinalSessionAggOperatorInfo(SOperatorInfo* downstream _error: if (pInfo != NULL) { - destroyStreamSessionAggOperatorInfo(pInfo, numOfCols); + destroyStreamSessionAggOperatorInfo(pInfo, pOperator->exprSupp.numOfExprs); } taosMemoryFreeClear(pInfo); @@ -2999,7 +3039,7 @@ _error: void destroyStreamStateOperatorInfo(void* param, int32_t numOfOutput) { SStreamStateAggOperatorInfo* pInfo = (SStreamStateAggOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); destroyStreamAggSupporter(&pInfo->streamAggSup); cleanupGroupResInfo(&pInfo->groupResInfo); if (pInfo->pChildren != NULL) { @@ -3207,7 +3247,7 @@ static void doStreamStateAggImpl(SOperatorInfo* pOperator, SSDataBlock* pSDataBl SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; SStreamStateAggOperatorInfo* pInfo = pOperator->info; bool masterScan = true; - int32_t numOfOutput = pOperator->numOfExprs; + int32_t numOfOutput = pOperator->exprSupp.numOfExprs; int64_t groupId = pSDataBlock->info.groupId; int64_t code = TSDB_CODE_SUCCESS; int32_t step = 1; @@ -3239,7 +3279,7 @@ static void doStreamStateAggImpl(SOperatorInfo* pOperator, SSDataBlock* pSDataBl deleteWindow(pAggSup->pCurWins, winIndex); continue; } - code = doOneStateWindowAgg(pInfo, pSDataBlock, &pCurWin->winInfo, &pResult, i, winRows, numOfOutput, pTaskInfo); + code = doOneStateWindowAgg(pInfo, pSDataBlock, &pCurWin->winInfo, &pResult, i, winRows, numOfOutput, pOperator); if (code != TSDB_CODE_SUCCESS || pResult == NULL) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } @@ -3261,6 +3301,7 @@ static SSDataBlock* doStreamStateAgg(SOperatorInfo* pOperator) { return NULL; } + SExprSupp* pSup = &pOperator->exprSupp; SStreamStateAggOperatorInfo* pInfo = pOperator->info; SOptrBasicInfo* pBInfo = &pInfo->binfo; if (pOperator->status == OP_RES_TO_RETURN) { @@ -3269,7 +3310,7 @@ static SSDataBlock* doStreamStateAgg(SOperatorInfo* pOperator) { return pInfo->pDelRes; } doBuildResultDatablock(pOperator, pBInfo, &pInfo->groupResInfo, pInfo->streamAggSup.pResultBuf); - if (pBInfo->pRes->info.rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (pBInfo->pRes->info.rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); } return pBInfo->pRes->info.rows == 0 ? NULL : pBInfo->pRes; @@ -3295,7 +3336,7 @@ static SSDataBlock* doStreamStateAgg(SOperatorInfo* pOperator) { } // the pDataBlock are always the same one, no need to call this again - setInputDataBlock(pOperator, pBInfo->pCtx, pBlock, TSDB_ORDER_ASC, MAIN_SCAN, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, TSDB_ORDER_ASC, MAIN_SCAN, true); doStreamStateAggImpl(pOperator, pBlock, pSeUpdated, pInfo->pSeDeleted); pInfo->twAggSup.maxTs = TMAX(pInfo->twAggSup.maxTs, pBlock->info.window.ekey); } @@ -3307,8 +3348,8 @@ static SSDataBlock* doStreamStateAgg(SOperatorInfo* pOperator) { copyUpdateResult(pSeUpdated, pUpdated); taosHashCleanup(pSeUpdated); - finalizeUpdatedResult(pOperator->numOfExprs, pInfo->streamAggSup.pResultBuf, pUpdated, - pInfo->binfo.rowCellInfoOffset); + finalizeUpdatedResult(pOperator->exprSupp.numOfExprs, pInfo->streamAggSup.pResultBuf, pUpdated, + pSup->rowEntryInfoOffset); initMultiResInfoFromArrayList(&pInfo->groupResInfo, pUpdated); blockDataEnsureCapacity(pInfo->binfo.pRes, pOperator->resultInfo.capacity); doBuildDeleteDataBlock(pInfo->pSeDeleted, pInfo->pDelRes, &pInfo->pDelIterator); @@ -3337,12 +3378,14 @@ SOperatorInfo* createStreamStateAggOperatorInfo(SOperatorInfo* downstream, SPhys goto _error; } + SExprSupp* pSup = &pOperator->exprSupp; + int32_t numOfCols = 0; SExprInfo* pExprInfo = createExprInfo(pStateNode->window.pFuncs, NULL, &numOfCols); pInfo->stateCol = extractColumnFromColumnNode(pColNode); initResultSizeInfo(pOperator, 4096); - initResultRowInfo(&pInfo->binfo.resultRowInfo, 8); + initResultRowInfo(&pInfo->binfo.resultRowInfo); pInfo->twAggSup = (STimeWindowAggSupp){ .waterMark = pStateNode->window.watermark, .calTrigger = pStateNode->window.triggerType, @@ -3350,12 +3393,12 @@ SOperatorInfo* createStreamStateAggOperatorInfo(SOperatorInfo* downstream, SPhys }; initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pTaskInfo->window); - code = initBiasicInfo(&pInfo->binfo, pExprInfo, numOfCols, pResBlock); + code = initBasicInfoEx(&pInfo->binfo, pSup, pExprInfo, numOfCols, pResBlock); if (code != TSDB_CODE_SUCCESS) { goto _error; } - code = initStateAggSupporter(&pInfo->streamAggSup, "StreamStateAggOperatorInfo", pInfo->binfo.pCtx, numOfCols); + code = initStateAggSupporter(&pInfo->streamAggSup, "StreamStateAggOperatorInfo", pSup->pCtx, numOfCols); if (code != TSDB_CODE_SUCCESS) { goto _error; } @@ -3365,7 +3408,7 @@ SOperatorInfo* createStreamStateAggOperatorInfo(SOperatorInfo* downstream, SPhys goto _error; } - initDummyFunction(pInfo->pDummyCtx, pInfo->binfo.pCtx, numOfCols); + initDummyFunction(pInfo->pDummyCtx, pSup->pCtx, numOfCols); pInfo->primaryTsIndex = tsSlotId; pInfo->order = TSDB_ORDER_ASC; _hash_fn_t hashFn = taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY); @@ -3379,8 +3422,8 @@ SOperatorInfo* createStreamStateAggOperatorInfo(SOperatorInfo* downstream, SPhys pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_STREAM_STATE; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; - pOperator->numOfExprs = numOfCols; - pOperator->pExpr = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfCols; + pOperator->exprSupp.pExprInfo = pExprInfo; pOperator->pTaskInfo = pTaskInfo; pOperator->info = pInfo; pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doStreamStateAgg, NULL, NULL, @@ -3418,6 +3461,8 @@ static int32_t outputMergeIntervalResult(SOperatorInfo* pOperatorInfo, uint64_t SMergeIntervalAggOperatorInfo* miaInfo = pOperatorInfo->info; SIntervalAggOperatorInfo* iaInfo = &miaInfo->intervalAggOperatorInfo; SExecTaskInfo* pTaskInfo = pOperatorInfo->pTaskInfo; + + SExprSupp* pSup = &pOperatorInfo->exprSupp; bool ascScan = (iaInfo->order == TSDB_ORDER_ASC); SET_RES_WINDOW_KEY(iaInfo->aggSup.keyBuf, &wstartTs, TSDB_KEYSIZE, tableGroupId); @@ -3425,8 +3470,8 @@ static int32_t outputMergeIntervalResult(SOperatorInfo* pOperatorInfo, uint64_t GET_RES_WINDOW_KEY_LEN(TSDB_KEYSIZE)); ASSERT(p1 != NULL); - finalizeResultRowIntoResultDataBlock(iaInfo->aggSup.pResultBuf, p1, iaInfo->binfo.pCtx, pOperatorInfo->pExpr, - pOperatorInfo->numOfExprs, iaInfo->binfo.rowCellInfoOffset, pResultBlock, + finalizeResultRowIntoResultDataBlock(iaInfo->aggSup.pResultBuf, p1, pSup->pCtx, pSup->pExprInfo, + pSup->numOfExprs, pSup->rowEntryInfoOffset, pResultBlock, pTaskInfo); taosHashRemove(iaInfo->aggSup.pResultRowHashTable, iaInfo->aggSup.keyBuf, GET_RES_WINDOW_KEY_LEN(TSDB_KEYSIZE)); @@ -3439,9 +3484,10 @@ static void doMergeIntervalAggImpl(SOperatorInfo* pOperatorInfo, SResultRowInfo* SIntervalAggOperatorInfo* iaInfo = &miaInfo->intervalAggOperatorInfo; SExecTaskInfo* pTaskInfo = pOperatorInfo->pTaskInfo; + SExprSupp* pSup = &pOperatorInfo->exprSupp; int32_t startPos = 0; - int32_t numOfOutput = pOperatorInfo->numOfExprs; + int32_t numOfOutput = pSup->numOfExprs; int64_t* tsCols = extractTsCol(pBlock, iaInfo); uint64_t tableGroupId = pBlock->info.groupId; TSKEY blockStartTs = getStartTsKey(&pBlock->info.window, tsCols); @@ -3453,8 +3499,8 @@ static void doMergeIntervalAggImpl(SOperatorInfo* pOperatorInfo, SResultRowInfo* //TODO: remove the hash table usage (groupid + winkey => result row position) int32_t ret = - setTimeWindowOutputBuf(pResultRowInfo, &win, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, iaInfo->binfo.pCtx, - numOfOutput, iaInfo->binfo.rowCellInfoOffset, &iaInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(pResultRowInfo, &win, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &iaInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS || pResult == NULL) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } @@ -3471,7 +3517,7 @@ static void doMergeIntervalAggImpl(SOperatorInfo* pOperatorInfo, SResultRowInfo* continue; } else { updateTimeWindowInfo(&iaInfo->twAggSup.timeWindowData, &currWin, true); - doApplyFunctions(pTaskInfo, iaInfo->binfo.pCtx, &currWin, &iaInfo->twAggSup.timeWindowData, startPos, + doApplyFunctions(pTaskInfo, pSup->pCtx, &currWin, &iaInfo->twAggSup.timeWindowData, startPos, currPos - startPos, tsCols, pBlock->info.rows, numOfOutput, iaInfo->order); outputMergeIntervalResult(pOperatorInfo, tableGroupId, pResultBlock, currTs); @@ -3480,15 +3526,15 @@ static void doMergeIntervalAggImpl(SOperatorInfo* pOperatorInfo, SResultRowInfo* currWin.skey = currTs; currWin.ekey = taosTimeAdd(currWin.skey, iaInfo->interval.interval, iaInfo->interval.intervalUnit, iaInfo->interval.precision) - 1; startPos = currPos; - ret = setTimeWindowOutputBuf(pResultRowInfo, &currWin, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, iaInfo->binfo.pCtx, - numOfOutput, iaInfo->binfo.rowCellInfoOffset, &iaInfo->aggSup, pTaskInfo); + ret = setTimeWindowOutputBuf(pResultRowInfo, &currWin, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &iaInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS || pResult == NULL) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } } } updateTimeWindowInfo(&iaInfo->twAggSup.timeWindowData, &currWin, true); - doApplyFunctions(pTaskInfo, iaInfo->binfo.pCtx, &currWin, &iaInfo->twAggSup.timeWindowData, startPos, + doApplyFunctions(pTaskInfo, pSup->pCtx, &currWin, &iaInfo->twAggSup.timeWindowData, startPos, currPos - startPos, tsCols, pBlock->info.rows, numOfOutput, iaInfo->order); outputMergeIntervalResult(pOperatorInfo, tableGroupId, pResultBlock, currTs); @@ -3503,6 +3549,7 @@ static SSDataBlock* doMergeIntervalAgg(SOperatorInfo* pOperator) { return NULL; } + SExprSupp* pSup = &pOperator->exprSupp; SSDataBlock* pRes = iaInfo->binfo.pRes; blockDataCleanup(pRes); blockDataEnsureCapacity(pRes, pOperator->resultInfo.capacity); @@ -3533,7 +3580,7 @@ static SSDataBlock* doMergeIntervalAgg(SOperatorInfo* pOperator) { } getTableScanInfo(pOperator, &iaInfo->order, &scanFlag); - setInputDataBlock(pOperator, iaInfo->binfo.pCtx, pBlock, iaInfo->order, scanFlag, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, iaInfo->order, scanFlag, true); doMergeIntervalAggImpl(pOperator, &iaInfo->binfo.resultRowInfo, pBlock, scanFlag, pRes); if (pRes->info.rows >= pOperator->resultInfo.threshold) { @@ -3563,42 +3610,40 @@ SOperatorInfo* createMergeIntervalOperatorInfo(SOperatorInfo* downstream, SExprI } SIntervalAggOperatorInfo* iaInfo = &miaInfo->intervalAggOperatorInfo; + SExprSupp* pSup = &pOperator->exprSupp; - iaInfo->win = pTaskInfo->window; - iaInfo->order = TSDB_ORDER_ASC; - iaInfo->interval = *pInterval; - + iaInfo->win = pTaskInfo->window; + iaInfo->order = TSDB_ORDER_ASC; + iaInfo->interval = *pInterval; iaInfo->execModel = pTaskInfo->execModel; - iaInfo->primaryTsIndex = primaryTsSlotId; size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; initResultSizeInfo(pOperator, 4096); - int32_t code = - initAggInfo(&iaInfo->binfo, &iaInfo->aggSup, pExprInfo, numOfCols, pResBlock, keyBufSize, pTaskInfo->id.str); + int32_t code = initAggInfo(&pOperator->exprSupp, &iaInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str); + initBasicInfo(&iaInfo->binfo, pResBlock); initExecTimeWindowInfo(&iaInfo->twAggSup.timeWindowData, &iaInfo->win); - iaInfo->timeWindowInterpo = timeWindowinterpNeeded(iaInfo->binfo.pCtx, numOfCols, iaInfo); + iaInfo->timeWindowInterpo = timeWindowinterpNeeded(pSup->pCtx, numOfCols, iaInfo); if (iaInfo->timeWindowInterpo) { iaInfo->binfo.resultRowInfo.openWindow = tdListNew(sizeof(SResultRowPosition)); } - // iaInfo->pTableQueryInfo = initTableQueryInfo(pTableGroupInfo); - if (code != TSDB_CODE_SUCCESS /* || iaInfo->pTableQueryInfo == NULL*/) { + if (code != TSDB_CODE_SUCCESS) { goto _error; } - initResultRowInfo(&iaInfo->binfo.resultRowInfo, (int32_t)1); + initResultRowInfo(&iaInfo->binfo.resultRowInfo); pOperator->name = "TimeMergeIntervalAggOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_MERGE_INTERVAL; pOperator->blocking = false; pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExprInfo; + pOperator->exprSupp.pExprInfo = pExprInfo; pOperator->pTaskInfo = pTaskInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->info = miaInfo; pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doMergeIntervalAgg, NULL, NULL, diff --git a/source/libs/executor/src/tsort.c b/source/libs/executor/src/tsort.c index 1502387360..f21cad2dd6 100644 --- a/source/libs/executor/src/tsort.c +++ b/source/libs/executor/src/tsort.c @@ -71,7 +71,7 @@ SSDataBlock* tsortGetSortedDataBlock(const SSortHandle* pSortHandle) { * @param type * @return */ -SSortHandle* tsortCreateSortHandle(SArray* pSortInfo, SArray* pIndexMap, int32_t type, int32_t pageSize, int32_t numOfPages, SSDataBlock* pBlock, const char* idstr) { +SSortHandle* tsortCreateSortHandle(SArray* pSortInfo, int32_t type, int32_t pageSize, int32_t numOfPages, SSDataBlock* pBlock, const char* idstr) { SSortHandle* pSortHandle = taosMemoryCalloc(1, sizeof(SSortHandle)); pSortHandle->type = type; @@ -195,6 +195,11 @@ static int32_t doAddToBuf(SSDataBlock* pDataBlock, SSortHandle* pHandle) { return doAddNewExternalMemSource(pHandle->pBuf, pHandle->pOrderedSource, pBlock, &pHandle->sourceId); } +static void setCurrentSourceIsDone(SSortSource* pSource, SSortHandle* pHandle) { + pSource->src.rowIndex = -1; + ++pHandle->numOfCompletedSources; +} + static int32_t sortComparInit(SMsortComparParam* cmpParam, SArray* pSources, int32_t startIndex, int32_t endIndex, SSortHandle* pHandle) { cmpParam->pSources = taosArrayGet(pSources, startIndex); cmpParam->numOfSources = (endIndex - startIndex + 1); @@ -205,8 +210,10 @@ static int32_t sortComparInit(SMsortComparParam* cmpParam, SArray* pSources, int for (int32_t i = 0; i < cmpParam->numOfSources; ++i) { SSortSource* pSource = cmpParam->pSources[i]; + // set current source is done if (taosArrayGetSize(pSource->pageIdList) == 0) { - return TSDB_CODE_SUCCESS; + setCurrentSourceIsDone(pSource, pHandle); + continue; } SPageInfo* pPgInfo = *(SPageInfo**)taosArrayGet(pSource->pageIdList, pSource->pageIndex); @@ -233,10 +240,9 @@ static int32_t sortComparInit(SMsortComparParam* cmpParam, SArray* pSources, int SSortSource* pSource = cmpParam->pSources[i]; pSource->src.pBlock = pHandle->fetchfp(pSource->param); - // set current source id done + // set current source is done if (pSource->src.pBlock == NULL) { - pSource->src.rowIndex = -1; - ++pHandle->numOfCompletedSources; + setCurrentSourceIsDone(pSource, pHandle); } } } diff --git a/source/libs/executor/test/sortTests.cpp b/source/libs/executor/test/sortTests.cpp index c037fae75f..66ed078bbe 100644 --- a/source/libs/executor/test/sortTests.cpp +++ b/source/libs/executor/test/sortTests.cpp @@ -209,7 +209,7 @@ TEST(testCase, inMem_sort_Test) { SArray* orderInfo = taosArrayInit(1, sizeof(SBlockOrderInfo)); taosArrayPush(orderInfo, &oi); - SSortHandle* phandle = tsortCreateSortHandle(orderInfo, NULL, SORT_SINGLESOURCE_SORT, 1024, 5, NULL, "test_abc"); + SSortHandle* phandle = tsortCreateSortHandle(orderInfo, SORT_SINGLESOURCE_SORT, 1024, 5, NULL, "test_abc"); tsortSetFetchRawDataFp(phandle, getSingleColDummyBlock, NULL, NULL); _info* pInfo = (_info*) taosMemoryCalloc(1, sizeof(_info)); @@ -298,7 +298,7 @@ TEST(testCase, external_mem_sort_Test) { SArray* orderInfo = taosArrayInit(1, sizeof(SBlockOrderInfo)); taosArrayPush(orderInfo, &oi); - SSortHandle* phandle = tsortCreateSortHandle(orderInfo, NULL, SORT_SINGLESOURCE_SORT, 128, 3, NULL, "test_abc"); + SSortHandle* phandle = tsortCreateSortHandle(orderInfo, SORT_SINGLESOURCE_SORT, 128, 3, NULL, "test_abc"); tsortSetFetchRawDataFp(phandle, getSingleColDummyBlock, NULL, NULL); SSortSource* ps = static_cast(taosMemoryCalloc(1, sizeof(SSortSource))); @@ -365,7 +365,7 @@ TEST(testCase, ordered_merge_sort_Test) { taosArrayPush(pBlock->pDataBlock, &colInfo); } - SSortHandle* phandle = tsortCreateSortHandle(orderInfo, NULL, SORT_MULTISOURCE_MERGE, 1024, 5, pBlock,"test_abc"); + SSortHandle* phandle = tsortCreateSortHandle(orderInfo, SORT_MULTISOURCE_MERGE, 1024, 5, pBlock,"test_abc"); tsortSetFetchRawDataFp(phandle, getSingleColDummyBlock, NULL, NULL); tsortSetComparFp(phandle, docomp); diff --git a/source/libs/nodes/src/nodesCloneFuncs.c b/source/libs/nodes/src/nodesCloneFuncs.c index 2e30b01357..d0eb1548c1 100644 --- a/source/libs/nodes/src/nodesCloneFuncs.c +++ b/source/libs/nodes/src/nodesCloneFuncs.c @@ -427,7 +427,7 @@ static SNode* logicWindowCopy(const SWindowLogicNode* pSrc, SWindowLogicNode* pD COPY_SCALAR_FIELD(triggerType); COPY_SCALAR_FIELD(watermark); COPY_SCALAR_FIELD(filesFactor); - COPY_SCALAR_FIELD(intervalAlgo); + COPY_SCALAR_FIELD(windowAlgo); return (SNode*)pDst; } @@ -538,6 +538,12 @@ static SNode* physiIntervalCopy(const SIntervalPhysiNode* pSrc, SIntervalPhysiNo return (SNode*)pDst; } +static SNode* physiSessionCopy(const SSessionWinodwPhysiNode* pSrc, SSessionWinodwPhysiNode* pDst) { + COPY_BASE_OBJECT_FIELD(window, physiWindowCopy); + COPY_SCALAR_FIELD(gap); + return (SNode*)pDst; +} + static SNode* dataBlockDescCopy(const SDataBlockDescNode* pSrc, SDataBlockDescNode* pDst) { COPY_SCALAR_FIELD(dataBlockId); CLONE_NODE_LIST_FIELD(pSlots); @@ -678,6 +684,9 @@ SNode* nodesCloneNode(const SNode* pNode) { case QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_INTERVAL: case QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_INTERVAL: return physiIntervalCopy((const SIntervalPhysiNode*)pNode, (SIntervalPhysiNode*)pDst); + case QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION: + case QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION: + return physiSessionCopy((const SSessionWinodwPhysiNode*)pNode, (SSessionWinodwPhysiNode*)pDst); default: break; } diff --git a/source/libs/nodes/src/nodesCodeFuncs.c b/source/libs/nodes/src/nodesCodeFuncs.c index 5e49a5b3c8..1b2eadd8e8 100644 --- a/source/libs/nodes/src/nodesCodeFuncs.c +++ b/source/libs/nodes/src/nodesCodeFuncs.c @@ -244,6 +244,10 @@ const char* nodesNodeName(ENodeType type) { return "PhysiSessionWindow"; case QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION: return "PhysiStreamSessionWindow"; + case QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION: + return "PhysiStreamSemiSessionWindow"; + case QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION: + return "PhysiStreamFinalSessionWindow"; case QUERY_NODE_PHYSICAL_PLAN_MERGE_STATE: return "PhysiStateWindow"; case QUERY_NODE_PHYSICAL_PLAN_STREAM_STATE: @@ -3996,6 +4000,8 @@ static int32_t specificNodeToJson(const void* pObj, SJson* pJson) { return physiFillNodeToJson(pObj, pJson); case QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION: case QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION: + case QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION: + case QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION: return physiSessionWindowNodeToJson(pObj, pJson); case QUERY_NODE_PHYSICAL_PLAN_MERGE_STATE: case QUERY_NODE_PHYSICAL_PLAN_STREAM_STATE: @@ -4129,6 +4135,8 @@ static int32_t jsonToSpecificNode(const SJson* pJson, void* pObj) { return jsonToPhysiFillNode(pJson, pObj); case QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION: case QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION: + case QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION: + case QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION: return jsonToPhysiSessionWindowNode(pJson, pObj); case QUERY_NODE_PHYSICAL_PLAN_MERGE_STATE: case QUERY_NODE_PHYSICAL_PLAN_STREAM_STATE: diff --git a/source/libs/nodes/src/nodesUtilFuncs.c b/source/libs/nodes/src/nodesUtilFuncs.c index c16b8f4c2c..6ebd0f2d39 100644 --- a/source/libs/nodes/src/nodesUtilFuncs.c +++ b/source/libs/nodes/src/nodesUtilFuncs.c @@ -150,6 +150,8 @@ SNode* nodesMakeNode(ENodeType type) { return makeNode(type, sizeof(SDropTopicStmt)); case QUERY_NODE_DROP_CGROUP_STMT: return makeNode(type, sizeof(SDropCGroupStmt)); + case QUERY_NODE_ALTER_LOCAL_STMT: + return makeNode(type, sizeof(SAlterLocalStmt)); case QUERY_NODE_EXPLAIN_STMT: return makeNode(type, sizeof(SExplainStmt)); case QUERY_NODE_DESCRIBE_STMT: @@ -206,11 +208,13 @@ SNode* nodesMakeNode(ENodeType type) { case QUERY_NODE_SHOW_APPS_STMT: case QUERY_NODE_SHOW_SCORES_STMT: case QUERY_NODE_SHOW_VARIABLE_STMT: - case QUERY_NODE_SHOW_CREATE_DATABASE_STMT: - case QUERY_NODE_SHOW_CREATE_TABLE_STMT: - case QUERY_NODE_SHOW_CREATE_STABLE_STMT: case QUERY_NODE_SHOW_TRANSACTIONS_STMT: return makeNode(type, sizeof(SShowStmt)); + case QUERY_NODE_SHOW_CREATE_DATABASE_STMT: + return makeNode(type, sizeof(SShowCreateDatabaseStmt)); + case QUERY_NODE_SHOW_CREATE_TABLE_STMT: + case QUERY_NODE_SHOW_CREATE_STABLE_STMT: + return makeNode(type, sizeof(SShowCreateTableStmt)); case QUERY_NODE_KILL_QUERY_STMT: return makeNode(type, sizeof(SKillQueryStmt)); case QUERY_NODE_KILL_TRANSACTION_STMT: @@ -288,6 +292,10 @@ SNode* nodesMakeNode(ENodeType type) { return makeNode(type, sizeof(SSessionWinodwPhysiNode)); case QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION: return makeNode(type, sizeof(SStreamSessionWinodwPhysiNode)); + case QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION: + return makeNode(type, sizeof(SStreamSemiSessionWinodwPhysiNode)); + case QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION: + return makeNode(type, sizeof(SStreamFinalSessionWinodwPhysiNode)); case QUERY_NODE_PHYSICAL_PLAN_MERGE_STATE: return makeNode(type, sizeof(SStateWinodwPhysiNode)); case QUERY_NODE_PHYSICAL_PLAN_STREAM_STATE: @@ -804,6 +812,7 @@ void nodesDestroyNode(SNode* pNode) { } case QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION: case QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION: + case QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION: case QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION: destroyWinodwPhysiNode((SWinodwPhysiNode*)pNode); break; diff --git a/source/libs/parser/inc/parAst.h b/source/libs/parser/inc/parAst.h index b8c6ffc843..b4f49b3e88 100644 --- a/source/libs/parser/inc/parAst.h +++ b/source/libs/parser/inc/parAst.h @@ -59,8 +59,8 @@ typedef enum EDatabaseOptionType { typedef enum ETableOptionType { TABLE_OPTION_COMMENT = 1, - TABLE_OPTION_FILE_FACTOR, - TABLE_OPTION_DELAY, + TABLE_OPTION_MAXDELAY, + TABLE_OPTION_WATERMARK, TABLE_OPTION_ROLLUP, TABLE_OPTION_TTL, TABLE_OPTION_SMA @@ -152,7 +152,7 @@ SNode* createAlterTableRenameCol(SAstCreateContext* pCxt, SNode* pRealTable, int SNode* createAlterTableSetTag(SAstCreateContext* pCxt, SNode* pRealTable, SToken* pTagName, SNode* pVal); SNode* createUseDatabaseStmt(SAstCreateContext* pCxt, SToken* pDbName); SNode* createShowStmt(SAstCreateContext* pCxt, ENodeType type, SNode* pDbName, SNode* pTbNamePattern); -SNode* createShowCreateDatabaseStmt(SAstCreateContext* pCxt, const SToken* pDbName); +SNode* createShowCreateDatabaseStmt(SAstCreateContext* pCxt, SToken* pDbName); SNode* createShowCreateTableStmt(SAstCreateContext* pCxt, ENodeType type, SNode* pRealTable); SNode* createCreateUserStmt(SAstCreateContext* pCxt, SToken* pUserName, const SToken* pPassword); SNode* createAlterUserStmt(SAstCreateContext* pCxt, SToken* pUserName, int8_t alterType, const SToken* pVal); diff --git a/source/libs/parser/inc/sql.y b/source/libs/parser/inc/sql.y index 0d9e6e4a73..476485904c 100644 --- a/source/libs/parser/inc/sql.y +++ b/source/libs/parser/inc/sql.y @@ -111,7 +111,7 @@ priv_level(A) ::= db_name(B) NK_DOT NK_STAR. /************************************************ create/drop/alter dnode *********************************************/ cmd ::= CREATE DNODE dnode_endpoint(A). { pCxt->pRootNode = createCreateDnodeStmt(pCxt, &A, NULL); } -cmd ::= CREATE DNODE dnode_host_name(A) PORT NK_INTEGER(B). { pCxt->pRootNode = createCreateDnodeStmt(pCxt, &A, &B); } +cmd ::= CREATE DNODE dnode_endpoint(A) PORT NK_INTEGER(B). { pCxt->pRootNode = createCreateDnodeStmt(pCxt, &A, &B); } cmd ::= DROP DNODE NK_INTEGER(A). { pCxt->pRootNode = createDropDnodeStmt(pCxt, &A); } cmd ::= DROP DNODE dnode_endpoint(A). { pCxt->pRootNode = createDropDnodeStmt(pCxt, &A); } cmd ::= ALTER DNODE NK_INTEGER(A) NK_STRING(B). { pCxt->pRootNode = createAlterDnodeStmt(pCxt, &A, &B, NULL); } @@ -122,11 +122,8 @@ cmd ::= ALTER ALL DNODES NK_STRING(A) NK_STRING(B). %type dnode_endpoint { SToken } %destructor dnode_endpoint { } dnode_endpoint(A) ::= NK_STRING(B). { A = B; } - -%type dnode_host_name { SToken } -%destructor dnode_host_name { } -dnode_host_name(A) ::= NK_ID(B). { A = B; } -dnode_host_name(A) ::= NK_IPTOKEN(B). { A = B; } +dnode_endpoint(A) ::= NK_ID(B). { A = B; } +dnode_endpoint(A) ::= NK_IPTOKEN(B). { A = B; } /************************************************ alter local *********************************************************/ cmd ::= ALTER LOCAL NK_STRING(A). { pCxt->pRootNode = createAlterLocalStmt(pCxt, &A, NULL); } @@ -317,8 +314,8 @@ tags_def(A) ::= TAGS NK_LP column_def_list(B) NK_RP. table_options(A) ::= . { A = createDefaultTableOptions(pCxt); } table_options(A) ::= table_options(B) COMMENT NK_STRING(C). { A = setTableOption(pCxt, B, TABLE_OPTION_COMMENT, &C); } -//table_options(A) ::= table_options(B) DELAY NK_INTEGER(C). { A = setTableOption(pCxt, B, TABLE_OPTION_DELAY, &C); } -table_options(A) ::= table_options(B) FILE_FACTOR NK_FLOAT(C). { A = setTableOption(pCxt, B, TABLE_OPTION_FILE_FACTOR, &C); } +table_options(A) ::= table_options(B) MAX_DELAY duration_list(C). { A = setTableOption(pCxt, B, TABLE_OPTION_MAXDELAY, C); } +table_options(A) ::= table_options(B) WATERMARK duration_list(C). { A = setTableOption(pCxt, B, TABLE_OPTION_WATERMARK, C); } table_options(A) ::= table_options(B) ROLLUP NK_LP rollup_func_list(C) NK_RP. { A = setTableOption(pCxt, B, TABLE_OPTION_ROLLUP, C); } table_options(A) ::= table_options(B) TTL NK_INTEGER(C). { A = setTableOption(pCxt, B, TABLE_OPTION_TTL, &C); } table_options(A) ::= table_options(B) SMA NK_LP col_name_list(C) NK_RP. { A = setTableOption(pCxt, B, TABLE_OPTION_SMA, C); } @@ -331,6 +328,11 @@ alter_table_options(A) ::= alter_table_options(B) alter_table_option(C). alter_table_option(A) ::= COMMENT NK_STRING(B). { A.type = TABLE_OPTION_COMMENT; A.val = B; } alter_table_option(A) ::= TTL NK_INTEGER(B). { A.type = TABLE_OPTION_TTL; A.val = B; } +%type duration_list { SNodeList* } +%destructor duration_list { nodesDestroyList($$); } +duration_list(A) ::= duration_literal(B). { A = createNodeList(pCxt, releaseRawExprNode(pCxt, B)); } +duration_list(A) ::= duration_list(B) NK_COMMA duration_literal(C). { A = addNodeToList(pCxt, B, releaseRawExprNode(pCxt, C)); } + %type rollup_func_list { SNodeList* } %destructor rollup_func_list { nodesDestroyList($$); } rollup_func_list(A) ::= rollup_func_name(B). { A = createNodeList(pCxt, B); } diff --git a/source/libs/parser/src/parAstCreater.c b/source/libs/parser/src/parAstCreater.c index 3002eacc2a..6ea1b090ad 100644 --- a/source/libs/parser/src/parAstCreater.c +++ b/source/libs/parser/src/parAstCreater.c @@ -18,6 +18,7 @@ #include "parAst.h" #include "parUtil.h" +#include "tglobal.h" #include "ttime.h" #define CHECK_OUT_OF_MEM(p) \ @@ -36,6 +37,14 @@ } \ } while (0) +#define COPY_STRING_FORM_ID_TOKEN(buf, pToken) strncpy(buf, (pToken)->z, TMIN((pToken)->n, sizeof(buf) - 1)) +#define COPY_STRING_FORM_STR_TOKEN(buf, pToken) \ + do { \ + if ((pToken)->n > 2) { \ + strncpy(buf, (pToken)->z + 1, TMIN((pToken)->n - 2, sizeof(buf) - 1)); \ + } \ + } while (0) + SToken nil_token = {.type = TK_NK_NIL, .n = 0, .z = NULL}; void initAstCreateContext(SParseContext* pParseCxt, SAstCreateContext* pCxt) { @@ -50,12 +59,6 @@ void initAstCreateContext(SParseContext* pParseCxt, SAstCreateContext* pCxt) { pCxt->errCode = TSDB_CODE_SUCCESS; } -static void copyStringFormStringToken(SToken* pToken, char* pBuf, int32_t len) { - if (pToken->n > 2) { - strncpy(pBuf, pToken->z + 1, TMIN(pToken->n - 2, len - 1)); - } -} - static void trimEscape(SToken* pName) { // todo need to deal with `ioo``ii` -> ioo`ii if (NULL != pName && pName->n > 1 && '`' == pName->z[0]) { @@ -108,50 +111,52 @@ static bool checkPassword(SAstCreateContext* pCxt, const SToken* pPasswordToken, return TSDB_CODE_SUCCESS == pCxt->errCode; } -static bool checkAndSplitEndpoint(SAstCreateContext* pCxt, const SToken* pEp, char* pFqdn, int32_t* pPort) { +static int32_t parsePort(SAstCreateContext* pCxt, const char* p, int32_t* pPort) { + *pPort = taosStr2Int32(p, NULL, 10); + if (*pPort >= UINT16_MAX || *pPort <= 0) { + return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_PORT); + } + return TSDB_CODE_SUCCESS; +} + +static int32_t parseEndpoint(SAstCreateContext* pCxt, const SToken* pEp, char* pFqdn, int32_t* pPort) { + if (pEp->n >= (NULL == pPort ? (TSDB_FQDN_LEN + 1 + 5) : TSDB_FQDN_LEN)) { // format 'fqdn:port' or 'fqdn' + return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NAME_OR_PASSWD_TOO_LONG); + } + + char ep[TSDB_FQDN_LEN + 1 + 5] = {0}; + COPY_STRING_FORM_ID_TOKEN(ep, pEp); + strdequote(ep); + strtrim(ep); + if (NULL == pPort) { + strcpy(pFqdn, ep); + return TSDB_CODE_SUCCESS; + } + char* pColon = strchr(ep, ':'); + if (NULL == pColon) { + *pPort = tsServerPort; + strcpy(pFqdn, ep); + return TSDB_CODE_SUCCESS; + } + strncpy(pFqdn, ep, pColon - ep); + return parsePort(pCxt, pColon + 1, pPort); +} + +static bool checkAndSplitEndpoint(SAstCreateContext* pCxt, const SToken* pEp, const SToken* pPortToken, char* pFqdn, + int32_t* pPort) { if (NULL == pEp) { pCxt->errCode = TSDB_CODE_PAR_SYNTAX_ERROR; - } else if (pEp->n >= TSDB_FQDN_LEN + 2 + 6) { // format 'fqdn:port' - pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NAME_OR_PASSWD_TOO_LONG); - } else { - char ep[TSDB_FQDN_LEN + 2 + 6]; - strncpy(ep, pEp->z, pEp->n); - strdequote(ep); - strtrim(ep); - char* pColon = strchr(ep, ':'); - if (NULL == pColon) { - pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_ENDPOINT); - } else { - strncpy(pFqdn, ep, pColon - ep); - *pPort = taosStr2Int32(pColon + 1, NULL, 10); - if (*pPort >= UINT16_MAX || *pPort <= 0) { - pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_PORT); - } - } + return false; } - return TSDB_CODE_SUCCESS == pCxt->errCode; -} -static bool checkFqdn(SAstCreateContext* pCxt, const SToken* pFqdn) { - if (NULL == pFqdn) { - pCxt->errCode = TSDB_CODE_PAR_SYNTAX_ERROR; - } else { - if (pFqdn->n >= TSDB_FQDN_LEN) { - pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NAME_OR_PASSWD_TOO_LONG); - } + if (NULL != pPortToken) { + pCxt->errCode = parsePort(pCxt, pPortToken->z, pPort); } - return TSDB_CODE_SUCCESS == pCxt->errCode; -} -static bool checkPort(SAstCreateContext* pCxt, const SToken* pPortToken, int32_t* pPort) { - if (NULL == pPortToken) { - pCxt->errCode = TSDB_CODE_PAR_SYNTAX_ERROR; - } else { - *pPort = taosStr2Int32(pPortToken->z, NULL, 10); - if (*pPort >= UINT16_MAX || *pPort <= 0) { - pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_PORT); - } + if (TSDB_CODE_SUCCESS == pCxt->errCode) { + pCxt->errCode = parseEndpoint(pCxt, pEp, pFqdn, (NULL != pPortToken ? NULL : pPort)); } + return TSDB_CODE_SUCCESS == pCxt->errCode; } @@ -274,9 +279,9 @@ SNode* createColumnNode(SAstCreateContext* pCxt, SToken* pTableAlias, SToken* pC SColumnNode* col = (SColumnNode*)nodesMakeNode(QUERY_NODE_COLUMN); CHECK_OUT_OF_MEM(col); if (NULL != pTableAlias) { - strncpy(col->tableAlias, pTableAlias->z, pTableAlias->n); + COPY_STRING_FORM_ID_TOKEN(col->tableAlias, pTableAlias); } - strncpy(col->colName, pColumnName->z, pColumnName->n); + COPY_STRING_FORM_ID_TOKEN(col->colName, pColumnName); return (SNode*)col; } @@ -417,7 +422,7 @@ SNode* createFunctionNode(SAstCreateContext* pCxt, const SToken* pFuncName, SNod } SFunctionNode* func = (SFunctionNode*)nodesMakeNode(QUERY_NODE_FUNCTION); CHECK_OUT_OF_MEM(func); - strncpy(func->functionName, pFuncName->z, pFuncName->n); + COPY_STRING_FORM_ID_TOKEN(func->functionName, pFuncName); func->pParameterList = pParameterList; return (SNode*)func; } @@ -464,16 +469,16 @@ SNode* createRealTableNode(SAstCreateContext* pCxt, SToken* pDbName, SToken* pTa SRealTableNode* realTable = (SRealTableNode*)nodesMakeNode(QUERY_NODE_REAL_TABLE); CHECK_OUT_OF_MEM(realTable); if (NULL != pDbName) { - strncpy(realTable->table.dbName, pDbName->z, pDbName->n); + COPY_STRING_FORM_ID_TOKEN(realTable->table.dbName, pDbName); } else { strcpy(realTable->table.dbName, pCxt->pQueryCxt->db); } if (NULL != pTableAlias && TK_NK_NIL != pTableAlias->type) { - strncpy(realTable->table.tableAlias, pTableAlias->z, pTableAlias->n); + COPY_STRING_FORM_ID_TOKEN(realTable->table.tableAlias, pTableAlias); } else { - strncpy(realTable->table.tableAlias, pTableName->z, pTableName->n); + COPY_STRING_FORM_ID_TOKEN(realTable->table.tableAlias, pTableName); } - strncpy(realTable->table.tableName, pTableName->z, pTableName->n); + COPY_STRING_FORM_ID_TOKEN(realTable->table.tableName, pTableName); return (SNode*)realTable; } @@ -483,7 +488,7 @@ SNode* createTempTableNode(SAstCreateContext* pCxt, SNode* pSubquery, const STok CHECK_OUT_OF_MEM(tempTable); tempTable->pSubquery = pSubquery; if (NULL != pTableAlias && TK_NK_NIL != pTableAlias->type) { - strncpy(tempTable->table.tableAlias, pTableAlias->z, pTableAlias->n); + COPY_STRING_FORM_ID_TOKEN(tempTable->table.tableAlias, pTableAlias); } else { sprintf(tempTable->table.tableAlias, "%p", tempTable); } @@ -785,8 +790,7 @@ SNode* setDatabaseOption(SAstCreateContext* pCxt, SNode* pOptions, EDatabaseOpti ((SDatabaseOptions*)pOptions)->pagesize = taosStr2Int32(((SToken*)pVal)->z, NULL, 10); break; case DB_OPTION_PRECISION: - copyStringFormStringToken((SToken*)pVal, ((SDatabaseOptions*)pOptions)->precisionStr, - sizeof(((SDatabaseOptions*)pOptions)->precisionStr)); + COPY_STRING_FORM_STR_TOKEN(((SDatabaseOptions*)pOptions)->precisionStr, (SToken*)pVal); break; case DB_OPTION_REPLICA: ((SDatabaseOptions*)pOptions)->replica = taosStr2Int8(((SToken*)pVal)->z, NULL, 10); @@ -835,7 +839,7 @@ SNode* createCreateDatabaseStmt(SAstCreateContext* pCxt, bool ignoreExists, STok } SCreateDatabaseStmt* pStmt = (SCreateDatabaseStmt*)nodesMakeNode(QUERY_NODE_CREATE_DATABASE_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->dbName, pDbName->z, pDbName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->dbName, pDbName); pStmt->ignoreExists = ignoreExists; pStmt->pOptions = (SDatabaseOptions*)pOptions; return (SNode*)pStmt; @@ -848,7 +852,7 @@ SNode* createDropDatabaseStmt(SAstCreateContext* pCxt, bool ignoreNotExists, STo } SDropDatabaseStmt* pStmt = (SDropDatabaseStmt*)nodesMakeNode(QUERY_NODE_DROP_DATABASE_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->dbName, pDbName->z, pDbName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->dbName, pDbName); pStmt->ignoreNotExists = ignoreNotExists; return (SNode*)pStmt; } @@ -860,7 +864,7 @@ SNode* createAlterDatabaseStmt(SAstCreateContext* pCxt, SToken* pDbName, SNode* } SAlterDatabaseStmt* pStmt = (SAlterDatabaseStmt*)nodesMakeNode(QUERY_NODE_ALTER_DATABASE_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->dbName, pDbName->z, pDbName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->dbName, pDbName); pStmt->pOptions = (SDatabaseOptions*)pOptions; return (SNode*)pStmt; } @@ -869,8 +873,10 @@ SNode* createDefaultTableOptions(SAstCreateContext* pCxt) { CHECK_PARSER_STATUS(pCxt); STableOptions* pOptions = (STableOptions*)nodesMakeNode(QUERY_NODE_TABLE_OPTIONS); CHECK_OUT_OF_MEM(pOptions); - // pOptions->delay = TSDB_DEFAULT_ROLLUP_DELAY; - pOptions->filesFactor = TSDB_DEFAULT_ROLLUP_FILE_FACTOR; + pOptions->maxDelay1 = -1; + pOptions->maxDelay2 = -1; + pOptions->watermark1 = TSDB_DEFAULT_ROLLUP_WATERMARK; + pOptions->watermark2 = TSDB_DEFAULT_ROLLUP_WATERMARK; pOptions->ttl = TSDB_DEFAULT_TABLE_TTL; return (SNode*)pOptions; } @@ -879,7 +885,6 @@ SNode* createAlterTableOptions(SAstCreateContext* pCxt) { CHECK_PARSER_STATUS(pCxt); STableOptions* pOptions = (STableOptions*)nodesMakeNode(QUERY_NODE_TABLE_OPTIONS); CHECK_OUT_OF_MEM(pOptions); - pOptions->delay = -1; pOptions->ttl = -1; return (SNode*)pOptions; } @@ -889,12 +894,14 @@ SNode* setTableOption(SAstCreateContext* pCxt, SNode* pOptions, ETableOptionType switch (type) { case TABLE_OPTION_COMMENT: if (checkComment(pCxt, (SToken*)pVal, true)) { - copyStringFormStringToken((SToken*)pVal, ((STableOptions*)pOptions)->comment, - sizeof(((STableOptions*)pOptions)->comment)); + COPY_STRING_FORM_STR_TOKEN(((STableOptions*)pOptions)->comment, (SToken*)pVal); } break; - case TABLE_OPTION_DELAY: - ((STableOptions*)pOptions)->delay = taosStr2Int32(((SToken*)pVal)->z, NULL, 10); + case TABLE_OPTION_MAXDELAY: + ((STableOptions*)pOptions)->pMaxDelay = pVal; + break; + case TABLE_OPTION_WATERMARK: + ((STableOptions*)pOptions)->pWatermark = pVal; break; case TABLE_OPTION_ROLLUP: ((STableOptions*)pOptions)->pRollupFuncs = pVal; @@ -918,7 +925,7 @@ SNode* createColumnDefNode(SAstCreateContext* pCxt, SToken* pColName, SDataType } SColumnDefNode* pCol = (SColumnDefNode*)nodesMakeNode(QUERY_NODE_COLUMN_DEF); CHECK_OUT_OF_MEM(pCol); - strncpy(pCol->colName, pColName->z, pColName->n); + COPY_STRING_FORM_ID_TOKEN(pCol->colName, pColName); pCol->dataType = dataType; if (NULL != pComment) { trimString(pComment->z, pComment->n, pCol->comments, sizeof(pCol->comments)); @@ -1033,7 +1040,7 @@ SNode* createAlterTableAddModifyCol(SAstCreateContext* pCxt, SNode* pRealTable, SAlterTableStmt* pStmt = (SAlterTableStmt*)nodesMakeNode(QUERY_NODE_ALTER_TABLE_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->alterType = alterType; - strncpy(pStmt->colName, pColName->z, pColName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->colName, pColName); pStmt->dataType = dataType; return createAlterTableStmtFinalize(pRealTable, pStmt); } @@ -1046,7 +1053,7 @@ SNode* createAlterTableDropCol(SAstCreateContext* pCxt, SNode* pRealTable, int8_ SAlterTableStmt* pStmt = (SAlterTableStmt*)nodesMakeNode(QUERY_NODE_ALTER_TABLE_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->alterType = alterType; - strncpy(pStmt->colName, pColName->z, pColName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->colName, pColName); return createAlterTableStmtFinalize(pRealTable, pStmt); } @@ -1059,8 +1066,8 @@ SNode* createAlterTableRenameCol(SAstCreateContext* pCxt, SNode* pRealTable, int SAlterTableStmt* pStmt = (SAlterTableStmt*)nodesMakeNode(QUERY_NODE_ALTER_TABLE_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->alterType = alterType; - strncpy(pStmt->colName, pOldColName->z, pOldColName->n); - strncpy(pStmt->newColName, pNewColName->z, pNewColName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->colName, pOldColName); + COPY_STRING_FORM_ID_TOKEN(pStmt->newColName, pNewColName); return createAlterTableStmtFinalize(pRealTable, pStmt); } @@ -1072,7 +1079,7 @@ SNode* createAlterTableSetTag(SAstCreateContext* pCxt, SNode* pRealTable, SToken SAlterTableStmt* pStmt = (SAlterTableStmt*)nodesMakeNode(QUERY_NODE_ALTER_TABLE_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->alterType = TSDB_ALTER_TABLE_UPDATE_TAG_VAL; - strncpy(pStmt->colName, pTagName->z, pTagName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->colName, pTagName); pStmt->pVal = (SValueNode*)pVal; return createAlterTableStmtFinalize(pRealTable, pStmt); } @@ -1084,7 +1091,7 @@ SNode* createUseDatabaseStmt(SAstCreateContext* pCxt, SToken* pDbName) { } SUseDatabaseStmt* pStmt = (SUseDatabaseStmt*)nodesMakeNode(QUERY_NODE_USE_DATABASE_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->dbName, pDbName->z, pDbName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->dbName, pDbName); return (SNode*)pStmt; } @@ -1107,18 +1114,24 @@ SNode* createShowStmt(SAstCreateContext* pCxt, ENodeType type, SNode* pDbName, S return (SNode*)pStmt; } -SNode* createShowCreateDatabaseStmt(SAstCreateContext* pCxt, const SToken* pDbName) { +SNode* createShowCreateDatabaseStmt(SAstCreateContext* pCxt, SToken* pDbName) { CHECK_PARSER_STATUS(pCxt); - SNode* pStmt = nodesMakeNode(QUERY_NODE_SHOW_CREATE_DATABASE_STMT); + if (!checkDbName(pCxt, pDbName, true)) { + return NULL; + } + SShowCreateDatabaseStmt* pStmt = (SShowCreateDatabaseStmt*)nodesMakeNode(QUERY_NODE_SHOW_CREATE_DATABASE_STMT); CHECK_OUT_OF_MEM(pStmt); - return pStmt; + COPY_STRING_FORM_ID_TOKEN(pStmt->dbName, pDbName); + return (SNode*)pStmt; } SNode* createShowCreateTableStmt(SAstCreateContext* pCxt, ENodeType type, SNode* pRealTable) { CHECK_PARSER_STATUS(pCxt); - SNode* pStmt = nodesMakeNode(type); + SShowCreateTableStmt* pStmt = (SShowCreateTableStmt*)nodesMakeNode(type); CHECK_OUT_OF_MEM(pStmt); - return pStmt; + strcpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName); + strcpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName); + return (SNode*)pStmt; } SNode* createCreateUserStmt(SAstCreateContext* pCxt, SToken* pUserName, const SToken* pPassword) { @@ -1129,7 +1142,7 @@ SNode* createCreateUserStmt(SAstCreateContext* pCxt, SToken* pUserName, const ST } SCreateUserStmt* pStmt = (SCreateUserStmt*)nodesMakeNode(QUERY_NODE_CREATE_USER_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->useName, pUserName->z, pUserName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->useName, pUserName); strcpy(pStmt->password, password); return (SNode*)pStmt; } @@ -1141,7 +1154,7 @@ SNode* createAlterUserStmt(SAstCreateContext* pCxt, SToken* pUserName, int8_t al } SAlterUserStmt* pStmt = (SAlterUserStmt*)nodesMakeNode(QUERY_NODE_ALTER_USER_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->useName, pUserName->z, pUserName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->useName, pUserName); if (TSDB_ALTER_USER_PASSWD == alterType) { char password[TSDB_USET_PASSWORD_LEN] = {0}; if (!checkPassword(pCxt, pVal, password)) { @@ -1161,29 +1174,18 @@ SNode* createDropUserStmt(SAstCreateContext* pCxt, SToken* pUserName) { } SDropUserStmt* pStmt = (SDropUserStmt*)nodesMakeNode(QUERY_NODE_DROP_USER_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->useName, pUserName->z, pUserName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->useName, pUserName); return (SNode*)pStmt; } SNode* createCreateDnodeStmt(SAstCreateContext* pCxt, const SToken* pFqdn, const SToken* pPort) { CHECK_PARSER_STATUS(pCxt); - int32_t port = 0; - char fqdn[TSDB_FQDN_LEN] = {0}; - if (NULL == pPort) { - if (!checkAndSplitEndpoint(pCxt, pFqdn, fqdn, &port)) { - return NULL; - } - } else if (!checkFqdn(pCxt, pFqdn) || !checkPort(pCxt, pPort, &port)) { - return NULL; - } SCreateDnodeStmt* pStmt = (SCreateDnodeStmt*)nodesMakeNode(QUERY_NODE_CREATE_DNODE_STMT); CHECK_OUT_OF_MEM(pStmt); - if (NULL == pPort) { - strcpy(pStmt->fqdn, fqdn); - } else { - strncpy(pStmt->fqdn, pFqdn->z, pFqdn->n); + if (!checkAndSplitEndpoint(pCxt, pFqdn, pPort, pStmt->fqdn, &pStmt->port)) { + nodesDestroyNode((SNode*)pStmt); + return NULL; } - pStmt->port = port; return (SNode*)pStmt; } @@ -1194,7 +1196,7 @@ SNode* createDropDnodeStmt(SAstCreateContext* pCxt, const SToken* pDnode) { if (TK_NK_INTEGER == pDnode->type) { pStmt->dnodeId = taosStr2Int32(pDnode->z, NULL, 10); } else { - if (!checkAndSplitEndpoint(pCxt, pDnode, pStmt->fqdn, &pStmt->port)) { + if (!checkAndSplitEndpoint(pCxt, pDnode, NULL, pStmt->fqdn, &pStmt->port)) { nodesDestroyNode((SNode*)pStmt); return NULL; } @@ -1225,8 +1227,8 @@ SNode* createCreateIndexStmt(SAstCreateContext* pCxt, EIndexType type, bool igno CHECK_OUT_OF_MEM(pStmt); pStmt->indexType = type; pStmt->ignoreExists = ignoreExists; - strncpy(pStmt->indexName, pIndexName->z, pIndexName->n); - strncpy(pStmt->tableName, pTableName->z, pTableName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->indexName, pIndexName); + COPY_STRING_FORM_ID_TOKEN(pStmt->tableName, pTableName); pStmt->pCols = pCols; pStmt->pOptions = (SIndexOptions*)pOptions; return (SNode*)pStmt; @@ -1252,8 +1254,8 @@ SNode* createDropIndexStmt(SAstCreateContext* pCxt, bool ignoreNotExists, SToken SDropIndexStmt* pStmt = (SDropIndexStmt*)nodesMakeNode(QUERY_NODE_DROP_INDEX_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->ignoreNotExists = ignoreNotExists; - strncpy(pStmt->indexName, pIndexName->z, pIndexName->n); - strncpy(pStmt->tableName, pTableName->z, pTableName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->indexName, pIndexName); + COPY_STRING_FORM_ID_TOKEN(pStmt->tableName, pTableName); return (SNode*)pStmt; } @@ -1278,14 +1280,14 @@ SNode* createCreateTopicStmt(SAstCreateContext* pCxt, bool ignoreExists, const S CHECK_PARSER_STATUS(pCxt); SCreateTopicStmt* pStmt = (SCreateTopicStmt*)nodesMakeNode(QUERY_NODE_CREATE_TOPIC_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->topicName, pTopicName->z, pTopicName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->topicName, pTopicName); pStmt->ignoreExists = ignoreExists; if (NULL != pRealTable) { strcpy(pStmt->subDbName, ((SRealTableNode*)pRealTable)->table.dbName); strcpy(pStmt->subSTbName, ((SRealTableNode*)pRealTable)->table.tableName); nodesDestroyNode(pRealTable); } else if (NULL != pSubDbName) { - strncpy(pStmt->subDbName, pSubDbName->z, pSubDbName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->subDbName, pSubDbName); } else { pStmt->pQuery = pQuery; } @@ -1296,7 +1298,7 @@ SNode* createDropTopicStmt(SAstCreateContext* pCxt, bool ignoreNotExists, const CHECK_PARSER_STATUS(pCxt); SDropTopicStmt* pStmt = (SDropTopicStmt*)nodesMakeNode(QUERY_NODE_DROP_TOPIC_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->topicName, pTopicName->z, pTopicName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->topicName, pTopicName); pStmt->ignoreNotExists = ignoreNotExists; return (SNode*)pStmt; } @@ -1307,8 +1309,8 @@ SNode* createDropCGroupStmt(SAstCreateContext* pCxt, bool ignoreNotExists, const SDropCGroupStmt* pStmt = (SDropCGroupStmt*)nodesMakeNode(QUERY_NODE_DROP_CGROUP_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->ignoreNotExists = ignoreNotExists; - strncpy(pStmt->topicName, pTopicName->z, pTopicName->n); - strncpy(pStmt->cgroup, pCGroupId->z, pCGroupId->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->topicName, pTopicName); + COPY_STRING_FORM_ID_TOKEN(pStmt->cgroup, pCGroupId); return (SNode*)pStmt; } @@ -1388,9 +1390,9 @@ SNode* createCreateFunctionStmt(SAstCreateContext* pCxt, bool ignoreExists, bool SCreateFunctionStmt* pStmt = (SCreateFunctionStmt*)nodesMakeNode(QUERY_NODE_CREATE_FUNCTION_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->ignoreExists = ignoreExists; - strncpy(pStmt->funcName, pFuncName->z, pFuncName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->funcName, pFuncName); pStmt->isAgg = aggFunc; - strncpy(pStmt->libraryPath, pLibPath->z + 1, pLibPath->n - 2); + COPY_STRING_FORM_STR_TOKEN(pStmt->libraryPath, pLibPath); pStmt->outputDt = dataType; pStmt->bufSize = bufSize; return (SNode*)pStmt; @@ -1401,7 +1403,7 @@ SNode* createDropFunctionStmt(SAstCreateContext* pCxt, bool ignoreNotExists, con SDropFunctionStmt* pStmt = (SDropFunctionStmt*)nodesMakeNode(QUERY_NODE_DROP_FUNCTION_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->ignoreNotExists = ignoreNotExists; - strncpy(pStmt->funcName, pFuncName->z, pFuncName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->funcName, pFuncName); return (SNode*)pStmt; } @@ -1418,7 +1420,7 @@ SNode* createCreateStreamStmt(SAstCreateContext* pCxt, bool ignoreExists, const CHECK_PARSER_STATUS(pCxt); SCreateStreamStmt* pStmt = (SCreateStreamStmt*)nodesMakeNode(QUERY_NODE_CREATE_STREAM_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->streamName, pStreamName->z, pStreamName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->streamName, pStreamName); if (NULL != pRealTable) { strcpy(pStmt->targetDbName, ((SRealTableNode*)pRealTable)->table.dbName); strcpy(pStmt->targetTabName, ((SRealTableNode*)pRealTable)->table.tableName); @@ -1434,7 +1436,7 @@ SNode* createDropStreamStmt(SAstCreateContext* pCxt, bool ignoreNotExists, const CHECK_PARSER_STATUS(pCxt); SDropStreamStmt* pStmt = (SDropStreamStmt*)nodesMakeNode(QUERY_NODE_DROP_STREAM_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->streamName, pStreamName->z, TMIN(pStreamName->n, sizeof(pStmt->streamName) - 1)); + COPY_STRING_FORM_ID_TOKEN(pStmt->streamName, pStreamName); pStmt->ignoreNotExists = ignoreNotExists; return (SNode*)pStmt; } @@ -1503,8 +1505,8 @@ SNode* createGrantStmt(SAstCreateContext* pCxt, int64_t privileges, SToken* pDbN SGrantStmt* pStmt = (SGrantStmt*)nodesMakeNode(QUERY_NODE_GRANT_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->privileges = privileges; - strncpy(pStmt->dbName, pDbName->z, pDbName->n); - strncpy(pStmt->userName, pUserName->z, pUserName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->dbName, pDbName); + COPY_STRING_FORM_ID_TOKEN(pStmt->userName, pUserName); return (SNode*)pStmt; } @@ -1516,8 +1518,8 @@ SNode* createRevokeStmt(SAstCreateContext* pCxt, int64_t privileges, SToken* pDb SRevokeStmt* pStmt = (SRevokeStmt*)nodesMakeNode(QUERY_NODE_REVOKE_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->privileges = privileges; - strncpy(pStmt->dbName, pDbName->z, pDbName->n); - strncpy(pStmt->userName, pUserName->z, pUserName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->dbName, pDbName); + COPY_STRING_FORM_ID_TOKEN(pStmt->userName, pUserName); return (SNode*)pStmt; } diff --git a/source/libs/parser/src/parAstParser.c b/source/libs/parser/src/parAstParser.c index 1e968fe1fa..e3218f972b 100644 --- a/source/libs/parser/src/parAstParser.c +++ b/source/libs/parser/src/parAstParser.c @@ -397,6 +397,14 @@ static int32_t collectMetaKeyFromShowVariables(SCollectMetaKeyCxt* pCxt, SShowSt pCxt->pMetaCache); } +static int32_t collectMetaKeyFromShowCreateDatabase(SCollectMetaKeyCxt* pCxt, SShowCreateDatabaseStmt* pStmt) { + return reserveDbCfgInCache(pCxt->pParseCxt->acctId, pStmt->dbName, pCxt->pMetaCache); +} + +static int32_t collectMetaKeyFromShowCreateTable(SCollectMetaKeyCxt* pCxt, SShowCreateTableStmt* pStmt) { + return reserveTableMetaInCache(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, pCxt->pMetaCache); +} + static int32_t collectMetaKeyFromShowApps(SCollectMetaKeyCxt* pCxt, SShowStmt* pStmt) { return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_PERFORMANCE_SCHEMA_DB, TSDB_PERFS_TABLE_APPS, pCxt->pMetaCache); @@ -478,6 +486,11 @@ static int32_t collectMetaKeyFromQuery(SCollectMetaKeyCxt* pCxt, SNode* pStmt) { return collectMetaKeyFromShowQueries(pCxt, (SShowStmt*)pStmt); case QUERY_NODE_SHOW_VARIABLE_STMT: return collectMetaKeyFromShowVariables(pCxt, (SShowStmt*)pStmt); + case QUERY_NODE_SHOW_CREATE_DATABASE_STMT: + return collectMetaKeyFromShowCreateDatabase(pCxt, (SShowCreateDatabaseStmt*)pStmt); + case QUERY_NODE_SHOW_CREATE_TABLE_STMT: + case QUERY_NODE_SHOW_CREATE_STABLE_STMT: + return collectMetaKeyFromShowCreateTable(pCxt, (SShowCreateTableStmt*)pStmt); case QUERY_NODE_SHOW_APPS_STMT: return collectMetaKeyFromShowApps(pCxt, (SShowStmt*)pStmt); case QUERY_NODE_SHOW_TRANSACTIONS_STMT: diff --git a/source/libs/parser/src/parTokenizer.c b/source/libs/parser/src/parTokenizer.c index 55ea7171fa..41d32204da 100644 --- a/source/libs/parser/src/parTokenizer.c +++ b/source/libs/parser/src/parTokenizer.c @@ -81,7 +81,7 @@ static SKeyword keywordTable[] = { {"DURATION", TK_DURATION}, {"EXISTS", TK_EXISTS}, {"EXPLAIN", TK_EXPLAIN}, - {"FILE_FACTOR", TK_FILE_FACTOR}, + // {"FILE_FACTOR", TK_FILE_FACTOR}, {"FILL", TK_FILL}, {"FIRST", TK_FIRST}, {"FLOAT", TK_FLOAT}, diff --git a/source/libs/parser/src/parTranslater.c b/source/libs/parser/src/parTranslater.c index 09d4bfab14..3cb885c8e1 100644 --- a/source/libs/parser/src/parTranslater.c +++ b/source/libs/parser/src/parTranslater.c @@ -1450,7 +1450,8 @@ static int32_t addMnodeToVgroupList(const SEpSet* pEpSet, SArray** pVgroupList) } static int32_t setSysTableVgroupList(STranslateContext* pCxt, SName* pName, SRealTableNode* pRealTable) { - if (0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_USER_TABLES)) { + if (0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_USER_TABLES) && + 0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_USER_TABLE_DISTRIBUTED)) { return TSDB_CODE_SUCCESS; } @@ -1531,7 +1532,8 @@ static bool joinTableIsSingleTable(SJoinTableNode* pJoinTable) { static bool isSingleTable(SRealTableNode* pRealTable) { int8_t tableType = pRealTable->pMeta->tableType; if (TSDB_SYSTEM_TABLE == tableType) { - return 0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_USER_TABLES); + return 0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_USER_TABLES) && + 0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_USER_TABLE_DISTRIBUTED); } return (TSDB_CHILD_TABLE == tableType || TSDB_NORMAL_TABLE == tableType); } @@ -2623,7 +2625,7 @@ static int32_t checkDbDaysOption(STranslateContext* pCxt, SDatabaseOptions* pOpt if (TIME_UNIT_MINUTE != pOptions->pDaysPerFile->unit && TIME_UNIT_HOUR != pOptions->pDaysPerFile->unit && TIME_UNIT_DAY != pOptions->pDaysPerFile->unit) { return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_OPTION_UNIT, "daysPerFile", - pOptions->pDaysPerFile->unit); + pOptions->pDaysPerFile->unit, TIME_UNIT_MINUTE, TIME_UNIT_HOUR, TIME_UNIT_DAY); } pOptions->daysPerFile = getBigintFromValueNode(pOptions->pDaysPerFile); } @@ -2925,14 +2927,6 @@ static SColumnDefNode* findColDef(SNodeList* pCols, const SColumnNode* pCol) { return NULL; } -static int32_t checTableFactorOption(STranslateContext* pCxt, float val) { - if (val < TSDB_MIN_ROLLUP_FILE_FACTOR || val > TSDB_MAX_ROLLUP_FILE_FACTOR) { - return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_F_RANGE_OPTION, "file_factor", val, - TSDB_MIN_ROLLUP_FILE_FACTOR, TSDB_MAX_ROLLUP_FILE_FACTOR); - } - return TSDB_CODE_SUCCESS; -} - static int32_t checkTableSmaOption(STranslateContext* pCxt, SCreateTableStmt* pStmt) { if (NULL != pStmt->pOptions->pSma) { SNode* pNode = NULL; @@ -3082,27 +3076,74 @@ static int32_t checkTableSchema(STranslateContext* pCxt, SCreateTableStmt* pStmt return code; } -static int32_t checkSchemalessDb(STranslateContext* pCxt, const char* pDbName) { - // if (0 != pCxt->pParseCxt->schemalessType) { - // return TSDB_CODE_SUCCESS; - // } - // SDbCfgInfo info = {0}; - // int32_t code = getDBCfg(pCxt, pDbName, &info); - // if (TSDB_CODE_SUCCESS == code) { - // code = info.schemaless ? TSDB_CODE_SML_INVALID_DB_CONF : TSDB_CODE_SUCCESS; - // } - // return code; - return TSDB_CODE_SUCCESS; +static int32_t getTableDelayOrWatermarkOption(STranslateContext* pCxt, const char* pName, int64_t minVal, + int64_t maxVal, SValueNode* pVal, int64_t* pMaxDelay) { + int32_t code = (DEAL_RES_ERROR == translateValue(pCxt, pVal) ? pCxt->errCode : TSDB_CODE_SUCCESS); + if (TSDB_CODE_SUCCESS == code && TIME_UNIT_MILLISECOND != pVal->unit && TIME_UNIT_SECOND != pVal->unit && + TIME_UNIT_MINUTE != pVal->unit) { + code = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_OPTION_UNIT, pName, pVal->unit, + TIME_UNIT_MILLISECOND, TIME_UNIT_SECOND, TIME_UNIT_MINUTE); + } + if (TSDB_CODE_SUCCESS == code) { + code = checkRangeOption(pCxt, pName, pVal->datum.i, minVal, maxVal); + } + if (TSDB_CODE_SUCCESS == code) { + *pMaxDelay = pVal->datum.i; + } + return code; +} + +static int32_t getTableMaxDelayOption(STranslateContext* pCxt, SValueNode* pVal, int64_t* pMaxDelay) { + return getTableDelayOrWatermarkOption(pCxt, "maxDelay", TSDB_MIN_ROLLUP_MAX_DELAY, TSDB_MAX_ROLLUP_MAX_DELAY, pVal, + pMaxDelay); +} + +static int32_t checkTableMaxDelayOption(STranslateContext* pCxt, STableOptions* pOptions) { + if (NULL == pOptions->pMaxDelay) { + return TSDB_CODE_SUCCESS; + } + + if (LIST_LENGTH(pOptions->pMaxDelay) > 2) { + return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_TABLE_OPTION, "maxdelay"); + } + + int32_t code = + getTableMaxDelayOption(pCxt, (SValueNode*)nodesListGetNode(pOptions->pMaxDelay, 0), &pOptions->maxDelay1); + if (TSDB_CODE_SUCCESS == code && 2 == LIST_LENGTH(pOptions->pMaxDelay)) { + code = getTableMaxDelayOption(pCxt, (SValueNode*)nodesListGetNode(pOptions->pMaxDelay, 1), &pOptions->maxDelay2); + } + + return code; +} + +static int32_t getTableWatermarkOption(STranslateContext* pCxt, SValueNode* pVal, int64_t* pMaxDelay) { + return getTableDelayOrWatermarkOption(pCxt, "watermark", TSDB_MIN_ROLLUP_WATERMARK, TSDB_MAX_ROLLUP_WATERMARK, pVal, + pMaxDelay); +} + +static int32_t checkTableWatermarkOption(STranslateContext* pCxt, STableOptions* pOptions) { + if (NULL == pOptions->pWatermark) { + return TSDB_CODE_SUCCESS; + } + + if (LIST_LENGTH(pOptions->pWatermark) > 2) { + return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_TABLE_OPTION, "watermark"); + } + + int32_t code = + getTableWatermarkOption(pCxt, (SValueNode*)nodesListGetNode(pOptions->pWatermark, 0), &pOptions->watermark1); + if (TSDB_CODE_SUCCESS == code && 2 == LIST_LENGTH(pOptions->pWatermark)) { + code = getTableWatermarkOption(pCxt, (SValueNode*)nodesListGetNode(pOptions->pWatermark, 1), &pOptions->watermark2); + } + + return code; } static int32_t checkCreateTable(STranslateContext* pCxt, SCreateTableStmt* pStmt) { - int32_t code = checkSchemalessDb(pCxt, pStmt->dbName); + int32_t code = checkTableMaxDelayOption(pCxt, pStmt->pOptions); if (TSDB_CODE_SUCCESS == code) { - code = checTableFactorOption(pCxt, pStmt->pOptions->filesFactor); + code = checkTableWatermarkOption(pCxt, pStmt->pOptions); } - // if (TSDB_CODE_SUCCESS == code) { - // code = checkRangeOption(pCxt, "delay", pStmt->pOptions->delay, TSDB_MIN_ROLLUP_DELAY, TSDB_MAX_ROLLUP_DELAY); - // } if (TSDB_CODE_SUCCESS == code) { code = checkTableRollupOption(pCxt, pStmt->pOptions->pRollupFuncs); } @@ -3345,8 +3386,10 @@ static int32_t buildRollupAst(STranslateContext* pCxt, SCreateTableStmt* pStmt, static int32_t buildCreateStbReq(STranslateContext* pCxt, SCreateTableStmt* pStmt, SMCreateStbReq* pReq) { pReq->igExists = pStmt->ignoreExists; - // pReq->delay = pStmt->pOptions->delay; - pReq->xFilesFactor = pStmt->pOptions->filesFactor; + pReq->delay1 = pStmt->pOptions->maxDelay1; + pReq->delay2 = pStmt->pOptions->maxDelay2; + pReq->watermark1 = pStmt->pOptions->watermark1; + pReq->watermark2 = pStmt->pOptions->watermark2; pReq->ttl = pStmt->pOptions->ttl; columnDefNodeToField(pStmt->pCols, &pReq->pColumns); columnDefNodeToField(pStmt->pTags, &pReq->pTags); @@ -3571,27 +3614,6 @@ static int32_t translateAlterDnode(STranslateContext* pCxt, SAlterDnodeStmt* pSt return buildCmdMsg(pCxt, TDMT_MND_CONFIG_DNODE, (FSerializeFunc)tSerializeSMCfgDnodeReq, &cfgReq); } -static int32_t nodeTypeToShowType(ENodeType nt) { - switch (nt) { - case QUERY_NODE_SHOW_CONNECTIONS_STMT: - return TSDB_MGMT_TABLE_CONNS; - case QUERY_NODE_SHOW_LICENCE_STMT: - return TSDB_MGMT_TABLE_GRANTS; - case QUERY_NODE_SHOW_QUERIES_STMT: - return TSDB_MGMT_TABLE_QUERIES; - case QUERY_NODE_SHOW_VARIABLE_STMT: - return TSDB_MGMT_TABLE_CONFIGS; - default: - break; - } - return 0; -} - -static int32_t translateShow(STranslateContext* pCxt, SShowStmt* pStmt) { - SShowReq showReq = {.type = nodeTypeToShowType(nodeType(pStmt))}; - return buildCmdMsg(pCxt, TDMT_MND_SHOW, (FSerializeFunc)tSerializeSShowReq, &showReq); -} - static int32_t getSmaIndexDstVgId(STranslateContext* pCxt, char* pTableName, int32_t* pVgId) { SVgroupInfo vg = {0}; int32_t code = getTableHashVgroup(pCxt, pCxt->pParseCxt->db, pTableName, &vg); @@ -4137,6 +4159,18 @@ static int32_t translateSplitVgroup(STranslateContext* pCxt, SSplitVgroupStmt* p return buildCmdMsg(pCxt, TDMT_MND_SPLIT_VGROUP, (FSerializeFunc)tSerializeSSplitVgroupReq, &req); } +static int32_t translateShowCreateDatabase(STranslateContext* pCxt, SShowCreateDatabaseStmt* pStmt) { + pStmt->pCfg = taosMemoryCalloc(1, sizeof(SDbCfgInfo)); + if (NULL == pStmt->pCfg) { + return TSDB_CODE_OUT_OF_MEMORY; + } + return getDBCfg(pCxt, pStmt->dbName, (SDbCfgInfo*)pStmt->pCfg); +} + +static int32_t translateShowCreateTable(STranslateContext* pCxt, SShowCreateTableStmt* pStmt) { + return getTableMeta(pCxt, pStmt->dbName, pStmt->tableName, &pStmt->pMeta); +} + static int32_t translateQuery(STranslateContext* pCxt, SNode* pNode) { int32_t code = TSDB_CODE_SUCCESS; switch (nodeType(pNode)) { @@ -4191,12 +4225,6 @@ static int32_t translateQuery(STranslateContext* pCxt, SNode* pNode) { case QUERY_NODE_ALTER_DNODE_STMT: code = translateAlterDnode(pCxt, (SAlterDnodeStmt*)pNode); break; - case QUERY_NODE_SHOW_CONNECTIONS_STMT: - case QUERY_NODE_SHOW_QUERIES_STMT: - case QUERY_NODE_SHOW_TOPICS_STMT: - case QUERY_NODE_SHOW_VARIABLE_STMT: - code = translateShow(pCxt, (SShowStmt*)pNode); - break; case QUERY_NODE_CREATE_INDEX_STMT: code = translateCreateIndex(pCxt, (SCreateIndexStmt*)pNode); break; @@ -4272,6 +4300,13 @@ static int32_t translateQuery(STranslateContext* pCxt, SNode* pNode) { case QUERY_NODE_SPLIT_VGROUP_STMT: code = translateSplitVgroup(pCxt, (SSplitVgroupStmt*)pNode); break; + case QUERY_NODE_SHOW_CREATE_DATABASE_STMT: + code = translateShowCreateDatabase(pCxt, (SShowCreateDatabaseStmt*)pNode); + break; + case QUERY_NODE_SHOW_CREATE_TABLE_STMT: + case QUERY_NODE_SHOW_CREATE_STABLE_STMT: + code = translateShowCreateTable(pCxt, (SShowCreateTableStmt*)pNode); + break; default: break; } @@ -4354,6 +4389,42 @@ static int32_t extractDescribeResultSchema(int32_t* numOfCols, SSchema** pSchema return TSDB_CODE_SUCCESS; } +static int32_t extractShowCreateDatabaseResultSchema(int32_t* numOfCols, SSchema** pSchema) { + *numOfCols = 2; + *pSchema = taosMemoryCalloc((*numOfCols), sizeof(SSchema)); + if (NULL == (*pSchema)) { + return TSDB_CODE_OUT_OF_MEMORY; + } + + (*pSchema)[0].type = TSDB_DATA_TYPE_BINARY; + (*pSchema)[0].bytes = TSDB_DB_NAME_LEN; + strcpy((*pSchema)[0].name, "Database"); + + (*pSchema)[1].type = TSDB_DATA_TYPE_BINARY; + (*pSchema)[1].bytes = TSDB_MAX_BINARY_LEN; + strcpy((*pSchema)[1].name, "Create Database"); + + return TSDB_CODE_SUCCESS; +} + +static int32_t extractShowCreateTableResultSchema(int32_t* numOfCols, SSchema** pSchema) { + *numOfCols = 2; + *pSchema = taosMemoryCalloc((*numOfCols), sizeof(SSchema)); + if (NULL == (*pSchema)) { + return TSDB_CODE_OUT_OF_MEMORY; + } + + (*pSchema)[0].type = TSDB_DATA_TYPE_BINARY; + (*pSchema)[0].bytes = TSDB_TABLE_NAME_LEN; + strcpy((*pSchema)[0].name, "Table"); + + (*pSchema)[1].type = TSDB_DATA_TYPE_BINARY; + (*pSchema)[1].bytes = TSDB_MAX_BINARY_LEN; + strcpy((*pSchema)[1].name, "Create Table"); + + return TSDB_CODE_SUCCESS; +} + int32_t extractResultSchema(const SNode* pRoot, int32_t* numOfCols, SSchema** pSchema) { if (NULL == pRoot) { return TSDB_CODE_SUCCESS; @@ -4367,6 +4438,11 @@ int32_t extractResultSchema(const SNode* pRoot, int32_t* numOfCols, SSchema** pS return extractExplainResultSchema(numOfCols, pSchema); case QUERY_NODE_DESCRIBE_STMT: return extractDescribeResultSchema(numOfCols, pSchema); + case QUERY_NODE_SHOW_CREATE_DATABASE_STMT: + return extractShowCreateDatabaseResultSchema(numOfCols, pSchema); + case QUERY_NODE_SHOW_CREATE_TABLE_STMT: + case QUERY_NODE_SHOW_CREATE_STABLE_STMT: + return extractShowCreateTableResultSchema(numOfCols, pSchema); default: break; } @@ -5014,10 +5090,7 @@ static int32_t rewriteCreateMultiTable(STranslateContext* pCxt, SQuery* pQuery) SNode* pNode; FOREACH(pNode, pStmt->pSubTables) { SCreateSubTableClause* pClause = (SCreateSubTableClause*)pNode; - code = checkSchemalessDb(pCxt, pClause->dbName); - if (TSDB_CODE_SUCCESS == code) { - code = rewriteCreateSubTable(pCxt, pClause, pVgroupHashmap); - } + code = rewriteCreateSubTable(pCxt, pClause, pVgroupHashmap); if (TSDB_CODE_SUCCESS != code) { taosHashCleanup(pVgroupHashmap); return code; @@ -5590,10 +5663,14 @@ static int32_t setQuery(STranslateContext* pCxt, SQuery* pQuery) { pQuery->msgType = toMsgType(((SVnodeModifOpStmt*)pQuery->pRoot)->sqlNodeType); break; case QUERY_NODE_DESCRIBE_STMT: + case QUERY_NODE_SHOW_CREATE_DATABASE_STMT: + case QUERY_NODE_SHOW_CREATE_TABLE_STMT: + case QUERY_NODE_SHOW_CREATE_STABLE_STMT: pQuery->execMode = QUERY_EXEC_MODE_LOCAL; pQuery->haveResultSet = true; break; case QUERY_NODE_RESET_QUERY_CACHE_STMT: + case QUERY_NODE_ALTER_LOCAL_STMT: pQuery->execMode = QUERY_EXEC_MODE_LOCAL; break; default: diff --git a/source/libs/parser/src/parUtil.c b/source/libs/parser/src/parUtil.c index d561b8ac4b..95cdf58d5a 100644 --- a/source/libs/parser/src/parUtil.c +++ b/source/libs/parser/src/parUtil.c @@ -90,7 +90,7 @@ static char* getSyntaxErrFormat(int32_t errCode) { case TSDB_CODE_PAR_GROUPBY_WINDOW_COEXIST: return "GROUP BY and WINDOW-clause can't be used together"; case TSDB_CODE_PAR_INVALID_OPTION_UNIT: - return "Invalid option %s unit: %c, only m, h, d allowed"; + return "Invalid option %s unit: %c, only %c, %c, %c allowed"; case TSDB_CODE_PAR_INVALID_KEEP_UNIT: return "Invalid option keep unit: %c, only m, h, d allowed"; case TSDB_CODE_PAR_AGG_FUNC_NESTING: @@ -194,6 +194,8 @@ static char* getSyntaxErrFormat(int32_t errCode) { return "%s function does not supportted in stream query"; case TSDB_CODE_PAR_GROUP_BY_NOT_ALLOWED_FUNC: return "%s function does not supportted in group query"; + case TSDB_CODE_PAR_INVALID_TABLE_OPTION: + return "Invalid option %s"; case TSDB_CODE_OUT_OF_MEMORY: return "Out of memory"; default: diff --git a/source/libs/parser/src/sql.c b/source/libs/parser/src/sql.c index e81610c1b4..a950b4efb0 100644 --- a/source/libs/parser/src/sql.c +++ b/source/libs/parser/src/sql.c @@ -104,25 +104,25 @@ #endif /************* Begin control #defines *****************************************/ #define YYCODETYPE unsigned short int -#define YYNOCODE 360 +#define YYNOCODE 359 #define YYACTIONTYPE unsigned short int #define ParseTOKENTYPE SToken typedef union { int yyinit; ParseTOKENTYPE yy0; - EFillMode yy54; - int32_t yy100; - bool yy137; - int64_t yy189; - SToken yy209; - ENullOrder yy217; - SDataType yy304; - EOperatorType yy380; - SNodeList* yy424; - EOrder yy578; - SAlterOption yy605; - EJoinType yy612; - SNode* yy632; + EOperatorType yy28; + int32_t yy42; + ENullOrder yy107; + EFillMode yy320; + SToken yy421; + SNodeList* yy530; + SAlterOption yy557; + EOrder yy610; + bool yy621; + EJoinType yy636; + int64_t yy669; + SNode* yy674; + SDataType yy690; } YYMINORTYPE; #ifndef YYSTACKDEPTH #define YYSTACKDEPTH 100 @@ -138,17 +138,17 @@ typedef union { #define ParseCTX_FETCH #define ParseCTX_STORE #define YYFALLBACK 1 -#define YYNSTATE 619 -#define YYNRULE 454 -#define YYNTOKEN 240 -#define YY_MAX_SHIFT 618 -#define YY_MIN_SHIFTREDUCE 906 -#define YY_MAX_SHIFTREDUCE 1359 -#define YY_ERROR_ACTION 1360 -#define YY_ACCEPT_ACTION 1361 -#define YY_NO_ACTION 1362 -#define YY_MIN_REDUCE 1363 -#define YY_MAX_REDUCE 1816 +#define YYNSTATE 622 +#define YYNRULE 459 +#define YYNTOKEN 239 +#define YY_MAX_SHIFT 621 +#define YY_MIN_SHIFTREDUCE 912 +#define YY_MAX_SHIFTREDUCE 1370 +#define YY_ERROR_ACTION 1371 +#define YY_ACCEPT_ACTION 1372 +#define YY_NO_ACTION 1373 +#define YY_MIN_REDUCE 1374 +#define YY_MAX_REDUCE 1832 /************* End control #defines *******************************************/ #define YY_NLOOKAHEAD ((int)(sizeof(yy_lookahead)/sizeof(yy_lookahead[0]))) @@ -215,607 +215,594 @@ typedef union { ** yy_default[] Default action for each state. ** *********** Begin parsing tables **********************************************/ -#define YY_ACTTAB_COUNT (2152) +#define YY_ACTTAB_COUNT (2098) static const YYACTIONTYPE yy_action[] = { - /* 0 */ 28, 231, 1663, 1650, 611, 610, 297, 1650, 358, 1485, - /* 10 */ 314, 1483, 35, 33, 352, 36, 34, 32, 31, 30, - /* 20 */ 306, 24, 1173, 1647, 533, 442, 441, 1647, 79, 1647, - /* 30 */ 1679, 36, 34, 32, 31, 30, 152, 493, 517, 1643, - /* 40 */ 1649, 115, 278, 1643, 1649, 1643, 1649, 1171, 516, 1486, - /* 50 */ 536, 533, 1633, 1494, 536, 1794, 536, 497, 14, 1746, - /* 60 */ 35, 33, 1300, 356, 1179, 1663, 114, 147, 306, 1692, - /* 70 */ 1173, 1791, 82, 1664, 519, 1666, 1667, 515, 532, 536, - /* 80 */ 1494, 1, 1732, 1743, 1794, 1314, 280, 1728, 36, 34, - /* 90 */ 32, 31, 30, 1679, 410, 1171, 1793, 1651, 1794, 520, - /* 100 */ 1791, 517, 309, 615, 112, 1583, 14, 1746, 35, 33, - /* 110 */ 149, 516, 1179, 1172, 1791, 1633, 306, 1647, 1173, 145, - /* 120 */ 1739, 1740, 532, 1744, 36, 34, 32, 31, 30, 2, - /* 130 */ 63, 1742, 1692, 1643, 1649, 84, 1664, 519, 1666, 1667, - /* 140 */ 515, 1364, 536, 1171, 536, 1732, 953, 1794, 952, 1731, - /* 150 */ 1728, 615, 1490, 72, 14, 392, 1174, 393, 1395, 148, - /* 160 */ 1179, 1172, 96, 1791, 532, 95, 94, 93, 92, 91, - /* 170 */ 90, 89, 88, 87, 1487, 954, 56, 2, 132, 201, - /* 180 */ 1375, 1177, 1178, 39, 1224, 1225, 1227, 1228, 1229, 1230, - /* 190 */ 1231, 512, 534, 1239, 1240, 1241, 1242, 1243, 1244, 615, - /* 200 */ 400, 952, 393, 1395, 1174, 55, 1197, 66, 133, 1172, - /* 210 */ 96, 150, 1451, 95, 94, 93, 92, 91, 90, 89, - /* 220 */ 88, 87, 65, 279, 1356, 38, 429, 194, 567, 1177, - /* 230 */ 1178, 1363, 1224, 1225, 1227, 1228, 1229, 1230, 1231, 512, - /* 240 */ 534, 1239, 1240, 1241, 1242, 1243, 1244, 566, 487, 565, - /* 250 */ 564, 563, 1174, 55, 63, 105, 104, 103, 102, 101, - /* 260 */ 100, 99, 98, 97, 936, 35, 33, 110, 36, 34, - /* 270 */ 32, 31, 30, 306, 1746, 1173, 1489, 1177, 1178, 1198, - /* 280 */ 1224, 1225, 1227, 1228, 1229, 1230, 1231, 512, 534, 1239, - /* 290 */ 1240, 1241, 1242, 1243, 1244, 55, 445, 444, 1741, 1538, - /* 300 */ 1171, 443, 940, 941, 111, 440, 296, 1355, 439, 438, - /* 310 */ 437, 1536, 569, 35, 33, 1245, 1679, 1179, 26, 310, - /* 320 */ 1386, 306, 391, 1173, 486, 395, 290, 130, 36, 34, - /* 330 */ 32, 31, 30, 1196, 8, 150, 1496, 1043, 559, 558, - /* 340 */ 557, 1047, 556, 1049, 1050, 555, 1052, 552, 1171, 1058, - /* 350 */ 549, 1060, 1061, 546, 543, 150, 615, 450, 482, 618, - /* 360 */ 485, 35, 33, 585, 583, 1179, 1172, 142, 397, 306, - /* 370 */ 1633, 1173, 458, 248, 1195, 291, 1324, 289, 288, 1532, - /* 380 */ 433, 318, 9, 150, 435, 107, 193, 399, 1574, 1576, - /* 390 */ 395, 607, 603, 599, 595, 247, 1171, 1472, 453, 157, - /* 400 */ 533, 129, 1361, 447, 615, 1425, 434, 572, 192, 1174, - /* 410 */ 1794, 1385, 357, 1179, 1172, 479, 1322, 1323, 1325, 1326, - /* 420 */ 80, 1571, 1792, 242, 61, 150, 1791, 60, 159, 1494, - /* 430 */ 9, 488, 483, 51, 1177, 1178, 50, 1224, 1225, 1227, - /* 440 */ 1228, 1229, 1230, 1231, 512, 534, 1239, 1240, 1241, 1242, - /* 450 */ 1243, 1244, 615, 55, 410, 316, 529, 1174, 54, 319, - /* 460 */ 322, 1633, 1172, 130, 150, 445, 444, 130, 1538, 562, - /* 470 */ 443, 382, 1496, 111, 440, 311, 1496, 439, 438, 437, - /* 480 */ 1536, 474, 1177, 1178, 203, 1224, 1225, 1227, 1228, 1229, - /* 490 */ 1230, 1231, 512, 534, 1239, 1240, 1241, 1242, 1243, 1244, - /* 500 */ 1384, 1794, 1147, 1199, 197, 1174, 1575, 1576, 36, 34, - /* 510 */ 32, 31, 30, 147, 1307, 161, 160, 1791, 35, 33, - /* 520 */ 1197, 1663, 1155, 1156, 195, 1470, 306, 351, 1173, 350, - /* 530 */ 1177, 1178, 493, 1224, 1225, 1227, 1228, 1229, 1230, 1231, - /* 540 */ 512, 534, 1239, 1240, 1241, 1242, 1243, 1244, 253, 1679, - /* 550 */ 1633, 1524, 533, 1171, 1383, 1621, 533, 517, 1382, 1250, - /* 560 */ 277, 114, 1195, 1005, 367, 1197, 1479, 516, 106, 375, - /* 570 */ 1179, 1633, 387, 466, 533, 431, 497, 1422, 1211, 469, - /* 580 */ 1007, 1494, 569, 150, 1381, 1494, 368, 2, 1692, 343, - /* 590 */ 388, 82, 1664, 519, 1666, 1667, 515, 1481, 536, 112, - /* 600 */ 331, 1732, 1477, 1494, 1633, 280, 1728, 198, 1633, 615, - /* 610 */ 345, 341, 533, 495, 144, 1739, 1740, 1794, 1744, 1172, - /* 620 */ 1794, 940, 941, 1538, 106, 1376, 11, 10, 222, 147, - /* 630 */ 317, 436, 147, 1791, 1633, 1536, 1791, 591, 590, 589, - /* 640 */ 321, 1494, 588, 587, 586, 116, 581, 580, 579, 578, - /* 650 */ 577, 576, 575, 574, 123, 570, 32, 31, 30, 1380, - /* 660 */ 386, 1299, 1174, 381, 380, 379, 378, 377, 374, 373, - /* 670 */ 372, 371, 370, 366, 365, 364, 363, 362, 361, 360, - /* 680 */ 359, 499, 1663, 520, 1379, 1226, 1197, 1177, 1178, 1584, - /* 690 */ 1224, 1225, 1227, 1228, 1229, 1230, 1231, 512, 534, 1239, - /* 700 */ 1240, 1241, 1242, 1243, 1244, 1264, 131, 1276, 1378, 1633, - /* 710 */ 1679, 259, 573, 533, 1466, 7, 511, 533, 496, 1471, - /* 720 */ 1200, 533, 459, 257, 53, 409, 1269, 52, 516, 1491, - /* 730 */ 1226, 533, 1633, 1610, 1633, 533, 36, 34, 32, 31, - /* 740 */ 30, 1377, 1494, 244, 162, 1374, 1494, 467, 1663, 1692, - /* 750 */ 1494, 501, 83, 1664, 519, 1666, 1667, 515, 1633, 536, - /* 760 */ 1494, 1373, 1732, 1794, 1494, 25, 299, 1728, 143, 55, - /* 770 */ 36, 34, 32, 31, 30, 147, 1679, 1469, 533, 1791, - /* 780 */ 223, 533, 281, 130, 517, 475, 1759, 435, 533, 1372, - /* 790 */ 530, 1633, 1497, 531, 516, 1633, 185, 1371, 1633, 183, - /* 800 */ 320, 476, 1370, 1369, 1663, 81, 1211, 1494, 1368, 434, - /* 810 */ 1494, 1633, 567, 504, 1262, 1692, 281, 1494, 274, 1664, - /* 820 */ 519, 1666, 1667, 515, 509, 536, 1751, 1295, 493, 561, - /* 830 */ 457, 566, 1679, 565, 564, 563, 59, 58, 355, 1633, - /* 840 */ 514, 156, 1452, 455, 1538, 207, 349, 1633, 1262, 584, - /* 850 */ 516, 1226, 1633, 1633, 1633, 1367, 1537, 114, 1633, 276, - /* 860 */ 1173, 226, 339, 1263, 337, 333, 329, 153, 324, 1298, - /* 870 */ 567, 1692, 1412, 1182, 273, 1664, 519, 1666, 1667, 515, - /* 880 */ 513, 536, 510, 1704, 1268, 1171, 1366, 1407, 1295, 566, - /* 890 */ 346, 565, 564, 563, 446, 112, 120, 1263, 187, 150, - /* 900 */ 189, 186, 1179, 188, 191, 1633, 1663, 190, 46, 448, - /* 910 */ 146, 1739, 1740, 1405, 1744, 11, 10, 1653, 1268, 1358, - /* 920 */ 1359, 480, 1181, 27, 304, 1257, 1258, 1259, 1260, 1261, - /* 930 */ 1265, 1266, 1267, 210, 1679, 451, 1633, 471, 502, 1185, - /* 940 */ 78, 615, 496, 37, 37, 460, 37, 1254, 233, 1321, - /* 950 */ 74, 1172, 516, 217, 1655, 1680, 1633, 27, 304, 1257, - /* 960 */ 1258, 1259, 1260, 1261, 1265, 1266, 1267, 118, 1663, 1396, - /* 970 */ 119, 1533, 120, 1692, 212, 46, 83, 1664, 519, 1666, - /* 980 */ 1667, 515, 977, 536, 1270, 1232, 1732, 1128, 1184, 235, - /* 990 */ 299, 1728, 143, 541, 1174, 428, 1679, 1762, 494, 978, - /* 1000 */ 1663, 225, 505, 119, 517, 228, 120, 230, 525, 3, - /* 1010 */ 1760, 241, 5, 1036, 516, 121, 252, 119, 1633, 1177, - /* 1020 */ 1178, 323, 1195, 326, 330, 286, 287, 1005, 1679, 249, - /* 1030 */ 1139, 369, 1573, 376, 1064, 1692, 517, 158, 83, 1664, - /* 1040 */ 519, 1666, 1667, 515, 1068, 536, 516, 1074, 1732, 384, - /* 1050 */ 1633, 389, 299, 1728, 1807, 383, 1072, 1201, 122, 385, - /* 1060 */ 1663, 390, 1204, 1766, 401, 398, 165, 1692, 402, 167, - /* 1070 */ 83, 1664, 519, 1666, 1667, 515, 1203, 536, 1205, 404, - /* 1080 */ 1732, 403, 170, 406, 299, 1728, 1807, 172, 1679, 407, - /* 1090 */ 1202, 175, 1663, 408, 62, 1789, 517, 411, 178, 430, - /* 1100 */ 432, 1484, 182, 86, 1480, 184, 516, 295, 124, 1179, - /* 1110 */ 1633, 125, 1482, 1478, 1615, 126, 127, 1614, 196, 461, - /* 1120 */ 1679, 199, 250, 202, 465, 462, 1200, 1692, 517, 468, - /* 1130 */ 83, 1664, 519, 1666, 1667, 515, 472, 536, 516, 205, - /* 1140 */ 1732, 481, 1633, 1663, 299, 1728, 1807, 497, 470, 478, - /* 1150 */ 473, 523, 1773, 1772, 298, 1750, 490, 208, 1763, 1692, - /* 1160 */ 211, 6, 264, 1664, 519, 1666, 1667, 515, 1753, 536, - /* 1170 */ 484, 1679, 216, 1663, 477, 113, 1295, 218, 1199, 517, - /* 1180 */ 40, 300, 1747, 506, 503, 18, 527, 1582, 1794, 516, - /* 1190 */ 137, 521, 522, 1633, 219, 1581, 526, 308, 497, 237, - /* 1200 */ 149, 1679, 528, 224, 1791, 251, 1663, 1495, 1713, 517, - /* 1210 */ 1692, 1810, 1790, 264, 1664, 519, 1666, 1667, 515, 516, - /* 1220 */ 536, 239, 71, 1633, 73, 539, 246, 254, 500, 614, - /* 1230 */ 136, 227, 1467, 229, 1679, 507, 1663, 265, 47, 1794, - /* 1240 */ 1692, 256, 517, 84, 1664, 519, 1666, 1667, 515, 258, - /* 1250 */ 536, 147, 516, 1732, 275, 1791, 1633, 508, 1728, 266, - /* 1260 */ 1627, 1626, 57, 1625, 1679, 325, 1622, 327, 328, 332, - /* 1270 */ 1166, 1167, 517, 1692, 154, 1620, 134, 1664, 519, 1666, - /* 1280 */ 1667, 515, 516, 536, 334, 335, 1633, 1663, 336, 1619, - /* 1290 */ 338, 1618, 340, 1617, 342, 1616, 1663, 344, 1600, 155, - /* 1300 */ 1142, 347, 1141, 1692, 348, 1594, 84, 1664, 519, 1666, - /* 1310 */ 1667, 515, 1593, 536, 353, 1679, 1732, 354, 1592, 498, - /* 1320 */ 1808, 1729, 1591, 517, 1679, 1111, 1566, 1565, 1564, 1563, - /* 1330 */ 1562, 1561, 517, 516, 1560, 1559, 1558, 1633, 1557, 1556, - /* 1340 */ 1555, 1554, 516, 1553, 1552, 1551, 1633, 1550, 1549, 117, - /* 1350 */ 1663, 1548, 1547, 1546, 1692, 1545, 1544, 269, 1664, 519, - /* 1360 */ 1666, 1667, 515, 1692, 536, 1543, 134, 1664, 519, 1666, - /* 1370 */ 1667, 515, 1113, 536, 1542, 1541, 1540, 1539, 1679, 1424, - /* 1380 */ 1392, 943, 163, 108, 942, 1391, 517, 1608, 1602, 1590, - /* 1390 */ 1663, 171, 1589, 394, 489, 140, 516, 164, 109, 169, - /* 1400 */ 1633, 396, 1579, 303, 45, 174, 1663, 1473, 1423, 1421, - /* 1410 */ 1809, 1419, 1417, 412, 413, 414, 417, 1692, 1679, 971, - /* 1420 */ 274, 1664, 519, 1666, 1667, 515, 514, 536, 416, 420, - /* 1430 */ 418, 421, 422, 1415, 1679, 426, 516, 424, 1404, 425, - /* 1440 */ 1633, 1403, 517, 1390, 1475, 1077, 181, 1078, 1474, 1413, - /* 1450 */ 1004, 582, 516, 1003, 1663, 584, 1633, 1692, 292, 305, - /* 1460 */ 273, 1664, 519, 1666, 1667, 515, 1408, 536, 1002, 1705, - /* 1470 */ 449, 1001, 998, 1692, 997, 996, 274, 1664, 519, 1666, - /* 1480 */ 1667, 515, 1679, 536, 293, 1406, 1663, 294, 1389, 452, - /* 1490 */ 517, 180, 1388, 454, 456, 85, 1607, 1149, 1601, 463, - /* 1500 */ 516, 1588, 1587, 141, 1633, 1586, 1578, 307, 49, 427, - /* 1510 */ 423, 419, 415, 179, 1679, 67, 1663, 204, 464, 4, - /* 1520 */ 37, 1692, 517, 206, 274, 1664, 519, 1666, 1667, 515, - /* 1530 */ 15, 536, 516, 128, 209, 43, 1633, 1320, 64, 200, - /* 1540 */ 135, 177, 213, 215, 1679, 22, 48, 1653, 214, 1313, - /* 1550 */ 68, 23, 517, 1692, 42, 1292, 260, 1664, 519, 1666, - /* 1560 */ 1667, 515, 516, 536, 221, 1291, 1633, 1663, 138, 1349, - /* 1570 */ 17, 1338, 1344, 1343, 301, 1348, 10, 1347, 302, 19, - /* 1580 */ 1255, 139, 1234, 1692, 151, 29, 268, 1664, 519, 1666, - /* 1590 */ 1667, 515, 12, 536, 1233, 1679, 20, 1219, 176, 16, - /* 1600 */ 168, 518, 173, 517, 405, 41, 13, 1663, 1577, 238, - /* 1610 */ 74, 524, 1652, 516, 232, 21, 234, 1633, 1318, 1189, - /* 1620 */ 236, 240, 166, 69, 70, 243, 1695, 540, 1236, 535, - /* 1630 */ 44, 1071, 1065, 538, 1692, 1679, 315, 270, 1664, 519, - /* 1640 */ 1666, 1667, 515, 517, 536, 542, 544, 1663, 1062, 1059, - /* 1650 */ 545, 547, 548, 516, 550, 1053, 553, 1633, 551, 1057, - /* 1660 */ 1051, 1056, 1663, 554, 1055, 1054, 75, 1042, 76, 560, - /* 1670 */ 1073, 77, 1070, 568, 1692, 1679, 969, 261, 1664, 519, - /* 1680 */ 1666, 1667, 515, 517, 536, 993, 1011, 245, 991, 571, - /* 1690 */ 1679, 986, 1008, 516, 990, 989, 988, 1633, 517, 987, - /* 1700 */ 985, 984, 1006, 981, 980, 979, 976, 975, 516, 974, - /* 1710 */ 1420, 592, 1633, 1663, 1692, 593, 594, 271, 1664, 519, - /* 1720 */ 1666, 1667, 515, 1418, 536, 1663, 597, 596, 598, 1692, - /* 1730 */ 1416, 600, 262, 1664, 519, 1666, 1667, 515, 601, 536, - /* 1740 */ 602, 1679, 604, 605, 606, 1402, 608, 609, 1401, 517, - /* 1750 */ 1414, 1387, 613, 1679, 612, 1175, 255, 616, 617, 516, - /* 1760 */ 1362, 517, 1362, 1633, 1362, 1663, 1362, 1362, 1362, 1362, - /* 1770 */ 1362, 516, 1362, 1362, 1362, 1633, 1663, 1362, 1362, 1362, - /* 1780 */ 1692, 1362, 1362, 272, 1664, 519, 1666, 1667, 515, 1362, - /* 1790 */ 536, 1362, 1692, 1679, 1362, 263, 1664, 519, 1666, 1667, - /* 1800 */ 515, 517, 536, 1362, 1679, 1362, 1362, 1362, 1362, 1362, - /* 1810 */ 1362, 516, 517, 1362, 1362, 1633, 1362, 1362, 1362, 1362, - /* 1820 */ 1362, 1362, 516, 1362, 1362, 1362, 1633, 1663, 1362, 1362, - /* 1830 */ 1362, 1362, 1692, 1362, 1362, 1675, 1664, 519, 1666, 1667, - /* 1840 */ 515, 1362, 536, 1692, 1362, 1362, 1674, 1664, 519, 1666, - /* 1850 */ 1667, 515, 1362, 536, 1362, 1679, 1362, 1663, 1362, 1362, - /* 1860 */ 1362, 1362, 1362, 517, 1362, 1362, 1362, 1362, 1362, 1362, - /* 1870 */ 1362, 1362, 1362, 516, 1362, 1362, 1362, 1633, 1362, 1362, - /* 1880 */ 1362, 1362, 1362, 1362, 1362, 1679, 1362, 1663, 1362, 1362, - /* 1890 */ 1362, 1362, 1362, 517, 1692, 1362, 1362, 1673, 1664, 519, - /* 1900 */ 1666, 1667, 515, 516, 536, 1362, 1362, 1633, 1362, 1362, - /* 1910 */ 1362, 1362, 1362, 1362, 1362, 1679, 1362, 1362, 1362, 1663, - /* 1920 */ 1362, 1362, 1362, 517, 1692, 1362, 1362, 284, 1664, 519, - /* 1930 */ 1666, 1667, 515, 516, 536, 1362, 1362, 1633, 1362, 1362, - /* 1940 */ 1362, 1362, 1362, 313, 312, 1362, 1362, 1679, 1362, 1362, - /* 1950 */ 1362, 1362, 1663, 1187, 1692, 517, 1362, 283, 1664, 519, - /* 1960 */ 1666, 1667, 515, 1362, 536, 516, 1362, 1362, 1362, 1633, - /* 1970 */ 1362, 1362, 1362, 1362, 1362, 1362, 1362, 1362, 1180, 1362, - /* 1980 */ 1679, 1362, 1362, 1362, 1362, 1663, 1692, 1362, 517, 285, - /* 1990 */ 1664, 519, 1666, 1667, 515, 1179, 536, 1362, 516, 1362, - /* 2000 */ 1362, 1362, 1633, 1362, 1362, 1362, 1362, 1362, 1362, 1362, - /* 2010 */ 1362, 1362, 1362, 1679, 1362, 1362, 493, 1362, 1362, 1692, - /* 2020 */ 1362, 517, 282, 1664, 519, 1666, 1667, 515, 1362, 536, - /* 2030 */ 1362, 516, 1362, 1362, 537, 1633, 1362, 1362, 1362, 1362, - /* 2040 */ 1362, 1362, 1362, 1362, 1183, 114, 493, 1362, 1362, 1362, - /* 2050 */ 1362, 1362, 1692, 1362, 1362, 267, 1664, 519, 1666, 1667, - /* 2060 */ 515, 1362, 536, 1362, 497, 1362, 1362, 1362, 1362, 1362, - /* 2070 */ 1362, 1362, 1362, 1362, 1362, 114, 1362, 1362, 1362, 1362, - /* 2080 */ 1362, 1362, 1362, 112, 1362, 1362, 1362, 1188, 1362, 1362, - /* 2090 */ 1362, 1362, 1362, 1362, 497, 1362, 1362, 1362, 220, 1739, - /* 2100 */ 492, 1362, 491, 1362, 1362, 1794, 1362, 1362, 1362, 1362, - /* 2110 */ 1362, 1362, 1191, 112, 1362, 1362, 1362, 149, 1362, 1362, - /* 2120 */ 1362, 1791, 1362, 534, 1239, 1240, 1362, 1362, 220, 1739, - /* 2130 */ 492, 1362, 491, 1362, 1362, 1794, 1362, 1362, 1362, 1362, - /* 2140 */ 1362, 1362, 1362, 1362, 1362, 1362, 1362, 147, 1362, 1362, - /* 2150 */ 1362, 1791, + /* 0 */ 1557, 321, 1679, 1666, 1550, 1552, 300, 396, 1666, 397, + /* 10 */ 1406, 317, 35, 33, 1663, 404, 1556, 397, 1406, 1663, + /* 20 */ 309, 1483, 1184, 1557, 36, 34, 32, 31, 30, 1695, + /* 30 */ 299, 536, 36, 34, 32, 31, 30, 521, 1810, 1555, + /* 40 */ 1659, 1665, 1649, 1810, 520, 1659, 1665, 1182, 28, 233, + /* 50 */ 1809, 540, 362, 1372, 1807, 150, 540, 501, 14, 1807, + /* 60 */ 35, 33, 1311, 395, 1190, 1679, 399, 536, 309, 1708, + /* 70 */ 1184, 159, 82, 1680, 523, 1682, 1683, 519, 414, 540, + /* 80 */ 57, 1, 1748, 536, 401, 1810, 283, 1744, 497, 281, + /* 90 */ 1206, 537, 1695, 537, 1325, 1182, 62, 1808, 1810, 61, + /* 100 */ 521, 1807, 106, 618, 154, 1649, 14, 520, 942, 435, + /* 110 */ 151, 326, 1190, 24, 1807, 414, 114, 1253, 1254, 1505, + /* 120 */ 501, 1505, 356, 36, 34, 32, 31, 30, 39, 2, + /* 130 */ 134, 497, 1708, 1695, 1462, 82, 1680, 523, 1682, 1683, + /* 140 */ 519, 490, 540, 524, 38, 1748, 946, 947, 1318, 283, + /* 150 */ 1744, 618, 1810, 1600, 1208, 112, 196, 56, 1185, 114, + /* 160 */ 1183, 1810, 1208, 1810, 149, 1253, 1254, 130, 1807, 499, + /* 170 */ 146, 1755, 1756, 149, 1760, 149, 574, 1807, 489, 1807, + /* 180 */ 508, 486, 1188, 1189, 347, 1235, 1236, 1238, 1239, 1240, + /* 190 */ 1241, 1242, 516, 538, 1250, 1251, 1252, 1255, 112, 36, + /* 200 */ 34, 32, 31, 30, 349, 345, 1185, 133, 1183, 1386, + /* 210 */ 152, 66, 282, 147, 1755, 1756, 1209, 1760, 524, 35, + /* 220 */ 33, 36, 34, 32, 31, 30, 312, 309, 1599, 1184, + /* 230 */ 1188, 1189, 1375, 1235, 1236, 1238, 1239, 1240, 1241, 1242, + /* 240 */ 516, 538, 1250, 1251, 1252, 1255, 36, 34, 32, 31, + /* 250 */ 30, 492, 487, 96, 1182, 73, 95, 94, 93, 92, + /* 260 */ 91, 90, 89, 88, 87, 14, 491, 35, 33, 1222, + /* 270 */ 26, 1190, 1433, 152, 1496, 309, 1498, 1184, 1193, 56, + /* 280 */ 36, 34, 32, 31, 30, 1663, 152, 403, 2, 1335, + /* 290 */ 399, 1049, 563, 562, 561, 1053, 560, 1055, 1056, 559, + /* 300 */ 1058, 556, 1182, 1064, 553, 1066, 1067, 550, 547, 1261, + /* 310 */ 618, 1659, 1665, 1091, 1092, 1208, 32, 31, 30, 1190, + /* 320 */ 1237, 589, 540, 1397, 1253, 1254, 483, 1333, 1334, 1336, + /* 330 */ 1337, 1209, 596, 595, 594, 324, 8, 593, 592, 591, + /* 340 */ 116, 586, 585, 584, 583, 582, 581, 580, 579, 124, + /* 350 */ 575, 1436, 578, 313, 1477, 355, 56, 354, 618, 1367, + /* 360 */ 1207, 131, 350, 1649, 572, 1185, 1396, 1183, 509, 1395, + /* 370 */ 1507, 96, 1253, 1254, 95, 94, 93, 92, 91, 90, + /* 380 */ 89, 88, 87, 122, 121, 569, 568, 567, 64, 1188, + /* 390 */ 1189, 1196, 1235, 1236, 1238, 1239, 1240, 1241, 1242, 516, + /* 400 */ 538, 1250, 1251, 1252, 1255, 497, 1649, 537, 152, 1649, + /* 410 */ 1501, 449, 448, 1185, 1557, 1183, 447, 1394, 360, 111, + /* 420 */ 444, 314, 1393, 443, 442, 441, 35, 33, 1256, 570, + /* 430 */ 1555, 1679, 1548, 114, 309, 1505, 1184, 1188, 1189, 1366, + /* 440 */ 1235, 1236, 1238, 1239, 1240, 1241, 1242, 516, 538, 1250, + /* 450 */ 1251, 1252, 1255, 1392, 1184, 449, 448, 1649, 1695, 1391, + /* 460 */ 447, 1182, 1649, 111, 444, 152, 521, 443, 442, 441, + /* 470 */ 197, 1649, 112, 520, 35, 33, 1551, 1552, 1190, 1182, + /* 480 */ 1190, 1237, 309, 1679, 1184, 152, 537, 148, 1755, 1756, + /* 490 */ 293, 1760, 959, 1649, 958, 9, 1190, 361, 1708, 1649, + /* 500 */ 64, 84, 1680, 523, 1682, 1683, 519, 1390, 540, 1182, + /* 510 */ 1695, 1748, 203, 110, 1505, 1747, 1744, 618, 500, 433, + /* 520 */ 473, 960, 1500, 1649, 571, 520, 1190, 1548, 1166, 1167, + /* 530 */ 187, 1253, 1254, 185, 537, 618, 316, 315, 56, 294, + /* 540 */ 67, 292, 291, 9, 437, 106, 1198, 1649, 439, 1482, + /* 550 */ 1708, 1310, 440, 83, 1680, 523, 1682, 1683, 519, 1011, + /* 560 */ 540, 1810, 1505, 1748, 959, 618, 958, 302, 1744, 145, + /* 570 */ 438, 1191, 1185, 149, 1183, 463, 1013, 1807, 1557, 1253, + /* 580 */ 1254, 225, 446, 445, 537, 320, 479, 1775, 1190, 386, + /* 590 */ 1185, 7, 1183, 960, 1555, 371, 1188, 1189, 1208, 1235, + /* 600 */ 1236, 1238, 1239, 1240, 1241, 1242, 516, 538, 1250, 1251, + /* 610 */ 1252, 1255, 1505, 319, 1188, 1189, 1810, 1275, 946, 947, + /* 620 */ 1185, 131, 1183, 322, 144, 79, 1389, 541, 149, 1388, + /* 630 */ 1507, 131, 1807, 35, 33, 163, 162, 1544, 115, 1280, + /* 640 */ 1507, 309, 572, 1184, 1188, 1189, 1497, 1235, 1236, 1238, + /* 650 */ 1239, 1240, 1241, 1242, 516, 538, 1250, 1251, 1252, 1255, + /* 660 */ 1385, 122, 121, 569, 568, 567, 1649, 152, 1182, 1649, + /* 670 */ 537, 1384, 11, 10, 25, 1667, 1383, 537, 280, 54, + /* 680 */ 1206, 372, 1199, 1762, 1194, 1190, 1663, 379, 413, 1382, + /* 690 */ 391, 36, 34, 32, 31, 30, 590, 588, 1505, 537, + /* 700 */ 1649, 1679, 2, 1210, 132, 1505, 1202, 1759, 392, 262, + /* 710 */ 1502, 1649, 1659, 1665, 1192, 1381, 1649, 538, 1250, 1251, + /* 720 */ 1374, 260, 53, 540, 618, 52, 1380, 1505, 1695, 1649, + /* 730 */ 1379, 256, 1762, 1762, 1535, 505, 521, 284, 1253, 1254, + /* 740 */ 1480, 1649, 164, 520, 105, 104, 103, 102, 101, 100, + /* 750 */ 99, 98, 97, 1287, 480, 1649, 1758, 1757, 1309, 1767, + /* 760 */ 1306, 1222, 1378, 1306, 1237, 566, 1649, 56, 1708, 1273, + /* 770 */ 1649, 278, 1680, 523, 1682, 1683, 519, 577, 540, 1185, + /* 780 */ 390, 1183, 1481, 385, 384, 383, 382, 381, 378, 377, + /* 790 */ 376, 375, 374, 370, 369, 368, 367, 366, 365, 364, + /* 800 */ 363, 1211, 1649, 1188, 1189, 81, 1235, 1236, 1238, 1239, + /* 810 */ 1240, 1241, 1242, 516, 538, 1250, 1251, 1252, 1255, 131, + /* 820 */ 1274, 537, 36, 34, 32, 31, 30, 1195, 1508, 537, + /* 830 */ 224, 1637, 1626, 572, 537, 1377, 60, 59, 359, 574, + /* 840 */ 471, 158, 1279, 189, 513, 534, 188, 353, 191, 1505, + /* 850 */ 439, 190, 122, 121, 569, 568, 567, 1505, 279, 537, + /* 860 */ 537, 343, 1505, 341, 337, 333, 155, 328, 284, 1494, + /* 870 */ 535, 246, 438, 1423, 1490, 1649, 335, 27, 307, 1268, + /* 880 */ 1269, 1270, 1271, 1272, 1276, 1277, 1278, 1505, 1505, 1418, + /* 890 */ 1679, 1590, 537, 1416, 193, 450, 152, 192, 1669, 461, + /* 900 */ 1273, 503, 161, 323, 120, 621, 46, 983, 212, 1369, + /* 910 */ 1370, 452, 459, 11, 10, 455, 78, 1695, 37, 251, + /* 920 */ 1505, 506, 1492, 1488, 984, 500, 75, 200, 209, 515, + /* 930 */ 1649, 142, 520, 565, 182, 1671, 37, 612, 608, 604, + /* 940 */ 600, 250, 470, 1387, 37, 475, 137, 1332, 235, 214, + /* 950 */ 118, 1274, 431, 427, 423, 419, 181, 1708, 228, 1281, + /* 960 */ 83, 1680, 523, 1682, 1683, 519, 80, 540, 1265, 244, + /* 970 */ 1748, 484, 1463, 1279, 302, 1744, 145, 1243, 119, 120, + /* 980 */ 46, 65, 545, 119, 179, 1139, 120, 107, 1412, 237, + /* 990 */ 464, 529, 119, 432, 1776, 1696, 219, 325, 1407, 1545, + /* 1000 */ 1778, 498, 533, 3, 1679, 5, 227, 230, 27, 307, + /* 1010 */ 1268, 1269, 1270, 1271, 1272, 1276, 1277, 1278, 327, 243, + /* 1020 */ 1042, 255, 1206, 1070, 1074, 232, 330, 1081, 1079, 478, + /* 1030 */ 1679, 1695, 205, 123, 614, 334, 289, 1150, 1011, 521, + /* 1040 */ 290, 252, 380, 178, 1649, 170, 520, 175, 373, 409, + /* 1050 */ 1592, 1158, 160, 199, 388, 387, 1679, 1695, 389, 1212, + /* 1060 */ 393, 394, 402, 1215, 405, 521, 167, 406, 168, 169, + /* 1070 */ 1649, 1708, 520, 1214, 83, 1680, 523, 1682, 1683, 519, + /* 1080 */ 407, 540, 1216, 1695, 1748, 408, 172, 410, 302, 1744, + /* 1090 */ 1823, 521, 174, 1213, 412, 177, 1649, 1708, 520, 1782, + /* 1100 */ 83, 1680, 523, 1682, 1683, 519, 411, 540, 1679, 63, + /* 1110 */ 1748, 415, 180, 434, 302, 1744, 1823, 436, 1495, 184, + /* 1120 */ 1491, 186, 1679, 1708, 125, 1805, 83, 1680, 523, 1682, + /* 1130 */ 1683, 519, 298, 540, 86, 1695, 1748, 126, 198, 1493, + /* 1140 */ 302, 1744, 1823, 521, 1489, 127, 1631, 253, 1649, 1695, + /* 1150 */ 520, 1766, 128, 1630, 472, 465, 469, 521, 1211, 466, + /* 1160 */ 527, 201, 1649, 501, 520, 204, 207, 476, 485, 477, + /* 1170 */ 474, 1789, 1779, 1679, 482, 1708, 301, 501, 269, 1680, + /* 1180 */ 523, 1682, 1683, 519, 210, 540, 488, 1788, 6, 1708, + /* 1190 */ 454, 213, 269, 1680, 523, 1682, 1683, 519, 494, 540, + /* 1200 */ 1695, 1306, 1769, 220, 1810, 462, 481, 113, 521, 1210, + /* 1210 */ 1763, 40, 510, 1649, 303, 520, 151, 18, 1810, 195, + /* 1220 */ 1807, 1598, 507, 530, 531, 218, 525, 1597, 221, 526, + /* 1230 */ 149, 457, 239, 241, 1807, 311, 451, 1679, 139, 532, + /* 1240 */ 1708, 194, 1826, 84, 1680, 523, 1682, 1683, 519, 1729, + /* 1250 */ 540, 1679, 254, 1748, 72, 1806, 1506, 512, 1744, 504, + /* 1260 */ 1679, 511, 74, 226, 1695, 1478, 51, 543, 229, 50, + /* 1270 */ 231, 257, 518, 617, 249, 47, 138, 1649, 1695, 520, + /* 1280 */ 1549, 261, 263, 270, 264, 259, 521, 1695, 1643, 1642, + /* 1290 */ 58, 1649, 1641, 520, 329, 521, 1638, 332, 331, 1177, + /* 1300 */ 1649, 1178, 520, 156, 1708, 336, 1636, 277, 1680, 523, + /* 1310 */ 1682, 1683, 519, 517, 540, 514, 1720, 1679, 1708, 338, + /* 1320 */ 339, 135, 1680, 523, 1682, 1683, 519, 1708, 540, 340, + /* 1330 */ 84, 1680, 523, 1682, 1683, 519, 1635, 540, 342, 1679, + /* 1340 */ 1748, 1634, 344, 1633, 1695, 1745, 346, 1632, 348, 1616, + /* 1350 */ 157, 351, 521, 352, 1153, 1152, 1610, 1649, 1609, 520, + /* 1360 */ 357, 358, 1608, 1607, 502, 1824, 1695, 1125, 1585, 1584, + /* 1370 */ 1583, 1582, 1581, 1580, 521, 1579, 1578, 1577, 1576, 1649, + /* 1380 */ 1575, 520, 1574, 1573, 1708, 1572, 1571, 273, 1680, 523, + /* 1390 */ 1682, 1683, 519, 1679, 540, 1570, 1569, 1568, 1567, 117, + /* 1400 */ 1566, 1565, 1564, 1563, 1562, 1679, 1708, 1561, 1560, 135, + /* 1410 */ 1680, 523, 1682, 1683, 519, 1127, 540, 1559, 1558, 1435, + /* 1420 */ 1695, 1403, 143, 949, 493, 108, 948, 165, 521, 1402, + /* 1430 */ 1624, 1618, 1695, 1649, 398, 520, 400, 166, 1606, 173, + /* 1440 */ 518, 1605, 109, 1595, 1484, 1649, 306, 520, 171, 1434, + /* 1450 */ 176, 1432, 418, 1825, 1430, 1428, 1679, 422, 1426, 426, + /* 1460 */ 1708, 416, 430, 278, 1680, 523, 1682, 1683, 519, 977, + /* 1470 */ 540, 417, 1708, 421, 420, 277, 1680, 523, 1682, 1683, + /* 1480 */ 519, 424, 540, 1695, 1721, 425, 428, 429, 1415, 1414, + /* 1490 */ 1401, 521, 1486, 1085, 1084, 1485, 1649, 1010, 520, 1009, + /* 1500 */ 1008, 587, 589, 1007, 1004, 1424, 45, 1679, 295, 308, + /* 1510 */ 1003, 183, 1002, 1419, 453, 296, 1417, 456, 297, 1400, + /* 1520 */ 458, 1679, 1399, 1708, 460, 1623, 278, 1680, 523, 1682, + /* 1530 */ 1683, 519, 85, 540, 1695, 1617, 49, 467, 1160, 1604, + /* 1540 */ 1603, 1602, 521, 129, 1594, 68, 206, 1649, 1695, 520, + /* 1550 */ 211, 4, 37, 15, 136, 43, 521, 1331, 215, 22, + /* 1560 */ 310, 1649, 1324, 520, 216, 217, 69, 23, 1669, 16, + /* 1570 */ 1679, 468, 223, 41, 1708, 208, 202, 278, 1680, 523, + /* 1580 */ 1682, 1683, 519, 48, 540, 1303, 1302, 1679, 1708, 42, + /* 1590 */ 140, 265, 1680, 523, 1682, 1683, 519, 1695, 540, 1360, + /* 1600 */ 17, 1355, 1349, 1354, 10, 521, 19, 304, 1359, 1358, + /* 1610 */ 1649, 305, 520, 1245, 1695, 29, 141, 153, 1244, 1266, + /* 1620 */ 234, 1230, 521, 12, 20, 1593, 21, 1649, 236, 520, + /* 1630 */ 240, 1329, 522, 238, 70, 1668, 1679, 1708, 245, 528, + /* 1640 */ 272, 1680, 523, 1682, 1683, 519, 75, 540, 1679, 13, + /* 1650 */ 71, 1711, 1200, 1247, 1708, 539, 44, 274, 1680, 523, + /* 1660 */ 1682, 1683, 519, 1695, 540, 1071, 544, 318, 542, 242, + /* 1670 */ 546, 521, 1068, 548, 1065, 1695, 1649, 549, 520, 551, + /* 1680 */ 554, 552, 555, 521, 557, 1059, 1057, 558, 1649, 1679, + /* 1690 */ 520, 1063, 1062, 1048, 76, 77, 1080, 55, 564, 247, + /* 1700 */ 1061, 1076, 1060, 1708, 975, 573, 266, 1680, 523, 1682, + /* 1710 */ 1683, 519, 999, 540, 1017, 1708, 1695, 576, 275, 1680, + /* 1720 */ 523, 1682, 1683, 519, 521, 540, 248, 997, 996, 1649, + /* 1730 */ 995, 520, 992, 994, 993, 991, 990, 1014, 1012, 987, + /* 1740 */ 986, 1679, 985, 982, 981, 980, 1431, 597, 598, 599, + /* 1750 */ 1679, 1429, 601, 602, 1427, 603, 1708, 1425, 605, 267, + /* 1760 */ 1680, 523, 1682, 1683, 519, 606, 540, 607, 1695, 609, + /* 1770 */ 610, 611, 1413, 613, 1398, 615, 521, 1695, 1186, 616, + /* 1780 */ 619, 1649, 258, 520, 620, 521, 1373, 1373, 1373, 1373, + /* 1790 */ 1649, 1373, 520, 1373, 1373, 1373, 1373, 1373, 1373, 1679, + /* 1800 */ 1373, 1373, 1373, 1373, 1373, 1373, 1373, 1373, 1708, 1373, + /* 1810 */ 1373, 276, 1680, 523, 1682, 1683, 519, 1708, 540, 1373, + /* 1820 */ 268, 1680, 523, 1682, 1683, 519, 1695, 540, 1373, 1373, + /* 1830 */ 1373, 1373, 1373, 1373, 521, 1373, 1373, 497, 1373, 1649, + /* 1840 */ 1679, 520, 1373, 1373, 1373, 1373, 1373, 1373, 1373, 1373, + /* 1850 */ 1373, 1373, 1373, 1373, 1373, 1373, 1373, 1679, 1373, 1373, + /* 1860 */ 1373, 1373, 1373, 1373, 1373, 114, 1708, 1695, 1373, 1691, + /* 1870 */ 1680, 523, 1682, 1683, 519, 521, 540, 1373, 1373, 1373, + /* 1880 */ 1649, 1373, 520, 1373, 1695, 501, 1373, 1373, 1373, 1373, + /* 1890 */ 1373, 1373, 521, 1373, 1373, 1373, 1373, 1649, 1373, 520, + /* 1900 */ 1373, 1373, 1373, 1373, 112, 1373, 1679, 1708, 1373, 1373, + /* 1910 */ 1690, 1680, 523, 1682, 1683, 519, 1373, 540, 1679, 222, + /* 1920 */ 1755, 496, 1373, 495, 1708, 1373, 1810, 1689, 1680, 523, + /* 1930 */ 1682, 1683, 519, 1695, 540, 1373, 1373, 1373, 151, 497, + /* 1940 */ 1373, 521, 1807, 1373, 1373, 1695, 1649, 1373, 520, 1373, + /* 1950 */ 1373, 1373, 1373, 521, 1373, 1373, 1373, 1373, 1649, 1679, + /* 1960 */ 520, 1373, 1373, 1373, 1373, 1373, 1373, 114, 1373, 1373, + /* 1970 */ 1373, 1373, 1373, 1708, 1373, 1373, 287, 1680, 523, 1682, + /* 1980 */ 1683, 519, 1373, 540, 1373, 1708, 1695, 501, 286, 1680, + /* 1990 */ 523, 1682, 1683, 519, 521, 540, 1373, 1373, 1373, 1649, + /* 2000 */ 1373, 520, 1373, 1373, 1373, 1373, 112, 1373, 1373, 1373, + /* 2010 */ 1373, 1679, 1373, 1373, 1373, 1373, 1373, 1373, 1373, 1373, + /* 2020 */ 1679, 222, 1755, 496, 1373, 495, 1708, 1373, 1810, 288, + /* 2030 */ 1680, 523, 1682, 1683, 519, 1373, 540, 1373, 1695, 1373, + /* 2040 */ 149, 1373, 1373, 1373, 1807, 1373, 521, 1695, 1373, 1373, + /* 2050 */ 1373, 1649, 1373, 520, 1373, 521, 1373, 1373, 1373, 1373, + /* 2060 */ 1649, 1373, 520, 1373, 1373, 1373, 1373, 1373, 1373, 1373, + /* 2070 */ 1373, 1373, 1373, 1373, 1373, 1373, 1373, 1373, 1708, 1373, + /* 2080 */ 1373, 285, 1680, 523, 1682, 1683, 519, 1708, 540, 1373, + /* 2090 */ 271, 1680, 523, 1682, 1683, 519, 1373, 540, }; static const YYCODETYPE yy_lookahead[] = { - /* 0 */ 324, 325, 243, 273, 251, 252, 276, 273, 250, 273, - /* 10 */ 276, 272, 12, 13, 298, 12, 13, 14, 15, 16, - /* 20 */ 20, 2, 22, 293, 250, 257, 258, 293, 253, 293, - /* 30 */ 271, 12, 13, 14, 15, 16, 262, 250, 279, 309, - /* 40 */ 310, 266, 284, 309, 310, 309, 310, 47, 289, 274, - /* 50 */ 320, 250, 293, 279, 320, 339, 320, 298, 58, 311, - /* 60 */ 12, 13, 14, 262, 64, 243, 279, 351, 20, 310, - /* 70 */ 22, 355, 313, 314, 315, 316, 317, 318, 20, 320, - /* 80 */ 279, 81, 323, 335, 339, 82, 327, 328, 12, 13, - /* 90 */ 14, 15, 16, 271, 57, 47, 351, 273, 339, 289, - /* 100 */ 355, 279, 292, 103, 317, 295, 58, 311, 12, 13, - /* 110 */ 351, 289, 64, 113, 355, 293, 20, 293, 22, 332, - /* 120 */ 333, 334, 20, 336, 12, 13, 14, 15, 16, 81, - /* 130 */ 255, 335, 310, 309, 310, 313, 314, 315, 316, 317, - /* 140 */ 318, 0, 320, 47, 320, 323, 20, 339, 22, 327, - /* 150 */ 328, 103, 277, 253, 58, 246, 156, 248, 249, 351, - /* 160 */ 64, 113, 21, 355, 20, 24, 25, 26, 27, 28, - /* 170 */ 29, 30, 31, 32, 274, 49, 4, 81, 242, 55, - /* 180 */ 244, 181, 182, 81, 184, 185, 186, 187, 188, 189, - /* 190 */ 190, 191, 192, 193, 194, 195, 196, 197, 198, 103, - /* 200 */ 246, 22, 248, 249, 156, 81, 20, 83, 256, 113, - /* 210 */ 21, 211, 260, 24, 25, 26, 27, 28, 29, 30, - /* 220 */ 31, 32, 165, 166, 148, 81, 47, 170, 93, 181, - /* 230 */ 182, 0, 184, 185, 186, 187, 188, 189, 190, 191, - /* 240 */ 192, 193, 194, 195, 196, 197, 198, 112, 20, 114, - /* 250 */ 115, 116, 156, 81, 255, 24, 25, 26, 27, 28, - /* 260 */ 29, 30, 31, 32, 4, 12, 13, 268, 12, 13, - /* 270 */ 14, 15, 16, 20, 311, 22, 277, 181, 182, 20, - /* 280 */ 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, - /* 290 */ 194, 195, 196, 197, 198, 81, 60, 61, 335, 271, - /* 300 */ 47, 65, 42, 43, 68, 69, 278, 231, 72, 73, - /* 310 */ 74, 283, 57, 12, 13, 14, 271, 64, 2, 263, - /* 320 */ 243, 20, 247, 22, 279, 250, 35, 271, 12, 13, - /* 330 */ 14, 15, 16, 20, 81, 211, 280, 94, 95, 96, - /* 340 */ 97, 98, 99, 100, 101, 102, 103, 104, 47, 106, - /* 350 */ 107, 108, 109, 110, 111, 211, 103, 4, 143, 19, - /* 360 */ 315, 12, 13, 257, 258, 64, 113, 270, 14, 20, - /* 370 */ 293, 22, 19, 33, 20, 84, 181, 86, 87, 282, - /* 380 */ 89, 281, 81, 211, 93, 45, 33, 247, 288, 289, - /* 390 */ 250, 51, 52, 53, 54, 55, 47, 0, 45, 55, - /* 400 */ 250, 145, 240, 50, 103, 0, 115, 64, 55, 156, - /* 410 */ 339, 243, 262, 64, 113, 220, 221, 222, 223, 224, - /* 420 */ 80, 279, 351, 83, 80, 211, 355, 83, 286, 279, - /* 430 */ 81, 216, 217, 80, 181, 182, 83, 184, 185, 186, - /* 440 */ 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, - /* 450 */ 197, 198, 103, 81, 57, 263, 116, 156, 3, 263, - /* 460 */ 298, 293, 113, 271, 211, 60, 61, 271, 271, 92, - /* 470 */ 65, 75, 280, 68, 69, 278, 280, 72, 73, 74, - /* 480 */ 283, 141, 181, 182, 144, 184, 185, 186, 187, 188, - /* 490 */ 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, - /* 500 */ 243, 339, 162, 20, 164, 156, 288, 289, 12, 13, - /* 510 */ 14, 15, 16, 351, 14, 119, 120, 355, 12, 13, - /* 520 */ 20, 243, 167, 168, 169, 0, 20, 155, 22, 157, - /* 530 */ 181, 182, 250, 184, 185, 186, 187, 188, 189, 190, - /* 540 */ 191, 192, 193, 194, 195, 196, 197, 198, 264, 271, - /* 550 */ 293, 267, 250, 47, 243, 0, 250, 279, 243, 14, - /* 560 */ 18, 279, 20, 47, 262, 20, 272, 289, 262, 27, - /* 570 */ 64, 293, 30, 302, 250, 269, 298, 0, 82, 298, - /* 580 */ 64, 279, 57, 211, 243, 279, 262, 81, 310, 151, - /* 590 */ 48, 313, 314, 315, 316, 317, 318, 272, 320, 317, - /* 600 */ 45, 323, 272, 279, 293, 327, 328, 272, 293, 103, - /* 610 */ 172, 173, 250, 331, 332, 333, 334, 339, 336, 113, - /* 620 */ 339, 42, 43, 271, 262, 244, 1, 2, 145, 351, - /* 630 */ 278, 269, 351, 355, 293, 283, 355, 60, 61, 62, - /* 640 */ 63, 279, 65, 66, 67, 68, 69, 70, 71, 72, - /* 650 */ 73, 74, 75, 76, 77, 78, 14, 15, 16, 243, - /* 660 */ 118, 4, 156, 121, 122, 123, 124, 125, 126, 127, - /* 670 */ 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, - /* 680 */ 138, 226, 243, 289, 243, 185, 20, 181, 182, 295, - /* 690 */ 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, - /* 700 */ 194, 195, 196, 197, 198, 139, 18, 82, 243, 293, - /* 710 */ 271, 23, 259, 250, 261, 37, 272, 250, 279, 0, - /* 720 */ 20, 250, 298, 35, 36, 262, 160, 39, 289, 262, - /* 730 */ 185, 250, 293, 262, 293, 250, 12, 13, 14, 15, - /* 740 */ 16, 243, 279, 262, 56, 243, 279, 262, 243, 310, - /* 750 */ 279, 41, 313, 314, 315, 316, 317, 318, 293, 320, - /* 760 */ 279, 243, 323, 339, 279, 199, 327, 328, 329, 81, - /* 770 */ 12, 13, 14, 15, 16, 351, 271, 0, 250, 355, - /* 780 */ 341, 250, 58, 271, 279, 346, 347, 93, 250, 243, - /* 790 */ 262, 293, 280, 262, 289, 293, 85, 243, 293, 88, - /* 800 */ 262, 296, 243, 243, 243, 117, 82, 279, 243, 115, - /* 810 */ 279, 293, 93, 41, 90, 310, 58, 279, 313, 314, - /* 820 */ 315, 316, 317, 318, 58, 320, 209, 210, 250, 272, - /* 830 */ 21, 112, 271, 114, 115, 116, 148, 149, 150, 293, - /* 840 */ 279, 153, 260, 34, 271, 145, 158, 293, 90, 41, - /* 850 */ 289, 185, 293, 293, 293, 243, 283, 279, 293, 171, - /* 860 */ 22, 358, 174, 139, 176, 177, 178, 179, 180, 212, - /* 870 */ 93, 310, 0, 47, 313, 314, 315, 316, 317, 318, - /* 880 */ 319, 320, 321, 322, 160, 47, 243, 0, 210, 112, - /* 890 */ 82, 114, 115, 116, 22, 317, 41, 139, 85, 211, - /* 900 */ 85, 88, 64, 88, 85, 293, 243, 88, 41, 22, - /* 910 */ 332, 333, 334, 0, 336, 1, 2, 44, 160, 196, - /* 920 */ 197, 349, 47, 199, 200, 201, 202, 203, 204, 205, - /* 930 */ 206, 207, 208, 41, 271, 22, 293, 82, 228, 113, - /* 940 */ 81, 103, 279, 41, 41, 306, 41, 181, 41, 82, - /* 950 */ 91, 113, 289, 343, 81, 271, 293, 199, 200, 201, - /* 960 */ 202, 203, 204, 205, 206, 207, 208, 41, 243, 249, - /* 970 */ 41, 282, 41, 310, 82, 41, 313, 314, 315, 316, - /* 980 */ 317, 318, 47, 320, 82, 82, 323, 82, 113, 82, - /* 990 */ 327, 328, 329, 41, 156, 251, 271, 312, 337, 64, - /* 1000 */ 243, 352, 230, 41, 279, 352, 41, 352, 82, 340, - /* 1010 */ 347, 82, 213, 82, 289, 41, 82, 41, 293, 181, - /* 1020 */ 182, 308, 20, 250, 45, 307, 257, 47, 271, 300, - /* 1030 */ 154, 250, 250, 287, 82, 310, 279, 40, 313, 314, - /* 1040 */ 315, 316, 317, 318, 82, 320, 289, 82, 323, 139, - /* 1050 */ 293, 250, 327, 328, 329, 285, 82, 20, 82, 285, - /* 1060 */ 243, 245, 20, 338, 304, 245, 255, 310, 289, 255, - /* 1070 */ 313, 314, 315, 316, 317, 318, 20, 320, 20, 299, - /* 1080 */ 323, 297, 255, 297, 327, 328, 329, 255, 271, 279, - /* 1090 */ 20, 255, 243, 290, 255, 338, 279, 250, 255, 245, - /* 1100 */ 271, 271, 271, 250, 271, 271, 289, 245, 271, 64, - /* 1110 */ 293, 271, 271, 271, 293, 271, 271, 293, 253, 163, - /* 1120 */ 271, 253, 304, 253, 289, 303, 20, 310, 279, 250, - /* 1130 */ 313, 314, 315, 316, 317, 318, 279, 320, 289, 253, - /* 1140 */ 323, 219, 293, 243, 327, 328, 329, 298, 297, 293, - /* 1150 */ 290, 218, 348, 348, 293, 338, 147, 294, 312, 310, - /* 1160 */ 294, 225, 313, 314, 315, 316, 317, 318, 345, 320, - /* 1170 */ 293, 271, 344, 243, 214, 279, 210, 308, 20, 279, - /* 1180 */ 40, 232, 311, 229, 227, 81, 291, 294, 339, 289, - /* 1190 */ 342, 293, 293, 293, 330, 294, 142, 293, 298, 279, - /* 1200 */ 351, 271, 290, 353, 355, 267, 243, 279, 326, 279, - /* 1210 */ 310, 359, 354, 313, 314, 315, 316, 317, 318, 289, - /* 1220 */ 320, 253, 253, 293, 81, 275, 253, 250, 354, 245, - /* 1230 */ 305, 353, 261, 353, 271, 354, 243, 265, 301, 339, - /* 1240 */ 310, 254, 279, 313, 314, 315, 316, 317, 318, 241, - /* 1250 */ 320, 351, 289, 323, 265, 355, 293, 327, 328, 265, - /* 1260 */ 0, 0, 40, 0, 271, 72, 0, 47, 175, 175, - /* 1270 */ 47, 47, 279, 310, 47, 0, 313, 314, 315, 316, - /* 1280 */ 317, 318, 289, 320, 47, 47, 293, 243, 175, 0, - /* 1290 */ 175, 0, 47, 0, 22, 0, 243, 47, 0, 81, - /* 1300 */ 113, 160, 156, 310, 159, 0, 313, 314, 315, 316, - /* 1310 */ 317, 318, 0, 320, 152, 271, 323, 151, 0, 356, - /* 1320 */ 357, 328, 0, 279, 271, 44, 0, 0, 0, 0, - /* 1330 */ 0, 0, 279, 289, 0, 0, 0, 293, 0, 0, - /* 1340 */ 0, 0, 289, 0, 0, 0, 293, 0, 0, 40, - /* 1350 */ 243, 0, 0, 0, 310, 0, 0, 313, 314, 315, - /* 1360 */ 316, 317, 318, 310, 320, 0, 313, 314, 315, 316, - /* 1370 */ 317, 318, 22, 320, 0, 0, 0, 0, 271, 0, - /* 1380 */ 0, 14, 40, 37, 14, 0, 279, 0, 0, 0, - /* 1390 */ 243, 147, 0, 44, 350, 41, 289, 38, 37, 37, - /* 1400 */ 293, 44, 0, 296, 90, 37, 243, 0, 0, 0, - /* 1410 */ 357, 0, 0, 47, 45, 37, 45, 310, 271, 59, - /* 1420 */ 313, 314, 315, 316, 317, 318, 279, 320, 47, 47, - /* 1430 */ 37, 45, 37, 0, 271, 37, 289, 47, 0, 45, - /* 1440 */ 293, 0, 279, 0, 0, 22, 88, 47, 0, 0, - /* 1450 */ 47, 41, 289, 47, 243, 41, 293, 310, 22, 296, - /* 1460 */ 313, 314, 315, 316, 317, 318, 0, 320, 47, 322, - /* 1470 */ 48, 47, 47, 310, 47, 47, 313, 314, 315, 316, - /* 1480 */ 317, 318, 271, 320, 22, 0, 243, 22, 0, 47, - /* 1490 */ 279, 33, 0, 22, 22, 20, 0, 47, 0, 22, - /* 1500 */ 289, 0, 0, 45, 293, 0, 0, 296, 145, 51, - /* 1510 */ 52, 53, 54, 55, 271, 81, 243, 37, 145, 41, - /* 1520 */ 41, 310, 279, 140, 313, 314, 315, 316, 317, 318, - /* 1530 */ 215, 320, 289, 161, 82, 41, 293, 82, 80, 142, - /* 1540 */ 81, 83, 81, 44, 271, 81, 145, 44, 41, 82, - /* 1550 */ 81, 41, 279, 310, 41, 82, 313, 314, 315, 316, - /* 1560 */ 317, 318, 289, 320, 44, 82, 293, 243, 44, 82, - /* 1570 */ 41, 82, 47, 47, 47, 47, 2, 47, 47, 41, - /* 1580 */ 181, 44, 82, 310, 44, 81, 313, 314, 315, 316, - /* 1590 */ 317, 318, 81, 320, 82, 271, 81, 22, 140, 215, - /* 1600 */ 142, 183, 144, 279, 146, 209, 215, 243, 0, 37, - /* 1610 */ 91, 143, 44, 289, 82, 81, 81, 293, 82, 22, - /* 1620 */ 81, 140, 164, 81, 81, 44, 81, 47, 82, 81, - /* 1630 */ 81, 113, 82, 92, 310, 271, 47, 313, 314, 315, - /* 1640 */ 316, 317, 318, 279, 320, 81, 47, 243, 82, 82, - /* 1650 */ 81, 47, 81, 289, 47, 82, 47, 293, 81, 105, - /* 1660 */ 82, 105, 243, 81, 105, 105, 81, 22, 81, 93, - /* 1670 */ 47, 81, 22, 58, 310, 271, 59, 313, 314, 315, - /* 1680 */ 316, 317, 318, 279, 320, 47, 64, 41, 47, 79, - /* 1690 */ 271, 22, 64, 289, 47, 47, 47, 293, 279, 47, - /* 1700 */ 47, 47, 47, 47, 47, 47, 47, 47, 289, 47, - /* 1710 */ 0, 47, 293, 243, 310, 45, 37, 313, 314, 315, - /* 1720 */ 316, 317, 318, 0, 320, 243, 45, 47, 37, 310, - /* 1730 */ 0, 47, 313, 314, 315, 316, 317, 318, 45, 320, - /* 1740 */ 37, 271, 47, 45, 37, 0, 47, 46, 0, 279, - /* 1750 */ 0, 0, 21, 271, 22, 22, 22, 21, 20, 289, - /* 1760 */ 360, 279, 360, 293, 360, 243, 360, 360, 360, 360, - /* 1770 */ 360, 289, 360, 360, 360, 293, 243, 360, 360, 360, - /* 1780 */ 310, 360, 360, 313, 314, 315, 316, 317, 318, 360, - /* 1790 */ 320, 360, 310, 271, 360, 313, 314, 315, 316, 317, - /* 1800 */ 318, 279, 320, 360, 271, 360, 360, 360, 360, 360, - /* 1810 */ 360, 289, 279, 360, 360, 293, 360, 360, 360, 360, - /* 1820 */ 360, 360, 289, 360, 360, 360, 293, 243, 360, 360, - /* 1830 */ 360, 360, 310, 360, 360, 313, 314, 315, 316, 317, - /* 1840 */ 318, 360, 320, 310, 360, 360, 313, 314, 315, 316, - /* 1850 */ 317, 318, 360, 320, 360, 271, 360, 243, 360, 360, - /* 1860 */ 360, 360, 360, 279, 360, 360, 360, 360, 360, 360, - /* 1870 */ 360, 360, 360, 289, 360, 360, 360, 293, 360, 360, - /* 1880 */ 360, 360, 360, 360, 360, 271, 360, 243, 360, 360, - /* 1890 */ 360, 360, 360, 279, 310, 360, 360, 313, 314, 315, - /* 1900 */ 316, 317, 318, 289, 320, 360, 360, 293, 360, 360, - /* 1910 */ 360, 360, 360, 360, 360, 271, 360, 360, 360, 243, - /* 1920 */ 360, 360, 360, 279, 310, 360, 360, 313, 314, 315, - /* 1930 */ 316, 317, 318, 289, 320, 360, 360, 293, 360, 360, - /* 1940 */ 360, 360, 360, 12, 13, 360, 360, 271, 360, 360, - /* 1950 */ 360, 360, 243, 22, 310, 279, 360, 313, 314, 315, - /* 1960 */ 316, 317, 318, 360, 320, 289, 360, 360, 360, 293, - /* 1970 */ 360, 360, 360, 360, 360, 360, 360, 360, 47, 360, - /* 1980 */ 271, 360, 360, 360, 360, 243, 310, 360, 279, 313, - /* 1990 */ 314, 315, 316, 317, 318, 64, 320, 360, 289, 360, - /* 2000 */ 360, 360, 293, 360, 360, 360, 360, 360, 360, 360, - /* 2010 */ 360, 360, 360, 271, 360, 360, 250, 360, 360, 310, - /* 2020 */ 360, 279, 313, 314, 315, 316, 317, 318, 360, 320, - /* 2030 */ 360, 289, 360, 360, 103, 293, 360, 360, 360, 360, - /* 2040 */ 360, 360, 360, 360, 113, 279, 250, 360, 360, 360, - /* 2050 */ 360, 360, 310, 360, 360, 313, 314, 315, 316, 317, - /* 2060 */ 318, 360, 320, 360, 298, 360, 360, 360, 360, 360, - /* 2070 */ 360, 360, 360, 360, 360, 279, 360, 360, 360, 360, - /* 2080 */ 360, 360, 360, 317, 360, 360, 360, 156, 360, 360, - /* 2090 */ 360, 360, 360, 360, 298, 360, 360, 360, 332, 333, - /* 2100 */ 334, 360, 336, 360, 360, 339, 360, 360, 360, 360, - /* 2110 */ 360, 360, 181, 317, 360, 360, 360, 351, 360, 360, - /* 2120 */ 360, 355, 360, 192, 193, 194, 360, 360, 332, 333, - /* 2130 */ 334, 360, 336, 360, 360, 339, 360, 360, 360, 360, - /* 2140 */ 360, 360, 360, 360, 360, 360, 360, 351, 360, 360, - /* 2150 */ 360, 355, 360, 360, 360, 360, 360, 360, 360, 360, - /* 2160 */ 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, - /* 2170 */ 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, - /* 2180 */ 360, 360, 360, 360, + /* 0 */ 269, 280, 242, 271, 283, 284, 274, 245, 271, 247, + /* 10 */ 248, 274, 12, 13, 282, 245, 285, 247, 248, 282, + /* 20 */ 20, 0, 22, 269, 12, 13, 14, 15, 16, 269, + /* 30 */ 276, 20, 12, 13, 14, 15, 16, 277, 338, 285, + /* 40 */ 308, 309, 282, 338, 284, 308, 309, 47, 323, 324, + /* 50 */ 350, 319, 249, 239, 354, 350, 319, 297, 58, 354, + /* 60 */ 12, 13, 14, 246, 64, 242, 249, 20, 20, 309, + /* 70 */ 22, 55, 312, 313, 314, 315, 316, 317, 57, 319, + /* 80 */ 4, 81, 322, 20, 14, 338, 326, 327, 249, 286, + /* 90 */ 20, 249, 269, 249, 82, 47, 80, 350, 338, 83, + /* 100 */ 277, 354, 260, 103, 260, 282, 58, 284, 4, 267, + /* 110 */ 350, 297, 64, 2, 354, 57, 277, 117, 118, 277, + /* 120 */ 297, 277, 297, 12, 13, 14, 15, 16, 81, 81, + /* 130 */ 254, 249, 309, 269, 258, 312, 313, 314, 315, 316, + /* 140 */ 317, 277, 319, 284, 81, 322, 42, 43, 14, 326, + /* 150 */ 327, 103, 338, 294, 20, 316, 113, 81, 158, 277, + /* 160 */ 160, 338, 20, 338, 350, 117, 118, 147, 354, 330, + /* 170 */ 331, 332, 333, 350, 335, 350, 57, 354, 314, 354, + /* 180 */ 41, 145, 182, 183, 153, 185, 186, 187, 188, 189, + /* 190 */ 190, 191, 192, 193, 194, 195, 196, 197, 316, 12, + /* 200 */ 13, 14, 15, 16, 173, 174, 158, 241, 160, 243, + /* 210 */ 210, 168, 169, 331, 332, 333, 20, 335, 284, 12, + /* 220 */ 13, 12, 13, 14, 15, 16, 292, 20, 294, 22, + /* 230 */ 182, 183, 0, 185, 186, 187, 188, 189, 190, 191, + /* 240 */ 192, 193, 194, 195, 196, 197, 12, 13, 14, 15, + /* 250 */ 16, 215, 216, 21, 47, 251, 24, 25, 26, 27, + /* 260 */ 28, 29, 30, 31, 32, 58, 20, 12, 13, 82, + /* 270 */ 2, 64, 0, 210, 271, 20, 272, 22, 47, 81, + /* 280 */ 12, 13, 14, 15, 16, 282, 210, 246, 81, 182, + /* 290 */ 249, 94, 95, 96, 97, 98, 99, 100, 101, 102, + /* 300 */ 103, 104, 47, 106, 107, 108, 109, 110, 111, 14, + /* 310 */ 103, 308, 309, 117, 118, 20, 14, 15, 16, 64, + /* 320 */ 186, 41, 319, 242, 117, 118, 219, 220, 221, 222, + /* 330 */ 223, 20, 60, 61, 62, 63, 81, 65, 66, 67, + /* 340 */ 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, + /* 350 */ 78, 0, 257, 261, 259, 157, 81, 159, 103, 150, + /* 360 */ 20, 269, 82, 282, 93, 158, 242, 160, 229, 242, + /* 370 */ 278, 21, 117, 118, 24, 25, 26, 27, 28, 29, + /* 380 */ 30, 31, 32, 112, 113, 114, 115, 116, 253, 182, + /* 390 */ 183, 160, 185, 186, 187, 188, 189, 190, 191, 192, + /* 400 */ 193, 194, 195, 196, 197, 249, 282, 249, 210, 282, + /* 410 */ 275, 60, 61, 158, 269, 160, 65, 242, 260, 68, + /* 420 */ 69, 276, 242, 72, 73, 74, 12, 13, 14, 279, + /* 430 */ 285, 242, 282, 277, 20, 277, 22, 182, 183, 230, + /* 440 */ 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, + /* 450 */ 195, 196, 197, 242, 22, 60, 61, 282, 269, 242, + /* 460 */ 65, 47, 282, 68, 69, 210, 277, 72, 73, 74, + /* 470 */ 112, 282, 316, 284, 12, 13, 283, 284, 64, 47, + /* 480 */ 64, 186, 20, 242, 22, 210, 249, 331, 332, 333, + /* 490 */ 35, 335, 20, 282, 22, 81, 64, 260, 309, 282, + /* 500 */ 253, 312, 313, 314, 315, 316, 317, 242, 319, 47, + /* 510 */ 269, 322, 55, 266, 277, 326, 327, 103, 277, 47, + /* 520 */ 297, 49, 275, 282, 279, 284, 64, 282, 170, 171, + /* 530 */ 85, 117, 118, 88, 249, 103, 12, 13, 81, 84, + /* 540 */ 83, 86, 87, 81, 89, 260, 22, 282, 93, 0, + /* 550 */ 309, 4, 267, 312, 313, 314, 315, 316, 317, 47, + /* 560 */ 319, 338, 277, 322, 20, 103, 22, 326, 327, 328, + /* 570 */ 115, 47, 158, 350, 160, 297, 64, 354, 269, 117, + /* 580 */ 118, 340, 255, 256, 249, 276, 345, 346, 64, 75, + /* 590 */ 158, 37, 160, 49, 285, 260, 182, 183, 20, 185, + /* 600 */ 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, + /* 610 */ 196, 197, 277, 261, 182, 183, 338, 141, 42, 43, + /* 620 */ 158, 269, 160, 261, 268, 251, 242, 103, 350, 242, + /* 630 */ 278, 269, 354, 12, 13, 121, 122, 281, 264, 163, + /* 640 */ 278, 20, 93, 22, 182, 183, 272, 185, 186, 187, + /* 650 */ 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, + /* 660 */ 242, 112, 113, 114, 115, 116, 282, 210, 47, 282, + /* 670 */ 249, 242, 1, 2, 198, 271, 242, 249, 18, 3, + /* 680 */ 20, 260, 158, 310, 160, 64, 282, 27, 260, 242, + /* 690 */ 30, 12, 13, 14, 15, 16, 255, 256, 277, 249, + /* 700 */ 282, 242, 81, 20, 18, 277, 182, 334, 48, 23, + /* 710 */ 260, 282, 308, 309, 47, 242, 282, 193, 194, 195, + /* 720 */ 0, 35, 36, 319, 103, 39, 242, 277, 269, 282, + /* 730 */ 242, 262, 310, 310, 265, 41, 277, 58, 117, 118, + /* 740 */ 0, 282, 56, 284, 24, 25, 26, 27, 28, 29, + /* 750 */ 30, 31, 32, 82, 295, 282, 334, 334, 211, 208, + /* 760 */ 209, 82, 242, 209, 186, 92, 282, 81, 309, 90, + /* 770 */ 282, 312, 313, 314, 315, 316, 317, 64, 319, 158, + /* 780 */ 120, 160, 0, 123, 124, 125, 126, 127, 128, 129, + /* 790 */ 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, + /* 800 */ 140, 20, 282, 182, 183, 119, 185, 186, 187, 188, + /* 810 */ 189, 190, 191, 192, 193, 194, 195, 196, 197, 269, + /* 820 */ 141, 249, 12, 13, 14, 15, 16, 160, 278, 249, + /* 830 */ 147, 0, 260, 93, 249, 242, 150, 151, 152, 57, + /* 840 */ 260, 155, 163, 85, 58, 260, 88, 161, 85, 277, + /* 850 */ 93, 88, 112, 113, 114, 115, 116, 277, 172, 249, + /* 860 */ 249, 175, 277, 177, 178, 179, 180, 181, 58, 270, + /* 870 */ 260, 260, 115, 0, 270, 282, 45, 198, 199, 200, + /* 880 */ 201, 202, 203, 204, 205, 206, 207, 277, 277, 0, + /* 890 */ 242, 277, 249, 0, 85, 22, 210, 88, 44, 21, + /* 900 */ 90, 225, 288, 260, 41, 19, 41, 47, 41, 117, + /* 910 */ 118, 22, 34, 1, 2, 22, 81, 269, 41, 33, + /* 920 */ 277, 227, 270, 270, 64, 277, 91, 270, 147, 270, + /* 930 */ 282, 45, 284, 270, 33, 81, 41, 51, 52, 53, + /* 940 */ 54, 55, 301, 243, 41, 82, 45, 82, 41, 82, + /* 950 */ 41, 141, 51, 52, 53, 54, 55, 309, 357, 82, + /* 960 */ 312, 313, 314, 315, 316, 317, 80, 319, 182, 83, + /* 970 */ 322, 348, 258, 163, 326, 327, 328, 82, 41, 41, + /* 980 */ 41, 80, 41, 41, 83, 82, 41, 41, 0, 82, + /* 990 */ 305, 82, 41, 250, 346, 269, 342, 250, 248, 281, + /* 1000 */ 311, 336, 116, 339, 242, 212, 351, 351, 198, 199, + /* 1010 */ 200, 201, 202, 203, 204, 205, 206, 207, 307, 82, + /* 1020 */ 82, 82, 20, 82, 82, 351, 249, 82, 82, 143, + /* 1030 */ 242, 269, 146, 82, 46, 45, 306, 156, 47, 277, + /* 1040 */ 255, 299, 289, 142, 282, 144, 284, 146, 249, 148, + /* 1050 */ 249, 165, 40, 167, 141, 287, 242, 269, 287, 20, + /* 1060 */ 249, 244, 244, 20, 303, 277, 253, 284, 167, 253, + /* 1070 */ 282, 309, 284, 20, 312, 313, 314, 315, 316, 317, + /* 1080 */ 296, 319, 20, 269, 322, 298, 253, 296, 326, 327, + /* 1090 */ 328, 277, 253, 20, 290, 253, 282, 309, 284, 337, + /* 1100 */ 312, 313, 314, 315, 316, 317, 277, 319, 242, 253, + /* 1110 */ 322, 249, 253, 244, 326, 327, 328, 269, 269, 269, + /* 1120 */ 269, 269, 242, 309, 269, 337, 312, 313, 314, 315, + /* 1130 */ 316, 317, 244, 319, 249, 269, 322, 269, 251, 269, + /* 1140 */ 326, 327, 328, 277, 269, 269, 282, 303, 282, 269, + /* 1150 */ 284, 337, 269, 282, 249, 166, 284, 277, 20, 302, + /* 1160 */ 217, 251, 282, 297, 284, 251, 251, 277, 218, 290, + /* 1170 */ 296, 347, 311, 242, 282, 309, 282, 297, 312, 313, + /* 1180 */ 314, 315, 316, 317, 293, 319, 282, 347, 224, 309, + /* 1190 */ 4, 293, 312, 313, 314, 315, 316, 317, 149, 319, + /* 1200 */ 269, 209, 344, 307, 338, 19, 213, 277, 277, 20, + /* 1210 */ 310, 40, 228, 282, 231, 284, 350, 81, 338, 33, + /* 1220 */ 354, 293, 226, 144, 291, 343, 282, 293, 329, 282, + /* 1230 */ 350, 45, 277, 251, 354, 282, 50, 242, 341, 290, + /* 1240 */ 309, 55, 358, 312, 313, 314, 315, 316, 317, 325, + /* 1250 */ 319, 242, 265, 322, 251, 353, 277, 326, 327, 353, + /* 1260 */ 242, 353, 81, 352, 269, 259, 80, 273, 352, 83, + /* 1270 */ 352, 249, 277, 244, 251, 300, 304, 282, 269, 284, + /* 1280 */ 282, 240, 263, 263, 263, 252, 277, 269, 0, 0, + /* 1290 */ 40, 282, 0, 284, 72, 277, 0, 176, 47, 47, + /* 1300 */ 282, 47, 284, 47, 309, 176, 0, 312, 313, 314, + /* 1310 */ 315, 316, 317, 318, 319, 320, 321, 242, 309, 47, + /* 1320 */ 47, 312, 313, 314, 315, 316, 317, 309, 319, 176, + /* 1330 */ 312, 313, 314, 315, 316, 317, 0, 319, 176, 242, + /* 1340 */ 322, 0, 47, 0, 269, 327, 22, 0, 47, 0, + /* 1350 */ 81, 163, 277, 162, 160, 158, 0, 282, 0, 284, + /* 1360 */ 154, 153, 0, 0, 355, 356, 269, 44, 0, 0, + /* 1370 */ 0, 0, 0, 0, 277, 0, 0, 0, 0, 282, + /* 1380 */ 0, 284, 0, 0, 309, 0, 0, 312, 313, 314, + /* 1390 */ 315, 316, 317, 242, 319, 0, 0, 0, 0, 40, + /* 1400 */ 0, 0, 0, 0, 0, 242, 309, 0, 0, 312, + /* 1410 */ 313, 314, 315, 316, 317, 22, 319, 0, 0, 0, + /* 1420 */ 269, 0, 41, 14, 349, 37, 14, 40, 277, 0, + /* 1430 */ 0, 0, 269, 282, 44, 284, 44, 38, 0, 149, + /* 1440 */ 277, 0, 37, 0, 0, 282, 295, 284, 37, 0, + /* 1450 */ 37, 0, 37, 356, 0, 0, 242, 37, 0, 37, + /* 1460 */ 309, 47, 37, 312, 313, 314, 315, 316, 317, 59, + /* 1470 */ 319, 45, 309, 45, 47, 312, 313, 314, 315, 316, + /* 1480 */ 317, 47, 319, 269, 321, 45, 47, 45, 0, 0, + /* 1490 */ 0, 277, 0, 47, 22, 0, 282, 47, 284, 47, + /* 1500 */ 47, 41, 41, 47, 47, 0, 90, 242, 22, 295, + /* 1510 */ 47, 88, 47, 0, 48, 22, 0, 47, 22, 0, + /* 1520 */ 22, 242, 0, 309, 22, 0, 312, 313, 314, 315, + /* 1530 */ 316, 317, 20, 319, 269, 0, 147, 22, 47, 0, + /* 1540 */ 0, 0, 277, 164, 0, 81, 37, 282, 269, 284, + /* 1550 */ 82, 41, 41, 214, 81, 41, 277, 82, 81, 81, + /* 1560 */ 295, 282, 82, 284, 41, 44, 81, 41, 44, 214, + /* 1570 */ 242, 147, 44, 208, 309, 142, 144, 312, 313, 314, + /* 1580 */ 315, 316, 317, 147, 319, 82, 82, 242, 309, 41, + /* 1590 */ 44, 312, 313, 314, 315, 316, 317, 269, 319, 82, + /* 1600 */ 41, 47, 82, 47, 2, 277, 41, 47, 47, 47, + /* 1610 */ 282, 47, 284, 82, 269, 81, 44, 44, 82, 182, + /* 1620 */ 82, 22, 277, 81, 81, 0, 81, 282, 81, 284, + /* 1630 */ 37, 82, 184, 81, 81, 44, 242, 309, 44, 145, + /* 1640 */ 312, 313, 314, 315, 316, 317, 91, 319, 242, 214, + /* 1650 */ 81, 81, 22, 82, 309, 81, 81, 312, 313, 314, + /* 1660 */ 315, 316, 317, 269, 319, 82, 47, 47, 92, 142, + /* 1670 */ 81, 277, 82, 47, 82, 269, 282, 81, 284, 47, + /* 1680 */ 47, 81, 81, 277, 47, 82, 82, 81, 282, 242, + /* 1690 */ 284, 105, 105, 22, 81, 81, 47, 81, 93, 41, + /* 1700 */ 105, 22, 105, 309, 59, 58, 312, 313, 314, 315, + /* 1710 */ 316, 317, 47, 319, 64, 309, 269, 79, 312, 313, + /* 1720 */ 314, 315, 316, 317, 277, 319, 41, 47, 47, 282, + /* 1730 */ 47, 284, 22, 47, 47, 47, 47, 64, 47, 47, + /* 1740 */ 47, 242, 47, 47, 47, 47, 0, 47, 45, 37, + /* 1750 */ 242, 0, 47, 45, 0, 37, 309, 0, 47, 312, + /* 1760 */ 313, 314, 315, 316, 317, 45, 319, 37, 269, 47, + /* 1770 */ 45, 37, 0, 47, 0, 22, 277, 269, 22, 21, + /* 1780 */ 21, 282, 22, 284, 20, 277, 359, 359, 359, 359, + /* 1790 */ 282, 359, 284, 359, 359, 359, 359, 359, 359, 242, + /* 1800 */ 359, 359, 359, 359, 359, 359, 359, 359, 309, 359, + /* 1810 */ 359, 312, 313, 314, 315, 316, 317, 309, 319, 359, + /* 1820 */ 312, 313, 314, 315, 316, 317, 269, 319, 359, 359, + /* 1830 */ 359, 359, 359, 359, 277, 359, 359, 249, 359, 282, + /* 1840 */ 242, 284, 359, 359, 359, 359, 359, 359, 359, 359, + /* 1850 */ 359, 359, 359, 359, 359, 359, 359, 242, 359, 359, + /* 1860 */ 359, 359, 359, 359, 359, 277, 309, 269, 359, 312, + /* 1870 */ 313, 314, 315, 316, 317, 277, 319, 359, 359, 359, + /* 1880 */ 282, 359, 284, 359, 269, 297, 359, 359, 359, 359, + /* 1890 */ 359, 359, 277, 359, 359, 359, 359, 282, 359, 284, + /* 1900 */ 359, 359, 359, 359, 316, 359, 242, 309, 359, 359, + /* 1910 */ 312, 313, 314, 315, 316, 317, 359, 319, 242, 331, + /* 1920 */ 332, 333, 359, 335, 309, 359, 338, 312, 313, 314, + /* 1930 */ 315, 316, 317, 269, 319, 359, 359, 359, 350, 249, + /* 1940 */ 359, 277, 354, 359, 359, 269, 282, 359, 284, 359, + /* 1950 */ 359, 359, 359, 277, 359, 359, 359, 359, 282, 242, + /* 1960 */ 284, 359, 359, 359, 359, 359, 359, 277, 359, 359, + /* 1970 */ 359, 359, 359, 309, 359, 359, 312, 313, 314, 315, + /* 1980 */ 316, 317, 359, 319, 359, 309, 269, 297, 312, 313, + /* 1990 */ 314, 315, 316, 317, 277, 319, 359, 359, 359, 282, + /* 2000 */ 359, 284, 359, 359, 359, 359, 316, 359, 359, 359, + /* 2010 */ 359, 242, 359, 359, 359, 359, 359, 359, 359, 359, + /* 2020 */ 242, 331, 332, 333, 359, 335, 309, 359, 338, 312, + /* 2030 */ 313, 314, 315, 316, 317, 359, 319, 359, 269, 359, + /* 2040 */ 350, 359, 359, 359, 354, 359, 277, 269, 359, 359, + /* 2050 */ 359, 282, 359, 284, 359, 277, 359, 359, 359, 359, + /* 2060 */ 282, 359, 284, 359, 359, 359, 359, 359, 359, 359, + /* 2070 */ 359, 359, 359, 359, 359, 359, 359, 359, 309, 359, + /* 2080 */ 359, 312, 313, 314, 315, 316, 317, 309, 319, 359, + /* 2090 */ 312, 313, 314, 315, 316, 317, 359, 319, }; -#define YY_SHIFT_COUNT (618) +#define YY_SHIFT_COUNT (621) #define YY_SHIFT_MIN (0) -#define YY_SHIFT_MAX (1931) +#define YY_SHIFT_MAX (1774) static const unsigned short int yy_shift_ofst[] = { - /* 0 */ 688, 0, 0, 48, 96, 96, 96, 96, 253, 253, - /* 10 */ 96, 96, 301, 349, 506, 349, 349, 349, 349, 349, - /* 20 */ 349, 349, 349, 349, 349, 349, 349, 349, 349, 349, - /* 30 */ 349, 349, 349, 349, 349, 349, 349, 349, 144, 144, - /* 40 */ 102, 102, 102, 1931, 1931, 1931, 1931, 372, 124, 214, - /* 50 */ 58, 58, 260, 260, 172, 214, 214, 58, 58, 58, - /* 60 */ 58, 58, 58, 58, 37, 58, 58, 186, 228, 259, - /* 70 */ 186, 58, 58, 186, 58, 186, 186, 259, 186, 58, - /* 80 */ 255, 542, 724, 758, 758, 189, 236, 838, 838, 838, - /* 90 */ 838, 838, 838, 838, 838, 838, 838, 838, 838, 838, - /* 100 */ 838, 838, 838, 838, 838, 838, 291, 126, 354, 354, - /* 110 */ 397, 516, 483, 483, 483, 525, 516, 313, 259, 186, - /* 120 */ 186, 259, 377, 343, 243, 243, 243, 243, 243, 243, - /* 130 */ 243, 340, 141, 405, 76, 195, 57, 215, 500, 545, - /* 140 */ 579, 179, 694, 700, 617, 678, 617, 455, 455, 455, - /* 150 */ 657, 666, 799, 1002, 979, 980, 876, 1002, 1002, 997, - /* 160 */ 910, 910, 1002, 1037, 1037, 1042, 37, 259, 37, 1056, - /* 170 */ 1058, 37, 1056, 37, 313, 1070, 37, 37, 1002, 37, - /* 180 */ 1037, 186, 186, 186, 186, 186, 186, 186, 186, 186, - /* 190 */ 186, 186, 1002, 1037, 1045, 1045, 1042, 255, 956, 259, - /* 200 */ 255, 1002, 1056, 255, 313, 1070, 255, 1106, 922, 933, - /* 210 */ 1045, 922, 933, 1045, 1045, 186, 936, 1009, 960, 799, - /* 220 */ 966, 313, 1158, 1140, 954, 957, 949, 954, 957, 954, - /* 230 */ 957, 1104, 933, 1045, 1045, 933, 1045, 1054, 313, 1070, - /* 240 */ 255, 377, 255, 313, 1143, 343, 1002, 255, 1037, 2152, - /* 250 */ 2152, 2152, 2152, 2152, 2152, 2152, 577, 1458, 231, 353, - /* 260 */ 3, 19, 316, 256, 496, 719, 777, 112, 112, 112, - /* 270 */ 112, 112, 112, 112, 112, 135, 438, 344, 396, 355, - /* 280 */ 625, 566, 642, 642, 642, 642, 555, 808, 711, 813, - /* 290 */ 815, 819, 872, 887, 913, 809, 855, 867, 892, 914, - /* 300 */ 723, 710, 772, 902, 766, 903, 873, 905, 907, 926, - /* 310 */ 929, 931, 826, 875, 934, 952, 962, 965, 974, 976, - /* 320 */ 859, 935, 1260, 1261, 1222, 1263, 1193, 1266, 1220, 1093, - /* 330 */ 1223, 1224, 1227, 1094, 1275, 1237, 1238, 1113, 1289, 1115, - /* 340 */ 1291, 1245, 1293, 1272, 1295, 1250, 1298, 1218, 1141, 1145, - /* 350 */ 1187, 1146, 1305, 1312, 1162, 1166, 1318, 1322, 1281, 1326, - /* 360 */ 1327, 1328, 1329, 1330, 1331, 1334, 1335, 1336, 1338, 1339, - /* 370 */ 1340, 1341, 1343, 1344, 1345, 1347, 1348, 1309, 1351, 1352, - /* 380 */ 1353, 1355, 1356, 1365, 1350, 1374, 1375, 1376, 1377, 1379, - /* 390 */ 1380, 1342, 1346, 1354, 1367, 1349, 1370, 1357, 1385, 1359, - /* 400 */ 1361, 1387, 1388, 1389, 1362, 1244, 1392, 1402, 1368, 1407, - /* 410 */ 1360, 1408, 1409, 1366, 1369, 1378, 1411, 1381, 1371, 1393, - /* 420 */ 1412, 1382, 1386, 1395, 1433, 1390, 1394, 1398, 1438, 1441, - /* 430 */ 1443, 1444, 1314, 1358, 1400, 1423, 1448, 1403, 1406, 1421, - /* 440 */ 1424, 1410, 1414, 1425, 1427, 1428, 1449, 1436, 1466, 1462, - /* 450 */ 1422, 1485, 1465, 1442, 1488, 1471, 1492, 1472, 1475, 1496, - /* 460 */ 1363, 1450, 1498, 1372, 1477, 1373, 1397, 1501, 1502, 1505, - /* 470 */ 1401, 1506, 1434, 1480, 1383, 1478, 1479, 1315, 1452, 1494, - /* 480 */ 1455, 1459, 1461, 1464, 1467, 1507, 1499, 1503, 1469, 1510, - /* 490 */ 1384, 1473, 1483, 1520, 1396, 1513, 1524, 1487, 1529, 1391, - /* 500 */ 1489, 1525, 1526, 1527, 1528, 1530, 1531, 1489, 1574, 1399, - /* 510 */ 1538, 1500, 1504, 1512, 1537, 1511, 1515, 1540, 1575, 1418, - /* 520 */ 1534, 1532, 1536, 1535, 1539, 1468, 1542, 1608, 1572, 1481, - /* 530 */ 1543, 1519, 1568, 1581, 1545, 1546, 1548, 1597, 1549, 1541, - /* 540 */ 1550, 1580, 1589, 1564, 1566, 1599, 1569, 1567, 1604, 1571, - /* 550 */ 1573, 1607, 1577, 1578, 1609, 1582, 1554, 1556, 1559, 1560, - /* 560 */ 1645, 1576, 1585, 1587, 1623, 1590, 1518, 1650, 1617, 1615, - /* 570 */ 1638, 1622, 1610, 1646, 1641, 1647, 1648, 1649, 1652, 1669, - /* 580 */ 1653, 1654, 1628, 1410, 1655, 1414, 1656, 1657, 1658, 1659, - /* 590 */ 1660, 1662, 1710, 1664, 1670, 1679, 1723, 1680, 1681, 1691, - /* 600 */ 1730, 1684, 1693, 1703, 1750, 1695, 1698, 1707, 1745, 1699, - /* 610 */ 1701, 1748, 1751, 1732, 1731, 1733, 1734, 1736, 1738, + /* 0 */ 686, 0, 0, 48, 207, 207, 207, 207, 255, 255, + /* 10 */ 207, 207, 414, 462, 621, 462, 462, 462, 462, 462, + /* 20 */ 462, 462, 462, 462, 462, 462, 462, 462, 462, 462, + /* 30 */ 462, 462, 462, 462, 462, 462, 462, 462, 63, 63, + /* 40 */ 47, 47, 47, 524, 524, 524, 524, 198, 457, 275, + /* 50 */ 11, 11, 104, 104, 76, 196, 275, 275, 11, 11, + /* 60 */ 11, 11, 11, 11, 11, 58, 11, 11, 142, 246, + /* 70 */ 311, 142, 11, 11, 142, 11, 142, 142, 142, 11, + /* 80 */ 119, 660, 679, 810, 810, 350, 395, 432, 432, 432, + /* 90 */ 432, 432, 432, 432, 432, 432, 432, 432, 432, 432, + /* 100 */ 432, 432, 432, 432, 432, 432, 455, 196, 70, 70, + /* 110 */ 21, 512, 683, 683, 683, 782, 512, 340, 311, 142, + /* 120 */ 142, 416, 416, 673, 713, 197, 197, 197, 197, 197, + /* 130 */ 197, 197, 886, 232, 351, 209, 107, 472, 43, 36, + /* 140 */ 134, 295, 544, 576, 757, 781, 551, 554, 551, 676, + /* 150 */ 676, 676, 547, 578, 793, 1002, 990, 991, 881, 1002, + /* 160 */ 1002, 1012, 913, 913, 1002, 1039, 1039, 1043, 58, 311, + /* 170 */ 58, 1053, 1062, 58, 1053, 58, 340, 1073, 58, 58, + /* 180 */ 1002, 58, 1039, 142, 142, 142, 142, 142, 142, 142, + /* 190 */ 142, 142, 142, 142, 1002, 1039, 416, 416, 1043, 119, + /* 200 */ 989, 311, 119, 1002, 1053, 119, 340, 1073, 119, 1138, + /* 210 */ 950, 943, 416, 950, 943, 416, 416, 142, 964, 1049, + /* 220 */ 993, 793, 992, 340, 1189, 1171, 984, 996, 983, 984, + /* 230 */ 996, 984, 996, 1136, 943, 416, 416, 943, 416, 1079, + /* 240 */ 340, 1073, 119, 673, 119, 340, 1181, 416, 713, 1002, + /* 250 */ 119, 1039, 2098, 2098, 2098, 2098, 2098, 2098, 2098, 272, + /* 260 */ 901, 720, 1186, 549, 740, 12, 111, 268, 20, 187, + /* 270 */ 271, 234, 234, 234, 234, 234, 234, 234, 234, 31, + /* 280 */ 16, 514, 358, 671, 476, 302, 302, 302, 302, 831, + /* 290 */ 280, 445, 758, 763, 809, 873, 889, 893, 878, 863, + /* 300 */ 865, 867, 912, 792, 694, 139, 877, 786, 895, 854, + /* 310 */ 903, 907, 909, 937, 938, 231, 667, 939, 941, 942, + /* 320 */ 945, 946, 951, 835, 860, 988, 1288, 1289, 1250, 1292, + /* 330 */ 1222, 1296, 1251, 1121, 1252, 1254, 1256, 1129, 1306, 1272, + /* 340 */ 1273, 1153, 1336, 1162, 1341, 1295, 1343, 1324, 1347, 1301, + /* 350 */ 1349, 1269, 1188, 1191, 1194, 1197, 1356, 1358, 1206, 1208, + /* 360 */ 1362, 1363, 1323, 1368, 1369, 1370, 1371, 1372, 1373, 1375, + /* 370 */ 1376, 1377, 1378, 1380, 1382, 1383, 1385, 1386, 1395, 1396, + /* 380 */ 1397, 1359, 1398, 1400, 1401, 1402, 1403, 1404, 1393, 1407, + /* 390 */ 1408, 1417, 1418, 1419, 1421, 1387, 1388, 1381, 1409, 1390, + /* 400 */ 1412, 1392, 1429, 1399, 1405, 1430, 1431, 1438, 1411, 1290, + /* 410 */ 1441, 1443, 1413, 1444, 1410, 1449, 1451, 1414, 1426, 1415, + /* 420 */ 1454, 1427, 1428, 1420, 1455, 1434, 1440, 1422, 1458, 1439, + /* 430 */ 1442, 1425, 1488, 1489, 1490, 1492, 1416, 1423, 1446, 1472, + /* 440 */ 1495, 1450, 1452, 1453, 1456, 1460, 1461, 1457, 1463, 1465, + /* 450 */ 1505, 1486, 1513, 1493, 1466, 1516, 1496, 1470, 1519, 1498, + /* 460 */ 1522, 1502, 1512, 1525, 1389, 1491, 1535, 1379, 1515, 1424, + /* 470 */ 1432, 1539, 1540, 1541, 1436, 1544, 1464, 1509, 1433, 1510, + /* 480 */ 1511, 1339, 1468, 1514, 1475, 1473, 1477, 1478, 1480, 1523, + /* 490 */ 1521, 1524, 1485, 1526, 1355, 1503, 1504, 1528, 1365, 1548, + /* 500 */ 1546, 1517, 1559, 1435, 1520, 1554, 1556, 1560, 1561, 1562, + /* 510 */ 1564, 1520, 1602, 1437, 1565, 1531, 1534, 1536, 1572, 1542, + /* 520 */ 1543, 1573, 1599, 1448, 1545, 1538, 1549, 1547, 1552, 1494, + /* 530 */ 1553, 1625, 1593, 1527, 1569, 1555, 1591, 1594, 1570, 1571, + /* 540 */ 1574, 1630, 1575, 1576, 1583, 1619, 1620, 1589, 1590, 1626, + /* 550 */ 1596, 1592, 1632, 1600, 1603, 1633, 1601, 1604, 1637, 1606, + /* 560 */ 1586, 1587, 1595, 1597, 1671, 1605, 1613, 1614, 1649, 1616, + /* 570 */ 1658, 1658, 1679, 1645, 1647, 1665, 1650, 1638, 1685, 1680, + /* 580 */ 1681, 1683, 1686, 1687, 1710, 1688, 1689, 1673, 1460, 1691, + /* 590 */ 1461, 1692, 1693, 1695, 1696, 1697, 1698, 1746, 1700, 1703, + /* 600 */ 1712, 1751, 1705, 1708, 1718, 1754, 1711, 1720, 1730, 1757, + /* 610 */ 1722, 1725, 1734, 1772, 1726, 1774, 1753, 1758, 1756, 1760, + /* 620 */ 1759, 1764, }; -#define YY_REDUCE_COUNT (255) -#define YY_REDUCE_MIN (-324) -#define YY_REDUCE_MAX (1796) +#define YY_REDUCE_COUNT (258) +#define YY_REDUCE_MIN (-300) +#define YY_REDUCE_MAX (1778) static const short yy_reduce_ofst[] = { - /* 0 */ 162, -241, 278, 439, 663, 725, 757, 817, 849, 900, - /* 10 */ -178, 930, 561, 963, 993, 505, 1044, 1053, 1107, 1147, - /* 20 */ 1163, 1211, 1243, 1273, 1324, 1364, 1404, 1419, 1470, 1482, - /* 30 */ 1522, 1533, 1584, 1614, 1644, 1676, 1709, 1742, 1766, 1796, - /* 40 */ 282, -213, 578, -270, -266, -264, -176, -284, 281, 424, - /* 50 */ 306, 362, -91, -46, -255, -192, 71, -226, -199, 150, - /* 60 */ 302, 324, 463, 467, -1, 471, 485, 28, 45, -190, - /* 70 */ 56, 528, 531, 197, 481, 192, 352, 100, 196, 538, - /* 80 */ -225, -242, -324, -324, -324, -64, -48, 77, 168, 257, - /* 90 */ 311, 315, 341, 416, 441, 465, 498, 502, 518, 546, - /* 100 */ 554, 559, 560, 565, 612, 643, 97, -247, 75, 140, - /* 110 */ -125, -232, -252, -204, -37, -100, 106, 142, 394, 512, - /* 120 */ 573, 218, 284, 453, -261, 294, 325, 330, 335, 444, - /* 130 */ 557, 271, 381, 582, 503, 572, 639, 610, 684, 684, - /* 140 */ 720, 744, 689, 685, 661, 661, 661, 649, 653, 655, - /* 150 */ 669, 684, 713, 773, 718, 769, 729, 781, 782, 746, - /* 160 */ 770, 774, 801, 816, 820, 760, 811, 779, 814, 784, - /* 170 */ 780, 827, 786, 832, 810, 803, 836, 839, 847, 843, - /* 180 */ 854, 829, 830, 831, 833, 834, 837, 840, 841, 842, - /* 190 */ 844, 845, 853, 862, 821, 824, 818, 865, 822, 835, - /* 200 */ 868, 879, 851, 870, 857, 860, 886, 846, 804, 863, - /* 210 */ 856, 805, 866, 861, 877, 684, 823, 828, 848, 869, - /* 220 */ 661, 896, 871, 864, 858, 850, 852, 874, 878, 881, - /* 230 */ 880, 882, 893, 898, 899, 901, 904, 895, 920, 912, - /* 240 */ 968, 938, 969, 928, 950, 971, 977, 973, 984, 937, - /* 250 */ 925, 972, 989, 994, 987, 1008, + /* 0 */ -186, -240, -177, 241, 648, 762, 788, 814, 866, 880, + /* 10 */ 189, 931, 995, 1009, 1018, 459, 1075, 1097, 1151, 1163, + /* 20 */ 1214, 1265, 1279, 1328, 1345, 1394, 1406, 1447, 1499, 1508, + /* 30 */ 1557, 1598, 1615, 1664, 1676, 1717, 1769, 1778, 1588, 1690, + /* 40 */ -161, -118, 156, -268, -263, 3, 404, -175, 223, 278, + /* 50 */ -158, 285, -238, -230, -300, -279, -295, -253, -156, 158, + /* 60 */ 237, 335, 421, 428, 450, 247, 572, 580, -246, -136, + /* 70 */ -66, 92, 585, 610, 145, 611, 352, 309, 362, 643, + /* 80 */ 374, -197, -275, -275, -275, -34, -124, 81, 124, 127, + /* 90 */ 175, 180, 211, 217, 265, 384, 387, 418, 429, 434, + /* 100 */ 447, 473, 484, 488, 520, 593, 356, 193, -183, 41, + /* 110 */ 135, 327, 373, 422, 423, 4, 441, 614, -141, 550, + /* 120 */ -269, 150, 245, 469, 95, 599, 604, 652, 653, 657, + /* 130 */ 659, 663, 641, 700, 714, 601, 623, 743, 685, 654, + /* 140 */ 726, 726, 747, 750, 718, 689, 665, 665, 665, 655, + /* 150 */ 656, 674, 664, 726, 711, 777, 730, 785, 742, 799, + /* 160 */ 801, 753, 768, 771, 811, 817, 818, 761, 813, 783, + /* 170 */ 816, 784, 787, 833, 791, 839, 829, 804, 842, 856, + /* 180 */ 862, 859, 869, 848, 849, 850, 851, 852, 855, 868, + /* 190 */ 870, 875, 876, 883, 885, 888, 864, 871, 844, 887, + /* 200 */ 857, 872, 910, 905, 874, 914, 890, 879, 915, 861, + /* 210 */ 824, 891, 892, 840, 898, 894, 904, 726, 858, 882, + /* 220 */ 897, 896, 665, 930, 900, 899, 902, 911, 884, 906, + /* 230 */ 916, 908, 918, 924, 928, 944, 947, 934, 953, 933, + /* 240 */ 955, 949, 982, 987, 1003, 979, 994, 998, 1006, 1022, + /* 250 */ 1023, 1029, 975, 972, 1019, 1020, 1021, 1033, 1041, }; static const YYACTIONTYPE yy_default[] = { - /* 0 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 10 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 20 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 30 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 40 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 50 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 60 */ 1360, 1360, 1360, 1360, 1429, 1360, 1360, 1360, 1360, 1360, - /* 70 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 80 */ 1427, 1567, 1360, 1734, 1360, 1360, 1360, 1360, 1360, 1360, - /* 90 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 100 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 110 */ 1429, 1360, 1745, 1745, 1745, 1427, 1360, 1360, 1360, 1360, - /* 120 */ 1360, 1360, 1523, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 130 */ 1360, 1603, 1360, 1360, 1811, 1360, 1609, 1769, 1360, 1360, - /* 140 */ 1360, 1360, 1476, 1761, 1737, 1751, 1738, 1796, 1796, 1796, - /* 150 */ 1754, 1360, 1765, 1360, 1360, 1360, 1595, 1360, 1360, 1572, - /* 160 */ 1569, 1569, 1360, 1360, 1360, 1360, 1429, 1360, 1429, 1360, - /* 170 */ 1360, 1429, 1360, 1429, 1360, 1360, 1429, 1429, 1360, 1429, - /* 180 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 190 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1427, 1605, 1360, - /* 200 */ 1427, 1360, 1360, 1427, 1360, 1360, 1427, 1360, 1776, 1774, - /* 210 */ 1360, 1776, 1774, 1360, 1360, 1360, 1788, 1784, 1767, 1765, - /* 220 */ 1751, 1360, 1360, 1360, 1802, 1798, 1814, 1802, 1798, 1802, - /* 230 */ 1798, 1360, 1774, 1360, 1360, 1774, 1360, 1580, 1360, 1360, - /* 240 */ 1427, 1360, 1427, 1360, 1492, 1360, 1360, 1427, 1360, 1597, - /* 250 */ 1611, 1526, 1526, 1526, 1430, 1365, 1360, 1360, 1360, 1360, - /* 260 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1678, 1787, 1786, - /* 270 */ 1710, 1709, 1708, 1706, 1677, 1488, 1360, 1360, 1360, 1360, - /* 280 */ 1360, 1360, 1671, 1672, 1670, 1669, 1360, 1360, 1360, 1360, - /* 290 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1735, - /* 300 */ 1360, 1799, 1803, 1360, 1360, 1360, 1654, 1360, 1360, 1360, - /* 310 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 320 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 330 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 340 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 350 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 360 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 370 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 380 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 390 */ 1360, 1360, 1360, 1394, 1360, 1360, 1360, 1360, 1360, 1360, - /* 400 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 410 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 420 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 430 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 440 */ 1360, 1457, 1456, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 450 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 460 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 470 */ 1360, 1360, 1360, 1360, 1360, 1758, 1768, 1360, 1360, 1360, - /* 480 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1654, 1360, 1785, - /* 490 */ 1360, 1744, 1740, 1360, 1360, 1736, 1360, 1360, 1797, 1360, - /* 500 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1730, 1360, - /* 510 */ 1703, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1665, - /* 520 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 530 */ 1360, 1360, 1653, 1360, 1694, 1360, 1360, 1360, 1360, 1360, - /* 540 */ 1360, 1360, 1360, 1520, 1360, 1360, 1360, 1360, 1360, 1360, - /* 550 */ 1360, 1360, 1360, 1360, 1360, 1360, 1505, 1503, 1502, 1501, - /* 560 */ 1360, 1498, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 570 */ 1360, 1360, 1360, 1449, 1360, 1360, 1360, 1360, 1360, 1360, - /* 580 */ 1360, 1360, 1360, 1440, 1360, 1439, 1360, 1360, 1360, 1360, - /* 590 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 600 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 610 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, + /* 0 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 10 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 20 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 30 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 40 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 50 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 60 */ 1371, 1371, 1371, 1371, 1371, 1440, 1371, 1371, 1371, 1371, + /* 70 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 80 */ 1438, 1586, 1371, 1750, 1371, 1371, 1371, 1371, 1371, 1371, + /* 90 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 100 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 110 */ 1440, 1371, 1761, 1761, 1761, 1438, 1371, 1371, 1371, 1371, + /* 120 */ 1371, 1371, 1371, 1534, 1371, 1371, 1371, 1371, 1371, 1371, + /* 130 */ 1371, 1371, 1619, 1371, 1371, 1827, 1371, 1371, 1625, 1785, + /* 140 */ 1371, 1371, 1371, 1371, 1487, 1777, 1753, 1767, 1754, 1812, + /* 150 */ 1812, 1812, 1770, 1371, 1781, 1371, 1371, 1371, 1611, 1371, + /* 160 */ 1371, 1591, 1588, 1588, 1371, 1371, 1371, 1371, 1440, 1371, + /* 170 */ 1440, 1371, 1371, 1440, 1371, 1440, 1371, 1371, 1440, 1440, + /* 180 */ 1371, 1440, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 190 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1438, + /* 200 */ 1621, 1371, 1438, 1371, 1371, 1438, 1371, 1371, 1438, 1371, + /* 210 */ 1792, 1790, 1371, 1792, 1790, 1371, 1371, 1371, 1804, 1800, + /* 220 */ 1783, 1781, 1767, 1371, 1371, 1371, 1818, 1814, 1830, 1818, + /* 230 */ 1814, 1818, 1814, 1371, 1790, 1371, 1371, 1790, 1371, 1596, + /* 240 */ 1371, 1371, 1438, 1371, 1438, 1371, 1503, 1371, 1371, 1371, + /* 250 */ 1438, 1371, 1613, 1627, 1537, 1537, 1537, 1441, 1376, 1371, + /* 260 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 270 */ 1499, 1694, 1803, 1802, 1726, 1725, 1724, 1722, 1693, 1371, + /* 280 */ 1371, 1371, 1371, 1371, 1371, 1687, 1688, 1686, 1685, 1371, + /* 290 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 300 */ 1371, 1371, 1751, 1371, 1815, 1819, 1371, 1371, 1371, 1670, + /* 310 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 320 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 330 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 340 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 350 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 360 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 370 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 380 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 390 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1405, 1371, 1371, + /* 400 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 410 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 420 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 430 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 440 */ 1371, 1371, 1371, 1371, 1371, 1468, 1467, 1371, 1371, 1371, + /* 450 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 460 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 470 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1774, + /* 480 */ 1784, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 490 */ 1371, 1670, 1371, 1801, 1371, 1760, 1756, 1371, 1371, 1752, + /* 500 */ 1371, 1371, 1813, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 510 */ 1371, 1371, 1746, 1371, 1719, 1371, 1371, 1371, 1371, 1371, + /* 520 */ 1371, 1371, 1371, 1681, 1371, 1371, 1371, 1371, 1371, 1371, + /* 530 */ 1371, 1371, 1371, 1371, 1371, 1371, 1669, 1371, 1710, 1371, + /* 540 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1531, 1371, 1371, + /* 550 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 560 */ 1516, 1514, 1513, 1512, 1371, 1509, 1371, 1371, 1371, 1371, + /* 570 */ 1540, 1539, 1371, 1371, 1371, 1371, 1371, 1371, 1460, 1371, + /* 580 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1451, 1371, + /* 590 */ 1450, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 600 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 610 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 620 */ 1371, 1371, }; /********** End of lemon-generated parsing tables *****************************/ @@ -947,11 +934,13 @@ static const YYCODETYPE yyFallback[] = { 0, /* BLOB => nothing */ 0, /* VARBINARY => nothing */ 0, /* DECIMAL => nothing */ - 0, /* FILE_FACTOR => nothing */ - 0, /* NK_FLOAT => nothing */ + 0, /* MAX_DELAY => nothing */ + 0, /* WATERMARK => nothing */ 0, /* ROLLUP => nothing */ 0, /* TTL => nothing */ 0, /* SMA => nothing */ + 0, /* FIRST => nothing */ + 0, /* LAST => nothing */ 0, /* SHOW => nothing */ 0, /* DATABASES => nothing */ 0, /* TABLES => nothing */ @@ -993,6 +982,7 @@ static const YYCODETYPE yyFallback[] = { 0, /* VERBOSE => nothing */ 0, /* NK_BOOL => nothing */ 0, /* RATIO => nothing */ + 0, /* NK_FLOAT => nothing */ 0, /* COMPACT => nothing */ 0, /* VNODES => nothing */ 0, /* IN => nothing */ @@ -1004,8 +994,6 @@ static const YYCODETYPE yyFallback[] = { 0, /* TRIGGER => nothing */ 0, /* AT_ONCE => nothing */ 0, /* WINDOW_CLOSE => nothing */ - 0, /* MAX_DELAY => nothing */ - 0, /* WATERMARK => nothing */ 0, /* KILL => nothing */ 0, /* CONNECTION => nothing */ 0, /* TRANSACTION => nothing */ @@ -1031,8 +1019,6 @@ static const YYCODETYPE yyFallback[] = { 0, /* TODAY => nothing */ 0, /* TIMEZONE => nothing */ 0, /* COUNT => nothing */ - 0, /* FIRST => nothing */ - 0, /* LAST => nothing */ 0, /* LAST_ROW => nothing */ 0, /* BETWEEN => nothing */ 0, /* IS => nothing */ @@ -1069,12 +1055,12 @@ static const YYCODETYPE yyFallback[] = { 0, /* ASC => nothing */ 0, /* NULLS => nothing */ 0, /* ID => nothing */ - 233, /* NK_BITNOT => ID */ - 233, /* INSERT => ID */ - 233, /* VALUES => ID */ - 233, /* IMPORT => ID */ - 233, /* NK_SEMI => ID */ - 233, /* FILE => ID */ + 232, /* NK_BITNOT => ID */ + 232, /* INSERT => ID */ + 232, /* VALUES => ID */ + 232, /* IMPORT => ID */ + 232, /* NK_SEMI => ID */ + 232, /* FILE => ID */ }; #endif /* YYFALLBACK */ @@ -1274,254 +1260,253 @@ static const char *const yyTokenName[] = { /* 109 */ "BLOB", /* 110 */ "VARBINARY", /* 111 */ "DECIMAL", - /* 112 */ "FILE_FACTOR", - /* 113 */ "NK_FLOAT", + /* 112 */ "MAX_DELAY", + /* 113 */ "WATERMARK", /* 114 */ "ROLLUP", /* 115 */ "TTL", /* 116 */ "SMA", - /* 117 */ "SHOW", - /* 118 */ "DATABASES", - /* 119 */ "TABLES", - /* 120 */ "STABLES", - /* 121 */ "MNODES", - /* 122 */ "MODULES", - /* 123 */ "QNODES", - /* 124 */ "FUNCTIONS", - /* 125 */ "INDEXES", - /* 126 */ "ACCOUNTS", - /* 127 */ "APPS", - /* 128 */ "CONNECTIONS", - /* 129 */ "LICENCE", - /* 130 */ "GRANTS", - /* 131 */ "QUERIES", - /* 132 */ "SCORES", - /* 133 */ "TOPICS", - /* 134 */ "VARIABLES", - /* 135 */ "BNODES", - /* 136 */ "SNODES", - /* 137 */ "CLUSTER", - /* 138 */ "TRANSACTIONS", - /* 139 */ "LIKE", - /* 140 */ "INDEX", - /* 141 */ "FULLTEXT", - /* 142 */ "FUNCTION", - /* 143 */ "INTERVAL", - /* 144 */ "TOPIC", - /* 145 */ "AS", - /* 146 */ "CONSUMER", - /* 147 */ "GROUP", - /* 148 */ "DESC", - /* 149 */ "DESCRIBE", - /* 150 */ "RESET", - /* 151 */ "QUERY", - /* 152 */ "CACHE", - /* 153 */ "EXPLAIN", - /* 154 */ "ANALYZE", - /* 155 */ "VERBOSE", - /* 156 */ "NK_BOOL", - /* 157 */ "RATIO", - /* 158 */ "COMPACT", - /* 159 */ "VNODES", - /* 160 */ "IN", - /* 161 */ "OUTPUTTYPE", - /* 162 */ "AGGREGATE", - /* 163 */ "BUFSIZE", - /* 164 */ "STREAM", - /* 165 */ "INTO", - /* 166 */ "TRIGGER", - /* 167 */ "AT_ONCE", - /* 168 */ "WINDOW_CLOSE", - /* 169 */ "MAX_DELAY", - /* 170 */ "WATERMARK", - /* 171 */ "KILL", - /* 172 */ "CONNECTION", - /* 173 */ "TRANSACTION", - /* 174 */ "BALANCE", - /* 175 */ "VGROUP", - /* 176 */ "MERGE", - /* 177 */ "REDISTRIBUTE", - /* 178 */ "SPLIT", - /* 179 */ "SYNCDB", - /* 180 */ "DELETE", - /* 181 */ "NULL", - /* 182 */ "NK_QUESTION", - /* 183 */ "NK_ARROW", - /* 184 */ "ROWTS", - /* 185 */ "TBNAME", - /* 186 */ "QSTARTTS", - /* 187 */ "QENDTS", - /* 188 */ "WSTARTTS", - /* 189 */ "WENDTS", - /* 190 */ "WDURATION", - /* 191 */ "CAST", - /* 192 */ "NOW", - /* 193 */ "TODAY", - /* 194 */ "TIMEZONE", - /* 195 */ "COUNT", - /* 196 */ "FIRST", - /* 197 */ "LAST", - /* 198 */ "LAST_ROW", - /* 199 */ "BETWEEN", - /* 200 */ "IS", - /* 201 */ "NK_LT", - /* 202 */ "NK_GT", - /* 203 */ "NK_LE", - /* 204 */ "NK_GE", - /* 205 */ "NK_NE", - /* 206 */ "MATCH", - /* 207 */ "NMATCH", - /* 208 */ "CONTAINS", - /* 209 */ "JOIN", - /* 210 */ "INNER", - /* 211 */ "SELECT", - /* 212 */ "DISTINCT", - /* 213 */ "WHERE", - /* 214 */ "PARTITION", - /* 215 */ "BY", - /* 216 */ "SESSION", - /* 217 */ "STATE_WINDOW", - /* 218 */ "SLIDING", - /* 219 */ "FILL", - /* 220 */ "VALUE", - /* 221 */ "NONE", - /* 222 */ "PREV", - /* 223 */ "LINEAR", - /* 224 */ "NEXT", - /* 225 */ "HAVING", - /* 226 */ "ORDER", - /* 227 */ "SLIMIT", - /* 228 */ "SOFFSET", - /* 229 */ "LIMIT", - /* 230 */ "OFFSET", - /* 231 */ "ASC", - /* 232 */ "NULLS", - /* 233 */ "ID", - /* 234 */ "NK_BITNOT", - /* 235 */ "INSERT", - /* 236 */ "VALUES", - /* 237 */ "IMPORT", - /* 238 */ "NK_SEMI", - /* 239 */ "FILE", - /* 240 */ "cmd", - /* 241 */ "account_options", - /* 242 */ "alter_account_options", - /* 243 */ "literal", - /* 244 */ "alter_account_option", - /* 245 */ "user_name", - /* 246 */ "privileges", - /* 247 */ "priv_level", - /* 248 */ "priv_type_list", - /* 249 */ "priv_type", - /* 250 */ "db_name", - /* 251 */ "dnode_endpoint", - /* 252 */ "dnode_host_name", - /* 253 */ "not_exists_opt", - /* 254 */ "db_options", - /* 255 */ "exists_opt", - /* 256 */ "alter_db_options", - /* 257 */ "integer_list", - /* 258 */ "variable_list", - /* 259 */ "retention_list", - /* 260 */ "alter_db_option", - /* 261 */ "retention", - /* 262 */ "full_table_name", - /* 263 */ "column_def_list", - /* 264 */ "tags_def_opt", - /* 265 */ "table_options", - /* 266 */ "multi_create_clause", - /* 267 */ "tags_def", - /* 268 */ "multi_drop_clause", - /* 269 */ "alter_table_clause", - /* 270 */ "alter_table_options", - /* 271 */ "column_name", - /* 272 */ "type_name", - /* 273 */ "signed_literal", - /* 274 */ "create_subtable_clause", - /* 275 */ "specific_tags_opt", - /* 276 */ "literal_list", - /* 277 */ "drop_table_clause", - /* 278 */ "col_name_list", - /* 279 */ "table_name", - /* 280 */ "column_def", - /* 281 */ "func_name_list", - /* 282 */ "alter_table_option", - /* 283 */ "col_name", - /* 284 */ "db_name_cond_opt", - /* 285 */ "like_pattern_opt", - /* 286 */ "table_name_cond", - /* 287 */ "from_db_opt", - /* 288 */ "func_name", - /* 289 */ "function_name", + /* 117 */ "FIRST", + /* 118 */ "LAST", + /* 119 */ "SHOW", + /* 120 */ "DATABASES", + /* 121 */ "TABLES", + /* 122 */ "STABLES", + /* 123 */ "MNODES", + /* 124 */ "MODULES", + /* 125 */ "QNODES", + /* 126 */ "FUNCTIONS", + /* 127 */ "INDEXES", + /* 128 */ "ACCOUNTS", + /* 129 */ "APPS", + /* 130 */ "CONNECTIONS", + /* 131 */ "LICENCE", + /* 132 */ "GRANTS", + /* 133 */ "QUERIES", + /* 134 */ "SCORES", + /* 135 */ "TOPICS", + /* 136 */ "VARIABLES", + /* 137 */ "BNODES", + /* 138 */ "SNODES", + /* 139 */ "CLUSTER", + /* 140 */ "TRANSACTIONS", + /* 141 */ "LIKE", + /* 142 */ "INDEX", + /* 143 */ "FULLTEXT", + /* 144 */ "FUNCTION", + /* 145 */ "INTERVAL", + /* 146 */ "TOPIC", + /* 147 */ "AS", + /* 148 */ "CONSUMER", + /* 149 */ "GROUP", + /* 150 */ "DESC", + /* 151 */ "DESCRIBE", + /* 152 */ "RESET", + /* 153 */ "QUERY", + /* 154 */ "CACHE", + /* 155 */ "EXPLAIN", + /* 156 */ "ANALYZE", + /* 157 */ "VERBOSE", + /* 158 */ "NK_BOOL", + /* 159 */ "RATIO", + /* 160 */ "NK_FLOAT", + /* 161 */ "COMPACT", + /* 162 */ "VNODES", + /* 163 */ "IN", + /* 164 */ "OUTPUTTYPE", + /* 165 */ "AGGREGATE", + /* 166 */ "BUFSIZE", + /* 167 */ "STREAM", + /* 168 */ "INTO", + /* 169 */ "TRIGGER", + /* 170 */ "AT_ONCE", + /* 171 */ "WINDOW_CLOSE", + /* 172 */ "KILL", + /* 173 */ "CONNECTION", + /* 174 */ "TRANSACTION", + /* 175 */ "BALANCE", + /* 176 */ "VGROUP", + /* 177 */ "MERGE", + /* 178 */ "REDISTRIBUTE", + /* 179 */ "SPLIT", + /* 180 */ "SYNCDB", + /* 181 */ "DELETE", + /* 182 */ "NULL", + /* 183 */ "NK_QUESTION", + /* 184 */ "NK_ARROW", + /* 185 */ "ROWTS", + /* 186 */ "TBNAME", + /* 187 */ "QSTARTTS", + /* 188 */ "QENDTS", + /* 189 */ "WSTARTTS", + /* 190 */ "WENDTS", + /* 191 */ "WDURATION", + /* 192 */ "CAST", + /* 193 */ "NOW", + /* 194 */ "TODAY", + /* 195 */ "TIMEZONE", + /* 196 */ "COUNT", + /* 197 */ "LAST_ROW", + /* 198 */ "BETWEEN", + /* 199 */ "IS", + /* 200 */ "NK_LT", + /* 201 */ "NK_GT", + /* 202 */ "NK_LE", + /* 203 */ "NK_GE", + /* 204 */ "NK_NE", + /* 205 */ "MATCH", + /* 206 */ "NMATCH", + /* 207 */ "CONTAINS", + /* 208 */ "JOIN", + /* 209 */ "INNER", + /* 210 */ "SELECT", + /* 211 */ "DISTINCT", + /* 212 */ "WHERE", + /* 213 */ "PARTITION", + /* 214 */ "BY", + /* 215 */ "SESSION", + /* 216 */ "STATE_WINDOW", + /* 217 */ "SLIDING", + /* 218 */ "FILL", + /* 219 */ "VALUE", + /* 220 */ "NONE", + /* 221 */ "PREV", + /* 222 */ "LINEAR", + /* 223 */ "NEXT", + /* 224 */ "HAVING", + /* 225 */ "ORDER", + /* 226 */ "SLIMIT", + /* 227 */ "SOFFSET", + /* 228 */ "LIMIT", + /* 229 */ "OFFSET", + /* 230 */ "ASC", + /* 231 */ "NULLS", + /* 232 */ "ID", + /* 233 */ "NK_BITNOT", + /* 234 */ "INSERT", + /* 235 */ "VALUES", + /* 236 */ "IMPORT", + /* 237 */ "NK_SEMI", + /* 238 */ "FILE", + /* 239 */ "cmd", + /* 240 */ "account_options", + /* 241 */ "alter_account_options", + /* 242 */ "literal", + /* 243 */ "alter_account_option", + /* 244 */ "user_name", + /* 245 */ "privileges", + /* 246 */ "priv_level", + /* 247 */ "priv_type_list", + /* 248 */ "priv_type", + /* 249 */ "db_name", + /* 250 */ "dnode_endpoint", + /* 251 */ "not_exists_opt", + /* 252 */ "db_options", + /* 253 */ "exists_opt", + /* 254 */ "alter_db_options", + /* 255 */ "integer_list", + /* 256 */ "variable_list", + /* 257 */ "retention_list", + /* 258 */ "alter_db_option", + /* 259 */ "retention", + /* 260 */ "full_table_name", + /* 261 */ "column_def_list", + /* 262 */ "tags_def_opt", + /* 263 */ "table_options", + /* 264 */ "multi_create_clause", + /* 265 */ "tags_def", + /* 266 */ "multi_drop_clause", + /* 267 */ "alter_table_clause", + /* 268 */ "alter_table_options", + /* 269 */ "column_name", + /* 270 */ "type_name", + /* 271 */ "signed_literal", + /* 272 */ "create_subtable_clause", + /* 273 */ "specific_tags_opt", + /* 274 */ "literal_list", + /* 275 */ "drop_table_clause", + /* 276 */ "col_name_list", + /* 277 */ "table_name", + /* 278 */ "column_def", + /* 279 */ "duration_list", + /* 280 */ "rollup_func_list", + /* 281 */ "alter_table_option", + /* 282 */ "duration_literal", + /* 283 */ "rollup_func_name", + /* 284 */ "function_name", + /* 285 */ "col_name", + /* 286 */ "db_name_cond_opt", + /* 287 */ "like_pattern_opt", + /* 288 */ "table_name_cond", + /* 289 */ "from_db_opt", /* 290 */ "index_name", /* 291 */ "index_options", /* 292 */ "func_list", - /* 293 */ "duration_literal", - /* 294 */ "sliding_opt", - /* 295 */ "func", - /* 296 */ "expression_list", - /* 297 */ "topic_name", - /* 298 */ "query_expression", - /* 299 */ "cgroup_name", - /* 300 */ "analyze_opt", - /* 301 */ "explain_options", - /* 302 */ "agg_func_opt", - /* 303 */ "bufsize_opt", - /* 304 */ "stream_name", - /* 305 */ "stream_options", - /* 306 */ "into_opt", - /* 307 */ "dnode_list", - /* 308 */ "where_clause_opt", - /* 309 */ "signed", - /* 310 */ "literal_func", - /* 311 */ "table_alias", - /* 312 */ "column_alias", - /* 313 */ "expression", - /* 314 */ "pseudo_column", - /* 315 */ "column_reference", - /* 316 */ "function_expression", - /* 317 */ "subquery", - /* 318 */ "star_func", - /* 319 */ "star_func_para_list", - /* 320 */ "noarg_func", - /* 321 */ "other_para_list", - /* 322 */ "star_func_para", - /* 323 */ "predicate", - /* 324 */ "compare_op", - /* 325 */ "in_op", - /* 326 */ "in_predicate_value", - /* 327 */ "boolean_value_expression", - /* 328 */ "boolean_primary", - /* 329 */ "common_expression", - /* 330 */ "from_clause", - /* 331 */ "table_reference_list", - /* 332 */ "table_reference", - /* 333 */ "table_primary", - /* 334 */ "joined_table", - /* 335 */ "alias_opt", - /* 336 */ "parenthesized_joined_table", - /* 337 */ "join_type", - /* 338 */ "search_condition", - /* 339 */ "query_specification", - /* 340 */ "set_quantifier_opt", - /* 341 */ "select_list", - /* 342 */ "partition_by_clause_opt", - /* 343 */ "twindow_clause_opt", - /* 344 */ "group_by_clause_opt", - /* 345 */ "having_clause_opt", - /* 346 */ "select_sublist", - /* 347 */ "select_item", - /* 348 */ "fill_opt", - /* 349 */ "fill_mode", - /* 350 */ "group_by_list", - /* 351 */ "query_expression_body", - /* 352 */ "order_by_clause_opt", - /* 353 */ "slimit_clause_opt", - /* 354 */ "limit_clause_opt", - /* 355 */ "query_primary", - /* 356 */ "sort_specification_list", - /* 357 */ "sort_specification", - /* 358 */ "ordering_specification_opt", - /* 359 */ "null_ordering_opt", + /* 293 */ "sliding_opt", + /* 294 */ "func", + /* 295 */ "expression_list", + /* 296 */ "topic_name", + /* 297 */ "query_expression", + /* 298 */ "cgroup_name", + /* 299 */ "analyze_opt", + /* 300 */ "explain_options", + /* 301 */ "agg_func_opt", + /* 302 */ "bufsize_opt", + /* 303 */ "stream_name", + /* 304 */ "stream_options", + /* 305 */ "into_opt", + /* 306 */ "dnode_list", + /* 307 */ "where_clause_opt", + /* 308 */ "signed", + /* 309 */ "literal_func", + /* 310 */ "table_alias", + /* 311 */ "column_alias", + /* 312 */ "expression", + /* 313 */ "pseudo_column", + /* 314 */ "column_reference", + /* 315 */ "function_expression", + /* 316 */ "subquery", + /* 317 */ "star_func", + /* 318 */ "star_func_para_list", + /* 319 */ "noarg_func", + /* 320 */ "other_para_list", + /* 321 */ "star_func_para", + /* 322 */ "predicate", + /* 323 */ "compare_op", + /* 324 */ "in_op", + /* 325 */ "in_predicate_value", + /* 326 */ "boolean_value_expression", + /* 327 */ "boolean_primary", + /* 328 */ "common_expression", + /* 329 */ "from_clause", + /* 330 */ "table_reference_list", + /* 331 */ "table_reference", + /* 332 */ "table_primary", + /* 333 */ "joined_table", + /* 334 */ "alias_opt", + /* 335 */ "parenthesized_joined_table", + /* 336 */ "join_type", + /* 337 */ "search_condition", + /* 338 */ "query_specification", + /* 339 */ "set_quantifier_opt", + /* 340 */ "select_list", + /* 341 */ "partition_by_clause_opt", + /* 342 */ "twindow_clause_opt", + /* 343 */ "group_by_clause_opt", + /* 344 */ "having_clause_opt", + /* 345 */ "select_sublist", + /* 346 */ "select_item", + /* 347 */ "fill_opt", + /* 348 */ "fill_mode", + /* 349 */ "group_by_list", + /* 350 */ "query_expression_body", + /* 351 */ "order_by_clause_opt", + /* 352 */ "slimit_clause_opt", + /* 353 */ "limit_clause_opt", + /* 354 */ "query_primary", + /* 355 */ "sort_specification_list", + /* 356 */ "sort_specification", + /* 357 */ "ordering_specification_opt", + /* 358 */ "null_ordering_opt", }; #endif /* defined(YYCOVERAGE) || !defined(NDEBUG) */ @@ -1568,7 +1553,7 @@ static const char *const yyRuleName[] = { /* 36 */ "priv_level ::= NK_STAR NK_DOT NK_STAR", /* 37 */ "priv_level ::= db_name NK_DOT NK_STAR", /* 38 */ "cmd ::= CREATE DNODE dnode_endpoint", - /* 39 */ "cmd ::= CREATE DNODE dnode_host_name PORT NK_INTEGER", + /* 39 */ "cmd ::= CREATE DNODE dnode_endpoint PORT NK_INTEGER", /* 40 */ "cmd ::= DROP DNODE NK_INTEGER", /* 41 */ "cmd ::= DROP DNODE dnode_endpoint", /* 42 */ "cmd ::= ALTER DNODE NK_INTEGER NK_STRING", @@ -1576,8 +1561,8 @@ static const char *const yyRuleName[] = { /* 44 */ "cmd ::= ALTER ALL DNODES NK_STRING", /* 45 */ "cmd ::= ALTER ALL DNODES NK_STRING NK_STRING", /* 46 */ "dnode_endpoint ::= NK_STRING", - /* 47 */ "dnode_host_name ::= NK_ID", - /* 48 */ "dnode_host_name ::= NK_IPTOKEN", + /* 47 */ "dnode_endpoint ::= NK_ID", + /* 48 */ "dnode_endpoint ::= NK_IPTOKEN", /* 49 */ "cmd ::= ALTER LOCAL NK_STRING", /* 50 */ "cmd ::= ALTER LOCAL NK_STRING NK_STRING", /* 51 */ "cmd ::= CREATE QNODE ON DNODE NK_INTEGER", @@ -1694,295 +1679,300 @@ static const char *const yyRuleName[] = { /* 162 */ "tags_def ::= TAGS NK_LP column_def_list NK_RP", /* 163 */ "table_options ::=", /* 164 */ "table_options ::= table_options COMMENT NK_STRING", - /* 165 */ "table_options ::= table_options FILE_FACTOR NK_FLOAT", - /* 166 */ "table_options ::= table_options ROLLUP NK_LP func_name_list NK_RP", - /* 167 */ "table_options ::= table_options TTL NK_INTEGER", - /* 168 */ "table_options ::= table_options SMA NK_LP col_name_list NK_RP", - /* 169 */ "alter_table_options ::= alter_table_option", - /* 170 */ "alter_table_options ::= alter_table_options alter_table_option", - /* 171 */ "alter_table_option ::= COMMENT NK_STRING", - /* 172 */ "alter_table_option ::= TTL NK_INTEGER", - /* 173 */ "col_name_list ::= col_name", - /* 174 */ "col_name_list ::= col_name_list NK_COMMA col_name", - /* 175 */ "col_name ::= column_name", - /* 176 */ "cmd ::= SHOW DNODES", - /* 177 */ "cmd ::= SHOW USERS", - /* 178 */ "cmd ::= SHOW DATABASES", - /* 179 */ "cmd ::= SHOW db_name_cond_opt TABLES like_pattern_opt", - /* 180 */ "cmd ::= SHOW db_name_cond_opt STABLES like_pattern_opt", - /* 181 */ "cmd ::= SHOW db_name_cond_opt VGROUPS", - /* 182 */ "cmd ::= SHOW MNODES", - /* 183 */ "cmd ::= SHOW MODULES", - /* 184 */ "cmd ::= SHOW QNODES", - /* 185 */ "cmd ::= SHOW FUNCTIONS", - /* 186 */ "cmd ::= SHOW INDEXES FROM table_name_cond from_db_opt", - /* 187 */ "cmd ::= SHOW STREAMS", - /* 188 */ "cmd ::= SHOW ACCOUNTS", - /* 189 */ "cmd ::= SHOW APPS", - /* 190 */ "cmd ::= SHOW CONNECTIONS", - /* 191 */ "cmd ::= SHOW LICENCE", - /* 192 */ "cmd ::= SHOW GRANTS", - /* 193 */ "cmd ::= SHOW CREATE DATABASE db_name", - /* 194 */ "cmd ::= SHOW CREATE TABLE full_table_name", - /* 195 */ "cmd ::= SHOW CREATE STABLE full_table_name", - /* 196 */ "cmd ::= SHOW QUERIES", - /* 197 */ "cmd ::= SHOW SCORES", - /* 198 */ "cmd ::= SHOW TOPICS", - /* 199 */ "cmd ::= SHOW VARIABLES", - /* 200 */ "cmd ::= SHOW BNODES", - /* 201 */ "cmd ::= SHOW SNODES", - /* 202 */ "cmd ::= SHOW CLUSTER", - /* 203 */ "cmd ::= SHOW TRANSACTIONS", - /* 204 */ "db_name_cond_opt ::=", - /* 205 */ "db_name_cond_opt ::= db_name NK_DOT", - /* 206 */ "like_pattern_opt ::=", - /* 207 */ "like_pattern_opt ::= LIKE NK_STRING", - /* 208 */ "table_name_cond ::= table_name", - /* 209 */ "from_db_opt ::=", - /* 210 */ "from_db_opt ::= FROM db_name", - /* 211 */ "func_name_list ::= func_name", - /* 212 */ "func_name_list ::= func_name_list NK_COMMA func_name", - /* 213 */ "func_name ::= function_name", - /* 214 */ "cmd ::= CREATE SMA INDEX not_exists_opt index_name ON table_name index_options", - /* 215 */ "cmd ::= CREATE FULLTEXT INDEX not_exists_opt index_name ON table_name NK_LP col_name_list NK_RP", - /* 216 */ "cmd ::= DROP INDEX exists_opt index_name ON table_name", - /* 217 */ "index_options ::=", - /* 218 */ "index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_RP sliding_opt", - /* 219 */ "index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt", - /* 220 */ "func_list ::= func", - /* 221 */ "func_list ::= func_list NK_COMMA func", - /* 222 */ "func ::= function_name NK_LP expression_list NK_RP", - /* 223 */ "cmd ::= CREATE TOPIC not_exists_opt topic_name AS query_expression", - /* 224 */ "cmd ::= CREATE TOPIC not_exists_opt topic_name AS DATABASE db_name", - /* 225 */ "cmd ::= CREATE TOPIC not_exists_opt topic_name AS STABLE full_table_name", - /* 226 */ "cmd ::= DROP TOPIC exists_opt topic_name", - /* 227 */ "cmd ::= DROP CONSUMER GROUP exists_opt cgroup_name ON topic_name", - /* 228 */ "cmd ::= DESC full_table_name", - /* 229 */ "cmd ::= DESCRIBE full_table_name", - /* 230 */ "cmd ::= RESET QUERY CACHE", - /* 231 */ "cmd ::= EXPLAIN analyze_opt explain_options query_expression", - /* 232 */ "analyze_opt ::=", - /* 233 */ "analyze_opt ::= ANALYZE", - /* 234 */ "explain_options ::=", - /* 235 */ "explain_options ::= explain_options VERBOSE NK_BOOL", - /* 236 */ "explain_options ::= explain_options RATIO NK_FLOAT", - /* 237 */ "cmd ::= COMPACT VNODES IN NK_LP integer_list NK_RP", - /* 238 */ "cmd ::= CREATE agg_func_opt FUNCTION not_exists_opt function_name AS NK_STRING OUTPUTTYPE type_name bufsize_opt", - /* 239 */ "cmd ::= DROP FUNCTION exists_opt function_name", - /* 240 */ "agg_func_opt ::=", - /* 241 */ "agg_func_opt ::= AGGREGATE", - /* 242 */ "bufsize_opt ::=", - /* 243 */ "bufsize_opt ::= BUFSIZE NK_INTEGER", - /* 244 */ "cmd ::= CREATE STREAM not_exists_opt stream_name stream_options into_opt AS query_expression", - /* 245 */ "cmd ::= DROP STREAM exists_opt stream_name", - /* 246 */ "into_opt ::=", - /* 247 */ "into_opt ::= INTO full_table_name", - /* 248 */ "stream_options ::=", - /* 249 */ "stream_options ::= stream_options TRIGGER AT_ONCE", - /* 250 */ "stream_options ::= stream_options TRIGGER WINDOW_CLOSE", - /* 251 */ "stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal", - /* 252 */ "stream_options ::= stream_options WATERMARK duration_literal", - /* 253 */ "cmd ::= KILL CONNECTION NK_INTEGER", - /* 254 */ "cmd ::= KILL QUERY NK_STRING", - /* 255 */ "cmd ::= KILL TRANSACTION NK_INTEGER", - /* 256 */ "cmd ::= BALANCE VGROUP", - /* 257 */ "cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER", - /* 258 */ "cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list", - /* 259 */ "cmd ::= SPLIT VGROUP NK_INTEGER", - /* 260 */ "dnode_list ::= DNODE NK_INTEGER", - /* 261 */ "dnode_list ::= dnode_list DNODE NK_INTEGER", - /* 262 */ "cmd ::= SYNCDB db_name REPLICA", - /* 263 */ "cmd ::= DELETE FROM full_table_name where_clause_opt", - /* 264 */ "cmd ::= query_expression", - /* 265 */ "literal ::= NK_INTEGER", - /* 266 */ "literal ::= NK_FLOAT", - /* 267 */ "literal ::= NK_STRING", - /* 268 */ "literal ::= NK_BOOL", - /* 269 */ "literal ::= TIMESTAMP NK_STRING", - /* 270 */ "literal ::= duration_literal", - /* 271 */ "literal ::= NULL", - /* 272 */ "literal ::= NK_QUESTION", - /* 273 */ "duration_literal ::= NK_VARIABLE", - /* 274 */ "signed ::= NK_INTEGER", - /* 275 */ "signed ::= NK_PLUS NK_INTEGER", - /* 276 */ "signed ::= NK_MINUS NK_INTEGER", - /* 277 */ "signed ::= NK_FLOAT", - /* 278 */ "signed ::= NK_PLUS NK_FLOAT", - /* 279 */ "signed ::= NK_MINUS NK_FLOAT", - /* 280 */ "signed_literal ::= signed", - /* 281 */ "signed_literal ::= NK_STRING", - /* 282 */ "signed_literal ::= NK_BOOL", - /* 283 */ "signed_literal ::= TIMESTAMP NK_STRING", - /* 284 */ "signed_literal ::= duration_literal", - /* 285 */ "signed_literal ::= NULL", - /* 286 */ "signed_literal ::= literal_func", - /* 287 */ "literal_list ::= signed_literal", - /* 288 */ "literal_list ::= literal_list NK_COMMA signed_literal", - /* 289 */ "db_name ::= NK_ID", - /* 290 */ "table_name ::= NK_ID", - /* 291 */ "column_name ::= NK_ID", - /* 292 */ "function_name ::= NK_ID", - /* 293 */ "table_alias ::= NK_ID", - /* 294 */ "column_alias ::= NK_ID", - /* 295 */ "user_name ::= NK_ID", - /* 296 */ "index_name ::= NK_ID", - /* 297 */ "topic_name ::= NK_ID", - /* 298 */ "stream_name ::= NK_ID", - /* 299 */ "cgroup_name ::= NK_ID", - /* 300 */ "expression ::= literal", - /* 301 */ "expression ::= pseudo_column", - /* 302 */ "expression ::= column_reference", - /* 303 */ "expression ::= function_expression", - /* 304 */ "expression ::= subquery", - /* 305 */ "expression ::= NK_LP expression NK_RP", - /* 306 */ "expression ::= NK_PLUS expression", - /* 307 */ "expression ::= NK_MINUS expression", - /* 308 */ "expression ::= expression NK_PLUS expression", - /* 309 */ "expression ::= expression NK_MINUS expression", - /* 310 */ "expression ::= expression NK_STAR expression", - /* 311 */ "expression ::= expression NK_SLASH expression", - /* 312 */ "expression ::= expression NK_REM expression", - /* 313 */ "expression ::= column_reference NK_ARROW NK_STRING", - /* 314 */ "expression_list ::= expression", - /* 315 */ "expression_list ::= expression_list NK_COMMA expression", - /* 316 */ "column_reference ::= column_name", - /* 317 */ "column_reference ::= table_name NK_DOT column_name", - /* 318 */ "pseudo_column ::= ROWTS", - /* 319 */ "pseudo_column ::= TBNAME", - /* 320 */ "pseudo_column ::= table_name NK_DOT TBNAME", - /* 321 */ "pseudo_column ::= QSTARTTS", - /* 322 */ "pseudo_column ::= QENDTS", - /* 323 */ "pseudo_column ::= WSTARTTS", - /* 324 */ "pseudo_column ::= WENDTS", - /* 325 */ "pseudo_column ::= WDURATION", - /* 326 */ "function_expression ::= function_name NK_LP expression_list NK_RP", - /* 327 */ "function_expression ::= star_func NK_LP star_func_para_list NK_RP", - /* 328 */ "function_expression ::= CAST NK_LP expression AS type_name NK_RP", - /* 329 */ "function_expression ::= literal_func", - /* 330 */ "literal_func ::= noarg_func NK_LP NK_RP", - /* 331 */ "literal_func ::= NOW", - /* 332 */ "noarg_func ::= NOW", - /* 333 */ "noarg_func ::= TODAY", - /* 334 */ "noarg_func ::= TIMEZONE", - /* 335 */ "star_func ::= COUNT", - /* 336 */ "star_func ::= FIRST", - /* 337 */ "star_func ::= LAST", - /* 338 */ "star_func ::= LAST_ROW", - /* 339 */ "star_func_para_list ::= NK_STAR", - /* 340 */ "star_func_para_list ::= other_para_list", - /* 341 */ "other_para_list ::= star_func_para", - /* 342 */ "other_para_list ::= other_para_list NK_COMMA star_func_para", - /* 343 */ "star_func_para ::= expression", - /* 344 */ "star_func_para ::= table_name NK_DOT NK_STAR", - /* 345 */ "predicate ::= expression compare_op expression", - /* 346 */ "predicate ::= expression BETWEEN expression AND expression", - /* 347 */ "predicate ::= expression NOT BETWEEN expression AND expression", - /* 348 */ "predicate ::= expression IS NULL", - /* 349 */ "predicate ::= expression IS NOT NULL", - /* 350 */ "predicate ::= expression in_op in_predicate_value", - /* 351 */ "compare_op ::= NK_LT", - /* 352 */ "compare_op ::= NK_GT", - /* 353 */ "compare_op ::= NK_LE", - /* 354 */ "compare_op ::= NK_GE", - /* 355 */ "compare_op ::= NK_NE", - /* 356 */ "compare_op ::= NK_EQ", - /* 357 */ "compare_op ::= LIKE", - /* 358 */ "compare_op ::= NOT LIKE", - /* 359 */ "compare_op ::= MATCH", - /* 360 */ "compare_op ::= NMATCH", - /* 361 */ "compare_op ::= CONTAINS", - /* 362 */ "in_op ::= IN", - /* 363 */ "in_op ::= NOT IN", - /* 364 */ "in_predicate_value ::= NK_LP expression_list NK_RP", - /* 365 */ "boolean_value_expression ::= boolean_primary", - /* 366 */ "boolean_value_expression ::= NOT boolean_primary", - /* 367 */ "boolean_value_expression ::= boolean_value_expression OR boolean_value_expression", - /* 368 */ "boolean_value_expression ::= boolean_value_expression AND boolean_value_expression", - /* 369 */ "boolean_primary ::= predicate", - /* 370 */ "boolean_primary ::= NK_LP boolean_value_expression NK_RP", - /* 371 */ "common_expression ::= expression", - /* 372 */ "common_expression ::= boolean_value_expression", - /* 373 */ "from_clause ::= FROM table_reference_list", - /* 374 */ "table_reference_list ::= table_reference", - /* 375 */ "table_reference_list ::= table_reference_list NK_COMMA table_reference", - /* 376 */ "table_reference ::= table_primary", - /* 377 */ "table_reference ::= joined_table", - /* 378 */ "table_primary ::= table_name alias_opt", - /* 379 */ "table_primary ::= db_name NK_DOT table_name alias_opt", - /* 380 */ "table_primary ::= subquery alias_opt", - /* 381 */ "table_primary ::= parenthesized_joined_table", - /* 382 */ "alias_opt ::=", - /* 383 */ "alias_opt ::= table_alias", - /* 384 */ "alias_opt ::= AS table_alias", - /* 385 */ "parenthesized_joined_table ::= NK_LP joined_table NK_RP", - /* 386 */ "parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP", - /* 387 */ "joined_table ::= table_reference join_type JOIN table_reference ON search_condition", - /* 388 */ "join_type ::=", - /* 389 */ "join_type ::= INNER", - /* 390 */ "query_specification ::= SELECT set_quantifier_opt select_list from_clause where_clause_opt partition_by_clause_opt twindow_clause_opt group_by_clause_opt having_clause_opt", - /* 391 */ "set_quantifier_opt ::=", - /* 392 */ "set_quantifier_opt ::= DISTINCT", - /* 393 */ "set_quantifier_opt ::= ALL", - /* 394 */ "select_list ::= NK_STAR", - /* 395 */ "select_list ::= select_sublist", - /* 396 */ "select_sublist ::= select_item", - /* 397 */ "select_sublist ::= select_sublist NK_COMMA select_item", - /* 398 */ "select_item ::= common_expression", - /* 399 */ "select_item ::= common_expression column_alias", - /* 400 */ "select_item ::= common_expression AS column_alias", - /* 401 */ "select_item ::= table_name NK_DOT NK_STAR", - /* 402 */ "where_clause_opt ::=", - /* 403 */ "where_clause_opt ::= WHERE search_condition", - /* 404 */ "partition_by_clause_opt ::=", - /* 405 */ "partition_by_clause_opt ::= PARTITION BY expression_list", - /* 406 */ "twindow_clause_opt ::=", - /* 407 */ "twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP", - /* 408 */ "twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP", - /* 409 */ "twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt", - /* 410 */ "twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt", - /* 411 */ "sliding_opt ::=", - /* 412 */ "sliding_opt ::= SLIDING NK_LP duration_literal NK_RP", - /* 413 */ "fill_opt ::=", - /* 414 */ "fill_opt ::= FILL NK_LP fill_mode NK_RP", - /* 415 */ "fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP", - /* 416 */ "fill_mode ::= NONE", - /* 417 */ "fill_mode ::= PREV", - /* 418 */ "fill_mode ::= NULL", - /* 419 */ "fill_mode ::= LINEAR", - /* 420 */ "fill_mode ::= NEXT", - /* 421 */ "group_by_clause_opt ::=", - /* 422 */ "group_by_clause_opt ::= GROUP BY group_by_list", - /* 423 */ "group_by_list ::= expression", - /* 424 */ "group_by_list ::= group_by_list NK_COMMA expression", - /* 425 */ "having_clause_opt ::=", - /* 426 */ "having_clause_opt ::= HAVING search_condition", - /* 427 */ "query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt", - /* 428 */ "query_expression_body ::= query_primary", - /* 429 */ "query_expression_body ::= query_expression_body UNION ALL query_expression_body", - /* 430 */ "query_expression_body ::= query_expression_body UNION query_expression_body", - /* 431 */ "query_primary ::= query_specification", - /* 432 */ "query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP", - /* 433 */ "order_by_clause_opt ::=", - /* 434 */ "order_by_clause_opt ::= ORDER BY sort_specification_list", - /* 435 */ "slimit_clause_opt ::=", - /* 436 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER", - /* 437 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER", - /* 438 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER", - /* 439 */ "limit_clause_opt ::=", - /* 440 */ "limit_clause_opt ::= LIMIT NK_INTEGER", - /* 441 */ "limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER", - /* 442 */ "limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER", - /* 443 */ "subquery ::= NK_LP query_expression NK_RP", - /* 444 */ "search_condition ::= common_expression", - /* 445 */ "sort_specification_list ::= sort_specification", - /* 446 */ "sort_specification_list ::= sort_specification_list NK_COMMA sort_specification", - /* 447 */ "sort_specification ::= expression ordering_specification_opt null_ordering_opt", - /* 448 */ "ordering_specification_opt ::=", - /* 449 */ "ordering_specification_opt ::= ASC", - /* 450 */ "ordering_specification_opt ::= DESC", - /* 451 */ "null_ordering_opt ::=", - /* 452 */ "null_ordering_opt ::= NULLS FIRST", - /* 453 */ "null_ordering_opt ::= NULLS LAST", + /* 165 */ "table_options ::= table_options MAX_DELAY duration_list", + /* 166 */ "table_options ::= table_options WATERMARK duration_list", + /* 167 */ "table_options ::= table_options ROLLUP NK_LP rollup_func_list NK_RP", + /* 168 */ "table_options ::= table_options TTL NK_INTEGER", + /* 169 */ "table_options ::= table_options SMA NK_LP col_name_list NK_RP", + /* 170 */ "alter_table_options ::= alter_table_option", + /* 171 */ "alter_table_options ::= alter_table_options alter_table_option", + /* 172 */ "alter_table_option ::= COMMENT NK_STRING", + /* 173 */ "alter_table_option ::= TTL NK_INTEGER", + /* 174 */ "duration_list ::= duration_literal", + /* 175 */ "duration_list ::= duration_list NK_COMMA duration_literal", + /* 176 */ "rollup_func_list ::= rollup_func_name", + /* 177 */ "rollup_func_list ::= rollup_func_list NK_COMMA rollup_func_name", + /* 178 */ "rollup_func_name ::= function_name", + /* 179 */ "rollup_func_name ::= FIRST", + /* 180 */ "rollup_func_name ::= LAST", + /* 181 */ "col_name_list ::= col_name", + /* 182 */ "col_name_list ::= col_name_list NK_COMMA col_name", + /* 183 */ "col_name ::= column_name", + /* 184 */ "cmd ::= SHOW DNODES", + /* 185 */ "cmd ::= SHOW USERS", + /* 186 */ "cmd ::= SHOW DATABASES", + /* 187 */ "cmd ::= SHOW db_name_cond_opt TABLES like_pattern_opt", + /* 188 */ "cmd ::= SHOW db_name_cond_opt STABLES like_pattern_opt", + /* 189 */ "cmd ::= SHOW db_name_cond_opt VGROUPS", + /* 190 */ "cmd ::= SHOW MNODES", + /* 191 */ "cmd ::= SHOW MODULES", + /* 192 */ "cmd ::= SHOW QNODES", + /* 193 */ "cmd ::= SHOW FUNCTIONS", + /* 194 */ "cmd ::= SHOW INDEXES FROM table_name_cond from_db_opt", + /* 195 */ "cmd ::= SHOW STREAMS", + /* 196 */ "cmd ::= SHOW ACCOUNTS", + /* 197 */ "cmd ::= SHOW APPS", + /* 198 */ "cmd ::= SHOW CONNECTIONS", + /* 199 */ "cmd ::= SHOW LICENCE", + /* 200 */ "cmd ::= SHOW GRANTS", + /* 201 */ "cmd ::= SHOW CREATE DATABASE db_name", + /* 202 */ "cmd ::= SHOW CREATE TABLE full_table_name", + /* 203 */ "cmd ::= SHOW CREATE STABLE full_table_name", + /* 204 */ "cmd ::= SHOW QUERIES", + /* 205 */ "cmd ::= SHOW SCORES", + /* 206 */ "cmd ::= SHOW TOPICS", + /* 207 */ "cmd ::= SHOW VARIABLES", + /* 208 */ "cmd ::= SHOW BNODES", + /* 209 */ "cmd ::= SHOW SNODES", + /* 210 */ "cmd ::= SHOW CLUSTER", + /* 211 */ "cmd ::= SHOW TRANSACTIONS", + /* 212 */ "db_name_cond_opt ::=", + /* 213 */ "db_name_cond_opt ::= db_name NK_DOT", + /* 214 */ "like_pattern_opt ::=", + /* 215 */ "like_pattern_opt ::= LIKE NK_STRING", + /* 216 */ "table_name_cond ::= table_name", + /* 217 */ "from_db_opt ::=", + /* 218 */ "from_db_opt ::= FROM db_name", + /* 219 */ "cmd ::= CREATE SMA INDEX not_exists_opt index_name ON table_name index_options", + /* 220 */ "cmd ::= CREATE FULLTEXT INDEX not_exists_opt index_name ON table_name NK_LP col_name_list NK_RP", + /* 221 */ "cmd ::= DROP INDEX exists_opt index_name ON table_name", + /* 222 */ "index_options ::=", + /* 223 */ "index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_RP sliding_opt", + /* 224 */ "index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt", + /* 225 */ "func_list ::= func", + /* 226 */ "func_list ::= func_list NK_COMMA func", + /* 227 */ "func ::= function_name NK_LP expression_list NK_RP", + /* 228 */ "cmd ::= CREATE TOPIC not_exists_opt topic_name AS query_expression", + /* 229 */ "cmd ::= CREATE TOPIC not_exists_opt topic_name AS DATABASE db_name", + /* 230 */ "cmd ::= CREATE TOPIC not_exists_opt topic_name AS STABLE full_table_name", + /* 231 */ "cmd ::= DROP TOPIC exists_opt topic_name", + /* 232 */ "cmd ::= DROP CONSUMER GROUP exists_opt cgroup_name ON topic_name", + /* 233 */ "cmd ::= DESC full_table_name", + /* 234 */ "cmd ::= DESCRIBE full_table_name", + /* 235 */ "cmd ::= RESET QUERY CACHE", + /* 236 */ "cmd ::= EXPLAIN analyze_opt explain_options query_expression", + /* 237 */ "analyze_opt ::=", + /* 238 */ "analyze_opt ::= ANALYZE", + /* 239 */ "explain_options ::=", + /* 240 */ "explain_options ::= explain_options VERBOSE NK_BOOL", + /* 241 */ "explain_options ::= explain_options RATIO NK_FLOAT", + /* 242 */ "cmd ::= COMPACT VNODES IN NK_LP integer_list NK_RP", + /* 243 */ "cmd ::= CREATE agg_func_opt FUNCTION not_exists_opt function_name AS NK_STRING OUTPUTTYPE type_name bufsize_opt", + /* 244 */ "cmd ::= DROP FUNCTION exists_opt function_name", + /* 245 */ "agg_func_opt ::=", + /* 246 */ "agg_func_opt ::= AGGREGATE", + /* 247 */ "bufsize_opt ::=", + /* 248 */ "bufsize_opt ::= BUFSIZE NK_INTEGER", + /* 249 */ "cmd ::= CREATE STREAM not_exists_opt stream_name stream_options into_opt AS query_expression", + /* 250 */ "cmd ::= DROP STREAM exists_opt stream_name", + /* 251 */ "into_opt ::=", + /* 252 */ "into_opt ::= INTO full_table_name", + /* 253 */ "stream_options ::=", + /* 254 */ "stream_options ::= stream_options TRIGGER AT_ONCE", + /* 255 */ "stream_options ::= stream_options TRIGGER WINDOW_CLOSE", + /* 256 */ "stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal", + /* 257 */ "stream_options ::= stream_options WATERMARK duration_literal", + /* 258 */ "cmd ::= KILL CONNECTION NK_INTEGER", + /* 259 */ "cmd ::= KILL QUERY NK_STRING", + /* 260 */ "cmd ::= KILL TRANSACTION NK_INTEGER", + /* 261 */ "cmd ::= BALANCE VGROUP", + /* 262 */ "cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER", + /* 263 */ "cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list", + /* 264 */ "cmd ::= SPLIT VGROUP NK_INTEGER", + /* 265 */ "dnode_list ::= DNODE NK_INTEGER", + /* 266 */ "dnode_list ::= dnode_list DNODE NK_INTEGER", + /* 267 */ "cmd ::= SYNCDB db_name REPLICA", + /* 268 */ "cmd ::= DELETE FROM full_table_name where_clause_opt", + /* 269 */ "cmd ::= query_expression", + /* 270 */ "literal ::= NK_INTEGER", + /* 271 */ "literal ::= NK_FLOAT", + /* 272 */ "literal ::= NK_STRING", + /* 273 */ "literal ::= NK_BOOL", + /* 274 */ "literal ::= TIMESTAMP NK_STRING", + /* 275 */ "literal ::= duration_literal", + /* 276 */ "literal ::= NULL", + /* 277 */ "literal ::= NK_QUESTION", + /* 278 */ "duration_literal ::= NK_VARIABLE", + /* 279 */ "signed ::= NK_INTEGER", + /* 280 */ "signed ::= NK_PLUS NK_INTEGER", + /* 281 */ "signed ::= NK_MINUS NK_INTEGER", + /* 282 */ "signed ::= NK_FLOAT", + /* 283 */ "signed ::= NK_PLUS NK_FLOAT", + /* 284 */ "signed ::= NK_MINUS NK_FLOAT", + /* 285 */ "signed_literal ::= signed", + /* 286 */ "signed_literal ::= NK_STRING", + /* 287 */ "signed_literal ::= NK_BOOL", + /* 288 */ "signed_literal ::= TIMESTAMP NK_STRING", + /* 289 */ "signed_literal ::= duration_literal", + /* 290 */ "signed_literal ::= NULL", + /* 291 */ "signed_literal ::= literal_func", + /* 292 */ "literal_list ::= signed_literal", + /* 293 */ "literal_list ::= literal_list NK_COMMA signed_literal", + /* 294 */ "db_name ::= NK_ID", + /* 295 */ "table_name ::= NK_ID", + /* 296 */ "column_name ::= NK_ID", + /* 297 */ "function_name ::= NK_ID", + /* 298 */ "table_alias ::= NK_ID", + /* 299 */ "column_alias ::= NK_ID", + /* 300 */ "user_name ::= NK_ID", + /* 301 */ "index_name ::= NK_ID", + /* 302 */ "topic_name ::= NK_ID", + /* 303 */ "stream_name ::= NK_ID", + /* 304 */ "cgroup_name ::= NK_ID", + /* 305 */ "expression ::= literal", + /* 306 */ "expression ::= pseudo_column", + /* 307 */ "expression ::= column_reference", + /* 308 */ "expression ::= function_expression", + /* 309 */ "expression ::= subquery", + /* 310 */ "expression ::= NK_LP expression NK_RP", + /* 311 */ "expression ::= NK_PLUS expression", + /* 312 */ "expression ::= NK_MINUS expression", + /* 313 */ "expression ::= expression NK_PLUS expression", + /* 314 */ "expression ::= expression NK_MINUS expression", + /* 315 */ "expression ::= expression NK_STAR expression", + /* 316 */ "expression ::= expression NK_SLASH expression", + /* 317 */ "expression ::= expression NK_REM expression", + /* 318 */ "expression ::= column_reference NK_ARROW NK_STRING", + /* 319 */ "expression_list ::= expression", + /* 320 */ "expression_list ::= expression_list NK_COMMA expression", + /* 321 */ "column_reference ::= column_name", + /* 322 */ "column_reference ::= table_name NK_DOT column_name", + /* 323 */ "pseudo_column ::= ROWTS", + /* 324 */ "pseudo_column ::= TBNAME", + /* 325 */ "pseudo_column ::= table_name NK_DOT TBNAME", + /* 326 */ "pseudo_column ::= QSTARTTS", + /* 327 */ "pseudo_column ::= QENDTS", + /* 328 */ "pseudo_column ::= WSTARTTS", + /* 329 */ "pseudo_column ::= WENDTS", + /* 330 */ "pseudo_column ::= WDURATION", + /* 331 */ "function_expression ::= function_name NK_LP expression_list NK_RP", + /* 332 */ "function_expression ::= star_func NK_LP star_func_para_list NK_RP", + /* 333 */ "function_expression ::= CAST NK_LP expression AS type_name NK_RP", + /* 334 */ "function_expression ::= literal_func", + /* 335 */ "literal_func ::= noarg_func NK_LP NK_RP", + /* 336 */ "literal_func ::= NOW", + /* 337 */ "noarg_func ::= NOW", + /* 338 */ "noarg_func ::= TODAY", + /* 339 */ "noarg_func ::= TIMEZONE", + /* 340 */ "star_func ::= COUNT", + /* 341 */ "star_func ::= FIRST", + /* 342 */ "star_func ::= LAST", + /* 343 */ "star_func ::= LAST_ROW", + /* 344 */ "star_func_para_list ::= NK_STAR", + /* 345 */ "star_func_para_list ::= other_para_list", + /* 346 */ "other_para_list ::= star_func_para", + /* 347 */ "other_para_list ::= other_para_list NK_COMMA star_func_para", + /* 348 */ "star_func_para ::= expression", + /* 349 */ "star_func_para ::= table_name NK_DOT NK_STAR", + /* 350 */ "predicate ::= expression compare_op expression", + /* 351 */ "predicate ::= expression BETWEEN expression AND expression", + /* 352 */ "predicate ::= expression NOT BETWEEN expression AND expression", + /* 353 */ "predicate ::= expression IS NULL", + /* 354 */ "predicate ::= expression IS NOT NULL", + /* 355 */ "predicate ::= expression in_op in_predicate_value", + /* 356 */ "compare_op ::= NK_LT", + /* 357 */ "compare_op ::= NK_GT", + /* 358 */ "compare_op ::= NK_LE", + /* 359 */ "compare_op ::= NK_GE", + /* 360 */ "compare_op ::= NK_NE", + /* 361 */ "compare_op ::= NK_EQ", + /* 362 */ "compare_op ::= LIKE", + /* 363 */ "compare_op ::= NOT LIKE", + /* 364 */ "compare_op ::= MATCH", + /* 365 */ "compare_op ::= NMATCH", + /* 366 */ "compare_op ::= CONTAINS", + /* 367 */ "in_op ::= IN", + /* 368 */ "in_op ::= NOT IN", + /* 369 */ "in_predicate_value ::= NK_LP expression_list NK_RP", + /* 370 */ "boolean_value_expression ::= boolean_primary", + /* 371 */ "boolean_value_expression ::= NOT boolean_primary", + /* 372 */ "boolean_value_expression ::= boolean_value_expression OR boolean_value_expression", + /* 373 */ "boolean_value_expression ::= boolean_value_expression AND boolean_value_expression", + /* 374 */ "boolean_primary ::= predicate", + /* 375 */ "boolean_primary ::= NK_LP boolean_value_expression NK_RP", + /* 376 */ "common_expression ::= expression", + /* 377 */ "common_expression ::= boolean_value_expression", + /* 378 */ "from_clause ::= FROM table_reference_list", + /* 379 */ "table_reference_list ::= table_reference", + /* 380 */ "table_reference_list ::= table_reference_list NK_COMMA table_reference", + /* 381 */ "table_reference ::= table_primary", + /* 382 */ "table_reference ::= joined_table", + /* 383 */ "table_primary ::= table_name alias_opt", + /* 384 */ "table_primary ::= db_name NK_DOT table_name alias_opt", + /* 385 */ "table_primary ::= subquery alias_opt", + /* 386 */ "table_primary ::= parenthesized_joined_table", + /* 387 */ "alias_opt ::=", + /* 388 */ "alias_opt ::= table_alias", + /* 389 */ "alias_opt ::= AS table_alias", + /* 390 */ "parenthesized_joined_table ::= NK_LP joined_table NK_RP", + /* 391 */ "parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP", + /* 392 */ "joined_table ::= table_reference join_type JOIN table_reference ON search_condition", + /* 393 */ "join_type ::=", + /* 394 */ "join_type ::= INNER", + /* 395 */ "query_specification ::= SELECT set_quantifier_opt select_list from_clause where_clause_opt partition_by_clause_opt twindow_clause_opt group_by_clause_opt having_clause_opt", + /* 396 */ "set_quantifier_opt ::=", + /* 397 */ "set_quantifier_opt ::= DISTINCT", + /* 398 */ "set_quantifier_opt ::= ALL", + /* 399 */ "select_list ::= NK_STAR", + /* 400 */ "select_list ::= select_sublist", + /* 401 */ "select_sublist ::= select_item", + /* 402 */ "select_sublist ::= select_sublist NK_COMMA select_item", + /* 403 */ "select_item ::= common_expression", + /* 404 */ "select_item ::= common_expression column_alias", + /* 405 */ "select_item ::= common_expression AS column_alias", + /* 406 */ "select_item ::= table_name NK_DOT NK_STAR", + /* 407 */ "where_clause_opt ::=", + /* 408 */ "where_clause_opt ::= WHERE search_condition", + /* 409 */ "partition_by_clause_opt ::=", + /* 410 */ "partition_by_clause_opt ::= PARTITION BY expression_list", + /* 411 */ "twindow_clause_opt ::=", + /* 412 */ "twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP", + /* 413 */ "twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP", + /* 414 */ "twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt", + /* 415 */ "twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt", + /* 416 */ "sliding_opt ::=", + /* 417 */ "sliding_opt ::= SLIDING NK_LP duration_literal NK_RP", + /* 418 */ "fill_opt ::=", + /* 419 */ "fill_opt ::= FILL NK_LP fill_mode NK_RP", + /* 420 */ "fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP", + /* 421 */ "fill_mode ::= NONE", + /* 422 */ "fill_mode ::= PREV", + /* 423 */ "fill_mode ::= NULL", + /* 424 */ "fill_mode ::= LINEAR", + /* 425 */ "fill_mode ::= NEXT", + /* 426 */ "group_by_clause_opt ::=", + /* 427 */ "group_by_clause_opt ::= GROUP BY group_by_list", + /* 428 */ "group_by_list ::= expression", + /* 429 */ "group_by_list ::= group_by_list NK_COMMA expression", + /* 430 */ "having_clause_opt ::=", + /* 431 */ "having_clause_opt ::= HAVING search_condition", + /* 432 */ "query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt", + /* 433 */ "query_expression_body ::= query_primary", + /* 434 */ "query_expression_body ::= query_expression_body UNION ALL query_expression_body", + /* 435 */ "query_expression_body ::= query_expression_body UNION query_expression_body", + /* 436 */ "query_primary ::= query_specification", + /* 437 */ "query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP", + /* 438 */ "order_by_clause_opt ::=", + /* 439 */ "order_by_clause_opt ::= ORDER BY sort_specification_list", + /* 440 */ "slimit_clause_opt ::=", + /* 441 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER", + /* 442 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER", + /* 443 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER", + /* 444 */ "limit_clause_opt ::=", + /* 445 */ "limit_clause_opt ::= LIMIT NK_INTEGER", + /* 446 */ "limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER", + /* 447 */ "limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER", + /* 448 */ "subquery ::= NK_LP query_expression NK_RP", + /* 449 */ "search_condition ::= common_expression", + /* 450 */ "sort_specification_list ::= sort_specification", + /* 451 */ "sort_specification_list ::= sort_specification_list NK_COMMA sort_specification", + /* 452 */ "sort_specification ::= expression ordering_specification_opt null_ordering_opt", + /* 453 */ "ordering_specification_opt ::=", + /* 454 */ "ordering_specification_opt ::= ASC", + /* 455 */ "ordering_specification_opt ::= DESC", + /* 456 */ "null_ordering_opt ::=", + /* 457 */ "null_ordering_opt ::= NULLS FIRST", + /* 458 */ "null_ordering_opt ::= NULLS LAST", }; #endif /* NDEBUG */ @@ -2109,174 +2099,174 @@ static void yy_destructor( */ /********* Begin destructor definitions ***************************************/ /* Default NON-TERMINAL Destructor */ - case 240: /* cmd */ - case 243: /* literal */ - case 254: /* db_options */ - case 256: /* alter_db_options */ - case 261: /* retention */ - case 262: /* full_table_name */ - case 265: /* table_options */ - case 269: /* alter_table_clause */ - case 270: /* alter_table_options */ - case 273: /* signed_literal */ - case 274: /* create_subtable_clause */ - case 277: /* drop_table_clause */ - case 280: /* column_def */ - case 283: /* col_name */ - case 284: /* db_name_cond_opt */ - case 285: /* like_pattern_opt */ - case 286: /* table_name_cond */ - case 287: /* from_db_opt */ - case 288: /* func_name */ + case 239: /* cmd */ + case 242: /* literal */ + case 252: /* db_options */ + case 254: /* alter_db_options */ + case 259: /* retention */ + case 260: /* full_table_name */ + case 263: /* table_options */ + case 267: /* alter_table_clause */ + case 268: /* alter_table_options */ + case 271: /* signed_literal */ + case 272: /* create_subtable_clause */ + case 275: /* drop_table_clause */ + case 278: /* column_def */ + case 282: /* duration_literal */ + case 283: /* rollup_func_name */ + case 285: /* col_name */ + case 286: /* db_name_cond_opt */ + case 287: /* like_pattern_opt */ + case 288: /* table_name_cond */ + case 289: /* from_db_opt */ case 291: /* index_options */ - case 293: /* duration_literal */ - case 294: /* sliding_opt */ - case 295: /* func */ - case 298: /* query_expression */ - case 301: /* explain_options */ - case 305: /* stream_options */ - case 306: /* into_opt */ - case 308: /* where_clause_opt */ - case 309: /* signed */ - case 310: /* literal_func */ - case 313: /* expression */ - case 314: /* pseudo_column */ - case 315: /* column_reference */ - case 316: /* function_expression */ - case 317: /* subquery */ - case 322: /* star_func_para */ - case 323: /* predicate */ - case 326: /* in_predicate_value */ - case 327: /* boolean_value_expression */ - case 328: /* boolean_primary */ - case 329: /* common_expression */ - case 330: /* from_clause */ - case 331: /* table_reference_list */ - case 332: /* table_reference */ - case 333: /* table_primary */ - case 334: /* joined_table */ - case 336: /* parenthesized_joined_table */ - case 338: /* search_condition */ - case 339: /* query_specification */ - case 343: /* twindow_clause_opt */ - case 345: /* having_clause_opt */ - case 347: /* select_item */ - case 348: /* fill_opt */ - case 351: /* query_expression_body */ - case 353: /* slimit_clause_opt */ - case 354: /* limit_clause_opt */ - case 355: /* query_primary */ - case 357: /* sort_specification */ + case 293: /* sliding_opt */ + case 294: /* func */ + case 297: /* query_expression */ + case 300: /* explain_options */ + case 304: /* stream_options */ + case 305: /* into_opt */ + case 307: /* where_clause_opt */ + case 308: /* signed */ + case 309: /* literal_func */ + case 312: /* expression */ + case 313: /* pseudo_column */ + case 314: /* column_reference */ + case 315: /* function_expression */ + case 316: /* subquery */ + case 321: /* star_func_para */ + case 322: /* predicate */ + case 325: /* in_predicate_value */ + case 326: /* boolean_value_expression */ + case 327: /* boolean_primary */ + case 328: /* common_expression */ + case 329: /* from_clause */ + case 330: /* table_reference_list */ + case 331: /* table_reference */ + case 332: /* table_primary */ + case 333: /* joined_table */ + case 335: /* parenthesized_joined_table */ + case 337: /* search_condition */ + case 338: /* query_specification */ + case 342: /* twindow_clause_opt */ + case 344: /* having_clause_opt */ + case 346: /* select_item */ + case 347: /* fill_opt */ + case 350: /* query_expression_body */ + case 352: /* slimit_clause_opt */ + case 353: /* limit_clause_opt */ + case 354: /* query_primary */ + case 356: /* sort_specification */ { - nodesDestroyNode((yypminor->yy632)); + nodesDestroyNode((yypminor->yy674)); } break; - case 241: /* account_options */ - case 242: /* alter_account_options */ - case 244: /* alter_account_option */ - case 303: /* bufsize_opt */ + case 240: /* account_options */ + case 241: /* alter_account_options */ + case 243: /* alter_account_option */ + case 302: /* bufsize_opt */ { } break; - case 245: /* user_name */ - case 247: /* priv_level */ - case 250: /* db_name */ - case 251: /* dnode_endpoint */ - case 252: /* dnode_host_name */ - case 271: /* column_name */ - case 279: /* table_name */ - case 289: /* function_name */ + case 244: /* user_name */ + case 246: /* priv_level */ + case 249: /* db_name */ + case 250: /* dnode_endpoint */ + case 269: /* column_name */ + case 277: /* table_name */ + case 284: /* function_name */ case 290: /* index_name */ - case 297: /* topic_name */ - case 299: /* cgroup_name */ - case 304: /* stream_name */ - case 311: /* table_alias */ - case 312: /* column_alias */ - case 318: /* star_func */ - case 320: /* noarg_func */ - case 335: /* alias_opt */ + case 296: /* topic_name */ + case 298: /* cgroup_name */ + case 303: /* stream_name */ + case 310: /* table_alias */ + case 311: /* column_alias */ + case 317: /* star_func */ + case 319: /* noarg_func */ + case 334: /* alias_opt */ { } break; - case 246: /* privileges */ - case 248: /* priv_type_list */ - case 249: /* priv_type */ + case 245: /* privileges */ + case 247: /* priv_type_list */ + case 248: /* priv_type */ { } break; - case 253: /* not_exists_opt */ - case 255: /* exists_opt */ - case 300: /* analyze_opt */ - case 302: /* agg_func_opt */ - case 340: /* set_quantifier_opt */ + case 251: /* not_exists_opt */ + case 253: /* exists_opt */ + case 299: /* analyze_opt */ + case 301: /* agg_func_opt */ + case 339: /* set_quantifier_opt */ { } break; - case 257: /* integer_list */ - case 258: /* variable_list */ - case 259: /* retention_list */ - case 263: /* column_def_list */ - case 264: /* tags_def_opt */ - case 266: /* multi_create_clause */ - case 267: /* tags_def */ - case 268: /* multi_drop_clause */ - case 275: /* specific_tags_opt */ - case 276: /* literal_list */ - case 278: /* col_name_list */ - case 281: /* func_name_list */ + case 255: /* integer_list */ + case 256: /* variable_list */ + case 257: /* retention_list */ + case 261: /* column_def_list */ + case 262: /* tags_def_opt */ + case 264: /* multi_create_clause */ + case 265: /* tags_def */ + case 266: /* multi_drop_clause */ + case 273: /* specific_tags_opt */ + case 274: /* literal_list */ + case 276: /* col_name_list */ + case 279: /* duration_list */ + case 280: /* rollup_func_list */ case 292: /* func_list */ - case 296: /* expression_list */ - case 307: /* dnode_list */ - case 319: /* star_func_para_list */ - case 321: /* other_para_list */ - case 341: /* select_list */ - case 342: /* partition_by_clause_opt */ - case 344: /* group_by_clause_opt */ - case 346: /* select_sublist */ - case 350: /* group_by_list */ - case 352: /* order_by_clause_opt */ - case 356: /* sort_specification_list */ + case 295: /* expression_list */ + case 306: /* dnode_list */ + case 318: /* star_func_para_list */ + case 320: /* other_para_list */ + case 340: /* select_list */ + case 341: /* partition_by_clause_opt */ + case 343: /* group_by_clause_opt */ + case 345: /* select_sublist */ + case 349: /* group_by_list */ + case 351: /* order_by_clause_opt */ + case 355: /* sort_specification_list */ { - nodesDestroyList((yypminor->yy424)); + nodesDestroyList((yypminor->yy530)); } break; - case 260: /* alter_db_option */ - case 282: /* alter_table_option */ + case 258: /* alter_db_option */ + case 281: /* alter_table_option */ { } break; - case 272: /* type_name */ + case 270: /* type_name */ { } break; - case 324: /* compare_op */ - case 325: /* in_op */ + case 323: /* compare_op */ + case 324: /* in_op */ { } break; - case 337: /* join_type */ + case 336: /* join_type */ { } break; - case 349: /* fill_mode */ + case 348: /* fill_mode */ { } break; - case 358: /* ordering_specification_opt */ + case 357: /* ordering_specification_opt */ { } break; - case 359: /* null_ordering_opt */ + case 358: /* null_ordering_opt */ { } @@ -2575,460 +2565,465 @@ static const struct { YYCODETYPE lhs; /* Symbol on the left-hand side of the rule */ signed char nrhs; /* Negative of the number of RHS symbols in the rule */ } yyRuleInfo[] = { - { 240, -6 }, /* (0) cmd ::= CREATE ACCOUNT NK_ID PASS NK_STRING account_options */ - { 240, -4 }, /* (1) cmd ::= ALTER ACCOUNT NK_ID alter_account_options */ - { 241, 0 }, /* (2) account_options ::= */ - { 241, -3 }, /* (3) account_options ::= account_options PPS literal */ - { 241, -3 }, /* (4) account_options ::= account_options TSERIES literal */ - { 241, -3 }, /* (5) account_options ::= account_options STORAGE literal */ - { 241, -3 }, /* (6) account_options ::= account_options STREAMS literal */ - { 241, -3 }, /* (7) account_options ::= account_options QTIME literal */ - { 241, -3 }, /* (8) account_options ::= account_options DBS literal */ - { 241, -3 }, /* (9) account_options ::= account_options USERS literal */ - { 241, -3 }, /* (10) account_options ::= account_options CONNS literal */ - { 241, -3 }, /* (11) account_options ::= account_options STATE literal */ - { 242, -1 }, /* (12) alter_account_options ::= alter_account_option */ - { 242, -2 }, /* (13) alter_account_options ::= alter_account_options alter_account_option */ - { 244, -2 }, /* (14) alter_account_option ::= PASS literal */ - { 244, -2 }, /* (15) alter_account_option ::= PPS literal */ - { 244, -2 }, /* (16) alter_account_option ::= TSERIES literal */ - { 244, -2 }, /* (17) alter_account_option ::= STORAGE literal */ - { 244, -2 }, /* (18) alter_account_option ::= STREAMS literal */ - { 244, -2 }, /* (19) alter_account_option ::= QTIME literal */ - { 244, -2 }, /* (20) alter_account_option ::= DBS literal */ - { 244, -2 }, /* (21) alter_account_option ::= USERS literal */ - { 244, -2 }, /* (22) alter_account_option ::= CONNS literal */ - { 244, -2 }, /* (23) alter_account_option ::= STATE literal */ - { 240, -5 }, /* (24) cmd ::= CREATE USER user_name PASS NK_STRING */ - { 240, -5 }, /* (25) cmd ::= ALTER USER user_name PASS NK_STRING */ - { 240, -5 }, /* (26) cmd ::= ALTER USER user_name PRIVILEGE NK_STRING */ - { 240, -3 }, /* (27) cmd ::= DROP USER user_name */ - { 240, -6 }, /* (28) cmd ::= GRANT privileges ON priv_level TO user_name */ - { 240, -6 }, /* (29) cmd ::= REVOKE privileges ON priv_level FROM user_name */ - { 246, -1 }, /* (30) privileges ::= ALL */ - { 246, -1 }, /* (31) privileges ::= priv_type_list */ - { 248, -1 }, /* (32) priv_type_list ::= priv_type */ - { 248, -3 }, /* (33) priv_type_list ::= priv_type_list NK_COMMA priv_type */ - { 249, -1 }, /* (34) priv_type ::= READ */ - { 249, -1 }, /* (35) priv_type ::= WRITE */ - { 247, -3 }, /* (36) priv_level ::= NK_STAR NK_DOT NK_STAR */ - { 247, -3 }, /* (37) priv_level ::= db_name NK_DOT NK_STAR */ - { 240, -3 }, /* (38) cmd ::= CREATE DNODE dnode_endpoint */ - { 240, -5 }, /* (39) cmd ::= CREATE DNODE dnode_host_name PORT NK_INTEGER */ - { 240, -3 }, /* (40) cmd ::= DROP DNODE NK_INTEGER */ - { 240, -3 }, /* (41) cmd ::= DROP DNODE dnode_endpoint */ - { 240, -4 }, /* (42) cmd ::= ALTER DNODE NK_INTEGER NK_STRING */ - { 240, -5 }, /* (43) cmd ::= ALTER DNODE NK_INTEGER NK_STRING NK_STRING */ - { 240, -4 }, /* (44) cmd ::= ALTER ALL DNODES NK_STRING */ - { 240, -5 }, /* (45) cmd ::= ALTER ALL DNODES NK_STRING NK_STRING */ - { 251, -1 }, /* (46) dnode_endpoint ::= NK_STRING */ - { 252, -1 }, /* (47) dnode_host_name ::= NK_ID */ - { 252, -1 }, /* (48) dnode_host_name ::= NK_IPTOKEN */ - { 240, -3 }, /* (49) cmd ::= ALTER LOCAL NK_STRING */ - { 240, -4 }, /* (50) cmd ::= ALTER LOCAL NK_STRING NK_STRING */ - { 240, -5 }, /* (51) cmd ::= CREATE QNODE ON DNODE NK_INTEGER */ - { 240, -5 }, /* (52) cmd ::= DROP QNODE ON DNODE NK_INTEGER */ - { 240, -5 }, /* (53) cmd ::= CREATE BNODE ON DNODE NK_INTEGER */ - { 240, -5 }, /* (54) cmd ::= DROP BNODE ON DNODE NK_INTEGER */ - { 240, -5 }, /* (55) cmd ::= CREATE SNODE ON DNODE NK_INTEGER */ - { 240, -5 }, /* (56) cmd ::= DROP SNODE ON DNODE NK_INTEGER */ - { 240, -5 }, /* (57) cmd ::= CREATE MNODE ON DNODE NK_INTEGER */ - { 240, -5 }, /* (58) cmd ::= DROP MNODE ON DNODE NK_INTEGER */ - { 240, -5 }, /* (59) cmd ::= CREATE DATABASE not_exists_opt db_name db_options */ - { 240, -4 }, /* (60) cmd ::= DROP DATABASE exists_opt db_name */ - { 240, -2 }, /* (61) cmd ::= USE db_name */ - { 240, -4 }, /* (62) cmd ::= ALTER DATABASE db_name alter_db_options */ - { 253, -3 }, /* (63) not_exists_opt ::= IF NOT EXISTS */ - { 253, 0 }, /* (64) not_exists_opt ::= */ - { 255, -2 }, /* (65) exists_opt ::= IF EXISTS */ - { 255, 0 }, /* (66) exists_opt ::= */ - { 254, 0 }, /* (67) db_options ::= */ - { 254, -3 }, /* (68) db_options ::= db_options BUFFER NK_INTEGER */ - { 254, -3 }, /* (69) db_options ::= db_options CACHELAST NK_INTEGER */ - { 254, -3 }, /* (70) db_options ::= db_options COMP NK_INTEGER */ - { 254, -3 }, /* (71) db_options ::= db_options DURATION NK_INTEGER */ - { 254, -3 }, /* (72) db_options ::= db_options DURATION NK_VARIABLE */ - { 254, -3 }, /* (73) db_options ::= db_options FSYNC NK_INTEGER */ - { 254, -3 }, /* (74) db_options ::= db_options MAXROWS NK_INTEGER */ - { 254, -3 }, /* (75) db_options ::= db_options MINROWS NK_INTEGER */ - { 254, -3 }, /* (76) db_options ::= db_options KEEP integer_list */ - { 254, -3 }, /* (77) db_options ::= db_options KEEP variable_list */ - { 254, -3 }, /* (78) db_options ::= db_options PAGES NK_INTEGER */ - { 254, -3 }, /* (79) db_options ::= db_options PAGESIZE NK_INTEGER */ - { 254, -3 }, /* (80) db_options ::= db_options PRECISION NK_STRING */ - { 254, -3 }, /* (81) db_options ::= db_options REPLICA NK_INTEGER */ - { 254, -3 }, /* (82) db_options ::= db_options STRICT NK_INTEGER */ - { 254, -3 }, /* (83) db_options ::= db_options WAL NK_INTEGER */ - { 254, -3 }, /* (84) db_options ::= db_options VGROUPS NK_INTEGER */ - { 254, -3 }, /* (85) db_options ::= db_options SINGLE_STABLE NK_INTEGER */ - { 254, -3 }, /* (86) db_options ::= db_options RETENTIONS retention_list */ - { 254, -3 }, /* (87) db_options ::= db_options SCHEMALESS NK_INTEGER */ - { 256, -1 }, /* (88) alter_db_options ::= alter_db_option */ - { 256, -2 }, /* (89) alter_db_options ::= alter_db_options alter_db_option */ - { 260, -2 }, /* (90) alter_db_option ::= BUFFER NK_INTEGER */ - { 260, -2 }, /* (91) alter_db_option ::= CACHELAST NK_INTEGER */ - { 260, -2 }, /* (92) alter_db_option ::= FSYNC NK_INTEGER */ - { 260, -2 }, /* (93) alter_db_option ::= KEEP integer_list */ - { 260, -2 }, /* (94) alter_db_option ::= KEEP variable_list */ - { 260, -2 }, /* (95) alter_db_option ::= PAGES NK_INTEGER */ - { 260, -2 }, /* (96) alter_db_option ::= REPLICA NK_INTEGER */ - { 260, -2 }, /* (97) alter_db_option ::= STRICT NK_INTEGER */ - { 260, -2 }, /* (98) alter_db_option ::= WAL NK_INTEGER */ - { 257, -1 }, /* (99) integer_list ::= NK_INTEGER */ - { 257, -3 }, /* (100) integer_list ::= integer_list NK_COMMA NK_INTEGER */ - { 258, -1 }, /* (101) variable_list ::= NK_VARIABLE */ - { 258, -3 }, /* (102) variable_list ::= variable_list NK_COMMA NK_VARIABLE */ - { 259, -1 }, /* (103) retention_list ::= retention */ - { 259, -3 }, /* (104) retention_list ::= retention_list NK_COMMA retention */ - { 261, -3 }, /* (105) retention ::= NK_VARIABLE NK_COLON NK_VARIABLE */ - { 240, -9 }, /* (106) cmd ::= CREATE TABLE not_exists_opt full_table_name NK_LP column_def_list NK_RP tags_def_opt table_options */ - { 240, -3 }, /* (107) cmd ::= CREATE TABLE multi_create_clause */ - { 240, -9 }, /* (108) cmd ::= CREATE STABLE not_exists_opt full_table_name NK_LP column_def_list NK_RP tags_def table_options */ - { 240, -3 }, /* (109) cmd ::= DROP TABLE multi_drop_clause */ - { 240, -4 }, /* (110) cmd ::= DROP STABLE exists_opt full_table_name */ - { 240, -3 }, /* (111) cmd ::= ALTER TABLE alter_table_clause */ - { 240, -3 }, /* (112) cmd ::= ALTER STABLE alter_table_clause */ - { 269, -2 }, /* (113) alter_table_clause ::= full_table_name alter_table_options */ - { 269, -5 }, /* (114) alter_table_clause ::= full_table_name ADD COLUMN column_name type_name */ - { 269, -4 }, /* (115) alter_table_clause ::= full_table_name DROP COLUMN column_name */ - { 269, -5 }, /* (116) alter_table_clause ::= full_table_name MODIFY COLUMN column_name type_name */ - { 269, -5 }, /* (117) alter_table_clause ::= full_table_name RENAME COLUMN column_name column_name */ - { 269, -5 }, /* (118) alter_table_clause ::= full_table_name ADD TAG column_name type_name */ - { 269, -4 }, /* (119) alter_table_clause ::= full_table_name DROP TAG column_name */ - { 269, -5 }, /* (120) alter_table_clause ::= full_table_name MODIFY TAG column_name type_name */ - { 269, -5 }, /* (121) alter_table_clause ::= full_table_name RENAME TAG column_name column_name */ - { 269, -6 }, /* (122) alter_table_clause ::= full_table_name SET TAG column_name NK_EQ signed_literal */ - { 266, -1 }, /* (123) multi_create_clause ::= create_subtable_clause */ - { 266, -2 }, /* (124) multi_create_clause ::= multi_create_clause create_subtable_clause */ - { 274, -10 }, /* (125) create_subtable_clause ::= not_exists_opt full_table_name USING full_table_name specific_tags_opt TAGS NK_LP literal_list NK_RP table_options */ - { 268, -1 }, /* (126) multi_drop_clause ::= drop_table_clause */ - { 268, -2 }, /* (127) multi_drop_clause ::= multi_drop_clause drop_table_clause */ - { 277, -2 }, /* (128) drop_table_clause ::= exists_opt full_table_name */ - { 275, 0 }, /* (129) specific_tags_opt ::= */ - { 275, -3 }, /* (130) specific_tags_opt ::= NK_LP col_name_list NK_RP */ - { 262, -1 }, /* (131) full_table_name ::= table_name */ - { 262, -3 }, /* (132) full_table_name ::= db_name NK_DOT table_name */ - { 263, -1 }, /* (133) column_def_list ::= column_def */ - { 263, -3 }, /* (134) column_def_list ::= column_def_list NK_COMMA column_def */ - { 280, -2 }, /* (135) column_def ::= column_name type_name */ - { 280, -4 }, /* (136) column_def ::= column_name type_name COMMENT NK_STRING */ - { 272, -1 }, /* (137) type_name ::= BOOL */ - { 272, -1 }, /* (138) type_name ::= TINYINT */ - { 272, -1 }, /* (139) type_name ::= SMALLINT */ - { 272, -1 }, /* (140) type_name ::= INT */ - { 272, -1 }, /* (141) type_name ::= INTEGER */ - { 272, -1 }, /* (142) type_name ::= BIGINT */ - { 272, -1 }, /* (143) type_name ::= FLOAT */ - { 272, -1 }, /* (144) type_name ::= DOUBLE */ - { 272, -4 }, /* (145) type_name ::= BINARY NK_LP NK_INTEGER NK_RP */ - { 272, -1 }, /* (146) type_name ::= TIMESTAMP */ - { 272, -4 }, /* (147) type_name ::= NCHAR NK_LP NK_INTEGER NK_RP */ - { 272, -2 }, /* (148) type_name ::= TINYINT UNSIGNED */ - { 272, -2 }, /* (149) type_name ::= SMALLINT UNSIGNED */ - { 272, -2 }, /* (150) type_name ::= INT UNSIGNED */ - { 272, -2 }, /* (151) type_name ::= BIGINT UNSIGNED */ - { 272, -1 }, /* (152) type_name ::= JSON */ - { 272, -4 }, /* (153) type_name ::= VARCHAR NK_LP NK_INTEGER NK_RP */ - { 272, -1 }, /* (154) type_name ::= MEDIUMBLOB */ - { 272, -1 }, /* (155) type_name ::= BLOB */ - { 272, -4 }, /* (156) type_name ::= VARBINARY NK_LP NK_INTEGER NK_RP */ - { 272, -1 }, /* (157) type_name ::= DECIMAL */ - { 272, -4 }, /* (158) type_name ::= DECIMAL NK_LP NK_INTEGER NK_RP */ - { 272, -6 }, /* (159) type_name ::= DECIMAL NK_LP NK_INTEGER NK_COMMA NK_INTEGER NK_RP */ - { 264, 0 }, /* (160) tags_def_opt ::= */ - { 264, -1 }, /* (161) tags_def_opt ::= tags_def */ - { 267, -4 }, /* (162) tags_def ::= TAGS NK_LP column_def_list NK_RP */ - { 265, 0 }, /* (163) table_options ::= */ - { 265, -3 }, /* (164) table_options ::= table_options COMMENT NK_STRING */ - { 265, -3 }, /* (165) table_options ::= table_options FILE_FACTOR NK_FLOAT */ - { 265, -5 }, /* (166) table_options ::= table_options ROLLUP NK_LP func_name_list NK_RP */ - { 265, -3 }, /* (167) table_options ::= table_options TTL NK_INTEGER */ - { 265, -5 }, /* (168) table_options ::= table_options SMA NK_LP col_name_list NK_RP */ - { 270, -1 }, /* (169) alter_table_options ::= alter_table_option */ - { 270, -2 }, /* (170) alter_table_options ::= alter_table_options alter_table_option */ - { 282, -2 }, /* (171) alter_table_option ::= COMMENT NK_STRING */ - { 282, -2 }, /* (172) alter_table_option ::= TTL NK_INTEGER */ - { 278, -1 }, /* (173) col_name_list ::= col_name */ - { 278, -3 }, /* (174) col_name_list ::= col_name_list NK_COMMA col_name */ - { 283, -1 }, /* (175) col_name ::= column_name */ - { 240, -2 }, /* (176) cmd ::= SHOW DNODES */ - { 240, -2 }, /* (177) cmd ::= SHOW USERS */ - { 240, -2 }, /* (178) cmd ::= SHOW DATABASES */ - { 240, -4 }, /* (179) cmd ::= SHOW db_name_cond_opt TABLES like_pattern_opt */ - { 240, -4 }, /* (180) cmd ::= SHOW db_name_cond_opt STABLES like_pattern_opt */ - { 240, -3 }, /* (181) cmd ::= SHOW db_name_cond_opt VGROUPS */ - { 240, -2 }, /* (182) cmd ::= SHOW MNODES */ - { 240, -2 }, /* (183) cmd ::= SHOW MODULES */ - { 240, -2 }, /* (184) cmd ::= SHOW QNODES */ - { 240, -2 }, /* (185) cmd ::= SHOW FUNCTIONS */ - { 240, -5 }, /* (186) cmd ::= SHOW INDEXES FROM table_name_cond from_db_opt */ - { 240, -2 }, /* (187) cmd ::= SHOW STREAMS */ - { 240, -2 }, /* (188) cmd ::= SHOW ACCOUNTS */ - { 240, -2 }, /* (189) cmd ::= SHOW APPS */ - { 240, -2 }, /* (190) cmd ::= SHOW CONNECTIONS */ - { 240, -2 }, /* (191) cmd ::= SHOW LICENCE */ - { 240, -2 }, /* (192) cmd ::= SHOW GRANTS */ - { 240, -4 }, /* (193) cmd ::= SHOW CREATE DATABASE db_name */ - { 240, -4 }, /* (194) cmd ::= SHOW CREATE TABLE full_table_name */ - { 240, -4 }, /* (195) cmd ::= SHOW CREATE STABLE full_table_name */ - { 240, -2 }, /* (196) cmd ::= SHOW QUERIES */ - { 240, -2 }, /* (197) cmd ::= SHOW SCORES */ - { 240, -2 }, /* (198) cmd ::= SHOW TOPICS */ - { 240, -2 }, /* (199) cmd ::= SHOW VARIABLES */ - { 240, -2 }, /* (200) cmd ::= SHOW BNODES */ - { 240, -2 }, /* (201) cmd ::= SHOW SNODES */ - { 240, -2 }, /* (202) cmd ::= SHOW CLUSTER */ - { 240, -2 }, /* (203) cmd ::= SHOW TRANSACTIONS */ - { 284, 0 }, /* (204) db_name_cond_opt ::= */ - { 284, -2 }, /* (205) db_name_cond_opt ::= db_name NK_DOT */ - { 285, 0 }, /* (206) like_pattern_opt ::= */ - { 285, -2 }, /* (207) like_pattern_opt ::= LIKE NK_STRING */ - { 286, -1 }, /* (208) table_name_cond ::= table_name */ - { 287, 0 }, /* (209) from_db_opt ::= */ - { 287, -2 }, /* (210) from_db_opt ::= FROM db_name */ - { 281, -1 }, /* (211) func_name_list ::= func_name */ - { 281, -3 }, /* (212) func_name_list ::= func_name_list NK_COMMA func_name */ - { 288, -1 }, /* (213) func_name ::= function_name */ - { 240, -8 }, /* (214) cmd ::= CREATE SMA INDEX not_exists_opt index_name ON table_name index_options */ - { 240, -10 }, /* (215) cmd ::= CREATE FULLTEXT INDEX not_exists_opt index_name ON table_name NK_LP col_name_list NK_RP */ - { 240, -6 }, /* (216) cmd ::= DROP INDEX exists_opt index_name ON table_name */ - { 291, 0 }, /* (217) index_options ::= */ - { 291, -9 }, /* (218) index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_RP sliding_opt */ - { 291, -11 }, /* (219) index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt */ - { 292, -1 }, /* (220) func_list ::= func */ - { 292, -3 }, /* (221) func_list ::= func_list NK_COMMA func */ - { 295, -4 }, /* (222) func ::= function_name NK_LP expression_list NK_RP */ - { 240, -6 }, /* (223) cmd ::= CREATE TOPIC not_exists_opt topic_name AS query_expression */ - { 240, -7 }, /* (224) cmd ::= CREATE TOPIC not_exists_opt topic_name AS DATABASE db_name */ - { 240, -7 }, /* (225) cmd ::= CREATE TOPIC not_exists_opt topic_name AS STABLE full_table_name */ - { 240, -4 }, /* (226) cmd ::= DROP TOPIC exists_opt topic_name */ - { 240, -7 }, /* (227) cmd ::= DROP CONSUMER GROUP exists_opt cgroup_name ON topic_name */ - { 240, -2 }, /* (228) cmd ::= DESC full_table_name */ - { 240, -2 }, /* (229) cmd ::= DESCRIBE full_table_name */ - { 240, -3 }, /* (230) cmd ::= RESET QUERY CACHE */ - { 240, -4 }, /* (231) cmd ::= EXPLAIN analyze_opt explain_options query_expression */ - { 300, 0 }, /* (232) analyze_opt ::= */ - { 300, -1 }, /* (233) analyze_opt ::= ANALYZE */ - { 301, 0 }, /* (234) explain_options ::= */ - { 301, -3 }, /* (235) explain_options ::= explain_options VERBOSE NK_BOOL */ - { 301, -3 }, /* (236) explain_options ::= explain_options RATIO NK_FLOAT */ - { 240, -6 }, /* (237) cmd ::= COMPACT VNODES IN NK_LP integer_list NK_RP */ - { 240, -10 }, /* (238) cmd ::= CREATE agg_func_opt FUNCTION not_exists_opt function_name AS NK_STRING OUTPUTTYPE type_name bufsize_opt */ - { 240, -4 }, /* (239) cmd ::= DROP FUNCTION exists_opt function_name */ - { 302, 0 }, /* (240) agg_func_opt ::= */ - { 302, -1 }, /* (241) agg_func_opt ::= AGGREGATE */ - { 303, 0 }, /* (242) bufsize_opt ::= */ - { 303, -2 }, /* (243) bufsize_opt ::= BUFSIZE NK_INTEGER */ - { 240, -8 }, /* (244) cmd ::= CREATE STREAM not_exists_opt stream_name stream_options into_opt AS query_expression */ - { 240, -4 }, /* (245) cmd ::= DROP STREAM exists_opt stream_name */ - { 306, 0 }, /* (246) into_opt ::= */ - { 306, -2 }, /* (247) into_opt ::= INTO full_table_name */ - { 305, 0 }, /* (248) stream_options ::= */ - { 305, -3 }, /* (249) stream_options ::= stream_options TRIGGER AT_ONCE */ - { 305, -3 }, /* (250) stream_options ::= stream_options TRIGGER WINDOW_CLOSE */ - { 305, -4 }, /* (251) stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal */ - { 305, -3 }, /* (252) stream_options ::= stream_options WATERMARK duration_literal */ - { 240, -3 }, /* (253) cmd ::= KILL CONNECTION NK_INTEGER */ - { 240, -3 }, /* (254) cmd ::= KILL QUERY NK_STRING */ - { 240, -3 }, /* (255) cmd ::= KILL TRANSACTION NK_INTEGER */ - { 240, -2 }, /* (256) cmd ::= BALANCE VGROUP */ - { 240, -4 }, /* (257) cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER */ - { 240, -4 }, /* (258) cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list */ - { 240, -3 }, /* (259) cmd ::= SPLIT VGROUP NK_INTEGER */ - { 307, -2 }, /* (260) dnode_list ::= DNODE NK_INTEGER */ - { 307, -3 }, /* (261) dnode_list ::= dnode_list DNODE NK_INTEGER */ - { 240, -3 }, /* (262) cmd ::= SYNCDB db_name REPLICA */ - { 240, -4 }, /* (263) cmd ::= DELETE FROM full_table_name where_clause_opt */ - { 240, -1 }, /* (264) cmd ::= query_expression */ - { 243, -1 }, /* (265) literal ::= NK_INTEGER */ - { 243, -1 }, /* (266) literal ::= NK_FLOAT */ - { 243, -1 }, /* (267) literal ::= NK_STRING */ - { 243, -1 }, /* (268) literal ::= NK_BOOL */ - { 243, -2 }, /* (269) literal ::= TIMESTAMP NK_STRING */ - { 243, -1 }, /* (270) literal ::= duration_literal */ - { 243, -1 }, /* (271) literal ::= NULL */ - { 243, -1 }, /* (272) literal ::= NK_QUESTION */ - { 293, -1 }, /* (273) duration_literal ::= NK_VARIABLE */ - { 309, -1 }, /* (274) signed ::= NK_INTEGER */ - { 309, -2 }, /* (275) signed ::= NK_PLUS NK_INTEGER */ - { 309, -2 }, /* (276) signed ::= NK_MINUS NK_INTEGER */ - { 309, -1 }, /* (277) signed ::= NK_FLOAT */ - { 309, -2 }, /* (278) signed ::= NK_PLUS NK_FLOAT */ - { 309, -2 }, /* (279) signed ::= NK_MINUS NK_FLOAT */ - { 273, -1 }, /* (280) signed_literal ::= signed */ - { 273, -1 }, /* (281) signed_literal ::= NK_STRING */ - { 273, -1 }, /* (282) signed_literal ::= NK_BOOL */ - { 273, -2 }, /* (283) signed_literal ::= TIMESTAMP NK_STRING */ - { 273, -1 }, /* (284) signed_literal ::= duration_literal */ - { 273, -1 }, /* (285) signed_literal ::= NULL */ - { 273, -1 }, /* (286) signed_literal ::= literal_func */ - { 276, -1 }, /* (287) literal_list ::= signed_literal */ - { 276, -3 }, /* (288) literal_list ::= literal_list NK_COMMA signed_literal */ - { 250, -1 }, /* (289) db_name ::= NK_ID */ - { 279, -1 }, /* (290) table_name ::= NK_ID */ - { 271, -1 }, /* (291) column_name ::= NK_ID */ - { 289, -1 }, /* (292) function_name ::= NK_ID */ - { 311, -1 }, /* (293) table_alias ::= NK_ID */ - { 312, -1 }, /* (294) column_alias ::= NK_ID */ - { 245, -1 }, /* (295) user_name ::= NK_ID */ - { 290, -1 }, /* (296) index_name ::= NK_ID */ - { 297, -1 }, /* (297) topic_name ::= NK_ID */ - { 304, -1 }, /* (298) stream_name ::= NK_ID */ - { 299, -1 }, /* (299) cgroup_name ::= NK_ID */ - { 313, -1 }, /* (300) expression ::= literal */ - { 313, -1 }, /* (301) expression ::= pseudo_column */ - { 313, -1 }, /* (302) expression ::= column_reference */ - { 313, -1 }, /* (303) expression ::= function_expression */ - { 313, -1 }, /* (304) expression ::= subquery */ - { 313, -3 }, /* (305) expression ::= NK_LP expression NK_RP */ - { 313, -2 }, /* (306) expression ::= NK_PLUS expression */ - { 313, -2 }, /* (307) expression ::= NK_MINUS expression */ - { 313, -3 }, /* (308) expression ::= expression NK_PLUS expression */ - { 313, -3 }, /* (309) expression ::= expression NK_MINUS expression */ - { 313, -3 }, /* (310) expression ::= expression NK_STAR expression */ - { 313, -3 }, /* (311) expression ::= expression NK_SLASH expression */ - { 313, -3 }, /* (312) expression ::= expression NK_REM expression */ - { 313, -3 }, /* (313) expression ::= column_reference NK_ARROW NK_STRING */ - { 296, -1 }, /* (314) expression_list ::= expression */ - { 296, -3 }, /* (315) expression_list ::= expression_list NK_COMMA expression */ - { 315, -1 }, /* (316) column_reference ::= column_name */ - { 315, -3 }, /* (317) column_reference ::= table_name NK_DOT column_name */ - { 314, -1 }, /* (318) pseudo_column ::= ROWTS */ - { 314, -1 }, /* (319) pseudo_column ::= TBNAME */ - { 314, -3 }, /* (320) pseudo_column ::= table_name NK_DOT TBNAME */ - { 314, -1 }, /* (321) pseudo_column ::= QSTARTTS */ - { 314, -1 }, /* (322) pseudo_column ::= QENDTS */ - { 314, -1 }, /* (323) pseudo_column ::= WSTARTTS */ - { 314, -1 }, /* (324) pseudo_column ::= WENDTS */ - { 314, -1 }, /* (325) pseudo_column ::= WDURATION */ - { 316, -4 }, /* (326) function_expression ::= function_name NK_LP expression_list NK_RP */ - { 316, -4 }, /* (327) function_expression ::= star_func NK_LP star_func_para_list NK_RP */ - { 316, -6 }, /* (328) function_expression ::= CAST NK_LP expression AS type_name NK_RP */ - { 316, -1 }, /* (329) function_expression ::= literal_func */ - { 310, -3 }, /* (330) literal_func ::= noarg_func NK_LP NK_RP */ - { 310, -1 }, /* (331) literal_func ::= NOW */ - { 320, -1 }, /* (332) noarg_func ::= NOW */ - { 320, -1 }, /* (333) noarg_func ::= TODAY */ - { 320, -1 }, /* (334) noarg_func ::= TIMEZONE */ - { 318, -1 }, /* (335) star_func ::= COUNT */ - { 318, -1 }, /* (336) star_func ::= FIRST */ - { 318, -1 }, /* (337) star_func ::= LAST */ - { 318, -1 }, /* (338) star_func ::= LAST_ROW */ - { 319, -1 }, /* (339) star_func_para_list ::= NK_STAR */ - { 319, -1 }, /* (340) star_func_para_list ::= other_para_list */ - { 321, -1 }, /* (341) other_para_list ::= star_func_para */ - { 321, -3 }, /* (342) other_para_list ::= other_para_list NK_COMMA star_func_para */ - { 322, -1 }, /* (343) star_func_para ::= expression */ - { 322, -3 }, /* (344) star_func_para ::= table_name NK_DOT NK_STAR */ - { 323, -3 }, /* (345) predicate ::= expression compare_op expression */ - { 323, -5 }, /* (346) predicate ::= expression BETWEEN expression AND expression */ - { 323, -6 }, /* (347) predicate ::= expression NOT BETWEEN expression AND expression */ - { 323, -3 }, /* (348) predicate ::= expression IS NULL */ - { 323, -4 }, /* (349) predicate ::= expression IS NOT NULL */ - { 323, -3 }, /* (350) predicate ::= expression in_op in_predicate_value */ - { 324, -1 }, /* (351) compare_op ::= NK_LT */ - { 324, -1 }, /* (352) compare_op ::= NK_GT */ - { 324, -1 }, /* (353) compare_op ::= NK_LE */ - { 324, -1 }, /* (354) compare_op ::= NK_GE */ - { 324, -1 }, /* (355) compare_op ::= NK_NE */ - { 324, -1 }, /* (356) compare_op ::= NK_EQ */ - { 324, -1 }, /* (357) compare_op ::= LIKE */ - { 324, -2 }, /* (358) compare_op ::= NOT LIKE */ - { 324, -1 }, /* (359) compare_op ::= MATCH */ - { 324, -1 }, /* (360) compare_op ::= NMATCH */ - { 324, -1 }, /* (361) compare_op ::= CONTAINS */ - { 325, -1 }, /* (362) in_op ::= IN */ - { 325, -2 }, /* (363) in_op ::= NOT IN */ - { 326, -3 }, /* (364) in_predicate_value ::= NK_LP expression_list NK_RP */ - { 327, -1 }, /* (365) boolean_value_expression ::= boolean_primary */ - { 327, -2 }, /* (366) boolean_value_expression ::= NOT boolean_primary */ - { 327, -3 }, /* (367) boolean_value_expression ::= boolean_value_expression OR boolean_value_expression */ - { 327, -3 }, /* (368) boolean_value_expression ::= boolean_value_expression AND boolean_value_expression */ - { 328, -1 }, /* (369) boolean_primary ::= predicate */ - { 328, -3 }, /* (370) boolean_primary ::= NK_LP boolean_value_expression NK_RP */ - { 329, -1 }, /* (371) common_expression ::= expression */ - { 329, -1 }, /* (372) common_expression ::= boolean_value_expression */ - { 330, -2 }, /* (373) from_clause ::= FROM table_reference_list */ - { 331, -1 }, /* (374) table_reference_list ::= table_reference */ - { 331, -3 }, /* (375) table_reference_list ::= table_reference_list NK_COMMA table_reference */ - { 332, -1 }, /* (376) table_reference ::= table_primary */ - { 332, -1 }, /* (377) table_reference ::= joined_table */ - { 333, -2 }, /* (378) table_primary ::= table_name alias_opt */ - { 333, -4 }, /* (379) table_primary ::= db_name NK_DOT table_name alias_opt */ - { 333, -2 }, /* (380) table_primary ::= subquery alias_opt */ - { 333, -1 }, /* (381) table_primary ::= parenthesized_joined_table */ - { 335, 0 }, /* (382) alias_opt ::= */ - { 335, -1 }, /* (383) alias_opt ::= table_alias */ - { 335, -2 }, /* (384) alias_opt ::= AS table_alias */ - { 336, -3 }, /* (385) parenthesized_joined_table ::= NK_LP joined_table NK_RP */ - { 336, -3 }, /* (386) parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP */ - { 334, -6 }, /* (387) joined_table ::= table_reference join_type JOIN table_reference ON search_condition */ - { 337, 0 }, /* (388) join_type ::= */ - { 337, -1 }, /* (389) join_type ::= INNER */ - { 339, -9 }, /* (390) query_specification ::= SELECT set_quantifier_opt select_list from_clause where_clause_opt partition_by_clause_opt twindow_clause_opt group_by_clause_opt having_clause_opt */ - { 340, 0 }, /* (391) set_quantifier_opt ::= */ - { 340, -1 }, /* (392) set_quantifier_opt ::= DISTINCT */ - { 340, -1 }, /* (393) set_quantifier_opt ::= ALL */ - { 341, -1 }, /* (394) select_list ::= NK_STAR */ - { 341, -1 }, /* (395) select_list ::= select_sublist */ - { 346, -1 }, /* (396) select_sublist ::= select_item */ - { 346, -3 }, /* (397) select_sublist ::= select_sublist NK_COMMA select_item */ - { 347, -1 }, /* (398) select_item ::= common_expression */ - { 347, -2 }, /* (399) select_item ::= common_expression column_alias */ - { 347, -3 }, /* (400) select_item ::= common_expression AS column_alias */ - { 347, -3 }, /* (401) select_item ::= table_name NK_DOT NK_STAR */ - { 308, 0 }, /* (402) where_clause_opt ::= */ - { 308, -2 }, /* (403) where_clause_opt ::= WHERE search_condition */ - { 342, 0 }, /* (404) partition_by_clause_opt ::= */ - { 342, -3 }, /* (405) partition_by_clause_opt ::= PARTITION BY expression_list */ - { 343, 0 }, /* (406) twindow_clause_opt ::= */ - { 343, -6 }, /* (407) twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP */ - { 343, -4 }, /* (408) twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP */ - { 343, -6 }, /* (409) twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt */ - { 343, -8 }, /* (410) twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt */ - { 294, 0 }, /* (411) sliding_opt ::= */ - { 294, -4 }, /* (412) sliding_opt ::= SLIDING NK_LP duration_literal NK_RP */ - { 348, 0 }, /* (413) fill_opt ::= */ - { 348, -4 }, /* (414) fill_opt ::= FILL NK_LP fill_mode NK_RP */ - { 348, -6 }, /* (415) fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP */ - { 349, -1 }, /* (416) fill_mode ::= NONE */ - { 349, -1 }, /* (417) fill_mode ::= PREV */ - { 349, -1 }, /* (418) fill_mode ::= NULL */ - { 349, -1 }, /* (419) fill_mode ::= LINEAR */ - { 349, -1 }, /* (420) fill_mode ::= NEXT */ - { 344, 0 }, /* (421) group_by_clause_opt ::= */ - { 344, -3 }, /* (422) group_by_clause_opt ::= GROUP BY group_by_list */ - { 350, -1 }, /* (423) group_by_list ::= expression */ - { 350, -3 }, /* (424) group_by_list ::= group_by_list NK_COMMA expression */ - { 345, 0 }, /* (425) having_clause_opt ::= */ - { 345, -2 }, /* (426) having_clause_opt ::= HAVING search_condition */ - { 298, -4 }, /* (427) query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt */ - { 351, -1 }, /* (428) query_expression_body ::= query_primary */ - { 351, -4 }, /* (429) query_expression_body ::= query_expression_body UNION ALL query_expression_body */ - { 351, -3 }, /* (430) query_expression_body ::= query_expression_body UNION query_expression_body */ - { 355, -1 }, /* (431) query_primary ::= query_specification */ - { 355, -6 }, /* (432) query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP */ - { 352, 0 }, /* (433) order_by_clause_opt ::= */ - { 352, -3 }, /* (434) order_by_clause_opt ::= ORDER BY sort_specification_list */ - { 353, 0 }, /* (435) slimit_clause_opt ::= */ - { 353, -2 }, /* (436) slimit_clause_opt ::= SLIMIT NK_INTEGER */ - { 353, -4 }, /* (437) slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER */ - { 353, -4 }, /* (438) slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER */ - { 354, 0 }, /* (439) limit_clause_opt ::= */ - { 354, -2 }, /* (440) limit_clause_opt ::= LIMIT NK_INTEGER */ - { 354, -4 }, /* (441) limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER */ - { 354, -4 }, /* (442) limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER */ - { 317, -3 }, /* (443) subquery ::= NK_LP query_expression NK_RP */ - { 338, -1 }, /* (444) search_condition ::= common_expression */ - { 356, -1 }, /* (445) sort_specification_list ::= sort_specification */ - { 356, -3 }, /* (446) sort_specification_list ::= sort_specification_list NK_COMMA sort_specification */ - { 357, -3 }, /* (447) sort_specification ::= expression ordering_specification_opt null_ordering_opt */ - { 358, 0 }, /* (448) ordering_specification_opt ::= */ - { 358, -1 }, /* (449) ordering_specification_opt ::= ASC */ - { 358, -1 }, /* (450) ordering_specification_opt ::= DESC */ - { 359, 0 }, /* (451) null_ordering_opt ::= */ - { 359, -2 }, /* (452) null_ordering_opt ::= NULLS FIRST */ - { 359, -2 }, /* (453) null_ordering_opt ::= NULLS LAST */ + { 239, -6 }, /* (0) cmd ::= CREATE ACCOUNT NK_ID PASS NK_STRING account_options */ + { 239, -4 }, /* (1) cmd ::= ALTER ACCOUNT NK_ID alter_account_options */ + { 240, 0 }, /* (2) account_options ::= */ + { 240, -3 }, /* (3) account_options ::= account_options PPS literal */ + { 240, -3 }, /* (4) account_options ::= account_options TSERIES literal */ + { 240, -3 }, /* (5) account_options ::= account_options STORAGE literal */ + { 240, -3 }, /* (6) account_options ::= account_options STREAMS literal */ + { 240, -3 }, /* (7) account_options ::= account_options QTIME literal */ + { 240, -3 }, /* (8) account_options ::= account_options DBS literal */ + { 240, -3 }, /* (9) account_options ::= account_options USERS literal */ + { 240, -3 }, /* (10) account_options ::= account_options CONNS literal */ + { 240, -3 }, /* (11) account_options ::= account_options STATE literal */ + { 241, -1 }, /* (12) alter_account_options ::= alter_account_option */ + { 241, -2 }, /* (13) alter_account_options ::= alter_account_options alter_account_option */ + { 243, -2 }, /* (14) alter_account_option ::= PASS literal */ + { 243, -2 }, /* (15) alter_account_option ::= PPS literal */ + { 243, -2 }, /* (16) alter_account_option ::= TSERIES literal */ + { 243, -2 }, /* (17) alter_account_option ::= STORAGE literal */ + { 243, -2 }, /* (18) alter_account_option ::= STREAMS literal */ + { 243, -2 }, /* (19) alter_account_option ::= QTIME literal */ + { 243, -2 }, /* (20) alter_account_option ::= DBS literal */ + { 243, -2 }, /* (21) alter_account_option ::= USERS literal */ + { 243, -2 }, /* (22) alter_account_option ::= CONNS literal */ + { 243, -2 }, /* (23) alter_account_option ::= STATE literal */ + { 239, -5 }, /* (24) cmd ::= CREATE USER user_name PASS NK_STRING */ + { 239, -5 }, /* (25) cmd ::= ALTER USER user_name PASS NK_STRING */ + { 239, -5 }, /* (26) cmd ::= ALTER USER user_name PRIVILEGE NK_STRING */ + { 239, -3 }, /* (27) cmd ::= DROP USER user_name */ + { 239, -6 }, /* (28) cmd ::= GRANT privileges ON priv_level TO user_name */ + { 239, -6 }, /* (29) cmd ::= REVOKE privileges ON priv_level FROM user_name */ + { 245, -1 }, /* (30) privileges ::= ALL */ + { 245, -1 }, /* (31) privileges ::= priv_type_list */ + { 247, -1 }, /* (32) priv_type_list ::= priv_type */ + { 247, -3 }, /* (33) priv_type_list ::= priv_type_list NK_COMMA priv_type */ + { 248, -1 }, /* (34) priv_type ::= READ */ + { 248, -1 }, /* (35) priv_type ::= WRITE */ + { 246, -3 }, /* (36) priv_level ::= NK_STAR NK_DOT NK_STAR */ + { 246, -3 }, /* (37) priv_level ::= db_name NK_DOT NK_STAR */ + { 239, -3 }, /* (38) cmd ::= CREATE DNODE dnode_endpoint */ + { 239, -5 }, /* (39) cmd ::= CREATE DNODE dnode_endpoint PORT NK_INTEGER */ + { 239, -3 }, /* (40) cmd ::= DROP DNODE NK_INTEGER */ + { 239, -3 }, /* (41) cmd ::= DROP DNODE dnode_endpoint */ + { 239, -4 }, /* (42) cmd ::= ALTER DNODE NK_INTEGER NK_STRING */ + { 239, -5 }, /* (43) cmd ::= ALTER DNODE NK_INTEGER NK_STRING NK_STRING */ + { 239, -4 }, /* (44) cmd ::= ALTER ALL DNODES NK_STRING */ + { 239, -5 }, /* (45) cmd ::= ALTER ALL DNODES NK_STRING NK_STRING */ + { 250, -1 }, /* (46) dnode_endpoint ::= NK_STRING */ + { 250, -1 }, /* (47) dnode_endpoint ::= NK_ID */ + { 250, -1 }, /* (48) dnode_endpoint ::= NK_IPTOKEN */ + { 239, -3 }, /* (49) cmd ::= ALTER LOCAL NK_STRING */ + { 239, -4 }, /* (50) cmd ::= ALTER LOCAL NK_STRING NK_STRING */ + { 239, -5 }, /* (51) cmd ::= CREATE QNODE ON DNODE NK_INTEGER */ + { 239, -5 }, /* (52) cmd ::= DROP QNODE ON DNODE NK_INTEGER */ + { 239, -5 }, /* (53) cmd ::= CREATE BNODE ON DNODE NK_INTEGER */ + { 239, -5 }, /* (54) cmd ::= DROP BNODE ON DNODE NK_INTEGER */ + { 239, -5 }, /* (55) cmd ::= CREATE SNODE ON DNODE NK_INTEGER */ + { 239, -5 }, /* (56) cmd ::= DROP SNODE ON DNODE NK_INTEGER */ + { 239, -5 }, /* (57) cmd ::= CREATE MNODE ON DNODE NK_INTEGER */ + { 239, -5 }, /* (58) cmd ::= DROP MNODE ON DNODE NK_INTEGER */ + { 239, -5 }, /* (59) cmd ::= CREATE DATABASE not_exists_opt db_name db_options */ + { 239, -4 }, /* (60) cmd ::= DROP DATABASE exists_opt db_name */ + { 239, -2 }, /* (61) cmd ::= USE db_name */ + { 239, -4 }, /* (62) cmd ::= ALTER DATABASE db_name alter_db_options */ + { 251, -3 }, /* (63) not_exists_opt ::= IF NOT EXISTS */ + { 251, 0 }, /* (64) not_exists_opt ::= */ + { 253, -2 }, /* (65) exists_opt ::= IF EXISTS */ + { 253, 0 }, /* (66) exists_opt ::= */ + { 252, 0 }, /* (67) db_options ::= */ + { 252, -3 }, /* (68) db_options ::= db_options BUFFER NK_INTEGER */ + { 252, -3 }, /* (69) db_options ::= db_options CACHELAST NK_INTEGER */ + { 252, -3 }, /* (70) db_options ::= db_options COMP NK_INTEGER */ + { 252, -3 }, /* (71) db_options ::= db_options DURATION NK_INTEGER */ + { 252, -3 }, /* (72) db_options ::= db_options DURATION NK_VARIABLE */ + { 252, -3 }, /* (73) db_options ::= db_options FSYNC NK_INTEGER */ + { 252, -3 }, /* (74) db_options ::= db_options MAXROWS NK_INTEGER */ + { 252, -3 }, /* (75) db_options ::= db_options MINROWS NK_INTEGER */ + { 252, -3 }, /* (76) db_options ::= db_options KEEP integer_list */ + { 252, -3 }, /* (77) db_options ::= db_options KEEP variable_list */ + { 252, -3 }, /* (78) db_options ::= db_options PAGES NK_INTEGER */ + { 252, -3 }, /* (79) db_options ::= db_options PAGESIZE NK_INTEGER */ + { 252, -3 }, /* (80) db_options ::= db_options PRECISION NK_STRING */ + { 252, -3 }, /* (81) db_options ::= db_options REPLICA NK_INTEGER */ + { 252, -3 }, /* (82) db_options ::= db_options STRICT NK_INTEGER */ + { 252, -3 }, /* (83) db_options ::= db_options WAL NK_INTEGER */ + { 252, -3 }, /* (84) db_options ::= db_options VGROUPS NK_INTEGER */ + { 252, -3 }, /* (85) db_options ::= db_options SINGLE_STABLE NK_INTEGER */ + { 252, -3 }, /* (86) db_options ::= db_options RETENTIONS retention_list */ + { 252, -3 }, /* (87) db_options ::= db_options SCHEMALESS NK_INTEGER */ + { 254, -1 }, /* (88) alter_db_options ::= alter_db_option */ + { 254, -2 }, /* (89) alter_db_options ::= alter_db_options alter_db_option */ + { 258, -2 }, /* (90) alter_db_option ::= BUFFER NK_INTEGER */ + { 258, -2 }, /* (91) alter_db_option ::= CACHELAST NK_INTEGER */ + { 258, -2 }, /* (92) alter_db_option ::= FSYNC NK_INTEGER */ + { 258, -2 }, /* (93) alter_db_option ::= KEEP integer_list */ + { 258, -2 }, /* (94) alter_db_option ::= KEEP variable_list */ + { 258, -2 }, /* (95) alter_db_option ::= PAGES NK_INTEGER */ + { 258, -2 }, /* (96) alter_db_option ::= REPLICA NK_INTEGER */ + { 258, -2 }, /* (97) alter_db_option ::= STRICT NK_INTEGER */ + { 258, -2 }, /* (98) alter_db_option ::= WAL NK_INTEGER */ + { 255, -1 }, /* (99) integer_list ::= NK_INTEGER */ + { 255, -3 }, /* (100) integer_list ::= integer_list NK_COMMA NK_INTEGER */ + { 256, -1 }, /* (101) variable_list ::= NK_VARIABLE */ + { 256, -3 }, /* (102) variable_list ::= variable_list NK_COMMA NK_VARIABLE */ + { 257, -1 }, /* (103) retention_list ::= retention */ + { 257, -3 }, /* (104) retention_list ::= retention_list NK_COMMA retention */ + { 259, -3 }, /* (105) retention ::= NK_VARIABLE NK_COLON NK_VARIABLE */ + { 239, -9 }, /* (106) cmd ::= CREATE TABLE not_exists_opt full_table_name NK_LP column_def_list NK_RP tags_def_opt table_options */ + { 239, -3 }, /* (107) cmd ::= CREATE TABLE multi_create_clause */ + { 239, -9 }, /* (108) cmd ::= CREATE STABLE not_exists_opt full_table_name NK_LP column_def_list NK_RP tags_def table_options */ + { 239, -3 }, /* (109) cmd ::= DROP TABLE multi_drop_clause */ + { 239, -4 }, /* (110) cmd ::= DROP STABLE exists_opt full_table_name */ + { 239, -3 }, /* (111) cmd ::= ALTER TABLE alter_table_clause */ + { 239, -3 }, /* (112) cmd ::= ALTER STABLE alter_table_clause */ + { 267, -2 }, /* (113) alter_table_clause ::= full_table_name alter_table_options */ + { 267, -5 }, /* (114) alter_table_clause ::= full_table_name ADD COLUMN column_name type_name */ + { 267, -4 }, /* (115) alter_table_clause ::= full_table_name DROP COLUMN column_name */ + { 267, -5 }, /* (116) alter_table_clause ::= full_table_name MODIFY COLUMN column_name type_name */ + { 267, -5 }, /* (117) alter_table_clause ::= full_table_name RENAME COLUMN column_name column_name */ + { 267, -5 }, /* (118) alter_table_clause ::= full_table_name ADD TAG column_name type_name */ + { 267, -4 }, /* (119) alter_table_clause ::= full_table_name DROP TAG column_name */ + { 267, -5 }, /* (120) alter_table_clause ::= full_table_name MODIFY TAG column_name type_name */ + { 267, -5 }, /* (121) alter_table_clause ::= full_table_name RENAME TAG column_name column_name */ + { 267, -6 }, /* (122) alter_table_clause ::= full_table_name SET TAG column_name NK_EQ signed_literal */ + { 264, -1 }, /* (123) multi_create_clause ::= create_subtable_clause */ + { 264, -2 }, /* (124) multi_create_clause ::= multi_create_clause create_subtable_clause */ + { 272, -10 }, /* (125) create_subtable_clause ::= not_exists_opt full_table_name USING full_table_name specific_tags_opt TAGS NK_LP literal_list NK_RP table_options */ + { 266, -1 }, /* (126) multi_drop_clause ::= drop_table_clause */ + { 266, -2 }, /* (127) multi_drop_clause ::= multi_drop_clause drop_table_clause */ + { 275, -2 }, /* (128) drop_table_clause ::= exists_opt full_table_name */ + { 273, 0 }, /* (129) specific_tags_opt ::= */ + { 273, -3 }, /* (130) specific_tags_opt ::= NK_LP col_name_list NK_RP */ + { 260, -1 }, /* (131) full_table_name ::= table_name */ + { 260, -3 }, /* (132) full_table_name ::= db_name NK_DOT table_name */ + { 261, -1 }, /* (133) column_def_list ::= column_def */ + { 261, -3 }, /* (134) column_def_list ::= column_def_list NK_COMMA column_def */ + { 278, -2 }, /* (135) column_def ::= column_name type_name */ + { 278, -4 }, /* (136) column_def ::= column_name type_name COMMENT NK_STRING */ + { 270, -1 }, /* (137) type_name ::= BOOL */ + { 270, -1 }, /* (138) type_name ::= TINYINT */ + { 270, -1 }, /* (139) type_name ::= SMALLINT */ + { 270, -1 }, /* (140) type_name ::= INT */ + { 270, -1 }, /* (141) type_name ::= INTEGER */ + { 270, -1 }, /* (142) type_name ::= BIGINT */ + { 270, -1 }, /* (143) type_name ::= FLOAT */ + { 270, -1 }, /* (144) type_name ::= DOUBLE */ + { 270, -4 }, /* (145) type_name ::= BINARY NK_LP NK_INTEGER NK_RP */ + { 270, -1 }, /* (146) type_name ::= TIMESTAMP */ + { 270, -4 }, /* (147) type_name ::= NCHAR NK_LP NK_INTEGER NK_RP */ + { 270, -2 }, /* (148) type_name ::= TINYINT UNSIGNED */ + { 270, -2 }, /* (149) type_name ::= SMALLINT UNSIGNED */ + { 270, -2 }, /* (150) type_name ::= INT UNSIGNED */ + { 270, -2 }, /* (151) type_name ::= BIGINT UNSIGNED */ + { 270, -1 }, /* (152) type_name ::= JSON */ + { 270, -4 }, /* (153) type_name ::= VARCHAR NK_LP NK_INTEGER NK_RP */ + { 270, -1 }, /* (154) type_name ::= MEDIUMBLOB */ + { 270, -1 }, /* (155) type_name ::= BLOB */ + { 270, -4 }, /* (156) type_name ::= VARBINARY NK_LP NK_INTEGER NK_RP */ + { 270, -1 }, /* (157) type_name ::= DECIMAL */ + { 270, -4 }, /* (158) type_name ::= DECIMAL NK_LP NK_INTEGER NK_RP */ + { 270, -6 }, /* (159) type_name ::= DECIMAL NK_LP NK_INTEGER NK_COMMA NK_INTEGER NK_RP */ + { 262, 0 }, /* (160) tags_def_opt ::= */ + { 262, -1 }, /* (161) tags_def_opt ::= tags_def */ + { 265, -4 }, /* (162) tags_def ::= TAGS NK_LP column_def_list NK_RP */ + { 263, 0 }, /* (163) table_options ::= */ + { 263, -3 }, /* (164) table_options ::= table_options COMMENT NK_STRING */ + { 263, -3 }, /* (165) table_options ::= table_options MAX_DELAY duration_list */ + { 263, -3 }, /* (166) table_options ::= table_options WATERMARK duration_list */ + { 263, -5 }, /* (167) table_options ::= table_options ROLLUP NK_LP rollup_func_list NK_RP */ + { 263, -3 }, /* (168) table_options ::= table_options TTL NK_INTEGER */ + { 263, -5 }, /* (169) table_options ::= table_options SMA NK_LP col_name_list NK_RP */ + { 268, -1 }, /* (170) alter_table_options ::= alter_table_option */ + { 268, -2 }, /* (171) alter_table_options ::= alter_table_options alter_table_option */ + { 281, -2 }, /* (172) alter_table_option ::= COMMENT NK_STRING */ + { 281, -2 }, /* (173) alter_table_option ::= TTL NK_INTEGER */ + { 279, -1 }, /* (174) duration_list ::= duration_literal */ + { 279, -3 }, /* (175) duration_list ::= duration_list NK_COMMA duration_literal */ + { 280, -1 }, /* (176) rollup_func_list ::= rollup_func_name */ + { 280, -3 }, /* (177) rollup_func_list ::= rollup_func_list NK_COMMA rollup_func_name */ + { 283, -1 }, /* (178) rollup_func_name ::= function_name */ + { 283, -1 }, /* (179) rollup_func_name ::= FIRST */ + { 283, -1 }, /* (180) rollup_func_name ::= LAST */ + { 276, -1 }, /* (181) col_name_list ::= col_name */ + { 276, -3 }, /* (182) col_name_list ::= col_name_list NK_COMMA col_name */ + { 285, -1 }, /* (183) col_name ::= column_name */ + { 239, -2 }, /* (184) cmd ::= SHOW DNODES */ + { 239, -2 }, /* (185) cmd ::= SHOW USERS */ + { 239, -2 }, /* (186) cmd ::= SHOW DATABASES */ + { 239, -4 }, /* (187) cmd ::= SHOW db_name_cond_opt TABLES like_pattern_opt */ + { 239, -4 }, /* (188) cmd ::= SHOW db_name_cond_opt STABLES like_pattern_opt */ + { 239, -3 }, /* (189) cmd ::= SHOW db_name_cond_opt VGROUPS */ + { 239, -2 }, /* (190) cmd ::= SHOW MNODES */ + { 239, -2 }, /* (191) cmd ::= SHOW MODULES */ + { 239, -2 }, /* (192) cmd ::= SHOW QNODES */ + { 239, -2 }, /* (193) cmd ::= SHOW FUNCTIONS */ + { 239, -5 }, /* (194) cmd ::= SHOW INDEXES FROM table_name_cond from_db_opt */ + { 239, -2 }, /* (195) cmd ::= SHOW STREAMS */ + { 239, -2 }, /* (196) cmd ::= SHOW ACCOUNTS */ + { 239, -2 }, /* (197) cmd ::= SHOW APPS */ + { 239, -2 }, /* (198) cmd ::= SHOW CONNECTIONS */ + { 239, -2 }, /* (199) cmd ::= SHOW LICENCE */ + { 239, -2 }, /* (200) cmd ::= SHOW GRANTS */ + { 239, -4 }, /* (201) cmd ::= SHOW CREATE DATABASE db_name */ + { 239, -4 }, /* (202) cmd ::= SHOW CREATE TABLE full_table_name */ + { 239, -4 }, /* (203) cmd ::= SHOW CREATE STABLE full_table_name */ + { 239, -2 }, /* (204) cmd ::= SHOW QUERIES */ + { 239, -2 }, /* (205) cmd ::= SHOW SCORES */ + { 239, -2 }, /* (206) cmd ::= SHOW TOPICS */ + { 239, -2 }, /* (207) cmd ::= SHOW VARIABLES */ + { 239, -2 }, /* (208) cmd ::= SHOW BNODES */ + { 239, -2 }, /* (209) cmd ::= SHOW SNODES */ + { 239, -2 }, /* (210) cmd ::= SHOW CLUSTER */ + { 239, -2 }, /* (211) cmd ::= SHOW TRANSACTIONS */ + { 286, 0 }, /* (212) db_name_cond_opt ::= */ + { 286, -2 }, /* (213) db_name_cond_opt ::= db_name NK_DOT */ + { 287, 0 }, /* (214) like_pattern_opt ::= */ + { 287, -2 }, /* (215) like_pattern_opt ::= LIKE NK_STRING */ + { 288, -1 }, /* (216) table_name_cond ::= table_name */ + { 289, 0 }, /* (217) from_db_opt ::= */ + { 289, -2 }, /* (218) from_db_opt ::= FROM db_name */ + { 239, -8 }, /* (219) cmd ::= CREATE SMA INDEX not_exists_opt index_name ON table_name index_options */ + { 239, -10 }, /* (220) cmd ::= CREATE FULLTEXT INDEX not_exists_opt index_name ON table_name NK_LP col_name_list NK_RP */ + { 239, -6 }, /* (221) cmd ::= DROP INDEX exists_opt index_name ON table_name */ + { 291, 0 }, /* (222) index_options ::= */ + { 291, -9 }, /* (223) index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_RP sliding_opt */ + { 291, -11 }, /* (224) index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt */ + { 292, -1 }, /* (225) func_list ::= func */ + { 292, -3 }, /* (226) func_list ::= func_list NK_COMMA func */ + { 294, -4 }, /* (227) func ::= function_name NK_LP expression_list NK_RP */ + { 239, -6 }, /* (228) cmd ::= CREATE TOPIC not_exists_opt topic_name AS query_expression */ + { 239, -7 }, /* (229) cmd ::= CREATE TOPIC not_exists_opt topic_name AS DATABASE db_name */ + { 239, -7 }, /* (230) cmd ::= CREATE TOPIC not_exists_opt topic_name AS STABLE full_table_name */ + { 239, -4 }, /* (231) cmd ::= DROP TOPIC exists_opt topic_name */ + { 239, -7 }, /* (232) cmd ::= DROP CONSUMER GROUP exists_opt cgroup_name ON topic_name */ + { 239, -2 }, /* (233) cmd ::= DESC full_table_name */ + { 239, -2 }, /* (234) cmd ::= DESCRIBE full_table_name */ + { 239, -3 }, /* (235) cmd ::= RESET QUERY CACHE */ + { 239, -4 }, /* (236) cmd ::= EXPLAIN analyze_opt explain_options query_expression */ + { 299, 0 }, /* (237) analyze_opt ::= */ + { 299, -1 }, /* (238) analyze_opt ::= ANALYZE */ + { 300, 0 }, /* (239) explain_options ::= */ + { 300, -3 }, /* (240) explain_options ::= explain_options VERBOSE NK_BOOL */ + { 300, -3 }, /* (241) explain_options ::= explain_options RATIO NK_FLOAT */ + { 239, -6 }, /* (242) cmd ::= COMPACT VNODES IN NK_LP integer_list NK_RP */ + { 239, -10 }, /* (243) cmd ::= CREATE agg_func_opt FUNCTION not_exists_opt function_name AS NK_STRING OUTPUTTYPE type_name bufsize_opt */ + { 239, -4 }, /* (244) cmd ::= DROP FUNCTION exists_opt function_name */ + { 301, 0 }, /* (245) agg_func_opt ::= */ + { 301, -1 }, /* (246) agg_func_opt ::= AGGREGATE */ + { 302, 0 }, /* (247) bufsize_opt ::= */ + { 302, -2 }, /* (248) bufsize_opt ::= BUFSIZE NK_INTEGER */ + { 239, -8 }, /* (249) cmd ::= CREATE STREAM not_exists_opt stream_name stream_options into_opt AS query_expression */ + { 239, -4 }, /* (250) cmd ::= DROP STREAM exists_opt stream_name */ + { 305, 0 }, /* (251) into_opt ::= */ + { 305, -2 }, /* (252) into_opt ::= INTO full_table_name */ + { 304, 0 }, /* (253) stream_options ::= */ + { 304, -3 }, /* (254) stream_options ::= stream_options TRIGGER AT_ONCE */ + { 304, -3 }, /* (255) stream_options ::= stream_options TRIGGER WINDOW_CLOSE */ + { 304, -4 }, /* (256) stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal */ + { 304, -3 }, /* (257) stream_options ::= stream_options WATERMARK duration_literal */ + { 239, -3 }, /* (258) cmd ::= KILL CONNECTION NK_INTEGER */ + { 239, -3 }, /* (259) cmd ::= KILL QUERY NK_STRING */ + { 239, -3 }, /* (260) cmd ::= KILL TRANSACTION NK_INTEGER */ + { 239, -2 }, /* (261) cmd ::= BALANCE VGROUP */ + { 239, -4 }, /* (262) cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER */ + { 239, -4 }, /* (263) cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list */ + { 239, -3 }, /* (264) cmd ::= SPLIT VGROUP NK_INTEGER */ + { 306, -2 }, /* (265) dnode_list ::= DNODE NK_INTEGER */ + { 306, -3 }, /* (266) dnode_list ::= dnode_list DNODE NK_INTEGER */ + { 239, -3 }, /* (267) cmd ::= SYNCDB db_name REPLICA */ + { 239, -4 }, /* (268) cmd ::= DELETE FROM full_table_name where_clause_opt */ + { 239, -1 }, /* (269) cmd ::= query_expression */ + { 242, -1 }, /* (270) literal ::= NK_INTEGER */ + { 242, -1 }, /* (271) literal ::= NK_FLOAT */ + { 242, -1 }, /* (272) literal ::= NK_STRING */ + { 242, -1 }, /* (273) literal ::= NK_BOOL */ + { 242, -2 }, /* (274) literal ::= TIMESTAMP NK_STRING */ + { 242, -1 }, /* (275) literal ::= duration_literal */ + { 242, -1 }, /* (276) literal ::= NULL */ + { 242, -1 }, /* (277) literal ::= NK_QUESTION */ + { 282, -1 }, /* (278) duration_literal ::= NK_VARIABLE */ + { 308, -1 }, /* (279) signed ::= NK_INTEGER */ + { 308, -2 }, /* (280) signed ::= NK_PLUS NK_INTEGER */ + { 308, -2 }, /* (281) signed ::= NK_MINUS NK_INTEGER */ + { 308, -1 }, /* (282) signed ::= NK_FLOAT */ + { 308, -2 }, /* (283) signed ::= NK_PLUS NK_FLOAT */ + { 308, -2 }, /* (284) signed ::= NK_MINUS NK_FLOAT */ + { 271, -1 }, /* (285) signed_literal ::= signed */ + { 271, -1 }, /* (286) signed_literal ::= NK_STRING */ + { 271, -1 }, /* (287) signed_literal ::= NK_BOOL */ + { 271, -2 }, /* (288) signed_literal ::= TIMESTAMP NK_STRING */ + { 271, -1 }, /* (289) signed_literal ::= duration_literal */ + { 271, -1 }, /* (290) signed_literal ::= NULL */ + { 271, -1 }, /* (291) signed_literal ::= literal_func */ + { 274, -1 }, /* (292) literal_list ::= signed_literal */ + { 274, -3 }, /* (293) literal_list ::= literal_list NK_COMMA signed_literal */ + { 249, -1 }, /* (294) db_name ::= NK_ID */ + { 277, -1 }, /* (295) table_name ::= NK_ID */ + { 269, -1 }, /* (296) column_name ::= NK_ID */ + { 284, -1 }, /* (297) function_name ::= NK_ID */ + { 310, -1 }, /* (298) table_alias ::= NK_ID */ + { 311, -1 }, /* (299) column_alias ::= NK_ID */ + { 244, -1 }, /* (300) user_name ::= NK_ID */ + { 290, -1 }, /* (301) index_name ::= NK_ID */ + { 296, -1 }, /* (302) topic_name ::= NK_ID */ + { 303, -1 }, /* (303) stream_name ::= NK_ID */ + { 298, -1 }, /* (304) cgroup_name ::= NK_ID */ + { 312, -1 }, /* (305) expression ::= literal */ + { 312, -1 }, /* (306) expression ::= pseudo_column */ + { 312, -1 }, /* (307) expression ::= column_reference */ + { 312, -1 }, /* (308) expression ::= function_expression */ + { 312, -1 }, /* (309) expression ::= subquery */ + { 312, -3 }, /* (310) expression ::= NK_LP expression NK_RP */ + { 312, -2 }, /* (311) expression ::= NK_PLUS expression */ + { 312, -2 }, /* (312) expression ::= NK_MINUS expression */ + { 312, -3 }, /* (313) expression ::= expression NK_PLUS expression */ + { 312, -3 }, /* (314) expression ::= expression NK_MINUS expression */ + { 312, -3 }, /* (315) expression ::= expression NK_STAR expression */ + { 312, -3 }, /* (316) expression ::= expression NK_SLASH expression */ + { 312, -3 }, /* (317) expression ::= expression NK_REM expression */ + { 312, -3 }, /* (318) expression ::= column_reference NK_ARROW NK_STRING */ + { 295, -1 }, /* (319) expression_list ::= expression */ + { 295, -3 }, /* (320) expression_list ::= expression_list NK_COMMA expression */ + { 314, -1 }, /* (321) column_reference ::= column_name */ + { 314, -3 }, /* (322) column_reference ::= table_name NK_DOT column_name */ + { 313, -1 }, /* (323) pseudo_column ::= ROWTS */ + { 313, -1 }, /* (324) pseudo_column ::= TBNAME */ + { 313, -3 }, /* (325) pseudo_column ::= table_name NK_DOT TBNAME */ + { 313, -1 }, /* (326) pseudo_column ::= QSTARTTS */ + { 313, -1 }, /* (327) pseudo_column ::= QENDTS */ + { 313, -1 }, /* (328) pseudo_column ::= WSTARTTS */ + { 313, -1 }, /* (329) pseudo_column ::= WENDTS */ + { 313, -1 }, /* (330) pseudo_column ::= WDURATION */ + { 315, -4 }, /* (331) function_expression ::= function_name NK_LP expression_list NK_RP */ + { 315, -4 }, /* (332) function_expression ::= star_func NK_LP star_func_para_list NK_RP */ + { 315, -6 }, /* (333) function_expression ::= CAST NK_LP expression AS type_name NK_RP */ + { 315, -1 }, /* (334) function_expression ::= literal_func */ + { 309, -3 }, /* (335) literal_func ::= noarg_func NK_LP NK_RP */ + { 309, -1 }, /* (336) literal_func ::= NOW */ + { 319, -1 }, /* (337) noarg_func ::= NOW */ + { 319, -1 }, /* (338) noarg_func ::= TODAY */ + { 319, -1 }, /* (339) noarg_func ::= TIMEZONE */ + { 317, -1 }, /* (340) star_func ::= COUNT */ + { 317, -1 }, /* (341) star_func ::= FIRST */ + { 317, -1 }, /* (342) star_func ::= LAST */ + { 317, -1 }, /* (343) star_func ::= LAST_ROW */ + { 318, -1 }, /* (344) star_func_para_list ::= NK_STAR */ + { 318, -1 }, /* (345) star_func_para_list ::= other_para_list */ + { 320, -1 }, /* (346) other_para_list ::= star_func_para */ + { 320, -3 }, /* (347) other_para_list ::= other_para_list NK_COMMA star_func_para */ + { 321, -1 }, /* (348) star_func_para ::= expression */ + { 321, -3 }, /* (349) star_func_para ::= table_name NK_DOT NK_STAR */ + { 322, -3 }, /* (350) predicate ::= expression compare_op expression */ + { 322, -5 }, /* (351) predicate ::= expression BETWEEN expression AND expression */ + { 322, -6 }, /* (352) predicate ::= expression NOT BETWEEN expression AND expression */ + { 322, -3 }, /* (353) predicate ::= expression IS NULL */ + { 322, -4 }, /* (354) predicate ::= expression IS NOT NULL */ + { 322, -3 }, /* (355) predicate ::= expression in_op in_predicate_value */ + { 323, -1 }, /* (356) compare_op ::= NK_LT */ + { 323, -1 }, /* (357) compare_op ::= NK_GT */ + { 323, -1 }, /* (358) compare_op ::= NK_LE */ + { 323, -1 }, /* (359) compare_op ::= NK_GE */ + { 323, -1 }, /* (360) compare_op ::= NK_NE */ + { 323, -1 }, /* (361) compare_op ::= NK_EQ */ + { 323, -1 }, /* (362) compare_op ::= LIKE */ + { 323, -2 }, /* (363) compare_op ::= NOT LIKE */ + { 323, -1 }, /* (364) compare_op ::= MATCH */ + { 323, -1 }, /* (365) compare_op ::= NMATCH */ + { 323, -1 }, /* (366) compare_op ::= CONTAINS */ + { 324, -1 }, /* (367) in_op ::= IN */ + { 324, -2 }, /* (368) in_op ::= NOT IN */ + { 325, -3 }, /* (369) in_predicate_value ::= NK_LP expression_list NK_RP */ + { 326, -1 }, /* (370) boolean_value_expression ::= boolean_primary */ + { 326, -2 }, /* (371) boolean_value_expression ::= NOT boolean_primary */ + { 326, -3 }, /* (372) boolean_value_expression ::= boolean_value_expression OR boolean_value_expression */ + { 326, -3 }, /* (373) boolean_value_expression ::= boolean_value_expression AND boolean_value_expression */ + { 327, -1 }, /* (374) boolean_primary ::= predicate */ + { 327, -3 }, /* (375) boolean_primary ::= NK_LP boolean_value_expression NK_RP */ + { 328, -1 }, /* (376) common_expression ::= expression */ + { 328, -1 }, /* (377) common_expression ::= boolean_value_expression */ + { 329, -2 }, /* (378) from_clause ::= FROM table_reference_list */ + { 330, -1 }, /* (379) table_reference_list ::= table_reference */ + { 330, -3 }, /* (380) table_reference_list ::= table_reference_list NK_COMMA table_reference */ + { 331, -1 }, /* (381) table_reference ::= table_primary */ + { 331, -1 }, /* (382) table_reference ::= joined_table */ + { 332, -2 }, /* (383) table_primary ::= table_name alias_opt */ + { 332, -4 }, /* (384) table_primary ::= db_name NK_DOT table_name alias_opt */ + { 332, -2 }, /* (385) table_primary ::= subquery alias_opt */ + { 332, -1 }, /* (386) table_primary ::= parenthesized_joined_table */ + { 334, 0 }, /* (387) alias_opt ::= */ + { 334, -1 }, /* (388) alias_opt ::= table_alias */ + { 334, -2 }, /* (389) alias_opt ::= AS table_alias */ + { 335, -3 }, /* (390) parenthesized_joined_table ::= NK_LP joined_table NK_RP */ + { 335, -3 }, /* (391) parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP */ + { 333, -6 }, /* (392) joined_table ::= table_reference join_type JOIN table_reference ON search_condition */ + { 336, 0 }, /* (393) join_type ::= */ + { 336, -1 }, /* (394) join_type ::= INNER */ + { 338, -9 }, /* (395) query_specification ::= SELECT set_quantifier_opt select_list from_clause where_clause_opt partition_by_clause_opt twindow_clause_opt group_by_clause_opt having_clause_opt */ + { 339, 0 }, /* (396) set_quantifier_opt ::= */ + { 339, -1 }, /* (397) set_quantifier_opt ::= DISTINCT */ + { 339, -1 }, /* (398) set_quantifier_opt ::= ALL */ + { 340, -1 }, /* (399) select_list ::= NK_STAR */ + { 340, -1 }, /* (400) select_list ::= select_sublist */ + { 345, -1 }, /* (401) select_sublist ::= select_item */ + { 345, -3 }, /* (402) select_sublist ::= select_sublist NK_COMMA select_item */ + { 346, -1 }, /* (403) select_item ::= common_expression */ + { 346, -2 }, /* (404) select_item ::= common_expression column_alias */ + { 346, -3 }, /* (405) select_item ::= common_expression AS column_alias */ + { 346, -3 }, /* (406) select_item ::= table_name NK_DOT NK_STAR */ + { 307, 0 }, /* (407) where_clause_opt ::= */ + { 307, -2 }, /* (408) where_clause_opt ::= WHERE search_condition */ + { 341, 0 }, /* (409) partition_by_clause_opt ::= */ + { 341, -3 }, /* (410) partition_by_clause_opt ::= PARTITION BY expression_list */ + { 342, 0 }, /* (411) twindow_clause_opt ::= */ + { 342, -6 }, /* (412) twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP */ + { 342, -4 }, /* (413) twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP */ + { 342, -6 }, /* (414) twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt */ + { 342, -8 }, /* (415) twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt */ + { 293, 0 }, /* (416) sliding_opt ::= */ + { 293, -4 }, /* (417) sliding_opt ::= SLIDING NK_LP duration_literal NK_RP */ + { 347, 0 }, /* (418) fill_opt ::= */ + { 347, -4 }, /* (419) fill_opt ::= FILL NK_LP fill_mode NK_RP */ + { 347, -6 }, /* (420) fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP */ + { 348, -1 }, /* (421) fill_mode ::= NONE */ + { 348, -1 }, /* (422) fill_mode ::= PREV */ + { 348, -1 }, /* (423) fill_mode ::= NULL */ + { 348, -1 }, /* (424) fill_mode ::= LINEAR */ + { 348, -1 }, /* (425) fill_mode ::= NEXT */ + { 343, 0 }, /* (426) group_by_clause_opt ::= */ + { 343, -3 }, /* (427) group_by_clause_opt ::= GROUP BY group_by_list */ + { 349, -1 }, /* (428) group_by_list ::= expression */ + { 349, -3 }, /* (429) group_by_list ::= group_by_list NK_COMMA expression */ + { 344, 0 }, /* (430) having_clause_opt ::= */ + { 344, -2 }, /* (431) having_clause_opt ::= HAVING search_condition */ + { 297, -4 }, /* (432) query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt */ + { 350, -1 }, /* (433) query_expression_body ::= query_primary */ + { 350, -4 }, /* (434) query_expression_body ::= query_expression_body UNION ALL query_expression_body */ + { 350, -3 }, /* (435) query_expression_body ::= query_expression_body UNION query_expression_body */ + { 354, -1 }, /* (436) query_primary ::= query_specification */ + { 354, -6 }, /* (437) query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP */ + { 351, 0 }, /* (438) order_by_clause_opt ::= */ + { 351, -3 }, /* (439) order_by_clause_opt ::= ORDER BY sort_specification_list */ + { 352, 0 }, /* (440) slimit_clause_opt ::= */ + { 352, -2 }, /* (441) slimit_clause_opt ::= SLIMIT NK_INTEGER */ + { 352, -4 }, /* (442) slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER */ + { 352, -4 }, /* (443) slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER */ + { 353, 0 }, /* (444) limit_clause_opt ::= */ + { 353, -2 }, /* (445) limit_clause_opt ::= LIMIT NK_INTEGER */ + { 353, -4 }, /* (446) limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER */ + { 353, -4 }, /* (447) limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER */ + { 316, -3 }, /* (448) subquery ::= NK_LP query_expression NK_RP */ + { 337, -1 }, /* (449) search_condition ::= common_expression */ + { 355, -1 }, /* (450) sort_specification_list ::= sort_specification */ + { 355, -3 }, /* (451) sort_specification_list ::= sort_specification_list NK_COMMA sort_specification */ + { 356, -3 }, /* (452) sort_specification ::= expression ordering_specification_opt null_ordering_opt */ + { 357, 0 }, /* (453) ordering_specification_opt ::= */ + { 357, -1 }, /* (454) ordering_specification_opt ::= ASC */ + { 357, -1 }, /* (455) ordering_specification_opt ::= DESC */ + { 358, 0 }, /* (456) null_ordering_opt ::= */ + { 358, -2 }, /* (457) null_ordering_opt ::= NULLS FIRST */ + { 358, -2 }, /* (458) null_ordering_opt ::= NULLS LAST */ }; static void yy_accept(yyParser*); /* Forward Declaration */ @@ -3117,11 +3112,11 @@ static YYACTIONTYPE yy_reduce( YYMINORTYPE yylhsminor; case 0: /* cmd ::= CREATE ACCOUNT NK_ID PASS NK_STRING account_options */ { pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_EXPRIE_STATEMENT); } - yy_destructor(yypParser,241,&yymsp[0].minor); + yy_destructor(yypParser,240,&yymsp[0].minor); break; case 1: /* cmd ::= ALTER ACCOUNT NK_ID alter_account_options */ { pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_EXPRIE_STATEMENT); } - yy_destructor(yypParser,242,&yymsp[0].minor); + yy_destructor(yypParser,241,&yymsp[0].minor); break; case 2: /* account_options ::= */ { } @@ -3135,20 +3130,20 @@ static YYACTIONTYPE yy_reduce( case 9: /* account_options ::= account_options USERS literal */ yytestcase(yyruleno==9); case 10: /* account_options ::= account_options CONNS literal */ yytestcase(yyruleno==10); case 11: /* account_options ::= account_options STATE literal */ yytestcase(yyruleno==11); -{ yy_destructor(yypParser,241,&yymsp[-2].minor); +{ yy_destructor(yypParser,240,&yymsp[-2].minor); { } - yy_destructor(yypParser,243,&yymsp[0].minor); + yy_destructor(yypParser,242,&yymsp[0].minor); } break; case 12: /* alter_account_options ::= alter_account_option */ -{ yy_destructor(yypParser,244,&yymsp[0].minor); +{ yy_destructor(yypParser,243,&yymsp[0].minor); { } } break; case 13: /* alter_account_options ::= alter_account_options alter_account_option */ -{ yy_destructor(yypParser,242,&yymsp[-1].minor); +{ yy_destructor(yypParser,241,&yymsp[-1].minor); { } - yy_destructor(yypParser,244,&yymsp[0].minor); + yy_destructor(yypParser,243,&yymsp[0].minor); } break; case 14: /* alter_account_option ::= PASS literal */ @@ -3162,63 +3157,63 @@ static YYACTIONTYPE yy_reduce( case 22: /* alter_account_option ::= CONNS literal */ yytestcase(yyruleno==22); case 23: /* alter_account_option ::= STATE literal */ yytestcase(yyruleno==23); { } - yy_destructor(yypParser,243,&yymsp[0].minor); + yy_destructor(yypParser,242,&yymsp[0].minor); break; case 24: /* cmd ::= CREATE USER user_name PASS NK_STRING */ -{ pCxt->pRootNode = createCreateUserStmt(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy0); } +{ pCxt->pRootNode = createCreateUserStmt(pCxt, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy0); } break; case 25: /* cmd ::= ALTER USER user_name PASS NK_STRING */ -{ pCxt->pRootNode = createAlterUserStmt(pCxt, &yymsp[-2].minor.yy209, TSDB_ALTER_USER_PASSWD, &yymsp[0].minor.yy0); } +{ pCxt->pRootNode = createAlterUserStmt(pCxt, &yymsp[-2].minor.yy421, TSDB_ALTER_USER_PASSWD, &yymsp[0].minor.yy0); } break; case 26: /* cmd ::= ALTER USER user_name PRIVILEGE NK_STRING */ -{ pCxt->pRootNode = createAlterUserStmt(pCxt, &yymsp[-2].minor.yy209, TSDB_ALTER_USER_PRIVILEGES, &yymsp[0].minor.yy0); } +{ pCxt->pRootNode = createAlterUserStmt(pCxt, &yymsp[-2].minor.yy421, TSDB_ALTER_USER_PRIVILEGES, &yymsp[0].minor.yy0); } break; case 27: /* cmd ::= DROP USER user_name */ -{ pCxt->pRootNode = createDropUserStmt(pCxt, &yymsp[0].minor.yy209); } +{ pCxt->pRootNode = createDropUserStmt(pCxt, &yymsp[0].minor.yy421); } break; case 28: /* cmd ::= GRANT privileges ON priv_level TO user_name */ -{ pCxt->pRootNode = createGrantStmt(pCxt, yymsp[-4].minor.yy189, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209); } +{ pCxt->pRootNode = createGrantStmt(pCxt, yymsp[-4].minor.yy669, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy421); } break; case 29: /* cmd ::= REVOKE privileges ON priv_level FROM user_name */ -{ pCxt->pRootNode = createRevokeStmt(pCxt, yymsp[-4].minor.yy189, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209); } +{ pCxt->pRootNode = createRevokeStmt(pCxt, yymsp[-4].minor.yy669, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy421); } break; case 30: /* privileges ::= ALL */ -{ yymsp[0].minor.yy189 = PRIVILEGE_TYPE_ALL; } +{ yymsp[0].minor.yy669 = PRIVILEGE_TYPE_ALL; } break; case 31: /* privileges ::= priv_type_list */ case 32: /* priv_type_list ::= priv_type */ yytestcase(yyruleno==32); -{ yylhsminor.yy189 = yymsp[0].minor.yy189; } - yymsp[0].minor.yy189 = yylhsminor.yy189; +{ yylhsminor.yy669 = yymsp[0].minor.yy669; } + yymsp[0].minor.yy669 = yylhsminor.yy669; break; case 33: /* priv_type_list ::= priv_type_list NK_COMMA priv_type */ -{ yylhsminor.yy189 = yymsp[-2].minor.yy189 | yymsp[0].minor.yy189; } - yymsp[-2].minor.yy189 = yylhsminor.yy189; +{ yylhsminor.yy669 = yymsp[-2].minor.yy669 | yymsp[0].minor.yy669; } + yymsp[-2].minor.yy669 = yylhsminor.yy669; break; case 34: /* priv_type ::= READ */ -{ yymsp[0].minor.yy189 = PRIVILEGE_TYPE_READ; } +{ yymsp[0].minor.yy669 = PRIVILEGE_TYPE_READ; } break; case 35: /* priv_type ::= WRITE */ -{ yymsp[0].minor.yy189 = PRIVILEGE_TYPE_WRITE; } +{ yymsp[0].minor.yy669 = PRIVILEGE_TYPE_WRITE; } break; case 36: /* priv_level ::= NK_STAR NK_DOT NK_STAR */ -{ yylhsminor.yy209 = yymsp[-2].minor.yy0; } - yymsp[-2].minor.yy209 = yylhsminor.yy209; +{ yylhsminor.yy421 = yymsp[-2].minor.yy0; } + yymsp[-2].minor.yy421 = yylhsminor.yy421; break; case 37: /* priv_level ::= db_name NK_DOT NK_STAR */ -{ yylhsminor.yy209 = yymsp[-2].minor.yy209; } - yymsp[-2].minor.yy209 = yylhsminor.yy209; +{ yylhsminor.yy421 = yymsp[-2].minor.yy421; } + yymsp[-2].minor.yy421 = yylhsminor.yy421; break; case 38: /* cmd ::= CREATE DNODE dnode_endpoint */ -{ pCxt->pRootNode = createCreateDnodeStmt(pCxt, &yymsp[0].minor.yy209, NULL); } +{ pCxt->pRootNode = createCreateDnodeStmt(pCxt, &yymsp[0].minor.yy421, NULL); } break; - case 39: /* cmd ::= CREATE DNODE dnode_host_name PORT NK_INTEGER */ -{ pCxt->pRootNode = createCreateDnodeStmt(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy0); } + case 39: /* cmd ::= CREATE DNODE dnode_endpoint PORT NK_INTEGER */ +{ pCxt->pRootNode = createCreateDnodeStmt(pCxt, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy0); } break; case 40: /* cmd ::= DROP DNODE NK_INTEGER */ { pCxt->pRootNode = createDropDnodeStmt(pCxt, &yymsp[0].minor.yy0); } break; case 41: /* cmd ::= DROP DNODE dnode_endpoint */ -{ pCxt->pRootNode = createDropDnodeStmt(pCxt, &yymsp[0].minor.yy209); } +{ pCxt->pRootNode = createDropDnodeStmt(pCxt, &yymsp[0].minor.yy421); } break; case 42: /* cmd ::= ALTER DNODE NK_INTEGER NK_STRING */ { pCxt->pRootNode = createAlterDnodeStmt(pCxt, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy0, NULL); } @@ -3233,28 +3228,28 @@ static YYACTIONTYPE yy_reduce( { pCxt->pRootNode = createAlterDnodeStmt(pCxt, NULL, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy0); } break; case 46: /* dnode_endpoint ::= NK_STRING */ - case 47: /* dnode_host_name ::= NK_ID */ yytestcase(yyruleno==47); - case 48: /* dnode_host_name ::= NK_IPTOKEN */ yytestcase(yyruleno==48); - case 289: /* db_name ::= NK_ID */ yytestcase(yyruleno==289); - case 290: /* table_name ::= NK_ID */ yytestcase(yyruleno==290); - case 291: /* column_name ::= NK_ID */ yytestcase(yyruleno==291); - case 292: /* function_name ::= NK_ID */ yytestcase(yyruleno==292); - case 293: /* table_alias ::= NK_ID */ yytestcase(yyruleno==293); - case 294: /* column_alias ::= NK_ID */ yytestcase(yyruleno==294); - case 295: /* user_name ::= NK_ID */ yytestcase(yyruleno==295); - case 296: /* index_name ::= NK_ID */ yytestcase(yyruleno==296); - case 297: /* topic_name ::= NK_ID */ yytestcase(yyruleno==297); - case 298: /* stream_name ::= NK_ID */ yytestcase(yyruleno==298); - case 299: /* cgroup_name ::= NK_ID */ yytestcase(yyruleno==299); - case 332: /* noarg_func ::= NOW */ yytestcase(yyruleno==332); - case 333: /* noarg_func ::= TODAY */ yytestcase(yyruleno==333); - case 334: /* noarg_func ::= TIMEZONE */ yytestcase(yyruleno==334); - case 335: /* star_func ::= COUNT */ yytestcase(yyruleno==335); - case 336: /* star_func ::= FIRST */ yytestcase(yyruleno==336); - case 337: /* star_func ::= LAST */ yytestcase(yyruleno==337); - case 338: /* star_func ::= LAST_ROW */ yytestcase(yyruleno==338); -{ yylhsminor.yy209 = yymsp[0].minor.yy0; } - yymsp[0].minor.yy209 = yylhsminor.yy209; + case 47: /* dnode_endpoint ::= NK_ID */ yytestcase(yyruleno==47); + case 48: /* dnode_endpoint ::= NK_IPTOKEN */ yytestcase(yyruleno==48); + case 294: /* db_name ::= NK_ID */ yytestcase(yyruleno==294); + case 295: /* table_name ::= NK_ID */ yytestcase(yyruleno==295); + case 296: /* column_name ::= NK_ID */ yytestcase(yyruleno==296); + case 297: /* function_name ::= NK_ID */ yytestcase(yyruleno==297); + case 298: /* table_alias ::= NK_ID */ yytestcase(yyruleno==298); + case 299: /* column_alias ::= NK_ID */ yytestcase(yyruleno==299); + case 300: /* user_name ::= NK_ID */ yytestcase(yyruleno==300); + case 301: /* index_name ::= NK_ID */ yytestcase(yyruleno==301); + case 302: /* topic_name ::= NK_ID */ yytestcase(yyruleno==302); + case 303: /* stream_name ::= NK_ID */ yytestcase(yyruleno==303); + case 304: /* cgroup_name ::= NK_ID */ yytestcase(yyruleno==304); + case 337: /* noarg_func ::= NOW */ yytestcase(yyruleno==337); + case 338: /* noarg_func ::= TODAY */ yytestcase(yyruleno==338); + case 339: /* noarg_func ::= TIMEZONE */ yytestcase(yyruleno==339); + case 340: /* star_func ::= COUNT */ yytestcase(yyruleno==340); + case 341: /* star_func ::= FIRST */ yytestcase(yyruleno==341); + case 342: /* star_func ::= LAST */ yytestcase(yyruleno==342); + case 343: /* star_func ::= LAST_ROW */ yytestcase(yyruleno==343); +{ yylhsminor.yy421 = yymsp[0].minor.yy0; } + yymsp[0].minor.yy421 = yylhsminor.yy421; break; case 49: /* cmd ::= ALTER LOCAL NK_STRING */ { pCxt->pRootNode = createAlterLocalStmt(pCxt, &yymsp[0].minor.yy0, NULL); } @@ -3287,1155 +3282,1166 @@ static YYACTIONTYPE yy_reduce( { pCxt->pRootNode = createDropComponentNodeStmt(pCxt, QUERY_NODE_DROP_MNODE_STMT, &yymsp[0].minor.yy0); } break; case 59: /* cmd ::= CREATE DATABASE not_exists_opt db_name db_options */ -{ pCxt->pRootNode = createCreateDatabaseStmt(pCxt, yymsp[-2].minor.yy137, &yymsp[-1].minor.yy209, yymsp[0].minor.yy632); } +{ pCxt->pRootNode = createCreateDatabaseStmt(pCxt, yymsp[-2].minor.yy621, &yymsp[-1].minor.yy421, yymsp[0].minor.yy674); } break; case 60: /* cmd ::= DROP DATABASE exists_opt db_name */ -{ pCxt->pRootNode = createDropDatabaseStmt(pCxt, yymsp[-1].minor.yy137, &yymsp[0].minor.yy209); } +{ pCxt->pRootNode = createDropDatabaseStmt(pCxt, yymsp[-1].minor.yy621, &yymsp[0].minor.yy421); } break; case 61: /* cmd ::= USE db_name */ -{ pCxt->pRootNode = createUseDatabaseStmt(pCxt, &yymsp[0].minor.yy209); } +{ pCxt->pRootNode = createUseDatabaseStmt(pCxt, &yymsp[0].minor.yy421); } break; case 62: /* cmd ::= ALTER DATABASE db_name alter_db_options */ -{ pCxt->pRootNode = createAlterDatabaseStmt(pCxt, &yymsp[-1].minor.yy209, yymsp[0].minor.yy632); } +{ pCxt->pRootNode = createAlterDatabaseStmt(pCxt, &yymsp[-1].minor.yy421, yymsp[0].minor.yy674); } break; case 63: /* not_exists_opt ::= IF NOT EXISTS */ -{ yymsp[-2].minor.yy137 = true; } +{ yymsp[-2].minor.yy621 = true; } break; case 64: /* not_exists_opt ::= */ case 66: /* exists_opt ::= */ yytestcase(yyruleno==66); - case 232: /* analyze_opt ::= */ yytestcase(yyruleno==232); - case 240: /* agg_func_opt ::= */ yytestcase(yyruleno==240); - case 391: /* set_quantifier_opt ::= */ yytestcase(yyruleno==391); -{ yymsp[1].minor.yy137 = false; } + case 237: /* analyze_opt ::= */ yytestcase(yyruleno==237); + case 245: /* agg_func_opt ::= */ yytestcase(yyruleno==245); + case 396: /* set_quantifier_opt ::= */ yytestcase(yyruleno==396); +{ yymsp[1].minor.yy621 = false; } break; case 65: /* exists_opt ::= IF EXISTS */ -{ yymsp[-1].minor.yy137 = true; } +{ yymsp[-1].minor.yy621 = true; } break; case 67: /* db_options ::= */ -{ yymsp[1].minor.yy632 = createDefaultDatabaseOptions(pCxt); } +{ yymsp[1].minor.yy674 = createDefaultDatabaseOptions(pCxt); } break; case 68: /* db_options ::= db_options BUFFER NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_BUFFER, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_BUFFER, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 69: /* db_options ::= db_options CACHELAST NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_CACHELAST, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_CACHELAST, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 70: /* db_options ::= db_options COMP NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_COMP, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_COMP, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 71: /* db_options ::= db_options DURATION NK_INTEGER */ case 72: /* db_options ::= db_options DURATION NK_VARIABLE */ yytestcase(yyruleno==72); -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_DAYS, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_DAYS, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 73: /* db_options ::= db_options FSYNC NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_FSYNC, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_FSYNC, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 74: /* db_options ::= db_options MAXROWS NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_MAXROWS, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_MAXROWS, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 75: /* db_options ::= db_options MINROWS NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_MINROWS, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_MINROWS, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 76: /* db_options ::= db_options KEEP integer_list */ case 77: /* db_options ::= db_options KEEP variable_list */ yytestcase(yyruleno==77); -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_KEEP, yymsp[0].minor.yy424); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_KEEP, yymsp[0].minor.yy530); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 78: /* db_options ::= db_options PAGES NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_PAGES, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_PAGES, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 79: /* db_options ::= db_options PAGESIZE NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_PAGESIZE, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_PAGESIZE, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 80: /* db_options ::= db_options PRECISION NK_STRING */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_PRECISION, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_PRECISION, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 81: /* db_options ::= db_options REPLICA NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_REPLICA, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_REPLICA, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 82: /* db_options ::= db_options STRICT NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_STRICT, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_STRICT, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 83: /* db_options ::= db_options WAL NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_WAL, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_WAL, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 84: /* db_options ::= db_options VGROUPS NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_VGROUPS, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_VGROUPS, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 85: /* db_options ::= db_options SINGLE_STABLE NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_SINGLE_STABLE, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_SINGLE_STABLE, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 86: /* db_options ::= db_options RETENTIONS retention_list */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_RETENTIONS, yymsp[0].minor.yy424); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_RETENTIONS, yymsp[0].minor.yy530); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 87: /* db_options ::= db_options SCHEMALESS NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_SCHEMALESS, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_SCHEMALESS, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 88: /* alter_db_options ::= alter_db_option */ -{ yylhsminor.yy632 = createAlterDatabaseOptions(pCxt); yylhsminor.yy632 = setAlterDatabaseOption(pCxt, yylhsminor.yy632, &yymsp[0].minor.yy605); } - yymsp[0].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterDatabaseOptions(pCxt); yylhsminor.yy674 = setAlterDatabaseOption(pCxt, yylhsminor.yy674, &yymsp[0].minor.yy557); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; case 89: /* alter_db_options ::= alter_db_options alter_db_option */ -{ yylhsminor.yy632 = setAlterDatabaseOption(pCxt, yymsp[-1].minor.yy632, &yymsp[0].minor.yy605); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setAlterDatabaseOption(pCxt, yymsp[-1].minor.yy674, &yymsp[0].minor.yy557); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; case 90: /* alter_db_option ::= BUFFER NK_INTEGER */ -{ yymsp[-1].minor.yy605.type = DB_OPTION_BUFFER; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy557.type = DB_OPTION_BUFFER; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; case 91: /* alter_db_option ::= CACHELAST NK_INTEGER */ -{ yymsp[-1].minor.yy605.type = DB_OPTION_CACHELAST; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy557.type = DB_OPTION_CACHELAST; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; case 92: /* alter_db_option ::= FSYNC NK_INTEGER */ -{ yymsp[-1].minor.yy605.type = DB_OPTION_FSYNC; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy557.type = DB_OPTION_FSYNC; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; case 93: /* alter_db_option ::= KEEP integer_list */ case 94: /* alter_db_option ::= KEEP variable_list */ yytestcase(yyruleno==94); -{ yymsp[-1].minor.yy605.type = DB_OPTION_KEEP; yymsp[-1].minor.yy605.pList = yymsp[0].minor.yy424; } +{ yymsp[-1].minor.yy557.type = DB_OPTION_KEEP; yymsp[-1].minor.yy557.pList = yymsp[0].minor.yy530; } break; case 95: /* alter_db_option ::= PAGES NK_INTEGER */ -{ yymsp[-1].minor.yy605.type = DB_OPTION_PAGES; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy557.type = DB_OPTION_PAGES; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; case 96: /* alter_db_option ::= REPLICA NK_INTEGER */ -{ yymsp[-1].minor.yy605.type = DB_OPTION_REPLICA; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy557.type = DB_OPTION_REPLICA; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; case 97: /* alter_db_option ::= STRICT NK_INTEGER */ -{ yymsp[-1].minor.yy605.type = DB_OPTION_STRICT; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy557.type = DB_OPTION_STRICT; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; case 98: /* alter_db_option ::= WAL NK_INTEGER */ -{ yymsp[-1].minor.yy605.type = DB_OPTION_WAL; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy557.type = DB_OPTION_WAL; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; case 99: /* integer_list ::= NK_INTEGER */ -{ yylhsminor.yy424 = createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy424 = yylhsminor.yy424; +{ yylhsminor.yy530 = createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy530 = yylhsminor.yy530; break; case 100: /* integer_list ::= integer_list NK_COMMA NK_INTEGER */ - case 261: /* dnode_list ::= dnode_list DNODE NK_INTEGER */ yytestcase(yyruleno==261); -{ yylhsminor.yy424 = addNodeToList(pCxt, yymsp[-2].minor.yy424, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } - yymsp[-2].minor.yy424 = yylhsminor.yy424; + case 266: /* dnode_list ::= dnode_list DNODE NK_INTEGER */ yytestcase(yyruleno==266); +{ yylhsminor.yy530 = addNodeToList(pCxt, yymsp[-2].minor.yy530, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } + yymsp[-2].minor.yy530 = yylhsminor.yy530; break; case 101: /* variable_list ::= NK_VARIABLE */ -{ yylhsminor.yy424 = createNodeList(pCxt, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy424 = yylhsminor.yy424; +{ yylhsminor.yy530 = createNodeList(pCxt, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy530 = yylhsminor.yy530; break; case 102: /* variable_list ::= variable_list NK_COMMA NK_VARIABLE */ -{ yylhsminor.yy424 = addNodeToList(pCxt, yymsp[-2].minor.yy424, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } - yymsp[-2].minor.yy424 = yylhsminor.yy424; +{ yylhsminor.yy530 = addNodeToList(pCxt, yymsp[-2].minor.yy530, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } + yymsp[-2].minor.yy530 = yylhsminor.yy530; break; case 103: /* retention_list ::= retention */ case 123: /* multi_create_clause ::= create_subtable_clause */ yytestcase(yyruleno==123); case 126: /* multi_drop_clause ::= drop_table_clause */ yytestcase(yyruleno==126); case 133: /* column_def_list ::= column_def */ yytestcase(yyruleno==133); - case 173: /* col_name_list ::= col_name */ yytestcase(yyruleno==173); - case 211: /* func_name_list ::= func_name */ yytestcase(yyruleno==211); - case 220: /* func_list ::= func */ yytestcase(yyruleno==220); - case 287: /* literal_list ::= signed_literal */ yytestcase(yyruleno==287); - case 341: /* other_para_list ::= star_func_para */ yytestcase(yyruleno==341); - case 396: /* select_sublist ::= select_item */ yytestcase(yyruleno==396); - case 445: /* sort_specification_list ::= sort_specification */ yytestcase(yyruleno==445); -{ yylhsminor.yy424 = createNodeList(pCxt, yymsp[0].minor.yy632); } - yymsp[0].minor.yy424 = yylhsminor.yy424; + case 176: /* rollup_func_list ::= rollup_func_name */ yytestcase(yyruleno==176); + case 181: /* col_name_list ::= col_name */ yytestcase(yyruleno==181); + case 225: /* func_list ::= func */ yytestcase(yyruleno==225); + case 292: /* literal_list ::= signed_literal */ yytestcase(yyruleno==292); + case 346: /* other_para_list ::= star_func_para */ yytestcase(yyruleno==346); + case 401: /* select_sublist ::= select_item */ yytestcase(yyruleno==401); + case 450: /* sort_specification_list ::= sort_specification */ yytestcase(yyruleno==450); +{ yylhsminor.yy530 = createNodeList(pCxt, yymsp[0].minor.yy674); } + yymsp[0].minor.yy530 = yylhsminor.yy530; break; case 104: /* retention_list ::= retention_list NK_COMMA retention */ case 134: /* column_def_list ::= column_def_list NK_COMMA column_def */ yytestcase(yyruleno==134); - case 174: /* col_name_list ::= col_name_list NK_COMMA col_name */ yytestcase(yyruleno==174); - case 212: /* func_name_list ::= func_name_list NK_COMMA func_name */ yytestcase(yyruleno==212); - case 221: /* func_list ::= func_list NK_COMMA func */ yytestcase(yyruleno==221); - case 288: /* literal_list ::= literal_list NK_COMMA signed_literal */ yytestcase(yyruleno==288); - case 342: /* other_para_list ::= other_para_list NK_COMMA star_func_para */ yytestcase(yyruleno==342); - case 397: /* select_sublist ::= select_sublist NK_COMMA select_item */ yytestcase(yyruleno==397); - case 446: /* sort_specification_list ::= sort_specification_list NK_COMMA sort_specification */ yytestcase(yyruleno==446); -{ yylhsminor.yy424 = addNodeToList(pCxt, yymsp[-2].minor.yy424, yymsp[0].minor.yy632); } - yymsp[-2].minor.yy424 = yylhsminor.yy424; + case 177: /* rollup_func_list ::= rollup_func_list NK_COMMA rollup_func_name */ yytestcase(yyruleno==177); + case 182: /* col_name_list ::= col_name_list NK_COMMA col_name */ yytestcase(yyruleno==182); + case 226: /* func_list ::= func_list NK_COMMA func */ yytestcase(yyruleno==226); + case 293: /* literal_list ::= literal_list NK_COMMA signed_literal */ yytestcase(yyruleno==293); + case 347: /* other_para_list ::= other_para_list NK_COMMA star_func_para */ yytestcase(yyruleno==347); + case 402: /* select_sublist ::= select_sublist NK_COMMA select_item */ yytestcase(yyruleno==402); + case 451: /* sort_specification_list ::= sort_specification_list NK_COMMA sort_specification */ yytestcase(yyruleno==451); +{ yylhsminor.yy530 = addNodeToList(pCxt, yymsp[-2].minor.yy530, yymsp[0].minor.yy674); } + yymsp[-2].minor.yy530 = yylhsminor.yy530; break; case 105: /* retention ::= NK_VARIABLE NK_COLON NK_VARIABLE */ -{ yylhsminor.yy632 = createNodeListNodeEx(pCxt, createDurationValueNode(pCxt, &yymsp[-2].minor.yy0), createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createNodeListNodeEx(pCxt, createDurationValueNode(pCxt, &yymsp[-2].minor.yy0), createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 106: /* cmd ::= CREATE TABLE not_exists_opt full_table_name NK_LP column_def_list NK_RP tags_def_opt table_options */ case 108: /* cmd ::= CREATE STABLE not_exists_opt full_table_name NK_LP column_def_list NK_RP tags_def table_options */ yytestcase(yyruleno==108); -{ pCxt->pRootNode = createCreateTableStmt(pCxt, yymsp[-6].minor.yy137, yymsp[-5].minor.yy632, yymsp[-3].minor.yy424, yymsp[-1].minor.yy424, yymsp[0].minor.yy632); } +{ pCxt->pRootNode = createCreateTableStmt(pCxt, yymsp[-6].minor.yy621, yymsp[-5].minor.yy674, yymsp[-3].minor.yy530, yymsp[-1].minor.yy530, yymsp[0].minor.yy674); } break; case 107: /* cmd ::= CREATE TABLE multi_create_clause */ -{ pCxt->pRootNode = createCreateMultiTableStmt(pCxt, yymsp[0].minor.yy424); } +{ pCxt->pRootNode = createCreateMultiTableStmt(pCxt, yymsp[0].minor.yy530); } break; case 109: /* cmd ::= DROP TABLE multi_drop_clause */ -{ pCxt->pRootNode = createDropTableStmt(pCxt, yymsp[0].minor.yy424); } +{ pCxt->pRootNode = createDropTableStmt(pCxt, yymsp[0].minor.yy530); } break; case 110: /* cmd ::= DROP STABLE exists_opt full_table_name */ -{ pCxt->pRootNode = createDropSuperTableStmt(pCxt, yymsp[-1].minor.yy137, yymsp[0].minor.yy632); } +{ pCxt->pRootNode = createDropSuperTableStmt(pCxt, yymsp[-1].minor.yy621, yymsp[0].minor.yy674); } break; case 111: /* cmd ::= ALTER TABLE alter_table_clause */ case 112: /* cmd ::= ALTER STABLE alter_table_clause */ yytestcase(yyruleno==112); - case 264: /* cmd ::= query_expression */ yytestcase(yyruleno==264); -{ pCxt->pRootNode = yymsp[0].minor.yy632; } + case 269: /* cmd ::= query_expression */ yytestcase(yyruleno==269); +{ pCxt->pRootNode = yymsp[0].minor.yy674; } break; case 113: /* alter_table_clause ::= full_table_name alter_table_options */ -{ yylhsminor.yy632 = createAlterTableModifyOptions(pCxt, yymsp[-1].minor.yy632, yymsp[0].minor.yy632); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableModifyOptions(pCxt, yymsp[-1].minor.yy674, yymsp[0].minor.yy674); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; case 114: /* alter_table_clause ::= full_table_name ADD COLUMN column_name type_name */ -{ yylhsminor.yy632 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy632, TSDB_ALTER_TABLE_ADD_COLUMN, &yymsp[-1].minor.yy209, yymsp[0].minor.yy304); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy674, TSDB_ALTER_TABLE_ADD_COLUMN, &yymsp[-1].minor.yy421, yymsp[0].minor.yy690); } + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; case 115: /* alter_table_clause ::= full_table_name DROP COLUMN column_name */ -{ yylhsminor.yy632 = createAlterTableDropCol(pCxt, yymsp[-3].minor.yy632, TSDB_ALTER_TABLE_DROP_COLUMN, &yymsp[0].minor.yy209); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableDropCol(pCxt, yymsp[-3].minor.yy674, TSDB_ALTER_TABLE_DROP_COLUMN, &yymsp[0].minor.yy421); } + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; case 116: /* alter_table_clause ::= full_table_name MODIFY COLUMN column_name type_name */ -{ yylhsminor.yy632 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy632, TSDB_ALTER_TABLE_UPDATE_COLUMN_BYTES, &yymsp[-1].minor.yy209, yymsp[0].minor.yy304); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy674, TSDB_ALTER_TABLE_UPDATE_COLUMN_BYTES, &yymsp[-1].minor.yy421, yymsp[0].minor.yy690); } + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; case 117: /* alter_table_clause ::= full_table_name RENAME COLUMN column_name column_name */ -{ yylhsminor.yy632 = createAlterTableRenameCol(pCxt, yymsp[-4].minor.yy632, TSDB_ALTER_TABLE_UPDATE_COLUMN_NAME, &yymsp[-1].minor.yy209, &yymsp[0].minor.yy209); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableRenameCol(pCxt, yymsp[-4].minor.yy674, TSDB_ALTER_TABLE_UPDATE_COLUMN_NAME, &yymsp[-1].minor.yy421, &yymsp[0].minor.yy421); } + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; case 118: /* alter_table_clause ::= full_table_name ADD TAG column_name type_name */ -{ yylhsminor.yy632 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy632, TSDB_ALTER_TABLE_ADD_TAG, &yymsp[-1].minor.yy209, yymsp[0].minor.yy304); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy674, TSDB_ALTER_TABLE_ADD_TAG, &yymsp[-1].minor.yy421, yymsp[0].minor.yy690); } + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; case 119: /* alter_table_clause ::= full_table_name DROP TAG column_name */ -{ yylhsminor.yy632 = createAlterTableDropCol(pCxt, yymsp[-3].minor.yy632, TSDB_ALTER_TABLE_DROP_TAG, &yymsp[0].minor.yy209); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableDropCol(pCxt, yymsp[-3].minor.yy674, TSDB_ALTER_TABLE_DROP_TAG, &yymsp[0].minor.yy421); } + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; case 120: /* alter_table_clause ::= full_table_name MODIFY TAG column_name type_name */ -{ yylhsminor.yy632 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy632, TSDB_ALTER_TABLE_UPDATE_TAG_BYTES, &yymsp[-1].minor.yy209, yymsp[0].minor.yy304); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy674, TSDB_ALTER_TABLE_UPDATE_TAG_BYTES, &yymsp[-1].minor.yy421, yymsp[0].minor.yy690); } + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; case 121: /* alter_table_clause ::= full_table_name RENAME TAG column_name column_name */ -{ yylhsminor.yy632 = createAlterTableRenameCol(pCxt, yymsp[-4].minor.yy632, TSDB_ALTER_TABLE_UPDATE_TAG_NAME, &yymsp[-1].minor.yy209, &yymsp[0].minor.yy209); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableRenameCol(pCxt, yymsp[-4].minor.yy674, TSDB_ALTER_TABLE_UPDATE_TAG_NAME, &yymsp[-1].minor.yy421, &yymsp[0].minor.yy421); } + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; case 122: /* alter_table_clause ::= full_table_name SET TAG column_name NK_EQ signed_literal */ -{ yylhsminor.yy632 = createAlterTableSetTag(pCxt, yymsp[-5].minor.yy632, &yymsp[-2].minor.yy209, yymsp[0].minor.yy632); } - yymsp[-5].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableSetTag(pCxt, yymsp[-5].minor.yy674, &yymsp[-2].minor.yy421, yymsp[0].minor.yy674); } + yymsp[-5].minor.yy674 = yylhsminor.yy674; break; case 124: /* multi_create_clause ::= multi_create_clause create_subtable_clause */ case 127: /* multi_drop_clause ::= multi_drop_clause drop_table_clause */ yytestcase(yyruleno==127); -{ yylhsminor.yy424 = addNodeToList(pCxt, yymsp[-1].minor.yy424, yymsp[0].minor.yy632); } - yymsp[-1].minor.yy424 = yylhsminor.yy424; +{ yylhsminor.yy530 = addNodeToList(pCxt, yymsp[-1].minor.yy530, yymsp[0].minor.yy674); } + yymsp[-1].minor.yy530 = yylhsminor.yy530; break; case 125: /* create_subtable_clause ::= not_exists_opt full_table_name USING full_table_name specific_tags_opt TAGS NK_LP literal_list NK_RP table_options */ -{ yylhsminor.yy632 = createCreateSubTableClause(pCxt, yymsp[-9].minor.yy137, yymsp[-8].minor.yy632, yymsp[-6].minor.yy632, yymsp[-5].minor.yy424, yymsp[-2].minor.yy424, yymsp[0].minor.yy632); } - yymsp[-9].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createCreateSubTableClause(pCxt, yymsp[-9].minor.yy621, yymsp[-8].minor.yy674, yymsp[-6].minor.yy674, yymsp[-5].minor.yy530, yymsp[-2].minor.yy530, yymsp[0].minor.yy674); } + yymsp[-9].minor.yy674 = yylhsminor.yy674; break; case 128: /* drop_table_clause ::= exists_opt full_table_name */ -{ yylhsminor.yy632 = createDropTableClause(pCxt, yymsp[-1].minor.yy137, yymsp[0].minor.yy632); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createDropTableClause(pCxt, yymsp[-1].minor.yy621, yymsp[0].minor.yy674); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; case 129: /* specific_tags_opt ::= */ case 160: /* tags_def_opt ::= */ yytestcase(yyruleno==160); - case 404: /* partition_by_clause_opt ::= */ yytestcase(yyruleno==404); - case 421: /* group_by_clause_opt ::= */ yytestcase(yyruleno==421); - case 433: /* order_by_clause_opt ::= */ yytestcase(yyruleno==433); -{ yymsp[1].minor.yy424 = NULL; } + case 409: /* partition_by_clause_opt ::= */ yytestcase(yyruleno==409); + case 426: /* group_by_clause_opt ::= */ yytestcase(yyruleno==426); + case 438: /* order_by_clause_opt ::= */ yytestcase(yyruleno==438); +{ yymsp[1].minor.yy530 = NULL; } break; case 130: /* specific_tags_opt ::= NK_LP col_name_list NK_RP */ -{ yymsp[-2].minor.yy424 = yymsp[-1].minor.yy424; } +{ yymsp[-2].minor.yy530 = yymsp[-1].minor.yy530; } break; case 131: /* full_table_name ::= table_name */ -{ yylhsminor.yy632 = createRealTableNode(pCxt, NULL, &yymsp[0].minor.yy209, NULL); } - yymsp[0].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createRealTableNode(pCxt, NULL, &yymsp[0].minor.yy421, NULL); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; case 132: /* full_table_name ::= db_name NK_DOT table_name */ -{ yylhsminor.yy632 = createRealTableNode(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209, NULL); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createRealTableNode(pCxt, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy421, NULL); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 135: /* column_def ::= column_name type_name */ -{ yylhsminor.yy632 = createColumnDefNode(pCxt, &yymsp[-1].minor.yy209, yymsp[0].minor.yy304, NULL); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createColumnDefNode(pCxt, &yymsp[-1].minor.yy421, yymsp[0].minor.yy690, NULL); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; case 136: /* column_def ::= column_name type_name COMMENT NK_STRING */ -{ yylhsminor.yy632 = createColumnDefNode(pCxt, &yymsp[-3].minor.yy209, yymsp[-2].minor.yy304, &yymsp[0].minor.yy0); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createColumnDefNode(pCxt, &yymsp[-3].minor.yy421, yymsp[-2].minor.yy690, &yymsp[0].minor.yy0); } + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; case 137: /* type_name ::= BOOL */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_BOOL); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_BOOL); } break; case 138: /* type_name ::= TINYINT */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_TINYINT); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_TINYINT); } break; case 139: /* type_name ::= SMALLINT */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_SMALLINT); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_SMALLINT); } break; case 140: /* type_name ::= INT */ case 141: /* type_name ::= INTEGER */ yytestcase(yyruleno==141); -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_INT); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_INT); } break; case 142: /* type_name ::= BIGINT */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_BIGINT); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_BIGINT); } break; case 143: /* type_name ::= FLOAT */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_FLOAT); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_FLOAT); } break; case 144: /* type_name ::= DOUBLE */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_DOUBLE); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_DOUBLE); } break; case 145: /* type_name ::= BINARY NK_LP NK_INTEGER NK_RP */ -{ yymsp[-3].minor.yy304 = createVarLenDataType(TSDB_DATA_TYPE_BINARY, &yymsp[-1].minor.yy0); } +{ yymsp[-3].minor.yy690 = createVarLenDataType(TSDB_DATA_TYPE_BINARY, &yymsp[-1].minor.yy0); } break; case 146: /* type_name ::= TIMESTAMP */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_TIMESTAMP); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_TIMESTAMP); } break; case 147: /* type_name ::= NCHAR NK_LP NK_INTEGER NK_RP */ -{ yymsp[-3].minor.yy304 = createVarLenDataType(TSDB_DATA_TYPE_NCHAR, &yymsp[-1].minor.yy0); } +{ yymsp[-3].minor.yy690 = createVarLenDataType(TSDB_DATA_TYPE_NCHAR, &yymsp[-1].minor.yy0); } break; case 148: /* type_name ::= TINYINT UNSIGNED */ -{ yymsp[-1].minor.yy304 = createDataType(TSDB_DATA_TYPE_UTINYINT); } +{ yymsp[-1].minor.yy690 = createDataType(TSDB_DATA_TYPE_UTINYINT); } break; case 149: /* type_name ::= SMALLINT UNSIGNED */ -{ yymsp[-1].minor.yy304 = createDataType(TSDB_DATA_TYPE_USMALLINT); } +{ yymsp[-1].minor.yy690 = createDataType(TSDB_DATA_TYPE_USMALLINT); } break; case 150: /* type_name ::= INT UNSIGNED */ -{ yymsp[-1].minor.yy304 = createDataType(TSDB_DATA_TYPE_UINT); } +{ yymsp[-1].minor.yy690 = createDataType(TSDB_DATA_TYPE_UINT); } break; case 151: /* type_name ::= BIGINT UNSIGNED */ -{ yymsp[-1].minor.yy304 = createDataType(TSDB_DATA_TYPE_UBIGINT); } +{ yymsp[-1].minor.yy690 = createDataType(TSDB_DATA_TYPE_UBIGINT); } break; case 152: /* type_name ::= JSON */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_JSON); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_JSON); } break; case 153: /* type_name ::= VARCHAR NK_LP NK_INTEGER NK_RP */ -{ yymsp[-3].minor.yy304 = createVarLenDataType(TSDB_DATA_TYPE_VARCHAR, &yymsp[-1].minor.yy0); } +{ yymsp[-3].minor.yy690 = createVarLenDataType(TSDB_DATA_TYPE_VARCHAR, &yymsp[-1].minor.yy0); } break; case 154: /* type_name ::= MEDIUMBLOB */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_MEDIUMBLOB); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_MEDIUMBLOB); } break; case 155: /* type_name ::= BLOB */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_BLOB); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_BLOB); } break; case 156: /* type_name ::= VARBINARY NK_LP NK_INTEGER NK_RP */ -{ yymsp[-3].minor.yy304 = createVarLenDataType(TSDB_DATA_TYPE_VARBINARY, &yymsp[-1].minor.yy0); } +{ yymsp[-3].minor.yy690 = createVarLenDataType(TSDB_DATA_TYPE_VARBINARY, &yymsp[-1].minor.yy0); } break; case 157: /* type_name ::= DECIMAL */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_DECIMAL); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_DECIMAL); } break; case 158: /* type_name ::= DECIMAL NK_LP NK_INTEGER NK_RP */ -{ yymsp[-3].minor.yy304 = createDataType(TSDB_DATA_TYPE_DECIMAL); } +{ yymsp[-3].minor.yy690 = createDataType(TSDB_DATA_TYPE_DECIMAL); } break; case 159: /* type_name ::= DECIMAL NK_LP NK_INTEGER NK_COMMA NK_INTEGER NK_RP */ -{ yymsp[-5].minor.yy304 = createDataType(TSDB_DATA_TYPE_DECIMAL); } +{ yymsp[-5].minor.yy690 = createDataType(TSDB_DATA_TYPE_DECIMAL); } break; case 161: /* tags_def_opt ::= tags_def */ - case 340: /* star_func_para_list ::= other_para_list */ yytestcase(yyruleno==340); - case 395: /* select_list ::= select_sublist */ yytestcase(yyruleno==395); -{ yylhsminor.yy424 = yymsp[0].minor.yy424; } - yymsp[0].minor.yy424 = yylhsminor.yy424; + case 345: /* star_func_para_list ::= other_para_list */ yytestcase(yyruleno==345); + case 400: /* select_list ::= select_sublist */ yytestcase(yyruleno==400); +{ yylhsminor.yy530 = yymsp[0].minor.yy530; } + yymsp[0].minor.yy530 = yylhsminor.yy530; break; case 162: /* tags_def ::= TAGS NK_LP column_def_list NK_RP */ -{ yymsp[-3].minor.yy424 = yymsp[-1].minor.yy424; } +{ yymsp[-3].minor.yy530 = yymsp[-1].minor.yy530; } break; case 163: /* table_options ::= */ -{ yymsp[1].minor.yy632 = createDefaultTableOptions(pCxt); } +{ yymsp[1].minor.yy674 = createDefaultTableOptions(pCxt); } break; case 164: /* table_options ::= table_options COMMENT NK_STRING */ -{ yylhsminor.yy632 = setTableOption(pCxt, yymsp[-2].minor.yy632, TABLE_OPTION_COMMENT, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setTableOption(pCxt, yymsp[-2].minor.yy674, TABLE_OPTION_COMMENT, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 165: /* table_options ::= table_options FILE_FACTOR NK_FLOAT */ -{ yylhsminor.yy632 = setTableOption(pCxt, yymsp[-2].minor.yy632, TABLE_OPTION_FILE_FACTOR, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 165: /* table_options ::= table_options MAX_DELAY duration_list */ +{ yylhsminor.yy674 = setTableOption(pCxt, yymsp[-2].minor.yy674, TABLE_OPTION_MAXDELAY, yymsp[0].minor.yy530); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 166: /* table_options ::= table_options ROLLUP NK_LP func_name_list NK_RP */ -{ yylhsminor.yy632 = setTableOption(pCxt, yymsp[-4].minor.yy632, TABLE_OPTION_ROLLUP, yymsp[-1].minor.yy424); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; + case 166: /* table_options ::= table_options WATERMARK duration_list */ +{ yylhsminor.yy674 = setTableOption(pCxt, yymsp[-2].minor.yy674, TABLE_OPTION_WATERMARK, yymsp[0].minor.yy530); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 167: /* table_options ::= table_options TTL NK_INTEGER */ -{ yylhsminor.yy632 = setTableOption(pCxt, yymsp[-2].minor.yy632, TABLE_OPTION_TTL, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 167: /* table_options ::= table_options ROLLUP NK_LP rollup_func_list NK_RP */ +{ yylhsminor.yy674 = setTableOption(pCxt, yymsp[-4].minor.yy674, TABLE_OPTION_ROLLUP, yymsp[-1].minor.yy530); } + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; - case 168: /* table_options ::= table_options SMA NK_LP col_name_list NK_RP */ -{ yylhsminor.yy632 = setTableOption(pCxt, yymsp[-4].minor.yy632, TABLE_OPTION_SMA, yymsp[-1].minor.yy424); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; + case 168: /* table_options ::= table_options TTL NK_INTEGER */ +{ yylhsminor.yy674 = setTableOption(pCxt, yymsp[-2].minor.yy674, TABLE_OPTION_TTL, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 169: /* alter_table_options ::= alter_table_option */ -{ yylhsminor.yy632 = createAlterTableOptions(pCxt); yylhsminor.yy632 = setTableOption(pCxt, yylhsminor.yy632, yymsp[0].minor.yy605.type, &yymsp[0].minor.yy605.val); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 169: /* table_options ::= table_options SMA NK_LP col_name_list NK_RP */ +{ yylhsminor.yy674 = setTableOption(pCxt, yymsp[-4].minor.yy674, TABLE_OPTION_SMA, yymsp[-1].minor.yy530); } + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; - case 170: /* alter_table_options ::= alter_table_options alter_table_option */ -{ yylhsminor.yy632 = setTableOption(pCxt, yymsp[-1].minor.yy632, yymsp[0].minor.yy605.type, &yymsp[0].minor.yy605.val); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + case 170: /* alter_table_options ::= alter_table_option */ +{ yylhsminor.yy674 = createAlterTableOptions(pCxt); yylhsminor.yy674 = setTableOption(pCxt, yylhsminor.yy674, yymsp[0].minor.yy557.type, &yymsp[0].minor.yy557.val); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 171: /* alter_table_option ::= COMMENT NK_STRING */ -{ yymsp[-1].minor.yy605.type = TABLE_OPTION_COMMENT; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } + case 171: /* alter_table_options ::= alter_table_options alter_table_option */ +{ yylhsminor.yy674 = setTableOption(pCxt, yymsp[-1].minor.yy674, yymsp[0].minor.yy557.type, &yymsp[0].minor.yy557.val); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 172: /* alter_table_option ::= TTL NK_INTEGER */ -{ yymsp[-1].minor.yy605.type = TABLE_OPTION_TTL; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } + case 172: /* alter_table_option ::= COMMENT NK_STRING */ +{ yymsp[-1].minor.yy557.type = TABLE_OPTION_COMMENT; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; - case 175: /* col_name ::= column_name */ -{ yylhsminor.yy632 = createColumnNode(pCxt, NULL, &yymsp[0].minor.yy209); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 173: /* alter_table_option ::= TTL NK_INTEGER */ +{ yymsp[-1].minor.yy557.type = TABLE_OPTION_TTL; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; - case 176: /* cmd ::= SHOW DNODES */ + case 174: /* duration_list ::= duration_literal */ + case 319: /* expression_list ::= expression */ yytestcase(yyruleno==319); +{ yylhsminor.yy530 = createNodeList(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy674)); } + yymsp[0].minor.yy530 = yylhsminor.yy530; + break; + case 175: /* duration_list ::= duration_list NK_COMMA duration_literal */ + case 320: /* expression_list ::= expression_list NK_COMMA expression */ yytestcase(yyruleno==320); +{ yylhsminor.yy530 = addNodeToList(pCxt, yymsp[-2].minor.yy530, releaseRawExprNode(pCxt, yymsp[0].minor.yy674)); } + yymsp[-2].minor.yy530 = yylhsminor.yy530; + break; + case 178: /* rollup_func_name ::= function_name */ +{ yylhsminor.yy674 = createFunctionNode(pCxt, &yymsp[0].minor.yy421, NULL); } + yymsp[0].minor.yy674 = yylhsminor.yy674; + break; + case 179: /* rollup_func_name ::= FIRST */ + case 180: /* rollup_func_name ::= LAST */ yytestcase(yyruleno==180); +{ yylhsminor.yy674 = createFunctionNode(pCxt, &yymsp[0].minor.yy0, NULL); } + yymsp[0].minor.yy674 = yylhsminor.yy674; + break; + case 183: /* col_name ::= column_name */ +{ yylhsminor.yy674 = createColumnNode(pCxt, NULL, &yymsp[0].minor.yy421); } + yymsp[0].minor.yy674 = yylhsminor.yy674; + break; + case 184: /* cmd ::= SHOW DNODES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_DNODES_STMT, NULL, NULL); } break; - case 177: /* cmd ::= SHOW USERS */ + case 185: /* cmd ::= SHOW USERS */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_USERS_STMT, NULL, NULL); } break; - case 178: /* cmd ::= SHOW DATABASES */ + case 186: /* cmd ::= SHOW DATABASES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_DATABASES_STMT, NULL, NULL); } break; - case 179: /* cmd ::= SHOW db_name_cond_opt TABLES like_pattern_opt */ -{ pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_TABLES_STMT, yymsp[-2].minor.yy632, yymsp[0].minor.yy632); } + case 187: /* cmd ::= SHOW db_name_cond_opt TABLES like_pattern_opt */ +{ pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_TABLES_STMT, yymsp[-2].minor.yy674, yymsp[0].minor.yy674); } break; - case 180: /* cmd ::= SHOW db_name_cond_opt STABLES like_pattern_opt */ -{ pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_STABLES_STMT, yymsp[-2].minor.yy632, yymsp[0].minor.yy632); } + case 188: /* cmd ::= SHOW db_name_cond_opt STABLES like_pattern_opt */ +{ pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_STABLES_STMT, yymsp[-2].minor.yy674, yymsp[0].minor.yy674); } break; - case 181: /* cmd ::= SHOW db_name_cond_opt VGROUPS */ -{ pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_VGROUPS_STMT, yymsp[-1].minor.yy632, NULL); } + case 189: /* cmd ::= SHOW db_name_cond_opt VGROUPS */ +{ pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_VGROUPS_STMT, yymsp[-1].minor.yy674, NULL); } break; - case 182: /* cmd ::= SHOW MNODES */ + case 190: /* cmd ::= SHOW MNODES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_MNODES_STMT, NULL, NULL); } break; - case 183: /* cmd ::= SHOW MODULES */ + case 191: /* cmd ::= SHOW MODULES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_MODULES_STMT, NULL, NULL); } break; - case 184: /* cmd ::= SHOW QNODES */ + case 192: /* cmd ::= SHOW QNODES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_QNODES_STMT, NULL, NULL); } break; - case 185: /* cmd ::= SHOW FUNCTIONS */ + case 193: /* cmd ::= SHOW FUNCTIONS */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_FUNCTIONS_STMT, NULL, NULL); } break; - case 186: /* cmd ::= SHOW INDEXES FROM table_name_cond from_db_opt */ -{ pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_INDEXES_STMT, yymsp[-1].minor.yy632, yymsp[0].minor.yy632); } + case 194: /* cmd ::= SHOW INDEXES FROM table_name_cond from_db_opt */ +{ pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_INDEXES_STMT, yymsp[-1].minor.yy674, yymsp[0].minor.yy674); } break; - case 187: /* cmd ::= SHOW STREAMS */ + case 195: /* cmd ::= SHOW STREAMS */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_STREAMS_STMT, NULL, NULL); } break; - case 188: /* cmd ::= SHOW ACCOUNTS */ + case 196: /* cmd ::= SHOW ACCOUNTS */ { pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_EXPRIE_STATEMENT); } break; - case 189: /* cmd ::= SHOW APPS */ + case 197: /* cmd ::= SHOW APPS */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_APPS_STMT, NULL, NULL); } break; - case 190: /* cmd ::= SHOW CONNECTIONS */ + case 198: /* cmd ::= SHOW CONNECTIONS */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_CONNECTIONS_STMT, NULL, NULL); } break; - case 191: /* cmd ::= SHOW LICENCE */ - case 192: /* cmd ::= SHOW GRANTS */ yytestcase(yyruleno==192); + case 199: /* cmd ::= SHOW LICENCE */ + case 200: /* cmd ::= SHOW GRANTS */ yytestcase(yyruleno==200); { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_LICENCE_STMT, NULL, NULL); } break; - case 193: /* cmd ::= SHOW CREATE DATABASE db_name */ -{ pCxt->pRootNode = createShowCreateDatabaseStmt(pCxt, &yymsp[0].minor.yy209); } + case 201: /* cmd ::= SHOW CREATE DATABASE db_name */ +{ pCxt->pRootNode = createShowCreateDatabaseStmt(pCxt, &yymsp[0].minor.yy421); } break; - case 194: /* cmd ::= SHOW CREATE TABLE full_table_name */ -{ pCxt->pRootNode = createShowCreateTableStmt(pCxt, QUERY_NODE_SHOW_CREATE_TABLE_STMT, yymsp[0].minor.yy632); } + case 202: /* cmd ::= SHOW CREATE TABLE full_table_name */ +{ pCxt->pRootNode = createShowCreateTableStmt(pCxt, QUERY_NODE_SHOW_CREATE_TABLE_STMT, yymsp[0].minor.yy674); } break; - case 195: /* cmd ::= SHOW CREATE STABLE full_table_name */ -{ pCxt->pRootNode = createShowCreateTableStmt(pCxt, QUERY_NODE_SHOW_CREATE_STABLE_STMT, yymsp[0].minor.yy632); } + case 203: /* cmd ::= SHOW CREATE STABLE full_table_name */ +{ pCxt->pRootNode = createShowCreateTableStmt(pCxt, QUERY_NODE_SHOW_CREATE_STABLE_STMT, yymsp[0].minor.yy674); } break; - case 196: /* cmd ::= SHOW QUERIES */ + case 204: /* cmd ::= SHOW QUERIES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_QUERIES_STMT, NULL, NULL); } break; - case 197: /* cmd ::= SHOW SCORES */ + case 205: /* cmd ::= SHOW SCORES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_SCORES_STMT, NULL, NULL); } break; - case 198: /* cmd ::= SHOW TOPICS */ + case 206: /* cmd ::= SHOW TOPICS */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_TOPICS_STMT, NULL, NULL); } break; - case 199: /* cmd ::= SHOW VARIABLES */ + case 207: /* cmd ::= SHOW VARIABLES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_VARIABLE_STMT, NULL, NULL); } break; - case 200: /* cmd ::= SHOW BNODES */ + case 208: /* cmd ::= SHOW BNODES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_BNODES_STMT, NULL, NULL); } break; - case 201: /* cmd ::= SHOW SNODES */ + case 209: /* cmd ::= SHOW SNODES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_SNODES_STMT, NULL, NULL); } break; - case 202: /* cmd ::= SHOW CLUSTER */ + case 210: /* cmd ::= SHOW CLUSTER */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_CLUSTER_STMT, NULL, NULL); } break; - case 203: /* cmd ::= SHOW TRANSACTIONS */ + case 211: /* cmd ::= SHOW TRANSACTIONS */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_TRANSACTIONS_STMT, NULL, NULL); } break; - case 204: /* db_name_cond_opt ::= */ - case 209: /* from_db_opt ::= */ yytestcase(yyruleno==209); -{ yymsp[1].minor.yy632 = createDefaultDatabaseCondValue(pCxt); } + case 212: /* db_name_cond_opt ::= */ + case 217: /* from_db_opt ::= */ yytestcase(yyruleno==217); +{ yymsp[1].minor.yy674 = createDefaultDatabaseCondValue(pCxt); } break; - case 205: /* db_name_cond_opt ::= db_name NK_DOT */ -{ yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[-1].minor.yy209); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + case 213: /* db_name_cond_opt ::= db_name NK_DOT */ +{ yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[-1].minor.yy421); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 206: /* like_pattern_opt ::= */ - case 217: /* index_options ::= */ yytestcase(yyruleno==217); - case 246: /* into_opt ::= */ yytestcase(yyruleno==246); - case 402: /* where_clause_opt ::= */ yytestcase(yyruleno==402); - case 406: /* twindow_clause_opt ::= */ yytestcase(yyruleno==406); - case 411: /* sliding_opt ::= */ yytestcase(yyruleno==411); - case 413: /* fill_opt ::= */ yytestcase(yyruleno==413); - case 425: /* having_clause_opt ::= */ yytestcase(yyruleno==425); - case 435: /* slimit_clause_opt ::= */ yytestcase(yyruleno==435); - case 439: /* limit_clause_opt ::= */ yytestcase(yyruleno==439); -{ yymsp[1].minor.yy632 = NULL; } + case 214: /* like_pattern_opt ::= */ + case 222: /* index_options ::= */ yytestcase(yyruleno==222); + case 251: /* into_opt ::= */ yytestcase(yyruleno==251); + case 407: /* where_clause_opt ::= */ yytestcase(yyruleno==407); + case 411: /* twindow_clause_opt ::= */ yytestcase(yyruleno==411); + case 416: /* sliding_opt ::= */ yytestcase(yyruleno==416); + case 418: /* fill_opt ::= */ yytestcase(yyruleno==418); + case 430: /* having_clause_opt ::= */ yytestcase(yyruleno==430); + case 440: /* slimit_clause_opt ::= */ yytestcase(yyruleno==440); + case 444: /* limit_clause_opt ::= */ yytestcase(yyruleno==444); +{ yymsp[1].minor.yy674 = NULL; } break; - case 207: /* like_pattern_opt ::= LIKE NK_STRING */ -{ yymsp[-1].minor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0); } + case 215: /* like_pattern_opt ::= LIKE NK_STRING */ +{ yymsp[-1].minor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0); } break; - case 208: /* table_name_cond ::= table_name */ -{ yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy209); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 216: /* table_name_cond ::= table_name */ +{ yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy421); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 210: /* from_db_opt ::= FROM db_name */ -{ yymsp[-1].minor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy209); } + case 218: /* from_db_opt ::= FROM db_name */ +{ yymsp[-1].minor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy421); } break; - case 213: /* func_name ::= function_name */ -{ yylhsminor.yy632 = createFunctionNode(pCxt, &yymsp[0].minor.yy209, NULL); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 219: /* cmd ::= CREATE SMA INDEX not_exists_opt index_name ON table_name index_options */ +{ pCxt->pRootNode = createCreateIndexStmt(pCxt, INDEX_TYPE_SMA, yymsp[-4].minor.yy621, &yymsp[-3].minor.yy421, &yymsp[-1].minor.yy421, NULL, yymsp[0].minor.yy674); } break; - case 214: /* cmd ::= CREATE SMA INDEX not_exists_opt index_name ON table_name index_options */ -{ pCxt->pRootNode = createCreateIndexStmt(pCxt, INDEX_TYPE_SMA, yymsp[-4].minor.yy137, &yymsp[-3].minor.yy209, &yymsp[-1].minor.yy209, NULL, yymsp[0].minor.yy632); } + case 220: /* cmd ::= CREATE FULLTEXT INDEX not_exists_opt index_name ON table_name NK_LP col_name_list NK_RP */ +{ pCxt->pRootNode = createCreateIndexStmt(pCxt, INDEX_TYPE_FULLTEXT, yymsp[-6].minor.yy621, &yymsp[-5].minor.yy421, &yymsp[-3].minor.yy421, yymsp[-1].minor.yy530, NULL); } break; - case 215: /* cmd ::= CREATE FULLTEXT INDEX not_exists_opt index_name ON table_name NK_LP col_name_list NK_RP */ -{ pCxt->pRootNode = createCreateIndexStmt(pCxt, INDEX_TYPE_FULLTEXT, yymsp[-6].minor.yy137, &yymsp[-5].minor.yy209, &yymsp[-3].minor.yy209, yymsp[-1].minor.yy424, NULL); } + case 221: /* cmd ::= DROP INDEX exists_opt index_name ON table_name */ +{ pCxt->pRootNode = createDropIndexStmt(pCxt, yymsp[-3].minor.yy621, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy421); } break; - case 216: /* cmd ::= DROP INDEX exists_opt index_name ON table_name */ -{ pCxt->pRootNode = createDropIndexStmt(pCxt, yymsp[-3].minor.yy137, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209); } + case 223: /* index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_RP sliding_opt */ +{ yymsp[-8].minor.yy674 = createIndexOption(pCxt, yymsp[-6].minor.yy530, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), NULL, yymsp[0].minor.yy674); } break; - case 218: /* index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_RP sliding_opt */ -{ yymsp[-8].minor.yy632 = createIndexOption(pCxt, yymsp[-6].minor.yy424, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), NULL, yymsp[0].minor.yy632); } + case 224: /* index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt */ +{ yymsp[-10].minor.yy674 = createIndexOption(pCxt, yymsp[-8].minor.yy530, releaseRawExprNode(pCxt, yymsp[-4].minor.yy674), releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), yymsp[0].minor.yy674); } break; - case 219: /* index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt */ -{ yymsp[-10].minor.yy632 = createIndexOption(pCxt, yymsp[-8].minor.yy424, releaseRawExprNode(pCxt, yymsp[-4].minor.yy632), releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), yymsp[0].minor.yy632); } + case 227: /* func ::= function_name NK_LP expression_list NK_RP */ +{ yylhsminor.yy674 = createFunctionNode(pCxt, &yymsp[-3].minor.yy421, yymsp[-1].minor.yy530); } + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; - case 222: /* func ::= function_name NK_LP expression_list NK_RP */ -{ yylhsminor.yy632 = createFunctionNode(pCxt, &yymsp[-3].minor.yy209, yymsp[-1].minor.yy424); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; + case 228: /* cmd ::= CREATE TOPIC not_exists_opt topic_name AS query_expression */ +{ pCxt->pRootNode = createCreateTopicStmt(pCxt, yymsp[-3].minor.yy621, &yymsp[-2].minor.yy421, yymsp[0].minor.yy674, NULL, NULL); } break; - case 223: /* cmd ::= CREATE TOPIC not_exists_opt topic_name AS query_expression */ -{ pCxt->pRootNode = createCreateTopicStmt(pCxt, yymsp[-3].minor.yy137, &yymsp[-2].minor.yy209, yymsp[0].minor.yy632, NULL, NULL); } + case 229: /* cmd ::= CREATE TOPIC not_exists_opt topic_name AS DATABASE db_name */ +{ pCxt->pRootNode = createCreateTopicStmt(pCxt, yymsp[-4].minor.yy621, &yymsp[-3].minor.yy421, NULL, &yymsp[0].minor.yy421, NULL); } break; - case 224: /* cmd ::= CREATE TOPIC not_exists_opt topic_name AS DATABASE db_name */ -{ pCxt->pRootNode = createCreateTopicStmt(pCxt, yymsp[-4].minor.yy137, &yymsp[-3].minor.yy209, NULL, &yymsp[0].minor.yy209, NULL); } + case 230: /* cmd ::= CREATE TOPIC not_exists_opt topic_name AS STABLE full_table_name */ +{ pCxt->pRootNode = createCreateTopicStmt(pCxt, yymsp[-4].minor.yy621, &yymsp[-3].minor.yy421, NULL, NULL, yymsp[0].minor.yy674); } break; - case 225: /* cmd ::= CREATE TOPIC not_exists_opt topic_name AS STABLE full_table_name */ -{ pCxt->pRootNode = createCreateTopicStmt(pCxt, yymsp[-4].minor.yy137, &yymsp[-3].minor.yy209, NULL, NULL, yymsp[0].minor.yy632); } + case 231: /* cmd ::= DROP TOPIC exists_opt topic_name */ +{ pCxt->pRootNode = createDropTopicStmt(pCxt, yymsp[-1].minor.yy621, &yymsp[0].minor.yy421); } break; - case 226: /* cmd ::= DROP TOPIC exists_opt topic_name */ -{ pCxt->pRootNode = createDropTopicStmt(pCxt, yymsp[-1].minor.yy137, &yymsp[0].minor.yy209); } + case 232: /* cmd ::= DROP CONSUMER GROUP exists_opt cgroup_name ON topic_name */ +{ pCxt->pRootNode = createDropCGroupStmt(pCxt, yymsp[-3].minor.yy621, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy421); } break; - case 227: /* cmd ::= DROP CONSUMER GROUP exists_opt cgroup_name ON topic_name */ -{ pCxt->pRootNode = createDropCGroupStmt(pCxt, yymsp[-3].minor.yy137, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209); } + case 233: /* cmd ::= DESC full_table_name */ + case 234: /* cmd ::= DESCRIBE full_table_name */ yytestcase(yyruleno==234); +{ pCxt->pRootNode = createDescribeStmt(pCxt, yymsp[0].minor.yy674); } break; - case 228: /* cmd ::= DESC full_table_name */ - case 229: /* cmd ::= DESCRIBE full_table_name */ yytestcase(yyruleno==229); -{ pCxt->pRootNode = createDescribeStmt(pCxt, yymsp[0].minor.yy632); } - break; - case 230: /* cmd ::= RESET QUERY CACHE */ + case 235: /* cmd ::= RESET QUERY CACHE */ { pCxt->pRootNode = createResetQueryCacheStmt(pCxt); } break; - case 231: /* cmd ::= EXPLAIN analyze_opt explain_options query_expression */ -{ pCxt->pRootNode = createExplainStmt(pCxt, yymsp[-2].minor.yy137, yymsp[-1].minor.yy632, yymsp[0].minor.yy632); } + case 236: /* cmd ::= EXPLAIN analyze_opt explain_options query_expression */ +{ pCxt->pRootNode = createExplainStmt(pCxt, yymsp[-2].minor.yy621, yymsp[-1].minor.yy674, yymsp[0].minor.yy674); } break; - case 233: /* analyze_opt ::= ANALYZE */ - case 241: /* agg_func_opt ::= AGGREGATE */ yytestcase(yyruleno==241); - case 392: /* set_quantifier_opt ::= DISTINCT */ yytestcase(yyruleno==392); -{ yymsp[0].minor.yy137 = true; } + case 238: /* analyze_opt ::= ANALYZE */ + case 246: /* agg_func_opt ::= AGGREGATE */ yytestcase(yyruleno==246); + case 397: /* set_quantifier_opt ::= DISTINCT */ yytestcase(yyruleno==397); +{ yymsp[0].minor.yy621 = true; } break; - case 234: /* explain_options ::= */ -{ yymsp[1].minor.yy632 = createDefaultExplainOptions(pCxt); } + case 239: /* explain_options ::= */ +{ yymsp[1].minor.yy674 = createDefaultExplainOptions(pCxt); } break; - case 235: /* explain_options ::= explain_options VERBOSE NK_BOOL */ -{ yylhsminor.yy632 = setExplainVerbose(pCxt, yymsp[-2].minor.yy632, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 240: /* explain_options ::= explain_options VERBOSE NK_BOOL */ +{ yylhsminor.yy674 = setExplainVerbose(pCxt, yymsp[-2].minor.yy674, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 236: /* explain_options ::= explain_options RATIO NK_FLOAT */ -{ yylhsminor.yy632 = setExplainRatio(pCxt, yymsp[-2].minor.yy632, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 241: /* explain_options ::= explain_options RATIO NK_FLOAT */ +{ yylhsminor.yy674 = setExplainRatio(pCxt, yymsp[-2].minor.yy674, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 237: /* cmd ::= COMPACT VNODES IN NK_LP integer_list NK_RP */ -{ pCxt->pRootNode = createCompactStmt(pCxt, yymsp[-1].minor.yy424); } + case 242: /* cmd ::= COMPACT VNODES IN NK_LP integer_list NK_RP */ +{ pCxt->pRootNode = createCompactStmt(pCxt, yymsp[-1].minor.yy530); } break; - case 238: /* cmd ::= CREATE agg_func_opt FUNCTION not_exists_opt function_name AS NK_STRING OUTPUTTYPE type_name bufsize_opt */ -{ pCxt->pRootNode = createCreateFunctionStmt(pCxt, yymsp[-6].minor.yy137, yymsp[-8].minor.yy137, &yymsp[-5].minor.yy209, &yymsp[-3].minor.yy0, yymsp[-1].minor.yy304, yymsp[0].minor.yy100); } + case 243: /* cmd ::= CREATE agg_func_opt FUNCTION not_exists_opt function_name AS NK_STRING OUTPUTTYPE type_name bufsize_opt */ +{ pCxt->pRootNode = createCreateFunctionStmt(pCxt, yymsp[-6].minor.yy621, yymsp[-8].minor.yy621, &yymsp[-5].minor.yy421, &yymsp[-3].minor.yy0, yymsp[-1].minor.yy690, yymsp[0].minor.yy42); } break; - case 239: /* cmd ::= DROP FUNCTION exists_opt function_name */ -{ pCxt->pRootNode = createDropFunctionStmt(pCxt, yymsp[-1].minor.yy137, &yymsp[0].minor.yy209); } + case 244: /* cmd ::= DROP FUNCTION exists_opt function_name */ +{ pCxt->pRootNode = createDropFunctionStmt(pCxt, yymsp[-1].minor.yy621, &yymsp[0].minor.yy421); } break; - case 242: /* bufsize_opt ::= */ -{ yymsp[1].minor.yy100 = 0; } + case 247: /* bufsize_opt ::= */ +{ yymsp[1].minor.yy42 = 0; } break; - case 243: /* bufsize_opt ::= BUFSIZE NK_INTEGER */ -{ yymsp[-1].minor.yy100 = taosStr2Int32(yymsp[0].minor.yy0.z, NULL, 10); } + case 248: /* bufsize_opt ::= BUFSIZE NK_INTEGER */ +{ yymsp[-1].minor.yy42 = taosStr2Int32(yymsp[0].minor.yy0.z, NULL, 10); } break; - case 244: /* cmd ::= CREATE STREAM not_exists_opt stream_name stream_options into_opt AS query_expression */ -{ pCxt->pRootNode = createCreateStreamStmt(pCxt, yymsp[-5].minor.yy137, &yymsp[-4].minor.yy209, yymsp[-2].minor.yy632, yymsp[-3].minor.yy632, yymsp[0].minor.yy632); } + case 249: /* cmd ::= CREATE STREAM not_exists_opt stream_name stream_options into_opt AS query_expression */ +{ pCxt->pRootNode = createCreateStreamStmt(pCxt, yymsp[-5].minor.yy621, &yymsp[-4].minor.yy421, yymsp[-2].minor.yy674, yymsp[-3].minor.yy674, yymsp[0].minor.yy674); } break; - case 245: /* cmd ::= DROP STREAM exists_opt stream_name */ -{ pCxt->pRootNode = createDropStreamStmt(pCxt, yymsp[-1].minor.yy137, &yymsp[0].minor.yy209); } + case 250: /* cmd ::= DROP STREAM exists_opt stream_name */ +{ pCxt->pRootNode = createDropStreamStmt(pCxt, yymsp[-1].minor.yy621, &yymsp[0].minor.yy421); } break; - case 247: /* into_opt ::= INTO full_table_name */ - case 373: /* from_clause ::= FROM table_reference_list */ yytestcase(yyruleno==373); - case 403: /* where_clause_opt ::= WHERE search_condition */ yytestcase(yyruleno==403); - case 426: /* having_clause_opt ::= HAVING search_condition */ yytestcase(yyruleno==426); -{ yymsp[-1].minor.yy632 = yymsp[0].minor.yy632; } + case 252: /* into_opt ::= INTO full_table_name */ + case 378: /* from_clause ::= FROM table_reference_list */ yytestcase(yyruleno==378); + case 408: /* where_clause_opt ::= WHERE search_condition */ yytestcase(yyruleno==408); + case 431: /* having_clause_opt ::= HAVING search_condition */ yytestcase(yyruleno==431); +{ yymsp[-1].minor.yy674 = yymsp[0].minor.yy674; } break; - case 248: /* stream_options ::= */ -{ yymsp[1].minor.yy632 = createStreamOptions(pCxt); } + case 253: /* stream_options ::= */ +{ yymsp[1].minor.yy674 = createStreamOptions(pCxt); } break; - case 249: /* stream_options ::= stream_options TRIGGER AT_ONCE */ -{ ((SStreamOptions*)yymsp[-2].minor.yy632)->triggerType = STREAM_TRIGGER_AT_ONCE; yylhsminor.yy632 = yymsp[-2].minor.yy632; } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 254: /* stream_options ::= stream_options TRIGGER AT_ONCE */ +{ ((SStreamOptions*)yymsp[-2].minor.yy674)->triggerType = STREAM_TRIGGER_AT_ONCE; yylhsminor.yy674 = yymsp[-2].minor.yy674; } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 250: /* stream_options ::= stream_options TRIGGER WINDOW_CLOSE */ -{ ((SStreamOptions*)yymsp[-2].minor.yy632)->triggerType = STREAM_TRIGGER_WINDOW_CLOSE; yylhsminor.yy632 = yymsp[-2].minor.yy632; } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 255: /* stream_options ::= stream_options TRIGGER WINDOW_CLOSE */ +{ ((SStreamOptions*)yymsp[-2].minor.yy674)->triggerType = STREAM_TRIGGER_WINDOW_CLOSE; yylhsminor.yy674 = yymsp[-2].minor.yy674; } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 251: /* stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal */ -{ ((SStreamOptions*)yymsp[-3].minor.yy632)->triggerType = STREAM_TRIGGER_MAX_DELAY; ((SStreamOptions*)yymsp[-3].minor.yy632)->pDelay = releaseRawExprNode(pCxt, yymsp[0].minor.yy632); yylhsminor.yy632 = yymsp[-3].minor.yy632; } - yymsp[-3].minor.yy632 = yylhsminor.yy632; + case 256: /* stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal */ +{ ((SStreamOptions*)yymsp[-3].minor.yy674)->triggerType = STREAM_TRIGGER_MAX_DELAY; ((SStreamOptions*)yymsp[-3].minor.yy674)->pDelay = releaseRawExprNode(pCxt, yymsp[0].minor.yy674); yylhsminor.yy674 = yymsp[-3].minor.yy674; } + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; - case 252: /* stream_options ::= stream_options WATERMARK duration_literal */ -{ ((SStreamOptions*)yymsp[-2].minor.yy632)->pWatermark = releaseRawExprNode(pCxt, yymsp[0].minor.yy632); yylhsminor.yy632 = yymsp[-2].minor.yy632; } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 257: /* stream_options ::= stream_options WATERMARK duration_literal */ +{ ((SStreamOptions*)yymsp[-2].minor.yy674)->pWatermark = releaseRawExprNode(pCxt, yymsp[0].minor.yy674); yylhsminor.yy674 = yymsp[-2].minor.yy674; } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 253: /* cmd ::= KILL CONNECTION NK_INTEGER */ + case 258: /* cmd ::= KILL CONNECTION NK_INTEGER */ { pCxt->pRootNode = createKillStmt(pCxt, QUERY_NODE_KILL_CONNECTION_STMT, &yymsp[0].minor.yy0); } break; - case 254: /* cmd ::= KILL QUERY NK_STRING */ + case 259: /* cmd ::= KILL QUERY NK_STRING */ { pCxt->pRootNode = createKillQueryStmt(pCxt, &yymsp[0].minor.yy0); } break; - case 255: /* cmd ::= KILL TRANSACTION NK_INTEGER */ + case 260: /* cmd ::= KILL TRANSACTION NK_INTEGER */ { pCxt->pRootNode = createKillStmt(pCxt, QUERY_NODE_KILL_TRANSACTION_STMT, &yymsp[0].minor.yy0); } break; - case 256: /* cmd ::= BALANCE VGROUP */ + case 261: /* cmd ::= BALANCE VGROUP */ { pCxt->pRootNode = createBalanceVgroupStmt(pCxt); } break; - case 257: /* cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER */ + case 262: /* cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER */ { pCxt->pRootNode = createMergeVgroupStmt(pCxt, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy0); } break; - case 258: /* cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list */ -{ pCxt->pRootNode = createRedistributeVgroupStmt(pCxt, &yymsp[-1].minor.yy0, yymsp[0].minor.yy424); } + case 263: /* cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list */ +{ pCxt->pRootNode = createRedistributeVgroupStmt(pCxt, &yymsp[-1].minor.yy0, yymsp[0].minor.yy530); } break; - case 259: /* cmd ::= SPLIT VGROUP NK_INTEGER */ + case 264: /* cmd ::= SPLIT VGROUP NK_INTEGER */ { pCxt->pRootNode = createSplitVgroupStmt(pCxt, &yymsp[0].minor.yy0); } break; - case 260: /* dnode_list ::= DNODE NK_INTEGER */ -{ yymsp[-1].minor.yy424 = createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } + case 265: /* dnode_list ::= DNODE NK_INTEGER */ +{ yymsp[-1].minor.yy530 = createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } break; - case 262: /* cmd ::= SYNCDB db_name REPLICA */ -{ pCxt->pRootNode = createSyncdbStmt(pCxt, &yymsp[-1].minor.yy209); } + case 267: /* cmd ::= SYNCDB db_name REPLICA */ +{ pCxt->pRootNode = createSyncdbStmt(pCxt, &yymsp[-1].minor.yy421); } break; - case 263: /* cmd ::= DELETE FROM full_table_name where_clause_opt */ -{ pCxt->pRootNode = createDeleteStmt(pCxt, yymsp[-1].minor.yy632, yymsp[0].minor.yy632); } + case 268: /* cmd ::= DELETE FROM full_table_name where_clause_opt */ +{ pCxt->pRootNode = createDeleteStmt(pCxt, yymsp[-1].minor.yy674, yymsp[0].minor.yy674); } break; - case 265: /* literal ::= NK_INTEGER */ -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 270: /* literal ::= NK_INTEGER */ +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 266: /* literal ::= NK_FLOAT */ -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 271: /* literal ::= NK_FLOAT */ +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 267: /* literal ::= NK_STRING */ -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 272: /* literal ::= NK_STRING */ +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 268: /* literal ::= NK_BOOL */ -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_BOOL, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 273: /* literal ::= NK_BOOL */ +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_BOOL, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 269: /* literal ::= TIMESTAMP NK_STRING */ -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_TIMESTAMP, &yymsp[0].minor.yy0)); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + case 274: /* literal ::= TIMESTAMP NK_STRING */ +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_TIMESTAMP, &yymsp[0].minor.yy0)); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 270: /* literal ::= duration_literal */ - case 280: /* signed_literal ::= signed */ yytestcase(yyruleno==280); - case 300: /* expression ::= literal */ yytestcase(yyruleno==300); - case 301: /* expression ::= pseudo_column */ yytestcase(yyruleno==301); - case 302: /* expression ::= column_reference */ yytestcase(yyruleno==302); - case 303: /* expression ::= function_expression */ yytestcase(yyruleno==303); - case 304: /* expression ::= subquery */ yytestcase(yyruleno==304); - case 329: /* function_expression ::= literal_func */ yytestcase(yyruleno==329); - case 365: /* boolean_value_expression ::= boolean_primary */ yytestcase(yyruleno==365); - case 369: /* boolean_primary ::= predicate */ yytestcase(yyruleno==369); - case 371: /* common_expression ::= expression */ yytestcase(yyruleno==371); - case 372: /* common_expression ::= boolean_value_expression */ yytestcase(yyruleno==372); - case 374: /* table_reference_list ::= table_reference */ yytestcase(yyruleno==374); - case 376: /* table_reference ::= table_primary */ yytestcase(yyruleno==376); - case 377: /* table_reference ::= joined_table */ yytestcase(yyruleno==377); - case 381: /* table_primary ::= parenthesized_joined_table */ yytestcase(yyruleno==381); - case 428: /* query_expression_body ::= query_primary */ yytestcase(yyruleno==428); - case 431: /* query_primary ::= query_specification */ yytestcase(yyruleno==431); -{ yylhsminor.yy632 = yymsp[0].minor.yy632; } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 275: /* literal ::= duration_literal */ + case 285: /* signed_literal ::= signed */ yytestcase(yyruleno==285); + case 305: /* expression ::= literal */ yytestcase(yyruleno==305); + case 306: /* expression ::= pseudo_column */ yytestcase(yyruleno==306); + case 307: /* expression ::= column_reference */ yytestcase(yyruleno==307); + case 308: /* expression ::= function_expression */ yytestcase(yyruleno==308); + case 309: /* expression ::= subquery */ yytestcase(yyruleno==309); + case 334: /* function_expression ::= literal_func */ yytestcase(yyruleno==334); + case 370: /* boolean_value_expression ::= boolean_primary */ yytestcase(yyruleno==370); + case 374: /* boolean_primary ::= predicate */ yytestcase(yyruleno==374); + case 376: /* common_expression ::= expression */ yytestcase(yyruleno==376); + case 377: /* common_expression ::= boolean_value_expression */ yytestcase(yyruleno==377); + case 379: /* table_reference_list ::= table_reference */ yytestcase(yyruleno==379); + case 381: /* table_reference ::= table_primary */ yytestcase(yyruleno==381); + case 382: /* table_reference ::= joined_table */ yytestcase(yyruleno==382); + case 386: /* table_primary ::= parenthesized_joined_table */ yytestcase(yyruleno==386); + case 433: /* query_expression_body ::= query_primary */ yytestcase(yyruleno==433); + case 436: /* query_primary ::= query_specification */ yytestcase(yyruleno==436); +{ yylhsminor.yy674 = yymsp[0].minor.yy674; } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 271: /* literal ::= NULL */ -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_NULL, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 276: /* literal ::= NULL */ +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_NULL, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 272: /* literal ::= NK_QUESTION */ -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createPlaceholderValueNode(pCxt, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 277: /* literal ::= NK_QUESTION */ +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createPlaceholderValueNode(pCxt, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 273: /* duration_literal ::= NK_VARIABLE */ -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 278: /* duration_literal ::= NK_VARIABLE */ +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 274: /* signed ::= NK_INTEGER */ -{ yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 279: /* signed ::= NK_INTEGER */ +{ yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 275: /* signed ::= NK_PLUS NK_INTEGER */ -{ yymsp[-1].minor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0); } + case 280: /* signed ::= NK_PLUS NK_INTEGER */ +{ yymsp[-1].minor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0); } break; - case 276: /* signed ::= NK_MINUS NK_INTEGER */ + case 281: /* signed ::= NK_MINUS NK_INTEGER */ { SToken t = yymsp[-1].minor.yy0; t.n = (yymsp[0].minor.yy0.z + yymsp[0].minor.yy0.n) - yymsp[-1].minor.yy0.z; - yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &t); + yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &t); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 277: /* signed ::= NK_FLOAT */ -{ yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 282: /* signed ::= NK_FLOAT */ +{ yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 278: /* signed ::= NK_PLUS NK_FLOAT */ -{ yymsp[-1].minor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0); } + case 283: /* signed ::= NK_PLUS NK_FLOAT */ +{ yymsp[-1].minor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0); } break; - case 279: /* signed ::= NK_MINUS NK_FLOAT */ + case 284: /* signed ::= NK_MINUS NK_FLOAT */ { SToken t = yymsp[-1].minor.yy0; t.n = (yymsp[0].minor.yy0.z + yymsp[0].minor.yy0.n) - yymsp[-1].minor.yy0.z; - yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &t); + yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &t); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 281: /* signed_literal ::= NK_STRING */ -{ yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 286: /* signed_literal ::= NK_STRING */ +{ yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 282: /* signed_literal ::= NK_BOOL */ -{ yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BOOL, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 287: /* signed_literal ::= NK_BOOL */ +{ yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BOOL, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 283: /* signed_literal ::= TIMESTAMP NK_STRING */ -{ yymsp[-1].minor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_TIMESTAMP, &yymsp[0].minor.yy0); } + case 288: /* signed_literal ::= TIMESTAMP NK_STRING */ +{ yymsp[-1].minor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_TIMESTAMP, &yymsp[0].minor.yy0); } break; - case 284: /* signed_literal ::= duration_literal */ - case 286: /* signed_literal ::= literal_func */ yytestcase(yyruleno==286); - case 343: /* star_func_para ::= expression */ yytestcase(yyruleno==343); - case 398: /* select_item ::= common_expression */ yytestcase(yyruleno==398); - case 444: /* search_condition ::= common_expression */ yytestcase(yyruleno==444); -{ yylhsminor.yy632 = releaseRawExprNode(pCxt, yymsp[0].minor.yy632); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 289: /* signed_literal ::= duration_literal */ + case 291: /* signed_literal ::= literal_func */ yytestcase(yyruleno==291); + case 348: /* star_func_para ::= expression */ yytestcase(yyruleno==348); + case 403: /* select_item ::= common_expression */ yytestcase(yyruleno==403); + case 449: /* search_condition ::= common_expression */ yytestcase(yyruleno==449); +{ yylhsminor.yy674 = releaseRawExprNode(pCxt, yymsp[0].minor.yy674); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 285: /* signed_literal ::= NULL */ -{ yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_NULL, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 290: /* signed_literal ::= NULL */ +{ yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_NULL, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 305: /* expression ::= NK_LP expression NK_RP */ - case 370: /* boolean_primary ::= NK_LP boolean_value_expression NK_RP */ yytestcase(yyruleno==370); -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, releaseRawExprNode(pCxt, yymsp[-1].minor.yy632)); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 310: /* expression ::= NK_LP expression NK_RP */ + case 375: /* boolean_primary ::= NK_LP boolean_value_expression NK_RP */ yytestcase(yyruleno==375); +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, releaseRawExprNode(pCxt, yymsp[-1].minor.yy674)); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 306: /* expression ::= NK_PLUS expression */ + case 311: /* expression ::= NK_PLUS expression */ { - SToken t = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &t, releaseRawExprNode(pCxt, yymsp[0].minor.yy632)); + SToken t = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &t, releaseRawExprNode(pCxt, yymsp[0].minor.yy674)); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 307: /* expression ::= NK_MINUS expression */ + case 312: /* expression ::= NK_MINUS expression */ { - SToken t = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &t, createOperatorNode(pCxt, OP_TYPE_MINUS, releaseRawExprNode(pCxt, yymsp[0].minor.yy632), NULL)); + SToken t = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &t, createOperatorNode(pCxt, OP_TYPE_MINUS, releaseRawExprNode(pCxt, yymsp[0].minor.yy674), NULL)); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 308: /* expression ::= expression NK_PLUS expression */ + case 313: /* expression ::= expression NK_PLUS expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_ADD, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_ADD, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 309: /* expression ::= expression NK_MINUS expression */ + case 314: /* expression ::= expression NK_MINUS expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_SUB, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_SUB, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 310: /* expression ::= expression NK_STAR expression */ + case 315: /* expression ::= expression NK_STAR expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_MULTI, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_MULTI, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 311: /* expression ::= expression NK_SLASH expression */ + case 316: /* expression ::= expression NK_SLASH expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_DIV, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_DIV, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 312: /* expression ::= expression NK_REM expression */ + case 317: /* expression ::= expression NK_REM expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_REM, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_REM, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 313: /* expression ::= column_reference NK_ARROW NK_STRING */ + case 318: /* expression ::= column_reference NK_ARROW NK_STRING */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_JSON_GET_VALUE, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_JSON_GET_VALUE, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 314: /* expression_list ::= expression */ -{ yylhsminor.yy424 = createNodeList(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy632)); } - yymsp[0].minor.yy424 = yylhsminor.yy424; + case 321: /* column_reference ::= column_name */ +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy421, createColumnNode(pCxt, NULL, &yymsp[0].minor.yy421)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 315: /* expression_list ::= expression_list NK_COMMA expression */ -{ yylhsminor.yy424 = addNodeToList(pCxt, yymsp[-2].minor.yy424, releaseRawExprNode(pCxt, yymsp[0].minor.yy632)); } - yymsp[-2].minor.yy424 = yylhsminor.yy424; + case 322: /* column_reference ::= table_name NK_DOT column_name */ +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy421, createColumnNode(pCxt, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy421)); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 316: /* column_reference ::= column_name */ -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy209, createColumnNode(pCxt, NULL, &yymsp[0].minor.yy209)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 323: /* pseudo_column ::= ROWTS */ + case 324: /* pseudo_column ::= TBNAME */ yytestcase(yyruleno==324); + case 326: /* pseudo_column ::= QSTARTTS */ yytestcase(yyruleno==326); + case 327: /* pseudo_column ::= QENDTS */ yytestcase(yyruleno==327); + case 328: /* pseudo_column ::= WSTARTTS */ yytestcase(yyruleno==328); + case 329: /* pseudo_column ::= WENDTS */ yytestcase(yyruleno==329); + case 330: /* pseudo_column ::= WDURATION */ yytestcase(yyruleno==330); + case 336: /* literal_func ::= NOW */ yytestcase(yyruleno==336); +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[0].minor.yy0, NULL)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 317: /* column_reference ::= table_name NK_DOT column_name */ -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209, createColumnNode(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209)); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 325: /* pseudo_column ::= table_name NK_DOT TBNAME */ +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[0].minor.yy0, createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[-2].minor.yy421)))); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 318: /* pseudo_column ::= ROWTS */ - case 319: /* pseudo_column ::= TBNAME */ yytestcase(yyruleno==319); - case 321: /* pseudo_column ::= QSTARTTS */ yytestcase(yyruleno==321); - case 322: /* pseudo_column ::= QENDTS */ yytestcase(yyruleno==322); - case 323: /* pseudo_column ::= WSTARTTS */ yytestcase(yyruleno==323); - case 324: /* pseudo_column ::= WENDTS */ yytestcase(yyruleno==324); - case 325: /* pseudo_column ::= WDURATION */ yytestcase(yyruleno==325); - case 331: /* literal_func ::= NOW */ yytestcase(yyruleno==331); -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[0].minor.yy0, NULL)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 331: /* function_expression ::= function_name NK_LP expression_list NK_RP */ + case 332: /* function_expression ::= star_func NK_LP star_func_para_list NK_RP */ yytestcase(yyruleno==332); +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-3].minor.yy421, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[-3].minor.yy421, yymsp[-1].minor.yy530)); } + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; - case 320: /* pseudo_column ::= table_name NK_DOT TBNAME */ -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[0].minor.yy0, createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[-2].minor.yy209)))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 333: /* function_expression ::= CAST NK_LP expression AS type_name NK_RP */ +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-5].minor.yy0, &yymsp[0].minor.yy0, createCastFunctionNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy674), yymsp[-1].minor.yy690)); } + yymsp[-5].minor.yy674 = yylhsminor.yy674; break; - case 326: /* function_expression ::= function_name NK_LP expression_list NK_RP */ - case 327: /* function_expression ::= star_func NK_LP star_func_para_list NK_RP */ yytestcase(yyruleno==327); -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-3].minor.yy209, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[-3].minor.yy209, yymsp[-1].minor.yy424)); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; + case 335: /* literal_func ::= noarg_func NK_LP NK_RP */ +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[-2].minor.yy421, NULL)); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 328: /* function_expression ::= CAST NK_LP expression AS type_name NK_RP */ -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-5].minor.yy0, &yymsp[0].minor.yy0, createCastFunctionNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy632), yymsp[-1].minor.yy304)); } - yymsp[-5].minor.yy632 = yylhsminor.yy632; + case 344: /* star_func_para_list ::= NK_STAR */ +{ yylhsminor.yy530 = createNodeList(pCxt, createColumnNode(pCxt, NULL, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy530 = yylhsminor.yy530; break; - case 330: /* literal_func ::= noarg_func NK_LP NK_RP */ -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[-2].minor.yy209, NULL)); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 349: /* star_func_para ::= table_name NK_DOT NK_STAR */ + case 406: /* select_item ::= table_name NK_DOT NK_STAR */ yytestcase(yyruleno==406); +{ yylhsminor.yy674 = createColumnNode(pCxt, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 339: /* star_func_para_list ::= NK_STAR */ -{ yylhsminor.yy424 = createNodeList(pCxt, createColumnNode(pCxt, NULL, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy424 = yylhsminor.yy424; - break; - case 344: /* star_func_para ::= table_name NK_DOT NK_STAR */ - case 401: /* select_item ::= table_name NK_DOT NK_STAR */ yytestcase(yyruleno==401); -{ yylhsminor.yy632 = createColumnNode(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; - break; - case 345: /* predicate ::= expression compare_op expression */ - case 350: /* predicate ::= expression in_op in_predicate_value */ yytestcase(yyruleno==350); + case 350: /* predicate ::= expression compare_op expression */ + case 355: /* predicate ::= expression in_op in_predicate_value */ yytestcase(yyruleno==355); { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, yymsp[-1].minor.yy380, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, yymsp[-1].minor.yy28, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 346: /* predicate ::= expression BETWEEN expression AND expression */ + case 351: /* predicate ::= expression BETWEEN expression AND expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-4].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createBetweenAnd(pCxt, releaseRawExprNode(pCxt, yymsp[-4].minor.yy632), releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-4].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createBetweenAnd(pCxt, releaseRawExprNode(pCxt, yymsp[-4].minor.yy674), releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; - case 347: /* predicate ::= expression NOT BETWEEN expression AND expression */ + case 352: /* predicate ::= expression NOT BETWEEN expression AND expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-5].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createNotBetweenAnd(pCxt, releaseRawExprNode(pCxt, yymsp[-5].minor.yy632), releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-5].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createNotBetweenAnd(pCxt, releaseRawExprNode(pCxt, yymsp[-5].minor.yy674), releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-5].minor.yy632 = yylhsminor.yy632; + yymsp[-5].minor.yy674 = yylhsminor.yy674; break; - case 348: /* predicate ::= expression IS NULL */ + case 353: /* predicate ::= expression IS NULL */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_IS_NULL, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), NULL)); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_IS_NULL, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), NULL)); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 349: /* predicate ::= expression IS NOT NULL */ + case 354: /* predicate ::= expression IS NOT NULL */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-3].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_IS_NOT_NULL, releaseRawExprNode(pCxt, yymsp[-3].minor.yy632), NULL)); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-3].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_IS_NOT_NULL, releaseRawExprNode(pCxt, yymsp[-3].minor.yy674), NULL)); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; - case 351: /* compare_op ::= NK_LT */ -{ yymsp[0].minor.yy380 = OP_TYPE_LOWER_THAN; } + case 356: /* compare_op ::= NK_LT */ +{ yymsp[0].minor.yy28 = OP_TYPE_LOWER_THAN; } break; - case 352: /* compare_op ::= NK_GT */ -{ yymsp[0].minor.yy380 = OP_TYPE_GREATER_THAN; } + case 357: /* compare_op ::= NK_GT */ +{ yymsp[0].minor.yy28 = OP_TYPE_GREATER_THAN; } break; - case 353: /* compare_op ::= NK_LE */ -{ yymsp[0].minor.yy380 = OP_TYPE_LOWER_EQUAL; } + case 358: /* compare_op ::= NK_LE */ +{ yymsp[0].minor.yy28 = OP_TYPE_LOWER_EQUAL; } break; - case 354: /* compare_op ::= NK_GE */ -{ yymsp[0].minor.yy380 = OP_TYPE_GREATER_EQUAL; } + case 359: /* compare_op ::= NK_GE */ +{ yymsp[0].minor.yy28 = OP_TYPE_GREATER_EQUAL; } break; - case 355: /* compare_op ::= NK_NE */ -{ yymsp[0].minor.yy380 = OP_TYPE_NOT_EQUAL; } + case 360: /* compare_op ::= NK_NE */ +{ yymsp[0].minor.yy28 = OP_TYPE_NOT_EQUAL; } break; - case 356: /* compare_op ::= NK_EQ */ -{ yymsp[0].minor.yy380 = OP_TYPE_EQUAL; } + case 361: /* compare_op ::= NK_EQ */ +{ yymsp[0].minor.yy28 = OP_TYPE_EQUAL; } break; - case 357: /* compare_op ::= LIKE */ -{ yymsp[0].minor.yy380 = OP_TYPE_LIKE; } + case 362: /* compare_op ::= LIKE */ +{ yymsp[0].minor.yy28 = OP_TYPE_LIKE; } break; - case 358: /* compare_op ::= NOT LIKE */ -{ yymsp[-1].minor.yy380 = OP_TYPE_NOT_LIKE; } + case 363: /* compare_op ::= NOT LIKE */ +{ yymsp[-1].minor.yy28 = OP_TYPE_NOT_LIKE; } break; - case 359: /* compare_op ::= MATCH */ -{ yymsp[0].minor.yy380 = OP_TYPE_MATCH; } + case 364: /* compare_op ::= MATCH */ +{ yymsp[0].minor.yy28 = OP_TYPE_MATCH; } break; - case 360: /* compare_op ::= NMATCH */ -{ yymsp[0].minor.yy380 = OP_TYPE_NMATCH; } + case 365: /* compare_op ::= NMATCH */ +{ yymsp[0].minor.yy28 = OP_TYPE_NMATCH; } break; - case 361: /* compare_op ::= CONTAINS */ -{ yymsp[0].minor.yy380 = OP_TYPE_JSON_CONTAINS; } + case 366: /* compare_op ::= CONTAINS */ +{ yymsp[0].minor.yy28 = OP_TYPE_JSON_CONTAINS; } break; - case 362: /* in_op ::= IN */ -{ yymsp[0].minor.yy380 = OP_TYPE_IN; } + case 367: /* in_op ::= IN */ +{ yymsp[0].minor.yy28 = OP_TYPE_IN; } break; - case 363: /* in_op ::= NOT IN */ -{ yymsp[-1].minor.yy380 = OP_TYPE_NOT_IN; } + case 368: /* in_op ::= NOT IN */ +{ yymsp[-1].minor.yy28 = OP_TYPE_NOT_IN; } break; - case 364: /* in_predicate_value ::= NK_LP expression_list NK_RP */ -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, createNodeListNode(pCxt, yymsp[-1].minor.yy424)); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 369: /* in_predicate_value ::= NK_LP expression_list NK_RP */ +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, createNodeListNode(pCxt, yymsp[-1].minor.yy530)); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 366: /* boolean_value_expression ::= NOT boolean_primary */ + case 371: /* boolean_value_expression ::= NOT boolean_primary */ { - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_NOT, releaseRawExprNode(pCxt, yymsp[0].minor.yy632), NULL)); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_NOT, releaseRawExprNode(pCxt, yymsp[0].minor.yy674), NULL)); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 367: /* boolean_value_expression ::= boolean_value_expression OR boolean_value_expression */ + case 372: /* boolean_value_expression ::= boolean_value_expression OR boolean_value_expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_OR, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_OR, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 368: /* boolean_value_expression ::= boolean_value_expression AND boolean_value_expression */ + case 373: /* boolean_value_expression ::= boolean_value_expression AND boolean_value_expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_AND, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_AND, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 375: /* table_reference_list ::= table_reference_list NK_COMMA table_reference */ -{ yylhsminor.yy632 = createJoinTableNode(pCxt, JOIN_TYPE_INNER, yymsp[-2].minor.yy632, yymsp[0].minor.yy632, NULL); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 380: /* table_reference_list ::= table_reference_list NK_COMMA table_reference */ +{ yylhsminor.yy674 = createJoinTableNode(pCxt, JOIN_TYPE_INNER, yymsp[-2].minor.yy674, yymsp[0].minor.yy674, NULL); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 378: /* table_primary ::= table_name alias_opt */ -{ yylhsminor.yy632 = createRealTableNode(pCxt, NULL, &yymsp[-1].minor.yy209, &yymsp[0].minor.yy209); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + case 383: /* table_primary ::= table_name alias_opt */ +{ yylhsminor.yy674 = createRealTableNode(pCxt, NULL, &yymsp[-1].minor.yy421, &yymsp[0].minor.yy421); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 379: /* table_primary ::= db_name NK_DOT table_name alias_opt */ -{ yylhsminor.yy632 = createRealTableNode(pCxt, &yymsp[-3].minor.yy209, &yymsp[-1].minor.yy209, &yymsp[0].minor.yy209); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; + case 384: /* table_primary ::= db_name NK_DOT table_name alias_opt */ +{ yylhsminor.yy674 = createRealTableNode(pCxt, &yymsp[-3].minor.yy421, &yymsp[-1].minor.yy421, &yymsp[0].minor.yy421); } + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; - case 380: /* table_primary ::= subquery alias_opt */ -{ yylhsminor.yy632 = createTempTableNode(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy632), &yymsp[0].minor.yy209); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + case 385: /* table_primary ::= subquery alias_opt */ +{ yylhsminor.yy674 = createTempTableNode(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy674), &yymsp[0].minor.yy421); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 382: /* alias_opt ::= */ -{ yymsp[1].minor.yy209 = nil_token; } + case 387: /* alias_opt ::= */ +{ yymsp[1].minor.yy421 = nil_token; } break; - case 383: /* alias_opt ::= table_alias */ -{ yylhsminor.yy209 = yymsp[0].minor.yy209; } - yymsp[0].minor.yy209 = yylhsminor.yy209; + case 388: /* alias_opt ::= table_alias */ +{ yylhsminor.yy421 = yymsp[0].minor.yy421; } + yymsp[0].minor.yy421 = yylhsminor.yy421; break; - case 384: /* alias_opt ::= AS table_alias */ -{ yymsp[-1].minor.yy209 = yymsp[0].minor.yy209; } + case 389: /* alias_opt ::= AS table_alias */ +{ yymsp[-1].minor.yy421 = yymsp[0].minor.yy421; } break; - case 385: /* parenthesized_joined_table ::= NK_LP joined_table NK_RP */ - case 386: /* parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP */ yytestcase(yyruleno==386); -{ yymsp[-2].minor.yy632 = yymsp[-1].minor.yy632; } + case 390: /* parenthesized_joined_table ::= NK_LP joined_table NK_RP */ + case 391: /* parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP */ yytestcase(yyruleno==391); +{ yymsp[-2].minor.yy674 = yymsp[-1].minor.yy674; } break; - case 387: /* joined_table ::= table_reference join_type JOIN table_reference ON search_condition */ -{ yylhsminor.yy632 = createJoinTableNode(pCxt, yymsp[-4].minor.yy612, yymsp[-5].minor.yy632, yymsp[-2].minor.yy632, yymsp[0].minor.yy632); } - yymsp[-5].minor.yy632 = yylhsminor.yy632; + case 392: /* joined_table ::= table_reference join_type JOIN table_reference ON search_condition */ +{ yylhsminor.yy674 = createJoinTableNode(pCxt, yymsp[-4].minor.yy636, yymsp[-5].minor.yy674, yymsp[-2].minor.yy674, yymsp[0].minor.yy674); } + yymsp[-5].minor.yy674 = yylhsminor.yy674; break; - case 388: /* join_type ::= */ -{ yymsp[1].minor.yy612 = JOIN_TYPE_INNER; } + case 393: /* join_type ::= */ +{ yymsp[1].minor.yy636 = JOIN_TYPE_INNER; } break; - case 389: /* join_type ::= INNER */ -{ yymsp[0].minor.yy612 = JOIN_TYPE_INNER; } + case 394: /* join_type ::= INNER */ +{ yymsp[0].minor.yy636 = JOIN_TYPE_INNER; } break; - case 390: /* query_specification ::= SELECT set_quantifier_opt select_list from_clause where_clause_opt partition_by_clause_opt twindow_clause_opt group_by_clause_opt having_clause_opt */ + case 395: /* query_specification ::= SELECT set_quantifier_opt select_list from_clause where_clause_opt partition_by_clause_opt twindow_clause_opt group_by_clause_opt having_clause_opt */ { - yymsp[-8].minor.yy632 = createSelectStmt(pCxt, yymsp[-7].minor.yy137, yymsp[-6].minor.yy424, yymsp[-5].minor.yy632); - yymsp[-8].minor.yy632 = addWhereClause(pCxt, yymsp[-8].minor.yy632, yymsp[-4].minor.yy632); - yymsp[-8].minor.yy632 = addPartitionByClause(pCxt, yymsp[-8].minor.yy632, yymsp[-3].minor.yy424); - yymsp[-8].minor.yy632 = addWindowClauseClause(pCxt, yymsp[-8].minor.yy632, yymsp[-2].minor.yy632); - yymsp[-8].minor.yy632 = addGroupByClause(pCxt, yymsp[-8].minor.yy632, yymsp[-1].minor.yy424); - yymsp[-8].minor.yy632 = addHavingClause(pCxt, yymsp[-8].minor.yy632, yymsp[0].minor.yy632); + yymsp[-8].minor.yy674 = createSelectStmt(pCxt, yymsp[-7].minor.yy621, yymsp[-6].minor.yy530, yymsp[-5].minor.yy674); + yymsp[-8].minor.yy674 = addWhereClause(pCxt, yymsp[-8].minor.yy674, yymsp[-4].minor.yy674); + yymsp[-8].minor.yy674 = addPartitionByClause(pCxt, yymsp[-8].minor.yy674, yymsp[-3].minor.yy530); + yymsp[-8].minor.yy674 = addWindowClauseClause(pCxt, yymsp[-8].minor.yy674, yymsp[-2].minor.yy674); + yymsp[-8].minor.yy674 = addGroupByClause(pCxt, yymsp[-8].minor.yy674, yymsp[-1].minor.yy530); + yymsp[-8].minor.yy674 = addHavingClause(pCxt, yymsp[-8].minor.yy674, yymsp[0].minor.yy674); } break; - case 393: /* set_quantifier_opt ::= ALL */ -{ yymsp[0].minor.yy137 = false; } + case 398: /* set_quantifier_opt ::= ALL */ +{ yymsp[0].minor.yy621 = false; } break; - case 394: /* select_list ::= NK_STAR */ -{ yymsp[0].minor.yy424 = NULL; } + case 399: /* select_list ::= NK_STAR */ +{ yymsp[0].minor.yy530 = NULL; } break; - case 399: /* select_item ::= common_expression column_alias */ -{ yylhsminor.yy632 = setProjectionAlias(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy632), &yymsp[0].minor.yy209); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + case 404: /* select_item ::= common_expression column_alias */ +{ yylhsminor.yy674 = setProjectionAlias(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy674), &yymsp[0].minor.yy421); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 400: /* select_item ::= common_expression AS column_alias */ -{ yylhsminor.yy632 = setProjectionAlias(pCxt, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), &yymsp[0].minor.yy209); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 405: /* select_item ::= common_expression AS column_alias */ +{ yylhsminor.yy674 = setProjectionAlias(pCxt, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), &yymsp[0].minor.yy421); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 405: /* partition_by_clause_opt ::= PARTITION BY expression_list */ - case 422: /* group_by_clause_opt ::= GROUP BY group_by_list */ yytestcase(yyruleno==422); - case 434: /* order_by_clause_opt ::= ORDER BY sort_specification_list */ yytestcase(yyruleno==434); -{ yymsp[-2].minor.yy424 = yymsp[0].minor.yy424; } + case 410: /* partition_by_clause_opt ::= PARTITION BY expression_list */ + case 427: /* group_by_clause_opt ::= GROUP BY group_by_list */ yytestcase(yyruleno==427); + case 439: /* order_by_clause_opt ::= ORDER BY sort_specification_list */ yytestcase(yyruleno==439); +{ yymsp[-2].minor.yy530 = yymsp[0].minor.yy530; } break; - case 407: /* twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP */ -{ yymsp[-5].minor.yy632 = createSessionWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy632), releaseRawExprNode(pCxt, yymsp[-1].minor.yy632)); } + case 412: /* twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP */ +{ yymsp[-5].minor.yy674 = createSessionWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy674), releaseRawExprNode(pCxt, yymsp[-1].minor.yy674)); } break; - case 408: /* twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP */ -{ yymsp[-3].minor.yy632 = createStateWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy632)); } + case 413: /* twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP */ +{ yymsp[-3].minor.yy674 = createStateWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy674)); } break; - case 409: /* twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt */ -{ yymsp[-5].minor.yy632 = createIntervalWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy632), NULL, yymsp[-1].minor.yy632, yymsp[0].minor.yy632); } + case 414: /* twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt */ +{ yymsp[-5].minor.yy674 = createIntervalWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy674), NULL, yymsp[-1].minor.yy674, yymsp[0].minor.yy674); } break; - case 410: /* twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt */ -{ yymsp[-7].minor.yy632 = createIntervalWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-5].minor.yy632), releaseRawExprNode(pCxt, yymsp[-3].minor.yy632), yymsp[-1].minor.yy632, yymsp[0].minor.yy632); } + case 415: /* twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt */ +{ yymsp[-7].minor.yy674 = createIntervalWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-5].minor.yy674), releaseRawExprNode(pCxt, yymsp[-3].minor.yy674), yymsp[-1].minor.yy674, yymsp[0].minor.yy674); } break; - case 412: /* sliding_opt ::= SLIDING NK_LP duration_literal NK_RP */ -{ yymsp[-3].minor.yy632 = releaseRawExprNode(pCxt, yymsp[-1].minor.yy632); } + case 417: /* sliding_opt ::= SLIDING NK_LP duration_literal NK_RP */ +{ yymsp[-3].minor.yy674 = releaseRawExprNode(pCxt, yymsp[-1].minor.yy674); } break; - case 414: /* fill_opt ::= FILL NK_LP fill_mode NK_RP */ -{ yymsp[-3].minor.yy632 = createFillNode(pCxt, yymsp[-1].minor.yy54, NULL); } + case 419: /* fill_opt ::= FILL NK_LP fill_mode NK_RP */ +{ yymsp[-3].minor.yy674 = createFillNode(pCxt, yymsp[-1].minor.yy320, NULL); } break; - case 415: /* fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP */ -{ yymsp[-5].minor.yy632 = createFillNode(pCxt, FILL_MODE_VALUE, createNodeListNode(pCxt, yymsp[-1].minor.yy424)); } + case 420: /* fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP */ +{ yymsp[-5].minor.yy674 = createFillNode(pCxt, FILL_MODE_VALUE, createNodeListNode(pCxt, yymsp[-1].minor.yy530)); } break; - case 416: /* fill_mode ::= NONE */ -{ yymsp[0].minor.yy54 = FILL_MODE_NONE; } + case 421: /* fill_mode ::= NONE */ +{ yymsp[0].minor.yy320 = FILL_MODE_NONE; } break; - case 417: /* fill_mode ::= PREV */ -{ yymsp[0].minor.yy54 = FILL_MODE_PREV; } + case 422: /* fill_mode ::= PREV */ +{ yymsp[0].minor.yy320 = FILL_MODE_PREV; } break; - case 418: /* fill_mode ::= NULL */ -{ yymsp[0].minor.yy54 = FILL_MODE_NULL; } + case 423: /* fill_mode ::= NULL */ +{ yymsp[0].minor.yy320 = FILL_MODE_NULL; } break; - case 419: /* fill_mode ::= LINEAR */ -{ yymsp[0].minor.yy54 = FILL_MODE_LINEAR; } + case 424: /* fill_mode ::= LINEAR */ +{ yymsp[0].minor.yy320 = FILL_MODE_LINEAR; } break; - case 420: /* fill_mode ::= NEXT */ -{ yymsp[0].minor.yy54 = FILL_MODE_NEXT; } + case 425: /* fill_mode ::= NEXT */ +{ yymsp[0].minor.yy320 = FILL_MODE_NEXT; } break; - case 423: /* group_by_list ::= expression */ -{ yylhsminor.yy424 = createNodeList(pCxt, createGroupingSetNode(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); } - yymsp[0].minor.yy424 = yylhsminor.yy424; + case 428: /* group_by_list ::= expression */ +{ yylhsminor.yy530 = createNodeList(pCxt, createGroupingSetNode(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } + yymsp[0].minor.yy530 = yylhsminor.yy530; break; - case 424: /* group_by_list ::= group_by_list NK_COMMA expression */ -{ yylhsminor.yy424 = addNodeToList(pCxt, yymsp[-2].minor.yy424, createGroupingSetNode(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); } - yymsp[-2].minor.yy424 = yylhsminor.yy424; + case 429: /* group_by_list ::= group_by_list NK_COMMA expression */ +{ yylhsminor.yy530 = addNodeToList(pCxt, yymsp[-2].minor.yy530, createGroupingSetNode(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } + yymsp[-2].minor.yy530 = yylhsminor.yy530; break; - case 427: /* query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt */ + case 432: /* query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt */ { - yylhsminor.yy632 = addOrderByClause(pCxt, yymsp[-3].minor.yy632, yymsp[-2].minor.yy424); - yylhsminor.yy632 = addSlimitClause(pCxt, yylhsminor.yy632, yymsp[-1].minor.yy632); - yylhsminor.yy632 = addLimitClause(pCxt, yylhsminor.yy632, yymsp[0].minor.yy632); + yylhsminor.yy674 = addOrderByClause(pCxt, yymsp[-3].minor.yy674, yymsp[-2].minor.yy530); + yylhsminor.yy674 = addSlimitClause(pCxt, yylhsminor.yy674, yymsp[-1].minor.yy674); + yylhsminor.yy674 = addLimitClause(pCxt, yylhsminor.yy674, yymsp[0].minor.yy674); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; - case 429: /* query_expression_body ::= query_expression_body UNION ALL query_expression_body */ -{ yylhsminor.yy632 = createSetOperator(pCxt, SET_OP_TYPE_UNION_ALL, yymsp[-3].minor.yy632, yymsp[0].minor.yy632); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; + case 434: /* query_expression_body ::= query_expression_body UNION ALL query_expression_body */ +{ yylhsminor.yy674 = createSetOperator(pCxt, SET_OP_TYPE_UNION_ALL, yymsp[-3].minor.yy674, yymsp[0].minor.yy674); } + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; - case 430: /* query_expression_body ::= query_expression_body UNION query_expression_body */ -{ yylhsminor.yy632 = createSetOperator(pCxt, SET_OP_TYPE_UNION, yymsp[-2].minor.yy632, yymsp[0].minor.yy632); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 435: /* query_expression_body ::= query_expression_body UNION query_expression_body */ +{ yylhsminor.yy674 = createSetOperator(pCxt, SET_OP_TYPE_UNION, yymsp[-2].minor.yy674, yymsp[0].minor.yy674); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 432: /* query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP */ -{ yymsp[-5].minor.yy632 = yymsp[-4].minor.yy632; } - yy_destructor(yypParser,352,&yymsp[-3].minor); - yy_destructor(yypParser,353,&yymsp[-2].minor); - yy_destructor(yypParser,354,&yymsp[-1].minor); + case 437: /* query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP */ +{ yymsp[-5].minor.yy674 = yymsp[-4].minor.yy674; } + yy_destructor(yypParser,351,&yymsp[-3].minor); + yy_destructor(yypParser,352,&yymsp[-2].minor); + yy_destructor(yypParser,353,&yymsp[-1].minor); break; - case 436: /* slimit_clause_opt ::= SLIMIT NK_INTEGER */ - case 440: /* limit_clause_opt ::= LIMIT NK_INTEGER */ yytestcase(yyruleno==440); -{ yymsp[-1].minor.yy632 = createLimitNode(pCxt, &yymsp[0].minor.yy0, NULL); } + case 441: /* slimit_clause_opt ::= SLIMIT NK_INTEGER */ + case 445: /* limit_clause_opt ::= LIMIT NK_INTEGER */ yytestcase(yyruleno==445); +{ yymsp[-1].minor.yy674 = createLimitNode(pCxt, &yymsp[0].minor.yy0, NULL); } break; - case 437: /* slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER */ - case 441: /* limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER */ yytestcase(yyruleno==441); -{ yymsp[-3].minor.yy632 = createLimitNode(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0); } + case 442: /* slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER */ + case 446: /* limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER */ yytestcase(yyruleno==446); +{ yymsp[-3].minor.yy674 = createLimitNode(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0); } break; - case 438: /* slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER */ - case 442: /* limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER */ yytestcase(yyruleno==442); -{ yymsp[-3].minor.yy632 = createLimitNode(pCxt, &yymsp[0].minor.yy0, &yymsp[-2].minor.yy0); } + case 443: /* slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER */ + case 447: /* limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER */ yytestcase(yyruleno==447); +{ yymsp[-3].minor.yy674 = createLimitNode(pCxt, &yymsp[0].minor.yy0, &yymsp[-2].minor.yy0); } break; - case 443: /* subquery ::= NK_LP query_expression NK_RP */ -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, yymsp[-1].minor.yy632); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 448: /* subquery ::= NK_LP query_expression NK_RP */ +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, yymsp[-1].minor.yy674); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 447: /* sort_specification ::= expression ordering_specification_opt null_ordering_opt */ -{ yylhsminor.yy632 = createOrderByExprNode(pCxt, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), yymsp[-1].minor.yy578, yymsp[0].minor.yy217); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 452: /* sort_specification ::= expression ordering_specification_opt null_ordering_opt */ +{ yylhsminor.yy674 = createOrderByExprNode(pCxt, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), yymsp[-1].minor.yy610, yymsp[0].minor.yy107); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 448: /* ordering_specification_opt ::= */ -{ yymsp[1].minor.yy578 = ORDER_ASC; } + case 453: /* ordering_specification_opt ::= */ +{ yymsp[1].minor.yy610 = ORDER_ASC; } break; - case 449: /* ordering_specification_opt ::= ASC */ -{ yymsp[0].minor.yy578 = ORDER_ASC; } + case 454: /* ordering_specification_opt ::= ASC */ +{ yymsp[0].minor.yy610 = ORDER_ASC; } break; - case 450: /* ordering_specification_opt ::= DESC */ -{ yymsp[0].minor.yy578 = ORDER_DESC; } + case 455: /* ordering_specification_opt ::= DESC */ +{ yymsp[0].minor.yy610 = ORDER_DESC; } break; - case 451: /* null_ordering_opt ::= */ -{ yymsp[1].minor.yy217 = NULL_ORDER_DEFAULT; } + case 456: /* null_ordering_opt ::= */ +{ yymsp[1].minor.yy107 = NULL_ORDER_DEFAULT; } break; - case 452: /* null_ordering_opt ::= NULLS FIRST */ -{ yymsp[-1].minor.yy217 = NULL_ORDER_FIRST; } + case 457: /* null_ordering_opt ::= NULLS FIRST */ +{ yymsp[-1].minor.yy107 = NULL_ORDER_FIRST; } break; - case 453: /* null_ordering_opt ::= NULLS LAST */ -{ yymsp[-1].minor.yy217 = NULL_ORDER_LAST; } + case 458: /* null_ordering_opt ::= NULLS LAST */ +{ yymsp[-1].minor.yy107 = NULL_ORDER_LAST; } break; default: break; diff --git a/source/libs/parser/test/mockCatalogService.cpp b/source/libs/parser/test/mockCatalogService.cpp index 68f9e9d36d..9758da7899 100644 --- a/source/libs/parser/test/mockCatalogService.cpp +++ b/source/libs/parser/test/mockCatalogService.cpp @@ -285,7 +285,7 @@ class MockCatalogServiceImpl { } void createSmaIndex(const SMCreateSmaReq* pReq) { - STableIndexInfo info; + STableIndexInfo info = {0}; info.intervalUnit = pReq->intervalUnit; info.slidingUnit = pReq->slidingUnit; info.interval = pReq->interval; diff --git a/source/libs/parser/test/parInitialATest.cpp b/source/libs/parser/test/parInitialATest.cpp index cc23fbbc60..3247ff352e 100644 --- a/source/libs/parser/test/parInitialATest.cpp +++ b/source/libs/parser/test/parInitialATest.cpp @@ -43,7 +43,40 @@ TEST_F(ParserInitialATest, alterDatabase) { run("ALTER DATABASE wxy_db KEEP 2400"); } -// todo ALTER local +TEST_F(ParserInitialATest, alterLocal) { + useDb("root", "test"); + + pair expect; + + auto clearAlterLocal = [&]() { + expect.first.clear(); + expect.second.clear(); + }; + + auto setAlterLocalFunc = [&](const char* pConfig, const char* pValue = nullptr) { + expect.first.assign(pConfig); + if (nullptr != pValue) { + expect.second.assign(pValue); + } + }; + + setCheckDdlFunc([&](const SQuery* pQuery, ParserStage stage) { + ASSERT_EQ(nodeType(pQuery->pRoot), QUERY_NODE_ALTER_LOCAL_STMT); + ASSERT_EQ(pQuery->execMode, QUERY_EXEC_MODE_LOCAL); + SAlterLocalStmt* pStmt = (SAlterLocalStmt*)pQuery->pRoot; + ASSERT_EQ(string(pStmt->config), expect.first); + ASSERT_EQ(string(pStmt->value), expect.second); + }); + + setAlterLocalFunc("resetlog"); + run("ALTER LOCAL 'resetlog'"); + clearAlterLocal(); + + setAlterLocalFunc("querypolicy", "2"); + run("ALTER LOCAL 'querypolicy' '2'"); + clearAlterLocal(); +} + // todo ALTER stable /* diff --git a/source/libs/parser/test/parInitialCTest.cpp b/source/libs/parser/test/parInitialCTest.cpp index 0286dba363..1ab251fa48 100644 --- a/source/libs/parser/test/parInitialCTest.cpp +++ b/source/libs/parser/test/parInitialCTest.cpp @@ -14,7 +14,6 @@ */ #include "parTestUtil.h" -#include "ttime.h" using namespace std; @@ -242,9 +241,47 @@ TEST_F(ParserInitialCTest, createDatabaseSemanticCheck) { TEST_F(ParserInitialCTest, createDnode) { useDb("root", "test"); - run("CREATE DNODE abc1 PORT 7000"); + SCreateDnodeReq expect = {0}; - run("CREATE DNODE 1.1.1.1 PORT 9000"); + auto clearCreateDnodeReq = [&]() { memset(&expect, 0, sizeof(SCreateDnodeReq)); }; + + auto setCreateDnodeReqFunc = [&](const char* pFqdn, int32_t port = tsServerPort) { + strcpy(expect.fqdn, pFqdn); + expect.port = port; + }; + + setCheckDdlFunc([&](const SQuery* pQuery, ParserStage stage) { + ASSERT_EQ(nodeType(pQuery->pRoot), QUERY_NODE_CREATE_DNODE_STMT); + SCreateDnodeReq req = {0}; + ASSERT_TRUE(TSDB_CODE_SUCCESS == tDeserializeSCreateDnodeReq(pQuery->pCmdMsg->pMsg, pQuery->pCmdMsg->msgLen, &req)); + + ASSERT_EQ(std::string(req.fqdn), std::string(expect.fqdn)); + ASSERT_EQ(req.port, expect.port); + }); + + setCreateDnodeReqFunc("abc1", 7030); + run("CREATE DNODE 'abc1' PORT 7030"); + clearCreateDnodeReq(); + + setCreateDnodeReqFunc("1.1.1.1", 8030); + run("CREATE DNODE 1.1.1.1 PORT 8030"); + clearCreateDnodeReq(); + + setCreateDnodeReqFunc("host1", 9030); + run("CREATE DNODE host1 PORT 9030"); + clearCreateDnodeReq(); + + setCreateDnodeReqFunc("abc2", 7040); + run("CREATE DNODE 'abc2:7040'"); + clearCreateDnodeReq(); + + setCreateDnodeReqFunc("1.1.1.2"); + run("CREATE DNODE 1.1.1.2"); + clearCreateDnodeReq(); + + setCreateDnodeReqFunc("host2"); + run("CREATE DNODE host2"); + clearCreateDnodeReq(); } // CREATE [AGGREGATE] FUNCTION [IF NOT EXISTS] func_name AS library_path OUTPUTTYPE type_name [BUFSIZE value] @@ -322,13 +359,17 @@ TEST_F(ParserInitialCTest, createStable) { memset(&expect, 0, sizeof(SMCreateStbReq)); }; - auto setCreateStbReqFunc = [&](const char* pTbname, int8_t igExists = 0, - float xFilesFactor = TSDB_DEFAULT_ROLLUP_FILE_FACTOR, + auto setCreateStbReqFunc = [&](const char* pTbname, int8_t igExists = 0, int64_t delay1 = -1, int64_t delay2 = -1, + int64_t watermark1 = TSDB_DEFAULT_ROLLUP_WATERMARK, + int64_t watermark2 = TSDB_DEFAULT_ROLLUP_WATERMARK, int32_t ttl = TSDB_DEFAULT_TABLE_TTL, const char* pComment = nullptr) { int32_t len = snprintf(expect.name, sizeof(expect.name), "0.test.%s", pTbname); expect.name[len] = '\0'; expect.igExists = igExists; - expect.xFilesFactor = xFilesFactor; + expect.delay1 = delay1; + expect.delay2 = delay2; + expect.watermark1 = watermark1; + expect.watermark2 = watermark2; expect.ttl = ttl; if (nullptr != pComment) { expect.comment = strdup(pComment); @@ -366,8 +407,10 @@ TEST_F(ParserInitialCTest, createStable) { ASSERT_EQ(std::string(req.name), std::string(expect.name)); ASSERT_EQ(req.igExists, expect.igExists); - ASSERT_EQ(req.xFilesFactor, expect.xFilesFactor); - ASSERT_EQ(req.delay, expect.delay); + ASSERT_EQ(req.delay1, expect.delay1); + ASSERT_EQ(req.delay2, expect.delay2); + ASSERT_EQ(req.watermark1, expect.watermark1); + ASSERT_EQ(req.watermark2, expect.watermark2); ASSERT_EQ(req.ttl, expect.ttl); ASSERT_EQ(req.numOfColumns, expect.numOfColumns); ASSERT_EQ(req.numOfTags, expect.numOfTags); @@ -418,7 +461,8 @@ TEST_F(ParserInitialCTest, createStable) { run("CREATE STABLE t1(ts TIMESTAMP, c1 INT) TAGS(id INT)"); clearCreateStbReq(); - setCreateStbReqFunc("t1", 1, 0.1, 100, "test create table"); + setCreateStbReqFunc("t1", 1, 100 * MILLISECOND_PER_SECOND, 10 * MILLISECOND_PER_MINUTE, 10, + 1 * MILLISECOND_PER_MINUTE, 100, "test create table"); addFieldToCreateStbReqFunc(true, "ts", TSDB_DATA_TYPE_TIMESTAMP, 0, 0); addFieldToCreateStbReqFunc(true, "c1", TSDB_DATA_TYPE_INT); addFieldToCreateStbReqFunc(true, "c2", TSDB_DATA_TYPE_UINT); @@ -456,15 +500,20 @@ TEST_F(ParserInitialCTest, createStable) { "TAGS (a1 TIMESTAMP, a2 INT, a3 INT UNSIGNED, a4 BIGINT, a5 BIGINT UNSIGNED, a6 FLOAT, a7 DOUBLE, " "a8 BINARY(20), a9 SMALLINT, a10 SMALLINT UNSIGNED COMMENT 'test column comment', a11 TINYINT, " "a12 TINYINT UNSIGNED, a13 BOOL, a14 NCHAR(30), a15 VARCHAR(50)) " - "TTL 100 COMMENT 'test create table' SMA(c1, c2, c3) ROLLUP (MIN) FILE_FACTOR 0.1"); + "TTL 100 COMMENT 'test create table' SMA(c1, c2, c3) ROLLUP (MIN) MAX_DELAY 100s,10m WATERMARK 10a,1m"); clearCreateStbReq(); } TEST_F(ParserInitialCTest, createStableSemanticCheck) { useDb("root", "test"); - run("CREATE STABLE stb2 (ts TIMESTAMP, c1 INT) TAGS (tag1 INT) ROLLUP(CEIL) FILE_FACTOR 0.1", - TSDB_CODE_PAR_INVALID_ROLLUP_OPTION, PARSER_STAGE_TRANSLATE); + run("CREATE STABLE stb2 (ts TIMESTAMP, c1 INT) TAGS (tag1 INT) ROLLUP(CEIL)", TSDB_CODE_PAR_INVALID_ROLLUP_OPTION); + + run("CREATE STABLE stb2 (ts TIMESTAMP, c1 INT) TAGS (tag1 INT) ROLLUP(MAX) MAX_DELAY 0s WATERMARK 1m", + TSDB_CODE_PAR_INVALID_RANGE_OPTION); + + run("CREATE STABLE stb2 (ts TIMESTAMP, c1 INT) TAGS (tag1 INT) ROLLUP(MAX) MAX_DELAY 10s WATERMARK 18m", + TSDB_CODE_PAR_INVALID_RANGE_OPTION); } TEST_F(ParserInitialCTest, createStream) { @@ -477,7 +526,7 @@ TEST_F(ParserInitialCTest, createStream) { memset(&expect, 0, sizeof(SCMCreateStreamReq)); }; - auto setCreateStbReqFunc = + auto setCreateStreamReqFunc = [&](const char* pStream, const char* pSrcDb, const char* pSql, const char* pDstStb = nullptr, int8_t igExists = 0, int8_t triggerType = STREAM_TRIGGER_AT_ONCE, int64_t maxDelay = 0, int64_t watermark = 0) { snprintf(expect.name, sizeof(expect.name), "0.%s", pStream); @@ -509,21 +558,21 @@ TEST_F(ParserInitialCTest, createStream) { tFreeSCMCreateStreamReq(&req); }); - setCreateStbReqFunc("s1", "test", "create stream s1 as select * from t1"); + setCreateStreamReqFunc("s1", "test", "create stream s1 as select * from t1"); run("CREATE STREAM s1 AS SELECT * FROM t1"); clearCreateStreamReq(); - setCreateStbReqFunc("s1", "test", "create stream if not exists s1 as select * from t1", nullptr, 1); + setCreateStreamReqFunc("s1", "test", "create stream if not exists s1 as select * from t1", nullptr, 1); run("CREATE STREAM IF NOT EXISTS s1 AS SELECT * FROM t1"); clearCreateStreamReq(); - setCreateStbReqFunc("s1", "test", "create stream s1 into st1 as select * from t1", "st1"); + setCreateStreamReqFunc("s1", "test", "create stream s1 into st1 as select * from t1", "st1"); run("CREATE STREAM s1 INTO st1 AS SELECT * FROM t1"); clearCreateStreamReq(); - setCreateStbReqFunc("s1", "test", - "create stream if not exists s1 trigger max_delay 20s watermark 10s into st1 as select * from t1", - "st1", 1, STREAM_TRIGGER_MAX_DELAY, 20 * MILLISECOND_PER_SECOND, 10 * MILLISECOND_PER_SECOND); + setCreateStreamReqFunc( + "s1", "test", "create stream if not exists s1 trigger max_delay 20s watermark 10s into st1 as select * from t1", + "st1", 1, STREAM_TRIGGER_MAX_DELAY, 20 * MILLISECOND_PER_SECOND, 10 * MILLISECOND_PER_SECOND); run("CREATE STREAM IF NOT EXISTS s1 TRIGGER MAX_DELAY 20s WATERMARK 10s INTO st1 AS SELECT * FROM t1"); clearCreateStreamReq(); } @@ -552,7 +601,7 @@ TEST_F(ParserInitialCTest, createTable) { "TAGS (a1 TIMESTAMP, a2 INT, a3 INT UNSIGNED, a4 BIGINT, a5 BIGINT UNSIGNED, a6 FLOAT, a7 DOUBLE, a8 BINARY(20), " "a9 SMALLINT, a10 SMALLINT UNSIGNED COMMENT 'test column comment', a11 TINYINT, a12 TINYINT UNSIGNED, a13 BOOL, " "a14 NCHAR(30), a15 VARCHAR(50)) " - "TTL 100 COMMENT 'test create table' SMA(c1, c2, c3) ROLLUP (MIN) FILE_FACTOR 0.1"); + "TTL 100 COMMENT 'test create table' SMA(c1, c2, c3) ROLLUP (MIN)"); run("CREATE TABLE IF NOT EXISTS t1 USING st1 TAGS(1, 'wxy', NOW)"); diff --git a/source/libs/parser/test/parInitialDTest.cpp b/source/libs/parser/test/parInitialDTest.cpp index 8562926789..4ecbb7b6d8 100644 --- a/source/libs/parser/test/parInitialDTest.cpp +++ b/source/libs/parser/test/parInitialDTest.cpp @@ -64,8 +64,9 @@ TEST_F(ParserInitialDTest, dropConsumerGroup) { SMDropCgroupReq expect = {0}; - auto setDropCgroupReqFunc = [&](const char* pTopicName, const char* pCGroupName, int8_t igNotExists = 0) { - memset(&expect, 0, sizeof(SMDropCgroupReq)); + auto clearDropCgroupReq = [&]() { memset(&expect, 0, sizeof(SMDropCgroupReq)); }; + + auto setDropCgroupReq = [&](const char* pTopicName, const char* pCGroupName, int8_t igNotExists = 0) { snprintf(expect.topic, sizeof(expect.topic), "0.%s", pTopicName); strcpy(expect.cgroup, pCGroupName); expect.igNotExists = igNotExists; @@ -81,15 +82,51 @@ TEST_F(ParserInitialDTest, dropConsumerGroup) { ASSERT_EQ(req.igNotExists, expect.igNotExists); }); - setDropCgroupReqFunc("tp1", "cg1"); + setDropCgroupReq("tp1", "cg1"); run("DROP CONSUMER GROUP cg1 ON tp1"); + clearDropCgroupReq(); - setDropCgroupReqFunc("tp1", "cg1", 1); + setDropCgroupReq("tp1", "cg1", 1); run("DROP CONSUMER GROUP IF EXISTS cg1 ON tp1"); + clearDropCgroupReq(); } // todo DROP database + // todo DROP dnode +TEST_F(ParserInitialDTest, dropDnode) { + useDb("root", "test"); + + SDropDnodeReq expect = {0}; + + auto clearDropDnodeReq = [&]() { memset(&expect, 0, sizeof(SDropDnodeReq)); }; + + auto setDropDnodeReqById = [&](int32_t dnodeId) { expect.dnodeId = dnodeId; }; + + auto setDropDnodeReqByEndpoint = [&](const char* pFqdn, int32_t port) { + strcpy(expect.fqdn, pFqdn); + expect.port = port; + }; + + setCheckDdlFunc([&](const SQuery* pQuery, ParserStage stage) { + ASSERT_EQ(nodeType(pQuery->pRoot), QUERY_NODE_DROP_DNODE_STMT); + SDropDnodeReq req = {0}; + ASSERT_TRUE(TSDB_CODE_SUCCESS == tDeserializeSDropDnodeReq(pQuery->pCmdMsg->pMsg, pQuery->pCmdMsg->msgLen, &req)); + + ASSERT_EQ(req.dnodeId, expect.dnodeId); + ASSERT_EQ(std::string(req.fqdn), std::string(expect.fqdn)); + ASSERT_EQ(req.port, expect.port); + }); + + setDropDnodeReqById(1); + run("DROP DNODE 1"); + clearDropDnodeReq(); + + setDropDnodeReqByEndpoint("host1", 7030); + run("DROP DNODE 'host1:7030'"); + clearDropDnodeReq(); +} + // todo DROP function TEST_F(ParserInitialDTest, dropIndex) { diff --git a/source/libs/parser/test/parShowToUse.cpp b/source/libs/parser/test/parShowToUse.cpp index d468ff25e3..940b6ea8ac 100644 --- a/source/libs/parser/test/parShowToUse.cpp +++ b/source/libs/parser/test/parShowToUse.cpp @@ -24,9 +24,45 @@ class ParserShowToUseTest : public ParserDdlTest {}; // todo SHOW accounts // todo SHOW apps // todo SHOW connections -// todo SHOW create database -// todo SHOW create stable -// todo SHOW create table + +TEST_F(ParserShowToUseTest, showCreateDatabase) { + useDb("root", "test"); + + setCheckDdlFunc([&](const SQuery* pQuery, ParserStage stage) { + ASSERT_EQ(nodeType(pQuery->pRoot), QUERY_NODE_SHOW_CREATE_DATABASE_STMT); + ASSERT_EQ(pQuery->execMode, QUERY_EXEC_MODE_LOCAL); + ASSERT_TRUE(pQuery->haveResultSet); + ASSERT_NE(((SShowCreateDatabaseStmt*)pQuery->pRoot)->pCfg, nullptr); + }); + + run("SHOW CREATE DATABASE test"); +} + +TEST_F(ParserShowToUseTest, showCreateSTable) { + useDb("root", "test"); + + setCheckDdlFunc([&](const SQuery* pQuery, ParserStage stage) { + ASSERT_EQ(nodeType(pQuery->pRoot), QUERY_NODE_SHOW_CREATE_STABLE_STMT); + ASSERT_EQ(pQuery->execMode, QUERY_EXEC_MODE_LOCAL); + ASSERT_TRUE(pQuery->haveResultSet); + ASSERT_NE(((SShowCreateTableStmt*)pQuery->pRoot)->pMeta, nullptr); + }); + + run("SHOW CREATE STABLE st1"); +} + +TEST_F(ParserShowToUseTest, showCreateTable) { + useDb("root", "test"); + + setCheckDdlFunc([&](const SQuery* pQuery, ParserStage stage) { + ASSERT_EQ(nodeType(pQuery->pRoot), QUERY_NODE_SHOW_CREATE_TABLE_STMT); + ASSERT_EQ(pQuery->execMode, QUERY_EXEC_MODE_LOCAL); + ASSERT_TRUE(pQuery->haveResultSet); + ASSERT_NE(((SShowCreateTableStmt*)pQuery->pRoot)->pMeta, nullptr); + }); + + run("SHOW CREATE TABLE t1"); +} TEST_F(ParserShowToUseTest, showDatabases) { useDb("root", "test"); diff --git a/source/libs/parser/test/parTestUtil.h b/source/libs/parser/test/parTestUtil.h index afdb763448..16c3d05b38 100644 --- a/source/libs/parser/test/parTestUtil.h +++ b/source/libs/parser/test/parTestUtil.h @@ -20,8 +20,11 @@ #define ALLOW_FORBID_FUNC +#include "cmdnodes.h" #include "querynodes.h" #include "taoserror.h" +#include "tglobal.h" +#include "ttime.h" namespace ParserTest { diff --git a/source/libs/planner/src/planLogicCreater.c b/source/libs/planner/src/planLogicCreater.c index cbc74a2711..3cd5eeb655 100644 --- a/source/libs/planner/src/planLogicCreater.c +++ b/source/libs/planner/src/planLogicCreater.c @@ -548,6 +548,7 @@ static int32_t createWindowLogicNodeBySession(SLogicPlanContext* pCxt, SSessionW pWindow->winType = WINDOW_TYPE_SESSION; pWindow->sessionGap = ((SValueNode*)pSession->pGap)->datum.i; + pWindow->windowAlgo = pCxt->pPlanCxt->streamQuery ? SESSION_ALGO_STREAM_SINGLE : SESSION_ALGO_MERGE; pWindow->pTspk = nodesCloneNode((SNode*)pSession->pCol); if (NULL == pWindow->pTspk) { @@ -572,7 +573,7 @@ static int32_t createWindowLogicNodeByInterval(SLogicPlanContext* pCxt, SInterva pWindow->sliding = (NULL != pInterval->pSliding ? ((SValueNode*)pInterval->pSliding)->datum.i : pWindow->interval); pWindow->slidingUnit = (NULL != pInterval->pSliding ? ((SValueNode*)pInterval->pSliding)->unit : pWindow->intervalUnit); - pWindow->intervalAlgo = pCxt->pPlanCxt->streamQuery ? INTERVAL_ALGO_STREAM_SINGLE : INTERVAL_ALGO_HASH; + pWindow->windowAlgo = pCxt->pPlanCxt->streamQuery ? INTERVAL_ALGO_STREAM_SINGLE : INTERVAL_ALGO_HASH; pWindow->pTspk = nodesCloneNode(pInterval->pCol); if (NULL == pWindow->pTspk) { diff --git a/source/libs/planner/src/planPhysiCreater.c b/source/libs/planner/src/planPhysiCreater.c index 447665af85..80be19064c 100644 --- a/source/libs/planner/src/planPhysiCreater.c +++ b/source/libs/planner/src/planPhysiCreater.c @@ -35,7 +35,7 @@ typedef struct SPhysiPlanContext { int32_t errCode; int16_t nextDataBlockId; SArray* pLocationHelper; - SArray* pExecNodeList; // SArray + SArray* pExecNodeList; // SArray } SPhysiPlanContext; static int32_t getSlotKey(SNode* pNode, const char* pStmtName, char* pKey) { @@ -523,7 +523,8 @@ static int32_t createSystemTableScanPhysiNode(SPhysiPlanContext* pCxt, SSubplan* pScan->showRewrite = pScanLogicNode->showRewrite; pScan->accountId = pCxt->pPlanCxt->acctId; - if (0 == strcmp(pScanLogicNode->tableName.tname, TSDB_INS_TABLE_USER_TABLES)) { + if (0 == strcmp(pScanLogicNode->tableName.tname, TSDB_INS_TABLE_USER_TABLES) || + 0 == strcmp(pScanLogicNode->tableName.tname, TSDB_INS_TABLE_USER_TABLE_DISTRIBUTED)) { vgroupInfoToNodeAddr(pScanLogicNode->pVgroupList->vgroups, &pSubplan->execNode); } SQueryNodeLoad node = {.addr = {.nodeId = MNODE_HANDLE, .epSet = pCxt->pPlanCxt->mgmtEpSet}, .load = 0}; @@ -968,8 +969,8 @@ static int32_t createWindowPhysiNodeFinalize(SPhysiPlanContext* pCxt, SNodeList* return code; } -static ENodeType getIntervalOperatorType(EIntervalAlgorithm intervalAlgo) { - switch (intervalAlgo) { +static ENodeType getIntervalOperatorType(EWindowAlgorithm windowAlgo) { + switch (windowAlgo) { case INTERVAL_ALGO_HASH: return QUERY_NODE_PHYSICAL_PLAN_HASH_INTERVAL; case INTERVAL_ALGO_MERGE: @@ -980,6 +981,14 @@ static ENodeType getIntervalOperatorType(EIntervalAlgorithm intervalAlgo) { return QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_INTERVAL; case INTERVAL_ALGO_STREAM_SINGLE: return QUERY_NODE_PHYSICAL_PLAN_STREAM_INTERVAL; + case SESSION_ALGO_STREAM_FINAL: + return QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION; + case SESSION_ALGO_STREAM_SEMI: + return QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION; + case SESSION_ALGO_STREAM_SINGLE: + return QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION; + case SESSION_ALGO_MERGE: + return QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION; default: break; } @@ -989,7 +998,7 @@ static ENodeType getIntervalOperatorType(EIntervalAlgorithm intervalAlgo) { static int32_t createIntervalPhysiNode(SPhysiPlanContext* pCxt, SNodeList* pChildren, SWindowLogicNode* pWindowLogicNode, SPhysiNode** pPhyNode) { SIntervalPhysiNode* pInterval = (SIntervalPhysiNode*)makePhysiNode( - pCxt, (SLogicNode*)pWindowLogicNode, getIntervalOperatorType(pWindowLogicNode->intervalAlgo)); + pCxt, (SLogicNode*)pWindowLogicNode, getIntervalOperatorType(pWindowLogicNode->windowAlgo)); if (NULL == pInterval) { return TSDB_CODE_OUT_OF_MEMORY; } @@ -1006,8 +1015,7 @@ static int32_t createIntervalPhysiNode(SPhysiPlanContext* pCxt, SNodeList* pChil static int32_t createSessionWindowPhysiNode(SPhysiPlanContext* pCxt, SNodeList* pChildren, SWindowLogicNode* pWindowLogicNode, SPhysiNode** pPhyNode) { SSessionWinodwPhysiNode* pSession = (SSessionWinodwPhysiNode*)makePhysiNode( - pCxt, (SLogicNode*)pWindowLogicNode, - (pCxt->pPlanCxt->streamQuery ? QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION : QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION)); + pCxt, (SLogicNode*)pWindowLogicNode, getIntervalOperatorType(pWindowLogicNode->windowAlgo)); if (NULL == pSession) { return TSDB_CODE_OUT_OF_MEMORY; } diff --git a/source/libs/planner/src/planSpliter.c b/source/libs/planner/src/planSpliter.c index 3a9e1d7405..fe02dd6425 100644 --- a/source/libs/planner/src/planSpliter.c +++ b/source/libs/planner/src/planSpliter.c @@ -376,8 +376,8 @@ static int32_t stbSplSplitIntervalForBatch(SSplitContext* pCxt, SStableSplitInfo SLogicNode* pPartWindow = NULL; int32_t code = stbSplCreatePartWindowNode((SWindowLogicNode*)pInfo->pSplitNode, &pPartWindow); if (TSDB_CODE_SUCCESS == code) { - ((SWindowLogicNode*)pPartWindow)->intervalAlgo = INTERVAL_ALGO_HASH; - ((SWindowLogicNode*)pInfo->pSplitNode)->intervalAlgo = INTERVAL_ALGO_MERGE; + ((SWindowLogicNode*)pPartWindow)->windowAlgo = INTERVAL_ALGO_HASH; + ((SWindowLogicNode*)pInfo->pSplitNode)->windowAlgo = INTERVAL_ALGO_MERGE; SNodeList* pMergeKeys = NULL; code = stbSplCreateMergeKeysByPrimaryKey(((SWindowLogicNode*)pInfo->pSplitNode)->pTspk, &pMergeKeys); if (TSDB_CODE_SUCCESS == code) { @@ -400,8 +400,8 @@ static int32_t stbSplSplitIntervalForStream(SSplitContext* pCxt, SStableSplitInf SLogicNode* pPartWindow = NULL; int32_t code = stbSplCreatePartWindowNode((SWindowLogicNode*)pInfo->pSplitNode, &pPartWindow); if (TSDB_CODE_SUCCESS == code) { - ((SWindowLogicNode*)pPartWindow)->intervalAlgo = INTERVAL_ALGO_STREAM_SEMI; - ((SWindowLogicNode*)pInfo->pSplitNode)->intervalAlgo = INTERVAL_ALGO_STREAM_FINAL; + ((SWindowLogicNode*)pPartWindow)->windowAlgo = INTERVAL_ALGO_STREAM_SEMI; + ((SWindowLogicNode*)pInfo->pSplitNode)->windowAlgo = INTERVAL_ALGO_STREAM_FINAL; code = stbSplCreateExchangeNode(pCxt, pInfo->pSplitNode, pPartWindow); } if (TSDB_CODE_SUCCESS == code) { @@ -421,8 +421,29 @@ static int32_t stbSplSplitInterval(SSplitContext* pCxt, SStableSplitInfo* pInfo) } } +static int32_t stbSplSplitSessionForStream(SSplitContext* pCxt, SStableSplitInfo* pInfo) { + SLogicNode* pPartWindow = NULL; + int32_t code = stbSplCreatePartWindowNode((SWindowLogicNode*)pInfo->pSplitNode, &pPartWindow); + if (TSDB_CODE_SUCCESS == code) { + ((SWindowLogicNode*)pPartWindow)->windowAlgo = SESSION_ALGO_STREAM_SEMI; + ((SWindowLogicNode*)pInfo->pSplitNode)->windowAlgo = SESSION_ALGO_STREAM_FINAL; + code = stbSplCreateExchangeNode(pCxt, pInfo->pSplitNode, pPartWindow); + } + if (TSDB_CODE_SUCCESS == code) { + code = nodesListMakeStrictAppend(&pInfo->pSubplan->pChildren, + (SNode*)splCreateScanSubplan(pCxt, pPartWindow, SPLIT_FLAG_STABLE_SPLIT)); + } + pInfo->pSubplan->subplanType = SUBPLAN_TYPE_MERGE; + ++(pCxt->groupId); + return code; +} + static int32_t stbSplSplitSession(SSplitContext* pCxt, SStableSplitInfo* pInfo) { - return TSDB_CODE_PLAN_INTERNAL_ERROR; + if (pCxt->pPlanCxt->streamQuery) { + return stbSplSplitSessionForStream(pCxt, pInfo); + } else { + return TSDB_CODE_PLAN_INTERNAL_ERROR; + } } static int32_t stbSplSplitWindowNode(SSplitContext* pCxt, SStableSplitInfo* pInfo) { @@ -537,10 +558,12 @@ static int32_t stbSplCreateMergeKeys(SNodeList* pSortKeys, SNodeList* pTargets, SNode* pNode = NULL; FOREACH(pNode, pSortKeys) { SOrderByExprNode* pSortKey = (SOrderByExprNode*)pNode; + SExprNode* pSortExpr = (SExprNode*)pSortKey->pExpr; SNode* pTarget = NULL; bool found = false; FOREACH(pTarget, pTargets) { - if (0 == strcmp(((SExprNode*)pSortKey->pExpr)->aliasName, ((SColumnNode*)pTarget)->colName)) { + if ((QUERY_NODE_COLUMN == nodeType(pSortExpr) && nodesEqualNode((SNode*)pSortExpr, pTarget)) || + (0 == strcmp(pSortExpr->aliasName, ((SColumnNode*)pTarget)->colName))) { code = nodesListMakeStrictAppend(&pMergeKeys, stbSplCreateOrderByExpr(pSortKey, pTarget)); if (TSDB_CODE_SUCCESS != code) { break; @@ -549,7 +572,7 @@ static int32_t stbSplCreateMergeKeys(SNodeList* pSortKeys, SNodeList* pTargets, } } if (TSDB_CODE_SUCCESS == code && !found) { - SNode* pCol = stbSplCreateColumnNode((SExprNode*)pSortKey->pExpr); + SNode* pCol = stbSplCreateColumnNode(pSortExpr); code = nodesListMakeStrictAppend(&pMergeKeys, stbSplCreateOrderByExpr(pSortKey, pCol)); if (TSDB_CODE_SUCCESS == code) { code = nodesListStrictAppend(pTargets, pCol); diff --git a/source/libs/planner/test/planOrderByTest.cpp b/source/libs/planner/test/planOrderByTest.cpp index b065ad0a59..2ca662cf86 100644 --- a/source/libs/planner/test/planOrderByTest.cpp +++ b/source/libs/planner/test/planOrderByTest.cpp @@ -28,6 +28,8 @@ TEST_F(PlanOrderByTest, basic) { // ORDER BY key is not in the projection list run("SELECT c1 FROM t1 ORDER BY c2"); + run("SELECT c1 AS a FROM t1 ORDER BY a"); + run("SELECT c1 + 10 AS a FROM t1 ORDER BY a"); } @@ -59,4 +61,6 @@ TEST_F(PlanOrderByTest, stable) { run("SELECT c2 FROM st1 ORDER BY c1"); run("SELECT c2 FROM st1 PARTITION BY c2 ORDER BY c1"); + + run("SELECT c1 AS a FROM st1 ORDER BY a"); } diff --git a/source/libs/qcom/src/queryUtil.c b/source/libs/qcom/src/queryUtil.c index 9322d7e594..2120d24d26 100644 --- a/source/libs/qcom/src/queryUtil.c +++ b/source/libs/qcom/src/queryUtil.c @@ -154,7 +154,7 @@ int32_t asyncSendMsgToServerExt(void* pTransporter, SEpSet* epSet, int64_t* pTra .info.persistHandle = persistHandle, .code = 0}; assert(pInfo->fp != NULL); - + TRACE_SET_ROOTID(&rpcMsg.info.traceId, pInfo->requestId); rpcSendRequestWithCtx(pTransporter, epSet, &rpcMsg, pTransporterId, rpcCtx); return TSDB_CODE_SUCCESS; } @@ -204,14 +204,14 @@ void destroyQueryExecRes(SQueryExecRes* pRes) { switch (pRes->msgType) { case TDMT_VND_ALTER_TABLE: case TDMT_MND_ALTER_STB: { - tFreeSTableMetaRsp((STableMetaRsp *)pRes->res); + tFreeSTableMetaRsp((STableMetaRsp*)pRes->res); taosMemoryFreeClear(pRes->res); break; } case TDMT_VND_SUBMIT: { tFreeSSubmitRsp((SSubmitRsp*)pRes->res); break; - } + } case TDMT_VND_QUERY: { taosArrayDestroy((SArray*)pRes->res); break; @@ -220,5 +220,3 @@ void destroyQueryExecRes(SQueryExecRes* pRes) { qError("invalid exec result for request type %d", pRes->msgType); } } - - diff --git a/source/libs/stream/src/streamDispatch.c b/source/libs/stream/src/streamDispatch.c index 59ec2b5ceb..ca10e7d956 100644 --- a/source/libs/stream/src/streamDispatch.c +++ b/source/libs/stream/src/streamDispatch.c @@ -134,6 +134,7 @@ int32_t streamBuildDispatchMsg(SStreamTask* pTask, SStreamDataBlock* data, SRpcM int32_t sz = taosArrayGetSize(vgInfo); for (int32_t i = 0; i < sz; i++) { SVgroupInfo* pVgInfo = taosArrayGet(vgInfo, i); + ASSERT(pVgInfo->vgId > 0); if (hashValue >= pVgInfo->hashBegin && hashValue <= pVgInfo->hashEnd) { vgId = pVgInfo->vgId; downstreamTaskId = pVgInfo->taskId; diff --git a/source/libs/stream/src/streamTask.c b/source/libs/stream/src/streamTask.c index 7a7d9d15ad..a35e7679a1 100644 --- a/source/libs/stream/src/streamTask.c +++ b/source/libs/stream/src/streamTask.c @@ -70,7 +70,7 @@ int32_t tEncodeSStreamTask(SEncoder* pEncoder, const SStreamTask* pTask) { if (tEncodeSEpSet(pEncoder, &pTask->fixedEpDispatcher.epSet) < 0) return -1; } else if (pTask->dispatchType == TASK_DISPATCH__SHUFFLE) { if (tSerializeSUseDbRspImp(pEncoder, &pTask->shuffleDispatcher.dbInfo) < 0) return -1; - /*if (tEncodeI8(pEncoder, pTask->shuffleDispatcher.hashMethod) < 0) return -1;*/ + if (tEncodeCStr(pEncoder, pTask->shuffleDispatcher.stbFullName) < 0) return -1; } if (tEncodeI64(pEncoder, pTask->triggerParam) < 0) return -1; @@ -119,8 +119,8 @@ int32_t tDecodeSStreamTask(SDecoder* pDecoder, SStreamTask* pTask) { if (tDecodeI32(pDecoder, &pTask->fixedEpDispatcher.nodeId) < 0) return -1; if (tDecodeSEpSet(pDecoder, &pTask->fixedEpDispatcher.epSet) < 0) return -1; } else if (pTask->dispatchType == TASK_DISPATCH__SHUFFLE) { - /*if (tDecodeI8(pDecoder, &pTask->shuffleDispatcher.hashMethod) < 0) return -1;*/ if (tDeserializeSUseDbRspImp(pDecoder, &pTask->shuffleDispatcher.dbInfo) < 0) return -1; + if (tDecodeCStrTo(pDecoder, pTask->shuffleDispatcher.stbFullName) < 0) return -1; } if (tDecodeI64(pDecoder, &pTask->triggerParam) < 0) return -1; diff --git a/source/libs/sync/inc/syncInt.h b/source/libs/sync/inc/syncInt.h index b00c7cbda1..b910b59f83 100644 --- a/source/libs/sync/inc/syncInt.h +++ b/source/libs/sync/inc/syncInt.h @@ -171,7 +171,8 @@ void syncNodeClose(SSyncNode* pSyncNode); int32_t syncNodePropose(SSyncNode* pSyncNode, const SRpcMsg* pMsg, bool isWeak); // option -bool syncNodeSnapshotEnable(SSyncNode* pSyncNode); +bool syncNodeSnapshotEnable(SSyncNode* pSyncNode); +SyncIndex syncNodeGetSnapshotConfigIndex(SSyncNode* pSyncNode, SyncIndex snapshotLastApplyIndex); // ping -------------- int32_t syncNodePing(SSyncNode* pSyncNode, const SRaftId* destRaftId, SyncPing* pMsg); @@ -194,6 +195,7 @@ int32_t syncNodeSendMsgById(const SRaftId* destRaftId, SSyncNode* pSyncNode, SRp int32_t syncNodeSendMsgByInfo(const SNodeInfo* nodeInfo, SSyncNode* pSyncNode, SRpcMsg* pMsg); cJSON* syncNode2Json(const SSyncNode* pSyncNode); char* syncNode2Str(const SSyncNode* pSyncNode); +void syncNodeEventLog(const SSyncNode* pSyncNode, char* str); char* syncNode2SimpleStr(const SSyncNode* pSyncNode); bool syncNodeInConfig(SSyncNode* pSyncNode, const SSyncCfg* config); void syncNodeUpdateConfig(SSyncNode* pSyncNode, SSyncCfg* newConfig, SyncIndex lastConfigChangeIndex, bool* isDrop); diff --git a/source/libs/sync/inc/syncRaftCfg.h b/source/libs/sync/inc/syncRaftCfg.h index cd64402738..435ad98fb3 100644 --- a/source/libs/sync/inc/syncRaftCfg.h +++ b/source/libs/sync/inc/syncRaftCfg.h @@ -47,14 +47,16 @@ typedef struct SRaftCfg { SRaftCfg *raftCfgOpen(const char *path); int32_t raftCfgClose(SRaftCfg *pRaftCfg); int32_t raftCfgPersist(SRaftCfg *pRaftCfg); +int32_t raftCfgAddConfigIndex(SRaftCfg *pRaftCfg, SyncIndex configIndex); -cJSON * syncCfg2Json(SSyncCfg *pSyncCfg); -char * syncCfg2Str(SSyncCfg *pSyncCfg); +cJSON *syncCfg2Json(SSyncCfg *pSyncCfg); +char *syncCfg2Str(SSyncCfg *pSyncCfg); +char *syncCfg2SimpleStr(SSyncCfg *pSyncCfg); int32_t syncCfgFromJson(const cJSON *pRoot, SSyncCfg *pSyncCfg); int32_t syncCfgFromStr(const char *s, SSyncCfg *pSyncCfg); -cJSON * raftCfg2Json(SRaftCfg *pRaftCfg); -char * raftCfg2Str(SRaftCfg *pRaftCfg); +cJSON *raftCfg2Json(SRaftCfg *pRaftCfg); +char *raftCfg2Str(SRaftCfg *pRaftCfg); int32_t raftCfgFromJson(const cJSON *pRoot, SRaftCfg *pRaftCfg); int32_t raftCfgFromStr(const char *s, SRaftCfg *pRaftCfg); @@ -71,6 +73,7 @@ void syncCfgPrint(SSyncCfg *pCfg); void syncCfgPrint2(char *s, SSyncCfg *pCfg); void syncCfgLog(SSyncCfg *pCfg); void syncCfgLog2(char *s, SSyncCfg *pCfg); +void syncCfgLog3(char *s, SSyncCfg *pCfg); void raftCfgPrint(SRaftCfg *pCfg); void raftCfgPrint2(char *s, SRaftCfg *pCfg); diff --git a/source/libs/sync/inc/syncSnapshot.h b/source/libs/sync/inc/syncSnapshot.h index a6170a92e3..f6544dd79d 100644 --- a/source/libs/sync/inc/syncSnapshot.h +++ b/source/libs/sync/inc/syncSnapshot.h @@ -39,8 +39,8 @@ typedef struct SSyncSnapshotSender { bool start; int32_t seq; int32_t ack; - void * pReader; - void * pCurrentBlock; + void *pReader; + void *pCurrentBlock; int32_t blockLen; SSnapshot snapshot; SSyncCfg lastConfig; @@ -59,14 +59,15 @@ void snapshotSenderStart(SSyncSnapshotSender *pSender); void snapshotSenderStop(SSyncSnapshotSender *pSender); int32_t snapshotSend(SSyncSnapshotSender *pSender); int32_t snapshotReSend(SSyncSnapshotSender *pSender); -cJSON * snapshotSender2Json(SSyncSnapshotSender *pSender); -char * snapshotSender2Str(SSyncSnapshotSender *pSender); +cJSON *snapshotSender2Json(SSyncSnapshotSender *pSender); +char *snapshotSender2Str(SSyncSnapshotSender *pSender); +char *snapshotSender2SimpleStr(SSyncSnapshotSender *pSender, char *event); typedef struct SSyncSnapshotReceiver { bool start; int32_t ack; - void * pWriter; + void *pWriter; SyncTerm term; SyncTerm privateTerm; @@ -80,8 +81,9 @@ void snapshotReceiverDestroy(SSyncSnapshotReceiver *pReceiver) void snapshotReceiverStart(SSyncSnapshotReceiver *pReceiver, SyncTerm privateTerm, SRaftId fromId); bool snapshotReceiverIsStart(SSyncSnapshotReceiver *pReceiver); void snapshotReceiverStop(SSyncSnapshotReceiver *pReceiver, bool apply); -cJSON * snapshotReceiver2Json(SSyncSnapshotReceiver *pReceiver); -char * snapshotReceiver2Str(SSyncSnapshotReceiver *pReceiver); +cJSON *snapshotReceiver2Json(SSyncSnapshotReceiver *pReceiver); +char *snapshotReceiver2Str(SSyncSnapshotReceiver *pReceiver); +char *snapshotReceiver2SimpleStr(SSyncSnapshotReceiver *pReceiver, char *event); int32_t syncNodeOnSnapshotSendCb(SSyncNode *ths, SyncSnapshotSend *pMsg); int32_t syncNodeOnSnapshotRspCb(SSyncNode *ths, SyncSnapshotRsp *pMsg); diff --git a/source/libs/sync/src/syncAppendEntries.c b/source/libs/sync/src/syncAppendEntries.c index 7b2f79e24c..58e40668a2 100644 --- a/source/libs/sync/src/syncAppendEntries.c +++ b/source/libs/sync/src/syncAppendEntries.c @@ -208,8 +208,9 @@ int32_t syncNodeOnAppendEntriesCb(SSyncNode* ths, SyncAppendEntries* pMsg) { SRpcMsg rpcMsg; syncEntry2OriginalRpc(pRollBackEntry, &rpcMsg); - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.index = pRollBackEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pRollBackEntry->isWeak; cbMeta.code = 0; cbMeta.state = ths->state; @@ -234,8 +235,9 @@ int32_t syncNodeOnAppendEntriesCb(SSyncNode* ths, SyncAppendEntries* pMsg) { if (ths->pFsm != NULL) { // if (ths->pFsm->FpPreCommitCb != NULL && pAppendEntry->originalRpcType != TDMT_SYNC_NOOP) { if (ths->pFsm->FpPreCommitCb != NULL && syncUtilUserPreCommit(pAppendEntry->originalRpcType)) { - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.index = pAppendEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pAppendEntry->isWeak; cbMeta.code = 2; cbMeta.state = ths->state; @@ -266,8 +268,9 @@ int32_t syncNodeOnAppendEntriesCb(SSyncNode* ths, SyncAppendEntries* pMsg) { if (ths->pFsm != NULL) { // if (ths->pFsm->FpPreCommitCb != NULL && pAppendEntry->originalRpcType != TDMT_SYNC_NOOP) { if (ths->pFsm->FpPreCommitCb != NULL && syncUtilUserPreCommit(pAppendEntry->originalRpcType)) { - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.index = pAppendEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pAppendEntry->isWeak; cbMeta.code = 3; cbMeta.state = ths->state; @@ -326,358 +329,6 @@ int32_t syncNodeOnAppendEntriesCb(SSyncNode* ths, SyncAppendEntries* pMsg) { return ret; } -#if 0 -int32_t syncNodeOnAppendEntriesCb(SSyncNode* ths, SyncAppendEntries* pMsg) { - int32_t ret = 0; - - char logBuf[128] = {0}; - snprintf(logBuf, sizeof(logBuf), "==syncNodeOnAppendEntriesCb== term:%lu", ths->pRaftStore->currentTerm); - syncAppendEntriesLog2(logBuf, pMsg); - - if (pMsg->term > ths->pRaftStore->currentTerm) { - syncNodeUpdateTerm(ths, pMsg->term); - } - assert(pMsg->term <= ths->pRaftStore->currentTerm); - - // reset elect timer - if (pMsg->term == ths->pRaftStore->currentTerm) { - ths->leaderCache = pMsg->srcId; - syncNodeResetElectTimer(ths); - } - assert(pMsg->dataLen >= 0); - - SyncTerm localPreLogTerm = 0; - if (pMsg->prevLogIndex >= SYNC_INDEX_BEGIN && pMsg->prevLogIndex <= ths->pLogStore->getLastIndex(ths->pLogStore)) { - SSyncRaftEntry* pEntry = ths->pLogStore->getEntry(ths->pLogStore, pMsg->prevLogIndex); - assert(pEntry != NULL); - localPreLogTerm = pEntry->term; - syncEntryDestory(pEntry); - } - - bool logOK = - (pMsg->prevLogIndex == SYNC_INDEX_INVALID) || - ((pMsg->prevLogIndex >= SYNC_INDEX_BEGIN) && - (pMsg->prevLogIndex <= ths->pLogStore->getLastIndex(ths->pLogStore)) && (pMsg->prevLogTerm == localPreLogTerm)); - - // reject request - if ((pMsg->term < ths->pRaftStore->currentTerm) || - ((pMsg->term == ths->pRaftStore->currentTerm) && (ths->state == TAOS_SYNC_STATE_FOLLOWER) && !logOK)) { - sTrace( - "syncNodeOnAppendEntriesCb --> reject, pMsg->term:%lu, ths->pRaftStore->currentTerm:%lu, ths->state:%d, " - "logOK:%d", - pMsg->term, ths->pRaftStore->currentTerm, ths->state, logOK); - - SyncAppendEntriesReply* pReply = syncAppendEntriesReplyBuild(ths->vgId); - pReply->srcId = ths->myRaftId; - pReply->destId = pMsg->srcId; - pReply->term = ths->pRaftStore->currentTerm; - pReply->success = false; - pReply->matchIndex = SYNC_INDEX_INVALID; - - SRpcMsg rpcMsg; - syncAppendEntriesReply2RpcMsg(pReply, &rpcMsg); - syncNodeSendMsgById(&pReply->destId, ths, &rpcMsg); - syncAppendEntriesReplyDestroy(pReply); - - return ret; - } - - // return to follower state - if (pMsg->term == ths->pRaftStore->currentTerm && ths->state == TAOS_SYNC_STATE_CANDIDATE) { - sTrace( - "syncNodeOnAppendEntriesCb --> return to follower, pMsg->term:%lu, ths->pRaftStore->currentTerm:%lu, " - "ths->state:%d, logOK:%d", - pMsg->term, ths->pRaftStore->currentTerm, ths->state, logOK); - - syncNodeBecomeFollower(ths, "from candidate by append entries"); - - // ret or reply? - return ret; - } - - // accept request - if (pMsg->term == ths->pRaftStore->currentTerm && ths->state == TAOS_SYNC_STATE_FOLLOWER && logOK) { - // preIndex = -1, or has preIndex entry in local log - assert(pMsg->prevLogIndex <= ths->pLogStore->getLastIndex(ths->pLogStore)); - - // has extra entries (> preIndex) in local log - bool hasExtraEntries = pMsg->prevLogIndex < ths->pLogStore->getLastIndex(ths->pLogStore); - - // has entries in SyncAppendEntries msg - bool hasAppendEntries = pMsg->dataLen > 0; - - sTrace( - "syncNodeOnAppendEntriesCb --> accept, pMsg->term:%lu, ths->pRaftStore->currentTerm:%lu, ths->state:%d, " - "logOK:%d, hasExtraEntries:%d, hasAppendEntries:%d", - pMsg->term, ths->pRaftStore->currentTerm, ths->state, logOK, hasExtraEntries, hasAppendEntries); - - if (hasExtraEntries && hasAppendEntries) { - // not conflict by default - bool conflict = false; - - SyncIndex extraIndex = pMsg->prevLogIndex + 1; - SSyncRaftEntry* pExtraEntry = ths->pLogStore->getEntry(ths->pLogStore, extraIndex); - assert(pExtraEntry != NULL); - - SSyncRaftEntry* pAppendEntry = syncEntryDeserialize(pMsg->data, pMsg->dataLen); - assert(pAppendEntry != NULL); - - // log not match, conflict - assert(extraIndex == pAppendEntry->index); - if (pExtraEntry->term != pAppendEntry->term) { - conflict = true; - } - - if (conflict) { - // roll back - SyncIndex delBegin = ths->pLogStore->getLastIndex(ths->pLogStore); - SyncIndex delEnd = extraIndex; - - sTrace("syncNodeOnAppendEntriesCb --> conflict:%d, delBegin:%ld, delEnd:%ld", conflict, delBegin, delEnd); - - // notice! reverse roll back! - for (SyncIndex index = delEnd; index >= delBegin; --index) { - if (ths->pFsm->FpRollBackCb != NULL) { - SSyncRaftEntry* pRollBackEntry = ths->pLogStore->getEntry(ths->pLogStore, index); - assert(pRollBackEntry != NULL); - - // if (pRollBackEntry->msgType != TDMT_SYNC_NOOP) { - if (syncUtilUserRollback(pRollBackEntry->msgType)) { - SRpcMsg rpcMsg; - syncEntry2OriginalRpc(pRollBackEntry, &rpcMsg); - - SFsmCbMeta cbMeta; - cbMeta.index = pRollBackEntry->index; - cbMeta.isWeak = pRollBackEntry->isWeak; - cbMeta.code = 0; - cbMeta.state = ths->state; - cbMeta.seqNum = pRollBackEntry->seqNum; - ths->pFsm->FpRollBackCb(ths->pFsm, &rpcMsg, cbMeta); - rpcFreeCont(rpcMsg.pCont); - } - - syncEntryDestory(pRollBackEntry); - } - } - - // delete confict entries - ths->pLogStore->truncate(ths->pLogStore, extraIndex); - - // append new entries - ths->pLogStore->appendEntry(ths->pLogStore, pAppendEntry); - - // pre commit - SRpcMsg rpcMsg; - syncEntry2OriginalRpc(pAppendEntry, &rpcMsg); - if (ths->pFsm != NULL) { - // if (ths->pFsm->FpPreCommitCb != NULL && pAppendEntry->originalRpcType != TDMT_SYNC_NOOP) { - if (ths->pFsm->FpPreCommitCb != NULL && syncUtilUserPreCommit(pAppendEntry->originalRpcType)) { - SFsmCbMeta cbMeta; - cbMeta.index = pAppendEntry->index; - cbMeta.isWeak = pAppendEntry->isWeak; - cbMeta.code = 2; - cbMeta.state = ths->state; - cbMeta.seqNum = pAppendEntry->seqNum; - ths->pFsm->FpPreCommitCb(ths->pFsm, &rpcMsg, cbMeta); - } - } - rpcFreeCont(rpcMsg.pCont); - } - - // free memory - syncEntryDestory(pExtraEntry); - syncEntryDestory(pAppendEntry); - - } else if (hasExtraEntries && !hasAppendEntries) { - // do nothing - - } else if (!hasExtraEntries && hasAppendEntries) { - SSyncRaftEntry* pAppendEntry = syncEntryDeserialize(pMsg->data, pMsg->dataLen); - assert(pAppendEntry != NULL); - - // append new entries - ths->pLogStore->appendEntry(ths->pLogStore, pAppendEntry); - - // pre commit - SRpcMsg rpcMsg; - syncEntry2OriginalRpc(pAppendEntry, &rpcMsg); - if (ths->pFsm != NULL) { - // if (ths->pFsm->FpPreCommitCb != NULL && pAppendEntry->originalRpcType != TDMT_SYNC_NOOP) { - if (ths->pFsm->FpPreCommitCb != NULL && syncUtilUserPreCommit(pAppendEntry->originalRpcType)) { - SFsmCbMeta cbMeta; - cbMeta.index = pAppendEntry->index; - cbMeta.isWeak = pAppendEntry->isWeak; - cbMeta.code = 3; - cbMeta.state = ths->state; - cbMeta.seqNum = pAppendEntry->seqNum; - ths->pFsm->FpPreCommitCb(ths->pFsm, &rpcMsg, cbMeta); - } - } - rpcFreeCont(rpcMsg.pCont); - - // free memory - syncEntryDestory(pAppendEntry); - - } else if (!hasExtraEntries && !hasAppendEntries) { - // do nothing - - } else { - assert(0); - } - - SyncAppendEntriesReply* pReply = syncAppendEntriesReplyBuild(ths->vgId); - pReply->srcId = ths->myRaftId; - pReply->destId = pMsg->srcId; - pReply->term = ths->pRaftStore->currentTerm; - pReply->success = true; - - if (hasAppendEntries) { - pReply->matchIndex = pMsg->prevLogIndex + 1; - } else { - pReply->matchIndex = pMsg->prevLogIndex; - } - - SRpcMsg rpcMsg; - syncAppendEntriesReply2RpcMsg(pReply, &rpcMsg); - syncNodeSendMsgById(&pReply->destId, ths, &rpcMsg); - syncAppendEntriesReplyDestroy(pReply); - - // maybe update commit index from leader - if (pMsg->commitIndex > ths->commitIndex) { - // has commit entry in local - if (pMsg->commitIndex <= ths->pLogStore->getLastIndex(ths->pLogStore)) { - SyncIndex beginIndex = ths->commitIndex + 1; - SyncIndex endIndex = pMsg->commitIndex; - - // update commit index - ths->commitIndex = pMsg->commitIndex; - - // call back Wal - ths->pLogStore->updateCommitIndex(ths->pLogStore, ths->commitIndex); - - // execute fsm - if (ths->pFsm != NULL) { - for (SyncIndex i = beginIndex; i <= endIndex; ++i) { - if (i != SYNC_INDEX_INVALID) { - SSyncRaftEntry* pEntry = ths->pLogStore->getEntry(ths->pLogStore, i); - assert(pEntry != NULL); - - SRpcMsg rpcMsg; - syncEntry2OriginalRpc(pEntry, &rpcMsg); - - if (ths->pFsm->FpCommitCb != NULL && syncUtilUserCommit(pEntry->originalRpcType)) { - SFsmCbMeta cbMeta; - cbMeta.index = pEntry->index; - cbMeta.isWeak = pEntry->isWeak; - cbMeta.code = 0; - cbMeta.state = ths->state; - cbMeta.seqNum = pEntry->seqNum; - cbMeta.term = pEntry->term; - cbMeta.currentTerm = ths->pRaftStore->currentTerm; - cbMeta.flag = 0x11; - - SSnapshot snapshot; - ASSERT(ths->pFsm->FpGetSnapshot != NULL); - ths->pFsm->FpGetSnapshot(ths->pFsm, &snapshot); - - bool needExecute = true; - if (cbMeta.index <= snapshot.lastApplyIndex) { - needExecute = false; - } - - if (needExecute) { - ths->pFsm->FpCommitCb(ths->pFsm, &rpcMsg, cbMeta); - } - } - - // config change - if (pEntry->originalRpcType == TDMT_SYNC_CONFIG_CHANGE) { - SSyncCfg oldSyncCfg = ths->pRaftCfg->cfg; - - SSyncCfg newSyncCfg; - int32_t ret = syncCfgFromStr(rpcMsg.pCont, &newSyncCfg); - ASSERT(ret == 0); - - // update new config myIndex - bool hit = false; - for (int i = 0; i < newSyncCfg.replicaNum; ++i) { - if (strcmp(ths->myNodeInfo.nodeFqdn, (newSyncCfg.nodeInfo)[i].nodeFqdn) == 0 && - ths->myNodeInfo.nodePort == (newSyncCfg.nodeInfo)[i].nodePort) { - newSyncCfg.myIndex = i; - hit = true; - break; - } - } - - SReConfigCbMeta cbMeta = {0}; - bool isDrop; - - // I am in newConfig - if (hit) { - syncNodeUpdateConfig(ths, &newSyncCfg, pEntry->index, &isDrop); - - // change isStandBy to normal - if (!isDrop) { - if (ths->state == TAOS_SYNC_STATE_LEADER) { - syncNodeBecomeLeader(ths, "config change"); - } else { - syncNodeBecomeFollower(ths, "config change"); - } - } - - if (gRaftDetailLog) { - char* sOld = syncCfg2Str(&oldSyncCfg); - char* sNew = syncCfg2Str(&newSyncCfg); - sInfo("==config change== 0x11 old:%s new:%s isDrop:%d \n", sOld, sNew, isDrop); - taosMemoryFree(sOld); - taosMemoryFree(sNew); - } - } - - // always call FpReConfigCb - if (ths->pFsm->FpReConfigCb != NULL) { - cbMeta.code = 0; - cbMeta.currentTerm = ths->pRaftStore->currentTerm; - cbMeta.index = pEntry->index; - cbMeta.term = pEntry->term; - cbMeta.newCfg = newSyncCfg; - cbMeta.oldCfg = oldSyncCfg; - cbMeta.seqNum = pEntry->seqNum; - cbMeta.flag = 0x11; - cbMeta.isDrop = isDrop; - ths->pFsm->FpReConfigCb(ths->pFsm, &rpcMsg, cbMeta); - } - } - - // restore finish - if (pEntry->index == ths->pLogStore->getLastIndex(ths->pLogStore)) { - if (ths->restoreFinish == false) { - if (ths->pFsm->FpRestoreFinishCb != NULL) { - ths->pFsm->FpRestoreFinishCb(ths->pFsm); - } - ths->restoreFinish = true; - sInfo("==syncNodeOnAppendEntriesCb== restoreFinish set true %p vgId:%d", ths, ths->vgId); - - /* - tsem_post(&ths->restoreSem); - sInfo("==syncNodeOnAppendEntriesCb== RestoreFinish tsem_post %p", ths); - */ - } - } - - rpcFreeCont(rpcMsg.pCont); - syncEntryDestory(pEntry); - } - } - } - } - } - } - - return ret; -} -#endif - static int32_t syncNodeMakeLogSame(SSyncNode* ths, SyncAppendEntries* pMsg) { int32_t code; @@ -696,8 +347,9 @@ static int32_t syncNodeMakeLogSame(SSyncNode* ths, SyncAppendEntries* pMsg) { SRpcMsg rpcMsg; syncEntry2OriginalRpc(pRollBackEntry, &rpcMsg); - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.index = pRollBackEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pRollBackEntry->isWeak; cbMeta.code = 0; cbMeta.state = ths->state; @@ -713,8 +365,10 @@ static int32_t syncNodeMakeLogSame(SSyncNode* ths, SyncAppendEntries* pMsg) { // delete confict entries code = ths->pLogStore->syncLogTruncate(ths->pLogStore, delBegin); ASSERT(code == 0); - sDebug("vgId:%d sync event %s commitIndex:%ld currentTerm:%lu log truncate, from %ld to %ld", ths->vgId, - syncUtilState2String(ths->state), ths->commitIndex, ths->pRaftStore->currentTerm, delBegin, delEnd); + + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "log truncate, from %ld to %ld", delBegin, delEnd); + syncNodeEventLog(ths, eventLog); logStoreSimpleLog2("after syncNodeMakeLogSame", ths->pLogStore); return code; @@ -725,8 +379,9 @@ static int32_t syncNodePreCommit(SSyncNode* ths, SSyncRaftEntry* pEntry) { syncEntry2OriginalRpc(pEntry, &rpcMsg); if (ths->pFsm != NULL) { if (ths->pFsm->FpPreCommitCb != NULL && syncUtilUserPreCommit(pEntry->originalRpcType)) { - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.index = pEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pEntry->isWeak; cbMeta.code = 2; cbMeta.state = ths->state; @@ -1061,10 +716,10 @@ int32_t syncNodeOnAppendEntriesSnapshotCb(SSyncNode* ths, SyncAppendEntries* pMs SyncIndex commitEnd = snapshot.lastApplyIndex; ths->commitIndex = snapshot.lastApplyIndex; - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu commit by snapshot from index:%ld to index:%ld", - ths->vgId, syncUtilState2String(ths->state), ths->commitIndex, ths->pRaftStore->currentTerm, - commitBegin, commitEnd); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "commit by snapshot from index:%ld to index:%ld", commitBegin, + commitEnd); + syncNodeEventLog(ths, eventLog); } SyncIndex beginIndex = ths->commitIndex + 1; diff --git a/source/libs/sync/src/syncAppendEntriesReply.c b/source/libs/sync/src/syncAppendEntriesReply.c index f934f9a268..5e5e23b312 100644 --- a/source/libs/sync/src/syncAppendEntriesReply.c +++ b/source/libs/sync/src/syncAppendEntriesReply.c @@ -183,31 +183,9 @@ int32_t syncNodeOnAppendEntriesReplySnapshotCb(SSyncNode* ths, SyncAppendEntries pMsg->privateTerm < pSender->privateTerm) { snapshotSenderStart(pSender); - char host[128]; - uint16_t port; - syncUtilU642Addr(pSender->pSyncNode->replicasId[pSender->replicaIndex].addr, host, sizeof(host), &port); - - if (gRaftDetailLog) { - char* s = snapshotSender2Str(pSender); - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot send to %s:%d start sender first time, " - "lastApplyIndex:%ld " - "lastApplyTerm:%lu " - "lastConfigIndex:%ld privateTerm:%lu " - "sender:%s", - ths->vgId, syncUtilState2String(ths->state), ths->commitIndex, ths->pRaftStore->currentTerm, host, port, - pSender->snapshot.lastApplyIndex, pSender->snapshot.lastApplyTerm, pSender->snapshot.lastConfigIndex, - pSender->privateTerm, s); - taosMemoryFree(s); - } else { - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot send to %s:%d start sender first time, " - "lastApplyIndex:%ld " - "lastApplyTerm:%lu lastConfigIndex:%ld privateTerm:%lu", - ths->vgId, syncUtilState2String(ths->state), ths->commitIndex, ths->pRaftStore->currentTerm, host, port, - pSender->snapshot.lastApplyIndex, pSender->snapshot.lastApplyTerm, pSender->snapshot.lastConfigIndex, - pSender->privateTerm); - } + char* eventLog = snapshotSender2SimpleStr(pSender, "snapshot sender start"); + syncNodeEventLog(ths, eventLog); + taosMemoryFree(eventLog); } SyncIndex sentryIndex = pSender->snapshot.lastApplyIndex + 1; diff --git a/source/libs/sync/src/syncCommit.c b/source/libs/sync/src/syncCommit.c index d010728c78..738cbd3c2b 100644 --- a/source/libs/sync/src/syncCommit.c +++ b/source/libs/sync/src/syncCommit.c @@ -56,9 +56,10 @@ void syncMaybeAdvanceCommitIndex(SSyncNode* pSyncNode) { SyncIndex commitEnd = snapshot.lastApplyIndex; pSyncNode->commitIndex = snapshot.lastApplyIndex; - sDebug("vgId:%d sync event %s commitIndex:%ld currentTerm:%lu commit by snapshot from index:%ld to index:%ld", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, pSyncNode->commitIndex, snapshot.lastApplyIndex); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "commit by snapshot from index:%ld to index:%ld", pSyncNode->commitIndex, + snapshot.lastApplyIndex); + syncNodeEventLog(pSyncNode, eventLog); } // update commit index diff --git a/source/libs/sync/src/syncMain.c b/source/libs/sync/src/syncMain.c index 3e5dcfea42..7fb3ba4ac0 100644 --- a/source/libs/sync/src/syncMain.c +++ b/source/libs/sync/src/syncMain.c @@ -189,24 +189,6 @@ int32_t syncReconfigBuild(int64_t rid, const SSyncCfg* pNewCfg, SRpcMsg* pRpcMsg int32_t ret = 0; bool IamInNew = syncNodeInConfig(pSyncNode, pNewCfg); -#if 0 - for (int i = 0; i < pNewCfg->replicaNum; ++i) { - if (strcmp((pNewCfg->nodeInfo)[i].nodeFqdn, pSyncNode->myNodeInfo.nodeFqdn) == 0 && - (pNewCfg->nodeInfo)[i].nodePort == pSyncNode->myNodeInfo.nodePort) { - IamInNew = true; - } - - /* - SRaftId newId; - newId.addr = syncUtilAddr2U64((pNewCfg->nodeInfo)[i].nodeFqdn, (pNewCfg->nodeInfo)[i].nodePort); - newId.vgId = pSyncNode->vgId; - if (syncUtilSameId(&(pSyncNode->myRaftId), &newId)) { - IamInNew = true; - } - */ - } -#endif - if (!IamInNew) { taosReleaseRef(tsNodeRefId, pSyncNode->rid); terrno = TSDB_CODE_SYN_NOT_IN_NEW_CONFIG; @@ -235,27 +217,6 @@ int32_t syncReconfig(int64_t rid, const SSyncCfg* pNewCfg) { bool IamInNew = syncNodeInConfig(pSyncNode, pNewCfg); -#if 0 - for (int i = 0; i < pNewCfg->replicaNum; ++i) { - if (strcmp((pNewCfg->nodeInfo)[i].nodeFqdn, pSyncNode->myNodeInfo.nodeFqdn) == 0 && - (pNewCfg->nodeInfo)[i].nodePort == pSyncNode->myNodeInfo.nodePort) { - IamInNew = true; - } - - /* - // some problem in inet_addr - - SRaftId newId = EMPTY_RAFT_ID; - newId.addr = syncUtilAddr2U64((pNewCfg->nodeInfo)[i].nodeFqdn, (pNewCfg->nodeInfo)[i].nodePort); - newId.vgId = pSyncNode->vgId; - - if (syncUtilSameId(&(pSyncNode->myRaftId), &newId)) { - IamInNew = true; - } - */ - } -#endif - if (!IamInNew) { sError("sync reconfig error, not in new config"); taosReleaseRef(tsNodeRefId, pSyncNode->rid); @@ -414,7 +375,7 @@ int32_t syncGetSnapshotMeta(int64_t rid, struct SSnapshotMeta* sMeta) { assert(rid == pSyncNode->rid); sMeta->lastConfigIndex = pSyncNode->pRaftCfg->lastConfigIndex; - sTrace("sync get snapshot meta: lastConfigIndex:%ld", pSyncNode->pRaftCfg->lastConfigIndex); + sTrace("vgId:%d, get snapshot meta, lastConfigIndex:%" PRId64, pSyncNode->vgId, pSyncNode->pRaftCfg->lastConfigIndex); taosReleaseRef(tsNodeRefId, pSyncNode->rid); return 0; @@ -437,12 +398,28 @@ int32_t syncGetSnapshotMetaByIndex(int64_t rid, SyncIndex snapshotIndex, struct } } sMeta->lastConfigIndex = lastIndex; - sTrace("sync get snapshot meta by index:%ld lastConfigIndex:%ld", snapshotIndex, sMeta->lastConfigIndex); + sTrace("vgId:%d, get snapshot meta by index:%" PRId64 " lastConfigIndex:%" PRId64, pSyncNode->vgId, snapshotIndex, + sMeta->lastConfigIndex); taosReleaseRef(tsNodeRefId, pSyncNode->rid); return 0; } +SyncIndex syncNodeGetSnapshotConfigIndex(SSyncNode* pSyncNode, SyncIndex snapshotLastApplyIndex) { + ASSERT(pSyncNode->pRaftCfg->configIndexCount >= 1); + SyncIndex lastIndex = (pSyncNode->pRaftCfg->configIndexArr)[0]; + + for (int i = 0; i < pSyncNode->pRaftCfg->configIndexCount; ++i) { + if ((pSyncNode->pRaftCfg->configIndexArr)[i] > lastIndex && + (pSyncNode->pRaftCfg->configIndexArr)[i] <= snapshotLastApplyIndex) { + lastIndex = (pSyncNode->pRaftCfg->configIndexArr)[i]; + } + } + + sTrace("sync syncNodeGetSnapshotConfigIndex index:%ld lastConfigIndex:%ld", snapshotLastApplyIndex, lastIndex); + return lastIndex; +} + const char* syncGetMyRoleStr(int64_t rid) { const char* s = syncUtilState2String(syncGetMyRole(rid)); return s; @@ -598,9 +575,6 @@ int32_t syncPropose(int64_t rid, const SRpcMsg* pMsg, bool isWeak) { return -1; } assert(rid == pSyncNode->rid); - sDebug("vgId:%d sync event %s commitIndex:%ld currentTerm:%lu propose msgType:%s,%d", pSyncNode->vgId, - syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, pSyncNode->pRaftStore->currentTerm, - TMSG_INFO(pMsg->msgType), pMsg->msgType); ret = syncNodePropose(pSyncNode, pMsg, isWeak); taosReleaseRef(tsNodeRefId, pSyncNode->rid); @@ -609,9 +583,10 @@ int32_t syncPropose(int64_t rid, const SRpcMsg* pMsg, bool isWeak) { int32_t syncNodePropose(SSyncNode* pSyncNode, const SRpcMsg* pMsg, bool isWeak) { int32_t ret = 0; - sDebug("vgId:%d sync event %s commitIndex:%ld currentTerm:%lu propose msgType:%s,%d", pSyncNode->vgId, - syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, pSyncNode->pRaftStore->currentTerm, - TMSG_INFO(pMsg->msgType), pMsg->msgType); + + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "propose type:%s,%d", TMSG_INFO(pMsg->msgType), pMsg->msgType); + syncNodeEventLog(pSyncNode, eventLog); if (pSyncNode->state == TAOS_SYNC_STATE_LEADER) { SRespStub stub; @@ -854,11 +829,7 @@ SSyncNode* syncNodeOpen(const SSyncInfo* pOldSyncInfo) { // start raft // syncNodeBecomeFollower(pSyncNode); - // snapshot meta - // pSyncNode->sMeta.lastConfigIndex = -1; - - sDebug("vgId:%d sync event %s commitIndex:%ld currentTerm:%lu sync open", pSyncNode->vgId, - syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, pSyncNode->pRaftStore->currentTerm); + syncNodeEventLog(pSyncNode, "sync open"); return pSyncNode; } @@ -870,15 +841,6 @@ void syncNodeStart(SSyncNode* pSyncNode) { syncNodeBecomeLeader(pSyncNode, "one replica start"); // Raft 3.6.2 Committing entries from previous terms - - // use this now - syncNodeAppendNoop(pSyncNode); - syncMaybeAdvanceCommitIndex(pSyncNode); // maybe only one replica - - if (gRaftDetailLog) { - syncNodeLog2("==state change become leader immediately==", pSyncNode); - } - return; } @@ -905,8 +867,7 @@ void syncNodeStartStandBy(SSyncNode* pSyncNode) { } void syncNodeClose(SSyncNode* pSyncNode) { - sDebug("vgId:%d sync event %s commitIndex:%ld currentTerm:%lu sync close", pSyncNode->vgId, - syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, pSyncNode->pRaftStore->currentTerm); + syncNodeEventLog(pSyncNode, "sync close"); int32_t ret; assert(pSyncNode != NULL); @@ -1290,11 +1251,32 @@ char* syncNode2Str(const SSyncNode* pSyncNode) { return serialized; } +void syncNodeEventLog(const SSyncNode* pSyncNode, char* str) { + int32_t userStrLen = strlen(str); + if (userStrLen < 256) { + char logBuf[128 + 256]; + snprintf(logBuf, sizeof(logBuf), + "vgId:%d %s term:%lu commit:%ld standby:%d replica-num:%d lconfig:%ld sync event %s", pSyncNode->vgId, + syncUtilState2String(pSyncNode->state), pSyncNode->pRaftStore->currentTerm, pSyncNode->commitIndex, + pSyncNode->pRaftCfg->isStandBy, pSyncNode->replicaNum, pSyncNode->pRaftCfg->lastConfigIndex, str); + sDebug("%s", logBuf); + } else { + int len = 128 + userStrLen; + char* s = (char*)taosMemoryMalloc(len); + snprintf(s, len, "vgId:%d %s term:%lu commit:%ld standby:%d replica-num:%d lconfig:%ld sync event %s", + pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->pRaftStore->currentTerm, + pSyncNode->commitIndex, pSyncNode->pRaftCfg->isStandBy, pSyncNode->replicaNum, + pSyncNode->pRaftCfg->lastConfigIndex, str); + sDebug("%s", s); + taosMemoryFree(s); + } +} + char* syncNode2SimpleStr(const SSyncNode* pSyncNode) { int len = 256; char* s = (char*)taosMemoryMalloc(len); snprintf(s, len, - "syncNode: vgId:%d currentTerm:%lu, commitIndex:%ld, state:%d %s, isStandBy:%d, " + "syncNode: vgId:%d, currentTerm:%lu, commitIndex:%ld, state:%d %s, isStandBy:%d, " "electTimerLogicClock:%lu, " "electTimerLogicClockUser:%lu, " "electTimerMS:%d, replicaNum:%d", @@ -1345,12 +1327,10 @@ void syncNodeUpdateConfig(SSyncNode* pSyncNode, SSyncCfg* pNewConfig, SyncIndex SSyncSnapshotSender* oldSenders[TSDB_MAX_REPLICA]; for (int i = 0; i < TSDB_MAX_REPLICA; ++i) { oldSenders[i] = (pSyncNode->senders)[i]; - sDebug("vgId:%d sync event %s commitIndex:%ld currentTerm:%lu save senders %d, %p, privateTerm:%lu", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, i, oldSenders[i], oldSenders[i]->privateTerm); - if (gRaftDetailLog) { - ; - } + + char* eventLog = snapshotSender2SimpleStr(oldSenders[i], "snapshot sender save old"); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); } // init internal @@ -1399,12 +1379,14 @@ void syncNodeUpdateConfig(SSyncNode* pSyncNode, SSyncCfg* pNewConfig, SyncIndex char host[128]; uint16_t port; syncUtilU642Addr((pSyncNode->replicasId)[i].addr, host, sizeof(host), &port); - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu reset sender for %lu, newIndex:%d, %s:%d, %p, " - "privateTerm:%lu", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, (pSyncNode->replicasId)[i].addr, i, host, port, oldSenders[j], - oldSenders[j]->privateTerm); + + do { + char eventLog[256]; + snprintf(eventLog, sizeof(eventLog), "snapshot sender reset for %lu, newIndex:%d, %s:%d, %p", + (pSyncNode->replicasId)[i].addr, i, host, port, oldSenders[j]); + syncNodeEventLog(pSyncNode, eventLog); + } while (0); + (pSyncNode->senders)[i] = oldSenders[j]; oldSenders[j] = NULL; reset = true; @@ -1412,11 +1394,13 @@ void syncNodeUpdateConfig(SSyncNode* pSyncNode, SSyncCfg* pNewConfig, SyncIndex // reset replicaIndex int32_t oldreplicaIndex = (pSyncNode->senders)[i]->replicaIndex; (pSyncNode->senders)[i]->replicaIndex = i; - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu udpate replicaIndex from %d to %d, %s:%d, %p, " - "reset:%d", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, oldreplicaIndex, i, host, port, (pSyncNode->senders)[i], reset); + + do { + char eventLog[256]; + snprintf(eventLog, sizeof(eventLog), "snapshot sender udpate replicaIndex from %d to %d, %s:%d, %p, reset:%d", + oldreplicaIndex, i, host, port, (pSyncNode->senders)[i], reset); + syncNodeEventLog(pSyncNode, eventLog); + } while (0); } } } @@ -1425,10 +1409,10 @@ void syncNodeUpdateConfig(SSyncNode* pSyncNode, SSyncCfg* pNewConfig, SyncIndex for (int i = 0; i < TSDB_MAX_REPLICA; ++i) { if ((pSyncNode->senders)[i] == NULL) { (pSyncNode->senders)[i] = snapshotSenderCreate(pSyncNode, i); - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu create new sender %p replicaIndex:%d, privateTerm:%lu", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, (pSyncNode->senders)[i], i, (pSyncNode->senders)[i]->privateTerm); + + char* eventLog = snapshotSender2SimpleStr((pSyncNode->senders)[i], "snapshot sender create new"); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); } } @@ -1436,9 +1420,13 @@ void syncNodeUpdateConfig(SSyncNode* pSyncNode, SSyncCfg* pNewConfig, SyncIndex for (int i = 0; i < TSDB_MAX_REPLICA; ++i) { if (oldSenders[i] != NULL) { snapshotSenderDestroy(oldSenders[i]); - sDebug("vgId:%d sync event %s commitIndex:%ld currentTerm:%lu delete old sender %p replicaIndex:%d", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, oldSenders[i], i); + + do { + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "snapshot sender delete old %p replica-index:%d", oldSenders[i], i); + syncNodeEventLog(pSyncNode, eventLog); + } while (0); + oldSenders[i] = NULL; } } @@ -1446,24 +1434,6 @@ void syncNodeUpdateConfig(SSyncNode* pSyncNode, SSyncCfg* pNewConfig, SyncIndex bool IamInOld = syncNodeInConfig(pSyncNode, &oldConfig); bool IamInNew = syncNodeInConfig(pSyncNode, pNewConfig); -#if 0 - for (int i = 0; i < oldConfig.replicaNum; ++i) { - if (strcmp((oldConfig.nodeInfo)[i].nodeFqdn, pSyncNode->myNodeInfo.nodeFqdn) == 0 && - (oldConfig.nodeInfo)[i].nodePort == pSyncNode->myNodeInfo.nodePort) { - IamInOld = true; - break; - } - } - - for (int i = 0; i < newConfig->replicaNum; ++i) { - if (strcmp((newConfig->nodeInfo)[i].nodeFqdn, pSyncNode->myNodeInfo.nodeFqdn) == 0 && - (newConfig->nodeInfo)[i].nodePort == pSyncNode->myNodeInfo.nodePort) { - IamInNew = true; - break; - } - } -#endif - *isDrop = true; if (IamInOld && !IamInNew) { *isDrop = true; @@ -1508,13 +1478,6 @@ void syncNodeUpdateTerm(SSyncNode* pSyncNode, SyncTerm term) { } void syncNodeBecomeFollower(SSyncNode* pSyncNode, const char* debugStr) { - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu become follower, isStandBy:%d, replicaNum:%d, " - "restoreFinish:%d, %s", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, pSyncNode->pRaftCfg->isStandBy, pSyncNode->replicaNum, - pSyncNode->restoreFinish, debugStr); - // maybe clear leader cache if (pSyncNode->state == TAOS_SYNC_STATE_LEADER) { pSyncNode->leaderCache = EMPTY_RAFT_ID; @@ -1526,6 +1489,21 @@ void syncNodeBecomeFollower(SSyncNode* pSyncNode, const char* debugStr) { // reset elect timer syncNodeResetElectTimer(pSyncNode); + + // trace log + do { + int32_t debugStrLen = strlen(debugStr); + if (debugStrLen < 256) { + char eventLog[256 + 64]; + snprintf(eventLog, sizeof(eventLog), "become follower %s", debugStr); + syncNodeEventLog(pSyncNode, eventLog); + } else { + char* eventLog = taosMemoryMalloc(debugStrLen + 64); + snprintf(eventLog, debugStrLen, "become follower %s", debugStr); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); + } + } while (0); } // TLA+ Spec @@ -1550,13 +1528,6 @@ void syncNodeBecomeLeader(SSyncNode* pSyncNode, const char* debugStr) { // reset restoreFinish pSyncNode->restoreFinish = false; - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu become leader, isStandBy:%d, replicaNum:%d, " - "restoreFinish:%d, %s", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, pSyncNode->pRaftCfg->isStandBy, pSyncNode->replicaNum, - pSyncNode->restoreFinish, debugStr); - // state change pSyncNode->state = TAOS_SYNC_STATE_LEADER; @@ -1602,6 +1573,25 @@ void syncNodeBecomeLeader(SSyncNode* pSyncNode, const char* debugStr) { // start heartbeat timer syncNodeStartHeartbeatTimer(pSyncNode); + + // append noop + syncNodeAppendNoop(pSyncNode); + syncMaybeAdvanceCommitIndex(pSyncNode); // maybe only one replica + + // trace log + do { + int32_t debugStrLen = strlen(debugStr); + if (debugStrLen < 256) { + char eventLog[256 + 64]; + snprintf(eventLog, sizeof(eventLog), "become leader %s", debugStr); + syncNodeEventLog(pSyncNode, eventLog); + } else { + char* eventLog = taosMemoryMalloc(debugStrLen + 64); + snprintf(eventLog, debugStrLen, "become leader %s", debugStr); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); + } + } while (0); } void syncNodeCandidate2Leader(SSyncNode* pSyncNode) { @@ -1613,10 +1603,6 @@ void syncNodeCandidate2Leader(SSyncNode* pSyncNode) { // Raft 3.6.2 Committing entries from previous terms - // use this now - syncNodeAppendNoop(pSyncNode); - syncMaybeAdvanceCommitIndex(pSyncNode); // maybe only one replica - // do not use this // syncNodeEqNoop(pSyncNode); } @@ -1857,7 +1843,7 @@ static void syncNodeEqPingTimer(void* param, void* tmrId) { if (pSyncNode->FpEqMsg != NULL) { int32_t code = pSyncNode->FpEqMsg(pSyncNode->msgcb, &rpcMsg); if (code != 0) { - sError("vgId:%d sync enqueue ping msg error, code:%d", pSyncNode->vgId, code); + sError("vgId:%d, sync enqueue ping msg error, code:%d", pSyncNode->vgId, code); rpcFreeCont(rpcMsg.pCont); syncTimeoutDestroy(pSyncMsg); return; @@ -1891,7 +1877,7 @@ static void syncNodeEqElectTimer(void* param, void* tmrId) { if (pSyncNode->FpEqMsg != NULL) { int32_t code = pSyncNode->FpEqMsg(pSyncNode->msgcb, &rpcMsg); if (code != 0) { - sError("vgId:%d sync enqueue elect msg error, code:%d", pSyncNode->vgId, code); + sError("vgId:%d, sync enqueue elect msg error, code:%d", pSyncNode->vgId, code); rpcFreeCont(rpcMsg.pCont); syncTimeoutDestroy(pSyncMsg); return; @@ -1929,7 +1915,7 @@ static void syncNodeEqHeartbeatTimer(void* param, void* tmrId) { if (pSyncNode->FpEqMsg != NULL) { int32_t code = pSyncNode->FpEqMsg(pSyncNode->msgcb, &rpcMsg); if (code != 0) { - sError("vgId:%d sync enqueue timer msg error, code:%d", pSyncNode->vgId, code); + sError("vgId:%d, sync enqueue timer msg error, code:%d", pSyncNode->vgId, code); rpcFreeCont(rpcMsg.pCont); syncTimeoutDestroy(pSyncMsg); return; @@ -2067,8 +2053,9 @@ int32_t syncNodeOnClientRequestCb(SSyncNode* ths, SyncClientRequest* pMsg) { if (ths->pFsm != NULL) { // if (ths->pFsm->FpPreCommitCb != NULL && pEntry->originalRpcType != TDMT_SYNC_NOOP) { if (ths->pFsm->FpPreCommitCb != NULL && syncUtilUserPreCommit(pEntry->originalRpcType)) { - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.index = pEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pEntry->isWeak; cbMeta.code = 0; cbMeta.state = ths->state; @@ -2089,8 +2076,9 @@ int32_t syncNodeOnClientRequestCb(SSyncNode* ths, SyncClientRequest* pMsg) { if (ths->pFsm != NULL) { // if (ths->pFsm->FpPreCommitCb != NULL && pEntry->originalRpcType != TDMT_SYNC_NOOP) { if (ths->pFsm->FpPreCommitCb != NULL && syncUtilUserPreCommit(pEntry->originalRpcType)) { - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.index = pEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pEntry->isWeak; cbMeta.code = 1; cbMeta.state = ths->state; @@ -2129,36 +2117,33 @@ const char* syncStr(ESyncState state) { static int32_t syncDoLeaderTransfer(SSyncNode* ths, SRpcMsg* pRpcMsg, SSyncRaftEntry* pEntry) { SyncLeaderTransfer* pSyncLeaderTransfer = syncLeaderTransferFromRpcMsg2(pRpcMsg); - sDebug("vgId:%d sync event %s commitIndex:%ld currentTerm:%lu begin leader transfer", ths->vgId, - syncUtilState2String(ths->state), ths->commitIndex, ths->pRaftStore->currentTerm); + syncNodeEventLog(ths, "begin leader transfer"); - if (strcmp(pSyncLeaderTransfer->newNodeInfo.nodeFqdn, ths->myNodeInfo.nodeFqdn) == 0 && - pSyncLeaderTransfer->newNodeInfo.nodePort == ths->myNodeInfo.nodePort) { - sDebug("vgId:%d sync event %s commitIndex:%ld currentTerm:%lu maybe leader transfer to %s:%d %lu", ths->vgId, - syncUtilState2String(ths->state), ths->commitIndex, ths->pRaftStore->currentTerm, - pSyncLeaderTransfer->newNodeInfo.nodeFqdn, pSyncLeaderTransfer->newNodeInfo.nodePort, - pSyncLeaderTransfer->newLeaderId.addr); + bool sameId = syncUtilSameId(&(pSyncLeaderTransfer->newLeaderId), &(ths->myRaftId)); + bool sameNodeInfo = strcmp(pSyncLeaderTransfer->newNodeInfo.nodeFqdn, ths->myNodeInfo.nodeFqdn) == 0 && + pSyncLeaderTransfer->newNodeInfo.nodePort == ths->myNodeInfo.nodePort; + bool same = sameId || sameNodeInfo; + if (same) { // reset elect timer now! int32_t electMS = 1; int32_t ret = syncNodeRestartElectTimer(ths, electMS); ASSERT(ret == 0); + + char eventLog[256]; + snprintf(eventLog, sizeof(eventLog), "maybe leader transfer to %s:%d %lu", + pSyncLeaderTransfer->newNodeInfo.nodeFqdn, pSyncLeaderTransfer->newNodeInfo.nodePort, + pSyncLeaderTransfer->newLeaderId.addr); + syncNodeEventLog(ths, eventLog); } - /* - if (syncUtilSameId(&(pSyncLeaderTransfer->newLeaderId), &(ths->myRaftId))) { - // reset elect timer now! - int32_t electMS = 1; - int32_t ret = syncNodeRestartElectTimer(ths, electMS); - ASSERT(ret == 0); - } - */ if (ths->pFsm->FpLeaderTransferCb != NULL) { - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.code = 0; cbMeta.currentTerm = ths->pRaftStore->currentTerm; cbMeta.flag = 0; cbMeta.index = pEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pEntry->isWeak; cbMeta.seqNum = pEntry->seqNum; cbMeta.state = ths->state; @@ -2216,12 +2201,10 @@ static int32_t syncNodeConfigChange(SSyncNode* ths, SRpcMsg* pRpcMsg, SSyncRaftE // change isStandBy to normal if (!isDrop) { char tmpbuf[512]; - char* oldStr = syncCfg2Str(&oldSyncCfg); - char* newStr = syncCfg2Str(&newSyncCfg); - syncUtilJson2Line(oldStr); - syncUtilJson2Line(newStr); + char* oldStr = syncCfg2SimpleStr(&oldSyncCfg); + char* newStr = syncCfg2SimpleStr(&newSyncCfg); snprintf(tmpbuf, sizeof(tmpbuf), "config change from %d to %d, index:%ld, %s --> %s", oldSyncCfg.replicaNum, - newSyncCfg.replicaNum, pEntry->index, oldStr, newStr); + newSyncCfg.replicaNum, pEntry->index, oldStr, newStr); taosMemoryFree(oldStr); taosMemoryFree(newStr); @@ -2233,12 +2216,10 @@ static int32_t syncNodeConfigChange(SSyncNode* ths, SRpcMsg* pRpcMsg, SSyncRaftE } } else { char tmpbuf[512]; - char* oldStr = syncCfg2Str(&oldSyncCfg); - char* newStr = syncCfg2Str(&newSyncCfg); - syncUtilJson2Line(oldStr); - syncUtilJson2Line(newStr); + char* oldStr = syncCfg2SimpleStr(&oldSyncCfg); + char* newStr = syncCfg2SimpleStr(&newSyncCfg); snprintf(tmpbuf, sizeof(tmpbuf), "config change2 from %d to %d, index:%ld, %s --> %s", oldSyncCfg.replicaNum, - newSyncCfg.replicaNum, pEntry->index, oldStr, newStr); + newSyncCfg.replicaNum, pEntry->index, oldStr, newStr); taosMemoryFree(oldStr); taosMemoryFree(newStr); @@ -2260,6 +2241,7 @@ static int32_t syncNodeConfigChange(SSyncNode* ths, SRpcMsg* pRpcMsg, SSyncRaftE cbMeta.code = 0; cbMeta.currentTerm = ths->pRaftStore->currentTerm; cbMeta.index = pEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, pEntry->index); cbMeta.term = pEntry->term; cbMeta.newCfg = newSyncCfg; cbMeta.oldCfg = oldSyncCfg; @@ -2275,10 +2257,10 @@ static int32_t syncNodeConfigChange(SSyncNode* ths, SRpcMsg* pRpcMsg, SSyncRaftE int32_t syncNodeCommit(SSyncNode* ths, SyncIndex beginIndex, SyncIndex endIndex, uint64_t flag) { int32_t code = 0; ESyncState state = flag; - sDebug("vgId:%d sync event %s commitIndex:%ld currentTerm:%lu commit by wal from index:%" PRId64 " to index:%" PRId64 - ", %s", - ths->vgId, syncUtilState2String(ths->state), ths->commitIndex, ths->pRaftStore->currentTerm, beginIndex, - endIndex, syncUtilState2String(state)); + + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "commit by wal from index:%ld to index:%ld", beginIndex, endIndex); + syncNodeEventLog(ths, eventLog); // execute fsm if (ths->pFsm != NULL) { @@ -2294,8 +2276,9 @@ int32_t syncNodeCommit(SSyncNode* ths, SyncIndex beginIndex, SyncIndex endIndex, // user commit if (ths->pFsm->FpCommitCb != NULL && syncUtilUserCommit(pEntry->originalRpcType)) { - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.index = pEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pEntry->isWeak; cbMeta.code = 0; cbMeta.state = ths->state; @@ -2309,6 +2292,8 @@ int32_t syncNodeCommit(SSyncNode* ths, SyncIndex beginIndex, SyncIndex endIndex, // config change if (pEntry->originalRpcType == TDMT_SYNC_CONFIG_CHANGE) { + raftCfgAddConfigIndex(ths->pRaftCfg, pEntry->index); + raftCfgPersist(ths->pRaftCfg); code = syncNodeConfigChange(ths, &rpcMsg, pEntry); ASSERT(code == 0); } @@ -2327,9 +2312,10 @@ int32_t syncNodeCommit(SSyncNode* ths, SyncIndex beginIndex, SyncIndex endIndex, ths->pFsm->FpRestoreFinishCb(ths->pFsm); } ths->restoreFinish = true; - sDebug("vgId:%d sync event %s commitIndex:%ld currentTerm:%lu restore finish, %s, index:%ld", ths->vgId, - syncUtilState2String(ths->state), ths->commitIndex, ths->pRaftStore->currentTerm, - syncUtilState2String(ths->state), pEntry->index); + + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "restore finish, index:%ld", pEntry->index); + syncNodeEventLog(ths, eventLog); } } diff --git a/source/libs/sync/src/syncRaftCfg.c b/source/libs/sync/src/syncRaftCfg.c index 2d51f1f6f0..a8242d06e3 100644 --- a/source/libs/sync/src/syncRaftCfg.c +++ b/source/libs/sync/src/syncRaftCfg.c @@ -66,6 +66,13 @@ int32_t raftCfgPersist(SRaftCfg *pRaftCfg) { return 0; } +int32_t raftCfgAddConfigIndex(SRaftCfg *pRaftCfg, SyncIndex configIndex) { + ASSERT(pRaftCfg->configIndexCount <= MAX_CONFIG_INDEX_COUNT); + (pRaftCfg->configIndexArr)[pRaftCfg->configIndexCount] = configIndex; + ++(pRaftCfg->configIndexCount); + return 0; +} + cJSON *syncCfg2Json(SSyncCfg *pSyncCfg) { char u64buf[128] = {0}; cJSON *pRoot = cJSON_CreateObject(); @@ -94,6 +101,29 @@ char *syncCfg2Str(SSyncCfg *pSyncCfg) { return serialized; } +char *syncCfg2SimpleStr(SSyncCfg *pSyncCfg) { + int32_t len = 512; + char *s = taosMemoryMalloc(len); + memset(s, 0, len); + + snprintf(s, len, "{replica-num:%d, my-index:%d, ", pSyncCfg->replicaNum, pSyncCfg->myIndex); + char *p = s + strlen(s); + for (int i = 0; i < pSyncCfg->replicaNum; ++i) { + /* + if (p + 128 + 32 > s + len) { + break; + } + */ + char buf[128 + 32]; + snprintf(buf, sizeof(buf), "%s:%d, ", pSyncCfg->nodeInfo[i].nodeFqdn, pSyncCfg->nodeInfo[i].nodePort); + strncpy(p, buf, sizeof(buf)); + p = s + strlen(s); + } + strcpy(p - 2, "}"); + + return s; +} + int32_t syncCfgFromJson(const cJSON *pRoot, SSyncCfg *pSyncCfg) { memset(pSyncCfg, 0, sizeof(SSyncCfg)); // cJSON *pJson = cJSON_GetObjectItem(pRoot, "SSyncCfg"); @@ -277,6 +307,12 @@ void syncCfgLog2(char *s, SSyncCfg *pCfg) { taosMemoryFree(serialized); } +void syncCfgLog3(char *s, SSyncCfg *pCfg) { + char *serialized = syncCfg2SimpleStr(pCfg); + sTrace("syncCfgLog3 | len:%lu | %s | %s", strlen(serialized), s, serialized); + taosMemoryFree(serialized); +} + void raftCfgPrint(SRaftCfg *pCfg) { char *serialized = raftCfg2Str(pCfg); printf("raftCfgPrint | len:%lu | %s \n", strlen(serialized), serialized); diff --git a/source/libs/sync/src/syncRaftLog.c b/source/libs/sync/src/syncRaftLog.c index 87dfef5fdd..79d9b329c1 100644 --- a/source/libs/sync/src/syncRaftLog.c +++ b/source/libs/sync/src/syncRaftLog.c @@ -163,12 +163,10 @@ static int32_t raftLogAppendEntry(struct SSyncLogStore* pLogStore, SSyncRaftEntr walFsync(pWal, true); - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu write index:%ld, isStandBy:%d, msgType:%s,%d, " - "originalRpcType:%s,%d", - pData->pSyncNode->vgId, syncUtilState2String(pData->pSyncNode->state), pData->pSyncNode->commitIndex, - pData->pSyncNode->pRaftStore->currentTerm, pEntry->index, pData->pSyncNode->pRaftCfg->isStandBy, - TMSG_INFO(pEntry->msgType), pEntry->msgType, TMSG_INFO(pEntry->originalRpcType), pEntry->originalRpcType); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "write index:%ld, type:%s,%d, type2:%s,%d", pEntry->index, + TMSG_INFO(pEntry->msgType), pEntry->msgType, TMSG_INFO(pEntry->originalRpcType), pEntry->originalRpcType); + syncNodeEventLog(pData->pSyncNode, eventLog); return code; } @@ -320,16 +318,13 @@ int32_t logStoreAppendEntry(SSyncLogStore* pLogStore, SSyncRaftEntry* pEntry) { linuxErrMsg); ASSERT(0); } - // assert(code == 0); walFsync(pWal, true); - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu old write index:%ld, isStandBy:%d, msgType:%s,%d, " - "originalRpcType:%s,%d", - pData->pSyncNode->vgId, syncUtilState2String(pData->pSyncNode->state), pData->pSyncNode->commitIndex, - pData->pSyncNode->pRaftStore->currentTerm, pEntry->index, pData->pSyncNode->pRaftCfg->isStandBy, - TMSG_INFO(pEntry->msgType), pEntry->msgType, TMSG_INFO(pEntry->originalRpcType), pEntry->originalRpcType); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "old write index:%ld, type:%s,%d, type2:%s,%d", pEntry->index, + TMSG_INFO(pEntry->msgType), pEntry->msgType, TMSG_INFO(pEntry->originalRpcType), pEntry->originalRpcType); + syncNodeEventLog(pData->pSyncNode, eventLog); return code; } diff --git a/source/libs/sync/src/syncRespMgr.c b/source/libs/sync/src/syncRespMgr.c index 551c2611b0..354575d29e 100644 --- a/source/libs/sync/src/syncRespMgr.c +++ b/source/libs/sync/src/syncRespMgr.c @@ -46,11 +46,11 @@ int64_t syncRespMgrAdd(SSyncRespMgr *pObj, SRespStub *pStub) { taosHashPut(pObj->pRespHash, &keyCode, sizeof(keyCode), pStub, sizeof(SRespStub)); SSyncNode *pSyncNode = pObj->data; - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu resp mgr add, msgType:%s,%d seq:%lu handle:%p ahandle:%p", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, TMSG_INFO(pStub->rpcMsg.msgType), pStub->rpcMsg.msgType, keyCode, - pStub->rpcMsg.info.handle, pStub->rpcMsg.info.ahandle); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "resp mgr add, type:%s,%d, seq:%lu, handle:%p, ahandle:%p", + TMSG_INFO(pStub->rpcMsg.msgType), pStub->rpcMsg.msgType, keyCode, pStub->rpcMsg.info.handle, + pStub->rpcMsg.info.ahandle); + syncNodeEventLog(pSyncNode, eventLog); taosThreadMutexUnlock(&(pObj->mutex)); return keyCode; @@ -73,12 +73,11 @@ int32_t syncRespMgrGet(SSyncRespMgr *pObj, uint64_t index, SRespStub *pStub) { memcpy(pStub, pTmp, sizeof(SRespStub)); SSyncNode *pSyncNode = pObj->data; - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu resp mgr get, msgType:%s,%d seq:%lu handle:%p " - "ahandle:%p", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, TMSG_INFO(pStub->rpcMsg.msgType), pStub->rpcMsg.msgType, index, - pStub->rpcMsg.info.handle, pStub->rpcMsg.info.ahandle); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "resp mgr get, type:%s,%d, seq:%lu, handle:%p, ahandle:%p", + TMSG_INFO(pStub->rpcMsg.msgType), pStub->rpcMsg.msgType, index, pStub->rpcMsg.info.handle, + pStub->rpcMsg.info.ahandle); + syncNodeEventLog(pSyncNode, eventLog); taosThreadMutexUnlock(&(pObj->mutex)); return 1; // get one object @@ -95,12 +94,11 @@ int32_t syncRespMgrGetAndDel(SSyncRespMgr *pObj, uint64_t index, SRespStub *pStu memcpy(pStub, pTmp, sizeof(SRespStub)); SSyncNode *pSyncNode = pObj->data; - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu resp mgr get and del, msgType:%s,%d seq:%lu handle:%p " - "ahandle:%p", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, TMSG_INFO(pStub->rpcMsg.msgType), pStub->rpcMsg.msgType, index, - pStub->rpcMsg.info.handle, pStub->rpcMsg.info.ahandle); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "resp mgr get-and-del, type:%s,%d, seq:%lu, handle:%p, ahandle:%p", + TMSG_INFO(pStub->rpcMsg.msgType), pStub->rpcMsg.msgType, index, pStub->rpcMsg.info.handle, + pStub->rpcMsg.info.ahandle); + syncNodeEventLog(pSyncNode, eventLog); taosHashRemove(pObj->pRespHash, &index, sizeof(index)); taosThreadMutexUnlock(&(pObj->mutex)); diff --git a/source/libs/sync/src/syncSnapshot.c b/source/libs/sync/src/syncSnapshot.c index 4973d36295..d1bfc39d1e 100644 --- a/source/libs/sync/src/syncSnapshot.c +++ b/source/libs/sync/src/syncSnapshot.c @@ -50,6 +50,7 @@ SSyncSnapshotSender *snapshotSenderCreate(SSyncNode *pSyncNode, int32_t replicaI } else { sError("snapshotSenderCreate cannot create sender"); } + return pSender; } @@ -84,6 +85,10 @@ void snapshotSenderStart(SSyncSnapshotSender *pSender) { // get current snapshot info pSender->pSyncNode->pFsm->FpGetSnapshot(pSender->pSyncNode->pFsm, &(pSender->snapshot)); + + sTrace("snapshotSenderStart lastApplyIndex:%ld, lastApplyTerm:%lu, lastConfigIndex:%ld", + pSender->snapshot.lastApplyIndex, pSender->snapshot.lastApplyTerm, pSender->snapshot.lastConfigIndex); + if (pSender->snapshot.lastConfigIndex != SYNC_INDEX_INVALID) { /* SSyncRaftEntry *pEntry = NULL; @@ -134,90 +139,13 @@ void snapshotSenderStart(SSyncSnapshotSender *pSender) { syncSnapshotSend2RpcMsg(pMsg, &rpcMsg); syncNodeSendMsgById(&(pMsg->destId), pSender->pSyncNode, &rpcMsg); - char host[128]; - uint16_t port; - syncUtilU642Addr(pSender->pSyncNode->replicasId[pSender->replicaIndex].addr, host, sizeof(host), &port); - - if (gRaftDetailLog) { - char *msgStr = syncSnapshotSend2Str(pMsg); - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot send to %s:%d begin seq:%d ack:%d " - "lastApplyIndex:%ld " - "lastApplyTerm:%lu " - "lastConfigIndex:%ld privateTerm:%lu send " - "msg:%s", - pSender->pSyncNode->vgId, syncUtilState2String(pSender->pSyncNode->state), pSender->pSyncNode->commitIndex, - pSender->pSyncNode->pRaftStore->currentTerm, host, port, pSender->seq, pSender->ack, - pSender->snapshot.lastApplyIndex, pSender->snapshot.lastApplyTerm, pSender->snapshot.lastConfigIndex, - pSender->privateTerm, msgStr); - taosMemoryFree(msgStr); - } else { - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot send to %s:%d begin seq:%d ack:%d " - "lastApplyIndex:%ld " - "lastApplyTerm:%lu " - "lastConfigIndex:%ld privateTerm:%lu", - pSender->pSyncNode->vgId, syncUtilState2String(pSender->pSyncNode->state), pSender->pSyncNode->commitIndex, - pSender->pSyncNode->pRaftStore->currentTerm, host, port, pSender->seq, pSender->ack, - pSender->snapshot.lastApplyIndex, pSender->snapshot.lastApplyTerm, pSender->snapshot.lastConfigIndex, - pSender->privateTerm); - } + char *eventLog = snapshotSender2SimpleStr(pSender, "snapshot sender send"); + syncNodeEventLog(pSender->pSyncNode, eventLog); + taosMemoryFree(eventLog); syncSnapshotSendDestroy(pMsg); } -#if 0 -// when entry in snapshot, start sender -void snapshotSenderStart(SSyncSnapshotSender *pSender) { - if (!(pSender->start)) { - // start - snapshotSenderDoStart(pSender); - pSender->start = true; - } else { - // already start - ASSERT(pSender->pSyncNode->pRaftStore->currentTerm >= pSender->term); - - // if current term is higher, need start again - if (pSender->pSyncNode->pRaftStore->currentTerm > pSender->term) { - // force peer rollback - SyncSnapshotSend *pMsg = syncSnapshotSendBuild(0, pSender->pSyncNode->vgId); - pMsg->srcId = pSender->pSyncNode->myRaftId; - pMsg->destId = (pSender->pSyncNode->replicasId)[pSender->replicaIndex]; - pMsg->term = pSender->pSyncNode->pRaftStore->currentTerm; - pMsg->lastIndex = pSender->snapshot.lastApplyIndex; - pMsg->lastTerm = pSender->snapshot.lastApplyTerm; - pMsg->seq = SYNC_SNAPSHOT_SEQ_FORCE_CLOSE; - - SRpcMsg rpcMsg; - syncSnapshotSend2RpcMsg(pMsg, &rpcMsg); - syncNodeSendMsgById(&(pMsg->destId), pSender->pSyncNode, &rpcMsg); - - char *msgStr = syncSnapshotSend2Str(pMsg); - sTrace("snapshot send force close seq:%d ack:%d send msg:%s", pSender->seq, pSender->ack, msgStr); - taosMemoryFree(msgStr); - - syncSnapshotSendDestroy(pMsg); - - // close reader - int32_t ret = pSender->pSyncNode->pFsm->FpSnapshotStopRead(pSender->pSyncNode->pFsm, pSender->pReader); - ASSERT(ret == 0); - pSender->pReader = NULL; - - // start again - snapshotSenderDoStart(pSender); - pSender->start = true; - } else { - // current term, do nothing - ASSERT(pSender->pSyncNode->pRaftStore->currentTerm == pSender->term); - } - } - - char *s = snapshotSender2Str(pSender); - sInfo("snapshotSenderStart %s", s); - taosMemoryFree(s); -} -#endif - void snapshotSenderStop(SSyncSnapshotSender *pSender) { if (pSender->pReader != NULL) { int32_t ret = pSender->pSyncNode->pFsm->FpSnapshotStopRead(pSender->pSyncNode->pFsm, pSender->pReader); @@ -279,45 +207,15 @@ int32_t snapshotSend(SSyncSnapshotSender *pSender) { syncSnapshotSend2RpcMsg(pMsg, &rpcMsg); syncNodeSendMsgById(&(pMsg->destId), pSender->pSyncNode, &rpcMsg); - char host[128]; - uint16_t port; - syncUtilU642Addr(pSender->pSyncNode->replicasId[pSender->replicaIndex].addr, host, sizeof(host), &port); - if (pSender->seq == SYNC_SNAPSHOT_SEQ_END) { - if (gRaftDetailLog) { - char *msgStr = syncSnapshotSend2Str(pMsg); - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot send to %s:%d finish seq:%d ack:%d " - "lastApplyIndex:%ld " - "lastApplyTerm:%lu " - "lastConfigIndex:%ld privateTerm:%lu send " - "msg:%s", - pSender->pSyncNode->vgId, syncUtilState2String(pSender->pSyncNode->state), pSender->pSyncNode->commitIndex, - pSender->pSyncNode->pRaftStore->currentTerm, host, port, pSender->seq, pSender->ack, - pSender->snapshot.lastApplyIndex, pSender->snapshot.lastApplyTerm, pSender->snapshot.lastConfigIndex, - pSender->privateTerm, msgStr); - taosMemoryFree(msgStr); - } else { - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot send to %s:%d finish seq:%d ack:%d " - "lastApplyIndex:%ld " - "lastApplyTerm:%lu " - "lastConfigIndex:%ld privateTerm:%lu", - pSender->pSyncNode->vgId, syncUtilState2String(pSender->pSyncNode->state), pSender->pSyncNode->commitIndex, - pSender->pSyncNode->pRaftStore->currentTerm, host, port, pSender->seq, pSender->ack, - pSender->snapshot.lastApplyIndex, pSender->snapshot.lastApplyTerm, pSender->snapshot.lastConfigIndex, - pSender->privateTerm); - } + char *eventLog = snapshotSender2SimpleStr(pSender, "snapshot sender finish"); + syncNodeEventLog(pSender->pSyncNode, eventLog); + taosMemoryFree(eventLog); + } else { - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot send to %s:%d sending seq:%d ack:%d " - "lastApplyIndex:%ld " - "lastApplyTerm:%lu " - "lastConfigIndex:%ld privateTerm:%lu", - pSender->pSyncNode->vgId, syncUtilState2String(pSender->pSyncNode->state), pSender->pSyncNode->commitIndex, - pSender->pSyncNode->pRaftStore->currentTerm, host, port, pSender->seq, pSender->ack, - pSender->snapshot.lastApplyIndex, pSender->snapshot.lastApplyTerm, pSender->snapshot.lastConfigIndex, - pSender->privateTerm); + char *eventLog = snapshotSender2SimpleStr(pSender, "snapshot sender sending"); + syncNodeEventLog(pSender->pSyncNode, eventLog); + taosMemoryFree(eventLog); } syncSnapshotSendDestroy(pMsg); @@ -342,27 +240,9 @@ int32_t snapshotReSend(SSyncSnapshotSender *pSender) { syncSnapshotSend2RpcMsg(pMsg, &rpcMsg); syncNodeSendMsgById(&(pMsg->destId), pSender->pSyncNode, &rpcMsg); - char host[128]; - uint16_t port; - syncUtilU642Addr(pSender->pSyncNode->replicasId[pSender->replicaIndex].addr, host, sizeof(host), &port); - - if (gRaftDetailLog) { - char *msgStr = syncSnapshotSend2Str(pMsg); - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot send to %s:%d resend seq:%d ack:%d " - "privateTerm:%lu send " - "msg:%s", - pSender->pSyncNode->vgId, syncUtilState2String(pSender->pSyncNode->state), pSender->pSyncNode->commitIndex, - pSender->pSyncNode->pRaftStore->currentTerm, host, port, pSender->seq, pSender->ack, pSender->privateTerm, - msgStr); - taosMemoryFree(msgStr); - } else { - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot send to %s:%d resend seq:%d ack:%d " - "privateTerm:%lu", - pSender->pSyncNode->vgId, syncUtilState2String(pSender->pSyncNode->state), pSender->pSyncNode->commitIndex, - pSender->pSyncNode->pRaftStore->currentTerm, host, port, pSender->seq, pSender->ack, pSender->privateTerm); - } + char *eventLog = snapshotSender2SimpleStr(pSender, "snapshot sender resend"); + syncNodeEventLog(pSender->pSyncNode, eventLog); + taosMemoryFree(eventLog); syncSnapshotSendDestroy(pMsg); } @@ -421,11 +301,28 @@ cJSON *snapshotSender2Json(SSyncSnapshotSender *pSender) { char *snapshotSender2Str(SSyncSnapshotSender *pSender) { cJSON *pJson = snapshotSender2Json(pSender); - char * serialized = cJSON_Print(pJson); + char *serialized = cJSON_Print(pJson); cJSON_Delete(pJson); return serialized; } +char *snapshotSender2SimpleStr(SSyncSnapshotSender *pSender, char *event) { + int32_t len = 256; + char *s = taosMemoryMalloc(len); + + SRaftId destId = pSender->pSyncNode->replicasId[pSender->replicaIndex]; + char host[128]; + uint16_t port; + syncUtilU642Addr(destId.addr, host, sizeof(host), &port); + + snprintf(s, len, "%s %p laindex:%ld laterm:%lu lcindex:%ld seq:%d ack:%d finish:%d pterm:%lu replica-index:%d %s:%d", + event, pSender, pSender->snapshot.lastApplyIndex, pSender->snapshot.lastApplyTerm, + pSender->snapshot.lastConfigIndex, pSender->seq, pSender->ack, pSender->finish, pSender->privateTerm, + pSender->replicaIndex, host, port); + + return s; +} + // ------------------------------------- SSyncSnapshotReceiver *snapshotReceiverCreate(SSyncNode *pSyncNode, SRaftId fromId) { bool condition = (pSyncNode->pFsm->FpSnapshotStartWrite != NULL) && (pSyncNode->pFsm->FpSnapshotStopWrite != NULL) && @@ -542,7 +439,7 @@ cJSON *snapshotReceiver2Json(SSyncSnapshotReceiver *pReceiver) { cJSON_AddStringToObject(pFromId, "addr", u64buf); { uint64_t u64 = pReceiver->fromId.addr; - cJSON * pTmp = pFromId; + cJSON *pTmp = pFromId; char host[128] = {0}; uint16_t port; syncUtilU642Addr(u64, host, sizeof(host), &port); @@ -566,11 +463,26 @@ cJSON *snapshotReceiver2Json(SSyncSnapshotReceiver *pReceiver) { char *snapshotReceiver2Str(SSyncSnapshotReceiver *pReceiver) { cJSON *pJson = snapshotReceiver2Json(pReceiver); - char * serialized = cJSON_Print(pJson); + char *serialized = cJSON_Print(pJson); cJSON_Delete(pJson); return serialized; } +char *snapshotReceiver2SimpleStr(SSyncSnapshotReceiver *pReceiver, char *event) { + int32_t len = 256; + char *s = taosMemoryMalloc(len); + + SRaftId fromId = pReceiver->fromId; + char host[128]; + uint16_t port; + syncUtilU642Addr(fromId.addr, host, sizeof(host), &port); + + snprintf(s, len, "%s %p start:%d ack:%d term:%lu pterm:%lu %s:%d ", event, pReceiver, pReceiver->start, + pReceiver->ack, pReceiver->term, pReceiver->privateTerm, host, port); + + return s; +} + // receiver do something int32_t syncNodeOnSnapshotSendCb(SSyncNode *pSyncNode, SyncSnapshotSend *pMsg) { // get receiver @@ -587,31 +499,9 @@ int32_t syncNodeOnSnapshotSendCb(SSyncNode *pSyncNode, SyncSnapshotSend *pMsg) { pReceiver->ack = pMsg->seq; needRsp = true; - char host[128]; - uint16_t port; - syncUtilU642Addr(pMsg->srcId.addr, host, sizeof(host), &port); - - if (gRaftDetailLog) { - char *msgStr = syncSnapshotSend2Str(pMsg); - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot recv from %s:%d begin ack:%d, " - "lastIndex:%ld, " - "lastTerm:%lu, " - "lastConfigIndex:%ld, privateTerm:%lu, recv msg:%s", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, host, port, pReceiver->ack, pMsg->lastIndex, pMsg->lastTerm, - pMsg->lastConfigIndex, pReceiver->privateTerm, msgStr); - taosMemoryFree(msgStr); - } else { - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot recv from %s:%d begin ack:%d, " - "lastIndex:%ld, " - "lastTerm:%lu, " - "lastConfigIndex:%ld privateTerm:%lu", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, host, port, pReceiver->ack, pMsg->lastIndex, pMsg->lastTerm, - pMsg->lastConfigIndex, pReceiver->privateTerm); - } + char *eventLog = snapshotReceiver2SimpleStr(pReceiver, "snapshot receiver begin"); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); } else if (pMsg->seq == SYNC_SNAPSHOT_SEQ_END) { // end, finish FSM @@ -631,46 +521,29 @@ int32_t syncNodeOnSnapshotSendCb(SSyncNode *pSyncNode, SyncSnapshotSend *pMsg) { syncNodeUpdateNewConfigIndex(pSyncNode, &newSyncCfg); bool IamInNew = syncNodeInConfig(pSyncNode, &newSyncCfg); -#if 0 - // update new config myIndex - bool IamInNew = false; - SSyncCfg newSyncCfg = pMsg->lastConfig; - for (int i = 0; i < newSyncCfg.replicaNum; ++i) { - if (strcmp(pSyncNode->myNodeInfo.nodeFqdn, (newSyncCfg.nodeInfo)[i].nodeFqdn) == 0 && - pSyncNode->myNodeInfo.nodePort == (newSyncCfg.nodeInfo)[i].nodePort) { - newSyncCfg.myIndex = i; - IamInNew = true; - break; - } - } -#endif - - bool isDrop; + bool isDrop = false; if (IamInNew) { - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu update config by snapshot, lastIndex:%ld, " - "lastTerm:%lu, " - "lastConfigIndex:%ld ", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, pMsg->lastIndex, pMsg->lastTerm, pMsg->lastConfigIndex); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), + "update config by snapshot, lastIndex:%ld, lastTerm:%lu, lastConfigIndex:%ld", pMsg->lastIndex, + pMsg->lastTerm, pMsg->lastConfigIndex); + syncNodeEventLog(pSyncNode, eventLog); + syncNodeUpdateConfig(pSyncNode, &newSyncCfg, pMsg->lastConfigIndex, &isDrop); + } else { - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu do not update config by snapshot, I am not in " - "newCfg, " - "lastIndex:%ld, lastTerm:%lu, " - "lastConfigIndex:%ld ", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, pMsg->lastIndex, pMsg->lastTerm, pMsg->lastConfigIndex); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), + "do not update config by snapshot, not in new, lastIndex:%ld, lastTerm:%lu, lastConfigIndex:%ld", + pMsg->lastIndex, pMsg->lastTerm, pMsg->lastConfigIndex); + syncNodeEventLog(pSyncNode, eventLog); } // change isStandBy to normal if (!isDrop) { char tmpbuf[512]; - char *oldStr = syncCfg2Str(&oldSyncCfg); - char *newStr = syncCfg2Str(&newSyncCfg); - syncUtilJson2Line(oldStr); - syncUtilJson2Line(newStr); + char *oldStr = syncCfg2SimpleStr(&oldSyncCfg); + char *newStr = syncCfg2SimpleStr(&newSyncCfg); snprintf(tmpbuf, sizeof(tmpbuf), "config change3 from %d to %d, index:%ld, %s --> %s", oldSyncCfg.replicaNum, newSyncCfg.replicaNum, pMsg->lastConfigIndex, oldStr, newStr); taosMemoryFree(oldStr); @@ -687,88 +560,33 @@ int32_t syncNodeOnSnapshotSendCb(SSyncNode *pSyncNode, SyncSnapshotSend *pMsg) { SSnapshot snapshot; pSyncNode->pFsm->FpGetSnapshot(pSyncNode->pFsm, &snapshot); - char host[128]; - uint16_t port; - syncUtilU642Addr(pMsg->srcId.addr, host, sizeof(host), &port); - - if (gRaftDetailLog) { - char *logSimpleStr = logStoreSimple2Str(pSyncNode->pLogStore); - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot recv from %s:%d finish, update log begin " - "index:%ld, " - "snapshot.lastApplyIndex:%ld, " - "snapshot.lastApplyTerm:%lu, snapshot.lastConfigIndex:%ld, privateTerm:%lu, raft log:%s", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, host, port, pMsg->lastIndex + 1, snapshot.lastApplyIndex, - snapshot.lastApplyTerm, snapshot.lastConfigIndex, pReceiver->privateTerm, logSimpleStr); - taosMemoryFree(logSimpleStr); - } else { - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot recv from %s:%d finish, update log begin " - "index:%ld, " - "snapshot.lastApplyIndex:%ld, " - "snapshot.lastApplyTerm:%lu, snapshot.lastConfigIndex:%ld, privateTerm:%lu", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, host, port, pMsg->lastIndex + 1, snapshot.lastApplyIndex, - snapshot.lastApplyTerm, snapshot.lastConfigIndex, pReceiver->privateTerm); - } + do { + char *eventLog = snapshotReceiver2SimpleStr(pReceiver, "snapshot receiver finish"); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); + } while (0); pReceiver->pWriter = NULL; snapshotReceiverStop(pReceiver, true); pReceiver->ack = pMsg->seq; needRsp = true; - if (gRaftDetailLog) { - char *msgStr = syncSnapshotSend2Str(pMsg); - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot recv from %s:%d end ack:%d, " - "lastIndex:%ld, lastTerm:%lu, " - "lastConfigIndex:%ld, privateTerm:%lu, recv msg:%s", - pReceiver->pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pReceiver->pSyncNode->pRaftStore->currentTerm, host, port, pReceiver->ack, pMsg->lastIndex, - pMsg->lastTerm, pMsg->lastConfigIndex, pReceiver->privateTerm, msgStr); - taosMemoryFree(msgStr); - } else { - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot recv from %s:%d end ack:%d, " - "lastIndex:%ld, lastTerm:%lu, " - "lastConfigIndex:%ld, privateTerm:%lu", - pReceiver->pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pReceiver->pSyncNode->pRaftStore->currentTerm, host, port, pReceiver->ack, pMsg->lastIndex, - pMsg->lastTerm, pMsg->lastConfigIndex, pReceiver->privateTerm); - } + do { + char *eventLog = snapshotReceiver2SimpleStr(pReceiver, "snapshot receiver end"); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); + } while (0); } else if (pMsg->seq == SYNC_SNAPSHOT_SEQ_FORCE_CLOSE) { pSyncNode->pFsm->FpSnapshotStopWrite(pSyncNode->pFsm, pReceiver->pWriter, false); snapshotReceiverStop(pReceiver, false); needRsp = false; - char host[128]; - uint16_t port; - syncUtilU642Addr(pMsg->srcId.addr, host, sizeof(host), &port); - - if (gRaftDetailLog) { - char *msgStr = syncSnapshotSend2Str(pMsg); - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot recv from %s:%d force close ack:%d, " - "lastIndex:%ld, " - "lastTerm:%lu, " - "lastConfigIndex:%ld, privateTerm:%lu, recv " - "msg:%s", - pReceiver->pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pReceiver->pSyncNode->pRaftStore->currentTerm, host, port, pReceiver->ack, pMsg->lastIndex, - pMsg->lastTerm, pMsg->lastConfigIndex, pReceiver->privateTerm, msgStr); - taosMemoryFree(msgStr); - } else { - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot recv from %s:%d force close ack:%d, " - "lastIndex:%ld, " - "lastTerm:%lu, " - "lastConfigIndex:%ld, privateTerm:%lu", - pReceiver->pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pReceiver->pSyncNode->pRaftStore->currentTerm, host, port, pReceiver->ack, pMsg->lastIndex, - pMsg->lastTerm, pMsg->lastConfigIndex, pReceiver->privateTerm); - } + do { + char *eventLog = snapshotReceiver2SimpleStr(pReceiver, "snapshot receiver force close"); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); + } while (0); } else if (pMsg->seq > SYNC_SNAPSHOT_SEQ_BEGIN && pMsg->seq < SYNC_SNAPSHOT_SEQ_END) { // transfering @@ -780,31 +598,11 @@ int32_t syncNodeOnSnapshotSendCb(SSyncNode *pSyncNode, SyncSnapshotSend *pMsg) { } needRsp = true; - char host[128]; - uint16_t port; - syncUtilU642Addr(pMsg->srcId.addr, host, sizeof(host), &port); - - if (gRaftDetailLog) { - char *msgStr = syncSnapshotSend2Str(pMsg); - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot recv from %s:%d receiving ack:%d, " - "lastIndex:%ld, " - "lastTerm:%lu, " - "lastConfigIndex:%ld, privateTerm:%lu, recv msg:%s", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, host, port, pReceiver->ack, pMsg->lastIndex, pMsg->lastTerm, - pMsg->lastConfigIndex, pReceiver->privateTerm, msgStr); - taosMemoryFree(msgStr); - } else { - sDebug( - "vgId:%d sync event %s commitIndex:%ld currentTerm:%lu snapshot recv from %s:%d receiving ack:%d, " - "lastIndex:%ld, " - "lastTerm:%lu, " - "lastConfigIndex:%ld, privateTerm:%lu", - pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->commitIndex, - pSyncNode->pRaftStore->currentTerm, host, port, pReceiver->ack, pMsg->lastIndex, pMsg->lastTerm, - pMsg->lastConfigIndex, pReceiver->privateTerm); - } + do { + char *eventLog = snapshotReceiver2SimpleStr(pReceiver, "snapshot receiver receiving"); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); + } while (0); } else { ASSERT(0); diff --git a/source/libs/sync/test/syncRaftCfgTest.cpp b/source/libs/sync/test/syncRaftCfgTest.cpp index 9460f0ad7a..8171d266d4 100644 --- a/source/libs/sync/test/syncRaftCfgTest.cpp +++ b/source/libs/sync/test/syncRaftCfgTest.cpp @@ -55,6 +55,8 @@ SSyncCfg* createSyncCfg() { void test1() { SSyncCfg* pCfg = createSyncCfg(); syncCfgLog2((char*)__FUNCTION__, pCfg); + syncCfgLog3((char*)__FUNCTION__, pCfg); + taosMemoryFree(pCfg); } diff --git a/source/libs/transport/inc/transComm.h b/source/libs/transport/inc/transComm.h index e680e30042..146b127422 100644 --- a/source/libs/transport/inc/transComm.h +++ b/source/libs/transport/inc/transComm.h @@ -26,6 +26,7 @@ extern "C" { #include "transLog.h" #include "transportInt.h" #include "trpc.h" +#include "ttrace.h" #include "tutil.h" typedef void* queue[2]; @@ -140,6 +141,7 @@ typedef struct { char spi : 2; char user[TSDB_UNI_LEN]; + STraceId traceId; uint64_t ahandle; // ahandle assigned by client uint32_t code; // del later uint32_t msgType; diff --git a/source/libs/transport/inc/transLog.h b/source/libs/transport/inc/transLog.h index ebf231cfcf..9947ba803f 100644 --- a/source/libs/transport/inc/transLog.h +++ b/source/libs/transport/inc/transLog.h @@ -22,6 +22,7 @@ extern "C" { // clang-format off #include "tlog.h" +#include "ttrace.h" #define tFatal(...) do {if (rpcDebugFlag & DEBUG_FATAL){ taosPrintLog("RPC FATAL ", DEBUG_FATAL, rpcDebugFlag, __VA_ARGS__); }} while (0) #define tError(...)do { if (rpcDebugFlag & DEBUG_ERROR){ taosPrintLog("RPC ERROR ", DEBUG_ERROR, rpcDebugFlag, __VA_ARGS__); } } while(0) @@ -30,6 +31,10 @@ extern "C" { #define tDebug(...) do {if (rpcDebugFlag & DEBUG_DEBUG){ taosPrintLog("RPC ", DEBUG_DEBUG, rpcDebugFlag, __VA_ARGS__); }} while(0) #define tTrace(...) do {if (rpcDebugFlag & DEBUG_TRACE){ taosPrintLog("RPC ", DEBUG_TRACE, rpcDebugFlag, __VA_ARGS__); }} while(0) #define tDump(x, y) do {if (rpcDebugFlag & DEBUG_DUMP) { taosDumpData((unsigned char *)x, y); } } while(0) + +//#define tTR(param, ...) do { char buf[40] = {0};TRACE_TO_STR(trace, buf);tTrace("TRID: %s "param, buf, __VA_ARGS__);} while(0) +#define tGTrace(param, ...) do { char buf[40] = {0}; TRACE_TO_STR(trace, buf); tTrace(param ", GTID: %s", __VA_ARGS__, buf);} while(0) + // clang-format on #ifdef __cplusplus } diff --git a/source/libs/transport/src/trans.c b/source/libs/transport/src/trans.c index 925de2f321..1ec96f4a7a 100644 --- a/source/libs/transport/src/trans.c +++ b/source/libs/transport/src/trans.c @@ -163,6 +163,11 @@ void rpcSetDefaultAddr(void* thandle, const char* ip, const char* fqdn) { transSetDefaultAddr(thandle, ip, fqdn); } +// void rpcSetMsgTraceId(SRpcMsg* pMsg, STraceId uid) { +// SRpcHandleInfo* pInfo = &pMsg->info; +// pInfo->traceId = uid; +//} + int32_t rpcInit() { // impl later return 0; diff --git a/source/libs/transport/src/transCli.c b/source/libs/transport/src/transCli.c index 3d570e0860..912c8b59c2 100644 --- a/source/libs/transport/src/transCli.c +++ b/source/libs/transport/src/transCli.c @@ -48,9 +48,10 @@ typedef struct SCliMsg { STransConnCtx* ctx; STransMsg msg; queue q; - uint64_t st; STransMsgType type; - int sent; //(0: no send, 1: alread sent) + + uint64_t st; + int sent; //(0: no send, 1: alread sent) } SCliMsg; typedef struct SCliThrdObj { @@ -167,27 +168,27 @@ static void cliReleaseUnfinishedMsg(SCliConn* conn) { snprintf(key, sizeof(key), "%s:%d", ip, (int)port); \ } while (0) -#define CONN_HOST_THREAD_INDEX(conn) (conn ? ((SCliConn*)conn)->hThrdIdx : -1) -#define CONN_PERSIST_TIME(para) (para * 1000 * 10) -#define CONN_GET_HOST_THREAD(conn) (conn ? ((SCliConn*)conn)->hostThrd : NULL) -#define CONN_GET_INST_LABEL(conn) (((STrans*)(((SCliThrdObj*)(conn)->hostThrd)->pTransInst))->label) -#define CONN_SHOULD_RELEASE(conn, head) \ - do { \ - if ((head)->release == 1 && (head->msgLen) == sizeof(*head)) { \ - uint64_t ahandle = head->ahandle; \ - CONN_GET_MSGCTX_BY_AHANDLE(conn, ahandle); \ - conn->status = ConnRelease; \ - transClearBuffer(&conn->readBuf); \ - transFreeMsg(transContFromHead((char*)head)); \ - tDebug("cli conn %p receive release request, ref: %d", conn, T_REF_VAL_GET(conn)); \ - if (T_REF_VAL_GET(conn) > 1) { \ - transUnrefCliHandle(conn); \ - } \ - destroyCmsg(pMsg); \ - cliReleaseUnfinishedMsg(conn); \ - addConnToPool(((SCliThrdObj*)conn->hostThrd)->pool, conn); \ - return; \ - } \ +#define CONN_HOST_THREAD_IDX(conn) (conn ? ((SCliConn*)conn)->hThrdIdx : -1) +#define CONN_PERSIST_TIME(para) (para * 1000 * 10) +#define CONN_GET_HOST_THREAD(conn) (conn ? ((SCliConn*)conn)->hostThrd : NULL) +#define CONN_GET_INST_LABEL(conn) (((STrans*)(((SCliThrdObj*)(conn)->hostThrd)->pTransInst))->label) +#define CONN_SHOULD_RELEASE(conn, head) \ + do { \ + if ((head)->release == 1 && (head->msgLen) == sizeof(*head)) { \ + uint64_t ahandle = head->ahandle; \ + CONN_GET_MSGCTX_BY_AHANDLE(conn, ahandle); \ + conn->status = ConnRelease; \ + transClearBuffer(&conn->readBuf); \ + transFreeMsg(transContFromHead((char*)head)); \ + tDebug("%s conn %p receive release request, ref: %d", CONN_GET_INST_LABEL(conn), conn, T_REF_VAL_GET(conn)); \ + if (T_REF_VAL_GET(conn) > 1) { \ + transUnrefCliHandle(conn); \ + } \ + destroyCmsg(pMsg); \ + cliReleaseUnfinishedMsg(conn); \ + addConnToPool(((SCliThrdObj*)conn->hostThrd)->pool, conn); \ + return; \ + } \ } while (0) #define CONN_GET_MSGCTX_BY_AHANDLE(conn, ahandle) \ @@ -280,6 +281,7 @@ void cliHandleResp(SCliConn* conn) { transMsg.code = pHead->code; transMsg.msgType = pHead->msgType; transMsg.info.ahandle = NULL; + transMsg.info.traceId = pHead->traceId; SCliMsg* pMsg = NULL; STransConnCtx* pCtx = NULL; @@ -293,27 +295,28 @@ void cliHandleResp(SCliConn* conn) { if (transMsg.info.ahandle == NULL) { transMsg.info.ahandle = transCtxDumpBrokenlinkVal(&conn->ctx, (int32_t*)&(transMsg.msgType)); } - tDebug("cli conn %p construct ahandle %p, persist: 0", conn, transMsg.info.ahandle); + tDebug("%s conn %p construct ahandle %p, persist: 0", CONN_GET_INST_LABEL(conn), conn, transMsg.info.ahandle); } else { transMsg.info.ahandle = pCtx ? pCtx->ahandle : NULL; - tDebug("cli conn %p get ahandle %p, persist: 0", conn, transMsg.info.ahandle); + tDebug("%s conn %p get ahandle %p, persist: 0", CONN_GET_INST_LABEL(conn), conn, transMsg.info.ahandle); } } else { uint64_t ahandle = (uint64_t)pHead->ahandle; CONN_GET_MSGCTX_BY_AHANDLE(conn, ahandle); if (pMsg == NULL) { transMsg.info.ahandle = transCtxDumpVal(&conn->ctx, transMsg.msgType); - tDebug("cli conn %p construct ahandle %p by %s, persist: 1", conn, transMsg.info.ahandle, - TMSG_INFO(transMsg.msgType)); + tDebug("%s conn %p construct ahandle %p by %s, persist: 1", CONN_GET_INST_LABEL(conn), conn, + transMsg.info.ahandle, TMSG_INFO(transMsg.msgType)); if (!CONN_RELEASE_BY_SERVER(conn) && transMsg.info.ahandle == NULL) { transMsg.code = TSDB_CODE_RPC_NETWORK_UNAVAIL; transMsg.info.ahandle = transCtxDumpBrokenlinkVal(&conn->ctx, (int32_t*)&(transMsg.msgType)); - tDebug("cli conn %p construct ahandle %p due brokenlink, persist: 1", conn, transMsg.info.ahandle); + tDebug("%s conn %p construct ahandle %p due brokenlink, persist: 1", CONN_GET_INST_LABEL(conn), conn, + transMsg.info.ahandle); } } else { pCtx = pMsg ? pMsg->ctx : NULL; transMsg.info.ahandle = pCtx ? pCtx->ahandle : NULL; - tDebug("cli conn %p get ahandle %p, persist: 1", conn, transMsg.info.ahandle); + tDebug("%s conn %p get ahandle %p, persist: 1", CONN_GET_INST_LABEL(conn), conn, transMsg.info.ahandle); } } // buf's mem alread translated to transMsg.pCont @@ -321,26 +324,27 @@ void cliHandleResp(SCliConn* conn) { if (!CONN_NO_PERSIST_BY_APP(conn)) { transMsg.info.handle = conn; - tDebug("%s cli conn %p ref by app", CONN_GET_INST_LABEL(conn), conn); + tDebug("%s conn %p ref by app", CONN_GET_INST_LABEL(conn), conn); } - - tDebug("%s cli conn %p %s received from %s:%d, local info:%s:%d, msg size:%d code:0x%x", pTransInst->label, conn, - TMSG_INFO(pHead->msgType), taosInetNtoa(conn->addr.sin_addr), ntohs(conn->addr.sin_port), - taosInetNtoa(conn->localAddr.sin_addr), ntohs(conn->localAddr.sin_port), transMsg.contLen, transMsg.code); + // char buf[64] = {0}; + // TRACE_TO_STR(&transMsg.info.traceId, buf); + STraceId* trace = &transMsg.info.traceId; + tGTrace("conn %p %s received from %s:%d, local info: %s:%d, msg size: %d, code: %d", conn, TMSG_INFO(pHead->msgType), + taosInetNtoa(conn->addr.sin_addr), ntohs(conn->addr.sin_port), taosInetNtoa(conn->localAddr.sin_addr), + ntohs(conn->localAddr.sin_port), transMsg.contLen, transMsg.code); if (pCtx == NULL && CONN_NO_PERSIST_BY_APP(conn)) { - tTrace("except, server continue send while cli ignore it"); + tDebug("%s except, server continue send while cli ignore it", CONN_GET_INST_LABEL(conn)); // transUnrefCliHandle(conn); return; } if (CONN_RELEASE_BY_SERVER(conn) && transMsg.info.ahandle == NULL) { - tTrace("except, server continue send while cli ignore it"); + tDebug("%s except, server continue send while cli ignore it", CONN_GET_INST_LABEL(conn)); // transUnrefCliHandle(conn); return; } if (cliAppCb(conn, &transMsg, pMsg) != 0) { - tTrace("try to send req to next node"); return; } destroyCmsg(pMsg); @@ -363,7 +367,7 @@ void cliHandleResp(SCliConn* conn) { void cliHandleExcept(SCliConn* pConn) { if (transQueueEmpty(&pConn->cliMsgs)) { if (pConn->broken == true && CONN_NO_PERSIST_BY_APP(pConn)) { - tTrace("%s cli conn %p handle except, persist:0", CONN_GET_INST_LABEL(pConn), pConn); + tTrace("%s conn %p handle except, persist:0", CONN_GET_INST_LABEL(pConn), pConn); transUnrefCliHandle(pConn); return; } @@ -386,11 +390,11 @@ void cliHandleExcept(SCliConn* pConn) { if (pMsg == NULL && !CONN_NO_PERSIST_BY_APP(pConn)) { transMsg.info.ahandle = transCtxDumpVal(&pConn->ctx, transMsg.msgType); - tDebug("%s cli conn %p construct ahandle %p by %s", CONN_GET_INST_LABEL(pConn), pConn, transMsg.info.ahandle, + tDebug("%s conn %p construct ahandle %p by %s", CONN_GET_INST_LABEL(pConn), pConn, transMsg.info.ahandle, TMSG_INFO(transMsg.msgType)); if (transMsg.info.ahandle == NULL) { transMsg.info.ahandle = transCtxDumpBrokenlinkVal(&pConn->ctx, (int32_t*)&(transMsg.msgType)); - tDebug("%s cli conn %p construct ahandle %p due to brokenlink", CONN_GET_INST_LABEL(pConn), pConn, + tDebug("%s conn %p construct ahandle %p due to brokenlink", CONN_GET_INST_LABEL(pConn), pConn, transMsg.info.ahandle); } } else { @@ -404,11 +408,10 @@ void cliHandleExcept(SCliConn* pConn) { } } if (cliAppCb(pConn, &transMsg, pMsg) != 0) { - tTrace("try to send req to next node"); return; } destroyCmsg(pMsg); - tTrace("%s cli conn %p start to destroy", CONN_GET_INST_LABEL(pConn), pConn); + tTrace("%s conn %p start to destroy", CONN_GET_INST_LABEL(pConn), pConn); } while (!transQueueEmpty(&pConn->cliMsgs)); transUnrefCliHandle(pConn); } @@ -417,7 +420,7 @@ void cliTimeoutCb(uv_timer_t* handle) { SCliThrdObj* pThrd = handle->data; STrans* pTransInst = pThrd->pTransInst; int64_t currentTime = pThrd->nextTimeout; - tTrace("%s, cli conn timeout, try to remove expire conn from conn pool", pTransInst->label); + tTrace("%s conn timeout, try to remove expire conn from conn pool", pTransInst->label); SConnList* p = taosHashIterate((SHashObj*)pThrd->pool, NULL); while (p != NULL) { @@ -492,7 +495,7 @@ static void addConnToPool(void* pool, SCliConn* conn) { char key[128] = {0}; CONN_CONSTRUCT_HASH_KEY(key, conn->ip, conn->port); - tTrace("cli conn %p added to conn pool, read buf cap: %d", conn, conn->readBuf.cap); + tTrace("%s conn %p added to conn pool, read buf cap: %d", CONN_GET_INST_LABEL(conn), conn, conn->readBuf.cap); SConnList* plist = taosHashGet((SHashObj*)pool, key, strlen(key)); // list already create before @@ -515,10 +518,10 @@ static void cliRecvCb(uv_stream_t* handle, ssize_t nread, const uv_buf_t* buf) { if (nread > 0) { pBuf->len += nread; if (transReadComplete(pBuf)) { - tTrace("%s cli conn %p read complete", CONN_GET_INST_LABEL(conn), conn); + tTrace("%s conn %p read complete", CONN_GET_INST_LABEL(conn), conn); cliHandleResp(conn); } else { - tTrace("%s cli conn %p read partial packet, continue to read", CONN_GET_INST_LABEL(conn), conn); + tTrace("%s conn %p read partial packet, continue to read", CONN_GET_INST_LABEL(conn), conn); } return; } @@ -528,11 +531,11 @@ static void cliRecvCb(uv_stream_t* handle, ssize_t nread, const uv_buf_t* buf) { // ref http://docs.libuv.org/en/v1.x/stream.html?highlight=uv_read_start#c.uv_read_cb // nread might be 0, which does not indicate an error or EOF. This is equivalent to EAGAIN or EWOULDBLOCK under // read(2). - tTrace("%s cli conn %p read empty", CONN_GET_INST_LABEL(conn), conn); + tTrace("%s conn %p read empty", CONN_GET_INST_LABEL(conn), conn); return; } if (nread < 0) { - tError("%s cli conn %p read error: %s", CONN_GET_INST_LABEL(conn), conn, uv_err_name(nread)); + tError("%s conn %p read error: %s", CONN_GET_INST_LABEL(conn), conn, uv_err_name(nread)); conn->broken = true; cliHandleExcept(conn); } @@ -557,7 +560,7 @@ static SCliConn* cliCreateConn(SCliThrdObj* pThrd) { return conn; } static void cliDestroyConn(SCliConn* conn, bool clear) { - tTrace("%s cli conn %p remove from conn pool", CONN_GET_INST_LABEL(conn), conn); + tTrace("%s conn %p remove from conn pool", CONN_GET_INST_LABEL(conn), conn); QUEUE_REMOVE(&conn->conn); if (clear) { @@ -570,7 +573,7 @@ static void cliDestroy(uv_handle_t* handle) { taosMemoryFree(conn->stream); transCtxCleanup(&conn->ctx); transQueueDestroy(&conn->cliMsgs); - tTrace("%s cli conn %p destroy successfully", CONN_GET_INST_LABEL(conn), conn); + tTrace("%s conn %p destroy successfully", CONN_GET_INST_LABEL(conn), conn); transDestroyBuffer(&conn->readBuf); taosMemoryFree(conn); } @@ -597,14 +600,14 @@ static void cliSendCb(uv_write_t* req, int status) { SCliConn* pConn = req->data; if (status == 0) { - tTrace("%s cli conn %p data already was written out", CONN_GET_INST_LABEL(pConn), pConn); + tTrace("%s conn %p data already was written out", CONN_GET_INST_LABEL(pConn), pConn); } else { - tError("%s cli conn %p failed to write: %s", CONN_GET_INST_LABEL(pConn), pConn, uv_err_name(status)); + tError("%s conn %p failed to write: %s", CONN_GET_INST_LABEL(pConn), pConn, uv_err_name(status)); cliHandleExcept(pConn); return; } if (cliHandleNoResp(pConn) == true) { - tTrace("%s cli conn %p no resp required", CONN_GET_INST_LABEL(pConn), pConn); + tTrace("%s conn %p no resp required", CONN_GET_INST_LABEL(pConn), pConn); return; } uv_read_start((uv_stream_t*)pConn->stream, cliAllocRecvBufferCb, cliRecvCb); @@ -640,11 +643,16 @@ void cliSend(SCliConn* pConn) { pHead->msgLen = (int32_t)htonl((uint32_t)msgLen); pHead->release = REQUEST_RELEASE_HANDLE(pCliMsg) ? 1 : 0; memcpy(pHead->user, pTransInst->user, strlen(pTransInst->user)); + pHead->traceId = pMsg->info.traceId; uv_buf_t wb = uv_buf_init((char*)pHead, msgLen); - tDebug("%s cli conn %p %s is send to %s:%d, local info %s:%d", CONN_GET_INST_LABEL(pConn), pConn, - TMSG_INFO(pHead->msgType), taosInetNtoa(pConn->addr.sin_addr), ntohs(pConn->addr.sin_port), - taosInetNtoa(pConn->localAddr.sin_addr), ntohs(pConn->localAddr.sin_port)); + + // char buf[64] = {0}; + // TRACE_TO_STR(&pMsg->info.traceId, buf); + STraceId* trace = &pMsg->info.traceId; + tGTrace("conn %p %s is sent to %s:%d, local info %s:%d", pConn, TMSG_INFO(pHead->msgType), + taosInetNtoa(pConn->addr.sin_addr), ntohs(pConn->addr.sin_port), taosInetNtoa(pConn->localAddr.sin_addr), + ntohs(pConn->localAddr.sin_port)); if (pHead->persist == 1) { CONN_SET_PERSIST_BY_APP(pConn); @@ -662,7 +670,7 @@ void cliConnCb(uv_connect_t* req, int status) { // impl later SCliConn* pConn = req->data; if (status != 0) { - tError("%s cli conn %p failed to connect server: %s", CONN_GET_INST_LABEL(pConn), pConn, uv_strerror(status)); + tError("%s conn %p failed to connect server: %s", CONN_GET_INST_LABEL(pConn), pConn, uv_strerror(status)); cliHandleExcept(pConn); return; } @@ -672,7 +680,7 @@ void cliConnCb(uv_connect_t* req, int status) { addrlen = sizeof(pConn->localAddr); uv_tcp_getsockname((uv_tcp_t*)pConn->stream, (struct sockaddr*)&pConn->localAddr, &addrlen); - tTrace("%s cli conn %p connect to server successfully", CONN_GET_INST_LABEL(pConn), pConn); + tTrace("%s conn %p connect to server successfully", CONN_GET_INST_LABEL(pConn), pConn); assert(pConn->stream == req->handle); cliSend(pConn); @@ -691,7 +699,7 @@ static void cliHandleQuit(SCliMsg* pMsg, SCliThrdObj* pThrd) { } static void cliHandleRelease(SCliMsg* pMsg, SCliThrdObj* pThrd) { SCliConn* conn = pMsg->msg.info.handle; - tDebug("%s cli conn %p start to release to inst", CONN_GET_INST_LABEL(conn), conn); + tDebug("%s conn %p start to release to inst", CONN_GET_INST_LABEL(conn), conn); if (T_REF_VAL_GET(conn) == 2) { transUnrefCliHandle(conn); @@ -716,15 +724,15 @@ SCliConn* cliGetConn(SCliMsg* pMsg, SCliThrdObj* pThrd) { if (pMsg->msg.info.handle != NULL) { conn = (SCliConn*)(pMsg->msg.info.handle); if (conn != NULL) { - tTrace("%s cli conn %p reused", CONN_GET_INST_LABEL(conn), conn); + tTrace("%s conn %p reused", CONN_GET_INST_LABEL(conn), conn); } } else { STransConnCtx* pCtx = pMsg->ctx; conn = getConnFromPool(pThrd->pool, EPSET_GET_INUSE_IP(&pCtx->epSet), EPSET_GET_INUSE_PORT(&pCtx->epSet)); if (conn != NULL) { - tTrace("%s cli conn %p get from conn pool", CONN_GET_INST_LABEL(conn), conn); + tTrace("%s conn %p get from conn pool", CONN_GET_INST_LABEL(conn), conn); } else { - tTrace("not found conn in conn pool %p", pThrd->pool); + tTrace("%s not found conn in conn pool %p", ((STrans*)pThrd->pTransInst)->label, pThrd->pool); } } return conn; @@ -750,6 +758,8 @@ void cliHandleReq(SCliMsg* pMsg, SCliThrdObj* pThrd) { cliMayCvtFqdnToIp(&pCtx->epSet, &pThrd->cvtAddr); + transPrintEpSet(&pCtx->epSet); + SCliConn* conn = cliGetConn(pMsg, pThrd); if (conn != NULL) { conn->hThrdIdx = pCtx->hThrdIdx; @@ -768,11 +778,11 @@ void cliHandleReq(SCliMsg* pMsg, SCliThrdObj* pThrd) { int ret = transSetConnOption((uv_tcp_t*)conn->stream); if (ret) { - tError("%s cli conn %p failed to set conn option, errmsg %s", pTransInst->label, conn, uv_err_name(ret)); + tError("%s conn %p failed to set conn option, errmsg %s", pTransInst->label, conn, uv_err_name(ret)); } int fd = taosCreateSocketWithTimeOutOpt(TRANS_CONN_TIMEOUT); if (fd == -1) { - tTrace("%s cli conn %p failed to create socket", pTransInst->label, conn); + tTrace("%s conn %p failed to create socket", pTransInst->label, conn); cliHandleExcept(conn); return; } @@ -782,10 +792,10 @@ void cliHandleReq(SCliMsg* pMsg, SCliThrdObj* pThrd) { addr.sin_family = AF_INET; addr.sin_addr.s_addr = taosGetIpv4FromFqdn(conn->ip); addr.sin_port = (uint16_t)htons((uint16_t)conn->port); - tTrace("%s cli conn %p try to connect to %s:%d", pTransInst->label, conn, conn->ip, conn->port); + tTrace("%s conn %p try to connect to %s:%d", pTransInst->label, conn, conn->ip, conn->port); ret = uv_tcp_connect(&conn->connReq, (uv_tcp_t*)(conn->stream), (const struct sockaddr*)&addr, cliConnCb); if (ret != 0) { - tTrace("%s cli conn %p failed to connect to %s:%d, reason: %s", pTransInst->label, conn, conn->ip, conn->port, + tTrace("%s conn %p failed to connect to %s:%d, reason: %s", pTransInst->label, conn, conn->ip, conn->port, uv_err_name(ret)); cliHandleExcept(conn); return; @@ -944,14 +954,13 @@ int cliAppCb(SCliConn* pConn, STransMsg* pResp, SCliMsg* pMsg) { STrans* pTransInst = pThrd->pTransInst; if (pMsg == NULL || pMsg->ctx == NULL) { - tTrace("%s cli conn %p handle resp", pTransInst->label, pConn); + tTrace("%s conn %p handle resp", pTransInst->label, pConn); pTransInst->cfp(pTransInst->parent, pResp, NULL); return 0; } STransConnCtx* pCtx = pMsg->ctx; SEpSet* pEpSet = &pCtx->epSet; - transPrintEpSet(pEpSet); if (pCtx->retryCount == 0) { pCtx->origEpSet = pCtx->epSet; @@ -964,6 +973,7 @@ int cliAppCb(SCliConn* pConn, STransMsg* pResp, SCliMsg* pMsg) { (pResp->code == TSDB_CODE_RPC_NETWORK_UNAVAIL || pResp->code == TSDB_CODE_APP_NOT_READY || pResp->code == TSDB_CODE_NODE_NOT_DEPLOYED || pResp->code == TSDB_CODE_SYN_NOT_LEADER)) { pMsg->sent = 0; + tTrace("try to send req to next node"); pMsg->st = taosGetTimestampUs(); pCtx->retryCount += 1; if (pResp->code == TSDB_CODE_RPC_NETWORK_UNAVAIL) { @@ -974,22 +984,27 @@ int cliAppCb(SCliConn* pConn, STransMsg* pResp, SCliMsg* pMsg) { arg->param1 = pMsg; arg->param2 = pThrd; transDQSched(pThrd->delayQueue, doDelayTask, arg, TRANS_RETRY_INTERVAL); - tTrace("use local epset, current in use: %d, retry count:%d, limit: %d", pEpSet->inUse, pCtx->retryCount + 1, - pEpSet->numOfEps * 3); + transPrintEpSet(pEpSet); + tTrace("%s use local epset, inUse: %d, retry count:%d, limit: %d", pTransInst->label, pEpSet->inUse, + pCtx->retryCount + 1, pEpSet->numOfEps * 3); + transUnrefCliHandle(pConn); return -1; } } else if (pCtx->retryCount < TRANS_RETRY_COUNT_LIMIT) { if (pResp->contLen == 0) { pEpSet->inUse = (++pEpSet->inUse) % pEpSet->numOfEps; - tTrace("use local epset, current in use: %d, retry count:%d, limit: %d", pEpSet->inUse, pCtx->retryCount + 1, - TRANS_RETRY_COUNT_LIMIT); + transPrintEpSet(&pCtx->epSet); + tTrace("%s use local epset, inUse: %d, retry count:%d, limit: %d", pTransInst->label, pEpSet->inUse, + pCtx->retryCount + 1, TRANS_RETRY_COUNT_LIMIT); } else { SEpSet epSet = {0}; tDeserializeSEpSet(pResp->pCont, pResp->contLen, &epSet); pCtx->epSet = epSet; - tTrace("use remote epset, current in use: %d, retry count:%d, limit: %d", pEpSet->inUse, pCtx->retryCount + 1, - TRANS_RETRY_COUNT_LIMIT); + + transPrintEpSet(&pCtx->epSet); + tTrace("%s use remote epset, inUse: %d, retry count:%d, limit: %d", pTransInst->label, pEpSet->inUse, + pCtx->retryCount + 1, TRANS_RETRY_COUNT_LIMIT); } addConnToPool(pThrd->pool, pConn); @@ -1001,17 +1016,18 @@ int cliAppCb(SCliConn* pConn, STransMsg* pResp, SCliMsg* pMsg) { } } + STraceId* trace = &pResp->info.traceId; if (pCtx->pSem != NULL) { - tTrace("%s cli conn %p(sync) handle resp", pTransInst->label, pConn); + tGTrace("conn %p(sync) handle resp", pConn); if (pCtx->pRsp == NULL) { - tTrace("%s cli conn %p(sync) failed to resp, ignore", pTransInst->label, pConn); + tGTrace("conn %p(sync) failed to resp, ignore", pConn); } else { memcpy((char*)pCtx->pRsp, (char*)pResp, sizeof(*pResp)); } tsem_post(pCtx->pSem); pCtx->pRsp = NULL; } else { - tTrace("%s cli conn %p handle resp", pTransInst->label, pConn); + tGTrace("conn %p handle resp", pConn); if (pResp->code != 0 || pCtx->retryCount == 0 || transEpSetIsEqual(&pCtx->epSet, &pCtx->origEpSet)) { pTransInst->cfp(pTransInst->parent, pResp, NULL); } else { @@ -1039,6 +1055,7 @@ void transRefCliHandle(void* handle) { return; } int ref = T_REF_INC((SCliConn*)handle); + tTrace("%s conn %p ref %d", CONN_GET_INST_LABEL((SCliConn*)handle), handle, ref); UNUSED(ref); } void transUnrefCliHandle(void* handle) { @@ -1046,7 +1063,7 @@ void transUnrefCliHandle(void* handle) { return; } int ref = T_REF_DEC((SCliConn*)handle); - tDebug("%s cli conn %p ref %d", CONN_GET_INST_LABEL((SCliConn*)handle), handle, ref); + tTrace("%s conn %p ref %d", CONN_GET_INST_LABEL((SCliConn*)handle), handle, ref); if (ref == 0) { cliDestroyConn((SCliConn*)handle, true); } @@ -1067,16 +1084,17 @@ void transReleaseCliHandle(void* handle) { void transSendRequest(void* shandle, const SEpSet* pEpSet, STransMsg* pReq, STransCtx* ctx) { STrans* pTransInst = (STrans*)shandle; - int index = CONN_HOST_THREAD_INDEX((SCliConn*)pReq->info.handle); - if (index == -1) { - index = cliRBChoseIdx(pTransInst); + int idx = CONN_HOST_THREAD_IDX((SCliConn*)pReq->info.handle); + if (idx == -1) { + idx = cliRBChoseIdx(pTransInst); } + TRACE_SET_MSGID(&pReq->info.traceId, tGenIdPI64()); STransConnCtx* pCtx = taosMemoryCalloc(1, sizeof(STransConnCtx)); pCtx->epSet = *pEpSet; pCtx->ahandle = pReq->info.ahandle; pCtx->msgType = pReq->msgType; - pCtx->hThrdIdx = index; + pCtx->hThrdIdx = idx; if (ctx != NULL) { pCtx->appCtx = *ctx; @@ -1089,27 +1107,30 @@ void transSendRequest(void* shandle, const SEpSet* pEpSet, STransMsg* pReq, STra cliMsg->st = taosGetTimestampUs(); cliMsg->type = Normal; - SCliThrdObj* thrd = ((SCliObj*)pTransInst->tcphandle)->pThreadObj[index]; + SCliThrdObj* thrd = ((SCliObj*)pTransInst->tcphandle)->pThreadObj[idx]; - tDebug("send request at thread:%d, threadID: %08" PRId64 ", msg: %p, dst: %s:%d, app:%p", index, thrd->pid, pReq, - EPSET_GET_INUSE_IP(&pCtx->epSet), EPSET_GET_INUSE_PORT(&pCtx->epSet), pReq->info.ahandle); + STraceId* trace = &pReq->info.traceId; + tGTrace("%s send request at thread:%08" PRId64 ", dst: %s:%d, app:%p", pTransInst->label, thrd->pid, + EPSET_GET_INUSE_IP(&pCtx->epSet), EPSET_GET_INUSE_PORT(&pCtx->epSet), pReq->info.ahandle); ASSERT(transSendAsync(thrd->asyncPool, &(cliMsg->q)) == 0); } void transSendRecv(void* shandle, const SEpSet* pEpSet, STransMsg* pReq, STransMsg* pRsp) { STrans* pTransInst = (STrans*)shandle; - int index = CONN_HOST_THREAD_INDEX(pReq->info.handle); - if (index == -1) { - index = cliRBChoseIdx(pTransInst); + int idx = CONN_HOST_THREAD_IDX(pReq->info.handle); + if (idx == -1) { + idx = cliRBChoseIdx(pTransInst); } tsem_t* sem = taosMemoryCalloc(1, sizeof(tsem_t)); tsem_init(sem, 0, 0); + TRACE_SET_MSGID(&pReq->info.traceId, tGenIdPI64()); + STransConnCtx* pCtx = taosMemoryCalloc(1, sizeof(STransConnCtx)); pCtx->epSet = *pEpSet; pCtx->ahandle = pReq->info.ahandle; pCtx->msgType = pReq->msgType; - pCtx->hThrdIdx = index; + pCtx->hThrdIdx = idx; pCtx->pSem = sem; pCtx->pRsp = pRsp; @@ -1119,16 +1140,17 @@ void transSendRecv(void* shandle, const SEpSet* pEpSet, STransMsg* pReq, STransM cliMsg->st = taosGetTimestampUs(); cliMsg->type = Normal; - SCliThrdObj* thrd = ((SCliObj*)pTransInst->tcphandle)->pThreadObj[index]; - tDebug("send request at thread:%d, threadID:%08" PRId64 ", msg: %p, dst: %s:%d, app:%p", index, thrd->pid, pReq, - EPSET_GET_INUSE_IP(&pCtx->epSet), EPSET_GET_INUSE_PORT(&pCtx->epSet), pReq->info.ahandle); + SCliThrdObj* thrd = ((SCliObj*)pTransInst->tcphandle)->pThreadObj[idx]; + + STraceId* trace = &pReq->info.traceId; + tGTrace("%s send request at thread:%08" PRId64 ", dst: %s:%d, app:%p", pTransInst->label, thrd->pid, + EPSET_GET_INUSE_IP(&pCtx->epSet), EPSET_GET_INUSE_PORT(&pCtx->epSet), pReq->info.ahandle); transSendAsync(thrd->asyncPool, &(cliMsg->q)); tsem_wait(sem); tsem_destroy(sem); taosMemoryFree(sem); } - /* * **/ @@ -1151,7 +1173,7 @@ void transSetDefaultAddr(void* ahandle, const char* ip, const char* fqdn) { cliMsg->type = Update; SCliThrdObj* thrd = ((SCliObj*)pTransInst->tcphandle)->pThreadObj[i]; - tDebug("update epset at thread:%d, threadID:%08" PRId64 "", i, thrd->pid); + tDebug("%s update epset at thread:%08" PRId64 "", pTransInst->label, thrd->pid); transSendAsync(thrd->asyncPool, &(cliMsg->q)); } diff --git a/source/libs/transport/src/transComm.c b/source/libs/transport/src/transComm.c index a04e8b5fca..8cd7f9d827 100644 --- a/source/libs/transport/src/transComm.c +++ b/source/libs/transport/src/transComm.c @@ -376,17 +376,18 @@ static void transDQTimeout(uv_timer_t* timer) { SDelayQueue* queue = timer->data; tTrace("timer %p timeout", timer); uint64_t timeout = 0; + int64_t current = taosGetTimestampMs(); do { HeapNode* minNode = heapMin(queue->heap); if (minNode == NULL) break; SDelayTask* task = container_of(minNode, SDelayTask, node); - if (task->execTime <= taosGetTimestampMs()) { + if (task->execTime <= current) { heapRemove(queue->heap, minNode); task->func(task->arg); taosMemoryFree(task); timeout = 0; } else { - timeout = task->execTime - taosGetTimestampMs(); + timeout = task->execTime - current; break; } } while (1); @@ -442,7 +443,7 @@ int transDQSched(SDelayQueue* queue, void (*func)(void* arg), void* arg, uint64_ } } - tTrace("timer %p put task into queue, timeoutMs: %" PRIu64 "", queue->timer, timeoutMs); + tTrace("timer %p put task into delay queue, timeoutMs: %" PRIu64 "", queue->timer, timeoutMs); heapInsert(queue->heap, &task->node); uv_timer_start(queue->timer, transDQTimeout, timeoutMs, 0); return 0; @@ -453,11 +454,17 @@ void transPrintEpSet(SEpSet* pEpSet) { tTrace("NULL epset"); return; } - tTrace("epset begin inUse: %d", pEpSet->inUse); + char buf[512] = {0}; + int len = snprintf(buf, sizeof(buf), "epset { "); for (int i = 0; i < pEpSet->numOfEps; i++) { - tTrace("ip: %s, port: %d", pEpSet->eps[i].fqdn, pEpSet->eps[i].port); + if (i == pEpSet->numOfEps - 1) { + len += snprintf(buf + len, sizeof(buf) - len, "%d. %s:%d ", i, pEpSet->eps[i].fqdn, pEpSet->eps[i].port); + } else { + len += snprintf(buf + len, sizeof(buf) - len, "%d. %s:%d, ", i, pEpSet->eps[i].fqdn, pEpSet->eps[i].port); + } } - tTrace("epset end"); + len += snprintf(buf + len, sizeof(buf) - len, "}"); + tTrace("%s, inUse: %d", buf, pEpSet->inUse); } bool transEpSetIsEqual(SEpSet* a, SEpSet* b) { if (a->numOfEps != b->numOfEps || a->inUse != b->inUse) { diff --git a/source/libs/transport/src/transSvr.c b/source/libs/transport/src/transSvr.c index bd3781f9c7..f9558d7252 100644 --- a/source/libs/transport/src/transSvr.c +++ b/source/libs/transport/src/transSvr.c @@ -169,7 +169,7 @@ static bool addHandleToAcceptloop(void* arg); conn->status = ConnRelease; \ transClearBuffer(&conn->readBuf); \ transFreeMsg(transContFromHead((char*)head)); \ - tTrace("server conn %p received release request", conn); \ + tTrace("conn %p received release request", conn); \ \ STransMsg tmsg = {.code = 0, .info.handle = (void*)conn, .info.ahandle = NULL}; \ SSvrMsg* srvMsg = taosMemoryCalloc(1, sizeof(SSvrMsg)); \ @@ -181,7 +181,7 @@ static bool addHandleToAcceptloop(void* arg); return; \ } \ if (conn->regArg.init) { \ - tTrace("server conn %p release, notify server app", conn); \ + tTrace("conn %p release, notify server app", conn); \ STrans* pTransInst = conn->pTransInst; \ (*pTransInst->cfp)(pTransInst->parent, &(conn->regArg.msg), NULL); \ memset(&conn->regArg, 0, sizeof(conn->regArg)); \ @@ -209,25 +209,25 @@ static bool addHandleToAcceptloop(void* arg); #define ASYNC_CHECK_HANDLE(exh1, refId) \ do { \ if (refId > 0) { \ - tTrace("server handle step1"); \ + tTrace("handle step1"); \ SExHandle* exh2 = transAcquireExHandle(refMgt, refId); \ if (exh2 == NULL || refId != exh2->refId) { \ - tTrace("server handle %p except, may already freed, ignore msg, ref1: %" PRIu64 ", ref2 : %" PRIu64 "", exh1, \ + tTrace("handle %p except, may already freed, ignore msg, ref1: %" PRIu64 ", ref2 : %" PRIu64 "", exh1, \ exh2 ? exh2->refId : 0, refId); \ goto _return1; \ } \ } else if (refId == 0) { \ - tTrace("server handle step2"); \ + tTrace("handle step2"); \ SExHandle* exh2 = transAcquireExHandle(refMgt, refId); \ if (exh2 == NULL || refId != exh2->refId) { \ - tTrace("server handle %p except, may already freed, ignore msg, ref1: %" PRIu64 ", ref2 : %" PRIu64 "", exh1, \ - refId, exh2 ? exh2->refId : 0); \ + tTrace("handle %p except, may already freed, ignore msg, ref1: %" PRIu64 ", ref2 : %" PRIu64 "", exh1, refId, \ + exh2 ? exh2->refId : 0); \ goto _return1; \ } else { \ refId = exh1->refId; \ } \ } else if (refId < 0) { \ - tTrace("server handle step3"); \ + tTrace("handle step3"); \ goto _return2; \ } \ } while (0) @@ -270,29 +270,28 @@ static void uvHandleReq(SSvrConn* pConn) { transMsg.msgType = pHead->msgType; transMsg.code = pHead->code; - transMsg.info.ahandle = (void*)pHead->ahandle; - transMsg.info.handle = NULL; - - // transDestroyBuffer(&pConn->readBuf); transClearBuffer(&pConn->readBuf); + pConn->inType = pHead->msgType; if (pConn->status == ConnNormal) { if (pHead->persist == 1) { pConn->status = ConnAcquire; transRefSrvHandle(pConn); - tDebug("server conn %p acquired by server app", pConn); + tDebug("conn %p acquired by server app", pConn); } } + STraceId* trace = &pHead->traceId; if (pConn->status == ConnNormal && pHead->noResp == 0) { transRefSrvHandle(pConn); - tDebug("server conn %p %s received from %s:%d, local info:%s:%d, msg size:%d code:0x%x", pConn, - TMSG_INFO(transMsg.msgType), taosInetNtoa(pConn->addr.sin_addr), ntohs(pConn->addr.sin_port), - taosInetNtoa(pConn->localAddr.sin_addr), ntohs(pConn->localAddr.sin_port), transMsg.contLen, transMsg.code); + + tGTrace("conn %p %s received from %s:%d, local info: %s:%d, msg size: %d", pConn, TMSG_INFO(transMsg.msgType), + taosInetNtoa(pConn->addr.sin_addr), ntohs(pConn->addr.sin_port), taosInetNtoa(pConn->localAddr.sin_addr), + ntohs(pConn->localAddr.sin_port), transMsg.contLen); } else { - tDebug("server conn %p %s received from %s:%d, local info:%s:%d, msg size:%d, resp:%d code:0x%x", pConn, - TMSG_INFO(transMsg.msgType), taosInetNtoa(pConn->addr.sin_addr), ntohs(pConn->addr.sin_port), - taosInetNtoa(pConn->localAddr.sin_addr), ntohs(pConn->localAddr.sin_port), transMsg.contLen, pHead->noResp, - transMsg.code); + tGTrace("conn %p %s received from %s:%d, local info: %s:%d, msg size: %d, resp:%d, code: %d", pConn, + TMSG_INFO(transMsg.msgType), taosInetNtoa(pConn->addr.sin_addr), ntohs(pConn->addr.sin_port), + taosInetNtoa(pConn->localAddr.sin_addr), ntohs(pConn->localAddr.sin_port), transMsg.contLen, pHead->noResp, + transMsg.code); // no ref here } @@ -300,11 +299,14 @@ static void uvHandleReq(SSvrConn* pConn) { // 1. server application should not send resp on handle // 2. once send out data, cli conn released to conn pool immediately // 3. not mixed with persist - + transMsg.info.ahandle = (void*)pHead->ahandle; transMsg.info.handle = (void*)transAcquireExHandle(refMgt, pConn->refId); transMsg.info.refId = pConn->refId; - tTrace("server handle %p conn: %p translated to app, refId: %" PRIu64 "", transMsg.info.handle, pConn, pConn->refId); + transMsg.info.traceId = pHead->traceId; + + tGTrace("handle %p conn: %p translated to app, refId: %" PRIu64 "", transMsg.info.handle, pConn, pConn->refId); assert(transMsg.info.handle != NULL); + if (pHead->noResp == 1) { transMsg.info.refId = -1; } @@ -328,12 +330,12 @@ void uvOnRecvCb(uv_stream_t* cli, ssize_t nread, const uv_buf_t* buf) { SConnBuffer* pBuf = &conn->readBuf; if (nread > 0) { pBuf->len += nread; - tTrace("server conn %p read summary, total read: %d, current read: %d", conn, pBuf->len, (int)nread); + tTrace("conn %p total read: %d, current read: %d", conn, pBuf->len, (int)nread); if (transReadComplete(pBuf)) { - tTrace("server conn %p alread read complete packet", conn); + tTrace("conn %p alread read complete packet", conn); uvHandleReq(conn); } else { - tTrace("server %p read partial packet, continue to read", conn); + tTrace("conn %p read partial packet, continue to read", conn); } return; } @@ -341,12 +343,12 @@ void uvOnRecvCb(uv_stream_t* cli, ssize_t nread, const uv_buf_t* buf) { return; } - tError("server conn %p read error: %s", conn, uv_err_name(nread)); + tError("conn %p read error: %s", conn, uv_err_name(nread)); if (nread < 0) { conn->broken = true; if (conn->status == ConnAcquire) { if (conn->regArg.init) { - tTrace("server conn %p broken, notify server app", conn); + tTrace("conn %p broken, notify server app", conn); STrans* pTransInst = conn->pTransInst; (*pTransInst->cfp)(pTransInst->parent, &(conn->regArg.msg), NULL); memset(&conn->regArg, 0, sizeof(conn->regArg)); @@ -363,14 +365,14 @@ void uvAllocConnBufferCb(uv_handle_t* handle, size_t suggested_size, uv_buf_t* b void uvOnTimeoutCb(uv_timer_t* handle) { // opt SSvrConn* pConn = handle->data; - tError("server conn %p time out", pConn); + tError("conn %p time out", pConn); } void uvOnSendCb(uv_write_t* req, int status) { SSvrConn* conn = req->data; // transClearBuffer(&conn->readBuf); if (status == 0) { - tTrace("server conn %p data already was written on stream", conn); + tTrace("conn %p data already was written on stream", conn); if (!transQueueEmpty(&conn->srvMsgs)) { SSvrMsg* msg = transQueuePop(&conn->srvMsgs); // if (msg->type == Release && conn->status != ConnNormal) { @@ -407,7 +409,7 @@ void uvOnSendCb(uv_write_t* req, int status) { } } } else { - tError("server conn %p failed to write data, %s", conn, uv_err_name(status)); + tError("conn %p failed to write data, %s", conn, uv_err_name(status)); conn->broken = true; transUnrefSrvHandle(conn); } @@ -424,8 +426,6 @@ static void uvOnPipeWriteCb(uv_write_t* req, int status) { } static void uvPrepareSendData(SSvrMsg* smsg, uv_buf_t* wb) { - tTrace("server conn %p prepare to send resp", smsg->pConn); - SSvrConn* pConn = smsg->pConn; STransMsg* pMsg = &smsg->msg; if (pMsg->pCont == 0) { @@ -434,6 +434,7 @@ static void uvPrepareSendData(SSvrMsg* smsg, uv_buf_t* wb) { } STransMsgHead* pHead = transHeadFromCont(pMsg->pCont); pHead->ahandle = (uint64_t)pMsg->info.ahandle; + pHead->traceId = pMsg->info.traceId; if (pConn->status == ConnNormal) { pHead->msgType = pConn->inType + 1; @@ -454,9 +455,11 @@ static void uvPrepareSendData(SSvrMsg* smsg, uv_buf_t* wb) { char* msg = (char*)pHead; int32_t len = transMsgLenFromCont(pMsg->contLen); - tDebug("server conn %p %s is sent to %s:%d, local info: %s:%d, msglen:%d", pConn, TMSG_INFO(pHead->msgType), - taosInetNtoa(pConn->addr.sin_addr), ntohs(pConn->addr.sin_port), taosInetNtoa(pConn->localAddr.sin_addr), - ntohs(pConn->localAddr.sin_port), len); + + STraceId* trace = &pMsg->info.traceId; + tGTrace("conn %p %s is sent to %s:%d, local info: %s:%d, msglen:%d", pConn, TMSG_INFO(pHead->msgType), + taosInetNtoa(pConn->addr.sin_addr), ntohs(pConn->addr.sin_port), taosInetNtoa(pConn->localAddr.sin_addr), + ntohs(pConn->localAddr.sin_port), len); pHead->msgLen = htonl(len); wb->base = msg; @@ -545,7 +548,7 @@ void uvWorkerAsyncCb(uv_async_t* handle) { int64_t refId = transMsg.info.refId; SExHandle* exh2 = transAcquireExHandle(refMgt, refId); if (exh2 == NULL || exh1 != exh2) { - tTrace("server handle except msg %p, ignore it", exh1); + tTrace("handle except msg %p, ignore it", exh1); transReleaseExHandle(refMgt, refId); destroySmsg(msg); continue; @@ -583,18 +586,18 @@ static void uvShutDownCb(uv_shutdown_t* req, int status) { static void uvWorkDoTask(uv_work_t* req) { // doing time-consumeing task // only auth conn currently, add more func later - tTrace("server conn %p start to be processed in BG Thread", req->data); + tTrace("conn %p start to be processed in BG Thread", req->data); return; } static void uvWorkAfterTask(uv_work_t* req, int status) { if (status != 0) { - tTrace("server conn %p failed to processed ", req->data); + tTrace("conn %p failed to processed ", req->data); } // Done time-consumeing task // add more func later // this func called in main loop - tTrace("server conn %p already processed ", req->data); + tTrace("conn %p already processed ", req->data); taosMemoryFree(req); } @@ -629,7 +632,7 @@ void uvOnAcceptCb(uv_stream_t* stream, int status) { } } void uvOnConnectionCb(uv_stream_t* q, ssize_t nread, const uv_buf_t* buf) { - tTrace("server connection coming"); + tTrace("connection coming"); if (nread < 0) { if (nread != UV_EOF) { tError("read error %s", uv_err_name(nread)); @@ -678,18 +681,18 @@ void uvOnConnectionCb(uv_stream_t* q, ssize_t nread, const uv_buf_t* buf) { if (uv_accept(q, (uv_stream_t*)(pConn->pTcp)) == 0) { uv_os_fd_t fd; uv_fileno((const uv_handle_t*)pConn->pTcp, &fd); - tTrace("server conn %p created, fd: %d", pConn, fd); + tTrace("conn %p created, fd: %d", pConn, fd); int addrlen = sizeof(pConn->addr); if (0 != uv_tcp_getpeername(pConn->pTcp, (struct sockaddr*)&pConn->addr, &addrlen)) { - tError("server conn %p failed to get peer info", pConn); + tError("conn %p failed to get peer info", pConn); transUnrefSrvHandle(pConn); return; } addrlen = sizeof(pConn->localAddr); if (0 != uv_tcp_getsockname(pConn->pTcp, (struct sockaddr*)&pConn->localAddr, &addrlen)) { - tError("server conn %p failed to get local info", pConn); + tError("conn %p failed to get local info", pConn); transUnrefSrvHandle(pConn); return; } @@ -798,7 +801,7 @@ static SSvrConn* createConn(void* hThrd) { pConn->refId = exh->refId; transRefSrvHandle(pConn); - tTrace("server handle %p, conn %p created, refId: %" PRId64 "", exh, pConn, pConn->refId); + tTrace("handle %p, conn %p created, refId: %" PRId64 "", exh, pConn, pConn->refId); return pConn; } @@ -809,7 +812,7 @@ static void destroyConn(SSvrConn* conn, bool clear) { transDestroyBuffer(&conn->readBuf); if (clear) { - tTrace("server conn %p to be destroyed", conn); + tTrace("conn %p to be destroyed", conn); // uv_shutdown_t* req = taosMemoryMalloc(sizeof(uv_shutdown_t)); uv_close((uv_handle_t*)conn->pTcp, uvDestroyConn); // uv_close(conn->pTcp) @@ -845,7 +848,7 @@ static void uvDestroyConn(uv_handle_t* handle) { transReleaseExHandle(refMgt, conn->refId); transRemoveExHandle(refMgt, conn->refId); - tDebug("server conn %p destroy", conn); + tDebug("conn %p destroy", conn); // uv_timer_stop(&conn->pTimer); transQueueDestroy(&conn->srvMsgs); @@ -974,18 +977,18 @@ void uvHandleRelease(SSvrMsg* msg, SWorkThrdObj* thrd) { uvStartSendRespInternal(msg); return; } else if (conn->status == ConnRelease || conn->status == ConnNormal) { - tDebug("server conn %p already released, ignore release-msg", conn); + tDebug("conn %p already released, ignore release-msg", conn); } destroySmsg(msg); } void uvHandleResp(SSvrMsg* msg, SWorkThrdObj* thrd) { // send msg to client - tDebug("server conn %p start to send resp (2/2)", msg->pConn); + tDebug("conn %p start to send resp (2/2)", msg->pConn); uvStartSendResp(msg); } void uvHandleRegister(SSvrMsg* msg, SWorkThrdObj* thrd) { SSvrConn* conn = msg->pConn; - tDebug("server conn %p register brokenlink callback", conn); + tDebug("conn %p register brokenlink callback", conn); if (conn->status == ConnAcquire) { if (!transQueuePush(&conn->srvMsgs, msg)) { return; @@ -994,7 +997,7 @@ void uvHandleRegister(SSvrMsg* msg, SWorkThrdObj* thrd) { conn->regArg.notifyCount = 0; conn->regArg.init = 1; conn->regArg.msg = msg->msg; - tDebug("server conn %p register brokenlink callback succ", conn); + tDebug("conn %p register brokenlink callback succ", conn); if (conn->broken) { STrans* pTransInst = conn->pTransInst; @@ -1062,7 +1065,7 @@ void transRefSrvHandle(void* handle) { return; } int ref = T_REF_INC((SSvrConn*)handle); - tDebug("server conn %p ref count: %d", handle, ref); + tDebug("conn %p ref count: %d", handle, ref); } void transUnrefSrvHandle(void* handle) { @@ -1070,7 +1073,7 @@ void transUnrefSrvHandle(void* handle) { return; } int ref = T_REF_DEC((SSvrConn*)handle); - tDebug("server conn %p ref count: %d", handle, ref); + tDebug("conn %p ref count: %d", handle, ref); if (ref == 0) { destroyConn((SSvrConn*)handle, true); } @@ -1091,16 +1094,16 @@ void transReleaseSrvHandle(void* handle) { m->msg = tmsg; m->type = Release; - tTrace("server conn %p start to release", exh->handle); + tTrace("conn %p start to release", exh->handle); transSendAsync(pThrd->asyncPool, &m->q); transReleaseExHandle(refMgt, refId); return; _return1: - tTrace("server handle %p failed to send to release handle", exh); + tTrace("handle %p failed to send to release handle", exh); transReleaseExHandle(refMgt, refId); return; _return2: - tTrace("server handle %p failed to send to release handle", exh); + tTrace("handle %p failed to send to release handle", exh); return; } void transSendResponse(const STransMsg* msg) { @@ -1118,17 +1121,19 @@ void transSendResponse(const STransMsg* msg) { SSvrMsg* m = taosMemoryCalloc(1, sizeof(SSvrMsg)); m->msg = tmsg; m->type = Normal; - tDebug("server conn %p start to send resp (1/2)", exh->handle); + + STraceId* trace = (STraceId*)&msg->info.traceId; + tGTrace("conn %p start to send resp (1/2)", exh->handle); transSendAsync(pThrd->asyncPool, &m->q); transReleaseExHandle(refMgt, refId); return; _return1: - tTrace("server handle %p failed to send resp", exh); + tTrace("handle %p failed to send resp", exh); rpcFreeCont(msg->pCont); transReleaseExHandle(refMgt, refId); return; _return2: - tTrace("server handle %p failed to send resp", exh); + tTrace("handle %p failed to send resp", exh); rpcFreeCont(msg->pCont); return; } @@ -1146,18 +1151,19 @@ void transRegisterMsg(const STransMsg* msg) { SSvrMsg* m = taosMemoryCalloc(1, sizeof(SSvrMsg)); m->msg = tmsg; m->type = Register; - tTrace("server conn %p start to register brokenlink callback", exh->handle); + + tTrace("conn %p start to register brokenlink callback", exh->handle); transSendAsync(pThrd->asyncPool, &m->q); transReleaseExHandle(refMgt, refId); return; _return1: - tTrace("server handle %p failed to send to register brokenlink", exh); + tTrace("handle %p failed to register brokenlink", exh); rpcFreeCont(msg->pCont); transReleaseExHandle(refMgt, refId); return; _return2: - tTrace("server handle %p failed to send to register brokenlink", exh); + tTrace("handle %p failed to register brokenlink", exh); rpcFreeCont(msg->pCont); } diff --git a/source/os/src/osSocket.c b/source/os/src/osSocket.c index 4a0d9e2866..4d61e7036d 100644 --- a/source/os/src/osSocket.c +++ b/source/os/src/osSocket.c @@ -946,8 +946,7 @@ int32_t taosGetFqdn(char *fqdn) { #endif // __APPLE__ int32_t ret = getaddrinfo(hostname, NULL, &hints, &result); if (!result) { - // printf("failed to get fqdn, code:%d, reason:%s", ret, gai_strerror(ret)); - assert(0); + fprintf(stderr,"failed to get fqdn, code:%d, reason:%s", ret, gai_strerror(ret)); return -1; } diff --git a/source/util/src/tenv.c b/source/util/src/tenv.c index e717e82c5b..4fc0542816 100644 --- a/source/util/src/tenv.c +++ b/source/util/src/tenv.c @@ -72,6 +72,9 @@ int32_t taosEnvToCfg(const char *envStr, char *cfgStr) { if (cfgNameLen > 0) { memcpy(cfgStr, buf, cfgNameLen); memset(&cfgStr[cfgNameLen], ' ', p - cfgStr - cfgNameLen + 1); + } else { + *cfgStr = '\0'; + return -1; } } return strlen(cfgStr); diff --git a/source/util/src/terror.c b/source/util/src/terror.c index 64031956d3..7f949e5b27 100644 --- a/source/util/src/terror.c +++ b/source/util/src/terror.c @@ -580,9 +580,14 @@ TAOS_DEFINE_ERROR(TSDB_CODE_TSMA_NO_INDEX_IN_CACHE, "No tsma index in ca TAOS_DEFINE_ERROR(TSDB_CODE_RSMA_INVALID_ENV, "Invalid rsma env") TAOS_DEFINE_ERROR(TSDB_CODE_RSMA_INVALID_STAT, "Invalid rsma state") +//tq +TAOS_DEFINE_ERROR(TSDB_CODE_TQ_NO_COMMITTED_OFFSET, "No committed offset") + TAOS_DEFINE_ERROR(TSDB_CODE_INDEX_REBUILDING, "Index is rebuilding") +TAOS_DEFINE_ERROR(TSDB_CODE_TMQ_INVALID_MSG, "Invalid message") + #ifdef TAOS_ERROR_C }; #endif diff --git a/source/util/src/thash.c b/source/util/src/thash.c index 8bda59bba7..98762d8fb0 100644 --- a/source/util/src/thash.c +++ b/source/util/src/thash.c @@ -56,7 +56,7 @@ typedef struct SHashEntry { } SHashEntry; struct SHashObj { - SHashEntry **hashList; + SHashEntry ** hashList; size_t capacity; // number of slots int64_t size; // number of elements in hash table _hash_fn_t hashFp; // hash function @@ -65,7 +65,7 @@ struct SHashObj { SRWLatch lock; // read-write spin lock SHashLockTypeE type; // lock type bool enableUpdate; // enable update - SArray *pMemBlock; // memory block allocated for SHashEntry + SArray * pMemBlock; // memory block allocated for SHashEntry _hash_before_fn_t callbackFp; // function invoked before return the value to caller }; @@ -633,7 +633,7 @@ void taosHashTableResize(SHashObj *pHashObj) { } int64_t st = taosGetTimestampUs(); - void *pNewEntryList = taosMemoryRealloc(pHashObj->hashList, sizeof(void *) * newCapacity); + void * pNewEntryList = taosMemoryRealloc(pHashObj->hashList, sizeof(void *) * newCapacity); if (pNewEntryList == NULL) { // uDebug("cache resize failed due to out of memory, capacity remain:%zu", pHashObj->capacity); return; @@ -642,7 +642,7 @@ void taosHashTableResize(SHashObj *pHashObj) { pHashObj->hashList = pNewEntryList; size_t inc = newCapacity - pHashObj->capacity; - void *p = taosMemoryCalloc(inc, sizeof(SHashEntry)); + void * p = taosMemoryCalloc(inc, sizeof(SHashEntry)); for (int32_t i = 0; i < inc; ++i) { pHashObj->hashList[i + pHashObj->capacity] = (void *)((char *)p + i * sizeof(SHashEntry)); @@ -653,9 +653,9 @@ void taosHashTableResize(SHashObj *pHashObj) { pHashObj->capacity = newCapacity; for (int32_t idx = 0; idx < pHashObj->capacity; ++idx) { SHashEntry *pe = pHashObj->hashList[idx]; - SHashNode *pNode; - SHashNode *pNext; - SHashNode *pPrev = NULL; + SHashNode * pNode; + SHashNode * pNext; + SHashNode * pPrev = NULL; if (pe->num == 0) { assert(pe->next == NULL); diff --git a/source/util/src/ttrace.c b/source/util/src/ttrace.c new file mode 100644 index 0000000000..f183fd58fd --- /dev/null +++ b/source/util/src/ttrace.c @@ -0,0 +1,79 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ +#include "ttrace.h" +#include "taos.h" +#include "thash.h" +#include "tuuid.h" + +// clang-format off +//static TdThreadOnce init = PTHREAD_ONCE_INIT; +//static void * ids = NULL; +//static TdThreadMutex mtx; +// +//void traceInit() { +// ids = taosHashInit(4096, taosGetDefaultHashFunction(TSDB_DATA_TYPE_INT), true, HASH_ENTRY_LOCK); +// taosThreadMutexInit(&mtx, NULL); +//} +//void traceCreateEnv() { +// taosThreadOnce(&init, traceInit); +//} +//void traceDestroyEnv() { +// taosThreadMutexDestroy(&mtx); +// taosHashCleanup(ids); +//} +// +//STraceId traceInitId(STraceSubId *h, STraceSubId *l) { +// STraceId id = *h; +// id = ((id << 32) & 0xFFFFFFFF) | ((*l) & 0xFFFFFFFF); +// return id; +//} +//void traceId2Str(STraceId *id, char *buf) { +// int32_t f = (*id >> 32) & 0xFFFFFFFF; +// int32_t s = (*id) & 0xFFFFFFFF; +// sprintf(buf, "%d:%d", f, s); +//} +// +//void traceSetSubId(STraceId *id, STraceSubId *subId) { +// int32_t parent = ((*id >> 32) & 0xFFFFFFFF); +// taosThreadMutexLock(&mtx); +// taosHashPut(ids, subId, sizeof(*subId), &parent, sizeof(parent)); +// taosThreadMutexUnlock(&mtx); +//} +// +//STraceSubId traceGetParentId(STraceId *id) { +// int32_t parent = ((*id >> 32) & 0xFFFFFFFF); +// taosThreadMutexLock(&mtx); +// STraceSubId *p = taosHashGet(ids, (void *)&parent, sizeof(parent)); +// parent = *p; +// taosThreadMutexUnlock(&mtx); +// +// return parent; +//} +// +//STraceSubId traceGenSubId() { +// return tGenIdPI32(); +//} +//void traceSetRootId(STraceId *traceid, int64_t rootId) { +// traceId->rootId = rootId; +//} +//int64_t traceGetRootId(STraceId *traceId); +// +//void traceSetMsgId(STraceId *traceid, int64_t msgId); +//int64_t traceGetMsgId(STraceId *traceid); +// +//char *trace2Str(STraceId *id); +// +//void traceSetSubId(STraceId *id, int32_t *subId); +// clang-format on diff --git a/source/util/src/tuuid.c b/source/util/src/tuuid.c index 1f3cbd9573..9101aec949 100644 --- a/source/util/src/tuuid.c +++ b/source/util/src/tuuid.c @@ -49,7 +49,7 @@ int64_t tGenIdPI64(void) { } int64_t id; - + while (true) { int64_t ts = taosGetTimestampMs(); uint64_t pid = taosGetPId(); diff --git a/tests/script/jenkins/basic.txt b/tests/script/jenkins/basic.txt index 6fe1251ed7..a8ec0f5151 100644 --- a/tests/script/jenkins/basic.txt +++ b/tests/script/jenkins/basic.txt @@ -132,7 +132,7 @@ #./test.sh -f tsim/mnode/basic1.sim -m # --- sma -./test.sh -f tsim/sma/tsmaCreateInsertData.sim +./test.sh -f tsim/sma/tsmaCreateInsertQuery.sim ./test.sh -f tsim/sma/rsmaCreateInsertQuery.sim # --- valgrind diff --git a/tests/script/sh/copy_udf.bat b/tests/script/sh/copy_udf.bat new file mode 100644 index 0000000000..5144cf8d25 --- /dev/null +++ b/tests/script/sh/copy_udf.bat @@ -0,0 +1,23 @@ +@echo off + +echo Executing copy_udf.bat +set SCRIPT_DIR=%cd% +echo SCRIPT_DIR: %SCRIPT_DIR% + +cd ..\..\.. +set TAOS_DIR=%cd% +echo find udf library in %TAOS_DIR% +set UDF1_DIR=%TAOS_DIR%\debug\build\lib\udf1.dll +set UDF2_DIR=%TAOS_DIR%\debug\build\lib\udf2.dll + +echo %UDF1_DIR% +echo %UDF2_DIR% + +set UDF_TMP=C:\Windows\Temp\udf +rm -rf %UDF_TMP% +mkdir %UDF_TMP% + +echo Copy udf shared library files to %UDF_TMP% + +cp %UDF1_DIR% %UDF_TMP% +cp %UDF2_DIR% %UDF_TMP% diff --git a/tests/script/tsim/insert/update0.sim b/tests/script/tsim/insert/update0.sim index 0ba3e98c91..ed74188bcb 100644 --- a/tests/script/tsim/insert/update0.sim +++ b/tests/script/tsim/insert/update0.sim @@ -9,7 +9,7 @@ sql create database d0 keep 365000d,365000d,365000d sql use d0 print =============== create super table and register rsma -sql create table if not exists stb (ts timestamp, c1 int) tags (city binary(20),district binary(20)) rollup(min) file_factor 0.1; +sql create table if not exists stb (ts timestamp, c1 int) tags (city binary(20),district binary(20)) rollup(min); sql show stables if $rows != 1 then diff --git a/tests/script/tsim/query/udf.sim b/tests/script/tsim/query/udf.sim index 93cae4e391..d9821a0495 100644 --- a/tests/script/tsim/query/udf.sim +++ b/tests/script/tsim/query/udf.sim @@ -19,8 +19,14 @@ sql show databases; sql create table t (ts timestamp, f int); sql insert into t values(now, 1)(now+1s, 2); -sql create function udf1 as '/tmp/udf/libudf1.so' outputtype int bufSize 8; -sql create aggregate function udf2 as '/tmp/udf/libudf2.so' outputtype double bufSize 8; +system_content printf %OS% +if $system_content == Windows_NT then + sql create function udf1 as 'C:\\Windows\\Temp\\udf1.dll' outputtype int bufSize 8; + sql create aggregate function udf2 as 'C:\\Windows\\Temp\\udf2.dll' outputtype double bufSize 8; +else + sql create function udf1 as '/tmp/udf/libudf1.so' outputtype int bufSize 8; + sql create aggregate function udf2 as '/tmp/udf/libudf2.so' outputtype double bufSize 8; +endi sql show functions; if $rows != 2 then return -1 diff --git a/tests/script/tsim/sma/rsmaCreateInsertQuery.sim b/tests/script/tsim/sma/rsmaCreateInsertQuery.sim index f929dda18c..645b28f771 100644 --- a/tests/script/tsim/sma/rsmaCreateInsertQuery.sim +++ b/tests/script/tsim/sma/rsmaCreateInsertQuery.sim @@ -9,7 +9,7 @@ sql create database d0 retentions 15s:7d,1m:21d,15m:365d; sql use d0 print =============== create super table and register rsma -sql create table if not exists stb (ts timestamp, c1 int) tags (city binary(20),district binary(20)) rollup(min) file_factor 0.1; +sql create table if not exists stb (ts timestamp, c1 int) tags (city binary(20),district binary(20)) rollup(min); sql show stables if $rows != 1 then diff --git a/tests/script/tsim/sma/tsmaCreateInsertData.sim b/tests/script/tsim/sma/tsmaCreateInsertData.sim deleted file mode 100644 index 7a8cc2a555..0000000000 --- a/tests/script/tsim/sma/tsmaCreateInsertData.sim +++ /dev/null @@ -1,56 +0,0 @@ -system sh/stop_dnodes.sh -system sh/deploy.sh -n dnode1 -i 1 -system sh/exec.sh -n dnode1 -s start -sleep 50 -sql connect - -print =============== create database -sql create database d1 vgroups 1 -sql use d1 - -print =============== create super table, include column type for count/sum/min/max/first -sql create table if not exists stb (ts timestamp, c1 int, c2 float, c3 double) tags (t1 int unsigned) - -sql show stables -if $rows != 1 then - return -1 -endi - -print =============== create child table -sql create table ct1 using stb tags(1000) - -sql show tables -if $rows != 1 then - return -1 -endi - -print =============== insert data, mode1: one row one table in sql -sql insert into ct1 values(now+0s, 10, 2.0, 3.0) -sql insert into ct1 values(now+1s, 11, 2.1, 3.1)(now+2s, -12, -2.2, -3.2)(now+3s, -13, -2.3, -3.3) - - -print =============== create sma index from super table -sql create sma index sma_index_name1 on stb function(max(c1),max(c2),min(c1)) interval(5m,10s) sliding(2m) -print $data00 $data01 $data02 $data03 - -print =============== trigger stream to execute sma aggr task and insert sma data into sma store -sql insert into ct1 values(now+5s, 20, 20.0, 30.0) -#=================================================================== - -print =============== show streams ================================ -sql show streams; -print $data00 $data01 $data02 - -if $data00 != d1 then - return -1 -endi - -print =============== select * from ct1 from memory -sql select * from ct1; -print $data00 $data01 -if $rows != 5 then - print rows $rows != 5 - return -1 -endi - -system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/sma/tsmaCreateInsertQuery.sim b/tests/script/tsim/sma/tsmaCreateInsertQuery.sim new file mode 100644 index 0000000000..3a1ed62a72 --- /dev/null +++ b/tests/script/tsim/sma/tsmaCreateInsertQuery.sim @@ -0,0 +1,181 @@ +system sh/stop_dnodes.sh +system sh/deploy.sh -n dnode1 -i 1 +system sh/exec.sh -n dnode1 -s start +sleep 50 +sql connect + +print =============== create database +sql create database d1 vgroups 1 +sql use d1 + +print =============== create super table, include column type for count/sum/min/max/first +sql create table if not exists stb (ts timestamp, c1 int, c2 float, c3 double) tags (t1 int unsigned) + +sql show stables +if $rows != 1 then + return -1 +endi + +print =============== create child table +sql create table ct1 using stb tags(1000) + +sql show tables +if $rows != 1 then + return -1 +endi + +print =============== insert data, mode1: one row one table in sql +sql insert into ct1 values(now+0s, 10, 2.0, 3.0) +sql insert into ct1 values(now+1s, 11, 2.1, 3.1)(now+2s, -12, -2.2, -3.2)(now+3s, -13, -2.3, -3.3) + + +print =============== create sma index from super table +sql create sma index sma_index_name1 on stb function(max(c1),max(c2),min(c1)) interval(5m,10s) sliding(5m) +print $data00 $data01 $data02 $data03 + +print =============== trigger stream to execute sma aggr task and insert sma data into sma store +sql insert into ct1 values(now+5s, 20, 20.0, 30.0) +#=================================================================== + +print =============== show streams ================================ +sql show streams; +print $data00 $data01 $data02 + +if $data00 != d1 then + return -1 +endi + +print =============== select * from ct1 from memory +sql select * from ct1; +print $data00 $data01 $data02 $data03 +print $data10 $data11 $data12 $data13 +print $data20 $data21 $data22 $data23 +print $data30 $data31 $data32 $data33 +print $data40 $data41 $data42 $data43 +if $rows != 5 then + print rows $rows != 5 + return -1 +endi + +print =============== select * from stb from memory in designated vgroup +sql select _wstartts, _wendts, min(c1),max(c2),max(c1) from stb interval(5m,10s) sliding(5m); +print $data00 $data01 $data02 $data03 $data04 +if $rows != 1 then + print rows $rows != 1 + return -1 +endi + +if $data02 != -13 then + print data02 $data02 != -13 + return -1 +endi + +if $data03 != 20.00000 then + print data03 $data03 != 20.00000 + return -1 +endi + +if $data04 != 20 then + print data04 $data04 != 20 + return -1 +endi + +print =============== select * from stb from memory in common vgroups +sql select _wstartts, _wendts, min(c1),max(c2),max(c1),max(c3) from stb interval(5m,10s) sliding(5m); +print $data00 $data01 $data02 $data03 $data04 $data05 +if $rows != 1 then + print rows $rows != 1 + return -1 +endi + +if $data02 != -13 then + print data02 $data02 != -13 + return -1 +endi + +if $data03 != 20.00000 then + print data03 $data03 != 20.00000 + return -1 +endi + +if $data04 != 20 then + print data04 $data04 != 20 + return -1 +endi + +if $data05 != 30.000000000 then + print data05 $data05 != 30.000000000 + return -1 +endi + + + +system sh/exec.sh -n dnode1 -s stop -x SIGINT +system sh/exec.sh -n dnode1 -s start +sleep 50 + + +print =============== select * from ct1 from file +sql select * from ct1; +print $data00 $data01 $data02 $data03 +print $data10 $data11 $data12 $data13 +print $data20 $data21 $data22 $data23 +print $data30 $data31 $data32 $data33 +print $data40 $data41 $data42 $data43 +if $rows != 5 then + print rows $rows != 5 + return -1 +endi + +print =============== select * from stb from file in designated vgroup +sql select _wstartts, _wendts, min(c1),max(c2),max(c1) from stb interval(5m,10s) sliding(5m); +print $data00 $data01 $data02 $data03 $data04 +if $rows != 1 then + print rows $rows != 1 + return -1 +endi + +if $data02 != -13 then + print data02 $data02 != -13 + return -1 +endi + +if $data03 != 20.00000 then + print data03 $data03 != 20.00000 + return -1 +endi + +if $data04 != 20 then + print data04 $data04 != 20 + return -1 +endi + +print =============== select * from stb from file in common vgroups +sql select _wstartts, _wendts, min(c1),max(c2),max(c1),max(c3) from stb interval(5m,10s) sliding(5m); +print $data00 $data01 $data02 $data03 $data04 $data05 +if $rows != 1 then + print rows $rows != 1 + return -1 +endi + +if $data02 != -13 then + print data02 $data02 != -13 + return -1 +endi + +if $data03 != 20.00000 then + print data03 $data03 != 20.00000 + return -1 +endi + +if $data04 != 20 then + print data04 $data04 != 20 + return -1 +endi + +if $data05 != 30.000000000 then + print data05 $data05 != 30.000000000 + return -1 +endi + + diff --git a/tests/script/tsim/testsuit.sim b/tests/script/tsim/testsuit.sim index 90210fd436..1636bb38c8 100644 --- a/tests/script/tsim/testsuit.sim +++ b/tests/script/tsim/testsuit.sim @@ -1,35 +1,35 @@ -#run tsim/user/pass_alter.sim -#run tsim/user/basic1.sim -#run tsim/user/privilege2.sim -#run tsim/user/user_len.sim -#run tsim/user/privilege1.sim -#run tsim/user/pass_len.sim -#run tsim/table/basic1.sim -#run tsim/trans/lossdata1.sim -#run tsim/trans/create_db.sim -#run tsim/stable/alter_metrics.sim -#run tsim/stable/tag_modify.sim -#run tsim/stable/alter_comment.sim -#run tsim/stable/column_drop.sim -#run tsim/stable/column_modify.sim -#run tsim/stable/tag_rename.sim -#run tsim/stable/vnode3.sim -#run tsim/stable/metrics.sim -#run tsim/stable/alter_insert2.sim -#run tsim/stable/show.sim -#run tsim/stable/alter_import.sim -#run tsim/stable/tag_add.sim -#run tsim/stable/tag_drop.sim -#run tsim/stable/column_add.sim -#run tsim/stable/alter_count.sim -#run tsim/stable/values.sim +run tsim/user/pass_alter.sim +run tsim/user/basic1.sim +run tsim/user/privilege2.sim +run tsim/user/user_len.sim +run tsim/user/privilege1.sim +run tsim/user/pass_len.sim +run tsim/table/basic1.sim +run tsim/trans/lossdata1.sim +run tsim/trans/create_db.sim +run tsim/stable/alter_metrics.sim +run tsim/stable/tag_modify.sim +run tsim/stable/alter_comment.sim +run tsim/stable/column_drop.sim +run tsim/stable/column_modify.sim +run tsim/stable/tag_rename.sim +run tsim/stable/vnode3.sim +run tsim/stable/metrics.sim +run tsim/stable/alter_insert2.sim +run tsim/stable/show.sim +run tsim/stable/alter_import.sim +run tsim/stable/tag_add.sim +run tsim/stable/tag_drop.sim +run tsim/stable/column_add.sim +run tsim/stable/alter_count.sim +run tsim/stable/values.sim run tsim/stable/dnode3.sim -#run tsim/stable/alter_insert1.sim -#run tsim/stable/refcount.sim -#run tsim/stable/disk.sim +run tsim/stable/alter_insert1.sim +run tsim/stable/refcount.sim +run tsim/stable/disk.sim run tsim/db/basic1.sim run tsim/db/basic3.sim -#run tsim/db/basic7.sim +run tsim/db/basic7.sim run tsim/db/basic6.sim run tsim/db/create_all_options.sim run tsim/db/basic2.sim diff --git a/tests/system-test/1-insert/alter_stable.py b/tests/system-test/1-insert/alter_stable.py index c92efb403c..cd64e3ddfe 100644 --- a/tests/system-test/1-insert/alter_stable.py +++ b/tests/system-test/1-insert/alter_stable.py @@ -77,6 +77,7 @@ class TDTestCase: tdSql.error(f'alter stable {stbname} modify column c9 double') tdSql.error(f'alter stable {stbname} modify column c10 float') tdSql.error(f'alter stable {stbname} modify column c11 int') + tdSql.error(f'alter stable {stbname} drop tag t0') tdSql.execute(f'drop database {dbname}') def alter_stable_tag_check(self,dbname,stbname,tbname): @@ -126,6 +127,11 @@ class TDTestCase: for i in ['int','unsigned int','float','binary(10)','nchar(10)']: tdSql.error(f'alter stable {stbname} modify tag t8 {i}') tdSql.error(f'alter stable {stbname} modify tag t4 int') + tdSql.error(f'alter stable {stbname} drop column t0') + #!bug TD-16410 + # tdSql.error(f'alter stable {tbname} set tag t1=100 ') + # tdSql.execute(f'create table ntb (ts timestamp,c0 int)') + tdSql.error(f'alter stable ntb add column c2 ') tdSql.execute(f'drop database {dbname}') def run(self): diff --git a/tests/system-test/1-insert/alter_table.py b/tests/system-test/1-insert/alter_table.py index 3c0def86e4..a4e40d1b0b 100644 --- a/tests/system-test/1-insert/alter_table.py +++ b/tests/system-test/1-insert/alter_table.py @@ -52,8 +52,6 @@ class TDTestCase: tdSql.execute(f'create database if not exists {dbname}') stbname = self.get_long_name(length=3, mode="letters") tbname = self.get_long_name(length=3, mode="letters") - tdLog.info('--------------------------child table tag check--------------------------------------') - tdLog.info(f'-----------------create stable {stbname} and child table {tbname}-------------------') tdSql.execute(f'create stable if not exists {dbname}.{stbname} (col_ts timestamp, c1 int) tags (tag_ts timestamp, t1 tinyint, t2 smallint, t3 int, \ t4 bigint, t5 tinyint unsigned, t6 smallint unsigned, t7 int unsigned, t8 bigint unsigned, t9 float, t10 double, t11 bool,t12 binary(20),t13 nchar(20))') tdSql.execute(f'create table if not exists {dbname}.{tbname} using {dbname}.{stbname} tags(now, 1, 2, 3, 4, 5, 6, 7, 8, 9.9, 10.1, True,"abc123","涛思数据")') @@ -90,13 +88,16 @@ class TDTestCase: tdSql.checkData(0,15,tag_nchar) # bug TD-16211 insert length more than setting binary and nchar - # tag_binary = self.get_long_name(length=21, mode="letters") - # tag_nchar = self.get_long_name(length=21, mode="letters") - # tdSql.error(f'alter table {dbname}.{tbname} set tag t12 = "{tag_binary}"') - # tdSql.error(f'alter table {dbname}.{tbname} set tag t13 = "{tag_nchar}"') - + # error_tag_binary = self.get_long_name(length=21, mode="letters") + # error_tag_nchar = self.get_long_name(length=21, mode="letters") + # tdSql.error(f'alter table {dbname}.{tbname} set tag t12 = "{error_tag_binary}"') + # tdSql.error(f'alter table {dbname}.{tbname} set tag t13 = "{error_tag_nchar}"') + error_tag_binary = self.get_long_name(length=25, mode="letters") + error_tag_nchar = self.get_long_name(length=25, mode="letters") + tdSql.error(f'alter table {dbname}.{tbname} set tag t12 = "{error_tag_binary}"') + tdSql.error(f'alter table {dbname}.{tbname} set tag t13 = "{error_tag_nchar}"') # bug TD-16210 modify binary to nchar - # tdSql.error(f'alter table {dbname}.{tbname} modify tag t12 nchar(10)') + tdSql.error(f'alter table {dbname}.{tbname} modify tag t12 nchar(10)') tdSql.execute(f"drop database {dbname}") def alter_ntb_column_check(self): ''' @@ -125,6 +126,20 @@ class TDTestCase: tdSql.execute(f'alter table {dbname}.{tbname} drop column `c15`') tdSql.query(f'describe {dbname}.{tbname}') tdSql.checkRows(14) + #! TD-16422 + # tdSql.execute(f'alter table {dbname}.{tbname} add column c16 binary(10)') + # tdSql.query(f'describe {dbname}.{tbname}') + # tdSql.checkRows(15) + # tdSql.checkEqual(tdSql.queryResult[14][2],10) + # tdSql.execute(f'alter table {dbname}.{tbname} drop column c16') + + # tdSql.execute(f'alter table {dbname}.{tbname} add column c16 nchar(10)') + # tdSql.query(f'describe {dbname}.{tbname}') + # tdSql.checkRows(15) + # tdSql.checkEqual(tdSql.queryResult[14][2],10) + # tdSql.execute(f'alter table {dbname}.{tbname} drop column c16') + + tdSql.execute(f'alter table {dbname}.{tbname} modify column c12 binary(30)') tdSql.query(f'describe {dbname}.{tbname}') tdSql.checkData(12,2,30) diff --git a/tests/system-test/1-insert/insertWithMoreVgroup.py b/tests/system-test/1-insert/insertWithMoreVgroup.py index 7708ebb476..29c293c608 100644 --- a/tests/system-test/1-insert/insertWithMoreVgroup.py +++ b/tests/system-test/1-insert/insertWithMoreVgroup.py @@ -119,7 +119,7 @@ class TDTestCase: # tdLog.debug("spent %.2fs to create 1 stable and %d table, create speed is %.2f table/s... [OK]"% (spendTime,count,speedCreate)) return - def mutiThread_create_tables(self,host,dbname,stbname,vgroups,threadNumbers,childrowcount): + def mutiThread_create_tables(self,host,dbname,stbname,vgroups,threadNumbers,childcount): buildPath = self.getBuildPath() config = buildPath+ "../sim/dnode1/cfg/" @@ -128,7 +128,7 @@ class TDTestCase: tsql.execute("drop database if exists %s"%dbname) tsql.execute("create database %s vgroups %d"%(dbname,vgroups)) tsql.execute("use %s" %dbname) - count=int(childrowcount) + count=int(childcount) threads = [] for i in range(threadNumbers): tsql.execute("create stable %s%d(ts timestamp, c1 int, c2 binary(10)) tags(t1 int)"%(stbname,i)) @@ -264,19 +264,88 @@ class TDTestCase: speedCreate=count/spendTime tdLog.debug("spent %.2fs to create 1 stable and %d table, create speed is %.2f table/s... [OK]"% (spendTime,count,speedCreate)) return + + def checkData(self,dbname,stbname,stableCount,CtableCount,rowsPerSTable,): + tdSql.execute("use %s"%dbname) + tdSql.query("show stables") + tdSql.checkRows(stableCount) + tdSql.query("show tables") + tdSql.checkRows(CtableCount) + for i in range(stableCount): + tdSql.query("select count(*) from %s%d"%(stbname,i)) + tdSql.checkData(0,0,rowsPerSTable) + return + + # test case1 base def test_case1(self): + #stableCount=threadNumbersCtb + parameterDict = {'vgroups': 1, \ + 'threadNumbersCtb': 5, \ + 'threadNumbersIda': 5, \ + 'stableCount': 5, \ + 'tablesPerStb': 50, \ + 'rowsPerTable': 10, \ + 'dbname': 'db', \ + 'stbname': 'stb', \ + 'host': 'localhost', \ + 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 + tdLog.debug("-----create database and muti-thread create tables test------- ") #host,dbname,stbname,vgroups,threadNumbers,tcountStart,tcountStop #host, dbname, stbname, threadNumbers, chilCount, ts_start, childrowcount - self.mutiThread_create_tables(host="localhost",dbname="db",stbname="stb", vgroups=1, threadNumbers=5, childrowcount=50) - self.mutiThread_insert_data(host="localhost",dbname="db",stbname="stb", threadNumbers=5,chilCount=50,ts_start=self.ts,childrowcount=10) + self.mutiThread_create_tables( + host=parameterDict['host'], + dbname=parameterDict['dbname'], + stbname=parameterDict['stbname'], + vgroups=parameterDict['vgroups'], + threadNumbers=parameterDict['threadNumbersCtb'], + childcount=parameterDict['tablesPerStb']) - return + self.mutiThread_insert_data( + host=parameterDict['host'], + dbname=parameterDict['dbname'], + stbname=parameterDict['stbname'], + threadNumbers=parameterDict['threadNumbersIda'], + chilCount=parameterDict['tablesPerStb'], + ts_start=parameterDict['startTs'], + childrowcount=parameterDict['rowsPerTable']) + tableCount=parameterDict['threadNumbersCtb']*parameterDict['tablesPerStb'] + rowsPerStable=parameterDict['rowsPerTable']*parameterDict['tablesPerStb'] + self.checkData(dbname=parameterDict['dbname'],stbname=parameterDict['stbname'], stableCount=parameterDict['threadNumbersCtb'],CtableCount=tableCount,rowsPerSTable=rowsPerStable) + def test_case3(self): - self.taosBenchCreate("127.0.0.1","no","db1", "stb1", 1, 8, 1*10) + #stableCount=threadNumbersCtb + parameterDict = {'vgroups': 1, \ + 'threadNumbersCtb': 8, \ + 'stableCount': 5, \ + 'tablesPerStb': 10, \ + 'rowsPerTable': 100, \ + 'dbname': 'db1', \ + 'stbname': 'stb1', \ + 'host': 'localhost', \ + 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 + + self.taosBenchCreate( + parameterDict['host'], + "no", + parameterDict['dbname'], + parameterDict['stbname'], + parameterDict['vgroups'], + parameterDict['threadNumbersCtb'], + parameterDict['tablesPerStb']) + tableCount=parameterDict['threadNumbersCtb']*parameterDict['tablesPerStb'] + rowsPerStable=parameterDict['rowsPerTable']*parameterDict['tablesPerStb'] + + self.checkData( + dbname=parameterDict['dbname'], + stbname=parameterDict['stbname'], + stableCount=parameterDict['threadNumbersCtb'], + CtableCount=tableCount, + rowsPerSTable=rowsPerStable) + # self.taosBenchCreate("test209","no","db2", "stb2", 1, 8, 1*10000) # self.taosBenchCreate("chenhaoran02","no","db1", "stb1", 1, 8, 1*10000) @@ -320,14 +389,6 @@ class TDTestCase: # tdSql.execute("create qnode on dnode %s"%dnodeId) - - # self.taosBenchCreate("test209","no","db2", "stb2", 1, 8, 1*10000) - - # self.taosBenchCreate("chenhaoran02","no","db1", "stb1", 1, 8, 1*10000) - - # self.taosBenchCreate("db1", "stb1", 4, 5, 100*10000) - # self.taosBenchCreate("db1", "stb1", 1, 5, 100*10000) - # run case def run(self): diff --git a/tests/system-test/1-insert/manyVgroups.json b/tests/system-test/1-insert/manyVgroups.json index 5dea41476c..20ac320552 100644 --- a/tests/system-test/1-insert/manyVgroups.json +++ b/tests/system-test/1-insert/manyVgroups.json @@ -11,7 +11,7 @@ "confirm_parameter_prompt": "no", "insert_interval": 0, "interlace_rows": 0, - "num_of_records_per_req": 100, + "num_of_records_per_req": 100000, "databases": [ { "dbinfo": { @@ -29,7 +29,7 @@ "batch_create_tbl_num": 50000, "data_source": "rand", "insert_mode": "taosc", - "insert_rows": 1, + "insert_rows": 100, "interlace_rows": 0, "insert_interval": 0, "max_sql_len": 10000000, @@ -45,28 +45,28 @@ }, { "type": "DOUBLE", - "count": 100 + "count": 1 }, { "type": "BINARY", - "len": 400, - "count": 10 + "len": 40, + "count": 1 }, { "type": "nchar", - "len": 200, - "count": 20 + "len": 20, + "count": 1 } ], "tags": [ { "type": "TINYINT", - "count": 2 + "count": 1 }, { "type": "BINARY", "len": 16, - "count": 2 + "count": 1 } ] } diff --git a/tests/system-test/2-query/bottom.py b/tests/system-test/2-query/bottom.py index 008f59aa6a..1037b0a8f3 100644 --- a/tests/system-test/2-query/bottom.py +++ b/tests/system-test/2-query/bottom.py @@ -11,9 +11,12 @@ # -*- coding: utf-8 -*- +import random +import string from util.log import * from util.cases import * from util.sql import * +from util.common import * @@ -23,79 +26,89 @@ class TDTestCase: tdSql.init(conn.cursor()) self.rowNum = 10 + self.tbnum = 20 self.ts = 1537146000000 - - def run(self): + self.binary_str = 'taosdata' + self.nchar_str = '涛思数据' + def bottom_check_base(self): tdSql.prepare() - - tdSql.execute('''create table test(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double, - col7 bool, col8 binary(20), col9 nchar(20), col11 tinyint unsigned, col12 smallint unsigned, col13 int unsigned, col14 bigint unsigned) tags(loc nchar(20))''') - tdSql.execute("create table test1 using test tags('beijing')") + tdSql.execute('''create table stb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') + tdSql.execute("create table stb_1 using stb tags('beijing')") + column_list = ['col1','col2','col3','col4','col5','col6','col7','col8'] + error_column_list = ['col11','col12','col13'] + error_param_list = [0,101] for i in range(self.rowNum): - tdSql.execute("insert into test1 values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)" - % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1)) - - # bottom verifacation - tdSql.error("select bottom(ts, 10) from test") - tdSql.error("select bottom(col1, 0) from test") - tdSql.error("select bottom(col1, 101) from test") - tdSql.error("select bottom(col2, 0) from test") - tdSql.error("select bottom(col2, 101) from test") - tdSql.error("select bottom(col3, 0) from test") - tdSql.error("select bottom(col3, 101) from test") - tdSql.error("select bottom(col4, 0) from test") - tdSql.error("select bottom(col4, 101) from test") - tdSql.error("select bottom(col5, 0) from test") - tdSql.error("select bottom(col5, 101) from test") - tdSql.error("select bottom(col6, 0) from test") - tdSql.error("select bottom(col6, 101) from test") - tdSql.error("select bottom(col7, 10) from test") - tdSql.error("select bottom(col8, 10) from test") - tdSql.error("select bottom(col9, 10) from test") - - tdSql.query("select bottom(col1, 2) from test") + tdSql.execute(f"insert into stb_1 values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) + + for i in column_list: + tdSql.query(f'select bottom({i},2) from stb_1') + tdSql.checkRows(2) + tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) + for j in error_param_list: + tdSql.error(f'select bottom({i},{j}) from stb_1') + for i in error_column_list: + tdSql.error(f'select bottom({i},10) from stb_1') + tdSql.query("select ts,bottom(col1, 2),ts from stb_1 group by tbname") tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) - tdSql.query("select bottom(col2, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) - - tdSql.query("select bottom(col3, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) - - tdSql.query("select bottom(col4, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) - - tdSql.query("select bottom(col11, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) - - tdSql.query("select bottom(col12, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) - - tdSql.query("select bottom(col13, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) - - tdSql.query("select bottom(col13,50) from test") - tdSql.checkRows(10) - - tdSql.query("select bottom(col14, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) - tdSql.query("select ts,bottom(col1, 2) from test1") - tdSql.checkRows(2) - tdSql.query("select ts,bottom(col1, 2),ts from test group by tbname") - tdSql.checkRows(2) - - tdSql.query('select bottom(col2,1) from test interval(1y) order by col2') + tdSql.query('select bottom(col2,1) from stb_1 interval(1y) order by col2') tdSql.checkData(0,0,1) + tdSql.error('select * from stb_1 where bottom(col2,1)=1') + tdSql.execute('drop database db') + def bottom_check_distribute(self): + # prepare data for vgroup 4 + dbname = tdCom.getLongName(5, "letters") + stbname = tdCom.getLongName(5, "letters") + vgroup_num = 2 + child_table_num = 20 + tdSql.execute(f"create database if not exists {dbname} vgroups {vgroup_num}") + tdSql.execute(f'use {dbname}') + # build 20 child tables,every table insert 10 rows + tdSql.execute(f'''create table {stbname}(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') + for i in range(child_table_num): + tdSql.execute(f"create table {stbname}_{i} using {stbname} tags('beijing')") + tdSql.execute(f"insert into {stbname}_{i}(ts) values(%d)" % (self.ts - 1-i)) + column_list = ['col1','col2','col3','col4','col5','col6','col7','col8'] + error_column_list = ['col11','col12','col13'] + error_param_list = [0,101] + for i in [f'{stbname}', f'{dbname}.{stbname}']: + for j in column_list: + tdSql.query(f"select bottom({j},1) from {i}") + tdSql.checkRows(0) + tdSql.query('show tables') + vgroup_list = [] + for i in range(len(tdSql.queryResult)): + vgroup_list.append(tdSql.queryResult[i][6]) + vgroup_list_set = set(vgroup_list) - tdSql.error('select * from test where bottom(col2,1)=1') + for i in vgroup_list_set: + vgroups_num = vgroup_list.count(i) + if vgroups_num >=2: + tdLog.info(f'This scene with {vgroups_num} vgroups is ok!') + continue + else: + tdLog.exit(f'This scene does not meet the requirements with {vgroups_num} vgroup!\n') + for i in range(self.rowNum): + for j in range(child_table_num): + tdSql.execute(f"insert into {stbname}_{j} values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) + for i in column_list: + tdSql.query(f'select bottom({i},2) from {stbname}') + tdSql.checkRows(2) + tdSql.checkEqual(tdSql.queryResult,[(1,),(1,)]) + for j in error_param_list: + tdSql.error(f'select bottom({i},{j}) from {stbname}') + for i in error_column_list: + tdSql.error(f'select bottom({i},10) from {stbname}') + + tdSql.execute(f'drop database {dbname}') + def run(self): + + self.bottom_check_base() + self.bottom_check_distribute() def stop(self): diff --git a/tests/system-test/2-query/distribute_agg_apercentile.py b/tests/system-test/2-query/distribute_agg_apercentile.py new file mode 100644 index 0000000000..fd1455ce16 --- /dev/null +++ b/tests/system-test/2-query/distribute_agg_apercentile.py @@ -0,0 +1,198 @@ +from util.log import * +from util.cases import * +from util.sql import * +import numpy as np +import random + + +class TDTestCase: + updatecfgDict = {'debugFlag': 143 ,"cDebugFlag":143,"uDebugFlag":143 ,"rpcDebugFlag":143 , "tmrDebugFlag":143 , + "jniDebugFlag":143 ,"simDebugFlag":143,"dDebugFlag":143, "dDebugFlag":143,"vDebugFlag":143,"mDebugFlag":143,"qDebugFlag":143, + "wDebugFlag":143,"sDebugFlag":143,"tsdbDebugFlag":143,"tqDebugFlag":143 ,"fsDebugFlag":143 ,"fnDebugFlag":143, + "maxTablesPerVnode":2 ,"minTablesPerVnode":2,"tableIncStepPerVnode":2 } + + def init(self, conn, logSql): + tdLog.debug("start to execute %s" % __file__) + tdSql.init(conn.cursor()) + self.vnode_disbutes = None + self.ts = 1537146000000 + + def prepare_datas_of_distribute(self): + + # prepate datas for 20 tables distributed at different vgroups + tdSql.execute("create database if not exists testdb keep 3650 duration 1000 vgroups 5") + tdSql.execute(" use testdb ") + tdSql.execute( + '''create table stb1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + tags (t0 timestamp, t1 int, t2 bigint, t3 smallint, t4 tinyint, t5 float, t6 double, t7 bool, t8 binary(16),t9 nchar(32)) + ''' + ) + + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(20): + tdSql.execute(f'create table ct{i+1} using stb1 tags ( now(), {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, "binary{i}", "nchar{i}" )') + + for i in range(9): + tdSql.execute( + f"insert into ct1 values ( now()-{i*10}s, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + tdSql.execute( + f"insert into ct4 values ( now()-{i*90}d, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + + for i in range(1,21): + if i ==1 or i == 4: + continue + else: + tbname = "ct"+f'{i}' + for j in range(9): + tdSql.execute( + f"insert into {tbname} values ( now()-{(i+j)*10}s, {1*(j+i)}, {11111*(j+i)}, {111*(j+i)}, {11*(j)}, {1.11*(j+i)}, {11.11*(j+i)}, {(j+i)%2}, 'binary{j}', 'nchar{j}', now()+{1*j}a )" + ) + tdSql.execute("insert into ct1 values (now()-45s, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar0', now()+8a )") + tdSql.execute("insert into ct1 values (now()+10s, 9, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+15s, 9, -99999, -999, -99, -9.99, NULL, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+20s, 9, -99999, -999, NULL, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + + tdSql.execute("insert into ct4 values (now()-810d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()-400d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()+90d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + + tdSql.execute( + f'''insert into t1 values + ( '2020-04-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2020-10-21 01:01:01.000', 1, 11111, 111, 11, 1.11, 11.11, 1, "binary1", "nchar1", now()+1a ) + ( '2020-12-31 01:01:01.000', 2, 22222, 222, 22, 2.22, 22.22, 0, "binary2", "nchar2", now()+2a ) + ( '2021-01-01 01:01:06.000', 3, 33333, 333, 33, 3.33, 33.33, 0, "binary3", "nchar3", now()+3a ) + ( '2021-05-07 01:01:10.000', 4, 44444, 444, 44, 4.44, 44.44, 1, "binary4", "nchar4", now()+4a ) + ( '2021-07-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2021-09-30 01:01:16.000', 5, 55555, 555, 55, 5.55, 55.55, 0, "binary5", "nchar5", now()+5a ) + ( '2022-02-01 01:01:20.000', 6, 66666, 666, 66, 6.66, 66.66, 1, "binary6", "nchar6", now()+6a ) + ( '2022-10-28 01:01:26.000', 7, 00000, 000, 00, 0.00, 00.00, 1, "binary7", "nchar7", "1970-01-01 08:00:00.000" ) + ( '2022-12-01 01:01:30.000', 8, -88888, -888, -88, -8.88, -88.88, 0, "binary8", "nchar8", "1969-01-01 01:00:00.000" ) + ( '2022-12-31 01:01:36.000', 9, -99999999999999999, -999, -99, -9.99, -999999999999999999999.99, 1, "binary9", "nchar9", "1900-01-01 00:00:00.000" ) + ( '2023-02-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ''' + ) + + tdLog.info(" prepare data for distributed_aggregate done! ") + + def check_distribute_datas(self): + # get vgroup_ids of all + tdSql.query("show vgroups ") + vgroups = tdSql.queryResult + + vnode_tables={} + + for vgroup_id in vgroups: + vnode_tables[vgroup_id[0]]=[] + + + # check sub_table of per vnode ,make sure sub_table has been distributed + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + vnode_tables[table_name[6]].append(table_name[0]) + self.vnode_disbutes = vnode_tables + + count = 0 + for k ,v in vnode_tables.items(): + if len(v)>=2: + count+=1 + if count < 2: + tdLog.exit(" the datas of all not satisfy sub_table has been distributed ") + + def distribute_agg_query(self): + # basic filter + tdSql.query("select apercentile(c1 , 20) from stb1 where c1 is null") + tdSql.checkRows(0) + + tdSql.query("select apercentile(c1 , 20) from stb1 where t1=1") + tdSql.checkData(0,0,2.800000000) + + tdSql.query("select apercentile(c1+c2 ,100) from stb1 where c1 =1 ") + tdSql.checkData(0,0,11112.000000000) + + tdSql.query("select apercentile(c1 ,10 ) from stb1 where tbname=\"ct2\"") + tdSql.checkData(0,0,2.000000000) + + tdSql.query("select apercentile(c1,20) from stb1 partition by tbname") + tdSql.checkRows(20) + + tdSql.query("select apercentile(c1,20) from stb1 where t1> 4 partition by tbname") + tdSql.checkRows(15) + + # union all + tdSql.query("select apercentile(c1,20) from stb1 union all select apercentile(c1,20) from stb1 ") + tdSql.checkRows(2) + tdSql.checkData(0,0,7.389181281) + + # join + + tdSql.execute(" create database if not exists db ") + tdSql.execute(" use db ") + tdSql.execute(" create stable st (ts timestamp , c1 int ,c2 float) tags(t1 int) ") + tdSql.execute(" create table tb1 using st tags(1) ") + tdSql.execute(" create table tb2 using st tags(2) ") + + + for i in range(10): + ts = i*10 + self.ts + tdSql.execute(f" insert into tb1 values({ts},{i},{i}.0)") + tdSql.execute(f" insert into tb2 values({ts},{i},{i}.0)") + + tdSql.query("select apercentile(tb1.c1,100), apercentile(tb2.c2,100) from tb1, tb2 where tb1.ts=tb2.ts") + tdSql.checkRows(1) + tdSql.checkData(0,0,9.000000000) + tdSql.checkData(0,0,9.000000000) + + # group by + tdSql.execute(" use testdb ") + tdSql.query(" select max(c1),c1 from stb1 group by t1 ") + tdSql.checkRows(20) + tdSql.query(" select max(c1),c1 from stb1 group by c1 ") + tdSql.checkRows(30) + tdSql.query(" select max(c1),c2 from stb1 group by c2 ") + tdSql.checkRows(31) + + # partition by tbname or partition by tag + tdSql.query("select apercentile(c1 ,10)from stb1 partition by tbname") + query_data = tdSql.queryResult + + # nest query for support max + tdSql.query("select apercentile(c2+2,10)+1 from (select max(c1) c2 from stb1)") + tdSql.checkData(0,0,31.000000000) + tdSql.query("select apercentile(c1+2,10)+1 as c2 from (select ts ,c1 ,c2 from stb1)") + tdSql.checkData(0,0,7.560701700) + tdSql.query("select apercentile(a+2,10)+1 as c2 from (select ts ,abs(c1) a ,c2 from stb1)") + tdSql.checkData(0,0,7.560701700) + + # mixup with other functions + tdSql.query("select max(c1),count(c1),last(c2,c3),spread(c1), apercentile(c1,10) from stb1") + tdSql.checkData(0,0,28) + tdSql.checkData(0,1,184) + tdSql.checkData(0,2,-99999) + tdSql.checkData(0,3,-999) + tdSql.checkData(0,4,28.000000000) + tdSql.checkData(0,5,4.560701700) + + def run(self): + + self.prepare_datas_of_distribute() + self.check_distribute_datas() + self.distribute_agg_query() + + + def stop(self): + tdSql.close() + tdLog.success("%s successfully executed" % __file__) + +tdCases.addWindows(__file__, TDTestCase()) +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/system-test/2-query/distribute_agg_count.py b/tests/system-test/2-query/distribute_agg_count.py new file mode 100644 index 0000000000..2ac9c86df0 --- /dev/null +++ b/tests/system-test/2-query/distribute_agg_count.py @@ -0,0 +1,296 @@ +from util.log import * +from util.cases import * +from util.sql import * +import numpy as np +import random + + +class TDTestCase: + updatecfgDict = {'debugFlag': 143 ,"cDebugFlag":143,"uDebugFlag":143 ,"rpcDebugFlag":143 , "tmrDebugFlag":143 , + "jniDebugFlag":143 ,"simDebugFlag":143,"dDebugFlag":143, "dDebugFlag":143,"vDebugFlag":143,"mDebugFlag":143,"qDebugFlag":143, + "wDebugFlag":143,"sDebugFlag":143,"tsdbDebugFlag":143,"tqDebugFlag":143 ,"fsDebugFlag":143 ,"fnDebugFlag":143, + "maxTablesPerVnode":2 ,"minTablesPerVnode":2,"tableIncStepPerVnode":2 } + + def init(self, conn, logSql): + tdLog.debug("start to execute %s" % __file__) + tdSql.init(conn.cursor()) + self.vnode_disbutes = None + self.ts = 1537146000000 + + + def check_count_functions(self, tbname , col_name): + + max_sql = f"select count({col_name}) from {tbname};" + + same_sql = f"select sum(c) from (select {col_name} ,1 as c from {tbname} where {col_name} is not null) " + + tdSql.query(max_sql) + max_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if max_result !=same_result: + tdLog.exit(" count function work not as expected, sql : %s "% max_sql) + else: + tdLog.info(" count function work as expected, sql : %s "% max_sql) + + + def prepare_datas_of_distribute(self): + + # prepate datas for 20 tables distributed at different vgroups + tdSql.execute("create database if not exists testdb keep 3650 duration 1000 vgroups 5") + tdSql.execute(" use testdb ") + tdSql.execute( + '''create table stb1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + tags (t0 timestamp, t1 int, t2 bigint, t3 smallint, t4 tinyint, t5 float, t6 double, t7 bool, t8 binary(16),t9 nchar(32)) + ''' + ) + + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(20): + tdSql.execute(f'create table ct{i+1} using stb1 tags ( now(), {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, "binary{i}", "nchar{i}" )') + + for i in range(9): + tdSql.execute( + f"insert into ct1 values ( now()-{i*10}s, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + tdSql.execute( + f"insert into ct4 values ( now()-{i*90}d, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + + for i in range(1,21): + if i ==1 or i == 4: + continue + else: + tbname = "ct"+f'{i}' + for j in range(9): + tdSql.execute( + f"insert into {tbname} values ( now()-{(i+j)*10}s, {1*(j+i)}, {11111*(j+i)}, {111*(j+i)}, {11*(j)}, {1.11*(j+i)}, {11.11*(j+i)}, {(j+i)%2}, 'binary{j}', 'nchar{j}', now()+{1*j}a )" + ) + tdSql.execute("insert into ct1 values (now()-45s, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar0', now()+8a )") + tdSql.execute("insert into ct1 values (now()+10s, 9, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+15s, 9, -99999, -999, -99, -9.99, NULL, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+20s, 9, -99999, -999, NULL, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + + tdSql.execute("insert into ct4 values (now()-810d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()-400d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()+90d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + + tdSql.execute( + f'''insert into t1 values + ( '2020-04-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2020-10-21 01:01:01.000', 1, 11111, 111, 11, 1.11, 11.11, 1, "binary1", "nchar1", now()+1a ) + ( '2020-12-31 01:01:01.000', 2, 22222, 222, 22, 2.22, 22.22, 0, "binary2", "nchar2", now()+2a ) + ( '2021-01-01 01:01:06.000', 3, 33333, 333, 33, 3.33, 33.33, 0, "binary3", "nchar3", now()+3a ) + ( '2021-05-07 01:01:10.000', 4, 44444, 444, 44, 4.44, 44.44, 1, "binary4", "nchar4", now()+4a ) + ( '2021-07-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2021-09-30 01:01:16.000', 5, 55555, 555, 55, 5.55, 55.55, 0, "binary5", "nchar5", now()+5a ) + ( '2022-02-01 01:01:20.000', 6, 66666, 666, 66, 6.66, 66.66, 1, "binary6", "nchar6", now()+6a ) + ( '2022-10-28 01:01:26.000', 7, 00000, 000, 00, 0.00, 00.00, 1, "binary7", "nchar7", "1970-01-01 08:00:00.000" ) + ( '2022-12-01 01:01:30.000', 8, -88888, -888, -88, -8.88, -88.88, 0, "binary8", "nchar8", "1969-01-01 01:00:00.000" ) + ( '2022-12-31 01:01:36.000', 9, -99999999999999999, -999, -99, -9.99, -999999999999999999999.99, 1, "binary9", "nchar9", "1900-01-01 00:00:00.000" ) + ( '2023-02-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ''' + ) + + tdLog.info(" prepare data for distributed_aggregate done! ") + + def check_distribute_datas(self): + # get vgroup_ids of all + tdSql.query("show vgroups ") + vgroups = tdSql.queryResult + + vnode_tables={} + + for vgroup_id in vgroups: + vnode_tables[vgroup_id[0]]=[] + + + # check sub_table of per vnode ,make sure sub_table has been distributed + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + vnode_tables[table_name[6]].append(table_name[0]) + self.vnode_disbutes = vnode_tables + + count = 0 + for k ,v in vnode_tables.items(): + if len(v)>=2: + count+=1 + if count < 2: + tdLog.exit(" the datas of all not satisfy sub_table has been distributed ") + + def check_count_distribute_diff_vnode(self,col_name): + + vgroup_ids = [] + for k ,v in self.vnode_disbutes.items(): + if len(v)>=2: + vgroup_ids.append(k) + + distribute_tbnames = [] + + for vgroup_id in vgroup_ids: + vnode_tables = self.vnode_disbutes[vgroup_id] + distribute_tbnames.append(random.sample(vnode_tables,1)[0]) + tbname_ins = "" + for tbname in distribute_tbnames: + tbname_ins += "'%s' ,"%tbname + + tbname_filters = tbname_ins[:-1] + + max_sql = f"select count({col_name}) from stb1 where tbname in ({tbname_filters});" + + same_sql = f"select sum(c) from (select {col_name} ,1 as c from stb1 where tbname in ({tbname_filters}) and {col_name} is not null) " + + tdSql.query(max_sql) + max_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if max_result !=same_result: + tdLog.exit(" count function work not as expected, sql : %s "% max_sql) + else: + tdLog.info(" count function work as expected, sql : %s "% max_sql) + + def check_count_status(self): + # check max function work status + + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + tablenames.append(table_name[0]) + + tdSql.query("desc stb1") + col_names = tdSql.queryResult + + colnames = [] + for col_name in col_names: + if col_name[1] in ["INT" ,"BIGINT" ,"SMALLINT" ,"TINYINT" , "FLOAT" ,"DOUBLE"]: + colnames.append(col_name[0]) + + for tablename in tablenames: + for colname in colnames: + self.check_count_functions(tablename,colname) + + # check max function for different vnode + + for colname in colnames: + if colname.startswith("c"): + self.check_count_distribute_diff_vnode(colname) + else: + # self.check_count_distribute_diff_vnode(colname) # bug for tag + pass + + + def distribute_agg_query(self): + # basic filter + tdSql.query("select count(c1) from stb1 ") + tdSql.checkData(0,0,184) + + tdSql.query("select count(c1) from stb1 where t1=1") + tdSql.checkData(0,0,9) + + tdSql.query("select count(c1+c2) from stb1 where c1 =1 ") + tdSql.checkData(0,0,2) + + tdSql.query("select count(c1) from stb1 where tbname=\"ct2\"") + tdSql.checkData(0,0,9) + + tdSql.query("select count(c1) from stb1 partition by tbname") + tdSql.checkRows(20) + + tdSql.query("select count(c1) from stb1 where t1> 4 partition by tbname") + tdSql.checkRows(15) + + # union all + tdSql.query("select count(c1) from stb1 union all select count(c1) from stb1 ") + tdSql.checkRows(2) + tdSql.checkData(0,0,184) + + # join + + tdSql.execute(" create database if not exists db ") + tdSql.execute(" use db ") + tdSql.execute(" create stable st (ts timestamp , c1 int ,c2 float) tags(t1 int) ") + tdSql.execute(" create table tb1 using st tags(1) ") + tdSql.execute(" create table tb2 using st tags(2) ") + + + for i in range(10): + ts = i*10 + self.ts + tdSql.execute(f" insert into tb1 values({ts},{i},{i}.0)") + tdSql.execute(f" insert into tb2 values({ts},{i},{i}.0)") + + tdSql.query("select count(tb1.c1), count(tb2.c2) from tb1, tb2 where tb1.ts=tb2.ts") + tdSql.checkRows(1) + tdSql.checkData(0,0,10) + tdSql.checkData(0,1,10) + + # group by + tdSql.execute(" use testdb ") + + tdSql.query(" select count(*) from stb1 ") + tdSql.checkData(0,0,187) + tdSql.query(" select count(*) from stb1 group by t1 ") + tdSql.checkRows(20) + tdSql.query(" select count(*) from stb1 group by c1 ") + tdSql.checkRows(30) + tdSql.query(" select count(*) from stb1 group by c2 ") + tdSql.checkRows(31) + + # partition by tbname or partition by tag + tdSql.query("select max(c1),tbname from stb1 partition by tbname") + query_data = tdSql.queryResult + + for row in query_data: + tbname = row[1] + tdSql.query(" select max(c1) from %s "%tbname) + tdSql.checkData(0,0,row[0]) + + tdSql.query("select max(c1),tbname from stb1 partition by t1") + query_data = tdSql.queryResult + + for row in query_data: + tbname = row[1] + tdSql.query(" select max(c1) from %s "%tbname) + tdSql.checkData(0,0,row[0]) + + # nest query for support max + tdSql.query("select abs(c2+2)+1 from (select count(c1) c2 from stb1)") + tdSql.checkData(0,0,187.000000000) + tdSql.query("select count(c1+2) as c2 from (select ts ,c1 ,c2 from stb1)") + tdSql.checkData(0,0,184) + tdSql.query("select count(a+2) as c2 from (select ts ,abs(c1) a ,c2 from stb1)") + tdSql.checkData(0,0,184) + + # mixup with other functions + tdSql.query("select max(c1),count(c1),last(c2,c3) from stb1") + tdSql.checkData(0,0,28) + tdSql.checkData(0,1,184) + tdSql.checkData(0,2,-99999) + tdSql.checkData(0,3,-999) + + def run(self): + + self.prepare_datas_of_distribute() + self.check_distribute_datas() + self.check_count_status() + self.distribute_agg_query() + + + def stop(self): + tdSql.close() + tdLog.success("%s successfully executed" % __file__) + +tdCases.addWindows(__file__, TDTestCase()) +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/system-test/2-query/distribute_agg_max.py b/tests/system-test/2-query/distribute_agg_max.py new file mode 100644 index 0000000000..ae0ab5aafa --- /dev/null +++ b/tests/system-test/2-query/distribute_agg_max.py @@ -0,0 +1,293 @@ +from util.log import * +from util.cases import * +from util.sql import * +import numpy as np +import random + + +class TDTestCase: + updatecfgDict = {'debugFlag': 143 ,"cDebugFlag":143,"uDebugFlag":143 ,"rpcDebugFlag":143 , "tmrDebugFlag":143 , + "jniDebugFlag":143 ,"simDebugFlag":143,"dDebugFlag":143, "dDebugFlag":143,"vDebugFlag":143,"mDebugFlag":143,"qDebugFlag":143, + "wDebugFlag":143,"sDebugFlag":143,"tsdbDebugFlag":143,"tqDebugFlag":143 ,"fsDebugFlag":143 ,"fnDebugFlag":143, + "maxTablesPerVnode":2 ,"minTablesPerVnode":2,"tableIncStepPerVnode":2 } + + def init(self, conn, logSql): + tdLog.debug("start to execute %s" % __file__) + tdSql.init(conn.cursor()) + self.vnode_disbutes = None + self.ts = 1537146000000 + + + def check_max_functions(self, tbname , col_name): + + max_sql = f"select max({col_name}) from {tbname};" + + same_sql = f"select {col_name} from {tbname} order by {col_name} desc limit 1" + + tdSql.query(max_sql) + max_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if max_result !=same_result: + tdLog.exit(" max function work not as expected, sql : %s "% max_sql) + else: + tdLog.info(" max function work as expected, sql : %s "% max_sql) + + + def prepare_datas_of_distribute(self): + + # prepate datas for 20 tables distributed at different vgroups + tdSql.execute("create database if not exists testdb keep 3650 duration 1000 vgroups 5") + tdSql.execute(" use testdb ") + tdSql.execute( + '''create table stb1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + tags (t0 timestamp, t1 int, t2 bigint, t3 smallint, t4 tinyint, t5 float, t6 double, t7 bool, t8 binary(16),t9 nchar(32)) + ''' + ) + + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(20): + tdSql.execute(f'create table ct{i+1} using stb1 tags ( now(), {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, "binary{i}", "nchar{i}" )') + + for i in range(9): + tdSql.execute( + f"insert into ct1 values ( now()-{i*10}s, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + tdSql.execute( + f"insert into ct4 values ( now()-{i*90}d, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + + for i in range(1,21): + if i ==1 or i == 4: + continue + else: + tbname = "ct"+f'{i}' + for j in range(9): + tdSql.execute( + f"insert into {tbname} values ( now()-{(i+j)*10}s, {1*(j+i)}, {11111*(j+i)}, {111*(j+i)}, {11*(j)}, {1.11*(j+i)}, {11.11*(j+i)}, {(j+i)%2}, 'binary{j}', 'nchar{j}', now()+{1*j}a )" + ) + tdSql.execute("insert into ct1 values (now()-45s, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar0', now()+8a )") + tdSql.execute("insert into ct1 values (now()+10s, 9, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+15s, 9, -99999, -999, -99, -9.99, NULL, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+20s, 9, -99999, -999, NULL, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + + tdSql.execute("insert into ct4 values (now()-810d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()-400d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()+90d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + + tdSql.execute( + f'''insert into t1 values + ( '2020-04-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2020-10-21 01:01:01.000', 1, 11111, 111, 11, 1.11, 11.11, 1, "binary1", "nchar1", now()+1a ) + ( '2020-12-31 01:01:01.000', 2, 22222, 222, 22, 2.22, 22.22, 0, "binary2", "nchar2", now()+2a ) + ( '2021-01-01 01:01:06.000', 3, 33333, 333, 33, 3.33, 33.33, 0, "binary3", "nchar3", now()+3a ) + ( '2021-05-07 01:01:10.000', 4, 44444, 444, 44, 4.44, 44.44, 1, "binary4", "nchar4", now()+4a ) + ( '2021-07-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2021-09-30 01:01:16.000', 5, 55555, 555, 55, 5.55, 55.55, 0, "binary5", "nchar5", now()+5a ) + ( '2022-02-01 01:01:20.000', 6, 66666, 666, 66, 6.66, 66.66, 1, "binary6", "nchar6", now()+6a ) + ( '2022-10-28 01:01:26.000', 7, 00000, 000, 00, 0.00, 00.00, 1, "binary7", "nchar7", "1970-01-01 08:00:00.000" ) + ( '2022-12-01 01:01:30.000', 8, -88888, -888, -88, -8.88, -88.88, 0, "binary8", "nchar8", "1969-01-01 01:00:00.000" ) + ( '2022-12-31 01:01:36.000', 9, -99999999999999999, -999, -99, -9.99, -999999999999999999999.99, 1, "binary9", "nchar9", "1900-01-01 00:00:00.000" ) + ( '2023-02-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ''' + ) + + tdLog.info(" prepare data for distributed_aggregate done! ") + + def check_distribute_datas(self): + # get vgroup_ids of all + tdSql.query("show vgroups ") + vgroups = tdSql.queryResult + + vnode_tables={} + + for vgroup_id in vgroups: + vnode_tables[vgroup_id[0]]=[] + + + # check sub_table of per vnode ,make sure sub_table has been distributed + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + vnode_tables[table_name[6]].append(table_name[0]) + self.vnode_disbutes = vnode_tables + + count = 0 + for k ,v in vnode_tables.items(): + if len(v)>=2: + count+=1 + if count < 2: + tdLog.exit(" the datas of all not satisfy sub_table has been distributed ") + + def check_max_distribute_diff_vnode(self,col_name): + + vgroup_ids = [] + for k ,v in self.vnode_disbutes.items(): + if len(v)>=2: + vgroup_ids.append(k) + + distribute_tbnames = [] + + for vgroup_id in vgroup_ids: + vnode_tables = self.vnode_disbutes[vgroup_id] + distribute_tbnames.append(random.sample(vnode_tables,1)[0]) + tbname_ins = "" + for tbname in distribute_tbnames: + tbname_ins += "'%s' ,"%tbname + + tbname_filters = tbname_ins[:-1] + + max_sql = f"select max({col_name}) from stb1 where tbname in ({tbname_filters});" + + same_sql = f"select {col_name} from stb1 where tbname in ({tbname_filters}) order by {col_name} desc limit 1" + + tdSql.query(max_sql) + max_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if max_result !=same_result: + tdLog.exit(" max function work not as expected, sql : %s "% max_sql) + else: + tdLog.info(" max function work as expected, sql : %s "% max_sql) + + def check_max_status(self): + # check max function work status + + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + tablenames.append(table_name[0]) + + tdSql.query("desc stb1") + col_names = tdSql.queryResult + + colnames = [] + for col_name in col_names: + if col_name[1] in ["INT" ,"BIGINT" ,"SMALLINT" ,"TINYINT" , "FLOAT" ,"DOUBLE"]: + colnames.append(col_name[0]) + + for tablename in tablenames: + for colname in colnames: + self.check_max_functions(tablename,colname) + + # check max function for different vnode + + for colname in colnames: + if colname.startswith("c"): + self.check_max_distribute_diff_vnode(colname) + else: + # self.check_max_distribute_diff_vnode(colname) # bug for tag + pass + + + def distribute_agg_query(self): + # basic filter + tdSql.query("select max(c1) from stb1 where c1 is null") + tdSql.checkRows(0) + + tdSql.query("select max(c1) from stb1 where t1=1") + tdSql.checkData(0,0,10) + + tdSql.query("select max(c1+c2) from stb1 where c1 =1 ") + tdSql.checkData(0,0,11112.000000000) + + tdSql.query("select max(c1) from stb1 where tbname=\"ct2\"") + tdSql.checkData(0,0,10) + + tdSql.query("select max(c1) from stb1 partition by tbname") + tdSql.checkRows(20) + + tdSql.query("select max(c1) from stb1 where t1> 4 partition by tbname") + tdSql.checkRows(15) + + # union all + tdSql.query("select max(c1) from stb1 union all select max(c1) from stb1 ") + tdSql.checkRows(2) + tdSql.checkData(0,0,28) + + # join + + tdSql.execute(" create database if not exists db ") + tdSql.execute(" use db ") + tdSql.execute(" create stable st (ts timestamp , c1 int ,c2 float) tags(t1 int) ") + tdSql.execute(" create table tb1 using st tags(1) ") + tdSql.execute(" create table tb2 using st tags(2) ") + + + for i in range(10): + ts = i*10 + self.ts + tdSql.execute(f" insert into tb1 values({ts},{i},{i}.0)") + tdSql.execute(f" insert into tb2 values({ts},{i},{i}.0)") + + tdSql.query("select max(tb1.c1), tb2.c2 from tb1, tb2 where tb1.ts=tb2.ts") + tdSql.checkRows(1) + tdSql.checkData(0,0,9) + tdSql.checkData(0,0,9.00000) + + # group by + tdSql.execute(" use testdb ") + tdSql.query(" select max(c1),c1 from stb1 group by t1 ") + tdSql.checkRows(20) + tdSql.query(" select max(c1),c1 from stb1 group by c1 ") + tdSql.checkRows(30) + tdSql.query(" select max(c1),c2 from stb1 group by c2 ") + tdSql.checkRows(31) + + # partition by tbname or partition by tag + tdSql.query("select max(c1),tbname from stb1 partition by tbname") + query_data = tdSql.queryResult + + for row in query_data: + tbname = row[1] + tdSql.query(" select max(c1) from %s "%tbname) + tdSql.checkData(0,0,row[0]) + + tdSql.query("select max(c1),tbname from stb1 partition by t1") + query_data = tdSql.queryResult + + for row in query_data: + tbname = row[1] + tdSql.query(" select max(c1) from %s "%tbname) + tdSql.checkData(0,0,row[0]) + + # nest query for support max + tdSql.query("select abs(c2+2)+1 from (select max(c1) c2 from stb1)") + tdSql.checkData(0,0,31.000000000) + tdSql.query("select max(c1+2)+1 as c2 from (select ts ,c1 ,c2 from stb1)") + tdSql.checkData(0,0,31.000000000) + tdSql.query("select max(a+2)+1 as c2 from (select ts ,abs(c1) a ,c2 from stb1)") + tdSql.checkData(0,0,31.000000000) + + # mixup with other functions + tdSql.query("select max(c1),count(c1),last(c2,c3) from stb1") + tdSql.checkData(0,0,28) + tdSql.checkData(0,1,184) + tdSql.checkData(0,2,-99999) + tdSql.checkData(0,3,-999) + + def run(self): + + self.prepare_datas_of_distribute() + self.check_distribute_datas() + self.check_max_status() + self.distribute_agg_query() + + + def stop(self): + tdSql.close() + tdLog.success("%s successfully executed" % __file__) + +tdCases.addWindows(__file__, TDTestCase()) +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/system-test/2-query/distribute_agg_min.py b/tests/system-test/2-query/distribute_agg_min.py new file mode 100644 index 0000000000..8a458c74df --- /dev/null +++ b/tests/system-test/2-query/distribute_agg_min.py @@ -0,0 +1,294 @@ +from util.log import * +from util.cases import * +from util.sql import * +import numpy as np +import random + + +class TDTestCase: + updatecfgDict = {'debugFlag': 143 ,"cDebugFlag":143,"uDebugFlag":143 ,"rpcDebugFlag":143 , "tmrDebugFlag":143 , + "jniDebugFlag":143 ,"simDebugFlag":143,"dDebugFlag":143, "dDebugFlag":143,"vDebugFlag":143,"mDebugFlag":143,"qDebugFlag":143, + "wDebugFlag":143,"sDebugFlag":143,"tsdbDebugFlag":143,"tqDebugFlag":143 ,"fsDebugFlag":143 ,"fnDebugFlag":143, + "maxTablesPerVnode":2 ,"minTablesPerVnode":2,"tableIncStepPerVnode":2 } + + def init(self, conn, logSql): + tdLog.debug("start to execute %s" % __file__) + tdSql.init(conn.cursor()) + self.vnode_disbutes = None + self.ts = 1537146000000 + + + def check_min_functions(self, tbname , col_name): + + min_sql = f"select min({col_name}) from {tbname};" + + same_sql = f"select {col_name} from {tbname} where {col_name} is not null order by {col_name} asc limit 1" + + tdSql.query(min_sql) + min_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if min_result !=same_result: + tdLog.exit(" min function work not as expected, sql : %s "% min_sql) + else: + tdLog.info(" min function work as expected, sql : %s "% min_sql) + + + def prepare_datas_of_distribute(self): + + # prepate datas for 20 tables distributed at different vgroups + tdSql.execute("create database if not exists testdb keep 3650 duration 1000 vgroups 5") + tdSql.execute(" use testdb ") + tdSql.execute( + '''create table stb1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + tags (t0 timestamp, t1 int, t2 bigint, t3 smallint, t4 tinyint, t5 float, t6 double, t7 bool, t8 binary(16),t9 nchar(32)) + ''' + ) + + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(20): + tdSql.execute(f'create table ct{i+1} using stb1 tags ( now(), {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, "binary{i}", "nchar{i}" )') + + for i in range(9): + tdSql.execute( + f"insert into ct1 values ( now()-{i*10}s, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + tdSql.execute( + f"insert into ct4 values ( now()-{i*90}d, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + + for i in range(1,21): + if i ==1 or i == 4: + continue + else: + tbname = "ct"+f'{i}' + for j in range(9): + tdSql.execute( + f"insert into {tbname} values ( now()-{(i+j)*10}s, {1*(j+i)}, {11111*(j+i)}, {111*(j+i)}, {11*(j)}, {1.11*(j+i)}, {11.11*(j+i)}, {(j+i)%2}, 'binary{j}', 'nchar{j}', now()+{1*j}a )" + ) + tdSql.execute("insert into ct1 values (now()-45s, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar0', now()+8a )") + tdSql.execute("insert into ct1 values (now()+10s, 9, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+15s, 9, -99999, -999, -99, -9.99, NULL, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+20s, 9, -99999, -999, NULL, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + + tdSql.execute("insert into ct4 values (now()-810d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()-400d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()+90d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + + tdSql.execute( + f'''insert into t1 values + ( '2020-04-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2020-10-21 01:01:01.000', 1, 11111, 111, 11, 1.11, 11.11, 1, "binary1", "nchar1", now()+1a ) + ( '2020-12-31 01:01:01.000', 2, 22222, 222, 22, 2.22, 22.22, 0, "binary2", "nchar2", now()+2a ) + ( '2021-01-01 01:01:06.000', 3, 33333, 333, 33, 3.33, 33.33, 0, "binary3", "nchar3", now()+3a ) + ( '2021-05-07 01:01:10.000', 4, 44444, 444, 44, 4.44, 44.44, 1, "binary4", "nchar4", now()+4a ) + ( '2021-07-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2021-09-30 01:01:16.000', 5, 55555, 555, 55, 5.55, 55.55, 0, "binary5", "nchar5", now()+5a ) + ( '2022-02-01 01:01:20.000', 6, 66666, 666, 66, 6.66, 66.66, 1, "binary6", "nchar6", now()+6a ) + ( '2022-10-28 01:01:26.000', 7, 00000, 000, 00, 0.00, 00.00, 1, "binary7", "nchar7", "1970-01-01 08:00:00.000" ) + ( '2022-12-01 01:01:30.000', 8, -88888, -888, -88, -8.88, -88.88, 0, "binary8", "nchar8", "1969-01-01 01:00:00.000" ) + ( '2022-12-31 01:01:36.000', 9, -99999999999999999, -999, -99, -9.99, -999999999999999999999.99, 1, "binary9", "nchar9", "1900-01-01 00:00:00.000" ) + ( '2023-02-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ''' + ) + + tdLog.info(" prepare data for distributed_aggregate done! ") + + def check_distribute_datas(self): + # get vgroup_ids of all + tdSql.query("show vgroups ") + vgroups = tdSql.queryResult + + vnode_tables={} + + for vgroup_id in vgroups: + vnode_tables[vgroup_id[0]]=[] + + + # check sub_table of per vnode ,make sure sub_table has been distributed + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + vnode_tables[table_name[6]].append(table_name[0]) + self.vnode_disbutes = vnode_tables + + count = 0 + for k ,v in vnode_tables.items(): + if len(v)>=2: + count+=1 + if count < 2: + tdLog.exit(" the datas of all not satisfy sub_table has been distributed ") + + def check_min_distribute_diff_vnode(self,col_name): + + vgroup_ids = [] + for k ,v in self.vnode_disbutes.items(): + if len(v)>=2: + vgroup_ids.append(k) + + distribute_tbnames = [] + + for vgroup_id in vgroup_ids: + vnode_tables = self.vnode_disbutes[vgroup_id] + distribute_tbnames.append(random.sample(vnode_tables,1)[0]) + tbname_ins = "" + for tbname in distribute_tbnames: + tbname_ins += "'%s' ,"%tbname + + tbname_filters = tbname_ins[:-1] + + min_sql = f"select min({col_name}) from stb1 where tbname in ({tbname_filters});" + + same_sql = f"select {col_name} from stb1 where tbname in ({tbname_filters}) and {col_name} is not null order by {col_name} asc limit 1" + + tdSql.query(min_sql) + min_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if min_result !=same_result: + tdLog.exit(" min function work not as expected, sql : %s "% min_sql) + else: + tdLog.info(" min function work as expected, sql : %s "% min_sql) + + def check_min_status(self): + # check max function work status + + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + tablenames.append(table_name[0]) + + tdSql.query("desc stb1") + col_names = tdSql.queryResult + + colnames = [] + for col_name in col_names: + if col_name[1] in ["INT" ,"BIGINT" ,"SMALLINT" ,"TINYINT" , "FLOAT" ,"DOUBLE"]: + colnames.append(col_name[0]) + + for tablename in tablenames: + for colname in colnames: + self.check_min_functions(tablename,colname) + + # check max function for different vnode + + for colname in colnames: + if colname.startswith("c"): + self.check_min_distribute_diff_vnode(colname) + else: + # self.check_min_distribute_diff_vnode(colname) # bug for tag + pass + + + def distribute_agg_query(self): + # basic filter + tdSql.query("select min(c1) from stb1 where c1 is null") + tdSql.checkRows(0) + + tdSql.query("select min(c1) from stb1 where t1=1") + tdSql.checkData(0,0,2) + + tdSql.query("select min(c1+c2) from stb1 where c1 =1 ") + tdSql.checkData(0,0,11112.000000000) + + tdSql.query("select min(c1) from stb1 where tbname=\"ct2\"") + tdSql.checkData(0,0,2) + + tdSql.query("select min(c1) from stb1 partition by tbname") + tdSql.checkRows(20) + + tdSql.query("select min(c1) from stb1 where t1> 4 partition by tbname") + tdSql.checkRows(15) + + # union all + tdSql.query("select min(c1) from stb1 union all select min(c1) from stb1 ") + tdSql.checkRows(2) + tdSql.checkData(0,0,0) + + # join + + tdSql.execute(" create database if not exists db ") + tdSql.execute(" use db ") + tdSql.execute(" create stable st (ts timestamp , c1 int ,c2 float) tags(t1 int) ") + tdSql.execute(" create table tb1 using st tags(1) ") + tdSql.execute(" create table tb2 using st tags(2) ") + + + for i in range(10): + ts = i*10 + self.ts + tdSql.execute(f" insert into tb1 values({ts},{i},{i}.0)") + tdSql.execute(f" insert into tb2 values({ts},{i},{i}.0)") + + tdSql.query("select min(tb1.c1), tb2.c2 from tb1, tb2 where tb1.ts=tb2.ts") + tdSql.checkRows(1) + tdSql.checkData(0,0,0) + tdSql.checkData(0,0,0.00000) + + # group by + tdSql.execute(" use testdb ") + tdSql.query(" select min(c1),c1 from stb1 group by t1 ") + tdSql.checkRows(20) + tdSql.query(" select min(c1),c1 from stb1 group by c1 ") + tdSql.checkRows(30) + tdSql.query(" select min(c1),c2 from stb1 group by c2 ") + tdSql.checkRows(31) + + # partition by tbname or partition by tag + tdSql.query("select min(c1),tbname from stb1 partition by tbname") + query_data = tdSql.queryResult + + for row in query_data: + tbname = row[1] + tdSql.query(" select min(c1) from %s "%tbname) + tdSql.checkData(0,0,row[0]) + + tdSql.query("select min(c1),tbname from stb1 partition by t1") + query_data = tdSql.queryResult + + for row in query_data: + tbname = row[1] + tdSql.query(" select min(c1) from %s "%tbname) + tdSql.checkData(0,0,row[0]) + + # nest query for support max + tdSql.query("select abs(c2+2)+1 from (select min(c1) c2 from stb1)") + tdSql.checkData(0,0,3.000000000) + tdSql.query("select min(c1+2)+1 as c2 from (select ts ,c1 ,c2 from stb1)") + tdSql.checkData(0,0,3.000000000) + tdSql.query("select min(a+2)+1 as c2 from (select ts ,abs(c1) a ,c2 from stb1)") + tdSql.checkData(0,0,3.000000000) + + # mixup with other functions + tdSql.query("select max(c1),count(c1),last(c2,c3),min(c1) from stb1") + tdSql.checkData(0,0,28) + tdSql.checkData(0,1,184) + tdSql.checkData(0,2,-99999) + tdSql.checkData(0,3,-999) + tdSql.checkData(0,4,0) + + def run(self): + + self.prepare_datas_of_distribute() + self.check_distribute_datas() + self.check_min_status() + self.distribute_agg_query() + + + def stop(self): + tdSql.close() + tdLog.success("%s successfully executed" % __file__) + +tdCases.addWindows(__file__, TDTestCase()) +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/system-test/2-query/distribute_agg_spread.py b/tests/system-test/2-query/distribute_agg_spread.py new file mode 100644 index 0000000000..94f1a61d77 --- /dev/null +++ b/tests/system-test/2-query/distribute_agg_spread.py @@ -0,0 +1,281 @@ +from util.log import * +from util.cases import * +from util.sql import * +import numpy as np +import random + + +class TDTestCase: + updatecfgDict = {'debugFlag': 143 ,"cDebugFlag":143,"uDebugFlag":143 ,"rpcDebugFlag":143 , "tmrDebugFlag":143 , + "jniDebugFlag":143 ,"simDebugFlag":143,"dDebugFlag":143, "dDebugFlag":143,"vDebugFlag":143,"mDebugFlag":143,"qDebugFlag":143, + "wDebugFlag":143,"sDebugFlag":143,"tsdbDebugFlag":143,"tqDebugFlag":143 ,"fsDebugFlag":143 ,"fnDebugFlag":143, + "maxTablesPerVnode":2 ,"minTablesPerVnode":2,"tableIncStepPerVnode":2 } + + def init(self, conn, logSql): + tdLog.debug("start to execute %s" % __file__) + tdSql.init(conn.cursor()) + self.vnode_disbutes = None + self.ts = 1537146000000 + + + def check_spread_functions(self, tbname , col_name): + + spread_sql = f"select spread({col_name}) from {tbname};" + + same_sql = f"select max({col_name})-min({col_name}) from {tbname}" + + tdSql.query(spread_sql) + spread_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if spread_result !=same_result: + tdLog.exit(" max function work not as expected, sql : %s "% spread_sql) + else: + tdLog.info(" max function work as expected, sql : %s "% spread_sql) + + + def prepare_datas_of_distribute(self): + + # prepate datas for 20 tables distributed at different vgroups + tdSql.execute("create database if not exists testdb keep 3650 duration 1000 vgroups 5") + tdSql.execute(" use testdb ") + tdSql.execute( + '''create table stb1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + tags (t0 timestamp, t1 int, t2 bigint, t3 smallint, t4 tinyint, t5 float, t6 double, t7 bool, t8 binary(16),t9 nchar(32)) + ''' + ) + + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(20): + tdSql.execute(f'create table ct{i+1} using stb1 tags ( now(), {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, "binary{i}", "nchar{i}" )') + + for i in range(9): + tdSql.execute( + f"insert into ct1 values ( now()-{i*10}s, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + tdSql.execute( + f"insert into ct4 values ( now()-{i*90}d, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + + for i in range(1,21): + if i ==1 or i == 4: + continue + else: + tbname = "ct"+f'{i}' + for j in range(9): + tdSql.execute( + f"insert into {tbname} values ( now()-{(i+j)*10}s, {1*(j+i)}, {11111*(j+i)}, {111*(j+i)}, {11*(j)}, {1.11*(j+i)}, {11.11*(j+i)}, {(j+i)%2}, 'binary{j}', 'nchar{j}', now()+{1*j}a )" + ) + tdSql.execute("insert into ct1 values (now()-45s, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar0', now()+8a )") + tdSql.execute("insert into ct1 values (now()+10s, 9, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+15s, 9, -99999, -999, -99, -9.99, NULL, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+20s, 9, -99999, -999, NULL, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + + tdSql.execute("insert into ct4 values (now()-810d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()-400d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()+90d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + + tdSql.execute( + f'''insert into t1 values + ( '2020-04-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2020-10-21 01:01:01.000', 1, 11111, 111, 11, 1.11, 11.11, 1, "binary1", "nchar1", now()+1a ) + ( '2020-12-31 01:01:01.000', 2, 22222, 222, 22, 2.22, 22.22, 0, "binary2", "nchar2", now()+2a ) + ( '2021-01-01 01:01:06.000', 3, 33333, 333, 33, 3.33, 33.33, 0, "binary3", "nchar3", now()+3a ) + ( '2021-05-07 01:01:10.000', 4, 44444, 444, 44, 4.44, 44.44, 1, "binary4", "nchar4", now()+4a ) + ( '2021-07-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2021-09-30 01:01:16.000', 5, 55555, 555, 55, 5.55, 55.55, 0, "binary5", "nchar5", now()+5a ) + ( '2022-02-01 01:01:20.000', 6, 66666, 666, 66, 6.66, 66.66, 1, "binary6", "nchar6", now()+6a ) + ( '2022-10-28 01:01:26.000', 7, 00000, 000, 00, 0.00, 00.00, 1, "binary7", "nchar7", "1970-01-01 08:00:00.000" ) + ( '2022-12-01 01:01:30.000', 8, -88888, -888, -88, -8.88, -88.88, 0, "binary8", "nchar8", "1969-01-01 01:00:00.000" ) + ( '2022-12-31 01:01:36.000', 9, -99999999999999999, -999, -99, -9.99, -999999999999999999999.99, 1, "binary9", "nchar9", "1900-01-01 00:00:00.000" ) + ( '2023-02-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ''' + ) + + tdLog.info(" prepare data for distributed_aggregate done! ") + + def check_distribute_datas(self): + # get vgroup_ids of all + tdSql.query("show vgroups ") + vgroups = tdSql.queryResult + + vnode_tables={} + + for vgroup_id in vgroups: + vnode_tables[vgroup_id[0]]=[] + + + # check sub_table of per vnode ,make sure sub_table has been distributed + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + vnode_tables[table_name[6]].append(table_name[0]) + self.vnode_disbutes = vnode_tables + + count = 0 + for k ,v in vnode_tables.items(): + if len(v)>=2: + count+=1 + if count < 2: + tdLog.exit(" the datas of all not satisfy sub_table has been distributed ") + + def check_spread_distribute_diff_vnode(self,col_name): + + vgroup_ids = [] + for k ,v in self.vnode_disbutes.items(): + if len(v)>=2: + vgroup_ids.append(k) + + distribute_tbnames = [] + + for vgroup_id in vgroup_ids: + vnode_tables = self.vnode_disbutes[vgroup_id] + distribute_tbnames.append(random.sample(vnode_tables,1)[0]) + tbname_ins = "" + for tbname in distribute_tbnames: + tbname_ins += "'%s' ,"%tbname + + tbname_filters = tbname_ins[:-1] + + spread_sql = f"select spread({col_name}) from stb1 where tbname in ({tbname_filters})" + + same_sql = f"select max({col_name}) - min({col_name}) from stb1 where tbname in ({tbname_filters})" + + tdSql.query(spread_sql) + spread_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if spread_result !=same_result: + tdLog.exit(" spread function work not as expected, sql : %s "% spread_sql) + else: + tdLog.info(" spread function work as expected, sql : %s "% spread_sql) + + def check_spread_status(self): + # check max function work status + + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + tablenames.append(table_name[0]) + + tdSql.query("desc stb1") + col_names = tdSql.queryResult + + colnames = [] + for col_name in col_names: + if col_name[1] in ["INT" ,"BIGINT" ,"SMALLINT" ,"TINYINT" , "FLOAT" ,"DOUBLE"]: + colnames.append(col_name[0]) + + for tablename in tablenames: + for colname in colnames: + self.check_spread_functions(tablename,colname) + + # check max function for different vnode + + for colname in colnames: + if colname.startswith("c"): + self.check_spread_distribute_diff_vnode(colname) + else: + # self.check_spread_distribute_diff_vnode(colname) # bug for tag + pass + + + def distribute_agg_query(self): + # basic filter + tdSql.query("select spread(c1) from stb1 where c1 is null") + tdSql.checkRows(0) + + tdSql.query("select spread(c1) from stb1 where t1=1") + tdSql.checkData(0,0,8.000000000) + + tdSql.query("select spread(c1+c2) from stb1 where c1 =1 ") + tdSql.checkData(0,0,0.000000000) + + tdSql.query("select spread(c1) from stb1 where tbname=\"ct2\"") + tdSql.checkData(0,0,8.000000000) + + tdSql.query("select spread(c1) from stb1 partition by tbname") + tdSql.checkRows(20) + + tdSql.query("select spread(c1) from stb1 where t1> 4 partition by tbname") + tdSql.checkRows(15) + + # union all + tdSql.query("select spread(c1) from stb1 union all select max(c1)-min(c1) from stb1 ") + tdSql.checkRows(2) + tdSql.checkData(0,0,28.000000000) + + # join + + tdSql.execute(" create database if not exists db ") + tdSql.execute(" use db ") + tdSql.execute(" create stable st (ts timestamp , c1 int ,c2 float) tags(t1 int) ") + tdSql.execute(" create table tb1 using st tags(1) ") + tdSql.execute(" create table tb2 using st tags(2) ") + + + for i in range(10): + ts = i*10 + self.ts + tdSql.execute(f" insert into tb1 values({ts},{i},{i}.0)") + tdSql.execute(f" insert into tb2 values({ts},{i},{i}.0)") + + tdSql.query("select spread(tb1.c1), spread(tb2.c2) from tb1, tb2 where tb1.ts=tb2.ts") + tdSql.checkRows(1) + tdSql.checkData(0,0,9.000000000) + tdSql.checkData(0,0,9.00000) + + # group by + tdSql.execute(" use testdb ") + tdSql.query(" select max(c1),c1 from stb1 group by t1 ") + tdSql.checkRows(20) + tdSql.query(" select max(c1),c1 from stb1 group by c1 ") + tdSql.checkRows(30) + tdSql.query(" select max(c1),c2 from stb1 group by c2 ") + tdSql.checkRows(31) + + # partition by tbname or partition by tag + tdSql.query("select spread(c1) from stb1 partition by tbname") + query_data = tdSql.queryResult + + # nest query for support max + tdSql.query("select spread(c2+2)+1 from (select max(c1) c2 from stb1)") + tdSql.checkData(0,0,1.000000000) + tdSql.query("select spread(c1+2)+1 as c2 from (select ts ,c1 ,c2 from stb1)") + tdSql.checkData(0,0,29.000000000) + tdSql.query("select spread(a+2)+1 as c2 from (select ts ,abs(c1) a ,c2 from stb1)") + tdSql.checkData(0,0,29.000000000) + + # mixup with other functions + tdSql.query("select max(c1),count(c1),last(c2,c3),spread(c1) from stb1") + tdSql.checkData(0,0,28) + tdSql.checkData(0,1,184) + tdSql.checkData(0,2,-99999) + tdSql.checkData(0,3,-999) + tdSql.checkData(0,4,28.000000000) + + def run(self): + + self.prepare_datas_of_distribute() + self.check_distribute_datas() + self.check_spread_status() + self.distribute_agg_query() + + + def stop(self): + tdSql.close() + tdLog.success("%s successfully executed" % __file__) + +tdCases.addWindows(__file__, TDTestCase()) +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/system-test/2-query/distribute_agg_sum.py b/tests/system-test/2-query/distribute_agg_sum.py new file mode 100644 index 0000000000..add4d75c61 --- /dev/null +++ b/tests/system-test/2-query/distribute_agg_sum.py @@ -0,0 +1,278 @@ +from util.log import * +from util.cases import * +from util.sql import * +import numpy as np +import random ,os ,sys +import platform + + +class TDTestCase: + updatecfgDict = {'debugFlag': 143 ,"cDebugFlag":143,"uDebugFlag":143 ,"rpcDebugFlag":143 , "tmrDebugFlag":143 , + "jniDebugFlag":143 ,"simDebugFlag":143,"dDebugFlag":143, "dDebugFlag":143,"vDebugFlag":143,"mDebugFlag":143,"qDebugFlag":143, + "wDebugFlag":143,"sDebugFlag":143,"tsdbDebugFlag":143,"tqDebugFlag":143 ,"fsDebugFlag":143 ,"fnDebugFlag":143, + "maxTablesPerVnode":2 ,"minTablesPerVnode":2,"tableIncStepPerVnode":2 } + + def init(self, conn, logSql): + tdLog.debug("start to execute %s" % __file__) + tdSql.init(conn.cursor()) + self.vnode_disbutes = None + self.ts = 1537146000000 + + + def check_sum_functions(self, tbname , col_name): + + sum_sql = f"select sum({col_name}) from {tbname};" + + same_sql = f"select {col_name} from {tbname} where {col_name} is not null " + + tdSql.query(same_sql) + pre_data = np.array(tdSql.queryResult)[np.array(tdSql.queryResult) != None] + if (platform.system().lower() == 'windows' and pre_data.dtype == 'int32'): + pre_data = np.array(pre_data, dtype = 'int64') + pre_sum = np.sum(pre_data) + + tdSql.query(sum_sql) + tdSql.checkData(0,0,pre_sum) + + def prepare_datas_of_distribute(self): + + # prepate datas for 20 tables distributed at different vgroups + tdSql.execute("create database if not exists testdb keep 3650 duration 1000 vgroups 5") + tdSql.execute(" use testdb ") + tdSql.execute( + '''create table stb1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + tags (t0 timestamp, t1 int, t2 bigint, t3 smallint, t4 tinyint, t5 float, t6 double, t7 bool, t8 binary(16),t9 nchar(32)) + ''' + ) + + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(20): + tdSql.execute(f'create table ct{i+1} using stb1 tags ( now(), {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, "binary{i}", "nchar{i}" )') + + for i in range(9): + tdSql.execute( + f"insert into ct1 values ( now()-{i*10}s, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + tdSql.execute( + f"insert into ct4 values ( now()-{i*90}d, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + + for i in range(1,21): + if i ==1 or i == 4: + continue + else: + tbname = "ct"+f'{i}' + for j in range(9): + tdSql.execute( + f"insert into {tbname} values ( now()-{(i+j)*10}s, {1*(j+i)}, {11111*(j+i)}, {111*(j+i)}, {11*(j)}, {1.11*(j+i)}, {11.11*(j+i)}, {(j+i)%2}, 'binary{j}', 'nchar{j}', now()+{1*j}a )" + ) + tdSql.execute("insert into ct1 values (now()-45s, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar0', now()+8a )") + tdSql.execute("insert into ct1 values (now()+10s, 9, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+15s, 9, -99999, -999, -99, -9.99, NULL, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+20s, 9, -99999, -999, NULL, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + + tdSql.execute("insert into ct4 values (now()-810d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()-400d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()+90d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + + tdSql.execute( + f'''insert into t1 values + ( '2020-04-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2020-10-21 01:01:01.000', 1, 11111, 111, 11, 1.11, 11.11, 1, "binary1", "nchar1", now()+1a ) + ( '2020-12-31 01:01:01.000', 2, 22222, 222, 22, 2.22, 22.22, 0, "binary2", "nchar2", now()+2a ) + ( '2021-01-01 01:01:06.000', 3, 33333, 333, 33, 3.33, 33.33, 0, "binary3", "nchar3", now()+3a ) + ( '2021-05-07 01:01:10.000', 4, 44444, 444, 44, 4.44, 44.44, 1, "binary4", "nchar4", now()+4a ) + ( '2021-07-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2021-09-30 01:01:16.000', 5, 55555, 555, 55, 5.55, 55.55, 0, "binary5", "nchar5", now()+5a ) + ( '2022-02-01 01:01:20.000', 6, 66666, 666, 66, 6.66, 66.66, 1, "binary6", "nchar6", now()+6a ) + ( '2022-10-28 01:01:26.000', 7, 00000, 000, 00, 0.00, 00.00, 1, "binary7", "nchar7", "1970-01-01 08:00:00.000" ) + ( '2022-12-01 01:01:30.000', 8, -88888, -888, -88, -8.88, -88.88, 0, "binary8", "nchar8", "1969-01-01 01:00:00.000" ) + ( '2022-12-31 01:01:36.000', 9, -99999999999999999, -999, -99, -9.99, -999999999999999999999.99, 1, "binary9", "nchar9", "1900-01-01 00:00:00.000" ) + ( '2023-02-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ''' + ) + + tdLog.info(" prepare data for distributed_aggregate done! ") + + def check_distribute_datas(self): + # get vgroup_ids of all + tdSql.query("show vgroups ") + vgroups = tdSql.queryResult + + vnode_tables={} + + for vgroup_id in vgroups: + vnode_tables[vgroup_id[0]]=[] + + + # check sub_table of per vnode ,make sure sub_table has been distributed + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + vnode_tables[table_name[6]].append(table_name[0]) + self.vnode_disbutes = vnode_tables + + count = 0 + for k ,v in vnode_tables.items(): + if len(v)>=2: + count+=1 + if count < 2: + tdLog.exit(" the datas of all not satisfy sub_table has been distributed ") + + def check_sum_distribute_diff_vnode(self,col_name): + + vgroup_ids = [] + for k ,v in self.vnode_disbutes.items(): + if len(v)>=2: + vgroup_ids.append(k) + + distribute_tbnames = [] + + for vgroup_id in vgroup_ids: + vnode_tables = self.vnode_disbutes[vgroup_id] + distribute_tbnames.append(random.sample(vnode_tables,1)[0]) + tbname_ins = "" + for tbname in distribute_tbnames: + tbname_ins += "'%s' ,"%tbname + + tbname_filters = tbname_ins[:-1] + + sum_sql = f"select sum({col_name}) from stb1 where tbname in ({tbname_filters});" + + same_sql = f"select {col_name} from stb1 where tbname in ({tbname_filters}) and {col_name} is not null " + + tdSql.query(same_sql) + pre_data = np.array(tdSql.queryResult)[np.array(tdSql.queryResult) != None] + if (platform.system().lower() == 'windows' and pre_data.dtype == 'int32'): + pre_data = np.array(pre_data, dtype = 'int64') + pre_sum = np.sum(pre_data) + + tdSql.query(sum_sql) + tdSql.checkData(0,0,pre_sum) + + def check_sum_status(self): + # check max function work status + + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + tablenames.append(table_name[0]) + + tdSql.query("desc stb1") + col_names = tdSql.queryResult + + colnames = [] + for col_name in col_names: + if col_name[1] in ["INT" ,"BIGINT" ,"SMALLINT" ,"TINYINT" , "FLOAT" ,"DOUBLE"]: + colnames.append(col_name[0]) + + for tablename in tablenames: + for colname in colnames: + self.check_sum_functions(tablename,colname) + + # check max function for different vnode + + for colname in colnames: + if colname.startswith("c"): + self.check_sum_distribute_diff_vnode(colname) + else: + # self.check_sum_distribute_diff_vnode(colname) # bug for tag + pass + + + def distribute_agg_query(self): + # basic filter + tdSql.query(" select sum(c1) from stb1 ") + tdSql.checkData(0,0,2592) + + tdSql.query(" select sum(a) from (select sum(c1) a from stb1 partition by tbname) ") + tdSql.checkData(0,0,2592) + + tdSql.query(" select sum(c1) from stb1 where t1=1") + tdSql.checkData(0,0,54) + + tdSql.query("select sum(c1+c2) from stb1 where c1 =1 ") + tdSql.checkData(0,0,22224.000000000) + + tdSql.query("select sum(c1) from stb1 where tbname=\"ct2\"") + tdSql.checkData(0,0,54) + + tdSql.query("select sum(c1) from stb1 partition by tbname") + tdSql.checkRows(20) + + tdSql.query("select sum(c1) from stb1 where t1> 4 partition by tbname") + tdSql.checkRows(15) + + # union all + tdSql.query("select sum(c1) from stb1 union all select sum(c1) from stb1 ") + tdSql.checkRows(2) + tdSql.checkData(0,0,2592) + + tdSql.query("select sum(a) from (select sum(c1) a from stb1 union all select sum(c1) a from stb1)") + tdSql.checkRows(1) + tdSql.checkData(0,0,5184) + + # join + + tdSql.execute(" create database if not exists db ") + tdSql.execute(" use db ") + tdSql.execute(" create stable st (ts timestamp , c1 int ,c2 float) tags(t1 int) ") + tdSql.execute(" create table tb1 using st tags(1) ") + tdSql.execute(" create table tb2 using st tags(2) ") + + + for i in range(10): + ts = i*10 + self.ts + tdSql.execute(f" insert into tb1 values({ts},{i},{i}.0)") + tdSql.execute(f" insert into tb2 values({ts},{i},{i}.0)") + + tdSql.query("select sum(tb1.c1), sum(tb2.c2) from tb1, tb2 where tb1.ts=tb2.ts") + tdSql.checkRows(1) + tdSql.checkData(0,0,45) + tdSql.checkData(0,1,45.000000000) + + # group by + tdSql.execute(" use testdb ") + + # partition by tbname or partition by tag + tdSql.query("select sum(c1) from stb1 partition by tbname") + tdSql.checkRows(20) + + # nest query for support max + tdSql.query("select abs(c2+2)+1 from (select sum(c1) c2 from stb1)") + tdSql.checkData(0,0,2595.000000000) + tdSql.query("select sum(c1+2) as c2 from (select ts ,c1 ,c2 from stb1)") + tdSql.checkData(0,0,2960.000000000) + tdSql.query("select sum(a+2) as c2 from (select ts ,abs(c1) a ,c2 from stb1)") + tdSql.checkData(0,0,2960.000000000) + + # mixup with other functions + tdSql.query("select max(c1),count(c1),last(c2,c3),sum(c1+c2) from stb1") + tdSql.checkData(0,0,28) + tdSql.checkData(0,1,184) + tdSql.checkData(0,2,-99999) + tdSql.checkData(0,3,-999) + tdSql.checkData(0,4,28202310.000000000) + + def run(self): + + self.prepare_datas_of_distribute() + self.check_distribute_datas() + self.check_sum_status() + self.distribute_agg_query() + + + def stop(self): + tdSql.close() + tdLog.success("%s successfully executed" % __file__) + +tdCases.addWindows(__file__, TDTestCase()) +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/system-test/2-query/first.py b/tests/system-test/2-query/first.py index 7227d1afb5..e9a8cc950b 100644 --- a/tests/system-test/2-query/first.py +++ b/tests/system-test/2-query/first.py @@ -11,8 +11,11 @@ # -*- coding: utf-8 -*- +import random +import string import sys import taos +from util.common import * from util.log import * from util.cases import * from util.sql import * @@ -25,124 +28,159 @@ class TDTestCase: tdSql.init(conn.cursor()) self.rowNum = 10 + self.tbnum = 20 self.ts = 1537146000000 - - def run(self): + self.binary_str = 'taosdata' + self.nchar_str = '涛思数据' + + def first_check_base(self): tdSql.prepare() - - tdSql.execute('''create table test(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double, - col7 bool, col8 binary(20), col9 nchar(20), col11 tinyint unsigned, col12 smallint unsigned, col13 int unsigned, col14 bigint unsigned) tags(loc nchar(20))''') - tdSql.execute("create table test1 using test tags('beijing')") - tdSql.execute("insert into test1(ts) values(%d)" % (self.ts - 1)) - - # first verifacation - # bug TD-15957 - tdSql.query("select first(*) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 1, None) - - tdSql.query("select first(col1) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col2) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col3) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col4) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col11) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col12) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col13) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col14) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col5) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col6) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col7) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col8) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col9) from test1") - tdSql.checkRows(0) - + column_dict = { + 'col1': 'tinyint', + 'col2': 'smallint', + 'col3': 'int', + 'col4': 'bigint', + 'col5': 'tinyint unsigned', + 'col6': 'smallint unsigned', + 'col7': 'int unsigned', + 'col8': 'bigint unsigned', + 'col9': 'float', + 'col10': 'double', + 'col11': 'bool', + 'col12': 'binary(20)', + 'col13': 'nchar(20)' + } + tdSql.execute('''create table stb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') + tdSql.execute("create table stb_1 using stb tags('beijing')") + tdSql.execute("insert into stb_1(ts) values(%d)" % (self.ts - 1)) + column_list = ['col1','col2','col3','col4','col5','col6','col7','col8','col9','col10','col11','col12','col13'] + for i in ['stb_1','db.stb_1','stb_1','db.stb_1']: + tdSql.query(f"select first(*) from {i}") + tdSql.checkRows(1) + tdSql.checkData(0, 1, None) + #!bug TD-16561 + # for i in ['stb','db.stb']: + # tdSql.query(f"select first(*) from {i}") + # tdSql.checkRows(1) + # tdSql.checkData(0, 1, None) + for i in column_list: + for j in ['stb_1','db.stb_1','stb_1','db.stb_1']: + tdSql.query(f"select first({i}) from {j}") + tdSql.checkRows(0) for i in range(self.rowNum): - tdSql.execute("insert into test1 values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)" - % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1)) - - tdSql.query("select first(*) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 1, 1) - - tdSql.query("select first(col1) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 1) - - tdSql.query("select first(col2) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 1) - - tdSql.query("select first(col3) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 1) - - tdSql.query("select first(col4) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 1) - - tdSql.query("select first(col11) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 1) - - tdSql.query("select first(col12) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 1) - - tdSql.query("select first(col13) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 1) - - tdSql.query("select first(col14) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 1) - - tdSql.query("select first(col5) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 0.1) - - tdSql.query("select first(col6) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 0.1) - - tdSql.query("select first(col7) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, False) - - tdSql.query("select first(col8) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 'taosdata1') - - tdSql.query("select first(col9) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, '涛思数据1') - - - tdSql.query("select first(*),last(*) from test1 where ts < 23 interval(1s)") + tdSql.execute(f"insert into stb_1 values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) + for k, v in column_dict.items(): + for j in ['stb_1', 'db.stb_1', 'stb', 'db.stb']: + tdSql.query(f"select first({k}) from {j}") + tdSql.checkRows(1) + # tinyint,smallint,int,bigint,tinyint unsigned,smallint unsigned,int unsigned,bigint unsigned + if v == 'tinyint' or v == 'smallint' or v == 'int' or v == 'bigint' or v == 'tinyint unsigned' or v == 'smallint unsigned'\ + or v == 'int unsigned' or v == 'bigint unsigned': + tdSql.checkData(0, 0, 1) + # float,double + elif v == 'float' or v == 'double': + tdSql.checkData(0, 0, 0.1) + # bool + elif v == 'bool': + tdSql.checkData(0, 0, False) + # binary + elif 'binary' in v: + tdSql.checkData(0, 0, f'{self.binary_str}1') + # nchar + elif 'nchar' in v: + tdSql.checkData(0, 0, f'{self.nchar_str}1') + #!bug TD-16569 + tdSql.query("select first(*),last(*) from stb where ts < 23 interval(1s)") tdSql.checkRows(0) + tdSql.execute('drop database db') + def first_check_stb_distribute(self): + # prepare data for vgroup 4 + dbname = tdCom.getLongName(10, "letters") + stbname = tdCom.getLongName(5, "letters") + child_table_num = 20 + vgroup = 2 + column_dict = { + 'col1': 'tinyint', + 'col2': 'smallint', + 'col3': 'int', + 'col4': 'bigint', + 'col5': 'tinyint unsigned', + 'col6': 'smallint unsigned', + 'col7': 'int unsigned', + 'col8': 'bigint unsigned', + 'col9': 'float', + 'col10': 'double', + 'col11': 'bool', + 'col12': 'binary(20)', + 'col13': 'nchar(20)' + } + tdSql.execute(f"create database if not exists {dbname} vgroups {vgroup}") + tdSql.execute(f'use {dbname}') + # build 20 child tables,every table insert 10 rows + tdSql.execute(f'''create table {stbname}(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') + for i in range(child_table_num): + tdSql.execute(f"create table {stbname}_{i} using {stbname} tags('beijing')") + tdSql.execute(f"insert into {stbname}_{i}(ts) values(%d)" % (self.ts - 1-i)) + #!bug TD-16561 + # for i in [f'{stbname}', f'{dbname}.{stbname}']: + # tdSql.query(f"select first(*) from {i}") + # tdSql.checkRows(1) + # tdSql.checkData(0, 1, None) + tdSql.query('show tables') + vgroup_list = [] + for i in range(len(tdSql.queryResult)): + vgroup_list.append(tdSql.queryResult[i][6]) + vgroup_list_set = set(vgroup_list) + # print(vgroup_list_set) + # print(vgroup_list) + for i in vgroup_list_set: + vgroups_num = vgroup_list.count(i) + if vgroups_num >=2: + tdLog.info(f'This scene with {vgroups_num} vgroups is ok!') + continue + else: + tdLog.exit('This scene does not meet the requirements with {vgroups_num} vgroup!\n') + + for i in range(child_table_num): + for j in range(self.rowNum): + tdSql.execute(f"insert into {stbname}_{i} values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + j + i, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 0.1, j + 0.1, j % 2, j + 1, j + 1)) + + for k, v in column_dict.items(): + for j in [f'{stbname}_{i}', f'{dbname}.{stbname}_{i}', f'{stbname}', f'{dbname}.{stbname}']: + tdSql.query(f"select first({k}) from {j}") + tdSql.checkRows(1) + # tinyint,smallint,int,bigint,tinyint unsigned,smallint unsigned,int unsigned,bigint unsigned + if v == 'tinyint' or v == 'smallint' or v == 'int' or v == 'bigint' or v == 'tinyint unsigned' or v == 'smallint unsigned'\ + or v == 'int unsigned' or v == 'bigint unsigned': + tdSql.checkData(0, 0, 1) + # float,double + elif v == 'float' or v == 'double': + tdSql.checkData(0, 0, 0.1) + # bool + elif v == 'bool': + tdSql.checkData(0, 0, False) + # binary + elif 'binary' in v: + tdSql.checkData(0, 0, f'{self.binary_str}1') + # nchar + elif 'nchar' in v: + tdSql.checkData(0, 0, f'{self.nchar_str}1') + #!bug TD-16569 + # tdSql.query(f"select first(*),last(*) from {stbname} where ts < 23 interval(1s)") + # tdSql.checkRows(0) + tdSql.execute(f'drop database {dbname}') + + + + pass + def run(self): + self.first_check_base() + self.first_check_stb_distribute() + def stop(self): tdSql.close() diff --git a/tests/system-test/2-query/histogram.py b/tests/system-test/2-query/histogram.py index 0448952be8..bc061f8fb7 100644 --- a/tests/system-test/2-query/histogram.py +++ b/tests/system-test/2-query/histogram.py @@ -44,8 +44,7 @@ class TDTestCase: buildPath = root[:len(root) - len("/build/bin")] break return buildPath - - def run(self): + def histogram_check_base(self): print("running {}".format(__file__)) tdSql.execute("drop database if exists db") tdSql.execute("create database if not exists db") @@ -3183,6 +3182,72 @@ class TDTestCase: tdSql.execute('drop database db') + + def histogram_check_distribute(self): + dbname = "db" + stbname = "stb" + row_num = 10 + child_table_num = 20 + vgroups = 2 + user_input_json = "[1,3,5,7]" + ts = 1537146000000 + binary_str = 'taosdata' + nchar_str = '涛思数据' + column_dict = { + 'ts' : 'timestamp', + 'col1' : 'tinyint', + 'col2' : 'smallint', + 'col3' : 'int', + 'col4' : 'bigint', + 'col5' : 'tinyint unsigned', + 'col6' : 'smallint unsigned', + 'col7' : 'int unsigned', + 'col8' : 'bigint unsigned', + 'col9' : 'float', + 'col10': 'double', + 'col11': 'bool', + 'col12': 'binary(20)', + 'col13': 'nchar(20)' + } + tdSql.execute(f"create database if not exists {dbname} vgroups {vgroups}") + tdSql.execute(f'use {dbname}') + # build 20 child tables,every table insert 10 rows + tdSql.execute(f'''create table {stbname}(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') + for i in range(child_table_num): + tdSql.execute(f"create table {stbname}_{i} using {stbname} tags('beijing')") + tdSql.query('show tables') + vgroup_list = [] + for i in range(len(tdSql.queryResult)): + vgroup_list.append(tdSql.queryResult[i][6]) + vgroup_list_set = set(vgroup_list) + for i in vgroup_list_set: + vgroups_num = vgroup_list.count(i) + if vgroups_num >=2: + tdLog.info(f'This scene with {vgroups_num} vgroups is ok!') + continue + else: + tdLog.exit(f'This scene does not meet the requirements with {vgroups_num} vgroup!\n') + for i in range(child_table_num): + for j in range(row_num): + tdSql.execute(f"insert into {stbname}_{i} values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{binary_str}%d', '{nchar_str}%d')" + % (ts + j + i, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 0.1, j + 0.1, j % 2, j + 1, j + 1)) + # user_input + for k,v in column_dict.items(): + if v.lower() == 'tinyint' or v.lower() == 'smallint' or v.lower() == 'int' or v.lower() == 'bigint' or v.lower() =='float' or v.lower() =='double'\ + or v.lower() =='tinyint unsigned' or v.lower() =='smallint unsigned' or v.lower() =='int unsigned' or v.lower() =='bigint unsigned': + tdSql.query(f'select histogram({k}, "user_input", "{user_input_json}", 0) from {stbname}') + tdSql.checkRows(len(user_input_json[1:-1].split(','))-1) + elif 'binary' in v.lower() or 'nchar' in v.lower() or 'bool' == v.lower(): + tdSql.error(f'select histogram({k}, "user_input", "{user_input_json}", 0) from {stbname}') + + tdSql.execute(f'drop database {dbname}') + + + def run(self): + self.histogram_check_base() + self.histogram_check_distribute() + def stop(self): tdSql.close() tdLog.success("%s successfully executed" % __file__) diff --git a/tests/system-test/2-query/hyperloglog.py b/tests/system-test/2-query/hyperloglog.py index 337db140a1..8dd6bd2dda 100644 --- a/tests/system-test/2-query/hyperloglog.py +++ b/tests/system-test/2-query/hyperloglog.py @@ -214,6 +214,79 @@ class TDTestCase: for i in range(4): tdSql.execute(f'create table ct{i+1} using stb1 tags ( {i+1} )') { i % 32767 }, { i % 127}, { i * 1.11111 }, { i * 1000.1111 }, { i % 2} + def __create_stable(self,stbname='stb',column_dict={'ts':'timestamp','col1': 'tinyint','col2': 'smallint','col3': 'int', + 'col4': 'bigint','col5': 'tinyint unsigned','col6': 'smallint unsigned','col7': 'int unsigned', + 'col8': 'bigint unsigned','col9': 'float','col10': 'double','col11': 'bool','col12': 'binary(20)','col13': 'nchar(20)'}, + tag_dict={'ts_tag':'timestamp','t1': 'tinyint','t2': 'smallint','t3': 'int', + 't4': 'bigint','t5': 'tinyint unsigned','t6': 'smallint unsigned','t7': 'int unsigned', + 't8': 'bigint unsigned','t9': 'float','t10': 'double','t11': 'bool','t12': 'binary(20)','t13': 'nchar(20)'}): + column_sql = '' + tag_sql = '' + for k,v in column_dict.items(): + column_sql += f"{k} {v}," + for k,v in tag_dict.items(): + tag_sql += f"{k} {v}," + tdSql.execute(f'create table if not exists {stbname} ({column_sql[:-1]}) tags({tag_sql[:-1]})') + + def __insert_data(self): + + pass + + def __hyperloglog_check_distribute(self): + dbname = "dbtest" + stbname = "stb" + childtable_num = 20 + vgroups_num = 4 + row_num = 10 + ts = 1537146000000 + binary_str = 'taosdata' + nchar_str = '涛思数据' + column_dict = { + 'ts':'timestamp', + 'col1': 'tinyint', + 'col2': 'smallint', + 'col3': 'int', + 'col4': 'bigint', + 'col5': 'tinyint unsigned', + 'col6': 'smallint unsigned', + 'col7': 'int unsigned', + 'col8': 'bigint unsigned', + 'col9': 'float', + 'col10': 'double', + 'col11': 'bool', + 'col12': 'binary(20)', + 'col13': 'nchar(20)' + } + tag_dict = { + 'loc':'nchar(20)' + } + tdSql.execute(f"create database if not exists {dbname} vgroups {vgroups_num}") + tdSql.execute(f'use {dbname}') + self.__create_stable(stbname,column_dict,tag_dict) + for i in range(childtable_num): + tdSql.execute(f"create table {stbname}_{i} using {stbname} tags('beijing')") + tdSql.query('show tables') + vgroup_list = [] + for i in range(len(tdSql.queryResult)): + vgroup_list.append(tdSql.queryResult[i][6]) + vgroup_list_set = set(vgroup_list) + for i in vgroup_list_set: + vgroups_num = vgroup_list.count(i) + if vgroups_num >=2: + tdLog.info(f'This scene with {vgroups_num} vgroups is ok!') + continue + else: + tdLog.exit('This scene does not meet the requirements with {vgroups_num} vgroup!\n') + for i in range(row_num): + tdSql.execute(f"insert into stb_1 values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{binary_str}%d', '{nchar_str}%d')" + % (ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) + for k in column_dict.keys(): + tdSql.query(f"select hyperloglog({k}) from {stbname}") + tdSql.checkRows(1) + tdSql.query(f"select hyperloglog({k}) from {stbname} group by {k}") + + tdSql.execute(f'drop database {dbname}') + def __insert_data(self, rows): now_time = int(datetime.datetime.timestamp(datetime.datetime.now()) * 1000) @@ -311,6 +384,10 @@ class TDTestCase: tdLog.printNoPrefix("==========step4:after wal, all check again ") self.all_test() + tdLog.printNoPrefix("==========step5: distribute scene check") + self.__hyperloglog_check_distribute() + + def stop(self): tdSql.close() tdLog.success(f"{__file__} successfully executed") diff --git a/tests/system-test/2-query/last.py b/tests/system-test/2-query/last.py index 4ef13e9142..ee65d22a22 100644 --- a/tests/system-test/2-query/last.py +++ b/tests/system-test/2-query/last.py @@ -1,6 +1,9 @@ +import random +import string from util.log import * from util.cases import * from util.sql import * +from util.common import * import numpy as np @@ -10,416 +13,258 @@ class TDTestCase: tdSql.init(conn.cursor()) self.rowNum = 10 + self.tbnum = 20 self.ts = 1537146000000 - - def run(self): + self.binary_str = 'taosdata' + self.nchar_str = '涛思数据' + + def set_create_normaltable_sql(self, ntbname, column_dict): + column_sql = '' + for k, v in column_dict.items(): + column_sql += f"{k} {v}," + create_ntb_sql = f'create table {ntbname} (ts timestamp,{column_sql[:-1]})' + return create_ntb_sql + + def set_create_stable_sql(self,stbname,column_dict,tag_dict): + column_sql = '' + tag_sql = '' + for k,v in column_dict.items(): + column_sql += f"{k} {v}," + for k,v in tag_dict.items(): + tag_sql += f"{k} {v}," + create_stb_sql = f'create table {stbname} (ts timestamp,{column_sql[:-1]}) tags({tag_sql[:-1]})' + return create_stb_sql + + def last_check_stb_tb_base(self): tdSql.prepare() + stbname = tdCom.getLongName(5, "letters") + column_dict = { + 'col1': 'tinyint', + 'col2': 'smallint', + 'col3': 'int', + 'col4': 'bigint', + 'col5': 'tinyint unsigned', + 'col6': 'smallint unsigned', + 'col7': 'int unsigned', + 'col8': 'bigint unsigned', + 'col9': 'float', + 'col10': 'double', + 'col11': 'bool', + 'col12': 'binary(20)', + 'col13': 'nchar(20)' + } + tag_dict = { + 'loc':'nchar(20)' + } + tdSql.execute(self.set_create_stable_sql(stbname,column_dict,tag_dict)) - tdSql.execute('''create table stb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double, - col7 bool, col8 binary(20), col9 nchar(20), col11 tinyint unsigned, col12 smallint unsigned, col13 int unsigned, col14 bigint unsigned) tags(loc nchar(20))''') - tdSql.execute("create table stb_1 using stb tags('beijing')") - tdSql.execute("insert into stb_1(ts) values(%d)" % (self.ts - 1)) + tdSql.execute(f"create table {stbname}_1 using {stbname} tags('beijing')") + tdSql.execute(f"insert into {stbname}_1(ts) values(%d)" % (self.ts - 1)) - # last verifacation - tdSql.query("select last(*) from stb_1") + for i in [f'{stbname}_1', f'db.{stbname}_1']: + tdSql.query(f"select last(*) from {i}") + tdSql.checkRows(1) + tdSql.checkData(0, 1, None) + #!bug TD-16561 + # for i in ['stb','db.stb','stb','db.stb']: + # tdSql.query(f"select last(*) from {i}") + # tdSql.checkRows(1) + # tdSql.checkData(0, 1, None) + for i in column_dict.keys(): + for j in [f'{stbname}_1', f'db.{stbname}_1', f'{stbname}', f'db.{stbname}']: + tdSql.query(f"select last({i}) from {j}") + tdSql.checkRows(0) + tdSql.query(f"select last({list(column_dict.keys())[0]}) from {stbname}_1 group by {list(column_dict.keys())[-1]}") tdSql.checkRows(1) - tdSql.checkData(0, 1, None) - tdSql.query("select last(*) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 1, None) - tdSql.query("select last(col1) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col1) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col2) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col2) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col3) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col3) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col4) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col4) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col11) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col11) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col12) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col12) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col13) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col13) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col14) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col14) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col5) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col5) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col6) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col6) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col7) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col7) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col8) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col8) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col9) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col9) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select count(col1) from stb_1 group by col7") - tdSql.checkRows(1) - for i in range(self.rowNum): - tdSql.execute("insert into stb_1 values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)" - % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1)) + tdSql.execute(f"insert into {stbname}_1 values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) + for i in [f'{stbname}_1', f'db.{stbname}_1', f'{stbname}', f'db.{stbname}']: + tdSql.query(f"select last(*) from {i}") + tdSql.checkRows(1) + tdSql.checkData(0, 1, 10) + for k, v in column_dict.items(): + for j in [f'{stbname}_1', f'db.{stbname}_1', f'{stbname}', f'db.{stbname}']: + tdSql.query(f"select last({k}) from {j}") + tdSql.checkRows(1) + # tinyint,smallint,int,bigint,tinyint unsigned,smallint unsigned,int unsigned,bigint unsigned + if v.lower() == 'tinyint' or v.lower() == 'smallint' or v.lower() == 'int' or v.lower() == 'bigint' or v.lower() == 'tinyint unsigned' or v.lower() == 'smallint unsigned'\ + or v.lower() == 'int unsigned' or v.lower() == 'bigint unsigned': + tdSql.checkData(0, 0, 10) + # float,double + elif v.lower() == 'float' or v.lower() == 'double': + tdSql.checkData(0, 0, 9.1) + # bool + elif v.lower() == 'bool': + tdSql.checkData(0, 0, True) + # binary + elif 'binary' in v.lower(): + tdSql.checkData(0, 0, f'{self.binary_str}{self.rowNum}') + # nchar + elif 'nchar' in v.lower(): + tdSql.checkData(0, 0, f'{self.nchar_str}{self.rowNum}') + for i in [f'{stbname}_1', f'db.{stbname}_1', f'{stbname}', f'db.{stbname}']: + tdSql.query(f"select last({list(column_dict.keys())[0]},{list(column_dict.keys())[1]},{list(column_dict.keys())[2]}) from {stbname}_1") + tdSql.checkData(0, 2, 10) - tdSql.query("select last(*) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 1, 10) - tdSql.query("select last(*) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 1, 10) - tdSql.query("select last(col1) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col1) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col2) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col2) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col3) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col3) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col4) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col4) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col11) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col11) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col12) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col12) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col13) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col13) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col14) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col14) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col5) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col5) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col6) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col6) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col7) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, True) - tdSql.query("select last(col7) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, True) - tdSql.query("select last(col8) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 'taosdata10') - tdSql.query("select last(col8) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 'taosdata10') - tdSql.query("select last(col9) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, '涛思数据10') - tdSql.query("select last(col9) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, '涛思数据10') - tdSql.query("select last(col1,col2,col3) from stb_1") - tdSql.checkData(0,2,10) + tdSql.error(f"select {list(column_dict.keys())[0]} from {stbname} where last({list(column_dict.keys())[12]})='涛思数据10'") + tdSql.error(f"select {list(column_dict.keys())[0]} from {stbname}_1 where last({list(column_dict.keys())[12]})='涛思数据10'") + tdSql.execute('drop database db') - tdSql.query("select last(*) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 1, 10) - tdSql.query("select last(*) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 1, 10) - tdSql.query("select last(col1) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col1) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col2) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col2) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col3) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col3) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col4) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col4) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col11) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col11) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col12) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col12) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col13) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col13) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col14) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col14) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col5) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col5) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col6) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col6) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col7) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, True) - tdSql.query("select last(col7) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, True) - tdSql.query("select last(col8) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 'taosdata10') - tdSql.query("select last(col8) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 'taosdata10') - tdSql.query("select last(col9) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, '涛思数据10') - tdSql.query("select last(col9) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, '涛思数据10') - tdSql.query("select last(col1,col2,col3) from stb") - tdSql.checkData(0,2,10) - - - tdSql.execute('''create table ntb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double, - col7 bool, col8 binary(20), col9 nchar(20), col11 tinyint unsigned, col12 smallint unsigned, col13 int unsigned, col14 bigint unsigned)''') - tdSql.execute("insert into ntb(ts) values(%d)" % (self.ts - 1)) - tdSql.query("select last(*) from ntb") + def last_check_ntb_base(self): + tdSql.prepare() + ntbname = tdCom.getLongName(5, "letters") + column_dict = { + 'col1': 'tinyint', + 'col2': 'smallint', + 'col3': 'int', + 'col4': 'bigint', + 'col5': 'tinyint unsigned', + 'col6': 'smallint unsigned', + 'col7': 'int unsigned', + 'col8': 'bigint unsigned', + 'col9': 'float', + 'col10': 'double', + 'col11': 'bool', + 'col12': 'binary(20)', + 'col13': 'nchar(20)' + } + create_ntb_sql = self.set_create_normaltable_sql(ntbname, column_dict) + tdSql.execute(create_ntb_sql) + tdSql.execute(f"insert into {ntbname}(ts) values(%d)" % (self.ts - 1)) + tdSql.query(f"select last(*) from {ntbname}") tdSql.checkRows(1) tdSql.checkData(0, 1, None) - tdSql.query("select last(*) from db.ntb") + tdSql.query(f"select last(*) from db.{ntbname}") tdSql.checkRows(1) tdSql.checkData(0, 1, None) - tdSql.query("select last(col1) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col1) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col2) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col2) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col3) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col3) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col4) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col4) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col11) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col11) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col12) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col12) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col13) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col13) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col14) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col14) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col5) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col5) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col6) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col6) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col7) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col7) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col8) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col8) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col9) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col9) from db.ntb") - tdSql.checkRows(0) - + for i in column_dict.keys(): + for j in [f'{ntbname}', f'db.{ntbname}']: + tdSql.query(f"select last({i}) from {j}") + tdSql.checkRows(0) for i in range(self.rowNum): - tdSql.execute("insert into ntb values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)" - % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1)) - - tdSql.query("select last(*) from ntb") + tdSql.execute(f"insert into {ntbname} values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) + tdSql.query(f"select last(*) from {ntbname}") tdSql.checkRows(1) tdSql.checkData(0, 1, 10) - tdSql.query("select last(*) from db.ntb") + tdSql.query(f"select last(*) from db.{ntbname}") tdSql.checkRows(1) tdSql.checkData(0, 1, 10) - tdSql.query("select last(col1) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col1) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col2) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col2) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col3) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col3) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col4) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col4) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col11) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col11) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col12) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col12) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col13) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col13) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col14) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col14) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col5) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col5) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col6) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col6) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col7) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, True) - tdSql.query("select last(col7) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, True) - tdSql.query("select last(col8) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 'taosdata10') - tdSql.query("select last(col8) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 'taosdata10') - tdSql.query("select last(col9) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, '涛思数据10') - tdSql.query("select last(col9) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, '涛思数据10') - tdSql.query("select last(col1,col2,col3) from ntb") - tdSql.checkData(0,2,10) + for k, v in column_dict.items(): + for j in [f'{ntbname}', f'db.{ntbname}']: + tdSql.query(f"select last({k}) from {j}") + tdSql.checkRows(1) + # tinyint,smallint,int,bigint,tinyint unsigned,smallint unsigned,int unsigned,bigint unsigned + if v.lower() == 'tinyint' or v.lower() == 'smallint' or v.lower() == 'int' or v.lower() == 'bigint' or v.lower() == 'tinyint unsigned' or v.lower() == 'smallint unsigned'\ + or v.lower() == 'int unsigned' or v.lower() == 'bigint unsigned': + tdSql.checkData(0, 0, 10) + # float,double + elif v.lower() == 'float' or v.lower() == 'double': + tdSql.checkData(0, 0, 9.1) + # bool + elif v.lower() == 'bool': + tdSql.checkData(0, 0, True) + # binary + elif 'binary' in v.lower(): + tdSql.checkData(0, 0, f'{self.binary_str}{self.rowNum}') + # nchar + elif 'nchar' in v.lower(): + tdSql.checkData(0, 0, f'{self.nchar_str}{self.rowNum}') + + tdSql.error( + f"select {list(column_dict.keys())[0]} from {ntbname} where last({list(column_dict.keys())[9]})='涛思数据10'") + + def last_check_stb_distribute(self): + # prepare data for vgroup 4 + dbname = tdCom.getLongName(10, "letters") + stbname = tdCom.getLongName(5, "letters") + vgroup_num = 4 + column_dict = { + 'col1': 'tinyint', + 'col2': 'smallint', + 'col3': 'int', + 'col4': 'bigint', + 'col5': 'tinyint unsigned', + 'col6': 'smallint unsigned', + 'col7': 'int unsigned', + 'col8': 'bigint unsigned', + 'col9': 'float', + 'col10': 'double', + 'col11': 'bool', + 'col12': 'binary(20)', + 'col13': 'nchar(20)' + } + + tdSql.execute( + f"create database if not exists {dbname} vgroups {vgroup_num}") + tdSql.execute(f'use {dbname}') + + # build 20 child tables,every table insert 10 rows + tdSql.execute(f'''create table {stbname}(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') + for i in range(self.tbnum): + tdSql.execute( + f"create table {stbname}_{i} using {stbname} tags('beijing')") + tdSql.execute( + f"insert into {stbname}_{i}(ts) values(%d)" % (self.ts - 1-i)) + # for i in [f'{stbname}', f'{dbname}.{stbname}']: + # tdSql.query(f"select last(*) from {i}") + # tdSql.checkRows(1) + # tdSql.checkData(0, 1, None) + tdSql.query('show tables') + vgroup_list = [] + for i in range(len(tdSql.queryResult)): + vgroup_list.append(tdSql.queryResult[i][6]) + vgroup_list_set = set(vgroup_list) + for i in vgroup_list_set: + vgroups_num = vgroup_list.count(i) + if vgroups_num >= 2: + tdLog.info(f'This scene with {vgroups_num} vgroups is ok!') + continue + else: + tdLog.exit( + 'This scene does not meet the requirements with {vgroups_num} vgroup!\n') + + for i in range(self.tbnum): + for j in range(self.rowNum): + tdSql.execute(f"insert into {stbname}_{i} values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + j + i, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 0.1, j + 0.1, j % 2, j + 1, j + 1)) + for i in [f'{stbname}', f'{dbname}.{stbname}']: + tdSql.query(f"select last(*) from {i}") + tdSql.checkRows(1) + tdSql.checkData(0, 1, 10) + for k, v in column_dict.items(): + for j in [f'{stbname}', f'{dbname}.{stbname}']: + tdSql.query(f"select last({k}) from {j}") + tdSql.checkRows(1) + # tinyint,smallint,int,bigint,tinyint unsigned,smallint unsigned,int unsigned,bigint unsigned + if v.lower() == 'tinyint' or v.lower() == 'smallint' or v.lower() == 'int' or v.lower() == 'bigint' or v.lower() == 'tinyint unsigned' or v.lower() == 'smallint unsigned'\ + or v.lower() == 'int unsigned' or v.lower() == 'bigint unsigned': + tdSql.checkData(0, 0, 10) + # float,double + elif v.lower() == 'float' or v.lower() == 'double': + tdSql.checkData(0, 0, 9.1) + # bool + elif v.lower() == 'bool': + tdSql.checkData(0, 0, True) + # binary + elif 'binary' in v.lower(): + tdSql.checkData(0, 0, f'{self.binary_str}{self.rowNum}') + # nchar + elif 'nchar' in v.lower(): + tdSql.checkData(0, 0, f'{self.nchar_str}{self.rowNum}') + tdSql.execute(f'drop database {dbname}') + + def run(self): + self.last_check_stb_tb_base() + self.last_check_ntb_base() + self.last_check_stb_distribute() - tdSql.error("select col1 from stb where last(col9)='涛思数据10'") - tdSql.error("select col1 from ntb where last(col9)='涛思数据10'") - tdSql.error("select col1 from stb_1 where last(col9)='涛思数据10'") def stop(self): tdSql.close() tdLog.success("%s successfully executed" % __file__) + tdCases.addWindows(__file__, TDTestCase()) -tdCases.addLinux(__file__, TDTestCase()) \ No newline at end of file +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/system-test/2-query/max.py b/tests/system-test/2-query/max.py index 5342c7d449..46426a5a95 100644 --- a/tests/system-test/2-query/max.py +++ b/tests/system-test/2-query/max.py @@ -5,198 +5,213 @@ import numpy as np class TDTestCase: + updatecfgDict = {'debugFlag': 143 ,"cDebugFlag":143,"uDebugFlag":143 ,"rpcDebugFlag":143 , "tmrDebugFlag":143 , + "jniDebugFlag":143 ,"simDebugFlag":143,"dDebugFlag":143, "dDebugFlag":143,"vDebugFlag":143,"mDebugFlag":143,"qDebugFlag":143, + "wDebugFlag":143,"sDebugFlag":143,"tsdbDebugFlag":143,"tqDebugFlag":143 ,"fsDebugFlag":143 ,"fnDebugFlag":143, + "maxTablesPerVnode":2 ,"minTablesPerVnode":2,"tableIncStepPerVnode":2 } def init(self, conn, logSql): tdLog.debug("start to execute %s" % __file__) tdSql.init(conn.cursor()) self.rowNum = 10 self.ts = 1537146000000 - - def prepare_data(self): - - pass - def run(self): + self.binary_str = 'taosdata' + self.nchar_str = '涛思数据' + def max_check_stb_and_tb_base(self): tdSql.prepare() - intData = [] floatData = [] - - tdSql.execute('''create table stb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double, - col7 bool, col8 binary(20), col9 nchar(20), col11 tinyint unsigned, col12 smallint unsigned, col13 int unsigned, col14 bigint unsigned) tags(loc nchar(20))''') + tdSql.execute('''create table stb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') tdSql.execute("create table stb_1 using stb tags('beijing')") - tdSql.execute('''create table ntb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double, - col7 bool, col8 binary(20), col9 nchar(20), col11 tinyint unsigned, col12 smallint unsigned, col13 int unsigned, col14 bigint unsigned)''') for i in range(self.rowNum): - tdSql.execute("insert into ntb values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)" - % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1)) + tdSql.execute(f"insert into stb_1 values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) intData.append(i + 1) floatData.append(i + 0.1) - for i in range(self.rowNum): - tdSql.execute("insert into stb_1 values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)" - % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1)) - intData.append(i + 1) - floatData.append(i + 0.1) + for i in ['ts','col11','col12','col13']: + for j in ['db.stb','stb','db.stb_1','stb_1']: + tdSql.error(f'select max({i} from {j} )') - # max verifacation - tdSql.error("select max(ts) from stb_1") - tdSql.error("select max(ts) from db.stb_1") - tdSql.error("select max(col7) from stb_1") - tdSql.error("select max(col7) from db.stb_1") - tdSql.error("select max(col8) from stb_1") - tdSql.error("select max(col8) from db.stb_1") - tdSql.error("select max(col9) from stb_1") - tdSql.error("select max(col9) from db.stb_1") - - tdSql.query("select max(col1) from stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col1) from db.stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col2) from stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col2) from db.stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col3) from stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col3) from db.stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col4) from stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col4) from db.stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col11) from stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col11) from db.stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col12) from stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col12) from db.stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col13) from stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col13) from db.stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col14) from stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col14) from db.stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col5) from stb_1") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col5) from db.stb_1") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col6) from stb_1") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col6) from db.stb_1") - tdSql.checkData(0, 0, np.max(floatData)) + for i in range(1,11): + for j in ['db.stb','stb','db.stb_1','stb_1']: + tdSql.query(f"select max(col{i}) from {j}") + if i<9: + tdSql.checkData(0, 0, np.max(intData)) + elif i>=9: + tdSql.checkData(0, 0, np.max(floatData)) tdSql.query("select max(col1) from stb_1 where col2<=5") tdSql.checkData(0,0,5) - - - - tdSql.error("select max(ts) from stb") - tdSql.error("select max(ts) from db.stb") - tdSql.error("select max(col7) from stb") - tdSql.error("select max(col7) from db.stb") - tdSql.error("select max(col8) from stb") - tdSql.error("select max(col8) from db.stb") - tdSql.error("select max(col9) from stb") - tdSql.error("select max(col9) from db.stb") - - tdSql.query("select max(col1) from stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col1) from db.stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col2) from stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col2) from db.stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col3) from stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col3) from db.stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col4) from stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col4) from db.stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col11) from stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col11) from db.stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col12) from stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col12) from db.stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col13) from stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col13) from db.stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col14) from stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col14) from db.stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col5) from stb") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col5) from db.stb") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col6) from stb") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col6) from db.stb") - tdSql.checkData(0, 0, np.max(floatData)) tdSql.query("select max(col1) from stb where col2<=5") tdSql.checkData(0,0,5) - - - - tdSql.error("select max(ts) from ntb") - tdSql.error("select max(ts) from db.ntb") - tdSql.error("select max(col7) from ntb") - tdSql.error("select max(col7) from db.ntb") - tdSql.error("select max(col8) from ntb") - tdSql.error("select max(col8) from db.ntb") - tdSql.error("select max(col9) from ntb") - tdSql.error("select max(col9) from db.ntb") - - tdSql.query("select max(col1) from ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col1) from db.ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col2) from ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col2) from db.ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col3) from ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col3) from db.ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col4) from ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col4) from db.ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col11) from ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col11) from db.ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col12) from ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col12) from db.ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col13) from ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col13) from db.ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col14) from ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col14) from db.ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col5) from ntb") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col5) from db.ntb") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col6) from ntb") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col6) from db.ntb") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col1) from stb_1 where col2<=5") + tdSql.execute('drop database db') + + def max_check_ntb_base(self): + tdSql.prepare() + intData = [] + floatData = [] + tdSql.execute('''create table ntb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20))''') + for i in range(self.rowNum): + tdSql.execute(f"insert into ntb values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) + intData.append(i + 1) + floatData.append(i + 0.1) + for i in ['ts','col11','col12','col13']: + for j in ['db.ntb','ntb']: + tdSql.error(f'select max({i} from {j} )') + for i in range(1,11): + for j in ['db.ntb','ntb']: + tdSql.query(f"select max(col{i}) from {j}") + if i<9: + tdSql.checkData(0, 0, np.max(intData)) + elif i>=9: + tdSql.checkData(0, 0, np.max(floatData)) + tdSql.query("select max(col1) from ntb where col2<=5") tdSql.checkData(0,0,5) + tdSql.execute('drop database db') + + + def check_max_functions(self, tbname , col_name): + + max_sql = f"select max({col_name}) from {tbname};" + + same_sql = f"select {col_name} from {tbname} order by {col_name} desc limit 1" + + tdSql.query(max_sql) + max_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if max_result !=same_result: + tdLog.exit(" max function work not as expected, sql : %s "% max_sql) + else: + tdLog.info(" max function work as expected, sql : %s "% max_sql) + + + def support_distributed_aggregate(self): + + # prepate datas for 20 tables distributed at different vgroups + tdSql.execute("create database if not exists testdb keep 3650 duration 1000 vgroups 5") + tdSql.execute(" use testdb ") + tdSql.execute( + '''create table stb1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + tags (t0 timestamp, t1 int, t2 bigint, t3 smallint, t4 tinyint, t5 float, t6 double, t7 bool, t8 binary(16),t9 nchar(32)) + ''' + ) + + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(20): + tdSql.execute(f'create table ct{i+1} using stb1 tags ( now(), {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, "binary{i}", "nchar{i}" )') + + for i in range(9): + tdSql.execute( + f"insert into ct1 values ( now()-{i*10}s, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + tdSql.execute( + f"insert into ct4 values ( now()-{i*90}d, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + + for i in range(1,21): + if i ==1 or i == 4: + continue + else: + tbname = "ct"+f'{i}' + for j in range(9): + tdSql.execute( + f"insert into {tbname} values ( now()-{(i+j)*10}s, {1*(j+i)}, {11111*(j+i)}, {111*(j+i)}, {11*(j)}, {1.11*(j+i)}, {11.11*(j+i)}, {(j+i)%2}, 'binary{j}', 'nchar{j}', now()+{1*j}a )" + ) + tdSql.execute("insert into ct1 values (now()-45s, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar0', now()+8a )") + tdSql.execute("insert into ct1 values (now()+10s, 9, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+15s, 9, -99999, -999, -99, -9.99, NULL, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+20s, 9, -99999, -999, NULL, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + + tdSql.execute("insert into ct4 values (now()-810d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()-400d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()+90d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + + tdSql.execute( + f'''insert into t1 values + ( '2020-04-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2020-10-21 01:01:01.000', 1, 11111, 111, 11, 1.11, 11.11, 1, "binary1", "nchar1", now()+1a ) + ( '2020-12-31 01:01:01.000', 2, 22222, 222, 22, 2.22, 22.22, 0, "binary2", "nchar2", now()+2a ) + ( '2021-01-01 01:01:06.000', 3, 33333, 333, 33, 3.33, 33.33, 0, "binary3", "nchar3", now()+3a ) + ( '2021-05-07 01:01:10.000', 4, 44444, 444, 44, 4.44, 44.44, 1, "binary4", "nchar4", now()+4a ) + ( '2021-07-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2021-09-30 01:01:16.000', 5, 55555, 555, 55, 5.55, 55.55, 0, "binary5", "nchar5", now()+5a ) + ( '2022-02-01 01:01:20.000', 6, 66666, 666, 66, 6.66, 66.66, 1, "binary6", "nchar6", now()+6a ) + ( '2022-10-28 01:01:26.000', 7, 00000, 000, 00, 0.00, 00.00, 1, "binary7", "nchar7", "1970-01-01 08:00:00.000" ) + ( '2022-12-01 01:01:30.000', 8, -88888, -888, -88, -8.88, -88.88, 0, "binary8", "nchar8", "1969-01-01 01:00:00.000" ) + ( '2022-12-31 01:01:36.000', 9, -99999999999999999, -999, -99, -9.99, -999999999999999999999.99, 1, "binary9", "nchar9", "1900-01-01 00:00:00.000" ) + ( '2023-02-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ''' + ) + + tdLog.info(" prepare data for distributed_aggregate done! ") + + # get vgroup_ids of all + tdSql.query("show vgroups ") + vgroups = tdSql.queryResult + + vnode_tables={} + + for vgroup_id in vgroups: + vnode_tables[vgroup_id[0]]=[] + + + # check sub_table of per vnode ,make sure sub_table has been distributed + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + vnode_tables[table_name[6]].append(table_name[0]) + + count = 0 + for k ,v in vnode_tables.items(): + if len(v)>=2: + count+=1 + if count < 2: + tdLog.exit(" the datas of all not satisfy sub_table has been distributed ") + + # check max function work status + + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + tablenames.append(table_name[0]) + + tdSql.query("desc stb1") + col_names = tdSql.queryResult + + colnames = [] + for col_name in col_names: + if col_name[1] in ["INT" ,"BIGINT" ,"SMALLINT" ,"TINYINT" , "FLOAT" ,"DOUBLE"]: + colnames.append(col_name[0]) + + for tablename in tablenames: + for colname in colnames: + self.check_max_functions(tablename,colname) + + # max function with basic filter + print(vnode_tables) + + + def run(self): + + # max verifacation + self.max_check_stb_and_tb_base() + self.max_check_ntb_base() + + self.support_distributed_aggregate() + def stop(self): tdSql.close() tdLog.success("%s successfully executed" % __file__) diff --git a/tests/system-test/2-query/top.py b/tests/system-test/2-query/top.py index 146bb34937..83f535856e 100644 --- a/tests/system-test/2-query/top.py +++ b/tests/system-test/2-query/top.py @@ -11,6 +11,9 @@ # -*- coding: utf-8 -*- +import random +import string +from util.common import * from util.log import * from util.cases import * from util.sql import * @@ -22,82 +25,89 @@ class TDTestCase: tdSql.init(conn.cursor()) self.rowNum = 10 + self.tbnum = 20 self.ts = 1537146000000 - - def run(self): + self.binary_str = 'taosdata' + self.nchar_str = '涛思数据' + def top_check_base(self): tdSql.prepare() - - - - tdSql.execute('''create table test(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double, - col7 bool, col8 binary(20), col9 nchar(20), col11 tinyint unsigned, col12 smallint unsigned, col13 int unsigned, col14 bigint unsigned) tags(loc nchar(20))''') - tdSql.execute("create table test1 using test tags('beijing')") + tdSql.execute('''create table stb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') + tdSql.execute("create table stb_1 using stb tags('beijing')") for i in range(self.rowNum): - tdSql.execute("insert into test1 values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)" - % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1)) - - - # top verifacation - tdSql.error("select top(ts, 10) from test") - tdSql.error("select top(col1, 0) from test") - tdSql.error("select top(col1, 101) from test") - tdSql.error("select top(col2, 0) from test") - tdSql.error("select top(col2, 101) from test") - tdSql.error("select top(col3, 0) from test") - tdSql.error("select top(col3, 101) from test") - tdSql.error("select top(col4, 0) from test") - tdSql.error("select top(col4, 101) from test") - tdSql.error("select top(col5, 0) from test") - tdSql.error("select top(col5, 101) from test") - tdSql.error("select top(col6, 0) from test") - tdSql.error("select top(col6, 101) from test") - tdSql.error("select top(col7, 10) from test") - tdSql.error("select top(col8, 10) from test") - tdSql.error("select top(col9, 10) from test") - tdSql.error("select top(col11, 0) from test") - tdSql.error("select top(col11, 101) from test") - tdSql.error("select top(col12, 0) from test") - tdSql.error("select top(col12, 101) from test") - tdSql.error("select top(col13, 0) from test") - tdSql.error("select top(col13, 101) from test") - tdSql.error("select top(col14, 0) from test") - tdSql.error("select top(col14, 101) from test") - - tdSql.query("select top(col1, 2) from test") + tdSql.execute(f"insert into stb_1 values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) + column_list = ['col1','col2','col3','col4','col5','col6','col7','col8'] + error_column_list = ['col11','col12','col13'] + error_param_list = [0,101] + for i in column_list: + tdSql.query(f'select top({i},2) from stb_1') + tdSql.checkRows(2) + tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) + for j in error_param_list: + tdSql.error(f'select top({i},{j}) from stb_1') + for i in error_column_list: + tdSql.error(f'select top({i},10) from stb_1') + tdSql.query("select ts,top(col1, 2),ts from stb_1 group by tbname") tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) - tdSql.query("select top(col2, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) - tdSql.query("select top(col3, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) - tdSql.query("select top(col4, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) - tdSql.query("select top(col11, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) - tdSql.query("select top(col12, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) - tdSql.query("select top(col13, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) - tdSql.query("select top(col14, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) - tdSql.query("select ts,top(col1, 2),ts from test1") - tdSql.checkRows(2) - tdSql.query("select top(col14, 100) from test") - tdSql.checkRows(10) - tdSql.query("select ts,top(col1, 2),ts from test group by tbname") - tdSql.checkRows(2) - tdSql.query('select top(col2,1) from test interval(1y) order by col2') + tdSql.query('select top(col2,1) from stb_1 interval(1y) order by col2') tdSql.checkData(0,0,10) + tdSql.error("select * from stb_1 where top(col2,1)=1") + tdSql.execute('drop database db') + def top_check_stb_distribute(self): + # prepare data for vgroup 4 + dbname = tdCom.getLongName(10, "letters") + stbname = tdCom.getLongName(5, "letters") + tdSql.execute(f"create database if not exists {dbname} vgroups 2") + tdSql.execute(f'use {dbname}') + # build 20 child tables,every table insert 10 rows + tdSql.execute(f'''create table {stbname}(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') + for i in range(self.tbnum): + tdSql.execute(f"create table {stbname}_{i} using {stbname} tags('beijing')") + tdSql.execute(f"insert into {stbname}_{i}(ts) values(%d)" % (self.ts - 1-i)) + column_list = ['col1','col2','col3','col4','col5','col6','col7','col8'] + for i in [f'{stbname}', f'{dbname}.{stbname}']: + for j in column_list: + tdSql.query(f"select top({j},1) from {i}") + tdSql.checkRows(0) + tdSql.query('show tables') + vgroup_list = [] + for i in range(len(tdSql.queryResult)): + vgroup_list.append(tdSql.queryResult[i][6]) + vgroup_list_set = set(vgroup_list) - tdSql.error("select * from test where bottom(col2,1)=1") - tdSql.error("select top(col14, 0) from test;") + for i in vgroup_list_set: + vgroups_num = vgroup_list.count(i) + if vgroups_num >=2: + tdLog.info(f'This scene with {vgroups_num} vgroups is ok!') + continue + else: + tdLog.exit(f'This scene does not meet the requirements with {vgroups_num} vgroup!\n') + for i in range(self.rowNum): + for j in range(self.tbnum): + tdSql.execute(f"insert into {stbname}_{j} values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) + + error_column_list = ['col11','col12','col13'] + error_param_list = [0,101] + for i in column_list: + tdSql.query(f'select top({i},2) from {stbname}') + tdSql.checkRows(2) + tdSql.checkEqual(tdSql.queryResult,[(10,),(10,)]) + for j in error_param_list: + tdSql.error(f'select top({i},{j}) from {stbname}') + for i in error_column_list: + tdSql.error(f'select top({i},10) from {stbname}') + + tdSql.query(f"select ts,top(col1, 2),ts from {stbname} group by tbname") + tdSql.checkRows(2*self.tbnum) + tdSql.query(f'select top(col2,1) from {stbname} interval(1y) order by col2') + tdSql.checkData(0,0,10) + tdSql.error(f"select * from {stbname} where top(col2,1)=1") + def run(self): + self.top_check_base() + self.top_check_stb_distribute() def stop(self): tdSql.close() tdLog.success("%s successfully executed" % __file__) diff --git a/tests/system-test/6-cluster/5dnode3mnodeDrop.py b/tests/system-test/6-cluster/5dnode3mnodeDrop.py index f999a16b05..b98134f5e0 100644 --- a/tests/system-test/6-cluster/5dnode3mnodeDrop.py +++ b/tests/system-test/6-cluster/5dnode3mnodeDrop.py @@ -269,7 +269,8 @@ class TDTestCase: tdSql.query("show dnodes;") print(tdSql.queryResult) - # drop and follower of mnode + + # drop follower of mnode dropcount =0 while dropcount <= 10: for i in range(1,3): diff --git a/tests/system-test/6-cluster/5dnode3mnodeDropInsert.py b/tests/system-test/6-cluster/5dnode3mnodeDropInsert.py new file mode 100644 index 0000000000..cfa3920604 --- /dev/null +++ b/tests/system-test/6-cluster/5dnode3mnodeDropInsert.py @@ -0,0 +1,428 @@ +from ssl import ALERT_DESCRIPTION_CERTIFICATE_UNOBTAINABLE +import taos +import sys +import time +import os + +from util.log import * +from util.sql import * +from util.cases import * +from util.dnodes import TDDnodes +from util.dnodes import TDDnode +import time +import socket +import subprocess +from multiprocessing import Process +import threading +import time +import inspect +import ctypes +class MyDnodes(TDDnodes): + def __init__(self ,dnodes_lists): + super(MyDnodes,self).__init__() + self.dnodes = dnodes_lists # dnode must be TDDnode instance + self.simDeployed = False + + +class TDTestCase: + + def init(self,conn ,logSql): + tdLog.debug(f"start to excute {__file__}") + self.TDDnodes = None + self.ts = 1500000000000 + + def buildcluster(self,dnodenumber): + self.depoly_cluster(dnodenumber) + self.master_dnode = self.TDDnodes.dnodes[0] + self.host=self.master_dnode.cfgDict["fqdn"] + conn1 = taos.connect(self.master_dnode.cfgDict["fqdn"] , config=self.master_dnode.cfgDir) + tdSql.init(conn1.cursor()) + + + def getBuildPath(self): + selfPath = os.path.dirname(os.path.realpath(__file__)) + + if ("community" in selfPath): + projPath = selfPath[:selfPath.find("community")] + else: + projPath = selfPath[:selfPath.find("tests")] + + for root, dirs, files in os.walk(projPath): + if ("taosd" in files): + rootRealPath = os.path.dirname(os.path.realpath(root)) + if ("packaging" not in rootRealPath): + buildPath = root[:len(root) - len("/build/bin")] + break + return buildPath + + def _async_raise(self, tid, exctype): + """raises the exception, performs cleanup if needed""" + if not inspect.isclass(exctype): + exctype = type(exctype) + res = ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, ctypes.py_object(exctype)) + if res == 0: + raise ValueError("invalid thread id") + elif res != 1: + # """if it returns a number greater than one, you're in trouble, + # and you should call it again with exc=NULL to revert the effect""" + ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, None) + raise SystemError("PyThreadState_SetAsyncExc failed") + + def stop_thread(self,thread): + self._async_raise(thread.ident, SystemExit) + + + def createDbTbale(self,dbcountStart,dbcountStop,stbname,chilCount): + # fisrt add data : db\stable\childtable\general table + + for couti in range(dbcountStart,dbcountStop): + tdLog.debug("drop database if exists db%d" %couti) + tdSql.execute("drop database if exists db%d" %couti) + print("create database if not exists db%d replica 1 duration 300" %couti) + tdSql.execute("create database if not exists db%d replica 1 duration 300" %couti) + tdSql.execute("use db%d" %couti) + tdSql.execute( + '''create table %s + (ts timestamp, c1 int, c2 bigint,c3 binary(16), c4 timestamp) + tags (t1 int) + '''%stbname + ) + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(chilCount): + tdSql.execute(f'create table {stbname}_{i+1} using {stbname} tags ( {i+1} )') + + def insertTabaleData(self,dbcountStart,dbcountStop,stbname,chilCount,ts_start,rowCount): + # insert data : create childtable and data + + for couti in range(dbcountStart,dbcountStop): + tdSql.execute("use db%d" %couti) + pre_insert = "insert into " + sql = pre_insert + chilCount=int(chilCount) + allRows=chilCount*rowCount + tdLog.debug("doing insert data into stable-index:%s rows:%d ..."%(stbname, allRows)) + exeStartTime=time.time() + for i in range(0,chilCount): + sql += " %s_%d values "%(stbname,i) + for j in range(rowCount): + sql += "(%d, %d, %d,'taos_%d',%d) "%(ts_start + j*1000, j, j, j, ts_start + j*1000) + if j >0 and j%4000 == 0: + # print(sql) + tdSql.execute(sql) + sql = "insert into %s_%d values " %(stbname,i) + # end sql + if sql != pre_insert: + # print(sql) + print(len(sql)) + tdSql.execute(sql) + exeEndTime=time.time() + spendTime=exeEndTime-exeStartTime + speedInsert=allRows/spendTime + tdLog.debug("spent %.2fs to INSERT %d rows into %s , insert rate is %.2f rows/s... [OK]"% (spendTime,allRows,stbname,speedInsert)) + + def checkData(self,dbname,stbname,stableCount,CtableCount,rowsPerSTable,): + tdSql.execute("use %s"%dbname) + tdSql.query("show stables") + tdSql.checkRows(stableCount) + tdSql.query("show tables") + tdSql.checkRows(CtableCount) + for i in range(stableCount): + tdSql.query("select count(*) from %s%d"%(stbname,i)) + tdSql.checkData(0,0,rowsPerSTable) + return + + + def depoly_cluster(self ,dnodes_nums): + + testCluster = False + valgrind = 0 + hostname = socket.gethostname() + dnodes = [] + start_port = 6030 + start_port_sec = 6130 + for num in range(1, dnodes_nums+1): + dnode = TDDnode(num) + dnode.addExtraCfg("firstEp", f"{hostname}:{start_port}") + dnode.addExtraCfg("fqdn", f"{hostname}") + dnode.addExtraCfg("serverPort", f"{start_port + (num-1)*100}") + dnode.addExtraCfg("monitorFqdn", hostname) + dnode.addExtraCfg("monitorPort", 7043) + dnode.addExtraCfg("secondEp", f"{hostname}:{start_port_sec}") + dnodes.append(dnode) + + self.TDDnodes = MyDnodes(dnodes) + self.TDDnodes.init("") + self.TDDnodes.setTestCluster(testCluster) + self.TDDnodes.setValgrind(valgrind) + self.TDDnodes.stopAll() + for dnode in self.TDDnodes.dnodes: + self.TDDnodes.deploy(dnode.index,{}) + + for dnode in self.TDDnodes.dnodes: + self.TDDnodes.starttaosd(dnode.index) + + # create cluster + for dnode in self.TDDnodes.dnodes[1:]: + # print(dnode.cfgDict) + dnode_id = dnode.cfgDict["fqdn"] + ":" +dnode.cfgDict["serverPort"] + dnode_first_host = dnode.cfgDict["firstEp"].split(":")[0] + dnode_first_port = dnode.cfgDict["firstEp"].split(":")[-1] + cmd = f" taos -h {dnode_first_host} -P {dnode_first_port} -s ' create dnode \"{dnode_id} \" ' ;" + print(cmd) + os.system(cmd) + + time.sleep(2) + tdLog.info(" create cluster with %d dnode done! " %dnodes_nums) + + def checkdnodes(self,dnodenumber): + count=0 + while count < 10: + time.sleep(1) + statusReadyBumber=0 + tdSql.query("show dnodes;") + if tdSql.checkRows(dnodenumber) : + print("dnode is %d nodes"%dnodenumber) + for i in range(dnodenumber): + if tdSql.queryResult[i][4] !='ready' : + status=tdSql.queryResult[i][4] + print("dnode:%d status is %s "%(i,status)) + break + else: + statusReadyBumber+=1 + print(statusReadyBumber) + if statusReadyBumber == dnodenumber : + print("all of %d mnodes is ready in 10s "%dnodenumber) + return True + break + count+=1 + else: + print("%d mnodes is not ready in 10s "%dnodenumber) + return False + + + def check3mnode(self): + count=0 + while count < 10: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3) : + print("mnode is three nodes") + if tdSql.queryResult[0][2]=='leader' : + if tdSql.queryResult[1][2]=='follower': + if tdSql.queryResult[2][2]=='follower': + print("three mnodes is ready in 10s") + break + elif tdSql.queryResult[0][2]=='follower' : + if tdSql.queryResult[1][2]=='leader': + if tdSql.queryResult[2][2]=='follower': + print("three mnodes is ready in 10s") + break + elif tdSql.queryResult[0][2]=='follower' : + if tdSql.queryResult[1][2]=='follower': + if tdSql.queryResult[2][2]=='leader': + print("three mnodes is ready in 10s") + break + count+=1 + else: + print("three mnodes is not ready in 10s ") + return -1 + + tdSql.query("show mnodes;") + tdSql.checkRows(3) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,3,'ready') + tdSql.checkData(1,1,'%s:6130'%self.host) + tdSql.checkData(1,3,'ready') + tdSql.checkData(2,1,'%s:6230'%self.host) + tdSql.checkData(2,3,'ready') + + def check3mnode1off(self): + count=0 + while count < 10: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3) : + print("mnode is three nodes") + if tdSql.queryResult[0][2]=='offline' : + if tdSql.queryResult[1][2]=='leader': + if tdSql.queryResult[2][2]=='follower': + print("stop mnodes on dnode 2 successfully in 10s") + break + elif tdSql.queryResult[1][2]=='follower': + if tdSql.queryResult[2][2]=='leader': + print("stop mnodes on dnode 2 successfully in 10s") + break + count+=1 + else: + print("stop mnodes on dnode 2 failed in 10s ") + return -1 + tdSql.error("drop mnode on dnode 1;") + + tdSql.query("show mnodes;") + tdSql.checkRows(3) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,2,'offline') + tdSql.checkData(0,3,'ready') + tdSql.checkData(1,1,'%s:6130'%self.host) + tdSql.checkData(1,3,'ready') + tdSql.checkData(2,1,'%s:6230'%self.host) + tdSql.checkData(2,3,'ready') + + def check3mnode2off(self): + count=0 + while count < 40: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3) : + print("mnode is three nodes") + if tdSql.queryResult[0][2]=='leader' : + if tdSql.queryResult[1][2]=='offline': + if tdSql.queryResult[2][2]=='follower': + print("stop mnodes on dnode 2 successfully in 10s") + break + count+=1 + else: + print("stop mnodes on dnode 2 failed in 10s ") + return -1 + tdSql.error("drop mnode on dnode 2;") + + tdSql.query("show mnodes;") + tdSql.checkRows(3) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,2,'leader') + tdSql.checkData(0,3,'ready') + tdSql.checkData(1,1,'%s:6130'%self.host) + tdSql.checkData(1,2,'offline') + tdSql.checkData(1,3,'ready') + tdSql.checkData(2,1,'%s:6230'%self.host) + tdSql.checkData(2,2,'follower') + tdSql.checkData(2,3,'ready') + + def check3mnode3off(self): + count=0 + while count < 10: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3) : + print("mnode is three nodes") + if tdSql.queryResult[0][2]=='leader' : + if tdSql.queryResult[2][2]=='offline': + if tdSql.queryResult[1][2]=='follower': + print("stop mnodes on dnode 3 successfully in 10s") + break + count+=1 + else: + print("stop mnodes on dnode 3 failed in 10s") + return -1 + tdSql.error("drop mnode on dnode 3;") + tdSql.query("show mnodes;") + tdSql.checkRows(3) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,2,'leader') + tdSql.checkData(0,3,'ready') + tdSql.checkData(1,1,'%s:6130'%self.host) + tdSql.checkData(1,2,'follower') + tdSql.checkData(1,3,'ready') + tdSql.checkData(2,1,'%s:6230'%self.host) + tdSql.checkData(2,2,'offline') + tdSql.checkData(2,3,'ready') + + def five_dnode_three_mnode(self,dnodenumber): + # testcase parameters + vgroups=1 + dbcountStart=0 + dbcountStop=1 + dbname="db" + stbname="stb" + tablesPerStb=1000 + rowsPerTable=100 + startTs=1640966400000 # 2022-01-01 00:00:00.000 + + tdSql.query("show dnodes;") + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(4,1,'%s:6430'%self.host) + tdSql.checkData(0,4,'ready') + tdSql.checkData(4,4,'ready') + tdSql.query("show mnodes;") + tdSql.checkRows(1) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,2,'leader') + tdSql.checkData(0,3,'ready') + + # fisr add three mnodes; + tdSql.execute("create mnode on dnode 2") + tdSql.execute("create mnode on dnode 3") + + # fisrt check statut ready + self.check3mnode() + + tdSql.error("create mnode on dnode 2") + tdSql.query("show dnodes;") + print(tdSql.queryResult) + tdLog.debug("stop all of mnode ") + + # drop follower of mnode and insert data + self.createDbTbale(dbcountStart, dbcountStop,stbname,tablesPerStb) + #(method) insertTabaleData: (dbcountStart: Any, dbcountStop: Any, stbname: Any, chilCount: Any, ts_start: Any, rowCount: Any) -> None + threads=threading.Thread(target=self.insertTabaleData, args=( + dbcountStart, + dbcountStop, + stbname, + tablesPerStb, + startTs, + rowsPerTable)) + + threads.start() + dropcount =0 + while dropcount <= 10: + for i in range(1,3): + tdLog.debug("drop mnode on dnode %d"%(i+1)) + tdSql.execute("drop mnode on dnode %d"%(i+1)) + tdSql.query("show mnodes;") + count=0 + while count<10: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(2): + print("drop mnode %d successfully"%(i+1)) + break + count+=1 + tdLog.debug("create mnode on dnode %d"%(i+1)) + tdSql.execute("create mnode on dnode %d"%(i+1)) + count=0 + while count<10: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3): + print("drop mnode %d successfully"%(i+1)) + break + count+=1 + dropcount+=1 + threads.join() + self.check3mnode() + + + + def getConnection(self, dnode): + host = dnode.cfgDict["fqdn"] + port = dnode.cfgDict["serverPort"] + config_dir = dnode.cfgDir + return taos.connect(host=host, port=int(port), config=config_dir) + + + def run(self): + # print(self.master_dnode.cfgDict) + self.buildcluster(5) + self.five_dnode_three_mnode(5) + + def stop(self): + tdSql.close() + tdLog.success(f"{__file__} successfully executed") + +tdCases.addLinux(__file__, TDTestCase()) +tdCases.addWindows(__file__, TDTestCase()) \ No newline at end of file diff --git a/tests/system-test/6-cluster/5dnode3mnodeSeperate1VnodeStopInsert.py b/tests/system-test/6-cluster/5dnode3mnodeSeperate1VnodeStopInsert.py new file mode 100644 index 0000000000..1739db09af --- /dev/null +++ b/tests/system-test/6-cluster/5dnode3mnodeSeperate1VnodeStopInsert.py @@ -0,0 +1,377 @@ +from ssl import ALERT_DESCRIPTION_CERTIFICATE_UNOBTAINABLE +import taos +import sys +import time +import os + +from util.log import * +from util.sql import * +from util.cases import * +from util.dnodes import TDDnodes +from util.dnodes import TDDnode +import time +import socket +import subprocess +from multiprocessing import Process +import threading +import time +import inspect +import ctypes +class MyDnodes(TDDnodes): + def __init__(self ,dnodes_lists): + super(MyDnodes,self).__init__() + self.dnodes = dnodes_lists # dnode must be TDDnode instance + self.simDeployed = False + + +class TDTestCase: + + def init(self,conn ,logSql): + tdLog.debug(f"start to excute {__file__}") + self.TDDnodes = None + + def buildcluster(self,dnodenumber): + self.depoly_cluster(dnodenumber) + self.master_dnode = self.TDDnodes.dnodes[0] + self.host=self.master_dnode.cfgDict["fqdn"] + conn1 = taos.connect(self.master_dnode.cfgDict["fqdn"] , config=self.master_dnode.cfgDir) + tdSql.init(conn1.cursor()) + + + def getBuildPath(self): + selfPath = os.path.dirname(os.path.realpath(__file__)) + + if ("community" in selfPath): + projPath = selfPath[:selfPath.find("community")] + else: + projPath = selfPath[:selfPath.find("tests")] + + for root, dirs, files in os.walk(projPath): + if ("taosd" in files): + rootRealPath = os.path.dirname(os.path.realpath(root)) + if ("packaging" not in rootRealPath): + buildPath = root[:len(root) - len("/build/bin")] + break + return buildPath + + def _async_raise(self, tid, exctype): + """raises the exception, performs cleanup if needed""" + if not inspect.isclass(exctype): + exctype = type(exctype) + res = ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, ctypes.py_object(exctype)) + if res == 0: + raise ValueError("invalid thread id") + elif res != 1: + # """if it returns a number greater than one, you're in trouble, + # and you should call it again with exc=NULL to revert the effect""" + ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, None) + raise SystemError("PyThreadState_SetAsyncExc failed") + + def stop_thread(self,thread): + self._async_raise(thread.ident, SystemExit) + + + def insert_data(self,countstart,countstop): + # fisrt add data : db\stable\childtable\general table + + for couti in range(countstart,countstop): + tdLog.debug("drop database if exists db%d" %couti) + tdSql.execute("drop database if exists db%d" %couti) + print("create database if not exists db%d replica 1 duration 300" %couti) + tdSql.execute("create database if not exists db%d replica 1 duration 300" %couti) + tdSql.execute("use db%d" %couti) + tdSql.execute( + '''create table stb1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + tags (t1 int) + ''' + ) + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(4): + tdSql.execute(f'create table ct{i+1} using stb1 tags ( {i+1} )') + + def checkData(self,dbname,stbname,stableCount,CtableCount,rowsPerSTable,): + tdSql.execute("use %s"%dbname) + tdSql.query("show stables") + tdSql.checkRows(stableCount) + tdSql.query("show tables") + tdSql.checkRows(CtableCount) + for i in range(stableCount): + tdSql.query("select count(*) from %s%d"%(stbname,i)) + tdSql.checkData(0,0,rowsPerSTable) + return + + def depoly_cluster(self ,dnodes_nums=5,independent=True): + + testCluster = False + valgrind = 0 + hostname = socket.gethostname() + dnodes = [] + start_port = 6030 + start_port_sec = 6130 + for num in range(1, dnodes_nums+1): + dnode = TDDnode(num) + dnode.addExtraCfg("firstEp", f"{hostname}:{start_port}") + dnode.addExtraCfg("fqdn", f"{hostname}") + dnode.addExtraCfg("serverPort", f"{start_port + (num-1)*100}") + dnode.addExtraCfg("monitorFqdn", hostname) + dnode.addExtraCfg("monitorPort", 7043) + dnode.addExtraCfg("secondEp", f"{hostname}:{start_port_sec}") + # configure three dnoe don't support vnodes + if independent and (num < 4): + dnode.addExtraCfg("supportVnodes", 0) + + dnodes.append(dnode) + + self.TDDnodes = MyDnodes(dnodes) + self.TDDnodes.init("") + self.TDDnodes.setTestCluster(testCluster) + self.TDDnodes.setValgrind(valgrind) + self.TDDnodes.stopAll() + for dnode in self.TDDnodes.dnodes: + self.TDDnodes.deploy(dnode.index,{}) + + for dnode in self.TDDnodes.dnodes: + self.TDDnodes.starttaosd(dnode.index) + + # create cluster + for dnode in self.TDDnodes.dnodes[1:]: + # print(dnode.cfgDict) + dnode_id = dnode.cfgDict["fqdn"] + ":" +dnode.cfgDict["serverPort"] + dnode_first_host = dnode.cfgDict["firstEp"].split(":")[0] + dnode_first_port = dnode.cfgDict["firstEp"].split(":")[-1] + cmd = f" taos -h {dnode_first_host} -P {dnode_first_port} -s ' create dnode \"{dnode_id} \" ' ;" + print(cmd) + os.system(cmd) + + time.sleep(2) + tdLog.info(" create cluster with %d dnode done! " %dnodes_nums) + + def checkdnodes(self,dnodenumber): + count=0 + while count < 100: + time.sleep(1) + statusReadyBumber=0 + tdSql.query("show dnodes;") + if tdSql.checkRows(dnodenumber) : + print("dnode is %d nodes"%dnodenumber) + for i in range(dnodenumber): + if tdSql.queryResult[i][4] !='ready' : + status=tdSql.queryResult[i][4] + print("dnode:%d status is %s "%(i,status)) + break + else: + statusReadyBumber+=1 + print(statusReadyBumber) + if statusReadyBumber == dnodenumber : + print("all of %d mnodes is ready in 10s "%dnodenumber) + return True + break + count+=1 + else: + print("%d mnodes is not ready in 10s "%dnodenumber) + return False + + + def check3mnode(self): + count=0 + while count < 10: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3) : + print("mnode is three nodes") + if tdSql.queryResult[0][2]=='leader' : + if tdSql.queryResult[1][2]=='follower': + if tdSql.queryResult[2][2]=='follower': + print("three mnodes is ready in 10s") + break + elif tdSql.queryResult[0][2]=='follower' : + if tdSql.queryResult[1][2]=='leader': + if tdSql.queryResult[2][2]=='follower': + print("three mnodes is ready in 10s") + break + elif tdSql.queryResult[0][2]=='follower' : + if tdSql.queryResult[1][2]=='follower': + if tdSql.queryResult[2][2]=='leader': + print("three mnodes is ready in 10s") + break + count+=1 + else: + print("three mnodes is not ready in 10s ") + return -1 + + tdSql.query("show mnodes;") + tdSql.checkRows(3) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,3,'ready') + tdSql.checkData(1,1,'%s:6130'%self.host) + tdSql.checkData(1,3,'ready') + tdSql.checkData(2,1,'%s:6230'%self.host) + tdSql.checkData(2,3,'ready') + + def check3mnode1off(self): + count=0 + while count < 10: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3) : + print("mnode is three nodes") + if tdSql.queryResult[0][2]=='offline' : + if tdSql.queryResult[1][2]=='leader': + if tdSql.queryResult[2][2]=='follower': + print("stop mnodes on dnode 2 successfully in 10s") + break + elif tdSql.queryResult[1][2]=='follower': + if tdSql.queryResult[2][2]=='leader': + print("stop mnodes on dnode 2 successfully in 10s") + break + count+=1 + else: + print("stop mnodes on dnode 2 failed in 10s ") + return -1 + tdSql.error("drop mnode on dnode 1;") + + tdSql.query("show mnodes;") + tdSql.checkRows(3) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,2,'offline') + tdSql.checkData(0,3,'ready') + tdSql.checkData(1,1,'%s:6130'%self.host) + tdSql.checkData(1,3,'ready') + tdSql.checkData(2,1,'%s:6230'%self.host) + tdSql.checkData(2,3,'ready') + + def check3mnode2off(self): + count=0 + while count < 40: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3) : + print("mnode is three nodes") + if tdSql.queryResult[0][2]=='leader' : + if tdSql.queryResult[1][2]=='offline': + if tdSql.queryResult[2][2]=='follower': + print("stop mnodes on dnode 2 successfully in 10s") + break + count+=1 + else: + print("stop mnodes on dnode 2 failed in 10s ") + return -1 + tdSql.error("drop mnode on dnode 2;") + + tdSql.query("show mnodes;") + tdSql.checkRows(3) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,2,'leader') + tdSql.checkData(0,3,'ready') + tdSql.checkData(1,1,'%s:6130'%self.host) + tdSql.checkData(1,2,'offline') + tdSql.checkData(1,3,'ready') + tdSql.checkData(2,1,'%s:6230'%self.host) + tdSql.checkData(2,2,'follower') + tdSql.checkData(2,3,'ready') + + def check3mnode3off(self): + count=0 + while count < 10: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3) : + print("mnode is three nodes") + if tdSql.queryResult[0][2]=='leader' : + if tdSql.queryResult[2][2]=='offline': + if tdSql.queryResult[1][2]=='follower': + print("stop mnodes on dnode 3 successfully in 10s") + break + count+=1 + else: + print("stop mnodes on dnode 3 failed in 10s") + return -1 + tdSql.error("drop mnode on dnode 3;") + tdSql.query("show mnodes;") + tdSql.checkRows(3) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,2,'leader') + tdSql.checkData(0,3,'ready') + tdSql.checkData(1,1,'%s:6130'%self.host) + tdSql.checkData(1,2,'follower') + tdSql.checkData(1,3,'ready') + tdSql.checkData(2,1,'%s:6230'%self.host) + tdSql.checkData(2,2,'offline') + tdSql.checkData(2,3,'ready') + + def five_dnode_three_mnode(self,dnodenumber): + tdSql.query("show dnodes;") + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(4,1,'%s:6430'%self.host) + tdSql.checkData(0,4,'ready') + tdSql.checkData(4,4,'ready') + tdSql.query("show mnodes;") + tdSql.checkRows(1) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,2,'leader') + tdSql.checkData(0,3,'ready') + + # fisr add three mnodes; + tdSql.execute("create mnode on dnode 2") + tdSql.execute("create mnode on dnode 3") + + # fisrt check statut ready + self.check3mnode() + + tdSql.error("create mnode on dnode 2") + tdSql.query("show dnodes;") + print(tdSql.queryResult) + tdLog.debug("stop all of mnode ") + + # seperate vnode and mnode in different dnodes. + # create database and stable + stopcount =0 + while stopcount < 2: + for i in range(dnodenumber): + # threads=[] + # threads = MyThreadFunc(self.insert_data(i*2,i*2+2)) + threads=threading.Thread(target=self.insert_data, args=(i,i+1)) + threads.start() + self.TDDnodes.stoptaosd(i+1) + self.TDDnodes.starttaosd(i+1) + + if self.checkdnodes(5): + print("123") + threads.join() + else: + print("456") + self.stop_thread(threads) + assert 1 == 2 ,"some dnode started failed" + return False + # self.check3mnode() + self.check3mnode() + + + stopcount+=1 + self.check3mnode() + + + def getConnection(self, dnode): + host = dnode.cfgDict["fqdn"] + port = dnode.cfgDict["serverPort"] + config_dir = dnode.cfgDir + return taos.connect(host=host, port=int(port), config=config_dir) + + + def run(self): + # print(self.master_dnode.cfgDict) + self.buildcluster(5) + self.five_dnode_three_mnode(5) + + def stop(self): + tdSql.close() + tdLog.success(f"{__file__} successfully executed") + +tdCases.addLinux(__file__, TDTestCase()) +tdCases.addWindows(__file__, TDTestCase()) \ No newline at end of file diff --git a/tests/system-test/7-tmq/basic5.py b/tests/system-test/7-tmq/basic5.py index e44f327995..4ed3be967e 100644 --- a/tests/system-test/7-tmq/basic5.py +++ b/tests/system-test/7-tmq/basic5.py @@ -192,7 +192,7 @@ class TDTestCase: time.sleep(1) tdLog.info("start consume processor") - pollDelay = 100 + pollDelay = 20 showMsg = 1 showRow = 1 @@ -208,7 +208,7 @@ class TDTestCase: os.system(shellCmd) # wait for data ready - prepareEnvThread.join() + # prepareEnvThread.join() tdLog.info("insert process end, and start to check consume result") while 1: diff --git a/tests/system-test/7-tmq/subscribeDb0.py b/tests/system-test/7-tmq/subscribeDb0.py index c9f256ed74..4e8fb04517 100644 --- a/tests/system-test/7-tmq/subscribeDb0.py +++ b/tests/system-test/7-tmq/subscribeDb0.py @@ -322,176 +322,6 @@ class TDTestCase: tdLog.printNoPrefix("======== test case 5 end ...... ") - def tmqCase6(self, cfgPath, buildPath): - tdLog.printNoPrefix("======== test case 6: Produce while one consumers to subscribe tow topic, Each contains one db") - tdLog.info("step 1: create database, stb, ctb and insert data") - # create and start thread - parameterDict = {'cfg': '', \ - 'dbName': 'db60', \ - 'vgroups': 4, \ - 'stbName': 'stb', \ - 'ctbNum': 10, \ - 'rowsPerTbl': 5000, \ - 'batchNum': 100, \ - 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 - parameterDict['cfg'] = cfgPath - - self.initConsumerTable() - - tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict['dbName'], parameterDict['vgroups'])) - - prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) - prepareEnvThread.start() - - parameterDict2 = {'cfg': '', \ - 'dbName': 'db61', \ - 'vgroups': 4, \ - 'stbName': 'stb2', \ - 'ctbNum': 10, \ - 'rowsPerTbl': 5000, \ - 'batchNum': 100, \ - 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 - parameterDict['cfg'] = cfgPath - - tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict2['dbName'], parameterDict2['vgroups'])) - - prepareEnvThread2 = threading.Thread(target=self.prepareEnv, kwargs=parameterDict2) - prepareEnvThread2.start() - - tdLog.info("create topics from db") - topicName1 = 'topic_db60' - topicName2 = 'topic_db61' - - tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) - tdSql.execute("create topic %s as database %s" %(topicName2, parameterDict2['dbName'])) - - consumerId = 0 - expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] + parameterDict2["rowsPerTbl"] * parameterDict2["ctbNum"] - topicList = topicName1 + ',' + topicName2 - ifcheckdata = 0 - ifManualCommit = 0 - keyList = 'group.id:cgrp1,\ - enable.auto.commit:false,\ - auto.commit.interval.ms:6000,\ - auto.offset.reset:earliest' - self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) - - #consumerId = 1 - #self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) - - event.wait() - - tdLog.info("start consume processor") - pollDelay = 100 - showMsg = 1 - showRow = 1 - self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - - # wait for data ready - prepareEnvThread.join() - prepareEnvThread2.join() - - tdLog.info("insert process end, and start to check consume result") - expectRows = 1 - resultList = self.selectConsumeResult(expectRows) - totalConsumeRows = 0 - for i in range(expectRows): - totalConsumeRows += resultList[i] - - if totalConsumeRows != expectrowcnt: - tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) - tdLog.exit("tmq consume rows error!") - - tdSql.query("drop topic %s"%topicName1) - tdSql.query("drop topic %s"%topicName2) - - tdLog.printNoPrefix("======== test case 6 end ...... ") - - def tmqCase7(self, cfgPath, buildPath): - tdLog.printNoPrefix("======== test case 7: Produce while two consumers to subscribe tow topic, Each contains one db") - tdLog.info("step 1: create database, stb, ctb and insert data") - # create and start thread - parameterDict = {'cfg': '', \ - 'dbName': 'db70', \ - 'vgroups': 4, \ - 'stbName': 'stb', \ - 'ctbNum': 10, \ - 'rowsPerTbl': 5000, \ - 'batchNum': 100, \ - 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 - parameterDict['cfg'] = cfgPath - - self.initConsumerTable() - - tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict['dbName'], parameterDict['vgroups'])) - - prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) - prepareEnvThread.start() - - parameterDict2 = {'cfg': '', \ - 'dbName': 'db71', \ - 'vgroups': 4, \ - 'stbName': 'stb2', \ - 'ctbNum': 10, \ - 'rowsPerTbl': 5000, \ - 'batchNum': 100, \ - 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 - parameterDict['cfg'] = cfgPath - - tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict2['dbName'], parameterDict2['vgroups'])) - - prepareEnvThread2 = threading.Thread(target=self.prepareEnv, kwargs=parameterDict2) - prepareEnvThread2.start() - - tdLog.info("create topics from db") - topicName1 = 'topic_db60' - topicName2 = 'topic_db61' - - tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) - tdSql.execute("create topic %s as database %s" %(topicName2, parameterDict2['dbName'])) - - consumerId = 0 - expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] + parameterDict2["rowsPerTbl"] * parameterDict2["ctbNum"] - topicList = topicName1 + ',' + topicName2 - ifcheckdata = 0 - ifManualCommit = 1 - keyList = 'group.id:cgrp1,\ - enable.auto.commit:false,\ - auto.commit.interval.ms:6000,\ - auto.offset.reset:earliest' - self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) - - consumerId = 1 - self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) - - event.wait() - - tdLog.info("start consume processor") - pollDelay = 100 - showMsg = 1 - showRow = 1 - self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - - # wait for data ready - prepareEnvThread.join() - prepareEnvThread2.join() - - tdLog.info("insert process end, and start to check consume result") - expectRows = 2 - resultList = self.selectConsumeResult(expectRows) - totalConsumeRows = 0 - for i in range(expectRows): - totalConsumeRows += resultList[i] - - if totalConsumeRows != expectrowcnt: - tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) - tdLog.exit("tmq consume rows error!") - - tdSql.query("drop topic %s"%topicName1) - tdSql.query("drop topic %s"%topicName2) - - tdLog.printNoPrefix("======== test case 7 end ...... ") - def run(self): tdSql.prepare() @@ -505,8 +335,6 @@ class TDTestCase: self.tmqCase4(cfgPath, buildPath) self.tmqCase5(cfgPath, buildPath) - self.tmqCase6(cfgPath, buildPath) - self.tmqCase7(cfgPath, buildPath) def stop(self): diff --git a/tests/system-test/7-tmq/subscribeDb1.py b/tests/system-test/7-tmq/subscribeDb1.py index ed92a429ae..28a341f8f3 100644 --- a/tests/system-test/7-tmq/subscribeDb1.py +++ b/tests/system-test/7-tmq/subscribeDb1.py @@ -72,10 +72,10 @@ class TDTestCase: if tdSql.getRows() == expectRows: break else: - time.sleep(5) - + time.sleep(5) + for i in range(expectRows): - tdLog.info ("ts: %s, consume id: %d, consume msgs: %d, consume rows: %d"%(tdSql.getData(i , 0), tdSql.getData(i , 1), tdSql.getData(i , 2), tdSql.getData(i , 3))) + tdLog.info ("consume id: %d, consume msgs: %d, consume rows: %d"%(tdSql.getData(i , 1), tdSql.getData(i , 2), tdSql.getData(i , 3))) resultList.append(tdSql.getData(i , 3)) return resultList @@ -85,7 +85,7 @@ class TDTestCase: logFile = cfgPath + '/../log/valgrind-tmq.log' shellCmd = 'nohup valgrind --log-file=' + logFile shellCmd += '--tool=memcheck --leak-check=full --show-reachable=no --track-origins=yes --show-leak-kinds=all --num-callers=20 -v --workaround-gcc296-bugs=yes ' - + if (platform.system().lower() == 'windows'): shellCmd = 'mintty -h never -w hide ' + buildPath + '\\build\\bin\\tmq_sim.exe -c ' + cfgPath shellCmd += " -y %d -d %s -g %d -r %d -w %s "%(pollDelay, dbName, showMsg, showRow, cdbName) @@ -97,7 +97,7 @@ class TDTestCase: tdLog.info(shellCmd) os.system(shellCmd) - def create_tables(self,tsql, dbName,vgroups,stbName,ctbNum,rowsPerTbl): + def create_tables(self,tsql, dbName,vgroups,stbName,ctbNum): tsql.execute("create database if not exists %s vgroups %d"%(dbName, vgroups)) tsql.execute("use %s" %dbName) tsql.execute("create table if not exists %s (ts timestamp, c1 bigint, c2 binary(16)) tags(t1 int)"%stbName) @@ -151,8 +151,7 @@ class TDTestCase: parameterDict["dbName"],\ parameterDict["vgroups"],\ parameterDict["stbName"],\ - parameterDict["ctbNum"],\ - parameterDict["rowsPerTbl"]) + parameterDict["ctbNum"]) self.insert_data(tsql,\ parameterDict["dbName"],\ @@ -163,16 +162,16 @@ class TDTestCase: parameterDict["startTs"]) return - def tmqCase8(self, cfgPath, buildPath): - tdLog.printNoPrefix("======== test case 8: Produce while one consume to subscribe one db, inclue 1 stb") + def tmqCase6(self, cfgPath, buildPath): + tdLog.printNoPrefix("======== test case 6: Produce while one consumers to subscribe tow topic, Each contains one db") tdLog.info("step 1: create database, stb, ctb and insert data") # create and start thread parameterDict = {'cfg': '', \ - 'dbName': 'db8', \ + 'dbName': 'db60', \ 'vgroups': 4, \ 'stbName': 'stb', \ 'ctbNum': 10, \ - 'rowsPerTbl': 10000, \ + 'rowsPerTbl': 5000, \ 'batchNum': 100, \ 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 parameterDict['cfg'] = cfgPath @@ -183,14 +182,32 @@ class TDTestCase: prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) prepareEnvThread.start() - + + parameterDict2 = {'cfg': '', \ + 'dbName': 'db61', \ + 'vgroups': 4, \ + 'stbName': 'stb2', \ + 'ctbNum': 10, \ + 'rowsPerTbl': 5000, \ + 'batchNum': 100, \ + 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 + parameterDict['cfg'] = cfgPath + + tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict2['dbName'], parameterDict2['vgroups'])) + + prepareEnvThread2 = threading.Thread(target=self.prepareEnv, kwargs=parameterDict2) + prepareEnvThread2.start() + tdLog.info("create topics from db") - topicName1 = 'topic_db1' + topicName1 = 'topic_db60' + topicName2 = 'topic_db61' + + tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) + tdSql.execute("create topic %s as database %s" %(topicName2, parameterDict2['dbName'])) - tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) consumerId = 0 - expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] / 2 - topicList = topicName1 + expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] + parameterDict2["rowsPerTbl"] * parameterDict2["ctbNum"] + topicList = topicName1 + ',' + topicName2 ifcheckdata = 0 ifManualCommit = 0 keyList = 'group.id:cgrp1,\ @@ -199,6 +216,9 @@ class TDTestCase: auto.offset.reset:earliest' self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + #consumerId = 1 + #self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + event.wait() tdLog.info("start consume processor") @@ -208,7 +228,8 @@ class TDTestCase: self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) # wait for data ready - prepareEnvThread.join() + prepareEnvThread.join() + prepareEnvThread2.join() tdLog.info("insert process end, and start to check consume result") expectRows = 1 @@ -221,36 +242,21 @@ class TDTestCase: tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) tdLog.exit("tmq consume rows error!") - - tdLog.info("again start consume processer") - self.initConsumerTable() - expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] - self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) - self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - expectRows = 1 - resultList = self.selectConsumeResult(expectRows) - totalConsumeRows = 0 - for i in range(expectRows): - totalConsumeRows += resultList[i] - - if totalConsumeRows != expectrowcnt: - tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) - tdLog.exit("tmq consume rows error!") - tdSql.query("drop topic %s"%topicName1) + tdSql.query("drop topic %s"%topicName2) - tdLog.printNoPrefix("======== test case 8 end ...... ") + tdLog.printNoPrefix("======== test case 6 end ...... ") - def tmqCase9(self, cfgPath, buildPath): - tdLog.printNoPrefix("======== test case 9: Produce while one consume to subscribe one db, inclue 1 stb") + def tmqCase7(self, cfgPath, buildPath): + tdLog.printNoPrefix("======== test case 7: Produce while two consumers to subscribe tow topic, Each contains one db") tdLog.info("step 1: create database, stb, ctb and insert data") # create and start thread parameterDict = {'cfg': '', \ - 'dbName': 'db9', \ + 'dbName': 'db70', \ 'vgroups': 4, \ 'stbName': 'stb', \ 'ctbNum': 10, \ - 'rowsPerTbl': 10000, \ + 'rowsPerTbl': 5000, \ 'batchNum': 100, \ 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 parameterDict['cfg'] = cfgPath @@ -261,14 +267,32 @@ class TDTestCase: prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) prepareEnvThread.start() - + + parameterDict2 = {'cfg': '', \ + 'dbName': 'db71', \ + 'vgroups': 4, \ + 'stbName': 'stb2', \ + 'ctbNum': 10, \ + 'rowsPerTbl': 5000, \ + 'batchNum': 100, \ + 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 + parameterDict['cfg'] = cfgPath + + tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict2['dbName'], parameterDict2['vgroups'])) + + prepareEnvThread2 = threading.Thread(target=self.prepareEnv, kwargs=parameterDict2) + prepareEnvThread2.start() + tdLog.info("create topics from db") - topicName1 = 'topic_db1' + topicName1 = 'topic_db60' + topicName2 = 'topic_db61' + + tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) + tdSql.execute("create topic %s as database %s" %(topicName2, parameterDict2['dbName'])) - tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) consumerId = 0 - expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] / 2 - topicList = topicName1 + expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] + parameterDict2["rowsPerTbl"] * parameterDict2["ctbNum"] + topicList = topicName1 + ',' + topicName2 ifcheckdata = 0 ifManualCommit = 1 keyList = 'group.id:cgrp1,\ @@ -277,86 +301,7 @@ class TDTestCase: auto.offset.reset:earliest' self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) - event.wait() - - tdLog.info("start consume processor") - pollDelay = 100 - showMsg = 1 - showRow = 1 - self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - - # wait for data ready - prepareEnvThread.join() - - tdLog.info("insert process end, and start to check consume result") - expectRows = 1 - resultList = self.selectConsumeResult(expectRows) - totalConsumeRows = 0 - for i in range(expectRows): - totalConsumeRows += resultList[i] - - tdSql.query("select count(*) from %s.%s" %(parameterDict['dbName'], parameterDict['stbName'])) - countOfStb = tdSql.getData(0,0) - print ("====total rows of stb: %d"%countOfStb) - - tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) - if totalConsumeRows < expectrowcnt: - tdLog.exit("tmq consume rows error!") - - tdLog.info("again start consume processer") - self.initConsumerTable() - expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] - self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) - self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - expectRows = 1 - resultList = self.selectConsumeResult(expectRows) - totalConsumeRows2 = 0 - for i in range(expectRows): - totalConsumeRows2 += resultList[i] - - tdLog.info("firstly act consume rows: %d"%(totalConsumeRows)) - tdLog.info("secondly act consume rows: %d, expect consume rows: %d"%(totalConsumeRows2, expectrowcnt)) - if totalConsumeRows + totalConsumeRows2 != expectrowcnt: - tdLog.exit("tmq consume rows error!") - - tdSql.query("drop topic %s"%topicName1) - - tdLog.printNoPrefix("======== test case 9 end ...... ") - - def tmqCase10(self, cfgPath, buildPath): - tdLog.printNoPrefix("======== test case 10: Produce while one consume to subscribe one db, inclue 1 stb") - tdLog.info("step 1: create database, stb, ctb and insert data") - # create and start thread - parameterDict = {'cfg': '', \ - 'dbName': 'db10', \ - 'vgroups': 4, \ - 'stbName': 'stb', \ - 'ctbNum': 10, \ - 'rowsPerTbl': 10000, \ - 'batchNum': 100, \ - 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 - parameterDict['cfg'] = cfgPath - - self.initConsumerTable() - - tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict['dbName'], parameterDict['vgroups'])) - - prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) - prepareEnvThread.start() - - tdLog.info("create topics from db") - topicName1 = 'topic_db1' - - tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) - consumerId = 0 - expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] - topicList = topicName1 - ifcheckdata = 0 - ifManualCommit = 1 - keyList = 'group.id:cgrp1,\ - enable.auto.commit:false,\ - auto.commit.interval.ms:6000,\ - auto.offset.reset:earliest' + consumerId = 1 self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) event.wait() @@ -367,23 +312,12 @@ class TDTestCase: showRow = 1 self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - time.sleep(2) - tdLog.info("pkill consume processor") - if (platform.system().lower() == 'windows'): - os.system("TASKKILL /F /IM tmq_sim.exe") - else: - os.system('pkill tmq_sim') - expectRows = 0 - resultList = self.selectConsumeResult(expectRows) - # wait for data ready prepareEnvThread.join() + prepareEnvThread2.join() + tdLog.info("insert process end, and start to check consume result") - - tdLog.info("again start consume processer") - self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - - expectRows = 1 + expectRows = 2 resultList = self.selectConsumeResult(expectRows) totalConsumeRows = 0 for i in range(expectRows): @@ -393,85 +327,10 @@ class TDTestCase: tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) tdLog.exit("tmq consume rows error!") - time.sleep(15) tdSql.query("drop topic %s"%topicName1) + tdSql.query("drop topic %s"%topicName2) - tdLog.printNoPrefix("======== test case 10 end ...... ") - - def tmqCase11(self, cfgPath, buildPath): - tdLog.printNoPrefix("======== test case 11: Produce while one consume to subscribe one db, inclue 1 stb") - tdLog.info("step 1: create database, stb, ctb and insert data") - # create and start thread - parameterDict = {'cfg': '', \ - 'dbName': 'db11', \ - 'vgroups': 4, \ - 'stbName': 'stb', \ - 'ctbNum': 10, \ - 'rowsPerTbl': 10000, \ - 'batchNum': 100, \ - 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 - parameterDict['cfg'] = cfgPath - - self.initConsumerTable() - - tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict['dbName'], parameterDict['vgroups'])) - - prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) - prepareEnvThread.start() - - tdLog.info("create topics from db") - topicName1 = 'topic_db1' - - tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) - consumerId = 0 - expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] - topicList = topicName1 - ifcheckdata = 0 - ifManualCommit = 1 - keyList = 'group.id:cgrp1,\ - enable.auto.commit:true,\ - auto.commit.interval.ms:1000,\ - auto.offset.reset:earliest' - self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) - - event.wait() - - tdLog.info("start consume processor") - pollDelay = 20 - showMsg = 1 - showRow = 1 - self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - - time.sleep(3) - tdLog.info("pkill consume processor") - if (platform.system().lower() == 'windows'): - os.system("TASKKILL /F /IM tmq_sim.exe") - else: - os.system('pkill tmq_sim') - expectRows = 0 - resultList = self.selectConsumeResult(expectRows) - - # wait for data ready - prepareEnvThread.join() - tdLog.info("insert process end, and start to check consume result") - - tdLog.info("again start consume processer") - self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - - expectRows = 1 - resultList = self.selectConsumeResult(expectRows) - totalConsumeRows = 0 - for i in range(expectRows): - totalConsumeRows += resultList[i] - - if totalConsumeRows >= expectrowcnt or totalConsumeRows <= 0: - tdLog.info("act consume rows: %d, expect consume rows between %d and 0"%(totalConsumeRows, expectrowcnt)) - tdLog.exit("tmq consume rows error!") - - time.sleep(15) - tdSql.query("drop topic %s"%topicName1) - - tdLog.printNoPrefix("======== test case 11 end ...... ") + tdLog.printNoPrefix("======== test case 7 end ...... ") def run(self): tdSql.prepare() @@ -484,10 +343,9 @@ class TDTestCase: cfgPath = buildPath + "/../sim/psim/cfg" tdLog.info("cfgPath: %s" % cfgPath) - self.tmqCase8(cfgPath, buildPath) - self.tmqCase9(cfgPath, buildPath) - self.tmqCase10(cfgPath, buildPath) - self.tmqCase11(cfgPath, buildPath) + self.tmqCase6(cfgPath, buildPath) + self.tmqCase7(cfgPath, buildPath) + def stop(self): tdSql.close() diff --git a/tests/system-test/7-tmq/subscribeDb2.py b/tests/system-test/7-tmq/subscribeDb2.py new file mode 100644 index 0000000000..af31e802b3 --- /dev/null +++ b/tests/system-test/7-tmq/subscribeDb2.py @@ -0,0 +1,347 @@ + +import taos +import sys +import time +import socket +import os +import threading + +from util.log import * +from util.sql import * +from util.cases import * +from util.dnodes import * + +class TDTestCase: + hostname = socket.gethostname() + #rpcDebugFlagVal = '143' + #clientCfgDict = {'serverPort': '', 'firstEp': '', 'secondEp':'', 'rpcDebugFlag':'135', 'fqdn':''} + #clientCfgDict["rpcDebugFlag"] = rpcDebugFlagVal + #updatecfgDict = {'clientCfg': {}, 'serverPort': '', 'firstEp': '', 'secondEp':'', 'rpcDebugFlag':'135', 'fqdn':''} + #updatecfgDict["rpcDebugFlag"] = rpcDebugFlagVal + #print ("===================: ", updatecfgDict) + + def init(self, conn, logSql): + tdLog.debug(f"start to excute {__file__}") + tdSql.init(conn.cursor()) + #tdSql.init(conn.cursor(), logSql) # output sql.txt file + + def getBuildPath(self): + selfPath = os.path.dirname(os.path.realpath(__file__)) + + if ("community" in selfPath): + projPath = selfPath[:selfPath.find("community")] + else: + projPath = selfPath[:selfPath.find("tests")] + + for root, dirs, files in os.walk(projPath): + if ("taosd" in files or "taosd.exe" in files): + rootRealPath = os.path.dirname(os.path.realpath(root)) + if ("packaging" not in rootRealPath): + buildPath = root[:len(root) - len("/build/bin")] + break + return buildPath + + def newcur(self,cfg,host,port): + user = "root" + password = "taosdata" + con=taos.connect(host=host, user=user, password=password, config=cfg ,port=port) + cur=con.cursor() + print(cur) + return cur + + def initConsumerTable(self,cdbName='cdb'): + tdLog.info("create consume database, and consume info table, and consume result table") + tdSql.query("create database if not exists %s vgroups 1"%(cdbName)) + tdSql.query("drop table if exists %s.consumeinfo "%(cdbName)) + tdSql.query("drop table if exists %s.consumeresult "%(cdbName)) + + tdSql.query("create table %s.consumeinfo (ts timestamp, consumerid int, topiclist binary(1024), keylist binary(1024), expectmsgcnt bigint, ifcheckdata int, ifmanualcommit int)"%cdbName) + tdSql.query("create table %s.consumeresult (ts timestamp, consumerid int, consummsgcnt bigint, consumrowcnt bigint, checkresult int)"%cdbName) + + def insertConsumerInfo(self,consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifmanualcommit,cdbName='cdb'): + sql = "insert into %s.consumeinfo values "%cdbName + sql += "(now, %d, '%s', '%s', %d, %d, %d)"%(consumerId, topicList, keyList, expectrowcnt, ifcheckdata, ifmanualcommit) + tdLog.info("consume info sql: %s"%sql) + tdSql.query(sql) + + def selectConsumeResult(self,expectRows,cdbName='cdb'): + resultList=[] + while 1: + tdSql.query("select * from %s.consumeresult"%cdbName) + #tdLog.info("row: %d, %l64d, %l64d"%(tdSql.getData(0, 1),tdSql.getData(0, 2),tdSql.getData(0, 3)) + if tdSql.getRows() == expectRows: + break + else: + time.sleep(5) + + for i in range(expectRows): + tdLog.info ("ts: %s, consume id: %d, consume msgs: %d, consume rows: %d"%(tdSql.getData(i , 0), tdSql.getData(i , 1), tdSql.getData(i , 2), tdSql.getData(i , 3))) + resultList.append(tdSql.getData(i , 3)) + + return resultList + + def startTmqSimProcess(self,buildPath,cfgPath,pollDelay,dbName,showMsg=1,showRow=1,cdbName='cdb',valgrind=0): + if valgrind == 1: + logFile = cfgPath + '/../log/valgrind-tmq.log' + shellCmd = 'nohup valgrind --log-file=' + logFile + shellCmd += '--tool=memcheck --leak-check=full --show-reachable=no --track-origins=yes --show-leak-kinds=all --num-callers=20 -v --workaround-gcc296-bugs=yes ' + + if (platform.system().lower() == 'windows'): + shellCmd = 'mintty -h never -w hide ' + buildPath + '\\build\\bin\\tmq_sim.exe -c ' + cfgPath + shellCmd += " -y %d -d %s -g %d -r %d -w %s "%(pollDelay, dbName, showMsg, showRow, cdbName) + shellCmd += "> nul 2>&1 &" + else: + shellCmd = 'nohup ' + buildPath + '/build/bin/tmq_sim -c ' + cfgPath + shellCmd += " -y %d -d %s -g %d -r %d -w %s "%(pollDelay, dbName, showMsg, showRow, cdbName) + shellCmd += "> /dev/null 2>&1 &" + tdLog.info(shellCmd) + os.system(shellCmd) + + def create_tables(self,tsql, dbName,vgroups,stbName,ctbNum,rowsPerTbl): + tsql.execute("create database if not exists %s vgroups %d"%(dbName, vgroups)) + tsql.execute("use %s" %dbName) + tsql.execute("create table if not exists %s (ts timestamp, c1 bigint, c2 binary(16)) tags(t1 int)"%stbName) + pre_create = "create table" + sql = pre_create + #tdLog.debug("doing create one stable %s and %d child table in %s ..." %(stbname, count ,dbname)) + for i in range(ctbNum): + sql += " %s_%d using %s tags(%d)"%(stbName,i,stbName,i+1) + if (i > 0) and (i%100 == 0): + tsql.execute(sql) + sql = pre_create + if sql != pre_create: + tsql.execute(sql) + + event.set() + tdLog.debug("complete to create database[%s], stable[%s] and %d child tables" %(dbName, stbName, ctbNum)) + return + + def insert_data(self,tsql,dbName,stbName,ctbNum,rowsPerTbl,batchNum,startTs): + tdLog.debug("start to insert data ............") + tsql.execute("use %s" %dbName) + pre_insert = "insert into " + sql = pre_insert + + t = time.time() + startTs = int(round(t * 1000)) + #tdLog.debug("doing insert data into stable:%s rows:%d ..."%(stbName, allRows)) + for i in range(ctbNum): + sql += " %s_%d values "%(stbName,i) + for j in range(rowsPerTbl): + sql += "(%d, %d, 'tmqrow_%d') "%(startTs + j, j, j) + if (j > 0) and ((j%batchNum == 0) or (j == rowsPerTbl - 1)): + tsql.execute(sql) + if j < rowsPerTbl - 1: + sql = "insert into %s_%d values " %(stbName,i) + else: + sql = "insert into " + #end sql + if sql != pre_insert: + #print("insert sql:%s"%sql) + tsql.execute(sql) + tdLog.debug("insert data ............ [OK]") + return + + def prepareEnv(self, **parameterDict): + print ("input parameters:") + print (parameterDict) + # create new connector for my thread + tsql=self.newcur(parameterDict['cfg'], 'localhost', 6030) + self.create_tables(tsql,\ + parameterDict["dbName"],\ + parameterDict["vgroups"],\ + parameterDict["stbName"],\ + parameterDict["ctbNum"],\ + parameterDict["rowsPerTbl"]) + + self.insert_data(tsql,\ + parameterDict["dbName"],\ + parameterDict["stbName"],\ + parameterDict["ctbNum"],\ + parameterDict["rowsPerTbl"],\ + parameterDict["batchNum"],\ + parameterDict["startTs"]) + return + + def tmqCase8(self, cfgPath, buildPath): + tdLog.printNoPrefix("======== test case 8: Produce while one consume to subscribe one db, inclue 1 stb") + tdLog.info("step 1: create database, stb, ctb and insert data") + # create and start thread + parameterDict = {'cfg': '', \ + 'dbName': 'db8', \ + 'vgroups': 4, \ + 'stbName': 'stb', \ + 'ctbNum': 10, \ + 'rowsPerTbl': 10000, \ + 'batchNum': 100, \ + 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 + parameterDict['cfg'] = cfgPath + + self.initConsumerTable() + + tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict['dbName'], parameterDict['vgroups'])) + + prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) + prepareEnvThread.start() + + tdLog.info("create topics from db") + topicName1 = 'topic_db1' + + tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) + consumerId = 0 + expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] / 2 + topicList = topicName1 + ifcheckdata = 0 + ifManualCommit = 0 + keyList = 'group.id:cgrp1,\ + enable.auto.commit:false,\ + auto.commit.interval.ms:6000,\ + auto.offset.reset:earliest' + self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + + event.wait() + + tdLog.info("start consume processor") + pollDelay = 100 + showMsg = 1 + showRow = 1 + self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) + + # wait for data ready + prepareEnvThread.join() + + tdLog.info("insert process end, and start to check consume result") + expectRows = 1 + resultList = self.selectConsumeResult(expectRows) + totalConsumeRows = 0 + for i in range(expectRows): + totalConsumeRows += resultList[i] + + if totalConsumeRows != expectrowcnt: + tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) + tdLog.exit("tmq consume rows error!") + + + tdLog.info("again start consume processer") + self.initConsumerTable() + expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] + self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) + expectRows = 1 + resultList = self.selectConsumeResult(expectRows) + totalConsumeRows = 0 + for i in range(expectRows): + totalConsumeRows += resultList[i] + + if totalConsumeRows != expectrowcnt: + tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) + tdLog.exit("tmq consume rows error!") + + tdSql.query("drop topic %s"%topicName1) + + tdLog.printNoPrefix("======== test case 8 end ...... ") + + def tmqCase9(self, cfgPath, buildPath): + tdLog.printNoPrefix("======== test case 9: Produce while one consume to subscribe one db, inclue 1 stb") + tdLog.info("step 1: create database, stb, ctb and insert data") + # create and start thread + parameterDict = {'cfg': '', \ + 'dbName': 'db9', \ + 'vgroups': 4, \ + 'stbName': 'stb', \ + 'ctbNum': 10, \ + 'rowsPerTbl': 10000, \ + 'batchNum': 100, \ + 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 + parameterDict['cfg'] = cfgPath + + self.initConsumerTable() + + tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict['dbName'], parameterDict['vgroups'])) + + prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) + prepareEnvThread.start() + + tdLog.info("create topics from db") + topicName1 = 'topic_db1' + + tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) + consumerId = 0 + expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] / 2 + topicList = topicName1 + ifcheckdata = 0 + ifManualCommit = 1 + keyList = 'group.id:cgrp1,\ + enable.auto.commit:false,\ + auto.commit.interval.ms:6000,\ + auto.offset.reset:earliest' + self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + + event.wait() + + tdLog.info("start consume processor") + pollDelay = 100 + showMsg = 1 + showRow = 1 + self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) + + # wait for data ready + prepareEnvThread.join() + + tdLog.info("insert process end, and start to check consume result") + expectRows = 1 + resultList = self.selectConsumeResult(expectRows) + totalConsumeRows = 0 + for i in range(expectRows): + totalConsumeRows += resultList[i] + + tdSql.query("select count(*) from %s.%s" %(parameterDict['dbName'], parameterDict['stbName'])) + countOfStb = tdSql.getData(0,0) + print ("====total rows of stb: %d"%countOfStb) + + tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) + if totalConsumeRows < expectrowcnt: + tdLog.exit("tmq consume rows error!") + + tdLog.info("again start consume processer") + self.initConsumerTable() + expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] + self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) + expectRows = 1 + resultList = self.selectConsumeResult(expectRows) + totalConsumeRows2 = 0 + for i in range(expectRows): + totalConsumeRows2 += resultList[i] + + tdLog.info("firstly act consume rows: %d"%(totalConsumeRows)) + tdLog.info("secondly act consume rows: %d, expect consume rows: %d"%(totalConsumeRows2, expectrowcnt)) + if totalConsumeRows + totalConsumeRows2 != expectrowcnt: + tdLog.exit("tmq consume rows error!") + + tdSql.query("drop topic %s"%topicName1) + + tdLog.printNoPrefix("======== test case 9 end ...... ") + + def run(self): + tdSql.prepare() + + buildPath = self.getBuildPath() + if (buildPath == ""): + tdLog.exit("taosd not found!") + else: + tdLog.info("taosd found in %s" % buildPath) + cfgPath = buildPath + "/../sim/psim/cfg" + tdLog.info("cfgPath: %s" % cfgPath) + + self.tmqCase8(cfgPath, buildPath) + self.tmqCase9(cfgPath, buildPath) + + def stop(self): + tdSql.close() + tdLog.success(f"{__file__} successfully executed") + +event = threading.Event() + +tdCases.addLinux(__file__, TDTestCase()) +tdCases.addWindows(__file__, TDTestCase()) diff --git a/tests/system-test/7-tmq/subscribeDb3.py b/tests/system-test/7-tmq/subscribeDb3.py new file mode 100644 index 0000000000..6973f4c51f --- /dev/null +++ b/tests/system-test/7-tmq/subscribeDb3.py @@ -0,0 +1,337 @@ + +import taos +import sys +import time +import socket +import os +import threading + +from util.log import * +from util.sql import * +from util.cases import * +from util.dnodes import * + +class TDTestCase: + hostname = socket.gethostname() + #rpcDebugFlagVal = '143' + #clientCfgDict = {'serverPort': '', 'firstEp': '', 'secondEp':'', 'rpcDebugFlag':'135', 'fqdn':''} + #clientCfgDict["rpcDebugFlag"] = rpcDebugFlagVal + #updatecfgDict = {'clientCfg': {}, 'serverPort': '', 'firstEp': '', 'secondEp':'', 'rpcDebugFlag':'135', 'fqdn':''} + #updatecfgDict["rpcDebugFlag"] = rpcDebugFlagVal + #print ("===================: ", updatecfgDict) + + def init(self, conn, logSql): + tdLog.debug(f"start to excute {__file__}") + tdSql.init(conn.cursor()) + #tdSql.init(conn.cursor(), logSql) # output sql.txt file + + def getBuildPath(self): + selfPath = os.path.dirname(os.path.realpath(__file__)) + + if ("community" in selfPath): + projPath = selfPath[:selfPath.find("community")] + else: + projPath = selfPath[:selfPath.find("tests")] + + for root, dirs, files in os.walk(projPath): + if ("taosd" in files or "taosd.exe" in files): + rootRealPath = os.path.dirname(os.path.realpath(root)) + if ("packaging" not in rootRealPath): + buildPath = root[:len(root) - len("/build/bin")] + break + return buildPath + + def newcur(self,cfg,host,port): + user = "root" + password = "taosdata" + con=taos.connect(host=host, user=user, password=password, config=cfg ,port=port) + cur=con.cursor() + print(cur) + return cur + + def initConsumerTable(self,cdbName='cdb'): + tdLog.info("create consume database, and consume info table, and consume result table") + tdSql.query("create database if not exists %s vgroups 1"%(cdbName)) + tdSql.query("drop table if exists %s.consumeinfo "%(cdbName)) + tdSql.query("drop table if exists %s.consumeresult "%(cdbName)) + + tdSql.query("create table %s.consumeinfo (ts timestamp, consumerid int, topiclist binary(1024), keylist binary(1024), expectmsgcnt bigint, ifcheckdata int, ifmanualcommit int)"%cdbName) + tdSql.query("create table %s.consumeresult (ts timestamp, consumerid int, consummsgcnt bigint, consumrowcnt bigint, checkresult int)"%cdbName) + + def insertConsumerInfo(self,consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifmanualcommit,cdbName='cdb'): + sql = "insert into %s.consumeinfo values "%cdbName + sql += "(now, %d, '%s', '%s', %d, %d, %d)"%(consumerId, topicList, keyList, expectrowcnt, ifcheckdata, ifmanualcommit) + tdLog.info("consume info sql: %s"%sql) + tdSql.query(sql) + + def selectConsumeResult(self,expectRows,cdbName='cdb'): + resultList=[] + while 1: + tdSql.query("select * from %s.consumeresult"%cdbName) + #tdLog.info("row: %d, %l64d, %l64d"%(tdSql.getData(0, 1),tdSql.getData(0, 2),tdSql.getData(0, 3)) + if tdSql.getRows() == expectRows: + break + else: + time.sleep(5) + + for i in range(expectRows): + tdLog.info ("ts: %s, consume id: %d, consume msgs: %d, consume rows: %d"%(tdSql.getData(i , 0), tdSql.getData(i , 1), tdSql.getData(i , 2), tdSql.getData(i , 3))) + resultList.append(tdSql.getData(i , 3)) + + return resultList + + def startTmqSimProcess(self,buildPath,cfgPath,pollDelay,dbName,showMsg=1,showRow=1,cdbName='cdb',valgrind=0): + if valgrind == 1: + logFile = cfgPath + '/../log/valgrind-tmq.log' + shellCmd = 'nohup valgrind --log-file=' + logFile + shellCmd += '--tool=memcheck --leak-check=full --show-reachable=no --track-origins=yes --show-leak-kinds=all --num-callers=20 -v --workaround-gcc296-bugs=yes ' + + if (platform.system().lower() == 'windows'): + shellCmd = 'mintty -h never -w hide ' + buildPath + '\\build\\bin\\tmq_sim.exe -c ' + cfgPath + shellCmd += " -y %d -d %s -g %d -r %d -w %s "%(pollDelay, dbName, showMsg, showRow, cdbName) + shellCmd += "> nul 2>&1 &" + else: + shellCmd = 'nohup ' + buildPath + '/build/bin/tmq_sim -c ' + cfgPath + shellCmd += " -y %d -d %s -g %d -r %d -w %s "%(pollDelay, dbName, showMsg, showRow, cdbName) + shellCmd += "> /dev/null 2>&1 &" + tdLog.info(shellCmd) + os.system(shellCmd) + + def create_tables(self,tsql, dbName,vgroups,stbName,ctbNum,rowsPerTbl): + tsql.execute("create database if not exists %s vgroups %d"%(dbName, vgroups)) + tsql.execute("use %s" %dbName) + tsql.execute("create table if not exists %s (ts timestamp, c1 bigint, c2 binary(16)) tags(t1 int)"%stbName) + pre_create = "create table" + sql = pre_create + #tdLog.debug("doing create one stable %s and %d child table in %s ..." %(stbname, count ,dbname)) + for i in range(ctbNum): + sql += " %s_%d using %s tags(%d)"%(stbName,i,stbName,i+1) + if (i > 0) and (i%100 == 0): + tsql.execute(sql) + sql = pre_create + if sql != pre_create: + tsql.execute(sql) + + event.set() + tdLog.debug("complete to create database[%s], stable[%s] and %d child tables" %(dbName, stbName, ctbNum)) + return + + def insert_data(self,tsql,dbName,stbName,ctbNum,rowsPerTbl,batchNum,startTs): + tdLog.debug("start to insert data ............") + tsql.execute("use %s" %dbName) + pre_insert = "insert into " + sql = pre_insert + + t = time.time() + startTs = int(round(t * 1000)) + #tdLog.debug("doing insert data into stable:%s rows:%d ..."%(stbName, allRows)) + for i in range(ctbNum): + sql += " %s_%d values "%(stbName,i) + for j in range(rowsPerTbl): + sql += "(%d, %d, 'tmqrow_%d') "%(startTs + j, j, j) + if (j > 0) and ((j%batchNum == 0) or (j == rowsPerTbl - 1)): + tsql.execute(sql) + if j < rowsPerTbl - 1: + sql = "insert into %s_%d values " %(stbName,i) + else: + sql = "insert into " + #end sql + if sql != pre_insert: + #print("insert sql:%s"%sql) + tsql.execute(sql) + tdLog.debug("insert data ............ [OK]") + return + + def prepareEnv(self, **parameterDict): + print ("input parameters:") + print (parameterDict) + # create new connector for my thread + tsql=self.newcur(parameterDict['cfg'], 'localhost', 6030) + self.create_tables(tsql,\ + parameterDict["dbName"],\ + parameterDict["vgroups"],\ + parameterDict["stbName"],\ + parameterDict["ctbNum"],\ + parameterDict["rowsPerTbl"]) + + self.insert_data(tsql,\ + parameterDict["dbName"],\ + parameterDict["stbName"],\ + parameterDict["ctbNum"],\ + parameterDict["rowsPerTbl"],\ + parameterDict["batchNum"],\ + parameterDict["startTs"]) + return + + def tmqCase10(self, cfgPath, buildPath): + tdLog.printNoPrefix("======== test case 10: Produce while one consume to subscribe one db, inclue 1 stb") + tdLog.info("step 1: create database, stb, ctb and insert data") + # create and start thread + parameterDict = {'cfg': '', \ + 'dbName': 'db10', \ + 'vgroups': 4, \ + 'stbName': 'stb', \ + 'ctbNum': 10, \ + 'rowsPerTbl': 10000, \ + 'batchNum': 100, \ + 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 + parameterDict['cfg'] = cfgPath + + self.initConsumerTable() + + tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict['dbName'], parameterDict['vgroups'])) + + prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) + prepareEnvThread.start() + + tdLog.info("create topics from db") + topicName1 = 'topic_db1' + + tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) + consumerId = 0 + expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] + topicList = topicName1 + ifcheckdata = 0 + ifManualCommit = 1 + keyList = 'group.id:cgrp1,\ + enable.auto.commit:false,\ + auto.commit.interval.ms:6000,\ + auto.offset.reset:earliest' + self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + + event.wait() + + tdLog.info("start consume processor") + pollDelay = 100 + showMsg = 1 + showRow = 1 + self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) + + time.sleep(2) + tdLog.info("pkill consume processor") + if (platform.system().lower() == 'windows'): + os.system("TASKKILL /F /IM tmq_sim.exe") + else: + os.system('pkill tmq_sim') + expectRows = 0 + resultList = self.selectConsumeResult(expectRows) + + # wait for data ready + prepareEnvThread.join() + tdLog.info("insert process end, and start to check consume result") + + tdLog.info("again start consume processer") + self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) + + expectRows = 1 + resultList = self.selectConsumeResult(expectRows) + totalConsumeRows = 0 + for i in range(expectRows): + totalConsumeRows += resultList[i] + + if totalConsumeRows != expectrowcnt: + tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) + tdLog.exit("tmq consume rows error!") + + time.sleep(15) + tdSql.query("drop topic %s"%topicName1) + + tdLog.printNoPrefix("======== test case 10 end ...... ") + + def tmqCase11(self, cfgPath, buildPath): + tdLog.printNoPrefix("======== test case 11: Produce while one consume to subscribe one db, inclue 1 stb") + tdLog.info("step 1: create database, stb, ctb and insert data") + # create and start thread + parameterDict = {'cfg': '', \ + 'dbName': 'db11', \ + 'vgroups': 4, \ + 'stbName': 'stb', \ + 'ctbNum': 10, \ + 'rowsPerTbl': 10000, \ + 'batchNum': 100, \ + 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 + parameterDict['cfg'] = cfgPath + + self.initConsumerTable() + + tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict['dbName'], parameterDict['vgroups'])) + + prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) + prepareEnvThread.start() + + tdLog.info("create topics from db") + topicName1 = 'topic_db1' + + tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) + consumerId = 0 + expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] + topicList = topicName1 + ifcheckdata = 0 + ifManualCommit = 1 + keyList = 'group.id:cgrp1,\ + enable.auto.commit:true,\ + auto.commit.interval.ms:1000,\ + auto.offset.reset:earliest' + self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + + event.wait() + + tdLog.info("start consume processor") + pollDelay = 20 + showMsg = 1 + showRow = 1 + self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) + + time.sleep(6) + tdLog.info("pkill consume processor") + if (platform.system().lower() == 'windows'): + os.system("TASKKILL /F /IM tmq_sim.exe") + else: + os.system('pkill tmq_sim') + expectRows = 0 + resultList = self.selectConsumeResult(expectRows) + + # wait for data ready + prepareEnvThread.join() + tdLog.info("insert process end, and start to check consume result") + + tdLog.info("again start consume processer") + self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) + + expectRows = 1 + resultList = self.selectConsumeResult(expectRows) + totalConsumeRows = 0 + for i in range(expectRows): + totalConsumeRows += resultList[i] + + if totalConsumeRows >= expectrowcnt or totalConsumeRows <= 0: + tdLog.info("act consume rows: %d, expect consume rows between %d and 0"%(totalConsumeRows, expectrowcnt)) + tdLog.exit("tmq consume rows error!") + + time.sleep(15) + tdSql.query("drop topic %s"%topicName1) + + tdLog.printNoPrefix("======== test case 11 end ...... ") + + def run(self): + tdSql.prepare() + + buildPath = self.getBuildPath() + if (buildPath == ""): + tdLog.exit("taosd not found!") + else: + tdLog.info("taosd found in %s" % buildPath) + cfgPath = buildPath + "/../sim/psim/cfg" + tdLog.info("cfgPath: %s" % cfgPath) + + self.tmqCase10(cfgPath, buildPath) + self.tmqCase11(cfgPath, buildPath) + + def stop(self): + tdSql.close() + tdLog.success(f"{__file__} successfully executed") + +event = threading.Event() + +tdCases.addLinux(__file__, TDTestCase()) +tdCases.addWindows(__file__, TDTestCase()) diff --git a/tests/system-test/fulltest.sh b/tests/system-test/fulltest.sh index b9f7e0f2eb..b5004d41ce 100755 --- a/tests/system-test/fulltest.sh +++ b/tests/system-test/fulltest.sh @@ -98,17 +98,25 @@ python3 ./test.py -f 2-query/stateduration.py python3 ./test.py -f 2-query/function_stateduration.py python3 ./test.py -f 2-query/statecount.py python3 ./test.py -f 2-query/tail.py +python3 ./test.py -f 2-query/distribute_agg_count.py +python3 ./test.py -f 2-query/distribute_agg_max.py +python3 ./test.py -f 2-query/distribute_agg_min.py +python3 ./test.py -f 2-query/distribute_agg_sum.py +python3 ./test.py -f 2-query/distribute_agg_spread.py +python3 ./test.py -f 2-query/distribute_agg_apercentile.py python3 ./test.py -f 6-cluster/5dnode1mnode.py python3 ./test.py -f 6-cluster/5dnode2mnode.py #python3 ./test.py -f 6-cluster/5dnode3mnodeStop.py -python3 ./test.py -f 6-cluster/5dnode3mnodeDrop.py +#python3 ./test.py -f 6-cluster/5dnode3mnodeDrop.py # BUG python3 ./test.py -f 6-cluster/5dnode3mnodeStopInsert.py python3 ./test.py -f 7-tmq/basic5.py python3 ./test.py -f 7-tmq/subscribeDb.py python3 ./test.py -f 7-tmq/subscribeDb0.py python3 ./test.py -f 7-tmq/subscribeDb1.py +python3 ./test.py -f 7-tmq/subscribeDb2.py +python3 ./test.py -f 7-tmq/subscribeDb3.py python3 ./test.py -f 7-tmq/subscribeStb.py python3 ./test.py -f 7-tmq/subscribeStb0.py python3 ./test.py -f 7-tmq/subscribeStb1.py diff --git a/tests/test/c/sdbDump.c b/tests/test/c/sdbDump.c index e5986cf4dd..612b870b7e 100644 --- a/tests/test/c/sdbDump.c +++ b/tests/test/c/sdbDump.c @@ -283,7 +283,8 @@ void dumpTrans(SSdb *pSdb, SJson *json) { tjsonAddIntegerToObject(item, "conflict", pObj->conflict); tjsonAddIntegerToObject(item, "exec", pObj->exec); tjsonAddStringToObject(item, "createdTime", i642str(pObj->createdTime)); - tjsonAddStringToObject(item, "dbname", pObj->dbname); + tjsonAddStringToObject(item, "dbname1", pObj->dbname1); + tjsonAddStringToObject(item, "dbname2", pObj->dbname2); tjsonAddIntegerToObject(item, "commitLogNum", taosArrayGetSize(pObj->commitActions)); tjsonAddIntegerToObject(item, "redoActionNum", taosArrayGetSize(pObj->redoActions)); tjsonAddIntegerToObject(item, "undoActionNum", taosArrayGetSize(pObj->undoActions)); @@ -294,8 +295,9 @@ void dumpTrans(SSdb *pSdb, SJson *json) { void dumpHeader(SSdb *pSdb, SJson *json) { tjsonAddIntegerToObject(json, "sver", 1); - tjsonAddStringToObject(json, "curVer", i642str(pSdb->curVer)); - tjsonAddStringToObject(json, "curTerm", i642str(pSdb->curTerm)); + tjsonAddStringToObject(json, "applyIndex", i642str(pSdb->applyIndex)); + tjsonAddStringToObject(json, "applyTerm", i642str(pSdb->applyTerm)); + tjsonAddStringToObject(json, "applyConfig", i642str(pSdb->applyConfig)); SJson *maxIdsJson = tjsonCreateObject(); tjsonAddItemToObject(json, "maxIds", maxIdsJson); diff --git a/tests/test/c/tmqDemo.c b/tests/test/c/tmqDemo.c index 5a972e071a..02fd3c1396 100644 --- a/tests/test/c/tmqDemo.c +++ b/tests/test/c/tmqDemo.c @@ -353,8 +353,8 @@ tmq_list_t* build_topic_list() { void sync_consume_loop(tmq_t* tmq, tmq_list_t* topics) { static const int MIN_COMMIT_COUNT = 1000; - int msg_count = 0; - tmq_resp_err_t err; + int msg_count = 0; + int32_t err; if ((err = tmq_subscribe(tmq, topics))) { fprintf(stderr, "%% Failed to start consuming topics: %s\n", tmq_err2str(err)); @@ -379,7 +379,7 @@ void sync_consume_loop(tmq_t* tmq, tmq_list_t* topics) { } void perf_loop(tmq_t* tmq, tmq_list_t* topics, int32_t totalMsgs, int64_t walLogSize) { - tmq_resp_err_t err; + int32_t err; if ((err = tmq_subscribe(tmq, topics))) { fprintf(stderr, "%% Failed to start consuming topics: %s\n", tmq_err2str(err)); diff --git a/tests/test/c/tmqSim.c b/tests/test/c/tmqSim.c index 8455bd9890..0f78a003d6 100644 --- a/tests/test/c/tmqSim.c +++ b/tests/test/c/tmqSim.c @@ -62,7 +62,7 @@ typedef struct { tmq_t* tmq; tmq_list_t* topicList; - + int32_t numOfVgroups; int32_t rowsOfPerVgroups[MAX_VGROUP_CNT][2]; // [i][0]: vgroup id, [i][1]: rows of consume int64_t ts; @@ -74,7 +74,7 @@ typedef struct { char cdbName[32]; char dbName[32]; int32_t showMsgFlag; - int32_t showRowFlag; + int32_t showRowFlag; int32_t saveRowFlag; int32_t consumeDelay; // unit s int32_t numOfThread; @@ -108,26 +108,20 @@ static void printHelp() { } char* getCurrentTimeString(char* timeString) { - time_t tTime = taosGetTimestampSec(); + time_t tTime = taosGetTimestampSec(); struct tm tm = *taosLocalTime(&tTime, NULL); - sprintf(timeString, "%d-%02d-%02d %02d:%02d:%02d", - tm.tm_year + 1900, - tm.tm_mon + 1, - tm.tm_mday, - tm.tm_hour, - tm.tm_min, - tm.tm_sec); + sprintf(timeString, "%d-%02d-%02d %02d:%02d:%02d", tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday, tm.tm_hour, + tm.tm_min, tm.tm_sec); return timeString; } - void initLogFile() { char filename[256]; - char tmpString[128]; + char tmpString[128]; - sprintf(filename,"%s/../log/tmqlog_%s.txt", configDir, getCurrentTimeString(tmpString)); - //sprintf(filename, "%s/../log/tmqlog.txt", configDir); + sprintf(filename, "%s/../log/tmqlog_%s.txt", configDir, getCurrentTimeString(tmpString)); + // sprintf(filename, "%s/../log/tmqlog.txt", configDir); #ifdef WINDOWS for (int i = 2; i < sizeof(filename); i++) { if (filename[i] == ':') filename[i] = '-'; @@ -249,17 +243,18 @@ void addRowsToVgroupId(SThreadInfo* pInfo, int32_t vgroupId, int32_t rows) { for (i = 0; i < pInfo->numOfVgroups; i++) { if (vgroupId == pInfo->rowsOfPerVgroups[i][0]) { pInfo->rowsOfPerVgroups[i][1] += rows; - return; - } + return; + } } pInfo->rowsOfPerVgroups[pInfo->numOfVgroups][0] = vgroupId; pInfo->rowsOfPerVgroups[pInfo->numOfVgroups][1] += rows; pInfo->numOfVgroups++; - + taosFprintfFile(g_fp, "consume id %d, add one new vogroup id: %d\n", pInfo->consumerId, vgroupId); if (pInfo->numOfVgroups > MAX_VGROUP_CNT) { - taosFprintfFile(g_fp, "====consume id %d, vgroup num %d over than 32. new vgroupId: %d\n", pInfo->consumerId, pInfo->numOfVgroups, vgroupId); + taosFprintfFile(g_fp, "====consume id %d, vgroup num %d over than 32. new vgroupId: %d\n", pInfo->consumerId, + pInfo->numOfVgroups, vgroupId); taosCloseFile(&g_fp); exit(-1); } @@ -277,7 +272,8 @@ int32_t saveConsumeContentToTbl(SThreadInfo* pInfo, char* buf) { TAOS* pConn = taos_connect(NULL, "root", "taosdata", NULL, 0); assert(pConn != NULL); - sprintf(sqlStr, "insert into %s.content_%d values (%"PRId64", \'%s\')", g_stConfInfo.cdbName, pInfo->consumerId, pInfo->ts++, buf); + sprintf(sqlStr, "insert into %s.content_%d values (%" PRId64 ", \'%s\')", g_stConfInfo.cdbName, pInfo->consumerId, + pInfo->ts++, buf); TAOS_RES* pRes = taos_query(pConn, sqlStr); if (taos_errno(pRes) != 0) { pError("error in insert consume result, reason:%s\n", taos_errstr(pRes)); @@ -295,12 +291,13 @@ int32_t saveConsumeContentToTbl(SThreadInfo* pInfo, char* buf) { static int32_t msg_process(TAOS_RES* msg, SThreadInfo* pInfo, int32_t msgIndex) { char buf[1024]; int32_t totalRows = 0; - + // printf("topic: %s\n", tmq_get_topic_name(msg)); int32_t vgroupId = tmq_get_vgroup_id(msg); - + taosFprintfFile(g_fp, "msg index:%" PRId64 ", consumerId: %d\n", msgIndex, pInfo->consumerId); - //taosFprintfFile(g_fp, "topic: %s, vgroupId: %d, tableName: %s\n", tmq_get_topic_name(msg), vgroupId, tmq_get_table_name(msg)); + // taosFprintfFile(g_fp, "topic: %s, vgroupId: %d, tableName: %s\n", tmq_get_topic_name(msg), vgroupId, + // tmq_get_table_name(msg)); taosFprintfFile(g_fp, "topic: %s, vgroupId: %d\n", tmq_get_topic_name(msg), vgroupId); while (1) { @@ -316,9 +313,9 @@ static int32_t msg_process(TAOS_RES* msg, SThreadInfo* pInfo, int32_t msgIndex) const char* tbName = tmq_get_table_name(msg); if (0 != g_stConfInfo.showRowFlag) { - taosFprintfFile(g_fp, "tbname:%s, rows[%d]: %s\n", (tbName != NULL ? tbName:"null table"), totalRows, buf); - if (0 != g_stConfInfo.saveRowFlag) { - saveConsumeContentToTbl(pInfo, buf); + taosFprintfFile(g_fp, "tbname:%s, rows[%d]: %s\n", (tbName != NULL ? tbName : "null table"), totalRows, buf); + if (0 != g_stConfInfo.saveRowFlag) { + saveConsumeContentToTbl(pInfo, buf); } } @@ -326,7 +323,7 @@ static int32_t msg_process(TAOS_RES* msg, SThreadInfo* pInfo, int32_t msgIndex) } addRowsToVgroupId(pInfo, vgroupId, totalRows); - + return totalRows; } @@ -342,8 +339,8 @@ int queryDB(TAOS* taos, char* command) { return 0; } -static void tmq_commit_cb_print(tmq_t* tmq, tmq_resp_err_t resp, tmq_topic_vgroup_list_t* offsets, void* param) { - pError("tmq_commit_cb_print() commit %d\n", resp); +static void tmq_commit_cb_print(tmq_t* tmq, int32_t code, void* param) { + pError("tmq_commit_cb_print() commit %d\n", code); } void build_consumer(SThreadInfo* pInfo) { @@ -401,16 +398,11 @@ int32_t saveConsumeResult(SThreadInfo* pInfo) { int64_t now = taosGetTimestampMs(); // schema: ts timestamp, consumerid int, consummsgcnt bigint, checkresult int - sprintf(sqlStr, "insert into %s.consumeresult values (%"PRId64", %d, %" PRId64 ", %" PRId64 ", %d)", - g_stConfInfo.cdbName, - now, - pInfo->consumerId, - pInfo->consumeMsgCnt, - pInfo->consumeRowCnt, - pInfo->checkresult); + sprintf(sqlStr, "insert into %s.consumeresult values (%" PRId64 ", %d, %" PRId64 ", %" PRId64 ", %d)", + g_stConfInfo.cdbName, now, pInfo->consumerId, pInfo->consumeMsgCnt, pInfo->consumeRowCnt, pInfo->checkresult); char tmpString[128]; - taosFprintfFile(g_fp, "%s, consume id %d result: %s\n", getCurrentTimeString(tmpString), pInfo->consumerId ,sqlStr); + taosFprintfFile(g_fp, "%s, consume id %d result: %s\n", getCurrentTimeString(tmpString), pInfo->consumerId, sqlStr); TAOS_RES* pRes = taos_query(pConn, sqlStr); if (taos_errno(pRes) != 0) { @@ -421,7 +413,7 @@ int32_t saveConsumeResult(SThreadInfo* pInfo) { taos_free_result(pRes); - #if 0 +#if 0 // vgroups for (i = 0; i < pInfo->numOfVgroups; i++) { // schema: ts timestamp, consumerid int, consummsgcnt bigint, checkresult int @@ -445,19 +437,20 @@ int32_t saveConsumeResult(SThreadInfo* pInfo) { taos_free_result(pRes); } - #endif +#endif return 0; } void loop_consume(SThreadInfo* pInfo) { - tmq_resp_err_t err; + int32_t code; int64_t totalMsgs = 0; int64_t totalRows = 0; char tmpString[128]; - taosFprintfFile(g_fp, "%s consumer id %d start to loop pull msg\n", getCurrentTimeString(tmpString), pInfo->consumerId); + taosFprintfFile(g_fp, "%s consumer id %d start to loop pull msg\n", getCurrentTimeString(tmpString), + pInfo->consumerId); pInfo->ts = taosGetTimestampMs(); @@ -473,7 +466,7 @@ void loop_consume(SThreadInfo* pInfo) { totalMsgs++; if (totalRows >= pInfo->expectMsgCnt) { - char tmpString[128]; + char tmpString[128]; taosFprintfFile(g_fp, "%s over than expect rows, so break consume\n", getCurrentTimeString(tmpString)); break; } @@ -503,8 +496,8 @@ void* consumeThreadFunc(void* param) { return NULL; } - tmq_resp_err_t err = tmq_subscribe(pInfo->tmq, pInfo->topicList); - if (err) { + int32_t err = tmq_subscribe(pInfo->tmq, pInfo->topicList); + if (err != 0) { pError("tmq_subscribe() fail, reason: %s\n", tmq_err2str(err)); exit(-1); } @@ -519,17 +512,19 @@ void* consumeThreadFunc(void* param) { pPrint("tmq_commit() manual commit when consume end.\n"); /*tmq_commit(pInfo->tmq, NULL, 0);*/ tmq_commit_sync(pInfo->tmq, NULL); + taosFprintfFile(g_fp, "tmq_commit() manual commit over.\n"); + pPrint("tmq_commit() manual commit over.\n"); } err = tmq_unsubscribe(pInfo->tmq); - if (err) { + if (err != 0) { pError("tmq_unsubscribe() fail, reason: %s\n", tmq_err2str(err)); /*pInfo->consumeMsgCnt = -1;*/ /*return NULL;*/ } err = tmq_consumer_close(pInfo->tmq); - if (err) { + if (err != 0) { pError("tmq_consumer_close() fail, reason: %s\n", tmq_err2str(err)); /*exit(-1);*/ } diff --git a/tests/tsim/src/simExe.c b/tests/tsim/src/simExe.c index dbc3c2c460..f97a13d2c5 100644 --- a/tests/tsim/src/simExe.c +++ b/tests/tsim/src/simExe.c @@ -458,11 +458,17 @@ bool simExecuteSystemContentCmd(SScript *script, char *option) { char buf[4096] = {0}; char buf1[4096 + 512] = {0}; char filename[400] = {0}; - sprintf(filename, "%s/%s.tmp", simScriptDir, script->fileName); + sprintf(filename, "%s" TD_DIRSEP "%s.tmp", simScriptDir, script->fileName); +#ifdef WINDOWS + sprintf(buf, "cd %s && ", simScriptDir); + simVisuallizeOption(script, option, buf + strlen(buf)); + sprintf(buf1, "%s > %s 2>nul", buf, filename); +#else sprintf(buf, "cd %s; ", simScriptDir); simVisuallizeOption(script, option, buf + strlen(buf)); sprintf(buf1, "%s > %s 2>/dev/null", buf, filename); +#endif sprintf(script->system_exit_code, "%d", system(buf1)); simStoreSystemContentResult(script, filename); diff --git a/tests/tsim/src/simParse.c b/tests/tsim/src/simParse.c index 638c4a1ccb..5b6dda4dae 100644 --- a/tests/tsim/src/simParse.c +++ b/tests/tsim/src/simParse.c @@ -206,7 +206,7 @@ SScript *simParseScript(char *fileName) { for (int32_t i = 0; i < cmdlen; ++i) { if (buffer[i] == '\r' || buffer[i] == '\n') { - buffer[i] = ' '; + buffer[i] = '\0'; } } diff --git a/tools/taos-tools b/tools/taos-tools index 3d5aa76f8c..28a49b447f 160000 --- a/tools/taos-tools +++ b/tools/taos-tools @@ -1 +1 @@ -Subproject commit 3d5aa76f8c718dcffa100b45e4cbf313d499c356 +Subproject commit 28a49b447f71c4f014ebbac858b7215b897d57fd diff --git a/tools/taosadapter b/tools/taosadapter new file mode 160000 index 0000000000..29926478ed --- /dev/null +++ b/tools/taosadapter @@ -0,0 +1 @@ +Subproject commit 29926478edd87533a043f91c1a9ed0e27671e626