diff --git a/README-CN.md b/README-CN.md index e30e38ae78..0b7e42d4fa 100644 --- a/README-CN.md +++ b/README-CN.md @@ -303,14 +303,14 @@ Query OK, 2 row(s) in set (0.001700s) TDengine 提供了丰富的应用程序开发接口,其中包括 C/C++、Java、Python、Go、Node.js、C# 、RESTful 等,便于用户快速开发应用: -- [Java](https://docs.taosdata.com/reference/connector/java/) -- [C/C++](https://www.taosdata.com/cn/documentation/connector#c-cpp) -- [Python](https://docs.taosdata.com/reference/connector/python/) -- [Go](https://docs.taosdata.com/reference/connector/go/) -- [Node.js](https://docs.taosdata.com/reference/connector/node/) -- [Rust](https://docs.taosdata.com/reference/connector/rust/) -- [C#](https://docs.taosdata.com/reference/connector/csharp/) -- [RESTful API](https://docs.taosdata.com/reference/rest-api/) +- [Java](https://docs.taosdata.com/connector/java/) +- [C/C++](https://docs.taosdata.com/connector/cpp/) +- [Python](https://docs.taosdata.com/connector/python/) +- [Go](https://docs.taosdata.com/connector/go/) +- [Node.js](https://docs.taosdata.com/connector/node/) +- [Rust](https://docs.taosdata.com/connector/rust/) +- [C#](https://docs.taosdata.com/connector/csharp/) +- [RESTful API](https://docs.taosdata.com/connector/rest-api/) # 成为社区贡献者 diff --git a/README.md b/README.md index 02dd9984e8..611d97aac9 100644 --- a/README.md +++ b/README.md @@ -19,29 +19,29 @@ English | [简体中文](README-CN.md) | We are hiring, check [here](https://tde # What is TDengine? -TDengine is an open source, high-performance, cloud native time-series database optimized for Internet of Things (IoT), Connected Cars, and Industrial IoT. It enables efficient, real-time data ingestion, processing, and monitoring of TB and even PB scale data per day, generated by billions of sensors and data collectors. TDengine differentiates itself from other time-seires databases with the following advantages: +TDengine is an open source, high-performance, cloud native [time-series database](https://tdengine.com/tsdb/what-is-a-time-series-database/) optimized for Internet of Things (IoT), Connected Cars, and Industrial IoT. It enables efficient, real-time data ingestion, processing, and monitoring of TB and even PB scale data per day, generated by billions of sensors and data collectors. TDengine differentiates itself from other time-seires databases with the following advantages: -- **High-Performance**: TDengine is the only time-series database to solve the high cardinality issue to support billions of data collection points while out performing other time-series databases for data ingestion, querying and data compression. +- **[High-Performance](https://tdengine.com/tdengine/high-performance-time-series-database/)**: TDengine is the only time-series database to solve the high cardinality issue to support billions of data collection points while out performing other time-series databases for data ingestion, querying and data compression. -- **Simplified Solution**: Through built-in caching, stream processing and data subscription features, TDengine provides a simplified solution for time-series data processing. It reduces system design complexity and operation costs significantly. +- **[Simplified Solution](https://tdengine.com/tdengine/simplified-time-series-data-solution/)**: Through built-in caching, stream processing and data subscription features, TDengine provides a simplified solution for time-series data processing. It reduces system design complexity and operation costs significantly. -- **Cloud Native**: Through native distributed design, sharding and partitioning, separation of compute and storage, RAFT, support for kubernetes deployment and full observability, TDengine is a cloud native Time-Series Database and can be deployed on public, private or hybrid clouds. +- **[Cloud Native](https://tdengine.com/tdengine/cloud-native-time-series-database/)**: Through native distributed design, sharding and partitioning, separation of compute and storage, RAFT, support for kubernetes deployment and full observability, TDengine is a cloud native Time-Series Database and can be deployed on public, private or hybrid clouds. -- **Ease of Use**: For administrators, TDengine significantly reduces the effort to deploy and maintain. For developers, it provides a simple interface, simplified solution and seamless integrations for third party tools. For data users, it gives easy data access. +- **[Ease of Use](https://docs.tdengine.com/get-started/docker/)**: For administrators, TDengine significantly reduces the effort to deploy and maintain. For developers, it provides a simple interface, simplified solution and seamless integrations for third party tools. For data users, it gives easy data access. -- **Easy Data Analytics**: Through super tables, storage and compute separation, data partitioning by time interval, pre-computation and other means, TDengine makes it easy to explore, format, and get access to data in a highly efficient way. +- **[Easy Data Analytics](https://tdengine.com/tdengine/time-series-data-analytics-made-easy/)**: Through super tables, storage and compute separation, data partitioning by time interval, pre-computation and other means, TDengine makes it easy to explore, format, and get access to data in a highly efficient way. -- **Open Source**: TDengine’s core modules, including cluster feature, are all available under open source licenses. It has gathered 18.8k stars on GitHub. There is an active developer community, and over 139k running instances worldwide. +- **[Open Source](https://tdengine.com/tdengine/open-source-time-series-database/)**: TDengine’s core modules, including cluster feature, are all available under open source licenses. It has gathered 18.8k stars on GitHub. There is an active developer community, and over 139k running instances worldwide. # Documentation -For user manual, system design and architecture, please refer to [TDengine Documentation](https://docs.taosdata.com) ([TDengine 文档](https://docs.taosdata.com)) +For user manual, system design and architecture, please refer to [TDengine Documentation](https://docs.tdengine.com) ([TDengine 文档](https://docs.taosdata.com)) # Building -At the moment, TDengine server supports running on Linux, Windows systems.Any OS application can also choose the RESTful interface of taosAdapter to connect the taosd service . TDengine supports X64/ARM64 CPU , and it will support MIPS64, Alpha64, ARM32, RISC-V and other CPU architectures in the future. +At the moment, TDengine server supports running on Linux and Windows systems. Any application can also choose the RESTful interface provided by taosAdapter to connect the taosd service . TDengine supports X64/ARM64 CPU, and it will support MIPS64, Alpha64, ARM32, RISC-V and other CPU architectures in the future. -You can choose to install through source code according to your needs, [container](https://docs.taosdata.com/get-started/docker/), [installation package](https://docs.taosdata.com/get-started/package/) or [Kubernetes](https://docs.taosdata.com/deployment/k8s/) to install. This quick guide only applies to installing from source. +You can choose to install through source code, [container](https://docs.tdengine.com/get-started/docker/), [installation package](https://docs.tdengine.com/get-started/package/) or [Kubernetes](https://docs.tdengine.com/deployment/k8s/). This quick guide only applies to installing from source. TDengine provide a few useful tools such as taosBenchmark (was named taosdemo) and taosdump. They were part of TDengine. By default, TDengine compiling does not include taosTools. You can use `cmake .. -DBUILD_TOOLS=true` to make them be compiled with TDengine. @@ -256,6 +256,7 @@ After building successfully, TDengine can be installed by: nmake install ``` + ## Quick Run @@ -304,14 +306,14 @@ Query OK, 2 row(s) in set (0.001700s) TDengine provides abundant developing tools for users to develop on TDengine. Follow the links below to find your desired connectors and relevant documentation. -- [Java](https://docs.taosdata.com/reference/connector/java/) -- [C/C++](https://docs.taosdata.com/reference/connector/cpp/) -- [Python](https://docs.taosdata.com/reference/connector/python/) -- [Go](https://docs.taosdata.com/reference/connector/go/) -- [Node.js](https://docs.taosdata.com/reference/connector/node/) -- [Rust](https://docs.taosdata.com/reference/connector/rust/) -- [C#](https://docs.taosdata.com/reference/connector/csharp/) -- [RESTful API](https://docs.taosdata.com/reference/rest-api/) +- [Java](https://docs.tdengine.com/reference/connector/java/) +- [C/C++](https://docs.tdengine.com/reference/connector/cpp/) +- [Python](https://docs.tdengine.com/reference/connector/python/) +- [Go](https://docs.tdengine.com/reference/connector/go/) +- [Node.js](https://docs.tdengine.com/reference/connector/node/) +- [Rust](https://docs.tdengine.com/reference/connector/rust/) +- [C#](https://docs.tdengine.com/reference/connector/csharp/) +- [RESTful API](https://docs.tdengine.com/reference/rest-api/) # Contribute to TDengine diff --git a/cmake/cmake.define b/cmake/cmake.define index 376a55d396..5d64815a9a 100644 --- a/cmake/cmake.define +++ b/cmake/cmake.define @@ -2,8 +2,6 @@ cmake_minimum_required(VERSION 3.0) set(CMAKE_VERBOSE_MAKEFILE OFF) -SET(BUILD_SHARED_LIBS "OFF") - #set output directory SET(LIBRARY_OUTPUT_PATH ${PROJECT_BINARY_DIR}/build/lib) SET(EXECUTABLE_OUTPUT_PATH ${PROJECT_BINARY_DIR}/build/bin) diff --git a/cmake/cmake.install b/cmake/cmake.install index 6dc6864975..fd1e080dda 100644 --- a/cmake/cmake.install +++ b/cmake/cmake.install @@ -1,3 +1,19 @@ +SET(PREPARE_ENV_CMD "prepare_env_cmd") +SET(PREPARE_ENV_TARGET "prepare_env_target") +ADD_CUSTOM_COMMAND(OUTPUT ${PREPARE_ENV_CMD} + POST_BUILD + COMMAND echo "make test directory" + DEPENDS taosd + COMMAND ${CMAKE_COMMAND} -E make_directory ${TD_TESTS_OUTPUT_DIR}/cfg/ + COMMAND ${CMAKE_COMMAND} -E make_directory ${TD_TESTS_OUTPUT_DIR}/log/ + COMMAND ${CMAKE_COMMAND} -E make_directory ${TD_TESTS_OUTPUT_DIR}/data/ + COMMAND ${CMAKE_COMMAND} -E echo dataDir ${TD_TESTS_OUTPUT_DIR}/data > ${TD_TESTS_OUTPUT_DIR}/cfg/taos.cfg + COMMAND ${CMAKE_COMMAND} -E echo logDir ${TD_TESTS_OUTPUT_DIR}/log >> ${TD_TESTS_OUTPUT_DIR}/cfg/taos.cfg + COMMAND ${CMAKE_COMMAND} -E echo charset UTF-8 >> ${TD_TESTS_OUTPUT_DIR}/cfg/taos.cfg + COMMAND ${CMAKE_COMMAND} -E echo monitor 0 >> ${TD_TESTS_OUTPUT_DIR}/cfg/taos.cfg + COMMENT "prepare taosd environment") +ADD_CUSTOM_TARGET(${PREPARE_ENV_TARGET} ALL WORKING_DIRECTORY ${TD_EXECUTABLE_OUTPUT_PATH} DEPENDS ${PREPARE_ENV_CMD}) + IF (TD_LINUX) SET(TD_MAKE_INSTALL_SH "${TD_SOURCE_DIR}/packaging/tools/make_install.sh") INSTALL(CODE "MESSAGE(\"make install script: ${TD_MAKE_INSTALL_SH}\")") diff --git a/cmake/cmake.options b/cmake/cmake.options index bec64f7bf0..3baccde4d7 100644 --- a/cmake/cmake.options +++ b/cmake/cmake.options @@ -90,6 +90,12 @@ ELSE () ENDIF () ENDIF () +option( + BUILD_SHARED_LIBS + "" + OFF + ) + option( RUST_BINDINGS "If build with rust-bindings" diff --git a/cmake/taostools_CMakeLists.txt.in b/cmake/taostools_CMakeLists.txt.in index 2d9b00eee7..c0de75c6dd 100644 --- a/cmake/taostools_CMakeLists.txt.in +++ b/cmake/taostools_CMakeLists.txt.in @@ -2,7 +2,7 @@ # taos-tools ExternalProject_Add(taos-tools GIT_REPOSITORY https://github.com/taosdata/taos-tools.git - GIT_TAG 833b721 + GIT_TAG aa45ad4 SOURCE_DIR "${TD_SOURCE_DIR}/tools/taos-tools" BINARY_DIR "" #BUILD_IN_SOURCE TRUE diff --git a/docs/en/02-intro/index.md b/docs/en/02-intro/index.md index b4636e54a6..51df831948 100644 --- a/docs/en/02-intro/index.md +++ b/docs/en/02-intro/index.md @@ -11,21 +11,33 @@ This section introduces the major features, competitive advantages, typical use- The major features are listed below: -1. While TDengine supports [using SQL to insert](/develop/insert-data/sql-writing), it also supports [Schemaless writing](/reference/schemaless/) just like NoSQL databases. TDengine also supports standard protocols like [InfluxDB LINE](/develop/insert-data/influxdb-line),[OpenTSDB Telnet](/develop/insert-data/opentsdb-telnet), [OpenTSDB JSON ](/develop/insert-data/opentsdb-json) among others. -2. TDengine supports seamless integration with third-party data collection agents like [Telegraf](/third-party/telegraf),[Prometheus](/third-party/prometheus),[StatsD](/third-party/statsd),[collectd](/third-party/collectd),[icinga2](/third-party/icinga2), [TCollector](/third-party/tcollector), [EMQX](/third-party/emq-broker), [HiveMQ](/third-party/hive-mq-broker). These agents can write data into TDengine with simple configuration and without a single line of code. -3. Support for [all kinds of queries](/develop/query-data), including aggregation, nested query, downsampling, interpolation and others. -4. Support for [user defined functions](/develop/udf). -5. Support for [caching](/develop/cache). TDengine always saves the last data point in cache, so Redis is not needed in some scenarios. -6. Support for [continuous query](../develop/stream). -7. Support for [data subscription](../develop/tmq) with the capability to specify filter conditions. -8. Support for [cluster](../deployment/), with the capability of increasing processing power by adding more nodes. High availability is supported by replication. -9. Provides an interactive [command-line interface](/reference/taos-shell) for management, maintenance and ad-hoc queries. -10. Provides many ways to [import](/operation/import) and [export](/operation/export) data. -11. Provides [monitoring](/operation/monitor) on running instances of TDengine. -12. Provides [connectors](/reference/connector/) for [C/C++](/reference/connector/cpp), [Java](/reference/connector/java), [Python](/reference/connector/python), [Go](/reference/connector/go), [Rust](/reference/connector/rust), [Node.js](/reference/connector/node) and other programming languages. -13. Provides a [REST API](/reference/rest-api/). -14. Supports seamless integration with [Grafana](/third-party/grafana) for visualization. -15. Supports seamless integration with Google Data Studio. +1. Insert data + * supports [using SQL to insert](../develop/insert-data/sql-writing). + * supports [schemaless writing](../reference/schemaless/) just like NoSQL databases. It also supports standard protocols like [InfluxDB LINE](../develop/insert-data/influxdb-line),[OpenTSDB Telnet](../develop/insert-data/opentsdb-telnet), [OpenTSDB JSON ](../develop/insert-data/opentsdb-json) among others. + * supports seamless integration with third-party tools like [Telegraf](../third-party/telegraf/), [Prometheus](../third-party/prometheus/), [collectd](../third-party/collectd/), [StatsD](../third-party/statsd/), [TCollector](../third-party/tcollector/) and [icinga2/](../third-party/icinga2/), they can write data into TDengine with simple configuration and without a single line of code. +2. Query data + * supports standard [SQL](../taos-sql/), including nested query. + * supports [time series specific functions](../taos-sql/function/#time-series-extensions) and [time series specific queries](../taos-sql/distinguished), like downsampling, interpolation, cumulated sum, time weighted average, state window, session window and many others. + * supports [user defined functions](../taos-sql/udf). +3. [Caching](../develop/cache/): TDengine always saves the last data point in cache, so Redis is not needed for time-series data processing. +4. [Stream Processing](../develop/stream/): not only is the continuous query is supported, but TDengine also supports even driven stream processing, so Flink or spark is not needed for time-series daata processing. +5. [Data Dubscription](../develop/tmq/): application can subscribe a table or a set of tables. API is the same as Kafka, but you can specify filter conditions. +6. Visualization + * supports seamless integration with [Grafana](../third-party/grafana/) for visualization. + * supports seamless integration with Google Data Studio. +7. Cluster + * supports [cluster](../deployment/) with the capability of increasing processing power by adding more nodes. + * supports [deployment on Kubernetes](../deployment/k8s/) + * supports high availability via data replication. +8. Administration + * provides [monitoring](../operation/monitor) on running instances of TDengine. + * provides many ways to [import](../operation/import) and [export](../operation/export) data. +9. Tools + * provides an interactive [command-line interface](../reference/taos-shell) for management, maintenance and ad-hoc queries. + * provides a tool [taosBenchmark](../reference/taosbenchmark/) for testing the performance of TDengine. +10. Programming + * provides [connectors](../reference/connector/) for [C/C++](../reference/connector/cpp), [Java](../reference/connector/java), [Python](../reference/connector/python), [Go](../reference/connector/go), [Rust](../reference/connector/rust), [Node.js](../reference/connector/node) and other programming languages. + * provides a [REST API](../reference/rest-api/). For more details on features, please read through the entire documentation. diff --git a/docs/en/04-concept/index.md b/docs/en/04-concept/index.md index 44dcad82fc..b0a0c25d85 100644 --- a/docs/en/04-concept/index.md +++ b/docs/en/04-concept/index.md @@ -104,15 +104,15 @@ Each row contains the device ID, time stamp, collected metrics (current, voltage ## Metric -Metric refers to the physical quantity collected by sensors, equipment or other types of data collection devices, such as current, voltage, temperature, pressure, GPS position, etc., which change with time, and the data type can be integer, float, Boolean, or strings. As time goes by, the amount of collected metric data stored increases. +Metric refers to the physical quantity collected by sensors, equipment or other types of data collection devices, such as current, voltage, temperature, pressure, GPS position, etc., which change with time, and the data type can be integer, float, Boolean, or strings. As time goes by, the amount of collected metric data stored increases. In the smart meters example, current, voltage and phase are the metrics. ## Label/Tag -Label/Tag refers to the static properties of sensors, equipment or other types of data collection devices, which do not change with time, such as device model, color, fixed location of the device, etc. The data type can be any type. Although static, TDengine allows users to add, delete or update tag values at any time. Unlike the collected metric data, the amount of tag data stored does not change over time. +Label/Tag refers to the static properties of sensors, equipment or other types of data collection devices, which do not change with time, such as device model, color, fixed location of the device, etc. The data type can be any type. Although static, TDengine allows users to add, delete or update tag values at any time. Unlike the collected metric data, the amount of tag data stored does not change over time. In the meters example, `location` and `groupid` are the tags. ## Data Collection Point -Data Collection Point (DCP) refers to hardware or software that collects metrics based on preset time periods or triggered by events. A data collection point can collect one or multiple metrics, but these metrics are collected at the same time and have the same time stamp. For some complex equipment, there are often multiple data collection points, and the sampling rate of each collection point may be different, and fully independent. For example, for a car, there could be a data collection point to collect GPS position metrics, a data collection point to collect engine status metrics, and a data collection point to collect the environment metrics inside the car. So in this example the car would have three data collection points. +Data Collection Point (DCP) refers to hardware or software that collects metrics based on preset time periods or triggered by events. A data collection point can collect one or multiple metrics, but these metrics are collected at the same time and have the same time stamp. For some complex equipment, there are often multiple data collection points, and the sampling rate of each collection point may be different, and fully independent. For example, for a car, there could be a data collection point to collect GPS position metrics, a data collection point to collect engine status metrics, and a data collection point to collect the environment metrics inside the car. So in this example the car would have three data collection points. In the smart meters example, d1001, d1002, d1003, and d1004 are the data collection points. ## Table @@ -137,7 +137,7 @@ The design of one table for one data collection point will require a huge number STable is a template for a type of data collection point. A STable contains a set of data collection points (tables) that have the same schema or data structure, but with different static attributes (tags). To describe a STable, in addition to defining the table structure of the metrics, it is also necessary to define the schema of its tags. The data type of tags can be int, float, string, and there can be multiple tags, which can be added, deleted, or modified afterward. If the whole system has N different types of data collection points, N STables need to be established. -In the design of TDengine, **a table is used to represent a specific data collection point, and STable is used to represent a set of data collection points of the same type**. +In the design of TDengine, **a table is used to represent a specific data collection point, and STable is used to represent a set of data collection points of the same type**. In the smart meters example, we can create a super table named `meters`. ## Subtable @@ -156,11 +156,13 @@ The relationship between a STable and the subtables created based on this STable Queries can be executed on both a table (subtable) and a STable. For a query on a STable, TDengine will treat the data in all its subtables as a whole data set for processing. TDengine will first find the subtables that meet the tag filter conditions, then scan the time-series data of these subtables to perform aggregation operation, which reduces the number of data sets to be scanned which in turn greatly improves the performance of data aggregation across multiple DCPs. In essence, querying a supertable is a very efficient aggregate query on multiple DCPs of the same type. -In TDengine, it is recommended to use a subtable instead of a regular table for a DCP. +In TDengine, it is recommended to use a subtable instead of a regular table for a DCP. In the smart meters example, we can create subtables like d1001, d1002, d1003, and d1004 under super table meters. + +To better understand the data model using metri, tags, super table and subtable, please refer to the diagram below which demonstrates the data model of the smart meters example. ![Meters Data Model Diagram](./supertable.webp) ## Database -A database is a collection of tables. TDengine allows a running instance to have multiple databases, and each database can be configured with different storage policies. The [characteristics of time-series data](https://www.taosdata.com/blog/2019/07/09/86.html) from different data collection points may be different. Characteristics include collection frequency, retention policy and others which determine how you create and configure the database. For e.g. days to keep, number of replicas, data block size, whether data updates are allowed and other configurable parameters would be determined by the characteristics of your data and your business requirements. In order for TDengine to work with maximum efficiency in various scenarios, TDengine recommends that STables with different data characteristics be created in different databases. +A database is a collection of tables. TDengine allows a running instance to have multiple databases, and each database can be configured with different storage policies. The [characteristics of time-series data](https://tdengine.com/tsdb/characteristics-of-time-series-data/) from different data collection points may be different. Characteristics include collection frequency, retention policy and others which determine how you create and configure the database. For e.g. days to keep, number of replicas, data block size, whether data updates are allowed and other configurable parameters would be determined by the characteristics of your data and your business requirements. In order for TDengine to work with maximum efficiency in various scenarios, TDengine recommends that STables with different data characteristics be created in different databases. In a database, there can be one or more STables, but a STable belongs to only one database. All tables owned by a STable are stored in only one database. diff --git a/docs/en/04-concept/supertable.webp b/docs/en/04-concept/supertable.webp new file mode 100644 index 0000000000..764b8f3de7 Binary files /dev/null and b/docs/en/04-concept/supertable.webp differ diff --git a/docs/en/07-develop/01-connect/index.md b/docs/en/07-develop/01-connect/index.md index 2053706421..901fe69d24 100644 --- a/docs/en/07-develop/01-connect/index.md +++ b/docs/en/07-develop/01-connect/index.md @@ -279,6 +279,6 @@ Prior to establishing connection, please make sure TDengine is already running a :::tip -If the connection fails, in most cases it's caused by improper configuration for FQDN or firewall. Please refer to the section "Unable to establish connection" in [FAQ](https://docs.taosdata.com/train-faq/faq). +If the connection fails, in most cases it's caused by improper configuration for FQDN or firewall. Please refer to the section "Unable to establish connection" in [FAQ](https://docs.tdengine.com/train-faq/faq). ::: diff --git a/docs/en/07-develop/03-insert-data/05-high-volume.md b/docs/en/07-develop/03-insert-data/05-high-volume.md new file mode 100644 index 0000000000..8163ae03b2 --- /dev/null +++ b/docs/en/07-develop/03-insert-data/05-high-volume.md @@ -0,0 +1,444 @@ +--- +sidebar_label: High Performance Writing +title: High Performance Writing +--- + +import Tabs from "@theme/Tabs"; +import TabItem from "@theme/TabItem"; + +This chapter introduces how to write data into TDengine with high throughput. + +## How to achieve high performance data writing + +To achieve high performance writing, there are a few aspects to consider. In the following sections we will describe these important factors in achieving high performance writing. + +### Application Program + +From the perspective of application program, you need to consider: + +1. The data size of each single write, also known as batch size. Generally speaking, higher batch size generates better writing performance. However, once the batch size is over a specific value, you will not get any additional benefit anymore. When using SQL to write into TDengine, it's better to put as much as possible data in single SQL. The maximum SQL length supported by TDengine is 1,048,576 bytes, i.e. 1 MB. It can be configured by parameter `maxSQLLength` on client side, and the default value is 65,480. + +2. The number of concurrent connections. Normally more connections can get better result. However, once the number of connections exceeds the processing ability of the server side, the performance may downgrade. + +3. The distribution of data to be written across tables or sub-tables. Writing to single table in one batch is more efficient than writing to multiple tables in one batch. + +4. Data Writing Protocol. + - Prameter binding mode is more efficient than SQL because it doesn't have the cost of parsing SQL. + - Writing to known existing tables is more efficient than wirting to uncertain tables in automatic creating mode because the later needs to check whether the table exists or not before actually writing data into it + - Writing in SQL is more efficient than writing in schemaless mode because schemaless writing creats table automatically and may alter table schema + +Application programs need to take care of the above factors and try to take advantage of them. The application progam should write to single table in each write batch. The batch size needs to be tuned to a proper value on a specific system. The number of concurrent connections needs to be tuned to a proper value too to achieve the best writing throughput. + +### Data Source + +Application programs need to read data from data source then write into TDengine. If you meet one or more of below situations, you need to setup message queues between the threads for reading from data source and the threads for writing into TDengine. + +1. There are multiple data sources, the data generation speed of each data source is much slower than the speed of single writing thread. In this case, the purpose of message queues is to consolidate the data from multiple data sources together to increase the batch size of single write. +2. The speed of data generation from single data source is much higher than the speed of single writing thread. The purpose of message queue in this case is to provide buffer so that data is not lost and multiple writing threads can get data from the buffer. +3. The data for single table are from multiple data source. In this case the purpose of message queues is to combine the data for single table together to improve the write efficiency. + +If the data source is Kafka, then the appication program is a consumer of Kafka, you can benefit from some kafka features to achieve high performance writing: + +1. Put the data for a table in single partition of single topic so that it's easier to put the data for each table together and write in batch +2. Subscribe multiple topics to accumulate data together. +3. Add more consumers to gain more concurrency and throughput. +4. Incrase the size of single fetch to increase the size of write batch. + +### Tune TDengine + +TDengine is a distributed and high performance time series database, there are also some ways to tune TDengine to get better writing performance. + +1. Set proper number of `vgroups` according to available CPU cores. Normally, we recommend 2 \* number_of_cores as a starting point. If the verification result shows this is not enough to utilize CPU resources, you can use a higher value. +2. Set proper `minTablesPerVnode`, `tableIncStepPerVnode`, and `maxVgroupsPerDb` according to the number of tables so that tables are distributed even across vgroups. The purpose is to balance the workload among all vnodes so that system resources can be utilized better to get higher performance. + +For more performance tuning parameters, please refer to [Configuration Parameters](../../../reference/config). + +## Sample Programs + +This section will introduce the sample programs to demonstrate how to write into TDengine with high performance. + +### Scenario + +Below are the scenario for the sample programs of high performance wrting. + +- Application program reads data from data source, the sample program simulates a data source by generating data +- The speed of single writing thread is much slower than the speed of generating data, so the program starts multiple writing threads while each thread establish a connection to TDengine and each thread has a message queue of fixed size. +- Application program maps the received data to different writing threads based on table name to make sure all the data for each table is always processed by a specific writing thread. +- Each writing thread writes the received data into TDengine once the message queue becomes empty or the read data meets a threshold. + +![Thread Model of High Performance Writing into TDengine](highvolume.webp) + +### Sample Programs + +The sample programs listed in this section are based on the scenario described previously. If your scenarios is different, please try to adjust the code based on the principles described in this chapter. + +The sample programs assume the source data is for all the different sub tables in same super table (meters). The super table has been created before the sample program starts to writing data. Sub tables are created automatically according to received data. If there are multiple super tables in your case, please try to adjust the part of creating table automatically. + + + + +**Program Inventory** + +| Class | Description | +| ---------------- | ----------------------------------------------------------------------------------------------------- | +| FastWriteExample | Main Program | +| ReadTask | Read data from simulated data source and put into a queue according to the hash value of table name | +| WriteTask | Read data from Queue, compose a wirte batch and write into TDengine | +| MockDataSource | Generate data for some sub tables of super table meters | +| SQLWriter | WriteTask uses this class to compose SQL, create table automatically, check SQL length and write data | +| StmtWriter | Write in Parameter binding mode (Not finished yet) | +| DataBaseMonitor | Calculate the writing speed and output on console every 10 seconds | + +Below is the list of complete code of the classes in above table and more detailed description. + +
+FastWriteExample +The main Program is responsible for: + +1. Create message queues +2. Start writing threads +3. Start reading threads +4. Otuput writing speed every 10 seconds + +The main program provides 4 parameters for tuning: + +1. The number of reading threads, default value is 1 +2. The number of writing threads, default alue is 2 +3. The total number of tables in the generated data, default value is 1000. These tables are distributed evenly across all writing threads. If the number of tables is very big, it will cost much time to firstly create these tables. +4. The batch size of single write, default value is 3,000 + +The capacity of message queue also impacts performance and can be tuned by modifying program. Normally it's always better to have a larger message queue. A larger message queue means lower possibility of being blocked when enqueueing and higher throughput. But a larger message queue consumes more memory space. The default value used in the sample programs is already big enoug. + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/FastWriteExample.java}} +``` + +
+ +
+ReadTask + +ReadTask reads data from data source. Each ReadTask is associated with a simulated data source, each data source generates data for a group of specific tables, and the data of any table is only generated from a single specific data source. + +ReadTask puts data in message queue in blocking mode. That means, the putting operation is blocked if the message queue is full. + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/ReadTask.java}} +``` + +
+ +
+WriteTask + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/WriteTask.java}} +``` + +
+ +
+ +MockDataSource + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/MockDataSource.java}} +``` + +
+ +
+ +SQLWriter + +SQLWriter class encapsulates the logic of composing SQL and writing data. Please be noted that the tables have not been created before writing, but are created automatically when catching the exception of table doesn't exist. For other exceptions caught, the SQL which caused the exception are logged for you to debug. + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/SQLWriter.java}} +``` + +
+ +
+ +DataBaseMonitor + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/DataBaseMonitor.java}} +``` + +
+ +**Steps to Launch** + +
+Launch Java Sample Program + +You need to set environment variable `TDENGINE_JDBC_URL` before launching the program. If TDengine Server is setup on localhost, then the default value for user name, password and port can be used, like below: + +``` +TDENGINE_JDBC_URL="jdbc:TAOS://localhost:6030?user=root&password=taosdata" +``` + +**Launch in IDE** + +1. Clone TDengine repolitory + ``` + git clone git@github.com:taosdata/TDengine.git --depth 1 + ``` +2. Use IDE to open `docs/examples/java` directory +3. Configure environment variable `TDENGINE_JDBC_URL`, you can also configure it before launching the IDE, if so you can skip this step. +4. Run class `com.taos.example.highvolume.FastWriteExample` + +**Launch on server** + +If you want to launch the sample program on a remote server, please follow below steps: + +1. Package the sample programs. Execute below command under directory `TDengine/docs/examples/java` : + ``` + mvn package + ``` +2. Create `examples/java` directory on the server + ``` + mkdir -p examples/java + ``` +3. Copy dependencies (below commands assume you are working on a local Windows host and try to launch on a remote Linux host) + - Copy dependent packages + ``` + scp -r .\target\lib @:~/examples/java + ``` + - Copy the jar of sample programs + ``` + scp -r .\target\javaexample-1.0.jar @:~/examples/java + ``` +4. Configure environment variable + Edit `~/.bash_profile` or `~/.bashrc` and add below: + + ``` + export TDENGINE_JDBC_URL="jdbc:TAOS://localhost:6030?user=root&password=taosdata" + ``` + + If your TDengine server is not deployed on localhost or doesn't use default port, you need to change the above URL to correct value in your environment. + +5. Launch the sample program + + ``` + java -classpath lib/*:javaexample-1.0.jar com.taos.example.highvolume.FastWriteExample + ``` + +6. The sample program doesn't exit unless you press CTRL + C to terminate it. + Below is the output of running on a server of 16 cores, 64GB memory and SSD hard disk. + + ``` + root@vm85$ java -classpath lib/*:javaexample-1.0.jar com.taos.example.highvolume.FastWriteExample 2 12 + 18:56:35.896 [main] INFO c.t.e.highvolume.FastWriteExample - readTaskCount=2, writeTaskCount=12 tableCount=1000 maxBatchSize=3000 + 18:56:36.011 [WriteThread-0] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.015 [WriteThread-0] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.021 [WriteThread-1] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.022 [WriteThread-1] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.031 [WriteThread-2] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.032 [WriteThread-2] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.041 [WriteThread-3] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.042 [WriteThread-3] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.093 [WriteThread-4] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.094 [WriteThread-4] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.099 [WriteThread-5] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.100 [WriteThread-5] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.100 [WriteThread-6] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.101 [WriteThread-6] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.103 [WriteThread-7] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.104 [WriteThread-7] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.105 [WriteThread-8] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.107 [WriteThread-8] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.108 [WriteThread-9] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.109 [WriteThread-9] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.156 [WriteThread-10] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.157 [WriteThread-11] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.158 [WriteThread-10] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.158 [ReadThread-0] INFO com.taos.example.highvolume.ReadTask - started + 18:56:36.158 [ReadThread-1] INFO com.taos.example.highvolume.ReadTask - started + 18:56:36.158 [WriteThread-11] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:46.369 [main] INFO c.t.e.highvolume.FastWriteExample - count=18554448 speed=1855444 + 18:56:56.946 [main] INFO c.t.e.highvolume.FastWriteExample - count=39059660 speed=2050521 + 18:57:07.322 [main] INFO c.t.e.highvolume.FastWriteExample - count=59403604 speed=2034394 + 18:57:18.032 [main] INFO c.t.e.highvolume.FastWriteExample - count=80262938 speed=2085933 + 18:57:28.432 [main] INFO c.t.e.highvolume.FastWriteExample - count=101139906 speed=2087696 + 18:57:38.921 [main] INFO c.t.e.highvolume.FastWriteExample - count=121807202 speed=2066729 + 18:57:49.375 [main] INFO c.t.e.highvolume.FastWriteExample - count=142952417 speed=2114521 + 18:58:00.689 [main] INFO c.t.e.highvolume.FastWriteExample - count=163650306 speed=2069788 + 18:58:11.646 [main] INFO c.t.e.highvolume.FastWriteExample - count=185019808 speed=2136950 + ``` + +
+ +
+ + +**Program Inventory** + +Sample programs in Python uses multi-process and cross-process message queues. + +| Function/CLass | Description | +| ---------------------------- | --------------------------------------------------------------------------- | +| main Function | Program entry point, create child processes and message queues | +| run_monitor_process Function | Create database, super table, calculate writing speed and output to console | +| run_read_task Function | Read data and distribute to message queues | +| MockDataSource Class | Simulate data source, return next 1,000 rows of each table | +| run_write_task Function | Read as much as possible data from message queue and write in batch | +| SQLWriter Class | Write in SQL and create table utomatically | +| StmtWriter Class | Write in parameter binding mode (not finished yet) | + +
+main function + +`main` function is responsible for creating message queues and fork child processes, there are 3 kinds of child processes: + +1. Monitoring process, initializes database and calculating writing speed +2. Reading process (n), reads data from data source +3. Writing process (m), wirtes data into TDengine + +`main` function provides 5 parameters: + +1. The number of reading tasks, default value is 1 +2. The number of writing tasks, default value is 1 +3. The number of tables, default value is 1,000 +4. The capacity of message queue, default value is 1,000,000 bytes +5. The batch size in single write, default value is 3000 + +```python +{{#include docs/examples/python/fast_write_example.py:main}} +``` + +
+ +
+run_monitor_process + +Monitoring process initilizes database and monitoring writing speed. + +```python +{{#include docs/examples/python/fast_write_example.py:monitor}} +``` + +
+ +
+ +run_read_task function + +Reading process reads data from other data system and distributes to the message queue allocated for it. + +```python +{{#include docs/examples/python/fast_write_example.py:read}} +``` + +
+ +
+ +MockDataSource + +Below is the simulated data source, we assume table name exists in each generated data. + +```python +{{#include docs/examples/python/mockdatasource.py}} +``` + +
+ +
+run_write_task function + +Writing process tries to read as much as possible data from message queue and writes in batch. + +```python +{{#include docs/examples/python/fast_write_example.py:write}} +``` + +
+ +
+ +SQLWriter class encapsulates the logic of composing SQL and writing data. Please be noted that the tables have not been created before writing, but are created automatically when catching the exception of table doesn't exist. For other exceptions caught, the SQL which caused the exception are logged for you to debug. This class also checks the SQL length, if the SQL length is closed to `maxSQLLength` the SQL will be executed immediately. To improve writing efficiency, it's better to increase `maxSQLLength` properly. + +SQLWriter + +```python +{{#include docs/examples/python/sql_writer.py}} +``` + +
+ +**Steps to Launch** + +
+ +Launch Sample Program in Python + +1. Prerequisities + + - TDengine client driver has been installed + - Python3 has been installed, the the version >= 3.8 + - TDengine Python connector `taospy` has been installed + +2. Install faster-fifo to replace python builtin multiprocessing.Queue + + ``` + pip3 install faster-fifo + ``` + +3. Click the "Copy" in the above sample programs to copy `fast_write_example.py` 、 `sql_writer.py` and `mockdatasource.py`. + +4. Execute the program + + ``` + python3 fast_write_example.py + ``` + + Below is the output of running on a server of 16 cores, 64GB memory and SSD hard disk. + + ``` + root@vm85$ python3 fast_write_example.py 8 8 + 2022-07-14 19:13:45,869 [root] - READ_TASK_COUNT=8, WRITE_TASK_COUNT=8, TABLE_COUNT=1000, QUEUE_SIZE=1000000, MAX_BATCH_SIZE=3000 + 2022-07-14 19:13:48,882 [root] - WriteTask-0 started with pid 718347 + 2022-07-14 19:13:48,883 [root] - WriteTask-1 started with pid 718348 + 2022-07-14 19:13:48,884 [root] - WriteTask-2 started with pid 718349 + 2022-07-14 19:13:48,884 [root] - WriteTask-3 started with pid 718350 + 2022-07-14 19:13:48,885 [root] - WriteTask-4 started with pid 718351 + 2022-07-14 19:13:48,885 [root] - WriteTask-5 started with pid 718352 + 2022-07-14 19:13:48,886 [root] - WriteTask-6 started with pid 718353 + 2022-07-14 19:13:48,886 [root] - WriteTask-7 started with pid 718354 + 2022-07-14 19:13:48,887 [root] - ReadTask-0 started with pid 718355 + 2022-07-14 19:13:48,888 [root] - ReadTask-1 started with pid 718356 + 2022-07-14 19:13:48,889 [root] - ReadTask-2 started with pid 718357 + 2022-07-14 19:13:48,889 [root] - ReadTask-3 started with pid 718358 + 2022-07-14 19:13:48,890 [root] - ReadTask-4 started with pid 718359 + 2022-07-14 19:13:48,891 [root] - ReadTask-5 started with pid 718361 + 2022-07-14 19:13:48,892 [root] - ReadTask-6 started with pid 718364 + 2022-07-14 19:13:48,893 [root] - ReadTask-7 started with pid 718365 + 2022-07-14 19:13:56,042 [DataBaseMonitor] - count=6676310 speed=667631.0 + 2022-07-14 19:14:06,196 [DataBaseMonitor] - count=20004310 speed=1332800.0 + 2022-07-14 19:14:16,366 [DataBaseMonitor] - count=32290310 speed=1228600.0 + 2022-07-14 19:14:26,527 [DataBaseMonitor] - count=44438310 speed=1214800.0 + 2022-07-14 19:14:36,673 [DataBaseMonitor] - count=56608310 speed=1217000.0 + 2022-07-14 19:14:46,834 [DataBaseMonitor] - count=68757310 speed=1214900.0 + 2022-07-14 19:14:57,280 [DataBaseMonitor] - count=80992310 speed=1223500.0 + 2022-07-14 19:15:07,689 [DataBaseMonitor] - count=93805310 speed=1281300.0 + 2022-07-14 19:15:18,020 [DataBaseMonitor] - count=106111310 speed=1230600.0 + 2022-07-14 19:15:28,356 [DataBaseMonitor] - count=118394310 speed=1228300.0 + 2022-07-14 19:15:38,690 [DataBaseMonitor] - count=130742310 speed=1234800.0 + 2022-07-14 19:15:49,000 [DataBaseMonitor] - count=143051310 speed=1230900.0 + 2022-07-14 19:15:59,323 [DataBaseMonitor] - count=155276310 speed=1222500.0 + 2022-07-14 19:16:09,649 [DataBaseMonitor] - count=167603310 speed=1232700.0 + 2022-07-14 19:16:19,995 [DataBaseMonitor] - count=179976310 speed=1237300.0 + ``` + +
+ +:::note +Don't establish connection to TDengine in the parent process if using Python connector in multi-process way, otherwise all the connections in child processes are blocked always. This is a known issue. + +::: + +
+
diff --git a/docs/en/07-develop/03-insert-data/highvolume.webp b/docs/en/07-develop/03-insert-data/highvolume.webp new file mode 100644 index 0000000000..46dfc74ae3 Binary files /dev/null and b/docs/en/07-develop/03-insert-data/highvolume.webp differ diff --git a/docs/en/07-develop/07-tmq.mdx b/docs/en/07-develop/07-tmq.mdx index ceeea64fca..17b3f5caa0 100644 --- a/docs/en/07-develop/07-tmq.mdx +++ b/docs/en/07-develop/07-tmq.mdx @@ -16,7 +16,7 @@ import CDemo from "./_sub_c.mdx"; TDengine provides data subscription and consumption interfaces similar to message queue products. These interfaces make it easier for applications to obtain data written to TDengine either in real time and to process data in the order that events occurred. This simplifies your time-series data processing systems and reduces your costs because it is no longer necessary to deploy a message queue product such as Kafka. -To use TDengine data subscription, you define topics like in Kafka. However, a topic in TDengine is based on query conditions for an existing supertable, standard table, or subtable - in other words, a SELECT statement. You can use SQL to filter data by tag, table name, column, or expression and then perform a scalar function or user-defined function on the data. Aggregate functions are not supported. This gives TDengine data subscription more flexibility than similar products. The granularity of data can be controlled on demand by applications, while filtering and preprocessing are handled by TDengine instead of the application layer. This implementation reduces the amount of data transmitted and the complexity of applications. +To use TDengine data subscription, you define topics like in Kafka. However, a topic in TDengine is based on query conditions for an existing supertable, table, or subtable - in other words, a SELECT statement. You can use SQL to filter data by tag, table name, column, or expression and then perform a scalar function or user-defined function on the data. Aggregate functions are not supported. This gives TDengine data subscription more flexibility than similar products. The granularity of data can be controlled on demand by applications, while filtering and preprocessing are handled by TDengine instead of the application layer. This implementation reduces the amount of data transmitted and the complexity of applications. By subscribing to a topic, a consumer can obtain the latest data in that topic in real time. Multiple consumers can be formed into a consumer group that consumes messages together. Consumer groups enable faster speed through multi-threaded, distributed data consumption. Note that consumers in different groups that are subscribed to the same topic do not consume messages together. A single consumer can subscribe to multiple topics. If the data in a supertable is sharded across multiple vnodes, consumer groups can consume it much more efficiently than single consumers. TDengine also includes an acknowledgement mechanism that ensures at-least-once delivery in complicated environments where machines may crash or restart. diff --git a/docs/en/07-develop/08-cache.md b/docs/en/07-develop/08-cache.md index 4892c21c9d..82a4787016 100644 --- a/docs/en/07-develop/08-cache.md +++ b/docs/en/07-develop/08-cache.md @@ -20,11 +20,11 @@ In theory, larger cache sizes are always better. However, at a certain point, it ## Read Cache -When you create a database, you can configure whether the latest data from every subtable is cached. To do so, set the *cachelast* parameter as follows: -- 0: Caching is disabled. -- 1: The latest row of data in each subtable is cached. This option significantly improves the performance of the `LAST_ROW` function -- 2: The latest non-null value in each column of each subtable is cached. This option significantly improves the performance of the `LAST` function in normal situations, such as WHERE, ORDER BY, GROUP BY, and INTERVAL statements. -- 3: Rows and columns are both cached. This option is equivalent to simultaneously enabling options 1 and 2. +When you create a database, you can configure whether the latest data from every subtable is cached. To do so, set the *cachemodel* parameter as follows: +- none: Caching is disabled. +- last_row: The latest row of data in each subtable is cached. This option significantly improves the performance of the `LAST_ROW` function +- last_value: The latest non-null value in each column of each subtable is cached. This option significantly improves the performance of the `LAST` function in normal situations, such as WHERE, ORDER BY, GROUP BY, and INTERVAL statements. +- both: Rows and columns are both cached. This option is equivalent to simultaneously enabling option last_row and last_value. ## Metadata Cache diff --git a/docs/en/10-deployment/01-deploy.md b/docs/en/10-deployment/01-deploy.md index bfbb547bd4..5dfcd3108d 100644 --- a/docs/en/10-deployment/01-deploy.md +++ b/docs/en/10-deployment/01-deploy.md @@ -39,18 +39,18 @@ To get the hostname on any host, the command `hostname -f` can be executed. On the physical machine running the application, ping the dnode that is running taosd. If the dnode is not accessible, the application cannot connect to taosd. In this case, verify the DNS and hosts settings on the physical node running the application. -The end point of each dnode is the output hostname and port, such as h1.taosdata.com:6030. +The end point of each dnode is the output hostname and port, such as h1.tdengine.com:6030. ### Step 5 -Modify the TDengine configuration file `/etc/taos/taos.cfg` on each node. Assuming the first dnode of TDengine cluster is "h1.taosdata.com:6030", its `taos.cfg` is configured as following. +Modify the TDengine configuration file `/etc/taos/taos.cfg` on each node. Assuming the first dnode of TDengine cluster is "h1.tdengine.com:6030", its `taos.cfg` is configured as following. ```c // firstEp is the end point to connect to when any dnode starts -firstEp h1.taosdata.com:6030 +firstEp h1.tdengine.com:6030 // must be configured to the FQDN of the host where the dnode is launched -fqdn h1.taosdata.com +fqdn h1.tdengine.com // the port used by the dnode, default is 6030 serverPort 6030 @@ -76,13 +76,13 @@ The first dnode can be started following the instructions in [Get Started](/get- taos> show dnodes; id | endpoint | vnodes | support_vnodes | status | create_time | note | ============================================================================================================================================ -1 | h1.taosdata.com:6030 | 0 | 1024 | ready | 2022-07-16 10:50:42.673 | | +1 | h1.tdengine.com:6030 | 0 | 1024 | ready | 2022-07-16 10:50:42.673 | | Query OK, 1 rows affected (0.007984s) ``` -From the above output, it is shown that the end point of the started dnode is "h1.taosdata.com:6030", which is the `firstEp` of the cluster. +From the above output, it is shown that the end point of the started dnode is "h1.tdengine.com:6030", which is the `firstEp` of the cluster. ## Add DNODE @@ -90,7 +90,7 @@ There are a few steps necessary to add other dnodes in the cluster. Second, we can start `taosd` as instructed in [Get Started](/get-started/). -Then, on the first dnode i.e. h1.taosdata.com in our example, use TDengine CLI `taos` to execute the following command: +Then, on the first dnode i.e. h1.tdengine.com in our example, use TDengine CLI `taos` to execute the following command: ```sql CREATE DNODE "h2.taos.com:6030"; @@ -98,7 +98,7 @@ CREATE DNODE "h2.taos.com:6030"; This adds the end point of the new dnode (from Step 4) into the end point list of the cluster. In the command "fqdn:port" should be quoted using double quotes. Change `"h2.taos.com:6030"` to the end point of your new dnode. -Then on the first dnode h1.taosdata.com, execute `show dnodes` in `taos` +Then on the first dnode h1.tdengine.com, execute `show dnodes` in `taos` ```sql SHOW DNODES; @@ -114,7 +114,9 @@ The above process can be repeated to add more dnodes in the cluster. Any node that is in the cluster and online can be the firstEp of new nodes. Nodes use the firstEp parameter only when joining a cluster for the first time. After a node has joined the cluster, it stores the latest mnode in its end point list and no longer makes use of firstEp. -However, firstEp is used by clients that connect to the cluster. For example, if you run `taos shell` without arguments, it connects to the firstEp by default. + +However, firstEp is used by clients that connect to the cluster. For example, if you run TDengine CLI `taos` without arguments, it connects to the firstEp by default. + Two dnodes that are launched without a firstEp value operate independently of each other. It is not possible to add one dnode to the other dnode and form a cluster. It is also not possible to form two independent clusters into a new cluster. ::: diff --git a/docs/en/10-deployment/03-k8s.md b/docs/en/10-deployment/03-k8s.md index b3f71ed5bd..b0aa677713 100644 --- a/docs/en/10-deployment/03-k8s.md +++ b/docs/en/10-deployment/03-k8s.md @@ -9,6 +9,7 @@ TDengine is a cloud-native time-series database that can be deployed on Kubernet Before deploying TDengine on Kubernetes, perform the following: +* Current steps are compatible with Kubernetes v1.5 and later version. * Install and configure minikube, kubectl, and helm. * Install and deploy Kubernetes and ensure that it can be accessed and used normally. Update any container registries or other services as necessary. @@ -100,7 +101,7 @@ spec: # Must set if you want a cluster. - name: TAOS_FIRST_EP value: "$(STS_NAME)-0.$(SERVICE_NAME).$(STS_NAMESPACE).svc.cluster.local:$(TAOS_SERVER_PORT)" - # TAOS_FQND should always be setted in k8s env. + # TAOS_FQDN should always be set in k8s env. - name: TAOS_FQDN value: "$(POD_NAME).$(SERVICE_NAME).$(STS_NAMESPACE).svc.cluster.local" volumeMounts: diff --git a/docs/en/10-deployment/05-helm.md b/docs/en/10-deployment/05-helm.md index 48cd9df32c..a4fa681000 100644 --- a/docs/en/10-deployment/05-helm.md +++ b/docs/en/10-deployment/05-helm.md @@ -152,7 +152,7 @@ clusterDomainSuffix: "" # converting an upper-snake-cased variable like `TAOS_DEBUG_FLAG`, # to a camelCase taos config variable `debugFlag`. # -# See the variable list at https://www.taosdata.com/cn/documentation/administrator . +# See the [Configuration Variables](../../reference/config) # # Note: # 1. firstEp/secondEp: should not be setted here, it's auto generated at scale-up. @@ -170,71 +170,21 @@ taoscfg: # number of replications, for cluster only TAOS_REPLICA: "1" - - # number of days per DB file - # TAOS_DAYS: "10" - - # number of days to keep DB file, default is 10 years. - #TAOS_KEEP: "3650" - - # cache block size (Mbyte) - #TAOS_CACHE: "16" - - # number of cache blocks per vnode - #TAOS_BLOCKS: "6" - - # minimum rows of records in file block - #TAOS_MIN_ROWS: "100" - - # maximum rows of records in file block - #TAOS_MAX_ROWS: "4096" - # - # TAOS_NUM_OF_THREADS_PER_CORE: number of threads per CPU core - #TAOS_NUM_OF_THREADS_PER_CORE: "1.0" + # TAOS_NUM_OF_RPC_THREADS: number of threads for RPC + #TAOS_NUM_OF_RPC_THREADS: "2" + # # TAOS_NUM_OF_COMMIT_THREADS: number of threads to commit cache data #TAOS_NUM_OF_COMMIT_THREADS: "4" - # - # TAOS_RATIO_OF_QUERY_CORES: - # the proportion of total CPU cores available for query processing - # 2.0: the query threads will be set to double of the CPU cores. - # 1.0: all CPU cores are available for query processing [default]. - # 0.5: only half of the CPU cores are available for query. - # 0.0: only one core available. - #TAOS_RATIO_OF_QUERY_CORES: "1.0" - - # - # TAOS_KEEP_COLUMN_NAME: - # the last_row/first/last aggregator will not change the original column name in the result fields - #TAOS_KEEP_COLUMN_NAME: "0" - - # enable/disable backuping vnode directory when removing vnode - #TAOS_VNODE_BAK: "1" - # enable/disable installation / usage report #TAOS_TELEMETRY_REPORTING: "1" - # enable/disable load balancing - #TAOS_BALANCE: "1" - - # max timer control blocks - #TAOS_MAX_TMR_CTRL: "512" - # time interval of system monitor, seconds #TAOS_MONITOR_INTERVAL: "30" - # number of seconds allowed for a dnode to be offline, for cluster only - #TAOS_OFFLINE_THRESHOLD: "8640000" - - # RPC re-try timer, millisecond - #TAOS_RPC_TIMER: "1000" - - # RPC maximum time for ack, seconds. - #TAOS_RPC_MAX_TIME: "600" - # time interval of dnode status reporting to mnode, seconds, for cluster only #TAOS_STATUS_INTERVAL: "1" @@ -245,37 +195,7 @@ taoscfg: #TAOS_MIN_SLIDING_TIME: "10" # minimum time window, milli-second - #TAOS_MIN_INTERVAL_TIME: "10" - - # maximum delay before launching a stream computation, milli-second - #TAOS_MAX_STREAM_COMP_DELAY: "20000" - - # maximum delay before launching a stream computation for the first time, milli-second - #TAOS_MAX_FIRST_STREAM_COMP_DELAY: "10000" - - # retry delay when a stream computation fails, milli-second - #TAOS_RETRY_STREAM_COMP_DELAY: "10" - - # the delayed time for launching a stream computation, from 0.1(default, 10% of whole computing time window) to 0.9 - #TAOS_STREAM_COMP_DELAY_RATIO: "0.1" - - # max number of vgroups per db, 0 means configured automatically - #TAOS_MAX_VGROUPS_PER_DB: "0" - - # max number of tables per vnode - #TAOS_MAX_TABLES_PER_VNODE: "1000000" - - # the number of acknowledgments required for successful data writing - #TAOS_QUORUM: "1" - - # enable/disable compression - #TAOS_COMP: "2" - - # write ahead log (WAL) level, 0: no wal; 1: write wal, but no fysnc; 2: write wal, and call fsync - #TAOS_WAL_LEVEL: "1" - - # if walLevel is set to 2, the cycle of fsync being executed, if set to 0, fsync is called right away - #TAOS_FSYNC: "3000" + #TAOS_MIN_INTERVAL_TIME: "1" # the compressed rpc message, option: # -1 (no compression) @@ -283,17 +203,8 @@ taoscfg: # > 0 (rpc message body which larger than this value will be compressed) #TAOS_COMPRESS_MSG_SIZE: "-1" - # max length of an SQL - #TAOS_MAX_SQL_LENGTH: "1048576" - - # the maximum number of records allowed for super table time sorting - #TAOS_MAX_NUM_OF_ORDERED_RES: "100000" - # max number of connections allowed in dnode - #TAOS_MAX_SHELL_CONNS: "5000" - - # max number of connections allowed in client - #TAOS_MAX_CONNECTIONS: "5000" + #TAOS_MAX_SHELL_CONNS: "50000" # stop writing logs when the disk size of the log folder is less than this value #TAOS_MINIMAL_LOG_DIR_G_B: "0.1" @@ -313,21 +224,8 @@ taoscfg: # enable/disable system monitor #TAOS_MONITOR: "1" - # enable/disable recording the SQL statements via restful interface - #TAOS_HTTP_ENABLE_RECORD_SQL: "0" - - # number of threads used to process http requests - #TAOS_HTTP_MAX_THREADS: "2" - - # maximum number of rows returned by the restful interface - #TAOS_RESTFUL_ROW_LIMIT: "10240" - - # The following parameter is used to limit the maximum number of lines in log files. - # max number of lines per log filters - # numOfLogLines 10000000 - # enable/disable async log - #TAOS_ASYNC_LOG: "0" + #TAOS_ASYNC_LOG: "1" # # time of keeping log files, days @@ -344,25 +242,8 @@ taoscfg: # debug flag for all log type, take effect when non-zero value\ #TAOS_DEBUG_FLAG: "143" - # enable/disable recording the SQL in taos client - #TAOS_ENABLE_RECORD_SQL: "0" - # generate core file when service crash #TAOS_ENABLE_CORE_FILE: "1" - - # maximum display width of binary and nchar fields in the shell. The parts exceeding this limit will be hidden - #TAOS_MAX_BINARY_DISPLAY_WIDTH: "30" - - # enable/disable stream (continuous query) - #TAOS_STREAM: "1" - - # in retrieve blocking model, only in 50% query threads will be used in query processing in dnode - #TAOS_RETRIEVE_BLOCKING_MODEL: "0" - - # the maximum allowed query buffer size in MB during query processing for each data node - # -1 no limit (default) - # 0 no query allowed, queries are disabled - #TAOS_QUERY_BUFFER_SIZE: "-1" ``` ## Scaling Out diff --git a/docs/en/12-taos-sql/01-data-type.md b/docs/en/12-taos-sql/01-data-type.md index b830994ac9..876de50f35 100644 --- a/docs/en/12-taos-sql/01-data-type.md +++ b/docs/en/12-taos-sql/01-data-type.md @@ -11,7 +11,7 @@ When using TDengine to store and query data, the most important part of the data - The format must be `YYYY-MM-DD HH:mm:ss.MS`, the default time precision is millisecond (ms), for example `2017-08-12 18:25:58.128` - Internal function `now` can be used to get the current timestamp on the client side - The current timestamp of the client side is applied when `now` is used to insert data -- Epoch Time:timestamp can also be a long integer number, which means the number of seconds, milliseconds or nanoseconds, depending on the time precision, from 1970-01-01 00:00:00.000 (UTC/GMT) +- Epoch Time:timestamp can also be a long integer number, which means the number of seconds, milliseconds or nanoseconds, depending on the time precision, from UTC 1970-01-01 00:00:00. - Add/subtract operations can be carried out on timestamps. For example `now-2h` means 2 hours prior to the time at which query is executed. The units of time in operations can be b(nanosecond), u(microsecond), a(millisecond), s(second), m(minute), h(hour), d(day), or w(week). So `select * from t1 where ts > now-2w and ts <= now-1w` means the data between two weeks ago and one week ago. The time unit can also be n (calendar month) or y (calendar year) when specifying the time window for down sampling operations. Time precision in TDengine can be set by the `PRECISION` parameter when executing `CREATE DATABASE`. The default time precision is millisecond. In the statement below, the precision is set to nanonseconds. diff --git a/docs/en/12-taos-sql/03-table.md b/docs/en/12-taos-sql/03-table.md index bf32cf171b..5a2c8ed6ee 100644 --- a/docs/en/12-taos-sql/03-table.md +++ b/docs/en/12-taos-sql/03-table.md @@ -57,7 +57,7 @@ table_option: { 3. MAX_DELAY: specifies the maximum latency for pushing computation results. The default value is 15 minutes or the value of the INTERVAL parameter, whichever is smaller. Enter a value between 0 and 15 minutes in milliseconds, seconds, or minutes. You can enter multiple values separated by commas (,). Note: Retain the default value if possible. Configuring a small MAX_DELAY may cause results to be frequently pushed, affecting storage and query performance. This parameter applies only to supertables and takes effect only when the RETENTIONS parameter has been specified for the database. 4. ROLLUP: specifies aggregate functions to roll up. Rolling up a function provides downsampled results based on multiple axes. This parameter applies only to supertables and takes effect only when the RETENTIONS parameter has been specified for the database. You can specify only one function to roll up. The rollup takes effect on all columns except TS. Enter one of the following values: avg, sum, min, max, last, or first. 5. SMA: specifies functions on which to enable small materialized aggregates (SMA). SMA is user-defined precomputation of aggregates based on data blocks. Enter one of the following values: max, min, or sum This parameter can be used with supertables and standard tables. -6. TTL: specifies the time to live (TTL) for the table. If the period specified by the TTL parameter elapses without any data being written to the table, TDengine will automatically delete the table. Note: The system may not delete the table at the exact moment that the TTL expires. Enter a value in days. The default value is 0. Note: The TTL parameter has a higher priority than the KEEP parameter. If a table is marked for deletion because the TTL has expired, it will be deleted even if the time specified by the KEEP parameter has not elapsed. This parameter can be used with standard tables and subtables. +6. TTL: specifies the time to live (TTL) for the table. If TTL is specified when creatinga table, after the time period for which the table has been existing is over TTL, TDengine will automatically delete the table. Please be noted that the system may not delete the table at the exact moment that the TTL expires but guarantee there is such a system and finally the table will be deleted. The unit of TTL is in days. The default value is 0, i.e. never expire. ## Create Subtables diff --git a/docs/en/12-taos-sql/10-function.md b/docs/en/12-taos-sql/10-function.md index d35fd31099..d6905c84a1 100644 --- a/docs/en/12-taos-sql/10-function.md +++ b/docs/en/12-taos-sql/10-function.md @@ -1139,7 +1139,7 @@ SELECT STATECOUNT(field_name, oper, val) FROM { tb_name | stb_name } [WHERE clau **Applicable parameter values**: -- oper : Can be one of `LT` (lower than), `GT` (greater than), `LE` (lower than or equal to), `GE` (greater than or equal to), `NE` (not equal to), `EQ` (equal to), the value is case insensitive +- oper : Can be one of `'LT'` (lower than), `'GT'` (greater than), `'LE'` (lower than or equal to), `'GE'` (greater than or equal to), `'NE'` (not equal to), `'EQ'` (equal to), the value is case insensitive, the value must be in quotes. - val : Numeric types **Return value type**: Integer @@ -1166,7 +1166,7 @@ SELECT stateDuration(field_name, oper, val, unit) FROM { tb_name | stb_name } [W **Applicable parameter values**: -- oper : Can be one of `LT` (lower than), `GT` (greater than), `LE` (lower than or equal to), `GE` (greater than or equal to), `NE` (not equal to), `EQ` (equal to), the value is case insensitive +- oper : Can be one of `'LT'` (lower than), `'GT'` (greater than), `'LE'` (lower than or equal to), `'GE'` (greater than or equal to), `'NE'` (not equal to), `'EQ'` (equal to), the value is case insensitive, the value must be in quotes. - val : Numeric types - unit: The unit of time interval. Enter one of the following options: 1b (nanoseconds), 1u (microseconds), 1a (milliseconds), 1s (seconds), 1m (minutes), 1h (hours), 1d (days), or 1w (weeks) If you do not enter a unit of time, the precision of the current database is used by default. diff --git a/docs/en/12-taos-sql/19-limit.md b/docs/en/12-taos-sql/19-limit.md index 0486ea3094..678c38a22e 100644 --- a/docs/en/12-taos-sql/19-limit.md +++ b/docs/en/12-taos-sql/19-limit.md @@ -30,7 +30,7 @@ The following characters cannot occur in a password: single quotation marks ('), - Maximum number of columns is 4096. There must be at least 2 columns, and the first column must be timestamp. - The maximum length of a tag name is 64 bytes - Maximum number of tags is 128. There must be at least 1 tag. The total length of tag values cannot exceed 16 KB. -- Maximum length of single SQL statement is 1 MB (1048576 bytes). It can be configured in the parameter `maxSQLLength` in the client side, the applicable range is [65480, 1048576]. +- Maximum length of single SQL statement is 1 MB (1048576 bytes). - At most 4096 columns can be returned by `SELECT`. Functions in the query statement constitute columns. An error is returned if the limit is exceeded. - Maximum numbers of databases, STables, tables are dependent only on the system resources. - The number of replicas can only be 1 or 3. diff --git a/docs/en/12-taos-sql/24-show.md b/docs/en/12-taos-sql/24-show.md index 6b56161322..c9adb0cf78 100644 --- a/docs/en/12-taos-sql/24-show.md +++ b/docs/en/12-taos-sql/24-show.md @@ -194,7 +194,7 @@ Shows information about streams in the system. SHOW SUBSCRIPTIONS; ``` -Shows all subscriptions in the current database. +Shows all subscriptions in the system. ## SHOW TABLES diff --git a/docs/en/12-taos-sql/25-grant.md b/docs/en/12-taos-sql/25-grant.md index 37438ee780..b9a3fa2321 100644 --- a/docs/en/12-taos-sql/25-grant.md +++ b/docs/en/12-taos-sql/25-grant.md @@ -1,6 +1,7 @@ --- -sidebar_label: Permissions Management -title: Permissions Management +sidebar_label: Access Control +title: User and Access Control +description: Manage user and user's permission --- This document describes how to manage permissions in TDengine. diff --git a/docs/en/12-taos-sql/index.md b/docs/en/12-taos-sql/index.md index e243cd2318..a5ffc9dc8d 100644 --- a/docs/en/12-taos-sql/index.md +++ b/docs/en/12-taos-sql/index.md @@ -1,6 +1,6 @@ --- title: TDengine SQL -description: "The syntax supported by TDengine SQL " +description: 'The syntax supported by TDengine SQL ' --- This section explains the syntax of SQL to perform operations on databases, tables and STables, insert data, select data and use functions. We also provide some tips that can be used in TDengine SQL. If you have previous experience with SQL this section will be fairly easy to understand. If you do not have previous experience with SQL, you'll come to appreciate the simplicity and power of SQL. TDengine SQL has been enhanced in version 3.0, and the query engine has been rearchitected. For information about how TDengine SQL has changed, see [Changes in TDengine 3.0](../taos-sql/changes). @@ -15,7 +15,7 @@ Syntax Specifications used in this chapter: - | means one of a few options, excluding | itself. - … means the item prior to it can be repeated multiple times. -To better demonstrate the syntax, usage and rules of TAOS SQL, hereinafter it's assumed that there is a data set of data from electric meters. Each meter collects 3 data measurements: current, voltage, phase. The data model is shown below: +To better demonstrate the syntax, usage and rules of TDengine SQL, hereinafter it's assumed that there is a data set of data from electric meters. Each meter collects 3 data measurements: current, voltage, phase. The data model is shown below: ``` taos> DESCRIBE meters; diff --git a/docs/en/13-operation/01-pkg-install.md b/docs/en/13-operation/01-pkg-install.md index a8d8d7b474..d7713b943f 100644 --- a/docs/en/13-operation/01-pkg-install.md +++ b/docs/en/13-operation/01-pkg-install.md @@ -1,12 +1,12 @@ --- -title: Install & Uninstall +title: Install and Uninstall description: Install, Uninstall, Start, Stop and Upgrade --- import Tabs from "@theme/Tabs"; import TabItem from "@theme/TabItem"; -TDengine community version provides deb and rpm packages for users to choose from, based on their system environment. The deb package supports Debian, Ubuntu and derivative systems. The rpm package supports CentOS, RHEL, SUSE and derivative systems. Furthermore, a tar.gz package is provided for TDengine Enterprise customers. +This document gives more information about installing, uninstalling, and upgrading TDengine. ## Install @@ -15,11 +15,48 @@ About details of installing TDenine, please refer to [Installation Guide](../../ ## Uninstall + + +Apt-get package of TDengine can be uninstalled as below: + +```bash +$ sudo apt-get remove tdengine +Reading package lists... Done +Building dependency tree +Reading state information... Done +The following packages will be REMOVED: + tdengine +0 upgraded, 0 newly installed, 1 to remove and 18 not upgraded. +After this operation, 68.3 MB disk space will be freed. +Do you want to continue? [Y/n] y +(Reading database ... 135625 files and directories currently installed.) +Removing tdengine (3.0.0.0) ... +TDengine is removed successfully! + +``` + +Apt-get package of taosTools can be uninstalled as below: + +``` +$ sudo apt remove taostools +Reading package lists... Done +Building dependency tree +Reading state information... Done +The following packages will be REMOVED: + taostools +0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded. +After this operation, 68.3 MB disk space will be freed. +Do you want to continue? [Y/n] +(Reading database ... 147973 files and directories currently installed.) +Removing taostools (2.1.2) ... +``` + + Deb package of TDengine can be uninstalled as below: -```bash +``` $ sudo dpkg -r tdengine (Reading database ... 137504 files and directories currently installed.) Removing tdengine (3.0.0.0) ... @@ -27,6 +64,14 @@ TDengine is removed successfully! ``` +Deb package of taosTools can be uninstalled as below: + +``` +$ sudo dpkg -r taostools +(Reading database ... 147973 files and directories currently installed.) +Removing taostools (2.1.2) ... +``` + @@ -38,6 +83,13 @@ $ sudo rpm -e tdengine TDengine is removed successfully! ``` +RPM package of taosTools can be uninstalled as below: + +``` +sudo rpm -e taostools +taosToole is removed successfully! +``` + @@ -46,115 +98,69 @@ tar.gz package of TDengine can be uninstalled as below: ``` $ rmtaos -Nginx for TDengine is running, stopping it... TDengine is removed successfully! +``` -taosKeeper is removed successfully! +tar.gz package of taosTools can be uninstalled as below: + +``` +$ rmtaostools +Start to uninstall taos tools ... + +taos tools is uninstalled successfully! ``` + + +Run C:\TDengine\unins000.exe to uninstall TDengine on a Windows system. -:::note +:::info -- We strongly recommend not to use multiple kinds of installation packages on a single host TDengine. -- After deb package is installed, if the installation directory is removed manually, uninstall or reinstall will not work. This issue can be resolved by using the command below which cleans up TDengine package information. You can then reinstall if needed. +- We strongly recommend not to use multiple kinds of installation packages on a single host TDengine. The packages may affect each other and cause errors. -```bash - $ sudo rm -f /var/lib/dpkg/info/tdengine* -``` +- After deb package is installed, if the installation directory is removed manually, uninstall or reinstall will not work. This issue can be resolved by using the command below which cleans up TDengine package information. -- After rpm package is installed, if the installation directory is removed manually, uninstall or reinstall will not work. This issue can be resolved by using the command below which cleans up TDengine package information. You can then reinstall if needed. + ``` + $ sudo rm -f /var/lib/dpkg/info/tdengine* + ``` -```bash - $ sudo rpm -e --noscripts tdengine -``` +You can then reinstall if needed. + +- After rpm package is installed, if the installation directory is removed manually, uninstall or reinstall will not work. This issue can be resolved by using the command below which cleans up TDengine package information. + + ``` + $ sudo rpm -e --noscripts tdengine + ``` + +You can then reinstall if needed. ::: -## Installation Directory - -TDengine is installed at /usr/local/taos if successful. - -```bash -$ cd /usr/local/taos -$ ll -$ ll -total 28 -drwxr-xr-x 7 root root 4096 Feb 22 09:34 ./ -drwxr-xr-x 12 root root 4096 Feb 22 09:34 ../ -drwxr-xr-x 2 root root 4096 Feb 22 09:34 bin/ -drwxr-xr-x 2 root root 4096 Feb 22 09:34 cfg/ -lrwxrwxrwx 1 root root 13 Feb 22 09:34 data -> /var/lib/taos/ -drwxr-xr-x 2 root root 4096 Feb 22 09:34 driver/ -drwxr-xr-x 10 root root 4096 Feb 22 09:34 examples/ -drwxr-xr-x 2 root root 4096 Feb 22 09:34 include/ -lrwxrwxrwx 1 root root 13 Feb 22 09:34 log -> /var/log/taos/ -``` - -During the installation process: - -- Configuration directory, data directory, and log directory are created automatically if they don't exist -- The default configuration file is located at /etc/taos/taos.cfg, which is a copy of /usr/local/taos/cfg/taos.cfg -- The default data directory is /var/lib/taos, which is a soft link to /usr/local/taos/data -- The default log directory is /var/log/taos, which is a soft link to /usr/local/taos/log -- The executables at /usr/local/taos/bin are linked to /usr/bin -- The DLL files at /usr/local/taos/driver are linked to /usr/lib -- The header files at /usr/local/taos/include are linked to /usr/include - -:::note +Uninstalling and Modifying Files - When TDengine is uninstalled, the configuration /etc/taos/taos.cfg, data directory /var/lib/taos, log directory /var/log/taos are kept. They can be deleted manually with caution, because data can't be recovered. Please follow data integrity, security, backup or relevant SOPs before deleting any data. + - When reinstalling TDengine, if the default configuration file /etc/taos/taos.cfg exists, it will be kept and the configuration file in the installation package will be renamed to taos.cfg.orig and stored at /usr/local/taos/cfg to be used as configuration sample. Otherwise the configuration file in the installation package will be installed to /etc/taos/taos.cfg and used. -## Start and Stop - -Linux system services `systemd`, `systemctl` or `service` are used to start, stop and restart TDengine. The server process of TDengine is `taosd`, which is started automatically after the Linux system is started. System operators can use `systemd`, `systemctl` or `service` to start, stop or restart TDengine server. - -For example, if using `systemctl` , the commands to start, stop, restart and check TDengine server are below: - -- Start server:`systemctl start taosd` - -- Stop server:`systemctl stop taosd` - -- Restart server:`systemctl restart taosd` - -- Check server status:`systemctl status taosd` - -Another component named as `taosAdapter` is to provide HTTP service for TDengine, it should be started and stopped using `systemctl`. - -If the server process is OK, the output of `systemctl status` is like below: - -``` -Active: active (running) -``` - -Otherwise, the output is as below: - -``` -Active: inactive (dead) -``` ## Upgrade - There are two aspects in upgrade operation: upgrade installation package and upgrade a running server. To upgrade a package, follow the steps mentioned previously to first uninstall the old version then install the new version. Upgrading a running server is much more complex. First please check the version number of the old version and the new version. The version number of TDengine consists of 4 sections, only if the first 3 sections match can the old version be upgraded to the new version. The steps of upgrading a running server are as below: - - Stop inserting data - Make sure all data is persisted to disk -- Make some simple queries (Such as total rows in stables, tables and so on. Note down the values. Follow best practices and relevant SOPs.) - Stop the cluster of TDengine - Uninstall old version and install new version - Start the cluster of TDengine -- Execute simple queries, such as the ones executed prior to installing the new package, to make sure there is no data loss +- Execute simple queries, such as the ones executed prior to installing the new package, to make sure there is no data loss - Run some simple data insertion statements to make sure the cluster works well - Restore business services :::warning - TDengine doesn't guarantee any lower version is compatible with the data generated by a higher version, so it's never recommended to downgrade the version. ::: diff --git a/docs/en/13-operation/02-planning.mdx b/docs/en/13-operation/02-planning.mdx index c1baf92dbf..2dffa7bb87 100644 --- a/docs/en/13-operation/02-planning.mdx +++ b/docs/en/13-operation/02-planning.mdx @@ -1,40 +1,32 @@ --- +sidebar_label: Resource Planning title: Resource Planning --- It is important to plan computing and storage resources if using TDengine to build an IoT, time-series or Big Data platform. How to plan the CPU, memory and disk resources required, will be described in this chapter. -## Memory Requirement of Server Side +## Server Memory Requirements -By default, the number of vgroups created for each database is the same as the number of CPU cores. This can be configured by the parameter `maxVgroupsPerDb`. Each vnode in a vgroup stores one replica. Each vnode consumes a fixed amount of memory, i.e. `blocks` \* `cache`. In addition, some memory is required for tag values associated with each table. A fixed amount of memory is required for each cluster. So, the memory required for each DB can be calculated using the formula below: +Each database creates a fixed number of vgroups. This number is 2 by default and can be configured with the `vgroups` parameter. The number of replicas can be controlled with the `replica` parameter. Each replica requires one vnode per vgroup. Altogether, the memory required by each database depends on the following configuration options: + +- vgroups +- replica +- buffer +- pages +- pagesize +- cachesize + +For more information, see [Database](../../taos-sql/database). + +The memory required by a database is therefore greater than or equal to: ``` -Database Memory Size = maxVgroupsPerDb * replica * (blocks * cache + 10MB) + numOfTables * (tagSizePerTable + 0.5KB) +vgroups * replica * (buffer + pages * pagesize + cachesize) ``` -For example, assuming the default value of `maxVgroupPerDB` is 64, the default value of `cache` is 16M, the default value of `blocks` is 6, there are 100,000 tables in a DB, the replica number is 1, total length of tag values is 256 bytes, the total memory required for this DB is: 64 \* 1 \* (16 \* 6 + 10) + 100000 \* (0.25 + 0.5) / 1000 = 6792M. +However, note that this requirement is spread over all dnodes in the cluster, not on a single physical machine. The physical servers that run dnodes meet the requirement together. If a cluster has multiple databases, the memory required increases accordingly. In complex environments where dnodes were added after initial deployment in response to increasing resource requirements, load may not be balanced among the original dnodes and newer dnodes. In this situation, the actual status of your dnodes is more important than theoretical calculations. -In the real operation of TDengine, we are more concerned about the memory used by each TDengine server process `taosd`. - -``` - taosd_memory = vnode_memory + mnode_memory + query_memory -``` - -In the above formula: - -1. "vnode_memory" of a `taosd` process is the memory used by all vnodes hosted by this `taosd` process. It can be roughly calculated by firstly adding up the total memory of all DBs whose memory usage can be derived according to the formula for Database Memory Size, mentioned above, then dividing by number of dnodes and multiplying the number of replicas. - -``` - vnode_memory = (sum(Database Memory Size) / number_of_dnodes) * replica -``` - -2. "mnode_memory" of a `taosd` process is the memory consumed by a mnode. If there is one (and only one) mnode hosted in a `taosd` process, the memory consumed by "mnode" is "0.2KB \* the total number of tables in the cluster". - -3. "query_memory" is the memory used when processing query requests. Each ongoing query consumes at least "0.2 KB \* total number of involved tables". - -Please note that the above formulas can only be used to estimate the minimum memory requirement, instead of maximum memory usage. In a real production environment, it's better to reserve some redundance beyond the estimated minimum memory requirement. If memory is abundant, it's suggested to increase the value of parameter `blocks` to speed up data insertion and data query. - -## Memory Requirement of Client Side +## Client Memory Requirements For the client programs using TDengine client driver `taosc` to connect to the server side there is a memory requirement as well. @@ -56,10 +48,10 @@ So, at least 3GB needs to be reserved for such a client. The CPU resources required depend on two aspects: -- **Data Insertion** Each dnode of TDengine can process at least 10,000 insertion requests in one second, while each insertion request can have multiple rows. The difference in computing resource consumed, between inserting 1 row at a time, and inserting 10 rows at a time is very small. So, the more the number of rows that can be inserted one time, the higher the efficiency. Inserting in batch also imposes requirements on the client side which needs to cache rows to insert in batch once the number of cached rows reaches a threshold. +- **Data Insertion** Each dnode of TDengine can process at least 10,000 insertion requests in one second, while each insertion request can have multiple rows. The difference in computing resource consumed, between inserting 1 row at a time, and inserting 10 rows at a time is very small. So, the more the number of rows that can be inserted one time, the higher the efficiency. If each insert request contains more than 200 records, a single core can process more than 1 million records per second. Inserting in batch also imposes requirements on the client side which needs to cache rows to insert in batch once the number of cached rows reaches a threshold. - **Data Query** High efficiency query is provided in TDengine, but it's hard to estimate the CPU resource required because the queries used in different use cases and the frequency of queries vary significantly. It can only be verified with the query statements, query frequency, data size to be queried, and other requirements provided by users. -In short, the CPU resource required for data insertion can be estimated but it's hard to do so for query use cases. In real operation, it's suggested to control CPU usage below 50%. If this threshold is exceeded, it's a reminder for system operator to add more nodes in the cluster to expand resources. +In short, the CPU resource required for data insertion can be estimated but it's hard to do so for query use cases. If possible, ensure that CPU usage remains below 50%. If this threshold is exceeded, it's a reminder for system operator to add more nodes in the cluster to expand resources. ## Disk Requirement @@ -77,6 +69,6 @@ To increase performance, multiple disks can be setup for parallel data reading o ## Number of Hosts -A host can be either physical or virtual. The total memory, total CPU, total disk required can be estimated according to the formulae mentioned previously. Then, according to the system resources that a single host can provide, assuming all hosts have the same resources, the number of hosts can be derived easily. +A host can be either physical or virtual. The total memory, total CPU, total disk required can be estimated according to the formulae mentioned previously. If the number of data replicas is not 1, the required resources are multiplied by the number of replicas. -**Quick Estimation for CPU, Memory and Disk** Please refer to [Resource Estimate](https://www.taosdata.com/config/config.html). +Then, according to the system resources that a single host can provide, assuming all hosts have the same resources, the number of hosts can be derived easily. diff --git a/docs/en/13-operation/03-tolerance.md b/docs/en/13-operation/03-tolerance.md index d4d48d7fcd..21a5a90282 100644 --- a/docs/en/13-operation/03-tolerance.md +++ b/docs/en/13-operation/03-tolerance.md @@ -1,6 +1,5 @@ --- -sidebar_label: Fault Tolerance -title: Fault Tolerance & Disaster Recovery +title: Fault Tolerance and Disaster Recovery --- ## Fault Tolerance @@ -11,22 +10,21 @@ When a data block is received by TDengine, the original data block is first writ There are 2 configuration parameters related to WAL: -- walLevel: - - 0:wal is disabled - - 1:wal is enabled without fsync - - 2:wal is enabled with fsync -- fsync:This parameter is only valid when walLevel is set to 2. It specifies the interval, in milliseconds, of invoking fsync. If set to 0, it means fsync is invoked immediately once WAL is written. +- wal_level: Specifies the WAL level. 1 indicates that WAL is enabled but fsync is disabled. 2 indicates that WAL and fsync are both enabled. The default value is 1. +- wal_fsync_period: This parameter is only valid when wal_level is set to 2. It specifies the interval, in milliseconds, of invoking fsync. If set to 0, it means fsync is invoked immediately once WAL is written. -To achieve absolutely no data loss, walLevel should be set to 2 and fsync should be set to 1. There is a performance penalty to the data ingestion rate. However, if the concurrent data insertion threads on the client side can reach a big enough number, for example 50, the data ingestion performance will be still good enough. Our verification shows that the drop is only 30% when fsync is set to 3,000 milliseconds. +To achieve absolutely no data loss, set wal_level to 2 and wal_fsync_period to 0. There is a performance penalty to the data ingestion rate. However, if the concurrent data insertion threads on the client side can reach a big enough number, for example 50, the data ingestion performance will be still good enough. Our verification shows that the drop is only 30% when wal_fsync_period is set to 3000 milliseconds. ## Disaster Recovery -TDengine uses replication to provide high availability and disaster recovery capability. +TDengine uses replication to provide high availability. -A TDengine cluster is managed by mnode. To ensure the high availability of mnode, multiple replicas can be configured by the system parameter `numOfMnodes`. The data replication between mnode replicas is performed in a synchronous way to guarantee metadata consistency. +A TDengine cluster is managed by mnodes. You can configure up to three mnodes to ensure high availability. The data replication between mnode replicas is performed in a synchronous way to guarantee metadata consistency. -The number of replicas for time series data in TDengine is associated with each database. There can be many databases in a cluster and each database can be configured with a different number of replicas. When creating a database, parameter `replica` is used to configure the number of replications. To achieve high availability, `replica` needs to be higher than 1. +The number of replicas for time series data in TDengine is associated with each database. There can be many databases in a cluster and each database can be configured with a different number of replicas. When creating a database, the parameter `replica` is used to specify the number of replicas. To achieve high availability, set `replica` to 3. The number of dnodes in a TDengine cluster must NOT be lower than the number of replicas for any database, otherwise it would fail when trying to create a table. As long as the dnodes of a TDengine cluster are deployed on different physical machines and the replica number is higher than 1, high availability can be achieved without any other assistance. For disaster recovery, dnodes of a TDengine cluster should be deployed in geographically different data centers. + +Alternatively, you can use taosX to synchronize the data from one TDengine cluster to another cluster in a remote location. However, taosX is only available in TDengine enterprise version, for more information please contact tdengine.com. diff --git a/docs/en/13-operation/17-diagnose.md b/docs/en/13-operation/17-diagnose.md index 2b474fddba..d01d12e831 100644 --- a/docs/en/13-operation/17-diagnose.md +++ b/docs/en/13-operation/17-diagnose.md @@ -13,110 +13,59 @@ Diagnostic steps: 1. If the port range to be diagnosed is being occupied by a `taosd` server process, please first stop `taosd. 2. On the server side, execute command `taos -n server -P -l ` to monitor the port range starting from the port specified by `-P` parameter with the role of "server". 3. On the client side, execute command `taos -n client -h -P -l ` to send a testing package to the specified server and port. - --l : The size of the testing package, in bytes. The value range is [11, 64,000] and default value is 1,000. Please note that the package length must be same in the above 2 commands executed on server side and client side respectively. + +-l : The size of the testing package, in bytes. The value range is [11, 64,000] and default value is 1,000. +Please note that the package length must be same in the above 2 commands executed on server side and client side respectively. Output of the server side for the example is below: ```bash -# taos -n server -P 6000 -12/21 14:50:13.522509 0x7f536f455200 UTL work as server, host:172.27.0.7 startPort:6000 endPort:6011 pkgLen:1000 - -12/21 14:50:13.522659 0x7f5352242700 UTL TCP server at port:6000 is listening -12/21 14:50:13.522727 0x7f5351240700 UTL TCP server at port:6001 is listening +# taos -n server -P 6030 -l 1000 +network test server is initialized, port:6030 +request is received, size:1000 +request is received, size:1000 ... ... ... -12/21 14:50:13.523954 0x7f5342fed700 UTL TCP server at port:6011 is listening -12/21 14:50:13.523989 0x7f53437ee700 UTL UDP server at port:6010 is listening -12/21 14:50:13.524019 0x7f53427ec700 UTL UDP server at port:6011 is listening -12/21 14:50:22.192849 0x7f5352242700 UTL TCP: read:1000 bytes from 172.27.0.8 at 6000 -12/21 14:50:22.192993 0x7f5352242700 UTL TCP: write:1000 bytes to 172.27.0.8 at 6000 -12/21 14:50:22.237082 0x7f5351a41700 UTL UDP: recv:1000 bytes from 172.27.0.8 at 6000 -12/21 14:50:22.237203 0x7f5351a41700 UTL UDP: send:1000 bytes to 172.27.0.8 at 6000 -12/21 14:50:22.237450 0x7f5351240700 UTL TCP: read:1000 bytes from 172.27.0.8 at 6001 -12/21 14:50:22.237576 0x7f5351240700 UTL TCP: write:1000 bytes to 172.27.0.8 at 6001 -12/21 14:50:22.281038 0x7f5350a3f700 UTL UDP: recv:1000 bytes from 172.27.0.8 at 6001 -12/21 14:50:22.281141 0x7f5350a3f700 UTL UDP: send:1000 bytes to 172.27.0.8 at 6001 -... -... -... -12/21 14:50:22.677443 0x7f5342fed700 UTL TCP: read:1000 bytes from 172.27.0.8 at 6011 -12/21 14:50:22.677576 0x7f5342fed700 UTL TCP: write:1000 bytes to 172.27.0.8 at 6011 -12/21 14:50:22.721144 0x7f53427ec700 UTL UDP: recv:1000 bytes from 172.27.0.8 at 6011 -12/21 14:50:22.721261 0x7f53427ec700 UTL UDP: send:1000 bytes to 172.27.0.8 at 6011 +request is received, size:1000 +request is received, size:1000 ``` Output of the client side for the example is below: ```bash # taos -n client -h 172.27.0.7 -P 6000 -12/21 14:50:22.192434 0x7fc95d859200 UTL work as client, host:172.27.0.7 startPort:6000 endPort:6011 pkgLen:1000 +taos -n client -h v3s2 -P 6030 -l 1000 +network test client is initialized, the server is v3s2:6030 +request is sent, size:1000 +response is received, size:1000 +request is sent, size:1000 +response is received, size:1000 +... +... +... +request is sent, size:1000 +response is received, size:1000 +request is sent, size:1000 +response is received, size:1000 -12/21 14:50:22.192472 0x7fc95d859200 UTL server ip:172.27.0.7 is resolved from host:172.27.0.7 -12/21 14:50:22.236869 0x7fc95d859200 UTL successed to test TCP port:6000 -12/21 14:50:22.237215 0x7fc95d859200 UTL successed to test UDP port:6000 -... -... -... -12/21 14:50:22.676891 0x7fc95d859200 UTL successed to test TCP port:6010 -12/21 14:50:22.677240 0x7fc95d859200 UTL successed to test UDP port:6010 -12/21 14:50:22.720893 0x7fc95d859200 UTL successed to test TCP port:6011 -12/21 14:50:22.721274 0x7fc95d859200 UTL successed to test UDP port:6011 +total succ: 100/100 cost: 16.23 ms speed: 5.87 MB/s ``` The output needs to be checked carefully for the system operator to find the root cause and resolve the problem. -## Startup Status and RPC Diagnostic - -`taos -n startup -h ` can be used to check the startup status of a `taosd` process. This is a common task which should be performed by a system operator, especially in the case of a cluster, to determine whether `taosd` has been started successfully. - -`taos -n rpc -h ` can be used to check whether the port of a started `taosd` can be accessed or not. If `taosd` process doesn't respond or is working abnormally, this command can be used to initiate a rpc communication with the specified fqdn to determine whether it's a network problem or whether `taosd` is abnormal. - -## Sync and Arbitrator Diagnostic - -```bash -taos -n sync -P 6040 -h -taos -n sync -P 6042 -h -``` - -The above commands can be executed in a Linux shell to check whether the port for sync is working well and whether the sync module on the server side is working well. Additionally, `-P 6042` is used to check whether the arbitrator is configured properly and is working well. - -## Network Speed Diagnostic - -`taos -n speed -h -P 6030 -N 10 -l 10000000 -S TCP` - -From version 2.2.0.0 onwards, the above command can be executed in a Linux shell to test network speed. The command sends uncompressed packages to a running `taosd` server process or a simulated server process started by `taos -n server` to test the network speed. Parameters can be used when testing network speed are as below: - --n:When set to "speed", it means testing network speed. --h:The FQDN or IP of the server process to be connected to; if not set, the FQDN configured in `taos.cfg` is used. --P:The port of the server process to connect to, the default value is 6030. --N:The number of packages that will be sent in the test, range is [1,10000], default value is 100. --l:The size of each package in bytes, range is [1024, 1024 \* 1024 \* 1024], default value is 1024. --S:The type of network packages to send, can be either TCP or UDP, default value is TCP. - -## FQDN Resolution Diagnostic - -`taos -n fqdn -h ` - -From version 2.2.0.0 onward, the above command can be executed in a Linux shell to test the resolution speed of FQDN. It can be used to try to resolve a FQDN to an IP address and record the time spent in this process. The parameters that can be used for this purpose are as below: - --n:When set to "fqdn", it means testing the speed of resolving FQDN. --h:The FQDN to be resolved. If not set, the `FQDN` parameter in `taos.cfg` is used by default. - ## Server Log -The parameter `debugFlag` is used to control the log level of the `taosd` server process. The default value is 131. For debugging and tracing, it needs to be set to either 135 or 143 respectively. +The parameter `debugFlag` is used to control the log level of the `taosd` server process. The default value is 131. For debugging and tracing, it needs to be set to either 135 or 143 respectively. -Once this parameter is set to 135 or 143, the log file grows very quickly especially when there is a huge volume of data insertion and data query requests. If all the logs are stored together, some important information may be missed very easily and so on the server side, important information is stored in a different place from other logs. - -- The log at level of INFO, WARNING and ERROR is stored in `taosinfo` so that it is easy to find important information -- The log at level of DEBUG (135) and TRACE (143) and other information not handled by `taosinfo` are stored in `taosdlog` +Once this parameter is set to 135 or 143, the log file grows very quickly especially when there is a huge volume of data insertion and data query requests. Ensure that the disk drive on which logs are stored has sufficient space. ## Client Log -An independent log file, named as "taoslog+" is generated for each client program, i.e. a client process. The default value of `debugFlag` is also 131 and only logs at level of INFO/ERROR/WARNING are recorded. As stated above, for debugging and tracing, it needs to be changed to 135 or 143 respectively, so that logs at DEBUG or TRACE level can be recorded. +An independent log file, named as "taoslog+" is generated for each client program, i.e. a client process. The parameter `debugFlag` is used to control the log level. The default value is 131. For debugging and tracing, it needs to be set to either 135 or 143 respectively. + +The default value of `debugFlag` is also 131 and only logs at level of INFO/ERROR/WARNING are recorded. As stated above, for debugging and tracing, it needs to be changed to 135 or 143 respectively, so that logs at DEBUG or TRACE level can be recorded. The maximum length of a single log file is controlled by parameter `numOfLogLines` and only 2 log files are kept for each `taosd` server process. -Log files are written in an async way to minimize the workload on disk, but the trade off for performance is that a few log lines may be lost in some extreme conditions. +Log files are written in an async way to minimize the workload on disk, but the trade off for performance is that a few log lines may be lost in some extreme conditions. You can configure asynclog to 0 when needed for troubleshooting purposes to ensure that no log information is lost. diff --git a/docs/en/14-reference/02-rest-api/02-rest-api.mdx b/docs/en/14-reference/02-rest-api/02-rest-api.mdx index 74ba78b7fc..ce28ee87d9 100644 --- a/docs/en/14-reference/02-rest-api/02-rest-api.mdx +++ b/docs/en/14-reference/02-rest-api/02-rest-api.mdx @@ -18,12 +18,12 @@ If the TDengine server is already installed, it can be verified as follows: The following example is in an Ubuntu environment and uses the `curl` tool to verify that the REST interface is working. Note that the `curl` tool may need to be installed in your environment. -The following example lists all databases on the host h1.taosdata.com. To use it in your environment, replace `h1.taosdata.com` and `6041` (the default port) with the actual running TDengine service FQDN and port number. +The following example lists all databases on the host h1.tdengine.com. To use it in your environment, replace `h1.tdengine.com` and `6041` (the default port) with the actual running TDengine service FQDN and port number. ```bash curl -L -H "Authorization: Basic cm9vdDp0YW9zZGF0YQ==" \ -d "select name, ntables, status from information_schema.ins_databases;" \ - h1.taosdata.com:6041/rest/sql + h1.tdengine.com:6041/rest/sql ``` The following return value results indicate that the verification passed. diff --git a/docs/en/14-reference/03-connector/04-java.mdx b/docs/en/14-reference/03-connector/04-java.mdx index 0f977393f1..129d90ea85 100644 --- a/docs/en/14-reference/03-connector/04-java.mdx +++ b/docs/en/14-reference/03-connector/04-java.mdx @@ -133,8 +133,6 @@ The configuration parameters in the URL are as follows: - batchfetch: true: pulls result sets in batches when executing queries; false: pulls result sets row by row. The default value is true. Enabling batch pulling and obtaining a batch of data can improve query performance when the query data volume is large. - batchErrorIgnore:true: When executing statement executeBatch, if there is a SQL execution failure in the middle, the following SQL will continue to be executed. false: No more statements after the failed SQL are executed. The default value is: false. -For more information about JDBC native connections, see [Video Tutorial](https://www.taosdata.com/blog/2020/11/11/1955.html). - **Connect using the TDengine client-driven configuration file ** When you use a JDBC native connection to connect to a TDengine cluster, you can use the TDengine client driver configuration file to specify parameters such as `firstEp` and `secondEp` of the cluster in the configuration file as below: diff --git a/docs/en/14-reference/03-connector/09-csharp.mdx b/docs/en/14-reference/03-connector/09-csharp.mdx index c745b8dd1a..823e907599 100644 --- a/docs/en/14-reference/03-connector/09-csharp.mdx +++ b/docs/en/14-reference/03-connector/09-csharp.mdx @@ -172,7 +172,6 @@ namespace TDengineExample `Taos` is an ADO.NET connector for TDengine, supporting Linux and Windows platforms. Community contributor `Maikebing@@maikebing contributes the connector`. Please refer to: * Interface download: -* Usage notes: ## Frequently Asked Questions diff --git a/docs/en/14-reference/03-connector/_preparation.mdx b/docs/en/14-reference/03-connector/_preparation.mdx index 07ebdbca3d..c6e42ce023 100644 --- a/docs/en/14-reference/03-connector/_preparation.mdx +++ b/docs/en/14-reference/03-connector/_preparation.mdx @@ -2,7 +2,7 @@ :::info -Since the TDengine client driver is written in C, using the native connection requires loading the client driver shared library file, which is usually included in the TDengine installer. You can install either standard TDengine server installation package or [TDengine client installation package](/get-started/). For Windows development, you need to install the corresponding [Windows client](https://www.taosdata.com/cn/all-downloads/#TDengine-Windows-Client) for TDengine. +Since the TDengine client driver is written in C, using the native connection requires loading the client driver shared library file, which is usually included in the TDengine installer. You can install either standard TDengine server installation package or [TDengine client installation package](/get-started/). For Windows development, you need to install the corresponding Windows client, please refer to [Install TDengine](../../get-started/package). - libtaos.so: After successful installation of TDengine on a Linux system, the dependent Linux version of the client driver `libtaos.so` file will be automatically linked to `/usr/lib/libtaos.so`, which is included in the Linux scannable path and does not need to be specified separately. - taos.dll: After installing the client on Windows, the dependent Windows version of the client driver taos.dll file will be automatically copied to the system default search path C:/Windows/System32, again without the need to specify it separately. diff --git a/docs/en/14-reference/06-taosdump.md b/docs/en/14-reference/06-taosdump.md index 2105ba83fa..e73441a96b 100644 --- a/docs/en/14-reference/06-taosdump.md +++ b/docs/en/14-reference/06-taosdump.md @@ -116,5 +116,4 @@ Usage: taosdump [OPTION...] dbname [tbname ...] Mandatory or optional arguments to long options are also mandatory or optional for any corresponding short options. -Report bugs to . ``` diff --git a/docs/en/14-reference/07-tdinsight/index.md b/docs/en/14-reference/07-tdinsight/index.md index e74c9de7b2..2e56203525 100644 --- a/docs/en/14-reference/07-tdinsight/index.md +++ b/docs/en/14-reference/07-tdinsight/index.md @@ -263,7 +263,7 @@ Once the import is complete, the full page view of TDinsight is shown below. ## TDinsight dashboard details -The TDinsight dashboard is designed to provide the usage and status of TDengine-related resources [dnodes, mnodes, vnodes](https://www.taosdata.com/cn/documentation/architecture#cluster) or databases. +The TDinsight dashboard is designed to provide the usage and status of TDengine-related resources [dnodes, mnodes, vnodes](../../taos-sql/node/) or databases. Details of the metrics are as follows. diff --git a/docs/en/14-reference/11-docker/index.md b/docs/en/14-reference/11-docker/index.md index b3c3cddd9a..7cd1e810dc 100644 --- a/docs/en/14-reference/11-docker/index.md +++ b/docs/en/14-reference/11-docker/index.md @@ -72,7 +72,7 @@ Next, ensure the hostname "tdengine" is resolvable in `/etc/hosts`. echo 127.0.0.1 tdengine |sudo tee -a /etc/hosts ``` -Finally, the TDengine service can be accessed from the taos shell or any connector with "tdengine" as the server address. +Finally, the TDengine service can be accessed from the TDengine CLI or any connector with "tdengine" as the server address. ```shell taos -h tdengine -P 6030 @@ -116,7 +116,7 @@ If you want to start your application in a container, you need to add the corres FROM ubuntu:20.04 RUN apt-get update && apt-get install -y wget ENV TDENGINE_VERSION=3.0.0.0 -RUN wget -c https://www.taosdata.com/assets-download/TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ +RUN wget -c https://www.taosdata.com/assets-download/3.0/TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ && tar xvf TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ && cd TDengine-client-${TDENGINE_VERSION} \ && ./install_client.sh \ @@ -217,7 +217,7 @@ Here is the full Dockerfile: ```docker FROM golang:1.17.6-buster as builder ENV TDENGINE_VERSION=3.0.0.0 -RUN wget -c https://www.taosdata.com/assets-download/TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ +RUN wget -c https://www.taosdata.com/assets-download/3.0/TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ && tar xvf TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ && cd TDengine-client-${TDENGINE_VERSION} \ && ./install_client.sh \ @@ -233,7 +233,7 @@ RUN go build FROM ubuntu:20.04 RUN apt-get update && apt-get install -y wget ENV TDENGINE_VERSION=3.0.0.0 -RUN wget -c https://www.taosdata.com/assets-download/TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ +RUN wget -c https://www.taosdata.com/assets-download/3.0/TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ && tar xvf TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ && cd TDengine-client-${TDENGINE_VERSION} \ && ./install_client.sh \ diff --git a/docs/en/14-reference/12-config/index.md b/docs/en/14-reference/12-config/index.md index cb7daf3c47..02921c3f6a 100644 --- a/docs/en/14-reference/12-config/index.md +++ b/docs/en/14-reference/12-config/index.md @@ -1,16 +1,13 @@ --- -sidebar_label: Configuration title: Configuration Parameters description: "Configuration parameters for client and server in TDengine" --- -In this chapter, all the configuration parameters on both server and client side are described thoroughly. - ## Configuration File on Server Side On the server side, the actual service of TDengine is provided by an executable `taosd` whose parameters can be configured in file `taos.cfg` to meet the requirements of different use cases. The default location of `taos.cfg` is `/etc/taos`, but can be changed by using `-c` parameter on the CLI of `taosd`. For example, the configuration file can be put under `/home/user` and used like below -```bash +``` taosd -c /home/user ``` @@ -24,8 +21,6 @@ taosd -C TDengine CLI `taos` is the tool for users to interact with TDengine. It can share same configuration file as `taosd` or use a separate configuration file. When launching `taos`, parameter `-c` can be used to specify the location where its configuration file is. For example `taos -c /home/cfg` means `/home/cfg/taos.cfg` will be used. If `-c` is not used, the default location of the configuration file is `/etc/taos`. For more details please use `taos --help` to get. -From version 2.0.10.0 below commands can be used to show the configuration parameters of the client side. - ```bash taos -C ``` @@ -36,6 +31,11 @@ taos --dump-config # Configuration Parameters +:::note +The parameters described in this document by the effect that they have on the system. + +::: + :::note `taosd` needs to be restarted for the parameters changed in the configuration file to take effect. @@ -45,19 +45,19 @@ taos --dump-config ### firstEp -| Attribute | Description | -| ------------- | ---------------------------------------------------------------------------------------------------- | -| Applicable | Server and Client | +| Attribute | Description | +| -------- | -------------------------------------------------------------- | +| Applicable | Server and Client | | Meaning | The end point of the first dnode in the cluster to be connected to when `taosd` or `taos` is started | -| Default Value | localhost:6030 | +| Default | localhost:6030 | ### secondEp -| Attribute | Description | -| ------------- | ---------------------------------------------------------------------------------------------------------------------- | -| Applicable | Server and Client | +| Attribute | Description | +| -------- | ------------------------------------------------------------------------------------- | +| Applicable | Server and Client | | Meaning | The end point of the second dnode to be connected to if the firstEp is not available when `taosd` or `taos` is started | -| Default Value | None | +| Default | None | ### fqdn @@ -65,35 +65,28 @@ taos --dump-config | ------------- | ------------------------------------------------------------------------ | | Applicable | Server Only | | Meaning | The FQDN of the host where `taosd` will be started. It can be IP address | -| Default Value | The first hostname configured for the host | -| Note | It should be within 96 bytes | +| Default Value | The first hostname configured for the host | +| Note | It should be within 96 bytes | | ### serverPort -| Attribute | Description | -| ------------- | ------------------------------------------------------------------------------------------------------------------------------- | -| Applicable | Server Only | +| Attribute | Description | +| -------- | ----------------------------------------------------------------------------------------------------------------------- | +| Applicable | Server Only | | Meaning | The port for external access after `taosd` is started | | Default Value | 6030 | :::note -TDengine uses 13 continuous ports, both TCP and UDP, starting with the port specified by `serverPort`. You should ensure, in your firewall rules, that these ports are kept open. Below table describes the ports used by TDengine in details. - +- Ensure that your firewall rules do not block TCP port 6042 on any host in the cluster. Below table describes the ports used by TDengine in details. ::: - | Protocol | Default Port | Description | How to configure | | :------- | :----------- | :----------------------------------------------- | :--------------------------------------------------------------------------------------------- | -| TCP | 6030 | Communication between client and server | serverPort | -| TCP | 6035 | Communication among server nodes in cluster | serverPort+5 | -| TCP | 6040 | Data syncup among server nodes in cluster | serverPort+10 | -| TCP | 6041 | REST connection between client and server | Please refer to [taosAdapter](../taosadapter/) | -| TCP | 6042 | Service Port of Arbitrator | The parameter of Arbitrator | -| TCP | 6043 | Service Port of TaosKeeper | The parameter of TaosKeeper | -| TCP | 6044 | Data access port for StatsD | refer to [taosAdapter](../taosadapter/) | -| UDP | 6045 | Data access for statsd | refer to [taosAdapter](../taosadapter/) | -| TCP | 6060 | Port of Monitoring Service in Enterprise version | | -| UDP | 6030-6034 | Communication between client and server | serverPort | -| UDP | 6035-6039 | Communication among server nodes in cluster | serverPort | +| TCP | 6030 | Communication between client and server. In a multi-node cluster, communication between nodes. serverPort | +| TCP | 6041 | REST connection between client and server | Prior to 2.4.0.0: serverPort+11; After 2.4.0.0 refer to [taosAdapter](/reference/taosadapter/) | +| TCP | 6043 | Service Port of TaosKeeper | The parameter of TaosKeeper | +| TCP | 6044 | Data access port for StatsD | Configurable through taosAdapter parameters. +| UDP | 6045 | Data access for statsd | Configurable through taosAdapter parameters. +| TCP | 6060 | Port of Monitoring Service in Enterprise version | | ### maxShellConns @@ -104,104 +97,109 @@ TDengine uses 13 continuous ports, both TCP and UDP, starting with the port spec | Value Range | 10-50000000 | | Default Value | 5000 | -### maxConnections - -| Attribute | Description | -| ------------- | ----------------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | The maximum number of connections allowed by a database | -| Value Range | 1-100000 | -| Default Value | 5000 | -| Note | The maximum number of worker threads on the client side is maxConnections/100 | - -### rpcForceTcp - -| Attribute | Description | -| ------------- | ------------------------------------------------------------------- | -| Applicable | Server and Client | -| Meaning | TCP is used by force | -| Value Range | 0: disabled 1: enabled | -| Default Value | 0 | -| Note | It's suggested to configure to enable if network is not good enough | - ## Monitoring Parameters ### monitor -| Attribute | Description | -| ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | The switch for monitoring inside server. The workload of the hosts, including CPU, memory, disk, network, TTP requests, are collected and stored in a system builtin database `LOG` | -| Value Range | 0: monitoring disabled, 1: monitoring enabled | -| Default Value | 1 | +| Attribute | Description | +| -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Applicable | Server only | +| Meaning | The switch for monitoring inside server. The main object of monitoring is to collect information about load on physical nodes, including CPU usage, memory usage, disk usage, and network bandwidth. Monitoring information is sent over HTTP to the taosKeeper service specified by `monitorFqdn` and `monitorProt`. +| Value Range | 0: monitoring disabled, 1: monitoring enabled | +| Default | 1 | + +### monitorFqdn + +| Attribute | Description | +| -------- | -------------------------- | +| Applicable | Server Only | +| Meaning | FQDN of taosKeeper monitoring service | +| Default | None | + +### monitorPort + +| Attribute | Description | +| -------- | --------------------------- | +| Applicable | Server Only | +| Meaning | Port of taosKeeper monitoring service | +| Default Value | 6043 | ### monitorInterval -| Attribute | Description | -| ------------- | ------------------------------------------ | -| Applicable | Server Only | +| Attribute | Description | +| -------- | -------------------------------------------- | +| Applicable | Server Only | | Meaning | The interval of collecting system workload | | Unit | second | -| Value Range | 1-600 | -| Default Value | 30 | +| Value Range | 1-200000 | +| Default Value | 30 | ### telemetryReporting -| Attribute | Description | -| ------------- | ---------------------------------------------------------------------------- | -| Applicable | Server Only | +| Attribute | Description | +| -------- | ---------------------------------------- | +| Applicable | Server Only | | Meaning | Switch for allowing TDengine to collect and report service usage information | | Value Range | 0: Not allowed; 1: Allowed | -| Default Value | 1 | +| Default Value | 1 | ## Query Parameters -### queryBufferSize +### queryPolicy -| Attribute | Description | -| ------------- | ---------------------------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | The total memory size reserved for all queries | -| Unit | MB | -| Default Value | None | -| Note | It can be estimated by "maximum number of concurrent queries" _ "number of tables" _ 170 | +| Attribute | Description | +| -------- | ----------------------------- | +| Applicable | Client only | +| Meaning | Execution policy for query statements | +| Unit | None | +| Default | 1 | +| Notes | 1: Run queries on vnodes and not on qnodes | -### ratioOfQueryCores +2: Run subtasks without scan operators on qnodes and subtasks with scan operators on vnodes. + +3: Only run scan operators on vnodes; run all other operators on qnodes. + +### querySmaOptimize + +| Attribute | Description | +| -------- | -------------------- | +| Applicable | Client only | +| 含义 | SMA index optimization policy | +| Unit | None | +| Default Value | 0 | +| Notes | + +0: Disable SMA indexing and perform all queries on non-indexed data. + +1: Enable SMA indexing and perform queries from suitable statements on precomputation results.| -| Attribute | Description | -| ------------- | ----------------------------------------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | Maximum number of query threads | -| Default Value | 1 | -| Note | value range: float number between [0, 2] 0: only 1 query thread; >0: the times of the number of cores | ### maxNumOfDistinctRes -| Attribute | Description | -| ------------- | -------------------------------------------- | -| Applicable | Server Only | +| Attribute | Description | +| -------- | -------------------------------- | --- | +| Applicable | Server Only | | Meaning | The maximum number of distinct rows returned | | Value Range | [100,000 - 100,000,000] | | Default Value | 100,000 | -| Note | After version 2.3.0.0 | ## Locale Parameters ### timezone -| Attribute | Description | -| ------------- | ------------------------------- | -| Applicable | Server and Client | +| Attribute | Description | +| -------- | ------------------------------ | +| Applicable | Server and Client | | Meaning | TimeZone | | Default Value | TimeZone configured in the host | :::info -To handle the data insertion and data query from multiple timezones, Unix Timestamp is used and stored in TDengine. The timestamp generated from any timezones at same time is same in Unix timestamp. To make sure the time on client side can be converted to Unix timestamp correctly, the timezone must be set properly. +To handle the data insertion and data query from multiple timezones, Unix Timestamp is used and stored in TDengine. The timestamp generated from any timezones at same time is same in Unix timestamp. Note that Unix timestamps are converted and recorded on the client side. To make sure the time on client side can be converted to Unix timestamp correctly, the timezone must be set properly. -On Linux system, TDengine clients automatically obtain timezone from the host. Alternatively, the timezone can be configured explicitly in configuration file `taos.cfg` like below. +On Linux system, TDengine clients automatically obtain timezone from the host. Alternatively, the timezone can be configured explicitly in configuration file `taos.cfg` like below. For example: ``` -timezone UTC-7 +timezone UTC-8 timezone GMT-8 timezone Asia/Shanghai ``` @@ -239,11 +237,11 @@ To avoid the problems of using time strings, Unix timestamp can be used directly | Default Value | Locale configured in host | :::info -A specific type "nchar" is provided in TDengine to store non-ASCII characters such as Chinese, Japanese, and Korean. The characters to be stored in nchar type are firstly encoded in UCS4-LE before sending to server side. To store non-ASCII characters correctly, the encoding format of the client side needs to be set properly. +A specific type "nchar" is provided in TDengine to store non-ASCII characters such as Chinese, Japanese, and Korean. The characters to be stored in nchar type are firstly encoded in UCS4-LE before sending to server side. Note that the correct encoding is determined by the user. To store non-ASCII characters correctly, the encoding format of the client side needs to be set properly. The characters input on the client side are encoded using the default system encoding, which is UTF-8 on Linux, or GB18030 or GBK on some systems in Chinese, POSIX in docker, CP936 on Windows in Chinese. The encoding of the operating system in use must be set correctly so that the characters in nchar type can be converted to UCS4-LE. -The locale definition standard on Linux is: \_., for example, in "zh_CN.UTF-8", "zh" means Chinese, "CN" means China mainland, "UTF-8" means charset. On Linux and Mac OSX, the charset can be set by locale in the system. On Windows system another configuration parameter `charset` must be used to configure charset because the locale used on Windows is not POSIX standard. Of course, `charset` can also be used on Linux to specify the charset. +The locale definition standard on Linux is: \_., for example, in "zh_CN.UTF-8", "zh" means Chinese, "CN" means China mainland, "UTF-8" means charset. The charset indicates how to display the characters. On Linux and Mac OSX, the charset can be set by locale in the system. On Windows system another configuration parameter `charset` must be used to configure charset because the locale used on Windows is not POSIX standard. Of course, `charset` can also be used on Linux to specify the charset. ::: @@ -256,459 +254,136 @@ The locale definition standard on Linux is: \_., f | Default Value | charset set in the system | :::info -On Linux, if `charset` is not set in `taos.cfg`, when `taos` is started, the charset is obtained from system locale. If obtaining charset from system locale fails, `taos` would fail to start. So on Linux system, if system locale is set properly, it's not necessary to set `charset` in `taos.cfg`. For example: +On Linux, if `charset` is not set in `taos.cfg`, when `taos` is started, the charset is obtained from system locale. If obtaining charset from system locale fails, `taos` would fail to start. + +So on Linux system, if system locale is set properly, it's not necessary to set `charset` in `taos.cfg`. For example: ``` locale zh_CN.UTF-8 ``` -On a Linux system, if the charset contained in `locale` is not consistent with that set by `charset`, the later setting in the configuration file takes precedence. - -```title="Effective charset is GBK" -locale zh_CN.UTF-8 -charset GBK -``` - -```title="Effective charset is UTF-8" -charset GBK -locale zh_CN.UTF-8 -``` - On Windows system, it's not possible to obtain charset from system locale. If it's not set in configuration file `taos.cfg`, it would be default to CP936, same as set as below in `taos.cfg`. For example ``` charset CP936 ``` +Refer to the documentation for your operating system before changing the charset. + +On a Linux system, if the charset contained in `locale` is not consistent with that set by `charset`, the later setting in the configuration file takes precedence. + +``` +locale zh_CN.UTF-8 +charset GBK +``` + +The charset that takes effect is GBK. + +``` +charset GBK +locale zh_CN.UTF-8 +``` + +The charset that takes effect is UTF-8. + ::: ## Storage Parameters ### dataDir -| Attribute | Description | -| ------------- | ------------------------------------------- | +| Attribute | Description | +| -------- | ------------------------------------------ | | Applicable | Server Only | | Meaning | All data files are stored in this directory | | Default Value | /var/lib/taos | -### cache - -| Attribute | Description | -| ------------- | ----------------------------- | -| Applicable | Server Only | -| Meaning | The size of each memory block | -| Unit | MB | -| Default Value | 16 | - -### blocks - -| Attribute | Description | -| ------------- | -------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | The number of memory blocks of size `cache` used by each vnode | -| Default Value | 6 | - -### days - -| Attribute | Description | -| ------------- | ----------------------------------------------------- | -| Applicable | Server Only | -| Meaning | The time range of the data stored in single data file | -| Unit | day | -| Default Value | 10 | - -### keep - -| Attribute | Description | -| ------------- | -------------------------------------- | -| Applicable | Server Only | -| Meaning | The number of days for data to be kept | -| Unit | day | -| Default Value | 3650 | - -### minRows - -| Attribute | Description | -| ------------- | ------------------------------------------ | -| Applicable | Server Only | -| Meaning | minimum number of rows in single data file | -| Default Value | 100 | - -### maxRows - -| Attribute | Description | -| ------------- | ------------------------------------------ | -| Applicable | Server Only | -| Meaning | maximum number of rows in single data file | -| Default Value | 4096 | - -### walLevel - -| Attribute | Description | -| ------------- | ---------------------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | WAL level | -| Value Range | 0: wal disabled
1: wal enabled without fsync
2: wal enabled with fsync | -| Default Value | 1 | - -### fsync - -| Attribute | Description | -| ------------- | --------------------------------------------------------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | The waiting time for invoking fsync when walLevel is 2 | -| Unit | millisecond | -| Value Range | 0: no waiting time, fsync is performed immediately once WAL is written;
maximum value is 180000, i.e. 3 minutes | -| Default Value | 3000 | - -### update - -| Attribute | Description | -| ------------- | ------------------------------------------------------------------------------------------------------ | -| Applicable | Server Only | -| Meaning | If it's allowed to update existing data | -| Value Range | 0: not allowed
1: a row can only be updated as a whole
2: a part of columns can be updated | -| Default Value | 0 | -| Note | Not available from version 2.0.8.0 | - -### cacheLast - -| Attribute | Description | -| ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | Whether to cache the latest rows of each sub table in memory | -| Value Range | 0: not cached
1: the last row of each sub table is cached
2: the last non-null value of each column is cached
3: identical to both 1 and 2 are set | -| Default Value | 0 | - ### minimalTmpDirGB -| Attribute | Description | -| ------------- | ----------------------------------------------------------------------------------------------- | -| Applicable | Server and Client | +| Attribute | Description | +| -------- | ------------------------------------------------ | +| Applicable | Server and Client | | Meaning | When the available disk space in tmpDir is below this threshold, writing to tmpDir is suspended | -| Unit | GB | -| Default Value | 1.0 | +| Unit | GB | +| Default Value | 1.0 | ### minimalDataDirGB -| Attribute | Description | -| ------------- | ------------------------------------------------------------------------------------------------ | -| Applicable | Server Only | -| Meaning | hen the available disk space in dataDir is below this threshold, writing to dataDir is suspended | -| Unit | GB | -| Default Value | 2.0 | - -### vnodeBak - -| Attribute | Description | -| ------------- | --------------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | Whether to backup the corresponding vnode directory when a vnode is deleted | -| Value Range | 0: not backed up, 1: backup | -| Default Value | 1 | +| Attribute | Description | +| -------- | ------------------------------------------------ | +| Applicable | Server Only | +| Meaning | When the available disk space in dataDir is below this threshold, writing to dataDir is suspended | +| Unit | GB | +| Default Value | 2.0 | ## Cluster Parameters -### numOfMnodes +### supportVnodes -| Attribute | Description | -| ------------- | ------------------------------ | -| Applicable | Server Only | -| Meaning | The number of management nodes | -| Default Value | 3 | - -### replica - -| Attribute | Description | -| ------------- | -------------------------- | -| Applicable | Server Only | -| Meaning | The number of replications | -| Value Range | 1-3 | -| Default Value | 1 | - -### quorum - -| Attribute | Description | -| ------------- | ------------------------------------------------------------------------------------------ | -| Applicable | Server Only | -| Meaning | The number of required confirmations for data replication in case of multiple replications | -| Value Range | 1,2 | -| Default Value | 1 | - -### role - -| Attribute | Description | -| ------------- | --------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | The role of the dnode | -| Value Range | 0: both mnode and vnode
1: mnode only
2: dnode only | -| Default Value | 0 | - -### balance - -| Attribute | Description | -| ------------- | ------------------------ | -| Applicable | Server Only | -| Meaning | Automatic load balancing | -| Value Range | 0: disabled, 1: enabled | -| Default Value | 1 | - -### balanceInterval - -| Attribute | Description | -| ------------- | ----------------------------------------------- | -| Applicable | Server Only | -| Meaning | The interval for checking load balance by mnode | -| Unit | second | -| Value Range | 1-30000 | -| Default Value | 300 | - -### arbitrator - -| Attribute | Description | -| ------------- | -------------------------------------------------- | -| Applicable | Server Only | -| Meaning | End point of arbitrator, format is same as firstEp | -| Default Value | None | +| Attribute | Description | +| -------- | --------------------------- | +| Applicable | Server Only | +| Meaning | Maximum number of vnodes per dnode | +| Value Range | 0-4096 | +| Default Value | 256 | ## Time Parameters -### precision - -| Attribute | Description | -| ------------- | ------------------------------------------------- | -| Applicable | Server only | -| Meaning | Time precision used for each database | -| Value Range | ms: millisecond; us: microsecond ; ns: nanosecond | -| Default Value | ms | - -### rpcTimer - -| Attribute | Description | -| ------------- | ------------------ | -| Applicable | Server and Client | -| Meaning | rpc retry interval | -| Unit | milliseconds | -| Value Range | 100-3000 | -| Default Value | 300 | - -### rpcMaxTime - -| Attribute | Description | -| ------------- | ---------------------------------- | -| Applicable | Server and Client | -| Meaning | maximum wait time for rpc response | -| Unit | second | -| Value Range | 100-7200 | -| Default Value | 600 | - ### statusInterval -| Attribute | Description | -| ------------- | ----------------------------------------------- | -| Applicable | Server Only | +| Attribute | Description | +| -------- | --------------------------- | +| Applicable | Server Only | | Meaning | the interval of dnode reporting status to mnode | | Unit | second | -| Value Range | 1-10 | -| Default Value | 1 | +| Value Range | 1-10 | +| Default Value | 1 | ### shellActivityTimer -| Attribute | Description | -| ------------- | ------------------------------------------------------ | -| Applicable | Server and Client | -| Meaning | The interval for taos shell to send heartbeat to mnode | -| Unit | second | -| Value Range | 1-120 | -| Default Value | 3 | - -### tableMetaKeepTimer - -| Attribute | Description | -| ------------- | -------------------------------------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | The expiration time for metadata in cache, once it's reached the client would refresh the metadata | -| Unit | second | -| Value Range | 1-8640000 | -| Default Value | 7200 | - -### maxTmrCtrl - -| Attribute | Description | -| ------------- | ------------------------ | -| Applicable | Server and Client | -| Meaning | Maximum number of timers | -| Unit | None | -| Value Range | 8-2048 | -| Default Value | 512 | - -### offlineThreshold - -| Attribute | Description | -| ------------- | ----------------------------------------------------------------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | The expiration time for dnode online status, once it's reached before receiving status from a node, the dnode becomes offline | -| Unit | second | -| Value Range | 5-7200000 | -| Default Value | 86400\*10 (i.e. 10 days) | +| Attribute | Description | +| -------- | --------------------------------- | +| Applicable | Server and Client | +| Meaning | The interval for TDengine CLI to send heartbeat to mnode | +| Unit | second | +| Value Range | 1-120 | +| Default Value | 3 | ## Performance Optimization Parameters -### numOfThreadsPerCore - -| Attribute | Description | -| ------------- | ------------------------------------------- | -| Applicable | Server and Client | -| Meaning | The number of consumer threads per CPU core | -| Default Value | 1.0 | - -### ratioOfQueryThreads - -| Attribute | Description | -| ------------- | --------------------------------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | Maximum number of query threads | -| Value Range | 0: Only one query thread
1: Same as number of CPU cores
2: two times of CPU cores | -| Default Value | 1 | -| Note | This value can be a float number, 0.5 means half of the CPU cores | - -### maxVgroupsPerDb - -| Attribute | Description | -| ------------- | ------------------------------------ | -| Applicable | Server Only | -| Meaning | Maximum number of vnodes for each DB | -| Value Range | 0-8192 | -| Default Value | | - -### maxTablesPerVnode - -| Attribute | Description | -| ------------- | -------------------------------------- | -| Applicable | Server Only | -| Meaning | Maximum number of tables in each vnode | -| Default Value | 1000000 | - -### minTablesPerVnode - -| Attribute | Description | -| ------------- | -------------------------------------- | -| Applicable | Server Only | -| Meaning | Minimum number of tables in each vnode | -| Default Value | 1000 | - -### tableIncStepPerVnode - -| Attribute | Description | -| ------------- | ------------------------------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | When minTablesPerVnode is reached, the number of tables are allocated for a vnode each time | -| Default Value | 1000 | - -### maxNumOfOrderedRes - -| Attribute | Description | -| ------------- | ------------------------------------------- | -| Applicable | Server and Client | -| Meaning | Maximum number of rows ordered for a STable | -| Default Value | 100,000 | - -### mnodeEqualVnodeNum - -| Attribute | Description | -| ------------- | ----------------------------------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | The number of vnodes whose system resources consumption are considered as equal to single mnode | -| Default Value | 4 | - ### numOfCommitThreads -| Attribute | Description | -| ------------- | ----------------------------------------- | -| Applicable | Server Only | +| Attribute | Description | +| -------- | ---------------------- | +| Applicable | Server Only | | Meaning | Maximum of threads for committing to disk | -| Default Value | | +| Default Value | | ## Compression Parameters -### comp - -| Attribute | Description | -| ------------- | ------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | Whether data is compressed | -| Value Range | 0: uncompressed, 1: One phase compression, 2: Two phase compression | -| Default Value | 2 | - -### tsdbMetaCompactRatio - -| Attribute | Description | -| ------------- | ------------------------------------------------------------------------------------------- | -| Meaning | The threshold for percentage of redundant in meta file to trigger compression for meta file | -| Value Range | 0: no compression forever, [1-100]: The threshold percentage | -| Default Value | 0 | - ### compressMsgSize -| Attribute | Description | -| ------------- | -------------------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | The threshold for message size to compress the message.. | +| Attribute | Description | +| ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Applicable | Server Only | +| Meaning | The threshold for message size to compress the message. | Set the value to 64330 bytes for good message compression. | | Unit | bytes | | Value Range | 0: already compress; >0: compress when message exceeds it; -1: always uncompress | -| Default Value | -1 | +| Default Value | -1 | ### compressColData -| Attribute | Description | -| ------------- | ------------------------------------------------------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | The threshold for size of column data to trigger compression for the query result | +| Attribute | Description | +| -------- | --------------------------------------------------------------------------------------- | +| Applicable | Server Only | +| Meaning | The threshold for size of column data to trigger compression for the query result | | Unit | bytes | | Value Range | 0: always compress; >0: only compress when the size of any column data exceeds the threshold; -1: always uncompress | +| Default Value | -1 | | Default Value | -1 | -| Note | available from version 2.3.0.0 | +| Note | available from version 2.3.0.0 | | -### lossyColumns - -| Attribute | Description | -| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | The floating number types for lossy compression | -| Value Range | "": lossy compression is disabled
float: only for float
double: only for double
float \| double: for both float and double | -| Default Value | "" , i.e. disabled | - -### fPrecision - -| Attribute | Description | -| ------------- | ----------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | Compression precision for float type | -| Value Range | 0.1 ~ 0.00000001 | -| Default Value | 0.00000001 | -| Note | The fractional part lower than this value will be discarded | - -### dPrecision - -| Attribute | Description | -| ------------- | ----------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | Compression precision for double type | -| Value Range | 0.1 ~ 0.0000000000000001 | -| Default Value | 0.0000000000000001 | -| Note | The fractional part lower than this value will be discarded | - -## Continuous Query Parameters - -### stream - -| Attribute | Description | -| ------------- | ---------------------------------- | -| Applicable | Server Only | -| Meaning | Whether to enable continuous query | -| Value Range | 0: disabled
1: enabled | -| Default Value | 1 | +## Continuous Query Parameters | ### minSlidingTime @@ -730,375 +405,444 @@ charset CP936 | Value Range | 1-1000000 | | Default Value | 10 | -### maxStreamCompDelay - -| Attribute | Description | -| ------------- | ------------------------------------------------ | -| Applicable | Server Only | -| Meaning | Maximum delay before starting a continuous query | -| Unit | millisecond | -| Value Range | 10-1000000000 | -| Default Value | 20000 | - -### maxFirstStreamCompDelay - -| Attribute | Description | -| ------------- | -------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | Maximum delay time before starting a continuous query the first time | -| Unit | millisecond | -| Value Range | 10-1000000000 | -| Default Value | 10000 | - -### retryStreamCompDelay - -| Attribute | Description | -| ------------- | --------------------------------------------- | -| Applicable | Server Only | -| Meaning | Delay time before retrying a continuous query | -| Unit | millisecond | -| Value Range | 10-1000000000 | -| Default Value | 10 | - -### streamCompDelayRatio - -| Attribute | Description | -| ------------- | ------------------------------------------------------------------------ | -| Applicable | Server Only | -| Meaning | The delay ratio, with time window size as the base, for continuous query | -| Value Range | 0.1-0.9 | -| Default Value | 0.1 | - :::info To prevent system resource from being exhausted by multiple concurrent streams, a random delay is applied on each stream automatically. `maxFirstStreamCompDelay` is the maximum delay time before a continuous query is started the first time. `streamCompDelayRatio` is the ratio for calculating delay time, with the size of the time window as base. `maxStreamCompDelay` is the maximum delay time. The actual delay time is a random time not bigger than `maxStreamCompDelay`. If a continuous query fails, `retryStreamComDelay` is the delay time before retrying it, also not bigger than `maxStreamCompDelay`. ::: -## HTTP Parameters - -### http - -| Attribute | Description | -| ------------- | ------------------------------ | -| Applicable | Server Only | -| Meaning | Whether to enable http service | -| Value Range | 0: disabled, 1: enabled | -| Default Value | 1 | - -### httpEnableRecordSql - -| Attribute | Description | -| ------------- | ------------------------------------------------------------------------- | -| Applicable | Server Only | -| Meaning | Whether to record the SQL invocation through REST interface | -| Default Value | 0: false; 1: true | -| Note | The resulting files, i.e. httpnote.0/httpnote.1, are located under logDir | - -### httpMaxThreads - -| Attribute | Description | -| ------------- | -------------------------------------------- | -| Applicable | Server Only | -| Meaning | The number of threads for RESTFul interface. | -| Default Value | 2 | - -### restfulRowLimit - -| Attribute | Description | -| ------------- | ------------------------------------------------------------ | -| Applicable | Server Only | -| Meaning | Maximum number of rows returned each time by REST interface. | -| Default Value | 10240 | -| Note | Maximum value is 10,000,000 | - -### httpDBNameMandatory - -| Attribute | Description | -| ------------- | ---------------------------------------- | -| Applicable | Server Only | -| Meaning | Whether database name is required in URL | -| Value Range | 0:not required, 1: required | -| Default Value | 0 | -| Note | From version 2.3.0.0 | - ## Log Parameters ### logDir -| Attribute | Description | -| ------------- | ----------------------------------- | -| Applicable | Server and Client | +| Attribute | Description | +| -------- | -------------------------------------------------- | +| Applicable | Server and Client | | Meaning | The directory for writing log files | | Default Value | /var/log/taos | ### minimalLogDirGB -| Attribute | Description | -| ------------- | -------------------------------------------------------------------------------------------------- | -| Applicable | Server and Client | +| Attribute | Description | +| -------- | -------------------------------------------- | +| Applicable | Server and Client | | Meaning | When the available disk space in logDir is below this threshold, writing to log files is suspended | -| Unit | GB | -| Default Value | 1.0 | +| Unit | GB | +| Default Value | 1.0 | ### numOfLogLines -| Attribute | Description | -| ------------- | ------------------------------------------ | -| Applicable | Server and Client | +| Attribute | Description | +| -------- | ---------------------------- | +| Applicable | Server and Client | | Meaning | Maximum number of lines in single log file | -| Default Value | 10,000,000 | +| Default Value | 10000000 | ### asyncLog -| Attribute | Description | -| ------------- | ---------------------------- | -| Applicable | Server and Client | +| Attribute | Description | +| -------- | -------------------- | +| Applicable | Server and Client | | Meaning | The mode of writing log file | | Value Range | 0: sync way; 1: async way | -| Default Value | 1 | +| Default Value | 1 | ### logKeepDays -| Attribute | Description | -| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | -| Applicable | Server and Client | +| Attribute | Description | +| -------- | ----------------------------------------------------------------------------------- | +| Applicable | Server and Client | | Meaning | The number of days for log files to be kept | -| Unit | day | -| Default Value | 0 | +| Unit | day | +| Default Value | 0 | | Note | When it's bigger than 0, the log file would be renamed to "taosdlog.xxx" in which "xxx" is the timestamp when the file is changed last time | ### debugFlag -| Attribute | Description | -| ------------- | --------------------------------------------------------- | -| Applicable | Server and Client | +| Attribute | Description | +| -------- | ------------------------------------------------------------------------------------------------- | +| Applicable | Server and Client | | Meaning | Log level | | Value Range | 131: INFO/WARNING/ERROR; 135: plus DEBUG; 143: plus TRACE | | Default Value | 131 or 135, depending on the module | -### mDebugFlag - -| Attribute | Description | -| ------------- | ------------------ | -| Applicable | Server Only | -| Meaning | Log level of mnode | -| Value Range | same as debugFlag | -| Default Value | 135 | - -### dDebugFlag - -| Attribute | Description | -| ------------- | ------------------ | -| Applicable | Server and Client | -| Meaning | Log level of dnode | -| Value Range | same as debugFlag | -| Default Value | 135 | - -### sDebugFlag - -| Attribute | Description | -| ------------- | ------------------------ | -| Applicable | Server and Client | -| Meaning | Log level of sync module | -| Value Range | same as debugFlag | -| Default Value | 135 | - -### wDebugFlag - -| Attribute | Description | -| ------------- | ----------------------- | -| Applicable | Server and Client | -| Meaning | Log level of WAL module | -| Value Range | same as debugFlag | -| Default Value | 135 | - -### sdbDebugFlag - -| Attribute | Description | -| ------------- | ---------------------- | -| Applicable | Server and Client | -| Meaning | logLevel of sdb module | -| Value Range | same as debugFlag | -| Default Value | 135 | - -### rpcDebugFlag - -| Attribute | Description | -| ------------- | ----------------------- | -| Applicable | Server and Client | -| Meaning | Log level of rpc module | -| Value Range | Same as debugFlag | -| Default Value | | - ### tmrDebugFlag -| Attribute | Description | -| ------------- | ------------------------- | +| Attribute | Description | +| -------- | -------------------- | | Applicable | Server and Client | | Meaning | Log level of timer module | -| Value Range | Same as debugFlag | -| Default Value | | - -### cDebugFlag - -| Attribute | Description | -| ------------- | ------------------- | -| Applicable | Client Only | -| Meaning | Log level of Client | -| Value Range | Same as debugFlag | -| Default Value | | - -### jniDebugFlag - -| Attribute | Description | -| ------------- | ----------------------- | -| Applicable | Client Only | -| Meaning | Log level of jni module | -| Value Range | Same as debugFlag | -| Default Value | | - -### odbcDebugFlag - -| Attribute | Description | -| ------------- | ------------------------ | -| Applicable | Client Only | -| Meaning | Log level of odbc module | -| Value Range | Same as debugFlag | -| Default Value | | +| Value Range | same as debugFlag | +| Default Value | | ### uDebugFlag -| Attribute | Description | -| ------------- | -------------------------- | -| Applicable | Server and Client | +| Attribute | Description | +| -------- | ---------------------- | +| Applicable | Server and Client | | Meaning | Log level of common module | -| Value Range | Same as debugFlag | -| Default Value | | | +| Value Range | same as debugFlag | +| Default Value | | -### mqttDebugFlag +### rpcDebugFlag -| Attribute | Description | -| ------------- | ------------------------ | -| Applicable | Server Only | -| Meaning | Log level of mqtt module | -| Value Range | Same as debugFlag | -| Default Value | | +| Attribute | Description | +| -------- | -------------------- | +| Applicable | Server and Client | +| Meaning | Log level of rpc module | +| Value Range | same as debugFlag | +| Default Value | | -### monitorDebugFlag +### jniDebugFlag -| Attribute | Description | -| ------------- | ------------------------------ | -| Applicable | Server Only | -| Meaning | Log level of monitoring module | -| Value Range | Same as debugFlag | -| Default Value | | +| Attribute | Description | +| -------- | ------------------ | +| Applicable | Client Only | +| Meaning | Log level of jni module | +| Value Range | same as debugFlag | +| Default Value | | ### qDebugFlag -| Attribute | Description | -| ------------- | ------------------------- | +| Attribute | Description | +| -------- | -------------------- | | Applicable | Server and Client | -| Meaning | Log level of query module | -| Value Range | Same as debugFlag | -| Default Value | | +| Meaning | Log level of query module | +| Value Range | same as debugFlag | +| Default Value | | + +### cDebugFlag + +| Attribute | Description | +| -------- | --------------------- | +| Applicable | Client Only | +| Meaning | Log level of Client | +| Value Range | same as debugFlag | +| Default Value | | + +### dDebugFlag + +| Attribute | Description | +| -------- | -------------------- | +| Applicable | Server Only | +| Meaning | Log level of dnode | +| Value Range | same as debugFlag | +| Default Value | 135 | ### vDebugFlag -| Attribute | Description | -| ------------- | ------------------ | -| Applicable | Server and Client | +| Attribute | Description | +| -------- | -------------------- | +| Applicable | Server Only | | Meaning | Log level of vnode | -| Value Range | Same as debugFlag | -| Default Value | | +| Value Range | same as debugFlag | +| Default Value | | + +### mDebugFlag + +| Attribute | Description | +| -------- | -------------------- | +| Applicable | Server Only | +| Meaning | Log level of mnode module | +| Value Range | same as debugFlag | +| Default Value | 135 | + +### wDebugFlag + +| Attribute | Description | +| -------- | ------------------ | +| Applicable | Server Only | +| Meaning | Log level of WAL module | +| Value Range | same as debugFlag | +| Default Value | 135 | + +### sDebugFlag + +| Attribute | Description | +| -------- | -------------------- | +| Applicable | Server and Client | +| Meaning | Log level of sync module | +| Value Range | same as debugFlag | +| Default Value | 135 | ### tsdbDebugFlag -| Attribute | Description | -| ------------- | ------------------------ | -| Applicable | Server Only | -| Meaning | Log level of TSDB module | -| Value Range | Same as debugFlag | -| Default Value | | +| Attribute | Description | +| -------- | ------------------- | +| Applicable | Server Only | +| Meaning | Log level of TSDB module | +| Value Range | same as debugFlag | +| Default Value | | -### cqDebugFlag +### tqDebugFlag | Attribute | Description | -| ------------- | ------------------------------------ | -| Applicable | Server and Client | -| Meaning | Log level of continuous query module | -| Value Range | Same as debugFlag | -| Default Value | | +| -------- | ----------------- | +| Applicable | Server only | +| Meaning | Log level of TQ module | +| Value Range | same as debugFlag | +| Default Value | | -## Client Only +### fsDebugFlag -### maxSQLLength +| Attribute | Description | +| -------- | ----------------- | +| Applicable | Server only | +| Meaning | Log level of FS module | +| Value Range | same as debugFlag | +| Default Value | | + +### udfDebugFlag | Attribute | Description | -| ------------- | -------------------------------------- | -| Applicable | Client Only | -| Meaning | Maximum length of single SQL statement | -| Unit | bytes | -| Value Range | 65480-1048576 | -| Default Value | 1048576 | +| -------- | ------------------ | +| Applicable | Server Only | +| Meaning | Log level of UDF module | +| Value Range | same as debugFlag | +| Default Value | | -### tscEnableRecordSql +### smaDebugFlag -| Attribute | Description | -| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| Meaning | Whether to record SQL statements in file | -| Value Range | 0: false, 1: true | -| Default Value | 0 | -| Note | The generated files are named as "tscnote-xxxx.0/tscnote-xxx.1" in which "xxxx" is the pid of the client, and located at same place as client log | +| Attribute | Description | +| -------- | ------------------ | +| Applicable | Server Only | +| Meaning | Log level of SMA module | +| Value Range | same as debugFlag | +| Default Value | | -### maxBinaryDisplayWidth +### idxDebugFlag -| Attribute | Description | -| ------------- | --------------------------------------------------------------------------------------------------- | -| Meaning | Maximum display width of binary and nchar in taos shell. Anything beyond this limit would be hidden | -| Value Range | 5 - | -| Default Value | 30 | +| Attribute | Description | +| -------- | -------------------- | +| Applicable | Server Only | +| Meaning | Log level of index module | +| Value Range | same as debugFlag | +| Default Value | | -:::info -If the length of value exceeds `maxBinaryDisplayWidth`, then the actual display width is max(column name, maxBinaryDisplayLength); otherwise the actual display width is max(length of column name, length of column value). This parameter can also be changed dynamically using `set max_binary_display_width ` in TDengine CLI `taos`. +### tdbDebugFlag -::: +| Attribute | Description | +| -------- | ------------------ | +| Applicable | Server Only | +| Meaning | Log level of TDB module | +| Value Range | same as debugFlag | +| Default Value | | -### maxWildCardsLength +## Schemaless Parameters -| Attribute | Description | -| ------------- | ----------------------------------------------------- | -| Meaning | The maximum length for wildcard string used with LIKE | -| Unit | bytes | -| Value Range | 0-16384 | -| Default Value | 100 | -| Note | From version 2.1.6.1 | +### smlChildTableName -### clientMerge +| Attribute | Description | +| -------- | ------------------------- | +| Applicable | Client only | +| Meaning | Custom subtable name for schemaless writes | +| Type | String | +| Default Value | None | -| Attribute | Description | -| ------------- | --------------------------------------------------- | -| Meaning | Whether to filter out duplicate data on client side | -| Value Range | 0: false; 1: true | -| Default Value | 0 | -| Note | From version 2.3.0.0 | +### smlTagName -### maxRegexStringLen +| Attribute | Description | +| -------- | ------------------------------------ | +| Applicable | Client only | +| Meaning | Default tag for schemaless writes without tag value specified | +| Type | String | +| Default Value | _tag_null | + +### smlDataFormat | Attribute | Description | -| ------------- | ------------------------------------ | -| Meaning | Maximum length of regular expression | -| Value Range | [128, 16384] | -| Default Value | 128 | -| Note | From version 2.3.0.0 | +| -------- | ----------------------------- | +| Applicable | Client only | +| Meaning | Whether schemaless columns are consistently ordered | +| Value Range | 0: not consistent; 1: consistent. | +| Default | 1 | ## Other Parameters ### enableCoreFile -| Attribute | Description | -| ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Attribute | Description | +| -------- | ------------------------------------------------------------------------------------------------------------------------------------------ | | Applicable | Server and Client | | Meaning | Whether to generate core file when server crashes | | Value Range | 0: false, 1: true | | Default Value | 1 | | Note | The core file is generated under root directory `systemctl start taosd` is used to start, or under the working directory if `taosd` is started directly on Linux Shell. | + +### udf + +| Attribute | Description | +| -------- | ------------------ | +| Applicable | Server Only | +| Meaning | Whether the UDF service is enabled | +| Value Range | 0: disable UDF; 1: enabled UDF | +| Default Value | 1 | + +## Parameter Comparison of TDengine 2.x and 3.0 +| # | **Parameter** | **In 2.x** | **In 3.0** | +| --- | :-----------------: | --------------- | --------------- | +| 1 | firstEp | Yes | Yes | +| 2 | secondEp | Yes | Yes | +| 3 | fqdn | Yes | Yes | +| 4 | serverPort | Yes | Yes | +| 5 | maxShellConns | Yes | Yes | +| 6 | monitor | Yes | Yes | +| 7 | monitorFqdn | No | Yes | +| 8 | monitorPort | No | Yes | +| 9 | monitorInterval | Yes | Yes | +| 10 | monitorMaxLogs | No | Yes | +| 11 | monitorComp | No | Yes | +| 12 | telemetryReporting | Yes | Yes | +| 13 | telemetryInterval | No | Yes | +| 14 | telemetryServer | No | Yes | +| 15 | telemetryPort | No | Yes | +| 16 | queryPolicy | No | Yes | +| 17 | querySmaOptimize | No | Yes | +| 18 | queryBufferSize | Yes | Yes | +| 19 | maxNumOfDistinctRes | Yes | Yes | +| 20 | minSlidingTime | Yes | Yes | +| 21 | minIntervalTime | Yes | Yes | +| 22 | countAlwaysReturnValue | Yes | Yes | +| 23 | dataDir | Yes | Yes | +| 24 | minimalDataDirGB | Yes | Yes | +| 25 | supportVnodes | No | Yes | +| 26 | tempDir | Yes | Yes | +| 27 | minimalTmpDirGB | Yes | Yes | +| 28 | compressMsgSize | Yes | Yes | +| 29 | compressColData | Yes | Yes | +| 30 | smlChildTableName | Yes | Yes | +| 31 | smlTagName | Yes | Yes | +| 32 | smlDataFormat | No | Yes | +| 33 | statusInterval | Yes | Yes | +| 34 | shellActivityTimer | Yes | Yes | +| 35 | transPullupInterval | No | Yes | +| 36 | mqRebalanceInterval | No | Yes | +| 37 | ttlUnit | No | Yes | +| 38 | ttlPushInterval | No | Yes | +| 39 | numOfTaskQueueThreads | No | Yes | +| 40 | numOfRpcThreads | No | Yes | +| 41 | numOfCommitThreads | Yes | Yes | +| 42 | numOfMnodeReadThreads | No | Yes | +| 43 | numOfVnodeQueryThreads | No | Yes | +| 44 | numOfVnodeStreamThreads | No | Yes | +| 45 | numOfVnodeFetchThreads | No | Yes | +| 46 | numOfVnodeWriteThreads | No | Yes | +| 47 | numOfVnodeSyncThreads | No | Yes | +| 48 | numOfQnodeQueryThreads | No | Yes | +| 49 | numOfQnodeFetchThreads | No | Yes | +| 50 | numOfSnodeSharedThreads | No | Yes | +| 51 | numOfSnodeUniqueThreads | No | Yes | +| 52 | rpcQueueMemoryAllowed | No | Yes | +| 53 | logDir | Yes | Yes | +| 54 | minimalLogDirGB | Yes | Yes | +| 55 | numOfLogLines | Yes | Yes | +| 56 | asyncLog | Yes | Yes | +| 57 | logKeepDays | Yes | Yes | +| 58 | debugFlag | Yes | Yes | +| 59 | tmrDebugFlag | Yes | Yes | +| 60 | uDebugFlag | Yes | Yes | +| 61 | rpcDebugFlag | Yes | Yes | +| 62 | jniDebugFlag | Yes | Yes | +| 63 | qDebugFlag | Yes | Yes | +| 64 | cDebugFlag | Yes | Yes | +| 65 | dDebugFlag | Yes | Yes | +| 66 | vDebugFlag | Yes | Yes | +| 67 | mDebugFlag | Yes | Yes | +| 68 | wDebugFlag | Yes | Yes | +| 69 | sDebugFlag | Yes | Yes | +| 70 | tsdbDebugFlag | Yes | Yes | +| 71 | tqDebugFlag | No | Yes | +| 72 | fsDebugFlag | Yes | Yes | +| 73 | udfDebugFlag | No | Yes | +| 74 | smaDebugFlag | No | Yes | +| 75 | idxDebugFlag | No | Yes | +| 76 | tdbDebugFlag | No | Yes | +| 77 | metaDebugFlag | No | Yes | +| 78 | timezone | Yes | Yes | +| 79 | locale | Yes | Yes | +| 80 | charset | Yes | Yes | +| 81 | udf | Yes | Yes | +| 82 | enableCoreFile | Yes | Yes | +| 83 | arbitrator | Yes | No | +| 84 | numOfThreadsPerCore | Yes | No | +| 85 | numOfMnodes | Yes | No | +| 86 | vnodeBak | Yes | No | +| 87 | balance | Yes | No | +| 88 | balanceInterval | Yes | No | +| 89 | offlineThreshold | Yes | No | +| 90 | role | Yes | No | +| 91 | dnodeNopLoop | Yes | No | +| 92 | keepTimeOffset | Yes | No | +| 93 | rpcTimer | Yes | No | +| 94 | rpcMaxTime | Yes | No | +| 95 | rpcForceTcp | Yes | No | +| 96 | tcpConnTimeout | Yes | No | +| 97 | syncCheckInterval | Yes | No | +| 98 | maxTmrCtrl | Yes | No | +| 99 | monitorReplica | Yes | No | +| 100 | smlTagNullName | Yes | No | +| 101 | keepColumnName | Yes | No | +| 102 | ratioOfQueryCores | Yes | No | +| 103 | maxStreamCompDelay | Yes | No | +| 104 | maxFirstStreamCompDelay | Yes | No | +| 105 | retryStreamCompDelay | Yes | No | +| 106 | streamCompDelayRatio | Yes | No | +| 107 | maxVgroupsPerDb | Yes | No | +| 108 | maxTablesPerVnode | Yes | No | +| 109 | minTablesPerVnode | Yes | No | +| 110 | tableIncStepPerVnode | Yes | No | +| 111 | cache | Yes | No | +| 112 | blocks | Yes | No | +| 113 | days | Yes | No | +| 114 | keep | Yes | No | +| 115 | minRows | Yes | No | +| 116 | maxRows | Yes | No | +| 117 | quorum | Yes | No | +| 118 | comp | Yes | No | +| 119 | walLevel | Yes | No | +| 120 | fsync | Yes | No | +| 121 | replica | Yes | No | +| 122 | partitions | Yes | No | +| 123 | quorum | Yes | No | +| 124 | update | Yes | No | +| 125 | cachelast | Yes | No | +| 126 | maxSQLLength | Yes | No | +| 127 | maxWildCardsLength | Yes | No | +| 128 | maxRegexStringLen | Yes | No | +| 129 | maxNumOfOrderedRes | Yes | No | +| 130 | maxConnections | Yes | No | +| 131 | mnodeEqualVnodeNum | Yes | No | +| 132 | http | Yes | No | +| 133 | httpEnableRecordSql | Yes | No | +| 134 | httpMaxThreads | Yes | No | +| 135 | restfulRowLimit | Yes | No | +| 136 | httpDbNameMandatory | Yes | No | +| 137 | httpKeepAlive | Yes | No | +| 138 | enableRecordSql | Yes | No | +| 139 | maxBinaryDisplayWidth | Yes | No | +| 140 | stream | Yes | No | +| 141 | retrieveBlockingModel | Yes | No | +| 142 | tsdbMetaCompactRatio | Yes | No | +| 143 | defaultJSONStrType | Yes | No | +| 144 | walFlushSize | Yes | No | +| 145 | keepTimeOffset | Yes | No | +| 146 | flowctrl | Yes | No | +| 147 | slaveQuery | Yes | No | +| 148 | adjustMaster | Yes | No | +| 149 | topicBinaryLen | Yes | No | +| 150 | telegrafUseFieldNum | Yes | No | +| 151 | deadLockKillQuery | Yes | No | +| 152 | clientMerge | Yes | No | +| 153 | sdbDebugFlag | Yes | No | +| 154 | odbcDebugFlag | Yes | No | +| 155 | httpDebugFlag | Yes | No | +| 156 | monDebugFlag | Yes | No | +| 157 | cqDebugFlag | Yes | No | +| 158 | shortcutFlag | Yes | No | +| 159 | probeSeconds | Yes | No | +| 160 | probeKillSeconds | Yes | No | +| 161 | probeInterval | Yes | No | +| 162 | lossyColumns | Yes | No | +| 163 | fPrecision | Yes | No | +| 164 | dPrecision | Yes | No | +| 165 | maxRange | Yes | No | +| 166 | range | Yes | No | diff --git a/docs/en/14-reference/13-schemaless/13-schemaless.md b/docs/en/14-reference/13-schemaless/13-schemaless.md index 8b6a26ae52..4f50c38cbb 100644 --- a/docs/en/14-reference/13-schemaless/13-schemaless.md +++ b/docs/en/14-reference/13-schemaless/13-schemaless.md @@ -1,9 +1,10 @@ --- title: Schemaless Writing -description: "The Schemaless write method eliminates the need to create super tables/sub tables in advance and automatically creates the storage structure corresponding to the data, as it is written to the interface." +description: 'The Schemaless write method eliminates the need to create super tables/sub tables in advance and automatically creates the storage structure corresponding to the data, as it is written to the interface.' --- -In IoT applications, data is collected for many purposes such as intelligent control, business analysis, device monitoring and so on. Due to changes in business or functional requirements or changes in device hardware, the application logic and even the data collected may change. To provide the flexibility needed in such cases and in a rapidly changing IoT landscape, TDengine provides a series of interfaces for the schemaless writing method. These interfaces eliminate the need to create super tables and subtables in advance by automatically creating the storage structure corresponding to the data as the data is written to the interface. When necessary, schemaless writing will automatically add the required columns to ensure that the data written by the user is stored correctly. +In IoT applications, data is collected for many purposes such as intelligent control, business analysis, device monitoring and so on. Due to changes in business or functional requirements or changes in device hardware, the application logic and even the data collected may change. Schemaless writing automatically creates storage structures for your data as it is being written to TDengine, so that you do not need to create supertables in advance. When necessary, schemaless writing +will automatically add the required columns to ensure that the data written by the user is stored correctly. The schemaless writing method creates super tables and their corresponding subtables. These are completely indistinguishable from the super tables and subtables created directly via SQL. You can write data directly to them via SQL statements. Note that the names of tables created by schemaless writing are based on fixed mapping rules for tag values, so they are not explicitly ideographic and they lack readability. @@ -19,12 +20,12 @@ With the following formatting conventions, schemaless writing uses a single stri measurement,tag_set field_set timestamp ``` -where : +where: - measurement will be used as the data table name. It will be separated from tag_set by a comma. -- tag_set will be used as tag data in the format `=,=`, i.e. multiple tags' data can be separated by a comma. It is separated from field_set by space. -- field_set will be used as normal column data in the format of `=,=`, again using a comma to separate multiple normal columns of data. It is separated from the timestamp by a space. -- The timestamp is the primary key corresponding to the data in this row. +- `tag_set` will be used as tags, with format like `=,=` Enter a space between `tag_set` and `field_set`. +- `field_set`will be used as data columns, with format like `=,=` Enter a space between `field_set` and `timestamp`. +- `timestamp` is the primary key timestamp corresponding to this row of data All data in tag_set is automatically converted to the NCHAR data type and does not require double quotes ("). @@ -35,18 +36,20 @@ In the schemaless writing data line protocol, each data item in the field_set ne - Spaces, equal signs (=), commas (,), and double quotes (") need to be escaped with a backslash (\\) in front. (All refer to the ASCII character) - Numeric types will be distinguished from data types by the suffix. -| **Serial number** | **Postfix** | **Mapping type** | **Size (bytes)** | -| -------- | -------- | ------------ | -------------- | -| 1 | none or f64 | double | 8 | -| 2 | f32 | float | 4 | -| 3 | i8/u8 | TinyInt/UTinyInt | 1 | -| 4 | i16/u16 | SmallInt/USmallInt | 2 | -| 5 | i32/u32 | Int/UInt | 4 | -| 6 | i64/i/u64/u | Bigint/Bigint/UBigint/UBigint | 8 | +| **Serial number** | **Postfix** | **Mapping type** | **Size (bytes)** | +| ----------------- | ----------- | ----------------------------- | ---------------- | +| 1 | None or f64 | double | 8 | +| 2 | f32 | float | 4 | +| 3 | i8/u8 | TinyInt/UTinyInt | 1 | +| 4 | i16/u16 | SmallInt/USmallInt | 2 | +| 5 | i32/u32 | Int/UInt | 4 | +| 6 | i64/i/u64/u | BigInt/BigInt/UBigInt/UBigInt | 8 | - `t`, `T`, `true`, `True`, `TRUE`, `f`, `F`, `false`, and `False` will be handled directly as BOOL types. -For example, the following data rows indicate that the t1 label is "3" (NCHAR), the t2 label is "4" (NCHAR), and the t3 label is "t3" to the super table named `st` labeled "t3" (NCHAR), write c1 column as 3 (BIGINT), c2 column as false (BOOL), c3 column is "passit" (BINARY), c4 column is 4 (DOUBLE), and the primary key timestamp is 1626006833639000000 in one row. +For example, the following data rows indicate that the t1 label is "3" (NCHAR), the t2 label is "4" (NCHAR), and the t3 label +is "t3" to the super table named `st` labeled "t3" (NCHAR), write c1 column as 3 (BIGINT), c2 column as false (BOOL), c3 column +is "passit" (BINARY), c4 column is 4 (DOUBLE), and the primary key timestamp is 1626006833639000000 in one row. ```json st,t1=3,t2=4,t3=t3 c1=3i64,c3="passit",c2=false,c4=4f64 1626006833639000000 @@ -58,102 +61,105 @@ Note that if the wrong case is used when describing the data type suffix, or if Schemaless writes process row data according to the following principles. -1. You can use the following rules to generate the subtable names: first, combine the measurement name and the key and value of the label into the next string: +1. You can use the following rules to generate the subtable names: first, combine the measurement name and the key and value of the label into the next string: ```json "measurement,tag_key1=tag_value1,tag_key2=tag_value2" ``` Note that tag_key1, tag_key2 are not the original order of the tags entered by the user but the result of using the tag names in ascending order of the strings. Therefore, tag_key1 is not the first tag entered in the line protocol. -The string's MD5 hash value "md5_val" is calculated after the ranking is completed. The calculation result is then combined with the string to generate the table name: "t_md5_val". "t*" is a fixed prefix that every table generated by this mapping relationship has. +The string's MD5 hash value "md5_val" is calculated after the ranking is completed. The calculation result is then combined with the string to generate the table name: "t_md5_val". "t_" is a fixed prefix that every table generated by this mapping relationship has. +You can configure smlChildTableName to specify table names, for example, `smlChildTableName=tname`. You can insert `st,tname=cpul,t1=4 c1=3 1626006833639000000` and the cpu1 table will be automatically created. Note that if multiple rows have the same tname but different tag_set values, the tag_set of the first row is used to create the table and the others are ignored. 2. If the super table obtained by parsing the line protocol does not exist, this super table is created. -If the subtable obtained by the parse line protocol does not exist, Schemaless creates the sub-table according to the subtable name determined in steps 1 or 2. +3. If the subtable obtained by the parse line protocol does not exist, Schemaless creates the sub-table according to the subtable name determined in steps 1 or 2. 4. If the specified tag or regular column in the data row does not exist, the corresponding tag or regular column is added to the super table (only incremental). -5. If there are some tag columns or regular columns in the super table that are not specified to take values in a data row, then the values of these columns are set to NULL. +5. If there are some tag columns or regular columns in the super table that are not specified to take values in a data row, then the values of these columns are set to + NULL. 6. For BINARY or NCHAR columns, if the length of the value provided in a data row exceeds the column type limit, the maximum length of characters allowed to be stored in the column is automatically increased (only incremented and not decremented) to ensure complete preservation of the data. 7. Errors encountered throughout the processing will interrupt the writing process and return an error code. -8. In order to improve the efficiency of writing, it is assumed by default that the order of the fields in the same Super is the same (the first data contains all fields, and the following data is in this order). If the order is different, the parameter smlDataFormat needs to be configured to be false. Otherwise, the data is written in the same order, and the data in the library will be abnormal. +8. It is assumed that the order of field_set in a supertable is consistent, meaning that the first record contains all fields and subsequent records store fields in the same order. If the order is not consistent, set smlDataFormat to false. Otherwise, data will be written out of order and a database error will occur. :::tip -All processing logic of schemaless will still follow TDengine's underlying restrictions on data structures, such as the total length of each row of data cannot exceed 16k bytes. See [TAOS SQL Boundary Limits](/taos-sql/limit) for specific constraints in this area. +All processing logic of schemaless will still follow TDengine's underlying restrictions on data structures, such as the total length of each row of data cannot exceed +16KB. See [TDengine SQL Boundary Limits](/taos-sql/limit) for specific constraints in this area. + ::: ## Time resolution recognition Three specified modes are supported in the schemaless writing process, as follows: -| **Serial** | **Value** | **Description** | -| -------- | ------------------- | ------------------------------- | -| 1 | SML_LINE_PROTOCOL | InfluxDB Line Protocol | -| 2 | SML_TELNET_PROTOCOL | OpenTSDB Text Line Protocol | -| 3 | SML_JSON_PROTOCOL | JSON protocol format | +| **Serial** | **Value** | **Description** | +| ---------- | ------------------- | ---------------------- | +| 1 | SML_LINE_PROTOCOL | InfluxDB Line Protocol | +| 2 | SML_TELNET_PROTOCOL | OpenTSDB file protocol | +| 3 | SML_JSON_PROTOCOL | OpenTSDB JSON protocol | -In the SML_LINE_PROTOCOL parsing mode, the user is required to specify the time resolution of the input timestamp. The available time resolutions are shown in the following table. +In InfluxDB line protocol mode, you must specify the precision of the input timestamp. Valid precisions are described in the following table. -| **Serial Number** | **Time Resolution Definition** | **Meaning** | -| -------- | --------------------------------- | -------------- | -| 1 | TSDB_SML_TIMESTAMP_NOT_CONFIGURED | Not defined (invalid) | -| 2 | TSDB_SML_TIMESTAMP_HOURS | hour | -| 3 | TSDB_SML_TIMESTAMP_MINUTES | MINUTES -| 4 | TSDB_SML_TIMESTAMP_SECONDS | SECONDS -| 5 | TSDB_SML_TIMESTAMP_MILLI_SECONDS | milliseconds -| 6 | TSDB_SML_TIMESTAMP_MICRO_SECONDS | microseconds -| 7 | TSDB_SML_TIMESTAMP_NANO_SECONDS | nanoseconds | +| **No.** | **Precision** | **Description** | +| ------- | --------------------------------- | --------------------- | +| 1 | TSDB_SML_TIMESTAMP_NOT_CONFIGURED | Not defined (invalid) | +| 2 | TSDB_SML_TIMESTAMP_HOURS | Hours | +| 3 | TSDB_SML_TIMESTAMP_MINUTES | Minutes | +| 4 | TSDB_SML_TIMESTAMP_SECONDS | Seconds | +| 5 | TSDB_SML_TIMESTAMP_MILLI_SECONDS | Milliseconds | +| 6 | TSDB_SML_TIMESTAMP_MICRO_SECONDS | Microseconds | +| 7 | TSDB_SML_TIMESTAMP_NANO_SECONDS | Nanoseconds | -In SML_TELNET_PROTOCOL and SML_JSON_PROTOCOL modes, the time precision is determined based on the length of the timestamp (in the same way as the OpenTSDB standard operation), and the user-specified time resolution is ignored at this point. +In OpenTSDB file and JSON protocol modes, the precision of the timestamp is determined from its length in the standard OpenTSDB manner. User input is ignored. -## Data schema mapping rules +## Data Model Mapping -This section describes how data for line protocols are mapped to data with a schema. The data measurement in each line protocol is mapped as follows: -- The tag name in tag_set is the name of the tag in the data schema -- The name in field_set is the column's name. - -The following data is used as an example to illustrate the mapping rules. +This section describes how data in line protocol is mapped to a schema. The data measurement in each line is mapped to a +supertable name. The tag name in tag_set is the tag name in the schema, and the name in field_set is the column name in the schema. The following example shows how data is mapped: ```json st,t1=3,t2=4,t3=t3 c1=3i64,c3="passit",c2=false,c4=4f64 1626006833639000000 ``` -The row data mapping generates a super table: `st`, which contains three labels of type NCHAR: t1, t2, t3. Five data columns are ts (timestamp), c1 (bigint), c3 (binary), c2 (bool), c4 (bigint). The mapping becomes the following SQL statement. +This row is mapped to a supertable: `st` contains three NCHAR tags: t1, t2, and t3. Five columns are created: ts (timestamp), c1 (bigint), c3 (binary), c2 (bool), and c4 (bigint). The following SQL statement is generated: ```json create stable st (_ts timestamp, c1 bigint, c2 bool, c3 binary(6), c4 bigint) tags(t1 nchar(1), t2 nchar(1), t3 nchar(2)) ``` -## Data schema change handling +## Processing Schema Changes -This section describes the impact on the data schema for different line protocol data writing cases. +This section describes the impact on the schema caused by different data being written. -When writing to an explicitly identified field type using the line protocol, subsequent changes to the field's type definition will result in an explicit data schema error, i.e., will trigger a write API report error. As shown below, the +If you use line protocol to write to a specific tag field and then later change the field type, a schema error will ocur. This triggers an error on the write API. This is shown as follows: ```json -st,t1=3,t2=4,t3=t3 c1=3i64,c3="passit",c2=false,c4=4 1626006833639000000 -st,t1=3,t2=4,t3=t3 c1=3i64,c3="passit",c2=false,c4=4i 1626006833640000000 +st,t1=3,t2=4,t3=t3 c1=3i64,c3="passit",c2=false,c4=4 1626006833639000000 +st,t1=3,t2=4,t3=t3 c1=3i64,c3="passit",c2=false,c4=4i 1626006833640000000 ``` -The data type mapping in the first row defines column c4 as DOUBLE, but the data in the second row is declared as BIGINT by the numeric suffix, which triggers a parsing error with schemaless writing. +The first row defines c4 as a double. However, in the second row, the suffix indicates that the value of c4 is a bigint. This causes schemaless writing to throw an error. -If the line protocol before the column declares the data column as BINARY, the subsequent one requires a longer binary length, which triggers a super table schema change. +An error also occurs if data input into a binary column exceeds the defined length of the column. ```json -st,t1=3,t2=4,t3=t3 c1=3i64,c5="pass" 1626006833639000000 -st,t1=3,t2=4,t3=t3 c1=3i64,c5="passit" 1626006833640000000 +st,t1=3,t2=4,t3=t3 c1=3i64,c5="pass" 1626006833639000000 +st,t1=3,t2=4,t3=t3 c1=3i64,c5="passit" 1626006833640000000 ``` -The first line of the line protocol parsing will declare column c5 is a BINARY(4) field. The second line data write will parse column c5 as a BINARY column. But in the second line, c5's width is 6 so you need to increase the width of the BINARY field to be able to accommodate the new string. +The first row defines c5 as a binary(4). but the second row writes 6 bytes to it. This means that the length of the binary column must be expanded to contain the data. ```json -st,t1=3,t2=4,t3=t3 c1=3i64 1626006833639000000 -st,t1=3,t2=4,t3=t3 c1=3i64,c6="passit" 1626006833640000000 +st,t1=3,t2=4,t3=t3 c1=3i64 1626006833639000000 +st,t1=3,t2=4,t3=t3 c1=3i64,c6="passit" 1626006833640000000 ``` -The second line of data has an additional column c6 of type BINARY(6) compared to the first row. Then a column c6 of type BINARY(6) is automatically added at this point. +The preceding data includes a new entry, c6, with type binary(6). When this occurs, a new column c6 with type binary(6) is added automatically. -## Write integrity +## Write Integrity -TDengine provides idempotency guarantees for data writing, i.e., you can repeatedly call the API to write data with errors. However, it does not give atomicity guarantees for writing multiple rows of data. During the process of writing numerous rows of data in one batch, some data will be written successfully, and some data will fail. +TDengine guarantees the idempotency of data writes. This means that you can repeatedly call the API to perform write operations with bad data. However, TDengine does not guarantee the atomicity of multi-row writes. In a multi-row write, some data may be written successfully and other data unsuccessfully. -## Error code +##: Error Codes -If it is an error in the data itself during the schemaless writing process, the application will get `TSDB_CODE_TSC_LINE_SYNTAX_ERROR` error message, which indicates that the error occurred in writing. The other error codes are consistent with the TDengine and can be obtained via the `taos_errstr()` to get the specific cause of the error. +The TSDB_CODE_TSC_LINE_SYNTAX_ERROR indicates an error in the schemaless writing component. +This error occurs when writing text. For other errors, schemaless writing uses the standard TDengine error codes +found in taos_errstr. diff --git a/docs/en/20-third-party/01-grafana.mdx b/docs/en/20-third-party/01-grafana.mdx index 5dbeb31a23..e0fbefd5a8 100644 --- a/docs/en/20-third-party/01-grafana.mdx +++ b/docs/en/20-third-party/01-grafana.mdx @@ -6,9 +6,7 @@ title: Grafana import Tabs from "@theme/Tabs"; import TabItem from "@theme/TabItem"; -TDengine can be quickly integrated with the open-source data visualization system [Grafana](https://www.grafana.com/) to build a data monitoring and alerting system. The whole process does not require any code development. And you can visualize the contents of the data tables in TDengine on a dashboard. - -You can learn more about using the TDengine plugin on [GitHub](https://github.com/taosdata/grafanaplugin/blob/master/README.md). +TDengine can be quickly integrated with the open-source data visualization system [Grafana](https://www.grafana.com/) to build a data monitoring and alerting system. The whole process does not require any code development. And you can visualize the contents of the data tables in TDengine on a dashboard. You can learn more about using the TDengine plugin on [GitHub](https://github.com/taosdata/grafanaplugin/blob/master/README.md). ## Prerequisites @@ -65,7 +63,6 @@ Restart Grafana service and open Grafana in web-browser, usually - Follow the installation steps in [Grafana](https://grafana.com/grafana/plugins/tdengine-datasource/?tab=installation) with the [``grafana-cli`` command-line tool](https://grafana.com/docs/grafana/latest/administration/cli/) for plugin installation. @@ -76,7 +73,7 @@ grafana-cli plugins install tdengine-datasource sudo -u grafana grafana-cli plugins install tdengine-datasource ``` -Alternatively, you can manually download the .zip file from [GitHub](https://github.com/taosdata/grafanaplugin/releases/tag/latest) or [Grafana](https://grafana.com/grafana/plugins/tdengine-datasource/?tab=installation) and unpack it into your grafana plugins directory. +You can also download zip files from [GitHub](https://github.com/taosdata/grafanaplugin/releases/tag/latest) or [Grafana](https://grafana.com/grafana/plugins/tdengine-datasource/?tab=installation) and install manually. The commands are as follows: ```bash GF_VERSION=3.2.2 @@ -131,7 +128,7 @@ docker run -d \ grafana/grafana ``` -You can setup a zero-configuration stack for TDengine + Grafana by [docker-compose](https://docs.docker.com/compose/) and [Grafana provisioning](https://grafana.com/docs/grafana/latest/administration/provisioning/) file: +You can setup a zero-configuration stack for TDengine + Grafana by [docker-compose](https://docs.docker.com/compose/) and [Grafana provisioning](https://grafana.com/docs/grafana/latest/administration/provisioning/) file: 1. Save the provisioning configuration file to `tdengine.yml`. @@ -196,7 +193,7 @@ Go back to the main interface to create a dashboard and click Add Query to enter As shown above, select the `TDengine` data source in the `Query` and enter the corresponding SQL in the query box below for query. -- INPUT SQL: enter the statement to be queried (the result set of the SQL statement should be two columns and multiple rows), for example: `select avg(mem_system) from log.dn where ts >= $from and ts < $to interval($interval)`, where, from, to and interval are built-in variables of the TDengine plugin, indicating the range and time interval of queries fetched from the Grafana plugin panel. In addition to the built-in variables, custom template variables are also supported. +- INPUT SQL: Enter the desired query (the results being two columns and multiple rows), such as `select _wstart, avg(mem_system) from log.dnodes_info where ts >= $from and ts < $to interval($interval)`. In this statement, $from, $to, and $interval are variables that Grafana replaces with the query time range and interval. In addition to the built-in variables, custom template variables are also supported. - ALIAS BY: This allows you to set the current query alias. - GENERATE SQL: Clicking this button will automatically replace the corresponding variables and generate the final executed statement. @@ -208,7 +205,11 @@ Follow the default prompt to query the average system memory usage for the speci ### Importing the Dashboard -You can install TDinsight dashboard in data source configuration page (like `http://localhost:3000/datasources/edit/1/dashboards`) as a monitoring visualization tool for TDengine cluster. The dashboard is published in Grafana as [Dashboard 15167 - TDinsight](https://grafana.com/grafana/dashboards/15167). Check the [TDinsight User Manual](/reference/tdinsight/) for the details. +You can install TDinsight dashboard in data source configuration page (like `http://localhost:3000/datasources/edit/1/dashboards`) as a monitoring visualization tool for TDengine cluster. Ensure that you use TDinsight for 3.x. + +![TDengine Database Grafana plugine import dashboard](./import_dashboard.webp) + +A dashboard for TDengine 2.x has been published on Grafana: [Dashboard 15167 - TDinsight](https://grafana.com/grafana/dashboards/15167)) 。 Check the [TDinsight User Manual](/reference/tdinsight/) for the details. For more dashboards using TDengine data source, [search here in Grafana](https://grafana.com/grafana/dashboards/?dataSource=tdengine-datasource). Here is a sub list: diff --git a/docs/en/20-third-party/06-statsd.md b/docs/en/20-third-party/06-statsd.md index 1cddbcf63d..32b1bbb97a 100644 --- a/docs/en/20-third-party/06-statsd.md +++ b/docs/en/20-third-party/06-statsd.md @@ -1,6 +1,6 @@ --- sidebar_label: StatsD -title: StatsD writing +title: StatsD Writing --- import StatsD from "../14-reference/_statsd.mdx" @@ -12,8 +12,8 @@ You can write StatsD data to TDengine by simply modifying the configuration file ## Prerequisites To write StatsD data to TDengine requires the following preparations. -- The TDengine cluster has been deployed and is working properly -- taosAdapter is installed and running properly. Please refer to the [taosAdapter manual](/reference/taosadapter) for details. +1. The TDengine cluster is deployed and functioning properly +2. taosAdapter is installed and running properly. Please refer to the taosAdapter manual for details. - StatsD has been installed. To install StatsD, please refer to [official documentation](https://github.com/statsd/statsd) ## Configuration steps @@ -39,8 +39,12 @@ $ echo "foo:1|c" | nc -u -w0 127.0.0.1 8125 Use the TDengine CLI to verify that StatsD data is written to TDengine and can read out correctly. ``` -Welcome to the TDengine shell from Linux, Client Version:3.0.0.0 -Copyright (c) 2022 by TAOS Data, Inc. All rights reserved. +taos> show databases; + name | created_time | ntables | vgroups | replica | quorum | days | keep | cache(MB) | blocks | minrows | maxrows | wallevel | fsync | comp | cachelast | precision | update | status | +==================================================================================================================================================================================================================================================================================== + log | 2022-04-20 07:19:50.260 | 11 | 1 | 1 | 1 | 10 | 3650 | 16 | 6 | 100 | 4096 | 1 | 3000 | 2 | 0 | ms | 0 | ready | + statsd | 2022-04-20 09:54:51.220 | 1 | 1 | 1 | 1 | 10 | 3650 | 16 | 6 | 100 | 4096 | 1 | 3000 | 2 | 0 | ns | 2 | ready | +Query OK, 2 row(s) in set (0.003142s) taos> use statsd; Database changed. diff --git a/docs/en/20-third-party/09-emq-broker.md b/docs/en/20-third-party/09-emq-broker.md index 0900dd3d75..2ead1bbaf4 100644 --- a/docs/en/20-third-party/09-emq-broker.md +++ b/docs/en/20-third-party/09-emq-broker.md @@ -9,7 +9,7 @@ MQTT is a popular IoT data transfer protocol. [EMQX](https://github.com/emqx/emq The following preparations are required for EMQX to add TDengine data sources correctly. - The TDengine cluster is deployed and working properly -- taosAdapter is installed and running properly. Please refer to the [taosAdapter manual](/reference/taosadapter) for details. +- taosAdapter is installed and running properly. Please refer to the [taosAdapter manual](../../reference/taosadapter) for details. - If you use the emulated writers described later, you need to install the appropriate version of Node.js. V12 is recommended. ## Install and start EMQX @@ -28,8 +28,6 @@ USE test; CREATE TABLE sensor_data (ts TIMESTAMP, temperature FLOAT, humidity FLOAT, volume FLOAT, pm10 FLOAT, pm25 FLOAT, so2 FLOAT, no2 FLOAT, co FLOAT, sensor_id NCHAR(255), area TINYINT, coll_time TIMESTAMP); ``` -Note: The table schema is based on the blog [(In Chinese) Data Transfer, Storage, Presentation, EMQX + TDengine Build MQTT IoT Data Visualization Platform](https://www.taosdata.com/blog/2020/08/04/1722.html) as an example. Subsequent operations are carried out with this blog scenario too. Please modify it according to your actual application scenario. - ## Configuring EMQX Rules Since the configuration interface of EMQX differs from version to version, here is v4.4.5 as an example. For other versions, please refer to the corresponding official documentation. @@ -137,5 +135,5 @@ Use the TDengine CLI program to log in and query the appropriate databases and t ![TDengine Database EMQX result in taos](./emqx/check-result-in-taos.webp) -Please refer to the [TDengine official documentation](https://docs.taosdata.com/) for more details on how to use TDengine. +Please refer to the [TDengine official documentation](https://docs.tdengine.com/) for more details on how to use TDengine. EMQX Please refer to the [EMQX official documentation](https://www.emqx.io/docs/en/v4.4/rule/rule-engine.html) for details on how to use EMQX. diff --git a/docs/en/20-third-party/10-hive-mq-broker.md b/docs/en/20-third-party/10-hive-mq-broker.md index 333e00fa0e..828a62ac5b 100644 --- a/docs/en/20-third-party/10-hive-mq-broker.md +++ b/docs/en/20-third-party/10-hive-mq-broker.md @@ -1,6 +1,6 @@ --- sidebar_label: HiveMQ Broker -title: HiveMQ Broker writing +title: HiveMQ Broker Writing --- -[HiveMQ](https://www.hivemq.com/) is an MQTT broker that provides community and enterprise editions. HiveMQ is mainly for enterprise emerging machine-to-machine M2M communication and internal transport, meeting scalability, ease of management, and security features. HiveMQ provides an open-source plug-in development kit. MQTT data can be saved to TDengine via TDengine extension for HiveMQ. Please refer to the [HiveMQ extension - TDengine documentation](https://github.com/huskar-t/hivemq-tdengine-extension/blob/b62a26ecc164a310104df57691691b237e091c89/README_EN.md) for details on how to use it. \ No newline at end of file +[HiveMQ](https://www.hivemq.com/) is an MQTT broker that provides community and enterprise editions. HiveMQ is mainly for enterprise emerging machine-to-machine M2M communication and internal transport, meeting scalability, ease of management, and security features. HiveMQ provides an open-source plug-in development kit. MQTT data can be saved to TDengine via TDengine extension for HiveMQ. For more information, see [HiveMQ TDengine Extension](https://github.com/huskar-t/hivemq-tdengine-extension/blob/b62a26ecc164a310104df57691691b237e091c89/README_EN.md). diff --git a/docs/en/20-third-party/import_dashboard.webp b/docs/en/20-third-party/import_dashboard.webp new file mode 100644 index 0000000000..164e3f4690 Binary files /dev/null and b/docs/en/20-third-party/import_dashboard.webp differ diff --git a/docs/en/20-third-party/import_dashboard1.webp b/docs/en/20-third-party/import_dashboard1.webp new file mode 100644 index 0000000000..d4fb374ce8 Binary files /dev/null and b/docs/en/20-third-party/import_dashboard1.webp differ diff --git a/docs/en/20-third-party/import_dashboard2.webp b/docs/en/20-third-party/import_dashboard2.webp new file mode 100644 index 0000000000..9f74dc96be Binary files /dev/null and b/docs/en/20-third-party/import_dashboard2.webp differ diff --git a/docs/en/21-tdinternal/01-arch.md b/docs/en/21-tdinternal/01-arch.md index 44651c0496..2f876adffc 100644 --- a/docs/en/21-tdinternal/01-arch.md +++ b/docs/en/21-tdinternal/01-arch.md @@ -12,6 +12,7 @@ The design of TDengine is based on the assumption that any hardware or software Logical structure diagram of TDengine's distributed architecture is as follows: ![TDengine Database architecture diagram](structure.webp) +
Figure 1: TDengine architecture diagram
A complete TDengine system runs on one or more physical nodes. Logically, it includes data node (dnode), TDengine client driver (TAOSC) and application (app). There are one or more data nodes in the system, which form a cluster. The application interacts with the TDengine cluster through TAOSC's API. The following is a brief introduction to each logical unit. @@ -38,15 +39,16 @@ A complete TDengine system runs on one or more physical nodes. Logically, it inc **Cluster external connection**: TDengine cluster can accommodate a single, multiple or even thousands of data nodes. The application only needs to initiate a connection to any data node in the cluster. The network parameter required for connection is the End Point (FQDN plus configured port number) of a data node. When starting the application taos through CLI, the FQDN of the data node can be specified through the option `-h`, and the configured port number can be specified through `-p`. If the port is not configured, the system configuration parameter “serverPort” of TDengine will be adopted. -**Inter-cluster communication**: Data nodes connect with each other through TCP/UDP. When a data node starts, it will obtain the EP information of the dnode where the mnode is located, and then establish a connection with the mnode in the system to exchange information. There are three steps to obtain EP information of the mnode: +**Inter-cluster communication**: Data nodes connect with each other through TCP/UDP. When a data node starts, it will obtain the EP information of the dnode where the mnode is located, and then establish a connection with the mnode in the system to exchange information. There are three steps to obtain EP information of the mnode: -1. Check whether the mnodeEpList file exists, if it does not exist or cannot be opened normally to obtain EP information of the mnode, skip to the second step; +1. Check whether the mnodeEpList file exists, if it does not exist or cannot be opened normally to obtain EP information of the mnode, skip to the second step; 2. Check the system configuration file taos.cfg to obtain node configuration parameters “firstEp” and “secondEp” (the node specified by these two parameters can be a normal node without mnode, in this case, the node will try to redirect to the mnode node when connected). If these two configuration parameters do not exist or do not exist in taos.cfg, or are invalid, skip to the third step; 3. Set your own EP as a mnode EP and run it independently. After obtaining the mnode EP list, the data node initiates the connection. It will successfully join the working cluster after connection. If not successful, it will try the next item in the mnode EP list. If all attempts are made, but the connection still fails, sleep for a few seconds before trying again. **The choice of MNODE**: TDengine logically has a management node, but there is no separate execution code. The server-side only has one set of execution code, taosd. So which data node will be the management node? This is determined automatically by the system without any manual intervention. The principle is as follows: when a data node starts, it will check its End Point and compare it with the obtained mnode EP List. If its EP exists in it, the data node shall start the mnode module and become a mnode. If your own EP is not in the mnode EP List, the mnode module will not start. During the system operation, due to load balancing, downtime and other reasons, mnode may migrate to the new dnode, totally transparently and without manual intervention. The modification of configuration parameters is the decision made by mnode itself according to resources usage. -**Add new data nodes:** After the system has a data node, it has become a working system. There are two steps to add a new node into the cluster. +**Add new data nodes:** After the system has a data node, it has become a working system. There are two steps to add a new node into the cluster. + - Step1: Connect to the existing working data node using TDengine CLI, and then add the End Point of the new data node with the command "create dnode" - Step 2: In the system configuration parameter file taos.cfg of the new data node, set the “firstEp” and “secondEp” parameters to the EP of any two data nodes in the existing cluster. Please refer to the user tutorial for detailed steps. In this way, the cluster will be established step by step. @@ -57,6 +59,7 @@ A complete TDengine system runs on one or more physical nodes. Logically, it inc To explain the relationship between vnode, mnode, TAOSC and application and their respective roles, the following is an analysis of a typical data writing process. ![typical process of TDengine Database](message.webp) +
Figure 2: Typical process of TDengine
1. Application initiates a request to insert data through JDBC, ODBC, or other APIs. @@ -121,16 +124,17 @@ The load balancing process does not require any manual intervention, and it is t If a database has N replicas, a virtual node group has N virtual nodes. But only one is the Leader and all others are slaves. When the application writes a new record to system, only the Leader vnode can accept the writing request. If a follower vnode receives a writing request, the system will notifies TAOSC to redirect. -### Leader vnode Writing Process +### Leader vnode Writing Process Leader Vnode uses a writing process as follows: ![TDengine Database Leader Writing Process](write_master.webp) +
Figure 3: TDengine Leader writing process
1. Leader vnode receives the application data insertion request, verifies, and moves to next step; 2. If the system configuration parameter `“walLevel”` is greater than 0, vnode will write the original request packet into database log file WAL. If walLevel is set to 2 and fsync is set to 0, TDengine will make WAL data written immediately to ensure that even system goes down, all data can be recovered from database log file; -3. If there are multiple replicas, vnode will forward data packet to follower vnodes in the same virtual node group, and the forwarded packet has a version number with data; +3. If there are multiple replicas, vnode will forward data packet to follower vnodes in the same virtual node group, and the forwarded packet has a version number with data; 4. Write into memory and add the record to “skip list”; 5. Leader vnode returns a confirmation message to the application, indicating a successful write. 6. If any of Step 2, 3 or 4 fails, the error will directly return to the application. @@ -140,6 +144,7 @@ Leader Vnode uses a writing process as follows: For a follower vnode, the write process as follows: ![TDengine Database Follower Writing Process](write_slave.webp) +
Figure 4: TDengine Follower Writing Process
1. Follower vnode receives a data insertion request forwarded by Leader vnode; @@ -212,6 +217,7 @@ When data is written to disk, the system decideswhether to compress the data bas By default, TDengine saves all data in /var/lib/taos directory, and the data files of each vnode are saved in a different directory under this directory. In order to expand the storage space, minimize the bottleneck of file reading and improve the data throughput rate, TDengine can configure the system parameter “dataDir” to allow multiple mounted hard disks to be used by system at the same time. In addition, TDengine also provides the function of tiered data storage, i.e. storage on different storage media according to the time stamps of data files. For example, the latest data is stored on SSD, the data older than a week is stored on local hard disk, and data older than four weeks is stored on network storage device. This reduces storage costs and ensures efficient data access. The movement of data on different storage media is automatically done by the system and is completely transparent to applications. Tiered storage of data is also configured through the system parameter “dataDir”. dataDir format is as follows: + ``` dataDir data_path [tier_level] ``` @@ -270,6 +276,7 @@ For the data collected by device D1001, the number of records per hour is counte TDengine creates a separate table for each data collection point, but in practical applications, it is often necessary to aggregate data from different data collection points. In order to perform aggregation operations efficiently, TDengine introduces the concept of STable (super table). STable is used to represent a specific type of data collection point. It is a table set containing multiple tables. The schema of each table in the set is the same, but each table has its own static tag. There can be multiple tags which can be added, deleted and modified at any time. Applications can aggregate or statistically operate on all or a subset of tables under a STABLE by specifying tag filters. This greatly simplifies the development of applications. The process is shown in the following figure: ![TDengine Database Diagram of multi-table aggregation query](multi_tables.webp) +
Figure 5: Diagram of multi-table aggregation query
1. Application sends a query condition to system; @@ -279,9 +286,8 @@ TDengine creates a separate table for each data collection point, but in practic 5. Each vnode first finds the set of tables within its own node that meet the tag filters from memory, then scans the stored time-series data, completes corresponding aggregation calculations, and returns result to TAOSC; 6. TAOSC finally aggregates the results returned by multiple data nodes and send them back to application. -Since TDengine stores tag data and time-series data separately in vnode, by filtering tag data in memory, the set of tables that need to participate in aggregation operation is first found, which reduces the volume of data to be scanned and improves aggregation speed. At the same time, because the data is distributed in multiple vnodes/dnodes, the aggregation operation is carried out concurrently in multiple vnodes, which further improves the aggregation speed. Aggregation functions for ordinary tables and most operations are applicable to STables. The syntax is exactly the same. Please see TAOS SQL for details. +Since TDengine stores tag data and time-series data separately in vnode, by filtering tag data in memory, the set of tables that need to participate in aggregation operation is first found, which reduces the volume of data to be scanned and improves aggregation speed. At the same time, because the data is distributed in multiple vnodes/dnodes, the aggregation operation is carried out concurrently in multiple vnodes, which further improves the aggregation speed. Aggregation functions for ordinary tables and most operations are applicable to STables. The syntax is exactly the same. Please see TDengine SQL for details. ### Precomputation In order to effectively improve the performance of query processing, based-on the unchangeable feature of IoT data, statistical information of data stored in data block is recorded in the head of data block, including max value, min value, and sum. We call it a precomputing unit. If the query processing involves all the data of a whole data block, the pre-calculated results are directly used, and no need to read the data block contents at all. Since the amount of pre-calculated data is much smaller than the actual size of data block stored on disk, for query processing with disk IO as bottleneck, the use of pre-calculated results can greatly reduce the pressure of reading IO and accelerate the query process. The precomputation mechanism is similar to the BRIN (Block Range Index) of PostgreSQL. - diff --git a/docs/en/25-application/03-immigrate.md b/docs/en/25-application/03-immigrate.md index 9614574c71..1aabaa43e7 100644 --- a/docs/en/25-application/03-immigrate.md +++ b/docs/en/25-application/03-immigrate.md @@ -41,7 +41,7 @@ The agents deployed in the application nodes are responsible for providing opera - **TDengine installation and deployment** -First of all, please install TDengine. Download the latest stable version of TDengine from the official website and install it. For help with using various installation packages, please refer to the blog ["Installation and Uninstallation of TDengine Multiple Installation Packages"](https://www.taosdata.com/blog/2019/08/09/566.html). +First of all, please install TDengine. Download the latest stable version of TDengine from the official website and install it. For help with using various installation packages, please refer to [Install TDengine](../../get-started/package) Note that once the installation is complete, do not start the `taosd` service before properly configuring the parameters. @@ -51,7 +51,7 @@ TDengine version 2.4 and later version includes `taosAdapter`. taosAdapter is a Users can flexibly deploy taosAdapter instances, based on their requirements, to improve data writing throughput and provide guarantees for data writes in different application scenarios. -Through taosAdapter, users can directly write the data collected by `collectd` or `StatsD` to TDengine to achieve easy, convenient and seamless migration in application scenarios. taosAdapter also supports Telegraf, Icinga, TCollector, and node_exporter data. For more details, please refer to [taosAdapter](/reference/taosadapter/). +Through taosAdapter, users can directly write the data collected by `collectd` or `StatsD` to TDengine to achieve easy, convenient and seamless migration in application scenarios. taosAdapter also supports Telegraf, Icinga, TCollector, and node_exporter data. For more details, please refer to [taosAdapter](../../reference/taosadapter/). If using collectd, modify the configuration file in its default location `/etc/collectd/collectd.conf` to point to the IP address and port of the node where to deploy taosAdapter. For example, assuming the taosAdapter IP address is 192.168.1.130 and port 6046, configure it as follows. @@ -411,7 +411,7 @@ TDengine provides a wealth of help documents to explain many aspects of cluster ### Cluster Deployment -The first is TDengine installation. Download the latest stable version of TDengine from the official website, and install it. Please refer to the blog ["Installation and Uninstallation of Various Installation Packages of TDengine"](https://www.taosdata.com/blog/2019/08/09/566.html) for the various installation package formats. +The first is TDengine installation. Download the latest stable version of TDengine from the official website, and install it. Please refer to [Install TDengine](../../get-started/package) for more details. Note that once the installation is complete, do not immediately start the `taosd` service, but start it after correctly configuring the parameters. diff --git a/docs/en/27-train-faq/01-faq.md b/docs/en/27-train-faq/01-faq.md index c10bca1d05..733b418474 100644 --- a/docs/en/27-train-faq/01-faq.md +++ b/docs/en/27-train-faq/01-faq.md @@ -1,114 +1,163 @@ --- -sidebar_label: FAQ title: Frequently Asked Questions --- ## Submit an Issue -If the tips in FAQ don't help much, please submit an issue on [GitHub](https://github.com/taosdata/TDengine) to describe your problem. In your description please include the TDengine version, hardware and OS information, the steps to reproduce the problem and any other relevant information. It would be very helpful if you can package the contents in `/var/log/taos` and `/etc/taos` and upload. These two are the default directories used by TDengine. If you have changed the default directories in your configuration, please package the files in your configured directories. We recommended setting `debugFlag` to 135 in `taos.cfg`, restarting `taosd`, then reproducing the problem and collecting the logs. If you don't want to restart, an alternative way of setting `debugFlag` is executing `alter dnode debugFlag 135` command in TDengine CLI `taos`. During normal running, however, please make sure `debugFlag` is set to 131. +If your issue could not be resolved by reviewing this documentation, you can submit your issue on GitHub and receive support from the TDengine Team. When you submit an issue, attach the following directories from your TDengine deployment: + +1. The directory containing TDengine logs (`/var/log/taos` by default) +2. The directory containing TDengine configuration files (`/etc/taos` by default) + +In your GitHub issue, provide the version of TDengine and the operating system and environment for your deployment, the operations that you performed when the issue occurred, and the time of occurrence and affected tables. + +To obtain more debugging information, open `taos.cfg` and set the `debugFlag` parameter to `135`. Then restart TDengine Server and reproduce the issue. The debug-level logs generated help the TDengine Team to resolve your issue. If it is not possible to restart TDengine Server, you can run the following command in the TDengine CLI to set the debug flag: + +``` + alter dnode 'debugFlag' '135'; +``` + +You can run the `SHOW DNODES` command to determine the dnode ID. + +When debugging information is no longer needed, set `debugFlag` to 131. ## Frequently Asked Questions -### 1. How to upgrade to TDengine 2.0 from older version? +### 1. What are the best practices for upgrading a previous version of TDengine to version 3.0? -version 2.x is not compatible with version 1.x. With regard to the configuration and data files, please perform the following steps before upgrading. Please follow data integrity, security, backup and other relevant SOPs, best practices before removing/deleting any data. +TDengine 3.0 is not compatible with the configuration and data files from previous versions. Before upgrading, perform the following steps: -1. Delete configuration files: `sudo rm -rf /etc/taos/taos.cfg` -2. Delete log files: `sudo rm -rf /var/log/taos/` -3. Delete data files if the data doesn't need to be kept: `sudo rm -rf /var/lib/taos/` -4. Install latest 2.x version -5. If the data needs to be kept and migrated to newer version, please contact professional service at TDengine for assistance. +1. Run `sudo rm -rf /etc/taos/taos.cfg` to delete your configuration file. +2. Run `sudo rm -rf /var/log/taos/` to delete your log files. +3. Run `sudo rm -rf /var/lib/taos/` to delete your data files. +4. Install TDengine 3.0. +5. For assistance in migrating data to TDengine 3.0, contact [TDengine Support](https://tdengine.com/support). -### 2. How to handle "Unable to establish connection"? +### 4. How can I resolve the "Unable to establish connection" error? -When the client is unable to connect to the server, you can try the following ways to troubleshoot and resolve the problem. +This error indicates that the client could not connect to the server. Perform the following troubleshooting steps: -1. Check the network +1. Check the network. - - Check if the hosts where the client and server are running are accessible to each other, for example by `ping` command. - - Check if the TCP/UDP on port 6030-6042 are open for access if firewall is enabled. If possible, disable the firewall for diagnostics, but please ensure that you are following security and other relevant protocols. - - Check if the FQDN and serverPort are configured correctly in `taos.cfg` used by the server side. - - Check if the `firstEp` is set properly in the `taos.cfg` used by the client side. + - For machines deployed in the cloud, verify that your security group can access ports 6030 and 6031 (TCP and UDP). + - For virtual machines deployed locally, verify that the hosts where the client and server are running are accessible to each other. Do not use localhost as the hostname. + - For machines deployed on a corporate network, verify that your NAT configuration allows the server to respond to the client. -2. Make sure the client version and server version are same. +2. Verify that the client and server are running the same version of TDengine. -3. On server side, check the running status of `taosd` by executing `systemctl status taosd` . If your server is started using another way instead of `systemctl`, use the proper method to check whether the server process is running normally. +3. On the server, run `systemctl status taosd` to verify that taosd is running normally. If taosd is stopped, run `systemctl start taosd`. -4. If using connector of Python, Java, Go, Rust, C#, node.JS on Linux to connect to the server, please make sure `libtaos.so` is in directory `/usr/local/taos/driver` and `/usr/local/taos/driver` is in system lib search environment variable `LD_LIBRARY_PATH`. +4. Verify that the client is configured with the correct FQDN for the server. -5. If using connector on Windows, please make sure `C:\TDengine\driver\taos.dll` is in your system lib search path. We recommend putting `taos.dll` under `C:\Windows\System32`. +5. If the server cannot be reached with the `ping` command, verify that network and DNS or hosts file settings are correct. For a TDengine cluster, the client must be able to ping the FQDN of every node in the cluster. -6. Some advanced network diagnostics tools +6. Verify that your firewall settings allow all hosts in the cluster to communicate on ports 6030 and 6041 (TCP and UDP). You can run `ufw status` (Ubuntu) or `firewall-cmd --list-port` (CentOS) to check the configuration. - - On Linux system tool `nc` can be used to check whether the TCP/UDP can be accessible on a specified port - Check whether a UDP port is open: `nc -vuz {hostIP} {port} ` - Check whether a TCP port on server side is open: `nc -l {port}` - Check whether a TCP port on client side is open: `nc {hostIP} {port}` +7. If you are using the Python, Java, Go, Rust, C#, or Node.js connector on Linux to connect to the server, verify that `libtaos.so` is in the `/usr/local/taos/driver` directory and `/usr/local/taos/driver` is in the `LD_LIBRARY_PATH` environment variable. - - On Windows system `Test-NetConnection -ComputerName {fqdn} -Port {port}` on PowerShell can be used to check whether the port on server side is open for access. +8. If you are using Windows, verify that `C:\TDengine\driver\taos.dll` is in the `PATH` environment variable. If possible, move `taos.dll` to the `C:\Windows\System32` directory. -7. TDengine CLI `taos` can also be used to check network, please refer to [TDengine CLI](/reference/taos-shell). +9. On Linux systems, you can use the `nc` tool to check whether a port is accessible: + - To check whether a UDP port is open, run `nc -vuz {hostIP} {port}`. + - To check whether a TCP port on the server side is open, run `nc -l {port}`. + - To check whether a TCP port on client side is open, run `nc {hostIP} {port}`. -### 3. How to handle "Unexpected generic error in RPC" or "Unable to resolve FQDN" ? +10. On Windows systems, you can run `Test-NetConnection -ComputerName {fqdn} -Port {port}` in PowerShell to check whether a port on the server side is accessible. -This error is caused because the FQDN can't be resolved. Please try following ways: +11. You can also use the TDengine CLI to diagnose network issues. For more information, see [Problem Diagnostics](https://docs.tdengine.com/operation/diagnose/). -1. Check whether the FQDN is configured properly on the server side -2. If DSN server is configured in the network, please check whether it works; otherwise, check `/etc/hosts` to see whether the FQDN is configured with correct IP -3. If the network configuration on the server side is OK, try to ping the server from the client side. -4. If TDengine has been used before with an old hostname then the hostname has been changed, please check `/var/lib/taos/taos/dnode/dnodeEps.json`. Before setting up a new TDengine cluster, it's better to cleanup the directories configured. +### 5. How can I resolve the "Unable to resolve FQDN" error? -### 4. "Invalid SQL" is returned even though the Syntax is correct +Clients and dnodes must be able to resolve the FQDN of each required node. You can confirm your configuration as follows: -"Invalid SQL" is returned when the length of SQL statement exceeds maximum allowed length or the syntax is not correct. +1. Verify that the FQDN is configured properly on the server. +2. If your network has a DNS server, verify that it is operational. +3. If your network does not have a DNS server, verify that the FQDNs in the `hosts` file are correct. +4. On the client, use the `ping` command to test your connection to the server. If you cannot ping an FQDN, TDengine cannot reach it. +5. If TDengine has been previously installed and the `hostname` was modified, open `dnode.json` in the `data` folder and verify that the endpoint configuration is correct. The default location of the dnode file is `/var/lib/taos/dnode`. Ensure that you clean up previous installations before reinstalling TDengine. +6. Confirm whether FQDNs are preconfigured in `/etc/hosts` and `/etc/hostname`. -### 5. Whether validation queries are supported? +### 6. What is the most effective way to write data to TDengine? -It's suggested to use a builtin database named as `log` to monitor. +Writing data in batches provides higher efficiency in most situations. You can insert one or more data records into one or more tables in a single SQL statement. - +### 9. Why are table names not fully displayed? -### 6. Can I delete a record? +The number of columns in the TDengine CLI terminal display is limited. This can cause table names to be cut off, and if you use an incomplete name in a statement, the "Table does not exist" error will occur. You can increase the display size with the `maxBinaryDisplayWidth` parameter or the SQL statement `set max_binary_display_width`. You can also append `\G` to your SQL statement to bypass this limitation. -From version 2.6.0.0 Enterprise version, deleting data can be supported. +### 10. How can I migrate data? -### 7. How to create a table of over 1024 columns? +In TDengine, the `hostname` uniquely identifies a machine. When you move data files to a new machine, you must configure the new machine to have the same `host name` as the original machine. -From version 2.1.7.0, at most 4096 columns can be defined for a table. +:::note -### 8. How to improve the efficiency of inserting data? +The data structure of previous versions of TDengine is not compatible with version 3.0. To migrate from TDengine 1.x or 2.x to 3.0, you must export data from your older deployment and import it back into TDengine 3.0. -Inserting data in batch is a good practice. Single SQL statement can insert data for one or multiple tables in batch. +::: -### 9. JDBC Error: the executed SQL is not a DML or a DDL? +### 11. How can I temporary change the log level from the TDengine Client? -Please upgrade to latest JDBC driver, for details please refer to [Java Connector](/reference/connector/java) - -### 10. Failed to connect with error "invalid timestamp" - -The most common reason is that the time setting is not aligned on the client side and the server side. On Linux system, please use `ntpdate` command. On Windows system, please enable automatic sync in system time setting. - -### 11. Table name is not shown in full - -There is a display width setting in TDengine CLI `taos`. It can be controlled by configuration parameter `maxBinaryDisplayWidth`, or can be set using SQL command `set max_binary_display_width`. A more convenient way is to append `\G` in a SQL command to bypass this limitation. - -### 12. How to change log level temporarily? - -Below SQL command can be used to adjust log level temporarily +To change the log level for debugging purposes, you can use the following command: ```sql -ALTER LOCAL flag_name flag_value; +ALTER LOCAL local_option + +local_option: { + 'resetLog' + | 'rpcDebugFlag' value + | 'tmrDebugFlag' value + | 'cDebugFlag' value + | 'uDebugFlag' value + | 'debugFlag' value +} ``` - - flag_name can be: debugFlag,cDebugFlag,tmrDebugFlag,uDebugFlag,rpcDebugFlag - - flag_value can be: 131 (INFO/WARNING/ERROR), 135 (plus DEBUG), 143 (plus TRACE) - +Use `resetlog` to remove all logs generated on the local client. Use the other parameters to specify a log level for a specific component. -### 13. What to do if go compilation fails? +For each parameter, you can set the value to `131` (error and warning), `135` (error, warning, and debug), or `143` (error, warning, debug, and trace). -From version 2.3.0.0, a new component named `taosAdapter` is introduced. Its' developed in Go. If you want to compile from source code and meet go compilation problems, try to do below steps to resolve Go environment problems. +### Why do TDengine components written in Go fail to compile? -```sh -go env -w GO111MODULE=on -go env -w GOPROXY=https://goproxy.cn,direct -``` +TDengine includes taosAdapter, an independent component written in Go. This component provides the REST API as well as data access for other products such as Prometheus and Telegraf. +When using the develop branch, you must run `git submodule update --init --recursive` to download the taosAdapter repository and then compile it. + +TDengine Go components require Go version 1.14 or later. + +### 13. How can I query the storage space being used by my data? + +The TDengine data files are stored in `/var/lib/taos` by default. Log files are stored in `/var/log/taos`. + +To see how much space your data files occupy, run `du -sh /var/lib/taos/vnode --exclude='wal'`. This excludes the write-ahead log (WAL) because its size is relatively fixed while writes are occurring, and it is written to disk and cleared when you shut down TDengine. + +If you want to see how much space is occupied by a single database, first determine which vgroup is storing the database by running `show vgroups`. Then check `/var/lib/taos/vnode` for the files associated with the vgroup ID. + +### 15. How is timezone information processed for timestamps? + +TDengine uses the timezone of the client for timestamps. The server timezone does not affect timestamps. The client converts Unix timestamps in SQL statements to UTC before sending them to the server. When you query data on the server, it provides timestamps in UTC to the client, which converts them to its local time. + +Timestamps are processed as follows: + +1. The client uses its system timezone unless it has been configured otherwise. +2. A timezone configured in `taos.cfg` takes precedence over the system timezone. +3. A timezone explicitly specified when establishing a connection to TDengine through a connector takes precedence over `taos.cfg` and the system timezone. For example, the Java connector allows you to specify a timezone in the JDBC URL. +4. If you use an RFC 3339 timestamp (2013-04-12T15:52:01.123+08:00), or an ISO 8601 timestamp (2013-04-12T15:52:01.123+0800), the timezone specified in the timestamp is used instead of the timestamps configured using any other method. + +### 16. Which network ports are required by TDengine? + +See [serverPort](https://docs.tdengine.com/reference/config/#serverport) in Configuration Parameters. + +Note that ports are specified using 6030 as the default first port. If you change this port, all other ports change as well. + +### 17. Why do applications such as Grafana fail to connect to TDengine over the REST API? + +In TDengine, the REST API is provided by taosAdapter. Ensure that taosAdapter is running before you connect an application to TDengine over the REST API. You can run `systemctl start taosadapter` to start the service. + +Note that the log path for taosAdapter must be configured separately. The default path is `/var/log/taos`. You can choose one of eight log levels. The default is `info`. You can set the log level to `panic` to disable log output. You can modify the taosAdapter configuration file to change these settings. The default location is `/etc/taos/taosadapter.toml`. + +For more information, see [taosAdapter](https://docs.tdengine.com/reference/taosadapter/). + +### 18. How can I resolve out-of-memory (OOM) errors? + +OOM errors are thrown by the operating system when its memory, including swap, becomes insufficient and it needs to terminate processes to remain operational. Most OOM errors in TDengine occur for one of the following reasons: free memory is less than the value of `vm.min_free_kbytes` or free memory is less than the size of the request. If TDengine occupies reserved memory, an OOM error can occur even when free memory is sufficient. + +TDengine preallocates memory to each vnode. The number of vnodes per database is determined by the `vgroups` parameter, and the amount of memory per vnode is determined by the `buffer` parameter. To prevent OOM errors from occurring, ensure that you prepare sufficient memory on your hosts to support the number of vnodes that your deployment requires. Configure an appropriately sized swap space. If you continue to receive OOM errors, your SQL statements may be querying too much data for your system. TDengine Enterprise Edition includes optimized memory management that increases stability for enterprise customers. diff --git a/docs/examples/java/src/main/java/com/taos/example/RestInsertExample.java b/docs/examples/java/src/main/java/com/taos/example/RestInsertExample.java index af97fe4373..9d85bf2a94 100644 --- a/docs/examples/java/src/main/java/com/taos/example/RestInsertExample.java +++ b/docs/examples/java/src/main/java/com/taos/example/RestInsertExample.java @@ -16,14 +16,14 @@ public class RestInsertExample { private static List getRawData() { return Arrays.asList( - "d1001,2018-10-03 14:38:05.000,10.30000,219,0.31000,California.SanFrancisco,2", - "d1001,2018-10-03 14:38:15.000,12.60000,218,0.33000,California.SanFrancisco,2", - "d1001,2018-10-03 14:38:16.800,12.30000,221,0.31000,California.SanFrancisco,2", - "d1002,2018-10-03 14:38:16.650,10.30000,218,0.25000,California.SanFrancisco,3", - "d1003,2018-10-03 14:38:05.500,11.80000,221,0.28000,California.LosAngeles,2", - "d1003,2018-10-03 14:38:16.600,13.40000,223,0.29000,California.LosAngeles,2", - "d1004,2018-10-03 14:38:05.000,10.80000,223,0.29000,California.LosAngeles,3", - "d1004,2018-10-03 14:38:06.500,11.50000,221,0.35000,California.LosAngeles,3" + "d1001,2018-10-03 14:38:05.000,10.30000,219,0.31000,'California.SanFrancisco',2", + "d1001,2018-10-03 14:38:15.000,12.60000,218,0.33000,'California.SanFrancisco',2", + "d1001,2018-10-03 14:38:16.800,12.30000,221,0.31000,'California.SanFrancisco',2", + "d1002,2018-10-03 14:38:16.650,10.30000,218,0.25000,'California.SanFrancisco',3", + "d1003,2018-10-03 14:38:05.500,11.80000,221,0.28000,'California.LosAngeles',2", + "d1003,2018-10-03 14:38:16.600,13.40000,223,0.29000,'California.LosAngeles',2", + "d1004,2018-10-03 14:38:05.000,10.80000,223,0.29000,'California.LosAngeles',3", + "d1004,2018-10-03 14:38:06.500,11.50000,221,0.35000,'California.LosAngeles',3" ); } diff --git a/docs/examples/java/src/main/java/com/taos/example/SubscribeDemo.java b/docs/examples/java/src/main/java/com/taos/example/SubscribeDemo.java index 50e8b35771..179e6e6911 100644 --- a/docs/examples/java/src/main/java/com/taos/example/SubscribeDemo.java +++ b/docs/examples/java/src/main/java/com/taos/example/SubscribeDemo.java @@ -57,7 +57,7 @@ public class SubscribeDemo { properties.setProperty(TMQConstants.ENABLE_AUTO_COMMIT, "true"); properties.setProperty(TMQConstants.GROUP_ID, "test"); properties.setProperty(TMQConstants.VALUE_DESERIALIZER, - "com.taosdata.jdbc.MetersDeserializer"); + "com.taos.example.MetersDeserializer"); // poll data try (TaosConsumer consumer = new TaosConsumer<>(properties)) { diff --git a/docs/examples/java/src/main/java/com/taos/example/highvolume/DataBaseMonitor.java b/docs/examples/java/src/main/java/com/taos/example/highvolume/DataBaseMonitor.java new file mode 100644 index 0000000000..04b149a4b9 --- /dev/null +++ b/docs/examples/java/src/main/java/com/taos/example/highvolume/DataBaseMonitor.java @@ -0,0 +1,63 @@ +package com.taos.example.highvolume; + +import java.sql.*; + +/** + * Prepare target database. + * Count total records in database periodically so that we can estimate the writing speed. + */ +public class DataBaseMonitor { + private Connection conn; + private Statement stmt; + + public DataBaseMonitor init() throws SQLException { + if (conn == null) { + String jdbcURL = System.getenv("TDENGINE_JDBC_URL"); + conn = DriverManager.getConnection(jdbcURL); + stmt = conn.createStatement(); + } + return this; + } + + public void close() { + try { + stmt.close(); + } catch (SQLException e) { + } + try { + conn.close(); + } catch (SQLException e) { + } + } + + public void prepareDatabase() throws SQLException { + stmt.execute("DROP DATABASE IF EXISTS test"); + stmt.execute("CREATE DATABASE test"); + stmt.execute("CREATE STABLE test.meters (ts TIMESTAMP, current FLOAT, voltage INT, phase FLOAT) TAGS (location BINARY(64), groupId INT)"); + } + + public Long count() throws SQLException { + if (!stmt.isClosed()) { + ResultSet result = stmt.executeQuery("SELECT count(*) from test.meters"); + result.next(); + return result.getLong(1); + } + return null; + } + + /** + * show test.stables; + * + * name | created_time | columns | tags | tables | + * ============================================================================================ + * meters | 2022-07-20 08:39:30.902 | 4 | 2 | 620000 | + */ + public Long getTableCount() throws SQLException { + if (!stmt.isClosed()) { + ResultSet result = stmt.executeQuery("show test.stables"); + result.next(); + return result.getLong(5); + } + return null; + } +} \ No newline at end of file diff --git a/docs/examples/java/src/main/java/com/taos/example/highvolume/FastWriteExample.java b/docs/examples/java/src/main/java/com/taos/example/highvolume/FastWriteExample.java new file mode 100644 index 0000000000..41b59551ca --- /dev/null +++ b/docs/examples/java/src/main/java/com/taos/example/highvolume/FastWriteExample.java @@ -0,0 +1,70 @@ +package com.taos.example.highvolume; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.sql.*; +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.ArrayBlockingQueue; +import java.util.concurrent.BlockingQueue; + + +public class FastWriteExample { + final static Logger logger = LoggerFactory.getLogger(FastWriteExample.class); + + final static int taskQueueCapacity = 1000000; + final static List> taskQueues = new ArrayList<>(); + final static List readTasks = new ArrayList<>(); + final static List writeTasks = new ArrayList<>(); + final static DataBaseMonitor databaseMonitor = new DataBaseMonitor(); + + public static void stopAll() { + logger.info("shutting down"); + readTasks.forEach(task -> task.stop()); + writeTasks.forEach(task -> task.stop()); + databaseMonitor.close(); + } + + public static void main(String[] args) throws InterruptedException, SQLException { + int readTaskCount = args.length > 0 ? Integer.parseInt(args[0]) : 1; + int writeTaskCount = args.length > 1 ? Integer.parseInt(args[1]) : 3; + int tableCount = args.length > 2 ? Integer.parseInt(args[2]) : 1000; + int maxBatchSize = args.length > 3 ? Integer.parseInt(args[3]) : 3000; + + logger.info("readTaskCount={}, writeTaskCount={} tableCount={} maxBatchSize={}", + readTaskCount, writeTaskCount, tableCount, maxBatchSize); + + databaseMonitor.init().prepareDatabase(); + + // Create task queues, whiting tasks and start writing threads. + for (int i = 0; i < writeTaskCount; ++i) { + BlockingQueue queue = new ArrayBlockingQueue<>(taskQueueCapacity); + taskQueues.add(queue); + WriteTask task = new WriteTask(queue, maxBatchSize); + Thread t = new Thread(task); + t.setName("WriteThread-" + i); + t.start(); + } + + // create reading tasks and start reading threads + int tableCountPerTask = tableCount / readTaskCount; + for (int i = 0; i < readTaskCount; ++i) { + ReadTask task = new ReadTask(i, taskQueues, tableCountPerTask); + Thread t = new Thread(task); + t.setName("ReadThread-" + i); + t.start(); + } + + Runtime.getRuntime().addShutdownHook(new Thread(FastWriteExample::stopAll)); + + long lastCount = 0; + while (true) { + Thread.sleep(10000); + long numberOfTable = databaseMonitor.getTableCount(); + long count = databaseMonitor.count(); + logger.info("numberOfTable={} count={} speed={}", numberOfTable, count, (count - lastCount) / 10); + lastCount = count; + } + } +} \ No newline at end of file diff --git a/docs/examples/java/src/main/java/com/taos/example/highvolume/MockDataSource.java b/docs/examples/java/src/main/java/com/taos/example/highvolume/MockDataSource.java new file mode 100644 index 0000000000..6fe83f002e --- /dev/null +++ b/docs/examples/java/src/main/java/com/taos/example/highvolume/MockDataSource.java @@ -0,0 +1,53 @@ +package com.taos.example.highvolume; + +import java.util.Iterator; + +/** + * Generate test data + */ +class MockDataSource implements Iterator { + private String tbNamePrefix; + private int tableCount; + private long maxRowsPerTable = 1000000000L; + + // 100 milliseconds between two neighbouring rows. + long startMs = System.currentTimeMillis() - maxRowsPerTable * 100; + private int currentRow = 0; + private int currentTbId = -1; + + // mock values + String[] location = {"LosAngeles", "SanDiego", "Hollywood", "Compton", "San Francisco"}; + float[] current = {8.8f, 10.7f, 9.9f, 8.9f, 9.4f}; + int[] voltage = {119, 116, 111, 113, 118}; + float[] phase = {0.32f, 0.34f, 0.33f, 0.329f, 0.141f}; + + public MockDataSource(String tbNamePrefix, int tableCount) { + this.tbNamePrefix = tbNamePrefix; + this.tableCount = tableCount; + } + + @Override + public boolean hasNext() { + currentTbId += 1; + if (currentTbId == tableCount) { + currentTbId = 0; + currentRow += 1; + } + return currentRow < maxRowsPerTable; + } + + @Override + public String next() { + long ts = startMs + 100 * currentRow; + int groupId = currentTbId % 5 == 0 ? currentTbId / 5 : currentTbId / 5 + 1; + StringBuilder sb = new StringBuilder(tbNamePrefix + "_" + currentTbId + ","); // tbName + sb.append(ts).append(','); // ts + sb.append(current[currentRow % 5]).append(','); // current + sb.append(voltage[currentRow % 5]).append(','); // voltage + sb.append(phase[currentRow % 5]).append(','); // phase + sb.append(location[currentRow % 5]).append(','); // location + sb.append(groupId); // groupID + + return sb.toString(); + } +} \ No newline at end of file diff --git a/docs/examples/java/src/main/java/com/taos/example/highvolume/ReadTask.java b/docs/examples/java/src/main/java/com/taos/example/highvolume/ReadTask.java new file mode 100644 index 0000000000..a6fcfed1d2 --- /dev/null +++ b/docs/examples/java/src/main/java/com/taos/example/highvolume/ReadTask.java @@ -0,0 +1,58 @@ +package com.taos.example.highvolume; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.util.Iterator; +import java.util.List; +import java.util.concurrent.BlockingQueue; + +class ReadTask implements Runnable { + private final static Logger logger = LoggerFactory.getLogger(ReadTask.class); + private final int taskId; + private final List> taskQueues; + private final int queueCount; + private final int tableCount; + private boolean active = true; + + public ReadTask(int readTaskId, List> queues, int tableCount) { + this.taskId = readTaskId; + this.taskQueues = queues; + this.queueCount = queues.size(); + this.tableCount = tableCount; + } + + /** + * Assign data received to different queues. + * Here we use the suffix number in table name. + * You are expected to define your own rule in practice. + * + * @param line record received + * @return which queue to use + */ + public int getQueueId(String line) { + String tbName = line.substring(0, line.indexOf(',')); // For example: tb1_101 + String suffixNumber = tbName.split("_")[1]; + return Integer.parseInt(suffixNumber) % this.queueCount; + } + + @Override + public void run() { + logger.info("started"); + Iterator it = new MockDataSource("tb" + this.taskId, tableCount); + try { + while (it.hasNext() && active) { + String line = it.next(); + int queueId = getQueueId(line); + taskQueues.get(queueId).put(line); + } + } catch (Exception e) { + logger.error("Read Task Error", e); + } + } + + public void stop() { + logger.info("stop"); + this.active = false; + } +} \ No newline at end of file diff --git a/docs/examples/java/src/main/java/com/taos/example/highvolume/SQLWriter.java b/docs/examples/java/src/main/java/com/taos/example/highvolume/SQLWriter.java new file mode 100644 index 0000000000..c2989acdbe --- /dev/null +++ b/docs/examples/java/src/main/java/com/taos/example/highvolume/SQLWriter.java @@ -0,0 +1,205 @@ +package com.taos.example.highvolume; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.sql.*; +import java.util.HashMap; +import java.util.Map; + +/** + * A helper class encapsulate the logic of writing using SQL. + *

+ * The main interfaces are two methods: + *

    + *
  1. {@link SQLWriter#processLine}, which receive raw lines from WriteTask and group them by table names.
  2. + *
  3. {@link SQLWriter#flush}, which assemble INSERT statement and execute it.
  4. + *
+ *

+ * There is a technical skill worth mentioning: we create table as needed when "table does not exist" error occur instead of creating table automatically using syntax "INSET INTO tb USING stb". + * This ensure that checking table existence is a one-time-only operation. + *

+ * + *

+ */ +public class SQLWriter { + final static Logger logger = LoggerFactory.getLogger(SQLWriter.class); + + private Connection conn; + private Statement stmt; + + /** + * current number of buffered records + */ + private int bufferedCount = 0; + /** + * Maximum number of buffered records. + * Flush action will be triggered if bufferedCount reached this value, + */ + private int maxBatchSize; + + + /** + * Maximum SQL length. + */ + private int maxSQLLength; + + /** + * Map from table name to column values. For example: + * "tb001" -> "(1648432611249,2.1,114,0.09) (1648432611250,2.2,135,0.2)" + */ + private Map tbValues = new HashMap<>(); + + /** + * Map from table name to tag values in the same order as creating stable. + * Used for creating table. + */ + private Map tbTags = new HashMap<>(); + + public SQLWriter(int maxBatchSize) { + this.maxBatchSize = maxBatchSize; + } + + + /** + * Get Database Connection + * + * @return Connection + * @throws SQLException + */ + private static Connection getConnection() throws SQLException { + String jdbcURL = System.getenv("TDENGINE_JDBC_URL"); + return DriverManager.getConnection(jdbcURL); + } + + /** + * Create Connection and Statement + * + * @throws SQLException + */ + public void init() throws SQLException { + conn = getConnection(); + stmt = conn.createStatement(); + stmt.execute("use test"); + ResultSet rs = stmt.executeQuery("show variables"); + while (rs.next()) { + String configName = rs.getString(1); + if ("maxSQLLength".equals(configName)) { + maxSQLLength = Integer.parseInt(rs.getString(2)); + logger.info("maxSQLLength={}", maxSQLLength); + } + } + } + + /** + * Convert raw data to SQL fragments, group them by table name and cache them in a HashMap. + * Trigger writing when number of buffered records reached maxBachSize. + * + * @param line raw data get from task queue in format: tbName,ts,current,voltage,phase,location,groupId + */ + public void processLine(String line) throws SQLException { + bufferedCount += 1; + int firstComma = line.indexOf(','); + String tbName = line.substring(0, firstComma); + int lastComma = line.lastIndexOf(','); + int secondLastComma = line.lastIndexOf(',', lastComma - 1); + String value = "(" + line.substring(firstComma + 1, secondLastComma) + ") "; + if (tbValues.containsKey(tbName)) { + tbValues.put(tbName, tbValues.get(tbName) + value); + } else { + tbValues.put(tbName, value); + } + if (!tbTags.containsKey(tbName)) { + String location = line.substring(secondLastComma + 1, lastComma); + String groupId = line.substring(lastComma + 1); + String tagValues = "('" + location + "'," + groupId + ')'; + tbTags.put(tbName, tagValues); + } + if (bufferedCount == maxBatchSize) { + flush(); + } + } + + + /** + * Assemble INSERT statement using buffered SQL fragments in Map {@link SQLWriter#tbValues} and execute it. + * In case of "Table does not exit" exception, create all tables in the sql and retry the sql. + */ + public void flush() throws SQLException { + StringBuilder sb = new StringBuilder("INSERT INTO "); + for (Map.Entry entry : tbValues.entrySet()) { + String tableName = entry.getKey(); + String values = entry.getValue(); + String q = tableName + " values " + values + " "; + if (sb.length() + q.length() > maxSQLLength) { + executeSQL(sb.toString()); + logger.warn("increase maxSQLLength or decrease maxBatchSize to gain better performance"); + sb = new StringBuilder("INSERT INTO "); + } + sb.append(q); + } + executeSQL(sb.toString()); + tbValues.clear(); + bufferedCount = 0; + } + + private void executeSQL(String sql) throws SQLException { + try { + stmt.executeUpdate(sql); + } catch (SQLException e) { + // convert to error code defined in taoserror.h + int errorCode = e.getErrorCode() & 0xffff; + if (errorCode == 0x362 || errorCode == 0x218) { + // Table does not exist + createTables(); + executeSQL(sql); + } else { + logger.error("Execute SQL: {}", sql); + throw e; + } + } catch (Throwable throwable) { + logger.error("Execute SQL: {}", sql); + throw throwable; + } + } + + /** + * Create tables in batch using syntax: + *

+ * CREATE TABLE [IF NOT EXISTS] tb_name1 USING stb_name TAGS (tag_value1, ...) [IF NOT EXISTS] tb_name2 USING stb_name TAGS (tag_value2, ...) ...; + *

+ */ + private void createTables() throws SQLException { + StringBuilder sb = new StringBuilder("CREATE TABLE "); + for (String tbName : tbValues.keySet()) { + String tagValues = tbTags.get(tbName); + sb.append("IF NOT EXISTS ").append(tbName).append(" USING meters TAGS ").append(tagValues).append(" "); + } + String sql = sb.toString(); + try { + stmt.executeUpdate(sql); + } catch (Throwable throwable) { + logger.error("Execute SQL: {}", sql); + throw throwable; + } + } + + public boolean hasBufferedValues() { + return bufferedCount > 0; + } + + public int getBufferedCount() { + return bufferedCount; + } + + public void close() { + try { + stmt.close(); + } catch (SQLException e) { + } + try { + conn.close(); + } catch (SQLException e) { + } + } +} \ No newline at end of file diff --git a/docs/examples/java/src/main/java/com/taos/example/highvolume/StmtWriter.java b/docs/examples/java/src/main/java/com/taos/example/highvolume/StmtWriter.java new file mode 100644 index 0000000000..8ade06625d --- /dev/null +++ b/docs/examples/java/src/main/java/com/taos/example/highvolume/StmtWriter.java @@ -0,0 +1,4 @@ +package com.taos.example.highvolume; + +public class StmtWriter { +} diff --git a/docs/examples/java/src/main/java/com/taos/example/highvolume/WriteTask.java b/docs/examples/java/src/main/java/com/taos/example/highvolume/WriteTask.java new file mode 100644 index 0000000000..de9e5463d7 --- /dev/null +++ b/docs/examples/java/src/main/java/com/taos/example/highvolume/WriteTask.java @@ -0,0 +1,58 @@ +package com.taos.example.highvolume; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.util.concurrent.BlockingQueue; + +class WriteTask implements Runnable { + private final static Logger logger = LoggerFactory.getLogger(WriteTask.class); + private final int maxBatchSize; + + // the queue from which this writing task get raw data. + private final BlockingQueue queue; + + // A flag indicate whether to continue. + private boolean active = true; + + public WriteTask(BlockingQueue taskQueue, int maxBatchSize) { + this.queue = taskQueue; + this.maxBatchSize = maxBatchSize; + } + + @Override + public void run() { + logger.info("started"); + String line = null; // data getting from the queue just now. + SQLWriter writer = new SQLWriter(maxBatchSize); + try { + writer.init(); + while (active) { + line = queue.poll(); + if (line != null) { + // parse raw data and buffer the data. + writer.processLine(line); + } else if (writer.hasBufferedValues()) { + // write data immediately if no more data in the queue + writer.flush(); + } else { + // sleep a while to avoid high CPU usage if no more data in the queue and no buffered records, . + Thread.sleep(100); + } + } + if (writer.hasBufferedValues()) { + writer.flush(); + } + } catch (Exception e) { + String msg = String.format("line=%s, bufferedCount=%s", line, writer.getBufferedCount()); + logger.error(msg, e); + } finally { + writer.close(); + } + } + + public void stop() { + logger.info("stop"); + this.active = false; + } +} \ No newline at end of file diff --git a/docs/examples/java/src/test/java/com/taos/test/TestAll.java b/docs/examples/java/src/test/java/com/taos/test/TestAll.java index 42db24485a..8d201da074 100644 --- a/docs/examples/java/src/test/java/com/taos/test/TestAll.java +++ b/docs/examples/java/src/test/java/com/taos/test/TestAll.java @@ -23,16 +23,16 @@ public class TestAll { String jdbcUrl = "jdbc:TAOS://localhost:6030?user=root&password=taosdata"; try (Connection conn = DriverManager.getConnection(jdbcUrl)) { try (Statement stmt = conn.createStatement()) { - String sql = "INSERT INTO power.d1001 USING power.meters TAGS(California.SanFrancisco, 2) VALUES('2018-10-03 14:38:05.000',10.30000,219,0.31000)\n" + - " power.d1001 USING power.meters TAGS(California.SanFrancisco, 2) VALUES('2018-10-03 15:38:15.000',12.60000,218,0.33000)\n" + - " power.d1001 USING power.meters TAGS(California.SanFrancisco, 2) VALUES('2018-10-03 15:38:16.800',12.30000,221,0.31000)\n" + - " power.d1002 USING power.meters TAGS(California.SanFrancisco, 3) VALUES('2018-10-03 15:38:16.650',10.30000,218,0.25000)\n" + - " power.d1003 USING power.meters TAGS(California.LosAngeles, 2) VALUES('2018-10-03 15:38:05.500',11.80000,221,0.28000)\n" + - " power.d1003 USING power.meters TAGS(California.LosAngeles, 2) VALUES('2018-10-03 15:38:16.600',13.40000,223,0.29000)\n" + - " power.d1004 USING power.meters TAGS(California.LosAngeles, 3) VALUES('2018-10-03 15:38:05.000',10.80000,223,0.29000)\n" + - " power.d1004 USING power.meters TAGS(California.LosAngeles, 3) VALUES('2018-10-03 15:38:06.000',10.80000,223,0.29000)\n" + - " power.d1004 USING power.meters TAGS(California.LosAngeles, 3) VALUES('2018-10-03 15:38:07.000',10.80000,223,0.29000)\n" + - " power.d1004 USING power.meters TAGS(California.LosAngeles, 3) VALUES('2018-10-03 15:38:08.500',11.50000,221,0.35000)"; + String sql = "INSERT INTO power.d1001 USING power.meters TAGS('California.SanFrancisco', 2) VALUES('2018-10-03 14:38:05.000',10.30000,219,0.31000)\n" + + " power.d1001 USING power.meters TAGS('California.SanFrancisco', 2) VALUES('2018-10-03 15:38:15.000',12.60000,218,0.33000)\n" + + " power.d1001 USING power.meters TAGS('California.SanFrancisco', 2) VALUES('2018-10-03 15:38:16.800',12.30000,221,0.31000)\n" + + " power.d1002 USING power.meters TAGS('California.SanFrancisco', 3) VALUES('2018-10-03 15:38:16.650',10.30000,218,0.25000)\n" + + " power.d1003 USING power.meters TAGS('California.LosAngeles', 2) VALUES('2018-10-03 15:38:05.500',11.80000,221,0.28000)\n" + + " power.d1003 USING power.meters TAGS('California.LosAngeles', 2) VALUES('2018-10-03 15:38:16.600',13.40000,223,0.29000)\n" + + " power.d1004 USING power.meters TAGS('California.LosAngeles', 3) VALUES('2018-10-03 15:38:05.000',10.80000,223,0.29000)\n" + + " power.d1004 USING power.meters TAGS('California.LosAngeles', 3) VALUES('2018-10-03 15:38:06.000',10.80000,223,0.29000)\n" + + " power.d1004 USING power.meters TAGS('California.LosAngeles', 3) VALUES('2018-10-03 15:38:07.000',10.80000,223,0.29000)\n" + + " power.d1004 USING power.meters TAGS('California.LosAngeles', 3) VALUES('2018-10-03 15:38:08.500',11.50000,221,0.35000)"; stmt.execute(sql); } diff --git a/docs/examples/python/fast_write_example.py b/docs/examples/python/fast_write_example.py new file mode 100644 index 0000000000..c9d606388f --- /dev/null +++ b/docs/examples/python/fast_write_example.py @@ -0,0 +1,180 @@ +# install dependencies: +# recommend python >= 3.8 +# pip3 install faster-fifo +# + +import logging +import math +import sys +import time +import os +from multiprocessing import Process +from faster_fifo import Queue +from mockdatasource import MockDataSource +from queue import Empty +from typing import List + +logging.basicConfig(stream=sys.stdout, level=logging.DEBUG, format="%(asctime)s [%(name)s] - %(message)s") + +READ_TASK_COUNT = 1 +WRITE_TASK_COUNT = 1 +TABLE_COUNT = 1000 +QUEUE_SIZE = 1000000 +MAX_BATCH_SIZE = 3000 + +read_processes = [] +write_processes = [] + + +def get_connection(): + """ + If variable TDENGINE_FIRST_EP is provided then it will be used. If not, firstEP in /etc/taos/taos.cfg will be used. + You can also override the default username and password by supply variable TDENGINE_USER and TDENGINE_PASSWORD + """ + import taos + firstEP = os.environ.get("TDENGINE_FIRST_EP") + if firstEP: + host, port = firstEP.split(":") + else: + host, port = None, 0 + user = os.environ.get("TDENGINE_USER", "root") + password = os.environ.get("TDENGINE_PASSWORD", "taosdata") + return taos.connect(host=host, port=int(port), user=user, password=password) + + +# ANCHOR: read + +def run_read_task(task_id: int, task_queues: List[Queue]): + table_count_per_task = TABLE_COUNT // READ_TASK_COUNT + data_source = MockDataSource(f"tb{task_id}", table_count_per_task) + try: + for batch in data_source: + for table_id, rows in batch: + # hash data to different queue + i = table_id % len(task_queues) + # block putting forever when the queue is full + task_queues[i].put_many(rows, block=True, timeout=-1) + except KeyboardInterrupt: + pass + + +# ANCHOR_END: read + +# ANCHOR: write +def run_write_task(task_id: int, queue: Queue): + from sql_writer import SQLWriter + log = logging.getLogger(f"WriteTask-{task_id}") + writer = SQLWriter(get_connection) + lines = None + try: + while True: + try: + # get as many as possible + lines = queue.get_many(block=False, max_messages_to_get=MAX_BATCH_SIZE) + writer.process_lines(lines) + except Empty: + time.sleep(0.01) + except KeyboardInterrupt: + pass + except BaseException as e: + log.debug(f"lines={lines}") + raise e + + +# ANCHOR_END: write + +def set_global_config(): + argc = len(sys.argv) + if argc > 1: + global READ_TASK_COUNT + READ_TASK_COUNT = int(sys.argv[1]) + if argc > 2: + global WRITE_TASK_COUNT + WRITE_TASK_COUNT = int(sys.argv[2]) + if argc > 3: + global TABLE_COUNT + TABLE_COUNT = int(sys.argv[3]) + if argc > 4: + global QUEUE_SIZE + QUEUE_SIZE = int(sys.argv[4]) + if argc > 5: + global MAX_BATCH_SIZE + MAX_BATCH_SIZE = int(sys.argv[5]) + + +# ANCHOR: monitor +def run_monitor_process(): + log = logging.getLogger("DataBaseMonitor") + conn = get_connection() + conn.execute("DROP DATABASE IF EXISTS test") + conn.execute("CREATE DATABASE test") + conn.execute("CREATE STABLE test.meters (ts TIMESTAMP, current FLOAT, voltage INT, phase FLOAT) " + "TAGS (location BINARY(64), groupId INT)") + + def get_count(): + res = conn.query("SELECT count(*) FROM test.meters") + rows = res.fetch_all() + return rows[0][0] if rows else 0 + + last_count = 0 + while True: + time.sleep(10) + count = get_count() + log.info(f"count={count} speed={(count - last_count) / 10}") + last_count = count + + +# ANCHOR_END: monitor +# ANCHOR: main +def main(): + set_global_config() + logging.info(f"READ_TASK_COUNT={READ_TASK_COUNT}, WRITE_TASK_COUNT={WRITE_TASK_COUNT}, " + f"TABLE_COUNT={TABLE_COUNT}, QUEUE_SIZE={QUEUE_SIZE}, MAX_BATCH_SIZE={MAX_BATCH_SIZE}") + + monitor_process = Process(target=run_monitor_process) + monitor_process.start() + time.sleep(3) # waiting for database ready. + + task_queues: List[Queue] = [] + # create task queues + for i in range(WRITE_TASK_COUNT): + queue = Queue(max_size_bytes=QUEUE_SIZE) + task_queues.append(queue) + + # create write processes + for i in range(WRITE_TASK_COUNT): + p = Process(target=run_write_task, args=(i, task_queues[i])) + p.start() + logging.debug(f"WriteTask-{i} started with pid {p.pid}") + write_processes.append(p) + + # create read processes + for i in range(READ_TASK_COUNT): + queues = assign_queues(i, task_queues) + p = Process(target=run_read_task, args=(i, queues)) + p.start() + logging.debug(f"ReadTask-{i} started with pid {p.pid}") + read_processes.append(p) + + try: + monitor_process.join() + except KeyboardInterrupt: + monitor_process.terminate() + [p.terminate() for p in read_processes] + [p.terminate() for p in write_processes] + [q.close() for q in task_queues] + + +def assign_queues(read_task_id, task_queues): + """ + Compute target queues for a specific read task. + """ + ratio = WRITE_TASK_COUNT / READ_TASK_COUNT + from_index = math.floor(read_task_id * ratio) + end_index = math.ceil((read_task_id + 1) * ratio) + return task_queues[from_index:end_index] + + +if __name__ == '__main__': + main() +# ANCHOR_END: main diff --git a/docs/examples/python/mockdatasource.py b/docs/examples/python/mockdatasource.py new file mode 100644 index 0000000000..852860aec0 --- /dev/null +++ b/docs/examples/python/mockdatasource.py @@ -0,0 +1,49 @@ +import time + + +class MockDataSource: + samples = [ + "8.8,119,0.32,LosAngeles,0", + "10.7,116,0.34,SanDiego,1", + "9.9,111,0.33,Hollywood,2", + "8.9,113,0.329,Compton,3", + "9.4,118,0.141,San Francisco,4" + ] + + def __init__(self, tb_name_prefix, table_count): + self.table_name_prefix = tb_name_prefix + "_" + self.table_count = table_count + self.max_rows = 10000000 + self.current_ts = round(time.time() * 1000) - self.max_rows * 100 + # [(tableId, tableName, values),] + self.data = self._init_data() + + def _init_data(self): + lines = self.samples * (self.table_count // 5 + 1) + data = [] + for i in range(self.table_count): + table_name = self.table_name_prefix + str(i) + data.append((i, table_name, lines[i])) # tableId, row + return data + + def __iter__(self): + self.row = 0 + return self + + def __next__(self): + """ + next 1000 rows for each table. + return: {tableId:[row,...]} + """ + # generate 1000 timestamps + ts = [] + for _ in range(1000): + self.current_ts += 100 + ts.append(str(self.current_ts)) + # add timestamp to each row + # [(tableId, ["tableName,ts,current,voltage,phase,location,groupId"])] + result = [] + for table_id, table_name, values in self.data: + rows = [table_name + ',' + t + ',' + values for t in ts] + result.append((table_id, rows)) + return result diff --git a/docs/examples/python/sql_writer.py b/docs/examples/python/sql_writer.py new file mode 100644 index 0000000000..758167376b --- /dev/null +++ b/docs/examples/python/sql_writer.py @@ -0,0 +1,90 @@ +import logging +import taos + + +class SQLWriter: + log = logging.getLogger("SQLWriter") + + def __init__(self, get_connection_func): + self._tb_values = {} + self._tb_tags = {} + self._conn = get_connection_func() + self._max_sql_length = self.get_max_sql_length() + self._conn.execute("USE test") + + def get_max_sql_length(self): + rows = self._conn.query("SHOW variables").fetch_all() + for r in rows: + name = r[0] + if name == "maxSQLLength": + return int(r[1]) + return 1024 * 1024 + + def process_lines(self, lines: str): + """ + :param lines: [[tbName,ts,current,voltage,phase,location,groupId]] + """ + for line in lines: + ps = line.split(",") + table_name = ps[0] + value = '(' + ",".join(ps[1:-2]) + ') ' + if table_name in self._tb_values: + self._tb_values[table_name] += value + else: + self._tb_values[table_name] = value + + if table_name not in self._tb_tags: + location = ps[-2] + group_id = ps[-1] + tag_value = f"('{location}',{group_id})" + self._tb_tags[table_name] = tag_value + self.flush() + + def flush(self): + """ + Assemble INSERT statement and execute it. + When the sql length grows close to MAX_SQL_LENGTH, the sql will be executed immediately, and a new INSERT statement will be created. + In case of "Table does not exit" exception, tables in the sql will be created and the sql will be re-executed. + """ + sql = "INSERT INTO " + sql_len = len(sql) + buf = [] + for tb_name, values in self._tb_values.items(): + q = tb_name + " VALUES " + values + if sql_len + len(q) >= self._max_sql_length: + sql += " ".join(buf) + self.execute_sql(sql) + sql = "INSERT INTO " + sql_len = len(sql) + buf = [] + buf.append(q) + sql_len += len(q) + sql += " ".join(buf) + self.execute_sql(sql) + self._tb_values.clear() + + def execute_sql(self, sql): + try: + self._conn.execute(sql) + except taos.Error as e: + error_code = e.errno & 0xffff + # Table does not exit + if error_code == 9731: + self.create_tables() + else: + self.log.error("Execute SQL: %s", sql) + raise e + except BaseException as baseException: + self.log.error("Execute SQL: %s", sql) + raise baseException + + def create_tables(self): + sql = "CREATE TABLE " + for tb in self._tb_values.keys(): + tag_values = self._tb_tags[tb] + sql += "IF NOT EXISTS " + tb + " USING meters TAGS " + tag_values + " " + try: + self._conn.execute(sql) + except BaseException as e: + self.log.error("Execute SQL: %s", sql) + raise e diff --git a/docs/zh/02-intro.md b/docs/zh/02-intro.md index f726b4ea92..152d03d9e3 100644 --- a/docs/zh/02-intro.md +++ b/docs/zh/02-intro.md @@ -1,9 +1,10 @@ --- title: 产品简介 +description: 简要介绍 TDengine 的主要功能 toc_max_heading_level: 2 --- -TDengine 是一款[开源](https://www.taosdata.com/tdengine/open_source_time-series_database)、[高性能](https://www.taosdata.com/tdengine/fast)、[云原生](https://www.taosdata.com/tdengine/cloud_native_time-series_database)的时序数据库Time Series Database, TSDB)。TDengine 能被广泛运用于物联网、工业互联网、车联网、IT 运维、金融等领域。除核心的时序数据库功能外,TDengine 还提供[缓存](../develop/cache/)、[数据订阅](../develop/tmq)、[流式计算](../develop/stream)等功能,是一极简的时序数据处理平台,最大程度的减小系统设计的复杂度,降低研发和运营成本。 +TDengine 是一款开源、高性能、云原生的[时序数据库](https://tdengine.com/tsdb/),且针对物联网、车联网以及工业互联网进行了优化。TDengine 的代码,包括其集群功能,都在 GNU AGPL v3.0 下开源。除核心的时序数据库功能外,TDengine 还提供[缓存](../develop/cache/)、[数据订阅](../develop/tmq)、[流式计算](../develop/stream)等其它功能以降低系统复杂度及研发和运维成本。 本章节介绍TDengine的主要功能、竞争优势、适用场景、与其他数据库的对比测试等等,让大家对TDengine有个整体的了解。 @@ -11,47 +12,70 @@ TDengine 是一款[开源](https://www.taosdata.com/tdengine/open_source_time-se TDengine的主要功能如下: -1. 高速数据写入,除 [SQL 写入](../develop/insert-data/sql-writing)外,还支持 [Schemaless 写入](../reference/schemaless/),支持 [InfluxDB LINE 协议](../develop/insert-data/influxdb-line),[OpenTSDB Telnet](../develop/insert-data/opentsdb-telnet), [OpenTSDB JSON ](../develop/insert-data/opentsdb-json)等协议写入; -2. 第三方数据采集工具 [Telegraf](../third-party/telegraf),[Prometheus](../third-party/prometheus),[StatsD](../third-party/statsd),[collectd](../third-party/collectd),[icinga2](../third-party/icinga2), [TCollector](../third-party/tcollector), [EMQ](../third-party/emq-broker), [HiveMQ](../third-party/hive-mq-broker) 等都可以进行配置后,不用任何代码,即可将数据写入; -3. 支持[各种查询](../develop/query-data),包括聚合查询、嵌套查询、降采样查询、插值等 -4. 支持[用户自定义函数](../develop/udf) -5. 支持[缓存](../develop/cache),将每张表的最后一条记录缓存起来,这样无需 Redis -6. 支持[流式计算](../develop/stream)(Stream Processing) -7. 支持[数据订阅](../develop/tmq),而且可以指定过滤条件 -8. 支持[集群](../deployment/),可以通过多节点进行水平扩展,并通过多副本实现高可靠 -9. 提供[命令行程序](../reference/taos-shell),便于管理集群,检查系统状态,做即席查询 -10. 提供多种数据的[导入](../operation/import)、[导出](../operation/export) -11. 支持对[TDengine 集群本身的监控](../operation/monitor) -12. 提供各种语言的[连接器](../connector): 如 C/C++, Java, Go, Node.JS, Rust, Python, C# 等 -13. 支持 [REST 接口](../connector/rest-api/) -14. 支持与[ Grafana 无缝集成](../third-party/grafana) -15. 支持与 Google Data Studio 无缝集成 -16. 支持 [Kubernetes 部署](../deployment/k8s) +1. 写入数据,支持 + - [SQL 写入](../develop/insert-data/sql-writing) + - [Schemaless 写入](../reference/schemaless/),支持多种标准写入协议 + - [InfluxDB LINE 协议](../develop/insert-data/influxdb-line) + - [OpenTSDB Telnet 协议](../develop/insert-data/opentsdb-telnet) + - [OpenTSDB JSON 协议](../develop/insert-data/opentsdb-json) + - 与多种第三方工具的无缝集成,它们都可以仅通过配置而无需任何代码即可将数据写入 TDengine + - [Telegraf](../third-party/telegraf) + - [Prometheus](../third-party/prometheus) + - [StatsD](../third-party/statsd) + - [collectd](../third-party/collectd) + - [icinga2](../third-party/icinga2) + - [TCollector](../third-party/tcollector) + - [EMQ](../third-party/emq-broker) + - [HiveMQ](../third-party/hive-mq-broker) ; +2. 查询数据,支持 + - [标准SQL](../taos-sql),含嵌套查询 + - [时序数据特色函数](../taos-sql/function/#time-series-extensions) + - [时序顺序特色查询](../taos-sql/distinguished),例如降采样、插值、累加和、时间加权平均、状态窗口、会话窗口等 + - [用户自定义函数](../taos-sql/udf) +3. [缓存](../develop/cache),将每张表的最后一条记录缓存起来,这样无需 Redis 就能对时序数据进行高效处理 +4. [流式计算](../develop/stream)(Stream Processing),TDengine 不仅支持连续查询,还支持事件驱动的流式计算,这样在处理时序数据时就无需 Flink 或 Spark 这样流计算组件 +5. [数据订阅](../develop/tmq),应用程序可以订阅一张表或一组表的数据,API 与 Kafka 相同,而且可以指定过滤条件 +6. 可视化 + - 支持与 [Grafana](../third-party/grafana/) 的无缝集成 + - 支持与 Google Data Studio 的无缝集成 +7. 集群 + - 集群部署(../deployment/),可以通过增加节点进行水平扩展以提升处理能力 + - 可以通过 [Kubernets 部署 TDengine](../deployment/k8s/) + - 通过多副本提供高可用能力 +8. 管理 + - [监控](../operation/monitor)运行中的 TDengine 实例 + - 多种[数据导入](../operation/import)方式 + - 多种[数据导出](../operation/export)方式 +9. 工具 + - 提供交互式[命令行程序](../reference/taos-shell),便于管理集群,检查系统状态,做即席查询 + - 提供压力测试工具[taosBenchmark](../reference/taosbenchmark),用于测试 TDengine 的性能 +10. 编程 + - 提供各种语言的[连接器](../connector): 如 [C/C++](../connector/cpp), [Java](../connector/java), [Go](../connector/go), [Node.JS](../connector/node), [Rust](../connector/rust), [Python](../connector/python), [C#](../connector/csharp) 等 + - 支持 [REST 接口](../connector/rest-api/) -更多细小的功能,请阅读整个文档。 +更多细节功能,请阅读整个文档。 ## 竞争优势 -由于 TDengine 充分利用了[时序数据特点](https://www.taosdata.com/blog/2019/07/09/105.html),比如结构化、无需事务、很少删除或更新、写多读少等等,设计了全新的针对时序数据的存储引擎和计算引擎,因此与其他时序数据库相比,TDengine 有以下特点: +由于 TDengine 充分利用了[时序数据特点](https://www.taosdata.com/blog/2019/07/09/105.html),比如结构化、无需事务、很少删除或更新、写多读少等等,因此与其他时序数据库相比,TDengine 有以下特点: -- **[高性能](https://www.taosdata.com/tdengine/fast)**:通过创新的存储引擎设计,无论是数据写入还是查询,TDengine 的性能比通用数据库快 10 倍以上,也远超其他时序数据库,存储空间不及通用数据库的1/10。 +- **[高性能](https://www.taosdata.com/tdengine/fast)**:TDengine 是唯一一个解决了时序数据存储的高基数难题的时序数据库,支持上亿数据采集点,并在数据插入、查询和数据压缩上远胜其它时序数据库。 -- **[云原生](https://www.taosdata.com/tdengine/cloud_native_time-series_database)**:通过原生分布式的设计,充分利用云平台的优势,TDengine 提供了水平扩展能力,具备弹性、韧性和可观测性,支持k8s部署,可运行在公有云、私有云和混合云上。 +- **[极简时序数据平台](https://www.taosdata.com/tdengine/simplified_solution_for_time-series_data_processing)**:TDengine 内建缓存、流式计算和数据订阅等功能,为时序数据的处理提供了极简的解决方案,从而大幅降低了业务系统的设计复杂度和运维成本。 -- **[极简时序数据平台](https://www.taosdata.com/tdengine/simplified_solution_for_time-series_data_processing)**:TDengine 内建消息队列、缓存、流式计算等功能,应用无需再集成 Kafka/Redis/HBase/Spark 等软件,大幅降低系统的复杂度,降低应用开发和运营成本。 +- **[云原生](https://www.taosdata.com/tdengine/cloud_native_time-series_database)**:通过原生的分布式设计、数据分片和分区、存算分离、RAFT 协议、Kubernets 部署和完整的可观测性,TDengine 是一款云原生时序数据库并且能够部署在公有云、私有云和混合云上。 -- **[分析能力](https://www.taosdata.com/tdengine/easy_data_analytics)**:支持 SQL,同时为时序数据特有的分析提供SQL扩展。通过超级表、存储计算分离、分区分片、预计算、自定义函数等技术,TDengine 具备强大的分析能力。 +- **[简单易用](https://www.taosdata.com/tdengine/ease_of_use)**:对系统管理员来说,TDengine 大幅降低了管理和维护的代价。对开发者来说, TDengine 提供了简单的接口、极简的解决方案和与第三方工具的无缝集成。对数据分析专家来说,TDengine 提供了便捷的数据访问。 -- **[简单易用](https://www.taosdata.com/tdengine/ease_of_use)**:无任何依赖,安装、集群几秒搞定;提供REST以及各种语言连接器,与众多第三方工具无缝集成;提供命令行程序,便于管理和即席查询;提供各种运维工具。 +- **[分析能力](https://www.taosdata.com/tdengine/easy_data_analytics)**:通过超级表、存储计算分离、分区分片、预计算和其它技术,TDengine 能够高效地浏览、格式化和访问数据。 -- **[核心开源](https://www.taosdata.com/tdengine/open_source_time-series_database)**:TDengine 的核心代码包括集群功能全部开源,截止到2022年8月1日,全球超过 135.9k 个运行实例,GitHub Star 18.7k,Fork 4.4k,社区活跃。 +- **[核心开源](https://www.taosdata.com/tdengine/open_source_time-series_database)**:TDengine 的核心代码包括集群功能全部在开源协议下公开。全球超过 140k 个运行实例,GitHub Star 19k,且拥有一个活跃的开发者社区。 采用 TDengine,可将典型的物联网、车联网、工业互联网大数据平台的总拥有成本大幅降低。表现在几个方面: 1. 由于其超强性能,它能将系统需要的计算资源和存储资源大幅降低 2. 因为支持 SQL,能与众多第三方软件无缝集成,学习迁移成本大幅下降 3. 因为是一极简的时序数据平台,系统复杂度、研发和运营成本大幅降低 -4. 因为维护简单,运营维护成本能大幅降低 ## 技术生态 @@ -66,7 +90,7 @@ TDengine的主要功能如下: 上图中,左侧是各种数据采集或消息队列,包括 OPC-UA、MQTT、Telegraf、也包括 Kafka, 他们的数据将被源源不断的写入到 TDengine。右侧则是可视化、BI 工具、组态软件、应用程序。下侧则是 TDengine 自身提供的命令行程序 (CLI) 以及可视化管理管理。 -## 总体适用场景 +## 典型适用场景 作为一个高性能、分布式、支持 SQL 的时序数据库 (Database),TDengine 的典型适用场景包括但不限于 IoT、工业互联网、车联网、IT 运维、能源、金融证券等领域。需要指出的是,TDengine 是针对时序数据场景设计的专用数据库和专用大数据处理工具,因充分利用了时序大数据的特点,它无法用来处理网络爬虫、微博、微信、电商、ERP、CRM 等通用型数据。本文对适用场景做更多详细的分析。 diff --git a/docs/zh/04-concept/index.md b/docs/zh/04-concept/index.md index 8e97d4a2f4..89d3df9c97 100644 --- a/docs/zh/04-concept/index.md +++ b/docs/zh/04-concept/index.md @@ -1,5 +1,7 @@ --- +sidebar_label: 基本概念 title: 数据模型和基本概念 +description: TDengine 的数据模型和基本概念 --- 为了便于解释基本概念,便于撰写示例程序,整个 TDengine 文档以智能电表作为典型时序数据场景。假设每个智能电表采集电流、电压、相位三个量,有多个智能电表,每个电表有位置 location 和分组 group ID 的静态属性. 其采集的数据类似如下的表格: @@ -104,15 +106,15 @@ title: 数据模型和基本概念 ## 采集量 (Metric) -采集量是指传感器、设备或其他类型采集点采集的物理量,比如电流、电压、温度、压力、GPS 位置等,是随时间变化的,数据类型可以是整型、浮点型、布尔型,也可是字符串。随着时间的推移,存储的采集量的数据量越来越大。 +采集量是指传感器、设备或其他类型采集点采集的物理量,比如电流、电压、温度、压力、GPS 位置等,是随时间变化的,数据类型可以是整型、浮点型、布尔型,也可是字符串。随着时间的推移,存储的采集量的数据量越来越大。智能电表示例中的电流、电压、相位就是采集量。 ## 标签 (Label/Tag) -标签是指传感器、设备或其他类型采集点的静态属性,不是随时间变化的,比如设备型号、颜色、设备的所在地等,数据类型可以是任何类型。虽然是静态的,但 TDengine 容许用户修改、删除或增加标签值。与采集量不一样的是,随时间的推移,存储的标签的数据量不会有什么变化。 +标签是指传感器、设备或其他类型采集点的静态属性,不是随时间变化的,比如设备型号、颜色、设备的所在地等,数据类型可以是任何类型。虽然是静态的,但 TDengine 容许用户修改、删除或增加标签值。与采集量不一样的是,随时间的推移,存储的标签的数据量不会有什么变化。智能电表示例中的location与groupId就是标签。 ## 数据采集点 (Data Collection Point) -数据采集点是指按照预设时间周期或受事件触发采集物理量的硬件或软件。一个数据采集点可以采集一个或多个采集量,**但这些采集量都是同一时刻采集的,具有相同的时间戳**。对于复杂的设备,往往有多个数据采集点,每个数据采集点采集的周期都可能不一样,而且完全独立,不同步。比如对于一台汽车,有数据采集点专门采集 GPS 位置,有数据采集点专门采集发动机状态,有数据采集点专门采集车内的环境,这样一台汽车就有三个数据采集点。 +数据采集点是指按照预设时间周期或受事件触发采集物理量的硬件或软件。一个数据采集点可以采集一个或多个采集量,**但这些采集量都是同一时刻采集的,具有相同的时间戳**。对于复杂的设备,往往有多个数据采集点,每个数据采集点采集的周期都可能不一样,而且完全独立,不同步。比如对于一台汽车,有数据采集点专门采集 GPS 位置,有数据采集点专门采集发动机状态,有数据采集点专门采集车内的环境,这样一台汽车就有三个数据采集点。智能电表示例中的d1001, d1002, d1003, d1004等就是数据采集点。 ## 表 (Table) @@ -131,13 +133,14 @@ TDengine 建议用数据采集点的名字(如上表中的 D1001)来做表 对于复杂的设备,比如汽车,它有多个数据采集点,那么就需要为一台汽车建立多张表。 + ## 超级表 (STable) 由于一个数据采集点一张表,导致表的数量巨增,难以管理,而且应用经常需要做采集点之间的聚合操作,聚合的操作也变得复杂起来。为解决这个问题,TDengine 引入超级表(Super Table,简称为 STable)的概念。 超级表是指某一特定类型的数据采集点的集合。同一类型的数据采集点,其表的结构是完全一样的,但每个表(数据采集点)的静态属性(标签)是不一样的。描述一个超级表(某一特定类型的数据采集点的集合),除需要定义采集量的表结构之外,还需要定义其标签的 schema,标签的数据类型可以是整数、浮点数、字符串,标签可以有多个,可以事后增加、删除或修改。如果整个系统有 N 个不同类型的数据采集点,就需要建立 N 个超级表。 -在 TDengine 的设计里,**表用来代表一个具体的数据采集点,超级表用来代表一组相同类型的数据采集点集合**。 +在 TDengine 的设计里,**表用来代表一个具体的数据采集点,超级表用来代表一组相同类型的数据采集点集合**。智能电表示例中,我们可以创建一个超级表meters. ## 子表 (Subtable) @@ -156,7 +159,9 @@ TDengine 建议用数据采集点的名字(如上表中的 D1001)来做表 查询既可以在表上进行,也可以在超级表上进行。针对超级表的查询,TDengine 将把所有子表中的数据视为一个整体数据集进行处理,会先把满足标签过滤条件的表从超级表中找出来,然后再扫描这些表的时序数据,进行聚合操作,这样需要扫描的数据集会大幅减少,从而显著提高查询的性能。本质上,TDengine 通过对超级表查询的支持,实现了多个同类数据采集点的高效聚合。 -TDengine系统建议给一个数据采集点建表,需要通过超级表建表,而不是建普通表。 +TDengine系统建议给一个数据采集点建表,需要通过超级表建表,而不是建普通表。在智能电表的示例中,我们可以通过超级表meters创建子表d1001, d1002, d1003, d1004等。 + +为了更好地理解超级与子表的关系,可以参考下面关于智能电表数据模型的示意图。 ![智能电表数据模型示意图](./supertable.webp) ## 库 (database) diff --git a/docs/zh/04-concept/supertable.webp b/docs/zh/04-concept/supertable.webp new file mode 100644 index 0000000000..764b8f3de7 Binary files /dev/null and b/docs/zh/04-concept/supertable.webp differ diff --git a/docs/zh/05-get-started/01-docker.md b/docs/zh/05-get-started/01-docker.md index f0f09d4c7e..e2be419517 100644 --- a/docs/zh/05-get-started/01-docker.md +++ b/docs/zh/05-get-started/01-docker.md @@ -1,6 +1,7 @@ --- sidebar_label: Docker title: 通过 Docker 快速体验 TDengine +description: 使用 Docker 快速体验 TDengine 的高效写入和查询 --- 本节首先介绍如何通过 Docker 快速体验 TDengine,然后介绍如何在 Docker 环境下体验 TDengine 的写入和查询功能。如果你不熟悉 Docker,请使用[安装包的方式快速体验](../../get-started/package/)。如果您希望为 TDengine 贡献代码或对内部技术实现感兴趣,请参考 [TDengine GitHub 主页](https://github.com/taosdata/TDengine) 下载源码构建和安装. diff --git a/docs/zh/05-get-started/03-package.md b/docs/zh/05-get-started/03-package.md index a1c1802d77..3e0fb056a5 100644 --- a/docs/zh/05-get-started/03-package.md +++ b/docs/zh/05-get-started/03-package.md @@ -1,6 +1,7 @@ --- sidebar_label: 安装包 title: 使用安装包立即开始 +description: 使用安装包快速体验 TDengine --- import Tabs from "@theme/Tabs"; diff --git a/docs/zh/07-develop/01-connect/index.md b/docs/zh/07-develop/01-connect/index.md index 3e44e6c5da..075d99cfee 100644 --- a/docs/zh/07-develop/01-connect/index.md +++ b/docs/zh/07-develop/01-connect/index.md @@ -1,6 +1,6 @@ --- title: 建立连接 -description: "本节介绍如何使用连接器建立与 TDengine 的连接,给出连接器安装、连接的简单说明。" +description: 使用连接器建立与 TDengine 的连接,以及连接器的安装和连接 --- import Tabs from "@theme/Tabs"; diff --git a/docs/zh/07-develop/02-model/index.mdx b/docs/zh/07-develop/02-model/index.mdx index 1609eb5362..d66059c2cd 100644 --- a/docs/zh/07-develop/02-model/index.mdx +++ b/docs/zh/07-develop/02-model/index.mdx @@ -1,5 +1,7 @@ --- +sidebar_label: 数据建模 title: TDengine 数据建模 +description: TDengine 中如何建立数据模型 --- TDengine 采用类关系型数据模型,需要建库、建表。因此对于一个具体的应用场景,需要考虑库、超级表和普通表的设计。本节不讨论细致的语法规则,只介绍概念。 @@ -39,7 +41,7 @@ USE power; CREATE STABLE meters (ts timestamp, current float, voltage int, phase float) TAGS (location binary(64), groupId int); ``` -与创建普通表一样,创建超级表时,需要提供表名(示例中为 meters),表结构 Schema,即数据列的定义。第一列必须为时间戳(示例中为 ts),其他列为采集的物理量(示例中为 current, voltage, phase),数据类型可以为整型、浮点型、字符串等。除此之外,还需要提供标签的 schema (示例中为 location, groupId),标签的数据类型可以为整型、浮点型、字符串等。采集点的静态属性往往可以作为标签,比如采集点的地理位置、设备型号、设备组 ID、管理员 ID 等等。标签的 schema 可以事后增加、删除、修改。具体定义以及细节请见 [TAOS SQL 的超级表管理](/taos-sql/stable) 章节。 +与创建普通表一样,创建超级表时,需要提供表名(示例中为 meters),表结构 Schema,即数据列的定义。第一列必须为时间戳(示例中为 ts),其他列为采集的物理量(示例中为 current, voltage, phase),数据类型可以为整型、浮点型、字符串等。除此之外,还需要提供标签的 schema (示例中为 location, groupId),标签的数据类型可以为整型、浮点型、字符串等。采集点的静态属性往往可以作为标签,比如采集点的地理位置、设备型号、设备组 ID、管理员 ID 等等。标签的 schema 可以事后增加、删除、修改。具体定义以及细节请见 [TDengine SQL 的超级表管理](/taos-sql/stable) 章节。 每一种类型的数据采集点需要建立一个超级表,因此一个物联网系统,往往会有多个超级表。对于电网,我们就需要对智能电表、变压器、母线、开关等都建立一个超级表。在物联网中,一个设备就可能有多个数据采集点(比如一台风力发电的风机,有的采集点采集电流、电压等电参数,有的采集点采集温度、湿度、风向等环境参数),这个时候,对这一类型的设备,需要建立多张超级表。 @@ -53,7 +55,7 @@ TDengine 对每个数据采集点需要独立建表。与标准的关系型数 CREATE TABLE d1001 USING meters TAGS ("California.SanFrancisco", 2); ``` -其中 d1001 是表名,meters 是超级表的表名,后面紧跟标签 Location 的具体标签值 "California.SanFrancisco",标签 groupId 的具体标签值 2。虽然在创建表时,需要指定标签值,但可以事后修改。详细细则请见 [TAOS SQL 的表管理](/taos-sql/table) 章节。 +其中 d1001 是表名,meters 是超级表的表名,后面紧跟标签 Location 的具体标签值 "California.SanFrancisco",标签 groupId 的具体标签值 2。虽然在创建表时,需要指定标签值,但可以事后修改。详细细则请见 [TDengine SQL 的表管理](/taos-sql/table) 章节。 TDengine 建议将数据采集点的全局唯一 ID 作为表名(比如设备序列号)。但对于有的场景,并没有唯一的 ID,可以将多个 ID 组合成一个唯一的 ID。不建议将具有唯一性的 ID 作为标签值。 diff --git a/docs/zh/07-develop/03-insert-data/01-sql-writing.mdx b/docs/zh/07-develop/03-insert-data/01-sql-writing.mdx index 214cbdaa96..8818eaae3d 100644 --- a/docs/zh/07-develop/03-insert-data/01-sql-writing.mdx +++ b/docs/zh/07-develop/03-insert-data/01-sql-writing.mdx @@ -23,9 +23,10 @@ import PhpStmt from "./_php_stmt.mdx"; ## SQL 写入简介 -应用通过连接器执行 INSERT 语句来插入数据,用户还可以通过 TAOS Shell,手动输入 INSERT 语句插入数据。 +应用通过连接器执行 INSERT 语句来插入数据,用户还可以通过 TDengine CLI,手动输入 INSERT 语句插入数据。 ### 一次写入一条 + 下面这条 INSERT 就将一条记录写入到表 d1001 中: ```sql @@ -48,7 +49,7 @@ TDengine 也支持一次向多个表写入数据,比如下面这条命令就 INSERT INTO d1001 VALUES (1538548685000, 10.3, 219, 0.31) (1538548695000, 12.6, 218, 0.33) d1002 VALUES (1538548696800, 12.3, 221, 0.31); ``` -详细的 SQL INSERT 语法规则参考 [TAOS SQL 的数据写入](/taos-sql/insert)。 +详细的 SQL INSERT 语法规则参考 [TDengine SQL 的数据写入](/taos-sql/insert)。 :::info @@ -134,4 +135,3 @@ TDengine 也提供了支持参数绑定的 Prepare API,与 MySQL 类似,这
- diff --git a/docs/zh/07-develop/03-insert-data/05-high-volume.md b/docs/zh/07-develop/03-insert-data/05-high-volume.md new file mode 100644 index 0000000000..32be8cb890 --- /dev/null +++ b/docs/zh/07-develop/03-insert-data/05-high-volume.md @@ -0,0 +1,440 @@ +import Tabs from "@theme/Tabs"; +import TabItem from "@theme/TabItem"; + +# 高效写入 + +本节介绍如何高效地向 TDengine 写入数据。 + +## 高效写入原理 {#principle} + +### 客户端程序的角度 {#application-view} + +从客户端程序的角度来说,高效写入数据要考虑以下几个因素: + +1. 单次写入的数据量。一般来讲,每批次写入的数据量越大越高效(但超过一定阈值其优势会消失)。使用 SQL 写入 TDengine 时,尽量在一条 SQL 中拼接更多数据。目前,TDengine 支持的一条 SQL 的最大长度为 1,048,576(1M)个字符。可通过配置客户端参数 maxSQLLength(默认值为 65480)进行修改。 +2. 并发连接数。一般来讲,同时写入数据的并发连接数越多写入越高效(但超过一定阈值反而会下降,取决于服务端处理能力)。 +3. 数据在不同表(或子表)之间的分布,即要写入数据的相邻性。一般来说,每批次只向同一张表(或子表)写入数据比向多张表(或子表)写入数据要更高效; +4. 写入方式。一般来讲: + - 参数绑定写入比 SQL 写入更高效。因参数绑定方式避免了 SQL 解析。(但增加了 C 接口的调用次数,对于连接器也有性能损耗)。 + - SQL 写入不自动建表比自动建表更高效。因自动建表要频繁检查表是否存在 + - SQL 写入比无模式写入更高效。因无模式写入会自动建表且支持动态更改表结构 + +客户端程序要充分且恰当地利用以上几个因素。在单次写入中尽量只向同一张表(或子表)写入数据,每批次写入的数据量经过测试和调优设定为一个最适合当前系统处理能力的数值,并发写入的连接数同样经过测试和调优后设定为一个最适合当前系统处理能力的数值,以实现在当前系统中的最佳写入速度。 + +### 数据源的角度 {#datasource-view} + +客户端程序通常需要从数据源读数据再写入 TDengine。从数据源角度来说,以下几种情况需要在读线程和写线程之间增加队列: + +1. 有多个数据源,单个数据源生成数据的速度远小于单线程写入的速度,但数据量整体比较大。此时队列的作用是把多个数据源的数据汇聚到一起,增加单次写入的数据量。 +2. 单个数据源生成数据的速度远大于单线程写入的速度。此时队列的作用是增加写入的并发度。 +3. 单张表的数据分散在多个数据源。此时队列的作用是将同一张表的数据提前汇聚到一起,提高写入时数据的相邻性。 + +如果写应用的数据源是 Kafka, 写应用本身即 Kafka 的消费者,则可利用 Kafka 的特性实现高效写入。比如: + +1. 将同一张表的数据写到同一个 Topic 的同一个 Partition,增加数据的相邻性 +2. 通过订阅多个 Topic 实现数据汇聚 +3. 通过增加 Consumer 线程数增加写入的并发度 +4. 通过增加每次 fetch 的最大数据量来增加单次写入的最大数据量 + +### 服务器配置的角度 {#setting-view} + +从服务器配置的角度来说,也有很多优化写入性能的方法。 + +如果总表数不多(远小于核数乘以1000), 且无论怎么调节客户端程序,taosd 进程的 CPU 使用率都很低,那么很可能是因为表在各个 vgroup 分布不均。比如:数据库总表数是 1000 且 minTablesPerVnode 设置的也是 1000,那么所有的表都会分布在 1 个 vgroup 上。此时如果将 minTablesPerVnode 和 tablelncStepPerVnode 都设置成 100, 则可将表分布至 10 个 vgroup。(假设 maxVgroupsPerDb 大于等于 10)。 + +如果总表数比较大(比如大于500万),适当增加 maxVgroupsPerDb 也能显著提高建表的速度。maxVgroupsPerDb 默认值为 0, 自动配置为 CPU 的核数。 如果表的数量巨大,也建议调节 maxTablesPerVnode 参数,以免超过单个 vnode 建表的上限。 + +更多调优参数,请参考 [配置参考](../../../reference/config)部分。 + +## 高效写入示例 {#sample-code} + +### 场景设计 {#scenario} + +下面的示例程序展示了如何高效写入数据,场景设计如下: + +- TDengine 客户端程序从其它数据源不断读入数据,在示例程序中采用生成模拟数据的方式来模拟读取数据源 +- 单个连接向 TDengine 写入的速度无法与读数据的速度相匹配,因此客户端程序启动多个线程,每个线程都建立了与 TDengine 的连接,每个线程都有一个独占的固定大小的消息队列 +- 客户端程序将接收到的数据根据所属的表名(或子表名)HASH 到不同的线程,即写入该线程所对应的消息队列,以此确保属于某个表(或子表)的数据一定会被一个固定的线程处理 +- 各个子线程在将所关联的消息队列中的数据读空后或者读取数据量达到一个预定的阈值后将该批数据写入 TDengine,并继续处理后面接收到的数据 + +![TDengine 高效写入示例场景的线程模型](highvolume.webp) + +### 示例代码 {#code} + +这一部分是针对以上场景的示例代码。对于其它场景高效写入原理相同,不过代码需要适当修改。 + +本示例代码假设源数据属于同一张超级表(meters)的不同子表。程序在开始写入数据之前已经在 test 库创建了这个超级表。对于子表,将根据收到的数据,由应用程序自动创建。如果实际场景是多个超级表,只需修改写任务自动建表的代码。 + + + + +**程序清单** + +| 类名 | 功能说明 | +| ---------------- | --------------------------------------------------------------------------- | +| FastWriteExample | 主程序 | +| ReadTask | 从模拟源中读取数据,将表名经过 hash 后得到 Queue 的 index,写入对应的 Queue | +| WriteTask | 从 Queue 中获取数据,组成一个 Batch,写入 TDengine | +| MockDataSource | 模拟生成一定数量 meters 子表的数据 | +| SQLWriter | WriteTask 依赖这个类完成 SQL 拼接、自动建表、 SQL 写入、SQL 长度检查 | +| StmtWriter | 实现参数绑定方式批量写入(暂未完成) | +| DataBaseMonitor | 统计写入速度,并每隔 10 秒把当前写入速度打印到控制台 | + + +以下是各类的完整代码和更详细的功能说明。 + +
+FastWriteExample +主程序负责: + +1. 创建消息队列 +2. 启动写线程 +3. 启动读线程 +4. 每隔 10 秒统计一次写入速度 + +主程序默认暴露了 4 个参数,每次启动程序都可调节,用于测试和调优: + +1. 读线程个数。默认为 1。 +2. 写线程个数。默认为 3。 +3. 模拟生成的总表数。默认为 1000。将会平分给各个读线程。如果总表数较大,建表需要花费较长,开始统计的写入速度可能较慢。 +4. 每批最多写入记录数量。默认为 3000。 + +队列容量(taskQueueCapacity)也是与性能有关的参数,可通过修改程序调节。一般来讲,队列容量越大,入队被阻塞的概率越小,队列的吞吐量越大,但是内存占用也会越大。 示例程序默认值已经设置地足够大。 + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/FastWriteExample.java}} +``` + +
+ +
+ReadTask + +读任务负责从数据源读数据。每个读任务都关联了一个模拟数据源。每个模拟数据源可生成一点数量表的数据。不同的模拟数据源生成不同表的数据。 + +读任务采用阻塞的方式写消息队列。也就是说,一旦队列满了,写操作就会阻塞。 + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/ReadTask.java}} +``` + +
+ +
+WriteTask + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/WriteTask.java}} +``` + +
+ +
+ +MockDataSource + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/MockDataSource.java}} +``` + +
+ +
+ +SQLWriter + +SQLWriter 类封装了拼 SQL 和写数据的逻辑。注意,所有的表都没有提前创建,而是在 catch 到表不存在异常的时候,再以超级表为模板批量建表,然后重新执行 INSERT 语句。对于其它异常,这里简单地记录当时执行的 SQL 语句到日志中,你也可以记录更多线索到日志,已便排查错误和故障恢复。 + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/SQLWriter.java}} +``` + +
+ +
+ +DataBaseMonitor + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/DataBaseMonitor.java}} +``` + +
+ +**执行步骤** + +
+执行 Java 示例程序 + +执行程序前需配置环境变量 `TDENGINE_JDBC_URL`。如果 TDengine Server 部署在本机,且用户名、密码和端口都是默认值,那么可配置: + +``` +TDENGINE_JDBC_URL="jdbc:TAOS://localhost:6030?user=root&password=taosdata" +``` + +**本地集成开发环境执行示例程序** + +1. clone TDengine 仓库 + ``` + git clone git@github.com:taosdata/TDengine.git --depth 1 + ``` +2. 用集成开发环境打开 `docs/examples/java` 目录。 +3. 在开发环境中配置环境变量 `TDENGINE_JDBC_URL`。如果已配置了全局的环境变量 `TDENGINE_JDBC_URL` 可跳过这一步。 +4. 运行类 `com.taos.example.highvolume.FastWriteExample`。 + +**远程服务器上执行示例程序** + +若要在服务器上执行示例程序,可按照下面的步骤操作: + +1. 打包示例代码。在目录 TDengine/docs/examples/java 下执行: + ``` + mvn package + ``` +2. 远程服务器上创建 examples 目录: + ``` + mkdir -p examples/java + ``` +3. 复制依赖到服务器指定目录: + - 复制依赖包,只用复制一次 + ``` + scp -r .\target\lib @:~/examples/java + ``` + - 复制本程序的 jar 包,每次更新代码都需要复制 + ``` + scp -r .\target\javaexample-1.0.jar @:~/examples/java + ``` +4. 配置环境变量。 + 编辑 `~/.bash_profile` 或 `~/.bashrc` 添加如下内容例如: + + ``` + export TDENGINE_JDBC_URL="jdbc:TAOS://localhost:6030?user=root&password=taosdata" + ``` + + 以上使用的是本地部署 TDengine Server 时默认的 JDBC URL。你需要根据自己的实际情况更改。 + +5. 用 java 命令启动示例程序,命令模板: + + ``` + java -classpath lib/*:javaexample-1.0.jar com.taos.example.highvolume.FastWriteExample + ``` + +6. 结束测试程序。测试程序不会自动结束,在获取到当前配置下稳定的写入速度后,按 CTRL + C 结束程序。 + 下面是一次实际运行的日志输出,机器配置 16核 + 64G + 固态硬盘。 + + ``` + root@vm85$ java -classpath lib/*:javaexample-1.0.jar com.taos.example.highvolume.FastWriteExample 2 12 + 18:56:35.896 [main] INFO c.t.e.highvolume.FastWriteExample - readTaskCount=2, writeTaskCount=12 tableCount=1000 maxBatchSize=3000 + 18:56:36.011 [WriteThread-0] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.015 [WriteThread-0] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.021 [WriteThread-1] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.022 [WriteThread-1] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.031 [WriteThread-2] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.032 [WriteThread-2] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.041 [WriteThread-3] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.042 [WriteThread-3] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.093 [WriteThread-4] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.094 [WriteThread-4] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.099 [WriteThread-5] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.100 [WriteThread-5] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.100 [WriteThread-6] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.101 [WriteThread-6] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.103 [WriteThread-7] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.104 [WriteThread-7] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.105 [WriteThread-8] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.107 [WriteThread-8] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.108 [WriteThread-9] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.109 [WriteThread-9] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.156 [WriteThread-10] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.157 [WriteThread-11] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.158 [WriteThread-10] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.158 [ReadThread-0] INFO com.taos.example.highvolume.ReadTask - started + 18:56:36.158 [ReadThread-1] INFO com.taos.example.highvolume.ReadTask - started + 18:56:36.158 [WriteThread-11] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:46.369 [main] INFO c.t.e.highvolume.FastWriteExample - count=18554448 speed=1855444 + 18:56:56.946 [main] INFO c.t.e.highvolume.FastWriteExample - count=39059660 speed=2050521 + 18:57:07.322 [main] INFO c.t.e.highvolume.FastWriteExample - count=59403604 speed=2034394 + 18:57:18.032 [main] INFO c.t.e.highvolume.FastWriteExample - count=80262938 speed=2085933 + 18:57:28.432 [main] INFO c.t.e.highvolume.FastWriteExample - count=101139906 speed=2087696 + 18:57:38.921 [main] INFO c.t.e.highvolume.FastWriteExample - count=121807202 speed=2066729 + 18:57:49.375 [main] INFO c.t.e.highvolume.FastWriteExample - count=142952417 speed=2114521 + 18:58:00.689 [main] INFO c.t.e.highvolume.FastWriteExample - count=163650306 speed=2069788 + 18:58:11.646 [main] INFO c.t.e.highvolume.FastWriteExample - count=185019808 speed=2136950 + ``` + +
+ +
+ + +**程序清单** + +Python 示例程序中采用了多进程的架构,并使用了跨进程的消息队列。 + +| 函数或类 | 功能说明 | +| ------------------------ | -------------------------------------------------------------------- | +| main 函数 | 程序入口, 创建各个子进程和消息队列 | +| run_monitor_process 函数 | 创建数据库,超级表,统计写入速度并定时打印到控制台 | +| run_read_task 函数 | 读进程主要逻辑,负责从其它数据系统读数据,并分发数据到为之分配的队列 | +| MockDataSource 类 | 模拟数据源, 实现迭代器接口,每次批量返回每张表的接下来 1000 条数据 | +| run_write_task 函数 | 写进程主要逻辑。每次从队列中取出尽量多的数据,并批量写入 | +| SQLWriter类 | SQL 写入和自动建表 | +| StmtWriter 类 | 实现参数绑定方式批量写入(暂未完成) | + + +
+main 函数 + +main 函数负责创建消息队列和启动子进程,子进程有 3 类: + +1. 1 个监控进程,负责数据库初始化和统计写入速度 +2. n 个读进程,负责从其它数据系统读数据 +3. m 个写进程,负责写数据库 + +main 函数可以接收 5 个启动参数,依次是: + +1. 读任务(进程)数, 默认为 1 +2. 写任务(进程)数, 默认为 1 +3. 模拟生成的总表数,默认为 1000 +4. 队列大小(单位字节),默认为 1000000 +5. 每批最多写入记录数量, 默认为 3000 + +```python +{{#include docs/examples/python/fast_write_example.py:main}} +``` + +
+ +
+run_monitor_process + +监控进程负责初始化数据库,并监控当前的写入速度。 + +```python +{{#include docs/examples/python/fast_write_example.py:monitor}} +``` + +
+ +
+ +run_read_task 函数 + +读进程,负责从其它数据系统读数据,并分发数据到为之分配的队列。 + +```python +{{#include docs/examples/python/fast_write_example.py:read}} +``` + +
+ +
+ +MockDataSource + +以下是模拟数据源的实现,我们假设数据源生成的每一条数据都带有目标表名信息。实际中你可能需要一定的规则确定目标表名。 + +```python +{{#include docs/examples/python/mockdatasource.py}} +``` + +
+ +
+run_write_task 函数 + +写进程每次从队列中取出尽量多的数据,并批量写入。 + +```python +{{#include docs/examples/python/fast_write_example.py:write}} +``` + +
+ +
+ +SQLWriter 类封装了拼 SQL 和写数据的逻辑。所有的表都没有提前创建,而是在发生表不存在错误的时候,再以超级表为模板批量建表,然后重新执行 INSERT 语句。对于其它错误会记录当时执行的 SQL, 以便排查错误和故障恢复。这个类也对 SQL 是否超过最大长度限制做了检查,如果接近 SQL 最大长度限制(maxSQLLength),将会立即执行 SQL。为了减少 SQL 此时,建议将 maxSQLLength 适当调大。 + +SQLWriter + +```python +{{#include docs/examples/python/sql_writer.py}} +``` + +
+ +**执行步骤** + +
+ +执行 Python 示例程序 + +1. 前提条件 + + - 已安装 TDengine 客户端驱动 + - 已安装 Python3, 推荐版本 >= 3.8 + - 已安装 taospy + +2. 安装 faster-fifo 代替 python 内置的 multiprocessing.Queue + + ``` + pip3 install faster-fifo + ``` + +3. 点击上面的“查看源码”链接复制 `fast_write_example.py` 、 `sql_writer.py` 和 `mockdatasource.py` 三个文件。 + +4. 执行示例程序 + + ``` + python3 fast_write_example.py + ``` + + 下面是一次实际运行的输出, 机器配置 16核 + 64G + 固态硬盘。 + + ``` + root@vm85$ python3 fast_write_example.py 8 8 + 2022-07-14 19:13:45,869 [root] - READ_TASK_COUNT=8, WRITE_TASK_COUNT=8, TABLE_COUNT=1000, QUEUE_SIZE=1000000, MAX_BATCH_SIZE=3000 + 2022-07-14 19:13:48,882 [root] - WriteTask-0 started with pid 718347 + 2022-07-14 19:13:48,883 [root] - WriteTask-1 started with pid 718348 + 2022-07-14 19:13:48,884 [root] - WriteTask-2 started with pid 718349 + 2022-07-14 19:13:48,884 [root] - WriteTask-3 started with pid 718350 + 2022-07-14 19:13:48,885 [root] - WriteTask-4 started with pid 718351 + 2022-07-14 19:13:48,885 [root] - WriteTask-5 started with pid 718352 + 2022-07-14 19:13:48,886 [root] - WriteTask-6 started with pid 718353 + 2022-07-14 19:13:48,886 [root] - WriteTask-7 started with pid 718354 + 2022-07-14 19:13:48,887 [root] - ReadTask-0 started with pid 718355 + 2022-07-14 19:13:48,888 [root] - ReadTask-1 started with pid 718356 + 2022-07-14 19:13:48,889 [root] - ReadTask-2 started with pid 718357 + 2022-07-14 19:13:48,889 [root] - ReadTask-3 started with pid 718358 + 2022-07-14 19:13:48,890 [root] - ReadTask-4 started with pid 718359 + 2022-07-14 19:13:48,891 [root] - ReadTask-5 started with pid 718361 + 2022-07-14 19:13:48,892 [root] - ReadTask-6 started with pid 718364 + 2022-07-14 19:13:48,893 [root] - ReadTask-7 started with pid 718365 + 2022-07-14 19:13:56,042 [DataBaseMonitor] - count=6676310 speed=667631.0 + 2022-07-14 19:14:06,196 [DataBaseMonitor] - count=20004310 speed=1332800.0 + 2022-07-14 19:14:16,366 [DataBaseMonitor] - count=32290310 speed=1228600.0 + 2022-07-14 19:14:26,527 [DataBaseMonitor] - count=44438310 speed=1214800.0 + 2022-07-14 19:14:36,673 [DataBaseMonitor] - count=56608310 speed=1217000.0 + 2022-07-14 19:14:46,834 [DataBaseMonitor] - count=68757310 speed=1214900.0 + 2022-07-14 19:14:57,280 [DataBaseMonitor] - count=80992310 speed=1223500.0 + 2022-07-14 19:15:07,689 [DataBaseMonitor] - count=93805310 speed=1281300.0 + 2022-07-14 19:15:18,020 [DataBaseMonitor] - count=106111310 speed=1230600.0 + 2022-07-14 19:15:28,356 [DataBaseMonitor] - count=118394310 speed=1228300.0 + 2022-07-14 19:15:38,690 [DataBaseMonitor] - count=130742310 speed=1234800.0 + 2022-07-14 19:15:49,000 [DataBaseMonitor] - count=143051310 speed=1230900.0 + 2022-07-14 19:15:59,323 [DataBaseMonitor] - count=155276310 speed=1222500.0 + 2022-07-14 19:16:09,649 [DataBaseMonitor] - count=167603310 speed=1232700.0 + 2022-07-14 19:16:19,995 [DataBaseMonitor] - count=179976310 speed=1237300.0 + ``` + +
+ +:::note +使用 Python 连接器多进程连接 TDengine 的时候,有一个限制:不能在父进程中建立连接,所有连接只能在子进程中创建。 +如果在父进程中创建连接,子进程再创建连接就会一直阻塞。这是个已知问题。 + +::: + +
+
+ + diff --git a/docs/zh/07-develop/03-insert-data/highvolume.webp b/docs/zh/07-develop/03-insert-data/highvolume.webp new file mode 100644 index 0000000000..46dfc74ae3 Binary files /dev/null and b/docs/zh/07-develop/03-insert-data/highvolume.webp differ diff --git a/docs/zh/07-develop/03-insert-data/index.md b/docs/zh/07-develop/03-insert-data/index.md index 55a28e4a8b..f1e5ada4df 100644 --- a/docs/zh/07-develop/03-insert-data/index.md +++ b/docs/zh/07-develop/03-insert-data/index.md @@ -1,5 +1,7 @@ --- +sidebar_label: 写入数据 title: 写入数据 +description: TDengine 的各种写入方式 --- TDengine 支持多种写入协议,包括 SQL,InfluxDB Line 协议, OpenTSDB Telnet 协议,OpenTSDB JSON 格式协议。数据可以单条插入,也可以批量插入,可以插入一个数据采集点的数据,也可以同时插入多个数据采集点的数据。同时,TDengine 支持多线程插入,支持时间乱序数据插入,也支持历史数据插入。InfluxDB Line 协议、OpenTSDB Telnet 协议和 OpenTSDB JSON 格式协议是 TDengine 支持的三种无模式写入协议。使用无模式方式写入无需提前创建超级表和子表,并且引擎能自适用数据对表结构做调整。 diff --git a/docs/zh/07-develop/04-query-data/index.mdx b/docs/zh/07-develop/04-query-data/index.mdx index 2631d147a5..d6156c8a59 100644 --- a/docs/zh/07-develop/04-query-data/index.mdx +++ b/docs/zh/07-develop/04-query-data/index.mdx @@ -1,4 +1,5 @@ --- +sidebar_label: 查询数据 title: 查询数据 description: "主要查询功能,通过连接器执行同步查询和异步查询" --- @@ -43,7 +44,7 @@ Query OK, 2 row(s) in set (0.001100s) 为满足物联网场景的需求,TDengine 支持几个特殊的函数,比如 twa(时间加权平均),spread (最大值与最小值的差),last_row(最后一条记录)等,更多与物联网场景相关的函数将添加进来。 -具体的查询语法请看 [TAOS SQL 的数据查询](../../taos-sql/select) 章节。 +具体的查询语法请看 [TDengine SQL 的数据查询](../../taos-sql/select) 章节。 ## 多表聚合查询 @@ -51,7 +52,7 @@ Query OK, 2 row(s) in set (0.001100s) ### 示例一 -在 TAOS Shell,查找加利福尼亚州所有智能电表采集的电压平均值,并按照 location 分组。 +在 TDengine CLI,查找加利福尼亚州所有智能电表采集的电压平均值,并按照 location 分组。 ``` taos> SELECT AVG(voltage), location FROM meters GROUP BY location; @@ -64,7 +65,7 @@ Query OK, 2 rows in database (0.005995s) ### 示例二 -在 TAOS shell, 查找 groupId 为 2 的所有智能电表的记录条数,电流的最大值。 +在 TDengine CLI, 查找 groupId 为 2 的所有智能电表的记录条数,电流的最大值。 ``` taos> SELECT count(*), max(current) FROM meters where groupId = 2; @@ -74,7 +75,7 @@ taos> SELECT count(*), max(current) FROM meters where groupId = 2; Query OK, 1 row(s) in set (0.002136s) ``` -在 [TAOS SQL 的数据查询](../../taos-sql/select) 一章,查询类操作都会注明是否支持超级表。 +在 [TDengine SQL 的数据查询](../../taos-sql/select) 一章,查询类操作都会注明是否支持超级表。 ## 降采样查询、插值 @@ -122,7 +123,7 @@ Query OK, 6 rows in database (0.005515s) 如果一个时间间隔里,没有采集的数据,TDengine 还提供插值计算的功能。 -语法规则细节请见 [TAOS SQL 的按时间窗口切分聚合](../../taos-sql/distinguished) 章节。 +语法规则细节请见 [TDengine SQL 的按时间窗口切分聚合](../../taos-sql/distinguished) 章节。 ## 示例代码 diff --git a/docs/zh/07-develop/08-cache.md b/docs/zh/07-develop/08-cache.md index bd9da6062d..29e28e3dde 100644 --- a/docs/zh/07-develop/08-cache.md +++ b/docs/zh/07-develop/08-cache.md @@ -20,11 +20,11 @@ create database db0 vgroups 100 buffer 16MB ## 读缓存 -在创建数据库时可以选择是否缓存该数据库中每个子表的最新数据。由参数 cachelast 设置,分为三种情况: -- 0: 不缓存 -- 1: 缓存子表最近一行数据,这将显著改善 last_row 函数的性能 -- 2: 缓存子表每一列最近的非 NULL 值,这将显著改善无特殊影响(比如 WHERE, ORDER BY, GROUP BY, INTERVAL)时的 last 函数的性能 -- 3: 同时缓存行和列,即等同于上述 cachelast 值为 1 或 2 时的行为同时生效 +在创建数据库时可以选择是否缓存该数据库中每个子表的最新数据。由参数 cachemodel 设置,分为四种情况: +- none: 不缓存 +- last_row: 缓存子表最近一行数据,这将显著改善 last_row 函数的性能 +- last_value: 缓存子表每一列最近的非 NULL 值,这将显著改善无特殊影响(比如 WHERE, ORDER BY, GROUP BY, INTERVAL)时的 last 函数的性能 +- both: 同时缓存最近的行和列,即等同于上述 cachemodel 值为 last_row 和 last_value 的行为同时生效 ## 元数据缓存 diff --git a/docs/zh/07-develop/index.md b/docs/zh/07-develop/index.md index 20c0170844..efaffaea71 100644 --- a/docs/zh/07-develop/index.md +++ b/docs/zh/07-develop/index.md @@ -1,5 +1,7 @@ --- title: 开发指南 +sidebar_label: 开发指南 +description: 让开发者能够快速上手的指南 --- 开发一个应用,如果你准备采用TDengine作为时序数据处理的工具,那么有如下几个事情要做: diff --git a/docs/zh/08-connector/_01-error-code.md b/docs/zh/08-connector/_01-error-code.md index 53e006e108..3111d4bbf8 100644 --- a/docs/zh/08-connector/_01-error-code.md +++ b/docs/zh/08-connector/_01-error-code.md @@ -1,6 +1,7 @@ --- sidebar_label: 错误码 title: TDengine C/C++ 连接器错误码 +description: C/C++ 连接器的错误码列表和详细说明 --- 本文中详细列举了在使用 TDengine C/C++ 连接器时客户端可能得到的错误码以及所要采取的相应动作。其它语言的连接器在使用原生连接方式时也会所得到的返回码返回给连接器的调用者。 diff --git a/docs/zh/10-deployment/01-deploy.md b/docs/zh/10-deployment/01-deploy.md index 22a9c2ff8e..03b4ce30f9 100644 --- a/docs/zh/10-deployment/01-deploy.md +++ b/docs/zh/10-deployment/01-deploy.md @@ -1,6 +1,7 @@ --- sidebar_label: 手动部署 title: 集群部署和管理 +description: 使用命令行工具手动部署 TDengine 集群 --- ## 准备工作 @@ -70,7 +71,7 @@ serverPort 6030 ## 启动集群 -按照《立即开始》里的步骤,启动第一个数据节点,例如 h1.taosdata.com,然后执行 taos,启动 taos shell,从 shell 里执行命令“SHOW DNODES”,如下所示: +按照《立即开始》里的步骤,启动第一个数据节点,例如 h1.taosdata.com,然后执行 taos,启动 TDengine CLI,在其中执行命令 “SHOW DNODES”,如下所示: ``` taos> show dnodes; @@ -114,7 +115,7 @@ SHOW DNODES; 任何已经加入集群在线的数据节点,都可以作为后续待加入节点的 firstEp。 firstEp 这个参数仅仅在该数据节点首次加入集群时有作用,加入集群后,该数据节点会保存最新的 mnode 的 End Point 列表,不再依赖这个参数。 -接下来,配置文件中的 firstEp 参数就主要在客户端连接的时候使用了,例如 taos shell 如果不加参数,会默认连接由 firstEp 指定的节点。 +接下来,配置文件中的 firstEp 参数就主要在客户端连接的时候使用了,例如 TDengine CLI 如果不加参数,会默认连接由 firstEp 指定的节点。 两个没有配置 firstEp 参数的数据节点 dnode 启动后,会独立运行起来。这个时候,无法将其中一个数据节点加入到另外一个数据节点,形成集群。无法将两个独立的集群合并成为新的集群。 ::: diff --git a/docs/zh/10-deployment/03-k8s.md b/docs/zh/10-deployment/03-k8s.md index 396b834324..0cae59657c 100644 --- a/docs/zh/10-deployment/03-k8s.md +++ b/docs/zh/10-deployment/03-k8s.md @@ -1,6 +1,7 @@ --- sidebar_label: Kubernetes title: 在 Kubernetes 上部署 TDengine 集群 +description: 利用 Kubernetes 部署 TDengine 集群的详细指南 --- 作为面向云原生架构设计的时序数据库,TDengine 支持 Kubernetes 部署。这里介绍如何使用 YAML 文件一步一步从头创建一个 TDengine 集群,并重点介绍 Kubernetes 环境下 TDengine 的常用操作。 @@ -9,6 +10,7 @@ title: 在 Kubernetes 上部署 TDengine 集群 要使用 Kubernetes 部署管理 TDengine 集群,需要做好如下准备工作。 +* 本文适用 Kubernetes v1.5 以上版本 * 本文和下一章使用 minikube、kubectl 和 helm 等工具进行安装部署,请提前安装好相应软件 * Kubernetes 已经安装部署并能正常访问使用或更新必要的容器仓库或其他服务 @@ -365,7 +367,7 @@ kubectl scale statefulsets tdengine --replicas=1 ``` -在 taos shell 中的所有数据库操作将无法成功。 +在 TDengine CLI 中的所有数据库操作将无法成功。 ``` taos> show dnodes; diff --git a/docs/zh/10-deployment/05-helm.md b/docs/zh/10-deployment/05-helm.md index 9a723ff62f..9a3b21f092 100644 --- a/docs/zh/10-deployment/05-helm.md +++ b/docs/zh/10-deployment/05-helm.md @@ -1,6 +1,7 @@ --- sidebar_label: Helm title: 使用 Helm 部署 TDengine 集群 +description: 使用 Helm 部署 TDengine 集群的详细指南 --- Helm 是 Kubernetes 的包管理器,上一节使用 Kubernets 部署 TDengine 集群的操作已经足够简单,但 Helm 依然可以提供更强大的能力。 @@ -171,70 +172,19 @@ taoscfg: TAOS_REPLICA: "1" - # number of days per DB file - # TAOS_DAYS: "10" - - # number of days to keep DB file, default is 10 years. - #TAOS_KEEP: "3650" - - # cache block size (Mbyte) - #TAOS_CACHE: "16" - - # number of cache blocks per vnode - #TAOS_BLOCKS: "6" - - # minimum rows of records in file block - #TAOS_MIN_ROWS: "100" - - # maximum rows of records in file block - #TAOS_MAX_ROWS: "4096" - - # - # TAOS_NUM_OF_THREADS_PER_CORE: number of threads per CPU core - #TAOS_NUM_OF_THREADS_PER_CORE: "1.0" + # TAOS_NUM_OF_RPC_THREADS: number of threads for RPC + #TAOS_NUM_OF_RPC_THREADS: "2" # # TAOS_NUM_OF_COMMIT_THREADS: number of threads to commit cache data #TAOS_NUM_OF_COMMIT_THREADS: "4" - # - # TAOS_RATIO_OF_QUERY_CORES: - # the proportion of total CPU cores available for query processing - # 2.0: the query threads will be set to double of the CPU cores. - # 1.0: all CPU cores are available for query processing [default]. - # 0.5: only half of the CPU cores are available for query. - # 0.0: only one core available. - #TAOS_RATIO_OF_QUERY_CORES: "1.0" - - # - # TAOS_KEEP_COLUMN_NAME: - # the last_row/first/last aggregator will not change the original column name in the result fields - #TAOS_KEEP_COLUMN_NAME: "0" - - # enable/disable backuping vnode directory when removing vnode - #TAOS_VNODE_BAK: "1" - # enable/disable installation / usage report #TAOS_TELEMETRY_REPORTING: "1" - # enable/disable load balancing - #TAOS_BALANCE: "1" - - # max timer control blocks - #TAOS_MAX_TMR_CTRL: "512" - # time interval of system monitor, seconds #TAOS_MONITOR_INTERVAL: "30" - # number of seconds allowed for a dnode to be offline, for cluster only - #TAOS_OFFLINE_THRESHOLD: "8640000" - - # RPC re-try timer, millisecond - #TAOS_RPC_TIMER: "1000" - - # RPC maximum time for ack, seconds. - #TAOS_RPC_MAX_TIME: "600" - # time interval of dnode status reporting to mnode, seconds, for cluster only #TAOS_STATUS_INTERVAL: "1" @@ -245,37 +195,7 @@ taoscfg: #TAOS_MIN_SLIDING_TIME: "10" # minimum time window, milli-second - #TAOS_MIN_INTERVAL_TIME: "10" - - # maximum delay before launching a stream computation, milli-second - #TAOS_MAX_STREAM_COMP_DELAY: "20000" - - # maximum delay before launching a stream computation for the first time, milli-second - #TAOS_MAX_FIRST_STREAM_COMP_DELAY: "10000" - - # retry delay when a stream computation fails, milli-second - #TAOS_RETRY_STREAM_COMP_DELAY: "10" - - # the delayed time for launching a stream computation, from 0.1(default, 10% of whole computing time window) to 0.9 - #TAOS_STREAM_COMP_DELAY_RATIO: "0.1" - - # max number of vgroups per db, 0 means configured automatically - #TAOS_MAX_VGROUPS_PER_DB: "0" - - # max number of tables per vnode - #TAOS_MAX_TABLES_PER_VNODE: "1000000" - - # the number of acknowledgments required for successful data writing - #TAOS_QUORUM: "1" - - # enable/disable compression - #TAOS_COMP: "2" - - # write ahead log (WAL) level, 0: no wal; 1: write wal, but no fysnc; 2: write wal, and call fsync - #TAOS_WAL_LEVEL: "1" - - # if walLevel is set to 2, the cycle of fsync being executed, if set to 0, fsync is called right away - #TAOS_FSYNC: "3000" + #TAOS_MIN_INTERVAL_TIME: "1" # the compressed rpc message, option: # -1 (no compression) @@ -283,17 +203,8 @@ taoscfg: # > 0 (rpc message body which larger than this value will be compressed) #TAOS_COMPRESS_MSG_SIZE: "-1" - # max length of an SQL - #TAOS_MAX_SQL_LENGTH: "1048576" - - # the maximum number of records allowed for super table time sorting - #TAOS_MAX_NUM_OF_ORDERED_RES: "100000" - # max number of connections allowed in dnode - #TAOS_MAX_SHELL_CONNS: "5000" - - # max number of connections allowed in client - #TAOS_MAX_CONNECTIONS: "5000" + #TAOS_MAX_SHELL_CONNS: "50000" # stop writing logs when the disk size of the log folder is less than this value #TAOS_MINIMAL_LOG_DIR_G_B: "0.1" @@ -313,21 +224,8 @@ taoscfg: # enable/disable system monitor #TAOS_MONITOR: "1" - # enable/disable recording the SQL statements via restful interface - #TAOS_HTTP_ENABLE_RECORD_SQL: "0" - - # number of threads used to process http requests - #TAOS_HTTP_MAX_THREADS: "2" - - # maximum number of rows returned by the restful interface - #TAOS_RESTFUL_ROW_LIMIT: "10240" - - # The following parameter is used to limit the maximum number of lines in log files. - # max number of lines per log filters - # numOfLogLines 10000000 - # enable/disable async log - #TAOS_ASYNC_LOG: "0" + #TAOS_ASYNC_LOG: "1" # # time of keeping log files, days @@ -344,25 +242,8 @@ taoscfg: # debug flag for all log type, take effect when non-zero value\ #TAOS_DEBUG_FLAG: "143" - # enable/disable recording the SQL in taos client - #TAOS_ENABLE_RECORD_SQL: "0" - # generate core file when service crash #TAOS_ENABLE_CORE_FILE: "1" - - # maximum display width of binary and nchar fields in the shell. The parts exceeding this limit will be hidden - #TAOS_MAX_BINARY_DISPLAY_WIDTH: "30" - - # enable/disable stream (continuous query) - #TAOS_STREAM: "1" - - # in retrieve blocking model, only in 50% query threads will be used in query processing in dnode - #TAOS_RETRIEVE_BLOCKING_MODEL: "0" - - # the maximum allowed query buffer size in MB during query processing for each data node - # -1 no limit (default) - # 0 no query allowed, queries are disabled - #TAOS_QUERY_BUFFER_SIZE: "-1" ``` ## 扩容 diff --git a/docs/zh/10-deployment/index.md b/docs/zh/10-deployment/index.md index 96ac7b176d..4ff1add779 100644 --- a/docs/zh/10-deployment/index.md +++ b/docs/zh/10-deployment/index.md @@ -1,5 +1,7 @@ --- +sidebar_label: 部署集群 title: 部署集群 +description: 部署 TDengine 集群的多种方式 --- TDengine 支持集群,提供水平扩展的能力。如果需要获得更高的处理能力,只需要多增加节点即可。TDengine 采用虚拟节点技术,将一个节点虚拟化为多个虚拟节点,以实现负载均衡。同时,TDengine可以将多个节点上的虚拟节点组成虚拟节点组,通过多副本机制,以保证供系统的高可用。TDengine的集群功能完全开源。 diff --git a/docs/zh/12-taos-sql/01-data-type.md b/docs/zh/12-taos-sql/01-data-type.md index 628086f5a9..b8ef050fb7 100644 --- a/docs/zh/12-taos-sql/01-data-type.md +++ b/docs/zh/12-taos-sql/01-data-type.md @@ -11,7 +11,7 @@ description: "TDengine 支持的数据类型: 时间戳、浮点型、JSON 类 - 时间格式为 `YYYY-MM-DD HH:mm:ss.MS`,默认时间分辨率为毫秒。比如:`2017-08-12 18:25:58.128` - 内部函数 now 是客户端的当前时间 - 插入记录时,如果时间戳为 now,插入数据时使用提交这条记录的客户端的当前时间 -- Epoch Time:时间戳也可以是一个长整数,表示从格林威治时间 1970-01-01 00:00:00.000 (UTC/GMT) 开始的毫秒数(相应地,如果所在 Database 的时间精度设置为“微秒”,则长整型格式的时间戳含义也就对应于从格林威治时间 1970-01-01 00:00:00.000 (UTC/GMT) 开始的微秒数;纳秒精度逻辑类似。) +- Epoch Time:时间戳也可以是一个长整数,表示从 UTC 时间 1970-01-01 00:00:00 开始的毫秒数。相应地,如果所在 Database 的时间精度设置为“微秒”,则长整型格式的时间戳含义也就对应于从 UTC 时间 1970-01-01 00:00:00 开始的微秒数;纳秒精度逻辑类似。 - 时间可以加减,比如 now-2h,表明查询时刻向前推 2 个小时(最近 2 小时)。数字后面的时间单位可以是 b(纳秒)、u(微秒)、a(毫秒)、s(秒)、m(分)、h(小时)、d(天)、w(周)。 比如 `select * from t1 where ts > now-2w and ts <= now-1w`,表示查询两周前整整一周的数据。在指定降采样操作(down sampling)的时间窗口(interval)时,时间单位还可以使用 n (自然月) 和 y (自然年)。 TDengine 缺省的时间戳精度是毫秒,但通过在 `CREATE DATABASE` 时传递的 PRECISION 参数也可以支持微秒和纳秒。 diff --git a/docs/zh/12-taos-sql/03-table.md b/docs/zh/12-taos-sql/03-table.md index 0e104bb7b6..9c33c45efc 100644 --- a/docs/zh/12-taos-sql/03-table.md +++ b/docs/zh/12-taos-sql/03-table.md @@ -1,5 +1,7 @@ --- title: 表管理 +sidebar_label: 表 +description: 对表的各种管理操作 --- ## 创建表 @@ -8,27 +10,27 @@ title: 表管理 ```sql CREATE TABLE [IF NOT EXISTS] [db_name.]tb_name (create_definition [, create_definitionn] ...) [table_options] - + CREATE TABLE create_subtable_clause - + CREATE TABLE [IF NOT EXISTS] [db_name.]tb_name (create_definition [, create_definitionn] ...) [TAGS (create_definition [, create_definitionn] ...)] [table_options] - + create_subtable_clause: { create_subtable_clause [create_subtable_clause] ... | [IF NOT EXISTS] [db_name.]tb_name USING [db_name.]stb_name [(tag_name [, tag_name] ...)] TAGS (tag_value [, tag_value] ...) } - + create_definition: col_name column_definition - + column_definition: type_name [comment 'string_value'] - + table_options: table_option ... - + table_option: { COMMENT 'string_value' | WATERMARK duration[,duration] @@ -52,12 +54,13 @@ table_option: { 需要注意的是转义字符中的内容必须是可打印字符。 **参数说明** + 1. COMMENT:表注释。可用于超级表、子表和普通表。 -2. WATERMARK:指定窗口的关闭时间,默认值为 5 秒,最小单位毫秒,范围为0到15分钟,多个以逗号分隔。只可用于超级表,且只有当数据库使用了RETENTIONS参数时,才可以使用此表参数。 -3. MAX_DELAY:用于控制推送计算结果的最大延迟,默认值为 interval 的值(但不能超过最大值),最小单位毫秒,范围为1毫秒到15分钟,多个以逗号分隔。注:不建议 MAX_DELAY 设置太小,否则会过于频繁的推送结果,影响存储和查询性能,如无特殊需求,取默认值即可。只可用于超级表,且只有当数据库使用了RETENTIONS参数时,才可以使用此表参数。 -4. ROLLUP:Rollup 指定的聚合函数,提供基于多层级的降采样聚合结果。只可用于超级表。只有当数据库使用了RETENTIONS参数时,才可以使用此表参数。作用于超级表除TS列外的其它所有列,但是只能定义一个聚合函数。 聚合函数支持 avg, sum, min, max, last, first。 -5. SMA:Small Materialized Aggregates,提供基于数据块的自定义预计算功能。预计算类型包括MAX、MIN和SUM。可用于超级表/普通表。 -6. TTL:Time to Live,是用户用来指定表的生命周期的参数。如果在持续的TTL时间内,都没有数据写入该表,则TDengine系统会自动删除该表。这个TTL的时间只是一个大概时间,我们系统不保证到了时间一定会将其删除,而只保证存在这样一个机制。TTL单位是天,默认为0,表示不限制。用户需要注意,TTL优先级高于KEEP,即TTL时间满足删除机制时,即使当前数据的存在时间小于KEEP,此表也会被删除。只可用于子表和普通表。 +2. WATERMARK:指定窗口的关闭时间,默认值为 5 秒,最小单位毫秒,范围为 0 到 15 分钟,多个以逗号分隔。只可用于超级表,且只有当数据库使用了 RETENTIONS 参数时,才可以使用此表参数。 +3. MAX_DELAY:用于控制推送计算结果的最大延迟,默认值为 interval 的值(但不能超过最大值),最小单位毫秒,范围为 1 毫秒到 15 分钟,多个以逗号分隔。注:不建议 MAX_DELAY 设置太小,否则会过于频繁的推送结果,影响存储和查询性能,如无特殊需求,取默认值即可。只可用于超级表,且只有当数据库使用了 RETENTIONS 参数时,才可以使用此表参数。 +4. ROLLUP:Rollup 指定的聚合函数,提供基于多层级的降采样聚合结果。只可用于超级表。只有当数据库使用了 RETENTIONS 参数时,才可以使用此表参数。作用于超级表除 TS 列外的其它所有列,但是只能定义一个聚合函数。 聚合函数支持 avg, sum, min, max, last, first。 +5. SMA:Small Materialized Aggregates,提供基于数据块的自定义预计算功能。预计算类型包括 MAX、MIN 和 SUM。可用于超级表/普通表。 +6. TTL:Time to Live,是用户用来指定表的生命周期的参数。如果创建表时指定了这个参数,当该表的存在时间超过 TTL 指定的时间后,TDengine 自动删除该表。这个 TTL 的时间只是一个大概时间,系统不保证到了时间一定会将其删除,而只保证存在这样一个机制且最终一定会删除。TTL 单位是天,默认为 0,表示不限制,到期时间为表创建时间加上 TTL 时间。 ## 创建子表 @@ -87,7 +90,7 @@ CREATE TABLE [IF NOT EXISTS] tb_name1 USING stb_name TAGS (tag_value1, ...) [IF ```sql ALTER TABLE [db_name.]tb_name alter_table_clause - + alter_table_clause: { alter_table_options | ADD COLUMN col_name column_type @@ -95,10 +98,10 @@ alter_table_clause: { | MODIFY COLUMN col_name column_type | RENAME COLUMN old_col_name new_col_name } - + alter_table_options: alter_table_option ... - + alter_table_option: { TTL value | COMMENT 'string_value' @@ -108,6 +111,7 @@ alter_table_option: { **使用说明** 对普通表可以进行如下修改操作 + 1. ADD COLUMN:添加列。 2. DROP COLUMN:删除列。 3. MODIFY COLUMN:修改列定义,如果数据列的类型是可变长类型,那么可以使用此指令修改其宽度,只能改大,不能改小。 @@ -141,15 +145,15 @@ ALTER TABLE tb_name RENAME COLUMN old_col_name new_col_name ```sql ALTER TABLE [db_name.]tb_name alter_table_clause - + alter_table_clause: { alter_table_options | SET TAG tag_name = new_tag_value } - + alter_table_options: alter_table_option ... - + alter_table_option: { TTL value | COMMENT 'string_value' @@ -157,6 +161,7 @@ alter_table_option: { ``` **使用说明** + 1. 对子表的列和标签的修改,除了更改标签值以外,都要通过超级表才能进行。 ### 修改子表标签值 @@ -167,7 +172,7 @@ ALTER TABLE tb_name SET TAG tag_name=new_tag_value; ## 删除表 -可以在一条SQL语句中删除一个或多个普通表或子表。 +可以在一条 SQL 语句中删除一个或多个普通表或子表。 ```sql DROP TABLE [IF EXISTS] [db_name.]tb_name [, [IF EXISTS] [db_name.]tb_name] ... @@ -177,7 +182,7 @@ DROP TABLE [IF EXISTS] [db_name.]tb_name [, [IF EXISTS] [db_name.]tb_name] ... ### 显示所有表 -如下SQL语句可以列出当前数据库中的所有表名。 +如下 SQL 语句可以列出当前数据库中的所有表名。 ```sql SHOW TABLES [LIKE tb_name_wildchar]; diff --git a/docs/zh/12-taos-sql/04-stable.md b/docs/zh/12-taos-sql/04-stable.md index 59d9657694..450ff07fd8 100644 --- a/docs/zh/12-taos-sql/04-stable.md +++ b/docs/zh/12-taos-sql/04-stable.md @@ -1,6 +1,7 @@ --- sidebar_label: 超级表管理 title: 超级表 STable 管理 +description: 对超级表的各种管理操作 --- ## 创建超级表 diff --git a/docs/zh/12-taos-sql/05-insert.md b/docs/zh/12-taos-sql/05-insert.md index c91e70c481..59af9c55ed 100644 --- a/docs/zh/12-taos-sql/05-insert.md +++ b/docs/zh/12-taos-sql/05-insert.md @@ -1,6 +1,7 @@ --- sidebar_label: 数据写入 title: 数据写入 +description: 写入数据的详细语法 --- ## 写入语法 diff --git a/docs/zh/12-taos-sql/06-select.md b/docs/zh/12-taos-sql/06-select.md index 5312d7d2f3..d8ff3f04ed 100644 --- a/docs/zh/12-taos-sql/06-select.md +++ b/docs/zh/12-taos-sql/06-select.md @@ -1,6 +1,7 @@ --- sidebar_label: 数据查询 title: 数据查询 +description: 查询数据的详细语法 --- ## 查询语法 @@ -354,19 +355,15 @@ SELECT ... FROM (SELECT ... FROM ...) ...; :::info -- 目前仅支持一层嵌套,也即不能在子查询中再嵌入子查询。 -- 内层查询的返回结果将作为“虚拟表”供外层查询使用,此虚拟表可以使用 AS 语法做重命名,以便于外层查询中方便引用。 -- 目前不能在“连续查询”功能中使用子查询。 +- 内层查询的返回结果将作为“虚拟表”供外层查询使用,此虚拟表建议起别名,以便于外层查询中方便引用。 - 在内层和外层查询中,都支持普通的表间/超级表间 JOIN。内层查询的计算结果也可以再参与数据子表的 JOIN 操作。 -- 目前内层查询、外层查询均不支持 UNION 操作。 - 内层查询支持的功能特性与非嵌套的查询语句能力是一致的。 - 内层查询的 ORDER BY 子句一般没有意义,建议避免这样的写法以免无谓的资源消耗。 - 与非嵌套的查询语句相比,外层查询所能支持的功能特性存在如下限制: - 计算函数部分: - - 如果内层查询的结果数据未提供时间戳,那么计算过程依赖时间戳的函数在外层会无法正常工作。例如:TOP, BOTTOM, FIRST, LAST, DIFF。 - - 计算过程需要两遍扫描的函数,在外层查询中无法正常工作。例如:此类函数包括:STDDEV, PERCENTILE。 - - 外层查询中不支持 IN 算子,但在内层中可以使用。 - - 外层查询不支持 GROUP BY。 + - 如果内层查询的结果数据未提供时间戳,那么计算过程隐式依赖时间戳的函数在外层会无法正常工作。例如:INTERP, DERIVATIVE, IRATE, LAST_ROW, FIRST, LAST, TWA, STATEDURATION, TAIL, UNIQUE。 + - 如果内层查询的结果数据不是有效的时间序列,那么计算过程依赖数据为时间序列的函数在外层会无法正常工作。例如:LEASTSQUARES, ELAPSED, INTERP, DERIVATIVE, IRATE, TWA, DIFF, STATECOUNT, STATEDURATION, CSUM, MAVG, TAIL, UNIQUE。 + - 计算过程需要两遍扫描的函数,在外层查询中无法正常工作。例如:此类函数包括:PERCENTILE。 ::: diff --git a/docs/zh/12-taos-sql/10-function.md b/docs/zh/12-taos-sql/10-function.md index e99d915101..9f999181c4 100644 --- a/docs/zh/12-taos-sql/10-function.md +++ b/docs/zh/12-taos-sql/10-function.md @@ -1,6 +1,7 @@ --- sidebar_label: 函数 title: 函数 +description: TDengine 支持的函数列表 toc_max_heading_level: 4 --- @@ -1166,7 +1167,7 @@ SELECT stateDuration(field_name, oper, val, unit) FROM { tb_name | stb_name } [W **参数范围**: -- oper : "LT" (小于)、"GT"(大于)、"LE"(小于等于)、"GE"(大于等于)、"NE"(不等于)、"EQ"(等于),不区分大小写。 +- oper : `'LT'` (小于)、`'GT'`(大于)、`'LE'`(小于等于)、`'GE'`(大于等于)、`'NE'`(不等于)、`'EQ'`(等于),不区分大小写,但需要用`''`包括。 - val : 数值型 - unit : 时间长度的单位,可取值时间单位: 1b(纳秒), 1u(微秒),1a(毫秒),1s(秒),1m(分),1h(小时),1d(天), 1w(周)。如果省略,默认为当前数据库精度。 diff --git a/docs/zh/12-taos-sql/12-distinguished.md b/docs/zh/12-taos-sql/12-distinguished.md index 2dad49ece9..016c1929fe 100644 --- a/docs/zh/12-taos-sql/12-distinguished.md +++ b/docs/zh/12-taos-sql/12-distinguished.md @@ -1,15 +1,16 @@ --- sidebar_label: 时序数据特色查询 title: 时序数据特色查询 +description: TDengine 提供的时序数据特有的查询功能 --- TDengine 是专为时序数据而研发的大数据平台,存储和计算都针对时序数据的特定进行了量身定制,在支持标准 SQL 的基础之上,还提供了一系列贴合时序业务场景的特色查询语法,极大的方便时序场景的应用开发。 -TDengine 提供的特色查询包括标签切分查询和窗口切分查询。 +TDengine 提供的特色查询包括数据切分查询和窗口切分查询。 -## 标签切分查询 +## 数据切分查询 -超级表查询中,当需要针对标签进行数据切分然后在切分出的数据空间内再进行一系列的计算时使用标签切分子句,标签切分的语句如下: +当需要按一定的维度对数据进行切分然后在切分出的数据空间内再进行一系列的计算时使用数据切分子句,数据切分语句的语法如下: ```sql PARTITION BY part_list @@ -17,22 +18,23 @@ PARTITION BY part_list part_list 可以是任意的标量表达式,包括列、常量、标量函数和它们的组合。 -当 PARTITION BY 和标签一起使用时,TDengine 按如下方式处理标签切分子句: +TDengine 按如下方式处理数据切分子句: -- 标签切分子句位于 WHERE 子句之后,且不能和 JOIN 子句一起使用。 -- 标签切分子句将超级表数据按指定的标签组合进行切分,每个切分的分片进行指定的计算。计算由之后的子句定义(窗口子句、GROUP BY 子句或 SELECT 子句)。 -- 标签切分子句可以和窗口切分子句(或 GROUP BY 子句)一起使用,此时后面的子句作用在每个切分的分片上。例如,将数据按标签 location 进行分组,并对每个组按 10 分钟进行降采样,取其最大值。 +- 数据切分子句位于 WHERE 子句之后。 +- 数据切分子句将表数据按指定的维度进行切分,每个切分的分片进行指定的计算。计算由之后的子句定义(窗口子句、GROUP BY 子句或 SELECT 子句)。 +- 数据切分子句可以和窗口切分子句(或 GROUP BY 子句)一起使用,此时后面的子句作用在每个切分的分片上。例如,将数据按标签 location 进行分组,并对每个组按 10 分钟进行降采样,取其最大值。 ```sql select max(current) from meters partition by location interval(10m) ``` +数据切分子句最常见的用法就是在超级表查询中,按标签将子表数据进行切分,然后分别进行计算。特别是 PARTITION BY TBNAME 用法,它将每个子表的数据独立出来,形成一条条独立的时间序列,极大的方便了各种时序场景的统计分析。 ## 窗口切分查询 TDengine 支持按时间段窗口切分方式进行聚合结果查询,比如温度传感器每秒采集一次数据,但需查询每隔 10 分钟的温度平均值。这种场景下可以使用窗口子句来获得需要的查询结果。窗口子句用于针对查询的数据集合按照窗口切分成为查询子集并进行聚合,窗口包含时间窗口(time window)、状态窗口(status window)、会话窗口(session window)三种窗口。其中时间窗口又可划分为滑动时间窗口和翻转时间窗口。窗口切分查询语法如下: ```sql -SELECT function_list FROM tb_name +SELECT select_list FROM tb_name [WHERE where_condition] [SESSION(ts_col, tol_val)] [STATE_WINDOW(col)] @@ -42,19 +44,15 @@ SELECT function_list FROM tb_name 在上述语法中的具体限制如下 -### 窗口切分查询中使用函数的限制 - -- 在聚合查询中,function_list 位置允许使用聚合和选择函数,并要求每个函数仅输出单个结果(例如:COUNT、AVG、SUM、STDDEV、LEASTSQUARES、PERCENTILE、MIN、MAX、FIRST、LAST),而不能使用具有多行输出结果的函数(例如:DIFF 以及四则运算)。 -- 此外 LAST_ROW 查询也不能与窗口聚合同时出现。 -- 标量函数(如:CEIL/FLOOR 等)也不能使用在窗口聚合查询中。 - ### 窗口子句的规则 -- 窗口子句位于标签切分子句之后,GROUP BY 子句之前,且不可以和 GROUP BY 子句一起使用。 +- 窗口子句位于数据切分子句之后,GROUP BY 子句之前,且不可以和 GROUP BY 子句一起使用。 - 窗口子句将数据按窗口进行切分,对每个窗口进行 SELECT 列表中的表达式的计算,SELECT 列表中的表达式只能包含: - 常量。 - - 聚集函数。 + - _wstart伪列、_wend伪列和_wduration伪列。 + - 聚集函数(包括选择函数和可以由参数确定输出行数的时序特有函数)。 - 包含上面表达式的表达式。 + - 且至少包含一个聚集函数。 - 窗口子句不可以和 GROUP BY 子句一起使用。 - WHERE 语句可以指定查询的起止时间和其他过滤条件。 @@ -73,7 +71,7 @@ FILL 语句指定某一窗口区间数据缺失的情况下的填充模式。填 1. 使用 FILL 语句的时候可能生成大量的填充输出,务必指定查询的时间区间。针对每次查询,系统可返回不超过 1 千万条具有插值的结果。 2. 在时间维度聚合中,返回的结果中时间序列严格单调递增。 -3. 如果查询对象是超级表,则聚合函数会作用于该超级表下满足值过滤条件的所有表的数据。如果查询中没有使用 GROUP BY 语句,则返回的结果按照时间序列严格单调递增;如果查询中使用了 GROUP BY 语句分组,则返回结果中每个 GROUP 内不按照时间序列严格单调递增。 +3. 如果查询对象是超级表,则聚合函数会作用于该超级表下满足值过滤条件的所有表的数据。如果查询中没有使用 PARTITION BY 语句,则返回的结果按照时间序列严格单调递增;如果查询中使用了 PARTITION BY 语句分组,则返回结果中每个 PARTITION 内不按照时间序列严格单调递增。 ::: @@ -105,7 +103,7 @@ SELECT COUNT(*) FROM temp_tb_1 INTERVAL(1m) SLIDING(2m); ### 状态窗口 -使用整数(布尔值)或字符串来标识产生记录时候设备的状态量。产生的记录如果具有相同的状态量数值则归属于同一个状态窗口,数值改变后该窗口关闭。如下图所示,根据状态量确定的状态窗口分别是[2019-04-28 14:22:07,2019-04-28 14:22:10]和[2019-04-28 14:22:11,2019-04-28 14:22:12]两个。(状态窗口暂不支持对超级表使用) +使用整数(布尔值)或字符串来标识产生记录时候设备的状态量。产生的记录如果具有相同的状态量数值则归属于同一个状态窗口,数值改变后该窗口关闭。如下图所示,根据状态量确定的状态窗口分别是[2019-04-28 14:22:07,2019-04-28 14:22:10]和[2019-04-28 14:22:11,2019-04-28 14:22:12]两个。 ![TDengine Database 时间窗口示意图](./timewindow-3.webp) @@ -121,7 +119,7 @@ SELECT COUNT(*), FIRST(ts), status FROM temp_tb_1 STATE_WINDOW(status); ![TDengine Database 时间窗口示意图](./timewindow-2.webp) -在 tol_value 时间间隔范围内的结果都认为归属于同一个窗口,如果连续的两条记录的时间超过 tol_val,则自动开启下一个窗口。(会话窗口暂不支持对超级表使用) +在 tol_value 时间间隔范围内的结果都认为归属于同一个窗口,如果连续的两条记录的时间超过 tol_val,则自动开启下一个窗口。 ``` diff --git a/docs/zh/12-taos-sql/13-tmq.md b/docs/zh/12-taos-sql/13-tmq.md index b05d2bf680..571300ad8c 100644 --- a/docs/zh/12-taos-sql/13-tmq.md +++ b/docs/zh/12-taos-sql/13-tmq.md @@ -1,6 +1,7 @@ --- sidebar_label: 数据订阅 title: 数据订阅 +description: TDengine 消息队列提供的数据订阅功能 --- TDengine 3.0.0.0 开始对消息队列做了大幅的优化和增强以简化用户的解决方案。 diff --git a/docs/zh/12-taos-sql/14-stream.md b/docs/zh/12-taos-sql/14-stream.md index 28f52be59a..70b062a6ca 100644 --- a/docs/zh/12-taos-sql/14-stream.md +++ b/docs/zh/12-taos-sql/14-stream.md @@ -1,6 +1,7 @@ --- sidebar_label: 流式计算 title: 流式计算 +description: 流式计算的相关 SQL 的详细语法 --- diff --git a/docs/zh/12-taos-sql/16-operators.md b/docs/zh/12-taos-sql/16-operators.md index 22b78455fb..48e9991799 100644 --- a/docs/zh/12-taos-sql/16-operators.md +++ b/docs/zh/12-taos-sql/16-operators.md @@ -1,6 +1,7 @@ --- sidebar_label: 运算符 title: 运算符 +description: TDengine 支持的所有运算符 --- ## 算术运算符 diff --git a/docs/zh/12-taos-sql/17-json.md b/docs/zh/12-taos-sql/17-json.md index 4a4a8cca73..4cbd8eef36 100644 --- a/docs/zh/12-taos-sql/17-json.md +++ b/docs/zh/12-taos-sql/17-json.md @@ -1,6 +1,7 @@ --- sidebar_label: JSON 类型使用说明 title: JSON 类型使用说明 +description: 对 JSON 类型如何使用的详细说明 --- diff --git a/docs/zh/12-taos-sql/18-escape.md b/docs/zh/12-taos-sql/18-escape.md index d478340599..7e543743a3 100644 --- a/docs/zh/12-taos-sql/18-escape.md +++ b/docs/zh/12-taos-sql/18-escape.md @@ -1,5 +1,7 @@ --- title: 转义字符说明 +sidebar_label: 转义字符 +description: TDengine 中使用转义字符的详细规则 --- ## 转义字符表 diff --git a/docs/zh/12-taos-sql/19-limit.md b/docs/zh/12-taos-sql/19-limit.md index ff552fc977..0dbe00f800 100644 --- a/docs/zh/12-taos-sql/19-limit.md +++ b/docs/zh/12-taos-sql/19-limit.md @@ -1,6 +1,7 @@ --- sidebar_label: 命名与边界限制 title: 命名与边界限制 +description: 合法字符集和命名中的限制规则 --- ## 名称命名规则 @@ -30,7 +31,7 @@ title: 命名与边界限制 - 最多允许 4096 列,最少需要 2 列,第一列必须是时间戳。 - 标签名最大长度为 64 - 最多允许 128 个,至少要有 1 个标签,一个表中标签值的总长度不超过 16KB -- SQL 语句最大长度 1048576 个字符,也可通过客户端配置参数 maxSQLLength 修改,取值范围 65480 ~ 1048576 +- SQL 语句最大长度 1048576 个字符 - SELECT 语句的查询结果,最多允许返回 4096 列(语句中的函数调用可能也会占用一些列空间),超限时需要显式指定较少的返回数据列,以避免语句执行报错 - 库的数目,超级表的数目、表的数目,系统不做限制,仅受系统资源限制 - 数据库的副本数只能设置为 1 或 3 diff --git a/docs/zh/12-taos-sql/20-keywords.md b/docs/zh/12-taos-sql/20-keywords.md index cac29d7863..047c6b08c9 100644 --- a/docs/zh/12-taos-sql/20-keywords.md +++ b/docs/zh/12-taos-sql/20-keywords.md @@ -1,6 +1,7 @@ --- sidebar_label: 保留关键字 title: TDengine 保留关键字 +description: TDengine 保留关键字的详细列表 --- ## 保留关键字 diff --git a/docs/zh/12-taos-sql/21-node.md b/docs/zh/12-taos-sql/21-node.md index 4816daf420..d47dc8198f 100644 --- a/docs/zh/12-taos-sql/21-node.md +++ b/docs/zh/12-taos-sql/21-node.md @@ -1,6 +1,7 @@ --- sidebar_label: 集群管理 title: 集群管理 +description: 管理集群的 SQL 命令的详细解析 --- 组成 TDengine 集群的物理实体是 dnode (data node 的缩写),它是一个运行在操作系统之上的进程。在 dnode 中可以建立负责时序数据存储的 vnode (virtual node),在多节点集群环境下当某个数据库的 replica 为 3 时,该数据库中的每个 vgroup 由 3 个 vnode 组成;当数据库的 replica 为 1 时,该数据库中的每个 vgroup 由 1 个 vnode 组成。如果要想配置某个数据库为多副本,则集群中的 dnode 数量至少为 3。在 dnode 还可以创建 mnode (management node),单个集群中最多可以创建三个 mnode。在 TDengine 3.0.0.0 中为了支持存算分离,引入了一种新的逻辑节点 qnode (query node),qnode 和 vnode 既可以共存在一个 dnode 中,也可以完全分离在不同的 dnode 上。 diff --git a/docs/zh/12-taos-sql/22-meta.md b/docs/zh/12-taos-sql/22-meta.md index 8139b2fc55..e9cda45b0f 100644 --- a/docs/zh/12-taos-sql/22-meta.md +++ b/docs/zh/12-taos-sql/22-meta.md @@ -1,6 +1,7 @@ --- sidebar_label: 元数据 title: 存储元数据的 Information_Schema 数据库 +description: Information_Schema 数据库中存储了系统中所有的元数据信息 --- TDengine 内置了一个名为 `INFORMATION_SCHEMA` 的数据库,提供对数据库元数据、数据库系统信息和状态的访问,例如数据库或表的名称,当前执行的 SQL 语句等。该数据库存储有关 TDengine 维护的所有其他数据库的信息。它包含多个只读表。实际上,这些表都是视图,而不是基表,因此没有与它们关联的文件。所以对这些表只能查询,不能进行 INSERT 等写入操作。`INFORMATION_SCHEMA` 数据库旨在以一种更一致的方式来提供对 TDengine 支持的各种 SHOW 语句(如 SHOW TABLES、SHOW DATABASES)所提供的信息的访问。与 SHOW 语句相比,使用 SELECT ... FROM INFORMATION_SCHEMA.tablename 具有以下优点: diff --git a/docs/zh/12-taos-sql/23-perf.md b/docs/zh/12-taos-sql/23-perf.md index ac852ee150..e6ff4960a7 100644 --- a/docs/zh/12-taos-sql/23-perf.md +++ b/docs/zh/12-taos-sql/23-perf.md @@ -1,6 +1,7 @@ --- sidebar_label: 统计数据 title: 存储统计数据的 Performance_Schema 数据库 +description: Performance_Schema 数据库中存储了系统中的各种统计信息 --- TDengine 3.0 版本开始提供一个内置数据库 `performance_schema`,其中存储了与性能有关的统计数据。本节详细介绍其中的表和表结构。 diff --git a/docs/zh/12-taos-sql/24-show.md b/docs/zh/12-taos-sql/24-show.md index 75efd5f514..b4aafdaa0a 100644 --- a/docs/zh/12-taos-sql/24-show.md +++ b/docs/zh/12-taos-sql/24-show.md @@ -1,6 +1,7 @@ --- sidebar_label: SHOW 命令 title: 使用 SHOW 命令查看系统元数据 +description: SHOW 命令的完整列表 --- SHOW 命令可以用来获取简要的系统信息。若想获取系统中详细的各种元数据、系统信息和状态,请使用 select 语句查询 INFORMATION_SCHEMA 数据库中的表。 @@ -194,7 +195,7 @@ SHOW STREAMS; SHOW SUBSCRIPTIONS; ``` -显示当前数据库下的所有的订阅关系 +显示当前系统内所有的订阅关系 ## SHOW TABLES diff --git a/docs/zh/12-taos-sql/25-grant.md b/docs/zh/12-taos-sql/25-grant.md index c41a3fcfc9..7fb9447101 100644 --- a/docs/zh/12-taos-sql/25-grant.md +++ b/docs/zh/12-taos-sql/25-grant.md @@ -1,6 +1,7 @@ --- sidebar_label: 权限管理 title: 权限管理 +description: 企业版中才具有的权限管理功能 --- 本节讲述如何在 TDengine 中进行权限管理的相关操作。 @@ -8,14 +9,51 @@ title: 权限管理 ## 创建用户 ```sql -CREATE USER use_name PASS 'password'; +CREATE USER use_name PASS 'password' [SYSINFO {1|0}]; ``` 创建用户。 -use_name最长为23字节。 +use_name 最长为 23 字节。 -password最长为128字节,合法字符包括"a-zA-Z0-9!?$%^&*()_–+={[}]:;@~#|<,>.?/",不可以出现单双引号、撇号、反斜杠和空格,且不可以为空。 +password 最长为 128 字节,合法字符包括"a-zA-Z0-9!?$%^&*()_–+={[}]:;@~#|<,>.?/",不可以出现单双引号、撇号、反斜杠和空格,且不可以为空。 + +SYSINFO 表示用户是否可以查看系统信息。1 表示可以查看,0 表示不可以查看。系统信息包括服务端配置信息、服务端各种节点信息(如 DNODE、QNODE等)、存储相关的信息等。默认为可以查看系统信息。 + +例如,创建密码为123456且可以查看系统信息的用户test如下: + +```sql +taos> create user test pass '123456' sysinfo 1; +Query OK, 0 of 0 rows affected (0.001254s) +``` + +## 查看用户 + +```sql +SHOW USERS; +``` + +查看用户信息。 + +```sql +taos> show users; + name | super | enable | sysinfo | create_time | +================================================================================ + test | 0 | 1 | 1 | 2022-08-29 15:10:27.315 | + root | 1 | 1 | 1 | 2022-08-29 15:03:34.710 | +Query OK, 2 rows in database (0.001657s) +``` + +也可以通过查询INFORMATION_SCHEMA.INS_USERS系统表来查看用户信息,例如: + +```sql +taos> select * from information_schema.ins_users; + name | super | enable | sysinfo | create_time | +================================================================================ + test | 0 | 1 | 1 | 2022-08-29 15:10:27.315 | + root | 1 | 1 | 1 | 2022-08-29 15:03:34.710 | +Query OK, 2 rows in database (0.001953s) +``` ## 删除用户 @@ -36,9 +74,15 @@ alter_user_clause: { ``` - PASS:修改用户密码。 -- ENABLE:修改用户是否启用。1表示启用此用户,0表示禁用此用户。 -- SYSINFO:修改用户是否可查看系统信息。1表示可以查看系统信息,0表示不可以查看系统信息。 +- ENABLE:修改用户是否启用。1 表示启用此用户,0 表示禁用此用户。 +- SYSINFO:修改用户是否可查看系统信息。1 表示可以查看系统信息,0 表示不可以查看系统信息。 +例如,禁用 test 用户: + +```sql +taos> alter user test enable 0; +Query OK, 0 of 0 rows affected (0.001160s) +``` ## 授权 @@ -61,7 +105,7 @@ priv_level : { } ``` -对用户授权。 +对用户授权。授权功能只包含在企业版中。 授权级别支持到DATABASE,权限有READ和WRITE两种。 @@ -91,4 +135,4 @@ priv_level : { ``` -收回对用户的授权。 +收回对用户的授权。授权功能只包含在企业版中。 diff --git a/docs/zh/12-taos-sql/26-udf.md b/docs/zh/12-taos-sql/26-udf.md index 7ddcad298b..764fde6e1f 100644 --- a/docs/zh/12-taos-sql/26-udf.md +++ b/docs/zh/12-taos-sql/26-udf.md @@ -1,6 +1,7 @@ --- sidebar_label: 自定义函数 title: 用户自定义函数 +description: 使用 UDF 的详细指南 --- 除了 TDengine 的内置函数以外,用户还可以编写自己的函数逻辑并加入TDengine系统中。 diff --git a/docs/zh/12-taos-sql/27-index.md b/docs/zh/12-taos-sql/27-index.md index 2c0907723e..f88c6cf4ff 100644 --- a/docs/zh/12-taos-sql/27-index.md +++ b/docs/zh/12-taos-sql/27-index.md @@ -1,6 +1,7 @@ --- sidebar_label: 索引 title: 使用索引 +description: 索引功能的使用细节 --- TDengine 从 3.0.0.0 版本开始引入了索引功能,支持 SMA 索引和 FULLTEXT 索引。 diff --git a/docs/zh/12-taos-sql/28-recovery.md b/docs/zh/12-taos-sql/28-recovery.md index 72b220b8ff..582c373907 100644 --- a/docs/zh/12-taos-sql/28-recovery.md +++ b/docs/zh/12-taos-sql/28-recovery.md @@ -1,6 +1,7 @@ --- sidebar_label: 异常恢复 title: 异常恢复 +description: 如何终止出现问题的连接、查询和事务以使系统恢复正常 --- 在一个复杂的应用场景中,连接和查询任务等有可能进入一种错误状态或者耗时过长迟迟无法结束,此时需要有能够终止这些连接或任务的方法。 diff --git a/docs/zh/12-taos-sql/index.md b/docs/zh/12-taos-sql/index.md index 821679551c..739d26b224 100644 --- a/docs/zh/12-taos-sql/index.md +++ b/docs/zh/12-taos-sql/index.md @@ -1,11 +1,11 @@ --- -title: TAOS SQL -description: "TAOS SQL 支持的语法规则、主要查询功能、支持的 SQL 查询函数,以及常用技巧等内容" +title: TDengine SQL +description: 'TDengine SQL 支持的语法规则、主要查询功能、支持的 SQL 查询函数,以及常用技巧等内容' --- -本文档说明 TAOS SQL 支持的语法规则、主要查询功能、支持的 SQL 查询函数,以及常用技巧等内容。阅读本文档需要读者具有基本的 SQL 语言的基础。TDengine 3.0 版本相比 2.x 版本做了大量改进和优化,特别是查询引擎进行了彻底的重构,因此 SQL 语法相比 2.x 版本有很多变更。详细的变更内容请见 [3.0 版本语法变更](/taos-sql/changes) 章节 +本文档说明 TDengine SQL 支持的语法规则、主要查询功能、支持的 SQL 查询函数,以及常用技巧等内容。阅读本文档需要读者具有基本的 SQL 语言的基础。TDengine 3.0 版本相比 2.x 版本做了大量改进和优化,特别是查询引擎进行了彻底的重构,因此 SQL 语法相比 2.x 版本有很多变更。详细的变更内容请见 [3.0 版本语法变更](/taos-sql/changes) 章节 -TAOS SQL 是用户对 TDengine 进行数据写入和查询的主要工具。TAOS SQL 提供标准的 SQL 语法,并针对时序数据和业务的特点优化和新增了许多语法和功能。TAOS SQL 语句的最大长度为 1M。TAOS SQL 不支持关键字的缩写,例如 DELETE 不能缩写为 DEL。 +TDengine SQL 是用户对 TDengine 进行数据写入和查询的主要工具。TDengine SQL 提供标准的 SQL 语法,并针对时序数据和业务的特点优化和新增了许多语法和功能。TDengine SQL 语句的最大长度为 1M。TDengine SQL 不支持关键字的缩写,例如 DELETE 不能缩写为 DEL。 本章节 SQL 语法遵循如下约定: diff --git a/docs/zh/14-reference/07-tdinsight/index.mdx b/docs/zh/14-reference/07-tdinsight/index.mdx index 9548922e65..ecd6362143 100644 --- a/docs/zh/14-reference/07-tdinsight/index.mdx +++ b/docs/zh/14-reference/07-tdinsight/index.mdx @@ -1,6 +1,7 @@ --- -title: TDinsight - 基于Grafana的TDengine零依赖监控解决方案 +title: TDinsight sidebar_label: TDinsight +description: 基于Grafana的TDengine零依赖监控解决方案 --- TDinsight 是使用监控数据库和 [Grafana] 对 TDengine 进行监控的解决方案。 diff --git a/docs/zh/14-reference/11-docker/index.md b/docs/zh/14-reference/11-docker/index.md index 743fc2d32f..d712e9aba8 100644 --- a/docs/zh/14-reference/11-docker/index.md +++ b/docs/zh/14-reference/11-docker/index.md @@ -32,7 +32,7 @@ taos> show databases; Query OK, 2 rows in database (0.033802s) ``` -因为运行在容器中的 TDengine 服务端使用容器的 hostname 建立连接,使用 taos shell 或者各种连接器(例如 JDBC-JNI)从容器外访问容器内的 TDengine 比较复杂,所以上述方式是访问容器中 TDengine 服务的最简单的方法,适用于一些简单场景。如果在一些复杂场景下想要从容器化使用 taos shell 或者各种连接器访问容器中的 TDengine 服务,请参考下一节。 +因为运行在容器中的 TDengine 服务端使用容器的 hostname 建立连接,使用 TDengine CLI 或者各种连接器(例如 JDBC-JNI)从容器外访问容器内的 TDengine 比较复杂,所以上述方式是访问容器中 TDengine 服务的最简单的方法,适用于一些简单场景。如果在一些复杂场景下想要从容器化使用 TDengine CLI 或者各种连接器访问容器中的 TDengine 服务,请参考下一节。 ## 在 host 网络上启动 TDengine @@ -75,7 +75,7 @@ docker run -d \ echo 127.0.0.1 tdengine |sudo tee -a /etc/hosts ``` -最后,可以从 taos shell 或者任意连接器以 "tdengine" 为服务端地址访问 TDengine 服务。 +最后,可以从 TDengine CLI 或者任意连接器以 "tdengine" 为服务端地址访问 TDengine 服务。 ```shell taos -h tdengine -P 6030 @@ -354,7 +354,7 @@ test-docker_td-2_1 /tini -- /usr/bin/entrypoi ... Up test-docker_td-3_1 /tini -- /usr/bin/entrypoi ... Up ``` -4. 用 taos shell 查看 dnodes +4. 用 TDengine CLI 查看 dnodes ```shell diff --git a/docs/zh/14-reference/13-schemaless/13-schemaless.md b/docs/zh/14-reference/13-schemaless/13-schemaless.md index ae4280e26a..a33abafaf8 100644 --- a/docs/zh/14-reference/13-schemaless/13-schemaless.md +++ b/docs/zh/14-reference/13-schemaless/13-schemaless.md @@ -3,7 +3,7 @@ title: Schemaless 写入 description: 'Schemaless 写入方式,可以免于预先创建超级表/子表的步骤,随着数据写入接口能够自动创建与数据对应的存储结构' --- -在物联网应用中,常会采集比较多的数据项,用于实现智能控制、业务分析、设备监控等。由于应用逻辑的版本升级,或者设备自身的硬件调整等原因,数据采集项就有可能比较频繁地出现变动。为了在这种情况下方便地完成数据记录工作,TDengine提供调用 Schemaless 写入方式,可以免于预先创建超级表/子表的步骤,随着数据写入接口能够自动创建与数据对应的存储结构。并且在必要时,Schemaless +在物联网应用中,常会采集比较多的数据项,用于实现智能控制、业务分析、设备监控等。由于应用逻辑的版本升级,或者设备自身的硬件调整等原因,数据采集项就有可能比较频繁地出现变动。为了在这种情况下方便地完成数据记录工作,TDengine 提供调用 Schemaless 写入方式,可以免于预先创建超级表/子表的步骤,随着数据写入接口能够自动创建与数据对应的存储结构。并且在必要时,Schemaless 将自动增加必要的数据列,保证用户写入的数据可以被正确存储。 无模式写入方式建立的超级表及其对应的子表与通过 SQL 直接建立的超级表和子表完全没有区别,你也可以通过,SQL 语句直接向其中写入数据。需要注意的是,通过无模式写入方式建立的表,其表名是基于标签值按照固定的映射规则生成,所以无法明确地进行表意,缺乏可读性。 @@ -36,14 +36,14 @@ tag_set 中的所有的数据自动转化为 nchar 数据类型,并不需要 - 对空格、等号(=)、逗号(,)、双引号("),前面需要使用反斜杠(\)进行转义。(都指的是英文半角符号) - 数值类型将通过后缀来区分数据类型: -| **序号** | **后缀** | **映射类型** | **大小(字节)** | -| -------- | -------- | ------------ | -------------- | -| 1 | 无或 f64 | double | 8 | -| 2 | f32 | float | 4 | -| 3 | i8/u8 | TinyInt/UTinyInt | 1 | -| 4 | i16/u16 | SmallInt/USmallInt | 2 | -| 5 | i32/u32 | Int/UInt | 4 | -| 6 | i64/i/u64/u | BigInt/BigInt/UBigInt/UBigInt | 8 | +| **序号** | **后缀** | **映射类型** | **大小(字节)** | +| -------- | ----------- | ----------------------------- | -------------- | +| 1 | 无或 f64 | double | 8 | +| 2 | f32 | float | 4 | +| 3 | i8/u8 | TinyInt/UTinyInt | 1 | +| 4 | i16/u16 | SmallInt/USmallInt | 2 | +| 5 | i32/u32 | Int/UInt | 4 | +| 6 | i64/i/u64/u | BigInt/BigInt/UBigInt/UBigInt | 8 | - t, T, true, True, TRUE, f, F, false, False 将直接作为 BOOL 型来处理。 @@ -69,7 +69,7 @@ st,t1=3,t2=4,t3=t3 c1=3i64,c3="passit",c2=false,c4=4f64 1626006833639000000 需要注意的是,这里的 tag_key1, tag_key2 并不是用户输入的标签的原始顺序,而是使用了标签名称按照字符串升序排列后的结果。所以,tag_key1 并不是在行协议中输入的第一个标签。 排列完成以后计算该字符串的 MD5 散列值 "md5_val"。然后将计算的结果与字符串组合生成表名:“t_md5_val”。其中的 “t_” 是固定的前缀,每个通过该映射关系自动生成的表都具有该前缀。 -为了让用户可以指定生成的表名,可以通过配置smlChildTableName来指定(比如 配置smlChildTableName=tname 插入数据为st,tname=cpu1,t1=4 c1=3 1626006833639000000 则创建的表名为cpu1,注意如果多行数据tname相同,但是后面的tag_set不同,则使用第一次自动建表时指定的tag_set,其他的会忽略)。 +为了让用户可以指定生成的表名,可以通过配置 smlChildTableName 来指定(比如 配置 smlChildTableName=tname 插入数据为 st,tname=cpu1,t1=4 c1=3 1626006833639000000 则创建的表名为 cpu1,注意如果多行数据 tname 相同,但是后面的 tag_set 不同,则使用第一次自动建表时指定的 tag_set,其他的会忽略)。 2. 如果解析行协议获得的超级表不存在,则会创建这个超级表(不建议手动创建超级表,不然插入数据可能异常)。 3. 如果解析行协议获得子表不存在,则 Schemaless 会按照步骤 1 或 2 确定的子表名来创建子表。 @@ -78,11 +78,11 @@ st,t1=3,t2=4,t3=t3 c1=3i64,c3="passit",c2=false,c4=4f64 1626006833639000000 NULL。 6. 对 BINARY 或 NCHAR 列,如果数据行中所提供值的长度超出了列类型的限制,自动增加该列允许存储的字符长度上限(只增不减),以保证数据的完整保存。 7. 整个处理过程中遇到的错误会中断写入过程,并返回错误代码。 -8. 为了提高写入的效率,默认假设同一个超级表中field_set的顺序是一样的(第一条数据包含所有的field,后面的数据按照这个顺序),如果顺序不一样,需要配置参数smlDataFormat为false,否则,数据写入按照相同顺序写入,库中数据会异常。 +8. 为了提高写入的效率,默认假设同一个超级表中 field_set 的顺序是一样的(第一条数据包含所有的 field,后面的数据按照这个顺序),如果顺序不一样,需要配置参数 smlDataFormat 为 false,否则,数据写入按照相同顺序写入,库中数据会异常。 :::tip 无模式所有的处理逻辑,仍会遵循 TDengine 对数据结构的底层限制,例如每行数据的总长度不能超过 -16KB。这方面的具体限制约束请参见 [TAOS SQL 边界限制](/taos-sql/limit) +16KB。这方面的具体限制约束请参见 [TDengine SQL 边界限制](/taos-sql/limit) ::: diff --git a/docs/zh/14-reference/index.md b/docs/zh/14-reference/index.md index e9c0c4fe23..9d0a44af57 100644 --- a/docs/zh/14-reference/index.md +++ b/docs/zh/14-reference/index.md @@ -1,5 +1,6 @@ --- title: 参考手册 +description: TDengine 中的各种组件的详细说明 --- 参考手册是对 TDengine 本身、 TDengine 各语言连接器及自带的工具最详细的介绍。 diff --git a/docs/zh/17-operation/01-pkg-install.md b/docs/zh/17-operation/01-pkg-install.md index 5e4cc93130..6d93c1697b 100644 --- a/docs/zh/17-operation/01-pkg-install.md +++ b/docs/zh/17-operation/01-pkg-install.md @@ -47,43 +47,99 @@ lrwxrwxrwx 1 root root 13 Feb 22 09:34 log -> /var/log/taos/ -内容 TBD +TDengine 卸载命令如下: + +``` +$ sudo apt-get remove tdengine +Reading package lists... Done +Building dependency tree +Reading state information... Done +The following packages will be REMOVED: + tdengine +0 upgraded, 0 newly installed, 1 to remove and 18 not upgraded. +After this operation, 68.3 MB disk space will be freed. +Do you want to continue? [Y/n] y +(Reading database ... 135625 files and directories currently installed.) +Removing tdengine (3.0.0.0) ... +TDengine is removed successfully! + +``` + +taosTools 卸载命令如下: + +``` +$ sudo apt remove taostools +Reading package lists... Done +Building dependency tree +Reading state information... Done +The following packages will be REMOVED: + taostools +0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded. +After this operation, 68.3 MB disk space will be freed. +Do you want to continue? [Y/n] +(Reading database ... 147973 files and directories currently installed.) +Removing taostools (2.1.2) ... +``` -卸载命令如下: +TDengine 卸载命令如下: ``` $ sudo dpkg -r tdengine (Reading database ... 120119 files and directories currently installed.) -Removing tdengine (3.0.0.10002) ... +Removing tdengine (3.0.0.0) ... TDengine is removed successfully! ``` +taosTools 卸载命令如下: + +``` +$ sudo dpkg -r taostools +(Reading database ... 147973 files and directories currently installed.) +Removing taostools (2.1.2) ... +``` + -卸载命令如下: +卸载 TDengine 命令如下: ``` $ sudo rpm -e tdengine TDengine is removed successfully! ``` +卸载 taosTools 命令如下: + +``` +sudo rpm -e taostools +taosToole is removed successfully! +``` + -卸载命令如下: +卸载 TDengine 命令如下: ``` $ rmtaos TDengine is removed successfully! ``` +卸载 taosTools 命令如下: + +``` +$ rmtaostools +Start to uninstall taos tools ... + +taos tools is uninstalled successfully! +``` + 在 C:\TDengine 目录下,通过运行 unins000.exe 卸载程序来卸载 TDengine。 diff --git a/docs/zh/17-operation/02-planning.mdx b/docs/zh/17-operation/02-planning.mdx index 0d63c4eaf3..28e3f54020 100644 --- a/docs/zh/17-operation/02-planning.mdx +++ b/docs/zh/17-operation/02-planning.mdx @@ -1,6 +1,7 @@ --- sidebar_label: 容量规划 title: 容量规划 +description: 如何规划一个 TDengine 集群所需的物理资源 --- 使用 TDengine 来搭建一个物联网大数据平台,计算资源、存储资源需要根据业务场景进行规划。下面分别讨论系统运行所需要的内存、CPU 以及硬盘空间。 diff --git a/docs/zh/17-operation/03-tolerance.md b/docs/zh/17-operation/03-tolerance.md index 1ce485b042..79cf10c39a 100644 --- a/docs/zh/17-operation/03-tolerance.md +++ b/docs/zh/17-operation/03-tolerance.md @@ -1,5 +1,7 @@ --- title: 容错和灾备 +sidebar_label: 容错和灾备 +description: TDengine 的容错和灾备功能 --- ## 容错 diff --git a/docs/zh/17-operation/07-import.md b/docs/zh/17-operation/07-import.md index 7dee05720d..17945be595 100644 --- a/docs/zh/17-operation/07-import.md +++ b/docs/zh/17-operation/07-import.md @@ -1,5 +1,6 @@ --- title: 数据导入 +description: 如何导入外部数据到 TDengine --- TDengine 提供多种方便的数据导入功能,一种按脚本文件导入,一种按数据文件导入,一种是 taosdump 工具导入本身导出的文件。 diff --git a/docs/zh/17-operation/08-export.md b/docs/zh/17-operation/08-export.md index 042ecc7ba2..44247e28bd 100644 --- a/docs/zh/17-operation/08-export.md +++ b/docs/zh/17-operation/08-export.md @@ -1,12 +1,13 @@ --- title: 数据导出 +description: 如何导出 TDengine 中的数据 --- 为方便数据导出,TDengine 提供了两种导出方式,分别是按表导出和用 taosdump 导出。 ## 按表导出 CSV 文件 -如果用户需要导出一个表或一个 STable 中的数据,可在 taos shell 中运行: +如果用户需要导出一个表或一个 STable 中的数据,可在 TDengine CLI 中运行: ```sql select * from >> data.csv; diff --git a/docs/zh/17-operation/10-monitor.md b/docs/zh/17-operation/10-monitor.md index 9f0f06fde2..e936f35dca 100644 --- a/docs/zh/17-operation/10-monitor.md +++ b/docs/zh/17-operation/10-monitor.md @@ -1,5 +1,6 @@ --- title: 系统监控 +description: 监控 TDengine 的运行状态 --- TDengine 通过 [taosKeeper](/reference/taosKeeper/) 将服务器的 CPU、内存、硬盘空间、带宽、请求数、磁盘读写速度等信息定时写入指定数据库。TDengine 还将重要的系统操作(比如登录、创建、删除数据库等)日志以及各种错误报警信息进行记录。系统管理员可以从 CLI 直接查看这个数据库,也可以在 WEB 通过图形化界面查看这些监测信息。 diff --git a/docs/zh/17-operation/17-diagnose.md b/docs/zh/17-operation/17-diagnose.md index e6e9be7153..ec529096a7 100644 --- a/docs/zh/17-operation/17-diagnose.md +++ b/docs/zh/17-operation/17-diagnose.md @@ -1,5 +1,6 @@ --- title: 诊断及其他 +description: 一些常见问题的诊断技巧 --- ## 网络连接诊断 diff --git a/docs/zh/20-third-party/01-grafana.mdx b/docs/zh/20-third-party/01-grafana.mdx index becb1a70a9..83f3f8bb25 100644 --- a/docs/zh/20-third-party/01-grafana.mdx +++ b/docs/zh/20-third-party/01-grafana.mdx @@ -1,6 +1,7 @@ --- sidebar_label: Grafana title: Grafana +description: 使用 Grafana 与 TDengine 的详细说明 --- import Tabs from "@theme/Tabs"; diff --git a/docs/zh/20-third-party/02-prometheus.md b/docs/zh/20-third-party/02-prometheus.md index 0fe534b8df..eb6c3bf1d0 100644 --- a/docs/zh/20-third-party/02-prometheus.md +++ b/docs/zh/20-third-party/02-prometheus.md @@ -1,6 +1,7 @@ --- sidebar_label: Prometheus title: Prometheus +description: 使用 Prometheus 访问 TDengine --- import Prometheus from "../14-reference/_prometheus.mdx" diff --git a/docs/zh/20-third-party/03-telegraf.md b/docs/zh/20-third-party/03-telegraf.md index 88a69211c0..84883e665a 100644 --- a/docs/zh/20-third-party/03-telegraf.md +++ b/docs/zh/20-third-party/03-telegraf.md @@ -1,6 +1,7 @@ --- sidebar_label: Telegraf title: Telegraf 写入 +description: 使用 Telegraf 向 TDengine 写入数据 --- import Telegraf from "../14-reference/_telegraf.mdx" diff --git a/docs/zh/20-third-party/05-collectd.md b/docs/zh/20-third-party/05-collectd.md index 04892fd42e..cc2235f260 100644 --- a/docs/zh/20-third-party/05-collectd.md +++ b/docs/zh/20-third-party/05-collectd.md @@ -1,6 +1,7 @@ --- sidebar_label: collectd title: collectd 写入 +description: 使用 collected 向 TDengine 写入数据 --- import CollectD from "../14-reference/_collectd.mdx" diff --git a/docs/zh/20-third-party/06-statsd.md b/docs/zh/20-third-party/06-statsd.md index 260d011835..122c9fd94c 100644 --- a/docs/zh/20-third-party/06-statsd.md +++ b/docs/zh/20-third-party/06-statsd.md @@ -1,6 +1,7 @@ --- sidebar_label: StatsD title: StatsD 直接写入 +description: 使用 StatsD 向 TDengine 写入 --- import StatsD from "../14-reference/_statsd.mdx" diff --git a/docs/zh/20-third-party/07-icinga2.md b/docs/zh/20-third-party/07-icinga2.md index ed1f1404a7..06ead57655 100644 --- a/docs/zh/20-third-party/07-icinga2.md +++ b/docs/zh/20-third-party/07-icinga2.md @@ -1,6 +1,7 @@ --- sidebar_label: icinga2 title: icinga2 写入 +description: 使用 icinga2 写入 TDengine --- import Icinga2 from "../14-reference/_icinga2.mdx" diff --git a/docs/zh/20-third-party/08-tcollector.md b/docs/zh/20-third-party/08-tcollector.md index a1245e8c27..78d0b4a5df 100644 --- a/docs/zh/20-third-party/08-tcollector.md +++ b/docs/zh/20-third-party/08-tcollector.md @@ -1,6 +1,7 @@ --- sidebar_label: TCollector title: TCollector 写入 +description: 使用 TCollector 写入 TDengine --- import TCollector from "../14-reference/_tcollector.mdx" diff --git a/docs/zh/20-third-party/09-emq-broker.md b/docs/zh/20-third-party/09-emq-broker.md index f252e520a7..782a139e22 100644 --- a/docs/zh/20-third-party/09-emq-broker.md +++ b/docs/zh/20-third-party/09-emq-broker.md @@ -1,6 +1,7 @@ --- sidebar_label: EMQX Broker title: EMQX Broker 写入 +description: 使用 EMQX Broker 写入 TDengine --- MQTT 是流行的物联网数据传输协议,[EMQX](https://github.com/emqx/emqx)是一开源的 MQTT Broker 软件,无需任何代码,只需要在 EMQX Dashboard 里使用“规则”做简单配置,即可将 MQTT 的数据直接写入 TDengine。EMQX 支持通过 发送到 Web 服务的方式保存数据到 TDengine,也在企业版上提供原生的 TDengine 驱动实现直接保存。 diff --git a/docs/zh/20-third-party/10-hive-mq-broker.md b/docs/zh/20-third-party/10-hive-mq-broker.md index f75ed793d6..a388ff6daf 100644 --- a/docs/zh/20-third-party/10-hive-mq-broker.md +++ b/docs/zh/20-third-party/10-hive-mq-broker.md @@ -1,6 +1,7 @@ --- sidebar_label: HiveMQ Broker title: HiveMQ Broker 写入 +description: 使用 HivMQ Broker 写入 TDengine --- [HiveMQ](https://www.hivemq.com/) 是一个提供免费个人版和企业版的 MQTT 代理,主要用于企业和新兴的机器到机器 M2M 通讯和内部传输,满足可伸缩性、易管理和安全特性。HiveMQ 提供了开源的插件开发包。可以通过 HiveMQ extension - TDengine 保存数据到 TDengine。详细使用方法请参考 [HiveMQ extension - TDengine 说明文档](https://github.com/huskar-t/hivemq-tdengine-extension/blob/b62a26ecc164a310104df57691691b237e091c89/README.md)。 diff --git a/docs/zh/20-third-party/11-kafka.md b/docs/zh/20-third-party/11-kafka.md index 09bda4664f..1172f4fbc5 100644 --- a/docs/zh/20-third-party/11-kafka.md +++ b/docs/zh/20-third-party/11-kafka.md @@ -1,6 +1,7 @@ --- sidebar_label: Kafka -title: TDengine Kafka Connector 使用教程 +title: TDengine Kafka Connector +description: 使用 TDengine Kafka Connector 的详细指南 --- TDengine Kafka Connector 包含两个插件: TDengine Source Connector 和 TDengine Sink Connector。用户只需提供简单的配置文件,就可以将 Kafka 中指定 topic 的数据(批量或实时)同步到 TDengine, 或将 TDengine 中指定数据库的数据(批量或实时)同步到 Kafka。 diff --git a/docs/zh/21-tdinternal/01-arch.md b/docs/zh/21-tdinternal/01-arch.md index a910c584d6..704524fd21 100644 --- a/docs/zh/21-tdinternal/01-arch.md +++ b/docs/zh/21-tdinternal/01-arch.md @@ -1,6 +1,7 @@ --- sidebar_label: 整体架构 title: 整体架构 +description: TDengine 架构设计,包括:集群、存储、缓存与持久化、数据备份、多级存储等 --- ## 集群与基本逻辑单元 @@ -287,7 +288,7 @@ TDengine 对每个数据采集点单独建表,但在实际应用中经常需 7. vnode 返回本节点的查询计算结果; 8. qnode 完成多节点数据聚合后将最终查询结果返回给客户端; -由于 TDengine 在 vnode 内将标签数据与时序数据分离存储,通过在内存里过滤标签数据,先找到需要参与聚合操作的表的集合,将需要扫描的数据集大幅减少,大幅提升聚合计算速度。同时,由于数据分布在多个 vnode/dnode,聚合计算操作在多个 vnode 里并发进行,又进一步提升了聚合的速度。 对普通表的聚合函数以及绝大部分操作都适用于超级表,语法完全一样,细节请看 TAOS SQL。 +由于 TDengine 在 vnode 内将标签数据与时序数据分离存储,通过在内存里过滤标签数据,先找到需要参与聚合操作的表的集合,将需要扫描的数据集大幅减少,大幅提升聚合计算速度。同时,由于数据分布在多个 vnode/dnode,聚合计算操作在多个 vnode 里并发进行,又进一步提升了聚合的速度。 对普通表的聚合函数以及绝大部分操作都适用于超级表,语法完全一样,细节请看 TDengine SQL。 ### 预计算 diff --git a/docs/zh/21-tdinternal/03-high-availability.md b/docs/zh/21-tdinternal/03-high-availability.md index ba056b6f16..4cdf04f6d1 100644 --- a/docs/zh/21-tdinternal/03-high-availability.md +++ b/docs/zh/21-tdinternal/03-high-availability.md @@ -1,5 +1,6 @@ --- title: 高可用 +description: TDengine 的高可用设计 --- ## Vnode 的高可用性 diff --git a/docs/zh/21-tdinternal/05-load-balance.md b/docs/zh/21-tdinternal/05-load-balance.md index 2376dd3e61..07af2328d5 100644 --- a/docs/zh/21-tdinternal/05-load-balance.md +++ b/docs/zh/21-tdinternal/05-load-balance.md @@ -1,5 +1,6 @@ --- title: 负载均衡 +description: TDengine 的负载均衡设计 --- TDengine 中的负载均衡主要指对时序数据的处理的负载均衡。TDengine 采用 Hash 一致性算法将一个数据库中的所有表和子表的数据均衡分散在属于该数据库的所有 vgroup 中,每张表或子表只能由一个 vgroup 处理,一个 vgroup 可能负责处理多个表或子表。 @@ -7,7 +8,7 @@ TDengine 中的负载均衡主要指对时序数据的处理的负载均衡。TD 创建数据库时可以指定其中的 vgroup 的数量: ```sql -create database db0 vgroups 100; +create database db0 vgroups 20; ``` 如何指定合适的 vgroup 的数量,这取决于系统资源。假定系统中只计划建立一个数据库,则 vgroup 数量由集群中所有 dnode 所能使用的资源决定。原则上可用的 CPU 和 Memory 越多,可建立的 vgroup 也越多。但也要考虑到磁盘性能,过多的 vgroup 在磁盘性能达到上限后反而会拖累整个系统的性能。假如系统中会建立多个数据库,则多个数据库的 vgroup 之和取决于系统中可用资源的数量。要综合考虑多个数据库之间表的数量、写入频率、数据量等多个因素在多个数据库之间分配 vgroup。实际中建议首先根据系统资源配置选择一个初始的 vgroup 数量,比如 CPU 总核数的 2 倍,以此为起点通过测试找到最佳的 vgroup 数量配置,此为系统中的 vgroup 总数。如果有多个数据库的话,再根据各个数据库的表数和数据量对 vgroup 进行分配。 diff --git a/docs/zh/21-tdinternal/index.md b/docs/zh/21-tdinternal/index.md index 63a746623e..21f106edc9 100644 --- a/docs/zh/21-tdinternal/index.md +++ b/docs/zh/21-tdinternal/index.md @@ -1,5 +1,6 @@ --- title: 技术内幕 +description: TDengine 的内部设计 --- ```mdx-code-block diff --git a/docs/zh/25-application/01-telegraf.md b/docs/zh/25-application/01-telegraf.md index a949fa9721..4e9597f964 100644 --- a/docs/zh/25-application/01-telegraf.md +++ b/docs/zh/25-application/01-telegraf.md @@ -1,6 +1,7 @@ --- sidebar_label: TDengine + Telegraf + Grafana -title: 使用 TDengine + Telegraf + Grafana 快速搭建 IT 运维展示系统 +title: TDengine + Telegraf + Grafana +description: 使用 TDengine + Telegraf + Grafana 快速搭建 IT 运维展示系统 --- ## 背景介绍 diff --git a/docs/zh/25-application/02-collectd.md b/docs/zh/25-application/02-collectd.md index 6bdebd5030..c6230f48ab 100644 --- a/docs/zh/25-application/02-collectd.md +++ b/docs/zh/25-application/02-collectd.md @@ -1,6 +1,7 @@ --- sidebar_label: TDengine + collectd/StatsD + Grafana -title: 使用 TDengine + collectd/StatsD + Grafana 快速搭建 IT 运维监控系统 +title: TDengine + collectd/StatsD + Grafana +description: 使用 TDengine + collectd/StatsD + Grafana 快速搭建 IT 运维监控系统 --- ## 背景介绍 diff --git a/docs/zh/25-application/index.md b/docs/zh/25-application/index.md index 1305cf230f..76aa179927 100644 --- a/docs/zh/25-application/index.md +++ b/docs/zh/25-application/index.md @@ -1,5 +1,6 @@ --- title: 应用实践 +description: TDengine 配合其它开源组件的一些应用示例 --- ```mdx-code-block diff --git a/docs/zh/27-train-faq/01-faq.md b/docs/zh/27-train-faq/01-faq.md index 04ee011b93..0a46db4a28 100644 --- a/docs/zh/27-train-faq/01-faq.md +++ b/docs/zh/27-train-faq/01-faq.md @@ -1,5 +1,6 @@ --- title: 常见问题及反馈 +description: 一些常见问题的解决方法汇总 --- ## 问题反馈 @@ -115,7 +116,7 @@ charset UTF-8 ### 9. 表名显示不全 -由于 taos shell 在终端中显示宽度有限,有可能比较长的表名显示不全,如果按照显示的不全的表名进行相关操作会发生 Table does not exist 错误。解决方法可以是通过修改 taos.cfg 文件中的设置项 maxBinaryDisplayWidth, 或者直接输入命令 set max_binary_display_width 100。或者在命令结尾使用 \G 参数来调整结果的显示方式。 +由于 TDengine CLI 在终端中显示宽度有限,有可能比较长的表名显示不全,如果按照显示的不全的表名进行相关操作会发生 Table does not exist 错误。解决方法可以是通过修改 taos.cfg 文件中的设置项 maxBinaryDisplayWidth, 或者直接输入命令 set max_binary_display_width 100。或者在命令结尾使用 \G 参数来调整结果的显示方式。 ### 10. 如何进行数据迁移? diff --git a/docs/zh/27-train-faq/index.md b/docs/zh/27-train-faq/index.md index b42bff0288..e7159d98c8 100644 --- a/docs/zh/27-train-faq/index.md +++ b/docs/zh/27-train-faq/index.md @@ -1,5 +1,6 @@ --- title: FAQ 及其他 +description: 用户经常遇到的问题 --- ```mdx-code-block diff --git a/docs/zh/28-releases/01-tdengine.md b/docs/zh/28-releases/01-tdengine.md index 1e97572ca4..e3e1463131 100644 --- a/docs/zh/28-releases/01-tdengine.md +++ b/docs/zh/28-releases/01-tdengine.md @@ -1,6 +1,7 @@ --- sidebar_label: TDengine 发布历史 title: TDengine 发布历史 +description: TDengine 发布历史、Release Notes 及下载链接 --- import Release from "/components/ReleaseV3"; diff --git a/docs/zh/28-releases/02-tools.md b/docs/zh/28-releases/02-tools.md index 7513333040..61129d74e5 100644 --- a/docs/zh/28-releases/02-tools.md +++ b/docs/zh/28-releases/02-tools.md @@ -1,6 +1,7 @@ --- sidebar_label: taosTools 发布历史 title: taosTools 发布历史 +description: taosTools 的发布历史、Release Notes 和下载链接 --- import Release from "/components/ReleaseV3"; diff --git a/examples/JDBC/JDBCDemo/README-jdbc-windows.md b/examples/JDBC/JDBCDemo/README-jdbc-windows.md index 17c5c8df00..5a781f40f7 100644 --- a/examples/JDBC/JDBCDemo/README-jdbc-windows.md +++ b/examples/JDBC/JDBCDemo/README-jdbc-windows.md @@ -129,7 +129,7 @@ https://www.taosdata.com/cn/all-downloads/ 192.168.236.136 td01 ``` -配置完成后,在命令行内使用taos shell连接server端 +配置完成后,在命令行内使用TDengine CLI连接server端 ```shell C:\TDengine>taos -h td01 diff --git a/examples/c/tmq_taosx.c b/examples/c/tmq_taosx.c index d0def44269..491eda1ddb 100644 --- a/examples/c/tmq_taosx.c +++ b/examples/c/tmq_taosx.c @@ -163,6 +163,13 @@ int32_t init_env() { } taos_free_result(pRes); + pRes = taos_query(pConn, "create table if not exists ct4 using st1(t3) tags('ct4')"); + if (taos_errno(pRes) != 0) { + printf("failed to create child table ct4, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + pRes = taos_query(pConn, "insert into ct3 values(1626006833600, 5, 6, 'c') ct1 values(1626006833601, 2, 3, 'sds') (1626006833602, 4, 5, 'ddd') ct0 values(1626006833602, 4, 3, 'hwj') ct1 values(now+5s, 23, 32, 's21ds')"); if (taos_errno(pRes) != 0) { printf("failed to insert into ct3, reason:%s\n", taos_errstr(pRes)); @@ -379,6 +386,8 @@ tmq_t* build_consumer() { tmq_conf_set(conf, "td.connect.pass", "taosdata"); tmq_conf_set(conf, "msg.with.table.name", "true"); tmq_conf_set(conf, "enable.auto.commit", "true"); + tmq_conf_set(conf, "experimental.snapshot.enable", "true"); + /*tmq_conf_set(conf, "experimental.snapshot.enable", "true");*/ @@ -406,7 +415,7 @@ void basic_consume_loop(tmq_t* tmq, tmq_list_t* topics) { } int32_t cnt = 0; while (running) { - TAOS_RES* tmqmessage = tmq_consumer_poll(tmq, -1); + TAOS_RES* tmqmessage = tmq_consumer_poll(tmq, 1000); if (tmqmessage) { cnt++; msg_process(tmqmessage); diff --git a/examples/nodejs/README-win.md b/examples/nodejs/README-win.md index 75fec69413..e496be2f87 100644 --- a/examples/nodejs/README-win.md +++ b/examples/nodejs/README-win.md @@ -35,7 +35,7 @@ Python 2.7.18 下载地址:https://www.taosdata.com/cn/all-downloads/,选择一个合适的windows-client下载(client应该尽量与server端的版本保持一致) -使用client的taos shell连接server +使用client的TDengine CLI连接server ```shell >taos -h node5 diff --git a/include/common/systable.h b/include/common/systable.h index ed2e6a46c3..01c9807627 100644 --- a/include/common/systable.h +++ b/include/common/systable.h @@ -22,27 +22,27 @@ extern "C" { #ifndef TDENGINE_SYSTABLE_H #define TDENGINE_SYSTABLE_H -#define TSDB_INFORMATION_SCHEMA_DB "information_schema" -#define TSDB_INS_TABLE_DNODES "ins_dnodes" -#define TSDB_INS_TABLE_MNODES "ins_mnodes" -#define TSDB_INS_TABLE_MODULES "ins_modules" -#define TSDB_INS_TABLE_QNODES "ins_qnodes" -#define TSDB_INS_TABLE_BNODES "ins_bnodes" -#define TSDB_INS_TABLE_SNODES "ins_snodes" -#define TSDB_INS_TABLE_CLUSTER "ins_cluster" -#define TSDB_INS_TABLE_DATABASES "ins_databases" -#define TSDB_INS_TABLE_FUNCTIONS "ins_functions" -#define TSDB_INS_TABLE_INDEXES "ins_indexes" -#define TSDB_INS_TABLE_STABLES "ins_stables" -#define TSDB_INS_TABLE_TABLES "ins_tables" -#define TSDB_INS_TABLE_TAGS "ins_tags" -#define TSDB_INS_TABLE_TABLE_DISTRIBUTED "ins_table_distributed" -#define TSDB_INS_TABLE_USERS "ins_users" -#define TSDB_INS_TABLE_LICENCES "ins_grants" -#define TSDB_INS_TABLE_VGROUPS "ins_vgroups" -#define TSDB_INS_TABLE_VNODES "ins_vnodes" -#define TSDB_INS_TABLE_CONFIGS "ins_configs" -#define TSDB_INS_TABLE_DNODE_VARIABLES "ins_dnode_variables" +#define TSDB_INFORMATION_SCHEMA_DB "information_schema" +#define TSDB_INS_TABLE_DNODES "ins_dnodes" +#define TSDB_INS_TABLE_MNODES "ins_mnodes" +#define TSDB_INS_TABLE_MODULES "ins_modules" +#define TSDB_INS_TABLE_QNODES "ins_qnodes" +#define TSDB_INS_TABLE_BNODES "ins_bnodes" +#define TSDB_INS_TABLE_SNODES "ins_snodes" +#define TSDB_INS_TABLE_CLUSTER "ins_cluster" +#define TSDB_INS_TABLE_DATABASES "ins_databases" +#define TSDB_INS_TABLE_FUNCTIONS "ins_functions" +#define TSDB_INS_TABLE_INDEXES "ins_indexes" +#define TSDB_INS_TABLE_STABLES "ins_stables" +#define TSDB_INS_TABLE_TABLES "ins_tables" +#define TSDB_INS_TABLE_TAGS "ins_tags" +#define TSDB_INS_TABLE_TABLE_DISTRIBUTED "ins_table_distributed" +#define TSDB_INS_TABLE_USERS "ins_users" +#define TSDB_INS_TABLE_LICENCES "ins_grants" +#define TSDB_INS_TABLE_VGROUPS "ins_vgroups" +#define TSDB_INS_TABLE_VNODES "ins_vnodes" +#define TSDB_INS_TABLE_CONFIGS "ins_configs" +#define TSDB_INS_TABLE_DNODE_VARIABLES "ins_dnode_variables" #define TSDB_PERFORMANCE_SCHEMA_DB "performance_schema" #define TSDB_PERFS_TABLE_SMAS "perf_smas" @@ -60,16 +60,20 @@ typedef struct SSysDbTableSchema { const char* name; const int32_t type; const int32_t bytes; + const bool sysInfo; } SSysDbTableSchema; typedef struct SSysTableMeta { const char* name; const SSysDbTableSchema* schema; const int32_t colNum; + const bool sysInfo; } SSysTableMeta; void getInfosDbMeta(const SSysTableMeta** pInfosTableMeta, size_t* size); void getPerfDbMeta(const SSysTableMeta** pPerfsTableMeta, size_t* size); +void getVisibleInfosTablesNum(bool sysInfo, size_t* size); +bool invisibleColumn(bool sysInfo, int8_t tableType, int8_t flags); #ifdef __cplusplus } diff --git a/include/common/tcommon.h b/include/common/tcommon.h index a071516fbf..03672f96f3 100644 --- a/include/common/tcommon.h +++ b/include/common/tcommon.h @@ -184,6 +184,7 @@ typedef struct SQueryTableDataCond { STimeWindow twindows; int64_t startVersion; int64_t endVersion; + int64_t schemaVersion; } SQueryTableDataCond; int32_t tEncodeDataBlock(void** buf, const SSDataBlock* pBlock); diff --git a/include/common/tdataformat.h b/include/common/tdataformat.h index af7c88acde..df16f4f0ab 100644 --- a/include/common/tdataformat.h +++ b/include/common/tdataformat.h @@ -96,6 +96,7 @@ char *tTagValToData(const STagVal *pTagVal, bool isJson); int32_t tEncodeTag(SEncoder *pEncoder, const STag *pTag); int32_t tDecodeTag(SDecoder *pDecoder, STag **ppTag); int32_t tTagToValArray(const STag *pTag, SArray **ppArray); +void tTagSetCid(const STag *pTag, int16_t iTag, int16_t cid); void debugPrintSTag(STag *pTag, const char *tag, int32_t ln); // TODO: remove int32_t parseJsontoTagData(const char *json, SArray *pTagVals, STag **ppTag, void *pMsgBuf); diff --git a/include/common/tmsg.h b/include/common/tmsg.h index 8f199c72f7..681094471a 100644 --- a/include/common/tmsg.h +++ b/include/common/tmsg.h @@ -268,6 +268,35 @@ STSRow* tGetSubmitBlkNext(SSubmitBlkIter* pIter); // for debug int32_t tPrintFixedSchemaSubmitReq(SSubmitReq* pReq, STSchema* pSchema); +struct SSchema { + int8_t type; + int8_t flags; + col_id_t colId; + int32_t bytes; + char name[TSDB_COL_NAME_LEN]; +}; + + +typedef struct { + char tbName[TSDB_TABLE_NAME_LEN]; + char stbName[TSDB_TABLE_NAME_LEN]; + char dbFName[TSDB_DB_FNAME_LEN]; + int64_t dbId; + int32_t numOfTags; + int32_t numOfColumns; + int8_t precision; + int8_t tableType; + int32_t sversion; + int32_t tversion; + uint64_t suid; + uint64_t tuid; + int32_t vgId; + int8_t sysInfo; + SSchema* pSchemas; +} STableMetaRsp; + + + typedef struct { int32_t code; int8_t hashMeta; @@ -276,6 +305,7 @@ typedef struct { int32_t numOfRows; int32_t affectedRows; int64_t sver; + STableMetaRsp* pMeta; } SSubmitBlkRsp; typedef struct { @@ -290,19 +320,14 @@ typedef struct { int32_t tEncodeSSubmitRsp(SEncoder* pEncoder, const SSubmitRsp* pRsp); int32_t tDecodeSSubmitRsp(SDecoder* pDecoder, SSubmitRsp* pRsp); +void tFreeSSubmitBlkRsp(void* param); void tFreeSSubmitRsp(SSubmitRsp* pRsp); -#define COL_SMA_ON ((int8_t)0x1) -#define COL_IDX_ON ((int8_t)0x2) -#define COL_SET_NULL ((int8_t)0x10) -#define COL_SET_VAL ((int8_t)0x20) -struct SSchema { - int8_t type; - int8_t flags; - col_id_t colId; - int32_t bytes; - char name[TSDB_COL_NAME_LEN]; -}; +#define COL_SMA_ON ((int8_t)0x1) +#define COL_IDX_ON ((int8_t)0x2) +#define COL_SET_NULL ((int8_t)0x10) +#define COL_SET_VAL ((int8_t)0x20) +#define COL_IS_SYSINFO ((int8_t)0x40) #define COL_IS_SET(FLG) (((FLG) & (COL_SET_VAL | COL_SET_NULL)) != 0) #define COL_CLR_SET(FLG) ((FLG) &= (~(COL_SET_VAL | COL_SET_NULL))) @@ -472,6 +497,14 @@ int32_t tSerializeSMCreateStbReq(void* buf, int32_t bufLen, SMCreateStbReq* pReq int32_t tDeserializeSMCreateStbReq(void* buf, int32_t bufLen, SMCreateStbReq* pReq); void tFreeSMCreateStbReq(SMCreateStbReq* pReq); +typedef struct { + STableMetaRsp* pMeta; +} SMCreateStbRsp; + +int32_t tEncodeSMCreateStbRsp(SEncoder* pEncoder, const SMCreateStbRsp* pRsp); +int32_t tDecodeSMCreateStbRsp(SDecoder* pDecoder, SMCreateStbRsp* pRsp); +void tFreeSMCreateStbRsp(SMCreateStbRsp* pRsp); + typedef struct { char name[TSDB_TABLE_FNAME_LEN]; int8_t igNotExists; @@ -530,6 +563,7 @@ typedef struct { uint32_t connId; int32_t dnodeNum; int8_t superUser; + int8_t sysInfo; int8_t connType; SEpSet epSet; int32_t svrTimestamp; @@ -1239,23 +1273,6 @@ typedef struct { SVgroupInfo vgroups[]; } SVgroupsInfo; -typedef struct { - char tbName[TSDB_TABLE_NAME_LEN]; - char stbName[TSDB_TABLE_NAME_LEN]; - char dbFName[TSDB_DB_FNAME_LEN]; - int64_t dbId; - int32_t numOfTags; - int32_t numOfColumns; - int8_t precision; - int8_t tableType; - int32_t sversion; - int32_t tversion; - uint64_t suid; - uint64_t tuid; - int32_t vgId; - SSchema* pSchemas; -} STableMetaRsp; - typedef struct { STableMetaRsp* pMeta; } SMAlterStbRsp; @@ -1266,7 +1283,7 @@ void tFreeSMAlterStbRsp(SMAlterStbRsp* pRsp); int32_t tSerializeSTableMetaRsp(void* buf, int32_t bufLen, STableMetaRsp* pRsp); int32_t tDeserializeSTableMetaRsp(void* buf, int32_t bufLen, STableMetaRsp* pRsp); -void tFreeSTableMetaRsp(STableMetaRsp* pRsp); +void tFreeSTableMetaRsp(void* pRsp); void tFreeSTableIndexRsp(void* info); typedef struct { @@ -2028,11 +2045,13 @@ int tEncodeSVCreateTbBatchReq(SEncoder* pCoder, const SVCreateTbBatchReq* pReq); int tDecodeSVCreateTbBatchReq(SDecoder* pCoder, SVCreateTbBatchReq* pReq); typedef struct { - int32_t code; + int32_t code; + STableMetaRsp* pMeta; } SVCreateTbRsp, SVUpdateTbRsp; int tEncodeSVCreateTbRsp(SEncoder* pCoder, const SVCreateTbRsp* pRsp); int tDecodeSVCreateTbRsp(SDecoder* pCoder, SVCreateTbRsp* pRsp); +void tFreeSVCreateTbRsp(void* param); int32_t tSerializeSVCreateTbReq(void** buf, SVCreateTbReq* pReq); void* tDeserializeSVCreateTbReq(void* buf, SVCreateTbReq* pReq); @@ -2598,7 +2617,7 @@ enum { typedef struct { int8_t type; union { - // snapshot data + // snapshot struct { int64_t uid; int64_t ts; @@ -2658,15 +2677,6 @@ typedef struct { int32_t tSerializeSMDropSmaReq(void* buf, int32_t bufLen, SMDropSmaReq* pReq); int32_t tDeserializeSMDropSmaReq(void* buf, int32_t bufLen, SMDropSmaReq* pReq); -typedef struct { - int32_t vgId; - SEpSet epSet; -} SVgEpSet; - -typedef struct { - int32_t padding; -} SRSmaExecMsg; - typedef struct { int8_t version; // for compatibility(default 0) int8_t intervalUnit; // MACRO: TIME_UNIT_XXX @@ -2926,33 +2936,14 @@ static FORCE_INLINE void tDeleteSMqSubTopicEp(SMqSubTopicEp* pSubTopicEp) { typedef struct { SMqRspHead head; - int64_t reqOffset; - int64_t rspOffset; - STqOffsetVal reqOffsetNew; - STqOffsetVal rspOffsetNew; + STqOffsetVal rspOffset; int16_t resMsgType; int32_t metaRspLen; void* metaRsp; } SMqMetaRsp; -static FORCE_INLINE int32_t tEncodeSMqMetaRsp(void** buf, const SMqMetaRsp* pRsp) { - int32_t tlen = 0; - tlen += taosEncodeFixedI64(buf, pRsp->reqOffset); - tlen += taosEncodeFixedI64(buf, pRsp->rspOffset); - tlen += taosEncodeFixedI16(buf, pRsp->resMsgType); - tlen += taosEncodeFixedI32(buf, pRsp->metaRspLen); - tlen += taosEncodeBinary(buf, pRsp->metaRsp, pRsp->metaRspLen); - return tlen; -} - -static FORCE_INLINE void* tDecodeSMqMetaRsp(const void* buf, SMqMetaRsp* pRsp) { - buf = taosDecodeFixedI64(buf, &pRsp->reqOffset); - buf = taosDecodeFixedI64(buf, &pRsp->rspOffset); - buf = taosDecodeFixedI16(buf, &pRsp->resMsgType); - buf = taosDecodeFixedI32(buf, &pRsp->metaRspLen); - buf = taosDecodeBinary(buf, &pRsp->metaRsp, pRsp->metaRspLen); - return (void*)buf; -} +int32_t tEncodeSMqMetaRsp(SEncoder* pEncoder, const SMqMetaRsp* pRsp); +int32_t tDecodeSMqMetaRsp(SDecoder* pDecoder, SMqMetaRsp* pRsp); typedef struct { SMqRspHead head; diff --git a/include/libs/command/command.h b/include/libs/command/command.h index 8a4ecad37d..b3339a417b 100644 --- a/include/libs/command/command.h +++ b/include/libs/command/command.h @@ -17,12 +17,12 @@ #define TDENGINE_COMMAND_H #include "cmdnodes.h" -#include "tmsg.h" #include "plannodes.h" +#include "tmsg.h" typedef struct SExplainCtx SExplainCtx; -int32_t qExecCommand(SNode* pStmt, SRetrieveTableRsp** pRsp); +int32_t qExecCommand(bool sysInfoUser, SNode *pStmt, SRetrieveTableRsp **pRsp); int32_t qExecStaticExplain(SQueryPlan *pDag, SRetrieveTableRsp **pRsp); int32_t qExecExplainBegin(SQueryPlan *pDag, SExplainCtx **pCtx, int64_t startTs); diff --git a/include/libs/executor/executor.h b/include/libs/executor/executor.h index 1ce88905c2..25a6221fcb 100644 --- a/include/libs/executor/executor.h +++ b/include/libs/executor/executor.h @@ -41,6 +41,9 @@ typedef struct { bool initTableReader; bool initTqReader; int32_t numOfVgroups; + + void* sContext; // SSnapContext* + void* pStateBackend; } SReadHandle; @@ -181,11 +184,17 @@ int32_t qGetStreamScanStatus(qTaskInfo_t tinfo, uint64_t* uid, int64_t* ts); int32_t qStreamPrepareTsdbScan(qTaskInfo_t tinfo, uint64_t uid, int64_t ts); -int32_t qStreamPrepareScan(qTaskInfo_t tinfo, const STqOffsetVal* pOffset); +int32_t qStreamPrepareScan(qTaskInfo_t tinfo, STqOffsetVal* pOffset, int8_t subType); int32_t qStreamExtractOffset(qTaskInfo_t tinfo, STqOffsetVal* pOffset); -void* qStreamExtractMetaMsg(qTaskInfo_t tinfo); +SMqMetaRsp* qStreamExtractMetaMsg(qTaskInfo_t tinfo); + +int64_t qStreamExtractPrepareUid(qTaskInfo_t tinfo); + +const SSchemaWrapper* qExtractSchemaFromTask(qTaskInfo_t tinfo); + +const char* qExtractTbnameFromTask(qTaskInfo_t tinfo); void* qExtractReaderFromStreamScanner(void* scanner); diff --git a/include/libs/function/taosudf.h b/include/libs/function/taosudf.h index 5e84b87a81..2b2063e3f6 100644 --- a/include/libs/function/taosudf.h +++ b/include/libs/function/taosudf.h @@ -256,8 +256,9 @@ static FORCE_INLINE int32_t udfColDataSet(SUdfColumn* pColumn, uint32_t currentR typedef int32_t (*TUdfScalarProcFunc)(SUdfDataBlock* block, SUdfColumn *resultCol); typedef int32_t (*TUdfAggStartFunc)(SUdfInterBuf *buf); -typedef int32_t (*TUdfAggProcessFunc)(SUdfDataBlock* block, SUdfInterBuf *interBuf, SUdfInterBuf *newInterBuf); -typedef int32_t (*TUdfAggFinishFunc)(SUdfInterBuf* buf, SUdfInterBuf *resultData); +typedef int32_t (*TUdfAggProcessFunc)(SUdfDataBlock *block, SUdfInterBuf *interBuf, SUdfInterBuf *newInterBuf); +typedef int32_t (*TUdfAggMergeFunc)(SUdfInterBuf *inputBuf1, SUdfInterBuf *inputBuf2, SUdfInterBuf *outputBuf); +typedef int32_t (*TUdfAggFinishFunc)(SUdfInterBuf *buf, SUdfInterBuf *resultData); #ifdef __cplusplus } diff --git a/include/libs/nodes/plannodes.h b/include/libs/nodes/plannodes.h index 8661baceb2..6fd6a316eb 100644 --- a/include/libs/nodes/plannodes.h +++ b/include/libs/nodes/plannodes.h @@ -317,6 +317,7 @@ typedef struct SSystemTableScanPhysiNode { SEpSet mgmtEpSet; bool showRewrite; int32_t accountId; + bool sysInfo; } SSystemTableScanPhysiNode; typedef struct STableScanPhysiNode { diff --git a/include/libs/parser/parser.h b/include/libs/parser/parser.h index 717278d51d..95bde85864 100644 --- a/include/libs/parser/parser.h +++ b/include/libs/parser/parser.h @@ -49,6 +49,7 @@ typedef struct SParseContext { SStmtCallback* pStmtCb; const char* pUser; bool isSuperUser; + bool enableSysInfo; bool async; int8_t schemalessType; const char* svrVer; diff --git a/include/libs/planner/planner.h b/include/libs/planner/planner.h index d1a5c5db10..05caa7a7bb 100644 --- a/include/libs/planner/planner.h +++ b/include/libs/planner/planner.h @@ -38,6 +38,7 @@ typedef struct SPlanContext { char* pMsg; int32_t msgLen; const char* pUser; + bool sysInfo; } SPlanContext; // Create the physical plan for the query, according to the AST. diff --git a/include/libs/qcom/query.h b/include/libs/qcom/query.h index 34d870397f..1fa7dca7dc 100644 --- a/include/libs/qcom/query.h +++ b/include/libs/qcom/query.h @@ -215,6 +215,7 @@ void initQueryModuleMsgHandle(); const SSchema* tGetTbnameColumnSchema(); bool tIsValidSchema(struct SSchema* pSchema, int32_t numOfCols, int32_t numOfTags); +int32_t queryCreateCTableMetaFromMsg(STableMetaRsp *msg, SCTableMeta *pMeta); int32_t queryCreateTableMetaFromMsg(STableMetaRsp* msg, bool isSuperTable, STableMeta** pMeta); char* jobTaskStatusStr(int32_t status); diff --git a/include/os/os.h b/include/os/os.h index b036002f8a..71966061a1 100644 --- a/include/os/os.h +++ b/include/os/os.h @@ -79,6 +79,7 @@ extern "C" { #include #include +#include "taoserror.h" #include "osAtomic.h" #include "osDef.h" #include "osDir.h" diff --git a/include/os/osSemaphore.h b/include/os/osSemaphore.h index 7fca20d75e..e52da96f01 100644 --- a/include/os/osSemaphore.h +++ b/include/os/osSemaphore.h @@ -23,10 +23,9 @@ extern "C" { #include #if defined(_TD_DARWIN_64) - +#include // typedef struct tsem_s *tsem_t; -typedef struct bosal_sem_t *tsem_t; - +typedef dispatch_semaphore_t tsem_t; int tsem_init(tsem_t *sem, int pshared, unsigned int value); int tsem_wait(tsem_t *sem); diff --git a/include/util/tcompare.h b/include/util/tcompare.h index cc9e8ae464..c7a3ca20f2 100644 --- a/include/util/tcompare.h +++ b/include/util/tcompare.h @@ -105,6 +105,97 @@ int32_t compareStrPatternNotMatch(const void *pLeft, const void *pRight); int32_t compareWStrPatternMatch(const void *pLeft, const void *pRight); int32_t compareWStrPatternNotMatch(const void *pLeft, const void *pRight); +int32_t compareInt8Int16(const void *pLeft, const void *pRight); +int32_t compareInt8Int32(const void *pLeft, const void *pRight); +int32_t compareInt8Int64(const void *pLeft, const void *pRight); +int32_t compareInt8Float(const void *pLeft, const void *pRight); +int32_t compareInt8Double(const void *pLeft, const void *pRight); +int32_t compareInt8Uint8(const void *pLeft, const void *pRight); +int32_t compareInt8Uint16(const void *pLeft, const void *pRight); +int32_t compareInt8Uint32(const void *pLeft, const void *pRight); +int32_t compareInt8Uint64(const void *pLeft, const void *pRight); +int32_t compareInt16Int8(const void *pLeft, const void *pRight); +int32_t compareInt16Int32(const void *pLeft, const void *pRight); +int32_t compareInt16Int64(const void *pLeft, const void *pRight); +int32_t compareInt16Float(const void *pLeft, const void *pRight); +int32_t compareInt16Double(const void *pLeft, const void *pRight); +int32_t compareInt16Uint8(const void *pLeft, const void *pRight); +int32_t compareInt16Uint16(const void *pLeft, const void *pRight); +int32_t compareInt16Uint32(const void *pLeft, const void *pRight); +int32_t compareInt16Uint64(const void *pLeft, const void *pRight); +int32_t compareInt32Int8(const void *pLeft, const void *pRight); +int32_t compareInt32Int16(const void *pLeft, const void *pRight); +int32_t compareInt32Int64(const void *pLeft, const void *pRight); +int32_t compareInt32Float(const void *pLeft, const void *pRight); +int32_t compareInt32Double(const void *pLeft, const void *pRight); +int32_t compareInt32Uint8(const void *pLeft, const void *pRight); +int32_t compareInt32Uint16(const void *pLeft, const void *pRight); +int32_t compareInt32Uint32(const void *pLeft, const void *pRight); +int32_t compareInt32Uint64(const void *pLeft, const void *pRight); +int32_t compareInt64Int8(const void *pLeft, const void *pRight); +int32_t compareInt64Int16(const void *pLeft, const void *pRight); +int32_t compareInt64Int32(const void *pLeft, const void *pRight); +int32_t compareInt64Float(const void *pLeft, const void *pRight); +int32_t compareInt64Double(const void *pLeft, const void *pRight); +int32_t compareInt64Uint8(const void *pLeft, const void *pRight); +int32_t compareInt64Uint16(const void *pLeft, const void *pRight); +int32_t compareInt64Uint32(const void *pLeft, const void *pRight); +int32_t compareInt64Uint64(const void *pLeft, const void *pRight); +int32_t compareFloatInt8(const void *pLeft, const void *pRight); +int32_t compareFloatInt16(const void *pLeft, const void *pRight); +int32_t compareFloatInt32(const void *pLeft, const void *pRight); +int32_t compareFloatInt64(const void *pLeft, const void *pRight); +int32_t compareFloatDouble(const void *pLeft, const void *pRight); +int32_t compareFloatUint8(const void *pLeft, const void *pRight); +int32_t compareFloatUint16(const void *pLeft, const void *pRight); +int32_t compareFloatUint32(const void *pLeft, const void *pRight); +int32_t compareFloatUint64(const void *pLeft, const void *pRight); +int32_t compareDoubleInt8(const void *pLeft, const void *pRight); +int32_t compareDoubleInt16(const void *pLeft, const void *pRight); +int32_t compareDoubleInt32(const void *pLeft, const void *pRight); +int32_t compareDoubleInt64(const void *pLeft, const void *pRight); +int32_t compareDoubleFloat(const void *pLeft, const void *pRight); +int32_t compareDoubleUint8(const void *pLeft, const void *pRight); +int32_t compareDoubleUint16(const void *pLeft, const void *pRight); +int32_t compareDoubleUint32(const void *pLeft, const void *pRight); +int32_t compareDoubleUint64(const void *pLeft, const void *pRight); +int32_t compareUint8Int8(const void *pLeft, const void *pRight); +int32_t compareUint8Int16(const void *pLeft, const void *pRight); +int32_t compareUint8Int32(const void *pLeft, const void *pRight); +int32_t compareUint8Int64(const void *pLeft, const void *pRight); +int32_t compareUint8Float(const void *pLeft, const void *pRight); +int32_t compareUint8Double(const void *pLeft, const void *pRight); +int32_t compareUint8Uint16(const void *pLeft, const void *pRight); +int32_t compareUint8Uint32(const void *pLeft, const void *pRight); +int32_t compareUint8Uint64(const void *pLeft, const void *pRight); +int32_t compareUint16Int8(const void *pLeft, const void *pRight); +int32_t compareUint16Int16(const void *pLeft, const void *pRight); +int32_t compareUint16Int32(const void *pLeft, const void *pRight); +int32_t compareUint16Int64(const void *pLeft, const void *pRight); +int32_t compareUint16Float(const void *pLeft, const void *pRight); +int32_t compareUint16Double(const void *pLeft, const void *pRight); +int32_t compareUint16Uint8(const void *pLeft, const void *pRight); +int32_t compareUint16Uint32(const void *pLeft, const void *pRight); +int32_t compareUint16Uint64(const void *pLeft, const void *pRight); +int32_t compareUint32Int8(const void *pLeft, const void *pRight); +int32_t compareUint32Int16(const void *pLeft, const void *pRight); +int32_t compareUint32Int32(const void *pLeft, const void *pRight); +int32_t compareUint32Int64(const void *pLeft, const void *pRight); +int32_t compareUint32Float(const void *pLeft, const void *pRight); +int32_t compareUint32Double(const void *pLeft, const void *pRight); +int32_t compareUint32Uint8(const void *pLeft, const void *pRight); +int32_t compareUint32Uint16(const void *pLeft, const void *pRight); +int32_t compareUint32Uint64(const void *pLeft, const void *pRight); +int32_t compareUint64Int8(const void *pLeft, const void *pRight); +int32_t compareUint64Int16(const void *pLeft, const void *pRight); +int32_t compareUint64Int32(const void *pLeft, const void *pRight); +int32_t compareUint64Int64(const void *pLeft, const void *pRight); +int32_t compareUint64Float(const void *pLeft, const void *pRight); +int32_t compareUint64Double(const void *pLeft, const void *pRight); +int32_t compareUint64Uint8(const void *pLeft, const void *pRight); +int32_t compareUint64Uint16(const void *pLeft, const void *pRight); +int32_t compareUint64Uint32(const void *pLeft, const void *pRight); + __compar_fn_t getComparFunc(int32_t type, int32_t optr); __compar_fn_t getKeyComparFunc(int32_t keyType, int32_t order); int32_t doCompare(const char *a, const char *b, int32_t type, size_t size); diff --git a/include/util/thash.h b/include/util/thash.h index 781c22a56a..f4d09eb090 100644 --- a/include/util/thash.h +++ b/include/util/thash.h @@ -210,6 +210,8 @@ void taosHashSetEqualFp(SHashObj *pHashObj, _equal_fn_t fp); */ void taosHashSetFreeFp(SHashObj *pHashObj, _hash_free_fn_t fp); +int64_t taosHashGetCompTimes(SHashObj *pHashObj); + #ifdef __cplusplus } #endif diff --git a/packaging/MPtestJenkinsfile b/packaging/MPtestJenkinsfile new file mode 100644 index 0000000000..a003bf354c --- /dev/null +++ b/packaging/MPtestJenkinsfile @@ -0,0 +1,203 @@ +def sync_source(branch_name) { + sh ''' + hostname + ip addr|grep 192|awk '{print $2}'|sed "s/\\/.*//" + echo ''' + branch_name + ''' + ''' + sh ''' + cd ${TDINTERNAL_ROOT_DIR} + git reset --hard + git fetch || git fetch + git checkout ''' + branch_name + ''' -f + git branch + git pull || git pull + git log | head -n 20 + cd ${TDENGINE_ROOT_DIR} + git reset --hard + git fetch || git fetch + rm -rf examples/rust/ + git checkout ''' + branch_name + ''' -f + git branch + git pull || git pull + git log | head -n 20 + git submodule update --init --recursive + ''' + return 1 +} +def run_test() { + sh ''' + cd ${TDENGINE_ROOT_DIR}/packaging + + ''' + sh ''' + export LD_LIBRARY_PATH=${TDINTERNAL_ROOT_DIR}/debug/build/lib + ./fulltest.sh + ''' + sh ''' + cd ${TDENGINE_ROOT_DIR}/tests + ./test-all.sh b1fq + ''' +} +def build_run() { + sync_source("${BRANCH_NAME}") +} +pipeline { + agent none + parameters { + string ( + name:'version', + defaultValue:'3.0.0.1', + description: 'release version number,eg: 3.0.0.1 or 3.0.0.' + ) + string ( + name:'baseVersion', + defaultValue:'3.0.0.1', + description: 'This number of baseVerison is generally not modified.Now it is 3.0.0.1' + ) + string ( + name:'toolsVersion', + defaultValue:'2.1.2', + description: 'This number of baseVerison is generally not modified.Now it is 3.0.0.1' + ) + string ( + name:'toolsBaseVersion', + defaultValue:'2.1.2', + description: 'This number of baseVerison is generally not modified.Now it is 3.0.0.1' + ) + } + environment{ + WORK_DIR = '/var/lib/jenkins/workspace' + TDINTERNAL_ROOT_DIR = '/var/lib/jenkins/workspace/TDinternal' + TDENGINE_ROOT_DIR = '/var/lib/jenkins/workspace/TDinternal/community' + BRANCH_NAME = '3.0' + + TD_SERVER_TAR = "TDengine-server-${version}-Linux-x64.tar.gz" + BASE_TD_SERVER_TAR = "TDengine-server-${baseVersion}-Linux-x64.tar.gz" + + TD_SERVER_ARM_TAR = "TDengine-server-${version}-Linux-arm64.tar.gz" + BASE_TD_SERVER_ARM_TAR = "TDengine-server-${baseVersion}-Linux-arm64.tar.gz" + + TD_SERVER_LITE_TAR = "TDengine-server-${version}-Linux-x64-Lite.tar.gz" + BASE_TD_SERVER_LITE_TAR = "TDengine-server-${baseVersion}-Linux-x64-Lite.tar.gz" + + TD_CLIENT_TAR = "TDengine-client-${version}-Linux-x64.tar.gz" + BASE_TD_CLIENT_TAR = "TDengine-client-${baseVersion}-Linux-x64.tar.gz" + + TD_CLIENT_ARM_TAR = "TDengine-client-${version}-Linux-arm64.tar.gz" + BASE_TD_CLIENT_ARM_TAR = "TDengine-client-${baseVersion}-Linux-arm64.tar.gz" + + TD_CLIENT_LITE_TAR = "TDengine-client-${version}-Linux-x64-Lite.tar.gz" + BASE_TD_CLIENT_LITE_TAR = "TDengine-client-${baseVersion}-Linux-x64-Lite.tar.gz" + + TD_SERVER_RPM = "TDengine-server-${version}-Linux-x64.rpm" + + TD_SERVER_DEB = "TDengine-server-${version}-Linux-x64.deb" + + TD_SERVER_EXE = "TDengine-server-${version}-Windows-x64.exe" + + TD_CLIENT_EXE = "TDengine-client-${version}-Windows-x64.exe" + + TD_TOOLS_TAR = "taosTools-${toolsVersion}-Linux-x64.tar.gz" + + + } + stages { + stage ('RUN') { + parallel { + stage('ubuntu16') { + agent{label " ubuntu16 "} + steps { + timeout(time: 30, unit: 'MINUTES'){ + sync_source("${BRANCH_NAME}") + sh ''' + cd ${TDENGINE_ROOT_DIR}/packaging + bash testpackage.sh ${TD_SERVER_TAR} ${version} ${BASE_TD_SERVER_TAR} ${baseVersion} server + python3 checkPackageRuning.py + ''' + sh ''' + cd ${TDENGINE_ROOT_DIR}/packaging + bash testpackage.sh ${TD_SERVER_LITE_TAR} ${version} ${BASE_TD_SERVER_LITE_TAR} ${baseVersion} server + python3 checkPackageRuning.py + ''' + sh ''' + cd ${TDENGINE_ROOT_DIR}/packaging + bash testpackage.sh ${TD_SERVER_DEB} ${version} ${BASE_TD_SERVER_TAR} ${baseVersion} server + python3 checkPackageRuning.py + ''' + } + } + } + stage('ubuntu18') { + agent{label " ubuntu18 "} + steps { + timeout(time: 30, unit: 'MINUTES'){ + sync_source("${BRANCH_NAME}") + sh ''' + cd ${TDENGINE_ROOT_DIR}/packaging + bash testpackage.sh ${TD_SERVER_TAR} ${version} ${BASE_TD_SERVER_TAR} ${baseVersion} server + python3 checkPackageRuning.py + ''' + sh ''' + cd ${TDENGINE_ROOT_DIR}/packaging + bash testpackage.sh ${TD_SERVER_LITE_TAR} ${version} ${BASE_TD_SERVER_LITE_TAR} ${baseVersion} server + python3 checkPackageRuning.py + ''' + sh ''' + cd ${TDENGINE_ROOT_DIR}/packaging + bash testpackage.sh ${TD_SERVER_DEB} ${version} ${BASE_TD_SERVER_DEB} ${baseVersion} server + python3 checkPackageRuning.py + ''' + } + } + } + stage('centos7') { + agent{label " centos7_9 "} + steps { + timeout(time: 30, unit: 'MINUTES'){ + sync_source("${BRANCH_NAME}") + sh ''' + cd ${TDENGINE_ROOT_DIR}/packaging + bash testpackage.sh ${TD_SERVER_TAR} ${version} ${BASE_TD_SERVER_TAR} ${baseVersion} server + python3 checkPackageRuning.py + ''' + sh ''' + cd ${TDENGINE_ROOT_DIR}/packaging + bash testpackage.sh ${TD_SERVER_LITE_TAR} ${version} ${BASE_TD_SERVER_LITE_TAR} ${baseVersion} server + python3 checkPackageRuning.py + ''' + sh ''' + cd ${TDENGINE_ROOT_DIR}/packaging + bash testpackage.sh ${TD_SERVER_RPM} ${version} ${BASE_TD_SERVER_TAR} ${baseVersion} server + python3 checkPackageRuning.py + ''' + } + } + } + stage('centos8') { + agent{label " centos8_3 "} + steps { + timeout(time: 30, unit: 'MINUTES'){ + sync_source("${BRANCH_NAME}") + sh ''' + cd ${TDENGINE_ROOT_DIR}/packaging + bash testpackage.sh ${TD_SERVER_TAR} ${version} ${BASE_TD_SERVER_TAR} ${baseVersion} server + python3 checkPackageRuning.py + rmtaos + ''' + sh ''' + cd ${TDENGINE_ROOT_DIR}/packaging + bash testpackage.sh ${TD_SERVER_LITE_TAR} ${version} ${BASE_TD_SERVER_LITE_TAR} ${baseVersion} server + python3 checkPackageRuning.py + ''' + sh ''' + cd ${TDENGINE_ROOT_DIR}/packaging + bash testpackage.sh ${TD_SERVER_RPM} ${version} ${BASE_TD_SERVER_TAR} ${baseVersion} server + python3 checkPackageRuning.py + ''' + } + } + } + } + } + } +} \ No newline at end of file diff --git a/packaging/cfg/taos.cfg b/packaging/cfg/taos.cfg index aae2e7c856..87f465fdb9 100644 --- a/packaging/cfg/taos.cfg +++ b/packaging/cfg/taos.cfg @@ -38,7 +38,7 @@ # The interval of dnode reporting status to mnode # statusInterval 1 -# The interval for taos shell to send heartbeat to mnode +# The interval for TDengine CLI to send heartbeat to mnode # shellActivityTimer 3 # The minimum sliding window time, milli-second diff --git a/packaging/checkPackageRuning.py b/packaging/checkPackageRuning.py new file mode 100755 index 0000000000..c0d1e8b86c --- /dev/null +++ b/packaging/checkPackageRuning.py @@ -0,0 +1,103 @@ +#!/usr/bin/python +################################################################### +# Copyright (c) 2016 by TAOS Technologies, Inc. +# All rights reserved. +# +# This file is proprietary and confidential to TAOS Technologies. +# No part of this file may be reproduced, stored, transmitted, +# disclosed or used in any form or by any means other than as +# expressly provided by the written permission from Jianhui Tao +# +################################################################### +# install pip +# pip install src/connector/python/ + +# -*- coding: utf-8 -*- +import sys , os +import getopt +import subprocess +# from this import d +import time + +# install taospy + +out = subprocess.getoutput("pip3 show taospy|grep Version| awk -F ':' '{print $2}' ") +print("taospy version %s "%out) +if (out == "" ): + os.system("pip install git+https://github.com/taosdata/taos-connector-python.git") + print("install taos python connector") +else: + os.system("pip3 install --upgrade taospy ") + + + +# start taosd prepare +os.system("rm -rf /var/lib/taos/*") +os.system("systemctl restart taosd ") + +# wait a moment ,at least 5 seconds +time.sleep(5) + +# prepare data by taosBenchmark + +os.system("taosBenchmark -y -n 100 -t 100") + +import taos + +conn = taos.connect(host="localhost", + user="root", + password="taosdata", + database="test", + port=6030, + config="/etc/taos", # for windows the default value is C:\TDengine\cfg + timezone="Asia/Shanghai") # default your host's timezone + +server_version = conn.server_info +print("server_version", server_version) +client_version = conn.client_info +print("client_version", client_version) # 3.0.0.0 + +# Execute a sql and get its result set. It's useful for SELECT statement +result: taos.TaosResult = conn.query("SELECT count(*) from test.meters") + +data = result.fetch_all() + +if data[0][0] !=10000: + print(" taosBenchmark work not as expected ") + sys.exit(1) +else: + print(" taosBenchmark work as expected ") + +# test taosdump dump out data and dump in data + +# dump out datas +os.system("taosdump --version") +os.system("mkdir -p /tmp/dumpdata") +os.system("rm -rf /tmp/dumpdata/*") + + + +# dump data out +print("taosdump dump out data") + +os.system("taosdump -o /tmp/dumpdata -D test -y ") + +# drop database of test +print("drop database test") +os.system(" taos -s ' drop database test ;' ") + +# dump data in +print("taosdump dump data in") +os.system("taosdump -i /tmp/dumpdata -y ") + +result = conn.query("SELECT count(*) from test.meters") + +data = result.fetch_all() + +if data[0][0] !=10000: + print(" taosdump work not as expected ") + sys.exit(1) +else: + print(" taosdump work as expected ") + +conn.close() \ No newline at end of file diff --git a/packaging/deb/DEBIAN/prerm b/packaging/deb/DEBIAN/prerm index 4953102842..65f261db2c 100644 --- a/packaging/deb/DEBIAN/prerm +++ b/packaging/deb/DEBIAN/prerm @@ -1,6 +1,6 @@ #!/bin/bash -if [ $1 -eq "abort-upgrade" ]; then +if [ "$1"x = "abort-upgrade"x ]; then exit 0 fi diff --git a/packaging/docker/README.md b/packaging/docker/README.md index e41182f471..763ab73724 100644 --- a/packaging/docker/README.md +++ b/packaging/docker/README.md @@ -47,7 +47,7 @@ taos> show databases; Query OK, 1 row(s) in set (0.002843s) ``` -Since TDengine use container hostname to establish connections, it's a bit more complex to use taos shell and native connectors(such as JDBC-JNI) with TDengine container instance. This is the recommended way to expose ports and use TDengine with docker in simple cases. If you want to use taos shell or taosc/connectors smoothly outside the `tdengine` container, see next use cases that match you need. +Since TDengine use container hostname to establish connections, it's a bit more complex to use TDengine CLI and native connectors(such as JDBC-JNI) with TDengine container instance. This is the recommended way to expose ports and use TDengine with docker in simple cases. If you want to use TDengine CLI or taosc/connectors smoothly outside the `tdengine` container, see next use cases that match you need. ### Start with host network @@ -87,7 +87,7 @@ docker run -d \ This command starts a docker container with TDengine server running and maps the container's TCP ports from 6030 to 6049 to the host's ports from 6030 to 6049 with TCP protocol and UDP ports range 6030-6039 to the host's UDP ports 6030-6039. If the host is already running TDengine server and occupying the same port(s), you need to map the container's port to a different unused port segment. (Please see TDengine 2.0 Port Description for details). In order to support TDengine clients accessing TDengine server services, both TCP and UDP ports need to be exposed by default(unless `rpcForceTcp` is set to `1`). -If you want to use taos shell or native connectors([JDBC-JNI](https://www.taosdata.com/cn/documentation/connector/java), or [driver-go](https://github.com/taosdata/driver-go)), you need to make sure the `TAOS_FQDN` is resolvable at `/etc/hosts` or with custom DNS service. +If you want to use TDengine CLI or native connectors([JDBC-JNI](https://www.taosdata.com/cn/documentation/connector/java), or [driver-go](https://github.com/taosdata/driver-go)), you need to make sure the `TAOS_FQDN` is resolvable at `/etc/hosts` or with custom DNS service. If you set the `TAOS_FQDN` to host's hostname, it will works as using `hosts` network like previous use case. Otherwise, like in `-e TAOS_FQDN=tdengine`, you can add the hostname record `tdengine` into `/etc/hosts` (use `127.0.0.1` here in host path, if use TDengine client/application in other hosts, you should set the right ip to the host eg. `192.168.10.1`(check the real ip in host with `hostname -i` or `ip route list default`) to make the TDengine endpoint resolvable): @@ -158,7 +158,7 @@ When you build your application with docker, you should add the TDengine client FROM ubuntu:20.04 RUN apt-get update && apt-get install -y wget ENV TDENGINE_VERSION=2.4.0.0 -RUN wget -c https://www.taosdata.com/assets-download/TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ +RUN wget -c https://www.taosdata.com/assets-download/3.0/TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ && tar xvf TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ && cd TDengine-client-${TDENGINE_VERSION} \ && ./install_client.sh \ @@ -265,7 +265,7 @@ Full version of dockerfile could be: ```dockerfile FROM golang:1.17.6-buster as builder ENV TDENGINE_VERSION=2.4.0.0 -RUN wget -c https://www.taosdata.com/assets-download/TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ +RUN wget -c https://www.taosdata.com/assets-download/3.0/TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ && tar xvf TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ && cd TDengine-client-${TDENGINE_VERSION} \ && ./install_client.sh \ @@ -279,7 +279,7 @@ RUN go env && go mod tidy && go build FROM ubuntu:20.04 RUN apt-get update && apt-get install -y wget ENV TDENGINE_VERSION=2.4.0.0 -RUN wget -c https://www.taosdata.com/assets-download/TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ +RUN wget -c https://www.taosdata.com/assets-download/3.0/TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ && tar xvf TDengine-client-${TDENGINE_VERSION}-Linux-x64.tar.gz \ && cd TDengine-client-${TDENGINE_VERSION} \ && ./install_client.sh \ @@ -391,7 +391,7 @@ test_td-1_1 /usr/bin/entrypoint.sh taosd Up 6030/tcp, 6031/tcp, test_td-2_1 /usr/bin/entrypoint.sh taosd Up 6030/tcp, 6031/tcp, 6032/tcp, 6033/tcp, 6034/tcp, 6035/tcp, 6036/tcp, 6037/tcp, 6038/tcp, 6039/tcp, 6040/tcp, 6041/tcp, 6042/tcp ``` -Check dnodes with taos shell: +Check dnodes with TDengine CLI: ```bash $ docker-compose exec td-1 taos -s "show dnodes" diff --git a/packaging/testpackage.sh b/packaging/testpackage.sh new file mode 100755 index 0000000000..054c24eb5d --- /dev/null +++ b/packaging/testpackage.sh @@ -0,0 +1,133 @@ +#!/bin/sh + + +packgeName=$1 +version=$2 +originPackageName=$3 +originversion=$4 +testFile=$5 +subFile="taos.tar.gz" + +if [ ${testFile} = "server" ];then + tdPath="TDengine-server-${version}" + originTdpPath="TDengine-server-${originversion}" + installCmd="install.sh" +elif [ ${testFile} = "client" ];then + tdPath="TDengine-client-${version}" + originTdpPath="TDengine-client-${originversion}" + installCmd="install_client.sh" +elif [ ${testFile} = "tools" ];then + tdPath="taosTools-${version}" + originTdpPath="taosTools-${originversion}" + installCmd="install-taostools.sh" +fi + +function cmdInstall { +comd=$1 +if command -v ${comd} ;then + echo "${comd} is already installed" +else + if command -v apt ;then + apt-get install ${comd} -y + elif command -v yum ;then + yum -y install ${comd} + echo "you should install ${comd} manually" + fi +fi +} + + +echo "Uninstall all components of TDeingne" + +if command -v rmtaos ;then + echo "uninstall all components of TDeingne:rmtaos" + rmtaos +else + echo "os doesn't include TDengine " +fi + +if command -v rmtaostools ;then + echo "uninstall all components of TDeingne:rmtaostools" + rmtaostools +else + echo "os doesn't include rmtaostools " +fi + + +cmdInstall tree +cmdInstall wget + +echo "new workroom path" +installPath="/usr/local/src/packageTest" +oriInstallPath="/usr/local/src/packageTest/3.1" + +if [ ! -d ${installPath} ] ;then + mkdir -p ${installPath} +else + echo "${installPath} already exists" +fi + + +if [ ! -d ${oriInstallPath} ] ;then + mkdir -p ${oriInstallPath} +else + echo "${oriInstallPath} already exists" +fi + +echo "decompress installPackage" + +cd ${installPath} +wget https://www.taosdata.com/assets-download/3.0/${packgeName} +cd ${oriInstallPath} +wget https://www.taosdata.com/assets-download/3.0/${originPackageName} + + +if [[ ${packgeName} =~ "deb" ]];then + cd ${installPath} + echo "dpkg ${packgeName}" && dpkg -i ${packgeName} +elif [[ ${packgeName} =~ "rpm" ]];then + cd ${installPath} + echo "rpm ${packgeName}" && rpm -ivh ${packgeName} +elif [[ ${packgeName} =~ "tar" ]];then + echo "tar ${packgeName}" && tar -xvf ${packgeName} + cd ${oriInstallPath} + echo "tar -xvf ${originPackageName}" && tar -xvf ${originPackageName} + cd ${installPath} + echo "tar -xvf ${packgeName}" && tar -xvf ${packgeName} + + + if [ ${testFile} != "tools" ] ;then + cd ${installPath}/${tdPath} && tar vxf ${subFile} + cd ${oriInstallPath}/${originTdpPath} && tar vxf ${subFile} + fi + + echo "check installPackage File" + + cd ${installPath} + + tree ${oriInstallPath}/${originTdpPath} > ${originPackageName}_checkfile + tree ${installPath}/${tdPath} > ${packgeName}_checkfile + + diff ${packgeName}_checkfile ${originPackageName}_checkfile > ${installPath}/diffFile.log + diffNumbers=`cat ${installPath}/diffFile.log |wc -l ` + if [ ${diffNumbers} != 0 ];then + echo "The number and names of files have changed from the previous installation package" + echo `cat ${installPath}/diffFile.log` + exit -1 + fi + + cd ${installPath}/${tdPath} + if [ ${testFile} = "server" ];then + bash ${installCmd} -e no + else + bash ${installCmd} + fi + if [[ ${packgeName} =~ "Lite" ]];then + cd ${installPath} + wget https://www.taosdata.com/assets-download/3.0/taosTools-2.1.2-Linux-x64.tar.gz + tar xvf taosTools-2.1.2-Linux-x64.tar.gz + cd taosTools-2.1.2 && bash install-taostools.sh + fi + +fi + diff --git a/packaging/tools/make_install.sh b/packaging/tools/make_install.sh index 6a95ace99e..f554942ce3 100755 --- a/packaging/tools/make_install.sh +++ b/packaging/tools/make_install.sh @@ -381,8 +381,7 @@ function install_header() { ${install_main_dir}/include || ${csudo}cp -f ${source_dir}/include/client/taos.h ${source_dir}/include/common/taosdef.h ${source_dir}/include/util/taoserror.h ${source_dir}/include/libs/function/taosudf.h \ ${install_main_2_dir}/include && - ${csudo}chmod 644 ${install_main_dir}/include/* ||: - ${csudo}chmod 644 ${install_main_2_dir}/include/* + ${csudo}chmod 644 ${install_main_dir}/include/* || ${csudo}chmod 644 ${install_main_2_dir}/include/* fi } diff --git a/source/client/CMakeLists.txt b/source/client/CMakeLists.txt index f52edbe71f..e8e3c87849 100644 --- a/source/client/CMakeLists.txt +++ b/source/client/CMakeLists.txt @@ -27,11 +27,18 @@ else() INCLUDE_DIRECTORIES(jni/linux) endif() +set_target_properties( + taos + PROPERTIES + CLEAN_DIRECT_OUTPUT + 1 +) + set_target_properties( taos PROPERTIES VERSION ${TD_VER_NUMBER} - SOVERSION ${TD_VER_NUMBER} + SOVERSION 1 ) add_library(taos_static STATIC ${CLIENT_SRC}) diff --git a/source/client/inc/clientInt.h b/source/client/inc/clientInt.h index 855dfb15ee..07e5f75e87 100644 --- a/source/client/inc/clientInt.h +++ b/source/client/inc/clientInt.h @@ -95,15 +95,15 @@ typedef struct { } SClientHbMgr; typedef struct SQueryExecMetric { - int64_t start; // start timestamp, us + int64_t start; // start timestamp, us int64_t syntaxStart; // start to parse, us - int64_t syntaxEnd; // end to parse, us - int64_t ctgStart; // start to parse, us - int64_t ctgEnd; // end to parse, us + int64_t syntaxEnd; // end to parse, us + int64_t ctgStart; // start to parse, us + int64_t ctgEnd; // end to parse, us int64_t semanticEnd; int64_t execEnd; - int64_t send; // start to send to server, us - int64_t rsp; // receive response from server, us + int64_t send; // start to send to server, us + int64_t rsp; // receive response from server, us } SQueryExecMetric; struct SAppInstInfo { @@ -137,6 +137,7 @@ typedef struct STscObj { char db[TSDB_DB_FNAME_LEN]; char sVer[TSDB_VERSION_LEN]; char sDetailVer[128]; + int8_t sysInfo; int8_t connType; int32_t acctId; uint32_t connId; @@ -257,7 +258,7 @@ SRequestObj* execQuery(uint64_t connId, const char* sql, int sqlLen, bool valida TAOS_RES* taosQueryImpl(TAOS* taos, const char* sql, bool validateOnly); void taosAsyncQueryImpl(uint64_t connId, const char* sql, __taos_async_fn_t fp, void* param, bool validateOnly); -int32_t getVersion1BlockMetaSize(const char* p, int32_t numOfCols); +int32_t getVersion1BlockMetaSize(const char* p, int32_t numOfCols); static FORCE_INLINE SReqResultInfo* tmqGetCurResInfo(TAOS_RES* res) { SMqRspObj* msg = (SMqRspObj*)res; @@ -368,8 +369,9 @@ void launchAsyncQuery(SRequestObj* pRequest, SQuery* pQuery, SMetaData* int32_t refreshMeta(STscObj* pTscObj, SRequestObj* pRequest); int32_t updateQnodeList(SAppInstInfo* pInfo, SArray* pNodeList); void doAsyncQuery(SRequestObj* pRequest, bool forceUpdateMeta); -int32_t removeMeta(STscObj* pTscObj, SArray* tbList); // todo move to clientImpl.c and become a static function -int32_t handleAlterTbExecRes(void* res, struct SCatalog* pCatalog); // todo move to xxx +int32_t removeMeta(STscObj* pTscObj, SArray* tbList); +int32_t handleAlterTbExecRes(void* res, struct SCatalog* pCatalog); +int32_t handleCreateTbExecRes(void* res, SCatalog* pCatalog); bool qnodeRequired(SRequestObj* pRequest); #ifdef __cplusplus diff --git a/source/client/src/clientHb.c b/source/client/src/clientHb.c index 9475d1b51e..56e3527f96 100644 --- a/source/client/src/clientHb.c +++ b/source/client/src/clientHb.c @@ -145,7 +145,7 @@ static int32_t hbProcessStbInfoRsp(void *value, int32_t valueLen, struct SCatalo } static int32_t hbQueryHbRspHandle(SAppHbMgr *pAppHbMgr, SClientHbRsp *pRsp) { - SClientHbReq *pReq = taosHashGet(pAppHbMgr->activeInfo, &pRsp->connKey, sizeof(SClientHbKey)); + SClientHbReq *pReq = taosHashAcquire(pAppHbMgr->activeInfo, &pRsp->connKey, sizeof(SClientHbKey)); if (NULL == pReq) { tscWarn("pReq to get activeInfo, may be dropped, refId:%" PRIx64 ", type:%d", pRsp->connKey.tscRid, pRsp->connKey.connType); @@ -260,6 +260,8 @@ static int32_t hbQueryHbRspHandle(SAppHbMgr *pAppHbMgr, SClientHbRsp *pRsp) { } } + taosHashRelease(pAppHbMgr->activeInfo, pReq); + return TSDB_CODE_SUCCESS; } @@ -914,10 +916,11 @@ int hbRegisterConn(SAppHbMgr *pAppHbMgr, int64_t tscRefId, int64_t clusterId, in } void hbDeregisterConn(SAppHbMgr *pAppHbMgr, SClientHbKey connKey) { - SClientHbReq *pReq = taosHashGet(pAppHbMgr->activeInfo, &connKey, sizeof(SClientHbKey)); + SClientHbReq *pReq = taosHashAcquire(pAppHbMgr->activeInfo, &connKey, sizeof(SClientHbKey)); if (pReq) { tFreeClientHbReq(pReq); taosHashRemove(pAppHbMgr->activeInfo, &connKey, sizeof(SClientHbKey)); + taosHashRelease(pAppHbMgr->activeInfo, pReq); } if (NULL == pReq) { diff --git a/source/client/src/clientImpl.c b/source/client/src/clientImpl.c index 998b9cee5c..2baca1288f 100644 --- a/source/client/src/clientImpl.c +++ b/source/client/src/clientImpl.c @@ -215,6 +215,7 @@ int32_t parseSql(SRequestObj* pRequest, bool topicQuery, SQuery** pQuery, SStmtC .pUser = pTscObj->user, .schemalessType = pTscObj->schemalessType, .isSuperUser = (0 == strcmp(pTscObj->user, TSDB_DEFAULT_USER)), + .enableSysInfo = pTscObj->sysInfo, .svrVer = pTscObj->sVer, .nodeOffline = (pTscObj->pAppInfo->onlineDnodes < pTscObj->pAppInfo->totalDnodes)}; @@ -246,7 +247,7 @@ int32_t parseSql(SRequestObj* pRequest, bool topicQuery, SQuery** pQuery, SStmtC int32_t execLocalCmd(SRequestObj* pRequest, SQuery* pQuery) { SRetrieveTableRsp* pRsp = NULL; - int32_t code = qExecCommand(pQuery->pRoot, &pRsp); + int32_t code = qExecCommand(pRequest->pTscObj->sysInfo, pQuery->pRoot, &pRsp); if (TSDB_CODE_SUCCESS == code && NULL != pRsp) { code = setQueryResultFromRsp(&pRequest->body.resInfo, pRsp, false, true); } @@ -284,7 +285,7 @@ void asyncExecLocalCmd(SRequestObj* pRequest, SQuery* pQuery) { return; } - int32_t code = qExecCommand(pQuery->pRoot, &pRsp); + int32_t code = qExecCommand(pRequest->pTscObj->sysInfo, pQuery->pRoot, &pRsp); if (TSDB_CODE_SUCCESS == code && NULL != pRsp) { code = setQueryResultFromRsp(&pRequest->body.resInfo, pRsp, false, true); } @@ -419,7 +420,8 @@ int32_t getPlan(SRequestObj* pRequest, SQuery* pQuery, SQueryPlan** pPlan, SArra .showRewrite = pQuery->showRewrite, .pMsg = pRequest->msgBuf, .msgLen = ERROR_MSG_BUF_DEFAULT_SIZE, - .pUser = pRequest->pTscObj->user}; + .pUser = pRequest->pTscObj->user, + .sysInfo = pRequest->pTscObj->sysInfo}; return qCreateQueryPlan(&cxt, pPlan, pNodeList); } @@ -721,6 +723,12 @@ int32_t handleSubmitExecRes(SRequestObj* pRequest, void* res, SCatalog* pCatalog for (int32_t i = 0; i < pRsp->nBlocks; ++i) { SSubmitBlkRsp* blk = pRsp->pBlocks + i; + if (blk->pMeta) { + handleCreateTbExecRes(blk->pMeta, pCatalog); + tFreeSTableMetaRsp(blk->pMeta); + taosMemoryFreeClear(blk->pMeta); + } + if (NULL == blk->tblFName || 0 == blk->tblFName[0]) { continue; } @@ -780,6 +788,10 @@ int32_t handleAlterTbExecRes(void* res, SCatalog* pCatalog) { return catalogUpdateTableMeta(pCatalog, (STableMetaRsp*)res); } +int32_t handleCreateTbExecRes(void* res, SCatalog* pCatalog) { + return catalogUpdateTableMeta(pCatalog, (STableMetaRsp*)res); +} + int32_t handleQueryExecRsp(SRequestObj* pRequest) { if (NULL == pRequest->body.resInfo.execRes.res) { return TSDB_CODE_SUCCESS; @@ -802,6 +814,19 @@ int32_t handleQueryExecRsp(SRequestObj* pRequest) { code = handleAlterTbExecRes(pRes->res, pCatalog); break; } + case TDMT_VND_CREATE_TABLE: { + SArray* pList = (SArray*)pRes->res; + int32_t num = taosArrayGetSize(pList); + for (int32_t i = 0; i < num; ++i) { + void* res = taosArrayGetP(pList, i); + code = handleCreateTbExecRes(res, pCatalog); + } + break; + } + case TDMT_MND_CREATE_STB: { + code = handleCreateTbExecRes(pRes->res, pCatalog); + break; + } case TDMT_VND_SUBMIT: { atomic_add_fetch_64((int64_t*)&pAppInfo->summary.insertBytes, pRes->numOfBytes); @@ -861,17 +886,13 @@ void schedulerExecCb(SExecResult* pResult, void* param, int32_t code) { return; } - if (code == TSDB_CODE_SUCCESS) { - code = handleQueryExecRsp(pRequest); - ASSERT(pRequest->code == TSDB_CODE_SUCCESS); - pRequest->code = code; - } - tscDebug("schedulerExecCb request type %s", TMSG_INFO(pRequest->type)); - if (NEED_CLIENT_RM_TBLMETA_REQ(pRequest->type)) { + if (NEED_CLIENT_RM_TBLMETA_REQ(pRequest->type) && NULL == pRequest->body.resInfo.execRes.res) { removeMeta(pTscObj, pRequest->targetTableList); } + handleQueryExecRsp(pRequest); + // return to client pRequest->body.queryFp(pRequest->body.param, pRequest, code); } @@ -932,6 +953,10 @@ SRequestObj* launchQueryImpl(SRequestObj* pRequest, SQuery* pQuery, bool keepQue qDestroyQuery(pQuery); } + if (NEED_CLIENT_RM_TBLMETA_REQ(pRequest->type) && NULL == pRequest->body.resInfo.execRes.res) { + removeMeta(pRequest->pTscObj, pRequest->targetTableList); + } + handleQueryExecRsp(pRequest); if (NULL != pRequest && TSDB_CODE_SUCCESS != code) { @@ -992,7 +1017,8 @@ void launchAsyncQuery(SRequestObj* pRequest, SQuery* pQuery, SMetaData* pResultM .showRewrite = pQuery->showRewrite, .pMsg = pRequest->msgBuf, .msgLen = ERROR_MSG_BUF_DEFAULT_SIZE, - .pUser = pRequest->pTscObj->user}; + .pUser = pRequest->pTscObj->user, + .sysInfo = pRequest->pTscObj->sysInfo}; SAppInstInfo* pAppInfo = getAppInfo(pRequest); SQueryPlan* pDag = NULL; @@ -1129,10 +1155,6 @@ SRequestObj* execQuery(uint64_t connId, const char* sql, int sqlLen, bool valida inRetry = true; } while (retryNum++ < REQUEST_TOTAL_EXEC_TIMES); - if (NEED_CLIENT_RM_TBLMETA_REQ(pRequest->type)) { - removeMeta(pRequest->pTscObj, pRequest->targetTableList); - } - return pRequest; } @@ -1577,10 +1599,11 @@ static int32_t doConvertUCS4(SReqResultInfo* pResultInfo, int32_t numOfRows, int } int32_t getVersion1BlockMetaSize(const char* p, int32_t numOfCols) { - int32_t cols = *(int32_t*) (p + sizeof(int32_t) * 3); + int32_t cols = *(int32_t*)(p + sizeof(int32_t) * 3); ASSERT(numOfCols == cols); - return sizeof(int32_t) + sizeof(int32_t) + sizeof(int32_t)*3 + sizeof(uint64_t) + numOfCols * (sizeof(int8_t) + sizeof(int32_t)); + return sizeof(int32_t) + sizeof(int32_t) + sizeof(int32_t) * 3 + sizeof(uint64_t) + + numOfCols * (sizeof(int8_t) + sizeof(int32_t)); } static int32_t estimateJsonLen(SReqResultInfo* pResultInfo, int32_t numOfCols, int32_t numOfRows) { @@ -1952,7 +1975,7 @@ _OVER: int32_t appendTbToReq(SHashObj* pHash, int32_t pos1, int32_t len1, int32_t pos2, int32_t len2, const char* str, int32_t acctId, char* db) { - SName name; + SName name = {0}; if (len1 <= 0) { return -1; diff --git a/source/client/src/clientMain.c b/source/client/src/clientMain.c index 31ae443d5b..9ceb6e0683 100644 --- a/source/client/src/clientMain.c +++ b/source/client/src/clientMain.c @@ -759,6 +759,7 @@ int32_t createParseContext(const SRequestObj *pRequest, SParseContext **pCxt) { .pUser = pTscObj->user, .schemalessType = pTscObj->schemalessType, .isSuperUser = (0 == strcmp(pTscObj->user, TSDB_DEFAULT_USER)), + .enableSysInfo = pTscObj->sysInfo, .async = true, .svrVer = pTscObj->sVer, .nodeOffline = (pTscObj->pAppInfo->onlineDnodes < pTscObj->pAppInfo->totalDnodes)}; diff --git a/source/client/src/clientMsgHandler.c b/source/client/src/clientMsgHandler.c index 0c4cf23c4e..a7a16d484c 100644 --- a/source/client/src/clientMsgHandler.c +++ b/source/client/src/clientMsgHandler.c @@ -96,6 +96,7 @@ int32_t processConnectRsp(void* param, SDataBuf* pMsg, int32_t code) { connectRsp.epSet.eps[i].fqdn, connectRsp.epSet.eps[i].port, pTscObj->id); } + pTscObj->sysInfo = connectRsp.sysInfo; pTscObj->connId = connectRsp.connId; pTscObj->acctId = connectRsp.acctId; tstrncpy(pTscObj->sVer, connectRsp.sVer, tListLen(pTscObj->sVer)); @@ -232,13 +233,36 @@ int32_t processCreateSTableRsp(void* param, SDataBuf* pMsg, int32_t code) { assert(pMsg != NULL && param != NULL); SRequestObj* pRequest = param; - taosMemoryFree(pMsg->pData); if (code != TSDB_CODE_SUCCESS) { setErrno(pRequest, code); + } else { + SMCreateStbRsp createRsp = {0}; + SDecoder coder = {0}; + tDecoderInit(&coder, pMsg->pData, pMsg->len); + tDecodeSMCreateStbRsp(&coder, &createRsp); + tDecoderClear(&coder); + + pRequest->body.resInfo.execRes.msgType = TDMT_MND_CREATE_STB; + pRequest->body.resInfo.execRes.res = createRsp.pMeta; } + taosMemoryFree(pMsg->pData); + if (pRequest->body.queryFp != NULL) { - removeMeta(pRequest->pTscObj, pRequest->tableList); + SExecResult* pRes = &pRequest->body.resInfo.execRes; + + if (code == TSDB_CODE_SUCCESS) { + SCatalog* pCatalog = NULL; + int32_t ret = catalogGetHandle(pRequest->pTscObj->pAppInfo->clusterId, &pCatalog); + if (pRes->res != NULL) { + ret = handleCreateTbExecRes(pRes->res, pCatalog); + } + + if (ret != TSDB_CODE_SUCCESS) { + code = ret; + } + } + pRequest->body.queryFp(pRequest->body.param, pRequest, code); } else { tsem_post(&pRequest->body.rspSem); diff --git a/source/client/src/clientSml.c b/source/client/src/clientSml.c index 9f905a8352..4968c5c68d 100644 --- a/source/client/src/clientSml.c +++ b/source/client/src/clientSml.c @@ -85,8 +85,11 @@ typedef TSDB_SML_PROTOCOL_TYPE SMLProtocolType; typedef enum { SCHEMA_ACTION_NULL, - SCHEMA_ACTION_COLUMN, - SCHEMA_ACTION_TAG + SCHEMA_ACTION_CREATE_STABLE, + SCHEMA_ACTION_ADD_COLUMN, + SCHEMA_ACTION_ADD_TAG, + SCHEMA_ACTION_CHANGE_COLUMN_SIZE, + SCHEMA_ACTION_CHANGE_TAG_SIZE, } ESchemaAction; typedef struct { @@ -219,7 +222,7 @@ static int32_t smlBuildInvalidDataMsg(SSmlMsgBuf *pBuf, const char *msg1, const static int32_t smlGenerateSchemaAction(SSchema *colField, SHashObj *colHash, SSmlKv *kv, bool isTag, ESchemaAction *action, SSmlHandle *info) { - uint16_t *index = (uint16_t *)taosHashGet(colHash, kv->key, kv->keyLen); + uint16_t *index = colHash ? (uint16_t *)taosHashGet(colHash, kv->key, kv->keyLen) : NULL; if (index) { if (colField[*index].type != kv->type) { uError("SML:0x%" PRIx64 " point type and db type mismatch. key: %s. point type: %d, db type: %d", info->id, @@ -232,16 +235,16 @@ static int32_t smlGenerateSchemaAction(SSchema *colField, SHashObj *colHash, SSm (colField[*index].type == TSDB_DATA_TYPE_NCHAR && ((colField[*index].bytes - VARSTR_HEADER_SIZE) / TSDB_NCHAR_SIZE < kv->length))) { if (isTag) { - *action = SCHEMA_ACTION_TAG; + *action = SCHEMA_ACTION_CHANGE_TAG_SIZE; } else { - *action = SCHEMA_ACTION_COLUMN; + *action = SCHEMA_ACTION_CHANGE_COLUMN_SIZE; } } } else { if (isTag) { - *action = SCHEMA_ACTION_TAG; + *action = SCHEMA_ACTION_ADD_TAG; } else { - *action = SCHEMA_ACTION_COLUMN; + *action = SCHEMA_ACTION_ADD_COLUMN; } } return 0; @@ -310,9 +313,31 @@ static int32_t getBytes(uint8_t type, int32_t length){ } } +static int32_t smlBuildFieldsList(SSmlHandle *info, SSchema *schemaField, SHashObj *schemaHash, SArray *cols, SArray* results, int32_t numOfCols, bool isTag) { + for (int j = 0; j < taosArrayGetSize(cols); ++j) { + SSmlKv *kv = (SSmlKv *)taosArrayGetP(cols, j); + ESchemaAction action = SCHEMA_ACTION_NULL; + smlGenerateSchemaAction(schemaField, schemaHash, kv, isTag, &action, info); + if(action == SCHEMA_ACTION_ADD_COLUMN || action == SCHEMA_ACTION_ADD_TAG){ + SField field = {0}; + field.type = kv->type; + field.bytes = getBytes(kv->type, kv->length); + memcpy(field.name, kv->key, kv->keyLen); + taosArrayPush(results, &field); + }else if(action == SCHEMA_ACTION_CHANGE_COLUMN_SIZE || action == SCHEMA_ACTION_CHANGE_TAG_SIZE){ + uint16_t *index = (uint16_t *)taosHashGet(schemaHash, kv->key, kv->keyLen); + uint16_t newIndex = *index; + if(isTag) newIndex -= numOfCols; + SField *field = (SField *)taosArrayGet(results, newIndex); + field->bytes = getBytes(kv->type, kv->length); + } + } + return TSDB_CODE_SUCCESS; +} + //static int32_t smlSendMetaMsg(SSmlHandle *info, SName *pName, SSmlSTableMeta *sTableData, // int32_t colVer, int32_t tagVer, int8_t source, uint64_t suid){ -static int32_t smlSendMetaMsg(SSmlHandle *info, SName *pName, SSmlSTableMeta *sTableData, +static int32_t smlSendMetaMsg(SSmlHandle *info, SName *pName, SArray* pColumns, SArray* pTags, STableMeta *pTableMeta, ESchemaAction action){ SRequestObj* pRequest = NULL; @@ -320,6 +345,12 @@ static int32_t smlSendMetaMsg(SSmlHandle *info, SName *pName, SSmlSTableMeta *s int32_t code = TSDB_CODE_SUCCESS; SCmdMsgInfo pCmdMsg = {0}; + // put front for free + pReq.numOfColumns = taosArrayGetSize(pColumns); + pReq.pColumns = pColumns; + pReq.numOfTags = taosArrayGetSize(pTags); + pReq.pTags = pTags; + code = buildRequest(info->taos->id, "", 0, NULL, false, &pRequest); if (code != TSDB_CODE_SUCCESS) { goto end; @@ -330,91 +361,41 @@ static int32_t smlSendMetaMsg(SSmlHandle *info, SName *pName, SSmlSTableMeta *s goto end; } - if (action == SCHEMA_ACTION_NULL){ + if (action == SCHEMA_ACTION_CREATE_STABLE){ pReq.colVer = 1; pReq.tagVer = 1; pReq.suid = 0; pReq.source = TD_REQ_FROM_APP; - } else if (action == SCHEMA_ACTION_TAG){ + } else if (action == SCHEMA_ACTION_ADD_TAG || action == SCHEMA_ACTION_CHANGE_TAG_SIZE){ pReq.colVer = pTableMeta->sversion; pReq.tagVer = pTableMeta->tversion + 1; pReq.suid = pTableMeta->uid; pReq.source = TD_REQ_FROM_TAOX; - } else if (action == SCHEMA_ACTION_COLUMN){ + } else if (action == SCHEMA_ACTION_ADD_COLUMN || action == SCHEMA_ACTION_CHANGE_COLUMN_SIZE){ pReq.colVer = pTableMeta->sversion + 1; pReq.tagVer = pTableMeta->tversion; pReq.suid = pTableMeta->uid; pReq.source = TD_REQ_FROM_TAOX; } + if (pReq.numOfTags == 0){ + pReq.numOfTags = 1; + SField field = {0}; + field.type = TSDB_DATA_TYPE_NCHAR; + field.bytes = 1; + strcpy(field.name, tsSmlTagName); + taosArrayPush(pReq.pTags, &field); + } + pReq.commentLen = -1; pReq.igExists = true; tNameExtractFullName(pName, pReq.name); - if(action == SCHEMA_ACTION_NULL || action == SCHEMA_ACTION_COLUMN){ - pReq.numOfColumns = taosArrayGetSize(sTableData->cols); - pReq.pColumns = taosArrayInit(pReq.numOfColumns, sizeof(SField)); - for (int i = 0; i < pReq.numOfColumns; i++) { - SSmlKv *kv = (SSmlKv *)taosArrayGetP(sTableData->cols, i); - SField field = {0}; - field.type = kv->type; - field.bytes = getBytes(kv->type, kv->length); - memcpy(field.name, kv->key, kv->keyLen); - taosArrayPush(pReq.pColumns, &field); - } - }else if (action == SCHEMA_ACTION_TAG){ - pReq.numOfColumns = pTableMeta->tableInfo.numOfColumns; - pReq.pColumns = taosArrayInit(pReq.numOfColumns, sizeof(SField)); - for (int i = 0; i < pReq.numOfColumns; i++) { - SSchema *s = &pTableMeta->schema[i]; - SField field = {0}; - field.type = s->type; - field.bytes = s->bytes; - strcpy(field.name, s->name); - taosArrayPush(pReq.pColumns, &field); - } - } - - if(action == SCHEMA_ACTION_NULL || action == SCHEMA_ACTION_TAG){ - pReq.numOfTags = taosArrayGetSize(sTableData->tags); - if (pReq.numOfTags == 0){ - pReq.numOfTags = 1; - pReq.pTags = taosArrayInit(pReq.numOfTags, sizeof(SField)); - SField field = {0}; - field.type = TSDB_DATA_TYPE_NCHAR; - field.bytes = 1; - strcpy(field.name, tsSmlTagName); - taosArrayPush(pReq.pTags, &field); - }else{ - pReq.pTags = taosArrayInit(pReq.numOfTags, sizeof(SField)); - for (int i = 0; i < pReq.numOfTags; i++) { - SSmlKv *kv = (SSmlKv *)taosArrayGetP(sTableData->tags, i); - SField field = {0}; - field.type = kv->type; - field.bytes = getBytes(kv->type, kv->length); - memcpy(field.name, kv->key, kv->keyLen); - taosArrayPush(pReq.pTags, &field); - } - } - }else if (action == SCHEMA_ACTION_COLUMN){ - pReq.numOfTags = pTableMeta->tableInfo.numOfTags; - pReq.pTags = taosArrayInit(pReq.numOfTags, sizeof(SField)); - for (int i = 0; i < pReq.numOfTags; i++) { - SSchema *s = &pTableMeta->schema[i + pTableMeta->tableInfo.numOfColumns]; - SField field = {0}; - field.type = s->type; - field.bytes = s->bytes; - strcpy(field.name, s->name); - taosArrayPush(pReq.pTags, &field); - } - } - pCmdMsg.epSet = getEpSet_s(&info->taos->pAppInfo->mgmtEp); pCmdMsg.msgType = TDMT_MND_CREATE_STB; pCmdMsg.msgLen = tSerializeSMCreateStbReq(NULL, 0, &pReq); pCmdMsg.pMsg = taosMemoryMalloc(pCmdMsg.msgLen); if (NULL == pCmdMsg.pMsg) { - tFreeSMCreateStbReq(&pReq); code = TSDB_CODE_OUT_OF_MEMORY; goto end; } @@ -442,7 +423,10 @@ end: } static int32_t smlModifyDBSchemas(SSmlHandle *info) { - int32_t code = 0; + int32_t code = 0; + SHashObj *hashTmp = NULL; + STableMeta *pTableMeta = NULL; + SName pName = {TSDB_TABLE_NAME_T, info->taos->acctId, {0}, {0}}; strcpy(pName.dbname, info->pRequest->pDb); @@ -455,7 +439,6 @@ static int32_t smlModifyDBSchemas(SSmlHandle *info) { SSmlSTableMeta **tableMetaSml = (SSmlSTableMeta **)taosHashIterate(info->superTables, NULL); while (tableMetaSml) { SSmlSTableMeta *sTableData = *tableMetaSml; - STableMeta *pTableMeta = NULL; bool needCheckMeta = false; // for multi thread size_t superTableLen = 0; @@ -466,14 +449,19 @@ static int32_t smlModifyDBSchemas(SSmlHandle *info) { code = catalogGetSTableMeta(info->pCatalog, &conn, &pName, &pTableMeta); if (code == TSDB_CODE_PAR_TABLE_NOT_EXIST || code == TSDB_CODE_MND_STB_NOT_EXIST) { - code = smlSendMetaMsg(info, &pName, sTableData, NULL, SCHEMA_ACTION_NULL); + SArray* pColumns = taosArrayInit(taosArrayGetSize(sTableData->cols), sizeof(SField)); + SArray* pTags = taosArrayInit(taosArrayGetSize(sTableData->tags), sizeof(SField)); + smlBuildFieldsList(info, NULL, NULL, sTableData->tags, pTags, 0, true); + smlBuildFieldsList(info, NULL, NULL, sTableData->cols, pColumns, 0, false); + + code = smlSendMetaMsg(info, &pName, pColumns, pTags, NULL, SCHEMA_ACTION_CREATE_STABLE); if (code != TSDB_CODE_SUCCESS) { uError("SML:0x%" PRIx64 " smlSendMetaMsg failed. can not create %s", info->id, superTable); goto end; } info->cost.numOfCreateSTables++; } else if (code == TSDB_CODE_SUCCESS) { - SHashObj *hashTmp = taosHashInit(pTableMeta->tableInfo.numOfTags, + hashTmp = taosHashInit(pTableMeta->tableInfo.numOfTags, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_NO_LOCK); for (uint16_t i = pTableMeta->tableInfo.numOfColumns; i < pTableMeta->tableInfo.numOfColumns + pTableMeta->tableInfo.numOfTags; i++) { @@ -483,34 +471,70 @@ static int32_t smlModifyDBSchemas(SSmlHandle *info) { ESchemaAction action = SCHEMA_ACTION_NULL; code = smlProcessSchemaAction(info, pTableMeta->schema, hashTmp, sTableData->tags, &action, true); if (code != TSDB_CODE_SUCCESS) { - taosHashCleanup(hashTmp); goto end; } - if (action == SCHEMA_ACTION_TAG){ - code = smlSendMetaMsg(info, &pName, sTableData, pTableMeta, action); + if (action != SCHEMA_ACTION_NULL){ + SArray* pColumns = taosArrayInit(taosArrayGetSize(sTableData->cols) + pTableMeta->tableInfo.numOfColumns, sizeof(SField)); + SArray* pTags = taosArrayInit(taosArrayGetSize(sTableData->tags) + pTableMeta->tableInfo.numOfTags, sizeof(SField)); + + for (uint16_t i = 0; i < pTableMeta->tableInfo.numOfColumns + pTableMeta->tableInfo.numOfTags; i++) { + SField field = {0}; + field.type = pTableMeta->schema[i].type; + field.bytes = pTableMeta->schema[i].bytes; + strcpy(field.name, pTableMeta->schema[i].name); + if(i < pTableMeta->tableInfo.numOfColumns){ + taosArrayPush(pColumns, &field); + }else{ + taosArrayPush(pTags, &field); + } + } + smlBuildFieldsList(info, pTableMeta->schema, hashTmp, sTableData->tags, pTags, pTableMeta->tableInfo.numOfColumns, true); + + code = smlSendMetaMsg(info, &pName, pColumns, pTags, pTableMeta, action); if (code != TSDB_CODE_SUCCESS) { uError("SML:0x%" PRIx64 " smlSendMetaMsg failed. can not create %s", info->id, superTable); goto end; } } + taosMemoryFreeClear(pTableMeta); code = catalogRefreshTableMeta(info->pCatalog, &conn, &pName, -1); if (code != TSDB_CODE_SUCCESS) { goto end; } + code = catalogGetSTableMeta(info->pCatalog, &conn, &pName, &pTableMeta); + if (code != TSDB_CODE_SUCCESS) { + goto end; + } taosHashClear(hashTmp); - for (uint16_t i = 1; i < pTableMeta->tableInfo.numOfColumns; i++) { + for (uint16_t i = 0; i < pTableMeta->tableInfo.numOfColumns; i++) { taosHashPut(hashTmp, pTableMeta->schema[i].name, strlen(pTableMeta->schema[i].name), &i, SHORT_BYTES); } action = SCHEMA_ACTION_NULL; code = smlProcessSchemaAction(info, pTableMeta->schema, hashTmp, sTableData->cols, &action, false); - taosHashCleanup(hashTmp); if (code != TSDB_CODE_SUCCESS) { goto end; } - if (action == SCHEMA_ACTION_COLUMN){ - code = smlSendMetaMsg(info, &pName, sTableData, pTableMeta, action); + if (action != SCHEMA_ACTION_NULL){ + SArray* pColumns = taosArrayInit(taosArrayGetSize(sTableData->cols) + pTableMeta->tableInfo.numOfColumns, sizeof(SField)); + SArray* pTags = taosArrayInit(taosArrayGetSize(sTableData->tags) + pTableMeta->tableInfo.numOfTags, sizeof(SField)); + + for (uint16_t i = 0; i < pTableMeta->tableInfo.numOfColumns + pTableMeta->tableInfo.numOfTags; i++) { + SField field = {0}; + field.type = pTableMeta->schema[i].type; + field.bytes = pTableMeta->schema[i].bytes; + strcpy(field.name, pTableMeta->schema[i].name); + if(i < pTableMeta->tableInfo.numOfColumns){ + taosArrayPush(pColumns, &field); + }else{ + taosArrayPush(pTags, &field); + } + } + + smlBuildFieldsList(info, pTableMeta->schema, hashTmp, sTableData->cols, pColumns, pTableMeta->tableInfo.numOfColumns, false); + + code = smlSendMetaMsg(info, &pName, pColumns, pTags, pTableMeta, action); if (code != TSDB_CODE_SUCCESS) { uError("SML:0x%" PRIx64 " smlSendMetaMsg failed. can not create %s", info->id, superTable); goto end; @@ -526,7 +550,7 @@ static int32_t smlModifyDBSchemas(SSmlHandle *info) { uError("SML:0x%" PRIx64 " load table meta error: %s", info->id, tstrerror(code)); goto end; } - if (pTableMeta) taosMemoryFree(pTableMeta); + taosMemoryFreeClear(pTableMeta); code = catalogGetSTableMeta(info->pCatalog, &conn, &pName, &pTableMeta); if (code != TSDB_CODE_SUCCESS) { @@ -551,10 +575,13 @@ static int32_t smlModifyDBSchemas(SSmlHandle *info) { sTableData->tableMeta = pTableMeta; tableMetaSml = (SSmlSTableMeta **)taosHashIterate(info->superTables, tableMetaSml); + taosHashCleanup(hashTmp); } return 0; end: + taosHashCleanup(hashTmp); + taosMemoryFreeClear(pTableMeta); catalogRefreshTableMeta(info->pCatalog, &conn, &pName, 1); return code; } @@ -2057,10 +2084,6 @@ static int32_t smlParseInfluxLine(SSmlHandle *info, const char *sql) { if (info->dataFormat) taosArrayDestroy(cols); return ret; } - if (taosArrayGetSize(cols) > TSDB_MAX_COLUMNS) { - smlBuildInvalidDataMsg(&info->msgBuf, "too many columns than 4096", NULL); - return TSDB_CODE_PAR_TOO_MANY_COLUMNS; - } bool hasTable = true; SSmlTableInfo *tinfo = NULL; @@ -2094,6 +2117,11 @@ static int32_t smlParseInfluxLine(SSmlHandle *info, const char *sql) { return TSDB_CODE_PAR_INVALID_TAGS_NUM; } + if (taosArrayGetSize(cols) + taosArrayGetSize((*oneTable)->tags) > TSDB_MAX_COLUMNS) { + smlBuildInvalidDataMsg(&info->msgBuf, "too many columns than 4096", NULL); + return TSDB_CODE_PAR_TOO_MANY_COLUMNS; + } + (*oneTable)->sTableName = elements.measure; (*oneTable)->sTableNameLen = elements.measureLen; if (strlen((*oneTable)->childTableName) == 0) { diff --git a/source/client/src/taosx.c b/source/client/src/taosx.c index 677567e38f..f016120a1f 100644 --- a/source/client/src/taosx.c +++ b/source/client/src/taosx.c @@ -765,6 +765,29 @@ static int32_t taosCreateTable(TAOS* taos, void* meta, int32_t metaLen) { } taosArrayPush(pRequest->tableList, &pName); + // change tag cid to new cid + if(pCreateReq->type == TSDB_CHILD_TABLE){ + STableMeta* pTableMeta = NULL; + SName sName = {0}; + toName(pTscObj->acctId, pRequest->pDb, pCreateReq->ctb.name, &sName); + code = catalogGetTableMeta(pCatalog, &conn, &sName, &pTableMeta); + if(code != TSDB_CODE_SUCCESS){ + uError("taosCreateTable:catalogGetTableMeta failed. table name: %s", pCreateReq->ctb.name); + goto end; + } + + for(int32_t i = 0; i < taosArrayGetSize(pCreateReq->ctb.tagName); i++){ + char* tName = taosArrayGet(pCreateReq->ctb.tagName, i); + for(int32_t j = pTableMeta->tableInfo.numOfColumns; j < pTableMeta->tableInfo.numOfColumns + pTableMeta->tableInfo.numOfTags; j++){ + SSchema *tag = &pTableMeta->schema[j]; + if(strcmp(tag->name, tName) == 0 && tag->type != TSDB_DATA_TYPE_JSON){ + tTagSetCid((STag *)pCreateReq->ctb.pTag, i, tag->colId); + } + } + } + taosMemoryFreeClear(pTableMeta); + } + SVgroupCreateTableBatch* pTableBatch = taosHashGet(pVgroupHashmap, &pInfo.vgId, sizeof(pInfo.vgId)); if (pTableBatch == NULL) { SVgroupCreateTableBatch tBatch = {0}; @@ -1305,6 +1328,7 @@ static int32_t tmqWriteRaw(TAOS* taos, void* data, int32_t dataLen) { SQuery* pQuery = NULL; SMqRspObj rspObj = {0}; SDecoder decoder = {0}; + STableMeta* pTableMeta = NULL; terrno = TSDB_CODE_SUCCESS; SRequestObj* pRequest = (SRequestObj*)createRequest(*(int64_t*)taos, TSDB_SQL_INSERT); @@ -1361,24 +1385,6 @@ static int32_t tmqWriteRaw(TAOS* taos, void* data, int32_t dataLen) { goto end; } - uint16_t fLen = 0; - int32_t rowSize = 0; - int16_t nVar = 0; - for (int i = 0; i < pSW->nCols; i++) { - SSchema* schema = pSW->pSchema + i; - fLen += TYPE_BYTES[schema->type]; - rowSize += schema->bytes; - if (IS_VAR_DATA_TYPE(schema->type)) { - nVar++; - } - } - - int32_t rows = rspObj.resInfo.numOfRows; - int32_t extendedRowSize = rowSize + TD_ROW_HEAD_LEN - sizeof(TSKEY) + nVar * sizeof(VarDataOffsetT) + - (int32_t)TD_BITMAP_BYTES(pSW->nCols - 1); - int32_t schemaLen = 0; - int32_t submitLen = sizeof(SSubmitBlk) + schemaLen + rows * extendedRowSize; - const char* tbName = (const char*)taosArrayGetP(rspObj.rsp.blockTbName, rspObj.resIter); if (!tbName) { uError("WriteRaw: tbname is null"); @@ -1398,6 +1404,35 @@ static int32_t tmqWriteRaw(TAOS* taos, void* data, int32_t dataLen) { goto end; } + code = catalogGetTableMeta(pCatalog, &conn, &pName, &pTableMeta); + if (code == TSDB_CODE_PAR_TABLE_NOT_EXIST){ + uError("WriteRaw:catalogGetTableMeta table not exist. table name: %s", tbName); + code = TSDB_CODE_SUCCESS; + continue; + } + if (code != TSDB_CODE_SUCCESS) { + uError("WriteRaw:catalogGetTableMeta failed. table name: %s", tbName); + goto end; + } + + uint16_t fLen = 0; + int32_t rowSize = 0; + int16_t nVar = 0; + for (int i = 0; i < pTableMeta->tableInfo.numOfColumns; i++) { + SSchema* schema = &pTableMeta->schema[i]; + fLen += TYPE_BYTES[schema->type]; + rowSize += schema->bytes; + if (IS_VAR_DATA_TYPE(schema->type)) { + nVar++; + } + } + + int32_t rows = rspObj.resInfo.numOfRows; + int32_t extendedRowSize = rowSize + TD_ROW_HEAD_LEN - sizeof(TSKEY) + nVar * sizeof(VarDataOffsetT) + + (int32_t)TD_BITMAP_BYTES(pTableMeta->tableInfo.numOfColumns - 1); + int32_t schemaLen = 0; + int32_t submitLen = sizeof(SSubmitBlk) + schemaLen + rows * extendedRowSize; + SSubmitReq* subReq = NULL; SSubmitBlk* blk = NULL; void* hData = taosHashGet(pVgHash, &vgData.vg.vgId, sizeof(vgData.vg.vgId)); @@ -1430,23 +1465,25 @@ static int32_t tmqWriteRaw(TAOS* taos, void* data, int32_t dataLen) { blk = POINTER_SHIFT(vgData.data, sizeof(SSubmitReq)); } - STableMeta* pTableMeta = NULL; - code = catalogGetTableMeta(pCatalog, &conn, &pName, &pTableMeta); - if (code != TSDB_CODE_SUCCESS) { - uError("WriteRaw:catalogGetTableMeta failed. table name: %s", tbName); - goto end; - } + // pSW->pSchema should be same as pTableMeta->schema +// ASSERT(pSW->nCols == pTableMeta->tableInfo.numOfColumns); uint64_t suid = (TSDB_NORMAL_TABLE == pTableMeta->tableType ? 0 : pTableMeta->suid); uint64_t uid = pTableMeta->uid; - taosMemoryFreeClear(pTableMeta); + int16_t sver = pTableMeta->sversion; void* blkSchema = POINTER_SHIFT(blk, sizeof(SSubmitBlk)); STSRow* rowData = POINTER_SHIFT(blkSchema, schemaLen); SRowBuilder rb = {0}; - tdSRowInit(&rb, pSW->version); - tdSRowSetTpInfo(&rb, pSW->nCols, fLen); - int32_t dataLen = 0; + tdSRowInit(&rb, sver); + tdSRowSetTpInfo(&rb, pTableMeta->tableInfo.numOfColumns, fLen); + int32_t totalLen = 0; + + SHashObj* schemaHash = taosHashInit(16, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), false, HASH_NO_LOCK); + for (int i = 0; i < pSW->nCols; i++) { + SSchema* schema = &pSW->pSchema[i]; + taosHashPut(schemaHash, schema->name, strlen(schema->name), &i, sizeof(int32_t)); + } for (int32_t j = 0; j < rows; j++) { tdSRowResetBuf(&rb, rowData); @@ -1455,33 +1492,41 @@ static int32_t tmqWriteRaw(TAOS* taos, void* data, int32_t dataLen) { rspObj.resInfo.current += 1; int32_t offset = 0; - for (int32_t k = 0; k < pSW->nCols; k++) { - const SSchema* pColumn = &pSW->pSchema[k]; - char* data = rspObj.resInfo.row[k]; - if (!data) { + for (int32_t k = 0; k < pTableMeta->tableInfo.numOfColumns; k++) { + const SSchema* pColumn = &pTableMeta->schema[k]; + int32_t* index = taosHashGet(schemaHash, pColumn->name, strlen(pColumn->name)); + if(!index){ tdAppendColValToRow(&rb, pColumn->colId, pColumn->type, TD_VTYPE_NULL, NULL, false, offset, k); - } else { - if (IS_VAR_DATA_TYPE(pColumn->type)) { - data -= VARSTR_HEADER_SIZE; + }else{ + char* colData = rspObj.resInfo.row[*index]; + if (!colData) { + tdAppendColValToRow(&rb, pColumn->colId, pColumn->type, TD_VTYPE_NULL, NULL, false, offset, k); + } else { + if (IS_VAR_DATA_TYPE(pColumn->type)) { + colData -= VARSTR_HEADER_SIZE; + } + tdAppendColValToRow(&rb, pColumn->colId, pColumn->type, TD_VTYPE_NORM, colData, true, offset, k); } - tdAppendColValToRow(&rb, pColumn->colId, pColumn->type, TD_VTYPE_NORM, data, true, offset, k); } + offset += TYPE_BYTES[pColumn->type]; } tdSRowEnd(&rb); int32_t rowLen = TD_ROW_LEN(rowData); rowData = POINTER_SHIFT(rowData, rowLen); - dataLen += rowLen; + totalLen += rowLen; } + taosHashCleanup(schemaHash); blk->uid = htobe64(uid); blk->suid = htobe64(suid); - blk->sversion = htonl(pSW->version); + blk->sversion = htonl(sver); blk->schemaLen = htonl(schemaLen); blk->numOfRows = htonl(rows); - blk->dataLen = htonl(dataLen); - subReq->length += sizeof(SSubmitBlk) + schemaLen + dataLen; + blk->dataLen = htonl(totalLen); + subReq->length += sizeof(SSubmitBlk) + schemaLen + totalLen; subReq->numOfBlocks++; + taosMemoryFreeClear(pTableMeta); } pQuery = (SQuery*)nodesMakeNode(QUERY_NODE_QUERY); @@ -1535,6 +1580,7 @@ end: qDestroyQuery(pQuery); destroyRequest(pRequest); taosHashCleanup(pVgHash); + taosMemoryFreeClear(pTableMeta); return code; } diff --git a/source/client/src/tmq.c b/source/client/src/tmq.c index 7637ffbc80..fa657fcb10 100644 --- a/source/client/src/tmq.c +++ b/source/client/src/tmq.c @@ -1132,7 +1132,10 @@ int32_t tmqPollCb(void* param, SDataBuf* pMsg, int32_t code) { memcpy(&pRspWrapper->dataRsp, pMsg->pData, sizeof(SMqRspHead)); } else { ASSERT(rspType == TMQ_MSG_TYPE__POLL_META_RSP); - tDecodeSMqMetaRsp(POINTER_SHIFT(pMsg->pData, sizeof(SMqRspHead)), &pRspWrapper->metaRsp); + SDecoder decoder; + tDecoderInit(&decoder, POINTER_SHIFT(pMsg->pData, sizeof(SMqRspHead)), pMsg->len - sizeof(SMqRspHead)); + tDecodeSMqMetaRsp(&decoder, &pRspWrapper->metaRsp); + tDecoderClear(&decoder); memcpy(&pRspWrapper->metaRsp, pMsg->pData, sizeof(SMqRspHead)); } @@ -1581,8 +1584,7 @@ void* tmqHandleAllRsp(tmq_t* tmq, int64_t timeout, bool pollIfReset) { SMqClientVg* pVg = pollRspWrapper->vgHandle; /*printf("vgId:%d, offset %" PRId64 " up to %" PRId64 "\n", pVg->vgId, pVg->currentOffset, * rspMsg->msg.rspOffset);*/ - pVg->currentOffset.version = pollRspWrapper->metaRsp.rspOffset; - pVg->currentOffset.type = TMQ_OFFSET__LOG; + pVg->currentOffset = pollRspWrapper->metaRsp.rspOffset; atomic_store_32(&pVg->vgStatus, TMQ_VG_STATUS__IDLE); // build rsp SMqMetaRspObj* pRsp = tmqBuildMetaRspFromWrapper(pollRspWrapper); diff --git a/source/client/test/smlTest.cpp b/source/client/test/smlTest.cpp index 68a8b9d336..b62238ccf2 100644 --- a/source/client/test/smlTest.cpp +++ b/source/client/test/smlTest.cpp @@ -692,3 +692,52 @@ TEST(testCase, smlParseTelnetLine_diff_json_type2_Test) { ASSERT_NE(ret, 0); smlDestroyInfo(info); } + +TEST(testCase, sml_col_4096_Test) { + SSmlHandle *info = smlBuildSmlInfo(NULL, NULL, TSDB_SML_LINE_PROTOCOL, TSDB_SML_TIMESTAMP_NANO_SECONDS); + ASSERT_NE(info, nullptr); + + const char *sql[] = { + "spgwgvldxv,id=spgwgvldxv_1,t0=f c0=t,c1=t,c2=t,c3=t,c4=t,c5=t,c6=t,c7=t,c8=t,c9=t,c10=t,c11=t,c12=t,c13=t,c14=t,c15=t,c16=t,c17=t,c18=t,c19=t,c20=t,c21=t,c22=t,c23=t,c24=t,c25=t,c26=t,c27=t,c28=t,c29=t,c30=t,c31=t,c32=t,c33=t,c34=t,c35=t,c36=t,c37=t,c38=t,c39=t,c40=t,c41=t,c42=t,c43=t,c44=t,c45=t,c46=t,c47=t,c48=t,c49=t,c50=t,c51=t,c52=t,c53=t,c54=t,c55=t,c56=t,c57=t,c58=t,c59=t,c60=t,c61=t,c62=t,c63=t,c64=t,c65=t,c66=t,c67=t,c68=t,c69=t,c70=t,c71=t,c72=t,c73=t,c74=t,c75=t,c76=t,c77=t,c78=t,c79=t,c80=t,c81=t,c82=t,c83=t,c84=t,c85=t,c86=t,c87=t,c88=t,c89=t,c90=t,c91=t,c92=t,c93=t,c94=t,c95=t,c96=t,c97=t,c98=t,c99=t,c100=t," + "c101=t,c102=t,c103=t,c104=t,c105=t,c106=t,c107=t,c108=t,c109=t,c110=t,c111=t,c112=t,c113=t,c114=t,c115=t,c116=t,c117=t,c118=t,c119=t,c120=t,c121=t,c122=t,c123=t,c124=t,c125=t,c126=t,c127=t,c128=t,c129=t,c130=t,c131=t,c132=t,c133=t,c134=t,c135=t,c136=t,c137=t,c138=t,c139=t,c140=t,c141=t,c142=t,c143=t,c144=t,c145=t,c146=t,c147=t,c148=t,c149=t,c150=t,c151=t,c152=t,c153=t,c154=t,c155=t,c156=t,c157=t,c158=t,c159=t,c160=t,c161=t,c162=t,c163=t,c164=t,c165=t,c166=t,c167=t,c168=t,c169=t,c170=t,c171=t,c172=t,c173=t,c174=t,c175=t,c176=t,c177=t,c178=t,c179=t,c180=t,c181=t,c182=t,c183=t,c184=t,c185=t,c186=t,c187=t,c188=t,c189=t," + "c190=t,c191=t,c192=t,c193=t,c194=t,c195=t,c196=t,c197=t,c198=t,c199=t,c200=t,c201=t,c202=t,c203=t,c204=t,c205=t,c206=t,c207=t,c208=t,c209=t,c210=t,c211=t,c212=t,c213=t,c214=t,c215=t,c216=t,c217=t,c218=t,c219=t,c220=t,c221=t,c222=t,c223=t,c224=t,c225=t,c226=t,c227=t,c228=t,c229=t,c230=t,c231=t,c232=t,c233=t,c234=t,c235=t,c236=t,c237=t,c238=t,c239=t,c240=t,c241=t,c242=t,c243=t,c244=t,c245=t,c246=t,c247=t,c248=t,c249=t,c250=t,c251=t,c252=t,c253=t,c254=t,c255=t,c256=t,c257=t,c258=t,c259=t,c260=t,c261=t,c262=t,c263=t,c264=t,c265=t,c266=t,c267=t,c268=t,c269=t,c270=t,c271=t,c272=t,c273=t,c274=t,c275=t,c276=t,c277=t,c278=t," + "c279=t,c280=t,c281=t,c282=t,c283=t,c284=t,c285=t,c286=t,c287=t,c288=t,c289=t,c290=t,c291=t,c292=t,c293=t,c294=t,c295=t,c296=t,c297=t,c298=t,c299=t,c300=t,c301=t,c302=t,c303=t,c304=t,c305=t,c306=t,c307=t,c308=t,c309=t,c310=t,c311=t,c312=t,c313=t,c314=t,c315=t,c316=t,c317=t,c318=t,c319=t,c320=t,c321=t,c322=t,c323=t,c324=t,c325=t,c326=t,c327=t,c328=t,c329=t,c330=t,c331=t,c332=t,c333=t,c334=t,c335=t,c336=t,c337=t,c338=t,c339=t,c340=t,c341=t,c342=t,c343=t,c344=t,c345=t,c346=t,c347=t,c348=t,c349=t,c350=t,c351=t,c352=t,c353=t,c354=t,c355=t,c356=t,c357=t,c358=t,c359=t,c360=t,c361=t,c362=t,c363=t,c364=t,c365=t,c366=t,c367=t,c368=t,c369=t,c370=t,c371=t,c372=t,c373=t,c374=t,c375=t,c376=t,c377=t,c378=t,c379=t,c380=t,c381=t,c382=t,c383=t,c384=t,c385=t,c386=t,c387=t,c388=t,c389=t,c390=t,c391=t,c392=t,c393=t,c394=t,c395=t,c396=t,c397=t,c398=t,c399=t,c400=t,c401=t,c402=t,c403=t,c404=t,c405=t,c406=t,c407=t,c408=t,c409=t,c410=t,c411=t,c412=t,c413=t,c414=t,c415=t,c416=t,c417=t,c418=t,c419=t,c420=t,c421=t,c422=t,c423=t,c424=t,c425=t,c426=t,c427=t,c428=t,c429=t,c430=t,c431=t,c432=t,c433=t,c434=t,c435=t,c436=t,c437=t,c438=t,c439=t,c440=t,c441=t,c442=t,c443=t,c444=t,c445=t,c446=t," + "c447=t,c448=t,c449=t,c450=t,c451=t,c452=t,c453=t,c454=t,c455=t,c456=t,c457=t,c458=t,c459=t,c460=t,c461=t,c462=t,c463=t,c464=t,c465=t,c466=t,c467=t,c468=t,c469=t,c470=t,c471=t,c472=t,c473=t,c474=t,c475=t,c476=t,c477=t,c478=t,c479=t,c480=t,c481=t,c482=t,c483=t,c484=t,c485=t,c486=t,c487=t,c488=t,c489=t,c490=t,c491=t,c492=t,c493=t,c494=t,c495=t,c496=t,c497=t,c498=t,c499=t,c500=t,c501=t,c502=t,c503=t,c504=t,c505=t,c506=t,c507=t,c508=t,c509=t,c510=t,c511=t,c512=t,c513=t,c514=t,c515=t,c516=t,c517=t,c518=t,c519=t,c520=t,c521=t,c522=t,c523=t,c524=t,c525=t,c526=t,c527=t,c528=t,c529=t,c530=t,c531=t,c532=t,c533=t,c534=t,c535=t,c536=t,c537=t,c538=t,c539=t,c540=t,c541=t,c542=t,c543=t,c544=t,c545=t,c546=t,c547=t,c548=t,c549=t,c550=t,c551=t,c552=t,c553=t,c554=t,c555=t,c556=t,c557=t,c558=t,c559=t,c560=t,c561=t,c562=t,c563=t,c564=t,c565=t,c566=t,c567=t,c568=t,c569=t,c570=t,c571=t,c572=t,c573=t,c574=t,c575=t,c576=t,c577=t,c578=t,c579=t,c580=t,c581=t,c582=t,c583=t,c584=t,c585=t,c586=t,c587=t,c588=t,c589=t,c590=t,c591=t,c592=t,c593=t,c594=t,c595=t,c596=t,c597=t,c598=t,c599=t,c600=t,c601=t,c602=t,c603=t,c604=t,c605=t,c606=t,c607=t,c608=t,c609=t,c610=t,c611=t,c612=t,c613=t,c614=t," + "c615=t,c616=t,c617=t,c618=t,c619=t,c620=t,c621=t,c622=t,c623=t,c624=t,c625=t,c626=t,c627=t,c628=t,c629=t,c630=t,c631=t,c632=t,c633=t,c634=t,c635=t,c636=t,c637=t,c638=t,c639=t,c640=t,c641=t,c642=t,c643=t,c644=t,c645=t,c646=t,c647=t,c648=t,c649=t,c650=t,c651=t,c652=t,c653=t,c654=t,c655=t,c656=t,c657=t,c658=t,c659=t,c660=t,c661=t,c662=t,c663=t,c664=t,c665=t,c666=t,c667=t,c668=t,c669=t,c670=t,c671=t,c672=t,c673=t,c674=t,c675=t,c676=t,c677=t,c678=t,c679=t,c680=t,c681=t,c682=t,c683=t,c684=t,c685=t,c686=t,c687=t,c688=t,c689=t,c690=t,c691=t,c692=t,c693=t,c694=t,c695=t,c696=t,c697=t,c698=t,c699=t,c700=t,c701=t,c702=t,c703=t,c704=t,c705=t,c706=t,c707=t,c708=t,c709=t,c710=t,c711=t,c712=t,c713=t,c714=t,c715=t,c716=t,c717=t,c718=t,c719=t,c720=t,c721=t,c722=t,c723=t,c724=t,c725=t,c726=t,c727=t,c728=t,c729=t,c730=t,c731=t,c732=t,c733=t,c734=t,c735=t,c736=t,c737=t,c738=t,c739=t,c740=t,c741=t,c742=t,c743=t,c744=t,c745=t,c746=t,c747=t,c748=t,c749=t,c750=t,c751=t,c752=t,c753=t,c754=t,c755=t,c756=t,c757=t,c758=t,c759=t,c760=t,c761=t,c762=t,c763=t,c764=t,c765=t,c766=t,c767=t,c768=t,c769=t,c770=t,c771=t,c772=t,c773=t,c774=t,c775=t,c776=t,c777=t,c778=t,c779=t,c780=t,c781=t,c782=t," + "c783=t,c784=t,c785=t,c786=t,c787=t,c788=t,c789=t,c790=t,c791=t,c792=t,c793=t,c794=t,c795=t,c796=t,c797=t,c798=t,c799=t,c800=t,c801=t,c802=t,c803=t,c804=t,c805=t,c806=t,c807=t,c808=t,c809=t,c810=t,c811=t,c812=t,c813=t," + "c814=t,c815=t,c816=t,c817=t,c818=t,c819=t,c820=t,c821=t,c822=t,c823=t,c824=t,c825=t,c826=t,c827=t,c828=t,c829=t,c830=t,c831=t,c832=t,c833=t,c834=t,c835=t,c836=t,c837=t,c838=t,c839=t,c840=t,c841=t,c842=t,c843=t,c844=t,c845=t,c846=t,c847=t,c848=t,c849=t,c850=t,c851=t,c852=t,c853=t,c854=t,c855=t,c856=t,c857=t,c858=t,c859=t,c860=t,c861=t,c862=t," + "c863=t,c864=t,c865=t,c866=t,c867=t,c868=t,c869=t,c870=t,c871=t,c872=t,c873=t,c874=t,c875=t,c876=t,c877=t,c878=t,c879=t,c880=t,c881=t,c882=t,c883=t,c884=t,c885=t,c886=t,c887=t,c888=t,c889=t,c890=t,c891=t,c892=t,c893=t,c894=t,c895=t,c896=t,c897=t,c898=t,c899=t,c900=t,c901=t,c902=t,c903=t,c904=t,c905=t,c906=t,c907=t,c908=t,c909=t,c910=t,c911=t,c912=t,c913=t,c914=t,c915=t,c916=t,c917=t,c918=t,c919=t,c920=t,c921=t,c922=t,c923=t,c924=t,c925=t,c926=t,c927=t,c928=t,c929=t,c930=t,c931=t,c932=t,c933=t,c934=t,c935=t,c936=t,c937=t,c938=t,c939=t,c940=t,c941=t,c942=t,c943=t,c944=t,c945=t,c946=t,c947=t,c948=t,c949=t,c950=t,c951=t,c952=t,c953=t,c954=t,c955=t,c956=t,c957=t,c958=t,c959=t,c960=t,c961=t,c962=t,c963=t,c964=t,c965=t,c966=t,c967=t,c968=t,c969=t,c970=t,c971=t,c972=t,c973=t,c974=t,c975=t,c976=t,c977=t,c978=t,c979=t,c980=t,c981=t,c982=t,c983=t,c984=t,c985=t,c986=t,c987=t,c988=t,c989=t,c990=t,c991=t,c992=t,c993=t,c994=t,c995=t,c996=t,c997=t,c998=t,c999=t,c1000=t,c1001=t,c1002=t,c1003=t,c1004=t,c1005=t,c1006=t,c1007=t,c1008=t,c1009=t,c1010=t,c1011=t,c1012=t,c1013=t,c1014=t,c1015=t,c1016=t,c1017=t,c1018=t,c1019=t,c1020=t,c1021=t,c1022=t,c1023=t,c1024=t,c1025=t,c1026=t," + "c1027=t,c1028=t,c1029=t,c1030=t,c1031=t,c1032=t,c1033=t,c1034=t,c1035=t,c1036=t,c1037=t,c1038=t,c1039=t,c1040=t,c1041=t,c1042=t,c1043=t,c1044=t,c1045=t,c1046=t,c1047=t,c1048=t,c1049=t,c1050=t,c1051=t,c1052=t,c1053=t,c1054=t,c1055=t,c1056=t,c1057=t,c1058=t,c1059=t,c1060=t,c1061=t,c1062=t,c1063=t,c1064=t,c1065=t,c1066=t,c1067=t,c1068=t,c1069=t,c1070=t,c1071=t,c1072=t,c1073=t,c1074=t,c1075=t,c1076=t,c1077=t,c1078=t,c1079=t,c1080=t,c1081=t,c1082=t,c1083=t,c1084=t,c1085=t,c1086=t,c1087=t,c1088=t,c1089=t,c1090=t,c1091=t,c1092=t,c1093=t,c1094=t,c1095=t,c1096=t,c1097=t,c1098=t,c1099=t,c1100=t,c1101=t,c1102=t,c1103=t,c1104=t,c1105=t,c1106=t,c1107=t,c1108=t,c1109=t,c1110=t,c1111=t,c1112=t,c1113=t,c1114=t,c1115=t,c1116=t,c1117=t,c1118=t,c1119=t,c1120=t,c1121=t,c1122=t,c1123=t,c1124=t,c1125=t,c1126=t,c1127=t,c1128=t,c1129=t,c1130=t,c1131=t,c1132=t,c1133=t,c1134=t,c1135=t,c1136=t,c1137=t,c1138=t,c1139=t,c1140=t,c1141=t,c1142=t,c1143=t,c1144=t,c1145=t,c1146=t,c1147=t,c1148=t,c1149=t,c1150=t,c1151=t,c1152=t,c1153=t,c1154=t,c1155=t,c1156=t,c1157=t,c1158=t,c1159=t,c1160=t,c1161=t,c1162=t,c1163=t,c1164=t,c1165=t,c1166=t,c1167=t,c1168=t,c1169=t,c1170=t,c1171=t,c1172=t,c1173=t," + "c1174=t,c1175=t,c1176=t,c1177=t,c1178=t,c1179=t,c1180=t,c1181=t,c1182=t,c1183=t,c1184=t,c1185=t,c1186=t,c1187=t,c1188=t,c1189=t,c1190=t,c1191=t,c1192=t,c1193=t,c1194=t,c1195=t,c1196=t,c1197=t,c1198=t,c1199=t,c1200=t,c1201=t,c1202=t,c1203=t,c1204=t,c1205=t,c1206=t,c1207=t,c1208=t,c1209=t,c1210=t,c1211=t,c1212=t,c1213=t,c1214=t,c1215=t,c1216=t,c1217=t,c1218=t,c1219=t,c1220=t,c1221=t,c1222=t,c1223=t,c1224=t,c1225=t,c1226=t,c1227=t,c1228=t,c1229=t,c1230=t,c1231=t,c1232=t,c1233=t,c1234=t,c1235=t,c1236=t,c1237=t,c1238=t,c1239=t,c1240=t,c1241=t,c1242=t,c1243=t,c1244=t,c1245=t,c1246=t,c1247=t,c1248=t,c1249=t,c1250=t,c1251=t,c1252=t,c1253=t,c1254=t,c1255=t,c1256=t,c1257=t,c1258=t,c1259=t,c1260=t,c1261=t,c1262=t,c1263=t,c1264=t,c1265=t,c1266=t,c1267=t,c1268=t,c1269=t,c1270=t,c1271=t,c1272=t,c1273=t,c1274=t,c1275=t,c1276=t,c1277=t,c1278=t,c1279=t,c1280=t,c1281=t,c1282=t,c1283=t,c1284=t,c1285=t,c1286=t,c1287=t,c1288=t,c1289=t,c1290=t,c1291=t,c1292=t,c1293=t,c1294=t,c1295=t,c1296=t,c1297=t,c1298=t,c1299=t,c1300=t,c1301=t,c1302=t,c1303=t,c1304=t,c1305=t,c1306=t,c1307=t,c1308=t,c1309=t,c1310=t,c1311=t,c1312=t,c1313=t,c1314=t,c1315=t,c1316=t,c1317=t,c1318=t,c1319=t,c1320=t," + "c1321=t,c1322=t,c1323=t,c1324=t,c1325=t,c1326=t,c1327=t,c1328=t,c1329=t,c1330=t,c1331=t,c1332=t,c1333=t,c1334=t,c1335=t,c1336=t,c1337=t,c1338=t,c1339=t,c1340=t,c1341=t,c1342=t,c1343=t,c1344=t,c1345=t,c1346=t,c1347=t," + "c1348=t,c1349=t,c1350=t,c1351=t,c1352=t,c1353=t,c1354=t,c1355=t,c1356=t,c1357=t,c1358=t,c1359=t,c1360=t,c1361=t,c1362=t,c1363=t,c1364=t,c1365=t,c1366=t,c1367=t,c1368=t,c1369=t,c1370=t,c1371=t,c1372=t,c1373=t,c1374=t,c1375=t,c1376=t,c1377=t,c1378=t,c1379=t,c1380=t,c1381=t,c1382=t,c1383=t,c1384=t,c1385=t,c1386=t,c1387=t,c1388=t,c1389=t,c1390=t,c1391=t,c1392=t,c1393=t,c1394=t,c1395=t,c1396=t,c1397=t,c1398=t,c1399=t,c1400=t,c1401=t,c1402=t,c1403=t,c1404=t,c1405=t,c1406=t,c1407=t,c1408=t,c1409=t,c1410=t,c1411=t,c1412=t,c1413=t,c1414=t,c1415=t,c1416=t,c1417=t,c1418=t,c1419=t,c1420=t,c1421=t,c1422=t,c1423=t,c1424=t,c1425=t,c1426=t,c1427=t,c1428=t,c1429=t,c1430=t,c1431=t,c1432=t,c1433=t,c1434=t,c1435=t,c1436=t,c1437=t,c1438=t,c1439=t,c1440=t,c1441=t,c1442=t,c1443=t,c1444=t,c1445=t,c1446=t,c1447=t,c1448=t,c1449=t,c1450=t,c1451=t,c1452=t,c1453=t,c1454=t,c1455=t,c1456=t,c1457=t,c1458=t,c1459=t,c1460=t,c1461=t,c1462=t,c1463=t,c1464=t,c1465=t,c1466=t,c1467=t,c1468=t,c1469=t,c1470=t,c1471=t,c1472=t,c1473=t,c1474=t,c1475=t,c1476=t,c1477=t,c1478=t,c1479=t,c1480=t,c1481=t,c1482=t,c1483=t,c1484=t,c1485=t,c1486=t,c1487=t,c1488=t,c1489=t,c1490=t,c1491=t,c1492=t,c1493=t,c1494=t," + "c1495=t,c1496=t,c1497=t,c1498=t,c1499=t,c1500=t,c1501=t,c1502=t,c1503=t,c1504=t,c1505=t,c1506=t,c1507=t,c1508=t,c1509=t,c1510=t,c1511=t,c1512=t,c1513=t,c1514=t,c1515=t,c1516=t,c1517=t,c1518=t,c1519=t,c1520=t,c1521=t,c1522=t,c1523=t,c1524=t,c1525=t,c1526=t,c1527=t,c1528=t,c1529=t,c1530=t,c1531=t,c1532=t,c1533=t,c1534=t,c1535=t,c1536=t,c1537=t,c1538=t,c1539=t,c1540=t,c1541=t,c1542=t,c1543=t,c1544=t,c1545=t,c1546=t,c1547=t,c1548=t,c1549=t,c1550=t,c1551=t,c1552=t,c1553=t,c1554=t,c1555=t,c1556=t,c1557=t,c1558=t,c1559=t,c1560=t,c1561=t,c1562=t,c1563=t,c1564=t,c1565=t,c1566=t,c1567=t,c1568=t,c1569=t,c1570=t,c1571=t,c1572=t,c1573=t,c1574=t,c1575=t,c1576=t,c1577=t,c1578=t,c1579=t,c1580=t,c1581=t,c1582=t,c1583=t,c1584=t,c1585=t,c1586=t,c1587=t,c1588=t,c1589=t,c1590=t,c1591=t,c1592=t,c1593=t,c1594=t,c1595=t,c1596=t,c1597=t,c1598=t,c1599=t,c1600=t,c1601=t,c1602=t,c1603=t,c1604=t,c1605=t,c1606=t,c1607=t,c1608=t,c1609=t,c1610=t,c1611=t,c1612=t,c1613=t,c1614=t,c1615=t,c1616=t,c1617=t,c1618=t,c1619=t,c1620=t,c1621=t,c1622=t,c1623=t,c1624=t,c1625=t,c1626=t,c1627=t,c1628=t,c1629=t,c1630=t,c1631=t,c1632=t,c1633=t,c1634=t,c1635=t,c1636=t,c1637=t,c1638=t,c1639=t,c1640=t,c1641=t," + "c1642=t,c1643=t,c1644=t,c1645=t,c1646=t,c1647=t,c1648=t,c1649=t,c1650=t,c1651=t,c1652=t,c1653=t,c1654=t,c1655=t,c1656=t,c1657=t,c1658=t,c1659=t,c1660=t,c1661=t,c1662=t,c1663=t,c1664=t,c1665=t,c1666=t,c1667=t,c1668=t,c1669=t,c1670=t,c1671=t,c1672=t,c1673=t,c1674=t,c1675=t,c1676=t,c1677=t,c1678=t,c1679=t,c1680=t,c1681=t,c1682=t,c1683=t,c1684=t,c1685=t,c1686=t,c1687=t,c1688=t,c1689=t,c1690=t,c1691=t,c1692=t,c1693=t,c1694=t,c1695=t,c1696=t,c1697=t,c1698=t,c1699=t,c1700=t,c1701=t,c1702=t,c1703=t,c1704=t,c1705=t,c1706=t,c1707=t,c1708=t,c1709=t,c1710=t,c1711=t,c1712=t,c1713=t,c1714=t,c1715=t,c1716=t,c1717=t,c1718=t,c1719=t,c1720=t,c1721=t,c1722=t,c1723=t,c1724=t,c1725=t,c1726=t,c1727=t,c1728=t,c1729=t,c1730=t,c1731=t,c1732=t,c1733=t,c1734=t,c1735=t,c1736=t,c1737=t,c1738=t,c1739=t,c1740=t,c1741=t,c1742=t,c1743=t,c1744=t,c1745=t,c1746=t,c1747=t,c1748=t,c1749=t,c1750=t,c1751=t,c1752=t,c1753=t,c1754=t,c1755=t,c1756=t,c1757=t,c1758=t,c1759=t,c1760=t,c1761=t,c1762=t,c1763=t,c1764=t,c1765=t,c1766=t,c1767=t,c1768=t,c1769=t,c1770=t,c1771=t,c1772=t,c1773=t,c1774=t,c1775=t,c1776=t,c1777=t,c1778=t,c1779=t,c1780=t,c1781=t,c1782=t,c1783=t,c1784=t,c1785=t,c1786=t,c1787=t,c1788=t," + "c1789=t,c1790=t,c1791=t,c1792=t,c1793=t,c1794=t,c1795=t,c1796=t,c1797=t,c1798=t,c1799=t,c1800=t,c1801=t,c1802=t,c1803=t,c1804=t,c1805=t,c1806=t,c1807=t,c1808=t,c1809=t,c1810=t,c1811=t,c1812=t,c1813=t,c1814=t,c1815=t," + "c1816=t,c1817=t,c1818=t,c1819=t,c1820=t,c1821=t,c1822=t,c1823=t,c1824=t,c1825=t,c1826=t,c1827=t,c1828=t,c1829=t,c1830=t,c1831=t,c1832=t,c1833=t,c1834=t,c1835=t,c1836=t,c1837=t,c1838=t,c1839=t,c1840=t,c1841=t,c1842=t,c1843=t,c1844=t,c1845=t,c1846=t,c1847=t,c1848=t,c1849=t,c1850=t,c1851=t,c1852=t,c1853=t,c1854=t,c1855=t,c1856=t,c1857=t,c1858=t,c1859=t,c1860=t,c1861=t,c1862=t,c1863=t,c1864=t,c1865=t,c1866=t,c1867=t,c1868=t,c1869=t,c1870=t,c1871=t,c1872=t,c1873=t,c1874=t,c1875=t,c1876=t,c1877=t,c1878=t,c1879=t,c1880=t,c1881=t,c1882=t,c1883=t,c1884=t,c1885=t,c1886=t,c1887=t,c1888=t,c1889=t,c1890=t,c1891=t,c1892=t,c1893=t,c1894=t,c1895=t,c1896=t,c1897=t,c1898=t,c1899=t,c1900=t,c1901=t,c1902=t,c1903=t,c1904=t,c1905=t,c1906=t,c1907=t,c1908=t,c1909=t,c1910=t,c1911=t,c1912=t,c1913=t,c1914=t,c1915=t,c1916=t,c1917=t,c1918=t,c1919=t,c1920=t,c1921=t,c1922=t,c1923=t,c1924=t,c1925=t,c1926=t,c1927=t,c1928=t,c1929=t,c1930=t,c1931=t,c1932=t,c1933=t,c1934=t,c1935=t,c1936=t,c1937=t,c1938=t,c1939=t,c1940=t,c1941=t,c1942=t,c1943=t,c1944=t,c1945=t,c1946=t,c1947=t,c1948=t,c1949=t,c1950=t,c1951=t,c1952=t,c1953=t,c1954=t,c1955=t,c1956=t,c1957=t,c1958=t,c1959=t,c1960=t,c1961=t,c1962=t," + "c1963=t,c1964=t,c1965=t,c1966=t,c1967=t,c1968=t,c1969=t,c1970=t,c1971=t,c1972=t,c1973=t,c1974=t,c1975=t,c1976=t,c1977=t,c1978=t,c1979=t,c1980=t,c1981=t,c1982=t,c1983=t,c1984=t,c1985=t,c1986=t,c1987=t,c1988=t,c1989=t,c1990=t,c1991=t,c1992=t,c1993=t,c1994=t,c1995=t,c1996=t,c1997=t,c1998=t,c1999=t,c2000=t,c2001=t,c2002=t,c2003=t,c2004=t,c2005=t,c2006=t,c2007=t,c2008=t,c2009=t,c2010=t,c2011=t,c2012=t,c2013=t,c2014=t,c2015=t,c2016=t,c2017=t,c2018=t,c2019=t,c2020=t,c2021=t,c2022=t,c2023=t,c2024=t,c2025=t,c2026=t,c2027=t,c2028=t,c2029=t,c2030=t,c2031=t,c2032=t,c2033=t,c2034=t,c2035=t,c2036=t,c2037=t,c2038=t,c2039=t,c2040=t,c2041=t,c2042=t,c2043=t,c2044=t,c2045=t,c2046=t,c2047=t,c2048=t,c2049=t,c2050=t,c2051=t,c2052=t,c2053=t,c2054=t,c2055=t,c2056=t,c2057=t,c2058=t,c2059=t,c2060=t,c2061=t,c2062=t,c2063=t,c2064=t,c2065=t,c2066=t,c2067=t,c2068=t,c2069=t,c2070=t,c2071=t,c2072=t,c2073=t,c2074=t,c2075=t,c2076=t,c2077=t,c2078=t,c2079=t,c2080=t,c2081=t,c2082=t,c2083=t,c2084=t,c2085=t,c2086=t,c2087=t,c2088=t,c2089=t,c2090=t,c2091=t,c2092=t,c2093=t,c2094=t,c2095=t,c2096=t,c2097=t,c2098=t,c2099=t,c2100=t,c2101=t,c2102=t,c2103=t,c2104=t,c2105=t,c2106=t,c2107=t,c2108=t,c2109=t," + "c2110=t,c2111=t,c2112=t,c2113=t,c2114=t,c2115=t,c2116=t,c2117=t,c2118=t,c2119=t,c2120=t,c2121=t,c2122=t,c2123=t,c2124=t,c2125=t,c2126=t,c2127=t,c2128=t,c2129=t,c2130=t,c2131=t,c2132=t,c2133=t,c2134=t,c2135=t,c2136=t,c2137=t,c2138=t,c2139=t,c2140=t,c2141=t,c2142=t,c2143=t,c2144=t,c2145=t,c2146=t,c2147=t,c2148=t,c2149=t,c2150=t,c2151=t,c2152=t,c2153=t,c2154=t,c2155=t,c2156=t,c2157=t,c2158=t,c2159=t,c2160=t,c2161=t,c2162=t,c2163=t,c2164=t,c2165=t,c2166=t,c2167=t,c2168=t,c2169=t,c2170=t,c2171=t,c2172=t,c2173=t,c2174=t,c2175=t,c2176=t,c2177=t,c2178=t,c2179=t,c2180=t,c2181=t,c2182=t,c2183=t,c2184=t,c2185=t,c2186=t,c2187=t,c2188=t,c2189=t,c2190=t,c2191=t,c2192=t,c2193=t,c2194=t,c2195=t,c2196=t,c2197=t,c2198=t,c2199=t,c2200=t,c2201=t,c2202=t,c2203=t,c2204=t,c2205=t,c2206=t,c2207=t,c2208=t,c2209=t,c2210=t,c2211=t,c2212=t,c2213=t,c2214=t,c2215=t,c2216=t,c2217=t,c2218=t,c2219=t,c2220=t,c2221=t,c2222=t,c2223=t,c2224=t,c2225=t,c2226=t,c2227=t,c2228=t,c2229=t,c2230=t,c2231=t,c2232=t,c2233=t,c2234=t,c2235=t,c2236=t,c2237=t,c2238=t,c2239=t,c2240=t,c2241=t,c2242=t,c2243=t,c2244=t,c2245=t,c2246=t,c2247=t,c2248=t,c2249=t,c2250=t,c2251=t,c2252=t,c2253=t,c2254=t,c2255=t,c2256=t," + "c2257=t,c2258=t,c2259=t,c2260=t,c2261=t,c2262=t,c2263=t,c2264=t,c2265=t,c2266=t,c2267=t,c2268=t,c2269=t,c2270=t,c2271=t,c2272=t,c2273=t,c2274=t,c2275=t,c2276=t,c2277=t,c2278=t,c2279=t,c2280=t,c2281=t,c2282=t,c2283=t," + "c2284=t,c2285=t,c2286=t,c2287=t,c2288=t,c2289=t,c2290=t,c2291=t,c2292=t,c2293=t,c2294=t,c2295=t,c2296=t,c2297=t,c2298=t,c2299=t,c2300=t,c2301=t,c2302=t,c2303=t,c2304=t,c2305=t,c2306=t,c2307=t,c2308=t,c2309=t,c2310=t,c2311=t,c2312=t,c2313=t,c2314=t,c2315=t,c2316=t,c2317=t,c2318=t,c2319=t,c2320=t,c2321=t,c2322=t,c2323=t,c2324=t,c2325=t,c2326=t,c2327=t,c2328=t,c2329=t,c2330=t,c2331=t,c2332=t,c2333=t,c2334=t,c2335=t,c2336=t,c2337=t,c2338=t,c2339=t,c2340=t,c2341=t,c2342=t,c2343=t,c2344=t,c2345=t,c2346=t,c2347=t,c2348=t,c2349=t,c2350=t,c2351=t,c2352=t,c2353=t,c2354=t,c2355=t,c2356=t,c2357=t,c2358=t,c2359=t,c2360=t,c2361=t,c2362=t,c2363=t,c2364=t,c2365=t,c2366=t,c2367=t,c2368=t,c2369=t,c2370=t,c2371=t,c2372=t,c2373=t,c2374=t,c2375=t,c2376=t,c2377=t,c2378=t,c2379=t,c2380=t,c2381=t,c2382=t,c2383=t,c2384=t,c2385=t,c2386=t,c2387=t,c2388=t,c2389=t,c2390=t,c2391=t,c2392=t,c2393=t,c2394=t,c2395=t,c2396=t,c2397=t,c2398=t,c2399=t,c2400=t,c2401=t,c2402=t,c2403=t,c2404=t,c2405=t,c2406=t,c2407=t,c2408=t,c2409=t,c2410=t,c2411=t,c2412=t,c2413=t,c2414=t,c2415=t,c2416=t,c2417=t,c2418=t,c2419=t,c2420=t,c2421=t,c2422=t,c2423=t,c2424=t,c2425=t,c2426=t,c2427=t,c2428=t,c2429=t,c2430=t," + "c2431=t,c2432=t,c2433=t,c2434=t,c2435=t,c2436=t,c2437=t,c2438=t,c2439=t,c2440=t,c2441=t,c2442=t,c2443=t,c2444=t,c2445=t,c2446=t,c2447=t,c2448=t,c2449=t,c2450=t,c2451=t,c2452=t,c2453=t,c2454=t,c2455=t,c2456=t,c2457=t,c2458=t,c2459=t,c2460=t,c2461=t,c2462=t,c2463=t,c2464=t,c2465=t,c2466=t,c2467=t,c2468=t,c2469=t,c2470=t,c2471=t,c2472=t,c2473=t,c2474=t,c2475=t,c2476=t,c2477=t,c2478=t,c2479=t,c2480=t,c2481=t,c2482=t,c2483=t,c2484=t,c2485=t,c2486=t,c2487=t,c2488=t,c2489=t,c2490=t,c2491=t,c2492=t,c2493=t,c2494=t,c2495=t,c2496=t,c2497=t,c2498=t,c2499=t,c2500=t,c2501=t,c2502=t,c2503=t,c2504=t,c2505=t,c2506=t,c2507=t,c2508=t,c2509=t,c2510=t,c2511=t,c2512=t,c2513=t,c2514=t,c2515=t,c2516=t,c2517=t,c2518=t,c2519=t,c2520=t,c2521=t,c2522=t,c2523=t,c2524=t,c2525=t,c2526=t,c2527=t,c2528=t,c2529=t,c2530=t,c2531=t,c2532=t,c2533=t,c2534=t,c2535=t,c2536=t,c2537=t,c2538=t,c2539=t,c2540=t,c2541=t,c2542=t,c2543=t,c2544=t,c2545=t,c2546=t,c2547=t,c2548=t,c2549=t,c2550=t,c2551=t,c2552=t,c2553=t,c2554=t,c2555=t,c2556=t,c2557=t,c2558=t,c2559=t,c2560=t,c2561=t,c2562=t,c2563=t,c2564=t,c2565=t,c2566=t,c2567=t,c2568=t,c2569=t,c2570=t,c2571=t,c2572=t,c2573=t,c2574=t,c2575=t,c2576=t,c2577=t," + "c2578=t,c2579=t,c2580=t,c2581=t,c2582=t,c2583=t,c2584=t,c2585=t,c2586=t,c2587=t,c2588=t,c2589=t,c2590=t,c2591=t,c2592=t,c2593=t,c2594=t,c2595=t,c2596=t,c2597=t,c2598=t,c2599=t,c2600=t,c2601=t,c2602=t,c2603=t,c2604=t,c2605=t,c2606=t,c2607=t,c2608=t,c2609=t,c2610=t,c2611=t,c2612=t,c2613=t,c2614=t,c2615=t,c2616=t,c2617=t,c2618=t,c2619=t,c2620=t,c2621=t,c2622=t,c2623=t,c2624=t,c2625=t,c2626=t,c2627=t,c2628=t,c2629=t,c2630=t,c2631=t,c2632=t,c2633=t,c2634=t,c2635=t,c2636=t,c2637=t,c2638=t,c2639=t,c2640=t,c2641=t,c2642=t,c2643=t,c2644=t,c2645=t,c2646=t,c2647=t,c2648=t,c2649=t,c2650=t,c2651=t,c2652=t,c2653=t,c2654=t,c2655=t,c2656=t,c2657=t,c2658=t,c2659=t,c2660=t,c2661=t,c2662=t,c2663=t,c2664=t,c2665=t,c2666=t,c2667=t,c2668=t,c2669=t,c2670=t,c2671=t,c2672=t,c2673=t,c2674=t,c2675=t,c2676=t,c2677=t,c2678=t,c2679=t,c2680=t,c2681=t,c2682=t,c2683=t,c2684=t,c2685=t,c2686=t,c2687=t,c2688=t,c2689=t,c2690=t,c2691=t,c2692=t,c2693=t,c2694=t,c2695=t,c2696=t,c2697=t,c2698=t,c2699=t,c2700=t,c2701=t,c2702=t,c2703=t,c2704=t,c2705=t,c2706=t,c2707=t,c2708=t,c2709=t,c2710=t,c2711=t,c2712=t,c2713=t,c2714=t,c2715=t,c2716=t,c2717=t,c2718=t,c2719=t,c2720=t,c2721=t,c2722=t,c2723=t,c2724=t," + "c2725=t,c2726=t,c2727=t,c2728=t,c2729=t,c2730=t,c2731=t,c2732=t,c2733=t,c2734=t,c2735=t,c2736=t,c2737=t,c2738=t,c2739=t,c2740=t,c2741=t,c2742=t,c2743=t,c2744=t,c2745=t,c2746=t,c2747=t,c2748=t,c2749=t,c2750=t,c2751=t,c2752=t,c2753=t,c2754=t,c2755=t,c2756=t,c2757=t,c2758=t,c2759=t,c2760=t,c2761=t,c2762=t,c2763=t,c2764=t,c2765=t,c2766=t,c2767=t,c2768=t,c2769=t,c2770=t,c2771=t,c2772=t,c2773=t,c2774=t,c2775=t,c2776=t,c2777=t,c2778=t,c2779=t,c2780=t,c2781=t,c2782=t,c2783=t,c2784=t,c2785=t,c2786=t,c2787=t,c2788=t,c2789=t,c2790=t,c2791=t,c2792=t,c2793=t,c2794=t,c2795=t,c2796=t,c2797=t,c2798=t,c2799=t,c2800=t,c2801=t,c2802=t,c2803=t,c2804=t,c2805=t,c2806=t,c2807=t,c2808=t,c2809=t,c2810=t,c2811=t,c2812=t,c2813=t,c2814=t,c2815=t,c2816=t,c2817=t,c2818=t,c2819=t,c2820=t,c2821=t,c2822=t,c2823=t,c2824=t,c2825=t,c2826=t,c2827=t,c2828=t,c2829=t,c2830=t,c2831=t,c2832=t,c2833=t,c2834=t,c2835=t,c2836=t,c2837=t,c2838=t,c2839=t,c2840=t,c2841=t,c2842=t,c2843=t,c2844=t,c2845=t,c2846=t,c2847=t,c2848=t,c2849=t,c2850=t,c2851=t,c2852=t,c2853=t,c2854=t,c2855=t,c2856=t,c2857=t,c2858=t,c2859=t,c2860=t,c2861=t,c2862=t,c2863=t,c2864=t,c2865=t,c2866=t,c2867=t,c2868=t,c2869=t,c2870=t,c2871=t," + "c2872=t,c2873=t,c2874=t,c2875=t,c2876=t,c2877=t,c2878=t,c2879=t,c2880=t,c2881=t,c2882=t,c2883=t,c2884=t,c2885=t,c2886=t,c2887=t,c2888=t,c2889=t,c2890=t,c2891=t,c2892=t,c2893=t,c2894=t,c2895=t,c2896=t,c2897=t,c2898=t,c2899=t,c2900=t,c2901=t,c2902=t,c2903=t,c2904=t,c2905=t,c2906=t,c2907=t,c2908=t,c2909=t,c2910=t,c2911=t,c2912=t,c2913=t,c2914=t,c2915=t,c2916=t,c2917=t,c2918=t,c2919=t,c2920=t,c2921=t,c2922=t,c2923=t,c2924=t,c2925=t,c2926=t,c2927=t,c2928=t,c2929=t,c2930=t,c2931=t,c2932=t,c2933=t,c2934=t,c2935=t,c2936=t,c2937=t,c2938=t,c2939=t,c2940=t,c2941=t,c2942=t,c2943=t,c2944=t,c2945=t,c2946=t,c2947=t,c2948=t,c2949=t,c2950=t,c2951=t,c2952=t,c2953=t,c2954=t,c2955=t,c2956=t,c2957=t,c2958=t,c2959=t,c2960=t,c2961=t,c2962=t,c2963=t,c2964=t,c2965=t,c2966=t,c2967=t,c2968=t,c2969=t,c2970=t,c2971=t,c2972=t,c2973=t,c2974=t,c2975=t,c2976=t,c2977=t,c2978=t,c2979=t,c2980=t,c2981=t,c2982=t,c2983=t,c2984=t,c2985=t,c2986=t,c2987=t,c2988=t,c2989=t,c2990=t,c2991=t,c2992=t,c2993=t,c2994=t,c2995=t,c2996=t,c2997=t,c2998=t,c2999=t,c3000=t,c3001=t,c3002=t,c3003=t,c3004=t,c3005=t,c3006=t,c3007=t,c3008=t,c3009=t,c3010=t,c3011=t,c3012=t,c3013=t,c3014=t,c3015=t,c3016=t,c3017=t,c3018=t," + "c3019=t,c3020=t,c3021=t,c3022=t,c3023=t,c3024=t,c3025=t,c3026=t,c3027=t,c3028=t,c3029=t,c3030=t,c3031=t,c3032=t,c3033=t,c3034=t,c3035=t,c3036=t,c3037=t,c3038=t,c3039=t,c3040=t,c3041=t,c3042=t,c3043=t,c3044=t,c3045=t,c3046=t,c3047=t,c3048=t,c3049=t,c3050=t,c3051=t,c3052=t,c3053=t,c3054=t,c3055=t,c3056=t,c3057=t,c3058=t,c3059=t,c3060=t,c3061=t,c3062=t,c3063=t,c3064=t,c3065=t,c3066=t,c3067=t,c3068=t,c3069=t,c3070=t,c3071=t,c3072=t,c3073=t,c3074=t,c3075=t,c3076=t,c3077=t,c3078=t,c3079=t,c3080=t,c3081=t,c3082=t,c3083=t,c3084=t,c3085=t,c3086=t,c3087=t,c3088=t,c3089=t,c3090=t,c3091=t,c3092=t,c3093=t,c3094=t,c3095=t,c3096=t,c3097=t,c3098=t,c3099=t,c3100=t,c3101=t,c3102=t,c3103=t,c3104=t,c3105=t,c3106=t,c3107=t,c3108=t,c3109=t,c3110=t,c3111=t,c3112=t,c3113=t,c3114=t,c3115=t,c3116=t,c3117=t,c3118=t,c3119=t,c3120=t,c3121=t,c3122=t,c3123=t,c3124=t,c3125=t,c3126=t,c3127=t,c3128=t,c3129=t,c3130=t,c3131=t,c3132=t,c3133=t,c3134=t,c3135=t,c3136=t,c3137=t,c3138=t,c3139=t,c3140=t,c3141=t,c3142=t,c3143=t,c3144=t,c3145=t,c3146=t,c3147=t,c3148=t,c3149=t,c3150=t,c3151=t,c3152=t,c3153=t,c3154=t,c3155=t,c3156=t,c3157=t,c3158=t,c3159=t,c3160=t,c3161=t,c3162=t,c3163=t,c3164=t,c3165=t," + "c3166=t,c3167=t,c3168=t,c3169=t,c3170=t,c3171=t,c3172=t,c3173=t,c3174=t,c3175=t,c3176=t,c3177=t,c3178=t,c3179=t,c3180=t,c3181=t,c3182=t,c3183=t,c3184=t,c3185=t,c3186=t,c3187=t,c3188=t,c3189=t,c3190=t,c3191=t,c3192=t,c3193=t,c3194=t,c3195=t,c3196=t,c3197=t,c3198=t,c3199=t,c3200=t,c3201=t,c3202=t,c3203=t,c3204=t,c3205=t,c3206=t,c3207=t,c3208=t,c3209=t,c3210=t,c3211=t,c3212=t,c3213=t,c3214=t,c3215=t,c3216=t,c3217=t,c3218=t,c3219=t,c3220=t,c3221=t,c3222=t,c3223=t,c3224=t,c3225=t,c3226=t,c3227=t,c3228=t,c3229=t,c3230=t,c3231=t,c3232=t,c3233=t,c3234=t,c3235=t,c3236=t,c3237=t,c3238=t,c3239=t,c3240=t,c3241=t,c3242=t,c3243=t,c3244=t,c3245=t,c3246=t,c3247=t,c3248=t,c3249=t,c3250=t,c3251=t,c3252=t,c3253=t,c3254=t,c3255=t,c3256=t,c3257=t,c3258=t,c3259=t,c3260=t,c3261=t,c3262=t,c3263=t,c3264=t,c3265=t,c3266=t,c3267=t,c3268=t,c3269=t,c3270=t,c3271=t,c3272=t,c3273=t,c3274=t,c3275=t,c3276=t,c3277=t,c3278=t,c3279=t,c3280=t,c3281=t,c3282=t,c3283=t,c3284=t,c3285=t,c3286=t,c3287=t,c3288=t,c3289=t,c3290=t,c3291=t,c3292=t,c3293=t,c3294=t,c3295=t,c3296=t,c3297=t,c3298=t,c3299=t,c3300=t,c3301=t,c3302=t,c3303=t,c3304=t,c3305=t,c3306=t,c3307=t,c3308=t,c3309=t,c3310=t,c3311=t,c3312=t," + "c3313=t,c3314=t,c3315=t,c3316=t,c3317=t,c3318=t,c3319=t,c3320=t,c3321=t,c3322=t,c3323=t,c3324=t,c3325=t,c3326=t,c3327=t,c3328=t,c3329=t,c3330=t,c3331=t,c3332=t,c3333=t,c3334=t,c3335=t,c3336=t,c3337=t,c3338=t,c3339=t,c3340=t,c3341=t,c3342=t,c3343=t,c3344=t,c3345=t,c3346=t,c3347=t,c3348=t,c3349=t,c3350=t,c3351=t,c3352=t,c3353=t,c3354=t,c3355=t,c3356=t,c3357=t,c3358=t,c3359=t,c3360=t,c3361=t,c3362=t,c3363=t,c3364=t,c3365=t,c3366=t,c3367=t,c3368=t,c3369=t,c3370=t,c3371=t,c3372=t,c3373=t,c3374=t,c3375=t,c3376=t,c3377=t,c3378=t,c3379=t,c3380=t,c3381=t,c3382=t,c3383=t,c3384=t,c3385=t,c3386=t,c3387=t,c3388=t,c3389=t,c3390=t,c3391=t,c3392=t,c3393=t,c3394=t,c3395=t,c3396=t,c3397=t,c3398=t,c3399=t,c3400=t,c3401=t,c3402=t,c3403=t,c3404=t,c3405=t,c3406=t,c3407=t,c3408=t,c3409=t,c3410=t,c3411=t,c3412=t,c3413=t,c3414=t,c3415=t,c3416=t,c3417=t,c3418=t,c3419=t,c3420=t,c3421=t,c3422=t,c3423=t,c3424=t,c3425=t,c3426=t,c3427=t,c3428=t,c3429=t,c3430=t,c3431=t,c3432=t,c3433=t,c3434=t,c3435=t,c3436=t,c3437=t,c3438=t,c3439=t,c3440=t,c3441=t,c3442=t,c3443=t,c3444=t,c3445=t,c3446=t,c3447=t,c3448=t,c3449=t,c3450=t,c3451=t,c3452=t,c3453=t,c3454=t,c3455=t,c3456=t,c3457=t,c3458=t,c3459=t," + "c3460=t,c3461=t,c3462=t,c3463=t,c3464=t,c3465=t,c3466=t,c3467=t,c3468=t,c3469=t,c3470=t,c3471=t,c3472=t,c3473=t,c3474=t,c3475=t,c3476=t,c3477=t,c3478=t,c3479=t,c3480=t,c3481=t,c3482=t,c3483=t,c3484=t,c3485=t,c3486=t,c3487=t,c3488=t,c3489=t,c3490=t,c3491=t,c3492=t,c3493=t,c3494=t,c3495=t,c3496=t,c3497=t,c3498=t,c3499=t,c3500=t,c3501=t,c3502=t,c3503=t,c3504=t,c3505=t,c3506=t,c3507=t,c3508=t,c3509=t,c3510=t,c3511=t,c3512=t,c3513=t," + "c3514=t,c3515=t,c3516=t,c3517=t,c3518=t,c3519=t,c3520=t,c3521=t,c3522=t,c3523=t,c3524=t,c3525=t,c3526=t,c3527=t,c3528=t,c3529=t,c3530=t,c3531=t,c3532=t,c3533=t,c3534=t,c3535=t,c3536=t,c3537=t,c3538=t,c3539=t,c3540=t,c3541=t,c3542=t,c3543=t,c3544=t,c3545=t,c3546=t,c3547=t,c3548=t,c3549=t,c3550=t,c3551=t,c3552=t,c3553=t,c3554=t,c3555=t,c3556=t,c3557=t,c3558=t,c3559=t,c3560=t,c3561=t,c3562=t,c3563=t,c3564=t,c3565=t,c3566=t,c3567=t,c3568=t,c3569=t,c3570=t,c3571=t,c3572=t,c3573=t,c3574=t,c3575=t,c3576=t,c3577=t,c3578=t,c3579=t,c3580=t,c3581=t,c3582=t,c3583=t,c3584=t,c3585=t,c3586=t,c3587=t,c3588=t,c3589=t,c3590=t,c3591=t,c3592=t,c3593=t,c3594=t,c3595=t,c3596=t,c3597=t,c3598=t,c3599=t,c3600=t,c3601=t,c3602=t,c3603=t,c3604=t,c3605=t,c3606=t,c3607=t,c3608=t,c3609=t,c3610=t,c3611=t,c3612=t,c3613=t,c3614=t,c3615=t,c3616=t,c3617=t,c3618=t,c3619=t,c3620=t,c3621=t,c3622=t,c3623=t,c3624=t,c3625=t,c3626=t,c3627=t,c3628=t,c3629=t,c3630=t,c3631=t,c3632=t,c3633=t,c3634=t,c3635=t,c3636=t,c3637=t,c3638=t,c3639=t,c3640=t,c3641=t,c3642=t,c3643=t,c3644=t,c3645=t,c3646=t,c3647=t,c3648=t,c3649=t,c3650=t,c3651=t,c3652=t,c3653=t,c3654=t,c3655=t,c3656=t,c3657=t,c3658=t,c3659=t,c3660=t," + "c3661=t,c3662=t,c3663=t,c3664=t,c3665=t,c3666=t,c3667=t,c3668=t,c3669=t,c3670=t,c3671=t,c3672=t,c3673=t,c3674=t,c3675=t,c3676=t,c3677=t,c3678=t,c3679=t,c3680=t,c3681=t,c3682=t,c3683=t,c3684=t,c3685=t,c3686=t,c3687=t,c3688=t,c3689=t,c3690=t,c3691=t,c3692=t,c3693=t,c3694=t,c3695=t,c3696=t,c3697=t,c3698=t,c3699=t,c3700=t,c3701=t,c3702=t,c3703=t,c3704=t,c3705=t,c3706=t,c3707=t,c3708=t,c3709=t,c3710=t,c3711=t,c3712=t,c3713=t,c3714=t,c3715=t,c3716=t,c3717=t,c3718=t,c3719=t,c3720=t,c3721=t,c3722=t,c3723=t,c3724=t,c3725=t,c3726=t,c3727=t,c3728=t,c3729=t,c3730=t,c3731=t,c3732=t,c3733=t,c3734=t,c3735=t,c3736=t,c3737=t,c3738=t,c3739=t,c3740=t,c3741=t,c3742=t,c3743=t,c3744=t,c3745=t,c3746=t,c3747=t,c3748=t,c3749=t,c3750=t,c3751=t,c3752=t,c3753=t,c3754=t,c3755=t,c3756=t,c3757=t,c3758=t,c3759=t,c3760=t,c3761=t,c3762=t,c3763=t,c3764=t,c3765=t,c3766=t,c3767=t,c3768=t,c3769=t,c3770=t,c3771=t,c3772=t,c3773=t,c3774=t,c3775=t,c3776=t,c3777=t,c3778=t,c3779=t,c3780=t,c3781=t,c3782=t,c3783=t,c3784=t,c3785=t,c3786=t,c3787=t,c3788=t,c3789=t,c3790=t,c3791=t,c3792=t,c3793=t,c3794=t,c3795=t,c3796=t,c3797=t,c3798=t,c3799=t,c3800=t,c3801=t,c3802=t,c3803=t,c3804=t,c3805=t,c3806=t,c3807=t," + "c3808=t,c3809=t,c3810=t,c3811=t,c3812=t,c3813=t,c3814=t,c3815=t,c3816=t,c3817=t,c3818=t,c3819=t,c3820=t,c3821=t,c3822=t,c3823=t,c3824=t,c3825=t,c3826=t,c3827=t,c3828=t,c3829=t,c3830=t,c3831=t,c3832=t,c3833=t,c3834=t,c3835=t,c3836=t,c3837=t,c3838=t,c3839=t,c3840=t,c3841=t,c3842=t,c3843=t,c3844=t,c3845=t,c3846=t,c3847=t,c3848=t,c3849=t,c3850=t,c3851=t,c3852=t,c3853=t,c3854=t,c3855=t,c3856=t,c3857=t,c3858=t,c3859=t,c3860=t,c3861=t,c3862=t,c3863=t,c3864=t,c3865=t,c3866=t,c3867=t,c3868=t,c3869=t,c3870=t,c3871=t,c3872=t,c3873=t,c3874=t,c3875=t,c3876=t,c3877=t,c3878=t,c3879=t,c3880=t,c3881=t,c3882=t,c3883=t,c3884=t,c3885=t,c3886=t,c3887=t,c3888=t,c3889=t,c3890=t,c3891=t,c3892=t,c3893=t,c3894=t,c3895=t,c3896=t,c3897=t,c3898=t,c3899=t,c3900=t,c3901=t,c3902=t,c3903=t,c3904=t,c3905=t,c3906=t,c3907=t,c3908=t,c3909=t,c3910=t,c3911=t,c3912=t,c3913=t,c3914=t,c3915=t,c3916=t,c3917=t,c3918=t,c3919=t,c3920=t,c3921=t,c3922=t,c3923=t,c3924=t,c3925=t,c3926=t,c3927=t,c3928=t,c3929=t,c3930=t,c3931=t,c3932=t,c3933=t,c3934=t,c3935=t,c3936=t,c3937=t,c3938=t,c3939=t,c3940=t,c3941=t,c3942=t,c3943=t,c3944=t,c3945=t,c3946=t,c3947=t,c3948=t,c3949=t,c3950=t,c3951=t,c3952=t,c3953=t,c3954=t," + "c3955=t,c3956=t,c3957=t,c3958=t,c3959=t,c3960=t,c3961=t,c3962=t,c3963=t,c3964=t,c3965=t,c3966=t,c3967=t,c3968=t,c3969=t,c3970=t,c3971=t,c3972=t,c3973=t,c3974=t,c3975=t,c3976=t,c3977=t,c3978=t,c3979=t,c3980=t,c3981=t,c3982=t,c3983=t,c3984=t,c3985=t,c3986=t,c3987=t,c3988=t,c3989=t,c3990=t,c3991=t,c3992=t,c3993=t,c3994=t,c3995=t,c3996=t,c3997=t,c3998=t,c3999=t,c4000=t,c4001=t,c4002=t,c4003=t,c4004=t,c4005=t,c4006=t,c4007=t,c4008=t,c4009=t,c4010=t,c4011=t,c4012=t,c4013=t,c4014=t,c4015=t,c4016=t,c4017=t,c4018=t,c4019=t,c4020=t,c4021=t,c4022=t,c4023=t,c4024=t,c4025=t,c4026=t,c4027=t,c4028=t,c4029=t,c4030=t,c4031=t,c4032=t,c4033=t,c4034=t,c4035=t,c4036=t,c4037=t,c4038=t,c4039=t,c4040=t,c4041=t,c4042=t,c4043=t,c4044=t,c4045=t,c4046=t,c4047=t,c4048=t,c4049=t,c4050=t,c4051=t,c4052=t,c4053=t,c4054=t,c4055=t,c4056=t,c4057=t,c4058=t,c4059=t,c4060=t,c4061=t,c4062=t,c4063=t,c4064=t,c4065=t,c4066=t,c4067=t,c4068=t,c4069=t,c4070=t,c4071=t,c4072=t,c4073=t,c4074=t,c4075=t,c4076=t,c4077=t,c4078=t,c4079=t,c4080=t,c4081=t,c4082=t,c4083=t,c4084=t,c4085=t,c4086=t,c4087=t,c4088=t,c4089=t,c4090=t,c4091=t,c4092=t,c4093=t 1626006833640000000" + }; + + int ret = TSDB_CODE_SUCCESS; + for(int i = 0; i < sizeof(sql)/sizeof(sql[0]); i++){ + ret = smlParseInfluxLine(info, sql[i]); + if(ret != TSDB_CODE_SUCCESS) break; + } + ASSERT_NE(ret, 0); + smlDestroyInfo(info); +} diff --git a/source/common/src/systable.c b/source/common/src/systable.c index 68a77a9f33..9ca896c9ee 100644 --- a/source/common/src/systable.c +++ b/source/common/src/systable.c @@ -15,343 +15,345 @@ #include "systable.h" #include "taos.h" +#include "taosdef.h" #include "tdef.h" #include "tgrant.h" +#include "tmsg.h" #include "types.h" #define SYSTABLE_SCH_TABLE_NAME_LEN ((TSDB_TABLE_NAME_LEN - 1) + VARSTR_HEADER_SIZE) #define SYSTABLE_SCH_DB_NAME_LEN ((TSDB_DB_NAME_LEN - 1) + VARSTR_HEADER_SIZE) #define SYSTABLE_SCH_COL_NAME_LEN ((TSDB_COL_NAME_LEN - 1) + VARSTR_HEADER_SIZE) +// clang-format off static const SSysDbTableSchema dnodesSchema[] = { - {.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "endpoint", .bytes = TSDB_EP_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "vnodes", .bytes = 2, .type = TSDB_DATA_TYPE_SMALLINT}, - {.name = "support_vnodes", .bytes = 2, .type = TSDB_DATA_TYPE_SMALLINT}, - {.name = "status", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, - {.name = "note", .bytes = 256 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, + {.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "endpoint", .bytes = TSDB_EP_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "vnodes", .bytes = 2, .type = TSDB_DATA_TYPE_SMALLINT, .sysInfo = true}, + {.name = "support_vnodes", .bytes = 2, .type = TSDB_DATA_TYPE_SMALLINT, .sysInfo = true}, + {.name = "status", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = true}, + {.name = "note", .bytes = 256 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, }; static const SSysDbTableSchema mnodesSchema[] = { - {.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "endpoint", .bytes = TSDB_EP_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "role", .bytes = 12 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "status", .bytes = 9 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, + {.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "endpoint", .bytes = TSDB_EP_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "role", .bytes = 12 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "status", .bytes = 9 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = true}, }; static const SSysDbTableSchema modulesSchema[] = { - {.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "endpoint", .bytes = 134 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "module", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, + {.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "endpoint", .bytes = 134 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "module", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, }; static const SSysDbTableSchema qnodesSchema[] = { - {.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "endpoint", .bytes = TSDB_EP_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, + {.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "endpoint", .bytes = TSDB_EP_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = true}, }; static const SSysDbTableSchema snodesSchema[] = { - {.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "endpoint", .bytes = TSDB_EP_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, + {.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "endpoint", .bytes = TSDB_EP_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = true}, }; static const SSysDbTableSchema bnodesSchema[] = { - {.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "endpoint", .bytes = TSDB_EP_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, + {.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "endpoint", .bytes = TSDB_EP_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = true}, }; static const SSysDbTableSchema clusterSchema[] = { - {.name = "id", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT}, - {.name = "name", .bytes = TSDB_CLUSTER_ID_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "uptime", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, + {.name = "id", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT, .sysInfo = true}, + {.name = "name", .bytes = TSDB_CLUSTER_ID_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "uptime", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = true}, }; static const SSysDbTableSchema userDBSchema[] = { - {.name = "name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, - {.name = "vgroups", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "ntables", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT}, - {.name = "replica", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT}, - {.name = "strict", .bytes = TSDB_DB_STRICT_STR_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "duration", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "keep", .bytes = 32 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "buffer", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "pagesize", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "pages", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "minrows", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "maxrows", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "comp", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT}, - {.name = "precision", .bytes = 2 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "status", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "retentions", .bytes = 60 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "single_stable", .bytes = 1, .type = TSDB_DATA_TYPE_BOOL}, - {.name = "cachemodel", .bytes = TSDB_CACHE_MODEL_STR_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "cachesize", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "wal_level", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT}, - {.name = "wal_fsync_period", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "wal_retention_period", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "wal_retention_size", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT}, - {.name = "wal_roll_period", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "wal_segment_size", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT}, + {.name = "name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, + {.name = "vgroups", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "ntables", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT, .sysInfo = false}, + {.name = "replica", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT, .sysInfo = true}, + {.name = "strict", .bytes = TSDB_DB_STRICT_STR_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "duration", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "keep", .bytes = 32 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "buffer", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "pagesize", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "pages", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "minrows", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "maxrows", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "comp", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT, .sysInfo = true}, + {.name = "precision", .bytes = 2 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "status", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "retentions", .bytes = 60 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "single_stable", .bytes = 1, .type = TSDB_DATA_TYPE_BOOL, .sysInfo = true}, + {.name = "cachemodel", .bytes = TSDB_CACHE_MODEL_STR_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "cachesize", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "wal_level", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT, .sysInfo = true}, + {.name = "wal_fsync_period", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "wal_retention_period", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "wal_retention_size", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT, .sysInfo = true}, + {.name = "wal_roll_period", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "wal_segment_size", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT, .sysInfo = true}, }; static const SSysDbTableSchema userFuncSchema[] = { - {.name = "name", .bytes = TSDB_FUNC_NAME_LEN - 1 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "comment", .bytes = PATH_MAX - 1 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "aggregate", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "output_type", .bytes = TSDB_TYPE_STR_MAX_LEN - 1 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, - {.name = "code_len", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "bufsize", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, + {.name = "name", .bytes = TSDB_FUNC_NAME_LEN - 1 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "comment", .bytes = PATH_MAX - 1 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "aggregate", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false}, + {.name = "output_type", .bytes = TSDB_TYPE_STR_MAX_LEN - 1 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, + {.name = "code_len", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false}, + {.name = "bufsize", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false}, }; static const SSysDbTableSchema userIdxSchema[] = { - {.name = "index_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "db_name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "table_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "vgroup_id", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, + {.name = "index_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "db_name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "table_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "vgroup_id", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, }; static const SSysDbTableSchema userStbsSchema[] = { - {.name = "stable_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "db_name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, - {.name = "columns", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "tags", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "last_update", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, - {.name = "table_comment", .bytes = TSDB_TB_COMMENT_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "watermark", .bytes = 64 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "max_delay", .bytes = 64 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "rollup", .bytes = 128 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, + {.name = "stable_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "db_name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, + {.name = "columns", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false}, + {.name = "tags", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false}, + {.name = "last_update", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, + {.name = "table_comment", .bytes = TSDB_TB_COMMENT_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "watermark", .bytes = 64 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "max_delay", .bytes = 64 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "rollup", .bytes = 128 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, }; static const SSysDbTableSchema streamSchema[] = { - {.name = "stream_name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, - {.name = "sql", .bytes = TSDB_SHOW_SQL_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "status", .bytes = 20 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "source_db", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "target_db", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "target_table", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "watermark", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT}, - {.name = "trigger", .bytes = 20 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, + {.name = "stream_name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, + {.name = "sql", .bytes = TSDB_SHOW_SQL_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "status", .bytes = 20 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "source_db", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "target_db", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "target_table", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "watermark", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT, .sysInfo = false}, + {.name = "trigger", .bytes = 20 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, }; static const SSysDbTableSchema userTblsSchema[] = { - {.name = "table_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "db_name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, - {.name = "columns", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "stable_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "uid", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT}, - {.name = "vgroup_id", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "ttl", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "table_comment", .bytes = TSDB_TB_COMMENT_LEN - 1 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "type", .bytes = 21 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, + {.name = "table_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "db_name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, + {.name = "columns", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false}, + {.name = "stable_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "uid", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT, .sysInfo = false}, + {.name = "vgroup_id", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "ttl", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false}, + {.name = "table_comment", .bytes = TSDB_TB_COMMENT_LEN - 1 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "type", .bytes = 21 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, }; static const SSysDbTableSchema userTagsSchema[] = { - {.name = "table_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "db_name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "stable_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "tag_name", .bytes = TSDB_COL_NAME_LEN - 1 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "tag_type", .bytes = 32 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "tag_value", .bytes = TSDB_MAX_TAGS_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, + {.name = "table_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "db_name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "stable_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "tag_name", .bytes = TSDB_COL_NAME_LEN - 1 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "tag_type", .bytes = 32 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "tag_value", .bytes = TSDB_MAX_TAGS_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, }; static const SSysDbTableSchema userTblDistSchema[] = { - {.name = "db_name", .bytes = 32 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "table_name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "distributed_histogram", .bytes = 500 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "min_of_rows", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "max_of_rows", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "avg_of_rows", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "stddev_of_rows", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "rows", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT}, - {.name = "blocks", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "storage_size", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT}, - {.name = "compression_ratio", .bytes = 8, .type = TSDB_DATA_TYPE_DOUBLE}, - {.name = "rows_in_mem", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "seek_header_time", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, + {.name = "db_name", .bytes = 32 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "table_name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "distributed_histogram", .bytes = 500 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "min_of_rows", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "max_of_rows", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "avg_of_rows", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "stddev_of_rows", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "rows", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT, .sysInfo = true}, + {.name = "blocks", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "storage_size", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT, .sysInfo = true}, + {.name = "compression_ratio", .bytes = 8, .type = TSDB_DATA_TYPE_DOUBLE, .sysInfo = true}, + {.name = "rows_in_mem", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "seek_header_time", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, }; static const SSysDbTableSchema userUsersSchema[] = { - {.name = "name", .bytes = TSDB_USER_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "super", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT}, - {.name = "enable", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT}, - {.name = "sysinfo", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT}, - {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, + {.name = "name", .bytes = TSDB_USER_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "super", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT, .sysInfo = false}, + {.name = "enable", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT, .sysInfo = false}, + {.name = "sysinfo", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT, .sysInfo = false}, + {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, }; GRANTS_SCHEMA; static const SSysDbTableSchema vgroupsSchema[] = { - {.name = "vgroup_id", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "db_name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "tables", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "v1_dnode", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "v1_status", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "v2_dnode", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "v2_status", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "v3_dnode", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "v3_status", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "status", .bytes = 12 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "nfiles", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "file_size", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "tsma", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT}, + {.name = "vgroup_id", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "db_name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "tables", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "v1_dnode", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "v1_status", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "v2_dnode", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "v2_status", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "v3_dnode", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "v3_status", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "status", .bytes = 12 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "nfiles", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "file_size", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, + {.name = "tsma", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT, .sysInfo = true}, }; static const SSysDbTableSchema smaSchema[] = { - {.name = "sma_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, - {.name = "stable_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "vgroup_id", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, + {.name = "sma_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, + {.name = "stable_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "vgroup_id", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = true}, }; static const SSysDbTableSchema transSchema[] = { - {.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, - {.name = "stage", .bytes = TSDB_TRANS_STAGE_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "db1", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "db2", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "failed_times", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "last_exec_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, - {.name = "last_action_info", - .bytes = (TSDB_TRANS_ERROR_LEN - 1) + VARSTR_HEADER_SIZE, - .type = TSDB_DATA_TYPE_VARCHAR}, + {.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false}, + {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, + {.name = "stage", .bytes = TSDB_TRANS_STAGE_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "db1", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "db2", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "failed_times", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false}, + {.name = "last_exec_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, + {.name = "last_action_info", .bytes = (TSDB_TRANS_ERROR_LEN - 1) + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, }; static const SSysDbTableSchema configSchema[] = { - {.name = "name", .bytes = TSDB_CONFIG_OPTION_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "value", .bytes = TSDB_CONFIG_VALUE_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, + {.name = "name", .bytes = TSDB_CONFIG_OPTION_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "value", .bytes = TSDB_CONFIG_VALUE_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, }; static const SSysDbTableSchema variablesSchema[] = { {.name = "dnode_id", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "name", .bytes = TSDB_CONFIG_OPTION_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "value", .bytes = TSDB_CONFIG_VALUE_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, + {.name = "name", .bytes = TSDB_CONFIG_OPTION_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, + {.name = "value", .bytes = TSDB_CONFIG_VALUE_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = true}, }; static const SSysTableMeta infosMeta[] = { - {TSDB_INS_TABLE_DNODES, dnodesSchema, tListLen(dnodesSchema)}, - {TSDB_INS_TABLE_MNODES, mnodesSchema, tListLen(mnodesSchema)}, - {TSDB_INS_TABLE_MODULES, modulesSchema, tListLen(modulesSchema)}, - {TSDB_INS_TABLE_QNODES, qnodesSchema, tListLen(qnodesSchema)}, + {TSDB_INS_TABLE_DNODES, dnodesSchema, tListLen(dnodesSchema), true}, + {TSDB_INS_TABLE_MNODES, mnodesSchema, tListLen(mnodesSchema), true}, + {TSDB_INS_TABLE_MODULES, modulesSchema, tListLen(modulesSchema), true}, + {TSDB_INS_TABLE_QNODES, qnodesSchema, tListLen(qnodesSchema), true}, // {TSDB_INS_TABLE_SNODES, snodesSchema, tListLen(snodesSchema)}, // {TSDB_INS_TABLE_BNODES, bnodesSchema, tListLen(bnodesSchema)}, - {TSDB_INS_TABLE_CLUSTER, clusterSchema, tListLen(clusterSchema)}, - {TSDB_INS_TABLE_DATABASES, userDBSchema, tListLen(userDBSchema)}, - {TSDB_INS_TABLE_FUNCTIONS, userFuncSchema, tListLen(userFuncSchema)}, - {TSDB_INS_TABLE_INDEXES, userIdxSchema, tListLen(userIdxSchema)}, - {TSDB_INS_TABLE_STABLES, userStbsSchema, tListLen(userStbsSchema)}, - {TSDB_INS_TABLE_TABLES, userTblsSchema, tListLen(userTblsSchema)}, - {TSDB_INS_TABLE_TAGS, userTagsSchema, tListLen(userTagsSchema)}, + {TSDB_INS_TABLE_CLUSTER, clusterSchema, tListLen(clusterSchema), true}, + {TSDB_INS_TABLE_DATABASES, userDBSchema, tListLen(userDBSchema), false}, + {TSDB_INS_TABLE_FUNCTIONS, userFuncSchema, tListLen(userFuncSchema), false}, + {TSDB_INS_TABLE_INDEXES, userIdxSchema, tListLen(userIdxSchema), false}, + {TSDB_INS_TABLE_STABLES, userStbsSchema, tListLen(userStbsSchema), false}, + {TSDB_INS_TABLE_TABLES, userTblsSchema, tListLen(userTblsSchema), false}, + {TSDB_INS_TABLE_TAGS, userTagsSchema, tListLen(userTagsSchema), false}, // {TSDB_INS_TABLE_TABLE_DISTRIBUTED, userTblDistSchema, tListLen(userTblDistSchema)}, - {TSDB_INS_TABLE_USERS, userUsersSchema, tListLen(userUsersSchema)}, - {TSDB_INS_TABLE_LICENCES, grantsSchema, tListLen(grantsSchema)}, - {TSDB_INS_TABLE_VGROUPS, vgroupsSchema, tListLen(vgroupsSchema)}, - {TSDB_INS_TABLE_CONFIGS, configSchema, tListLen(configSchema)}, - {TSDB_INS_TABLE_DNODE_VARIABLES, variablesSchema, tListLen(variablesSchema)}, + {TSDB_INS_TABLE_USERS, userUsersSchema, tListLen(userUsersSchema), false}, + {TSDB_INS_TABLE_LICENCES, grantsSchema, tListLen(grantsSchema), true}, + {TSDB_INS_TABLE_VGROUPS, vgroupsSchema, tListLen(vgroupsSchema), true}, + {TSDB_INS_TABLE_CONFIGS, configSchema, tListLen(configSchema), true}, + {TSDB_INS_TABLE_DNODE_VARIABLES, variablesSchema, tListLen(variablesSchema), true}, }; static const SSysDbTableSchema connectionsSchema[] = { - {.name = "conn_id", .bytes = 4, .type = TSDB_DATA_TYPE_UINT}, - {.name = "user", .bytes = TSDB_USER_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY}, - {.name = "app", .bytes = TSDB_APP_NAME_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY}, - {.name = "pid", .bytes = 4, .type = TSDB_DATA_TYPE_UINT}, - {.name = "end_point", .bytes = TSDB_EP_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY}, - {.name = "login_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, - {.name = "last_access", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, + {.name = "conn_id", .bytes = 4, .type = TSDB_DATA_TYPE_UINT, .sysInfo = false}, + {.name = "user", .bytes = TSDB_USER_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false}, + {.name = "app", .bytes = TSDB_APP_NAME_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false}, + {.name = "pid", .bytes = 4, .type = TSDB_DATA_TYPE_UINT, .sysInfo = false}, + {.name = "end_point", .bytes = TSDB_EP_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false}, + {.name = "login_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, + {.name = "last_access", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, }; static const SSysDbTableSchema topicSchema[] = { - {.name = "topic_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_BINARY}, - {.name = "db_name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_BINARY}, - {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, - {.name = "sql", .bytes = TSDB_SHOW_SQL_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY}, + {.name = "topic_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false}, + {.name = "db_name", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false}, + {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, + {.name = "sql", .bytes = TSDB_SHOW_SQL_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false}, // TODO config }; static const SSysDbTableSchema consumerSchema[] = { - {.name = "consumer_id", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT}, - {.name = "consumer_group", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_BINARY}, - {.name = "client_id", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_BINARY}, - {.name = "status", .bytes = 20 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY}, - {.name = "topics", .bytes = TSDB_TOPIC_FNAME_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY}, - /*{.name = "end_point", .bytes = TSDB_IPv4ADDR_LEN + 6 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},*/ - {.name = "up_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, - {.name = "subscribe_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, - {.name = "rebalance_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, + {.name = "consumer_id", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT, .sysInfo = false}, + {.name = "consumer_group", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false}, + {.name = "client_id", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false}, + {.name = "status", .bytes = 20 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false}, + {.name = "topics", .bytes = TSDB_TOPIC_FNAME_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false}, + /*{.name = "end_point", .bytes = TSDB_IPv4ADDR_LEN + 6 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false},*/ + {.name = "up_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, + {.name = "subscribe_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, + {.name = "rebalance_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, }; static const SSysDbTableSchema subscriptionSchema[] = { - {.name = "topic_name", .bytes = TSDB_TOPIC_FNAME_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY}, - {.name = "consumer_group", .bytes = TSDB_CGROUP_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY}, - {.name = "vgroup_id", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "consumer_id", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT}, + {.name = "topic_name", .bytes = TSDB_TOPIC_FNAME_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false}, + {.name = "consumer_group", .bytes = TSDB_CGROUP_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false}, + {.name = "vgroup_id", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false}, + {.name = "consumer_id", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT, .sysInfo = false}, }; static const SSysDbTableSchema offsetSchema[] = { - {.name = "topic_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_BINARY}, - {.name = "group_id", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_BINARY}, - {.name = "vgroup_id", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "committed_offset", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT}, - {.name = "current_offset", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT}, - {.name = "skip_log_cnt", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT}, + {.name = "topic_name", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false}, + {.name = "group_id", .bytes = SYSTABLE_SCH_TABLE_NAME_LEN, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false}, + {.name = "vgroup_id", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false}, + {.name = "committed_offset", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT, .sysInfo = false}, + {.name = "current_offset", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT, .sysInfo = false}, + {.name = "skip_log_cnt", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT, .sysInfo = false}, }; static const SSysDbTableSchema querySchema[] = { - {.name = "kill_id", .bytes = TSDB_QUERY_ID_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "query_id", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT}, - {.name = "conn_id", .bytes = 4, .type = TSDB_DATA_TYPE_UINT}, - {.name = "app", .bytes = TSDB_APP_NAME_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "pid", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "user", .bytes = TSDB_USER_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "end_point", .bytes = TSDB_IPv4ADDR_LEN + 6 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, - {.name = "exec_usec", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT}, - {.name = "stable_query", .bytes = 1, .type = TSDB_DATA_TYPE_BOOL}, - {.name = "sub_num", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "sub_status", .bytes = TSDB_SHOW_SUBQUERY_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "sql", .bytes = TSDB_SHOW_SQL_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, + {.name = "kill_id", .bytes = TSDB_QUERY_ID_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "query_id", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT, .sysInfo = false}, + {.name = "conn_id", .bytes = 4, .type = TSDB_DATA_TYPE_UINT, .sysInfo = false}, + {.name = "app", .bytes = TSDB_APP_NAME_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "pid", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false}, + {.name = "user", .bytes = TSDB_USER_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "end_point", .bytes = TSDB_IPv4ADDR_LEN + 6 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, + {.name = "exec_usec", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT, .sysInfo = false}, + {.name = "stable_query", .bytes = 1, .type = TSDB_DATA_TYPE_BOOL, .sysInfo = false}, + {.name = "sub_num", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false}, + {.name = "sub_status", .bytes = TSDB_SHOW_SUBQUERY_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "sql", .bytes = TSDB_SHOW_SQL_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, }; static const SSysDbTableSchema appSchema[] = { - {.name = "app_id", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT}, - {.name = "ip", .bytes = TSDB_IPv4ADDR_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "pid", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, - {.name = "name", .bytes = TSDB_APP_NAME_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "start_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, - {.name = "insert_req", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT}, - {.name = "insert_row", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT}, - {.name = "insert_time", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT}, - {.name = "insert_bytes", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT}, - {.name = "fetch_bytes", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT}, - {.name = "query_time", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT}, - {.name = "slow_query", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT}, - {.name = "total_req", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT}, - {.name = "current_req", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT}, - {.name = "last_access", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, + {.name = "app_id", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT, .sysInfo = false}, + {.name = "ip", .bytes = TSDB_IPv4ADDR_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "pid", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false}, + {.name = "name", .bytes = TSDB_APP_NAME_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false}, + {.name = "start_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, + {.name = "insert_req", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT, .sysInfo = false}, + {.name = "insert_row", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT, .sysInfo = false}, + {.name = "insert_time", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT, .sysInfo = false}, + {.name = "insert_bytes", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT, .sysInfo = false}, + {.name = "fetch_bytes", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT, .sysInfo = false}, + {.name = "query_time", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT, .sysInfo = false}, + {.name = "slow_query", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT, .sysInfo = false}, + {.name = "total_req", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT, .sysInfo = false}, + {.name = "current_req", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT, .sysInfo = false}, + {.name = "last_access", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false}, }; static const SSysTableMeta perfsMeta[] = { - {TSDB_PERFS_TABLE_CONNECTIONS, connectionsSchema, tListLen(connectionsSchema)}, - {TSDB_PERFS_TABLE_QUERIES, querySchema, tListLen(querySchema)}, - {TSDB_PERFS_TABLE_TOPICS, topicSchema, tListLen(topicSchema)}, - {TSDB_PERFS_TABLE_CONSUMERS, consumerSchema, tListLen(consumerSchema)}, - {TSDB_PERFS_TABLE_SUBSCRIPTIONS, subscriptionSchema, tListLen(subscriptionSchema)}, + {TSDB_PERFS_TABLE_CONNECTIONS, connectionsSchema, tListLen(connectionsSchema), false}, + {TSDB_PERFS_TABLE_QUERIES, querySchema, tListLen(querySchema), false}, + {TSDB_PERFS_TABLE_TOPICS, topicSchema, tListLen(topicSchema), false}, + {TSDB_PERFS_TABLE_CONSUMERS, consumerSchema, tListLen(consumerSchema), false}, + {TSDB_PERFS_TABLE_SUBSCRIPTIONS, subscriptionSchema, tListLen(subscriptionSchema), false}, // {TSDB_PERFS_TABLE_OFFSETS, offsetSchema, tListLen(offsetSchema)}, - {TSDB_PERFS_TABLE_TRANS, transSchema, tListLen(transSchema)}, - {TSDB_PERFS_TABLE_SMAS, smaSchema, tListLen(smaSchema)}, - {TSDB_PERFS_TABLE_STREAMS, streamSchema, tListLen(streamSchema)}, - {TSDB_PERFS_TABLE_APPS, appSchema, tListLen(appSchema)}}; + {TSDB_PERFS_TABLE_TRANS, transSchema, tListLen(transSchema), false}, + // {TSDB_PERFS_TABLE_SMAS, smaSchema, tListLen(smaSchema), false}, + {TSDB_PERFS_TABLE_STREAMS, streamSchema, tListLen(streamSchema), false}, + {TSDB_PERFS_TABLE_APPS, appSchema, tListLen(appSchema), false}}; +// clang-format on void getInfosDbMeta(const SSysTableMeta** pInfosTableMeta, size_t* size) { if (pInfosTableMeta) { @@ -370,3 +372,26 @@ void getPerfDbMeta(const SSysTableMeta** pPerfsTableMeta, size_t* size) { *size = tListLen(perfsMeta); } } + +void getVisibleInfosTablesNum(bool sysInfo, size_t* size) { + if (sysInfo) { + getInfosDbMeta(NULL, size); + return; + } + *size = 0; + const SSysTableMeta* pMeta = NULL; + size_t totalNum = 0; + getInfosDbMeta(&pMeta, &totalNum); + for (size_t i = 0; i < totalNum; ++i) { + if (!pMeta[i].sysInfo) { + ++(*size); + } + } +} + +bool invisibleColumn(bool sysInfo, int8_t tableType, int8_t flags) { + if (sysInfo || TSDB_SYSTEM_TABLE != tableType) { + return false; + } + return 0 != (flags & COL_IS_SYSINFO); +} diff --git a/source/common/src/tdatablock.c b/source/common/src/tdatablock.c index c65e966046..c7f372f17b 100644 --- a/source/common/src/tdatablock.c +++ b/source/common/src/tdatablock.c @@ -1228,6 +1228,7 @@ void blockDataFreeRes(SSDataBlock* pBlock) { } taosArrayDestroy(pBlock->pDataBlock); + pBlock->pDataBlock = NULL; taosMemoryFreeClear(pBlock->pBlockAgg); memset(&pBlock->info, 0, sizeof(SDataBlockInfo)); } @@ -1706,8 +1707,8 @@ static char* formatTimestamp(char* buf, int64_t val, int precision) { } void blockDebugShowDataBlock(SSDataBlock* pBlock, const char* flag) { - SArray* dataBlocks = taosArrayInit(1, sizeof(SSDataBlock)); - taosArrayPush(dataBlocks, pBlock); + SArray* dataBlocks = taosArrayInit(1, sizeof(SSDataBlock*)); + taosArrayPush(dataBlocks, &pBlock); blockDebugShowDataBlocks(dataBlocks, flag); taosArrayDestroy(dataBlocks); } diff --git a/source/common/src/tdataformat.c b/source/common/src/tdataformat.c index 8eeab77a15..b40f449a05 100644 --- a/source/common/src/tdataformat.c +++ b/source/common/src/tdataformat.c @@ -1064,6 +1064,26 @@ _err: return code; } +void tTagSetCid(const STag *pTag, int16_t iTag, int16_t cid) { + uint8_t *p = NULL; + int8_t isLarge = pTag->flags & TD_TAG_LARGE; + int16_t offset = 0; + + if (isLarge) { + p = (uint8_t *)&((int16_t *)pTag->idx)[pTag->nTag]; + } else { + p = (uint8_t *)&pTag->idx[pTag->nTag]; + } + + if (isLarge) { + offset = ((int16_t *)pTag->idx)[iTag]; + } else { + offset = pTag->idx[iTag]; + } + + tPutI16v(p + offset, cid); +} + #if 1 // =================================================================================================================== int tdInitTSchemaBuilder(STSchemaBuilder *pBuilder, schema_ver_t version) { if (pBuilder == NULL) return -1; diff --git a/source/common/src/tglobal.c b/source/common/src/tglobal.c index bb2729c776..ee9d751555 100644 --- a/source/common/src/tglobal.c +++ b/source/common/src/tglobal.c @@ -420,6 +420,7 @@ static int32_t taosAddServerCfg(SConfig *pCfg) { if (cfgAddInt32(pCfg, "mqRebalanceInterval", tsMqRebalanceInterval, 1, 10000, 1) != 0) return -1; if (cfgAddInt32(pCfg, "ttlUnit", tsTtlUnit, 1, 86400 * 365, 1) != 0) return -1; if (cfgAddInt32(pCfg, "ttlPushInterval", tsTtlPushInterval, 1, 100000, 1) != 0) return -1; + if (cfgAddInt32(pCfg, "uptimeInterval", tsUptimeInterval, 1, 100000, 1) != 0) return -1; if (cfgAddBool(pCfg, "udf", tsStartUdfd, 0) != 0) return -1; GRANT_CFG_ADD; @@ -567,6 +568,7 @@ static int32_t taosSetServerCfg(SConfig *pCfg) { tsMqRebalanceInterval = cfgGetItem(pCfg, "mqRebalanceInterval")->i32; tsTtlUnit = cfgGetItem(pCfg, "ttlUnit")->i32; tsTtlPushInterval = cfgGetItem(pCfg, "ttlPushInterval")->i32; + tsUptimeInterval = cfgGetItem(pCfg, "uptimeInterval")->i32; tsStartUdfd = cfgGetItem(pCfg, "udf")->bval; diff --git a/source/common/src/tmsg.c b/source/common/src/tmsg.c index 533d924546..8dc4931573 100644 --- a/source/common/src/tmsg.c +++ b/source/common/src/tmsg.c @@ -3196,12 +3196,16 @@ static int32_t tDecodeSTableMetaRsp(SDecoder *pDecoder, STableMetaRsp *pRsp) { if (tDecodeI32(pDecoder, &pRsp->vgId) < 0) return -1; int32_t totalCols = pRsp->numOfTags + pRsp->numOfColumns; - pRsp->pSchemas = taosMemoryMalloc(sizeof(SSchema) * totalCols); - if (pRsp->pSchemas == NULL) return -1; + if (totalCols > 0) { + pRsp->pSchemas = taosMemoryMalloc(sizeof(SSchema) * totalCols); + if (pRsp->pSchemas == NULL) return -1; - for (int32_t i = 0; i < totalCols; ++i) { - SSchema *pSchema = &pRsp->pSchemas[i]; - if (tDecodeSSchema(pDecoder, pSchema) < 0) return -1; + for (int32_t i = 0; i < totalCols; ++i) { + SSchema *pSchema = &pRsp->pSchemas[i]; + if (tDecodeSSchema(pDecoder, pSchema) < 0) return -1; + } + } else { + pRsp->pSchemas = NULL; } return 0; @@ -3326,7 +3330,7 @@ int32_t tDeserializeSSTbHbRsp(void *buf, int32_t bufLen, SSTbHbRsp *pRsp) { return 0; } -void tFreeSTableMetaRsp(STableMetaRsp *pRsp) { taosMemoryFreeClear(pRsp->pSchemas); } +void tFreeSTableMetaRsp(void *pRsp) { taosMemoryFreeClear(((STableMetaRsp*)pRsp)->pSchemas); } void tFreeSTableIndexRsp(void *info) { if (NULL == info) { @@ -3630,6 +3634,7 @@ int32_t tSerializeSConnectRsp(void *buf, int32_t bufLen, SConnectRsp *pRsp) { if (tEncodeU32(&encoder, pRsp->connId) < 0) return -1; if (tEncodeI32(&encoder, pRsp->dnodeNum) < 0) return -1; if (tEncodeI8(&encoder, pRsp->superUser) < 0) return -1; + if (tEncodeI8(&encoder, pRsp->sysInfo) < 0) return -1; if (tEncodeI8(&encoder, pRsp->connType) < 0) return -1; if (tEncodeSEpSet(&encoder, &pRsp->epSet) < 0) return -1; if (tEncodeI32(&encoder, pRsp->svrTimestamp) < 0) return -1; @@ -3652,6 +3657,7 @@ int32_t tDeserializeSConnectRsp(void *buf, int32_t bufLen, SConnectRsp *pRsp) { if (tDecodeU32(&decoder, &pRsp->connId) < 0) return -1; if (tDecodeI32(&decoder, &pRsp->dnodeNum) < 0) return -1; if (tDecodeI8(&decoder, &pRsp->superUser) < 0) return -1; + if (tDecodeI8(&decoder, &pRsp->sysInfo) < 0) return -1; if (tDecodeI8(&decoder, &pRsp->connType) < 0) return -1; if (tDecodeSEpSet(&decoder, &pRsp->epSet) < 0) return -1; if (tDecodeI32(&decoder, &pRsp->svrTimestamp) < 0) return -1; @@ -5090,6 +5096,10 @@ int tEncodeSVCreateTbRsp(SEncoder *pCoder, const SVCreateTbRsp *pRsp) { if (tStartEncode(pCoder) < 0) return -1; if (tEncodeI32(pCoder, pRsp->code) < 0) return -1; + if (tEncodeI32(pCoder, pRsp->pMeta ? 1 : 0) < 0) return -1; + if (pRsp->pMeta) { + if (tEncodeSTableMetaRsp(pCoder, pRsp->pMeta) < 0) return -1; + } tEndEncode(pCoder); return 0; @@ -5100,10 +5110,32 @@ int tDecodeSVCreateTbRsp(SDecoder *pCoder, SVCreateTbRsp *pRsp) { if (tDecodeI32(pCoder, &pRsp->code) < 0) return -1; + int32_t meta = 0; + if (tDecodeI32(pCoder, &meta) < 0) return -1; + if (meta) { + pRsp->pMeta = taosMemoryCalloc(1, sizeof(STableMetaRsp)); + if (NULL == pRsp->pMeta) return -1; + if (tDecodeSTableMetaRsp(pCoder, pRsp->pMeta) < 0) return -1; + } else { + pRsp->pMeta = NULL; + } + tEndDecode(pCoder); return 0; } +void tFreeSVCreateTbRsp(void* param) { + if (NULL == param) { + return; + } + + SVCreateTbRsp* pRsp = (SVCreateTbRsp*)param; + if (pRsp->pMeta) { + taosMemoryFree(pRsp->pMeta->pSchemas); + taosMemoryFree(pRsp->pMeta); + } +} + // TDMT_VND_DROP_TABLE ================= static int32_t tEncodeSVDropTbReq(SEncoder *pCoder, const SVDropTbReq *pReq) { if (tStartEncode(pCoder) < 0) return -1; @@ -5292,6 +5324,10 @@ static int32_t tEncodeSSubmitBlkRsp(SEncoder *pEncoder, const SSubmitBlkRsp *pBl if (tEncodeI32v(pEncoder, pBlock->numOfRows) < 0) return -1; if (tEncodeI32v(pEncoder, pBlock->affectedRows) < 0) return -1; if (tEncodeI64v(pEncoder, pBlock->sver) < 0) return -1; + if (tEncodeI32(pEncoder, pBlock->pMeta ? 1 : 0) < 0) return -1; + if (pBlock->pMeta) { + if (tEncodeSTableMetaRsp(pEncoder, pBlock->pMeta) < 0) return -1; + } tEndEncode(pEncoder); return 0; @@ -5309,6 +5345,16 @@ static int32_t tDecodeSSubmitBlkRsp(SDecoder *pDecoder, SSubmitBlkRsp *pBlock) { if (tDecodeI32v(pDecoder, &pBlock->numOfRows) < 0) return -1; if (tDecodeI32v(pDecoder, &pBlock->affectedRows) < 0) return -1; if (tDecodeI64v(pDecoder, &pBlock->sver) < 0) return -1; + + int32_t meta = 0; + if (tDecodeI32(pDecoder, &meta) < 0) return -1; + if (meta) { + pBlock->pMeta = taosMemoryCalloc(1, sizeof(STableMetaRsp)); + if (NULL == pBlock->pMeta) return -1; + if (tDecodeSTableMetaRsp(pDecoder, pBlock->pMeta) < 0) return -1; + } else { + pBlock->pMeta = NULL; + } tEndDecode(pDecoder); return 0; @@ -5347,6 +5393,21 @@ int32_t tDecodeSSubmitRsp(SDecoder *pDecoder, SSubmitRsp *pRsp) { return 0; } +void tFreeSSubmitBlkRsp(void* param) { + if (NULL == param) { + return; + } + + SSubmitBlkRsp* pRsp = (SSubmitBlkRsp*)param; + + taosMemoryFree(pRsp->tblFName); + if (pRsp->pMeta) { + taosMemoryFree(pRsp->pMeta->pSchemas); + taosMemoryFree(pRsp->pMeta); + } +} + + void tFreeSSubmitRsp(SSubmitRsp *pRsp) { if (NULL == pRsp) return; @@ -5558,9 +5619,63 @@ void tFreeSMAlterStbRsp(SMAlterStbRsp *pRsp) { } } + +int32_t tEncodeSMCreateStbRsp(SEncoder *pEncoder, const SMCreateStbRsp *pRsp) { + if (tStartEncode(pEncoder) < 0) return -1; + if (tEncodeI32(pEncoder, pRsp->pMeta->pSchemas ? 1 : 0) < 0) return -1; + if (pRsp->pMeta->pSchemas) { + if (tEncodeSTableMetaRsp(pEncoder, pRsp->pMeta) < 0) return -1; + } + tEndEncode(pEncoder); + return 0; +} + +int32_t tDecodeSMCreateStbRsp(SDecoder *pDecoder, SMCreateStbRsp *pRsp) { + int32_t meta = 0; + if (tStartDecode(pDecoder) < 0) return -1; + if (tDecodeI32(pDecoder, &meta) < 0) return -1; + if (meta) { + pRsp->pMeta = taosMemoryCalloc(1, sizeof(STableMetaRsp)); + if (NULL == pRsp->pMeta) return -1; + if (tDecodeSTableMetaRsp(pDecoder, pRsp->pMeta) < 0) return -1; + } + tEndDecode(pDecoder); + return 0; +} + +int32_t tDeserializeSMCreateStbRsp(void *buf, int32_t bufLen, SMCreateStbRsp *pRsp) { + int32_t meta = 0; + SDecoder decoder = {0}; + tDecoderInit(&decoder, buf, bufLen); + + if (tStartDecode(&decoder) < 0) return -1; + if (tDecodeI32(&decoder, &meta) < 0) return -1; + if (meta) { + pRsp->pMeta = taosMemoryCalloc(1, sizeof(STableMetaRsp)); + if (NULL == pRsp->pMeta) return -1; + if (tDecodeSTableMetaRsp(&decoder, pRsp->pMeta) < 0) return -1; + } + tEndDecode(&decoder); + tDecoderClear(&decoder); + return 0; +} + +void tFreeSMCreateStbRsp(SMCreateStbRsp *pRsp) { + if (NULL == pRsp) { + return; + } + + if (pRsp->pMeta) { + taosMemoryFree(pRsp->pMeta->pSchemas); + taosMemoryFree(pRsp->pMeta); + } +} + + + int32_t tEncodeSTqOffsetVal(SEncoder *pEncoder, const STqOffsetVal *pOffsetVal) { if (tEncodeI8(pEncoder, pOffsetVal->type) < 0) return -1; - if (pOffsetVal->type == TMQ_OFFSET__SNAPSHOT_DATA) { + if (pOffsetVal->type == TMQ_OFFSET__SNAPSHOT_DATA || pOffsetVal->type == TMQ_OFFSET__SNAPSHOT_META) { if (tEncodeI64(pEncoder, pOffsetVal->uid) < 0) return -1; if (tEncodeI64(pEncoder, pOffsetVal->ts) < 0) return -1; } else if (pOffsetVal->type == TMQ_OFFSET__LOG) { @@ -5575,7 +5690,7 @@ int32_t tEncodeSTqOffsetVal(SEncoder *pEncoder, const STqOffsetVal *pOffsetVal) int32_t tDecodeSTqOffsetVal(SDecoder *pDecoder, STqOffsetVal *pOffsetVal) { if (tDecodeI8(pDecoder, &pOffsetVal->type) < 0) return -1; - if (pOffsetVal->type == TMQ_OFFSET__SNAPSHOT_DATA) { + if (pOffsetVal->type == TMQ_OFFSET__SNAPSHOT_DATA || pOffsetVal->type == TMQ_OFFSET__SNAPSHOT_META) { if (tDecodeI64(pDecoder, &pOffsetVal->uid) < 0) return -1; if (tDecodeI64(pDecoder, &pOffsetVal->ts) < 0) return -1; } else if (pOffsetVal->type == TMQ_OFFSET__LOG) { @@ -5597,10 +5712,8 @@ int32_t tFormatOffset(char *buf, int32_t maxLen, const STqOffsetVal *pVal) { snprintf(buf, maxLen, "offset(reset to latest)"); } else if (pVal->type == TMQ_OFFSET__LOG) { snprintf(buf, maxLen, "offset(log) ver:%" PRId64, pVal->version); - } else if (pVal->type == TMQ_OFFSET__SNAPSHOT_DATA) { + } else if (pVal->type == TMQ_OFFSET__SNAPSHOT_DATA || pVal->type == TMQ_OFFSET__SNAPSHOT_META) { snprintf(buf, maxLen, "offset(ss data) uid:%" PRId64 ", ts:%" PRId64, pVal->uid, pVal->ts); - } else if (pVal->type == TMQ_OFFSET__SNAPSHOT_META) { - snprintf(buf, maxLen, "offset(ss meta) uid:%" PRId64 ", ts:%" PRId64, pVal->uid, pVal->ts); } else { ASSERT(0); } @@ -5614,9 +5727,7 @@ bool tOffsetEqual(const STqOffsetVal *pLeft, const STqOffsetVal *pRight) { } else if (pLeft->type == TMQ_OFFSET__SNAPSHOT_DATA) { return pLeft->uid == pRight->uid && pLeft->ts == pRight->ts; } else if (pLeft->type == TMQ_OFFSET__SNAPSHOT_META) { - ASSERT(0); - // TODO - return pLeft->uid == pRight->uid && pLeft->ts == pRight->ts; + return pLeft->uid == pRight->uid; } else { ASSERT(0); /*ASSERT(pLeft->type == TMQ_OFFSET__RESET_NONE || pLeft->type == TMQ_OFFSET__RESET_EARLIEAST ||*/ @@ -5701,6 +5812,21 @@ int32_t tDecodeDeleteRes(SDecoder *pCoder, SDeleteRes *pRes) { if (tDecodeCStrTo(pCoder, pRes->tsColName) < 0) return -1; return 0; } + +int32_t tEncodeSMqMetaRsp(SEncoder* pEncoder, const SMqMetaRsp* pRsp) { + if (tEncodeSTqOffsetVal(pEncoder, &pRsp->rspOffset) < 0) return -1; + if(tEncodeI16(pEncoder, pRsp->resMsgType)) return -1; + if(tEncodeBinary(pEncoder, pRsp->metaRsp, pRsp->metaRspLen)) return -1; + return 0; +} + +int32_t tDecodeSMqMetaRsp(SDecoder* pDecoder, SMqMetaRsp* pRsp) { + if (tDecodeSTqOffsetVal(pDecoder, &pRsp->rspOffset) < 0) return -1; + if (tDecodeI16(pDecoder, &pRsp->resMsgType) < 0) return -1; + if (tDecodeBinaryAlloc(pDecoder, &pRsp->metaRsp, (uint64_t*)&pRsp->metaRspLen) < 0) return -1; + return 0; +} + int32_t tEncodeSMqDataRsp(SEncoder *pEncoder, const SMqDataRsp *pRsp) { if (tEncodeSTqOffsetVal(pEncoder, &pRsp->reqOffset) < 0) return -1; if (tEncodeSTqOffsetVal(pEncoder, &pRsp->rspOffset) < 0) return -1; diff --git a/source/dnode/mnode/impl/inc/mndInfoSchema.h b/source/dnode/mnode/impl/inc/mndInfoSchema.h index b10d92ee3d..4f98465cd1 100644 --- a/source/dnode/mnode/impl/inc/mndInfoSchema.h +++ b/source/dnode/mnode/impl/inc/mndInfoSchema.h @@ -24,7 +24,8 @@ extern "C" { int32_t mndInitInfos(SMnode *pMnode); void mndCleanupInfos(SMnode *pMnode); -int32_t mndBuildInsTableSchema(SMnode *pMnode, const char *dbFName, const char *tbName, STableMetaRsp *pRsp); +int32_t mndBuildInsTableSchema(SMnode *pMnode, const char *dbFName, const char *tbName, bool sysinfo, + STableMetaRsp *pRsp); int32_t mndBuildInsTableCfg(SMnode *pMnode, const char *dbFName, const char *tbName, STableCfgRsp *pRsp); #ifdef __cplusplus diff --git a/source/dnode/mnode/impl/inc/mndStb.h b/source/dnode/mnode/impl/inc/mndStb.h index 010199a89f..8f0d55e100 100644 --- a/source/dnode/mnode/impl/inc/mndStb.h +++ b/source/dnode/mnode/impl/inc/mndStb.h @@ -35,6 +35,7 @@ SDbObj *mndAcquireDbByStb(SMnode *pMnode, const char *stbName); int32_t mndBuildStbFromReq(SMnode *pMnode, SStbObj *pDst, SMCreateStbReq *pCreate, SDbObj *pDb); int32_t mndAddStbToTrans(SMnode *pMnode, STrans *pTrans, SDbObj *pDb, SStbObj *pStb); void mndFreeStb(SStbObj *pStb); +int32_t mndBuildSMCreateStbRsp(SMnode *pMnode, char* dbFName, char* stbFName, void **pCont, int32_t *pLen); void mndExtractDbNameFromStbFullName(const char *stbFullName, char *dst); void mndExtractTbNameFromStbFullName(const char *stbFullName, char *dst, int32_t dstSize); diff --git a/source/dnode/mnode/impl/src/mndDb.c b/source/dnode/mnode/impl/src/mndDb.c index 853ace79fd..8c1c3ba873 100644 --- a/source/dnode/mnode/impl/src/mndDb.c +++ b/source/dnode/mnode/impl/src/mndDb.c @@ -1731,7 +1731,7 @@ static int32_t mndRetrieveDbs(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBloc SDbObj infoschemaDb = {0}; setInformationSchemaDbCfg(&infoschemaDb); size_t numOfTables = 0; - getInfosDbMeta(NULL, &numOfTables); + getVisibleInfosTablesNum(sysinfo, &numOfTables); mndDumpDbInfoData(pMnode, pBlock, &infoschemaDb, pShow, numOfRows, numOfTables, true, 0, 1); numOfRows += 1; diff --git a/source/dnode/mnode/impl/src/mndInfoSchema.c b/source/dnode/mnode/impl/src/mndInfoSchema.c index bf33cf603f..09172115f8 100644 --- a/source/dnode/mnode/impl/src/mndInfoSchema.c +++ b/source/dnode/mnode/impl/src/mndInfoSchema.c @@ -14,8 +14,8 @@ */ #define _DEFAULT_SOURCE -#include "systable.h" #include "mndInt.h" +#include "systable.h" static int32_t mndInitInfosTableSchema(const SSysDbTableSchema *pSrc, int32_t colNum, SSchema **pDst) { SSchema *schema = taosMemoryCalloc(colNum, sizeof(SSchema)); @@ -29,6 +29,9 @@ static int32_t mndInitInfosTableSchema(const SSysDbTableSchema *pSrc, int32_t co schema[i].type = pSrc[i].type; schema[i].colId = i + 1; schema[i].bytes = pSrc[i].bytes; + if (pSrc[i].sysInfo) { + schema[i].flags |= COL_IS_SYSINFO; + } } *pDst = schema; @@ -43,13 +46,14 @@ static int32_t mndInsInitMeta(SHashObj *hash) { meta.sversion = 1; meta.tversion = 1; - size_t size = 0; - const SSysTableMeta* pInfosTableMeta = NULL; + size_t size = 0; + const SSysTableMeta *pInfosTableMeta = NULL; getInfosDbMeta(&pInfosTableMeta, &size); for (int32_t i = 0; i < size; ++i) { tstrncpy(meta.tbName, pInfosTableMeta[i].name, sizeof(meta.tbName)); meta.numOfColumns = pInfosTableMeta[i].colNum; + meta.sysInfo = pInfosTableMeta[i].sysInfo; if (mndInitInfosTableSchema(pInfosTableMeta[i].schema, pInfosTableMeta[i].colNum, &meta.pSchemas)) { return -1; @@ -64,14 +68,15 @@ static int32_t mndInsInitMeta(SHashObj *hash) { return 0; } -int32_t mndBuildInsTableSchema(SMnode *pMnode, const char *dbFName, const char *tbName, STableMetaRsp *pRsp) { +int32_t mndBuildInsTableSchema(SMnode *pMnode, const char *dbFName, const char *tbName, bool sysinfo, + STableMetaRsp *pRsp) { if (NULL == pMnode->infosMeta) { terrno = TSDB_CODE_APP_NOT_READY; return -1; } STableMetaRsp *pMeta = taosHashGet(pMnode->infosMeta, tbName, strlen(tbName)); - if (NULL == pMeta) { + if (NULL == pMeta || (!sysinfo && pMeta->sysInfo)) { mError("invalid information schema table name:%s", tbName); terrno = TSDB_CODE_MND_INVALID_SYS_TABLENAME; return -1; @@ -121,7 +126,6 @@ int32_t mndBuildInsTableCfg(SMnode *pMnode, const char *dbFName, const char *tbN return 0; } - int32_t mndInitInfos(SMnode *pMnode) { pMnode->infosMeta = taosHashInit(20, taosGetDefaultHashFunction(TSDB_DATA_TYPE_VARCHAR), false, HASH_NO_LOCK); if (pMnode->infosMeta == NULL) { diff --git a/source/dnode/mnode/impl/src/mndMain.c b/source/dnode/mnode/impl/src/mndMain.c index 65a539bc90..2221718023 100644 --- a/source/dnode/mnode/impl/src/mndMain.c +++ b/source/dnode/mnode/impl/src/mndMain.c @@ -132,7 +132,7 @@ static void *mndThreadFp(void *param) { mndCalMqRebalance(pMnode); } - if (lastTime % (tsTelemInterval * 10) == 1) { + if (lastTime % (tsTelemInterval * 10) == ((tsTelemInterval - 1) * 10)) { mndPullupTelem(pMnode); } diff --git a/source/dnode/mnode/impl/src/mndMnode.c b/source/dnode/mnode/impl/src/mndMnode.c index 4f07d9e014..71bda4d4f3 100644 --- a/source/dnode/mnode/impl/src/mndMnode.c +++ b/source/dnode/mnode/impl/src/mndMnode.c @@ -89,14 +89,14 @@ static int32_t mndCreateDefaultMnode(SMnode *pMnode) { if (pRaw == NULL) return -1; sdbSetRawStatus(pRaw, SDB_STATUS_READY); - mDebug("mnode:%d, will be created when deploying, raw:%p", mnodeObj.id, pRaw); + mInfo("mnode:%d, will be created when deploying, raw:%p", mnodeObj.id, pRaw); STrans *pTrans = mndTransCreate(pMnode, TRN_POLICY_RETRY, TRN_CONFLICT_GLOBAL, NULL); if (pTrans == NULL) { mError("mnode:%d, failed to create since %s", mnodeObj.id, terrstr()); return -1; } - mDebug("trans:%d, used to create mnode:%d", pTrans->id, mnodeObj.id); + mInfo("trans:%d, used to create mnode:%d", pTrans->id, mnodeObj.id); if (mndTransAppendCommitlog(pTrans, pRaw) != 0) { mError("trans:%d, failed to append commit log since %s", pTrans->id, terrstr()); @@ -365,7 +365,7 @@ static int32_t mndCreateMnode(SMnode *pMnode, SRpcMsg *pReq, SDnodeObj *pDnode, STrans *pTrans = mndTransCreate(pMnode, TRN_POLICY_RETRY, TRN_CONFLICT_GLOBAL, pReq); if (pTrans == NULL) goto _OVER; mndTransSetSerial(pTrans); - mDebug("trans:%d, used to create mnode:%d", pTrans->id, pCreate->dnodeId); + mInfo("trans:%d, used to create mnode:%d", pTrans->id, pCreate->dnodeId); if (mndSetCreateMnodeRedoLogs(pMnode, pTrans, &mnodeObj) != 0) goto _OVER; if (mndSetCreateMnodeCommitLogs(pMnode, pTrans, &mnodeObj) != 0) goto _OVER; @@ -392,7 +392,7 @@ static int32_t mndProcessCreateMnodeReq(SRpcMsg *pReq) { goto _OVER; } - mDebug("mnode:%d, start to create", createReq.dnodeId); + mInfo("mnode:%d, start to create", createReq.dnodeId); if (mndCheckOperPrivilege(pMnode, pReq->info.conn.user, MND_OPER_CREATE_MNODE) != 0) { goto _OVER; } @@ -574,7 +574,7 @@ static int32_t mndDropMnode(SMnode *pMnode, SRpcMsg *pReq, SMnodeObj *pObj) { pTrans = mndTransCreate(pMnode, TRN_POLICY_RETRY, TRN_CONFLICT_GLOBAL, pReq); if (pTrans == NULL) goto _OVER; mndTransSetSerial(pTrans); - mDebug("trans:%d, used to drop mnode:%d", pTrans->id, pObj->id); + mInfo("trans:%d, used to drop mnode:%d", pTrans->id, pObj->id); if (mndSetDropMnodeInfoToTrans(pMnode, pTrans, pObj) != 0) goto _OVER; if (mndTransPrepare(pMnode, pTrans) != 0) goto _OVER; @@ -597,7 +597,7 @@ static int32_t mndProcessDropMnodeReq(SRpcMsg *pReq) { goto _OVER; } - mDebug("mnode:%d, start to drop", dropReq.dnodeId); + mInfo("mnode:%d, start to drop", dropReq.dnodeId); if (mndCheckOperPrivilege(pMnode, pReq->info.conn.user, MND_OPER_DROP_MNODE) != 0) { goto _OVER; } @@ -732,7 +732,7 @@ static int32_t mndProcessAlterMnodeReq(SRpcMsg *pReq) { } } - mTrace("trans:-1, sync reconfig will be proposed"); + mInfo("trans:-1, sync reconfig will be proposed"); SSyncMgmt *pMgmt = &pMnode->syncMgmt; pMgmt->standby = 0; diff --git a/source/dnode/mnode/impl/src/mndProfile.c b/source/dnode/mnode/impl/src/mndProfile.c index e55c562e38..e8737e30c9 100644 --- a/source/dnode/mnode/impl/src/mndProfile.c +++ b/source/dnode/mnode/impl/src/mndProfile.c @@ -270,6 +270,7 @@ static int32_t mndProcessConnectReq(SRpcMsg *pReq) { SConnectRsp connectRsp = {0}; connectRsp.acctId = pUser->acctId; connectRsp.superUser = pUser->superUser; + connectRsp.sysInfo = pUser->sysInfo; connectRsp.clusterId = pMnode->clusterId; connectRsp.connId = pConn->id; connectRsp.connType = connReq.connType; diff --git a/source/dnode/mnode/impl/src/mndSma.c b/source/dnode/mnode/impl/src/mndSma.c index 2fb934aaad..8638cc5118 100644 --- a/source/dnode/mnode/impl/src/mndSma.c +++ b/source/dnode/mnode/impl/src/mndSma.c @@ -38,7 +38,6 @@ static SSdbRow *mndSmaActionDecode(SSdbRaw *pRaw); static int32_t mndSmaActionInsert(SSdb *pSdb, SSmaObj *pSma); static int32_t mndSmaActionDelete(SSdb *pSdb, SSmaObj *pSpSmatb); static int32_t mndSmaActionUpdate(SSdb *pSdb, SSmaObj *pOld, SSmaObj *pNew); -static int32_t mndSmaGetVgEpSet(SMnode *pMnode, SDbObj *pDb, SVgEpSet **ppVgEpSet, int32_t *numOfVgroups); static int32_t mndProcessCreateSmaReq(SRpcMsg *pReq); static int32_t mndProcessDropSmaReq(SRpcMsg *pReq); static int32_t mndProcessGetSmaReq(SRpcMsg *pReq); @@ -841,6 +840,7 @@ static int32_t mndDropSma(SMnode *pMnode, SRpcMsg *pReq, SDbObj *pDb, SSmaObj *p _OVER: mndTransDrop(pTrans); + mndReleaseStream(pMnode, pStream); mndReleaseVgroup(pMnode, pVgroup); mndReleaseStb(pMnode, pStb); return code; @@ -961,6 +961,7 @@ _OVER: mError("sma:%s, failed to drop since %s", dropReq.name, terrstr()); } + mndReleaseSma(pMnode, pSma); mndReleaseDb(pMnode, pDb); return code; } diff --git a/source/dnode/mnode/impl/src/mndStb.c b/source/dnode/mnode/impl/src/mndStb.c index ebec3d5ea6..dc8285740a 100644 --- a/source/dnode/mnode/impl/src/mndStb.c +++ b/source/dnode/mnode/impl/src/mndStb.c @@ -536,7 +536,7 @@ int32_t mndCheckCreateStbReq(SMCreateStbReq *pCreate) { return -1; } - if (pCreate->numOfColumns < TSDB_MIN_COLUMNS || pCreate->numOfColumns > TSDB_MAX_COLUMNS) { + if (pCreate->numOfColumns < TSDB_MIN_COLUMNS || pCreate->numOfTags + pCreate->numOfColumns > TSDB_MAX_COLUMNS) { terrno = TSDB_CODE_PAR_INVALID_COLUMNS_NUM; return -1; } @@ -1774,6 +1774,67 @@ static int32_t mndBuildSMAlterStbRsp(SDbObj *pDb, SStbObj *pObj, void **pCont, i return 0; } +int32_t mndBuildSMCreateStbRsp(SMnode *pMnode, char* dbFName, char* stbFName, void **pCont, int32_t *pLen) { + int32_t ret = -1; + SDbObj *pDb = mndAcquireDb(pMnode, dbFName); + if (NULL == pDb) { + return -1; + } + + SStbObj *pObj = mndAcquireStb(pMnode, stbFName); + if (NULL == pObj) { + goto _OVER; + } + + SEncoder ec = {0}; + uint32_t contLen = 0; + SMCreateStbRsp stbRsp = {0}; + SName name = {0}; + tNameFromString(&name, pObj->name, T_NAME_ACCT | T_NAME_DB | T_NAME_TABLE); + + stbRsp.pMeta = taosMemoryCalloc(1, sizeof(STableMetaRsp)); + if (NULL == stbRsp.pMeta) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + goto _OVER; + } + + ret = mndBuildStbSchemaImp(pDb, pObj, name.tname, stbRsp.pMeta); + if (ret) { + tFreeSMCreateStbRsp(&stbRsp); + goto _OVER; + } + + tEncodeSize(tEncodeSMCreateStbRsp, &stbRsp, contLen, ret); + if (ret) { + tFreeSMCreateStbRsp(&stbRsp); + goto _OVER; + } + + void *cont = taosMemoryMalloc(contLen); + tEncoderInit(&ec, cont, contLen); + tEncodeSMCreateStbRsp(&ec, &stbRsp); + tEncoderClear(&ec); + + tFreeSMCreateStbRsp(&stbRsp); + + *pCont = cont; + *pLen = contLen; + + ret = 0; + +_OVER: + if (pObj) { + mndReleaseStb(pMnode, pObj); + } + + if (pDb) { + mndReleaseDb(pMnode, pDb); + } + + return ret; +} + + static int32_t mndAlterStbImp(SMnode *pMnode, SRpcMsg *pReq, SDbObj *pDb, SStbObj *pStb, bool needRsp, void *alterOriData, int32_t alterOriDataLen) { int32_t code = -1; @@ -2157,6 +2218,10 @@ static int32_t mndProcessTableMetaReq(SRpcMsg *pReq) { STableInfoReq infoReq = {0}; STableMetaRsp metaRsp = {0}; + SUserObj *pUser = mndAcquireUser(pMnode, pReq->info.conn.user); + if (pUser == NULL) return 0; + bool sysinfo = pUser->sysInfo; + if (tDeserializeSTableInfoReq(pReq->pCont, pReq->contLen, &infoReq) != 0) { terrno = TSDB_CODE_INVALID_MSG; goto _OVER; @@ -2164,7 +2229,7 @@ static int32_t mndProcessTableMetaReq(SRpcMsg *pReq) { if (0 == strcmp(infoReq.dbFName, TSDB_INFORMATION_SCHEMA_DB)) { mDebug("information_schema table:%s.%s, start to retrieve meta", infoReq.dbFName, infoReq.tbName); - if (mndBuildInsTableSchema(pMnode, infoReq.dbFName, infoReq.tbName, &metaRsp) != 0) { + if (mndBuildInsTableSchema(pMnode, infoReq.dbFName, infoReq.tbName, sysinfo, &metaRsp) != 0) { goto _OVER; } } else if (0 == strcmp(infoReq.dbFName, TSDB_PERFORMANCE_SCHEMA_DB)) { @@ -2203,6 +2268,7 @@ _OVER: mError("stb:%s.%s, failed to retrieve meta since %s", infoReq.dbFName, infoReq.tbName, terrstr()); } + mndReleaseUser(pMnode, pUser); tFreeSTableMetaRsp(&metaRsp); return code; } diff --git a/source/dnode/mnode/impl/src/mndStream.c b/source/dnode/mnode/impl/src/mndStream.c index 6dc8e2072b..dd7a9e71ea 100644 --- a/source/dnode/mnode/impl/src/mndStream.c +++ b/source/dnode/mnode/impl/src/mndStream.c @@ -631,6 +631,7 @@ static int32_t mndProcessCreateStreamReq(SRpcMsg *pReq) { SStreamObj *pStream = NULL; SDbObj *pDb = NULL; SCMCreateStreamReq createStreamReq = {0}; + SStreamObj streamObj = {0}; if (tDeserializeSCMCreateStreamReq(pReq->pCont, pReq->contLen, &createStreamReq) != 0) { terrno = TSDB_CODE_INVALID_MSG; @@ -659,7 +660,6 @@ static int32_t mndProcessCreateStreamReq(SRpcMsg *pReq) { } // build stream obj from request - SStreamObj streamObj = {0}; if (mndBuildStreamObjFromCreateReq(pMnode, &streamObj, &createStreamReq) < 0) { /*ASSERT(0);*/ mError("stream:%s, failed to create since %s", createStreamReq.name, terrstr()); diff --git a/source/dnode/mnode/impl/src/mndSync.c b/source/dnode/mnode/impl/src/mndSync.c index b7129cf56e..e8b75e6a94 100644 --- a/source/dnode/mnode/impl/src/mndSync.c +++ b/source/dnode/mnode/impl/src/mndSync.c @@ -50,7 +50,7 @@ void mndSyncCommitMsg(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta cbM int32_t transId = sdbGetIdFromRaw(pMnode->pSdb, pRaw); pMgmt->errCode = cbMeta.code; - mDebug("trans:%d, is proposed, saved:%d code:0x%x, apply index:%" PRId64 " term:%" PRIu64 " config:%" PRId64 + mInfo("trans:%d, is proposed, saved:%d code:0x%x, apply index:%" PRId64 " term:%" PRIu64 " config:%" PRId64 " role:%s raw:%p", transId, pMgmt->transId, cbMeta.code, cbMeta.index, cbMeta.term, cbMeta.lastConfigIndex, syncStr(cbMeta.state), pRaw); @@ -68,7 +68,7 @@ void mndSyncCommitMsg(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta cbM if (pMgmt->errCode != 0) { mError("trans:%d, failed to propose since %s, post sem", transId, tstrerror(pMgmt->errCode)); } else { - mDebug("trans:%d, is proposed and post sem", transId, tstrerror(pMgmt->errCode)); + mInfo("trans:%d, is proposed and post sem", transId, tstrerror(pMgmt->errCode)); } pMgmt->transId = 0; taosWUnLockLatch(&pMgmt->lock); @@ -88,7 +88,7 @@ void mndSyncCommitMsg(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta cbM } int32_t mndSyncGetSnapshot(struct SSyncFSM *pFsm, SSnapshot *pSnapshot, void *pReaderParam, void **ppReader) { - mDebug("start to read snapshot from sdb in atomic way"); + mInfo("start to read snapshot from sdb in atomic way"); SMnode *pMnode = pFsm->data; return sdbStartRead(pMnode->pSdb, (SSdbIter **)ppReader, &pSnapshot->lastApplyIndex, &pSnapshot->lastApplyTerm, &pSnapshot->lastConfigIndex); @@ -118,7 +118,7 @@ void mndReConfig(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SReConfigCbMeta cbM SSyncMgmt *pMgmt = &pMnode->syncMgmt; pMgmt->errCode = cbMeta.code; - mDebug("trans:-1, sync reconfig is proposed, saved:%d code:0x%x, index:%" PRId64 " term:%" PRId64, pMgmt->transId, + mInfo("trans:-1, sync reconfig is proposed, saved:%d code:0x%x, index:%" PRId64 " term:%" PRId64, pMgmt->transId, cbMeta.code, cbMeta.index, cbMeta.term); taosWLockLatch(&pMgmt->lock); @@ -126,7 +126,7 @@ void mndReConfig(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SReConfigCbMeta cbM if (pMgmt->errCode != 0) { mError("trans:-1, failed to propose sync reconfig since %s, post sem", tstrerror(pMgmt->errCode)); } else { - mDebug("trans:-1, sync reconfig is proposed, saved:%d code:0x%x, index:%" PRId64 " term:%" PRId64 " post sem", + mInfo("trans:-1, sync reconfig is proposed, saved:%d code:0x%x, index:%" PRId64 " term:%" PRId64 " post sem", pMgmt->transId, cbMeta.code, cbMeta.index, cbMeta.term); } pMgmt->transId = 0; @@ -136,13 +136,13 @@ void mndReConfig(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SReConfigCbMeta cbM } int32_t mndSnapshotStartRead(struct SSyncFSM *pFsm, void *pParam, void **ppReader) { - mDebug("start to read snapshot from sdb"); + mInfo("start to read snapshot from sdb"); SMnode *pMnode = pFsm->data; return sdbStartRead(pMnode->pSdb, (SSdbIter **)ppReader, NULL, NULL, NULL); } int32_t mndSnapshotStopRead(struct SSyncFSM *pFsm, void *pReader) { - mDebug("stop to read snapshot from sdb"); + mInfo("stop to read snapshot from sdb"); SMnode *pMnode = pFsm->data; return sdbStopRead(pMnode->pSdb, pReader); } @@ -174,12 +174,12 @@ int32_t mndSnapshotDoWrite(struct SSyncFSM *pFsm, void *pWriter, void *pBuf, int void mndLeaderTransfer(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta cbMeta) { SMnode *pMnode = pFsm->data; atomic_store_8(&(pMnode->syncMgmt.leaderTransferFinish), 1); - mDebug("vgId:1, mnode leader transfer finish"); + mInfo("vgId:1, mnode leader transfer finish"); } static void mndBecomeFollower(struct SSyncFSM *pFsm) { SMnode *pMnode = pFsm->data; - mDebug("vgId:1, become follower and post sem"); + mInfo("vgId:1, become follower and post sem"); taosWLockLatch(&pMnode->syncMgmt.lock); if (pMnode->syncMgmt.transId != 0) { @@ -190,7 +190,7 @@ static void mndBecomeFollower(struct SSyncFSM *pFsm) { } static void mndBecomeLeader(struct SSyncFSM *pFsm) { - mDebug("vgId:1, become leader"); + mInfo("vgId:1, become leader"); SMnode *pMnode = pFsm->data; } @@ -228,7 +228,7 @@ int32_t mndInitSync(SMnode *pMnode) { syncInfo.isStandBy = pMgmt->standby; syncInfo.snapshotStrategy = SYNC_STRATEGY_STANDARD_SNAPSHOT; - mDebug("start to open mnode sync, standby:%d", pMgmt->standby); + mInfo("start to open mnode sync, standby:%d", pMgmt->standby); if (pMgmt->standby || pMgmt->replica.id > 0) { SSyncCfg *pCfg = &syncInfo.syncCfg; pCfg->replicaNum = 1; @@ -236,7 +236,7 @@ int32_t mndInitSync(SMnode *pMnode) { SNodeInfo *pNode = &pCfg->nodeInfo[0]; tstrncpy(pNode->nodeFqdn, pMgmt->replica.fqdn, sizeof(pNode->nodeFqdn)); pNode->nodePort = pMgmt->replica.port; - mDebug("mnode ep:%s:%u", pNode->nodeFqdn, pNode->nodePort); + mInfo("mnode ep:%s:%u", pNode->nodeFqdn, pNode->nodePort); } tsem_init(&pMgmt->syncSem, 0, 0); @@ -284,7 +284,7 @@ int32_t mndSyncPropose(SMnode *pMnode, SSdbRaw *pRaw, int32_t transId) { return -1; } else { pMgmt->transId = transId; - mDebug("trans:%d, will be proposed", pMgmt->transId); + mInfo("trans:%d, will be proposed", pMgmt->transId); taosWUnLockLatch(&pMgmt->lock); } @@ -314,7 +314,7 @@ void mndSyncStart(SMnode *pMnode) { SSyncMgmt *pMgmt = &pMnode->syncMgmt; syncSetMsgCb(pMgmt->sync, &pMnode->msgCb); syncStart(pMgmt->sync); - mDebug("mnode sync started, id:%" PRId64 " standby:%d", pMgmt->sync, pMgmt->standby); + mInfo("mnode sync started, id:%" PRId64 " standby:%d", pMgmt->sync, pMgmt->standby); } void mndSyncStop(SMnode *pMnode) { diff --git a/source/dnode/mnode/impl/src/mndTrans.c b/source/dnode/mnode/impl/src/mndTrans.c index c77a80cc82..9c4a5afb03 100644 --- a/source/dnode/mnode/impl/src/mndTrans.c +++ b/source/dnode/mnode/impl/src/mndTrans.c @@ -17,6 +17,7 @@ #include "mndTrans.h" #include "mndConsumer.h" #include "mndDb.h" +#include "mndStb.h" #include "mndPrivilege.h" #include "mndShow.h" #include "mndSync.h" @@ -455,11 +456,11 @@ static const char *mndTransStr(ETrnStage stage) { } static void mndTransTestStartFunc(SMnode *pMnode, void *param, int32_t paramLen) { - mDebug("test trans start, param:%s, len:%d", (char *)param, paramLen); + mInfo("test trans start, param:%s, len:%d", (char *)param, paramLen); } static void mndTransTestStopFunc(SMnode *pMnode, void *param, int32_t paramLen) { - mDebug("test trans stop, param:%s, len:%d", (char *)param, paramLen); + mInfo("test trans stop, param:%s, len:%d", (char *)param, paramLen); } static TransCbFp mndTransGetCbFp(ETrnFunc ftype) { @@ -706,7 +707,7 @@ int32_t mndSetRpcInfoForDbTrans(SMnode *pMnode, SRpcMsg *pMsg, EOperType oper, c if (pTrans->oper == oper) { if (strcasecmp(dbname, pTrans->dbname1) == 0) { - mDebug("trans:%d, db:%s oper:%d matched with input", pTrans->id, dbname, oper); + mInfo("trans:%d, db:%s oper:%d matched with input", pTrans->id, dbname, oper); if (pTrans->pRpcArray == NULL) { pTrans->pRpcArray = taosArrayInit(1, sizeof(SRpcHandleInfo)); } @@ -745,7 +746,7 @@ static int32_t mndTransSync(SMnode *pMnode, STrans *pTrans) { } sdbSetRawStatus(pRaw, SDB_STATUS_READY); - mDebug("trans:%d, sync to other mnodes, stage:%s", pTrans->id, mndTransStr(pTrans->stage)); + mInfo("trans:%d, sync to other mnodes, stage:%s", pTrans->id, mndTransStr(pTrans->stage)); int32_t code = mndSyncPropose(pMnode, pRaw, pTrans->id); if (code != 0) { mError("trans:%d, failed to sync since %s", pTrans->id, terrstr()); @@ -754,7 +755,7 @@ static int32_t mndTransSync(SMnode *pMnode, STrans *pTrans) { } sdbFreeRaw(pRaw); - mDebug("trans:%d, sync finished", pTrans->id); + mInfo("trans:%d, sync finished", pTrans->id); return 0; } @@ -820,12 +821,12 @@ int32_t mndTransPrepare(SMnode *pMnode, STrans *pTrans) { return -1; } - mDebug("trans:%d, prepare transaction", pTrans->id); + mInfo("trans:%d, prepare transaction", pTrans->id); if (mndTransSync(pMnode, pTrans) != 0) { mError("trans:%d, failed to prepare since %s", pTrans->id, terrstr()); return -1; } - mDebug("trans:%d, prepare finished", pTrans->id); + mInfo("trans:%d, prepare finished", pTrans->id); STrans *pNew = mndAcquireTrans(pMnode, pTrans->id); if (pNew == NULL) { @@ -846,22 +847,22 @@ int32_t mndTransPrepare(SMnode *pMnode, STrans *pTrans) { } static int32_t mndTransCommit(SMnode *pMnode, STrans *pTrans) { - mDebug("trans:%d, commit transaction", pTrans->id); + mInfo("trans:%d, commit transaction", pTrans->id); if (mndTransSync(pMnode, pTrans) != 0) { mError("trans:%d, failed to commit since %s", pTrans->id, terrstr()); return -1; } - mDebug("trans:%d, commit finished", pTrans->id); + mInfo("trans:%d, commit finished", pTrans->id); return 0; } static int32_t mndTransRollback(SMnode *pMnode, STrans *pTrans) { - mDebug("trans:%d, rollback transaction", pTrans->id); + mInfo("trans:%d, rollback transaction", pTrans->id); if (mndTransSync(pMnode, pTrans) != 0) { mError("trans:%d, failed to rollback since %s", pTrans->id, terrstr()); return -1; } - mDebug("trans:%d, rollback finished", pTrans->id); + mInfo("trans:%d, rollback finished", pTrans->id); return 0; } @@ -893,13 +894,36 @@ static void mndTransSendRpcRsp(SMnode *pMnode, STrans *pTrans) { for (int32_t i = 0; i < size; ++i) { SRpcHandleInfo *pInfo = taosArrayGet(pTrans->pRpcArray, i); if (pInfo->handle != NULL) { - mDebug("trans:%d, send rsp, code:0x%x stage:%s app:%p", pTrans->id, code, mndTransStr(pTrans->stage), + mInfo("trans:%d, send rsp, code:0x%x stage:%s app:%p", pTrans->id, code, mndTransStr(pTrans->stage), pInfo->ahandle); if (code == TSDB_CODE_RPC_NETWORK_UNAVAIL) { code = TSDB_CODE_MND_TRANS_NETWORK_UNAVAILL; } SRpcMsg rspMsg = {.code = code, .info = *pInfo}; + if (pTrans->originRpcType == TDMT_MND_CREATE_DB) { + mInfo("trans:%d, origin msgtype:%s", pTrans->id, TMSG_INFO(pTrans->originRpcType)); + SDbObj *pDb = mndAcquireDb(pMnode, pTrans->dbname1); + if (pDb != NULL) { + for (int32_t j = 0; j < 12; j++) { + bool ready = mndIsDbReady(pMnode, pDb); + if (!ready) { + mInfo("trans:%d, db:%s not ready yet, wait %d times", pTrans->id, pTrans->dbname1, j); + taosMsleep(1000); + } else { + break; + } + } + } + mndReleaseDb(pMnode, pDb); + } else if (pTrans->originRpcType == TDMT_MND_CREATE_STB) { + void *pCont = NULL; + int32_t contLen = 0; + if (0 == mndBuildSMCreateStbRsp(pMnode, pTrans->dbname1, pTrans->dbname2, &pCont, &contLen) != 0) { + mndTransSetRpcRsp(pTrans, pCont, contLen); + } + } + if (pTrans->rpcRspLen != 0) { void *rpcCont = rpcMallocCont(pTrans->rpcRspLen); if (rpcCont != NULL) { @@ -909,23 +933,6 @@ static void mndTransSendRpcRsp(SMnode *pMnode, STrans *pTrans) { } } - if (pTrans->originRpcType == TDMT_MND_CREATE_DB) { - mDebug("trans:%d, origin msgtype:%s", pTrans->id, TMSG_INFO(pTrans->originRpcType)); - SDbObj *pDb = mndAcquireDb(pMnode, pTrans->dbname1); - if (pDb != NULL) { - for (int32_t j = 0; j < 12; j++) { - bool ready = mndIsDbReady(pMnode, pDb); - if (!ready) { - mDebug("trans:%d, db:%s not ready yet, wait %d times", pTrans->id, pTrans->dbname1, j); - taosMsleep(1000); - } else { - break; - } - } - } - mndReleaseDb(pMnode, pDb); - } - tmsgSendRsp(&rspMsg); } } @@ -971,7 +978,7 @@ int32_t mndTransProcessRsp(SRpcMsg *pRsp) { pAction->errCode = pRsp->code; } - mDebug("trans:%d, %s:%d response is received, code:0x%x, accept:0x%x retry:0x%x", transId, + mInfo("trans:%d, %s:%d response is received, code:0x%x, accept:0x%x retry:0x%x", transId, mndTransStr(pAction->stage), action, pRsp->code, pAction->acceptableCode, pAction->retryCode); mndTransExecute(pMnode, pTrans); @@ -987,10 +994,10 @@ static void mndTransResetAction(SMnode *pMnode, STrans *pTrans, STransAction *pA if (pAction->errCode == TSDB_CODE_RPC_REDIRECT || pAction->errCode == TSDB_CODE_SYN_NEW_CONFIG_ERROR || pAction->errCode == TSDB_CODE_SYN_INTERNAL_ERROR || pAction->errCode == TSDB_CODE_SYN_NOT_LEADER) { pAction->epSet.inUse = (pAction->epSet.inUse + 1) % pAction->epSet.numOfEps; - mDebug("trans:%d, %s:%d execute status is reset and set epset inuse:%d", pTrans->id, mndTransStr(pAction->stage), + mInfo("trans:%d, %s:%d execute status is reset and set epset inuse:%d", pTrans->id, mndTransStr(pAction->stage), pAction->id, pAction->epSet.inUse); } else { - mDebug("trans:%d, %s:%d execute status is reset", pTrans->id, mndTransStr(pAction->stage), pAction->id); + mInfo("trans:%d, %s:%d execute status is reset", pTrans->id, mndTransStr(pAction->stage), pAction->id); } pAction->errCode = 0; } @@ -1017,7 +1024,7 @@ static int32_t mndTransWriteSingleLog(SMnode *pMnode, STrans *pTrans, STransActi pAction->rawWritten = true; pAction->errCode = 0; code = 0; - mDebug("trans:%d, %s:%d write to sdb, type:%s status:%s", pTrans->id, mndTransStr(pAction->stage), pAction->id, + mInfo("trans:%d, %s:%d write to sdb, type:%s status:%s", pTrans->id, mndTransStr(pAction->stage), pAction->id, sdbTableName(pAction->pRaw->type), sdbStatusName(pAction->pRaw->status)); pTrans->lastAction = pAction->id; @@ -1066,7 +1073,7 @@ static int32_t mndTransSendSingleMsg(SMnode *pMnode, STrans *pTrans, STransActio pAction->msgSent = 1; pAction->msgReceived = 0; pAction->errCode = 0; - mDebug("trans:%d, %s:%d is sent, %s", pTrans->id, mndTransStr(pAction->stage), pAction->id, detail); + mInfo("trans:%d, %s:%d is sent, %s", pTrans->id, mndTransStr(pAction->stage), pAction->id, detail); pTrans->lastAction = pAction->id; pTrans->lastMsgType = pAction->msgType; @@ -1093,7 +1100,7 @@ static int32_t mndTransSendSingleMsg(SMnode *pMnode, STrans *pTrans, STransActio static int32_t mndTransExecNullMsg(SMnode *pMnode, STrans *pTrans, STransAction *pAction) { pAction->rawWritten = 0; pAction->errCode = 0; - mDebug("trans:%d, %s:%d confirm action executed", pTrans->id, mndTransStr(pAction->stage), pAction->id); + mInfo("trans:%d, %s:%d confirm action executed", pTrans->id, mndTransStr(pAction->stage), pAction->id); pTrans->lastAction = pAction->id; pTrans->lastMsgType = pAction->msgType; @@ -1153,7 +1160,7 @@ static int32_t mndTransExecuteActions(SMnode *pMnode, STrans *pTrans, SArray *pA pTrans->lastMsgType = 0; memset(&pTrans->lastEpset, 0, sizeof(pTrans->lastEpset)); pTrans->lastErrorNo = 0; - mDebug("trans:%d, all %d actions execute successfully", pTrans->id, numOfActions); + mInfo("trans:%d, all %d actions execute successfully", pTrans->id, numOfActions); return 0; } else { mError("trans:%d, all %d actions executed, code:0x%x", pTrans->id, numOfActions, errCode & 0XFFFF); @@ -1168,7 +1175,7 @@ static int32_t mndTransExecuteActions(SMnode *pMnode, STrans *pTrans, SArray *pA return errCode; } } else { - mDebug("trans:%d, %d of %d actions executed", pTrans->id, numOfExecuted, numOfActions); + mInfo("trans:%d, %d of %d actions executed", pTrans->id, numOfExecuted, numOfActions); return TSDB_CODE_ACTION_IN_PROGRESS; } } @@ -1214,7 +1221,7 @@ static int32_t mndTransExecuteRedoActionsSerial(SMnode *pMnode, STrans *pTrans) code = pAction->errCode; mndTransResetAction(pMnode, pTrans, pAction); } else { - mDebug("trans:%d, %s:%d execute successfully", pTrans->id, mndTransStr(pAction->stage), action); + mInfo("trans:%d, %s:%d execute successfully", pTrans->id, mndTransStr(pAction->stage), action); } } else { code = TSDB_CODE_ACTION_IN_PROGRESS; @@ -1223,7 +1230,7 @@ static int32_t mndTransExecuteRedoActionsSerial(SMnode *pMnode, STrans *pTrans) if (pAction->errCode != 0 && pAction->errCode != pAction->acceptableCode) { code = pAction->errCode; } else { - mDebug("trans:%d, %s:%d write successfully", pTrans->id, mndTransStr(pAction->stage), action); + mInfo("trans:%d, %s:%d write successfully", pTrans->id, mndTransStr(pAction->stage), action); } } else { } @@ -1247,7 +1254,7 @@ static int32_t mndTransExecuteRedoActionsSerial(SMnode *pMnode, STrans *pTrans) if (code == 0) { pTrans->code = 0; pTrans->redoActionPos++; - mDebug("trans:%d, %s:%d is executed and need sync to other mnodes", pTrans->id, mndTransStr(pAction->stage), + mInfo("trans:%d, %s:%d is executed and need sync to other mnodes", pTrans->id, mndTransStr(pAction->stage), pAction->id); code = mndTransSync(pMnode, pTrans); if (code != 0) { @@ -1256,17 +1263,17 @@ static int32_t mndTransExecuteRedoActionsSerial(SMnode *pMnode, STrans *pTrans) mndTransStr(pAction->stage), pAction->id, terrstr()); } } else if (code == TSDB_CODE_ACTION_IN_PROGRESS) { - mDebug("trans:%d, %s:%d is in progress and wait it finish", pTrans->id, mndTransStr(pAction->stage), pAction->id); + mInfo("trans:%d, %s:%d is in progress and wait it finish", pTrans->id, mndTransStr(pAction->stage), pAction->id); break; } else if (code == pAction->retryCode) { - mDebug("trans:%d, %s:%d receive code:0x%x and retry", pTrans->id, mndTransStr(pAction->stage), pAction->id, code); + mInfo("trans:%d, %s:%d receive code:0x%x and retry", pTrans->id, mndTransStr(pAction->stage), pAction->id, code); taosMsleep(300); action--; continue; } else { terrno = code; pTrans->code = code; - mDebug("trans:%d, %s:%d receive code:0x%x and wait another schedule, failedTimes:%d", pTrans->id, + mInfo("trans:%d, %s:%d receive code:0x%x and wait another schedule, failedTimes:%d", pTrans->id, mndTransStr(pAction->stage), pAction->id, code, pTrans->failedTimes); break; } @@ -1278,7 +1285,7 @@ static int32_t mndTransExecuteRedoActionsSerial(SMnode *pMnode, STrans *pTrans) static bool mndTransPerformPrepareStage(SMnode *pMnode, STrans *pTrans) { bool continueExec = true; pTrans->stage = TRN_STAGE_REDO_ACTION; - mDebug("trans:%d, stage from prepare to redoAction", pTrans->id); + mInfo("trans:%d, stage from prepare to redoAction", pTrans->id); return continueExec; } @@ -1297,10 +1304,10 @@ static bool mndTransPerformRedoActionStage(SMnode *pMnode, STrans *pTrans) { if (code == 0) { pTrans->code = 0; pTrans->stage = TRN_STAGE_COMMIT; - mDebug("trans:%d, stage from redoAction to commit", pTrans->id); + mInfo("trans:%d, stage from redoAction to commit", pTrans->id); continueExec = true; } else if (code == TSDB_CODE_ACTION_IN_PROGRESS) { - mDebug("trans:%d, stage keep on redoAction since %s", pTrans->id, tstrerror(code)); + mInfo("trans:%d, stage keep on redoAction since %s", pTrans->id, tstrerror(code)); continueExec = false; } else { pTrans->failedTimes++; @@ -1340,7 +1347,7 @@ static bool mndTransPerformCommitStage(SMnode *pMnode, STrans *pTrans) { if (code == 0) { pTrans->code = 0; pTrans->stage = TRN_STAGE_COMMIT_ACTION; - mDebug("trans:%d, stage from commit to commitAction", pTrans->id); + mInfo("trans:%d, stage from commit to commitAction", pTrans->id); continueExec = true; } else { pTrans->code = terrno; @@ -1359,7 +1366,7 @@ static bool mndTransPerformCommitActionStage(SMnode *pMnode, STrans *pTrans) { if (code == 0) { pTrans->code = 0; pTrans->stage = TRN_STAGE_FINISHED; - mDebug("trans:%d, stage from commitAction to finished", pTrans->id); + mInfo("trans:%d, stage from commitAction to finished", pTrans->id); continueExec = true; } else { pTrans->code = terrno; @@ -1377,10 +1384,10 @@ static bool mndTransPerformUndoActionStage(SMnode *pMnode, STrans *pTrans) { if (code == 0) { pTrans->stage = TRN_STAGE_FINISHED; - mDebug("trans:%d, stage from undoAction to finished", pTrans->id); + mInfo("trans:%d, stage from undoAction to finished", pTrans->id); continueExec = true; } else if (code == TSDB_CODE_ACTION_IN_PROGRESS) { - mDebug("trans:%d, stage keep on undoAction since %s", pTrans->id, tstrerror(code)); + mInfo("trans:%d, stage keep on undoAction since %s", pTrans->id, tstrerror(code)); continueExec = false; } else { pTrans->failedTimes++; @@ -1399,7 +1406,7 @@ static bool mndTransPerformRollbackStage(SMnode *pMnode, STrans *pTrans) { if (code == 0) { pTrans->stage = TRN_STAGE_UNDO_ACTION; - mDebug("trans:%d, stage from rollback to undoAction", pTrans->id); + mInfo("trans:%d, stage from rollback to undoAction", pTrans->id); continueExec = true; } else { pTrans->failedTimes++; @@ -1424,7 +1431,7 @@ static bool mndTransPerfromFinishedStage(SMnode *pMnode, STrans *pTrans) { mError("trans:%d, failed to write sdb since %s", pTrans->id, terrstr()); } - mDebug("trans:%d, execute finished, code:0x%x, failedTimes:%d", pTrans->id, pTrans->code, pTrans->failedTimes); + mInfo("trans:%d, execute finished, code:0x%x, failedTimes:%d", pTrans->id, pTrans->code, pTrans->failedTimes); return continueExec; } @@ -1432,7 +1439,7 @@ void mndTransExecute(SMnode *pMnode, STrans *pTrans) { bool continueExec = true; while (continueExec) { - mDebug("trans:%d, continue to execute, stage:%s", pTrans->id, mndTransStr(pTrans->stage)); + mInfo("trans:%d, continue to execute, stage:%s", pTrans->id, mndTransStr(pTrans->stage)); pTrans->lastExecTime = taosGetTimestampMs(); switch (pTrans->stage) { case TRN_STAGE_PREPARE: diff --git a/source/dnode/vnode/inc/vnode.h b/source/dnode/vnode/inc/vnode.h index 430aadb558..3a3cbe72ba 100644 --- a/source/dnode/vnode/inc/vnode.h +++ b/source/dnode/vnode/inc/vnode.h @@ -63,7 +63,7 @@ void vnodeGetInfo(SVnode *pVnode, const char **dbname, int32_t *vgId); int32_t vnodeProcessCreateTSma(SVnode *pVnode, void *pCont, uint32_t contLen); int32_t vnodeGetAllTableList(SVnode *pVnode, uint64_t uid, SArray *list); int32_t vnodeGetCtbIdList(SVnode *pVnode, int64_t suid, SArray *list); -int32_t vnodeGetStbIdList(SVnode *pVnode, int64_t suid, SArray* list); +int32_t vnodeGetStbIdList(SVnode *pVnode, int64_t suid, SArray *list); void *vnodeGetIdx(SVnode *pVnode); void *vnodeGetIvtIdx(SVnode *pVnode); @@ -96,7 +96,7 @@ int32_t metaGetTableTags(SMeta *pMeta, uint64_t suid, SArray *uidList, SHash int32_t metaReadNext(SMetaReader *pReader); const void *metaGetTableTagVal(void *tag, int16_t type, STagVal *tagVal); int metaGetTableNameByUid(void *meta, uint64_t uid, char *tbName); -bool metaIsTableExist(SMeta *pMeta, tb_uid_t uid); +bool metaIsTableExist(SMeta *pMeta, tb_uid_t uid); typedef struct SMetaFltParam { tb_uid_t suid; @@ -157,6 +157,31 @@ void tsdbCacheSetCapacity(SVnode *pVnode, size_t capacity); size_t tsdbCacheGetCapacity(SVnode *pVnode); // tq +typedef struct SMetaTableInfo{ + int64_t suid; + int64_t uid; + SSchemaWrapper *schema; + char tbName[TSDB_TABLE_NAME_LEN]; +}SMetaTableInfo; + +typedef struct SIdInfo{ + int64_t version; + int32_t index; +}SIdInfo; + +typedef struct SSnapContext { + SMeta *pMeta; + int64_t snapVersion; + TBC *pCur; + int64_t suid; + int8_t subType; + SHashObj *idVersion; + SHashObj *suidInfo; + SArray *idList; + int32_t index; + bool withMeta; + bool queryMetaOrData; // true-get meta, false-get data +}SSnapContext; typedef struct STqReader { int64_t ver; @@ -207,6 +232,12 @@ int32_t vnodeSnapWriterOpen(SVnode *pVnode, int64_t sver, int64_t ever, SVSnapWr int32_t vnodeSnapWriterClose(SVSnapWriter *pWriter, int8_t rollback, SSnapshot *pSnapshot); int32_t vnodeSnapWrite(SVSnapWriter *pWriter, uint8_t *pData, uint32_t nData); +int32_t buildSnapContext(SMeta* pMeta, int64_t snapVersion, int64_t suid, int8_t subType, bool withMeta, SSnapContext** ctxRet); +int32_t getMetafromSnapShot(SSnapContext* ctx, void **pBuf, int32_t *contLen, int16_t *type, int64_t *uid); +SMetaTableInfo getUidfromSnapShot(SSnapContext* ctx); +int32_t setForSnapShot(SSnapContext* ctx, int64_t uid); +int32_t destroySnapContext(SSnapContext* ctx); + // structs struct STsdbCfg { int8_t precision; @@ -226,6 +257,7 @@ typedef struct { int64_t numOfSTables; int64_t numOfCTables; int64_t numOfNTables; + int64_t numOfNTimeSeries; int64_t numOfTimeSeries; int64_t pointsWritten; int64_t totalStorage; diff --git a/source/dnode/vnode/src/inc/tq.h b/source/dnode/vnode/src/inc/tq.h index cb5ec7aabe..a97c8ff132 100644 --- a/source/dnode/vnode/src/inc/tq.h +++ b/source/dnode/vnode/src/inc/tq.h @@ -68,21 +68,21 @@ typedef struct { typedef struct { char* qmsg; - qTaskInfo_t task; } STqExecCol; typedef struct { - int64_t suid; + int64_t suid; } STqExecTb; typedef struct { - SHashObj* pFilterOutTbUid; + SHashObj* pFilterOutTbUid; } STqExecDb; typedef struct { int8_t subType; STqReader* pExecReader; + qTaskInfo_t task; union { STqExecCol execCol; STqExecTb execTb; @@ -101,7 +101,6 @@ typedef struct { int64_t snapshotVer; - // TODO remove SWalReader* pWalReader; SWalRef* pRef; @@ -141,7 +140,7 @@ int32_t tEncodeSTqHandle(SEncoder* pEncoder, const STqHandle* pHandle); int32_t tDecodeSTqHandle(SDecoder* pDecoder, STqHandle* pHandle); // tqRead -int64_t tqScan(STQ* pTq, const STqHandle* pHandle, SMqDataRsp* pRsp, STqOffsetVal* offset); +int64_t tqScan(STQ* pTq, const STqHandle* pHandle, SMqDataRsp* pRsp, SMqMetaRsp* pMetaRsp, STqOffsetVal* offset); int64_t tqFetchLog(STQ* pTq, STqHandle* pHandle, int64_t* fetchOffset, SWalCkHead** pHeadWithCkSum); // tqExec @@ -182,6 +181,11 @@ static FORCE_INLINE void tqOffsetResetToData(STqOffsetVal* pOffsetVal, int64_t u pOffsetVal->ts = ts; } +static FORCE_INLINE void tqOffsetResetToMeta(STqOffsetVal* pOffsetVal, int64_t uid) { + pOffsetVal->type = TMQ_OFFSET__SNAPSHOT_META; + pOffsetVal->uid = uid; +} + static FORCE_INLINE void tqOffsetResetToLog(STqOffsetVal* pOffsetVal, int64_t ver) { pOffsetVal->type = TMQ_OFFSET__LOG; pOffsetVal->version = ver; diff --git a/source/dnode/vnode/src/inc/vnodeInt.h b/source/dnode/vnode/src/inc/vnodeInt.h index 0b51b61c3a..9b252df58b 100644 --- a/source/dnode/vnode/src/inc/vnodeInt.h +++ b/source/dnode/vnode/src/inc/vnodeInt.h @@ -102,7 +102,7 @@ int metaCommit(SMeta* pMeta); int metaCreateSTable(SMeta* pMeta, int64_t version, SVCreateStbReq* pReq); int metaAlterSTable(SMeta* pMeta, int64_t version, SVCreateStbReq* pReq); int metaDropSTable(SMeta* pMeta, int64_t verison, SVDropStbReq* pReq, SArray* tbUidList); -int metaCreateTable(SMeta* pMeta, int64_t version, SVCreateTbReq* pReq); +int metaCreateTable(SMeta* pMeta, int64_t version, SVCreateTbReq* pReq, STableMetaRsp **pMetaRsp); int metaDropTable(SMeta* pMeta, int64_t version, SVDropTbReq* pReq, SArray* tbUids); int metaTtlDropTable(SMeta* pMeta, int64_t ttl, SArray* tbUids); int metaAlterTable(SMeta* pMeta, int64_t version, SVAlterTbReq* pReq, STableMetaRsp* pMetaRsp); diff --git a/source/dnode/vnode/src/meta/metaQuery.c b/source/dnode/vnode/src/meta/metaQuery.c index 805bc24d8c..9d3b4d82eb 100644 --- a/source/dnode/vnode/src/meta/metaQuery.c +++ b/source/dnode/vnode/src/meta/metaQuery.c @@ -619,7 +619,7 @@ int64_t metaGetTimeSeriesNum(SMeta *pMeta) { vnodeGetTimeSeriesNum(pMeta->pVnode, &num); pMeta->pVnode->config.vndStats.numOfTimeSeries = num; - return pMeta->pVnode->config.vndStats.numOfTimeSeries; + return pMeta->pVnode->config.vndStats.numOfTimeSeries + pMeta->pVnode->config.vndStats.numOfNTimeSeries; } typedef struct { @@ -887,6 +887,37 @@ const void *metaGetTableTagVal(void *pTag, int16_t type, STagVal *val) { if (!find) { return NULL; } + +#ifdef TAG_FILTER_DEBUG + if (IS_VAR_DATA_TYPE(val->type)) { + char* buf = taosMemoryCalloc(val->nData + 1, 1); + memcpy(buf, val->pData, val->nData); + metaDebug("metaTag table val varchar index:%d cid:%d type:%d value:%s", 1, val->cid, val->type, buf); + taosMemoryFree(buf); + } else { + double dval = 0; + GET_TYPED_DATA(dval, double, val->type, &val->i64); + metaDebug("metaTag table val number index:%d cid:%d type:%d value:%f", 1, val->cid, val->type, dval); + } + + SArray* pTagVals = NULL; + tTagToValArray((STag*)pTag, &pTagVals); + for (int i = 0; i < taosArrayGetSize(pTagVals); i++) { + STagVal* pTagVal = (STagVal*)taosArrayGet(pTagVals, i); + + if (IS_VAR_DATA_TYPE(pTagVal->type)) { + char* buf = taosMemoryCalloc(pTagVal->nData + 1, 1); + memcpy(buf, pTagVal->pData, pTagVal->nData); + metaDebug("metaTag table varchar index:%d cid:%d type:%d value:%s", i, pTagVal->cid, pTagVal->type, buf); + taosMemoryFree(buf); + } else { + double dval = 0; + GET_TYPED_DATA(dval, double, pTagVal->type, &pTagVal->i64); + metaDebug("metaTag table number index:%d cid:%d type:%d value:%f", i, pTagVal->cid, pTagVal->type, dval); + } + } +#endif + return val; } diff --git a/source/dnode/vnode/src/meta/metaSnapshot.c b/source/dnode/vnode/src/meta/metaSnapshot.c index 973c381407..0edbd092e6 100644 --- a/source/dnode/vnode/src/meta/metaSnapshot.c +++ b/source/dnode/vnode/src/meta/metaSnapshot.c @@ -195,3 +195,434 @@ _err: metaError("vgId:%d, vnode snapshot meta write failed since %s", TD_VID(pMeta->pVnode), tstrerror(code)); return code; } + +typedef struct STableInfoForChildTable{ + char *tableName; + SSchemaWrapper *schemaRow; + SSchemaWrapper *tagRow; +}STableInfoForChildTable; + +static void destroySTableInfoForChildTable(void* data) { + STableInfoForChildTable* pData = (STableInfoForChildTable*)data; + taosMemoryFree(pData->tableName); + tDeleteSSchemaWrapper(pData->schemaRow); + tDeleteSSchemaWrapper(pData->tagRow); +} + +static void MoveToSnapShotVersion(SSnapContext* ctx){ + tdbTbcClose(ctx->pCur); + tdbTbcOpen(ctx->pMeta->pTbDb, &ctx->pCur, NULL); + STbDbKey key = {.version = ctx->snapVersion, .uid = INT64_MAX}; + int c = 0; + tdbTbcMoveTo(ctx->pCur, &key, sizeof(key), &c); + if(c < 0){ + tdbTbcMoveToPrev(ctx->pCur); + } +} + +static int32_t MoveToPosition(SSnapContext* ctx, int64_t ver, int64_t uid){ + tdbTbcClose(ctx->pCur); + tdbTbcOpen(ctx->pMeta->pTbDb, &ctx->pCur, NULL); + STbDbKey key = {.version = ver, .uid = uid}; + int c = 0; + tdbTbcMoveTo(ctx->pCur, &key, sizeof(key), &c); + return c; +} + +static void MoveToFirst(SSnapContext* ctx){ + tdbTbcClose(ctx->pCur); + tdbTbcOpen(ctx->pMeta->pTbDb, &ctx->pCur, NULL); + tdbTbcMoveToFirst(ctx->pCur); +} + +static void saveSuperTableInfoForChildTable(SMetaEntry *me, SHashObj *suidInfo){ + STableInfoForChildTable* data = (STableInfoForChildTable*)taosHashGet(suidInfo, &me->uid, sizeof(tb_uid_t)); + if(data){ + return; + } + STableInfoForChildTable dataTmp = {0}; + dataTmp.tableName = strdup(me->name); + + dataTmp.schemaRow = tCloneSSchemaWrapper(&me->stbEntry.schemaRow); + dataTmp.tagRow = tCloneSSchemaWrapper(&me->stbEntry.schemaTag); + taosHashPut(suidInfo, &me->uid, sizeof(tb_uid_t), &dataTmp, sizeof(STableInfoForChildTable)); +} + +int32_t buildSnapContext(SMeta* pMeta, int64_t snapVersion, int64_t suid, int8_t subType, bool withMeta, SSnapContext** ctxRet){ + SSnapContext* ctx = taosMemoryCalloc(1, sizeof(SSnapContext)); + if(ctx == NULL) return -1; + *ctxRet = ctx; + ctx->pMeta = pMeta; + ctx->snapVersion = snapVersion; + ctx->suid = suid; + ctx->subType = subType; + ctx->queryMetaOrData = withMeta; + ctx->withMeta = withMeta; + ctx->idVersion = taosHashInit(100, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, HASH_NO_LOCK); + if(ctx->idVersion == NULL){ + return -1; + } + + ctx->suidInfo = taosHashInit(100, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, HASH_NO_LOCK); + if(ctx->suidInfo == NULL){ + return -1; + } + taosHashSetFreeFp(ctx->suidInfo, destroySTableInfoForChildTable); + + ctx->index = 0; + ctx->idList = taosArrayInit(100, sizeof(int64_t)); + void *pKey = NULL; + void *pVal = NULL; + int vLen = 0, kLen = 0; + + metaDebug("tmqsnap init snapVersion:%" PRIi64, ctx->snapVersion); + MoveToFirst(ctx); + while(1){ + int32_t ret = tdbTbcNext(ctx->pCur, &pKey, &kLen, &pVal, &vLen); + if (ret < 0) break; + STbDbKey *tmp = (STbDbKey*)pKey; + if (tmp->version > ctx->snapVersion) break; + + SIdInfo* idData = (SIdInfo*)taosHashGet(ctx->idVersion, &tmp->uid, sizeof(tb_uid_t)); + if(idData) { + continue; + } + + if (tdbTbGet(pMeta->pUidIdx, &tmp->uid, sizeof(tb_uid_t), NULL, NULL) < 0) { // check if table exist for now, need optimize later + continue; + } + + SDecoder dc = {0}; + SMetaEntry me = {0}; + tDecoderInit(&dc, pVal, vLen); + metaDecodeEntry(&dc, &me); + if(ctx->subType == TOPIC_SUB_TYPE__TABLE){ + if ((me.uid != ctx->suid && me.type == TSDB_SUPER_TABLE) || + (me.ctbEntry.suid != ctx->suid && me.type == TSDB_CHILD_TABLE)){ + tDecoderClear(&dc); + continue; + } + } + + taosArrayPush(ctx->idList, &tmp->uid); + metaDebug("tmqsnap init idlist name:%s, uid:%" PRIi64, me.name, tmp->uid); + SIdInfo info = {0}; + taosHashPut(ctx->idVersion, &tmp->uid, sizeof(tb_uid_t), &info, sizeof(SIdInfo)); + + tDecoderClear(&dc); + } + taosHashClear(ctx->idVersion); + + MoveToSnapShotVersion(ctx); + while(1){ + int32_t ret = tdbTbcPrev(ctx->pCur, &pKey, &kLen, &pVal, &vLen); + if (ret < 0) break; + + STbDbKey *tmp = (STbDbKey*)pKey; + SIdInfo* idData = (SIdInfo*)taosHashGet(ctx->idVersion, &tmp->uid, sizeof(tb_uid_t)); + if(idData){ + continue; + } + SIdInfo info = {.version = tmp->version, .index = 0}; + taosHashPut(ctx->idVersion, &tmp->uid, sizeof(tb_uid_t), &info, sizeof(SIdInfo)); + + SDecoder dc = {0}; + SMetaEntry me = {0}; + tDecoderInit(&dc, pVal, vLen); + metaDecodeEntry(&dc, &me); + if(ctx->subType == TOPIC_SUB_TYPE__TABLE){ + if ((me.uid != ctx->suid && me.type == TSDB_SUPER_TABLE) || + (me.ctbEntry.suid != ctx->suid && me.type == TSDB_CHILD_TABLE)){ + tDecoderClear(&dc); + continue; + } + } + + if ((ctx->subType == TOPIC_SUB_TYPE__DB && me.type == TSDB_SUPER_TABLE) + || (ctx->subType == TOPIC_SUB_TYPE__TABLE && me.uid == ctx->suid)) { + saveSuperTableInfoForChildTable(&me, ctx->suidInfo); + } + tDecoderClear(&dc); + } + + for(int i = 0; i < taosArrayGetSize(ctx->idList); i++){ + int64_t *uid = taosArrayGet(ctx->idList, i); + SIdInfo* idData = (SIdInfo*)taosHashGet(ctx->idVersion, uid, sizeof(int64_t)); + ASSERT(idData); + idData->index = i; + metaDebug("tmqsnap init idVersion uid:%" PRIi64 " version:%" PRIi64 " index:%d", *uid, idData->version, idData->index); + } + + return TDB_CODE_SUCCESS; +} + +int32_t destroySnapContext(SSnapContext* ctx){ + tdbTbcClose(ctx->pCur); + taosArrayDestroy(ctx->idList); + taosHashCleanup(ctx->idVersion); + taosHashCleanup(ctx->suidInfo); + taosMemoryFree(ctx); + return 0; +} + +static int32_t buildNormalChildTableInfo(SVCreateTbReq *req, void **pBuf, int32_t *contLen){ + int32_t ret = 0; + SVCreateTbBatchReq reqs = {0}; + + reqs.pArray = taosArrayInit(1, sizeof(struct SVCreateTbReq)); + if (NULL == reqs.pArray){ + ret = -1; + goto end; + } + taosArrayPush(reqs.pArray, req); + reqs.nReqs = 1; + + tEncodeSize(tEncodeSVCreateTbBatchReq, &reqs, *contLen, ret); + if(ret < 0){ + ret = -1; + goto end; + } + *contLen += sizeof(SMsgHead); + *pBuf = taosMemoryMalloc(*contLen); + if (NULL == *pBuf) { + ret = -1; + goto end; + } + SEncoder coder = {0}; + tEncoderInit(&coder, POINTER_SHIFT(*pBuf, sizeof(SMsgHead)), *contLen); + if (tEncodeSVCreateTbBatchReq(&coder, &reqs) < 0) { + taosMemoryFreeClear(*pBuf); + tEncoderClear(&coder); + ret = -1; + goto end; + } + tEncoderClear(&coder); + +end: + taosArrayDestroy(reqs.pArray); + return ret; +} + +static int32_t buildSuperTableInfo(SVCreateStbReq *req, void **pBuf, int32_t *contLen){ + int32_t ret = 0; + tEncodeSize(tEncodeSVCreateStbReq, req, *contLen, ret); + if (ret < 0) { + return -1; + } + + *contLen += sizeof(SMsgHead); + *pBuf = taosMemoryMalloc(*contLen); + if (NULL == *pBuf) { + return -1; + } + + SEncoder encoder = {0}; + tEncoderInit(&encoder, POINTER_SHIFT(*pBuf, sizeof(SMsgHead)), *contLen); + if (tEncodeSVCreateStbReq(&encoder, req) < 0) { + taosMemoryFreeClear(*pBuf); + tEncoderClear(&encoder); + return -1; + } + tEncoderClear(&encoder); + return 0; +} + +int32_t setForSnapShot(SSnapContext* ctx, int64_t uid){ + int c = 0; + + if(uid == 0){ + ctx->index = 0; + return c; + } + + SIdInfo* idInfo = (SIdInfo*)taosHashGet(ctx->idVersion, &uid, sizeof(tb_uid_t)); + if(!idInfo){ + return -1; + } + + ctx->index = idInfo->index; + + return c; +} + +int32_t getMetafromSnapShot(SSnapContext* ctx, void **pBuf, int32_t *contLen, int16_t *type, int64_t *uid){ + int32_t ret = 0; + void *pKey = NULL; + void *pVal = NULL; + int vLen = 0, kLen = 0; + + while(1){ + if(ctx->index >= taosArrayGetSize(ctx->idList)){ + metaDebug("tmqsnap get meta end"); + ctx->index = 0; + ctx->queryMetaOrData = false; // change to get data + return 0; + } + + int64_t* uidTmp = taosArrayGet(ctx->idList, ctx->index); + ctx->index++; + SIdInfo* idInfo = (SIdInfo*)taosHashGet(ctx->idVersion, uidTmp, sizeof(tb_uid_t)); + ASSERT(idInfo); + + *uid = *uidTmp; + ret = MoveToPosition(ctx, idInfo->version, *uidTmp); + if(ret == 0){ + break; + } + metaDebug("tmqsnap get meta not exist uid:%" PRIi64 " version:%" PRIi64, *uid, idInfo->version); + } + + tdbTbcGet(ctx->pCur, (const void**)&pKey, &kLen, (const void**)&pVal, &vLen); + SDecoder dc = {0}; + SMetaEntry me = {0}; + tDecoderInit(&dc, pVal, vLen); + metaDecodeEntry(&dc, &me); + metaDebug("tmqsnap get meta uid:%" PRIi64 " name:%s index:%d", *uid, me.name, ctx->index-1); + + if ((ctx->subType == TOPIC_SUB_TYPE__DB && me.type == TSDB_SUPER_TABLE) + || (ctx->subType == TOPIC_SUB_TYPE__TABLE && me.uid == ctx->suid)) { + SVCreateStbReq req = {0}; + req.name = me.name; + req.suid = me.uid; + req.schemaRow = me.stbEntry.schemaRow; + req.schemaTag = me.stbEntry.schemaTag; + req.schemaRow.version = 1; + req.schemaTag.version = 1; + + ret = buildSuperTableInfo(&req, pBuf, contLen); + *type = TDMT_VND_CREATE_STB; + + } else if ((ctx->subType == TOPIC_SUB_TYPE__DB && me.type == TSDB_CHILD_TABLE) + || (ctx->subType == TOPIC_SUB_TYPE__TABLE && me.type == TSDB_CHILD_TABLE && me.ctbEntry.suid == ctx->suid)) { + STableInfoForChildTable* data = (STableInfoForChildTable*)taosHashGet(ctx->suidInfo, &me.ctbEntry.suid, sizeof(tb_uid_t)); + ASSERT(data); + SVCreateTbReq req = {0}; + + req.type = TSDB_CHILD_TABLE; + req.name = me.name; + req.uid = me.uid; + req.commentLen = -1; + req.ctb.suid = me.ctbEntry.suid; + req.ctb.tagNum = data->tagRow->nCols; + req.ctb.name = data->tableName; + + SArray* tagName = taosArrayInit(req.ctb.tagNum, TSDB_COL_NAME_LEN); + STag* p = (STag*)me.ctbEntry.pTags; + if(tTagIsJson(p)){ + if (p->nTag != 0) { + SSchema* schema = &data->tagRow->pSchema[0]; + taosArrayPush(tagName, schema->name); + } + }else{ + SArray* pTagVals = NULL; + if (tTagToValArray((const STag*)p, &pTagVals) != 0) { + ASSERT(0); + } + int16_t nCols = taosArrayGetSize(pTagVals); + for (int j = 0; j < nCols; ++j) { + STagVal* pTagVal = (STagVal*)taosArrayGet(pTagVals, j); + for(int i = 0; i < data->tagRow->nCols; i++){ + SSchema *schema = &data->tagRow->pSchema[i]; + if(schema->colId == pTagVal->cid){ + taosArrayPush(tagName, schema->name); + } + } + } + } +// SIdInfo* sidInfo = (SIdInfo*)taosHashGet(ctx->idVersion, &me.ctbEntry.suid, sizeof(tb_uid_t)); +// if(sidInfo->version >= idInfo->version){ +// // need parse tag +// STag* p = (STag*)me.ctbEntry.pTags; +// SArray* pTagVals = NULL; +// if (tTagToValArray((const STag*)p, &pTagVals) != 0) { +// } +// +// int16_t nCols = taosArrayGetSize(pTagVals); +// for (int j = 0; j < nCols; ++j) { +// STagVal* pTagVal = (STagVal*)taosArrayGet(pTagVals, j); +// } +// }else{ + req.ctb.pTag = me.ctbEntry.pTags; +// } + + req.ctb.tagName = tagName; + ret = buildNormalChildTableInfo(&req, pBuf, contLen); + *type = TDMT_VND_CREATE_TABLE; + taosArrayDestroy(tagName); + } else if(ctx->subType == TOPIC_SUB_TYPE__DB){ + SVCreateTbReq req = {0}; + req.type = TSDB_NORMAL_TABLE; + req.name = me.name; + req.uid = me.uid; + req.commentLen = -1; + req.ntb.schemaRow = me.ntbEntry.schemaRow; + ret = buildNormalChildTableInfo(&req, pBuf, contLen); + *type = TDMT_VND_CREATE_TABLE; + } else{ + ASSERT(0); + } + tDecoderClear(&dc); + + return ret; +} + +SMetaTableInfo getUidfromSnapShot(SSnapContext* ctx){ + SMetaTableInfo result = {0}; + void *pKey = NULL; + void *pVal = NULL; + int vLen, kLen; + + while(1){ + if(ctx->index >= taosArrayGetSize(ctx->idList)){ + metaDebug("tmqsnap get uid info end"); + return result; + } + int64_t* uidTmp = taosArrayGet(ctx->idList, ctx->index); + ctx->index++; + SIdInfo* idInfo = (SIdInfo*)taosHashGet(ctx->idVersion, uidTmp, sizeof(tb_uid_t)); + ASSERT(idInfo); + + int32_t ret = MoveToPosition(ctx, idInfo->version, *uidTmp); + if(ret != 0) { + metaDebug("tmqsnap getUidfromSnapShot not exist uid:%" PRIi64 " version:%" PRIi64, *uidTmp, idInfo->version); + continue; + } + tdbTbcGet(ctx->pCur, (const void**)&pKey, &kLen, (const void**)&pVal, &vLen); + SDecoder dc = {0}; + SMetaEntry me = {0}; + tDecoderInit(&dc, pVal, vLen); + metaDecodeEntry(&dc, &me); + metaDebug("tmqsnap get uid info uid:%" PRIi64 " name:%s index:%d", me.uid, me.name, ctx->index-1); + + if (ctx->subType == TOPIC_SUB_TYPE__DB && me.type == TSDB_CHILD_TABLE){ + STableInfoForChildTable* data = (STableInfoForChildTable*)taosHashGet(ctx->suidInfo, &me.ctbEntry.suid, sizeof(tb_uid_t)); + result.uid = me.uid; + result.suid = me.ctbEntry.suid; + result.schema = tCloneSSchemaWrapper(data->schemaRow); + strcpy(result.tbName, me.name); + tDecoderClear(&dc); + break; + } else if (ctx->subType == TOPIC_SUB_TYPE__DB && me.type == TSDB_NORMAL_TABLE) { + result.uid = me.uid; + result.suid = 0; + strcpy(result.tbName, me.name); + result.schema = tCloneSSchemaWrapper(&me.ntbEntry.schemaRow); + tDecoderClear(&dc); + break; + } else if(ctx->subType == TOPIC_SUB_TYPE__TABLE && me.type == TSDB_CHILD_TABLE && me.ctbEntry.suid == ctx->suid) { + STableInfoForChildTable* data = (STableInfoForChildTable*)taosHashGet(ctx->suidInfo, &me.ctbEntry.suid, sizeof(tb_uid_t)); + result.uid = me.uid; + result.suid = me.ctbEntry.suid; + strcpy(result.tbName, me.name); + result.schema = tCloneSSchemaWrapper(data->schemaRow); + tDecoderClear(&dc); + break; + } else{ + metaDebug("tmqsnap get uid continue"); + tDecoderClear(&dc); + continue; + } + } + + return result; +} diff --git a/source/dnode/vnode/src/meta/metaTable.c b/source/dnode/vnode/src/meta/metaTable.c index aa107ab253..583a2e098f 100644 --- a/source/dnode/vnode/src/meta/metaTable.c +++ b/source/dnode/vnode/src/meta/metaTable.c @@ -99,6 +99,7 @@ static int metaSaveJsonVarToIdx(SMeta *pMeta, const SMetaEntry *pCtbEntry, const memcpy(val, (uint16_t *)&len, VARSTR_HEADER_SIZE); type = TSDB_DATA_TYPE_VARCHAR; term = indexTermCreate(suid, ADD_VALUE, type, key, nKey, val, len); + taosMemoryFree(val); } else if (pTagVal->nData == 0) { term = indexTermCreate(suid, ADD_VALUE, TSDB_DATA_TYPE_VARCHAR, key, nKey, pTagVal->pData, 0); } @@ -115,6 +116,7 @@ static int metaSaveJsonVarToIdx(SMeta *pMeta, const SMetaEntry *pCtbEntry, const indexMultiTermAdd(terms, term); } } + taosArrayDestroy(pTagVals); indexJsonPut(pMeta->pTagIvtIdx, terms, tuid); indexMultiTermDestroy(terms); #endif @@ -367,7 +369,7 @@ int metaAlterSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) { return 0; } -int metaCreateTable(SMeta *pMeta, int64_t version, SVCreateTbReq *pReq) { +int metaCreateTable(SMeta *pMeta, int64_t version, SVCreateTbReq *pReq, STableMetaRsp **pMetaRsp) { SMetaEntry me = {0}; SMetaReader mr = {0}; @@ -413,6 +415,25 @@ int metaCreateTable(SMeta *pMeta, int64_t version, SVCreateTbReq *pReq) { me.ctbEntry.suid = pReq->ctb.suid; me.ctbEntry.pTags = pReq->ctb.pTag; +#ifdef TAG_FILTER_DEBUG + SArray* pTagVals = NULL; + int32_t code = tTagToValArray((STag*)pReq->ctb.pTag, &pTagVals); + for (int i = 0; i < taosArrayGetSize(pTagVals); i++) { + STagVal* pTagVal = (STagVal*)taosArrayGet(pTagVals, i); + + if (IS_VAR_DATA_TYPE(pTagVal->type)) { + char* buf = taosMemoryCalloc(pTagVal->nData + 1, 1); + memcpy(buf, pTagVal->pData, pTagVal->nData); + metaDebug("metaTag table:%s varchar index:%d cid:%d type:%d value:%s", pReq->name, i, pTagVal->cid, pTagVal->type, buf); + taosMemoryFree(buf); + } else { + double val = 0; + GET_TYPED_DATA(val, double, pTagVal->type, &pTagVal->i64); + metaDebug("metaTag table:%s number index:%d cid:%d type:%d value:%f", pReq->name, i, pTagVal->cid, pTagVal->type, val); + } + } +#endif + ++pMeta->pVnode->config.vndStats.numOfCTables; } else { me.ntbEntry.ctime = pReq->ctime; @@ -423,10 +444,26 @@ int metaCreateTable(SMeta *pMeta, int64_t version, SVCreateTbReq *pReq) { me.ntbEntry.ncid = me.ntbEntry.schemaRow.pSchema[me.ntbEntry.schemaRow.nCols - 1].colId + 1; ++pMeta->pVnode->config.vndStats.numOfNTables; + pMeta->pVnode->config.vndStats.numOfNTimeSeries += me.ntbEntry.schemaRow.nCols - 1; } if (metaHandleEntry(pMeta, &me) < 0) goto _err; + if (pMetaRsp) { + *pMetaRsp = taosMemoryCalloc(1, sizeof(STableMetaRsp)); + + if (*pMetaRsp) { + if (me.type == TSDB_CHILD_TABLE) { + (*pMetaRsp)->tableType = TSDB_CHILD_TABLE; + (*pMetaRsp)->tuid = pReq->uid; + (*pMetaRsp)->suid = pReq->ctb.suid; + strcpy((*pMetaRsp)->tbName, pReq->name); + } else { + metaUpdateMetaRsp(pReq->uid, pReq->name, &pReq->ntb.schemaRow, *pMetaRsp); + } + } + } + metaDebug("vgId:%d, table:%s uid %" PRId64 " is created, type:%" PRId8, TD_VID(pMeta->pVnode), pReq->name, pReq->uid, pReq->type); return 0; @@ -516,6 +553,9 @@ static int metaDropTableByUid(SMeta *pMeta, tb_uid_t uid, int *type) { SDecoder dc = {0}; rc = tdbTbGet(pMeta->pUidIdx, &uid, sizeof(uid), &pData, &nData); + if (rc < 0) { + return -1; + } int64_t version = ((SUidIdxVal *)pData)[0].version; tdbTbGet(pMeta->pTbDb, &(STbDbKey){.version = version, .uid = uid}, sizeof(STbDbKey), &pData, &nData); @@ -562,6 +602,7 @@ static int metaDropTableByUid(SMeta *pMeta, tb_uid_t uid, int *type) { // drop schema.db (todo) --pMeta->pVnode->config.vndStats.numOfNTables; + pMeta->pVnode->config.vndStats.numOfNTimeSeries -= e.ntbEntry.schemaRow.nCols - 1; } else if (e.type == TSDB_SUPER_TABLE) { tdbTbDelete(pMeta->pSuidIdx, &e.uid, sizeof(tb_uid_t), &pMeta->txn); // drop schema.db (todo) @@ -664,6 +705,8 @@ static int metaAlterTableColumn(SMeta *pMeta, int64_t version, SVAlterTbReq *pAl pSchema->pSchema[entry.ntbEntry.schemaRow.nCols - 1].flags = pAlterTbReq->flags; pSchema->pSchema[entry.ntbEntry.schemaRow.nCols - 1].colId = entry.ntbEntry.ncid++; strcpy(pSchema->pSchema[entry.ntbEntry.schemaRow.nCols - 1].name, pAlterTbReq->colName); + + ++pMeta->pVnode->config.vndStats.numOfNTimeSeries; break; case TSDB_ALTER_TABLE_DROP_COLUMN: if (pColumn == NULL) { @@ -684,6 +727,8 @@ static int metaAlterTableColumn(SMeta *pMeta, int64_t version, SVAlterTbReq *pAl memmove(pColumn, pColumn + 1, tlen); } pSchema->nCols--; + + --pMeta->pVnode->config.vndStats.numOfNTimeSeries; break; case TSDB_ALTER_TABLE_UPDATE_COLUMN_BYTES: if (pColumn == NULL) { diff --git a/source/dnode/vnode/src/tq/tq.c b/source/dnode/vnode/src/tq/tq.c index 3ff59ac2c0..54f764c6b3 100644 --- a/source/dnode/vnode/src/tq/tq.c +++ b/source/dnode/vnode/src/tq/tq.c @@ -100,7 +100,13 @@ void tqClose(STQ* pTq) { } int32_t tqSendMetaPollRsp(STQ* pTq, const SRpcMsg* pMsg, const SMqPollReq* pReq, const SMqMetaRsp* pRsp) { - int32_t tlen = sizeof(SMqRspHead) + tEncodeSMqMetaRsp(NULL, pRsp); + int32_t len = 0; + int32_t code = 0; + tEncodeSize(tEncodeSMqMetaRsp, pRsp, len, code); + if (code < 0) { + return -1; + } + int32_t tlen = sizeof(SMqRspHead) + len; void* buf = rpcMallocCont(tlen); if (buf == NULL) { return -1; @@ -111,7 +117,11 @@ int32_t tqSendMetaPollRsp(STQ* pTq, const SRpcMsg* pMsg, const SMqPollReq* pReq, ((SMqRspHead*)buf)->consumerId = pReq->consumerId; void* abuf = POINTER_SHIFT(buf, sizeof(SMqRspHead)); - tEncodeSMqMetaRsp(&abuf, pRsp); + + SEncoder encoder = {0}; + tEncoderInit(&encoder, abuf, len); + tEncodeSMqMetaRsp(&encoder, pRsp); + tEncoderClear(&encoder); SRpcMsg resp = { .info = pMsg->info, @@ -121,9 +131,8 @@ int32_t tqSendMetaPollRsp(STQ* pTq, const SRpcMsg* pMsg, const SMqPollReq* pReq, }; tmsgSendRsp(&resp); - tqDebug("vgId:%d, from consumer:%" PRId64 ", (epoch %d) send rsp, res msg type %d, reqOffset:%" PRId64 - ", rspOffset:%" PRId64, - TD_VID(pTq->pVnode), pReq->consumerId, pReq->epoch, pRsp->resMsgType, pRsp->reqOffset, pRsp->rspOffset); + tqDebug("vgId:%d, from consumer:%" PRId64 ", (epoch %d) send rsp, res msg type %d, offset type:%d", + TD_VID(pTq->pVnode), pReq->consumerId, pReq->epoch, pRsp->resMsgType, pRsp->rspOffset.type); return 0; } @@ -202,7 +211,7 @@ int32_t tqProcessOffsetCommitReq(STQ* pTq, int64_t version, char* msg, int32_t m } tDecoderClear(&decoder); - if (offset.val.type == TMQ_OFFSET__SNAPSHOT_DATA) { + if (offset.val.type == TMQ_OFFSET__SNAPSHOT_DATA || offset.val.type == TMQ_OFFSET__SNAPSHOT_META) { tqDebug("receive offset commit msg to %s on vgId:%d, offset(type:snapshot) uid:%" PRId64 ", ts:%" PRId64, offset.subKey, TD_VID(pTq->pVnode), offset.val.uid, offset.val.ts); } else if (offset.val.type == TMQ_OFFSET__LOG) { @@ -297,7 +306,6 @@ static int32_t tqInitDataRsp(SMqDataRsp* pRsp, const SMqPollReq* pReq, int8_t su int32_t tqProcessPollReq(STQ* pTq, SRpcMsg* pMsg) { SMqPollReq* pReq = pMsg->pCont; int64_t consumerId = pReq->consumerId; - int64_t timeout = pReq->timeout; int32_t reqEpoch = pReq->epoch; int32_t code = 0; STqOffsetVal reqOffset = pReq->reqOffset; @@ -349,12 +357,11 @@ int32_t tqProcessPollReq(STQ* pTq, SRpcMsg* pMsg) { TD_VID(pTq->pVnode), formatBuf); } else { if (reqOffset.type == TMQ_OFFSET__RESET_EARLIEAST) { - if (pReq->useSnapshot && pHandle->execHandle.subType == TOPIC_SUB_TYPE__COLUMN) { - if (!pHandle->fetchMeta) { - tqOffsetResetToData(&fetchOffsetNew, 0, 0); + if (pReq->useSnapshot){ + if (pHandle->fetchMeta){ + tqOffsetResetToMeta(&fetchOffsetNew, 0); } else { - // reset to meta - ASSERT(0); + tqOffsetResetToData(&fetchOffsetNew, 0, 0); } } else { tqOffsetResetToLog(&fetchOffsetNew, walGetFirstVer(pTq->pVnode->pWal)); @@ -378,28 +385,34 @@ int32_t tqProcessPollReq(STQ* pTq, SRpcMsg* pMsg) { } } - // 3.query - if (pHandle->execHandle.subType == TOPIC_SUB_TYPE__COLUMN) { - /*if (fetchOffsetNew.type == TMQ_OFFSET__LOG) {*/ - /*fetchOffsetNew.version++;*/ - /*}*/ - if (tqScan(pTq, pHandle, &dataRsp, &fetchOffsetNew) < 0) { - ASSERT(0); - code = -1; + if(pHandle->execHandle.subType == TOPIC_SUB_TYPE__COLUMN || fetchOffsetNew.type != TMQ_OFFSET__LOG){ + SMqMetaRsp metaRsp = {0}; + tqScan(pTq, pHandle, &dataRsp, &metaRsp, &fetchOffsetNew); + + if(metaRsp.metaRspLen > 0){ + if (tqSendMetaPollRsp(pTq, pMsg, pReq, &metaRsp) < 0) { + code = -1; + } + tqDebug("tmq poll: consumer %ld, subkey %s, vg %d, send meta offset type:%d,uid:%ld,version:%ld", consumerId, pHandle->subKey, + TD_VID(pTq->pVnode), metaRsp.rspOffset.type, metaRsp.rspOffset.uid, metaRsp.rspOffset.version); + taosMemoryFree(metaRsp.metaRsp); goto OVER; } - if (dataRsp.blockNum == 0) { - // TODO add to async task pool - /*dataRsp.rspOffset.version--;*/ + + if (dataRsp.blockNum > 0){ + if (tqSendDataRsp(pTq, pMsg, pReq, &dataRsp) < 0) { + code = -1; + } + goto OVER; + }else{ + fetchOffsetNew = dataRsp.rspOffset; } - if (tqSendDataRsp(pTq, pMsg, pReq, &dataRsp) < 0) { - code = -1; - } - goto OVER; + + tqDebug("tmq poll: consumer %ld, subkey %s, vg %d, send data blockNum:%d, offset type:%d,uid:%ld,version:%ld", consumerId, pHandle->subKey, + TD_VID(pTq->pVnode), dataRsp.blockNum, dataRsp.rspOffset.type, dataRsp.rspOffset.uid, dataRsp.rspOffset.version); } - if (pHandle->execHandle.subType != TOPIC_SUB_TYPE__COLUMN) { - ASSERT(fetchOffsetNew.type == TMQ_OFFSET__LOG); + if (pHandle->execHandle.subType != TOPIC_SUB_TYPE__COLUMN && fetchOffsetNew.type == TMQ_OFFSET__LOG) { int64_t fetchVer = fetchOffsetNew.version + 1; pCkHead = taosMemoryMalloc(sizeof(SWalCkHead) + 2048); if (pCkHead == NULL) { @@ -413,7 +426,7 @@ int32_t tqProcessPollReq(STQ* pTq, SRpcMsg* pMsg) { consumerEpoch = atomic_load_32(&pHandle->epoch); if (consumerEpoch > reqEpoch) { tqWarn("tmq poll: consumer %" PRId64 " (epoch %d), subkey %s, vg %d offset %" PRId64 - ", found new consumer epoch %d, discard req epoch %d", + ", found new consumer epoch %d, discard req epoch %d", consumerId, pReq->epoch, pHandle->subKey, TD_VID(pTq->pVnode), fetchVer, consumerEpoch, reqEpoch); break; } @@ -422,7 +435,6 @@ int32_t tqProcessPollReq(STQ* pTq, SRpcMsg* pMsg) { // TODO add push mgr tqOffsetResetToLog(&dataRsp.rspOffset, fetchVer); - ASSERT(dataRsp.rspOffset.version >= dataRsp.reqOffset.version); if (tqSendDataRsp(pTq, pMsg, pReq, &dataRsp) < 0) { code = -1; } @@ -444,8 +456,6 @@ int32_t tqProcessPollReq(STQ* pTq, SRpcMsg* pMsg) { // TODO continue scan until meeting batch requirement if (dataRsp.blockNum > 0 /* threshold */) { tqOffsetResetToLog(&dataRsp.rspOffset, fetchVer); - ASSERT(dataRsp.rspOffset.version >= dataRsp.reqOffset.version); - if (tqSendDataRsp(pTq, pMsg, pReq, &dataRsp) < 0) { code = -1; } @@ -459,11 +469,7 @@ int32_t tqProcessPollReq(STQ* pTq, SRpcMsg* pMsg) { ASSERT(IS_META_MSG(pHead->msgType)); tqDebug("fetch meta msg, ver:%" PRId64 ", type:%d", pHead->version, pHead->msgType); SMqMetaRsp metaRsp = {0}; - /*metaRsp.reqOffset = pReq->reqOffset.version;*/ - metaRsp.rspOffset = fetchVer; - /*metaRsp.rspOffsetNew.version = fetchVer;*/ - tqOffsetResetToLog(&metaRsp.reqOffsetNew, pReq->reqOffset.version); - tqOffsetResetToLog(&metaRsp.rspOffsetNew, fetchVer); + tqOffsetResetToLog(&metaRsp.rspOffset, fetchVer); metaRsp.resMsgType = pHead->msgType; metaRsp.metaRspLen = pHead->bodyLen; metaRsp.metaRsp = pHead->body; @@ -477,6 +483,11 @@ int32_t tqProcessPollReq(STQ* pTq, SRpcMsg* pMsg) { } } + // send empty to client + if (tqSendDataRsp(pTq, pMsg, pReq, &dataRsp) < 0) { + code = -1; + } + OVER: if (pCkHead) taosMemoryFree(pCkHead); // TODO wrap in destroy func @@ -561,6 +572,7 @@ int32_t tqProcessVgChangeReq(STQ* pTq, int64_t version, char* msg, int32_t msgLe pHandle->execHandle.subType = req.subType; pHandle->fetchMeta = req.withMeta; + // TODO version should be assigned and refed during preprocess SWalRef* pRef = walRefCommittedVer(pTq->pVnode->pWal); if (pRef == NULL) { @@ -570,36 +582,42 @@ int32_t tqProcessVgChangeReq(STQ* pTq, int64_t version, char* msg, int32_t msgLe int64_t ver = pRef->refVer; pHandle->pRef = pRef; + SReadHandle handle = { + .meta = pTq->pVnode->pMeta, + .vnode = pTq->pVnode, + .initTableReader = true, + .initTqReader = true, + .version = ver, + }; + pHandle->snapshotVer = ver; + if (pHandle->execHandle.subType == TOPIC_SUB_TYPE__COLUMN) { pHandle->execHandle.execCol.qmsg = req.qmsg; - pHandle->snapshotVer = ver; req.qmsg = NULL; - SReadHandle handle = { - .meta = pTq->pVnode->pMeta, - .vnode = pTq->pVnode, - .initTableReader = true, - .initTqReader = true, - .version = ver, - }; - pHandle->execHandle.execCol.task = + + pHandle->execHandle.task = qCreateQueueExecTaskInfo(pHandle->execHandle.execCol.qmsg, &handle, &pHandle->execHandle.numOfCols, &pHandle->execHandle.pSchemaWrapper); - ASSERT(pHandle->execHandle.execCol.task); + ASSERT(pHandle->execHandle.task); void* scanner = NULL; - qExtractStreamScanner(pHandle->execHandle.execCol.task, &scanner); + qExtractStreamScanner(pHandle->execHandle.task, &scanner); ASSERT(scanner); pHandle->execHandle.pExecReader = qExtractReaderFromStreamScanner(scanner); ASSERT(pHandle->execHandle.pExecReader); } else if (pHandle->execHandle.subType == TOPIC_SUB_TYPE__DB) { pHandle->pWalReader = walOpenReader(pTq->pVnode->pWal, NULL); - pHandle->execHandle.pExecReader = tqOpenReader(pTq->pVnode); pHandle->execHandle.execDb.pFilterOutTbUid = taosHashInit(64, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), false, HASH_NO_LOCK); + buildSnapContext(handle.meta, handle.version, 0, pHandle->execHandle.subType, pHandle->fetchMeta, (SSnapContext **)(&handle.sContext)); + + pHandle->execHandle.task = + qCreateQueueExecTaskInfo(NULL, &handle, NULL, NULL); } else if (pHandle->execHandle.subType == TOPIC_SUB_TYPE__TABLE) { pHandle->pWalReader = walOpenReader(pTq->pVnode->pWal, NULL); pHandle->execHandle.execTb.suid = req.suid; + SArray* tbUidList = taosArrayInit(0, sizeof(int64_t)); vnodeGetCtbIdList(pTq->pVnode, req.suid, tbUidList); tqDebug("vgId:%d, tq try to get all ctb, suid:%" PRId64, pTq->pVnode->config.vgId, req.suid); @@ -610,6 +628,10 @@ int32_t tqProcessVgChangeReq(STQ* pTq, int64_t version, char* msg, int32_t msgLe pHandle->execHandle.pExecReader = tqOpenReader(pTq->pVnode); tqReaderSetTbUidList(pHandle->execHandle.pExecReader, tbUidList); taosArrayDestroy(tbUidList); + + buildSnapContext(handle.meta, handle.version, req.suid, pHandle->execHandle.subType, pHandle->fetchMeta, (SSnapContext **)(&handle.sContext)); + pHandle->execHandle.task = + qCreateQueueExecTaskInfo(NULL, &handle, NULL, NULL); } taosHashPut(pTq->pHandle, req.subKey, strlen(req.subKey), pHandle, sizeof(STqHandle)); tqDebug("try to persist handle %s consumer %" PRId64, req.subKey, pHandle->consumerId); diff --git a/source/dnode/vnode/src/tq/tqExec.c b/source/dnode/vnode/src/tq/tqExec.c index 435bbb77b8..a0b8141cfb 100644 --- a/source/dnode/vnode/src/tq/tqExec.c +++ b/source/dnode/vnode/src/tq/tqExec.c @@ -60,18 +60,18 @@ static int32_t tqAddTbNameToRsp(const STQ* pTq, int64_t uid, SMqDataRsp* pRsp) { return 0; } -int64_t tqScan(STQ* pTq, const STqHandle* pHandle, SMqDataRsp* pRsp, STqOffsetVal* pOffset) { +int64_t tqScan(STQ* pTq, const STqHandle* pHandle, SMqDataRsp* pRsp, SMqMetaRsp* pMetaRsp, STqOffsetVal* pOffset) { const STqExecHandle* pExec = &pHandle->execHandle; - qTaskInfo_t task = pExec->execCol.task; + qTaskInfo_t task = pExec->task; - if (qStreamPrepareScan(task, pOffset) < 0) { + if (qStreamPrepareScan(task, pOffset, pHandle->execHandle.subType) < 0) { tqDebug("prepare scan failed, return"); if (pOffset->type == TMQ_OFFSET__LOG) { pRsp->rspOffset = *pOffset; return 0; } else { tqOffsetResetToLog(pOffset, pHandle->snapshotVer); - if (qStreamPrepareScan(task, pOffset) < 0) { + if (qStreamPrepareScan(task, pOffset, pHandle->execHandle.subType) < 0) { tqDebug("prepare scan failed, return"); pRsp->rspOffset = *pOffset; return 0; @@ -83,24 +83,39 @@ int64_t tqScan(STQ* pTq, const STqHandle* pHandle, SMqDataRsp* pRsp, STqOffsetVa while (1) { SSDataBlock* pDataBlock = NULL; uint64_t ts = 0; - tqDebug("task start to execute"); + tqDebug("tmqsnap task start to execute"); if (qExecTask(task, &pDataBlock, &ts) < 0) { ASSERT(0); } - tqDebug("task execute end, get %p", pDataBlock); + tqDebug("tmqsnap task execute end, get %p", pDataBlock); if (pDataBlock != NULL) { if (pRsp->withTbName) { + int64_t uid = 0; if (pOffset->type == TMQ_OFFSET__LOG) { - int64_t uid = pExec->pExecReader->msgIter.uid; + uid = pExec->pExecReader->msgIter.uid; if (tqAddTbNameToRsp(pTq, uid, pRsp) < 0) { continue; } } else { - pRsp->withTbName = 0; + char* tbName = strdup(qExtractTbnameFromTask(task)); + taosArrayPush(pRsp->blockTbName, &tbName); } } - tqAddBlockDataToRsp(pDataBlock, pRsp, pExec->numOfCols); + if(pRsp->withSchema){ + if (pOffset->type == TMQ_OFFSET__LOG) { + tqAddBlockSchemaToRsp(pExec, pRsp); + }else{ + SSchemaWrapper* pSW = tCloneSSchemaWrapper(qExtractSchemaFromTask(task)); + taosArrayPush(pRsp->blockSchema, &pSW); + } + } + + if(pHandle->execHandle.subType == TOPIC_SUB_TYPE__COLUMN){ + tqAddBlockDataToRsp(pDataBlock, pRsp, pExec->numOfCols); + }else{ + tqAddBlockDataToRsp(pDataBlock, pRsp, taosArrayGetSize(pDataBlock->pDataBlock)); + } pRsp->blockNum++; if (pOffset->type == TMQ_OFFSET__LOG) { continue; @@ -110,39 +125,51 @@ int64_t tqScan(STQ* pTq, const STqHandle* pHandle, SMqDataRsp* pRsp, STqOffsetVa } } - if (pRsp->blockNum == 0 && pOffset->type == TMQ_OFFSET__SNAPSHOT_DATA) { - tqDebug("vgId: %d, tsdb consume over, switch to wal, ver %" PRId64, TD_VID(pTq->pVnode), - pHandle->snapshotVer + 1); - tqOffsetResetToLog(pOffset, pHandle->snapshotVer); - qStreamPrepareScan(task, pOffset); - continue; - } - - void* meta = qStreamExtractMetaMsg(task); - if (meta != NULL) { - // tq add meta to rsp - } - - if (qStreamExtractOffset(task, &pRsp->rspOffset) < 0) { - ASSERT(0); - } - - ASSERT(pRsp->rspOffset.type != 0); - -#if 0 - if (pRsp->reqOffset.type == TMQ_OFFSET__LOG) { - if (pRsp->blockNum > 0) { - ASSERT(pRsp->rspOffset.version > pRsp->reqOffset.version); - } else { - ASSERT(pRsp->rspOffset.version >= pRsp->reqOffset.version); + if(pHandle->execHandle.subType == TOPIC_SUB_TYPE__COLUMN){ + if (pRsp->blockNum == 0 && pOffset->type == TMQ_OFFSET__SNAPSHOT_DATA) { + tqDebug("vgId: %d, tsdb consume over, switch to wal, ver %" PRId64, TD_VID(pTq->pVnode), + pHandle->snapshotVer + 1); + tqOffsetResetToLog(pOffset, pHandle->snapshotVer); + qStreamPrepareScan(task, pOffset, pHandle->execHandle.subType); + continue; } + }else{ + if (pDataBlock == NULL && pOffset->type == TMQ_OFFSET__SNAPSHOT_DATA){ + if(qStreamExtractPrepareUid(task) != 0){ + continue; + } + tqDebug("tmqsnap vgId: %d, tsdb consume over, switch to wal, ver %" PRId64, TD_VID(pTq->pVnode), + pHandle->snapshotVer + 1); + break; + } + + if (pRsp->blockNum > 0){ + tqDebug("tmqsnap task exec exited, get data"); + break; + } + + SMqMetaRsp* tmp = qStreamExtractMetaMsg(task); + if(tmp->rspOffset.type == TMQ_OFFSET__SNAPSHOT_DATA){ + tqOffsetResetToData(pOffset, tmp->rspOffset.uid, tmp->rspOffset.ts); + qStreamPrepareScan(task, pOffset, pHandle->execHandle.subType); + tmp->rspOffset.type = TMQ_OFFSET__SNAPSHOT_META; + tqDebug("tmqsnap task exec change to get data"); + continue; + } + + *pMetaRsp = *tmp; + tqDebug("tmqsnap task exec exited, get meta"); } -#endif tqDebug("task exec exited"); break; } + if (qStreamExtractOffset(task, &pRsp->rspOffset) < 0) { + ASSERT(0); + } + + ASSERT(pRsp->rspOffset.type != 0); return 0; } diff --git a/source/dnode/vnode/src/tq/tqMeta.c b/source/dnode/vnode/src/tq/tqMeta.c index 405bc669bd..a192d1f863 100644 --- a/source/dnode/vnode/src/tq/tqMeta.c +++ b/source/dnode/vnode/src/tq/tqMeta.c @@ -249,27 +249,34 @@ int32_t tqMetaRestoreHandle(STQ* pTq) { } walRefVer(handle.pRef, handle.snapshotVer); - if (handle.execHandle.subType == TOPIC_SUB_TYPE__COLUMN) { - SReadHandle reader = { - .meta = pTq->pVnode->pMeta, - .vnode = pTq->pVnode, - .initTableReader = true, - .initTqReader = true, - .version = handle.snapshotVer, - }; + SReadHandle reader = { + .meta = pTq->pVnode->pMeta, + .vnode = pTq->pVnode, + .initTableReader = true, + .initTqReader = true, + .version = handle.snapshotVer, + }; - handle.execHandle.execCol.task = qCreateQueueExecTaskInfo( + if (handle.execHandle.subType == TOPIC_SUB_TYPE__COLUMN) { + + handle.execHandle.task = qCreateQueueExecTaskInfo( handle.execHandle.execCol.qmsg, &reader, &handle.execHandle.numOfCols, &handle.execHandle.pSchemaWrapper); - ASSERT(handle.execHandle.execCol.task); + ASSERT(handle.execHandle.task); void* scanner = NULL; - qExtractStreamScanner(handle.execHandle.execCol.task, &scanner); + qExtractStreamScanner(handle.execHandle.task, &scanner); ASSERT(scanner); handle.execHandle.pExecReader = qExtractReaderFromStreamScanner(scanner); ASSERT(handle.execHandle.pExecReader); } else { + handle.pWalReader = walOpenReader(pTq->pVnode->pWal, NULL); handle.execHandle.execDb.pFilterOutTbUid = taosHashInit(64, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), false, HASH_NO_LOCK); +// handle.execHandle.pExecReader = tqOpenReader(pTq->pVnode); + buildSnapContext(reader.meta, reader.version, 0, handle.execHandle.subType, handle.fetchMeta, (SSnapContext **)(&reader.sContext)); + + handle.execHandle.task = + qCreateQueueExecTaskInfo(NULL, &reader, NULL, NULL); } tqDebug("tq restore %s consumer %" PRId64 " vgId:%d", handle.subKey, handle.consumerId, TD_VID(pTq->pVnode)); taosHashPut(pTq->pHandle, pKey, kLen, &handle, sizeof(STqHandle)); diff --git a/source/dnode/vnode/src/tq/tqRead.c b/source/dnode/vnode/src/tq/tqRead.c index e6a331f20e..6e2a6fdb71 100644 --- a/source/dnode/vnode/src/tq/tqRead.c +++ b/source/dnode/vnode/src/tq/tqRead.c @@ -68,7 +68,7 @@ int64_t tqFetchLog(STQ* pTq, STqHandle* pHandle, int64_t* fetchOffset, SWalCkHea offset++; } } -END: + END: taosThreadMutexUnlock(&pHandle->pWalReader->mutex); return code; } @@ -398,7 +398,7 @@ int32_t tqUpdateTbUidList(STQ* pTq, const SArray* tbUidList, bool isAdd) { if (pIter == NULL) break; STqHandle* pExec = (STqHandle*)pIter; if (pExec->execHandle.subType == TOPIC_SUB_TYPE__COLUMN) { - int32_t code = qUpdateQualifiedTableId(pExec->execHandle.execCol.task, tbUidList, isAdd); + int32_t code = qUpdateQualifiedTableId(pExec->execHandle.task, tbUidList, isAdd); ASSERT(code == 0); } else if (pExec->execHandle.subType == TOPIC_SUB_TYPE__DB) { if (!isAdd) { diff --git a/source/dnode/vnode/src/tsdb/tsdbCache.c b/source/dnode/vnode/src/tsdb/tsdbCache.c index b9f3897674..61c6877555 100644 --- a/source/dnode/vnode/src/tsdb/tsdbCache.c +++ b/source/dnode/vnode/src/tsdb/tsdbCache.c @@ -476,7 +476,7 @@ static int32_t getNextRowFromFSLast(void *iter, TSDBROW **ppRow) { if (code) goto _err; if (!state->aBlockL) { - state->aBlockL = taosArrayInit(0, sizeof(SBlockIdx)); + state->aBlockL = taosArrayInit(0, sizeof(SBlockL)); } else { taosArrayClear(state->aBlockL); } diff --git a/source/dnode/vnode/src/tsdb/tsdbCommit.c b/source/dnode/vnode/src/tsdb/tsdbCommit.c index 020f3b0bc6..04a6de8472 100644 --- a/source/dnode/vnode/src/tsdb/tsdbCommit.c +++ b/source/dnode/vnode/src/tsdb/tsdbCommit.c @@ -835,6 +835,9 @@ static int32_t tsdbMergeCommitLast(SCommitter *pCommitter, STbDataIter *pIter) { // set block data schema if need if (pBlockData->suid == 0 && pBlockData->uid == 0) { + code = tsdbCommitterUpdateTableSchema(pCommitter, pTbData->suid, pTbData->uid); + if (code) goto _err; + code = tBlockDataInit(pBlockData, pTbData->suid, pTbData->suid ? 0 : pTbData->uid, pCommitter->skmTable.pTSchema); if (code) goto _err; @@ -963,7 +966,20 @@ static int32_t tsdbCommitTableData(SCommitter *pCommitter, STbData *pTbData) { pRow = NULL; } - if (pRow == NULL) goto _exit; + if (pRow == NULL) { + if (pCommitter->dReader.pBlockIdx && tTABLEIDCmprFn(pCommitter->dReader.pBlockIdx, pTbData) == 0) { + SBlockIdx blockIdx = {.suid = pTbData->suid, .uid = pTbData->uid}; + code = tsdbWriteBlock(pCommitter->dWriter.pWriter, &pCommitter->dReader.mBlock, &blockIdx); + if (code) goto _err; + + if (taosArrayPush(pCommitter->dWriter.aBlockIdx, &blockIdx) == NULL) { + code = TSDB_CODE_OUT_OF_MEMORY; + goto _err; + } + } + + goto _exit; + } int32_t iBlock = 0; SBlock block; diff --git a/source/dnode/vnode/src/tsdb/tsdbRead.c b/source/dnode/vnode/src/tsdb/tsdbRead.c index be8e809a72..a92e8189a1 100644 --- a/source/dnode/vnode/src/tsdb/tsdbRead.c +++ b/source/dnode/vnode/src/tsdb/tsdbRead.c @@ -3349,10 +3349,10 @@ int32_t tsdbReaderOpen(SVnode* pVnode, SQueryTableDataCond* pCond, SArray* pTabl } if (pCond->suid != 0) { - pReader->pSchema = metaGetTbTSchema(pReader->pTsdb->pVnode->pMeta, pReader->suid, -1); + pReader->pSchema = metaGetTbTSchema(pReader->pTsdb->pVnode->pMeta, pReader->suid, pCond->schemaVersion); } else if (taosArrayGetSize(pTableList) > 0) { STableKeyInfo* pKey = taosArrayGet(pTableList, 0); - pReader->pSchema = metaGetTbTSchema(pReader->pTsdb->pVnode->pMeta, pKey->uid, -1); + pReader->pSchema = metaGetTbTSchema(pReader->pTsdb->pVnode->pMeta, pKey->uid, pCond->schemaVersion); } int32_t numOfTables = taosArrayGetSize(pTableList); diff --git a/source/dnode/vnode/src/vnd/vnodeCfg.c b/source/dnode/vnode/src/vnd/vnodeCfg.c index 4418ce20e8..580ab8bc93 100644 --- a/source/dnode/vnode/src/vnd/vnodeCfg.c +++ b/source/dnode/vnode/src/vnd/vnodeCfg.c @@ -117,6 +117,7 @@ int vnodeEncodeConfig(const void *pObj, SJson *pJson) { if (tjsonAddIntegerToObject(pJson, "vndStats.ctables", pCfg->vndStats.numOfCTables) < 0) return -1; if (tjsonAddIntegerToObject(pJson, "vndStats.ntables", pCfg->vndStats.numOfNTables) < 0) return -1; if (tjsonAddIntegerToObject(pJson, "vndStats.timeseries", pCfg->vndStats.numOfTimeSeries) < 0) return -1; + if (tjsonAddIntegerToObject(pJson, "vndStats.ntimeseries", pCfg->vndStats.numOfNTimeSeries) < 0) return -1; SJson *pNodeInfoArr = tjsonCreateArray(); tjsonAddItemToObject(pJson, "syncCfg.nodeInfo", pNodeInfoArr); @@ -224,6 +225,8 @@ int vnodeDecodeConfig(const SJson *pJson, void *pObj) { if (code < 0) return -1; tjsonGetNumberValue(pJson, "vndStats.timeseries", pCfg->vndStats.numOfTimeSeries, code); if (code < 0) return -1; + tjsonGetNumberValue(pJson, "vndStats.ntimeseries", pCfg->vndStats.numOfNTimeSeries, code); + if (code < 0) return -1; SJson *pNodeInfoArr = tjsonGetObjectItem(pJson, "syncCfg.nodeInfo"); int arraySize = tjsonGetArraySize(pNodeInfoArr); diff --git a/source/dnode/vnode/src/vnd/vnodeSvr.c b/source/dnode/vnode/src/vnd/vnodeSvr.c index 495220b5de..85feecff1a 100644 --- a/source/dnode/vnode/src/vnd/vnodeSvr.c +++ b/source/dnode/vnode/src/vnd/vnodeSvr.c @@ -368,6 +368,10 @@ void smaHandleRes(void *pVnode, int64_t smaId, const SArray *data) { } void vnodeUpdateMetaRsp(SVnode *pVnode, STableMetaRsp *pMetaRsp) { + if (NULL == pMetaRsp) { + return; + } + strcpy(pMetaRsp->dbFName, pVnode->config.dbname); pMetaRsp->dbId = pVnode->config.dbId; pMetaRsp->vgId = TD_VID(pVnode); @@ -512,7 +516,7 @@ static int32_t vnodeProcessCreateTbReq(SVnode *pVnode, int64_t version, void *pR } // do create table - if (metaCreateTable(pVnode->pMeta, version, pCreateReq) < 0) { + if (metaCreateTable(pVnode->pMeta, version, pCreateReq, &cRsp.pMeta) < 0) { if (pCreateReq->flags & TD_CREATE_IF_NOT_EXISTS && terrno == TSDB_CODE_TDB_TABLE_ALREADY_EXIST) { cRsp.code = TSDB_CODE_SUCCESS; } else { @@ -522,6 +526,7 @@ static int32_t vnodeProcessCreateTbReq(SVnode *pVnode, int64_t version, void *pR cRsp.code = TSDB_CODE_SUCCESS; tdFetchTbUidList(pVnode->pSma, &pStore, pCreateReq->ctb.suid, pCreateReq->uid); taosArrayPush(tbUids, &pCreateReq->uid); + vnodeUpdateMetaRsp(pVnode, cRsp.pMeta); } taosArrayPush(rsp.pArray, &cRsp); @@ -550,7 +555,7 @@ _exit: pCreateReq = req.pReqs + iReq; taosArrayDestroy(pCreateReq->ctb.tagName); } - taosArrayDestroy(rsp.pArray); + taosArrayDestroyEx(rsp.pArray, tFreeSVCreateTbRsp); taosArrayDestroy(tbUids); tDecoderClear(&decoder); tEncoderClear(&encoder); @@ -862,7 +867,7 @@ static int32_t vnodeProcessSubmitReq(SVnode *pVnode, int64_t version, void *pReq goto _exit; } - if (metaCreateTable(pVnode->pMeta, version, &createTbReq) < 0) { + if (metaCreateTable(pVnode->pMeta, version, &createTbReq, &submitBlkRsp.pMeta) < 0) { if (terrno != TSDB_CODE_TDB_TABLE_ALREADY_EXIST) { submitBlkRsp.code = terrno; pRsp->code = terrno; @@ -870,6 +875,10 @@ static int32_t vnodeProcessSubmitReq(SVnode *pVnode, int64_t version, void *pReq taosArrayDestroy(createTbReq.ctb.tagName); goto _exit; } + } else { + if (NULL != submitBlkRsp.pMeta) { + vnodeUpdateMetaRsp(pVnode, submitBlkRsp.pMeta); + } } taosArrayPush(newTbUids, &createTbReq.uid); @@ -913,11 +922,7 @@ _exit: tEncodeSSubmitRsp(&encoder, &submitRsp); tEncoderClear(&encoder); - for (int32_t i = 0; i < taosArrayGetSize(submitRsp.pArray); i++) { - taosMemoryFree(((SSubmitBlkRsp *)taosArrayGet(submitRsp.pArray, i))[0].tblFName); - } - - taosArrayDestroy(submitRsp.pArray); + taosArrayDestroyEx(submitRsp.pArray, tFreeSSubmitBlkRsp); // TODO: the partial success scenario and the error case // => If partial success, extract the success submitted rows and reconstruct a new submit msg, and push to level diff --git a/source/libs/catalog/inc/catalogInt.h b/source/libs/catalog/inc/catalogInt.h index 777dcd0592..9b62581051 100644 --- a/source/libs/catalog/inc/catalogInt.h +++ b/source/libs/catalog/inc/catalogInt.h @@ -188,7 +188,7 @@ typedef struct SCtgTbCache { typedef struct SCtgVgCache { SRWLatch vgLock; - SDBVgInfo *vgInfo; + SDBVgInfo *vgInfo; } SCtgVgCache; typedef struct SCtgDBCache { @@ -224,7 +224,7 @@ typedef struct SCtgUserAuth { } SCtgUserAuth; typedef struct SCatalog { - uint64_t clusterId; + uint64_t clusterId; SHashObj *userCache; //key:user, value:SCtgUserAuth SHashObj *dbCache; //key:dbname, value:SCtgDBCache SCtgRentMgmt dbRent; @@ -253,9 +253,9 @@ typedef struct SCtgJob { int32_t jobResCode; int32_t taskIdx; SRWLatch taskLock; - + uint64_t queryId; - SCatalog* pCtg; + SCatalog* pCtg; SRequestConnInfo conn; void* userParam; catalogCallback userFp; @@ -279,7 +279,7 @@ typedef struct SCtgMsgCtx { void* lastOut; void* out; char* target; - SHashObj* pBatchs; + SHashObj* pBatchs; } SCtgMsgCtx; @@ -364,7 +364,7 @@ typedef struct SCtgCacheStat { uint64_t numOfMetaHit; uint64_t numOfMetaMiss; uint64_t numOfIndexHit; - uint64_t numOfIndexMiss; + uint64_t numOfIndexMiss; uint64_t numOfUserHit; uint64_t numOfUserMiss; uint64_t numOfClear; @@ -451,7 +451,7 @@ typedef struct SCtgCacheOperation { int32_t opId; void *data; bool syncOp; - tsem_t rspSem; + tsem_t rspSem; bool stopQueue; bool unLocked; } SCtgCacheOperation; @@ -466,7 +466,7 @@ typedef struct SCtgQueue { bool stopQueue; SCtgQNode *head; SCtgQNode *tail; - tsem_t reqSem; + tsem_t reqSem; uint64_t qRemainNum; } SCtgQueue; @@ -475,7 +475,7 @@ typedef struct SCatalogMgmt { int32_t jobPool; SRWLatch lock; SCtgQueue queue; - TdThread updateThread; + TdThread updateThread; SHashObj *pCluster; //key: clusterId, value: SCatalog* SCatalogStat stat; SCatalogCfg cfg; @@ -528,8 +528,8 @@ typedef struct SCtgOperation { #define CTG_META_SIZE(pMeta) (sizeof(STableMeta) + ((pMeta)->tableInfo.numOfTags + (pMeta)->tableInfo.numOfColumns) * sizeof(SSchema)) -#define CTG_TABLE_NOT_EXIST(code) (code == CTG_ERR_CODE_TABLE_NOT_EXIST) -#define CTG_DB_NOT_EXIST(code) (code == TSDB_CODE_MND_DB_NOT_EXIST) +#define CTG_TABLE_NOT_EXIST(code) (code == CTG_ERR_CODE_TABLE_NOT_EXIST) +#define CTG_DB_NOT_EXIST(code) (code == TSDB_CODE_MND_DB_NOT_EXIST) #define ctgFatal(param, ...) qFatal("CTG:%p " param, pCtg, __VA_ARGS__) #define ctgError(param, ...) qError("CTG:%p " param, pCtg, __VA_ARGS__) @@ -576,7 +576,7 @@ typedef struct SCtgOperation { } \ } while (0) - + #define CTG_ERR_RET(c) do { int32_t _code = c; if (_code != TSDB_CODE_SUCCESS) { terrno = _code; return _code; } } while (0) #define CTG_RET(c) do { int32_t _code = c; if (_code != TSDB_CODE_SUCCESS) { terrno = _code; } return _code; } while (0) #define CTG_ERR_JRET(c) do { code = c; if (code != TSDB_CODE_SUCCESS) { terrno = code; goto _return; } } while (0) diff --git a/source/libs/catalog/src/catalog.c b/source/libs/catalog/src/catalog.c index b6e958e192..7b32eadcd4 100644 --- a/source/libs/catalog/src/catalog.c +++ b/source/libs/catalog/src/catalog.c @@ -270,13 +270,22 @@ int32_t ctgUpdateTbMeta(SCatalog* pCtg, STableMetaRsp* rspMsg, bool syncOp) { int32_t code = 0; strcpy(output->dbFName, rspMsg->dbFName); - strcpy(output->tbName, rspMsg->tbName); output->dbId = rspMsg->dbId; - SET_META_TYPE_TABLE(output->metaType); + if (TSDB_CHILD_TABLE == rspMsg->tableType && NULL == rspMsg->pSchemas) { + strcpy(output->ctbName, rspMsg->tbName); - CTG_ERR_JRET(queryCreateTableMetaFromMsg(rspMsg, rspMsg->tableType == TSDB_SUPER_TABLE, &output->tbMeta)); + SET_META_TYPE_CTABLE(output->metaType); + + CTG_ERR_JRET(queryCreateCTableMetaFromMsg(rspMsg, &output->ctbMeta)); + } else { + strcpy(output->tbName, rspMsg->tbName); + + SET_META_TYPE_TABLE(output->metaType); + + CTG_ERR_JRET(queryCreateTableMetaFromMsg(rspMsg, rspMsg->tableType == TSDB_SUPER_TABLE, &output->tbMeta)); + } CTG_ERR_JRET(ctgUpdateTbMetaEnqueue(pCtg, output, syncOp)); diff --git a/source/libs/catalog/src/ctgAsync.c b/source/libs/catalog/src/ctgAsync.c index 64ca85edf4..585b33930c 100644 --- a/source/libs/catalog/src/ctgAsync.c +++ b/source/libs/catalog/src/ctgAsync.c @@ -39,7 +39,7 @@ int32_t ctgInitGetTbMetaTask(SCtgJob *pJob, int32_t taskIdx, void* param) { taosMemoryFree(task.taskCtx); CTG_ERR_RET(TSDB_CODE_OUT_OF_MEMORY); } - + memcpy(ctx->pName, name, sizeof(*name)); ctx->flag = CTG_FLAG_UNKNOWN_STB; @@ -69,7 +69,7 @@ int32_t ctgInitGetTbMetasTask(SCtgJob *pJob, int32_t taskIdx, void* param) { taosArrayPush(pJob->pTasks, &task); - qDebug("QID:0x%" PRIx64 " the %dth task type %s initialized, dbNum:%d, tbNum:%d", + qDebug("QID:0x%" PRIx64 " the %dth task type %s initialized, dbNum:%d, tbNum:%d", pJob->queryId, taskIdx, ctgTaskTypeStr(task.type), taosArrayGetSize(ctx->pNames), pJob->tbMetaNum); return TSDB_CODE_SUCCESS; @@ -89,7 +89,7 @@ int32_t ctgInitGetDbVgTask(SCtgJob *pJob, int32_t taskIdx, void* param) { } SCtgDbVgCtx* ctx = task.taskCtx; - + memcpy(ctx->dbFName, dbFName, sizeof(ctx->dbFName)); taosArrayPush(pJob->pTasks, &task); @@ -113,7 +113,7 @@ int32_t ctgInitGetDbCfgTask(SCtgJob *pJob, int32_t taskIdx, void* param) { } SCtgDbCfgCtx* ctx = task.taskCtx; - + memcpy(ctx->dbFName, dbFName, sizeof(ctx->dbFName)); taosArrayPush(pJob->pTasks, &task); @@ -137,7 +137,7 @@ int32_t ctgInitGetDbInfoTask(SCtgJob *pJob, int32_t taskIdx, void* param) { } SCtgDbInfoCtx* ctx = task.taskCtx; - + memcpy(ctx->dbFName, dbFName, sizeof(ctx->dbFName)); taosArrayPush(pJob->pTasks, &task); @@ -167,7 +167,7 @@ int32_t ctgInitGetTbHashTask(SCtgJob *pJob, int32_t taskIdx, void* param) { taosMemoryFree(task.taskCtx); CTG_ERR_RET(TSDB_CODE_OUT_OF_MEMORY); } - + memcpy(ctx->pName, name, sizeof(*name)); tNameGetFullDbName(ctx->pName, ctx->dbFName); @@ -197,7 +197,7 @@ int32_t ctgInitGetTbHashsTask(SCtgJob *pJob, int32_t taskIdx, void* param) { taosArrayPush(pJob->pTasks, &task); - qDebug("QID:0x%" PRIx64 " the %dth task type %s initialized, dbNum:%d, tbNum:%d", + qDebug("QID:0x%" PRIx64 " the %dth task type %s initialized, dbNum:%d, tbNum:%d", pJob->queryId, taskIdx, ctgTaskTypeStr(task.type), taosArrayGetSize(ctx->pNames), pJob->tbHashNum); return TSDB_CODE_SUCCESS; @@ -248,7 +248,7 @@ int32_t ctgInitGetIndexTask(SCtgJob *pJob, int32_t taskIdx, void* param) { } SCtgIndexCtx* ctx = task.taskCtx; - + strcpy(ctx->indexFName, name); taosArrayPush(pJob->pTasks, &task); @@ -272,7 +272,7 @@ int32_t ctgInitGetUdfTask(SCtgJob *pJob, int32_t taskIdx, void* param) { } SCtgUdfCtx* ctx = task.taskCtx; - + strcpy(ctx->udfName, name); taosArrayPush(pJob->pTasks, &task); @@ -296,7 +296,7 @@ int32_t ctgInitGetUserTask(SCtgJob *pJob, int32_t taskIdx, void* param) { } SCtgUserCtx* ctx = task.taskCtx; - + memcpy(&ctx->user, user, sizeof(*user)); taosArrayPush(pJob->pTasks, &task); @@ -339,7 +339,7 @@ int32_t ctgInitGetTbIndexTask(SCtgJob *pJob, int32_t taskIdx, void* param) { taosMemoryFree(task.taskCtx); CTG_ERR_RET(TSDB_CODE_OUT_OF_MEMORY); } - + memcpy(ctx->pName, name, sizeof(*name)); taosArrayPush(pJob->pTasks, &task); @@ -368,7 +368,7 @@ int32_t ctgInitGetTbCfgTask(SCtgJob *pJob, int32_t taskIdx, void* param) { taosMemoryFree(task.taskCtx); CTG_ERR_RET(TSDB_CODE_OUT_OF_MEMORY); } - + memcpy(ctx->pName, name, sizeof(*name)); taosArrayPush(pJob->pTasks, &task); @@ -387,7 +387,7 @@ int32_t ctgHandleForceUpdate(SCatalog* pCtg, int32_t taskNum, SCtgJob *pJob, con taosHashCleanup(pTb); CTG_ERR_RET(TSDB_CODE_OUT_OF_MEMORY); } - + for (int32_t i = 0; i < pJob->dbVgNum; ++i) { char* dbFName = taosArrayGet(pReq->pDbVgroup, i); taosHashPut(pDb, dbFName, strlen(dbFName), dbFName, TSDB_DB_FNAME_LEN); @@ -474,7 +474,7 @@ int32_t ctgInitTask(SCtgJob *pJob, CTG_TASK_TYPE type, void* param, int32_t *tas if (taskId) { *taskId = tid; } - + return TSDB_CODE_SUCCESS; } @@ -510,7 +510,7 @@ int32_t ctgInitJob(SCatalog* pCtg, SRequestConnInfo *pConn, SCtgJob** job, const pJob->pCtg = pCtg; pJob->conn = *pConn; pJob->userParam = param; - + pJob->tbMetaNum = tbMetaNum; pJob->tbHashNum = tbHashNum; pJob->qnodeNum = qnodeNum; @@ -844,20 +844,20 @@ int32_t ctgDumpSvrVer(SCtgTask* pTask) { pJob->jobRes.pSvrVer->code = pTask->code; pJob->jobRes.pSvrVer->pRes = pTask->res; - + return TSDB_CODE_SUCCESS; } int32_t ctgCallSubCb(SCtgTask *pTask) { int32_t code = 0; - + CTG_LOCK(CTG_WRITE, &pTask->lock); - + int32_t parentNum = taosArrayGetSize(pTask->pParents); for (int32_t i = 0; i < parentNum; ++i) { SCtgMsgCtx *pMsgCtx = CTG_GET_TASK_MSGCTX(pTask, -1); SCtgTask* pParent = taosArrayGetP(pTask->pParents, i); - + pParent->subRes.code = pTask->code; if (TSDB_CODE_SUCCESS == pTask->code) { code = (*gCtgAsyncFps[pTask->type].cloneFp)(pTask, &pParent->subRes.res); @@ -868,22 +868,22 @@ int32_t ctgCallSubCb(SCtgTask *pTask) { SCtgMsgCtx *pParMsgCtx = CTG_GET_TASK_MSGCTX(pParent, -1); - pParMsgCtx->pBatchs = pMsgCtx->pBatchs; + pParMsgCtx->pBatchs = pMsgCtx->pBatchs; CTG_ERR_JRET(pParent->subRes.fp(pParent)); } - + _return: CTG_UNLOCK(CTG_WRITE, &pTask->lock); - CTG_RET(code); + CTG_RET(code); } int32_t ctgCallUserCb(void* param) { SCtgJob* pJob = (SCtgJob*)param; qDebug("QID:0x%" PRIx64 " ctg start to call user cb with rsp %s", pJob->queryId, tstrerror(pJob->jobResCode)); - + (*pJob->userFp)(&pJob->jobRes, pJob->userParam, pJob->jobResCode); qDebug("QID:0x%" PRIx64 " ctg end to call user cb", pJob->queryId); @@ -922,9 +922,9 @@ _return: //taosSsleep(2); //qDebug("QID:0x%" PRIx64 " ctg after sleep", pJob->queryId); - + taosAsyncExec(ctgCallUserCb, pJob, NULL); - + CTG_RET(code); } @@ -932,7 +932,7 @@ int32_t ctgHandleGetTbMetaRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf int32_t code = 0; SCtgDBCache *dbCache = NULL; SCtgTask* pTask = tReq->pTask; - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SRequestConnInfo* pConn = &pTask->pJob->conn; SCtgMsgCtx* pMsgCtx = CTG_GET_TASK_MSGCTX(pTask, tReq->msgIdx); SCtgTbMetaCtx* ctx = (SCtgTbMetaCtx*)pTask->taskCtx; @@ -958,38 +958,38 @@ int32_t ctgHandleGetTbMetaRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf } case TDMT_MND_TABLE_META: { STableMetaOutput* pOut = (STableMetaOutput*)pMsgCtx->out; - + if (CTG_IS_META_NULL(pOut->metaType)) { if (CTG_FLAG_IS_STB(flag)) { char dbFName[TSDB_DB_FNAME_LEN] = {0}; tNameGetFullDbName(pName, dbFName); - + CTG_ERR_RET(ctgAcquireVgInfoFromCache(pCtg, dbFName, &dbCache)); if (NULL != dbCache) { SVgroupInfo vgInfo = {0}; CTG_ERR_JRET(ctgGetVgInfoFromHashValue(pCtg, dbCache->vgCache.vgInfo, pName, &vgInfo)); - + ctgDebug("will refresh tbmeta, supposed to be stb, tbName:%s, flag:%d", tNameGetTableName(pName), flag); - *vgId = vgInfo.vgId; + *vgId = vgInfo.vgId; CTG_ERR_JRET(ctgGetTbMetaFromVnode(pCtg, pConn, pName, &vgInfo, NULL, tReq)); ctgReleaseVgInfoToCache(pCtg, dbCache); } else { SBuildUseDBInput input = {0}; - + tstrncpy(input.db, dbFName, tListLen(input.db)); input.vgVersion = CTG_DEFAULT_INVALID_VERSION; - + CTG_ERR_JRET(ctgGetDBVgInfoFromMnode(pCtg, pConn, &input, NULL, tReq)); } return TSDB_CODE_SUCCESS; } - + ctgError("no tbmeta got, tbName:%s", tNameGetTableName(pName)); ctgRemoveTbMetaFromCache(pCtg, pName, false); - + CTG_ERR_JRET(CTG_ERR_CODE_TABLE_NOT_EXIST); } @@ -998,12 +998,12 @@ int32_t ctgHandleGetTbMetaRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf STableMetaOutput* pLastOut = (STableMetaOutput*)pMsgCtx->out; TSWAP(pLastOut->tbMeta, pOut->tbMeta); } - + break; } case TDMT_VND_TABLE_META: { STableMetaOutput* pOut = (STableMetaOutput*)pMsgCtx->out; - + if (CTG_IS_META_NULL(pOut->metaType)) { ctgError("no tbmeta got, tbNmae:%s", tNameGetTableName(pName)); ctgRemoveTbMetaFromCache(pCtg, pName, false); @@ -1013,12 +1013,12 @@ int32_t ctgHandleGetTbMetaRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf if (CTG_FLAG_IS_STB(flag)) { break; } - + if (CTG_IS_META_TABLE(pOut->metaType) && TSDB_SUPER_TABLE == pOut->tbMeta->tableType) { ctgDebug("will continue to refresh tbmeta since got stb, tbName:%s", tNameGetTableName(pName)); - + taosMemoryFreeClear(pOut->tbMeta); - + CTG_RET(ctgGetTbMetaFromMnode(pCtg, pConn, pName, NULL, tReq)); } else if (CTG_IS_META_BOTH(pOut->metaType)) { int32_t exist = 0; @@ -1029,13 +1029,13 @@ int32_t ctgHandleGetTbMetaRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf stbCtx.flag = flag; stbCtx.pName = &stbName; - taosMemoryFreeClear(pOut->tbMeta); + taosMemoryFreeClear(pOut->tbMeta); CTG_ERR_JRET(ctgReadTbMetaFromCache(pCtg, &stbCtx, &pOut->tbMeta)); if (pOut->tbMeta) { exist = 1; } } - + if (0 == exist) { TSWAP(pMsgCtx->lastOut, pMsgCtx->out); CTG_RET(ctgGetTbMetaFromMnodeImpl(pCtg, pConn, pOut->dbFName, pOut->tbName, NULL, tReq)); @@ -1056,7 +1056,7 @@ int32_t ctgHandleGetTbMetaRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf if (CTG_IS_META_BOTH(pOut->metaType)) { memcpy(pOut->tbMeta, &pOut->ctbMeta, sizeof(pOut->ctbMeta)); } - + /* else if (CTG_IS_META_CTABLE(pOut->metaType)) { SName stbName = *pName; @@ -1064,7 +1064,7 @@ int32_t ctgHandleGetTbMetaRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf SCtgTbMetaCtx stbCtx = {0}; stbCtx.flag = flag; stbCtx.pName = &stbName; - + CTG_ERR_JRET(ctgReadTbMetaFromCache(pCtg, &stbCtx, &pOut->tbMeta)); if (NULL == pOut->tbMeta) { ctgDebug("stb no longer exist, stbName:%s", stbName.tname); @@ -1088,7 +1088,7 @@ _return: if (pTask->res || code) { ctgHandleTaskEnd(pTask, code); } - + CTG_RET(code); } @@ -1097,7 +1097,7 @@ int32_t ctgHandleGetTbMetasRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBu int32_t code = 0; SCtgDBCache *dbCache = NULL; SCtgTask* pTask = tReq->pTask; - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SRequestConnInfo* pConn = &pTask->pJob->conn; SCtgMsgCtx* pMsgCtx = CTG_GET_TASK_MSGCTX(pTask, tReq->msgIdx); SCtgTbMetasCtx* ctx = (SCtgTbMetasCtx*)pTask->taskCtx; @@ -1125,38 +1125,38 @@ int32_t ctgHandleGetTbMetasRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBu } case TDMT_MND_TABLE_META: { STableMetaOutput* pOut = (STableMetaOutput*)pMsgCtx->out; - + if (CTG_IS_META_NULL(pOut->metaType)) { if (CTG_FLAG_IS_STB(flag)) { char dbFName[TSDB_DB_FNAME_LEN] = {0}; tNameGetFullDbName(pName, dbFName); - + CTG_ERR_RET(ctgAcquireVgInfoFromCache(pCtg, dbFName, &dbCache)); if (NULL != dbCache) { SVgroupInfo vgInfo = {0}; CTG_ERR_JRET(ctgGetVgInfoFromHashValue(pCtg, dbCache->vgCache.vgInfo, pName, &vgInfo)); - + ctgDebug("will refresh tbmeta, supposed to be stb, tbName:%s, flag:%d", tNameGetTableName(pName), flag); - *vgId = vgInfo.vgId; + *vgId = vgInfo.vgId; CTG_ERR_JRET(ctgGetTbMetaFromVnode(pCtg, pConn, pName, &vgInfo, NULL, tReq)); ctgReleaseVgInfoToCache(pCtg, dbCache); } else { SBuildUseDBInput input = {0}; - + tstrncpy(input.db, dbFName, tListLen(input.db)); input.vgVersion = CTG_DEFAULT_INVALID_VERSION; - + CTG_ERR_JRET(ctgGetDBVgInfoFromMnode(pCtg, pConn, &input, NULL, tReq)); } return TSDB_CODE_SUCCESS; } - + ctgError("no tbmeta got, tbName:%s", tNameGetTableName(pName)); ctgRemoveTbMetaFromCache(pCtg, pName, false); - + CTG_ERR_JRET(CTG_ERR_CODE_TABLE_NOT_EXIST); } @@ -1165,12 +1165,12 @@ int32_t ctgHandleGetTbMetasRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBu STableMetaOutput* pLastOut = (STableMetaOutput*)pMsgCtx->out; TSWAP(pLastOut->tbMeta, pOut->tbMeta); } - + break; } case TDMT_VND_TABLE_META: { STableMetaOutput* pOut = (STableMetaOutput*)pMsgCtx->out; - + if (CTG_IS_META_NULL(pOut->metaType)) { ctgError("no tbmeta got, tbNmae:%s", tNameGetTableName(pName)); ctgRemoveTbMetaFromCache(pCtg, pName, false); @@ -1180,12 +1180,12 @@ int32_t ctgHandleGetTbMetasRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBu if (CTG_FLAG_IS_STB(flag)) { break; } - + if (CTG_IS_META_TABLE(pOut->metaType) && TSDB_SUPER_TABLE == pOut->tbMeta->tableType) { ctgDebug("will continue to refresh tbmeta since got stb, tbName:%s", tNameGetTableName(pName)); - + taosMemoryFreeClear(pOut->tbMeta); - + CTG_RET(ctgGetTbMetaFromMnode(pCtg, pConn, pName, NULL, tReq)); } else if (CTG_IS_META_BOTH(pOut->metaType)) { int32_t exist = 0; @@ -1196,14 +1196,14 @@ int32_t ctgHandleGetTbMetasRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBu stbCtx.flag = flag; stbCtx.pName = &stbName; - taosMemoryFreeClear(pOut->tbMeta); + taosMemoryFreeClear(pOut->tbMeta); CTG_ERR_JRET(ctgReadTbMetaFromCache(pCtg, &stbCtx, &pOut->tbMeta)); if (pOut->tbMeta) { ctgDebug("use cached stb meta, tbName:%s", tNameGetTableName(pName)); exist = 1; } } - + if (0 == exist) { TSWAP(pMsgCtx->lastOut, pMsgCtx->out); CTG_RET(ctgGetTbMetaFromMnodeImpl(pCtg, pConn, pOut->dbFName, pOut->tbName, NULL, tReq)); @@ -1224,7 +1224,7 @@ int32_t ctgHandleGetTbMetasRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBu if (CTG_IS_META_BOTH(pOut->metaType)) { memcpy(pOut->tbMeta, &pOut->ctbMeta, sizeof(pOut->ctbMeta)); } - + /* else if (CTG_IS_META_CTABLE(pOut->metaType)) { SName stbName = *pName; @@ -1232,7 +1232,7 @@ int32_t ctgHandleGetTbMetasRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBu SCtgTbMetaCtx stbCtx = {0}; stbCtx.flag = flag; stbCtx.pName = &stbName; - + CTG_ERR_JRET(ctgReadTbMetaFromCache(pCtg, &stbCtx, &pOut->tbMeta)); if (NULL == pOut->tbMeta) { ctgDebug("stb no longer exist, stbName:%s", stbName.tname); @@ -1273,7 +1273,7 @@ _return: if (pTask->res && taskDone) { ctgHandleTaskEnd(pTask, code); } - + CTG_RET(code); } @@ -1282,7 +1282,7 @@ int32_t ctgHandleGetDbVgRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf * int32_t code = 0; SCtgTask* pTask = tReq->pTask; SCtgDbVgCtx* ctx = (SCtgDbVgCtx*)pTask->taskCtx; - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; CTG_ERR_JRET(ctgProcessRspMsg(pTask->msgCtx.out, reqType, pMsg->pData, pMsg->len, rspCode, pTask->msgCtx.target)); @@ -1290,7 +1290,7 @@ int32_t ctgHandleGetDbVgRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf * case TDMT_MND_USE_DB: { SUseDbOutput* pOut = (SUseDbOutput*)pTask->msgCtx.out; SDBVgInfo* pDb = NULL; - + CTG_ERR_JRET(ctgGenerateVgList(pCtg, pOut->dbVgroup->vgHash, (SArray**)&pTask->res)); CTG_ERR_JRET(cloneDbVgInfo(pOut->dbVgroup, &pDb)); @@ -1316,7 +1316,7 @@ int32_t ctgHandleGetTbHashRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf int32_t code = 0; SCtgTask* pTask = tReq->pTask; SCtgTbHashCtx* ctx = (SCtgTbHashCtx*)pTask->taskCtx; - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; CTG_ERR_JRET(ctgProcessRspMsg(pTask->msgCtx.out, reqType, pMsg->pData, pMsg->len, rspCode, pTask->msgCtx.target)); @@ -1330,7 +1330,7 @@ int32_t ctgHandleGetTbHashRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf } CTG_ERR_JRET(ctgGetVgInfoFromHashValue(pCtg, pOut->dbVgroup, ctx->pName, (SVgroupInfo*)pTask->res)); - + CTG_ERR_JRET(ctgUpdateVgroupEnqueue(pCtg, ctx->dbFName, pOut->dbId, pOut->dbVgroup, false)); pOut->dbVgroup = NULL; @@ -1354,7 +1354,7 @@ int32_t ctgHandleGetTbHashsRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBu int32_t code = 0; SCtgTask* pTask = tReq->pTask; SCtgTbHashsCtx* ctx = (SCtgTbHashsCtx*)pTask->taskCtx; - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SCtgMsgCtx* pMsgCtx = CTG_GET_TASK_MSGCTX(pTask, tReq->msgIdx); SCtgFetch* pFetch = taosArrayGet(ctx->pFetchs, tReq->msgIdx); bool taskDone = false; @@ -1367,7 +1367,7 @@ int32_t ctgHandleGetTbHashsRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBu STablesReq* pReq = taosArrayGet(ctx->pNames, pFetch->dbIdx); CTG_ERR_JRET(ctgGetVgInfosFromHashValue(pCtg, tReq, pOut->dbVgroup, ctx, pMsgCtx->target, pReq->pTables, true)); - + CTG_ERR_JRET(ctgUpdateVgroupEnqueue(pCtg, pMsgCtx->target, pOut->dbId, pOut->dbVgroup, false)); pOut->dbVgroup = NULL; @@ -1394,7 +1394,7 @@ _return: pRes->code = code; pRes->pRes = NULL; } - + if (0 == atomic_sub_fetch_32(&ctx->fetchNum, 1)) { TSWAP(pTask->res, ctx->pResList); taskDone = true; @@ -1419,9 +1419,9 @@ int32_t ctgHandleGetTbIndexRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBu CTG_ERR_JRET(ctgCloneTableIndex(pOut->pIndex, &pInfo)); pTask->res = pInfo; - SCtgTbIndexCtx* ctx = pTask->taskCtx; + SCtgTbIndexCtx* ctx = pTask->taskCtx; CTG_ERR_JRET(ctgUpdateTbIndexEnqueue(pTask->pJob->pCtg, (STableIndex**)&pTask->msgCtx.out, false)); - + _return: if (TSDB_CODE_MND_DB_INDEX_NOT_EXIST == code) { @@ -1438,7 +1438,7 @@ int32_t ctgHandleGetTbCfgRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf CTG_ERR_JRET(ctgProcessRspMsg(&pTask->msgCtx.out, reqType, pMsg->pData, pMsg->len, rspCode, pTask->msgCtx.target)); TSWAP(pTask->res, pTask->msgCtx.out); - + _return: ctgHandleTaskEnd(pTask, code); @@ -1452,7 +1452,7 @@ int32_t ctgHandleGetDbCfgRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf CTG_ERR_JRET(ctgProcessRspMsg(pTask->msgCtx.out, reqType, pMsg->pData, pMsg->len, rspCode, pTask->msgCtx.target)); TSWAP(pTask->res, pTask->msgCtx.out); - + _return: ctgHandleTaskEnd(pTask, code); @@ -1471,7 +1471,7 @@ int32_t ctgHandleGetQnodeRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf CTG_ERR_JRET(ctgProcessRspMsg(pTask->msgCtx.out, reqType, pMsg->pData, pMsg->len, rspCode, pTask->msgCtx.target)); TSWAP(pTask->res, pTask->msgCtx.out); - + _return: ctgHandleTaskEnd(pTask, code); @@ -1485,7 +1485,7 @@ int32_t ctgHandleGetDnodeRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf CTG_ERR_JRET(ctgProcessRspMsg(&pTask->msgCtx.out, reqType, pMsg->pData, pMsg->len, rspCode, pTask->msgCtx.target)); TSWAP(pTask->res, pTask->msgCtx.out); - + _return: ctgHandleTaskEnd(pTask, code); @@ -1499,7 +1499,7 @@ int32_t ctgHandleGetIndexRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf CTG_ERR_JRET(ctgProcessRspMsg(pTask->msgCtx.out, reqType, pMsg->pData, pMsg->len, rspCode, pTask->msgCtx.target)); TSWAP(pTask->res, pTask->msgCtx.out); - + _return: ctgHandleTaskEnd(pTask, code); @@ -1513,7 +1513,7 @@ int32_t ctgHandleGetUdfRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf *p CTG_ERR_JRET(ctgProcessRspMsg(pTask->msgCtx.out, reqType, pMsg->pData, pMsg->len, rspCode, pTask->msgCtx.target)); TSWAP(pTask->res, pTask->msgCtx.out); - + _return: ctgHandleTaskEnd(pTask, code); @@ -1525,7 +1525,7 @@ int32_t ctgHandleGetUserRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf * int32_t code = 0; SCtgTask* pTask = tReq->pTask; SCtgUserCtx* ctx = (SCtgUserCtx*)pTask->taskCtx; - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; bool pass = false; SGetUserAuthRsp* pOut = (SGetUserAuthRsp*)pTask->msgCtx.out; @@ -1573,7 +1573,7 @@ int32_t ctgHandleGetSvrVerRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBuf CTG_ERR_JRET(ctgProcessRspMsg(&pTask->msgCtx.out, reqType, pMsg->pData, pMsg->len, rspCode, pTask->msgCtx.target)); TSWAP(pTask->res, pTask->msgCtx.out); - + _return: ctgHandleTaskEnd(pTask, code); @@ -1583,7 +1583,7 @@ _return: int32_t ctgAsyncRefreshTbMeta(SCtgTaskReq *tReq, int32_t flag, SName* pName, int32_t* vgId) { SCtgTask* pTask = tReq->pTask; - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SRequestConnInfo* pConn = &pTask->pJob->conn; int32_t code = 0; @@ -1603,7 +1603,7 @@ int32_t ctgAsyncRefreshTbMeta(SCtgTaskReq *tReq, int32_t flag, SName* pName, int SCtgDBCache *dbCache = NULL; char dbFName[TSDB_DB_FNAME_LEN] = {0}; tNameGetFullDbName(pName, dbFName); - + CTG_ERR_RET(ctgAcquireVgInfoFromCache(pCtg, dbFName, &dbCache)); if (dbCache) { SVgroupInfo vgInfo = {0}; @@ -1632,7 +1632,7 @@ _return: } int32_t ctgLaunchGetTbMetaTask(SCtgTask *pTask) { - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SRequestConnInfo* pConn = &pTask->pJob->conn; SCtgJob* pJob = pTask->pJob; SCtgMsgCtx *pMsgCtx = CTG_GET_TASK_MSGCTX(pTask, -1); @@ -1649,14 +1649,14 @@ int32_t ctgLaunchGetTbMetaTask(SCtgTask *pTask) { SCtgTbMetaCtx* pCtx = (SCtgTbMetaCtx*)pTask->taskCtx; SCtgTaskReq tReq; tReq.pTask = pTask; - tReq.msgIdx = -1; + tReq.msgIdx = -1; CTG_ERR_RET(ctgAsyncRefreshTbMeta(&tReq, pCtx->flag, pCtx->pName, &pCtx->vgId)); return TSDB_CODE_SUCCESS; } int32_t ctgLaunchGetTbMetasTask(SCtgTask *pTask) { - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SRequestConnInfo* pConn = &pTask->pJob->conn; SCtgTbMetasCtx* pCtx = (SCtgTbMetasCtx*)pTask->taskCtx; SCtgJob* pJob = pTask->pJob; @@ -1670,18 +1670,18 @@ int32_t ctgLaunchGetTbMetasTask(SCtgTask *pTask) { CTG_ERR_RET(ctgGetTbMetasFromCache(pCtg, pConn, pCtx, i, &fetchIdx, baseResIdx, pReq->pTables)); baseResIdx += taosArrayGetSize(pReq->pTables); } - + pCtx->fetchNum = taosArrayGetSize(pCtx->pFetchs); if (pCtx->fetchNum <= 0) { TSWAP(pTask->res, pCtx->pResList); - + CTG_ERR_RET(ctgHandleTaskEnd(pTask, 0)); return TSDB_CODE_SUCCESS; } - + pTask->msgCtxs = taosArrayInit(pCtx->fetchNum, sizeof(SCtgMsgCtx)); taosArraySetSize(pTask->msgCtxs, pCtx->fetchNum); - + for (int32_t i = 0; i < pCtx->fetchNum; ++i) { SCtgFetch* pFetch = taosArrayGet(pCtx->pFetchs, i); SName* pName = ctgGetFetchName(pCtx->pNames, pFetch); @@ -1689,19 +1689,19 @@ int32_t ctgLaunchGetTbMetasTask(SCtgTask *pTask) { if (NULL == pMsgCtx->pBatchs) { pMsgCtx->pBatchs = pJob->pBatchs; } - + SCtgTaskReq tReq; tReq.pTask = pTask; - tReq.msgIdx = pFetch->fetchIdx; + tReq.msgIdx = pFetch->fetchIdx; CTG_ERR_RET(ctgAsyncRefreshTbMeta(&tReq, pFetch->flag, pName, &pFetch->vgId)); } - + return TSDB_CODE_SUCCESS; } int32_t ctgLaunchGetDbVgTask(SCtgTask *pTask) { int32_t code = 0; - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SRequestConnInfo* pConn = &pTask->pJob->conn; SCtgDBCache *dbCache = NULL; SCtgDbVgCtx* pCtx = (SCtgDbVgCtx*)pTask->taskCtx; @@ -1710,18 +1710,18 @@ int32_t ctgLaunchGetDbVgTask(SCtgTask *pTask) { if (NULL == pMsgCtx->pBatchs) { pMsgCtx->pBatchs = pJob->pBatchs; } - + CTG_ERR_RET(ctgAcquireVgInfoFromCache(pCtg, pCtx->dbFName, &dbCache)); if (NULL != dbCache) { CTG_ERR_JRET(ctgGenerateVgList(pCtg, dbCache->vgCache.vgInfo->vgHash, (SArray**)&pTask->res)); ctgReleaseVgInfoToCache(pCtg, dbCache); dbCache = NULL; - + CTG_ERR_JRET(ctgHandleTaskEnd(pTask, 0)); } else { SBuildUseDBInput input = {0}; - + tstrncpy(input.db, pCtx->dbFName, tListLen(input.db)); input.vgVersion = CTG_DEFAULT_INVALID_VERSION; @@ -1742,7 +1742,7 @@ _return: int32_t ctgLaunchGetTbHashTask(SCtgTask *pTask) { int32_t code = 0; - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SRequestConnInfo* pConn = &pTask->pJob->conn; SCtgDBCache *dbCache = NULL; SCtgTbHashCtx* pCtx = (SCtgTbHashCtx*)pTask->taskCtx; @@ -1751,7 +1751,7 @@ int32_t ctgLaunchGetTbHashTask(SCtgTask *pTask) { if (NULL == pMsgCtx->pBatchs) { pMsgCtx->pBatchs = pJob->pBatchs; } - + CTG_ERR_RET(ctgAcquireVgInfoFromCache(pCtg, pCtx->dbFName, &dbCache)); if (NULL != dbCache) { pTask->res = taosMemoryMalloc(sizeof(SVgroupInfo)); @@ -1762,17 +1762,17 @@ int32_t ctgLaunchGetTbHashTask(SCtgTask *pTask) { ctgReleaseVgInfoToCache(pCtg, dbCache); dbCache = NULL; - + CTG_ERR_JRET(ctgHandleTaskEnd(pTask, 0)); } else { SBuildUseDBInput input = {0}; - + tstrncpy(input.db, pCtx->dbFName, tListLen(input.db)); input.vgVersion = CTG_DEFAULT_INVALID_VERSION; SCtgTaskReq tReq; tReq.pTask = pTask; - tReq.msgIdx = -1; + tReq.msgIdx = -1; CTG_ERR_RET(ctgGetDBVgInfoFromMnode(pCtg, pConn, &input, NULL, &tReq)); } @@ -1786,16 +1786,16 @@ _return: } int32_t ctgLaunchGetTbHashsTask(SCtgTask *pTask) { - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SRequestConnInfo* pConn = &pTask->pJob->conn; SCtgTbHashsCtx* pCtx = (SCtgTbHashsCtx*)pTask->taskCtx; SCtgDBCache *dbCache = NULL; - SCtgJob* pJob = pTask->pJob; + SCtgJob* pJob = pTask->pJob; int32_t dbNum = taosArrayGetSize(pCtx->pNames); int32_t fetchIdx = 0; int32_t baseResIdx = 0; int32_t code = 0; - + for (int32_t i = 0; i < dbNum; ++i) { STablesReq* pReq = taosArrayGet(pCtx->pNames, i); @@ -1804,7 +1804,7 @@ int32_t ctgLaunchGetTbHashsTask(SCtgTask *pTask) { if (NULL != dbCache) { SCtgTaskReq tReq; tReq.pTask = pTask; - tReq.msgIdx = -1; + tReq.msgIdx = -1; CTG_ERR_JRET(ctgGetVgInfosFromHashValue(pCtg, &tReq, dbCache->vgCache.vgInfo, pCtx, pReq->dbFName, pReq->pTables, false)); ctgReleaseVgInfoToCache(pCtg, dbCache); @@ -1815,21 +1815,21 @@ int32_t ctgLaunchGetTbHashsTask(SCtgTask *pTask) { ctgAddFetch(&pCtx->pFetchs, i, -1, &fetchIdx, baseResIdx, 0); baseResIdx += taosArrayGetSize(pReq->pTables); - taosArraySetSize(pCtx->pResList, baseResIdx); + taosArraySetSize(pCtx->pResList, baseResIdx); } } pCtx->fetchNum = taosArrayGetSize(pCtx->pFetchs); if (pCtx->fetchNum <= 0) { TSWAP(pTask->res, pCtx->pResList); - + CTG_ERR_RET(ctgHandleTaskEnd(pTask, 0)); return TSDB_CODE_SUCCESS; } - + pTask->msgCtxs = taosArrayInit(pCtx->fetchNum, sizeof(SCtgMsgCtx)); taosArraySetSize(pTask->msgCtxs, pCtx->fetchNum); - + for (int32_t i = 0; i < pCtx->fetchNum; ++i) { SCtgFetch* pFetch = taosArrayGet(pCtx->pFetchs, i); STablesReq* pReq = taosArrayGet(pCtx->pNames, pFetch->dbIdx); @@ -1837,10 +1837,10 @@ int32_t ctgLaunchGetTbHashsTask(SCtgTask *pTask) { if (NULL == pMsgCtx->pBatchs) { pMsgCtx->pBatchs = pJob->pBatchs; } - + SBuildUseDBInput input = {0}; strcpy(input.db, pReq->dbFName); - + input.vgVersion = CTG_DEFAULT_INVALID_VERSION; SCtgTaskReq tReq; @@ -1854,14 +1854,14 @@ _return: if (dbCache) { ctgReleaseVgInfoToCache(pCtg, dbCache); } - + return code; } int32_t ctgLaunchGetTbIndexTask(SCtgTask *pTask) { int32_t code = 0; - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SRequestConnInfo* pConn = &pTask->pJob->conn; SCtgTbIndexCtx* pCtx = (SCtgTbIndexCtx*)pTask->taskCtx; SArray* pRes = NULL; @@ -1874,18 +1874,18 @@ int32_t ctgLaunchGetTbIndexTask(SCtgTask *pTask) { CTG_ERR_RET(ctgReadTbIndexFromCache(pCtg, pCtx->pName, &pRes)); if (pRes) { pTask->res = pRes; - + CTG_ERR_RET(ctgHandleTaskEnd(pTask, 0)); return TSDB_CODE_SUCCESS; } - + CTG_ERR_RET(ctgGetTbIndexFromMnode(pCtg, pConn, pCtx->pName, NULL, pTask)); return TSDB_CODE_SUCCESS; } int32_t ctgLaunchGetTbCfgTask(SCtgTask *pTask) { int32_t code = 0; - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SRequestConnInfo* pConn = &pTask->pJob->conn; SCtgTbCfgCtx* pCtx = (SCtgTbCfgCtx*)pTask->taskCtx; SArray* pRes = NULL; @@ -1915,7 +1915,7 @@ int32_t ctgLaunchGetTbCfgTask(SCtgTask *pTask) { return TSDB_CODE_SUCCESS; } } - + CTG_ERR_JRET(ctgGetTableCfgFromVnode(pCtg, pConn, pCtx->pName, pCtx->pVgInfo, NULL, pTask)); } @@ -1926,13 +1926,13 @@ _return: if (CTG_TASK_LAUNCHED == pTask->status) { ctgHandleTaskEnd(pTask, code); } - + CTG_RET(code); } int32_t ctgLaunchGetQnodeTask(SCtgTask *pTask) { - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SRequestConnInfo* pConn = &pTask->pJob->conn; SCtgJob* pJob = pTask->pJob; SCtgMsgCtx *pMsgCtx = CTG_GET_TASK_MSGCTX(pTask, -1); @@ -1945,7 +1945,7 @@ int32_t ctgLaunchGetQnodeTask(SCtgTask *pTask) { } int32_t ctgLaunchGetDnodeTask(SCtgTask *pTask) { - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SRequestConnInfo* pConn = &pTask->pJob->conn; SCtgJob* pJob = pTask->pJob; SCtgMsgCtx *pMsgCtx = CTG_GET_TASK_MSGCTX(pTask, -1); @@ -1959,7 +1959,7 @@ int32_t ctgLaunchGetDnodeTask(SCtgTask *pTask) { int32_t ctgLaunchGetDbCfgTask(SCtgTask *pTask) { - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SRequestConnInfo* pConn = &pTask->pJob->conn; SCtgDbCfgCtx* pCtx = (SCtgDbCfgCtx*)pTask->taskCtx; SCtgJob* pJob = pTask->pJob; @@ -1975,7 +1975,7 @@ int32_t ctgLaunchGetDbCfgTask(SCtgTask *pTask) { int32_t ctgLaunchGetDbInfoTask(SCtgTask *pTask) { int32_t code = 0; - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SCtgDBCache *dbCache = NULL; SCtgDbInfoCtx* pCtx = (SCtgDbInfoCtx*)pTask->taskCtx; SCtgJob* pJob = pTask->pJob; @@ -2014,7 +2014,7 @@ _return: } int32_t ctgLaunchGetIndexTask(SCtgTask *pTask) { - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SRequestConnInfo* pConn = &pTask->pJob->conn; SCtgIndexCtx* pCtx = (SCtgIndexCtx*)pTask->taskCtx; SCtgJob* pJob = pTask->pJob; @@ -2029,7 +2029,7 @@ int32_t ctgLaunchGetIndexTask(SCtgTask *pTask) { } int32_t ctgLaunchGetUdfTask(SCtgTask *pTask) { - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SRequestConnInfo* pConn = &pTask->pJob->conn; SCtgUdfCtx* pCtx = (SCtgUdfCtx*)pTask->taskCtx; SCtgJob* pJob = pTask->pJob; @@ -2044,7 +2044,7 @@ int32_t ctgLaunchGetUdfTask(SCtgTask *pTask) { } int32_t ctgLaunchGetUserTask(SCtgTask *pTask) { - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SRequestConnInfo* pConn = &pTask->pJob->conn; SCtgUserCtx* pCtx = (SCtgUserCtx*)pTask->taskCtx; bool inCache = false; @@ -2054,7 +2054,7 @@ int32_t ctgLaunchGetUserTask(SCtgTask *pTask) { if (NULL == pMsgCtx->pBatchs) { pMsgCtx->pBatchs = pJob->pBatchs; } - + CTG_ERR_RET(ctgChkAuthFromCache(pCtg, pCtx->user.user, pCtx->user.dbFName, pCtx->user.type, &inCache, &pass)); if (inCache) { pTask->res = taosMemoryCalloc(1, sizeof(bool)); @@ -2062,7 +2062,7 @@ int32_t ctgLaunchGetUserTask(SCtgTask *pTask) { CTG_ERR_RET(TSDB_CODE_OUT_OF_MEMORY); } *(bool*)pTask->res = pass; - + CTG_ERR_RET(ctgHandleTaskEnd(pTask, 0)); return TSDB_CODE_SUCCESS; } @@ -2073,7 +2073,7 @@ int32_t ctgLaunchGetUserTask(SCtgTask *pTask) { } int32_t ctgLaunchGetSvrVerTask(SCtgTask *pTask) { - SCatalog* pCtg = pTask->pJob->pCtg; + SCatalog* pCtg = pTask->pJob->pCtg; SRequestConnInfo* pConn = &pTask->pJob->conn; SCtgJob* pJob = pTask->pJob; SCtgMsgCtx *pMsgCtx = CTG_GET_TASK_MSGCTX(pTask, -1); @@ -2096,7 +2096,7 @@ int32_t ctgRelaunchGetTbMetaTask(SCtgTask *pTask) { int32_t ctgGetTbCfgCb(SCtgTask *pTask) { int32_t code = 0; - + CTG_ERR_JRET(pTask->subRes.code); SCtgTbCfgCtx* pCtx = (SCtgTbCfgCtx*)pTask->taskCtx; @@ -2104,7 +2104,7 @@ int32_t ctgGetTbCfgCb(SCtgTask *pTask) { pCtx->tbType = ((STableMeta*)pTask->subRes.res)->tableType; } else if (CTG_TASK_GET_DB_VGROUP == pTask->subRes.type) { SDBVgInfo* pDb = (SDBVgInfo*)pTask->subRes.res; - + pCtx->pVgInfo = taosMemoryCalloc(1, sizeof(SVgroupInfo)); CTG_ERR_JRET(ctgGetVgInfoFromHashValue(pTask->pJob->pCtg, pDb, pCtx->pName, pCtx->pVgInfo)); } @@ -2167,7 +2167,7 @@ SCtgAsyncFps gCtgAsyncFps[] = { int32_t ctgMakeAsyncRes(SCtgJob *pJob) { int32_t code = 0; int32_t taskNum = taosArrayGetSize(pJob->pTasks); - + for (int32_t i = 0; i < taskNum; ++i) { SCtgTask *pTask = taosArrayGet(pJob->pTasks, i); CTG_ERR_RET((*gCtgAsyncFps[pTask->type].dumpResFp)(pTask)); @@ -2180,16 +2180,16 @@ int32_t ctgSearchExistingTask(SCtgJob *pJob, CTG_TASK_TYPE type, void* param, in bool equal = false; SCtgTask* pTask = NULL; int32_t code = 0; - + CTG_LOCK(CTG_READ, &pJob->taskLock); - + int32_t taskNum = taosArrayGetSize(pJob->pTasks); for (int32_t i = 0; i < taskNum; ++i) { pTask = taosArrayGet(pJob->pTasks, i); if (type != pTask->type) { continue; } - + CTG_ERR_JRET((*gCtgAsyncFps[type].compFp)(pTask, param, &equal)); if (equal) { break; @@ -2208,7 +2208,7 @@ _return: int32_t ctgSetSubTaskCb(SCtgTask *pSub, SCtgTask *pTask) { int32_t code = 0; - + CTG_LOCK(CTG_WRITE, &pSub->lock); if (CTG_TASK_DONE == pSub->status) { pTask->subRes.code = pSub->code; @@ -2216,7 +2216,7 @@ int32_t ctgSetSubTaskCb(SCtgTask *pSub, SCtgTask *pTask) { SCtgMsgCtx *pMsgCtx = CTG_GET_TASK_MSGCTX(pTask, -1); SCtgMsgCtx *pSubMsgCtx = CTG_GET_TASK_MSGCTX(pSub, -1); pMsgCtx->pBatchs = pSubMsgCtx->pBatchs; - + CTG_ERR_JRET(pTask->subRes.fp(pTask)); } else { if (NULL == pSub->pParents) { @@ -2230,7 +2230,7 @@ _return: CTG_UNLOCK(CTG_WRITE, &pSub->lock); - CTG_RET(code); + CTG_RET(code); } @@ -2242,13 +2242,13 @@ int32_t ctgLaunchSubTask(SCtgTask *pTask, CTG_TASK_TYPE type, ctgSubTaskCbFp fp, ctgClearSubTaskRes(&pTask->subRes); pTask->subRes.type = type; pTask->subRes.fp = fp; - + CTG_ERR_RET(ctgSearchExistingTask(pJob, type, param, &subTaskId)); if (subTaskId < 0) { CTG_ERR_RET(ctgInitTask(pJob, type, param, &subTaskId)); newTask = true; } - + SCtgTask* pSub = taosArrayGet(pJob->pTasks, subTaskId); CTG_ERR_RET(ctgSetSubTaskCb(pSub, pTask)); @@ -2267,21 +2267,21 @@ int32_t ctgLaunchSubTask(SCtgTask *pTask, CTG_TASK_TYPE type, ctgSubTaskCbFp fp, int32_t ctgLaunchJob(SCtgJob *pJob) { int32_t taskNum = taosArrayGetSize(pJob->pTasks); - + for (int32_t i = 0; i < taskNum; ++i) { SCtgTask *pTask = taosArrayGet(pJob->pTasks, i); qDebug("QID:0x%" PRIx64 " ctg launch [%dth] task", pJob->queryId, pTask->taskId); CTG_ERR_RET((*gCtgAsyncFps[pTask->type].launchFp)(pTask)); - + pTask->status = CTG_TASK_LAUNCHED; } if (taskNum <= 0) { qDebug("QID:0x%" PRIx64 " ctg call user callback with rsp %s", pJob->queryId, tstrerror(pJob->jobResCode)); - + taosAsyncExec(ctgCallUserCb, pJob, NULL); -#if CTG_BATCH_FETCH +#if CTG_BATCH_FETCH } else { ctgLaunchBatchs(pJob->pCtg, pJob, pJob->pBatchs); #endif diff --git a/source/libs/command/src/command.c b/source/libs/command/src/command.c index 1b2489acd6..7d259fe06c 100644 --- a/source/libs/command/src/command.c +++ b/source/libs/command/src/command.c @@ -17,6 +17,7 @@ #include "catalog.h" #include "commandInt.h" #include "scheduler.h" +#include "systable.h" #include "tdatablock.h" #include "tglobal.h" #include "tgrant.h" @@ -75,46 +76,41 @@ static SSDataBlock* buildDescResultDataBlock() { return pBlock; } -static void setDescResultIntoDataBlock(SSDataBlock* pBlock, int32_t numOfRows, STableMeta* pMeta) { +static void setDescResultIntoDataBlock(bool sysInfoUser, SSDataBlock* pBlock, int32_t numOfRows, STableMeta* pMeta) { blockDataEnsureCapacity(pBlock, numOfRows); - pBlock->info.rows = numOfRows; + pBlock->info.rows = 0; // field SColumnInfoData* pCol1 = taosArrayGet(pBlock->pDataBlock, 0); - char buf[DESCRIBE_RESULT_FIELD_LEN] = {0}; - for (int32_t i = 0; i < numOfRows; ++i) { - STR_TO_VARSTR(buf, pMeta->schema[i].name); - colDataAppend(pCol1, i, buf, false); - } - // Type SColumnInfoData* pCol2 = taosArrayGet(pBlock->pDataBlock, 1); - for (int32_t i = 0; i < numOfRows; ++i) { - STR_TO_VARSTR(buf, tDataTypes[pMeta->schema[i].type].name); - colDataAppend(pCol2, i, buf, false); - } - // Length SColumnInfoData* pCol3 = taosArrayGet(pBlock->pDataBlock, 2); - for (int32_t i = 0; i < numOfRows; ++i) { - int32_t bytes = getSchemaBytes(pMeta->schema + i); - colDataAppend(pCol3, i, (const char*)&bytes, false); - } - // Note SColumnInfoData* pCol4 = taosArrayGet(pBlock->pDataBlock, 3); + char buf[DESCRIBE_RESULT_FIELD_LEN] = {0}; for (int32_t i = 0; i < numOfRows; ++i) { + if (invisibleColumn(sysInfoUser, pMeta->tableType, pMeta->schema[i].flags)) { + continue; + } + STR_TO_VARSTR(buf, pMeta->schema[i].name); + colDataAppend(pCol1, pBlock->info.rows, buf, false); + STR_TO_VARSTR(buf, tDataTypes[pMeta->schema[i].type].name); + colDataAppend(pCol2, pBlock->info.rows, buf, false); + int32_t bytes = getSchemaBytes(pMeta->schema + i); + colDataAppend(pCol3, pBlock->info.rows, (const char*)&bytes, false); STR_TO_VARSTR(buf, i >= pMeta->tableInfo.numOfColumns ? "TAG" : ""); - colDataAppend(pCol4, i, buf, false); + colDataAppend(pCol4, pBlock->info.rows, buf, false); + ++(pBlock->info.rows); } } -static int32_t execDescribe(SNode* pStmt, SRetrieveTableRsp** pRsp) { +static int32_t execDescribe(bool sysInfoUser, SNode* pStmt, SRetrieveTableRsp** pRsp) { SDescribeStmt* pDesc = (SDescribeStmt*)pStmt; int32_t numOfRows = TABLE_TOTAL_COL_NUM(pDesc->pMeta); SSDataBlock* pBlock = buildDescResultDataBlock(); - setDescResultIntoDataBlock(pBlock, numOfRows, pDesc->pMeta); + setDescResultIntoDataBlock(sysInfoUser, pBlock, numOfRows, pDesc->pMeta); return buildRetrieveTableRsp(pBlock, DESCRIBE_RESULT_COLS, pRsp); } @@ -665,10 +661,10 @@ static int32_t execSelectWithoutFrom(SSelectStmt* pSelect, SRetrieveTableRsp** p return code; } -int32_t qExecCommand(SNode* pStmt, SRetrieveTableRsp** pRsp) { +int32_t qExecCommand(bool sysInfoUser, SNode* pStmt, SRetrieveTableRsp** pRsp) { switch (nodeType(pStmt)) { case QUERY_NODE_DESCRIBE_STMT: - return execDescribe(pStmt, pRsp); + return execDescribe(sysInfoUser, pStmt, pRsp); case QUERY_NODE_RESET_QUERY_CACHE_STMT: return execResetQueryCache(); case QUERY_NODE_SHOW_CREATE_DATABASE_STMT: diff --git a/source/libs/executor/inc/executil.h b/source/libs/executor/inc/executil.h index a25933d15e..9e7fcc2227 100644 --- a/source/libs/executor/inc/executil.h +++ b/source/libs/executor/inc/executil.h @@ -22,6 +22,7 @@ #include "tbuffer.h" #include "tcommon.h" #include "tpagedbuf.h" +#include "tsimplehash.h" #define T_LONG_JMP(_obj, _c) \ do { \ @@ -106,7 +107,7 @@ static FORCE_INLINE void setResultBufPageDirty(SDiskbasedBuf* pBuf, SResultRowPo setBufPageDirty(pPage, true); } -void initGroupedResultInfo(SGroupResInfo* pGroupResInfo, SHashObj* pHashmap, int32_t order); +void initGroupedResultInfo(SGroupResInfo* pGroupResInfo, SSHashObj* pHashmap, int32_t order); void cleanupGroupResInfo(SGroupResInfo* pGroupResInfo); void initMultiResInfoFromArrayList(SGroupResInfo* pGroupResInfo, SArray* pArrayList); diff --git a/source/libs/executor/inc/executorimpl.h b/source/libs/executor/inc/executorimpl.h index 594955d469..e1c9c6a114 100644 --- a/source/libs/executor/inc/executorimpl.h +++ b/source/libs/executor/inc/executorimpl.h @@ -142,7 +142,9 @@ typedef struct { //TODO remove prepareStatus STqOffsetVal prepareStatus; // for tmq STqOffsetVal lastStatus; // for tmq - void* metaBlk; // for tmq fetching meta + SMqMetaRsp metaRsp; // for tmq fetching meta + SSchemaWrapper *schema; + char tbName[TSDB_TABLE_NAME_LEN]; SSDataBlock* pullOverBlk; // for streaming SWalFilterCond cond; int64_t lastScanUid; @@ -491,6 +493,19 @@ typedef struct SStreamScanInfo { SNode* pTagIndexCond; } SStreamScanInfo; +typedef struct SStreamRawScanInfo{ +// int8_t subType; +// bool withMeta; +// int64_t suid; +// int64_t snapVersion; +// void *metaInfo; +// void *dataInfo; + SVnode* vnode; + SSDataBlock pRes; // result SSDataBlock + STsdbReader* dataReader; + SSnapContext* sContext; +}SStreamRawScanInfo; + typedef struct SSysTableScanInfo { SRetrieveMetaTableRsp* pRsp; SRetrieveTableReq req; @@ -499,6 +514,7 @@ typedef struct SSysTableScanInfo { SReadHandle readHandle; int32_t accountId; const char* pUser; + bool sysInfo; bool showRewrite; SNode* pCondition; // db_name filter condition, to discard data that are not in current database SMTbCursor* pCur; // cursor for iterate the local table meta store. @@ -932,6 +948,8 @@ SOperatorInfo* createDataBlockInfoScanOperator(void* dataReader, SReadHandle* re SOperatorInfo* createStreamScanOperatorInfo(SReadHandle* pHandle, STableScanPhysiNode* pTableScanNode, SNode* pTagCond, SExecTaskInfo* pTaskInfo); +SOperatorInfo* createRawScanOperatorInfo(SReadHandle* pHandle, SExecTaskInfo* pTaskInfo); + SOperatorInfo* createFillOperatorInfo(SOperatorInfo* downstream, SFillPhysiNode* pPhyFillNode, SExecTaskInfo* pTaskInfo); SOperatorInfo* createStatewindowOperatorInfo(SOperatorInfo* downstream, SStateWinodwPhysiNode* pStateNode, SExecTaskInfo* pTaskInfo); SOperatorInfo* createPartitionOperatorInfo(SOperatorInfo* downstream, SPartitionPhysiNode* pPartNode, SExecTaskInfo* pTaskInfo); diff --git a/source/libs/executor/inc/tsimplehash.h b/source/libs/executor/inc/tsimplehash.h index 4c5a80e2f1..27191e3b7e 100644 --- a/source/libs/executor/inc/tsimplehash.h +++ b/source/libs/executor/inc/tsimplehash.h @@ -28,7 +28,7 @@ typedef void (*_hash_free_fn_t)(void *); /** * @brief single thread hash - * + * */ typedef struct SSHashObj SSHashObj; @@ -52,13 +52,13 @@ int32_t tSimpleHashPrint(const SSHashObj *pHashObj); /** * @brief put element into hash table, if the element with the same key exists, update it - * - * @param pHashObj - * @param key - * @param keyLen - * @param data - * @param dataLen - * @return int32_t + * + * @param pHashObj + * @param key + * @param keyLen + * @param data + * @param dataLen + * @return int32_t */ int32_t tSimpleHashPut(SSHashObj *pHashObj, const void *key, size_t keyLen, const void *data, size_t dataLen); @@ -80,6 +80,18 @@ void *tSimpleHashGet(SSHashObj *pHashObj, const void *key, size_t keyLen); */ int32_t tSimpleHashRemove(SSHashObj *pHashObj, const void *key, size_t keyLen); +/** + * remove item with the specified key during hash iterate + * + * @param pHashObj + * @param key + * @param keyLen + * @param pIter + * @param iter + * @return int32_t + */ +int32_t tSimpleHashIterateRemove(SSHashObj *pHashObj, const void *key, size_t keyLen, void **pIter, int32_t *iter); + /** * Clear the hash table. * @param pHashObj @@ -99,13 +111,27 @@ void tSimpleHashCleanup(SSHashObj *pHashObj); */ size_t tSimpleHashGetMemSize(const SSHashObj *pHashObj); +#pragma pack(push, 4) +typedef struct SHNode{ + struct SHNode *next; + uint32_t keyLen : 20; + uint32_t dataLen : 12; + char data[]; +} SHNode; +#pragma pack(pop) + /** * Get the corresponding key information for a given data in hash table * @param data * @param keyLen * @return */ -void *tSimpleHashGetKey(void *data, size_t* keyLen); +static FORCE_INLINE void *tSimpleHashGetKey(void *data, size_t *keyLen) { + SHNode *node = (SHNode *)((char *)data - offsetof(SHNode, data)); + if (keyLen) *keyLen = node->keyLen; + + return POINTER_SHIFT(data, node->dataLen); +} /** * Create the hash table iterator @@ -116,17 +142,6 @@ void *tSimpleHashGetKey(void *data, size_t* keyLen); */ void *tSimpleHashIterate(const SSHashObj *pHashObj, void *data, int32_t *iter); -/** - * Create the hash table iterator - * - * @param pHashObj - * @param data - * @param key - * @param iter - * @return void* - */ -void *tSimpleHashIterateKV(const SSHashObj *pHashObj, void *data, void **key, int32_t *iter); - #ifdef __cplusplus } #endif diff --git a/source/libs/executor/src/executil.c b/source/libs/executor/src/executil.c index c6bc120c62..80c1494f8d 100644 --- a/source/libs/executor/src/executil.c +++ b/source/libs/executor/src/executil.c @@ -83,7 +83,7 @@ int32_t resultrowComparAsc(const void* p1, const void* p2) { static int32_t resultrowComparDesc(const void* p1, const void* p2) { return resultrowComparAsc(p2, p1); } -void initGroupedResultInfo(SGroupResInfo* pGroupResInfo, SHashObj* pHashmap, int32_t order) { +void initGroupedResultInfo(SGroupResInfo* pGroupResInfo, SSHashObj* pHashmap, int32_t order) { if (pGroupResInfo->pRows != NULL) { taosArrayDestroy(pGroupResInfo->pRows); } @@ -92,9 +92,10 @@ void initGroupedResultInfo(SGroupResInfo* pGroupResInfo, SHashObj* pHashmap, int void* pData = NULL; pGroupResInfo->pRows = taosArrayInit(10, POINTER_BYTES); - size_t keyLen = 0; - while ((pData = taosHashIterate(pHashmap, pData)) != NULL) { - void* key = taosHashGetKey(pData, &keyLen); + size_t keyLen = 0; + int32_t iter = 0; + while ((pData = tSimpleHashIterate(pHashmap, pData, &iter)) != NULL) { + void* key = tSimpleHashGetKey(pData, &keyLen); SResKeyPos* p = taosMemoryMalloc(keyLen + sizeof(SResultRowPosition)); @@ -348,7 +349,7 @@ static int32_t createResultData(SDataType* pType, int32_t numOfRows, SScalarPara int32_t code = colInfoDataEnsureCapacity(pColumnData, numOfRows); if (code != TSDB_CODE_SUCCESS) { - terrno = TSDB_CODE_OUT_OF_MEMORY; + terrno = code; taosMemoryFree(pColumnData); return terrno; } @@ -366,6 +367,7 @@ static SColumnInfoData* getColInfoResult(void* metaHandle, uint64_t suid, SArray SScalarParam output = {0}; tagFilterAssist ctx = {0}; + ctx.colHash = taosHashInit(4, taosGetDefaultHashFunction(TSDB_DATA_TYPE_SMALLINT), false, HASH_NO_LOCK); if (ctx.colHash == NULL) { terrno = TSDB_CODE_OUT_OF_MEMORY; @@ -473,6 +475,7 @@ static SColumnInfoData* getColInfoResult(void* metaHandle, uint64_t suid, SArray if (code != TSDB_CODE_SUCCESS) { terrno = code; qError("failed to create result, reason:%s", tstrerror(code)); + terrno = code; goto end; } @@ -1297,6 +1300,7 @@ int32_t initQueryTableDataCond(SQueryTableDataCond* pCond, const STableScanPhysi pCond->type = TIMEWINDOW_RANGE_CONTAINED; pCond->startVersion = -1; pCond->endVersion = -1; + pCond->schemaVersion = -1; // pCond->type = pTableScanNode->scanFlag; int32_t j = 0; diff --git a/source/libs/executor/src/executor.c b/source/libs/executor/src/executor.c index fe1f4911ca..124f4b44b0 100644 --- a/source/libs/executor/src/executor.c +++ b/source/libs/executor/src/executor.c @@ -140,7 +140,23 @@ int32_t qSetMultiStreamInput(qTaskInfo_t tinfo, const void* pBlocks, size_t numO qTaskInfo_t qCreateQueueExecTaskInfo(void* msg, SReadHandle* readers, int32_t* numOfCols, SSchemaWrapper** pSchema) { if (msg == NULL) { // TODO create raw scan - return NULL; + + SExecTaskInfo* pTaskInfo = taosMemoryCalloc(1, sizeof(SExecTaskInfo)); + if (NULL == pTaskInfo) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + return NULL; + } + setTaskStatus(pTaskInfo, TASK_NOT_COMPLETED); + + pTaskInfo->cost.created = taosGetTimestampMs(); + pTaskInfo->execModel = OPTR_EXEC_MODEL_QUEUE; + pTaskInfo->pRoot = createRawScanOperatorInfo(readers, pTaskInfo); + if(NULL == pTaskInfo->pRoot){ + terrno = TSDB_CODE_OUT_OF_MEMORY; + taosMemoryFree(pTaskInfo); + return NULL; + } + return pTaskInfo; } struct SSubplan* pPlan = NULL; @@ -669,15 +685,26 @@ void* qExtractReaderFromStreamScanner(void* scanner) { return (void*)pInfo->tqReader; } -const SSchemaWrapper* qExtractSchemaFromStreamScanner(void* scanner) { - SStreamScanInfo* pInfo = scanner; - return pInfo->tqReader->pSchemaWrapper; +const SSchemaWrapper* qExtractSchemaFromTask(qTaskInfo_t tinfo) { + SExecTaskInfo* pTaskInfo = (SExecTaskInfo*)tinfo; + return pTaskInfo->streamInfo.schema; } -void* qStreamExtractMetaMsg(qTaskInfo_t tinfo) { +const char* qExtractTbnameFromTask(qTaskInfo_t tinfo) { + SExecTaskInfo* pTaskInfo = (SExecTaskInfo*)tinfo; + return pTaskInfo->streamInfo.tbName; +} + +SMqMetaRsp* qStreamExtractMetaMsg(qTaskInfo_t tinfo) { SExecTaskInfo* pTaskInfo = (SExecTaskInfo*)tinfo; ASSERT(pTaskInfo->execModel == OPTR_EXEC_MODEL_QUEUE); - return pTaskInfo->streamInfo.metaBlk; + return &pTaskInfo->streamInfo.metaRsp; +} + +int64_t qStreamExtractPrepareUid(qTaskInfo_t tinfo) { + SExecTaskInfo* pTaskInfo = (SExecTaskInfo*)tinfo; + ASSERT(pTaskInfo->execModel == OPTR_EXEC_MODEL_QUEUE); + return pTaskInfo->streamInfo.prepareStatus.uid; } int32_t qStreamExtractOffset(qTaskInfo_t tinfo, STqOffsetVal* pOffset) { @@ -687,102 +714,164 @@ int32_t qStreamExtractOffset(qTaskInfo_t tinfo, STqOffsetVal* pOffset) { return 0; } -int32_t qStreamPrepareScan(qTaskInfo_t tinfo, const STqOffsetVal* pOffset) { +int32_t initQueryTableDataCondForTmq(SQueryTableDataCond* pCond, SSnapContext* sContext, SMetaTableInfo mtInfo) { + memset(pCond, 0, sizeof(SQueryTableDataCond)); + pCond->order = TSDB_ORDER_ASC; + pCond->numOfCols = mtInfo.schema->nCols; + pCond->colList = taosMemoryCalloc(pCond->numOfCols, sizeof(SColumnInfo)); + if (pCond->colList == NULL) { + terrno = TSDB_CODE_QRY_OUT_OF_MEMORY; + return terrno; + } + + pCond->twindows = (STimeWindow){.skey = INT64_MIN, .ekey = INT64_MAX}; + pCond->suid = mtInfo.suid; + pCond->type = TIMEWINDOW_RANGE_CONTAINED; + pCond->startVersion = -1; + pCond->endVersion = sContext->snapVersion; + pCond->schemaVersion = sContext->snapVersion; + + for (int32_t i = 0; i < pCond->numOfCols; ++i) { + pCond->colList[i].type = mtInfo.schema->pSchema[i].type; + pCond->colList[i].bytes = mtInfo.schema->pSchema[i].bytes; + pCond->colList[i].colId = mtInfo.schema->pSchema[i].colId; + } + + return TSDB_CODE_SUCCESS; +} + +int32_t qStreamPrepareScan(qTaskInfo_t tinfo, STqOffsetVal* pOffset, int8_t subType) { SExecTaskInfo* pTaskInfo = (SExecTaskInfo*)tinfo; SOperatorInfo* pOperator = pTaskInfo->pRoot; ASSERT(pTaskInfo->execModel == OPTR_EXEC_MODEL_QUEUE); pTaskInfo->streamInfo.prepareStatus = *pOffset; - if (!tOffsetEqual(pOffset, &pTaskInfo->streamInfo.lastStatus)) { - while (1) { - uint16_t type = pOperator->operatorType; - pOperator->status = OP_OPENED; - // TODO add more check - if (type != QUERY_NODE_PHYSICAL_PLAN_STREAM_SCAN) { - ASSERT(pOperator->numOfDownstream == 1); - pOperator = pOperator->pDownstream[0]; - } + if (tOffsetEqual(pOffset, &pTaskInfo->streamInfo.lastStatus)) { + return 0; + } + if (subType == TOPIC_SUB_TYPE__COLUMN) { + uint16_t type = pOperator->operatorType; + pOperator->status = OP_OPENED; + // TODO add more check + if (type != QUERY_NODE_PHYSICAL_PLAN_STREAM_SCAN) { + ASSERT(pOperator->numOfDownstream == 1); + pOperator = pOperator->pDownstream[0]; + } - SStreamScanInfo* pInfo = pOperator->info; - if (pOffset->type == TMQ_OFFSET__LOG) { - STableScanInfo* pTSInfo = pInfo->pTableScanOp->info; - tsdbReaderClose(pTSInfo->dataReader); - pTSInfo->dataReader = NULL; + SStreamScanInfo* pInfo = pOperator->info; + if (pOffset->type == TMQ_OFFSET__LOG) { + STableScanInfo* pTSInfo = pInfo->pTableScanOp->info; + tsdbReaderClose(pTSInfo->dataReader); + pTSInfo->dataReader = NULL; #if 0 - if (tOffsetEqual(pOffset, &pTaskInfo->streamInfo.lastStatus) && - pInfo->tqReader->pWalReader->curVersion != pOffset->version) { - qError("prepare scan ver %" PRId64 " actual ver %" PRId64 ", last %" PRId64, pOffset->version, - pInfo->tqReader->pWalReader->curVersion, pTaskInfo->streamInfo.lastStatus.version); - ASSERT(0); - } -#endif - if (tqSeekVer(pInfo->tqReader, pOffset->version + 1) < 0) { - return -1; - } - ASSERT(pInfo->tqReader->pWalReader->curVersion == pOffset->version + 1); - } else if (pOffset->type == TMQ_OFFSET__SNAPSHOT_DATA) { - /*pInfo->blockType = STREAM_INPUT__TABLE_SCAN;*/ - int64_t uid = pOffset->uid; - int64_t ts = pOffset->ts; - - if (uid == 0) { - if (taosArrayGetSize(pTaskInfo->tableqinfoList.pTableList) != 0) { - STableKeyInfo* pTableInfo = taosArrayGet(pTaskInfo->tableqinfoList.pTableList, 0); - uid = pTableInfo->uid; - ts = INT64_MIN; - } else { - return -1; - } - } - - /*if (pTaskInfo->streamInfo.lastStatus.type != TMQ_OFFSET__SNAPSHOT_DATA ||*/ - /*pTaskInfo->streamInfo.lastStatus.uid != uid || pTaskInfo->streamInfo.lastStatus.ts != ts) {*/ - STableScanInfo* pTableScanInfo = pInfo->pTableScanOp->info; - int32_t tableSz = taosArrayGetSize(pTaskInfo->tableqinfoList.pTableList); - -#ifndef NDEBUG - - qDebug("switch to next table %" PRId64 " (cursor %d), %" PRId64 " rows returned", uid, - pTableScanInfo->currentTable, pInfo->pTableScanOp->resultInfo.totalRows); - pInfo->pTableScanOp->resultInfo.totalRows = 0; -#endif - - bool found = false; - for (int32_t i = 0; i < tableSz; i++) { - STableKeyInfo* pTableInfo = taosArrayGet(pTaskInfo->tableqinfoList.pTableList, i); - if (pTableInfo->uid == uid) { - found = true; - pTableScanInfo->currentTable = i; - break; - } - } - - // TODO after dropping table, table may be not found - ASSERT(found); - - if (pTableScanInfo->dataReader == NULL) { - if (tsdbReaderOpen(pTableScanInfo->readHandle.vnode, &pTableScanInfo->cond, - pTaskInfo->tableqinfoList.pTableList, &pTableScanInfo->dataReader, NULL) < 0 || - pTableScanInfo->dataReader == NULL) { - ASSERT(0); - } - } - - tsdbSetTableId(pTableScanInfo->dataReader, uid); - int64_t oldSkey = pTableScanInfo->cond.twindows.skey; - pTableScanInfo->cond.twindows.skey = ts + 1; - tsdbReaderReset(pTableScanInfo->dataReader, &pTableScanInfo->cond); - pTableScanInfo->cond.twindows.skey = oldSkey; - pTableScanInfo->scanTimes = 0; - - qDebug("tsdb reader offset seek to uid %" PRId64 " ts %" PRId64 ", table cur set to %d , all table num %d", uid, - ts, pTableScanInfo->currentTable, tableSz); - /*}*/ - - } else { + if (tOffsetEqual(pOffset, &pTaskInfo->streamInfo.lastStatus) && + pInfo->tqReader->pWalReader->curVersion != pOffset->version) { + qError("prepare scan ver %" PRId64 " actual ver %" PRId64 ", last %" PRId64, pOffset->version, + pInfo->tqReader->pWalReader->curVersion, pTaskInfo->streamInfo.lastStatus.version); ASSERT(0); } - return 0; +#endif + if (tqSeekVer(pInfo->tqReader, pOffset->version + 1) < 0) { + return -1; + } + ASSERT(pInfo->tqReader->pWalReader->curVersion == pOffset->version + 1); + } else if (pOffset->type == TMQ_OFFSET__SNAPSHOT_DATA) { + /*pInfo->blockType = STREAM_INPUT__TABLE_SCAN;*/ + int64_t uid = pOffset->uid; + int64_t ts = pOffset->ts; + + if (uid == 0) { + if (taosArrayGetSize(pTaskInfo->tableqinfoList.pTableList) != 0) { + STableKeyInfo* pTableInfo = taosArrayGet(pTaskInfo->tableqinfoList.pTableList, 0); + uid = pTableInfo->uid; + ts = INT64_MIN; + } else { + return -1; + } + } + + /*if (pTaskInfo->streamInfo.lastStatus.type != TMQ_OFFSET__SNAPSHOT_DATA ||*/ + /*pTaskInfo->streamInfo.lastStatus.uid != uid || pTaskInfo->streamInfo.lastStatus.ts != ts) {*/ + STableScanInfo* pTableScanInfo = pInfo->pTableScanOp->info; + int32_t tableSz = taosArrayGetSize(pTaskInfo->tableqinfoList.pTableList); + +#ifndef NDEBUG + qDebug("switch to next table %" PRId64 " (cursor %d), %" PRId64 " rows returned", uid, + pTableScanInfo->currentTable, pInfo->pTableScanOp->resultInfo.totalRows); + pInfo->pTableScanOp->resultInfo.totalRows = 0; +#endif + + bool found = false; + for (int32_t i = 0; i < tableSz; i++) { + STableKeyInfo* pTableInfo = taosArrayGet(pTaskInfo->tableqinfoList.pTableList, i); + if (pTableInfo->uid == uid) { + found = true; + pTableScanInfo->currentTable = i; + break; + } + } + + // TODO after dropping table, table may be not found + ASSERT(found); + + if (pTableScanInfo->dataReader == NULL) { + if (tsdbReaderOpen(pTableScanInfo->readHandle.vnode, &pTableScanInfo->cond, + pTaskInfo->tableqinfoList.pTableList, &pTableScanInfo->dataReader, NULL) < 0 || + pTableScanInfo->dataReader == NULL) { + ASSERT(0); + } + } + + tsdbSetTableId(pTableScanInfo->dataReader, uid); + int64_t oldSkey = pTableScanInfo->cond.twindows.skey; + pTableScanInfo->cond.twindows.skey = ts + 1; + tsdbReaderReset(pTableScanInfo->dataReader, &pTableScanInfo->cond); + pTableScanInfo->cond.twindows.skey = oldSkey; + pTableScanInfo->scanTimes = 0; + + qDebug("tsdb reader offset seek to uid %" PRId64 " ts %" PRId64 ", table cur set to %d , all table num %d", uid, + ts, pTableScanInfo->currentTable, tableSz); + /*}*/ + } else { + ASSERT(0); } + }else if (pOffset->type == TMQ_OFFSET__SNAPSHOT_DATA){ + SStreamRawScanInfo* pInfo = pOperator->info; + SSnapContext* sContext = pInfo->sContext; + if(setForSnapShot(sContext, pOffset->uid) != 0) { + qError("setDataForSnapShot error. uid:%"PRIi64, pOffset->uid); + return -1; + } + + SMetaTableInfo mtInfo = getUidfromSnapShot(sContext); + tsdbReaderClose(pInfo->dataReader); + pInfo->dataReader = NULL; + cleanupQueryTableDataCond(&pTaskInfo->streamInfo.tableCond); + taosArrayDestroy(pTaskInfo->tableqinfoList.pTableList); + if(mtInfo.uid == 0) return 0; // no data + + initQueryTableDataCondForTmq(&pTaskInfo->streamInfo.tableCond, sContext, mtInfo); + pTaskInfo->streamInfo.tableCond.twindows.skey = pOffset->ts; + pTaskInfo->tableqinfoList.pTableList = taosArrayInit(1, sizeof(STableKeyInfo)); + taosArrayPush(pTaskInfo->tableqinfoList.pTableList, &(STableKeyInfo){.uid = mtInfo.uid, .groupId = 0}); + tsdbReaderOpen(pInfo->vnode, &pTaskInfo->streamInfo.tableCond, pTaskInfo->tableqinfoList.pTableList, &pInfo->dataReader, NULL); + + strcpy(pTaskInfo->streamInfo.tbName, mtInfo.tbName); + tDeleteSSchemaWrapper(pTaskInfo->streamInfo.schema); + pTaskInfo->streamInfo.schema = mtInfo.schema; + qDebug("tmqsnap qStreamPrepareScan snapshot data uid %ld ts %ld", mtInfo.uid, pOffset->ts); + }else if(pOffset->type == TMQ_OFFSET__SNAPSHOT_META){ + SStreamRawScanInfo* pInfo = pOperator->info; + SSnapContext* sContext = pInfo->sContext; + if(setForSnapShot(sContext, pOffset->uid) != 0) { + qError("setForSnapShot error. uid:%"PRIi64" ,version:%"PRIi64, pOffset->uid); + return -1; + } + qDebug("tmqsnap qStreamPrepareScan snapshot meta uid %ld ts %ld", pOffset->uid); + }else if (pOffset->type == TMQ_OFFSET__LOG) { + SStreamRawScanInfo* pInfo = pOperator->info; + tsdbReaderClose(pInfo->dataReader); + pInfo->dataReader = NULL; + qDebug("tmqsnap qStreamPrepareScan snapshot log"); } return 0; } diff --git a/source/libs/executor/src/executorimpl.c b/source/libs/executor/src/executorimpl.c index 30d7283264..19daa7e96a 100644 --- a/source/libs/executor/src/executorimpl.c +++ b/source/libs/executor/src/executorimpl.c @@ -231,7 +231,7 @@ SResultRow* doSetResultOutBufByKey(SDiskbasedBuf* pResultBuf, SResultRowInfo* pR SET_RES_WINDOW_KEY(pSup->keyBuf, pData, bytes, groupId); SResultRowPosition* p1 = - (SResultRowPosition*)taosHashGet(pSup->pResultRowHashTable, pSup->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes)); + (SResultRowPosition*)tSimpleHashGet(pSup->pResultRowHashTable, pSup->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes)); SResultRow* pResult = NULL; @@ -267,7 +267,7 @@ SResultRow* doSetResultOutBufByKey(SDiskbasedBuf* pResultBuf, SResultRowInfo* pR // add a new result set for a new group SResultRowPosition pos = {.pageId = pResult->pageId, .offset = pResult->offset}; - taosHashPut(pSup->pResultRowHashTable, pSup->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes), &pos, + tSimpleHashPut(pSup->pResultRowHashTable, pSup->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes), &pos, sizeof(SResultRowPosition)); } @@ -276,7 +276,7 @@ SResultRow* doSetResultOutBufByKey(SDiskbasedBuf* pResultBuf, SResultRowInfo* pR // too many time window in query if (pTaskInfo->execModel == OPTR_EXEC_MODEL_BATCH && - taosHashGetSize(pSup->pResultRowHashTable) > MAX_INTERVAL_TIME_WINDOW) { + tSimpleHashGetSize(pSup->pResultRowHashTable) > MAX_INTERVAL_TIME_WINDOW) { T_LONG_JMP(pTaskInfo->env, TSDB_CODE_QRY_TOO_MANY_TIMEWINDOW); } @@ -2997,7 +2997,7 @@ int32_t aggEncodeResultRow(SOperatorInfo* pOperator, char** result, int32_t* len } SOptrBasicInfo* pInfo = (SOptrBasicInfo*)(pOperator->info); SAggSupporter* pSup = (SAggSupporter*)POINTER_SHIFT(pOperator->info, sizeof(SOptrBasicInfo)); - int32_t size = taosHashGetSize(pSup->pResultRowHashTable); + int32_t size = tSimpleHashGetSize(pSup->pResultRowHashTable); size_t keyLen = sizeof(uint64_t) * 2; // estimate the key length int32_t totalSize = sizeof(int32_t) + sizeof(int32_t) + size * (sizeof(int32_t) + keyLen + sizeof(int32_t) + pSup->resultRowSize); @@ -3024,10 +3024,11 @@ int32_t aggEncodeResultRow(SOperatorInfo* pOperator, char** result, int32_t* len SResultRow* pRow = (SResultRow*)((char*)pPage + pos->offset); setBufPageDirty(pPage, true); releaseBufPage(pSup->pResultBuf, pPage); - - void* pIter = taosHashIterate(pSup->pResultRowHashTable, NULL); - while (pIter) { - void* key = taosHashGetKey(pIter, &keyLen); + + int32_t iter = 0; + void* pIter = NULL; + while ((pIter = tSimpleHashIterate(pSup->pResultRowHashTable, pIter, &iter))) { + void* key = tSimpleHashGetKey(pIter, &keyLen); SResultRowPosition* p1 = (SResultRowPosition*)pIter; pPage = (SFilePage*)getBufPage(pSup->pResultBuf, p1->pageId); @@ -3058,8 +3059,6 @@ int32_t aggEncodeResultRow(SOperatorInfo* pOperator, char** result, int32_t* len offset += sizeof(int32_t); memcpy(*result + offset, pRow, pSup->resultRowSize); offset += pSup->resultRowSize; - - pIter = taosHashIterate(pSup->pResultRowHashTable, pIter); } *(int32_t*)(*result) = offset; @@ -3094,7 +3093,7 @@ int32_t aggDecodeResultRow(SOperatorInfo* pOperator, char* result) { // add a new result set for a new group SResultRowPosition pos = {.pageId = resultRow->pageId, .offset = resultRow->offset}; - taosHashPut(pSup->pResultRowHashTable, result + offset, keyLen, &pos, sizeof(SResultRowPosition)); + tSimpleHashPut(pSup->pResultRowHashTable, result + offset, keyLen, &pos, sizeof(SResultRowPosition)); offset += keyLen; int32_t valueLen = *(int32_t*)(result + offset); @@ -3211,6 +3210,7 @@ static void doHandleRemainBlockForNewGroupImpl(SOperatorInfo* pOperator, SFillOp Q_STATUS_EQUAL(pTaskInfo->status, TASK_COMPLETED) ? pInfo->win.ekey : pInfo->existNewGroupBlock->info.window.ekey; taosResetFillInfo(pInfo->pFillInfo, getFillInfoStart(pInfo->pFillInfo)); + blockDataCleanup(pInfo->pRes); doApplyScalarCalculation(pOperator, pInfo->existNewGroupBlock, order, scanFlag); taosFillSetStartInfo(pInfo->pFillInfo, pInfo->pRes->info.rows, ekey); @@ -3273,7 +3273,6 @@ static SSDataBlock* doFillImpl(SOperatorInfo* pOperator) { SSDataBlock* pResBlock = pInfo->pFinalRes; blockDataCleanup(pResBlock); - blockDataCleanup(pInfo->pRes); int32_t order = TSDB_ORDER_ASC; int32_t scanFlag = MAIN_SCAN; @@ -3297,6 +3296,8 @@ static SSDataBlock* doFillImpl(SOperatorInfo* pOperator) { taosFillSetStartInfo(pInfo->pFillInfo, 0, pInfo->win.ekey); } else { blockDataUpdateTsWindow(pBlock, pInfo->primarySrcSlotId); + + blockDataCleanup(pInfo->pRes); doApplyScalarCalculation(pOperator, pBlock, order, scanFlag); if (pInfo->curGroupId == 0 || pInfo->curGroupId == pInfo->pRes->info.groupId) { @@ -3339,7 +3340,6 @@ static SSDataBlock* doFillImpl(SOperatorInfo* pOperator) { assert(pBlock != NULL); blockDataCleanup(pResBlock); - blockDataCleanup(pInfo->pRes); doHandleRemainBlockForNewGroupImpl(pOperator, pInfo, pResultInfo, pTaskInfo); if (pResBlock->info.rows > pResultInfo->threshold) { @@ -3439,7 +3439,7 @@ int32_t doInitAggInfoSup(SAggSupporter* pAggSup, SqlFunctionCtx* pCtx, int32_t n pAggSup->currentPageId = -1; pAggSup->resultRowSize = getResultRowSize(pCtx, numOfOutput); pAggSup->keyBuf = taosMemoryCalloc(1, keyBufSize + POINTER_BYTES + sizeof(int64_t)); - pAggSup->pResultRowHashTable = taosHashInit(10, hashFn, true, HASH_NO_LOCK); + pAggSup->pResultRowHashTable = tSimpleHashInit(10, hashFn); if (pAggSup->keyBuf == NULL || pAggSup->pResultRowHashTable == NULL) { return TSDB_CODE_OUT_OF_MEMORY; @@ -3466,7 +3466,7 @@ int32_t doInitAggInfoSup(SAggSupporter* pAggSup, SqlFunctionCtx* pCtx, int32_t n void cleanupAggSup(SAggSupporter* pAggSup) { taosMemoryFreeClear(pAggSup->keyBuf); - taosHashCleanup(pAggSup->pResultRowHashTable); + tSimpleHashCleanup(pAggSup->pResultRowHashTable); destroyDiskbasedBuf(pAggSup->pResultBuf); } @@ -3993,6 +3993,7 @@ static int32_t initTableblockDistQueryCond(uint64_t uid, SQueryTableDataCond* pC pCond->type = TIMEWINDOW_RANGE_CONTAINED; pCond->startVersion = -1; pCond->endVersion = -1; + pCond->schemaVersion = -1; return TSDB_CODE_SUCCESS; } diff --git a/source/libs/executor/src/scanoperator.c b/source/libs/executor/src/scanoperator.c index db5405d86c..d4c98adb7c 100644 --- a/source/libs/executor/src/scanoperator.c +++ b/source/libs/executor/src/scanoperator.c @@ -36,8 +36,8 @@ #define SWITCH_ORDER(n) (((n) = ((n) == TSDB_ORDER_ASC) ? TSDB_ORDER_DESC : TSDB_ORDER_ASC)) static int32_t buildSysDbTableInfo(const SSysTableScanInfo* pInfo, int32_t capacity); -static int32_t buildDbTableInfoBlock(const SSDataBlock* p, const SSysTableMeta* pSysDbTableMeta, size_t size, - const char* dbName); +static int32_t buildDbTableInfoBlock(bool sysInfo, const SSDataBlock* p, const SSysTableMeta* pSysDbTableMeta, + size_t size, const char* dbName); static bool processBlockWithProbability(const SSampleExecInfo* pInfo); @@ -178,8 +178,8 @@ static SResultRow* getTableGroupOutputBuf(SOperatorInfo* pOperator, uint64_t gro STableScanInfo* pTableScanInfo = pOperator->info; - SResultRowPosition* p1 = (SResultRowPosition*)taosHashGet(pTableScanInfo->pdInfo.pAggSup->pResultRowHashTable, buf, - GET_RES_WINDOW_KEY_LEN(sizeof(groupId))); + SResultRowPosition* p1 = (SResultRowPosition*)tSimpleHashGet(pTableScanInfo->pdInfo.pAggSup->pResultRowHashTable, buf, + GET_RES_WINDOW_KEY_LEN(sizeof(groupId))); if (p1 == NULL) { return NULL; @@ -1334,9 +1334,9 @@ static SSDataBlock* doStreamScan(SOperatorInfo* pOperator) { } } else if (ret.fetchType == FETCH_TYPE__META) { ASSERT(0); - pTaskInfo->streamInfo.lastStatus = ret.offset; - pTaskInfo->streamInfo.metaBlk = ret.meta; - return NULL; +// pTaskInfo->streamInfo.lastStatus = ret.offset; +// pTaskInfo->streamInfo.metaBlk = ret.meta; +// return NULL; } else if (ret.fetchType == FETCH_TYPE__NONE) { pTaskInfo->streamInfo.lastStatus = ret.offset; ASSERT(pTaskInfo->streamInfo.lastStatus.version >= pTaskInfo->streamInfo.prepareStatus.version); @@ -1357,10 +1357,6 @@ static SSDataBlock* doStreamScan(SOperatorInfo* pOperator) { } qDebug("stream scan tsdb return null"); return NULL; - } else if (pTaskInfo->streamInfo.prepareStatus.type == TMQ_OFFSET__SNAPSHOT_META) { - // TODO scan meta - ASSERT(0); - return NULL; } if (pTaskInfo->streamInfo.recoverStep == STREAM_RECOVER_STEP__PREPARE) { @@ -1545,11 +1541,6 @@ static SSDataBlock* doStreamScan(SOperatorInfo* pOperator) { } } -static SSDataBlock* doRawScan(SOperatorInfo* pInfo) { - // - return NULL; -} - static SArray* extractTableIdList(const STableListInfo* pTableGroupInfo) { SArray* tableIdList = taosArrayInit(4, sizeof(uint64_t)); @@ -1562,17 +1553,160 @@ static SArray* extractTableIdList(const STableListInfo* pTableGroupInfo) { return tableIdList; } +static SSDataBlock* doRawScan(SOperatorInfo* pOperator) { +// NOTE: this operator does never check if current status is done or not + SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; + SStreamRawScanInfo* pInfo = pOperator->info; + pTaskInfo->streamInfo.metaRsp.metaRspLen = 0; // use metaRspLen !=0 to judge if data is meta + pTaskInfo->streamInfo.metaRsp.metaRsp = NULL; + + qDebug("tmqsnap doRawScan called"); + if(pTaskInfo->streamInfo.prepareStatus.type == TMQ_OFFSET__SNAPSHOT_DATA){ + SSDataBlock* pBlock = &pInfo->pRes; + + if (pInfo->dataReader && tsdbNextDataBlock(pInfo->dataReader)) { + if (isTaskKilled(pTaskInfo)) { + longjmp(pTaskInfo->env, TSDB_CODE_TSC_QUERY_CANCELLED); + } + + tsdbRetrieveDataBlockInfo(pInfo->dataReader, &pBlock->info); + + SArray* pCols = tsdbRetrieveDataBlock(pInfo->dataReader, NULL); + pBlock->pDataBlock = pCols; + if (pCols == NULL) { + longjmp(pTaskInfo->env, terrno); + } + + qDebug("tmqsnap doRawScan get data uid:%ld", pBlock->info.uid); + pTaskInfo->streamInfo.lastStatus.type = TMQ_OFFSET__SNAPSHOT_DATA; + pTaskInfo->streamInfo.lastStatus.uid = pBlock->info.uid; + pTaskInfo->streamInfo.lastStatus.ts = pBlock->info.window.ekey; + return pBlock; + } + + SMetaTableInfo mtInfo = getUidfromSnapShot(pInfo->sContext); + if (mtInfo.uid == 0){ //read snapshot done, change to get data from wal + qDebug("tmqsnap read snapshot done, change to get data from wal"); + pTaskInfo->streamInfo.prepareStatus.uid = mtInfo.uid; + pTaskInfo->streamInfo.lastStatus.type = TMQ_OFFSET__LOG; + pTaskInfo->streamInfo.lastStatus.version = pInfo->sContext->snapVersion; + tDeleteSSchemaWrapper(pTaskInfo->streamInfo.schema); + }else{ + pTaskInfo->streamInfo.prepareStatus.uid = mtInfo.uid; + pTaskInfo->streamInfo.prepareStatus.ts = INT64_MIN; + qDebug("tmqsnap change get data uid:%ld", mtInfo.uid); + qStreamPrepareScan(pTaskInfo, &pTaskInfo->streamInfo.prepareStatus, pInfo->sContext->subType); + strcpy(pTaskInfo->streamInfo.tbName, mtInfo.tbName); + tDeleteSSchemaWrapper(pTaskInfo->streamInfo.schema); + pTaskInfo->streamInfo.schema = mtInfo.schema; + } + qDebug("tmqsnap stream scan tsdb return null"); + return NULL; + }else if(pTaskInfo->streamInfo.prepareStatus.type == TMQ_OFFSET__SNAPSHOT_META){ + SSnapContext *sContext = pInfo->sContext; + void* data = NULL; + int32_t dataLen = 0; + int16_t type = 0; + int64_t uid = 0; + if(getMetafromSnapShot(sContext, &data, &dataLen, &type, &uid) < 0){ + qError("tmqsnap getMetafromSnapShot error"); + taosMemoryFreeClear(data); + return NULL; + } + + if(!sContext->queryMetaOrData){ // change to get data next poll request + pTaskInfo->streamInfo.lastStatus.type = TMQ_OFFSET__SNAPSHOT_META; + pTaskInfo->streamInfo.lastStatus.uid = uid; + pTaskInfo->streamInfo.metaRsp.rspOffset.type = TMQ_OFFSET__SNAPSHOT_DATA; + pTaskInfo->streamInfo.metaRsp.rspOffset.uid = 0; + pTaskInfo->streamInfo.metaRsp.rspOffset.ts = INT64_MIN; + }else{ + pTaskInfo->streamInfo.lastStatus.type = TMQ_OFFSET__SNAPSHOT_META; + pTaskInfo->streamInfo.lastStatus.uid = uid; + pTaskInfo->streamInfo.metaRsp.rspOffset = pTaskInfo->streamInfo.lastStatus; + pTaskInfo->streamInfo.metaRsp.resMsgType = type; + pTaskInfo->streamInfo.metaRsp.metaRspLen = dataLen; + pTaskInfo->streamInfo.metaRsp.metaRsp = data; + } + + return NULL; + } +// else if (pTaskInfo->streamInfo.prepareStatus.type == TMQ_OFFSET__LOG) { +// int64_t fetchVer = pTaskInfo->streamInfo.prepareStatus.version + 1; +// +// while(1){ +// if (tqFetchLog(pInfo->tqReader->pWalReader, pInfo->sContext->withMeta, &fetchVer, &pInfo->pCkHead) < 0) { +// qDebug("tmqsnap tmq poll: consumer log end. offset %" PRId64, fetchVer); +// pTaskInfo->streamInfo.lastStatus.version = fetchVer; +// pTaskInfo->streamInfo.lastStatus.type = TMQ_OFFSET__LOG; +// return NULL; +// } +// SWalCont* pHead = &pInfo->pCkHead->head; +// qDebug("tmqsnap tmq poll: consumer log offset %" PRId64 " msgType %d", fetchVer, pHead->msgType); +// +// if (pHead->msgType == TDMT_VND_SUBMIT) { +// SSubmitReq* pCont = (SSubmitReq*)&pHead->body; +// tqReaderSetDataMsg(pInfo->tqReader, pCont, 0); +// SSDataBlock* block = tqLogScanExec(pInfo->sContext->subType, pInfo->tqReader, pInfo->pFilterOutTbUid, &pInfo->pRes); +// if(block){ +// pTaskInfo->streamInfo.lastStatus.type = TMQ_OFFSET__LOG; +// pTaskInfo->streamInfo.lastStatus.version = fetchVer; +// qDebug("tmqsnap fetch data msg, ver:%" PRId64 ", type:%d", pHead->version, pHead->msgType); +// return block; +// }else{ +// fetchVer++; +// } +// } else{ +// ASSERT(pInfo->sContext->withMeta); +// ASSERT(IS_META_MSG(pHead->msgType)); +// qDebug("tmqsnap fetch meta msg, ver:%" PRId64 ", type:%d", pHead->version, pHead->msgType); +// pTaskInfo->streamInfo.metaRsp.rspOffset.version = fetchVer; +// pTaskInfo->streamInfo.metaRsp.rspOffset.type = TMQ_OFFSET__LOG; +// pTaskInfo->streamInfo.metaRsp.resMsgType = pHead->msgType; +// pTaskInfo->streamInfo.metaRsp.metaRspLen = pHead->bodyLen; +// pTaskInfo->streamInfo.metaRsp.metaRsp = taosMemoryMalloc(pHead->bodyLen); +// memcpy(pTaskInfo->streamInfo.metaRsp.metaRsp, pHead->body, pHead->bodyLen); +// return NULL; +// } +// } + return NULL; +} + +static void destroyRawScanOperatorInfo(void* param) { + SStreamRawScanInfo* pRawScan = (SStreamRawScanInfo*)param; + tsdbReaderClose(pRawScan->dataReader); + destroySnapContext(pRawScan->sContext); + taosMemoryFree(pRawScan); +} + // for subscribing db or stb (not including column), // if this scan is used, meta data can be return // and schemas are decided when scanning -SOperatorInfo* createRawScanOperatorInfo(SReadHandle* pHandle, STableScanPhysiNode* pTableScanNode, - SExecTaskInfo* pTaskInfo, STimeWindowAggSupp* pTwSup) { +SOperatorInfo* createRawScanOperatorInfo(SReadHandle* pHandle, SExecTaskInfo* pTaskInfo) { // create operator // create tb reader // create meta reader // create tq reader - return NULL; + SStreamRawScanInfo* pInfo = taosMemoryCalloc(1, sizeof(SStreamRawScanInfo)); + SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); + if (pInfo == NULL || pOperator == NULL) { + terrno = TSDB_CODE_QRY_OUT_OF_MEMORY; + return NULL; + } + + pInfo->vnode = pHandle->vnode; + + pInfo->sContext = pHandle->sContext; + pOperator->name = "RawStreamScanOperator"; +// pOperator->blocking = false; +// pOperator->status = OP_NOT_OPENED; + pOperator->info = pInfo; + pOperator->pTaskInfo = pTaskInfo; + + pOperator->fpSet = createOperatorFpSet(NULL, doRawScan, NULL, NULL, destroyRawScanOperatorInfo, + NULL, NULL, NULL); + return pOperator; } static void destroyStreamScanOperatorInfo(void* param) { @@ -2428,10 +2562,10 @@ int32_t buildSysDbTableInfo(const SSysTableScanInfo* pInfo, int32_t capacity) { const SSysTableMeta* pSysDbTableMeta = NULL; getInfosDbMeta(&pSysDbTableMeta, &size); - p->info.rows = buildDbTableInfoBlock(p, pSysDbTableMeta, size, TSDB_INFORMATION_SCHEMA_DB); + p->info.rows = buildDbTableInfoBlock(pInfo->sysInfo, p, pSysDbTableMeta, size, TSDB_INFORMATION_SCHEMA_DB); getPerfDbMeta(&pSysDbTableMeta, &size); - p->info.rows = buildDbTableInfoBlock(p, pSysDbTableMeta, size, TSDB_PERFORMANCE_SCHEMA_DB); + p->info.rows = buildDbTableInfoBlock(pInfo->sysInfo, p, pSysDbTableMeta, size, TSDB_PERFORMANCE_SCHEMA_DB); pInfo->pRes->info.rows = p->info.rows; relocateColumnData(pInfo->pRes, pInfo->scanCols, p->pDataBlock, false); @@ -2440,13 +2574,16 @@ int32_t buildSysDbTableInfo(const SSysTableScanInfo* pInfo, int32_t capacity) { return pInfo->pRes->info.rows; } -int32_t buildDbTableInfoBlock(const SSDataBlock* p, const SSysTableMeta* pSysDbTableMeta, size_t size, +int32_t buildDbTableInfoBlock(bool sysInfo, const SSDataBlock* p, const SSysTableMeta* pSysDbTableMeta, size_t size, const char* dbName) { char n[TSDB_TABLE_FNAME_LEN + VARSTR_HEADER_SIZE] = {0}; int32_t numOfRows = p->info.rows; for (int32_t i = 0; i < size; ++i) { const SSysTableMeta* pm = &pSysDbTableMeta[i]; + if (!sysInfo && pm->sysInfo) { + continue; + } SColumnInfoData* pColInfoData = taosArrayGet(p->pDataBlock, 0); @@ -2500,6 +2637,7 @@ SOperatorInfo* createSysTableScanOperatorInfo(void* readHandle, SSystemTableScan pInfo->accountId = pScanPhyNode->accountId; pInfo->pUser = taosMemoryStrDup((void*)pUser); + pInfo->sysInfo = pScanPhyNode->sysInfo; pInfo->showRewrite = pScanPhyNode->showRewrite; pInfo->pRes = pResBlock; pInfo->pCondition = pScanNode->node.pConditions; diff --git a/source/libs/executor/src/timewindowoperator.c b/source/libs/executor/src/timewindowoperator.c index f4ebf17646..9ea6b4c42f 100644 --- a/source/libs/executor/src/timewindowoperator.c +++ b/source/libs/executor/src/timewindowoperator.c @@ -33,11 +33,16 @@ typedef struct SPullWindowInfo { uint64_t groupId; } SPullWindowInfo; +typedef struct SOpenWindowInfo { + SResultRowPosition pos; + uint64_t groupId; +} SOpenWindowInfo; + static SSDataBlock* doStreamSessionAgg(SOperatorInfo* pOperator); static int64_t* extractTsCol(SSDataBlock* pBlock, const SIntervalAggOperatorInfo* pInfo); -static SResultRowPosition addToOpenWindowList(SResultRowInfo* pResultRowInfo, const SResultRow* pResult); +static SResultRowPosition addToOpenWindowList(SResultRowInfo* pResultRowInfo, const SResultRow* pResult, uint64_t groupId); static void doCloseWindow(SResultRowInfo* pResultRowInfo, const SIntervalAggOperatorInfo* pInfo, SResultRow* pResult); ///* @@ -598,14 +603,14 @@ static void doInterpUnclosedTimeWindow(SOperatorInfo* pOperatorInfo, int32_t num int32_t startPos = 0; int32_t numOfOutput = pSup->numOfExprs; - uint64_t groupId = pBlock->info.groupId; SResultRow* pResult = NULL; while (1) { SListNode* pn = tdListGetHead(pResultRowInfo->openWindow); - - SResultRowPosition* p1 = (SResultRowPosition*)pn->data; + SOpenWindowInfo* pOpenWin = (SOpenWindowInfo *)pn->data; + uint64_t groupId = pOpenWin->groupId; + SResultRowPosition* p1 = &pOpenWin->pos; if (p->pageId == p1->pageId && p->offset == p1->offset) { break; } @@ -631,8 +636,10 @@ static void doInterpUnclosedTimeWindow(SOperatorInfo* pOperatorInfo, int32_t num SGroupKeys* pTsKey = taosArrayGet(pInfo->pPrevValues, 0); int64_t prevTs = *(int64_t*)pTsKey->pData; - doTimeWindowInterpolation(pInfo->pPrevValues, pBlock->pDataBlock, prevTs, -1, tsCols[startPos], startPos, w.ekey, - RESULT_ROW_END_INTERP, pSup); + if (groupId == pBlock->info.groupId) { + doTimeWindowInterpolation(pInfo->pPrevValues, pBlock->pDataBlock, prevTs, -1, tsCols[startPos], startPos, w.ekey, + RESULT_ROW_END_INTERP, pSup); + } setResultRowInterpo(pResult, RESULT_ROW_END_INTERP); setNotInterpoWindowKey(pSup->pCtx, numOfExprs, RESULT_ROW_START_INTERP); @@ -966,7 +973,7 @@ static void hashIntervalAgg(SOperatorInfo* pOperatorInfo, SResultRowInfo* pResul // prev time window not interpolation yet. if (pInfo->timeWindowInterpo) { - SResultRowPosition pos = addToOpenWindowList(pResultRowInfo, pResult); + SResultRowPosition pos = addToOpenWindowList(pResultRowInfo, pResult, tableGroupId); doInterpUnclosedTimeWindow(pOperatorInfo, numOfOutput, pResultRowInfo, pBlock, scanFlag, tsCols, &pos); // restore current time window @@ -1018,10 +1025,18 @@ static void hashIntervalAgg(SOperatorInfo* pOperatorInfo, SResultRowInfo* pResul ekey = ascScan ? nextWin.ekey : nextWin.skey; forwardRows = getNumOfRowsInTimeWindow(&pBlock->info, tsCols, startPos, ekey, binarySearchForKey, NULL, pInfo->inputOrder); - // window start(end) key interpolation doWindowBorderInterpolation(pInfo, pBlock, pResult, &nextWin, startPos, forwardRows, pSup); - + //TODO: add to open window? how to close the open windows after input blocks exhausted? +#if 0 + if ((ascScan && ekey <= pBlock->info.window.ekey) || + (!ascScan && ekey >= pBlock->info.window.skey)) { + // window start(end) key interpolation + doWindowBorderInterpolation(pInfo, pBlock, pResult, &nextWin, startPos, forwardRows, pSup); + } else if (pInfo->timeWindowInterpo) { + addToOpenWindowList(pResultRowInfo, pResult, tableGroupId); + } +#endif updateTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &nextWin, true); doApplyFunctions(pTaskInfo, pSup->pCtx, &pInfo->twAggSup.timeWindowData, startPos, forwardRows, pBlock->info.rows, numOfOutput); @@ -1041,20 +1056,23 @@ void doCloseWindow(SResultRowInfo* pResultRowInfo, const SIntervalAggOperatorInf } } -SResultRowPosition addToOpenWindowList(SResultRowInfo* pResultRowInfo, const SResultRow* pResult) { - SResultRowPosition pos = (SResultRowPosition){.pageId = pResult->pageId, .offset = pResult->offset}; +SResultRowPosition addToOpenWindowList(SResultRowInfo* pResultRowInfo, const SResultRow* pResult, uint64_t groupId) { + SOpenWindowInfo openWin = {0}; + openWin.pos.pageId = pResult->pageId; + openWin.pos.offset = pResult->offset; + openWin.groupId = groupId; SListNode* pn = tdListGetTail(pResultRowInfo->openWindow); if (pn == NULL) { - tdListAppend(pResultRowInfo->openWindow, &pos); - return pos; + tdListAppend(pResultRowInfo->openWindow, &openWin); + return openWin.pos; } - SResultRowPosition* px = (SResultRowPosition*)pn->data; - if (px->pageId != pos.pageId || px->offset != pos.offset) { - tdListAppend(pResultRowInfo->openWindow, &pos); + SOpenWindowInfo * px = (SOpenWindowInfo *)pn->data; + if (px->pos.pageId != openWin.pos.pageId || px->pos.offset != openWin.pos.offset || px->groupId != openWin.groupId) { + tdListAppend(pResultRowInfo->openWindow, &openWin); } - return pos; + return openWin.pos; } int64_t* extractTsCol(SSDataBlock* pBlock, const SIntervalAggOperatorInfo* pInfo) { @@ -1381,7 +1399,7 @@ bool doClearWindow(SAggSupporter* pAggSup, SExprSupp* pSup, char* pData, int16_t int32_t numOfOutput) { SET_RES_WINDOW_KEY(pAggSup->keyBuf, pData, bytes, groupId); SResultRowPosition* p1 = - (SResultRowPosition*)taosHashGet(pAggSup->pResultRowHashTable, pAggSup->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes)); + (SResultRowPosition*)tSimpleHashGet(pAggSup->pResultRowHashTable, pAggSup->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes)); if (!p1) { // window has been closed return false; @@ -1394,14 +1412,14 @@ bool doDeleteIntervalWindow(SAggSupporter* pAggSup, TSKEY ts, uint64_t groupId) size_t bytes = sizeof(TSKEY); SET_RES_WINDOW_KEY(pAggSup->keyBuf, &ts, bytes, groupId); SResultRowPosition* p1 = - (SResultRowPosition*)taosHashGet(pAggSup->pResultRowHashTable, pAggSup->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes)); + (SResultRowPosition*)tSimpleHashGet(pAggSup->pResultRowHashTable, pAggSup->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes)); if (!p1) { // window has been closed return false; } // SFilePage* bufPage = getBufPage(pAggSup->pResultBuf, p1->pageId); // dBufSetBufPageRecycled(pAggSup->pResultBuf, bufPage); - taosHashRemove(pAggSup->pResultRowHashTable, pAggSup->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes)); + tSimpleHashRemove(pAggSup->pResultRowHashTable, pAggSup->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes)); return true; } @@ -1451,11 +1469,13 @@ static void doClearWindows(SAggSupporter* pAggSup, SExprSupp* pSup1, SInterval* } } -static int32_t getAllIntervalWindow(SHashObj* pHashMap, SHashObj* resWins) { - void* pIte = NULL; - size_t keyLen = 0; - while ((pIte = taosHashIterate(pHashMap, pIte)) != NULL) { - void* key = taosHashGetKey(pIte, &keyLen); +static int32_t getAllIntervalWindow(SSHashObj* pHashMap, SHashObj* resWins) { + + void* pIte = NULL; + size_t keyLen = 0; + int32_t iter = 0; + while ((pIte = tSimpleHashIterate(pHashMap, pIte, &iter)) != NULL) { + void* key = tSimpleHashGetKey(pIte, &keyLen); uint64_t groupId = *(uint64_t*)key; ASSERT(keyLen == GET_RES_WINDOW_KEY_LEN(sizeof(TSKEY))); TSKEY ts = *(int64_t*)((char*)key + sizeof(uint64_t)); @@ -1468,14 +1488,15 @@ static int32_t getAllIntervalWindow(SHashObj* pHashMap, SHashObj* resWins) { return TSDB_CODE_SUCCESS; } -static int32_t closeIntervalWindow(SHashObj* pHashMap, STimeWindowAggSupp* pSup, SInterval* pInterval, +static int32_t closeIntervalWindow(SSHashObj* pHashMap, STimeWindowAggSupp* pSup, SInterval* pInterval, SHashObj* pPullDataMap, SHashObj* closeWins, SArray* pRecyPages, SDiskbasedBuf* pDiscBuf) { qDebug("===stream===close interval window"); - void* pIte = NULL; - size_t keyLen = 0; - while ((pIte = taosHashIterate(pHashMap, pIte)) != NULL) { - void* key = taosHashGetKey(pIte, &keyLen); + void* pIte = NULL; + size_t keyLen = 0; + int32_t iter = 0; + while ((pIte = tSimpleHashIterate(pHashMap, pIte, &iter)) != NULL) { + void* key = tSimpleHashGetKey(pIte, &keyLen); uint64_t groupId = *(uint64_t*)key; ASSERT(keyLen == GET_RES_WINDOW_KEY_LEN(sizeof(TSKEY))); TSKEY ts = *(int64_t*)((char*)key + sizeof(uint64_t)); @@ -1513,7 +1534,7 @@ static int32_t closeIntervalWindow(SHashObj* pHashMap, STimeWindowAggSupp* pSup, } char keyBuf[GET_RES_WINDOW_KEY_LEN(sizeof(TSKEY))]; SET_RES_WINDOW_KEY(keyBuf, &ts, sizeof(TSKEY), groupId); - taosHashRemove(pHashMap, keyBuf, keyLen); + tSimpleHashIterateRemove(pHashMap, keyBuf, keyLen, &pIte, &iter); } } return TSDB_CODE_SUCCESS; @@ -1885,7 +1906,7 @@ SOperatorInfo* createIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo* pInfo->timeWindowInterpo = timeWindowinterpNeeded(pSup->pCtx, numOfCols, pInfo); if (pInfo->timeWindowInterpo) { - pInfo->binfo.resultRowInfo.openWindow = tdListNew(sizeof(SResultRowPosition)); + pInfo->binfo.resultRowInfo.openWindow = tdListNew(sizeof(SOpenWindowInfo)); if (pInfo->binfo.resultRowInfo.openWindow == NULL) { goto _error; } @@ -2145,6 +2166,7 @@ static void genInterpolationResult(STimeSliceOperatorInfo* pSliceInfo, SExprSupp // todo set the correct primary timestamp column // output the result + bool hasInterp = true; for (int32_t j = 0; j < pExprSup->numOfExprs; ++j) { SExprInfo* pExprInfo = &pExprSup->pExprInfo[j]; int32_t srcSlot = pExprInfo->base.pParam[0].pCol->slotId; @@ -2156,7 +2178,6 @@ static void genInterpolationResult(STimeSliceOperatorInfo* pSliceInfo, SExprSupp switch (pSliceInfo->fillType) { case TSDB_FILL_NULL: { colDataAppendNULL(pDst, rows); - pResBlock->info.rows += 1; break; } @@ -2176,7 +2197,6 @@ static void genInterpolationResult(STimeSliceOperatorInfo* pSliceInfo, SExprSupp GET_TYPED_DATA(v, int64_t, pVar->nType, &pVar->i); colDataAppend(pDst, rows, (char*)&v, false); } - pResBlock->info.rows += 1; break; } @@ -2190,6 +2210,7 @@ static void genInterpolationResult(STimeSliceOperatorInfo* pSliceInfo, SExprSupp // before interp range, do not fill if (start.key == INT64_MIN || end.key == INT64_MAX) { + hasInterp = false; break; } @@ -2201,28 +2222,27 @@ static void genInterpolationResult(STimeSliceOperatorInfo* pSliceInfo, SExprSupp } taosMemoryFree(current.val); - pResBlock->info.rows += 1; break; } case TSDB_FILL_PREV: { if (!pSliceInfo->isPrevRowSet) { + hasInterp = false; break; } SGroupKeys* pkey = taosArrayGet(pSliceInfo->pPrevRow, srcSlot); colDataAppend(pDst, rows, pkey->pData, false); - pResBlock->info.rows += 1; break; } case TSDB_FILL_NEXT: { if (!pSliceInfo->isNextRowSet) { + hasInterp = false; break; } SGroupKeys* pkey = taosArrayGet(pSliceInfo->pNextRow, srcSlot); colDataAppend(pDst, rows, pkey->pData, false); - pResBlock->info.rows += 1; break; } @@ -2231,6 +2251,11 @@ static void genInterpolationResult(STimeSliceOperatorInfo* pSliceInfo, SExprSupp break; } } + + if (hasInterp) { + pResBlock->info.rows += 1; + } + } static int32_t initPrevRowsKeeper(STimeSliceOperatorInfo* pInfo, SSDataBlock* pBlock) { @@ -2411,6 +2436,11 @@ static SSDataBlock* doTimeslice(SOperatorInfo* pOperator) { SColumnInfoData* pSrc = taosArrayGet(pBlock->pDataBlock, srcSlot); SColumnInfoData* pDst = taosArrayGet(pResBlock->pDataBlock, dstSlot); + if (colDataIsNull_s(pSrc, i)) { + colDataAppendNULL(pDst, pResBlock->info.rows); + continue; + } + char* v = colDataGetData(pSrc, i); colDataAppend(pDst, pResBlock->info.rows, v, false); } @@ -2847,7 +2877,7 @@ bool hasIntervalWindow(SAggSupporter* pSup, TSKEY ts, uint64_t groupId) { int32_t bytes = sizeof(TSKEY); SET_RES_WINDOW_KEY(pSup->keyBuf, &ts, bytes, groupId); SResultRowPosition* p1 = - (SResultRowPosition*)taosHashGet(pSup->pResultRowHashTable, pSup->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes)); + (SResultRowPosition*)tSimpleHashGet(pSup->pResultRowHashTable, pSup->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes)); return p1 != NULL; } @@ -2888,7 +2918,7 @@ static void rebuildIntervalWindow(SStreamFinalIntervalOperatorInfo* pInfo, SExpr bool isDeletedWindow(STimeWindow* pWin, uint64_t groupId, SAggSupporter* pSup) { SET_RES_WINDOW_KEY(pSup->keyBuf, &pWin->skey, sizeof(int64_t), groupId); - SResultRowPosition* p1 = (SResultRowPosition*)taosHashGet(pSup->pResultRowHashTable, pSup->keyBuf, + SResultRowPosition* p1 = (SResultRowPosition*)tSimpleHashGet(pSup->pResultRowHashTable, pSup->keyBuf, GET_RES_WINDOW_KEY_LEN(sizeof(int64_t))); return p1 == NULL; } @@ -3017,7 +3047,7 @@ static void doHashInterval(SOperatorInfo* pOperatorInfo, SSDataBlock* pSDataBloc } static void clearStreamIntervalOperator(SStreamFinalIntervalOperatorInfo* pInfo) { - taosHashClear(pInfo->aggSup.pResultRowHashTable); + tSimpleHashClear(pInfo->aggSup.pResultRowHashTable); clearDiskbasedBuf(pInfo->aggSup.pResultBuf); initResultRowInfo(&pInfo->binfo.resultRowInfo); } @@ -4918,14 +4948,14 @@ static int32_t outputMergeAlignedIntervalResult(SOperatorInfo* pOperatorInfo, ui SExprSupp* pSup = &pOperatorInfo->exprSupp; SET_RES_WINDOW_KEY(iaInfo->aggSup.keyBuf, &wstartTs, TSDB_KEYSIZE, tableGroupId); - SResultRowPosition* p1 = (SResultRowPosition*)taosHashGet(iaInfo->aggSup.pResultRowHashTable, iaInfo->aggSup.keyBuf, + SResultRowPosition* p1 = (SResultRowPosition*)tSimpleHashGet(iaInfo->aggSup.pResultRowHashTable, iaInfo->aggSup.keyBuf, GET_RES_WINDOW_KEY_LEN(TSDB_KEYSIZE)); ASSERT(p1 != NULL); finalizeResultRowIntoResultDataBlock(iaInfo->aggSup.pResultBuf, p1, pSup->pCtx, pSup->pExprInfo, pSup->numOfExprs, pSup->rowEntryInfoOffset, pResultBlock, pTaskInfo); - taosHashRemove(iaInfo->aggSup.pResultRowHashTable, iaInfo->aggSup.keyBuf, GET_RES_WINDOW_KEY_LEN(TSDB_KEYSIZE)); - ASSERT(taosHashGetSize(iaInfo->aggSup.pResultRowHashTable) == 0); + tSimpleHashRemove(iaInfo->aggSup.pResultRowHashTable, iaInfo->aggSup.keyBuf, GET_RES_WINDOW_KEY_LEN(TSDB_KEYSIZE)); + ASSERT(tSimpleHashGetSize(iaInfo->aggSup.pResultRowHashTable) == 0); return TSDB_CODE_SUCCESS; } @@ -4948,7 +4978,7 @@ static void doMergeAlignedIntervalAggImpl(SOperatorInfo* pOperatorInfo, SResultR // there is an result exists if (miaInfo->curTs != INT64_MIN) { - ASSERT(taosHashGetSize(iaInfo->aggSup.pResultRowHashTable) == 1); + ASSERT(tSimpleHashGetSize(iaInfo->aggSup.pResultRowHashTable) == 1); if (ts != miaInfo->curTs) { outputMergeAlignedIntervalResult(pOperatorInfo, tableGroupId, pResultBlock, miaInfo->curTs); @@ -4956,7 +4986,7 @@ static void doMergeAlignedIntervalAggImpl(SOperatorInfo* pOperatorInfo, SResultR } } else { miaInfo->curTs = ts; - ASSERT(taosHashGetSize(iaInfo->aggSup.pResultRowHashTable) == 0); + ASSERT(tSimpleHashGetSize(iaInfo->aggSup.pResultRowHashTable) == 0); } STimeWindow win = {0}; @@ -5032,7 +5062,7 @@ static void doMergeAlignedIntervalAgg(SOperatorInfo* pOperator) { if (pBlock == NULL) { // close last unfinalized time window if (miaInfo->curTs != INT64_MIN) { - ASSERT(taosHashGetSize(iaInfo->aggSup.pResultRowHashTable) == 1); + ASSERT(tSimpleHashGetSize(iaInfo->aggSup.pResultRowHashTable) == 1); outputMergeAlignedIntervalResult(pOperator, miaInfo->groupId, pRes, miaInfo->curTs); miaInfo->curTs = INT64_MIN; } @@ -5149,7 +5179,7 @@ SOperatorInfo* createMergeAlignedIntervalOperatorInfo(SOperatorInfo* downstream, iaInfo->timeWindowInterpo = timeWindowinterpNeeded(pSup->pCtx, num, iaInfo); if (iaInfo->timeWindowInterpo) { - iaInfo->binfo.resultRowInfo.openWindow = tdListNew(sizeof(SResultRowPosition)); + iaInfo->binfo.resultRowInfo.openWindow = tdListNew(sizeof(SOpenWindowInfo)); } initResultRowInfo(&iaInfo->binfo.resultRowInfo); @@ -5213,12 +5243,12 @@ static int32_t finalizeWindowResult(SOperatorInfo* pOperatorInfo, uint64_t table SExprSupp* pExprSup = &pOperatorInfo->exprSupp; SET_RES_WINDOW_KEY(iaInfo->aggSup.keyBuf, &win->skey, TSDB_KEYSIZE, tableGroupId); - SResultRowPosition* p1 = (SResultRowPosition*)taosHashGet(iaInfo->aggSup.pResultRowHashTable, iaInfo->aggSup.keyBuf, + SResultRowPosition* p1 = (SResultRowPosition*)tSimpleHashGet(iaInfo->aggSup.pResultRowHashTable, iaInfo->aggSup.keyBuf, GET_RES_WINDOW_KEY_LEN(TSDB_KEYSIZE)); ASSERT(p1 != NULL); finalizeResultRowIntoResultDataBlock(iaInfo->aggSup.pResultBuf, p1, pExprSup->pCtx, pExprSup->pExprInfo, pExprSup->numOfExprs, pExprSup->rowEntryInfoOffset, pResultBlock, pTaskInfo); - taosHashRemove(iaInfo->aggSup.pResultRowHashTable, iaInfo->aggSup.keyBuf, GET_RES_WINDOW_KEY_LEN(TSDB_KEYSIZE)); + tSimpleHashRemove(iaInfo->aggSup.pResultRowHashTable, iaInfo->aggSup.keyBuf, GET_RES_WINDOW_KEY_LEN(TSDB_KEYSIZE)); return TSDB_CODE_SUCCESS; } @@ -5284,7 +5314,7 @@ static void doMergeIntervalAggImpl(SOperatorInfo* pOperatorInfo, SResultRowInfo* // prev time window not interpolation yet. if (iaInfo->timeWindowInterpo) { - SResultRowPosition pos = addToOpenWindowList(pResultRowInfo, pResult); + SResultRowPosition pos = addToOpenWindowList(pResultRowInfo, pResult, tableGroupId); doInterpUnclosedTimeWindow(pOperatorInfo, numOfOutput, pResultRowInfo, pBlock, scanFlag, tsCols, &pos); // restore current time window @@ -5459,9 +5489,10 @@ SOperatorInfo* createMergeIntervalOperatorInfo(SOperatorInfo* downstream, SMerge initBasicInfo(&pIntervalInfo->binfo, pResBlock); initExecTimeWindowInfo(&pIntervalInfo->twAggSup.timeWindowData, &pIntervalInfo->win); + pIntervalInfo->timeWindowInterpo = timeWindowinterpNeeded(pExprSupp->pCtx, num, pIntervalInfo); if (pIntervalInfo->timeWindowInterpo) { - pIntervalInfo->binfo.resultRowInfo.openWindow = tdListNew(sizeof(SResultRowPosition)); + pIntervalInfo->binfo.resultRowInfo.openWindow = tdListNew(sizeof(SOpenWindowInfo)); if (pIntervalInfo->binfo.resultRowInfo.openWindow == NULL) { goto _error; } diff --git a/source/libs/executor/src/tsimplehash.c b/source/libs/executor/src/tsimplehash.c index 6b2edf0d5e..8cd376e092 100644 --- a/source/libs/executor/src/tsimplehash.c +++ b/source/libs/executor/src/tsimplehash.c @@ -31,21 +31,12 @@ taosMemoryFreeClear(_n); \ } while (0); -#pragma pack(push, 4) -typedef struct SHNode { - struct SHNode *next; - uint32_t keyLen : 20; - uint32_t dataLen : 12; - char data[]; -} SHNode; -#pragma pack(pop) - struct SSHashObj { SHNode **hashList; size_t capacity; // number of slots - int64_t size; // number of elements in hash table - _hash_fn_t hashFp; // hash function - _equal_fn_t equalFp; // equal function + int64_t size; // number of elements in hash table + _hash_fn_t hashFp; // hash function + _equal_fn_t equalFp; // equal function }; static FORCE_INLINE int32_t taosHashCapacity(int32_t length) { @@ -76,7 +67,6 @@ SSHashObj *tSimpleHashInit(size_t capacity, _hash_fn_t fn) { pHashObj->hashFp = fn; ASSERT((pHashObj->capacity & (pHashObj->capacity - 1)) == 0); - pHashObj->hashList = (SHNode **)taosMemoryCalloc(pHashObj->capacity, sizeof(void *)); if (!pHashObj->hashList) { taosMemoryFree(pHashObj); @@ -285,6 +275,44 @@ int32_t tSimpleHashRemove(SSHashObj *pHashObj, const void *key, size_t keyLen) { return TSDB_CODE_SUCCESS; } +int32_t tSimpleHashIterateRemove(SSHashObj *pHashObj, const void *key, size_t keyLen, void **pIter, int32_t *iter) { + if (!pHashObj || !key) { + return TSDB_CODE_FAILED; + } + + uint32_t hashVal = (*pHashObj->hashFp)(key, (uint32_t)keyLen); + + int32_t slot = HASH_INDEX(hashVal, pHashObj->capacity); + + SHNode *pNode = pHashObj->hashList[slot]; + SHNode *pPrev = NULL; + while (pNode) { + if ((*(pHashObj->equalFp))(GET_SHASH_NODE_KEY(pNode, pNode->dataLen), key, keyLen) == 0) { + if (!pPrev) { + pHashObj->hashList[slot] = pNode->next; + } else { + pPrev->next = pNode->next; + } + + if (*pIter == (void *)GET_SHASH_NODE_DATA(pNode)) { + if (!pPrev) { + *pIter = NULL; + } else { + *pIter = GET_SHASH_NODE_DATA(pPrev); + } + } + + FREE_HASH_NODE(pNode); + atomic_sub_fetch_64(&pHashObj->size, 1); + break; + } + pPrev = pNode; + pNode = pNode->next; + } + + return TSDB_CODE_SUCCESS; +} + void tSimpleHashClear(SSHashObj *pHashObj) { if (!pHashObj || taosHashTableEmpty(pHashObj)) { return; @@ -302,6 +330,7 @@ void tSimpleHashClear(SSHashObj *pHashObj) { FREE_HASH_NODE(pNode); pNode = pNext; } + pHashObj->hashList[i] = NULL; } atomic_store_64(&pHashObj->size, 0); } @@ -324,15 +353,6 @@ size_t tSimpleHashGetMemSize(const SSHashObj *pHashObj) { return (pHashObj->capacity * sizeof(void *)) + sizeof(SHNode) * tSimpleHashGetSize(pHashObj) + sizeof(SSHashObj); } -void *tSimpleHashGetKey(void *data, size_t *keyLen) { - SHNode *node = (SHNode *)((char *)data - offsetof(SHNode, data)); - if (keyLen) { - *keyLen = node->keyLen; - } - - return POINTER_SHIFT(data, node->dataLen); -} - void *tSimpleHashIterate(const SSHashObj *pHashObj, void *data, int32_t *iter) { if (!pHashObj) { return NULL; @@ -341,7 +361,7 @@ void *tSimpleHashIterate(const SSHashObj *pHashObj, void *data, int32_t *iter) { SHNode *pNode = NULL; if (!data) { - for (int32_t i = 0; i < pHashObj->capacity; ++i) { + for (int32_t i = *iter; i < pHashObj->capacity; ++i) { pNode = pHashObj->hashList[i]; if (!pNode) { continue; @@ -368,52 +388,5 @@ void *tSimpleHashIterate(const SSHashObj *pHashObj, void *data, int32_t *iter) { return GET_SHASH_NODE_DATA(pNode); } - return NULL; -} - -void *tSimpleHashIterateKV(const SSHashObj *pHashObj, void *data, void **key, int32_t *iter) { - if (!pHashObj) { - return NULL; - } - - SHNode *pNode = NULL; - - if (!data) { - for (int32_t i = 0; i < pHashObj->capacity; ++i) { - pNode = pHashObj->hashList[i]; - if (!pNode) { - continue; - } - *iter = i; - if (key) { - *key = GET_SHASH_NODE_KEY(pNode, pNode->dataLen); - } - return GET_SHASH_NODE_DATA(pNode); - } - return NULL; - } - - pNode = (SHNode *)((char *)data - offsetof(SHNode, data)); - - if (pNode->next) { - if (key) { - *key = GET_SHASH_NODE_KEY(pNode->next, pNode->next->dataLen); - } - return GET_SHASH_NODE_DATA(pNode->next); - } - - ++(*iter); - for (int32_t i = *iter; i < pHashObj->capacity; ++i) { - pNode = pHashObj->hashList[i]; - if (!pNode) { - continue; - } - *iter = i; - if (key) { - *key = GET_SHASH_NODE_KEY(pNode, pNode->dataLen); - } - return GET_SHASH_NODE_DATA(pNode); - } - return NULL; } \ No newline at end of file diff --git a/source/libs/executor/test/tSimpleHashTests.cpp b/source/libs/executor/test/tSimpleHashTests.cpp index acb6d434b4..3bf339ef90 100644 --- a/source/libs/executor/test/tSimpleHashTests.cpp +++ b/source/libs/executor/test/tSimpleHashTests.cpp @@ -30,7 +30,7 @@ // return RUN_ALL_TESTS(); // } -TEST(testCase, tSimpleHashTest) { +TEST(testCase, tSimpleHashTest_intKey) { SSHashObj *pHashObj = tSimpleHashInit(8, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT)); @@ -57,12 +57,14 @@ TEST(testCase, tSimpleHashTest) { int32_t iter = 0; int64_t keySum = 0; int64_t dataSum = 0; + size_t kLen = 0; while ((data = tSimpleHashIterate(pHashObj, data, &iter))) { - void *key = tSimpleHashGetKey(data, NULL); + void *key = tSimpleHashGetKey(data, &kLen); + ASSERT_EQ(keyLen, kLen); keySum += *(int64_t *)key; dataSum += *(int64_t *)data; } - + ASSERT_EQ(keySum, dataSum); ASSERT_EQ(keySum, originKeySum); @@ -74,4 +76,69 @@ TEST(testCase, tSimpleHashTest) { tSimpleHashCleanup(pHashObj); } + +TEST(testCase, tSimpleHashTest_binaryKey) { + SSHashObj *pHashObj = + tSimpleHashInit(8, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT)); + + assert(pHashObj != nullptr); + + ASSERT_EQ(0, tSimpleHashGetSize(pHashObj)); + + typedef struct { + int64_t suid; + int64_t uid; + } SCombineKey; + + size_t keyLen = sizeof(SCombineKey); + size_t dataLen = sizeof(int64_t); + + int64_t originDataSum = 0; + SCombineKey combineKey = {0}; + for (int64_t i = 1; i <= 100; ++i) { + combineKey.suid = i; + combineKey.uid = i + 1; + tSimpleHashPut(pHashObj, (const void *)&combineKey, keyLen, (const void *)&i, dataLen); + originDataSum += i; + ASSERT_EQ(i, tSimpleHashGetSize(pHashObj)); + } + + for (int64_t i = 1; i <= 100; ++i) { + combineKey.suid = i; + combineKey.uid = i + 1; + void *data = tSimpleHashGet(pHashObj, (const void *)&combineKey, keyLen); + ASSERT_EQ(i, *(int64_t *)data); + } + + void *data = NULL; + int32_t iter = 0; + int64_t keySum = 0; + int64_t dataSum = 0; + size_t kLen = 0; + while ((data = tSimpleHashIterate(pHashObj, data, &iter))) { + void *key = tSimpleHashGetKey(data, &kLen); + ASSERT_EQ(keyLen, kLen); + dataSum += *(int64_t *)data; + } + + ASSERT_EQ(originDataSum, dataSum); + + tSimpleHashRemove(pHashObj, (const void *)&combineKey, keyLen); + + while ((data = tSimpleHashIterate(pHashObj, data, &iter))) { + void *key = tSimpleHashGetKey(data, &kLen); + ASSERT_EQ(keyLen, kLen); + } + + for (int64_t i = 1; i <= 99; ++i) { + combineKey.suid = i; + combineKey.uid = i + 1; + tSimpleHashRemove(pHashObj, (const void *)&combineKey, keyLen); + ASSERT_EQ(99 - i, tSimpleHashGetSize(pHashObj)); + } + + tSimpleHashCleanup(pHashObj); +} + + #pragma GCC diagnostic pop \ No newline at end of file diff --git a/source/libs/function/src/builtins.c b/source/libs/function/src/builtins.c index ed82e4cb50..b7cd02befd 100644 --- a/source/libs/function/src/builtins.c +++ b/source/libs/function/src/builtins.c @@ -303,7 +303,7 @@ static int32_t translateInOutStr(SFunctionNode* pFunc, char* pErrBuf, int32_t le } SExprNode* pPara1 = (SExprNode*)nodesListGetNode(pFunc->pParameterList, 0); - if (!IS_VAR_DATA_TYPE(pPara1->resType.type)) { + if (!IS_STR_DATA_TYPE(pPara1->resType.type)) { return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); } @@ -317,7 +317,7 @@ static int32_t translateTrimStr(SFunctionNode* pFunc, char* pErrBuf, int32_t len } SExprNode* pPara1 = (SExprNode*)nodesListGetNode(pFunc->pParameterList, 0); - if (!IS_VAR_DATA_TYPE(pPara1->resType.type)) { + if (!IS_STR_DATA_TYPE(pPara1->resType.type)) { return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); } @@ -546,7 +546,7 @@ static int32_t translateApercentile(SFunctionNode* pFunc, char* pErrBuf, int32_t // param2 if (3 == numOfParams) { uint8_t para3Type = ((SExprNode*)nodesListGetNode(pFunc->pParameterList, 2))->resType.type; - if (!IS_VAR_DATA_TYPE(para3Type)) { + if (!IS_STR_DATA_TYPE(para3Type)) { return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); } @@ -593,7 +593,7 @@ static int32_t translateApercentileImpl(SFunctionNode* pFunc, char* pErrBuf, int // param2 if (3 == numOfParams) { uint8_t para3Type = ((SExprNode*)nodesListGetNode(pFunc->pParameterList, 2))->resType.type; - if (!IS_VAR_DATA_TYPE(para3Type)) { + if (!IS_STR_DATA_TYPE(para3Type)) { return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); } @@ -1388,7 +1388,7 @@ static int32_t translateSample(SFunctionNode* pFunc, char* pErrBuf, int32_t len) } // set result type - if (IS_VAR_DATA_TYPE(colType)) { + if (IS_STR_DATA_TYPE(colType)) { pFunc->node.resType = (SDataType){.bytes = pCol->resType.bytes, .type = colType}; } else { pFunc->node.resType = (SDataType){.bytes = tDataTypes[colType].bytes, .type = colType}; @@ -1431,7 +1431,7 @@ static int32_t translateTail(SFunctionNode* pFunc, char* pErrBuf, int32_t len) { } // set result type - if (IS_VAR_DATA_TYPE(colType)) { + if (IS_STR_DATA_TYPE(colType)) { pFunc->node.resType = (SDataType){.bytes = pCol->resType.bytes, .type = colType}; } else { pFunc->node.resType = (SDataType){.bytes = tDataTypes[colType].bytes, .type = colType}; @@ -1514,7 +1514,7 @@ static int32_t translateInterp(SFunctionNode* pFunc, char* pErrBuf, int32_t len) for (int32_t i = 1; i < 3; ++i) { nodeType = nodeType(nodesListGetNode(pFunc->pParameterList, i)); paraType = ((SExprNode*)nodesListGetNode(pFunc->pParameterList, i))->resType.type; - if (!IS_VAR_DATA_TYPE(paraType) || QUERY_NODE_VALUE != nodeType) { + if (!IS_STR_DATA_TYPE(paraType) || QUERY_NODE_VALUE != nodeType) { return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); } @@ -1682,7 +1682,7 @@ static int32_t translateLength(SFunctionNode* pFunc, char* pErrBuf, int32_t len) return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName); } - if (!IS_VAR_DATA_TYPE(((SExprNode*)nodesListGetNode(pFunc->pParameterList, 0))->resType.type)) { + if (!IS_STR_DATA_TYPE(((SExprNode*)nodesListGetNode(pFunc->pParameterList, 0))->resType.type)) { return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); } @@ -1714,7 +1714,7 @@ static int32_t translateConcatImpl(SFunctionNode* pFunc, char* pErrBuf, int32_t for (int32_t i = 0; i < numOfParams; ++i) { SNode* pPara = nodesListGetNode(pFunc->pParameterList, i); uint8_t paraType = ((SExprNode*)pPara)->resType.type; - if (!IS_VAR_DATA_TYPE(paraType) && !IS_NULL_TYPE(paraType)) { + if (!IS_STR_DATA_TYPE(paraType) && !IS_NULL_TYPE(paraType)) { return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); } if (TSDB_DATA_TYPE_NCHAR == paraType) { @@ -1770,7 +1770,7 @@ static int32_t translateSubstr(SFunctionNode* pFunc, char* pErrBuf, int32_t len) uint8_t para0Type = pPara0->resType.type; uint8_t para1Type = pPara1->resType.type; - if (!IS_VAR_DATA_TYPE(para0Type) || !IS_INTEGER_TYPE(para1Type)) { + if (!IS_STR_DATA_TYPE(para0Type) || !IS_INTEGER_TYPE(para1Type)) { return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); } @@ -1802,7 +1802,7 @@ static int32_t translateCast(SFunctionNode* pFunc, char* pErrBuf, int32_t len) { uint8_t para2Type = pFunc->node.resType.type; int32_t para2Bytes = pFunc->node.resType.bytes; - if (IS_VAR_DATA_TYPE(para2Type)) { + if (IS_STR_DATA_TYPE(para2Type)) { para2Bytes -= VARSTR_HEADER_SIZE; } if (para2Bytes <= 0 || para2Bytes > 4096) { // cast dst var type length limits to 4096 bytes @@ -1859,7 +1859,7 @@ static int32_t translateToUnixtimestamp(SFunctionNode* pFunc, char* pErrBuf, int return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName); } - if (!IS_VAR_DATA_TYPE(((SExprNode*)nodesListGetNode(pFunc->pParameterList, 0))->resType.type)) { + if (!IS_STR_DATA_TYPE(((SExprNode*)nodesListGetNode(pFunc->pParameterList, 0))->resType.type)) { return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); } @@ -1878,7 +1878,7 @@ static int32_t translateTimeTruncate(SFunctionNode* pFunc, char* pErrBuf, int32_ uint8_t para1Type = ((SExprNode*)nodesListGetNode(pFunc->pParameterList, 0))->resType.type; uint8_t para2Type = ((SExprNode*)nodesListGetNode(pFunc->pParameterList, 1))->resType.type; - if ((!IS_VAR_DATA_TYPE(para1Type) && !IS_INTEGER_TYPE(para1Type) && TSDB_DATA_TYPE_TIMESTAMP != para1Type) || + if ((!IS_STR_DATA_TYPE(para1Type) && !IS_INTEGER_TYPE(para1Type) && TSDB_DATA_TYPE_TIMESTAMP != para1Type) || !IS_INTEGER_TYPE(para2Type)) { return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); } @@ -1911,7 +1911,7 @@ static int32_t translateTimeDiff(SFunctionNode* pFunc, char* pErrBuf, int32_t le for (int32_t i = 0; i < 2; ++i) { uint8_t paraType = ((SExprNode*)nodesListGetNode(pFunc->pParameterList, i))->resType.type; - if (!IS_VAR_DATA_TYPE(paraType) && !IS_INTEGER_TYPE(paraType) && TSDB_DATA_TYPE_TIMESTAMP != paraType) { + if (!IS_STR_DATA_TYPE(paraType) && !IS_INTEGER_TYPE(paraType) && TSDB_DATA_TYPE_TIMESTAMP != paraType) { return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); } } diff --git a/source/libs/function/src/udfd.c b/source/libs/function/src/udfd.c index 1cbc78df48..5b27e030b9 100644 --- a/source/libs/function/src/udfd.c +++ b/source/libs/function/src/udfd.c @@ -84,6 +84,7 @@ typedef struct SUdf { TUdfAggStartFunc aggStartFunc; TUdfAggProcessFunc aggProcFunc; TUdfAggFinishFunc aggFinishFunc; + TUdfAggMergeFunc aggMergeFunc; TUdfInitFunc initFunc; TUdfDestroyFunc destroyFunc; @@ -271,6 +272,15 @@ void udfdProcessCallRequest(SUvUdfWork *uvUdf, SUdfRequest *request) { break; } + case TSDB_UDF_CALL_AGG_MERGE: { + SUdfInterBuf outBuf = {.buf = taosMemoryMalloc(udf->bufSize), .bufLen = udf->bufSize, .numOfResult = 0}; + code = udf->aggMergeFunc(&call->interBuf, &call->interBuf2, &outBuf); + freeUdfInterBuf(&call->interBuf); + freeUdfInterBuf(&call->interBuf2); + subRsp->resultBuf = outBuf; + + break; + } case TSDB_UDF_CALL_AGG_FIN: { SUdfInterBuf outBuf = {.buf = taosMemoryMalloc(udf->bufSize), .bufLen = udf->bufSize, .numOfResult = 0}; code = udf->aggFinishFunc(&call->interBuf, &outBuf); @@ -309,6 +319,10 @@ void udfdProcessCallRequest(SUvUdfWork *uvUdf, SUdfRequest *request) { freeUdfInterBuf(&subRsp->resultBuf); break; } + case TSDB_UDF_CALL_AGG_MERGE: { + freeUdfInterBuf(&subRsp->resultBuf); + break; + } case TSDB_UDF_CALL_AGG_FIN: { freeUdfInterBuf(&subRsp->resultBuf); break; @@ -560,7 +574,11 @@ int32_t udfdLoadUdf(char *udfName, SUdf *udf) { strncpy(finishFuncName, processFuncName, strlen(processFuncName)); strncat(finishFuncName, finishSuffix, strlen(finishSuffix)); uv_dlsym(&udf->lib, finishFuncName, (void **)(&udf->aggFinishFunc)); - // TODO: merge + char mergeFuncName[TSDB_FUNC_NAME_LEN + 6] = {0}; + char *mergeSuffix = "_merge"; + strncpy(finishFuncName, processFuncName, strlen(processFuncName)); + strncat(finishFuncName, mergeSuffix, strlen(mergeSuffix)); + uv_dlsym(&udf->lib, finishFuncName, (void **)(&udf->aggMergeFunc)); } return 0; } diff --git a/source/libs/nodes/src/nodesCloneFuncs.c b/source/libs/nodes/src/nodesCloneFuncs.c index 9390d129df..83bccbffb4 100644 --- a/source/libs/nodes/src/nodesCloneFuncs.c +++ b/source/libs/nodes/src/nodesCloneFuncs.c @@ -545,6 +545,7 @@ static int32_t physiSysTableScanCopy(const SSystemTableScanPhysiNode* pSrc, SSys COPY_OBJECT_FIELD(mgmtEpSet, sizeof(SEpSet)); COPY_SCALAR_FIELD(showRewrite); COPY_SCALAR_FIELD(accountId); + COPY_SCALAR_FIELD(sysInfo); return TSDB_CODE_SUCCESS; } diff --git a/source/libs/nodes/src/nodesCodeFuncs.c b/source/libs/nodes/src/nodesCodeFuncs.c index 0f32001c47..822bdec365 100644 --- a/source/libs/nodes/src/nodesCodeFuncs.c +++ b/source/libs/nodes/src/nodesCodeFuncs.c @@ -1654,6 +1654,7 @@ static int32_t jsonToPhysiTableScanNode(const SJson* pJson, void* pObj) { static const char* jkSysTableScanPhysiPlanMnodeEpSet = "MnodeEpSet"; static const char* jkSysTableScanPhysiPlanShowRewrite = "ShowRewrite"; static const char* jkSysTableScanPhysiPlanAccountId = "AccountId"; +static const char* jkSysTableScanPhysiPlanSysInfo = "SysInfo"; static int32_t physiSysTableScanNodeToJson(const void* pObj, SJson* pJson) { const SSystemTableScanPhysiNode* pNode = (const SSystemTableScanPhysiNode*)pObj; @@ -1668,6 +1669,9 @@ static int32_t physiSysTableScanNodeToJson(const void* pObj, SJson* pJson) { if (TSDB_CODE_SUCCESS == code) { code = tjsonAddIntegerToObject(pJson, jkSysTableScanPhysiPlanAccountId, pNode->accountId); } + if (TSDB_CODE_SUCCESS == code) { + code = tjsonAddBoolToObject(pJson, jkSysTableScanPhysiPlanSysInfo, pNode->sysInfo); + } return code; } @@ -1684,7 +1688,9 @@ static int32_t jsonToPhysiSysTableScanNode(const SJson* pJson, void* pObj) { } if (TSDB_CODE_SUCCESS == code) { tjsonGetNumberValue(pJson, jkSysTableScanPhysiPlanAccountId, pNode->accountId, code); - ; + } + if (TSDB_CODE_SUCCESS == code) { + code = tjsonGetBoolValue(pJson, jkSysTableScanPhysiPlanSysInfo, &pNode->sysInfo); } return code; diff --git a/source/libs/nodes/src/nodesToSQLFuncs.c b/source/libs/nodes/src/nodesToSQLFuncs.c index e521c57c3d..9325d02886 100644 --- a/source/libs/nodes/src/nodesToSQLFuncs.c +++ b/source/libs/nodes/src/nodesToSQLFuncs.c @@ -135,7 +135,12 @@ int32_t nodesNodeToSQL(SNode *pNode, char *buf, int32_t bufSize, int32_t *len) { NODES_ERR_RET(TSDB_CODE_QRY_APP_ERROR); } - *len += snprintf(buf + *len, bufSize - *len, "%s", t); + int32_t tlen = strlen(t); + if (tlen > 32) { + *len += snprintf(buf + *len, bufSize - *len, "%.*s...%s", 32, t, t + tlen - 1); + } else { + *len += snprintf(buf + *len, bufSize - *len, "%s", t); + } taosMemoryFree(t); return TSDB_CODE_SUCCESS; @@ -199,12 +204,17 @@ int32_t nodesNodeToSQL(SNode *pNode, char *buf, int32_t bufSize, int32_t *len) { SNodeListNode *pListNode = (SNodeListNode *)pNode; SNode *node = NULL; bool first = true; + int32_t num = 0; *len += snprintf(buf + *len, bufSize - *len, "("); FOREACH(node, pListNode->pNodeList) { if (!first) { *len += snprintf(buf + *len, bufSize - *len, ", "); + if (++num >= 10) { + *len += snprintf(buf + *len, bufSize - *len, "..."); + break; + } } NODES_ERR_RET(nodesNodeToSQL(node, buf, bufSize, len)); first = false; diff --git a/source/libs/parser/inc/sql.y b/source/libs/parser/inc/sql.y index 56e68d8374..9bff061d02 100644 --- a/source/libs/parser/inc/sql.y +++ b/source/libs/parser/inc/sql.y @@ -495,12 +495,9 @@ bufsize_opt(A) ::= BUFSIZE NK_INTEGER(B). /************************************************ create/drop stream **************************************************/ cmd ::= CREATE STREAM not_exists_opt(E) stream_name(A) - stream_options(B) into_opt(C) AS query_expression(D). { pCxt->pRootNode = createCreateStreamStmt(pCxt, E, &A, C, B, D); } + stream_options(B) INTO full_table_name(C) AS query_expression(D). { pCxt->pRootNode = createCreateStreamStmt(pCxt, E, &A, C, B, D); } cmd ::= DROP STREAM exists_opt(A) stream_name(B). { pCxt->pRootNode = createDropStreamStmt(pCxt, A, &B); } -into_opt(A) ::= . { A = NULL; } -into_opt(A) ::= INTO full_table_name(B). { A = B; } - stream_options(A) ::= . { A = createStreamOptions(pCxt); } stream_options(A) ::= stream_options(B) TRIGGER AT_ONCE. { ((SStreamOptions*)B)->triggerType = STREAM_TRIGGER_AT_ONCE; A = B; } stream_options(A) ::= stream_options(B) TRIGGER WINDOW_CLOSE. { ((SStreamOptions*)B)->triggerType = STREAM_TRIGGER_WINDOW_CLOSE; A = B; } diff --git a/source/libs/parser/src/parAstParser.c b/source/libs/parser/src/parAstParser.c index ffa7729745..82b5842663 100644 --- a/source/libs/parser/src/parAstParser.c +++ b/source/libs/parser/src/parAstParser.c @@ -464,6 +464,9 @@ static int32_t collectMetaKeyFromShowCreateTable(SCollectMetaKeyCxt* pCxt, SShow if (TSDB_CODE_SUCCESS == code) { code = reserveDbCfgInCache(pCxt->pParseCxt->acctId, pStmt->dbName, pCxt->pMetaCache); } + if (TSDB_CODE_SUCCESS == code) { + code = reserveUserAuthInCacheExt(pCxt->pParseCxt->pUser, &name, AUTH_TYPE_READ, pCxt->pMetaCache); + } return code; } diff --git a/source/libs/parser/src/parAuthenticator.c b/source/libs/parser/src/parAuthenticator.c index befc822808..d9a5761d99 100644 --- a/source/libs/parser/src/parAuthenticator.c +++ b/source/libs/parser/src/parAuthenticator.c @@ -96,6 +96,10 @@ static int32_t authInsert(SAuthCxt* pCxt, SInsertStmt* pInsert) { return code; } +static int32_t authShowCreateTable(SAuthCxt* pCxt, SShowCreateTableStmt* pStmt) { + return checkAuth(pCxt, pStmt->dbName, AUTH_TYPE_READ); +} + static int32_t authQuery(SAuthCxt* pCxt, SNode* pStmt) { switch (nodeType(pStmt)) { case QUERY_NODE_SET_OPERATOR: @@ -108,6 +112,24 @@ static int32_t authQuery(SAuthCxt* pCxt, SNode* pStmt) { return authDelete(pCxt, (SDeleteStmt*)pStmt); case QUERY_NODE_INSERT_STMT: return authInsert(pCxt, (SInsertStmt*)pStmt); + case QUERY_NODE_SHOW_DNODES_STMT: + case QUERY_NODE_SHOW_MNODES_STMT: + case QUERY_NODE_SHOW_MODULES_STMT: + case QUERY_NODE_SHOW_QNODES_STMT: + case QUERY_NODE_SHOW_SNODES_STMT: + case QUERY_NODE_SHOW_BNODES_STMT: + case QUERY_NODE_SHOW_CLUSTER_STMT: + case QUERY_NODE_SHOW_LICENCES_STMT: + case QUERY_NODE_SHOW_VGROUPS_STMT: + case QUERY_NODE_SHOW_VARIABLES_STMT: + case QUERY_NODE_SHOW_CREATE_DATABASE_STMT: + case QUERY_NODE_SHOW_TABLE_DISTRIBUTED_STMT: + case QUERY_NODE_SHOW_VNODES_STMT: + case QUERY_NODE_SHOW_SCORES_STMT: + return !pCxt->pParseCxt->enableSysInfo ? TSDB_CODE_PAR_PERMISSION_DENIED : TSDB_CODE_SUCCESS; + case QUERY_NODE_SHOW_CREATE_TABLE_STMT: + case QUERY_NODE_SHOW_CREATE_STABLE_STMT: + return authShowCreateTable(pCxt, (SShowCreateTableStmt*)pStmt); default: break; } diff --git a/source/libs/parser/src/parInsert.c b/source/libs/parser/src/parInsert.c index 4e32672697..c9115d90e1 100644 --- a/source/libs/parser/src/parInsert.c +++ b/source/libs/parser/src/parInsert.c @@ -1715,7 +1715,7 @@ static int32_t skipUsingClause(SInsertParseSyntaxCxt* pCxt) { } static int32_t collectTableMetaKey(SInsertParseSyntaxCxt* pCxt, bool isStable, int32_t tableNo, SToken* pTbToken) { - SName name; + SName name = {0}; CHECK_CODE(createSName(&name, pTbToken, pCxt->pComCxt->acctId, pCxt->pComCxt->db, &pCxt->msg)); CHECK_CODE(reserveTableMetaInCacheForInsert(&name, isStable ? CATALOG_REQ_TYPE_META : CATALOG_REQ_TYPE_BOTH, tableNo, pCxt->pMetaCache)); @@ -1730,7 +1730,7 @@ static int32_t checkTableName(const char* pTableName, SMsgBuf* pMsgBuf) { } static int32_t collectAutoCreateTableMetaKey(SInsertParseSyntaxCxt* pCxt, int32_t tableNo, SToken* pTbToken) { - SName name; + SName name = {0}; CHECK_CODE(createSName(&name, pTbToken, pCxt->pComCxt->acctId, pCxt->pComCxt->db, &pCxt->msg)); CHECK_CODE(checkTableName(name.tname, &pCxt->msg)); CHECK_CODE(reserveTableMetaInCacheForInsert(&name, CATALOG_REQ_TYPE_VGROUP, tableNo, pCxt->pMetaCache)); diff --git a/source/libs/parser/src/parTranslater.c b/source/libs/parser/src/parTranslater.c index 8a1d8763bf..d938325ef2 100644 --- a/source/libs/parser/src/parTranslater.c +++ b/source/libs/parser/src/parTranslater.c @@ -784,6 +784,9 @@ static int32_t createColumnsByTable(STranslateContext* pCxt, const STableNode* p int32_t nums = pMeta->tableInfo.numOfColumns + (igTags ? 0 : ((TSDB_SUPER_TABLE == pMeta->tableType) ? pMeta->tableInfo.numOfTags : 0)); for (int32_t i = 0; i < nums; ++i) { + if (invisibleColumn(pCxt->pParseCxt->enableSysInfo, pMeta->tableType, pMeta->schema[i].flags)) { + continue; + } SColumnNode* pCol = (SColumnNode*)nodesMakeNode(QUERY_NODE_COLUMN); if (NULL == pCol) { return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_OUT_OF_MEMORY); @@ -826,7 +829,8 @@ static int32_t findAndSetColumn(STranslateContext* pCxt, SColumnNode** pColRef, } int32_t nums = pMeta->tableInfo.numOfTags + pMeta->tableInfo.numOfColumns; for (int32_t i = 0; i < nums; ++i) { - if (0 == strcmp(pCol->colName, pMeta->schema[i].name)) { + if (0 == strcmp(pCol->colName, pMeta->schema[i].name) && + !invisibleColumn(pCxt->pParseCxt->enableSysInfo, pMeta->tableType, pMeta->schema[i].flags)) { setColumnInfoBySchema((SRealTableNode*)pTable, pMeta->schema + i, (i - pMeta->tableInfo.numOfColumns), pCol); *pFound = true; break; @@ -2192,14 +2196,14 @@ static int32_t translateTable(STranslateContext* pCxt, SNode* pTable) { code = setTableCacheLastMode(pCxt, &name, pRealTable); } } - pRealTable->table.precision = pRealTable->pMeta->tableInfo.precision; - pRealTable->table.singleTable = isSingleTable(pRealTable); if (TSDB_CODE_SUCCESS == code) { + pRealTable->table.precision = pRealTable->pMeta->tableInfo.precision; + pRealTable->table.singleTable = isSingleTable(pRealTable); + if (TSDB_SUPER_TABLE == pRealTable->pMeta->tableType) { + pCxt->stableQuery = true; + } code = addNamespace(pCxt, pRealTable); } - if (TSDB_SUPER_TABLE == pRealTable->pMeta->tableType) { - pCxt->stableQuery = true; - } break; } case QUERY_NODE_TEMP_TABLE: { @@ -2594,8 +2598,7 @@ static int32_t getQueryTimeRange(STranslateContext* pCxt, SNode* pWhere, STimeWi return code; } -static int32_t checkFill(STranslateContext* pCxt, SFillNode* pFill, SValueNode* pInterval, - bool isInterpFill) { +static int32_t checkFill(STranslateContext* pCxt, SFillNode* pFill, SValueNode* pInterval, bool isInterpFill) { if (FILL_MODE_NONE == pFill->mode) { if (isInterpFill) { return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_WRONG_VALUE_TYPE, "Unsupported fill type"); @@ -4998,7 +5001,7 @@ static int32_t checkCreateStream(STranslateContext* pCxt, SCreateStreamStmt* pSt return TSDB_CODE_SUCCESS; } - if (QUERY_NODE_SELECT_STMT != nodeType(pStmt->pQuery) || + if (QUERY_NODE_SELECT_STMT != nodeType(pStmt->pQuery) || NULL == ((SSelectStmt*)pStmt->pQuery)->pFromTable || QUERY_NODE_REAL_TABLE != nodeType(((SSelectStmt*)pStmt->pQuery)->pFromTable)) { return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_STREAM_QUERY, "Unsupported stream query"); } diff --git a/source/libs/parser/src/parser.c b/source/libs/parser/src/parser.c index 7e27132f3c..7ee6a5b223 100644 --- a/source/libs/parser/src/parser.c +++ b/source/libs/parser/src/parser.c @@ -136,8 +136,7 @@ static int32_t setValueByBindParam(SValueNode* pVal, TAOS_MULTI_BIND* pParam) { } static EDealRes rewriteQueryExprAliasImpl(SNode* pNode, void* pContext) { - if (nodesIsExprNode(pNode) && QUERY_NODE_COLUMN != nodeType(pNode) && '\0' == ((SExprNode*)pNode)->userAlias[0]) { - strcpy(((SExprNode*)pNode)->userAlias, ((SExprNode*)pNode)->aliasName); + if (nodesIsExprNode(pNode) && QUERY_NODE_COLUMN != nodeType(pNode)) { sprintf(((SExprNode*)pNode)->aliasName, "#%d", *(int32_t*)pContext); ++(*(int32_t*)pContext); } diff --git a/source/libs/parser/src/sql.c b/source/libs/parser/src/sql.c index c820e955d7..c4bd1aff04 100644 --- a/source/libs/parser/src/sql.c +++ b/source/libs/parser/src/sql.c @@ -104,26 +104,26 @@ #endif /************* Begin control #defines *****************************************/ #define YYCODETYPE unsigned short int -#define YYNOCODE 427 +#define YYNOCODE 426 #define YYACTIONTYPE unsigned short int #define ParseTOKENTYPE SToken typedef union { int yyinit; ParseTOKENTYPE yy0; - SAlterOption yy95; - EOperatorType yy198; - EOrder yy204; - int8_t yy215; - ENullOrder yy277; - bool yy313; - int64_t yy473; - SNodeList* yy544; - SToken yy617; - EJoinType yy708; - SDataType yy784; - EFillMode yy816; - SNode* yy840; - int32_t yy844; + SAlterOption yy5; + int8_t yy59; + int64_t yy69; + EJoinType yy156; + SNodeList* yy172; + EFillMode yy186; + SToken yy209; + int32_t yy232; + SNode* yy272; + bool yy293; + EOperatorType yy392; + ENullOrder yy493; + SDataType yy616; + EOrder yy818; } YYMINORTYPE; #ifndef YYSTACKDEPTH #define YYSTACKDEPTH 100 @@ -140,16 +140,16 @@ typedef union { #define ParseCTX_STORE #define YYFALLBACK 1 #define YYNSTATE 667 -#define YYNRULE 491 +#define YYNRULE 489 #define YYNTOKEN 305 #define YY_MAX_SHIFT 666 -#define YY_MIN_SHIFTREDUCE 973 -#define YY_MAX_SHIFTREDUCE 1463 -#define YY_ERROR_ACTION 1464 -#define YY_ACCEPT_ACTION 1465 -#define YY_NO_ACTION 1466 -#define YY_MIN_REDUCE 1467 -#define YY_MAX_REDUCE 1957 +#define YY_MIN_SHIFTREDUCE 972 +#define YY_MAX_SHIFTREDUCE 1460 +#define YY_ERROR_ACTION 1461 +#define YY_ACCEPT_ACTION 1462 +#define YY_NO_ACTION 1463 +#define YY_MIN_REDUCE 1464 +#define YY_MAX_REDUCE 1952 /************* End control #defines *******************************************/ #define YY_NLOOKAHEAD ((int)(sizeof(yy_lookahead)/sizeof(yy_lookahead[0]))) @@ -216,694 +216,650 @@ typedef union { ** yy_default[] Default action for each state. ** *********** Begin parsing tables **********************************************/ -#define YY_ACTTAB_COUNT (2548) +#define YY_ACTTAB_COUNT (2259) static const YYACTIONTYPE yy_action[] = { - /* 0 */ 526, 30, 261, 526, 549, 433, 526, 434, 1502, 11, - /* 10 */ 10, 117, 39, 37, 55, 1653, 1654, 117, 471, 378, - /* 20 */ 339, 1468, 1264, 1006, 476, 1023, 1290, 1022, 1607, 1791, - /* 30 */ 1598, 1607, 127, 1340, 1607, 1262, 441, 552, 434, 1502, - /* 40 */ 469, 1775, 107, 1779, 1290, 106, 105, 104, 103, 102, - /* 50 */ 101, 100, 99, 98, 1775, 1024, 1335, 1809, 150, 64, - /* 60 */ 1935, 14, 1567, 1010, 1011, 553, 1771, 1777, 1270, 450, - /* 70 */ 1761, 125, 577, 165, 39, 37, 1403, 1932, 571, 1771, - /* 80 */ 1777, 328, 339, 1529, 1264, 551, 161, 1877, 1878, 1, - /* 90 */ 1882, 571, 1659, 479, 478, 1340, 1823, 1262, 1376, 327, - /* 100 */ 95, 1792, 580, 1794, 1795, 576, 496, 571, 1657, 344, - /* 110 */ 1869, 663, 1652, 1654, 330, 1865, 160, 513, 1335, 494, - /* 120 */ 1935, 492, 1289, 14, 325, 1342, 1343, 1705, 164, 543, - /* 130 */ 1270, 1161, 1162, 1934, 33, 32, 1895, 1932, 40, 38, - /* 140 */ 36, 35, 34, 148, 63, 1479, 640, 639, 638, 637, + /* 0 */ 433, 1930, 434, 1499, 1593, 441, 526, 434, 1499, 513, + /* 10 */ 30, 260, 39, 37, 1929, 326, 325, 117, 1927, 1702, + /* 20 */ 339, 1465, 1261, 146, 471, 40, 38, 36, 35, 34, + /* 30 */ 1786, 552, 1606, 1337, 1604, 1259, 344, 552, 1287, 1649, + /* 40 */ 1651, 378, 107, 1774, 526, 106, 105, 104, 103, 102, + /* 50 */ 101, 100, 99, 98, 1770, 117, 1332, 432, 1804, 64, + /* 60 */ 436, 14, 476, 36, 35, 34, 553, 148, 1267, 1476, + /* 70 */ 450, 1756, 1604, 577, 39, 37, 1400, 1595, 1766, 1772, + /* 80 */ 328, 1930, 339, 1526, 1261, 1804, 217, 1286, 1770, 1, + /* 90 */ 571, 1005, 1656, 542, 164, 1337, 1818, 1259, 1927, 327, + /* 100 */ 95, 1787, 580, 1789, 1790, 576, 43, 571, 1654, 158, + /* 110 */ 1864, 663, 1766, 1772, 330, 1860, 159, 513, 1332, 63, + /* 120 */ 1930, 78, 1643, 14, 571, 1339, 1340, 1703, 163, 541, + /* 130 */ 1267, 1009, 1010, 165, 33, 32, 1890, 1927, 40, 38, + /* 140 */ 36, 35, 34, 543, 63, 63, 640, 639, 638, 637, /* 150 */ 349, 2, 636, 635, 128, 630, 629, 628, 627, 626, /* 160 */ 625, 624, 139, 620, 619, 618, 348, 347, 615, 614, - /* 170 */ 1265, 107, 1263, 663, 106, 105, 104, 103, 102, 101, - /* 180 */ 100, 99, 98, 1809, 36, 35, 34, 1342, 1343, 224, - /* 190 */ 225, 542, 384, 1268, 1269, 613, 1317, 1318, 1320, 1321, - /* 200 */ 1322, 1323, 1324, 1325, 573, 569, 1333, 1334, 1336, 1337, - /* 210 */ 1338, 1339, 1341, 1344, 1467, 1288, 1434, 33, 32, 482, - /* 220 */ 481, 40, 38, 36, 35, 34, 123, 168, 541, 303, - /* 230 */ 1465, 223, 1265, 84, 1263, 1264, 477, 480, 116, 115, - /* 240 */ 114, 113, 112, 111, 110, 109, 108, 305, 1262, 1023, - /* 250 */ 516, 1022, 22, 174, 1600, 1268, 1269, 1490, 1317, 1318, - /* 260 */ 1320, 1321, 1322, 1323, 1324, 1325, 573, 569, 1333, 1334, - /* 270 */ 1336, 1337, 1338, 1339, 1341, 1344, 39, 37, 1489, 1024, - /* 280 */ 538, 1270, 168, 526, 339, 71, 1264, 1488, 70, 354, - /* 290 */ 1244, 1245, 1708, 1791, 170, 211, 512, 1340, 1761, 1262, - /* 300 */ 1119, 602, 601, 600, 1123, 599, 1125, 1126, 598, 1128, - /* 310 */ 595, 1607, 1134, 592, 1136, 1137, 589, 586, 1935, 1761, - /* 320 */ 1335, 1809, 1584, 1270, 663, 14, 1659, 1935, 1761, 553, - /* 330 */ 1935, 166, 1270, 343, 1761, 1932, 577, 1935, 39, 37, - /* 340 */ 1933, 487, 1657, 165, 1932, 552, 339, 1932, 1264, 549, - /* 350 */ 165, 76, 305, 2, 1932, 516, 497, 544, 539, 1340, - /* 360 */ 1823, 1262, 1698, 159, 95, 1792, 580, 1794, 1795, 576, - /* 370 */ 210, 571, 63, 173, 1869, 663, 1646, 127, 330, 1865, - /* 380 */ 160, 552, 1335, 1265, 490, 1263, 419, 605, 484, 1342, - /* 390 */ 1343, 33, 32, 209, 1270, 40, 38, 36, 35, 34, - /* 400 */ 1896, 634, 632, 39, 37, 1345, 1268, 1269, 1487, 91, - /* 410 */ 622, 339, 1791, 1264, 42, 8, 125, 40, 38, 36, - /* 420 */ 35, 34, 124, 611, 1340, 58, 1262, 1596, 57, 49, - /* 430 */ 1599, 162, 1877, 1878, 1265, 1882, 1263, 663, 178, 177, - /* 440 */ 1809, 352, 137, 136, 608, 607, 606, 1335, 575, 1761, - /* 450 */ 43, 1342, 1343, 1761, 316, 577, 1486, 1268, 1269, 1270, - /* 460 */ 1317, 1318, 1320, 1321, 1322, 1323, 1324, 1325, 573, 569, - /* 470 */ 1333, 1334, 1336, 1337, 1338, 1339, 1341, 1344, 63, 1823, - /* 480 */ 9, 74, 1935, 294, 1792, 580, 1794, 1795, 576, 574, - /* 490 */ 571, 568, 1841, 1289, 122, 165, 1265, 1761, 1263, 1932, - /* 500 */ 33, 32, 663, 1602, 40, 38, 36, 35, 34, 317, - /* 510 */ 168, 315, 314, 1485, 473, 351, 1342, 1343, 475, 1268, - /* 520 */ 1269, 1291, 1317, 1318, 1320, 1321, 1322, 1323, 1324, 1325, - /* 530 */ 573, 569, 1333, 1334, 1336, 1337, 1338, 1339, 1341, 1344, - /* 540 */ 474, 1010, 1011, 33, 32, 1460, 1364, 40, 38, 36, - /* 550 */ 35, 34, 168, 168, 1761, 526, 1935, 1592, 377, 146, - /* 560 */ 376, 1265, 63, 1263, 26, 1532, 382, 168, 1610, 165, - /* 570 */ 33, 32, 217, 1932, 40, 38, 36, 35, 34, 218, - /* 580 */ 1484, 1791, 1414, 1607, 1268, 1269, 1594, 1317, 1318, 1320, - /* 590 */ 1321, 1322, 1323, 1324, 1325, 573, 569, 1333, 1334, 1336, - /* 600 */ 1337, 1338, 1339, 1341, 1344, 39, 37, 77, 27, 1809, - /* 610 */ 498, 1884, 63, 339, 78, 1264, 168, 578, 1369, 1483, - /* 620 */ 505, 1761, 1761, 373, 577, 1302, 1340, 28, 1262, 482, - /* 630 */ 481, 1482, 1459, 33, 32, 1881, 123, 40, 38, 36, - /* 640 */ 35, 34, 375, 371, 438, 1590, 477, 480, 1823, 1335, - /* 650 */ 1287, 1935, 96, 1792, 580, 1794, 1795, 576, 253, 571, - /* 660 */ 1761, 1270, 1869, 513, 165, 1481, 1868, 1865, 1932, 1081, - /* 670 */ 33, 32, 1761, 1706, 40, 38, 36, 35, 34, 666, - /* 680 */ 33, 32, 9, 526, 40, 38, 36, 35, 34, 1478, - /* 690 */ 1477, 33, 32, 268, 383, 40, 38, 36, 35, 34, - /* 700 */ 168, 1704, 1083, 300, 663, 432, 1761, 157, 436, 1698, - /* 710 */ 214, 1607, 656, 652, 648, 644, 266, 1582, 1342, 1343, - /* 720 */ 176, 33, 32, 307, 572, 40, 38, 36, 35, 34, - /* 730 */ 1761, 1761, 39, 37, 526, 604, 526, 302, 1476, 1287, - /* 740 */ 339, 549, 1264, 526, 307, 389, 412, 404, 92, 424, - /* 750 */ 168, 231, 1302, 1340, 405, 1262, 440, 1585, 74, 436, - /* 760 */ 1362, 1407, 1607, 1265, 1607, 1263, 397, 1289, 425, 127, - /* 770 */ 399, 1607, 1475, 1703, 1779, 300, 1335, 1889, 1396, 1761, - /* 780 */ 1603, 1362, 44, 4, 523, 1775, 1268, 1269, 1270, 1317, - /* 790 */ 1318, 1320, 1321, 1322, 1323, 1324, 1325, 573, 569, 1333, - /* 800 */ 1334, 1336, 1337, 1338, 1339, 1341, 1344, 390, 125, 2, - /* 810 */ 1771, 1777, 334, 1761, 1363, 7, 220, 450, 611, 386, - /* 820 */ 90, 526, 571, 163, 1877, 1878, 1659, 1882, 1424, 145, - /* 830 */ 87, 663, 448, 312, 1236, 1363, 213, 137, 136, 608, - /* 840 */ 607, 606, 1657, 1480, 1884, 1342, 1343, 423, 1474, 1607, + /* 170 */ 1262, 107, 1260, 663, 106, 105, 104, 103, 102, 101, + /* 180 */ 100, 99, 98, 440, 1775, 1287, 436, 1339, 1340, 223, + /* 190 */ 224, 11, 10, 1265, 1266, 1770, 1314, 1315, 1317, 1318, + /* 200 */ 1319, 1320, 1321, 1322, 573, 569, 1330, 1331, 1333, 1334, + /* 210 */ 1335, 1336, 1338, 1341, 1464, 496, 1431, 33, 32, 1766, + /* 220 */ 1772, 40, 38, 36, 35, 34, 526, 167, 494, 150, + /* 230 */ 492, 571, 1262, 1564, 1260, 1261, 210, 55, 116, 115, + /* 240 */ 114, 113, 112, 111, 110, 109, 108, 305, 1259, 63, + /* 250 */ 516, 1701, 22, 300, 1604, 1265, 1266, 167, 1314, 1315, + /* 260 */ 1317, 1318, 1319, 1320, 1321, 1322, 573, 569, 1330, 1331, + /* 270 */ 1333, 1334, 1335, 1336, 1338, 1341, 39, 37, 1650, 1651, + /* 280 */ 1373, 1267, 167, 167, 339, 552, 1261, 613, 1286, 49, + /* 290 */ 1160, 1161, 76, 305, 1786, 1487, 516, 1337, 1421, 1259, + /* 300 */ 1118, 602, 601, 600, 1122, 599, 1124, 1125, 598, 1127, + /* 310 */ 595, 342, 1133, 592, 1135, 1136, 589, 586, 1285, 146, + /* 320 */ 1332, 1581, 1804, 173, 663, 14, 479, 478, 1606, 377, + /* 330 */ 553, 376, 1267, 1656, 1930, 1756, 1756, 577, 39, 37, + /* 340 */ 312, 535, 1419, 1420, 1422, 1423, 339, 1928, 1261, 1654, + /* 350 */ 1705, 1927, 84, 2, 42, 71, 1656, 63, 70, 1337, + /* 360 */ 1818, 1259, 1267, 343, 95, 1787, 580, 1789, 1790, 576, + /* 370 */ 605, 571, 1654, 1597, 1864, 663, 345, 1589, 330, 1860, + /* 380 */ 159, 1288, 1332, 1262, 146, 1260, 1022, 167, 1021, 1339, + /* 390 */ 1340, 33, 32, 1606, 1267, 40, 38, 36, 35, 34, + /* 400 */ 1891, 1930, 384, 39, 37, 1342, 1265, 1266, 1486, 1485, + /* 410 */ 611, 339, 1786, 1261, 165, 8, 1023, 438, 1927, 634, + /* 420 */ 632, 1080, 611, 1284, 1337, 622, 1259, 167, 549, 137, + /* 430 */ 136, 608, 607, 606, 1262, 1695, 1260, 663, 1484, 303, + /* 440 */ 1804, 137, 136, 608, 607, 606, 172, 1332, 575, 1756, + /* 450 */ 1756, 1339, 1340, 1756, 1082, 577, 127, 1265, 1266, 1267, + /* 460 */ 1314, 1315, 1317, 1318, 1319, 1320, 1321, 1322, 573, 569, + /* 470 */ 1330, 1331, 1333, 1334, 1335, 1336, 1338, 1341, 1818, 1756, + /* 480 */ 9, 1591, 293, 1787, 580, 1789, 1790, 576, 574, 571, + /* 490 */ 568, 1836, 167, 74, 125, 167, 1262, 222, 1260, 33, + /* 500 */ 32, 1529, 663, 40, 38, 36, 35, 34, 551, 160, + /* 510 */ 1872, 1873, 1587, 1877, 1483, 1600, 1339, 1340, 252, 1265, + /* 520 */ 1266, 1579, 1314, 1315, 1317, 1318, 1319, 1320, 1321, 1322, + /* 530 */ 573, 569, 1330, 1331, 1333, 1334, 1335, 1336, 1338, 1341, + /* 540 */ 1700, 526, 300, 33, 32, 1457, 91, 40, 38, 36, + /* 550 */ 35, 34, 169, 167, 316, 1756, 1241, 1242, 623, 124, + /* 560 */ 1576, 1262, 1879, 1260, 26, 482, 481, 1596, 1462, 1604, + /* 570 */ 33, 32, 123, 1582, 40, 38, 36, 35, 34, 213, + /* 580 */ 1786, 1411, 477, 480, 1265, 1266, 1876, 1314, 1315, 1317, + /* 590 */ 1318, 1319, 1320, 1321, 1322, 573, 569, 1330, 1331, 1333, + /* 600 */ 1334, 1335, 1336, 1338, 1341, 39, 37, 475, 1804, 317, + /* 610 */ 146, 315, 314, 339, 473, 1261, 578, 1361, 475, 1607, + /* 620 */ 549, 1756, 611, 577, 28, 1299, 1337, 354, 1259, 474, + /* 630 */ 33, 32, 1456, 450, 40, 38, 36, 35, 34, 538, + /* 640 */ 474, 137, 136, 608, 607, 606, 1818, 1695, 127, 1332, + /* 650 */ 96, 1787, 580, 1789, 1790, 576, 572, 571, 175, 74, + /* 660 */ 1864, 1267, 526, 609, 1863, 1860, 1647, 1930, 554, 512, + /* 670 */ 33, 32, 122, 382, 40, 38, 36, 35, 34, 27, + /* 680 */ 164, 1599, 9, 1482, 1927, 1022, 125, 1021, 7, 1366, + /* 690 */ 1604, 33, 32, 1481, 1565, 40, 38, 36, 35, 34, + /* 700 */ 469, 250, 1872, 548, 663, 547, 33, 32, 1930, 1930, + /* 710 */ 40, 38, 36, 35, 34, 1023, 544, 539, 1339, 1340, + /* 720 */ 526, 166, 164, 307, 1756, 1927, 1927, 135, 487, 1404, + /* 730 */ 526, 383, 39, 37, 1756, 1286, 1480, 302, 1879, 1284, + /* 740 */ 339, 389, 1261, 497, 307, 526, 412, 604, 1604, 424, + /* 750 */ 526, 549, 1299, 1337, 1477, 1259, 404, 209, 1604, 61, + /* 760 */ 1359, 405, 1875, 1262, 373, 1260, 397, 1479, 425, 1478, + /* 770 */ 399, 490, 255, 1604, 54, 484, 1332, 1756, 1604, 127, + /* 780 */ 208, 1359, 1475, 375, 371, 419, 1265, 1266, 1267, 1314, + /* 790 */ 1315, 1317, 1318, 1319, 1320, 1321, 1322, 573, 569, 1330, + /* 800 */ 1331, 1333, 1334, 1335, 1336, 1338, 1341, 390, 1756, 2, + /* 810 */ 1756, 1397, 58, 526, 1360, 57, 1879, 125, 505, 386, + /* 820 */ 1289, 33, 32, 1756, 448, 40, 38, 36, 35, 34, + /* 830 */ 1474, 663, 161, 1872, 1873, 1360, 1877, 177, 176, 1505, + /* 840 */ 1874, 1604, 1347, 1009, 1010, 1339, 1340, 423, 1286, 1580, /* 850 */ 418, 417, 416, 415, 414, 411, 410, 409, 408, 407, - /* 860 */ 403, 402, 401, 400, 394, 393, 392, 391, 1880, 388, - /* 870 */ 387, 535, 1422, 1423, 1425, 1426, 29, 337, 1357, 1358, - /* 880 */ 1359, 1360, 1361, 1365, 1366, 1367, 1368, 1350, 61, 1761, - /* 890 */ 1265, 609, 1263, 1289, 1650, 1935, 1400, 29, 337, 1357, - /* 900 */ 1358, 1359, 1360, 1361, 1365, 1366, 1367, 1368, 166, 1583, - /* 910 */ 1791, 1473, 1932, 1268, 1269, 1472, 1317, 1318, 1320, 1321, - /* 920 */ 1322, 1323, 1324, 1325, 573, 569, 1333, 1334, 1336, 1337, - /* 930 */ 1338, 1339, 1341, 1344, 623, 147, 1579, 1791, 1809, 526, - /* 940 */ 279, 611, 610, 256, 1319, 1650, 578, 1884, 1471, 1470, - /* 950 */ 449, 1761, 1761, 577, 277, 60, 1761, 475, 59, 1292, - /* 960 */ 137, 136, 608, 607, 606, 1809, 554, 1607, 1289, 613, - /* 970 */ 1568, 1879, 135, 578, 181, 429, 427, 1823, 1761, 474, - /* 980 */ 577, 94, 1792, 580, 1794, 1795, 576, 536, 571, 1761, - /* 990 */ 1761, 1869, 1780, 554, 468, 306, 1865, 273, 53, 509, - /* 1000 */ 1637, 1659, 1396, 1775, 1823, 526, 63, 1935, 94, 1792, - /* 1010 */ 580, 1794, 1795, 576, 526, 571, 1604, 1658, 1869, 54, - /* 1020 */ 167, 1748, 306, 1865, 1932, 1736, 1519, 202, 1771, 1777, - /* 1030 */ 200, 336, 335, 1607, 1935, 1462, 1463, 558, 526, 526, - /* 1040 */ 571, 1278, 1607, 1273, 93, 526, 526, 165, 483, 506, - /* 1050 */ 510, 1932, 1340, 561, 1271, 326, 228, 522, 526, 204, - /* 1060 */ 526, 1791, 203, 146, 499, 526, 1607, 1607, 361, 524, - /* 1070 */ 1319, 525, 1609, 1607, 1607, 1335, 262, 41, 222, 68, - /* 1080 */ 67, 381, 342, 526, 172, 1272, 1607, 1270, 1607, 1809, - /* 1090 */ 146, 131, 245, 1607, 346, 206, 233, 578, 205, 1609, - /* 1100 */ 301, 567, 1761, 369, 577, 367, 363, 359, 356, 353, - /* 1110 */ 345, 1607, 1782, 208, 134, 135, 207, 1810, 146, 1514, - /* 1120 */ 1399, 1512, 51, 1791, 1213, 226, 237, 1609, 1823, 556, - /* 1130 */ 566, 51, 95, 1792, 580, 1794, 1795, 576, 519, 571, - /* 1140 */ 41, 485, 1869, 488, 168, 1319, 330, 1865, 1948, 11, - /* 1150 */ 10, 1809, 616, 41, 617, 1784, 350, 1903, 584, 578, - /* 1160 */ 134, 230, 1112, 1503, 1761, 1647, 577, 135, 119, 1421, - /* 1170 */ 134, 1899, 550, 240, 1069, 1791, 1067, 255, 1370, 250, - /* 1180 */ 1276, 258, 260, 3, 5, 355, 313, 1326, 1050, 1279, - /* 1190 */ 1823, 1274, 360, 1229, 95, 1792, 580, 1794, 1795, 576, - /* 1200 */ 272, 571, 269, 1809, 1869, 1140, 1508, 1144, 330, 1865, - /* 1210 */ 1948, 578, 1282, 1284, 1151, 1149, 1761, 138, 577, 1926, - /* 1220 */ 175, 1051, 1275, 1287, 569, 1333, 1334, 1336, 1337, 1338, - /* 1230 */ 1339, 1791, 385, 1354, 406, 1700, 413, 421, 420, 1293, - /* 1240 */ 559, 1791, 1823, 422, 426, 431, 95, 1792, 580, 1794, - /* 1250 */ 1795, 576, 428, 571, 658, 439, 1869, 430, 562, 1809, - /* 1260 */ 330, 1865, 1948, 1295, 442, 443, 184, 578, 1294, 1809, - /* 1270 */ 186, 1888, 1761, 1296, 577, 444, 445, 578, 189, 447, - /* 1280 */ 191, 72, 1761, 73, 577, 451, 470, 554, 195, 472, - /* 1290 */ 1791, 304, 1597, 199, 118, 1593, 1741, 554, 1823, 501, - /* 1300 */ 201, 140, 286, 1792, 580, 1794, 1795, 576, 1823, 571, - /* 1310 */ 141, 1595, 286, 1792, 580, 1794, 1795, 576, 1809, 571, - /* 1320 */ 1591, 142, 143, 212, 270, 500, 578, 215, 1935, 507, - /* 1330 */ 504, 1761, 511, 577, 322, 219, 534, 514, 1935, 132, - /* 1340 */ 1740, 167, 1710, 520, 517, 1932, 133, 324, 81, 521, - /* 1350 */ 1791, 165, 1292, 530, 271, 1932, 83, 1823, 1608, 235, - /* 1360 */ 1791, 96, 1792, 580, 1794, 1795, 576, 1900, 571, 537, - /* 1370 */ 239, 1869, 532, 1910, 6, 565, 1865, 533, 1809, 546, - /* 1380 */ 329, 1909, 540, 531, 529, 244, 578, 1891, 1809, 528, - /* 1390 */ 1396, 1761, 1291, 577, 154, 126, 578, 249, 563, 560, - /* 1400 */ 246, 1761, 48, 577, 1885, 247, 331, 248, 85, 1791, - /* 1410 */ 582, 1651, 1580, 265, 274, 659, 660, 1823, 1931, 662, - /* 1420 */ 52, 149, 1792, 580, 1794, 1795, 576, 1823, 571, 1951, - /* 1430 */ 153, 96, 1792, 580, 1794, 1795, 576, 1809, 571, 557, - /* 1440 */ 1755, 1869, 323, 287, 297, 578, 1866, 1850, 296, 254, - /* 1450 */ 1761, 276, 577, 564, 1754, 278, 257, 259, 65, 1753, - /* 1460 */ 1791, 1752, 66, 1749, 357, 555, 1949, 358, 1256, 1257, - /* 1470 */ 171, 362, 1747, 364, 365, 366, 1823, 1746, 1745, 368, - /* 1480 */ 295, 1792, 580, 1794, 1795, 576, 370, 571, 1809, 1744, - /* 1490 */ 372, 1743, 374, 527, 1232, 1231, 578, 1721, 1720, 379, - /* 1500 */ 380, 1761, 1201, 577, 1719, 1718, 1693, 129, 1692, 1691, - /* 1510 */ 1690, 69, 1791, 1689, 1688, 1687, 1686, 1685, 395, 396, - /* 1520 */ 1684, 398, 1791, 130, 1669, 1668, 1667, 1823, 1683, 1682, - /* 1530 */ 1681, 295, 1792, 580, 1794, 1795, 576, 1680, 571, 1791, - /* 1540 */ 1809, 1679, 1678, 1677, 1676, 1675, 1674, 1673, 578, 1672, - /* 1550 */ 1809, 1671, 1670, 1761, 1666, 577, 1665, 1664, 578, 1663, - /* 1560 */ 1203, 1662, 1661, 1761, 1660, 577, 1534, 1809, 179, 1533, - /* 1570 */ 1531, 1499, 120, 182, 180, 575, 1498, 158, 435, 1823, - /* 1580 */ 1761, 1013, 577, 290, 1792, 580, 1794, 1795, 576, 1823, - /* 1590 */ 571, 190, 1012, 149, 1792, 580, 1794, 1795, 576, 1791, - /* 1600 */ 571, 437, 1734, 183, 121, 1728, 1823, 1717, 1716, 1702, - /* 1610 */ 294, 1792, 580, 1794, 1795, 576, 1791, 571, 188, 1842, - /* 1620 */ 1586, 545, 1043, 1530, 1528, 452, 454, 1809, 1526, 453, - /* 1630 */ 456, 457, 338, 458, 1524, 578, 460, 462, 1950, 461, - /* 1640 */ 1761, 1522, 577, 465, 1809, 464, 1511, 1510, 1495, 340, - /* 1650 */ 466, 1588, 578, 1155, 1154, 1587, 50, 1761, 631, 577, - /* 1660 */ 1080, 1077, 633, 1520, 198, 1076, 1823, 1075, 1515, 1513, - /* 1670 */ 295, 1792, 580, 1794, 1795, 576, 318, 571, 319, 320, - /* 1680 */ 486, 1494, 1493, 1823, 1791, 489, 197, 295, 1792, 580, - /* 1690 */ 1794, 1795, 576, 491, 571, 1492, 493, 495, 97, 1733, - /* 1700 */ 152, 1238, 1791, 1727, 216, 467, 463, 459, 455, 196, - /* 1710 */ 56, 502, 1809, 144, 1715, 1713, 1714, 1712, 1711, 221, - /* 1720 */ 578, 1248, 15, 1709, 227, 1761, 79, 577, 1701, 503, - /* 1730 */ 1809, 321, 508, 80, 232, 518, 41, 87, 578, 229, - /* 1740 */ 47, 75, 16, 1761, 194, 577, 243, 242, 82, 25, - /* 1750 */ 17, 1823, 1436, 23, 234, 280, 1792, 580, 1794, 1795, - /* 1760 */ 576, 1791, 571, 236, 1418, 515, 238, 1782, 151, 1823, - /* 1770 */ 1420, 252, 241, 281, 1792, 580, 1794, 1795, 576, 24, - /* 1780 */ 571, 1413, 1393, 46, 1781, 86, 18, 155, 1392, 1809, - /* 1790 */ 1448, 1453, 1442, 1447, 332, 1452, 1451, 578, 333, 10, - /* 1800 */ 45, 1280, 1761, 1330, 577, 1355, 193, 187, 13, 192, - /* 1810 */ 1791, 19, 1328, 446, 1327, 156, 1826, 169, 570, 31, - /* 1820 */ 12, 20, 1310, 21, 583, 1141, 341, 1138, 1823, 185, - /* 1830 */ 587, 1791, 282, 1792, 580, 1794, 1795, 576, 1809, 571, - /* 1840 */ 585, 588, 581, 1135, 579, 590, 578, 1129, 593, 596, - /* 1850 */ 1118, 1761, 1127, 577, 591, 594, 597, 1133, 1132, 1809, - /* 1860 */ 1131, 1130, 88, 89, 263, 603, 1150, 578, 1146, 62, - /* 1870 */ 1041, 1072, 1761, 612, 577, 1071, 1070, 1823, 1068, 1066, - /* 1880 */ 1065, 289, 1792, 580, 1794, 1795, 576, 1064, 571, 1791, - /* 1890 */ 1087, 621, 264, 1062, 1061, 1060, 1059, 1058, 1823, 1791, - /* 1900 */ 1057, 1056, 291, 1792, 580, 1794, 1795, 576, 1047, 571, - /* 1910 */ 1084, 1082, 1053, 1052, 1049, 1048, 1046, 1809, 1527, 641, - /* 1920 */ 1525, 642, 643, 645, 647, 578, 1523, 1809, 649, 646, - /* 1930 */ 1761, 651, 577, 1521, 650, 578, 653, 655, 654, 1509, - /* 1940 */ 1761, 657, 577, 1491, 1003, 267, 661, 1466, 1466, 1266, - /* 1950 */ 275, 1791, 664, 1466, 665, 1466, 1823, 1466, 1466, 1466, - /* 1960 */ 283, 1792, 580, 1794, 1795, 576, 1823, 571, 1791, 1466, - /* 1970 */ 292, 1792, 580, 1794, 1795, 576, 1466, 571, 1466, 1809, - /* 1980 */ 1466, 1466, 1466, 1466, 1466, 1466, 1466, 578, 1466, 1466, - /* 1990 */ 1466, 1466, 1761, 1466, 577, 1466, 1809, 1466, 1466, 1466, - /* 2000 */ 1466, 1466, 1466, 1466, 578, 1466, 1466, 1466, 1466, 1761, - /* 2010 */ 1466, 577, 1466, 1466, 1466, 1466, 1466, 1791, 1823, 1466, - /* 2020 */ 1466, 1466, 284, 1792, 580, 1794, 1795, 576, 1466, 571, - /* 2030 */ 1466, 1466, 1466, 1466, 1791, 1823, 1466, 1466, 1466, 293, - /* 2040 */ 1792, 580, 1794, 1795, 576, 1809, 571, 1466, 1466, 1466, - /* 2050 */ 1466, 1466, 1466, 578, 1466, 1466, 1466, 1466, 1761, 1466, - /* 2060 */ 577, 1466, 1809, 1466, 1466, 1466, 1466, 1466, 1466, 1466, - /* 2070 */ 578, 1466, 1466, 1466, 1466, 1761, 1466, 577, 1466, 1466, - /* 2080 */ 1466, 1466, 1466, 1791, 1823, 1466, 1466, 1466, 285, 1792, - /* 2090 */ 580, 1794, 1795, 576, 1466, 571, 1466, 1466, 1466, 1466, - /* 2100 */ 1466, 1823, 1466, 1466, 1466, 298, 1792, 580, 1794, 1795, - /* 2110 */ 576, 1809, 571, 1466, 1466, 1466, 1466, 1466, 1466, 578, - /* 2120 */ 1466, 1466, 1466, 1466, 1761, 1466, 577, 1466, 1466, 1466, - /* 2130 */ 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1791, 1466, 1466, - /* 2140 */ 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1791, 1466, 1466, - /* 2150 */ 1823, 1466, 1466, 1466, 299, 1792, 580, 1794, 1795, 576, - /* 2160 */ 1466, 571, 1466, 1466, 1466, 1809, 1466, 1466, 1466, 1466, - /* 2170 */ 1466, 1466, 1466, 578, 1466, 1809, 1466, 1466, 1761, 1466, - /* 2180 */ 577, 1466, 1466, 578, 1466, 1466, 1466, 1466, 1761, 1466, - /* 2190 */ 577, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1791, 1466, - /* 2200 */ 1466, 1466, 1466, 1466, 1823, 1466, 1466, 1466, 1803, 1792, - /* 2210 */ 580, 1794, 1795, 576, 1823, 571, 1791, 1466, 1802, 1792, - /* 2220 */ 580, 1794, 1795, 576, 1466, 571, 1809, 1466, 1466, 1466, - /* 2230 */ 1466, 1466, 1466, 1466, 578, 1466, 1466, 1466, 1466, 1761, - /* 2240 */ 1466, 577, 1466, 1466, 1809, 1466, 1466, 1466, 1466, 1466, - /* 2250 */ 1466, 1466, 578, 1466, 1466, 1466, 1466, 1761, 1466, 577, - /* 2260 */ 1466, 1466, 1466, 1466, 1466, 1823, 1466, 1466, 1466, 1801, - /* 2270 */ 1792, 580, 1794, 1795, 576, 1791, 571, 1466, 1466, 1466, - /* 2280 */ 1466, 1466, 1466, 1823, 1466, 1466, 1466, 310, 1792, 580, - /* 2290 */ 1794, 1795, 576, 1466, 571, 1466, 1791, 1466, 1466, 1466, - /* 2300 */ 1466, 1466, 1466, 1809, 1466, 1466, 1466, 1466, 1466, 1466, - /* 2310 */ 1466, 578, 1466, 1466, 1466, 1466, 1761, 1466, 577, 1466, - /* 2320 */ 1466, 1466, 1466, 1466, 1809, 1466, 1466, 1466, 1466, 1466, - /* 2330 */ 1466, 1466, 578, 1466, 1466, 1466, 1466, 1761, 1466, 577, - /* 2340 */ 1466, 1466, 1823, 1466, 1466, 1466, 309, 1792, 580, 1794, - /* 2350 */ 1795, 576, 1791, 571, 1466, 1466, 1466, 1466, 1466, 1466, - /* 2360 */ 1466, 1466, 1791, 1823, 1466, 1466, 1466, 311, 1792, 580, - /* 2370 */ 1794, 1795, 576, 1466, 571, 1466, 1466, 1466, 1466, 1466, - /* 2380 */ 1809, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 578, 1466, - /* 2390 */ 1809, 1466, 1466, 1761, 549, 577, 1466, 1466, 578, 1466, - /* 2400 */ 1466, 1466, 1466, 1761, 1466, 577, 1466, 1466, 1466, 1466, - /* 2410 */ 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1823, - /* 2420 */ 1466, 1466, 127, 308, 1792, 580, 1794, 1795, 576, 1823, - /* 2430 */ 571, 1466, 1466, 288, 1792, 580, 1794, 1795, 576, 1466, - /* 2440 */ 571, 549, 554, 1466, 1466, 1466, 1466, 1466, 1466, 1466, - /* 2450 */ 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, - /* 2460 */ 1466, 125, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 127, - /* 2470 */ 1466, 1466, 1466, 1466, 1466, 1466, 251, 1877, 548, 1466, - /* 2480 */ 547, 1466, 1466, 1935, 1466, 1466, 1466, 1466, 1466, 554, - /* 2490 */ 1466, 1466, 1466, 1466, 1466, 1466, 167, 1466, 1466, 1466, - /* 2500 */ 1932, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 125, 1466, - /* 2510 */ 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, - /* 2520 */ 1466, 1466, 1466, 251, 1877, 548, 1466, 547, 1466, 1466, - /* 2530 */ 1935, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, - /* 2540 */ 1466, 1466, 1466, 165, 1466, 1466, 1466, 1932, + /* 860 */ 403, 402, 401, 400, 394, 393, 392, 391, 549, 388, + /* 870 */ 387, 1756, 1656, 616, 1473, 1393, 29, 337, 1354, 1355, + /* 880 */ 1356, 1357, 1358, 1362, 1363, 1364, 1365, 658, 1655, 468, + /* 890 */ 1262, 610, 1260, 1286, 1647, 1068, 127, 29, 337, 1354, + /* 900 */ 1355, 1356, 1357, 1358, 1362, 1363, 1364, 1365, 272, 613, + /* 910 */ 536, 1634, 1316, 1265, 1266, 1756, 1314, 1315, 1317, 1318, + /* 920 */ 1319, 1320, 1321, 1322, 573, 569, 1330, 1331, 1333, 1334, + /* 930 */ 1335, 1336, 1338, 1341, 125, 147, 1472, 1786, 561, 352, + /* 940 */ 278, 351, 1884, 1393, 1743, 482, 481, 1516, 1774, 162, + /* 950 */ 1872, 1873, 123, 1877, 276, 60, 1805, 232, 59, 1770, + /* 960 */ 526, 526, 477, 480, 1471, 1804, 44, 4, 244, 483, + /* 970 */ 145, 449, 1601, 578, 180, 429, 427, 1756, 1756, 1930, + /* 980 */ 577, 1930, 1500, 1766, 1772, 334, 1470, 1786, 1604, 1604, + /* 990 */ 526, 361, 164, 554, 164, 571, 1927, 216, 1927, 498, + /* 1000 */ 556, 499, 1469, 1818, 1468, 1756, 63, 94, 1787, 580, + /* 1010 */ 1789, 1790, 576, 526, 571, 1804, 558, 1864, 1604, 336, + /* 1020 */ 335, 306, 1860, 578, 506, 1316, 526, 1756, 1756, 1275, + /* 1030 */ 577, 201, 77, 1930, 199, 1396, 1644, 510, 526, 1930, + /* 1040 */ 1337, 1604, 1268, 1756, 93, 1756, 166, 1511, 1467, 227, + /* 1050 */ 1927, 350, 164, 1818, 1604, 1786, 1927, 95, 1787, 580, + /* 1060 */ 1789, 1790, 576, 1332, 571, 526, 1604, 1864, 41, 485, + /* 1070 */ 1316, 330, 1860, 1943, 526, 1267, 522, 53, 509, 68, + /* 1080 */ 67, 381, 1898, 1804, 171, 524, 221, 203, 526, 1756, + /* 1090 */ 202, 578, 205, 1604, 207, 204, 1756, 206, 577, 525, + /* 1100 */ 301, 1509, 1604, 369, 1894, 367, 363, 359, 356, 353, + /* 1110 */ 1270, 554, 131, 526, 1786, 1212, 1604, 1269, 566, 567, + /* 1120 */ 526, 1818, 134, 488, 261, 94, 1787, 580, 1789, 1790, + /* 1130 */ 576, 346, 571, 225, 135, 1864, 51, 550, 666, 306, + /* 1140 */ 1860, 1604, 1804, 562, 167, 236, 51, 323, 1604, 41, + /* 1150 */ 578, 1930, 267, 90, 1786, 1756, 617, 577, 41, 519, + /* 1160 */ 1777, 11, 10, 87, 164, 249, 156, 3, 1927, 229, + /* 1170 */ 254, 656, 652, 648, 644, 265, 584, 1276, 1066, 1271, + /* 1180 */ 1818, 1111, 1804, 1418, 294, 1787, 580, 1789, 1790, 576, + /* 1190 */ 578, 571, 239, 1367, 1786, 1756, 1323, 577, 1459, 1460, + /* 1200 */ 1279, 1281, 257, 1779, 259, 271, 134, 92, 135, 5, + /* 1210 */ 230, 1049, 569, 1330, 1331, 1333, 1334, 1335, 1336, 559, + /* 1220 */ 1818, 360, 1804, 1139, 95, 1787, 580, 1789, 1790, 576, + /* 1230 */ 578, 571, 268, 355, 1864, 1756, 119, 577, 330, 1860, + /* 1240 */ 1943, 134, 549, 523, 1050, 313, 1228, 1273, 174, 1921, + /* 1250 */ 385, 1351, 1284, 1143, 1272, 1150, 406, 413, 1697, 421, + /* 1260 */ 1818, 420, 422, 1786, 95, 1787, 580, 1789, 1790, 576, + /* 1270 */ 127, 571, 426, 428, 1864, 219, 430, 1290, 330, 1860, + /* 1280 */ 1943, 431, 439, 1148, 1292, 442, 183, 443, 138, 1883, + /* 1290 */ 554, 1804, 1291, 1235, 185, 212, 444, 1293, 188, 578, + /* 1300 */ 445, 190, 447, 72, 1756, 73, 577, 451, 125, 194, + /* 1310 */ 470, 472, 1594, 198, 118, 1590, 304, 1786, 200, 554, + /* 1320 */ 140, 269, 141, 250, 1872, 548, 1592, 547, 1588, 1818, + /* 1330 */ 1930, 142, 143, 285, 1787, 580, 1789, 1790, 576, 211, + /* 1340 */ 571, 500, 1736, 164, 214, 1804, 507, 1927, 504, 511, + /* 1350 */ 218, 322, 534, 578, 514, 520, 501, 1735, 1756, 1930, + /* 1360 */ 577, 132, 1707, 517, 324, 1289, 81, 1786, 521, 133, + /* 1370 */ 270, 83, 166, 554, 537, 1605, 1927, 530, 1905, 234, + /* 1380 */ 1895, 238, 6, 1818, 1786, 532, 533, 285, 1787, 580, + /* 1390 */ 1789, 1790, 576, 329, 571, 1804, 546, 531, 540, 529, + /* 1400 */ 528, 248, 1288, 578, 1393, 126, 563, 560, 1756, 48, + /* 1410 */ 577, 1880, 1804, 1930, 1904, 85, 1648, 331, 1577, 659, + /* 1420 */ 578, 582, 264, 660, 243, 1756, 164, 577, 153, 1886, + /* 1430 */ 1927, 247, 245, 1818, 1786, 246, 253, 96, 1787, 580, + /* 1440 */ 1789, 1790, 576, 1845, 571, 273, 662, 1864, 299, 275, + /* 1450 */ 1818, 565, 1860, 256, 149, 1787, 580, 1789, 1790, 576, + /* 1460 */ 1786, 571, 1804, 52, 1946, 1926, 557, 286, 296, 258, + /* 1470 */ 578, 564, 295, 1750, 277, 1756, 1749, 577, 65, 1748, + /* 1480 */ 1747, 66, 1744, 357, 358, 1253, 1254, 170, 1804, 362, + /* 1490 */ 1742, 364, 365, 527, 366, 1741, 578, 368, 555, 1944, + /* 1500 */ 1818, 1756, 1740, 577, 96, 1787, 580, 1789, 1790, 576, + /* 1510 */ 1786, 571, 370, 1739, 1864, 372, 1738, 1230, 374, 1861, + /* 1520 */ 1231, 1718, 1786, 379, 380, 1716, 1818, 1717, 1715, 1690, + /* 1530 */ 294, 1787, 580, 1789, 1790, 576, 1786, 571, 1804, 1689, + /* 1540 */ 1200, 129, 1688, 1687, 69, 1686, 578, 395, 1681, 396, + /* 1550 */ 1804, 1756, 1685, 577, 1684, 1683, 1682, 398, 578, 1680, + /* 1560 */ 1679, 1678, 1677, 1756, 1804, 577, 1676, 1675, 1674, 1673, + /* 1570 */ 1672, 1671, 575, 1670, 1669, 1668, 1818, 1756, 1667, 577, + /* 1580 */ 289, 1787, 580, 1789, 1790, 576, 130, 571, 1818, 1786, + /* 1590 */ 1666, 1665, 149, 1787, 580, 1789, 1790, 576, 1664, 571, + /* 1600 */ 1663, 1662, 1818, 1202, 1660, 1659, 293, 1787, 580, 1789, + /* 1610 */ 1790, 576, 1661, 571, 1658, 1837, 1657, 1804, 545, 1531, + /* 1620 */ 178, 1530, 338, 120, 181, 578, 196, 1528, 179, 1496, + /* 1630 */ 1756, 157, 577, 435, 1012, 437, 1011, 1945, 1495, 182, + /* 1640 */ 152, 121, 1786, 452, 453, 467, 463, 459, 455, 195, + /* 1650 */ 1731, 1725, 1714, 189, 1786, 1818, 187, 1713, 1699, 294, + /* 1660 */ 1787, 580, 1789, 1790, 576, 1583, 571, 1527, 1786, 1042, + /* 1670 */ 1804, 1525, 454, 1523, 456, 340, 458, 457, 578, 1521, + /* 1680 */ 460, 75, 1804, 1756, 193, 577, 462, 461, 1519, 464, + /* 1690 */ 578, 465, 466, 1508, 1507, 1756, 1804, 577, 1492, 1585, + /* 1700 */ 1153, 1154, 197, 1584, 578, 1079, 1074, 50, 1818, 1756, + /* 1710 */ 1517, 577, 294, 1787, 580, 1789, 1790, 576, 631, 571, + /* 1720 */ 1818, 1076, 1786, 633, 279, 1787, 580, 1789, 1790, 576, + /* 1730 */ 1075, 571, 1512, 318, 1818, 319, 1786, 1510, 280, 1787, + /* 1740 */ 580, 1789, 1790, 576, 320, 571, 192, 186, 1786, 191, + /* 1750 */ 1804, 486, 489, 446, 1491, 491, 1490, 1489, 578, 493, + /* 1760 */ 495, 97, 1730, 1756, 1804, 577, 1237, 56, 1724, 184, + /* 1770 */ 502, 1712, 578, 1710, 508, 503, 1804, 1756, 215, 577, + /* 1780 */ 1711, 1709, 321, 1708, 578, 15, 144, 220, 1818, 1756, + /* 1790 */ 1245, 577, 281, 1787, 580, 1789, 1790, 576, 1706, 571, + /* 1800 */ 1698, 226, 1818, 518, 79, 1786, 288, 1787, 580, 1789, + /* 1810 */ 1790, 576, 228, 571, 1818, 1786, 515, 80, 290, 1787, + /* 1820 */ 580, 1789, 1790, 576, 82, 571, 87, 41, 231, 23, + /* 1830 */ 47, 1786, 1433, 1804, 233, 241, 235, 1415, 237, 242, + /* 1840 */ 1417, 578, 16, 1804, 25, 1777, 1756, 151, 577, 240, + /* 1850 */ 24, 578, 46, 1410, 86, 1786, 1756, 17, 577, 1804, + /* 1860 */ 1390, 251, 1389, 1776, 154, 1450, 45, 578, 18, 1439, + /* 1870 */ 1445, 1818, 1756, 13, 577, 282, 1787, 580, 1789, 1790, + /* 1880 */ 576, 1818, 571, 1804, 1444, 291, 1787, 580, 1789, 1790, + /* 1890 */ 576, 578, 571, 332, 1449, 1448, 1756, 1818, 577, 333, + /* 1900 */ 10, 283, 1787, 580, 1789, 1790, 576, 1277, 571, 1352, + /* 1910 */ 19, 1786, 1821, 1307, 1327, 570, 155, 1325, 31, 581, + /* 1920 */ 1324, 1818, 12, 20, 168, 292, 1787, 580, 1789, 1790, + /* 1930 */ 576, 1786, 571, 21, 583, 1140, 341, 585, 579, 1804, + /* 1940 */ 1137, 587, 588, 590, 1134, 591, 593, 578, 596, 1132, + /* 1950 */ 594, 1786, 1756, 1128, 577, 1126, 1131, 597, 1117, 1804, + /* 1960 */ 1130, 88, 1149, 603, 1129, 89, 62, 578, 262, 1145, + /* 1970 */ 612, 1786, 1756, 1071, 577, 1070, 1040, 1818, 1069, 1804, + /* 1980 */ 1067, 284, 1787, 580, 1789, 1790, 576, 578, 571, 1065, + /* 1990 */ 1064, 1786, 1756, 1063, 577, 263, 1086, 1818, 1061, 1804, + /* 2000 */ 1060, 297, 1787, 580, 1789, 1790, 576, 578, 571, 621, + /* 2010 */ 1059, 1058, 1756, 1057, 577, 1056, 1055, 1818, 1083, 1804, + /* 2020 */ 1081, 298, 1787, 580, 1789, 1790, 576, 578, 571, 1052, + /* 2030 */ 1051, 1786, 1756, 1048, 577, 1047, 1046, 1818, 1045, 1524, + /* 2040 */ 641, 1798, 1787, 580, 1789, 1790, 576, 1786, 571, 642, + /* 2050 */ 643, 1522, 645, 646, 647, 1520, 649, 1818, 651, 1804, + /* 2060 */ 650, 1797, 1787, 580, 1789, 1790, 576, 578, 571, 1518, + /* 2070 */ 653, 654, 1756, 655, 577, 1804, 1506, 657, 1002, 1488, + /* 2080 */ 266, 661, 664, 578, 1263, 274, 665, 1463, 1756, 1463, + /* 2090 */ 577, 1463, 1463, 1463, 1463, 1463, 1463, 1818, 1786, 1463, + /* 2100 */ 1463, 1796, 1787, 580, 1789, 1790, 576, 1463, 571, 1463, + /* 2110 */ 1463, 1463, 1463, 1818, 1786, 1463, 1463, 310, 1787, 580, + /* 2120 */ 1789, 1790, 576, 1463, 571, 1463, 1804, 1463, 1463, 1463, + /* 2130 */ 1463, 1463, 1463, 1463, 578, 1463, 1463, 1463, 1463, 1756, + /* 2140 */ 1463, 577, 1804, 1463, 1463, 1463, 1463, 1463, 1463, 1463, + /* 2150 */ 578, 1463, 1463, 1463, 1463, 1756, 1463, 577, 1463, 1463, + /* 2160 */ 1463, 1463, 1463, 1463, 1818, 1786, 1463, 1463, 309, 1787, + /* 2170 */ 580, 1789, 1790, 576, 1463, 571, 1463, 1463, 1463, 1463, + /* 2180 */ 1818, 1786, 1463, 1463, 311, 1787, 580, 1789, 1790, 576, + /* 2190 */ 1463, 571, 1463, 1804, 1463, 1463, 1463, 1463, 1463, 1463, + /* 2200 */ 1463, 578, 1463, 1463, 1463, 1463, 1756, 1463, 577, 1804, + /* 2210 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 578, 1463, 1463, + /* 2220 */ 1463, 1463, 1756, 1463, 577, 1463, 1463, 1463, 1463, 1463, + /* 2230 */ 1463, 1818, 1463, 1463, 1463, 308, 1787, 580, 1789, 1790, + /* 2240 */ 576, 1463, 571, 1463, 1463, 1463, 1463, 1818, 1463, 1463, + /* 2250 */ 1463, 287, 1787, 580, 1789, 1790, 576, 1463, 571, }; static const YYCODETYPE yy_lookahead[] = { - /* 0 */ 316, 390, 391, 316, 316, 312, 316, 314, 315, 1, - /* 10 */ 2, 327, 12, 13, 327, 350, 351, 327, 334, 364, - /* 20 */ 20, 0, 22, 4, 334, 20, 20, 22, 344, 308, - /* 30 */ 338, 344, 344, 33, 344, 35, 312, 20, 314, 315, - /* 40 */ 35, 349, 21, 338, 20, 24, 25, 26, 27, 28, - /* 50 */ 29, 30, 31, 32, 349, 50, 56, 336, 321, 4, - /* 60 */ 405, 61, 325, 44, 45, 344, 374, 375, 68, 60, - /* 70 */ 349, 383, 351, 418, 12, 13, 14, 422, 386, 374, - /* 80 */ 375, 376, 20, 0, 22, 397, 398, 399, 400, 89, - /* 90 */ 402, 386, 336, 322, 323, 33, 375, 35, 90, 343, - /* 100 */ 379, 380, 381, 382, 383, 384, 21, 386, 352, 347, - /* 110 */ 389, 111, 350, 351, 393, 394, 395, 351, 56, 34, - /* 120 */ 405, 36, 20, 61, 358, 125, 126, 361, 407, 20, - /* 130 */ 68, 125, 126, 418, 8, 9, 415, 422, 12, 13, - /* 140 */ 14, 15, 16, 307, 89, 309, 63, 64, 65, 66, + /* 0 */ 312, 404, 314, 315, 337, 312, 316, 314, 315, 351, + /* 10 */ 389, 390, 12, 13, 417, 328, 358, 327, 421, 361, + /* 20 */ 20, 0, 22, 336, 334, 12, 13, 14, 15, 16, + /* 30 */ 308, 20, 345, 33, 344, 35, 347, 20, 20, 350, + /* 40 */ 351, 364, 21, 338, 316, 24, 25, 26, 27, 28, + /* 50 */ 29, 30, 31, 32, 349, 327, 56, 313, 336, 4, + /* 60 */ 316, 61, 334, 14, 15, 16, 344, 307, 68, 309, + /* 70 */ 60, 349, 344, 351, 12, 13, 14, 338, 373, 374, + /* 80 */ 375, 404, 20, 0, 22, 336, 56, 20, 349, 89, + /* 90 */ 385, 4, 336, 344, 417, 33, 374, 35, 421, 343, + /* 100 */ 378, 379, 380, 381, 382, 383, 89, 385, 352, 335, + /* 110 */ 388, 111, 373, 374, 392, 393, 394, 351, 56, 89, + /* 120 */ 404, 91, 348, 61, 385, 125, 126, 361, 406, 380, + /* 130 */ 68, 44, 45, 417, 8, 9, 414, 421, 12, 13, + /* 140 */ 14, 15, 16, 20, 89, 89, 63, 64, 65, 66, /* 150 */ 67, 89, 69, 70, 71, 72, 73, 74, 75, 76, /* 160 */ 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, /* 170 */ 170, 21, 172, 111, 24, 25, 26, 27, 28, 29, - /* 180 */ 30, 31, 32, 336, 14, 15, 16, 125, 126, 120, - /* 190 */ 121, 344, 316, 193, 194, 60, 196, 197, 198, 199, + /* 180 */ 30, 31, 32, 313, 338, 20, 316, 125, 126, 120, + /* 190 */ 121, 1, 2, 193, 194, 349, 196, 197, 198, 199, /* 200 */ 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, - /* 210 */ 210, 211, 212, 213, 0, 20, 90, 8, 9, 64, - /* 220 */ 65, 12, 13, 14, 15, 16, 71, 227, 381, 353, - /* 230 */ 305, 120, 170, 318, 172, 22, 81, 82, 24, 25, - /* 240 */ 26, 27, 28, 29, 30, 31, 32, 178, 35, 20, - /* 250 */ 181, 22, 43, 56, 339, 193, 194, 308, 196, 197, + /* 210 */ 210, 211, 212, 213, 0, 21, 90, 8, 9, 373, + /* 220 */ 374, 12, 13, 14, 15, 16, 316, 227, 34, 321, + /* 230 */ 36, 385, 170, 325, 172, 22, 121, 327, 24, 25, + /* 240 */ 26, 27, 28, 29, 30, 31, 32, 178, 35, 89, + /* 250 */ 181, 360, 43, 362, 344, 193, 194, 227, 196, 197, /* 260 */ 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, - /* 270 */ 208, 209, 210, 211, 212, 213, 12, 13, 308, 50, - /* 280 */ 155, 68, 227, 316, 20, 88, 22, 308, 91, 364, - /* 290 */ 179, 180, 0, 308, 327, 121, 364, 33, 349, 35, + /* 270 */ 208, 209, 210, 211, 212, 213, 12, 13, 350, 351, + /* 280 */ 90, 68, 227, 227, 20, 20, 22, 60, 20, 89, + /* 290 */ 125, 126, 177, 178, 308, 308, 181, 33, 193, 35, /* 300 */ 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, - /* 310 */ 112, 344, 114, 115, 116, 117, 118, 119, 405, 349, - /* 320 */ 56, 336, 0, 68, 111, 61, 336, 405, 349, 344, - /* 330 */ 405, 418, 68, 343, 349, 422, 351, 405, 12, 13, - /* 340 */ 418, 4, 352, 418, 422, 20, 20, 422, 22, 316, - /* 350 */ 418, 177, 178, 89, 422, 181, 19, 232, 233, 33, - /* 360 */ 375, 35, 344, 335, 379, 380, 381, 382, 383, 384, - /* 370 */ 33, 386, 89, 355, 389, 111, 348, 344, 393, 394, - /* 380 */ 395, 20, 56, 170, 47, 172, 77, 100, 51, 125, - /* 390 */ 126, 8, 9, 56, 68, 12, 13, 14, 15, 16, - /* 400 */ 415, 322, 323, 12, 13, 14, 193, 194, 308, 318, - /* 410 */ 68, 20, 308, 22, 89, 89, 383, 12, 13, 14, - /* 420 */ 15, 16, 331, 101, 33, 88, 35, 337, 91, 89, - /* 430 */ 339, 398, 399, 400, 170, 402, 172, 111, 129, 130, - /* 440 */ 336, 364, 120, 121, 122, 123, 124, 56, 344, 349, - /* 450 */ 89, 125, 126, 349, 37, 351, 308, 193, 194, 68, + /* 310 */ 112, 328, 114, 115, 116, 117, 118, 119, 20, 336, + /* 320 */ 56, 0, 336, 56, 111, 61, 322, 323, 345, 169, + /* 330 */ 344, 171, 68, 336, 404, 349, 349, 351, 12, 13, + /* 340 */ 343, 236, 237, 238, 239, 240, 20, 417, 22, 352, + /* 350 */ 0, 421, 318, 89, 89, 88, 336, 89, 91, 33, + /* 360 */ 374, 35, 68, 343, 378, 379, 380, 381, 382, 383, + /* 370 */ 100, 385, 352, 339, 388, 111, 328, 337, 392, 393, + /* 380 */ 394, 20, 56, 170, 336, 172, 20, 227, 22, 125, + /* 390 */ 126, 8, 9, 345, 68, 12, 13, 14, 15, 16, + /* 400 */ 414, 404, 316, 12, 13, 14, 193, 194, 308, 308, + /* 410 */ 101, 20, 308, 22, 417, 89, 50, 14, 421, 322, + /* 420 */ 323, 35, 101, 20, 33, 68, 35, 227, 316, 120, + /* 430 */ 121, 122, 123, 124, 170, 344, 172, 111, 308, 353, + /* 440 */ 336, 120, 121, 122, 123, 124, 355, 56, 344, 349, + /* 450 */ 349, 125, 126, 349, 68, 351, 344, 193, 194, 68, /* 460 */ 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, - /* 470 */ 206, 207, 208, 209, 210, 211, 212, 213, 89, 375, - /* 480 */ 89, 320, 405, 379, 380, 381, 382, 383, 384, 385, - /* 490 */ 386, 387, 388, 20, 333, 418, 170, 349, 172, 422, - /* 500 */ 8, 9, 111, 342, 12, 13, 14, 15, 16, 92, - /* 510 */ 227, 94, 95, 308, 97, 364, 125, 126, 101, 193, - /* 520 */ 194, 20, 196, 197, 198, 199, 200, 201, 202, 203, + /* 470 */ 206, 207, 208, 209, 210, 211, 212, 213, 374, 349, + /* 480 */ 89, 337, 378, 379, 380, 381, 382, 383, 384, 385, + /* 490 */ 386, 387, 227, 320, 382, 227, 170, 120, 172, 8, + /* 500 */ 9, 0, 111, 12, 13, 14, 15, 16, 396, 397, + /* 510 */ 398, 399, 337, 401, 308, 342, 125, 126, 157, 193, + /* 520 */ 194, 0, 196, 197, 198, 199, 200, 201, 202, 203, /* 530 */ 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, - /* 540 */ 123, 44, 45, 8, 9, 162, 152, 12, 13, 14, - /* 550 */ 15, 16, 227, 227, 349, 316, 405, 337, 169, 336, - /* 560 */ 171, 170, 89, 172, 2, 0, 327, 227, 345, 418, - /* 570 */ 8, 9, 56, 422, 12, 13, 14, 15, 16, 56, - /* 580 */ 308, 308, 90, 344, 193, 194, 337, 196, 197, 198, + /* 540 */ 360, 316, 362, 8, 9, 162, 318, 12, 13, 14, + /* 550 */ 15, 16, 327, 227, 37, 349, 179, 180, 324, 331, + /* 560 */ 326, 170, 376, 172, 2, 64, 65, 339, 305, 344, + /* 570 */ 8, 9, 71, 0, 12, 13, 14, 15, 16, 337, + /* 580 */ 308, 90, 81, 82, 193, 194, 400, 196, 197, 198, /* 590 */ 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, - /* 600 */ 209, 210, 211, 212, 213, 12, 13, 91, 214, 336, - /* 610 */ 364, 377, 89, 20, 91, 22, 227, 344, 224, 308, - /* 620 */ 368, 349, 349, 165, 351, 90, 33, 2, 35, 64, - /* 630 */ 65, 308, 249, 8, 9, 401, 71, 12, 13, 14, - /* 640 */ 15, 16, 184, 185, 14, 337, 81, 82, 375, 56, - /* 650 */ 20, 405, 379, 380, 381, 382, 383, 384, 157, 386, - /* 660 */ 349, 68, 389, 351, 418, 308, 393, 394, 422, 35, - /* 670 */ 8, 9, 349, 361, 12, 13, 14, 15, 16, 19, - /* 680 */ 8, 9, 89, 316, 12, 13, 14, 15, 16, 308, - /* 690 */ 308, 8, 9, 33, 327, 12, 13, 14, 15, 16, - /* 700 */ 227, 360, 68, 362, 111, 313, 349, 47, 316, 344, - /* 710 */ 337, 344, 52, 53, 54, 55, 56, 0, 125, 126, - /* 720 */ 355, 8, 9, 61, 337, 12, 13, 14, 15, 16, - /* 730 */ 349, 349, 12, 13, 316, 337, 316, 18, 308, 20, - /* 740 */ 20, 316, 22, 316, 61, 327, 27, 327, 88, 30, - /* 750 */ 227, 91, 90, 33, 327, 35, 313, 0, 320, 316, - /* 760 */ 98, 14, 344, 170, 344, 172, 47, 20, 49, 344, - /* 770 */ 51, 344, 308, 360, 338, 362, 56, 225, 226, 349, - /* 780 */ 342, 98, 42, 43, 124, 349, 193, 194, 68, 196, + /* 600 */ 209, 210, 211, 212, 213, 12, 13, 101, 336, 92, + /* 610 */ 336, 94, 95, 20, 97, 22, 344, 152, 101, 345, + /* 620 */ 316, 349, 101, 351, 2, 90, 33, 364, 35, 123, + /* 630 */ 8, 9, 249, 60, 12, 13, 14, 15, 16, 155, + /* 640 */ 123, 120, 121, 122, 123, 124, 374, 344, 344, 56, + /* 650 */ 378, 379, 380, 381, 382, 383, 337, 385, 355, 320, + /* 660 */ 388, 68, 316, 346, 392, 393, 349, 404, 364, 364, + /* 670 */ 8, 9, 333, 327, 12, 13, 14, 15, 16, 214, + /* 680 */ 417, 342, 89, 308, 421, 20, 382, 22, 39, 224, + /* 690 */ 344, 8, 9, 308, 325, 12, 13, 14, 15, 16, + /* 700 */ 35, 397, 398, 399, 111, 401, 8, 9, 404, 404, + /* 710 */ 12, 13, 14, 15, 16, 50, 232, 233, 125, 126, + /* 720 */ 316, 417, 417, 61, 349, 421, 421, 43, 4, 14, + /* 730 */ 316, 327, 12, 13, 349, 20, 308, 18, 376, 20, + /* 740 */ 20, 327, 22, 19, 61, 316, 27, 337, 344, 30, + /* 750 */ 316, 316, 90, 33, 309, 35, 327, 33, 344, 3, + /* 760 */ 98, 327, 400, 170, 165, 172, 47, 308, 49, 308, + /* 770 */ 51, 47, 424, 344, 90, 51, 56, 349, 344, 344, + /* 780 */ 56, 98, 308, 184, 185, 77, 193, 194, 68, 196, /* 790 */ 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, - /* 800 */ 207, 208, 209, 210, 211, 212, 213, 88, 383, 89, - /* 810 */ 374, 375, 376, 349, 152, 39, 156, 60, 101, 100, - /* 820 */ 89, 316, 386, 398, 399, 400, 336, 402, 193, 157, - /* 830 */ 99, 111, 327, 343, 174, 152, 176, 120, 121, 122, - /* 840 */ 123, 124, 352, 309, 377, 125, 126, 128, 308, 344, + /* 800 */ 207, 208, 209, 210, 211, 212, 213, 88, 349, 89, + /* 810 */ 349, 4, 88, 316, 152, 91, 376, 382, 368, 100, + /* 820 */ 20, 8, 9, 349, 327, 12, 13, 14, 15, 16, + /* 830 */ 308, 111, 397, 398, 399, 152, 401, 129, 130, 0, + /* 840 */ 400, 344, 14, 44, 45, 125, 126, 128, 20, 0, /* 850 */ 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, - /* 860 */ 141, 142, 143, 144, 145, 146, 147, 148, 401, 150, - /* 870 */ 151, 236, 237, 238, 239, 240, 214, 215, 216, 217, - /* 880 */ 218, 219, 220, 221, 222, 223, 224, 14, 3, 349, - /* 890 */ 170, 346, 172, 20, 349, 405, 4, 214, 215, 216, - /* 900 */ 217, 218, 219, 220, 221, 222, 223, 224, 418, 0, - /* 910 */ 308, 308, 422, 193, 194, 308, 196, 197, 198, 199, + /* 860 */ 141, 142, 143, 144, 145, 146, 147, 148, 316, 150, + /* 870 */ 151, 349, 336, 13, 308, 226, 214, 215, 216, 217, + /* 880 */ 218, 219, 220, 221, 222, 223, 224, 48, 352, 317, + /* 890 */ 170, 346, 172, 20, 349, 35, 344, 214, 215, 216, + /* 900 */ 217, 218, 219, 220, 221, 222, 223, 224, 329, 60, + /* 910 */ 415, 332, 197, 193, 194, 349, 196, 197, 198, 199, /* 920 */ 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, - /* 930 */ 210, 211, 212, 213, 324, 18, 326, 308, 336, 316, - /* 940 */ 23, 101, 346, 425, 197, 349, 344, 377, 308, 308, - /* 950 */ 327, 349, 349, 351, 37, 38, 349, 101, 41, 20, - /* 960 */ 120, 121, 122, 123, 124, 336, 364, 344, 20, 60, - /* 970 */ 325, 401, 43, 344, 57, 58, 59, 375, 349, 123, - /* 980 */ 351, 379, 380, 381, 382, 383, 384, 416, 386, 349, - /* 990 */ 349, 389, 338, 364, 317, 393, 394, 329, 157, 158, - /* 1000 */ 332, 336, 226, 349, 375, 316, 89, 405, 379, 380, - /* 1010 */ 381, 382, 383, 384, 316, 386, 327, 352, 389, 90, - /* 1020 */ 418, 0, 393, 394, 422, 327, 0, 93, 374, 375, - /* 1030 */ 96, 12, 13, 344, 405, 125, 126, 43, 316, 316, - /* 1040 */ 386, 22, 344, 35, 127, 316, 316, 418, 22, 327, - /* 1050 */ 327, 422, 33, 43, 35, 328, 327, 327, 316, 93, - /* 1060 */ 316, 308, 96, 336, 371, 316, 344, 344, 47, 327, - /* 1070 */ 197, 327, 345, 344, 344, 56, 327, 43, 43, 162, - /* 1080 */ 163, 164, 328, 316, 167, 35, 344, 68, 344, 336, - /* 1090 */ 336, 43, 412, 344, 327, 93, 157, 344, 96, 345, - /* 1100 */ 183, 61, 349, 186, 351, 188, 189, 190, 191, 192, - /* 1110 */ 328, 344, 46, 93, 43, 43, 96, 336, 336, 0, - /* 1120 */ 228, 0, 43, 308, 90, 90, 43, 345, 375, 244, - /* 1130 */ 111, 43, 379, 380, 381, 382, 383, 384, 90, 386, - /* 1140 */ 43, 22, 389, 22, 227, 197, 393, 394, 395, 1, - /* 1150 */ 2, 336, 13, 43, 13, 89, 317, 404, 43, 344, - /* 1160 */ 43, 90, 90, 315, 349, 348, 351, 43, 43, 90, - /* 1170 */ 43, 378, 403, 90, 35, 308, 35, 419, 90, 396, - /* 1180 */ 172, 419, 419, 406, 229, 373, 372, 90, 35, 170, - /* 1190 */ 375, 172, 47, 168, 379, 380, 381, 382, 383, 384, - /* 1200 */ 90, 386, 366, 336, 389, 90, 0, 90, 393, 394, - /* 1210 */ 395, 344, 193, 194, 90, 90, 349, 90, 351, 404, - /* 1220 */ 42, 68, 172, 20, 205, 206, 207, 208, 209, 210, - /* 1230 */ 211, 308, 356, 193, 316, 316, 356, 152, 354, 20, - /* 1240 */ 246, 308, 375, 354, 316, 310, 379, 380, 381, 382, - /* 1250 */ 383, 384, 316, 386, 48, 310, 389, 316, 248, 336, - /* 1260 */ 393, 394, 395, 20, 370, 351, 320, 344, 20, 336, - /* 1270 */ 320, 404, 349, 20, 351, 363, 365, 344, 320, 363, - /* 1280 */ 320, 320, 349, 320, 351, 316, 310, 364, 320, 336, - /* 1290 */ 308, 310, 336, 336, 316, 336, 349, 364, 375, 369, - /* 1300 */ 336, 336, 379, 380, 381, 382, 383, 384, 375, 386, - /* 1310 */ 336, 336, 379, 380, 381, 382, 383, 384, 336, 386, - /* 1320 */ 336, 336, 336, 318, 370, 175, 344, 318, 405, 316, - /* 1330 */ 351, 349, 316, 351, 363, 318, 234, 349, 405, 359, - /* 1340 */ 349, 418, 349, 154, 349, 422, 359, 349, 318, 357, - /* 1350 */ 308, 418, 20, 349, 332, 422, 318, 375, 344, 359, - /* 1360 */ 308, 379, 380, 381, 382, 383, 384, 378, 386, 235, - /* 1370 */ 359, 389, 349, 411, 241, 393, 394, 349, 336, 161, - /* 1380 */ 349, 411, 349, 243, 242, 413, 344, 414, 336, 230, - /* 1390 */ 226, 349, 20, 351, 411, 344, 344, 373, 247, 245, - /* 1400 */ 410, 349, 89, 351, 377, 409, 250, 408, 89, 308, - /* 1410 */ 340, 349, 326, 318, 316, 36, 311, 375, 421, 310, - /* 1420 */ 367, 379, 380, 381, 382, 383, 384, 375, 386, 426, - /* 1430 */ 362, 379, 380, 381, 382, 383, 384, 336, 386, 421, - /* 1440 */ 0, 389, 341, 330, 330, 344, 394, 392, 330, 420, - /* 1450 */ 349, 319, 351, 421, 0, 306, 420, 420, 177, 0, - /* 1460 */ 308, 0, 42, 0, 35, 423, 424, 187, 35, 35, - /* 1470 */ 35, 187, 0, 35, 35, 187, 375, 0, 0, 187, - /* 1480 */ 379, 380, 381, 382, 383, 384, 35, 386, 336, 0, - /* 1490 */ 22, 0, 35, 341, 172, 170, 344, 0, 0, 166, - /* 1500 */ 165, 349, 46, 351, 0, 0, 0, 42, 0, 0, - /* 1510 */ 0, 149, 308, 0, 0, 0, 0, 0, 144, 35, - /* 1520 */ 0, 144, 308, 42, 0, 0, 0, 375, 0, 0, - /* 1530 */ 0, 379, 380, 381, 382, 383, 384, 0, 386, 308, - /* 1540 */ 336, 0, 0, 0, 0, 0, 0, 0, 344, 0, - /* 1550 */ 336, 0, 0, 349, 0, 351, 0, 0, 344, 0, - /* 1560 */ 22, 0, 0, 349, 0, 351, 0, 336, 56, 0, - /* 1570 */ 0, 0, 39, 42, 56, 344, 0, 43, 46, 375, - /* 1580 */ 349, 14, 351, 379, 380, 381, 382, 383, 384, 375, - /* 1590 */ 386, 161, 14, 379, 380, 381, 382, 383, 384, 308, - /* 1600 */ 386, 46, 0, 40, 39, 0, 375, 0, 0, 0, - /* 1610 */ 379, 380, 381, 382, 383, 384, 308, 386, 39, 388, - /* 1620 */ 0, 417, 62, 0, 0, 35, 39, 336, 0, 47, - /* 1630 */ 35, 47, 341, 39, 0, 344, 35, 39, 424, 47, - /* 1640 */ 349, 0, 351, 47, 336, 35, 0, 0, 0, 341, - /* 1650 */ 39, 0, 344, 35, 22, 0, 98, 349, 43, 351, - /* 1660 */ 35, 35, 43, 0, 96, 35, 375, 22, 0, 0, - /* 1670 */ 379, 380, 381, 382, 383, 384, 22, 386, 22, 22, - /* 1680 */ 49, 0, 0, 375, 308, 35, 33, 379, 380, 381, - /* 1690 */ 382, 383, 384, 35, 386, 0, 35, 22, 20, 0, - /* 1700 */ 47, 35, 308, 0, 154, 52, 53, 54, 55, 56, - /* 1710 */ 157, 22, 336, 173, 0, 0, 0, 0, 0, 90, - /* 1720 */ 344, 35, 89, 0, 89, 349, 89, 351, 0, 157, - /* 1730 */ 336, 157, 159, 39, 46, 155, 43, 99, 344, 153, - /* 1740 */ 43, 88, 231, 349, 91, 351, 46, 43, 89, 43, - /* 1750 */ 231, 375, 90, 89, 89, 379, 380, 381, 382, 383, - /* 1760 */ 384, 308, 386, 90, 90, 182, 89, 46, 89, 375, - /* 1770 */ 90, 46, 89, 379, 380, 381, 382, 383, 384, 89, - /* 1780 */ 386, 90, 90, 43, 46, 89, 43, 46, 90, 336, - /* 1790 */ 35, 90, 90, 35, 35, 35, 35, 344, 35, 2, - /* 1800 */ 225, 22, 349, 90, 351, 193, 153, 154, 231, 156, - /* 1810 */ 308, 43, 90, 160, 90, 46, 89, 46, 89, 89, - /* 1820 */ 89, 89, 22, 89, 35, 90, 35, 90, 375, 176, - /* 1830 */ 35, 308, 379, 380, 381, 382, 383, 384, 336, 386, - /* 1840 */ 89, 89, 100, 90, 195, 35, 344, 90, 35, 35, - /* 1850 */ 22, 349, 90, 351, 89, 89, 89, 113, 113, 336, - /* 1860 */ 113, 113, 89, 89, 43, 101, 35, 344, 22, 89, - /* 1870 */ 62, 35, 349, 61, 351, 35, 35, 375, 35, 35, - /* 1880 */ 35, 379, 380, 381, 382, 383, 384, 35, 386, 308, - /* 1890 */ 68, 87, 43, 35, 35, 22, 35, 22, 375, 308, - /* 1900 */ 35, 35, 379, 380, 381, 382, 383, 384, 22, 386, - /* 1910 */ 68, 35, 35, 35, 35, 35, 35, 336, 0, 35, - /* 1920 */ 0, 47, 39, 35, 39, 344, 0, 336, 35, 47, - /* 1930 */ 349, 39, 351, 0, 47, 344, 35, 39, 47, 0, - /* 1940 */ 349, 35, 351, 0, 35, 22, 21, 427, 427, 22, - /* 1950 */ 22, 308, 21, 427, 20, 427, 375, 427, 427, 427, - /* 1960 */ 379, 380, 381, 382, 383, 384, 375, 386, 308, 427, - /* 1970 */ 379, 380, 381, 382, 383, 384, 427, 386, 427, 336, - /* 1980 */ 427, 427, 427, 427, 427, 427, 427, 344, 427, 427, - /* 1990 */ 427, 427, 349, 427, 351, 427, 336, 427, 427, 427, - /* 2000 */ 427, 427, 427, 427, 344, 427, 427, 427, 427, 349, - /* 2010 */ 427, 351, 427, 427, 427, 427, 427, 308, 375, 427, - /* 2020 */ 427, 427, 379, 380, 381, 382, 383, 384, 427, 386, - /* 2030 */ 427, 427, 427, 427, 308, 375, 427, 427, 427, 379, - /* 2040 */ 380, 381, 382, 383, 384, 336, 386, 427, 427, 427, - /* 2050 */ 427, 427, 427, 344, 427, 427, 427, 427, 349, 427, - /* 2060 */ 351, 427, 336, 427, 427, 427, 427, 427, 427, 427, - /* 2070 */ 344, 427, 427, 427, 427, 349, 427, 351, 427, 427, - /* 2080 */ 427, 427, 427, 308, 375, 427, 427, 427, 379, 380, - /* 2090 */ 381, 382, 383, 384, 427, 386, 427, 427, 427, 427, - /* 2100 */ 427, 375, 427, 427, 427, 379, 380, 381, 382, 383, - /* 2110 */ 384, 336, 386, 427, 427, 427, 427, 427, 427, 344, - /* 2120 */ 427, 427, 427, 427, 349, 427, 351, 427, 427, 427, - /* 2130 */ 427, 427, 427, 427, 427, 427, 427, 308, 427, 427, - /* 2140 */ 427, 427, 427, 427, 427, 427, 427, 308, 427, 427, - /* 2150 */ 375, 427, 427, 427, 379, 380, 381, 382, 383, 384, - /* 2160 */ 427, 386, 427, 427, 427, 336, 427, 427, 427, 427, - /* 2170 */ 427, 427, 427, 344, 427, 336, 427, 427, 349, 427, - /* 2180 */ 351, 427, 427, 344, 427, 427, 427, 427, 349, 427, - /* 2190 */ 351, 427, 427, 427, 427, 427, 427, 427, 308, 427, - /* 2200 */ 427, 427, 427, 427, 375, 427, 427, 427, 379, 380, - /* 2210 */ 381, 382, 383, 384, 375, 386, 308, 427, 379, 380, - /* 2220 */ 381, 382, 383, 384, 427, 386, 336, 427, 427, 427, - /* 2230 */ 427, 427, 427, 427, 344, 427, 427, 427, 427, 349, - /* 2240 */ 427, 351, 427, 427, 336, 427, 427, 427, 427, 427, - /* 2250 */ 427, 427, 344, 427, 427, 427, 427, 349, 427, 351, - /* 2260 */ 427, 427, 427, 427, 427, 375, 427, 427, 427, 379, - /* 2270 */ 380, 381, 382, 383, 384, 308, 386, 427, 427, 427, - /* 2280 */ 427, 427, 427, 375, 427, 427, 427, 379, 380, 381, - /* 2290 */ 382, 383, 384, 427, 386, 427, 308, 427, 427, 427, - /* 2300 */ 427, 427, 427, 336, 427, 427, 427, 427, 427, 427, - /* 2310 */ 427, 344, 427, 427, 427, 427, 349, 427, 351, 427, - /* 2320 */ 427, 427, 427, 427, 336, 427, 427, 427, 427, 427, - /* 2330 */ 427, 427, 344, 427, 427, 427, 427, 349, 427, 351, - /* 2340 */ 427, 427, 375, 427, 427, 427, 379, 380, 381, 382, - /* 2350 */ 383, 384, 308, 386, 427, 427, 427, 427, 427, 427, - /* 2360 */ 427, 427, 308, 375, 427, 427, 427, 379, 380, 381, - /* 2370 */ 382, 383, 384, 427, 386, 427, 427, 427, 427, 427, - /* 2380 */ 336, 427, 427, 427, 427, 427, 427, 427, 344, 427, - /* 2390 */ 336, 427, 427, 349, 316, 351, 427, 427, 344, 427, - /* 2400 */ 427, 427, 427, 349, 427, 351, 427, 427, 427, 427, - /* 2410 */ 427, 427, 427, 427, 427, 427, 427, 427, 427, 375, - /* 2420 */ 427, 427, 344, 379, 380, 381, 382, 383, 384, 375, - /* 2430 */ 386, 427, 427, 379, 380, 381, 382, 383, 384, 427, - /* 2440 */ 386, 316, 364, 427, 427, 427, 427, 427, 427, 427, - /* 2450 */ 427, 427, 427, 427, 427, 427, 427, 427, 427, 427, - /* 2460 */ 427, 383, 427, 427, 427, 427, 427, 427, 427, 344, - /* 2470 */ 427, 427, 427, 427, 427, 427, 398, 399, 400, 427, - /* 2480 */ 402, 427, 427, 405, 427, 427, 427, 427, 427, 364, - /* 2490 */ 427, 427, 427, 427, 427, 427, 418, 427, 427, 427, - /* 2500 */ 422, 427, 427, 427, 427, 427, 427, 427, 383, 427, - /* 2510 */ 427, 427, 427, 427, 427, 427, 427, 427, 427, 427, - /* 2520 */ 427, 427, 427, 398, 399, 400, 427, 402, 427, 427, - /* 2530 */ 405, 427, 427, 427, 427, 427, 427, 427, 427, 427, - /* 2540 */ 427, 427, 427, 418, 427, 427, 427, 422, + /* 930 */ 210, 211, 212, 213, 382, 18, 308, 308, 43, 364, + /* 940 */ 23, 364, 225, 226, 0, 64, 65, 0, 338, 397, + /* 950 */ 398, 399, 71, 401, 37, 38, 336, 157, 41, 349, + /* 960 */ 316, 316, 81, 82, 308, 336, 42, 43, 411, 22, + /* 970 */ 157, 327, 327, 344, 57, 58, 59, 349, 349, 404, + /* 980 */ 351, 404, 315, 373, 374, 375, 308, 308, 344, 344, + /* 990 */ 316, 47, 417, 364, 417, 385, 421, 56, 421, 364, + /* 1000 */ 244, 327, 308, 374, 308, 349, 89, 378, 379, 380, + /* 1010 */ 381, 382, 383, 316, 385, 336, 43, 388, 344, 12, + /* 1020 */ 13, 392, 393, 344, 327, 197, 316, 349, 349, 22, + /* 1030 */ 351, 93, 91, 404, 96, 228, 348, 327, 316, 404, + /* 1040 */ 33, 344, 35, 349, 127, 349, 417, 0, 308, 327, + /* 1050 */ 421, 317, 417, 374, 344, 308, 421, 378, 379, 380, + /* 1060 */ 381, 382, 383, 56, 385, 316, 344, 388, 43, 22, + /* 1070 */ 197, 392, 393, 394, 316, 68, 327, 157, 158, 162, + /* 1080 */ 163, 164, 403, 336, 167, 327, 43, 93, 316, 349, + /* 1090 */ 96, 344, 93, 344, 93, 96, 349, 96, 351, 327, + /* 1100 */ 183, 0, 344, 186, 377, 188, 189, 190, 191, 192, + /* 1110 */ 35, 364, 43, 316, 308, 90, 344, 35, 111, 61, + /* 1120 */ 316, 374, 43, 22, 327, 378, 379, 380, 381, 382, + /* 1130 */ 383, 327, 385, 90, 43, 388, 43, 402, 19, 392, + /* 1140 */ 393, 344, 336, 248, 227, 43, 43, 341, 344, 43, + /* 1150 */ 344, 404, 33, 89, 308, 349, 13, 351, 43, 90, + /* 1160 */ 46, 1, 2, 99, 417, 395, 47, 405, 421, 90, + /* 1170 */ 418, 52, 53, 54, 55, 56, 43, 170, 35, 172, + /* 1180 */ 374, 90, 336, 90, 378, 379, 380, 381, 382, 383, + /* 1190 */ 344, 385, 90, 90, 308, 349, 90, 351, 125, 126, + /* 1200 */ 193, 194, 418, 89, 418, 90, 43, 88, 43, 229, + /* 1210 */ 91, 35, 205, 206, 207, 208, 209, 210, 211, 246, + /* 1220 */ 374, 47, 336, 90, 378, 379, 380, 381, 382, 383, + /* 1230 */ 344, 385, 366, 372, 388, 349, 43, 351, 392, 393, + /* 1240 */ 394, 43, 316, 124, 68, 371, 168, 172, 42, 403, + /* 1250 */ 356, 193, 20, 90, 172, 90, 316, 356, 316, 152, + /* 1260 */ 374, 354, 354, 308, 378, 379, 380, 381, 382, 383, + /* 1270 */ 344, 385, 316, 316, 388, 156, 316, 20, 392, 393, + /* 1280 */ 394, 310, 310, 90, 20, 370, 320, 351, 90, 403, + /* 1290 */ 364, 336, 20, 174, 320, 176, 363, 20, 320, 344, + /* 1300 */ 365, 320, 363, 320, 349, 320, 351, 316, 382, 320, + /* 1310 */ 310, 336, 336, 336, 316, 336, 310, 308, 336, 364, + /* 1320 */ 336, 370, 336, 397, 398, 399, 336, 401, 336, 374, + /* 1330 */ 404, 336, 336, 378, 379, 380, 381, 382, 383, 318, + /* 1340 */ 385, 175, 349, 417, 318, 336, 316, 421, 351, 316, + /* 1350 */ 318, 363, 234, 344, 349, 154, 369, 349, 349, 404, + /* 1360 */ 351, 359, 349, 349, 349, 20, 318, 308, 357, 359, + /* 1370 */ 332, 318, 417, 364, 235, 344, 421, 349, 410, 359, + /* 1380 */ 377, 359, 241, 374, 308, 349, 349, 378, 379, 380, + /* 1390 */ 381, 382, 383, 349, 385, 336, 161, 243, 349, 242, + /* 1400 */ 230, 372, 20, 344, 226, 344, 247, 245, 349, 89, + /* 1410 */ 351, 376, 336, 404, 410, 89, 349, 250, 326, 36, + /* 1420 */ 344, 340, 318, 311, 412, 349, 417, 351, 410, 413, + /* 1430 */ 421, 407, 409, 374, 308, 408, 419, 378, 379, 380, + /* 1440 */ 381, 382, 383, 391, 385, 316, 310, 388, 362, 319, + /* 1450 */ 374, 392, 393, 419, 378, 379, 380, 381, 382, 383, + /* 1460 */ 308, 385, 336, 367, 425, 420, 420, 330, 330, 419, + /* 1470 */ 344, 420, 330, 0, 306, 349, 0, 351, 177, 0, + /* 1480 */ 0, 42, 0, 35, 187, 35, 35, 35, 336, 187, + /* 1490 */ 0, 35, 35, 341, 187, 0, 344, 187, 422, 423, + /* 1500 */ 374, 349, 0, 351, 378, 379, 380, 381, 382, 383, + /* 1510 */ 308, 385, 35, 0, 388, 22, 0, 170, 35, 393, + /* 1520 */ 172, 0, 308, 166, 165, 0, 374, 0, 0, 0, + /* 1530 */ 378, 379, 380, 381, 382, 383, 308, 385, 336, 0, + /* 1540 */ 46, 42, 0, 0, 149, 0, 344, 144, 0, 35, + /* 1550 */ 336, 349, 0, 351, 0, 0, 0, 144, 344, 0, + /* 1560 */ 0, 0, 0, 349, 336, 351, 0, 0, 0, 0, + /* 1570 */ 0, 0, 344, 0, 0, 0, 374, 349, 0, 351, + /* 1580 */ 378, 379, 380, 381, 382, 383, 42, 385, 374, 308, + /* 1590 */ 0, 0, 378, 379, 380, 381, 382, 383, 0, 385, + /* 1600 */ 0, 0, 374, 22, 0, 0, 378, 379, 380, 381, + /* 1610 */ 382, 383, 0, 385, 0, 387, 0, 336, 416, 0, + /* 1620 */ 56, 0, 341, 39, 42, 344, 33, 0, 56, 0, + /* 1630 */ 349, 43, 351, 46, 14, 46, 14, 423, 0, 40, + /* 1640 */ 47, 39, 308, 35, 47, 52, 53, 54, 55, 56, + /* 1650 */ 0, 0, 0, 161, 308, 374, 39, 0, 0, 378, + /* 1660 */ 379, 380, 381, 382, 383, 0, 385, 0, 308, 62, + /* 1670 */ 336, 0, 39, 0, 35, 341, 39, 47, 344, 0, + /* 1680 */ 35, 88, 336, 349, 91, 351, 39, 47, 0, 35, + /* 1690 */ 344, 47, 39, 0, 0, 349, 336, 351, 0, 0, + /* 1700 */ 22, 35, 96, 0, 344, 35, 22, 98, 374, 349, + /* 1710 */ 0, 351, 378, 379, 380, 381, 382, 383, 43, 385, + /* 1720 */ 374, 35, 308, 43, 378, 379, 380, 381, 382, 383, + /* 1730 */ 35, 385, 0, 22, 374, 22, 308, 0, 378, 379, + /* 1740 */ 380, 381, 382, 383, 22, 385, 153, 154, 308, 156, + /* 1750 */ 336, 49, 35, 160, 0, 35, 0, 0, 344, 35, + /* 1760 */ 22, 20, 0, 349, 336, 351, 35, 157, 0, 176, + /* 1770 */ 22, 0, 344, 0, 159, 157, 336, 349, 154, 351, + /* 1780 */ 0, 0, 157, 0, 344, 89, 173, 90, 374, 349, + /* 1790 */ 35, 351, 378, 379, 380, 381, 382, 383, 0, 385, + /* 1800 */ 0, 89, 374, 155, 89, 308, 378, 379, 380, 381, + /* 1810 */ 382, 383, 153, 385, 374, 308, 182, 39, 378, 379, + /* 1820 */ 380, 381, 382, 383, 89, 385, 99, 43, 46, 89, + /* 1830 */ 43, 308, 90, 336, 89, 43, 90, 90, 89, 46, + /* 1840 */ 90, 344, 231, 336, 43, 46, 349, 89, 351, 89, + /* 1850 */ 89, 344, 43, 90, 89, 308, 349, 231, 351, 336, + /* 1860 */ 90, 46, 90, 46, 46, 90, 225, 344, 43, 90, + /* 1870 */ 35, 374, 349, 231, 351, 378, 379, 380, 381, 382, + /* 1880 */ 383, 374, 385, 336, 35, 378, 379, 380, 381, 382, + /* 1890 */ 383, 344, 385, 35, 35, 35, 349, 374, 351, 35, + /* 1900 */ 2, 378, 379, 380, 381, 382, 383, 22, 385, 193, + /* 1910 */ 43, 308, 89, 22, 90, 89, 46, 90, 89, 100, + /* 1920 */ 90, 374, 89, 89, 46, 378, 379, 380, 381, 382, + /* 1930 */ 383, 308, 385, 89, 35, 90, 35, 89, 195, 336, + /* 1940 */ 90, 35, 89, 35, 90, 89, 35, 344, 35, 113, + /* 1950 */ 89, 308, 349, 90, 351, 90, 113, 89, 22, 336, + /* 1960 */ 113, 89, 35, 101, 113, 89, 89, 344, 43, 22, + /* 1970 */ 61, 308, 349, 35, 351, 35, 62, 374, 35, 336, + /* 1980 */ 35, 378, 379, 380, 381, 382, 383, 344, 385, 35, + /* 1990 */ 35, 308, 349, 35, 351, 43, 68, 374, 35, 336, + /* 2000 */ 35, 378, 379, 380, 381, 382, 383, 344, 385, 87, + /* 2010 */ 22, 35, 349, 22, 351, 35, 35, 374, 68, 336, + /* 2020 */ 35, 378, 379, 380, 381, 382, 383, 344, 385, 35, + /* 2030 */ 35, 308, 349, 35, 351, 35, 22, 374, 35, 0, + /* 2040 */ 35, 378, 379, 380, 381, 382, 383, 308, 385, 47, + /* 2050 */ 39, 0, 35, 47, 39, 0, 35, 374, 39, 336, + /* 2060 */ 47, 378, 379, 380, 381, 382, 383, 344, 385, 0, + /* 2070 */ 35, 47, 349, 39, 351, 336, 0, 35, 35, 0, + /* 2080 */ 22, 21, 21, 344, 22, 22, 20, 426, 349, 426, + /* 2090 */ 351, 426, 426, 426, 426, 426, 426, 374, 308, 426, + /* 2100 */ 426, 378, 379, 380, 381, 382, 383, 426, 385, 426, + /* 2110 */ 426, 426, 426, 374, 308, 426, 426, 378, 379, 380, + /* 2120 */ 381, 382, 383, 426, 385, 426, 336, 426, 426, 426, + /* 2130 */ 426, 426, 426, 426, 344, 426, 426, 426, 426, 349, + /* 2140 */ 426, 351, 336, 426, 426, 426, 426, 426, 426, 426, + /* 2150 */ 344, 426, 426, 426, 426, 349, 426, 351, 426, 426, + /* 2160 */ 426, 426, 426, 426, 374, 308, 426, 426, 378, 379, + /* 2170 */ 380, 381, 382, 383, 426, 385, 426, 426, 426, 426, + /* 2180 */ 374, 308, 426, 426, 378, 379, 380, 381, 382, 383, + /* 2190 */ 426, 385, 426, 336, 426, 426, 426, 426, 426, 426, + /* 2200 */ 426, 344, 426, 426, 426, 426, 349, 426, 351, 336, + /* 2210 */ 426, 426, 426, 426, 426, 426, 426, 344, 426, 426, + /* 2220 */ 426, 426, 349, 426, 351, 426, 426, 426, 426, 426, + /* 2230 */ 426, 374, 426, 426, 426, 378, 379, 380, 381, 382, + /* 2240 */ 383, 426, 385, 426, 426, 426, 426, 374, 426, 426, + /* 2250 */ 426, 378, 379, 380, 381, 382, 383, 426, 385, 426, + /* 2260 */ 426, 426, 426, 426, 426, 426, 426, 426, 426, 426, + /* 2270 */ 426, 426, 426, 426, 426, 426, 426, 426, 426, 426, + /* 2280 */ 426, 426, 426, 426, 426, 426, 426, 426, 426, 426, + /* 2290 */ 426, 426, 426, 426, 426, 426, 426, 426, 426, 426, + /* 2300 */ 426, 426, 426, 426, 426, 426, 426, 426, 426, 426, + /* 2310 */ 426, 426, 426, 426, 426, 426, 426, 426, 426, 426, + /* 2320 */ 426, 426, 426, 426, 426, 426, 426, 426, 426, 426, + /* 2330 */ 426, 426, 426, 426, 426, 426, 426, 426, 426, 426, + /* 2340 */ 426, 426, 426, 426, 426, 426, 426, 426, 426, 426, + /* 2350 */ 426, 426, 426, 426, 426, 426, 426, 426, 426, 426, + /* 2360 */ 426, 426, 426, 426, 426, 426, 426, 426, 426, 426, + /* 2370 */ 426, 426, 426, 426, 426, 426, 426, 426, 426, 426, + /* 2380 */ 426, 426, 426, 426, 426, 426, 426, 426, 426, 426, + /* 2390 */ 426, 426, }; #define YY_SHIFT_COUNT (666) #define YY_SHIFT_MIN (0) -#define YY_SHIFT_MAX (1943) +#define YY_SHIFT_MAX (2079) static const unsigned short int yy_shift_ofst[] = { /* 0 */ 917, 0, 0, 62, 62, 264, 264, 264, 326, 326, /* 10 */ 264, 264, 391, 593, 720, 593, 593, 593, 593, 593, /* 20 */ 593, 593, 593, 593, 593, 593, 593, 593, 593, 593, /* 30 */ 593, 593, 593, 593, 593, 593, 593, 593, 593, 593, - /* 40 */ 593, 593, 325, 325, 361, 361, 361, 1019, 1019, 473, - /* 50 */ 1019, 1019, 389, 523, 283, 340, 283, 17, 17, 19, - /* 60 */ 19, 55, 6, 283, 283, 17, 17, 17, 17, 17, - /* 70 */ 17, 17, 17, 17, 17, 9, 17, 17, 17, 24, - /* 80 */ 17, 17, 102, 17, 17, 102, 109, 17, 102, 102, - /* 90 */ 102, 17, 135, 719, 662, 683, 683, 150, 213, 213, + /* 40 */ 593, 593, 265, 265, 17, 17, 17, 1007, 1007, 268, + /* 50 */ 1007, 1007, 160, 30, 56, 200, 56, 11, 11, 87, + /* 60 */ 87, 55, 165, 56, 56, 11, 11, 11, 11, 11, + /* 70 */ 11, 11, 11, 11, 11, 10, 11, 11, 11, 18, + /* 80 */ 11, 11, 67, 11, 11, 67, 123, 11, 67, 67, + /* 90 */ 67, 11, 227, 719, 662, 683, 683, 150, 213, 213, /* 100 */ 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, - /* 110 */ 213, 213, 213, 213, 213, 213, 213, 417, 155, 6, - /* 120 */ 630, 630, 757, 634, 909, 501, 501, 501, 634, 195, - /* 130 */ 195, 24, 292, 292, 102, 102, 255, 255, 287, 342, - /* 140 */ 198, 198, 198, 198, 198, 198, 198, 660, 21, 383, - /* 150 */ 565, 635, 5, 174, 125, 747, 873, 229, 497, 856, - /* 160 */ 939, 552, 776, 552, 740, 885, 885, 885, 892, 948, - /* 170 */ 955, 1145, 1025, 1178, 1203, 1203, 1178, 1085, 1085, 1203, - /* 180 */ 1203, 1203, 1219, 1219, 1243, 9, 24, 9, 1248, 1253, - /* 190 */ 9, 1248, 9, 9, 9, 1203, 9, 1219, 102, 102, - /* 200 */ 102, 102, 102, 102, 102, 102, 102, 102, 102, 1203, - /* 210 */ 1219, 255, 1243, 135, 1150, 24, 135, 1203, 1203, 1248, - /* 220 */ 135, 1102, 255, 255, 255, 255, 1102, 255, 1189, 135, - /* 230 */ 287, 135, 195, 1332, 255, 1134, 1102, 255, 255, 1134, - /* 240 */ 1102, 255, 255, 102, 1133, 1218, 1134, 1140, 1142, 1159, - /* 250 */ 955, 1164, 195, 1372, 1151, 1154, 1156, 1151, 1154, 1151, - /* 260 */ 1154, 1313, 1319, 255, 342, 1203, 135, 1379, 1219, 2548, - /* 270 */ 2548, 2548, 2548, 2548, 2548, 2548, 83, 1653, 214, 337, - /* 280 */ 126, 209, 492, 562, 625, 672, 535, 322, 713, 713, - /* 290 */ 713, 713, 713, 713, 713, 713, 717, 840, 405, 405, - /* 300 */ 69, 458, 197, 309, 85, 111, 8, 394, 170, 170, - /* 310 */ 170, 170, 929, 1021, 934, 966, 1002, 1020, 1026, 1119, - /* 320 */ 1121, 516, 841, 1034, 1035, 1048, 1071, 1072, 1079, 1083, - /* 330 */ 1148, 910, 994, 1010, 1088, 1008, 1050, 1040, 1097, 1066, - /* 340 */ 1110, 1115, 1117, 1124, 1125, 1127, 731, 1139, 1141, 1153, - /* 350 */ 1206, 1440, 1454, 1281, 1459, 1461, 1420, 1463, 1429, 1280, - /* 360 */ 1433, 1434, 1435, 1284, 1472, 1438, 1439, 1288, 1477, 1292, - /* 370 */ 1478, 1451, 1489, 1468, 1491, 1457, 1322, 1325, 1497, 1498, - /* 380 */ 1333, 1335, 1504, 1505, 1456, 1506, 1465, 1508, 1509, 1510, - /* 390 */ 1362, 1513, 1514, 1515, 1516, 1517, 1374, 1484, 1520, 1377, - /* 400 */ 1528, 1529, 1530, 1537, 1541, 1542, 1543, 1544, 1545, 1546, - /* 410 */ 1547, 1549, 1551, 1552, 1481, 1524, 1525, 1526, 1554, 1556, - /* 420 */ 1557, 1538, 1559, 1561, 1562, 1564, 1566, 1512, 1569, 1518, - /* 430 */ 1570, 1571, 1531, 1533, 1534, 1567, 1532, 1578, 1555, 1576, - /* 440 */ 1563, 1565, 1602, 1605, 1607, 1579, 1430, 1608, 1609, 1620, - /* 450 */ 1560, 1623, 1624, 1590, 1582, 1587, 1628, 1595, 1584, 1594, - /* 460 */ 1634, 1601, 1592, 1598, 1641, 1610, 1596, 1611, 1646, 1647, - /* 470 */ 1648, 1651, 1558, 1568, 1618, 1632, 1655, 1625, 1615, 1619, - /* 480 */ 1626, 1630, 1645, 1663, 1654, 1668, 1656, 1631, 1669, 1657, - /* 490 */ 1650, 1681, 1658, 1682, 1661, 1695, 1675, 1678, 1699, 1553, - /* 500 */ 1666, 1703, 1540, 1689, 1572, 1550, 1714, 1715, 1574, 1573, - /* 510 */ 1716, 1717, 1718, 1633, 1629, 1686, 1583, 1723, 1635, 1580, - /* 520 */ 1637, 1728, 1694, 1586, 1659, 1638, 1688, 1693, 1511, 1664, - /* 530 */ 1662, 1665, 1673, 1674, 1677, 1697, 1680, 1679, 1683, 1690, - /* 540 */ 1691, 1704, 1700, 1721, 1696, 1706, 1519, 1692, 1698, 1725, - /* 550 */ 1575, 1740, 1738, 1741, 1701, 1743, 1577, 1702, 1755, 1758, - /* 560 */ 1759, 1760, 1761, 1763, 1702, 1797, 1779, 1612, 1768, 1727, - /* 570 */ 1713, 1729, 1722, 1730, 1724, 1769, 1731, 1732, 1771, 1800, - /* 580 */ 1649, 1734, 1742, 1735, 1789, 1791, 1751, 1737, 1795, 1752, - /* 590 */ 1753, 1810, 1765, 1757, 1813, 1766, 1762, 1814, 1767, 1744, - /* 600 */ 1745, 1747, 1748, 1828, 1764, 1773, 1774, 1831, 1780, 1821, - /* 610 */ 1821, 1846, 1808, 1812, 1836, 1840, 1841, 1843, 1844, 1845, - /* 620 */ 1852, 1822, 1804, 1849, 1858, 1859, 1873, 1861, 1875, 1865, - /* 630 */ 1866, 1842, 1615, 1876, 1619, 1877, 1878, 1879, 1880, 1886, - /* 640 */ 1881, 1918, 1884, 1874, 1883, 1920, 1888, 1882, 1885, 1926, - /* 650 */ 1893, 1887, 1892, 1933, 1901, 1891, 1898, 1939, 1906, 1909, - /* 660 */ 1943, 1923, 1925, 1927, 1928, 1931, 1934, + /* 110 */ 213, 213, 213, 213, 213, 213, 213, 517, 881, 165, + /* 120 */ 403, 403, 573, 386, 849, 361, 361, 361, 386, 298, + /* 130 */ 298, 18, 350, 350, 67, 67, 294, 294, 270, 357, + /* 140 */ 198, 198, 198, 198, 198, 198, 198, 1119, 21, 383, + /* 150 */ 501, 105, 665, 484, 715, 828, 366, 799, 506, 800, + /* 160 */ 717, 649, 717, 924, 756, 756, 756, 807, 873, 980, + /* 170 */ 1174, 1078, 1206, 1232, 1232, 1206, 1107, 1107, 1232, 1232, + /* 180 */ 1232, 1257, 1257, 1264, 10, 18, 10, 1272, 1277, 10, + /* 190 */ 1272, 10, 10, 10, 1232, 10, 1257, 67, 67, 67, + /* 200 */ 67, 67, 67, 67, 67, 67, 67, 67, 1232, 1257, + /* 210 */ 294, 1264, 227, 1166, 18, 227, 1232, 1232, 1272, 227, + /* 220 */ 1118, 294, 294, 294, 294, 1118, 294, 1201, 227, 270, + /* 230 */ 227, 298, 1345, 294, 1139, 1118, 294, 294, 1139, 1118, + /* 240 */ 294, 294, 67, 1141, 1235, 1139, 1154, 1157, 1170, 980, + /* 250 */ 1178, 298, 1382, 1159, 1162, 1167, 1159, 1162, 1159, 1162, + /* 260 */ 1320, 1326, 294, 357, 1232, 227, 1383, 1257, 2259, 2259, + /* 270 */ 2259, 2259, 2259, 2259, 2259, 83, 1593, 214, 724, 126, + /* 280 */ 209, 491, 562, 622, 813, 535, 321, 698, 698, 698, + /* 290 */ 698, 698, 698, 698, 698, 521, 309, 13, 13, 115, + /* 300 */ 69, 599, 267, 708, 194, 377, 190, 465, 49, 49, + /* 310 */ 49, 49, 684, 944, 938, 994, 999, 1001, 947, 1047, + /* 320 */ 1101, 941, 920, 1025, 1043, 1069, 1079, 1091, 1093, 1102, + /* 330 */ 1160, 1073, 973, 895, 1103, 1075, 1082, 1058, 1106, 1114, + /* 340 */ 1115, 1133, 1163, 1165, 1193, 1198, 1064, 860, 1143, 1176, + /* 350 */ 839, 1473, 1476, 1301, 1479, 1480, 1439, 1482, 1448, 1297, + /* 360 */ 1450, 1451, 1452, 1302, 1490, 1456, 1457, 1307, 1495, 1310, + /* 370 */ 1502, 1477, 1513, 1493, 1516, 1483, 1348, 1347, 1521, 1527, + /* 380 */ 1357, 1359, 1525, 1528, 1494, 1529, 1499, 1539, 1542, 1543, + /* 390 */ 1395, 1545, 1552, 1554, 1555, 1556, 1403, 1514, 1548, 1413, + /* 400 */ 1559, 1560, 1561, 1562, 1566, 1567, 1568, 1569, 1570, 1571, + /* 410 */ 1573, 1574, 1575, 1578, 1544, 1590, 1591, 1598, 1600, 1601, + /* 420 */ 1612, 1581, 1604, 1605, 1614, 1616, 1619, 1564, 1621, 1572, + /* 430 */ 1627, 1629, 1582, 1584, 1588, 1620, 1587, 1622, 1589, 1638, + /* 440 */ 1599, 1602, 1650, 1651, 1652, 1617, 1492, 1657, 1658, 1665, + /* 450 */ 1607, 1667, 1671, 1608, 1597, 1633, 1673, 1639, 1630, 1637, + /* 460 */ 1679, 1645, 1640, 1647, 1688, 1654, 1644, 1653, 1693, 1694, + /* 470 */ 1698, 1699, 1609, 1606, 1666, 1678, 1703, 1670, 1675, 1680, + /* 480 */ 1686, 1695, 1684, 1710, 1711, 1732, 1713, 1702, 1737, 1722, + /* 490 */ 1717, 1754, 1720, 1756, 1724, 1757, 1738, 1741, 1762, 1610, + /* 500 */ 1731, 1768, 1613, 1748, 1618, 1624, 1771, 1773, 1625, 1615, + /* 510 */ 1780, 1781, 1783, 1696, 1697, 1755, 1634, 1798, 1712, 1648, + /* 520 */ 1715, 1800, 1778, 1659, 1735, 1727, 1782, 1784, 1611, 1740, + /* 530 */ 1742, 1745, 1746, 1747, 1749, 1787, 1750, 1758, 1760, 1761, + /* 540 */ 1763, 1792, 1793, 1799, 1765, 1801, 1626, 1770, 1772, 1815, + /* 550 */ 1641, 1809, 1817, 1818, 1775, 1825, 1642, 1779, 1835, 1849, + /* 560 */ 1858, 1859, 1860, 1864, 1779, 1898, 1885, 1716, 1867, 1823, + /* 570 */ 1824, 1826, 1827, 1829, 1830, 1870, 1833, 1834, 1878, 1891, + /* 580 */ 1743, 1844, 1819, 1845, 1899, 1901, 1848, 1850, 1906, 1853, + /* 590 */ 1854, 1908, 1856, 1863, 1911, 1861, 1865, 1913, 1868, 1836, + /* 600 */ 1843, 1847, 1851, 1936, 1862, 1872, 1876, 1927, 1877, 1925, + /* 610 */ 1925, 1947, 1914, 1909, 1938, 1940, 1943, 1945, 1954, 1955, + /* 620 */ 1958, 1928, 1922, 1952, 1963, 1965, 1988, 1976, 1991, 1980, + /* 630 */ 1981, 1950, 1675, 1985, 1680, 1994, 1995, 1998, 2000, 2014, + /* 640 */ 2003, 2039, 2005, 2002, 2011, 2051, 2017, 2006, 2015, 2055, + /* 650 */ 2021, 2013, 2019, 2069, 2035, 2024, 2034, 2076, 2042, 2043, + /* 660 */ 2079, 2058, 2060, 2062, 2063, 2061, 2066, }; -#define YY_REDUCE_COUNT (275) -#define YY_REDUCE_MIN (-389) -#define YY_REDUCE_MAX (2125) +#define YY_REDUCE_COUNT (274) +#define YY_REDUCE_MIN (-403) +#define YY_REDUCE_MAX (1873) static const short yy_reduce_ofst[] = { - /* 0 */ -75, 602, 629, -279, -15, 753, 815, 867, 923, 933, - /* 10 */ 273, 982, 104, 1042, 1052, 1101, 1152, 1204, 1214, 1231, - /* 20 */ 1291, 1308, 1376, 1394, 1453, 1502, 1523, 1581, 1591, 1643, - /* 30 */ 1660, 1709, 1726, 1775, 1829, 1839, 1890, 1908, 1967, 1988, - /* 40 */ 2044, 2054, 2078, 2125, -312, 33, 425, -295, 436, 490, - /* 50 */ -308, 654, -345, -68, 77, 151, 246, -316, -310, -307, - /* 60 */ -276, -285, -238, -87, -78, -313, -33, 239, 367, 418, - /* 70 */ 420, 427, 505, 623, 689, 161, 698, 722, 723, -234, - /* 80 */ 729, 730, 727, 742, 744, -244, -153, 749, 754, -10, - /* 90 */ 782, 767, 91, -124, -389, -389, -389, -164, -51, -30, - /* 100 */ -21, 100, 148, 205, 272, 311, 323, 357, 381, 382, - /* 110 */ 430, 464, 540, 603, 607, 640, 641, 28, -263, -335, - /* 120 */ 392, 443, 438, -229, -85, 234, 467, 570, 79, 18, - /* 130 */ 365, 312, 341, 413, 223, 665, 545, 596, 668, 610, - /* 140 */ 90, 220, 249, 308, 373, 387, 398, 252, 534, 518, - /* 150 */ 645, 571, 677, 693, 680, 781, 781, 839, 848, 817, - /* 160 */ 793, 769, 769, 769, 783, 758, 762, 763, 777, 781, - /* 170 */ 812, 814, 836, 876, 918, 919, 880, 884, 889, 928, - /* 180 */ 936, 941, 935, 945, 894, 946, 914, 950, 912, 911, - /* 190 */ 958, 916, 960, 961, 963, 969, 968, 976, 953, 956, - /* 200 */ 957, 959, 964, 965, 974, 975, 984, 985, 986, 978, - /* 210 */ 981, 947, 954, 1005, 930, 979, 1009, 1013, 1016, 971, - /* 220 */ 1017, 980, 988, 991, 993, 995, 987, 998, 992, 1030, - /* 230 */ 1022, 1038, 1014, 989, 1004, 962, 1000, 1023, 1028, 970, - /* 240 */ 1011, 1031, 1033, 781, 973, 972, 983, 990, 996, 999, - /* 250 */ 1024, 769, 1051, 1027, 997, 1029, 1003, 1018, 1036, 1032, - /* 260 */ 1037, 1055, 1070, 1062, 1086, 1098, 1095, 1105, 1109, 1053, - /* 270 */ 1068, 1113, 1114, 1118, 1132, 1149, + /* 0 */ 263, 629, 747, -278, -14, 679, 846, 886, 955, 1009, + /* 10 */ 272, 1059, 104, 1076, 1126, 806, 1152, 1202, 1214, 1228, + /* 20 */ 1281, 1334, 1346, 1360, 1414, 1428, 1440, 1497, 1507, 1523, + /* 30 */ 1547, 1603, 1623, 1643, 1663, 1683, 1723, 1739, 1790, 1806, + /* 40 */ 1857, 1873, 304, 926, 112, 435, 552, -295, 610, -3, + /* 50 */ -261, -154, -323, 305, 575, 577, 635, -310, -272, -312, + /* 60 */ -307, -403, -311, -284, -70, -90, 225, 346, 404, 414, + /* 70 */ 429, 434, 497, 644, 645, 339, 674, 697, 710, -342, + /* 80 */ 722, 749, -313, 758, 772, -244, -251, 797, -17, 20, + /* 90 */ 48, 804, 228, 86, -379, -379, -379, -240, -13, 100, + /* 100 */ 101, 130, 206, 375, 385, 428, 459, 461, 474, 522, + /* 110 */ 566, 628, 656, 678, 694, 696, 740, -226, -92, -72, + /* 120 */ -256, -130, 173, 4, 34, 186, 362, 440, 97, 91, + /* 130 */ 303, -234, -109, 180, 274, 536, 317, 545, 579, 234, + /* 140 */ -333, 40, 144, 175, 242, 319, 410, 450, 445, 348, + /* 150 */ 369, 495, 572, 557, 620, 620, 734, 667, 688, 727, + /* 160 */ 735, 735, 735, 770, 752, 784, 786, 762, 620, 861, + /* 170 */ 874, 866, 894, 940, 942, 901, 907, 908, 956, 957, + /* 180 */ 960, 971, 972, 915, 966, 936, 974, 933, 935, 978, + /* 190 */ 939, 981, 983, 985, 991, 989, 1000, 975, 976, 977, + /* 200 */ 979, 982, 984, 986, 990, 992, 995, 996, 998, 1006, + /* 210 */ 993, 951, 1021, 987, 997, 1026, 1030, 1033, 988, 1032, + /* 220 */ 1002, 1005, 1008, 1013, 1014, 1010, 1015, 1011, 1048, 1038, + /* 230 */ 1053, 1031, 1003, 1028, 968, 1020, 1036, 1037, 1004, 1022, + /* 240 */ 1044, 1049, 620, 1016, 1012, 1018, 1023, 1027, 1024, 1029, + /* 250 */ 735, 1061, 1035, 1045, 1017, 1039, 1046, 1034, 1051, 1050, + /* 260 */ 1052, 1081, 1067, 1092, 1129, 1104, 1112, 1136, 1096, 1086, + /* 270 */ 1137, 1138, 1142, 1130, 1168, }; static const YYACTIONTYPE yy_default[] = { - /* 0 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 10 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 20 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 30 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 40 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 50 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 60 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 70 */ 1464, 1464, 1464, 1464, 1464, 1538, 1464, 1464, 1464, 1464, - /* 80 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 90 */ 1464, 1464, 1536, 1694, 1464, 1871, 1464, 1464, 1464, 1464, - /* 100 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 110 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 120 */ 1464, 1464, 1538, 1464, 1536, 1883, 1883, 1883, 1464, 1464, - /* 130 */ 1464, 1464, 1737, 1737, 1464, 1464, 1464, 1464, 1636, 1464, - /* 140 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1729, 1464, 1952, - /* 150 */ 1464, 1464, 1464, 1735, 1906, 1464, 1464, 1464, 1464, 1589, - /* 160 */ 1898, 1875, 1889, 1876, 1873, 1937, 1937, 1937, 1892, 1464, - /* 170 */ 1902, 1464, 1722, 1699, 1464, 1464, 1699, 1696, 1696, 1464, - /* 180 */ 1464, 1464, 1464, 1464, 1464, 1538, 1464, 1538, 1464, 1464, - /* 190 */ 1538, 1464, 1538, 1538, 1538, 1464, 1538, 1464, 1464, 1464, - /* 200 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 210 */ 1464, 1464, 1464, 1536, 1731, 1464, 1536, 1464, 1464, 1464, - /* 220 */ 1536, 1911, 1464, 1464, 1464, 1464, 1911, 1464, 1464, 1536, - /* 230 */ 1464, 1536, 1464, 1464, 1464, 1913, 1911, 1464, 1464, 1913, - /* 240 */ 1911, 1464, 1464, 1464, 1925, 1921, 1913, 1929, 1927, 1904, - /* 250 */ 1902, 1889, 1464, 1464, 1943, 1939, 1955, 1943, 1939, 1943, - /* 260 */ 1939, 1464, 1605, 1464, 1464, 1464, 1536, 1496, 1464, 1724, - /* 270 */ 1737, 1639, 1639, 1639, 1539, 1469, 1464, 1464, 1464, 1464, - /* 280 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1808, 1924, - /* 290 */ 1923, 1847, 1846, 1845, 1843, 1807, 1464, 1601, 1806, 1805, - /* 300 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1799, 1800, - /* 310 */ 1798, 1797, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 320 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 330 */ 1872, 1464, 1940, 1944, 1464, 1464, 1464, 1464, 1464, 1783, - /* 340 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 350 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 360 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 370 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 380 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 390 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 400 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 410 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 420 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 430 */ 1464, 1464, 1464, 1464, 1501, 1464, 1464, 1464, 1464, 1464, - /* 440 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 450 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 460 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 470 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1573, 1572, - /* 480 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 490 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 500 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 510 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1741, 1464, 1464, - /* 520 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1905, 1464, 1464, - /* 530 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 540 */ 1464, 1464, 1464, 1783, 1464, 1922, 1464, 1882, 1878, 1464, - /* 550 */ 1464, 1874, 1782, 1464, 1464, 1938, 1464, 1464, 1464, 1464, - /* 560 */ 1464, 1464, 1464, 1464, 1464, 1867, 1464, 1464, 1840, 1825, - /* 570 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 580 */ 1793, 1464, 1464, 1464, 1464, 1464, 1633, 1464, 1464, 1464, - /* 590 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1618, - /* 600 */ 1616, 1615, 1614, 1464, 1611, 1464, 1464, 1464, 1464, 1642, - /* 610 */ 1641, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 620 */ 1464, 1464, 1464, 1557, 1464, 1464, 1464, 1464, 1464, 1464, - /* 630 */ 1464, 1464, 1549, 1464, 1548, 1464, 1464, 1464, 1464, 1464, - /* 640 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 650 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, - /* 660 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 0 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 10 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 20 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 30 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 40 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 50 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 60 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 70 */ 1461, 1461, 1461, 1461, 1461, 1535, 1461, 1461, 1461, 1461, + /* 80 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 90 */ 1461, 1461, 1533, 1691, 1461, 1866, 1461, 1461, 1461, 1461, + /* 100 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 110 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 120 */ 1461, 1461, 1535, 1461, 1533, 1878, 1878, 1878, 1461, 1461, + /* 130 */ 1461, 1461, 1732, 1732, 1461, 1461, 1461, 1461, 1633, 1461, + /* 140 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1726, 1461, 1947, + /* 150 */ 1461, 1461, 1461, 1901, 1461, 1461, 1461, 1461, 1586, 1893, + /* 160 */ 1870, 1884, 1871, 1868, 1932, 1932, 1932, 1887, 1461, 1897, + /* 170 */ 1461, 1719, 1696, 1461, 1461, 1696, 1693, 1693, 1461, 1461, + /* 180 */ 1461, 1461, 1461, 1461, 1535, 1461, 1535, 1461, 1461, 1535, + /* 190 */ 1461, 1535, 1535, 1535, 1461, 1535, 1461, 1461, 1461, 1461, + /* 200 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 210 */ 1461, 1461, 1533, 1728, 1461, 1533, 1461, 1461, 1461, 1533, + /* 220 */ 1906, 1461, 1461, 1461, 1461, 1906, 1461, 1461, 1533, 1461, + /* 230 */ 1533, 1461, 1461, 1461, 1908, 1906, 1461, 1461, 1908, 1906, + /* 240 */ 1461, 1461, 1461, 1920, 1916, 1908, 1924, 1922, 1899, 1897, + /* 250 */ 1884, 1461, 1461, 1938, 1934, 1950, 1938, 1934, 1938, 1934, + /* 260 */ 1461, 1602, 1461, 1461, 1461, 1533, 1493, 1461, 1721, 1732, + /* 270 */ 1636, 1636, 1636, 1536, 1466, 1461, 1461, 1461, 1461, 1461, + /* 280 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1803, 1919, 1918, + /* 290 */ 1842, 1841, 1840, 1838, 1802, 1461, 1598, 1801, 1800, 1461, + /* 300 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1794, 1795, + /* 310 */ 1793, 1792, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 320 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 330 */ 1867, 1461, 1935, 1939, 1461, 1461, 1461, 1461, 1461, 1778, + /* 340 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 350 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 360 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 370 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 380 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 390 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 400 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 410 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 420 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 430 */ 1461, 1461, 1461, 1461, 1498, 1461, 1461, 1461, 1461, 1461, + /* 440 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 450 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 460 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 470 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1570, 1569, + /* 480 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 490 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 500 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 510 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1736, 1461, 1461, + /* 520 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1900, 1461, 1461, + /* 530 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 540 */ 1461, 1461, 1461, 1778, 1461, 1917, 1461, 1877, 1873, 1461, + /* 550 */ 1461, 1869, 1777, 1461, 1461, 1933, 1461, 1461, 1461, 1461, + /* 560 */ 1461, 1461, 1461, 1461, 1461, 1862, 1461, 1461, 1835, 1820, + /* 570 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 580 */ 1788, 1461, 1461, 1461, 1461, 1461, 1630, 1461, 1461, 1461, + /* 590 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1615, + /* 600 */ 1613, 1612, 1611, 1461, 1608, 1461, 1461, 1461, 1461, 1639, + /* 610 */ 1638, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 620 */ 1461, 1461, 1461, 1554, 1461, 1461, 1461, 1461, 1461, 1461, + /* 630 */ 1461, 1461, 1546, 1461, 1545, 1461, 1461, 1461, 1461, 1461, + /* 640 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 650 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, 1461, + /* 660 */ 1461, 1461, 1461, 1461, 1461, 1461, 1461, }; /********** End of lemon-generated parsing tables *****************************/ @@ -1686,62 +1642,61 @@ static const char *const yyTokenName[] = { /* 368 */ "agg_func_opt", /* 369 */ "bufsize_opt", /* 370 */ "stream_name", - /* 371 */ "into_opt", - /* 372 */ "dnode_list", - /* 373 */ "where_clause_opt", - /* 374 */ "signed", - /* 375 */ "literal_func", - /* 376 */ "literal_list", - /* 377 */ "table_alias", - /* 378 */ "column_alias", - /* 379 */ "expression", - /* 380 */ "pseudo_column", - /* 381 */ "column_reference", - /* 382 */ "function_expression", - /* 383 */ "subquery", - /* 384 */ "star_func", - /* 385 */ "star_func_para_list", - /* 386 */ "noarg_func", - /* 387 */ "other_para_list", - /* 388 */ "star_func_para", - /* 389 */ "predicate", - /* 390 */ "compare_op", - /* 391 */ "in_op", - /* 392 */ "in_predicate_value", - /* 393 */ "boolean_value_expression", - /* 394 */ "boolean_primary", - /* 395 */ "common_expression", - /* 396 */ "from_clause_opt", - /* 397 */ "table_reference_list", - /* 398 */ "table_reference", - /* 399 */ "table_primary", - /* 400 */ "joined_table", - /* 401 */ "alias_opt", - /* 402 */ "parenthesized_joined_table", - /* 403 */ "join_type", - /* 404 */ "search_condition", - /* 405 */ "query_specification", - /* 406 */ "set_quantifier_opt", - /* 407 */ "select_list", - /* 408 */ "partition_by_clause_opt", - /* 409 */ "range_opt", - /* 410 */ "every_opt", - /* 411 */ "fill_opt", - /* 412 */ "twindow_clause_opt", - /* 413 */ "group_by_clause_opt", - /* 414 */ "having_clause_opt", - /* 415 */ "select_item", - /* 416 */ "fill_mode", - /* 417 */ "group_by_list", - /* 418 */ "query_expression_body", - /* 419 */ "order_by_clause_opt", - /* 420 */ "slimit_clause_opt", - /* 421 */ "limit_clause_opt", - /* 422 */ "query_primary", - /* 423 */ "sort_specification_list", - /* 424 */ "sort_specification", - /* 425 */ "ordering_specification_opt", - /* 426 */ "null_ordering_opt", + /* 371 */ "dnode_list", + /* 372 */ "where_clause_opt", + /* 373 */ "signed", + /* 374 */ "literal_func", + /* 375 */ "literal_list", + /* 376 */ "table_alias", + /* 377 */ "column_alias", + /* 378 */ "expression", + /* 379 */ "pseudo_column", + /* 380 */ "column_reference", + /* 381 */ "function_expression", + /* 382 */ "subquery", + /* 383 */ "star_func", + /* 384 */ "star_func_para_list", + /* 385 */ "noarg_func", + /* 386 */ "other_para_list", + /* 387 */ "star_func_para", + /* 388 */ "predicate", + /* 389 */ "compare_op", + /* 390 */ "in_op", + /* 391 */ "in_predicate_value", + /* 392 */ "boolean_value_expression", + /* 393 */ "boolean_primary", + /* 394 */ "common_expression", + /* 395 */ "from_clause_opt", + /* 396 */ "table_reference_list", + /* 397 */ "table_reference", + /* 398 */ "table_primary", + /* 399 */ "joined_table", + /* 400 */ "alias_opt", + /* 401 */ "parenthesized_joined_table", + /* 402 */ "join_type", + /* 403 */ "search_condition", + /* 404 */ "query_specification", + /* 405 */ "set_quantifier_opt", + /* 406 */ "select_list", + /* 407 */ "partition_by_clause_opt", + /* 408 */ "range_opt", + /* 409 */ "every_opt", + /* 410 */ "fill_opt", + /* 411 */ "twindow_clause_opt", + /* 412 */ "group_by_clause_opt", + /* 413 */ "having_clause_opt", + /* 414 */ "select_item", + /* 415 */ "fill_mode", + /* 416 */ "group_by_list", + /* 417 */ "query_expression_body", + /* 418 */ "order_by_clause_opt", + /* 419 */ "slimit_clause_opt", + /* 420 */ "limit_clause_opt", + /* 421 */ "query_primary", + /* 422 */ "sort_specification_list", + /* 423 */ "sort_specification", + /* 424 */ "ordering_specification_opt", + /* 425 */ "null_ordering_opt", }; #endif /* defined(YYCOVERAGE) || !defined(NDEBUG) */ @@ -2015,231 +1970,229 @@ static const char *const yyRuleName[] = { /* 263 */ "agg_func_opt ::= AGGREGATE", /* 264 */ "bufsize_opt ::=", /* 265 */ "bufsize_opt ::= BUFSIZE NK_INTEGER", - /* 266 */ "cmd ::= CREATE STREAM not_exists_opt stream_name stream_options into_opt AS query_expression", + /* 266 */ "cmd ::= CREATE STREAM not_exists_opt stream_name stream_options INTO full_table_name AS query_expression", /* 267 */ "cmd ::= DROP STREAM exists_opt stream_name", - /* 268 */ "into_opt ::=", - /* 269 */ "into_opt ::= INTO full_table_name", - /* 270 */ "stream_options ::=", - /* 271 */ "stream_options ::= stream_options TRIGGER AT_ONCE", - /* 272 */ "stream_options ::= stream_options TRIGGER WINDOW_CLOSE", - /* 273 */ "stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal", - /* 274 */ "stream_options ::= stream_options WATERMARK duration_literal", - /* 275 */ "stream_options ::= stream_options IGNORE EXPIRED NK_INTEGER", - /* 276 */ "cmd ::= KILL CONNECTION NK_INTEGER", - /* 277 */ "cmd ::= KILL QUERY NK_STRING", - /* 278 */ "cmd ::= KILL TRANSACTION NK_INTEGER", - /* 279 */ "cmd ::= BALANCE VGROUP", - /* 280 */ "cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER", - /* 281 */ "cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list", - /* 282 */ "cmd ::= SPLIT VGROUP NK_INTEGER", - /* 283 */ "dnode_list ::= DNODE NK_INTEGER", - /* 284 */ "dnode_list ::= dnode_list DNODE NK_INTEGER", - /* 285 */ "cmd ::= DELETE FROM full_table_name where_clause_opt", - /* 286 */ "cmd ::= query_expression", - /* 287 */ "cmd ::= INSERT INTO full_table_name NK_LP col_name_list NK_RP query_expression", - /* 288 */ "cmd ::= INSERT INTO full_table_name query_expression", - /* 289 */ "literal ::= NK_INTEGER", - /* 290 */ "literal ::= NK_FLOAT", - /* 291 */ "literal ::= NK_STRING", - /* 292 */ "literal ::= NK_BOOL", - /* 293 */ "literal ::= TIMESTAMP NK_STRING", - /* 294 */ "literal ::= duration_literal", - /* 295 */ "literal ::= NULL", - /* 296 */ "literal ::= NK_QUESTION", - /* 297 */ "duration_literal ::= NK_VARIABLE", - /* 298 */ "signed ::= NK_INTEGER", - /* 299 */ "signed ::= NK_PLUS NK_INTEGER", - /* 300 */ "signed ::= NK_MINUS NK_INTEGER", - /* 301 */ "signed ::= NK_FLOAT", - /* 302 */ "signed ::= NK_PLUS NK_FLOAT", - /* 303 */ "signed ::= NK_MINUS NK_FLOAT", - /* 304 */ "signed_literal ::= signed", - /* 305 */ "signed_literal ::= NK_STRING", - /* 306 */ "signed_literal ::= NK_BOOL", - /* 307 */ "signed_literal ::= TIMESTAMP NK_STRING", - /* 308 */ "signed_literal ::= duration_literal", - /* 309 */ "signed_literal ::= NULL", - /* 310 */ "signed_literal ::= literal_func", - /* 311 */ "signed_literal ::= NK_QUESTION", - /* 312 */ "literal_list ::= signed_literal", - /* 313 */ "literal_list ::= literal_list NK_COMMA signed_literal", - /* 314 */ "db_name ::= NK_ID", - /* 315 */ "table_name ::= NK_ID", - /* 316 */ "column_name ::= NK_ID", - /* 317 */ "function_name ::= NK_ID", - /* 318 */ "table_alias ::= NK_ID", - /* 319 */ "column_alias ::= NK_ID", - /* 320 */ "user_name ::= NK_ID", - /* 321 */ "topic_name ::= NK_ID", - /* 322 */ "stream_name ::= NK_ID", - /* 323 */ "cgroup_name ::= NK_ID", - /* 324 */ "expression ::= literal", - /* 325 */ "expression ::= pseudo_column", - /* 326 */ "expression ::= column_reference", - /* 327 */ "expression ::= function_expression", - /* 328 */ "expression ::= subquery", - /* 329 */ "expression ::= NK_LP expression NK_RP", - /* 330 */ "expression ::= NK_PLUS expression", - /* 331 */ "expression ::= NK_MINUS expression", - /* 332 */ "expression ::= expression NK_PLUS expression", - /* 333 */ "expression ::= expression NK_MINUS expression", - /* 334 */ "expression ::= expression NK_STAR expression", - /* 335 */ "expression ::= expression NK_SLASH expression", - /* 336 */ "expression ::= expression NK_REM expression", - /* 337 */ "expression ::= column_reference NK_ARROW NK_STRING", - /* 338 */ "expression ::= expression NK_BITAND expression", - /* 339 */ "expression ::= expression NK_BITOR expression", - /* 340 */ "expression_list ::= expression", - /* 341 */ "expression_list ::= expression_list NK_COMMA expression", - /* 342 */ "column_reference ::= column_name", - /* 343 */ "column_reference ::= table_name NK_DOT column_name", - /* 344 */ "pseudo_column ::= ROWTS", - /* 345 */ "pseudo_column ::= TBNAME", - /* 346 */ "pseudo_column ::= table_name NK_DOT TBNAME", - /* 347 */ "pseudo_column ::= QSTART", - /* 348 */ "pseudo_column ::= QEND", - /* 349 */ "pseudo_column ::= QDURATION", - /* 350 */ "pseudo_column ::= WSTART", - /* 351 */ "pseudo_column ::= WEND", - /* 352 */ "pseudo_column ::= WDURATION", - /* 353 */ "function_expression ::= function_name NK_LP expression_list NK_RP", - /* 354 */ "function_expression ::= star_func NK_LP star_func_para_list NK_RP", - /* 355 */ "function_expression ::= CAST NK_LP expression AS type_name NK_RP", - /* 356 */ "function_expression ::= literal_func", - /* 357 */ "literal_func ::= noarg_func NK_LP NK_RP", - /* 358 */ "literal_func ::= NOW", - /* 359 */ "noarg_func ::= NOW", - /* 360 */ "noarg_func ::= TODAY", - /* 361 */ "noarg_func ::= TIMEZONE", - /* 362 */ "noarg_func ::= DATABASE", - /* 363 */ "noarg_func ::= CLIENT_VERSION", - /* 364 */ "noarg_func ::= SERVER_VERSION", - /* 365 */ "noarg_func ::= SERVER_STATUS", - /* 366 */ "noarg_func ::= CURRENT_USER", - /* 367 */ "noarg_func ::= USER", - /* 368 */ "star_func ::= COUNT", - /* 369 */ "star_func ::= FIRST", - /* 370 */ "star_func ::= LAST", - /* 371 */ "star_func ::= LAST_ROW", - /* 372 */ "star_func_para_list ::= NK_STAR", - /* 373 */ "star_func_para_list ::= other_para_list", - /* 374 */ "other_para_list ::= star_func_para", - /* 375 */ "other_para_list ::= other_para_list NK_COMMA star_func_para", - /* 376 */ "star_func_para ::= expression", - /* 377 */ "star_func_para ::= table_name NK_DOT NK_STAR", - /* 378 */ "predicate ::= expression compare_op expression", - /* 379 */ "predicate ::= expression BETWEEN expression AND expression", - /* 380 */ "predicate ::= expression NOT BETWEEN expression AND expression", - /* 381 */ "predicate ::= expression IS NULL", - /* 382 */ "predicate ::= expression IS NOT NULL", - /* 383 */ "predicate ::= expression in_op in_predicate_value", - /* 384 */ "compare_op ::= NK_LT", - /* 385 */ "compare_op ::= NK_GT", - /* 386 */ "compare_op ::= NK_LE", - /* 387 */ "compare_op ::= NK_GE", - /* 388 */ "compare_op ::= NK_NE", - /* 389 */ "compare_op ::= NK_EQ", - /* 390 */ "compare_op ::= LIKE", - /* 391 */ "compare_op ::= NOT LIKE", - /* 392 */ "compare_op ::= MATCH", - /* 393 */ "compare_op ::= NMATCH", - /* 394 */ "compare_op ::= CONTAINS", - /* 395 */ "in_op ::= IN", - /* 396 */ "in_op ::= NOT IN", - /* 397 */ "in_predicate_value ::= NK_LP literal_list NK_RP", - /* 398 */ "boolean_value_expression ::= boolean_primary", - /* 399 */ "boolean_value_expression ::= NOT boolean_primary", - /* 400 */ "boolean_value_expression ::= boolean_value_expression OR boolean_value_expression", - /* 401 */ "boolean_value_expression ::= boolean_value_expression AND boolean_value_expression", - /* 402 */ "boolean_primary ::= predicate", - /* 403 */ "boolean_primary ::= NK_LP boolean_value_expression NK_RP", - /* 404 */ "common_expression ::= expression", - /* 405 */ "common_expression ::= boolean_value_expression", - /* 406 */ "from_clause_opt ::=", - /* 407 */ "from_clause_opt ::= FROM table_reference_list", - /* 408 */ "table_reference_list ::= table_reference", - /* 409 */ "table_reference_list ::= table_reference_list NK_COMMA table_reference", - /* 410 */ "table_reference ::= table_primary", - /* 411 */ "table_reference ::= joined_table", - /* 412 */ "table_primary ::= table_name alias_opt", - /* 413 */ "table_primary ::= db_name NK_DOT table_name alias_opt", - /* 414 */ "table_primary ::= subquery alias_opt", - /* 415 */ "table_primary ::= parenthesized_joined_table", - /* 416 */ "alias_opt ::=", - /* 417 */ "alias_opt ::= table_alias", - /* 418 */ "alias_opt ::= AS table_alias", - /* 419 */ "parenthesized_joined_table ::= NK_LP joined_table NK_RP", - /* 420 */ "parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP", - /* 421 */ "joined_table ::= table_reference join_type JOIN table_reference ON search_condition", - /* 422 */ "join_type ::=", - /* 423 */ "join_type ::= INNER", - /* 424 */ "query_specification ::= SELECT set_quantifier_opt select_list from_clause_opt where_clause_opt partition_by_clause_opt range_opt every_opt fill_opt twindow_clause_opt group_by_clause_opt having_clause_opt", - /* 425 */ "set_quantifier_opt ::=", - /* 426 */ "set_quantifier_opt ::= DISTINCT", - /* 427 */ "set_quantifier_opt ::= ALL", - /* 428 */ "select_list ::= select_item", - /* 429 */ "select_list ::= select_list NK_COMMA select_item", - /* 430 */ "select_item ::= NK_STAR", - /* 431 */ "select_item ::= common_expression", - /* 432 */ "select_item ::= common_expression column_alias", - /* 433 */ "select_item ::= common_expression AS column_alias", - /* 434 */ "select_item ::= table_name NK_DOT NK_STAR", - /* 435 */ "where_clause_opt ::=", - /* 436 */ "where_clause_opt ::= WHERE search_condition", - /* 437 */ "partition_by_clause_opt ::=", - /* 438 */ "partition_by_clause_opt ::= PARTITION BY expression_list", - /* 439 */ "twindow_clause_opt ::=", - /* 440 */ "twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP", - /* 441 */ "twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP", - /* 442 */ "twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt", - /* 443 */ "twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt", - /* 444 */ "sliding_opt ::=", - /* 445 */ "sliding_opt ::= SLIDING NK_LP duration_literal NK_RP", - /* 446 */ "fill_opt ::=", - /* 447 */ "fill_opt ::= FILL NK_LP fill_mode NK_RP", - /* 448 */ "fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP", - /* 449 */ "fill_mode ::= NONE", - /* 450 */ "fill_mode ::= PREV", - /* 451 */ "fill_mode ::= NULL", - /* 452 */ "fill_mode ::= LINEAR", - /* 453 */ "fill_mode ::= NEXT", - /* 454 */ "group_by_clause_opt ::=", - /* 455 */ "group_by_clause_opt ::= GROUP BY group_by_list", - /* 456 */ "group_by_list ::= expression", - /* 457 */ "group_by_list ::= group_by_list NK_COMMA expression", - /* 458 */ "having_clause_opt ::=", - /* 459 */ "having_clause_opt ::= HAVING search_condition", - /* 460 */ "range_opt ::=", - /* 461 */ "range_opt ::= RANGE NK_LP expression NK_COMMA expression NK_RP", - /* 462 */ "every_opt ::=", - /* 463 */ "every_opt ::= EVERY NK_LP duration_literal NK_RP", - /* 464 */ "query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt", - /* 465 */ "query_expression_body ::= query_primary", - /* 466 */ "query_expression_body ::= query_expression_body UNION ALL query_expression_body", - /* 467 */ "query_expression_body ::= query_expression_body UNION query_expression_body", - /* 468 */ "query_primary ::= query_specification", - /* 469 */ "query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP", - /* 470 */ "order_by_clause_opt ::=", - /* 471 */ "order_by_clause_opt ::= ORDER BY sort_specification_list", - /* 472 */ "slimit_clause_opt ::=", - /* 473 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER", - /* 474 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER", - /* 475 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER", - /* 476 */ "limit_clause_opt ::=", - /* 477 */ "limit_clause_opt ::= LIMIT NK_INTEGER", - /* 478 */ "limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER", - /* 479 */ "limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER", - /* 480 */ "subquery ::= NK_LP query_expression NK_RP", - /* 481 */ "search_condition ::= common_expression", - /* 482 */ "sort_specification_list ::= sort_specification", - /* 483 */ "sort_specification_list ::= sort_specification_list NK_COMMA sort_specification", - /* 484 */ "sort_specification ::= expression ordering_specification_opt null_ordering_opt", - /* 485 */ "ordering_specification_opt ::=", - /* 486 */ "ordering_specification_opt ::= ASC", - /* 487 */ "ordering_specification_opt ::= DESC", - /* 488 */ "null_ordering_opt ::=", - /* 489 */ "null_ordering_opt ::= NULLS FIRST", - /* 490 */ "null_ordering_opt ::= NULLS LAST", + /* 268 */ "stream_options ::=", + /* 269 */ "stream_options ::= stream_options TRIGGER AT_ONCE", + /* 270 */ "stream_options ::= stream_options TRIGGER WINDOW_CLOSE", + /* 271 */ "stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal", + /* 272 */ "stream_options ::= stream_options WATERMARK duration_literal", + /* 273 */ "stream_options ::= stream_options IGNORE EXPIRED NK_INTEGER", + /* 274 */ "cmd ::= KILL CONNECTION NK_INTEGER", + /* 275 */ "cmd ::= KILL QUERY NK_STRING", + /* 276 */ "cmd ::= KILL TRANSACTION NK_INTEGER", + /* 277 */ "cmd ::= BALANCE VGROUP", + /* 278 */ "cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER", + /* 279 */ "cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list", + /* 280 */ "cmd ::= SPLIT VGROUP NK_INTEGER", + /* 281 */ "dnode_list ::= DNODE NK_INTEGER", + /* 282 */ "dnode_list ::= dnode_list DNODE NK_INTEGER", + /* 283 */ "cmd ::= DELETE FROM full_table_name where_clause_opt", + /* 284 */ "cmd ::= query_expression", + /* 285 */ "cmd ::= INSERT INTO full_table_name NK_LP col_name_list NK_RP query_expression", + /* 286 */ "cmd ::= INSERT INTO full_table_name query_expression", + /* 287 */ "literal ::= NK_INTEGER", + /* 288 */ "literal ::= NK_FLOAT", + /* 289 */ "literal ::= NK_STRING", + /* 290 */ "literal ::= NK_BOOL", + /* 291 */ "literal ::= TIMESTAMP NK_STRING", + /* 292 */ "literal ::= duration_literal", + /* 293 */ "literal ::= NULL", + /* 294 */ "literal ::= NK_QUESTION", + /* 295 */ "duration_literal ::= NK_VARIABLE", + /* 296 */ "signed ::= NK_INTEGER", + /* 297 */ "signed ::= NK_PLUS NK_INTEGER", + /* 298 */ "signed ::= NK_MINUS NK_INTEGER", + /* 299 */ "signed ::= NK_FLOAT", + /* 300 */ "signed ::= NK_PLUS NK_FLOAT", + /* 301 */ "signed ::= NK_MINUS NK_FLOAT", + /* 302 */ "signed_literal ::= signed", + /* 303 */ "signed_literal ::= NK_STRING", + /* 304 */ "signed_literal ::= NK_BOOL", + /* 305 */ "signed_literal ::= TIMESTAMP NK_STRING", + /* 306 */ "signed_literal ::= duration_literal", + /* 307 */ "signed_literal ::= NULL", + /* 308 */ "signed_literal ::= literal_func", + /* 309 */ "signed_literal ::= NK_QUESTION", + /* 310 */ "literal_list ::= signed_literal", + /* 311 */ "literal_list ::= literal_list NK_COMMA signed_literal", + /* 312 */ "db_name ::= NK_ID", + /* 313 */ "table_name ::= NK_ID", + /* 314 */ "column_name ::= NK_ID", + /* 315 */ "function_name ::= NK_ID", + /* 316 */ "table_alias ::= NK_ID", + /* 317 */ "column_alias ::= NK_ID", + /* 318 */ "user_name ::= NK_ID", + /* 319 */ "topic_name ::= NK_ID", + /* 320 */ "stream_name ::= NK_ID", + /* 321 */ "cgroup_name ::= NK_ID", + /* 322 */ "expression ::= literal", + /* 323 */ "expression ::= pseudo_column", + /* 324 */ "expression ::= column_reference", + /* 325 */ "expression ::= function_expression", + /* 326 */ "expression ::= subquery", + /* 327 */ "expression ::= NK_LP expression NK_RP", + /* 328 */ "expression ::= NK_PLUS expression", + /* 329 */ "expression ::= NK_MINUS expression", + /* 330 */ "expression ::= expression NK_PLUS expression", + /* 331 */ "expression ::= expression NK_MINUS expression", + /* 332 */ "expression ::= expression NK_STAR expression", + /* 333 */ "expression ::= expression NK_SLASH expression", + /* 334 */ "expression ::= expression NK_REM expression", + /* 335 */ "expression ::= column_reference NK_ARROW NK_STRING", + /* 336 */ "expression ::= expression NK_BITAND expression", + /* 337 */ "expression ::= expression NK_BITOR expression", + /* 338 */ "expression_list ::= expression", + /* 339 */ "expression_list ::= expression_list NK_COMMA expression", + /* 340 */ "column_reference ::= column_name", + /* 341 */ "column_reference ::= table_name NK_DOT column_name", + /* 342 */ "pseudo_column ::= ROWTS", + /* 343 */ "pseudo_column ::= TBNAME", + /* 344 */ "pseudo_column ::= table_name NK_DOT TBNAME", + /* 345 */ "pseudo_column ::= QSTART", + /* 346 */ "pseudo_column ::= QEND", + /* 347 */ "pseudo_column ::= QDURATION", + /* 348 */ "pseudo_column ::= WSTART", + /* 349 */ "pseudo_column ::= WEND", + /* 350 */ "pseudo_column ::= WDURATION", + /* 351 */ "function_expression ::= function_name NK_LP expression_list NK_RP", + /* 352 */ "function_expression ::= star_func NK_LP star_func_para_list NK_RP", + /* 353 */ "function_expression ::= CAST NK_LP expression AS type_name NK_RP", + /* 354 */ "function_expression ::= literal_func", + /* 355 */ "literal_func ::= noarg_func NK_LP NK_RP", + /* 356 */ "literal_func ::= NOW", + /* 357 */ "noarg_func ::= NOW", + /* 358 */ "noarg_func ::= TODAY", + /* 359 */ "noarg_func ::= TIMEZONE", + /* 360 */ "noarg_func ::= DATABASE", + /* 361 */ "noarg_func ::= CLIENT_VERSION", + /* 362 */ "noarg_func ::= SERVER_VERSION", + /* 363 */ "noarg_func ::= SERVER_STATUS", + /* 364 */ "noarg_func ::= CURRENT_USER", + /* 365 */ "noarg_func ::= USER", + /* 366 */ "star_func ::= COUNT", + /* 367 */ "star_func ::= FIRST", + /* 368 */ "star_func ::= LAST", + /* 369 */ "star_func ::= LAST_ROW", + /* 370 */ "star_func_para_list ::= NK_STAR", + /* 371 */ "star_func_para_list ::= other_para_list", + /* 372 */ "other_para_list ::= star_func_para", + /* 373 */ "other_para_list ::= other_para_list NK_COMMA star_func_para", + /* 374 */ "star_func_para ::= expression", + /* 375 */ "star_func_para ::= table_name NK_DOT NK_STAR", + /* 376 */ "predicate ::= expression compare_op expression", + /* 377 */ "predicate ::= expression BETWEEN expression AND expression", + /* 378 */ "predicate ::= expression NOT BETWEEN expression AND expression", + /* 379 */ "predicate ::= expression IS NULL", + /* 380 */ "predicate ::= expression IS NOT NULL", + /* 381 */ "predicate ::= expression in_op in_predicate_value", + /* 382 */ "compare_op ::= NK_LT", + /* 383 */ "compare_op ::= NK_GT", + /* 384 */ "compare_op ::= NK_LE", + /* 385 */ "compare_op ::= NK_GE", + /* 386 */ "compare_op ::= NK_NE", + /* 387 */ "compare_op ::= NK_EQ", + /* 388 */ "compare_op ::= LIKE", + /* 389 */ "compare_op ::= NOT LIKE", + /* 390 */ "compare_op ::= MATCH", + /* 391 */ "compare_op ::= NMATCH", + /* 392 */ "compare_op ::= CONTAINS", + /* 393 */ "in_op ::= IN", + /* 394 */ "in_op ::= NOT IN", + /* 395 */ "in_predicate_value ::= NK_LP literal_list NK_RP", + /* 396 */ "boolean_value_expression ::= boolean_primary", + /* 397 */ "boolean_value_expression ::= NOT boolean_primary", + /* 398 */ "boolean_value_expression ::= boolean_value_expression OR boolean_value_expression", + /* 399 */ "boolean_value_expression ::= boolean_value_expression AND boolean_value_expression", + /* 400 */ "boolean_primary ::= predicate", + /* 401 */ "boolean_primary ::= NK_LP boolean_value_expression NK_RP", + /* 402 */ "common_expression ::= expression", + /* 403 */ "common_expression ::= boolean_value_expression", + /* 404 */ "from_clause_opt ::=", + /* 405 */ "from_clause_opt ::= FROM table_reference_list", + /* 406 */ "table_reference_list ::= table_reference", + /* 407 */ "table_reference_list ::= table_reference_list NK_COMMA table_reference", + /* 408 */ "table_reference ::= table_primary", + /* 409 */ "table_reference ::= joined_table", + /* 410 */ "table_primary ::= table_name alias_opt", + /* 411 */ "table_primary ::= db_name NK_DOT table_name alias_opt", + /* 412 */ "table_primary ::= subquery alias_opt", + /* 413 */ "table_primary ::= parenthesized_joined_table", + /* 414 */ "alias_opt ::=", + /* 415 */ "alias_opt ::= table_alias", + /* 416 */ "alias_opt ::= AS table_alias", + /* 417 */ "parenthesized_joined_table ::= NK_LP joined_table NK_RP", + /* 418 */ "parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP", + /* 419 */ "joined_table ::= table_reference join_type JOIN table_reference ON search_condition", + /* 420 */ "join_type ::=", + /* 421 */ "join_type ::= INNER", + /* 422 */ "query_specification ::= SELECT set_quantifier_opt select_list from_clause_opt where_clause_opt partition_by_clause_opt range_opt every_opt fill_opt twindow_clause_opt group_by_clause_opt having_clause_opt", + /* 423 */ "set_quantifier_opt ::=", + /* 424 */ "set_quantifier_opt ::= DISTINCT", + /* 425 */ "set_quantifier_opt ::= ALL", + /* 426 */ "select_list ::= select_item", + /* 427 */ "select_list ::= select_list NK_COMMA select_item", + /* 428 */ "select_item ::= NK_STAR", + /* 429 */ "select_item ::= common_expression", + /* 430 */ "select_item ::= common_expression column_alias", + /* 431 */ "select_item ::= common_expression AS column_alias", + /* 432 */ "select_item ::= table_name NK_DOT NK_STAR", + /* 433 */ "where_clause_opt ::=", + /* 434 */ "where_clause_opt ::= WHERE search_condition", + /* 435 */ "partition_by_clause_opt ::=", + /* 436 */ "partition_by_clause_opt ::= PARTITION BY expression_list", + /* 437 */ "twindow_clause_opt ::=", + /* 438 */ "twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP", + /* 439 */ "twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP", + /* 440 */ "twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt", + /* 441 */ "twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt", + /* 442 */ "sliding_opt ::=", + /* 443 */ "sliding_opt ::= SLIDING NK_LP duration_literal NK_RP", + /* 444 */ "fill_opt ::=", + /* 445 */ "fill_opt ::= FILL NK_LP fill_mode NK_RP", + /* 446 */ "fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP", + /* 447 */ "fill_mode ::= NONE", + /* 448 */ "fill_mode ::= PREV", + /* 449 */ "fill_mode ::= NULL", + /* 450 */ "fill_mode ::= LINEAR", + /* 451 */ "fill_mode ::= NEXT", + /* 452 */ "group_by_clause_opt ::=", + /* 453 */ "group_by_clause_opt ::= GROUP BY group_by_list", + /* 454 */ "group_by_list ::= expression", + /* 455 */ "group_by_list ::= group_by_list NK_COMMA expression", + /* 456 */ "having_clause_opt ::=", + /* 457 */ "having_clause_opt ::= HAVING search_condition", + /* 458 */ "range_opt ::=", + /* 459 */ "range_opt ::= RANGE NK_LP expression NK_COMMA expression NK_RP", + /* 460 */ "every_opt ::=", + /* 461 */ "every_opt ::= EVERY NK_LP duration_literal NK_RP", + /* 462 */ "query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt", + /* 463 */ "query_expression_body ::= query_primary", + /* 464 */ "query_expression_body ::= query_expression_body UNION ALL query_expression_body", + /* 465 */ "query_expression_body ::= query_expression_body UNION query_expression_body", + /* 466 */ "query_primary ::= query_specification", + /* 467 */ "query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP", + /* 468 */ "order_by_clause_opt ::=", + /* 469 */ "order_by_clause_opt ::= ORDER BY sort_specification_list", + /* 470 */ "slimit_clause_opt ::=", + /* 471 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER", + /* 472 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER", + /* 473 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER", + /* 474 */ "limit_clause_opt ::=", + /* 475 */ "limit_clause_opt ::= LIMIT NK_INTEGER", + /* 476 */ "limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER", + /* 477 */ "limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER", + /* 478 */ "subquery ::= NK_LP query_expression NK_RP", + /* 479 */ "search_condition ::= common_expression", + /* 480 */ "sort_specification_list ::= sort_specification", + /* 481 */ "sort_specification_list ::= sort_specification_list NK_COMMA sort_specification", + /* 482 */ "sort_specification ::= expression ordering_specification_opt null_ordering_opt", + /* 483 */ "ordering_specification_opt ::=", + /* 484 */ "ordering_specification_opt ::= ASC", + /* 485 */ "ordering_specification_opt ::= DESC", + /* 486 */ "null_ordering_opt ::=", + /* 487 */ "null_ordering_opt ::= NULLS FIRST", + /* 488 */ "null_ordering_opt ::= NULLS LAST", }; #endif /* NDEBUG */ @@ -2393,42 +2346,41 @@ static void yy_destructor( case 362: /* stream_options */ case 364: /* query_expression */ case 367: /* explain_options */ - case 371: /* into_opt */ - case 373: /* where_clause_opt */ - case 374: /* signed */ - case 375: /* literal_func */ - case 379: /* expression */ - case 380: /* pseudo_column */ - case 381: /* column_reference */ - case 382: /* function_expression */ - case 383: /* subquery */ - case 388: /* star_func_para */ - case 389: /* predicate */ - case 392: /* in_predicate_value */ - case 393: /* boolean_value_expression */ - case 394: /* boolean_primary */ - case 395: /* common_expression */ - case 396: /* from_clause_opt */ - case 397: /* table_reference_list */ - case 398: /* table_reference */ - case 399: /* table_primary */ - case 400: /* joined_table */ - case 402: /* parenthesized_joined_table */ - case 404: /* search_condition */ - case 405: /* query_specification */ - case 409: /* range_opt */ - case 410: /* every_opt */ - case 411: /* fill_opt */ - case 412: /* twindow_clause_opt */ - case 414: /* having_clause_opt */ - case 415: /* select_item */ - case 418: /* query_expression_body */ - case 420: /* slimit_clause_opt */ - case 421: /* limit_clause_opt */ - case 422: /* query_primary */ - case 424: /* sort_specification */ + case 372: /* where_clause_opt */ + case 373: /* signed */ + case 374: /* literal_func */ + case 378: /* expression */ + case 379: /* pseudo_column */ + case 380: /* column_reference */ + case 381: /* function_expression */ + case 382: /* subquery */ + case 387: /* star_func_para */ + case 388: /* predicate */ + case 391: /* in_predicate_value */ + case 392: /* boolean_value_expression */ + case 393: /* boolean_primary */ + case 394: /* common_expression */ + case 395: /* from_clause_opt */ + case 396: /* table_reference_list */ + case 397: /* table_reference */ + case 398: /* table_primary */ + case 399: /* joined_table */ + case 401: /* parenthesized_joined_table */ + case 403: /* search_condition */ + case 404: /* query_specification */ + case 408: /* range_opt */ + case 409: /* every_opt */ + case 410: /* fill_opt */ + case 411: /* twindow_clause_opt */ + case 413: /* having_clause_opt */ + case 414: /* select_item */ + case 417: /* query_expression_body */ + case 419: /* slimit_clause_opt */ + case 420: /* limit_clause_opt */ + case 421: /* query_primary */ + case 423: /* sort_specification */ { - nodesDestroyNode((yypminor->yy840)); + nodesDestroyNode((yypminor->yy272)); } break; case 306: /* account_options */ @@ -2449,11 +2401,11 @@ static void yy_destructor( case 363: /* topic_name */ case 365: /* cgroup_name */ case 370: /* stream_name */ - case 377: /* table_alias */ - case 378: /* column_alias */ - case 384: /* star_func */ - case 386: /* noarg_func */ - case 401: /* alias_opt */ + case 376: /* table_alias */ + case 377: /* column_alias */ + case 383: /* star_func */ + case 385: /* noarg_func */ + case 400: /* alias_opt */ { } @@ -2474,7 +2426,7 @@ static void yy_destructor( case 320: /* exists_opt */ case 366: /* analyze_opt */ case 368: /* agg_func_opt */ - case 406: /* set_quantifier_opt */ + case 405: /* set_quantifier_opt */ { } @@ -2493,18 +2445,18 @@ static void yy_destructor( case 346: /* duration_list */ case 347: /* rollup_func_list */ case 358: /* func_list */ - case 372: /* dnode_list */ - case 376: /* literal_list */ - case 385: /* star_func_para_list */ - case 387: /* other_para_list */ - case 407: /* select_list */ - case 408: /* partition_by_clause_opt */ - case 413: /* group_by_clause_opt */ - case 417: /* group_by_list */ - case 419: /* order_by_clause_opt */ - case 423: /* sort_specification_list */ + case 371: /* dnode_list */ + case 375: /* literal_list */ + case 384: /* star_func_para_list */ + case 386: /* other_para_list */ + case 406: /* select_list */ + case 407: /* partition_by_clause_opt */ + case 412: /* group_by_clause_opt */ + case 416: /* group_by_list */ + case 418: /* order_by_clause_opt */ + case 422: /* sort_specification_list */ { - nodesDestroyList((yypminor->yy544)); + nodesDestroyList((yypminor->yy172)); } break; case 325: /* alter_db_option */ @@ -2518,28 +2470,28 @@ static void yy_destructor( } break; - case 390: /* compare_op */ - case 391: /* in_op */ + case 389: /* compare_op */ + case 390: /* in_op */ { } break; - case 403: /* join_type */ + case 402: /* join_type */ { } break; - case 416: /* fill_mode */ + case 415: /* fill_mode */ { } break; - case 425: /* ordering_specification_opt */ + case 424: /* ordering_specification_opt */ { } break; - case 426: /* null_ordering_opt */ + case 425: /* null_ordering_opt */ { } @@ -3104,231 +3056,229 @@ static const struct { { 368, -1 }, /* (263) agg_func_opt ::= AGGREGATE */ { 369, 0 }, /* (264) bufsize_opt ::= */ { 369, -2 }, /* (265) bufsize_opt ::= BUFSIZE NK_INTEGER */ - { 305, -8 }, /* (266) cmd ::= CREATE STREAM not_exists_opt stream_name stream_options into_opt AS query_expression */ + { 305, -9 }, /* (266) cmd ::= CREATE STREAM not_exists_opt stream_name stream_options INTO full_table_name AS query_expression */ { 305, -4 }, /* (267) cmd ::= DROP STREAM exists_opt stream_name */ - { 371, 0 }, /* (268) into_opt ::= */ - { 371, -2 }, /* (269) into_opt ::= INTO full_table_name */ - { 362, 0 }, /* (270) stream_options ::= */ - { 362, -3 }, /* (271) stream_options ::= stream_options TRIGGER AT_ONCE */ - { 362, -3 }, /* (272) stream_options ::= stream_options TRIGGER WINDOW_CLOSE */ - { 362, -4 }, /* (273) stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal */ - { 362, -3 }, /* (274) stream_options ::= stream_options WATERMARK duration_literal */ - { 362, -4 }, /* (275) stream_options ::= stream_options IGNORE EXPIRED NK_INTEGER */ - { 305, -3 }, /* (276) cmd ::= KILL CONNECTION NK_INTEGER */ - { 305, -3 }, /* (277) cmd ::= KILL QUERY NK_STRING */ - { 305, -3 }, /* (278) cmd ::= KILL TRANSACTION NK_INTEGER */ - { 305, -2 }, /* (279) cmd ::= BALANCE VGROUP */ - { 305, -4 }, /* (280) cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER */ - { 305, -4 }, /* (281) cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list */ - { 305, -3 }, /* (282) cmd ::= SPLIT VGROUP NK_INTEGER */ - { 372, -2 }, /* (283) dnode_list ::= DNODE NK_INTEGER */ - { 372, -3 }, /* (284) dnode_list ::= dnode_list DNODE NK_INTEGER */ - { 305, -4 }, /* (285) cmd ::= DELETE FROM full_table_name where_clause_opt */ - { 305, -1 }, /* (286) cmd ::= query_expression */ - { 305, -7 }, /* (287) cmd ::= INSERT INTO full_table_name NK_LP col_name_list NK_RP query_expression */ - { 305, -4 }, /* (288) cmd ::= INSERT INTO full_table_name query_expression */ - { 308, -1 }, /* (289) literal ::= NK_INTEGER */ - { 308, -1 }, /* (290) literal ::= NK_FLOAT */ - { 308, -1 }, /* (291) literal ::= NK_STRING */ - { 308, -1 }, /* (292) literal ::= NK_BOOL */ - { 308, -2 }, /* (293) literal ::= TIMESTAMP NK_STRING */ - { 308, -1 }, /* (294) literal ::= duration_literal */ - { 308, -1 }, /* (295) literal ::= NULL */ - { 308, -1 }, /* (296) literal ::= NK_QUESTION */ - { 349, -1 }, /* (297) duration_literal ::= NK_VARIABLE */ - { 374, -1 }, /* (298) signed ::= NK_INTEGER */ - { 374, -2 }, /* (299) signed ::= NK_PLUS NK_INTEGER */ - { 374, -2 }, /* (300) signed ::= NK_MINUS NK_INTEGER */ - { 374, -1 }, /* (301) signed ::= NK_FLOAT */ - { 374, -2 }, /* (302) signed ::= NK_PLUS NK_FLOAT */ - { 374, -2 }, /* (303) signed ::= NK_MINUS NK_FLOAT */ - { 338, -1 }, /* (304) signed_literal ::= signed */ - { 338, -1 }, /* (305) signed_literal ::= NK_STRING */ - { 338, -1 }, /* (306) signed_literal ::= NK_BOOL */ - { 338, -2 }, /* (307) signed_literal ::= TIMESTAMP NK_STRING */ - { 338, -1 }, /* (308) signed_literal ::= duration_literal */ - { 338, -1 }, /* (309) signed_literal ::= NULL */ - { 338, -1 }, /* (310) signed_literal ::= literal_func */ - { 338, -1 }, /* (311) signed_literal ::= NK_QUESTION */ - { 376, -1 }, /* (312) literal_list ::= signed_literal */ - { 376, -3 }, /* (313) literal_list ::= literal_list NK_COMMA signed_literal */ - { 316, -1 }, /* (314) db_name ::= NK_ID */ - { 344, -1 }, /* (315) table_name ::= NK_ID */ - { 336, -1 }, /* (316) column_name ::= NK_ID */ - { 351, -1 }, /* (317) function_name ::= NK_ID */ - { 377, -1 }, /* (318) table_alias ::= NK_ID */ - { 378, -1 }, /* (319) column_alias ::= NK_ID */ - { 310, -1 }, /* (320) user_name ::= NK_ID */ - { 363, -1 }, /* (321) topic_name ::= NK_ID */ - { 370, -1 }, /* (322) stream_name ::= NK_ID */ - { 365, -1 }, /* (323) cgroup_name ::= NK_ID */ - { 379, -1 }, /* (324) expression ::= literal */ - { 379, -1 }, /* (325) expression ::= pseudo_column */ - { 379, -1 }, /* (326) expression ::= column_reference */ - { 379, -1 }, /* (327) expression ::= function_expression */ - { 379, -1 }, /* (328) expression ::= subquery */ - { 379, -3 }, /* (329) expression ::= NK_LP expression NK_RP */ - { 379, -2 }, /* (330) expression ::= NK_PLUS expression */ - { 379, -2 }, /* (331) expression ::= NK_MINUS expression */ - { 379, -3 }, /* (332) expression ::= expression NK_PLUS expression */ - { 379, -3 }, /* (333) expression ::= expression NK_MINUS expression */ - { 379, -3 }, /* (334) expression ::= expression NK_STAR expression */ - { 379, -3 }, /* (335) expression ::= expression NK_SLASH expression */ - { 379, -3 }, /* (336) expression ::= expression NK_REM expression */ - { 379, -3 }, /* (337) expression ::= column_reference NK_ARROW NK_STRING */ - { 379, -3 }, /* (338) expression ::= expression NK_BITAND expression */ - { 379, -3 }, /* (339) expression ::= expression NK_BITOR expression */ - { 341, -1 }, /* (340) expression_list ::= expression */ - { 341, -3 }, /* (341) expression_list ::= expression_list NK_COMMA expression */ - { 381, -1 }, /* (342) column_reference ::= column_name */ - { 381, -3 }, /* (343) column_reference ::= table_name NK_DOT column_name */ - { 380, -1 }, /* (344) pseudo_column ::= ROWTS */ - { 380, -1 }, /* (345) pseudo_column ::= TBNAME */ - { 380, -3 }, /* (346) pseudo_column ::= table_name NK_DOT TBNAME */ - { 380, -1 }, /* (347) pseudo_column ::= QSTART */ - { 380, -1 }, /* (348) pseudo_column ::= QEND */ - { 380, -1 }, /* (349) pseudo_column ::= QDURATION */ - { 380, -1 }, /* (350) pseudo_column ::= WSTART */ - { 380, -1 }, /* (351) pseudo_column ::= WEND */ - { 380, -1 }, /* (352) pseudo_column ::= WDURATION */ - { 382, -4 }, /* (353) function_expression ::= function_name NK_LP expression_list NK_RP */ - { 382, -4 }, /* (354) function_expression ::= star_func NK_LP star_func_para_list NK_RP */ - { 382, -6 }, /* (355) function_expression ::= CAST NK_LP expression AS type_name NK_RP */ - { 382, -1 }, /* (356) function_expression ::= literal_func */ - { 375, -3 }, /* (357) literal_func ::= noarg_func NK_LP NK_RP */ - { 375, -1 }, /* (358) literal_func ::= NOW */ - { 386, -1 }, /* (359) noarg_func ::= NOW */ - { 386, -1 }, /* (360) noarg_func ::= TODAY */ - { 386, -1 }, /* (361) noarg_func ::= TIMEZONE */ - { 386, -1 }, /* (362) noarg_func ::= DATABASE */ - { 386, -1 }, /* (363) noarg_func ::= CLIENT_VERSION */ - { 386, -1 }, /* (364) noarg_func ::= SERVER_VERSION */ - { 386, -1 }, /* (365) noarg_func ::= SERVER_STATUS */ - { 386, -1 }, /* (366) noarg_func ::= CURRENT_USER */ - { 386, -1 }, /* (367) noarg_func ::= USER */ - { 384, -1 }, /* (368) star_func ::= COUNT */ - { 384, -1 }, /* (369) star_func ::= FIRST */ - { 384, -1 }, /* (370) star_func ::= LAST */ - { 384, -1 }, /* (371) star_func ::= LAST_ROW */ - { 385, -1 }, /* (372) star_func_para_list ::= NK_STAR */ - { 385, -1 }, /* (373) star_func_para_list ::= other_para_list */ - { 387, -1 }, /* (374) other_para_list ::= star_func_para */ - { 387, -3 }, /* (375) other_para_list ::= other_para_list NK_COMMA star_func_para */ - { 388, -1 }, /* (376) star_func_para ::= expression */ - { 388, -3 }, /* (377) star_func_para ::= table_name NK_DOT NK_STAR */ - { 389, -3 }, /* (378) predicate ::= expression compare_op expression */ - { 389, -5 }, /* (379) predicate ::= expression BETWEEN expression AND expression */ - { 389, -6 }, /* (380) predicate ::= expression NOT BETWEEN expression AND expression */ - { 389, -3 }, /* (381) predicate ::= expression IS NULL */ - { 389, -4 }, /* (382) predicate ::= expression IS NOT NULL */ - { 389, -3 }, /* (383) predicate ::= expression in_op in_predicate_value */ - { 390, -1 }, /* (384) compare_op ::= NK_LT */ - { 390, -1 }, /* (385) compare_op ::= NK_GT */ - { 390, -1 }, /* (386) compare_op ::= NK_LE */ - { 390, -1 }, /* (387) compare_op ::= NK_GE */ - { 390, -1 }, /* (388) compare_op ::= NK_NE */ - { 390, -1 }, /* (389) compare_op ::= NK_EQ */ - { 390, -1 }, /* (390) compare_op ::= LIKE */ - { 390, -2 }, /* (391) compare_op ::= NOT LIKE */ - { 390, -1 }, /* (392) compare_op ::= MATCH */ - { 390, -1 }, /* (393) compare_op ::= NMATCH */ - { 390, -1 }, /* (394) compare_op ::= CONTAINS */ - { 391, -1 }, /* (395) in_op ::= IN */ - { 391, -2 }, /* (396) in_op ::= NOT IN */ - { 392, -3 }, /* (397) in_predicate_value ::= NK_LP literal_list NK_RP */ - { 393, -1 }, /* (398) boolean_value_expression ::= boolean_primary */ - { 393, -2 }, /* (399) boolean_value_expression ::= NOT boolean_primary */ - { 393, -3 }, /* (400) boolean_value_expression ::= boolean_value_expression OR boolean_value_expression */ - { 393, -3 }, /* (401) boolean_value_expression ::= boolean_value_expression AND boolean_value_expression */ - { 394, -1 }, /* (402) boolean_primary ::= predicate */ - { 394, -3 }, /* (403) boolean_primary ::= NK_LP boolean_value_expression NK_RP */ - { 395, -1 }, /* (404) common_expression ::= expression */ - { 395, -1 }, /* (405) common_expression ::= boolean_value_expression */ - { 396, 0 }, /* (406) from_clause_opt ::= */ - { 396, -2 }, /* (407) from_clause_opt ::= FROM table_reference_list */ - { 397, -1 }, /* (408) table_reference_list ::= table_reference */ - { 397, -3 }, /* (409) table_reference_list ::= table_reference_list NK_COMMA table_reference */ - { 398, -1 }, /* (410) table_reference ::= table_primary */ - { 398, -1 }, /* (411) table_reference ::= joined_table */ - { 399, -2 }, /* (412) table_primary ::= table_name alias_opt */ - { 399, -4 }, /* (413) table_primary ::= db_name NK_DOT table_name alias_opt */ - { 399, -2 }, /* (414) table_primary ::= subquery alias_opt */ - { 399, -1 }, /* (415) table_primary ::= parenthesized_joined_table */ - { 401, 0 }, /* (416) alias_opt ::= */ - { 401, -1 }, /* (417) alias_opt ::= table_alias */ - { 401, -2 }, /* (418) alias_opt ::= AS table_alias */ - { 402, -3 }, /* (419) parenthesized_joined_table ::= NK_LP joined_table NK_RP */ - { 402, -3 }, /* (420) parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP */ - { 400, -6 }, /* (421) joined_table ::= table_reference join_type JOIN table_reference ON search_condition */ - { 403, 0 }, /* (422) join_type ::= */ - { 403, -1 }, /* (423) join_type ::= INNER */ - { 405, -12 }, /* (424) query_specification ::= SELECT set_quantifier_opt select_list from_clause_opt where_clause_opt partition_by_clause_opt range_opt every_opt fill_opt twindow_clause_opt group_by_clause_opt having_clause_opt */ - { 406, 0 }, /* (425) set_quantifier_opt ::= */ - { 406, -1 }, /* (426) set_quantifier_opt ::= DISTINCT */ - { 406, -1 }, /* (427) set_quantifier_opt ::= ALL */ - { 407, -1 }, /* (428) select_list ::= select_item */ - { 407, -3 }, /* (429) select_list ::= select_list NK_COMMA select_item */ - { 415, -1 }, /* (430) select_item ::= NK_STAR */ - { 415, -1 }, /* (431) select_item ::= common_expression */ - { 415, -2 }, /* (432) select_item ::= common_expression column_alias */ - { 415, -3 }, /* (433) select_item ::= common_expression AS column_alias */ - { 415, -3 }, /* (434) select_item ::= table_name NK_DOT NK_STAR */ - { 373, 0 }, /* (435) where_clause_opt ::= */ - { 373, -2 }, /* (436) where_clause_opt ::= WHERE search_condition */ - { 408, 0 }, /* (437) partition_by_clause_opt ::= */ - { 408, -3 }, /* (438) partition_by_clause_opt ::= PARTITION BY expression_list */ - { 412, 0 }, /* (439) twindow_clause_opt ::= */ - { 412, -6 }, /* (440) twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP */ - { 412, -4 }, /* (441) twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP */ - { 412, -6 }, /* (442) twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt */ - { 412, -8 }, /* (443) twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt */ - { 359, 0 }, /* (444) sliding_opt ::= */ - { 359, -4 }, /* (445) sliding_opt ::= SLIDING NK_LP duration_literal NK_RP */ - { 411, 0 }, /* (446) fill_opt ::= */ - { 411, -4 }, /* (447) fill_opt ::= FILL NK_LP fill_mode NK_RP */ - { 411, -6 }, /* (448) fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP */ - { 416, -1 }, /* (449) fill_mode ::= NONE */ - { 416, -1 }, /* (450) fill_mode ::= PREV */ - { 416, -1 }, /* (451) fill_mode ::= NULL */ - { 416, -1 }, /* (452) fill_mode ::= LINEAR */ - { 416, -1 }, /* (453) fill_mode ::= NEXT */ - { 413, 0 }, /* (454) group_by_clause_opt ::= */ - { 413, -3 }, /* (455) group_by_clause_opt ::= GROUP BY group_by_list */ - { 417, -1 }, /* (456) group_by_list ::= expression */ - { 417, -3 }, /* (457) group_by_list ::= group_by_list NK_COMMA expression */ - { 414, 0 }, /* (458) having_clause_opt ::= */ - { 414, -2 }, /* (459) having_clause_opt ::= HAVING search_condition */ - { 409, 0 }, /* (460) range_opt ::= */ - { 409, -6 }, /* (461) range_opt ::= RANGE NK_LP expression NK_COMMA expression NK_RP */ - { 410, 0 }, /* (462) every_opt ::= */ - { 410, -4 }, /* (463) every_opt ::= EVERY NK_LP duration_literal NK_RP */ - { 364, -4 }, /* (464) query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt */ - { 418, -1 }, /* (465) query_expression_body ::= query_primary */ - { 418, -4 }, /* (466) query_expression_body ::= query_expression_body UNION ALL query_expression_body */ - { 418, -3 }, /* (467) query_expression_body ::= query_expression_body UNION query_expression_body */ - { 422, -1 }, /* (468) query_primary ::= query_specification */ - { 422, -6 }, /* (469) query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP */ - { 419, 0 }, /* (470) order_by_clause_opt ::= */ - { 419, -3 }, /* (471) order_by_clause_opt ::= ORDER BY sort_specification_list */ - { 420, 0 }, /* (472) slimit_clause_opt ::= */ - { 420, -2 }, /* (473) slimit_clause_opt ::= SLIMIT NK_INTEGER */ - { 420, -4 }, /* (474) slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER */ - { 420, -4 }, /* (475) slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER */ - { 421, 0 }, /* (476) limit_clause_opt ::= */ - { 421, -2 }, /* (477) limit_clause_opt ::= LIMIT NK_INTEGER */ - { 421, -4 }, /* (478) limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER */ - { 421, -4 }, /* (479) limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER */ - { 383, -3 }, /* (480) subquery ::= NK_LP query_expression NK_RP */ - { 404, -1 }, /* (481) search_condition ::= common_expression */ - { 423, -1 }, /* (482) sort_specification_list ::= sort_specification */ - { 423, -3 }, /* (483) sort_specification_list ::= sort_specification_list NK_COMMA sort_specification */ - { 424, -3 }, /* (484) sort_specification ::= expression ordering_specification_opt null_ordering_opt */ - { 425, 0 }, /* (485) ordering_specification_opt ::= */ - { 425, -1 }, /* (486) ordering_specification_opt ::= ASC */ - { 425, -1 }, /* (487) ordering_specification_opt ::= DESC */ - { 426, 0 }, /* (488) null_ordering_opt ::= */ - { 426, -2 }, /* (489) null_ordering_opt ::= NULLS FIRST */ - { 426, -2 }, /* (490) null_ordering_opt ::= NULLS LAST */ + { 362, 0 }, /* (268) stream_options ::= */ + { 362, -3 }, /* (269) stream_options ::= stream_options TRIGGER AT_ONCE */ + { 362, -3 }, /* (270) stream_options ::= stream_options TRIGGER WINDOW_CLOSE */ + { 362, -4 }, /* (271) stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal */ + { 362, -3 }, /* (272) stream_options ::= stream_options WATERMARK duration_literal */ + { 362, -4 }, /* (273) stream_options ::= stream_options IGNORE EXPIRED NK_INTEGER */ + { 305, -3 }, /* (274) cmd ::= KILL CONNECTION NK_INTEGER */ + { 305, -3 }, /* (275) cmd ::= KILL QUERY NK_STRING */ + { 305, -3 }, /* (276) cmd ::= KILL TRANSACTION NK_INTEGER */ + { 305, -2 }, /* (277) cmd ::= BALANCE VGROUP */ + { 305, -4 }, /* (278) cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER */ + { 305, -4 }, /* (279) cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list */ + { 305, -3 }, /* (280) cmd ::= SPLIT VGROUP NK_INTEGER */ + { 371, -2 }, /* (281) dnode_list ::= DNODE NK_INTEGER */ + { 371, -3 }, /* (282) dnode_list ::= dnode_list DNODE NK_INTEGER */ + { 305, -4 }, /* (283) cmd ::= DELETE FROM full_table_name where_clause_opt */ + { 305, -1 }, /* (284) cmd ::= query_expression */ + { 305, -7 }, /* (285) cmd ::= INSERT INTO full_table_name NK_LP col_name_list NK_RP query_expression */ + { 305, -4 }, /* (286) cmd ::= INSERT INTO full_table_name query_expression */ + { 308, -1 }, /* (287) literal ::= NK_INTEGER */ + { 308, -1 }, /* (288) literal ::= NK_FLOAT */ + { 308, -1 }, /* (289) literal ::= NK_STRING */ + { 308, -1 }, /* (290) literal ::= NK_BOOL */ + { 308, -2 }, /* (291) literal ::= TIMESTAMP NK_STRING */ + { 308, -1 }, /* (292) literal ::= duration_literal */ + { 308, -1 }, /* (293) literal ::= NULL */ + { 308, -1 }, /* (294) literal ::= NK_QUESTION */ + { 349, -1 }, /* (295) duration_literal ::= NK_VARIABLE */ + { 373, -1 }, /* (296) signed ::= NK_INTEGER */ + { 373, -2 }, /* (297) signed ::= NK_PLUS NK_INTEGER */ + { 373, -2 }, /* (298) signed ::= NK_MINUS NK_INTEGER */ + { 373, -1 }, /* (299) signed ::= NK_FLOAT */ + { 373, -2 }, /* (300) signed ::= NK_PLUS NK_FLOAT */ + { 373, -2 }, /* (301) signed ::= NK_MINUS NK_FLOAT */ + { 338, -1 }, /* (302) signed_literal ::= signed */ + { 338, -1 }, /* (303) signed_literal ::= NK_STRING */ + { 338, -1 }, /* (304) signed_literal ::= NK_BOOL */ + { 338, -2 }, /* (305) signed_literal ::= TIMESTAMP NK_STRING */ + { 338, -1 }, /* (306) signed_literal ::= duration_literal */ + { 338, -1 }, /* (307) signed_literal ::= NULL */ + { 338, -1 }, /* (308) signed_literal ::= literal_func */ + { 338, -1 }, /* (309) signed_literal ::= NK_QUESTION */ + { 375, -1 }, /* (310) literal_list ::= signed_literal */ + { 375, -3 }, /* (311) literal_list ::= literal_list NK_COMMA signed_literal */ + { 316, -1 }, /* (312) db_name ::= NK_ID */ + { 344, -1 }, /* (313) table_name ::= NK_ID */ + { 336, -1 }, /* (314) column_name ::= NK_ID */ + { 351, -1 }, /* (315) function_name ::= NK_ID */ + { 376, -1 }, /* (316) table_alias ::= NK_ID */ + { 377, -1 }, /* (317) column_alias ::= NK_ID */ + { 310, -1 }, /* (318) user_name ::= NK_ID */ + { 363, -1 }, /* (319) topic_name ::= NK_ID */ + { 370, -1 }, /* (320) stream_name ::= NK_ID */ + { 365, -1 }, /* (321) cgroup_name ::= NK_ID */ + { 378, -1 }, /* (322) expression ::= literal */ + { 378, -1 }, /* (323) expression ::= pseudo_column */ + { 378, -1 }, /* (324) expression ::= column_reference */ + { 378, -1 }, /* (325) expression ::= function_expression */ + { 378, -1 }, /* (326) expression ::= subquery */ + { 378, -3 }, /* (327) expression ::= NK_LP expression NK_RP */ + { 378, -2 }, /* (328) expression ::= NK_PLUS expression */ + { 378, -2 }, /* (329) expression ::= NK_MINUS expression */ + { 378, -3 }, /* (330) expression ::= expression NK_PLUS expression */ + { 378, -3 }, /* (331) expression ::= expression NK_MINUS expression */ + { 378, -3 }, /* (332) expression ::= expression NK_STAR expression */ + { 378, -3 }, /* (333) expression ::= expression NK_SLASH expression */ + { 378, -3 }, /* (334) expression ::= expression NK_REM expression */ + { 378, -3 }, /* (335) expression ::= column_reference NK_ARROW NK_STRING */ + { 378, -3 }, /* (336) expression ::= expression NK_BITAND expression */ + { 378, -3 }, /* (337) expression ::= expression NK_BITOR expression */ + { 341, -1 }, /* (338) expression_list ::= expression */ + { 341, -3 }, /* (339) expression_list ::= expression_list NK_COMMA expression */ + { 380, -1 }, /* (340) column_reference ::= column_name */ + { 380, -3 }, /* (341) column_reference ::= table_name NK_DOT column_name */ + { 379, -1 }, /* (342) pseudo_column ::= ROWTS */ + { 379, -1 }, /* (343) pseudo_column ::= TBNAME */ + { 379, -3 }, /* (344) pseudo_column ::= table_name NK_DOT TBNAME */ + { 379, -1 }, /* (345) pseudo_column ::= QSTART */ + { 379, -1 }, /* (346) pseudo_column ::= QEND */ + { 379, -1 }, /* (347) pseudo_column ::= QDURATION */ + { 379, -1 }, /* (348) pseudo_column ::= WSTART */ + { 379, -1 }, /* (349) pseudo_column ::= WEND */ + { 379, -1 }, /* (350) pseudo_column ::= WDURATION */ + { 381, -4 }, /* (351) function_expression ::= function_name NK_LP expression_list NK_RP */ + { 381, -4 }, /* (352) function_expression ::= star_func NK_LP star_func_para_list NK_RP */ + { 381, -6 }, /* (353) function_expression ::= CAST NK_LP expression AS type_name NK_RP */ + { 381, -1 }, /* (354) function_expression ::= literal_func */ + { 374, -3 }, /* (355) literal_func ::= noarg_func NK_LP NK_RP */ + { 374, -1 }, /* (356) literal_func ::= NOW */ + { 385, -1 }, /* (357) noarg_func ::= NOW */ + { 385, -1 }, /* (358) noarg_func ::= TODAY */ + { 385, -1 }, /* (359) noarg_func ::= TIMEZONE */ + { 385, -1 }, /* (360) noarg_func ::= DATABASE */ + { 385, -1 }, /* (361) noarg_func ::= CLIENT_VERSION */ + { 385, -1 }, /* (362) noarg_func ::= SERVER_VERSION */ + { 385, -1 }, /* (363) noarg_func ::= SERVER_STATUS */ + { 385, -1 }, /* (364) noarg_func ::= CURRENT_USER */ + { 385, -1 }, /* (365) noarg_func ::= USER */ + { 383, -1 }, /* (366) star_func ::= COUNT */ + { 383, -1 }, /* (367) star_func ::= FIRST */ + { 383, -1 }, /* (368) star_func ::= LAST */ + { 383, -1 }, /* (369) star_func ::= LAST_ROW */ + { 384, -1 }, /* (370) star_func_para_list ::= NK_STAR */ + { 384, -1 }, /* (371) star_func_para_list ::= other_para_list */ + { 386, -1 }, /* (372) other_para_list ::= star_func_para */ + { 386, -3 }, /* (373) other_para_list ::= other_para_list NK_COMMA star_func_para */ + { 387, -1 }, /* (374) star_func_para ::= expression */ + { 387, -3 }, /* (375) star_func_para ::= table_name NK_DOT NK_STAR */ + { 388, -3 }, /* (376) predicate ::= expression compare_op expression */ + { 388, -5 }, /* (377) predicate ::= expression BETWEEN expression AND expression */ + { 388, -6 }, /* (378) predicate ::= expression NOT BETWEEN expression AND expression */ + { 388, -3 }, /* (379) predicate ::= expression IS NULL */ + { 388, -4 }, /* (380) predicate ::= expression IS NOT NULL */ + { 388, -3 }, /* (381) predicate ::= expression in_op in_predicate_value */ + { 389, -1 }, /* (382) compare_op ::= NK_LT */ + { 389, -1 }, /* (383) compare_op ::= NK_GT */ + { 389, -1 }, /* (384) compare_op ::= NK_LE */ + { 389, -1 }, /* (385) compare_op ::= NK_GE */ + { 389, -1 }, /* (386) compare_op ::= NK_NE */ + { 389, -1 }, /* (387) compare_op ::= NK_EQ */ + { 389, -1 }, /* (388) compare_op ::= LIKE */ + { 389, -2 }, /* (389) compare_op ::= NOT LIKE */ + { 389, -1 }, /* (390) compare_op ::= MATCH */ + { 389, -1 }, /* (391) compare_op ::= NMATCH */ + { 389, -1 }, /* (392) compare_op ::= CONTAINS */ + { 390, -1 }, /* (393) in_op ::= IN */ + { 390, -2 }, /* (394) in_op ::= NOT IN */ + { 391, -3 }, /* (395) in_predicate_value ::= NK_LP literal_list NK_RP */ + { 392, -1 }, /* (396) boolean_value_expression ::= boolean_primary */ + { 392, -2 }, /* (397) boolean_value_expression ::= NOT boolean_primary */ + { 392, -3 }, /* (398) boolean_value_expression ::= boolean_value_expression OR boolean_value_expression */ + { 392, -3 }, /* (399) boolean_value_expression ::= boolean_value_expression AND boolean_value_expression */ + { 393, -1 }, /* (400) boolean_primary ::= predicate */ + { 393, -3 }, /* (401) boolean_primary ::= NK_LP boolean_value_expression NK_RP */ + { 394, -1 }, /* (402) common_expression ::= expression */ + { 394, -1 }, /* (403) common_expression ::= boolean_value_expression */ + { 395, 0 }, /* (404) from_clause_opt ::= */ + { 395, -2 }, /* (405) from_clause_opt ::= FROM table_reference_list */ + { 396, -1 }, /* (406) table_reference_list ::= table_reference */ + { 396, -3 }, /* (407) table_reference_list ::= table_reference_list NK_COMMA table_reference */ + { 397, -1 }, /* (408) table_reference ::= table_primary */ + { 397, -1 }, /* (409) table_reference ::= joined_table */ + { 398, -2 }, /* (410) table_primary ::= table_name alias_opt */ + { 398, -4 }, /* (411) table_primary ::= db_name NK_DOT table_name alias_opt */ + { 398, -2 }, /* (412) table_primary ::= subquery alias_opt */ + { 398, -1 }, /* (413) table_primary ::= parenthesized_joined_table */ + { 400, 0 }, /* (414) alias_opt ::= */ + { 400, -1 }, /* (415) alias_opt ::= table_alias */ + { 400, -2 }, /* (416) alias_opt ::= AS table_alias */ + { 401, -3 }, /* (417) parenthesized_joined_table ::= NK_LP joined_table NK_RP */ + { 401, -3 }, /* (418) parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP */ + { 399, -6 }, /* (419) joined_table ::= table_reference join_type JOIN table_reference ON search_condition */ + { 402, 0 }, /* (420) join_type ::= */ + { 402, -1 }, /* (421) join_type ::= INNER */ + { 404, -12 }, /* (422) query_specification ::= SELECT set_quantifier_opt select_list from_clause_opt where_clause_opt partition_by_clause_opt range_opt every_opt fill_opt twindow_clause_opt group_by_clause_opt having_clause_opt */ + { 405, 0 }, /* (423) set_quantifier_opt ::= */ + { 405, -1 }, /* (424) set_quantifier_opt ::= DISTINCT */ + { 405, -1 }, /* (425) set_quantifier_opt ::= ALL */ + { 406, -1 }, /* (426) select_list ::= select_item */ + { 406, -3 }, /* (427) select_list ::= select_list NK_COMMA select_item */ + { 414, -1 }, /* (428) select_item ::= NK_STAR */ + { 414, -1 }, /* (429) select_item ::= common_expression */ + { 414, -2 }, /* (430) select_item ::= common_expression column_alias */ + { 414, -3 }, /* (431) select_item ::= common_expression AS column_alias */ + { 414, -3 }, /* (432) select_item ::= table_name NK_DOT NK_STAR */ + { 372, 0 }, /* (433) where_clause_opt ::= */ + { 372, -2 }, /* (434) where_clause_opt ::= WHERE search_condition */ + { 407, 0 }, /* (435) partition_by_clause_opt ::= */ + { 407, -3 }, /* (436) partition_by_clause_opt ::= PARTITION BY expression_list */ + { 411, 0 }, /* (437) twindow_clause_opt ::= */ + { 411, -6 }, /* (438) twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP */ + { 411, -4 }, /* (439) twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP */ + { 411, -6 }, /* (440) twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt */ + { 411, -8 }, /* (441) twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt */ + { 359, 0 }, /* (442) sliding_opt ::= */ + { 359, -4 }, /* (443) sliding_opt ::= SLIDING NK_LP duration_literal NK_RP */ + { 410, 0 }, /* (444) fill_opt ::= */ + { 410, -4 }, /* (445) fill_opt ::= FILL NK_LP fill_mode NK_RP */ + { 410, -6 }, /* (446) fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP */ + { 415, -1 }, /* (447) fill_mode ::= NONE */ + { 415, -1 }, /* (448) fill_mode ::= PREV */ + { 415, -1 }, /* (449) fill_mode ::= NULL */ + { 415, -1 }, /* (450) fill_mode ::= LINEAR */ + { 415, -1 }, /* (451) fill_mode ::= NEXT */ + { 412, 0 }, /* (452) group_by_clause_opt ::= */ + { 412, -3 }, /* (453) group_by_clause_opt ::= GROUP BY group_by_list */ + { 416, -1 }, /* (454) group_by_list ::= expression */ + { 416, -3 }, /* (455) group_by_list ::= group_by_list NK_COMMA expression */ + { 413, 0 }, /* (456) having_clause_opt ::= */ + { 413, -2 }, /* (457) having_clause_opt ::= HAVING search_condition */ + { 408, 0 }, /* (458) range_opt ::= */ + { 408, -6 }, /* (459) range_opt ::= RANGE NK_LP expression NK_COMMA expression NK_RP */ + { 409, 0 }, /* (460) every_opt ::= */ + { 409, -4 }, /* (461) every_opt ::= EVERY NK_LP duration_literal NK_RP */ + { 364, -4 }, /* (462) query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt */ + { 417, -1 }, /* (463) query_expression_body ::= query_primary */ + { 417, -4 }, /* (464) query_expression_body ::= query_expression_body UNION ALL query_expression_body */ + { 417, -3 }, /* (465) query_expression_body ::= query_expression_body UNION query_expression_body */ + { 421, -1 }, /* (466) query_primary ::= query_specification */ + { 421, -6 }, /* (467) query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP */ + { 418, 0 }, /* (468) order_by_clause_opt ::= */ + { 418, -3 }, /* (469) order_by_clause_opt ::= ORDER BY sort_specification_list */ + { 419, 0 }, /* (470) slimit_clause_opt ::= */ + { 419, -2 }, /* (471) slimit_clause_opt ::= SLIMIT NK_INTEGER */ + { 419, -4 }, /* (472) slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER */ + { 419, -4 }, /* (473) slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER */ + { 420, 0 }, /* (474) limit_clause_opt ::= */ + { 420, -2 }, /* (475) limit_clause_opt ::= LIMIT NK_INTEGER */ + { 420, -4 }, /* (476) limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER */ + { 420, -4 }, /* (477) limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER */ + { 382, -3 }, /* (478) subquery ::= NK_LP query_expression NK_RP */ + { 403, -1 }, /* (479) search_condition ::= common_expression */ + { 422, -1 }, /* (480) sort_specification_list ::= sort_specification */ + { 422, -3 }, /* (481) sort_specification_list ::= sort_specification_list NK_COMMA sort_specification */ + { 423, -3 }, /* (482) sort_specification ::= expression ordering_specification_opt null_ordering_opt */ + { 424, 0 }, /* (483) ordering_specification_opt ::= */ + { 424, -1 }, /* (484) ordering_specification_opt ::= ASC */ + { 424, -1 }, /* (485) ordering_specification_opt ::= DESC */ + { 425, 0 }, /* (486) null_ordering_opt ::= */ + { 425, -2 }, /* (487) null_ordering_opt ::= NULLS FIRST */ + { 425, -2 }, /* (488) null_ordering_opt ::= NULLS LAST */ }; static void yy_accept(yyParser*); /* Forward Declaration */ @@ -3465,69 +3415,69 @@ static YYACTIONTYPE yy_reduce( yy_destructor(yypParser,308,&yymsp[0].minor); break; case 24: /* cmd ::= CREATE USER user_name PASS NK_STRING sysinfo_opt */ -{ pCxt->pRootNode = createCreateUserStmt(pCxt, &yymsp[-3].minor.yy617, &yymsp[-1].minor.yy0, yymsp[0].minor.yy215); } +{ pCxt->pRootNode = createCreateUserStmt(pCxt, &yymsp[-3].minor.yy209, &yymsp[-1].minor.yy0, yymsp[0].minor.yy59); } break; case 25: /* cmd ::= ALTER USER user_name PASS NK_STRING */ -{ pCxt->pRootNode = createAlterUserStmt(pCxt, &yymsp[-2].minor.yy617, TSDB_ALTER_USER_PASSWD, &yymsp[0].minor.yy0); } +{ pCxt->pRootNode = createAlterUserStmt(pCxt, &yymsp[-2].minor.yy209, TSDB_ALTER_USER_PASSWD, &yymsp[0].minor.yy0); } break; case 26: /* cmd ::= ALTER USER user_name ENABLE NK_INTEGER */ -{ pCxt->pRootNode = createAlterUserStmt(pCxt, &yymsp[-2].minor.yy617, TSDB_ALTER_USER_ENABLE, &yymsp[0].minor.yy0); } +{ pCxt->pRootNode = createAlterUserStmt(pCxt, &yymsp[-2].minor.yy209, TSDB_ALTER_USER_ENABLE, &yymsp[0].minor.yy0); } break; case 27: /* cmd ::= ALTER USER user_name SYSINFO NK_INTEGER */ -{ pCxt->pRootNode = createAlterUserStmt(pCxt, &yymsp[-2].minor.yy617, TSDB_ALTER_USER_SYSINFO, &yymsp[0].minor.yy0); } +{ pCxt->pRootNode = createAlterUserStmt(pCxt, &yymsp[-2].minor.yy209, TSDB_ALTER_USER_SYSINFO, &yymsp[0].minor.yy0); } break; case 28: /* cmd ::= DROP USER user_name */ -{ pCxt->pRootNode = createDropUserStmt(pCxt, &yymsp[0].minor.yy617); } +{ pCxt->pRootNode = createDropUserStmt(pCxt, &yymsp[0].minor.yy209); } break; case 29: /* sysinfo_opt ::= */ -{ yymsp[1].minor.yy215 = 1; } +{ yymsp[1].minor.yy59 = 1; } break; case 30: /* sysinfo_opt ::= SYSINFO NK_INTEGER */ -{ yymsp[-1].minor.yy215 = taosStr2Int8(yymsp[0].minor.yy0.z, NULL, 10); } +{ yymsp[-1].minor.yy59 = taosStr2Int8(yymsp[0].minor.yy0.z, NULL, 10); } break; case 31: /* cmd ::= GRANT privileges ON priv_level TO user_name */ -{ pCxt->pRootNode = createGrantStmt(pCxt, yymsp[-4].minor.yy473, &yymsp[-2].minor.yy617, &yymsp[0].minor.yy617); } +{ pCxt->pRootNode = createGrantStmt(pCxt, yymsp[-4].minor.yy69, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209); } break; case 32: /* cmd ::= REVOKE privileges ON priv_level FROM user_name */ -{ pCxt->pRootNode = createRevokeStmt(pCxt, yymsp[-4].minor.yy473, &yymsp[-2].minor.yy617, &yymsp[0].minor.yy617); } +{ pCxt->pRootNode = createRevokeStmt(pCxt, yymsp[-4].minor.yy69, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209); } break; case 33: /* privileges ::= ALL */ -{ yymsp[0].minor.yy473 = PRIVILEGE_TYPE_ALL; } +{ yymsp[0].minor.yy69 = PRIVILEGE_TYPE_ALL; } break; case 34: /* privileges ::= priv_type_list */ case 35: /* priv_type_list ::= priv_type */ yytestcase(yyruleno==35); -{ yylhsminor.yy473 = yymsp[0].minor.yy473; } - yymsp[0].minor.yy473 = yylhsminor.yy473; +{ yylhsminor.yy69 = yymsp[0].minor.yy69; } + yymsp[0].minor.yy69 = yylhsminor.yy69; break; case 36: /* priv_type_list ::= priv_type_list NK_COMMA priv_type */ -{ yylhsminor.yy473 = yymsp[-2].minor.yy473 | yymsp[0].minor.yy473; } - yymsp[-2].minor.yy473 = yylhsminor.yy473; +{ yylhsminor.yy69 = yymsp[-2].minor.yy69 | yymsp[0].minor.yy69; } + yymsp[-2].minor.yy69 = yylhsminor.yy69; break; case 37: /* priv_type ::= READ */ -{ yymsp[0].minor.yy473 = PRIVILEGE_TYPE_READ; } +{ yymsp[0].minor.yy69 = PRIVILEGE_TYPE_READ; } break; case 38: /* priv_type ::= WRITE */ -{ yymsp[0].minor.yy473 = PRIVILEGE_TYPE_WRITE; } +{ yymsp[0].minor.yy69 = PRIVILEGE_TYPE_WRITE; } break; case 39: /* priv_level ::= NK_STAR NK_DOT NK_STAR */ -{ yylhsminor.yy617 = yymsp[-2].minor.yy0; } - yymsp[-2].minor.yy617 = yylhsminor.yy617; +{ yylhsminor.yy209 = yymsp[-2].minor.yy0; } + yymsp[-2].minor.yy209 = yylhsminor.yy209; break; case 40: /* priv_level ::= db_name NK_DOT NK_STAR */ -{ yylhsminor.yy617 = yymsp[-2].minor.yy617; } - yymsp[-2].minor.yy617 = yylhsminor.yy617; +{ yylhsminor.yy209 = yymsp[-2].minor.yy209; } + yymsp[-2].minor.yy209 = yylhsminor.yy209; break; case 41: /* cmd ::= CREATE DNODE dnode_endpoint */ -{ pCxt->pRootNode = createCreateDnodeStmt(pCxt, &yymsp[0].minor.yy617, NULL); } +{ pCxt->pRootNode = createCreateDnodeStmt(pCxt, &yymsp[0].minor.yy209, NULL); } break; case 42: /* cmd ::= CREATE DNODE dnode_endpoint PORT NK_INTEGER */ -{ pCxt->pRootNode = createCreateDnodeStmt(pCxt, &yymsp[-2].minor.yy617, &yymsp[0].minor.yy0); } +{ pCxt->pRootNode = createCreateDnodeStmt(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy0); } break; case 43: /* cmd ::= DROP DNODE NK_INTEGER */ { pCxt->pRootNode = createDropDnodeStmt(pCxt, &yymsp[0].minor.yy0); } break; case 44: /* cmd ::= DROP DNODE dnode_endpoint */ -{ pCxt->pRootNode = createDropDnodeStmt(pCxt, &yymsp[0].minor.yy617); } +{ pCxt->pRootNode = createDropDnodeStmt(pCxt, &yymsp[0].minor.yy209); } break; case 45: /* cmd ::= ALTER DNODE NK_INTEGER NK_STRING */ { pCxt->pRootNode = createAlterDnodeStmt(pCxt, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy0, NULL); } @@ -3544,31 +3494,31 @@ static YYACTIONTYPE yy_reduce( case 49: /* dnode_endpoint ::= NK_STRING */ case 50: /* dnode_endpoint ::= NK_ID */ yytestcase(yyruleno==50); case 51: /* dnode_endpoint ::= NK_IPTOKEN */ yytestcase(yyruleno==51); - case 314: /* db_name ::= NK_ID */ yytestcase(yyruleno==314); - case 315: /* table_name ::= NK_ID */ yytestcase(yyruleno==315); - case 316: /* column_name ::= NK_ID */ yytestcase(yyruleno==316); - case 317: /* function_name ::= NK_ID */ yytestcase(yyruleno==317); - case 318: /* table_alias ::= NK_ID */ yytestcase(yyruleno==318); - case 319: /* column_alias ::= NK_ID */ yytestcase(yyruleno==319); - case 320: /* user_name ::= NK_ID */ yytestcase(yyruleno==320); - case 321: /* topic_name ::= NK_ID */ yytestcase(yyruleno==321); - case 322: /* stream_name ::= NK_ID */ yytestcase(yyruleno==322); - case 323: /* cgroup_name ::= NK_ID */ yytestcase(yyruleno==323); - case 359: /* noarg_func ::= NOW */ yytestcase(yyruleno==359); - case 360: /* noarg_func ::= TODAY */ yytestcase(yyruleno==360); - case 361: /* noarg_func ::= TIMEZONE */ yytestcase(yyruleno==361); - case 362: /* noarg_func ::= DATABASE */ yytestcase(yyruleno==362); - case 363: /* noarg_func ::= CLIENT_VERSION */ yytestcase(yyruleno==363); - case 364: /* noarg_func ::= SERVER_VERSION */ yytestcase(yyruleno==364); - case 365: /* noarg_func ::= SERVER_STATUS */ yytestcase(yyruleno==365); - case 366: /* noarg_func ::= CURRENT_USER */ yytestcase(yyruleno==366); - case 367: /* noarg_func ::= USER */ yytestcase(yyruleno==367); - case 368: /* star_func ::= COUNT */ yytestcase(yyruleno==368); - case 369: /* star_func ::= FIRST */ yytestcase(yyruleno==369); - case 370: /* star_func ::= LAST */ yytestcase(yyruleno==370); - case 371: /* star_func ::= LAST_ROW */ yytestcase(yyruleno==371); -{ yylhsminor.yy617 = yymsp[0].minor.yy0; } - yymsp[0].minor.yy617 = yylhsminor.yy617; + case 312: /* db_name ::= NK_ID */ yytestcase(yyruleno==312); + case 313: /* table_name ::= NK_ID */ yytestcase(yyruleno==313); + case 314: /* column_name ::= NK_ID */ yytestcase(yyruleno==314); + case 315: /* function_name ::= NK_ID */ yytestcase(yyruleno==315); + case 316: /* table_alias ::= NK_ID */ yytestcase(yyruleno==316); + case 317: /* column_alias ::= NK_ID */ yytestcase(yyruleno==317); + case 318: /* user_name ::= NK_ID */ yytestcase(yyruleno==318); + case 319: /* topic_name ::= NK_ID */ yytestcase(yyruleno==319); + case 320: /* stream_name ::= NK_ID */ yytestcase(yyruleno==320); + case 321: /* cgroup_name ::= NK_ID */ yytestcase(yyruleno==321); + case 357: /* noarg_func ::= NOW */ yytestcase(yyruleno==357); + case 358: /* noarg_func ::= TODAY */ yytestcase(yyruleno==358); + case 359: /* noarg_func ::= TIMEZONE */ yytestcase(yyruleno==359); + case 360: /* noarg_func ::= DATABASE */ yytestcase(yyruleno==360); + case 361: /* noarg_func ::= CLIENT_VERSION */ yytestcase(yyruleno==361); + case 362: /* noarg_func ::= SERVER_VERSION */ yytestcase(yyruleno==362); + case 363: /* noarg_func ::= SERVER_STATUS */ yytestcase(yyruleno==363); + case 364: /* noarg_func ::= CURRENT_USER */ yytestcase(yyruleno==364); + case 365: /* noarg_func ::= USER */ yytestcase(yyruleno==365); + case 366: /* star_func ::= COUNT */ yytestcase(yyruleno==366); + case 367: /* star_func ::= FIRST */ yytestcase(yyruleno==367); + case 368: /* star_func ::= LAST */ yytestcase(yyruleno==368); + case 369: /* star_func ::= LAST_ROW */ yytestcase(yyruleno==369); +{ yylhsminor.yy209 = yymsp[0].minor.yy0; } + yymsp[0].minor.yy209 = yylhsminor.yy209; break; case 52: /* cmd ::= ALTER LOCAL NK_STRING */ { pCxt->pRootNode = createAlterLocalStmt(pCxt, &yymsp[0].minor.yy0, NULL); } @@ -3601,189 +3551,189 @@ static YYACTIONTYPE yy_reduce( { pCxt->pRootNode = createDropComponentNodeStmt(pCxt, QUERY_NODE_DROP_MNODE_STMT, &yymsp[0].minor.yy0); } break; case 62: /* cmd ::= CREATE DATABASE not_exists_opt db_name db_options */ -{ pCxt->pRootNode = createCreateDatabaseStmt(pCxt, yymsp[-2].minor.yy313, &yymsp[-1].minor.yy617, yymsp[0].minor.yy840); } +{ pCxt->pRootNode = createCreateDatabaseStmt(pCxt, yymsp[-2].minor.yy293, &yymsp[-1].minor.yy209, yymsp[0].minor.yy272); } break; case 63: /* cmd ::= DROP DATABASE exists_opt db_name */ -{ pCxt->pRootNode = createDropDatabaseStmt(pCxt, yymsp[-1].minor.yy313, &yymsp[0].minor.yy617); } +{ pCxt->pRootNode = createDropDatabaseStmt(pCxt, yymsp[-1].minor.yy293, &yymsp[0].minor.yy209); } break; case 64: /* cmd ::= USE db_name */ -{ pCxt->pRootNode = createUseDatabaseStmt(pCxt, &yymsp[0].minor.yy617); } +{ pCxt->pRootNode = createUseDatabaseStmt(pCxt, &yymsp[0].minor.yy209); } break; case 65: /* cmd ::= ALTER DATABASE db_name alter_db_options */ -{ pCxt->pRootNode = createAlterDatabaseStmt(pCxt, &yymsp[-1].minor.yy617, yymsp[0].minor.yy840); } +{ pCxt->pRootNode = createAlterDatabaseStmt(pCxt, &yymsp[-1].minor.yy209, yymsp[0].minor.yy272); } break; case 66: /* cmd ::= FLUSH DATABASE db_name */ -{ pCxt->pRootNode = createFlushDatabaseStmt(pCxt, &yymsp[0].minor.yy617); } +{ pCxt->pRootNode = createFlushDatabaseStmt(pCxt, &yymsp[0].minor.yy209); } break; case 67: /* cmd ::= TRIM DATABASE db_name */ -{ pCxt->pRootNode = createTrimDatabaseStmt(pCxt, &yymsp[0].minor.yy617); } +{ pCxt->pRootNode = createTrimDatabaseStmt(pCxt, &yymsp[0].minor.yy209); } break; case 68: /* not_exists_opt ::= IF NOT EXISTS */ -{ yymsp[-2].minor.yy313 = true; } +{ yymsp[-2].minor.yy293 = true; } break; case 69: /* not_exists_opt ::= */ case 71: /* exists_opt ::= */ yytestcase(yyruleno==71); case 255: /* analyze_opt ::= */ yytestcase(yyruleno==255); case 262: /* agg_func_opt ::= */ yytestcase(yyruleno==262); - case 425: /* set_quantifier_opt ::= */ yytestcase(yyruleno==425); -{ yymsp[1].minor.yy313 = false; } + case 423: /* set_quantifier_opt ::= */ yytestcase(yyruleno==423); +{ yymsp[1].minor.yy293 = false; } break; case 70: /* exists_opt ::= IF EXISTS */ -{ yymsp[-1].minor.yy313 = true; } +{ yymsp[-1].minor.yy293 = true; } break; case 72: /* db_options ::= */ -{ yymsp[1].minor.yy840 = createDefaultDatabaseOptions(pCxt); } +{ yymsp[1].minor.yy272 = createDefaultDatabaseOptions(pCxt); } break; case 73: /* db_options ::= db_options BUFFER NK_INTEGER */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_BUFFER, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_BUFFER, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 74: /* db_options ::= db_options CACHEMODEL NK_STRING */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_CACHEMODEL, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_CACHEMODEL, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 75: /* db_options ::= db_options CACHESIZE NK_INTEGER */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_CACHESIZE, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_CACHESIZE, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 76: /* db_options ::= db_options COMP NK_INTEGER */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_COMP, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_COMP, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 77: /* db_options ::= db_options DURATION NK_INTEGER */ case 78: /* db_options ::= db_options DURATION NK_VARIABLE */ yytestcase(yyruleno==78); -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_DAYS, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_DAYS, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 79: /* db_options ::= db_options MAXROWS NK_INTEGER */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_MAXROWS, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_MAXROWS, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 80: /* db_options ::= db_options MINROWS NK_INTEGER */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_MINROWS, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_MINROWS, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 81: /* db_options ::= db_options KEEP integer_list */ case 82: /* db_options ::= db_options KEEP variable_list */ yytestcase(yyruleno==82); -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_KEEP, yymsp[0].minor.yy544); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_KEEP, yymsp[0].minor.yy172); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 83: /* db_options ::= db_options PAGES NK_INTEGER */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_PAGES, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_PAGES, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 84: /* db_options ::= db_options PAGESIZE NK_INTEGER */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_PAGESIZE, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_PAGESIZE, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 85: /* db_options ::= db_options PRECISION NK_STRING */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_PRECISION, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_PRECISION, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 86: /* db_options ::= db_options REPLICA NK_INTEGER */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_REPLICA, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_REPLICA, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 87: /* db_options ::= db_options STRICT NK_STRING */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_STRICT, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_STRICT, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 88: /* db_options ::= db_options VGROUPS NK_INTEGER */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_VGROUPS, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_VGROUPS, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 89: /* db_options ::= db_options SINGLE_STABLE NK_INTEGER */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_SINGLE_STABLE, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_SINGLE_STABLE, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 90: /* db_options ::= db_options RETENTIONS retention_list */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_RETENTIONS, yymsp[0].minor.yy544); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_RETENTIONS, yymsp[0].minor.yy172); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 91: /* db_options ::= db_options SCHEMALESS NK_INTEGER */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_SCHEMALESS, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_SCHEMALESS, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 92: /* db_options ::= db_options WAL_LEVEL NK_INTEGER */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_WAL, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_WAL, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 93: /* db_options ::= db_options WAL_FSYNC_PERIOD NK_INTEGER */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_FSYNC, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_FSYNC, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 94: /* db_options ::= db_options WAL_RETENTION_PERIOD NK_INTEGER */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_WAL_RETENTION_PERIOD, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_WAL_RETENTION_PERIOD, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 95: /* db_options ::= db_options WAL_RETENTION_PERIOD NK_MINUS NK_INTEGER */ { SToken t = yymsp[-1].minor.yy0; t.n = (yymsp[0].minor.yy0.z + yymsp[0].minor.yy0.n) - yymsp[-1].minor.yy0.z; - yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-3].minor.yy840, DB_OPTION_WAL_RETENTION_PERIOD, &t); + yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-3].minor.yy272, DB_OPTION_WAL_RETENTION_PERIOD, &t); } - yymsp[-3].minor.yy840 = yylhsminor.yy840; + yymsp[-3].minor.yy272 = yylhsminor.yy272; break; case 96: /* db_options ::= db_options WAL_RETENTION_SIZE NK_INTEGER */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_WAL_RETENTION_SIZE, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_WAL_RETENTION_SIZE, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 97: /* db_options ::= db_options WAL_RETENTION_SIZE NK_MINUS NK_INTEGER */ { SToken t = yymsp[-1].minor.yy0; t.n = (yymsp[0].minor.yy0.z + yymsp[0].minor.yy0.n) - yymsp[-1].minor.yy0.z; - yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-3].minor.yy840, DB_OPTION_WAL_RETENTION_SIZE, &t); + yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-3].minor.yy272, DB_OPTION_WAL_RETENTION_SIZE, &t); } - yymsp[-3].minor.yy840 = yylhsminor.yy840; + yymsp[-3].minor.yy272 = yylhsminor.yy272; break; case 98: /* db_options ::= db_options WAL_ROLL_PERIOD NK_INTEGER */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_WAL_ROLL_PERIOD, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_WAL_ROLL_PERIOD, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 99: /* db_options ::= db_options WAL_SEGMENT_SIZE NK_INTEGER */ -{ yylhsminor.yy840 = setDatabaseOption(pCxt, yymsp[-2].minor.yy840, DB_OPTION_WAL_SEGMENT_SIZE, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setDatabaseOption(pCxt, yymsp[-2].minor.yy272, DB_OPTION_WAL_SEGMENT_SIZE, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 100: /* alter_db_options ::= alter_db_option */ -{ yylhsminor.yy840 = createAlterDatabaseOptions(pCxt); yylhsminor.yy840 = setAlterDatabaseOption(pCxt, yylhsminor.yy840, &yymsp[0].minor.yy95); } - yymsp[0].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createAlterDatabaseOptions(pCxt); yylhsminor.yy272 = setAlterDatabaseOption(pCxt, yylhsminor.yy272, &yymsp[0].minor.yy5); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; case 101: /* alter_db_options ::= alter_db_options alter_db_option */ -{ yylhsminor.yy840 = setAlterDatabaseOption(pCxt, yymsp[-1].minor.yy840, &yymsp[0].minor.yy95); } - yymsp[-1].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setAlterDatabaseOption(pCxt, yymsp[-1].minor.yy272, &yymsp[0].minor.yy5); } + yymsp[-1].minor.yy272 = yylhsminor.yy272; break; case 102: /* alter_db_option ::= CACHEMODEL NK_STRING */ -{ yymsp[-1].minor.yy95.type = DB_OPTION_CACHEMODEL; yymsp[-1].minor.yy95.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy5.type = DB_OPTION_CACHEMODEL; yymsp[-1].minor.yy5.val = yymsp[0].minor.yy0; } break; case 103: /* alter_db_option ::= CACHESIZE NK_INTEGER */ -{ yymsp[-1].minor.yy95.type = DB_OPTION_CACHESIZE; yymsp[-1].minor.yy95.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy5.type = DB_OPTION_CACHESIZE; yymsp[-1].minor.yy5.val = yymsp[0].minor.yy0; } break; case 104: /* alter_db_option ::= WAL_FSYNC_PERIOD NK_INTEGER */ -{ yymsp[-1].minor.yy95.type = DB_OPTION_FSYNC; yymsp[-1].minor.yy95.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy5.type = DB_OPTION_FSYNC; yymsp[-1].minor.yy5.val = yymsp[0].minor.yy0; } break; case 105: /* alter_db_option ::= KEEP integer_list */ case 106: /* alter_db_option ::= KEEP variable_list */ yytestcase(yyruleno==106); -{ yymsp[-1].minor.yy95.type = DB_OPTION_KEEP; yymsp[-1].minor.yy95.pList = yymsp[0].minor.yy544; } +{ yymsp[-1].minor.yy5.type = DB_OPTION_KEEP; yymsp[-1].minor.yy5.pList = yymsp[0].minor.yy172; } break; case 107: /* alter_db_option ::= WAL_LEVEL NK_INTEGER */ -{ yymsp[-1].minor.yy95.type = DB_OPTION_WAL; yymsp[-1].minor.yy95.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy5.type = DB_OPTION_WAL; yymsp[-1].minor.yy5.val = yymsp[0].minor.yy0; } break; case 108: /* integer_list ::= NK_INTEGER */ -{ yylhsminor.yy544 = createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy544 = yylhsminor.yy544; +{ yylhsminor.yy172 = createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy172 = yylhsminor.yy172; break; case 109: /* integer_list ::= integer_list NK_COMMA NK_INTEGER */ - case 284: /* dnode_list ::= dnode_list DNODE NK_INTEGER */ yytestcase(yyruleno==284); -{ yylhsminor.yy544 = addNodeToList(pCxt, yymsp[-2].minor.yy544, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } - yymsp[-2].minor.yy544 = yylhsminor.yy544; + case 282: /* dnode_list ::= dnode_list DNODE NK_INTEGER */ yytestcase(yyruleno==282); +{ yylhsminor.yy172 = addNodeToList(pCxt, yymsp[-2].minor.yy172, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } + yymsp[-2].minor.yy172 = yylhsminor.yy172; break; case 110: /* variable_list ::= NK_VARIABLE */ -{ yylhsminor.yy544 = createNodeList(pCxt, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy544 = yylhsminor.yy544; +{ yylhsminor.yy172 = createNodeList(pCxt, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy172 = yylhsminor.yy172; break; case 111: /* variable_list ::= variable_list NK_COMMA NK_VARIABLE */ -{ yylhsminor.yy544 = addNodeToList(pCxt, yymsp[-2].minor.yy544, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } - yymsp[-2].minor.yy544 = yylhsminor.yy544; +{ yylhsminor.yy172 = addNodeToList(pCxt, yymsp[-2].minor.yy172, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } + yymsp[-2].minor.yy172 = yylhsminor.yy172; break; case 112: /* retention_list ::= retention */ case 132: /* multi_create_clause ::= create_subtable_clause */ yytestcase(yyruleno==132); @@ -3792,266 +3742,266 @@ static YYACTIONTYPE yy_reduce( case 185: /* rollup_func_list ::= rollup_func_name */ yytestcase(yyruleno==185); case 190: /* col_name_list ::= col_name */ yytestcase(yyruleno==190); case 238: /* func_list ::= func */ yytestcase(yyruleno==238); - case 312: /* literal_list ::= signed_literal */ yytestcase(yyruleno==312); - case 374: /* other_para_list ::= star_func_para */ yytestcase(yyruleno==374); - case 428: /* select_list ::= select_item */ yytestcase(yyruleno==428); - case 482: /* sort_specification_list ::= sort_specification */ yytestcase(yyruleno==482); -{ yylhsminor.yy544 = createNodeList(pCxt, yymsp[0].minor.yy840); } - yymsp[0].minor.yy544 = yylhsminor.yy544; + case 310: /* literal_list ::= signed_literal */ yytestcase(yyruleno==310); + case 372: /* other_para_list ::= star_func_para */ yytestcase(yyruleno==372); + case 426: /* select_list ::= select_item */ yytestcase(yyruleno==426); + case 480: /* sort_specification_list ::= sort_specification */ yytestcase(yyruleno==480); +{ yylhsminor.yy172 = createNodeList(pCxt, yymsp[0].minor.yy272); } + yymsp[0].minor.yy172 = yylhsminor.yy172; break; case 113: /* retention_list ::= retention_list NK_COMMA retention */ case 143: /* column_def_list ::= column_def_list NK_COMMA column_def */ yytestcase(yyruleno==143); case 186: /* rollup_func_list ::= rollup_func_list NK_COMMA rollup_func_name */ yytestcase(yyruleno==186); case 191: /* col_name_list ::= col_name_list NK_COMMA col_name */ yytestcase(yyruleno==191); case 239: /* func_list ::= func_list NK_COMMA func */ yytestcase(yyruleno==239); - case 313: /* literal_list ::= literal_list NK_COMMA signed_literal */ yytestcase(yyruleno==313); - case 375: /* other_para_list ::= other_para_list NK_COMMA star_func_para */ yytestcase(yyruleno==375); - case 429: /* select_list ::= select_list NK_COMMA select_item */ yytestcase(yyruleno==429); - case 483: /* sort_specification_list ::= sort_specification_list NK_COMMA sort_specification */ yytestcase(yyruleno==483); -{ yylhsminor.yy544 = addNodeToList(pCxt, yymsp[-2].minor.yy544, yymsp[0].minor.yy840); } - yymsp[-2].minor.yy544 = yylhsminor.yy544; + case 311: /* literal_list ::= literal_list NK_COMMA signed_literal */ yytestcase(yyruleno==311); + case 373: /* other_para_list ::= other_para_list NK_COMMA star_func_para */ yytestcase(yyruleno==373); + case 427: /* select_list ::= select_list NK_COMMA select_item */ yytestcase(yyruleno==427); + case 481: /* sort_specification_list ::= sort_specification_list NK_COMMA sort_specification */ yytestcase(yyruleno==481); +{ yylhsminor.yy172 = addNodeToList(pCxt, yymsp[-2].minor.yy172, yymsp[0].minor.yy272); } + yymsp[-2].minor.yy172 = yylhsminor.yy172; break; case 114: /* retention ::= NK_VARIABLE NK_COLON NK_VARIABLE */ -{ yylhsminor.yy840 = createNodeListNodeEx(pCxt, createDurationValueNode(pCxt, &yymsp[-2].minor.yy0), createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createNodeListNodeEx(pCxt, createDurationValueNode(pCxt, &yymsp[-2].minor.yy0), createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 115: /* cmd ::= CREATE TABLE not_exists_opt full_table_name NK_LP column_def_list NK_RP tags_def_opt table_options */ case 117: /* cmd ::= CREATE STABLE not_exists_opt full_table_name NK_LP column_def_list NK_RP tags_def table_options */ yytestcase(yyruleno==117); -{ pCxt->pRootNode = createCreateTableStmt(pCxt, yymsp[-6].minor.yy313, yymsp[-5].minor.yy840, yymsp[-3].minor.yy544, yymsp[-1].minor.yy544, yymsp[0].minor.yy840); } +{ pCxt->pRootNode = createCreateTableStmt(pCxt, yymsp[-6].minor.yy293, yymsp[-5].minor.yy272, yymsp[-3].minor.yy172, yymsp[-1].minor.yy172, yymsp[0].minor.yy272); } break; case 116: /* cmd ::= CREATE TABLE multi_create_clause */ -{ pCxt->pRootNode = createCreateMultiTableStmt(pCxt, yymsp[0].minor.yy544); } +{ pCxt->pRootNode = createCreateMultiTableStmt(pCxt, yymsp[0].minor.yy172); } break; case 118: /* cmd ::= DROP TABLE multi_drop_clause */ -{ pCxt->pRootNode = createDropTableStmt(pCxt, yymsp[0].minor.yy544); } +{ pCxt->pRootNode = createDropTableStmt(pCxt, yymsp[0].minor.yy172); } break; case 119: /* cmd ::= DROP STABLE exists_opt full_table_name */ -{ pCxt->pRootNode = createDropSuperTableStmt(pCxt, yymsp[-1].minor.yy313, yymsp[0].minor.yy840); } +{ pCxt->pRootNode = createDropSuperTableStmt(pCxt, yymsp[-1].minor.yy293, yymsp[0].minor.yy272); } break; case 120: /* cmd ::= ALTER TABLE alter_table_clause */ - case 286: /* cmd ::= query_expression */ yytestcase(yyruleno==286); -{ pCxt->pRootNode = yymsp[0].minor.yy840; } + case 284: /* cmd ::= query_expression */ yytestcase(yyruleno==284); +{ pCxt->pRootNode = yymsp[0].minor.yy272; } break; case 121: /* cmd ::= ALTER STABLE alter_table_clause */ -{ pCxt->pRootNode = setAlterSuperTableType(yymsp[0].minor.yy840); } +{ pCxt->pRootNode = setAlterSuperTableType(yymsp[0].minor.yy272); } break; case 122: /* alter_table_clause ::= full_table_name alter_table_options */ -{ yylhsminor.yy840 = createAlterTableModifyOptions(pCxt, yymsp[-1].minor.yy840, yymsp[0].minor.yy840); } - yymsp[-1].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createAlterTableModifyOptions(pCxt, yymsp[-1].minor.yy272, yymsp[0].minor.yy272); } + yymsp[-1].minor.yy272 = yylhsminor.yy272; break; case 123: /* alter_table_clause ::= full_table_name ADD COLUMN column_name type_name */ -{ yylhsminor.yy840 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy840, TSDB_ALTER_TABLE_ADD_COLUMN, &yymsp[-1].minor.yy617, yymsp[0].minor.yy784); } - yymsp[-4].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy272, TSDB_ALTER_TABLE_ADD_COLUMN, &yymsp[-1].minor.yy209, yymsp[0].minor.yy616); } + yymsp[-4].minor.yy272 = yylhsminor.yy272; break; case 124: /* alter_table_clause ::= full_table_name DROP COLUMN column_name */ -{ yylhsminor.yy840 = createAlterTableDropCol(pCxt, yymsp[-3].minor.yy840, TSDB_ALTER_TABLE_DROP_COLUMN, &yymsp[0].minor.yy617); } - yymsp[-3].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createAlterTableDropCol(pCxt, yymsp[-3].minor.yy272, TSDB_ALTER_TABLE_DROP_COLUMN, &yymsp[0].minor.yy209); } + yymsp[-3].minor.yy272 = yylhsminor.yy272; break; case 125: /* alter_table_clause ::= full_table_name MODIFY COLUMN column_name type_name */ -{ yylhsminor.yy840 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy840, TSDB_ALTER_TABLE_UPDATE_COLUMN_BYTES, &yymsp[-1].minor.yy617, yymsp[0].minor.yy784); } - yymsp[-4].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy272, TSDB_ALTER_TABLE_UPDATE_COLUMN_BYTES, &yymsp[-1].minor.yy209, yymsp[0].minor.yy616); } + yymsp[-4].minor.yy272 = yylhsminor.yy272; break; case 126: /* alter_table_clause ::= full_table_name RENAME COLUMN column_name column_name */ -{ yylhsminor.yy840 = createAlterTableRenameCol(pCxt, yymsp[-4].minor.yy840, TSDB_ALTER_TABLE_UPDATE_COLUMN_NAME, &yymsp[-1].minor.yy617, &yymsp[0].minor.yy617); } - yymsp[-4].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createAlterTableRenameCol(pCxt, yymsp[-4].minor.yy272, TSDB_ALTER_TABLE_UPDATE_COLUMN_NAME, &yymsp[-1].minor.yy209, &yymsp[0].minor.yy209); } + yymsp[-4].minor.yy272 = yylhsminor.yy272; break; case 127: /* alter_table_clause ::= full_table_name ADD TAG column_name type_name */ -{ yylhsminor.yy840 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy840, TSDB_ALTER_TABLE_ADD_TAG, &yymsp[-1].minor.yy617, yymsp[0].minor.yy784); } - yymsp[-4].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy272, TSDB_ALTER_TABLE_ADD_TAG, &yymsp[-1].minor.yy209, yymsp[0].minor.yy616); } + yymsp[-4].minor.yy272 = yylhsminor.yy272; break; case 128: /* alter_table_clause ::= full_table_name DROP TAG column_name */ -{ yylhsminor.yy840 = createAlterTableDropCol(pCxt, yymsp[-3].minor.yy840, TSDB_ALTER_TABLE_DROP_TAG, &yymsp[0].minor.yy617); } - yymsp[-3].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createAlterTableDropCol(pCxt, yymsp[-3].minor.yy272, TSDB_ALTER_TABLE_DROP_TAG, &yymsp[0].minor.yy209); } + yymsp[-3].minor.yy272 = yylhsminor.yy272; break; case 129: /* alter_table_clause ::= full_table_name MODIFY TAG column_name type_name */ -{ yylhsminor.yy840 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy840, TSDB_ALTER_TABLE_UPDATE_TAG_BYTES, &yymsp[-1].minor.yy617, yymsp[0].minor.yy784); } - yymsp[-4].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy272, TSDB_ALTER_TABLE_UPDATE_TAG_BYTES, &yymsp[-1].minor.yy209, yymsp[0].minor.yy616); } + yymsp[-4].minor.yy272 = yylhsminor.yy272; break; case 130: /* alter_table_clause ::= full_table_name RENAME TAG column_name column_name */ -{ yylhsminor.yy840 = createAlterTableRenameCol(pCxt, yymsp[-4].minor.yy840, TSDB_ALTER_TABLE_UPDATE_TAG_NAME, &yymsp[-1].minor.yy617, &yymsp[0].minor.yy617); } - yymsp[-4].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createAlterTableRenameCol(pCxt, yymsp[-4].minor.yy272, TSDB_ALTER_TABLE_UPDATE_TAG_NAME, &yymsp[-1].minor.yy209, &yymsp[0].minor.yy209); } + yymsp[-4].minor.yy272 = yylhsminor.yy272; break; case 131: /* alter_table_clause ::= full_table_name SET TAG column_name NK_EQ signed_literal */ -{ yylhsminor.yy840 = createAlterTableSetTag(pCxt, yymsp[-5].minor.yy840, &yymsp[-2].minor.yy617, yymsp[0].minor.yy840); } - yymsp[-5].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createAlterTableSetTag(pCxt, yymsp[-5].minor.yy272, &yymsp[-2].minor.yy209, yymsp[0].minor.yy272); } + yymsp[-5].minor.yy272 = yylhsminor.yy272; break; case 133: /* multi_create_clause ::= multi_create_clause create_subtable_clause */ case 136: /* multi_drop_clause ::= multi_drop_clause drop_table_clause */ yytestcase(yyruleno==136); -{ yylhsminor.yy544 = addNodeToList(pCxt, yymsp[-1].minor.yy544, yymsp[0].minor.yy840); } - yymsp[-1].minor.yy544 = yylhsminor.yy544; +{ yylhsminor.yy172 = addNodeToList(pCxt, yymsp[-1].minor.yy172, yymsp[0].minor.yy272); } + yymsp[-1].minor.yy172 = yylhsminor.yy172; break; case 134: /* create_subtable_clause ::= not_exists_opt full_table_name USING full_table_name specific_cols_opt TAGS NK_LP expression_list NK_RP table_options */ -{ yylhsminor.yy840 = createCreateSubTableClause(pCxt, yymsp[-9].minor.yy313, yymsp[-8].minor.yy840, yymsp[-6].minor.yy840, yymsp[-5].minor.yy544, yymsp[-2].minor.yy544, yymsp[0].minor.yy840); } - yymsp[-9].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createCreateSubTableClause(pCxt, yymsp[-9].minor.yy293, yymsp[-8].minor.yy272, yymsp[-6].minor.yy272, yymsp[-5].minor.yy172, yymsp[-2].minor.yy172, yymsp[0].minor.yy272); } + yymsp[-9].minor.yy272 = yylhsminor.yy272; break; case 137: /* drop_table_clause ::= exists_opt full_table_name */ -{ yylhsminor.yy840 = createDropTableClause(pCxt, yymsp[-1].minor.yy313, yymsp[0].minor.yy840); } - yymsp[-1].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createDropTableClause(pCxt, yymsp[-1].minor.yy293, yymsp[0].minor.yy272); } + yymsp[-1].minor.yy272 = yylhsminor.yy272; break; case 138: /* specific_cols_opt ::= */ case 169: /* tags_def_opt ::= */ yytestcase(yyruleno==169); - case 437: /* partition_by_clause_opt ::= */ yytestcase(yyruleno==437); - case 454: /* group_by_clause_opt ::= */ yytestcase(yyruleno==454); - case 470: /* order_by_clause_opt ::= */ yytestcase(yyruleno==470); -{ yymsp[1].minor.yy544 = NULL; } + case 435: /* partition_by_clause_opt ::= */ yytestcase(yyruleno==435); + case 452: /* group_by_clause_opt ::= */ yytestcase(yyruleno==452); + case 468: /* order_by_clause_opt ::= */ yytestcase(yyruleno==468); +{ yymsp[1].minor.yy172 = NULL; } break; case 139: /* specific_cols_opt ::= NK_LP col_name_list NK_RP */ -{ yymsp[-2].minor.yy544 = yymsp[-1].minor.yy544; } +{ yymsp[-2].minor.yy172 = yymsp[-1].minor.yy172; } break; case 140: /* full_table_name ::= table_name */ -{ yylhsminor.yy840 = createRealTableNode(pCxt, NULL, &yymsp[0].minor.yy617, NULL); } - yymsp[0].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createRealTableNode(pCxt, NULL, &yymsp[0].minor.yy209, NULL); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; case 141: /* full_table_name ::= db_name NK_DOT table_name */ -{ yylhsminor.yy840 = createRealTableNode(pCxt, &yymsp[-2].minor.yy617, &yymsp[0].minor.yy617, NULL); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createRealTableNode(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209, NULL); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 144: /* column_def ::= column_name type_name */ -{ yylhsminor.yy840 = createColumnDefNode(pCxt, &yymsp[-1].minor.yy617, yymsp[0].minor.yy784, NULL); } - yymsp[-1].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createColumnDefNode(pCxt, &yymsp[-1].minor.yy209, yymsp[0].minor.yy616, NULL); } + yymsp[-1].minor.yy272 = yylhsminor.yy272; break; case 145: /* column_def ::= column_name type_name COMMENT NK_STRING */ -{ yylhsminor.yy840 = createColumnDefNode(pCxt, &yymsp[-3].minor.yy617, yymsp[-2].minor.yy784, &yymsp[0].minor.yy0); } - yymsp[-3].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createColumnDefNode(pCxt, &yymsp[-3].minor.yy209, yymsp[-2].minor.yy616, &yymsp[0].minor.yy0); } + yymsp[-3].minor.yy272 = yylhsminor.yy272; break; case 146: /* type_name ::= BOOL */ -{ yymsp[0].minor.yy784 = createDataType(TSDB_DATA_TYPE_BOOL); } +{ yymsp[0].minor.yy616 = createDataType(TSDB_DATA_TYPE_BOOL); } break; case 147: /* type_name ::= TINYINT */ -{ yymsp[0].minor.yy784 = createDataType(TSDB_DATA_TYPE_TINYINT); } +{ yymsp[0].minor.yy616 = createDataType(TSDB_DATA_TYPE_TINYINT); } break; case 148: /* type_name ::= SMALLINT */ -{ yymsp[0].minor.yy784 = createDataType(TSDB_DATA_TYPE_SMALLINT); } +{ yymsp[0].minor.yy616 = createDataType(TSDB_DATA_TYPE_SMALLINT); } break; case 149: /* type_name ::= INT */ case 150: /* type_name ::= INTEGER */ yytestcase(yyruleno==150); -{ yymsp[0].minor.yy784 = createDataType(TSDB_DATA_TYPE_INT); } +{ yymsp[0].minor.yy616 = createDataType(TSDB_DATA_TYPE_INT); } break; case 151: /* type_name ::= BIGINT */ -{ yymsp[0].minor.yy784 = createDataType(TSDB_DATA_TYPE_BIGINT); } +{ yymsp[0].minor.yy616 = createDataType(TSDB_DATA_TYPE_BIGINT); } break; case 152: /* type_name ::= FLOAT */ -{ yymsp[0].minor.yy784 = createDataType(TSDB_DATA_TYPE_FLOAT); } +{ yymsp[0].minor.yy616 = createDataType(TSDB_DATA_TYPE_FLOAT); } break; case 153: /* type_name ::= DOUBLE */ -{ yymsp[0].minor.yy784 = createDataType(TSDB_DATA_TYPE_DOUBLE); } +{ yymsp[0].minor.yy616 = createDataType(TSDB_DATA_TYPE_DOUBLE); } break; case 154: /* type_name ::= BINARY NK_LP NK_INTEGER NK_RP */ -{ yymsp[-3].minor.yy784 = createVarLenDataType(TSDB_DATA_TYPE_BINARY, &yymsp[-1].minor.yy0); } +{ yymsp[-3].minor.yy616 = createVarLenDataType(TSDB_DATA_TYPE_BINARY, &yymsp[-1].minor.yy0); } break; case 155: /* type_name ::= TIMESTAMP */ -{ yymsp[0].minor.yy784 = createDataType(TSDB_DATA_TYPE_TIMESTAMP); } +{ yymsp[0].minor.yy616 = createDataType(TSDB_DATA_TYPE_TIMESTAMP); } break; case 156: /* type_name ::= NCHAR NK_LP NK_INTEGER NK_RP */ -{ yymsp[-3].minor.yy784 = createVarLenDataType(TSDB_DATA_TYPE_NCHAR, &yymsp[-1].minor.yy0); } +{ yymsp[-3].minor.yy616 = createVarLenDataType(TSDB_DATA_TYPE_NCHAR, &yymsp[-1].minor.yy0); } break; case 157: /* type_name ::= TINYINT UNSIGNED */ -{ yymsp[-1].minor.yy784 = createDataType(TSDB_DATA_TYPE_UTINYINT); } +{ yymsp[-1].minor.yy616 = createDataType(TSDB_DATA_TYPE_UTINYINT); } break; case 158: /* type_name ::= SMALLINT UNSIGNED */ -{ yymsp[-1].minor.yy784 = createDataType(TSDB_DATA_TYPE_USMALLINT); } +{ yymsp[-1].minor.yy616 = createDataType(TSDB_DATA_TYPE_USMALLINT); } break; case 159: /* type_name ::= INT UNSIGNED */ -{ yymsp[-1].minor.yy784 = createDataType(TSDB_DATA_TYPE_UINT); } +{ yymsp[-1].minor.yy616 = createDataType(TSDB_DATA_TYPE_UINT); } break; case 160: /* type_name ::= BIGINT UNSIGNED */ -{ yymsp[-1].minor.yy784 = createDataType(TSDB_DATA_TYPE_UBIGINT); } +{ yymsp[-1].minor.yy616 = createDataType(TSDB_DATA_TYPE_UBIGINT); } break; case 161: /* type_name ::= JSON */ -{ yymsp[0].minor.yy784 = createDataType(TSDB_DATA_TYPE_JSON); } +{ yymsp[0].minor.yy616 = createDataType(TSDB_DATA_TYPE_JSON); } break; case 162: /* type_name ::= VARCHAR NK_LP NK_INTEGER NK_RP */ -{ yymsp[-3].minor.yy784 = createVarLenDataType(TSDB_DATA_TYPE_VARCHAR, &yymsp[-1].minor.yy0); } +{ yymsp[-3].minor.yy616 = createVarLenDataType(TSDB_DATA_TYPE_VARCHAR, &yymsp[-1].minor.yy0); } break; case 163: /* type_name ::= MEDIUMBLOB */ -{ yymsp[0].minor.yy784 = createDataType(TSDB_DATA_TYPE_MEDIUMBLOB); } +{ yymsp[0].minor.yy616 = createDataType(TSDB_DATA_TYPE_MEDIUMBLOB); } break; case 164: /* type_name ::= BLOB */ -{ yymsp[0].minor.yy784 = createDataType(TSDB_DATA_TYPE_BLOB); } +{ yymsp[0].minor.yy616 = createDataType(TSDB_DATA_TYPE_BLOB); } break; case 165: /* type_name ::= VARBINARY NK_LP NK_INTEGER NK_RP */ -{ yymsp[-3].minor.yy784 = createVarLenDataType(TSDB_DATA_TYPE_VARBINARY, &yymsp[-1].minor.yy0); } +{ yymsp[-3].minor.yy616 = createVarLenDataType(TSDB_DATA_TYPE_VARBINARY, &yymsp[-1].minor.yy0); } break; case 166: /* type_name ::= DECIMAL */ -{ yymsp[0].minor.yy784 = createDataType(TSDB_DATA_TYPE_DECIMAL); } +{ yymsp[0].minor.yy616 = createDataType(TSDB_DATA_TYPE_DECIMAL); } break; case 167: /* type_name ::= DECIMAL NK_LP NK_INTEGER NK_RP */ -{ yymsp[-3].minor.yy784 = createDataType(TSDB_DATA_TYPE_DECIMAL); } +{ yymsp[-3].minor.yy616 = createDataType(TSDB_DATA_TYPE_DECIMAL); } break; case 168: /* type_name ::= DECIMAL NK_LP NK_INTEGER NK_COMMA NK_INTEGER NK_RP */ -{ yymsp[-5].minor.yy784 = createDataType(TSDB_DATA_TYPE_DECIMAL); } +{ yymsp[-5].minor.yy616 = createDataType(TSDB_DATA_TYPE_DECIMAL); } break; case 170: /* tags_def_opt ::= tags_def */ - case 373: /* star_func_para_list ::= other_para_list */ yytestcase(yyruleno==373); -{ yylhsminor.yy544 = yymsp[0].minor.yy544; } - yymsp[0].minor.yy544 = yylhsminor.yy544; + case 371: /* star_func_para_list ::= other_para_list */ yytestcase(yyruleno==371); +{ yylhsminor.yy172 = yymsp[0].minor.yy172; } + yymsp[0].minor.yy172 = yylhsminor.yy172; break; case 171: /* tags_def ::= TAGS NK_LP column_def_list NK_RP */ -{ yymsp[-3].minor.yy544 = yymsp[-1].minor.yy544; } +{ yymsp[-3].minor.yy172 = yymsp[-1].minor.yy172; } break; case 172: /* table_options ::= */ -{ yymsp[1].minor.yy840 = createDefaultTableOptions(pCxt); } +{ yymsp[1].minor.yy272 = createDefaultTableOptions(pCxt); } break; case 173: /* table_options ::= table_options COMMENT NK_STRING */ -{ yylhsminor.yy840 = setTableOption(pCxt, yymsp[-2].minor.yy840, TABLE_OPTION_COMMENT, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setTableOption(pCxt, yymsp[-2].minor.yy272, TABLE_OPTION_COMMENT, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 174: /* table_options ::= table_options MAX_DELAY duration_list */ -{ yylhsminor.yy840 = setTableOption(pCxt, yymsp[-2].minor.yy840, TABLE_OPTION_MAXDELAY, yymsp[0].minor.yy544); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setTableOption(pCxt, yymsp[-2].minor.yy272, TABLE_OPTION_MAXDELAY, yymsp[0].minor.yy172); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 175: /* table_options ::= table_options WATERMARK duration_list */ -{ yylhsminor.yy840 = setTableOption(pCxt, yymsp[-2].minor.yy840, TABLE_OPTION_WATERMARK, yymsp[0].minor.yy544); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setTableOption(pCxt, yymsp[-2].minor.yy272, TABLE_OPTION_WATERMARK, yymsp[0].minor.yy172); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 176: /* table_options ::= table_options ROLLUP NK_LP rollup_func_list NK_RP */ -{ yylhsminor.yy840 = setTableOption(pCxt, yymsp[-4].minor.yy840, TABLE_OPTION_ROLLUP, yymsp[-1].minor.yy544); } - yymsp[-4].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setTableOption(pCxt, yymsp[-4].minor.yy272, TABLE_OPTION_ROLLUP, yymsp[-1].minor.yy172); } + yymsp[-4].minor.yy272 = yylhsminor.yy272; break; case 177: /* table_options ::= table_options TTL NK_INTEGER */ -{ yylhsminor.yy840 = setTableOption(pCxt, yymsp[-2].minor.yy840, TABLE_OPTION_TTL, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setTableOption(pCxt, yymsp[-2].minor.yy272, TABLE_OPTION_TTL, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 178: /* table_options ::= table_options SMA NK_LP col_name_list NK_RP */ -{ yylhsminor.yy840 = setTableOption(pCxt, yymsp[-4].minor.yy840, TABLE_OPTION_SMA, yymsp[-1].minor.yy544); } - yymsp[-4].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setTableOption(pCxt, yymsp[-4].minor.yy272, TABLE_OPTION_SMA, yymsp[-1].minor.yy172); } + yymsp[-4].minor.yy272 = yylhsminor.yy272; break; case 179: /* alter_table_options ::= alter_table_option */ -{ yylhsminor.yy840 = createAlterTableOptions(pCxt); yylhsminor.yy840 = setTableOption(pCxt, yylhsminor.yy840, yymsp[0].minor.yy95.type, &yymsp[0].minor.yy95.val); } - yymsp[0].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createAlterTableOptions(pCxt); yylhsminor.yy272 = setTableOption(pCxt, yylhsminor.yy272, yymsp[0].minor.yy5.type, &yymsp[0].minor.yy5.val); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; case 180: /* alter_table_options ::= alter_table_options alter_table_option */ -{ yylhsminor.yy840 = setTableOption(pCxt, yymsp[-1].minor.yy840, yymsp[0].minor.yy95.type, &yymsp[0].minor.yy95.val); } - yymsp[-1].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setTableOption(pCxt, yymsp[-1].minor.yy272, yymsp[0].minor.yy5.type, &yymsp[0].minor.yy5.val); } + yymsp[-1].minor.yy272 = yylhsminor.yy272; break; case 181: /* alter_table_option ::= COMMENT NK_STRING */ -{ yymsp[-1].minor.yy95.type = TABLE_OPTION_COMMENT; yymsp[-1].minor.yy95.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy5.type = TABLE_OPTION_COMMENT; yymsp[-1].minor.yy5.val = yymsp[0].minor.yy0; } break; case 182: /* alter_table_option ::= TTL NK_INTEGER */ -{ yymsp[-1].minor.yy95.type = TABLE_OPTION_TTL; yymsp[-1].minor.yy95.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy5.type = TABLE_OPTION_TTL; yymsp[-1].minor.yy5.val = yymsp[0].minor.yy0; } break; case 183: /* duration_list ::= duration_literal */ - case 340: /* expression_list ::= expression */ yytestcase(yyruleno==340); -{ yylhsminor.yy544 = createNodeList(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy840)); } - yymsp[0].minor.yy544 = yylhsminor.yy544; + case 338: /* expression_list ::= expression */ yytestcase(yyruleno==338); +{ yylhsminor.yy172 = createNodeList(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy272)); } + yymsp[0].minor.yy172 = yylhsminor.yy172; break; case 184: /* duration_list ::= duration_list NK_COMMA duration_literal */ - case 341: /* expression_list ::= expression_list NK_COMMA expression */ yytestcase(yyruleno==341); -{ yylhsminor.yy544 = addNodeToList(pCxt, yymsp[-2].minor.yy544, releaseRawExprNode(pCxt, yymsp[0].minor.yy840)); } - yymsp[-2].minor.yy544 = yylhsminor.yy544; + case 339: /* expression_list ::= expression_list NK_COMMA expression */ yytestcase(yyruleno==339); +{ yylhsminor.yy172 = addNodeToList(pCxt, yymsp[-2].minor.yy172, releaseRawExprNode(pCxt, yymsp[0].minor.yy272)); } + yymsp[-2].minor.yy172 = yylhsminor.yy172; break; case 187: /* rollup_func_name ::= function_name */ -{ yylhsminor.yy840 = createFunctionNode(pCxt, &yymsp[0].minor.yy617, NULL); } - yymsp[0].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createFunctionNode(pCxt, &yymsp[0].minor.yy209, NULL); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; case 188: /* rollup_func_name ::= FIRST */ case 189: /* rollup_func_name ::= LAST */ yytestcase(yyruleno==189); -{ yylhsminor.yy840 = createFunctionNode(pCxt, &yymsp[0].minor.yy0, NULL); } - yymsp[0].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createFunctionNode(pCxt, &yymsp[0].minor.yy0, NULL); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; case 192: /* col_name ::= column_name */ -{ yylhsminor.yy840 = createColumnNode(pCxt, NULL, &yymsp[0].minor.yy617); } - yymsp[0].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createColumnNode(pCxt, NULL, &yymsp[0].minor.yy209); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; case 193: /* cmd ::= SHOW DNODES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_DNODES_STMT); } @@ -4063,13 +4013,13 @@ static YYACTIONTYPE yy_reduce( { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_DATABASES_STMT); } break; case 196: /* cmd ::= SHOW db_name_cond_opt TABLES like_pattern_opt */ -{ pCxt->pRootNode = createShowStmtWithCond(pCxt, QUERY_NODE_SHOW_TABLES_STMT, yymsp[-2].minor.yy840, yymsp[0].minor.yy840, OP_TYPE_LIKE); } +{ pCxt->pRootNode = createShowStmtWithCond(pCxt, QUERY_NODE_SHOW_TABLES_STMT, yymsp[-2].minor.yy272, yymsp[0].minor.yy272, OP_TYPE_LIKE); } break; case 197: /* cmd ::= SHOW db_name_cond_opt STABLES like_pattern_opt */ -{ pCxt->pRootNode = createShowStmtWithCond(pCxt, QUERY_NODE_SHOW_STABLES_STMT, yymsp[-2].minor.yy840, yymsp[0].minor.yy840, OP_TYPE_LIKE); } +{ pCxt->pRootNode = createShowStmtWithCond(pCxt, QUERY_NODE_SHOW_STABLES_STMT, yymsp[-2].minor.yy272, yymsp[0].minor.yy272, OP_TYPE_LIKE); } break; case 198: /* cmd ::= SHOW db_name_cond_opt VGROUPS */ -{ pCxt->pRootNode = createShowStmtWithCond(pCxt, QUERY_NODE_SHOW_VGROUPS_STMT, yymsp[-1].minor.yy840, NULL, OP_TYPE_LIKE); } +{ pCxt->pRootNode = createShowStmtWithCond(pCxt, QUERY_NODE_SHOW_VGROUPS_STMT, yymsp[-1].minor.yy272, NULL, OP_TYPE_LIKE); } break; case 199: /* cmd ::= SHOW MNODES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_MNODES_STMT); } @@ -4084,7 +4034,7 @@ static YYACTIONTYPE yy_reduce( { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_FUNCTIONS_STMT); } break; case 203: /* cmd ::= SHOW INDEXES FROM table_name_cond from_db_opt */ -{ pCxt->pRootNode = createShowStmtWithCond(pCxt, QUERY_NODE_SHOW_INDEXES_STMT, yymsp[0].minor.yy840, yymsp[-1].minor.yy840, OP_TYPE_EQUAL); } +{ pCxt->pRootNode = createShowStmtWithCond(pCxt, QUERY_NODE_SHOW_INDEXES_STMT, yymsp[0].minor.yy272, yymsp[-1].minor.yy272, OP_TYPE_EQUAL); } break; case 204: /* cmd ::= SHOW STREAMS */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_STREAMS_STMT); } @@ -4103,13 +4053,13 @@ static YYACTIONTYPE yy_reduce( { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_LICENCES_STMT); } break; case 210: /* cmd ::= SHOW CREATE DATABASE db_name */ -{ pCxt->pRootNode = createShowCreateDatabaseStmt(pCxt, &yymsp[0].minor.yy617); } +{ pCxt->pRootNode = createShowCreateDatabaseStmt(pCxt, &yymsp[0].minor.yy209); } break; case 211: /* cmd ::= SHOW CREATE TABLE full_table_name */ -{ pCxt->pRootNode = createShowCreateTableStmt(pCxt, QUERY_NODE_SHOW_CREATE_TABLE_STMT, yymsp[0].minor.yy840); } +{ pCxt->pRootNode = createShowCreateTableStmt(pCxt, QUERY_NODE_SHOW_CREATE_TABLE_STMT, yymsp[0].minor.yy272); } break; case 212: /* cmd ::= SHOW CREATE STABLE full_table_name */ -{ pCxt->pRootNode = createShowCreateTableStmt(pCxt, QUERY_NODE_SHOW_CREATE_STABLE_STMT, yymsp[0].minor.yy840); } +{ pCxt->pRootNode = createShowCreateTableStmt(pCxt, QUERY_NODE_SHOW_CREATE_STABLE_STMT, yymsp[0].minor.yy272); } break; case 213: /* cmd ::= SHOW QUERIES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_QUERIES_STMT); } @@ -4142,7 +4092,7 @@ static YYACTIONTYPE yy_reduce( { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_TRANSACTIONS_STMT); } break; case 223: /* cmd ::= SHOW TABLE DISTRIBUTED full_table_name */ -{ pCxt->pRootNode = createShowTableDistributedStmt(pCxt, yymsp[0].minor.yy840); } +{ pCxt->pRootNode = createShowTableDistributedStmt(pCxt, yymsp[0].minor.yy272); } break; case 224: /* cmd ::= SHOW CONSUMERS */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_CONSUMERS_STMT); } @@ -4151,713 +4101,711 @@ static YYACTIONTYPE yy_reduce( { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_SUBSCRIPTIONS_STMT); } break; case 226: /* cmd ::= SHOW TAGS FROM table_name_cond from_db_opt */ -{ pCxt->pRootNode = createShowStmtWithCond(pCxt, QUERY_NODE_SHOW_TAGS_STMT, yymsp[0].minor.yy840, yymsp[-1].minor.yy840, OP_TYPE_EQUAL); } +{ pCxt->pRootNode = createShowStmtWithCond(pCxt, QUERY_NODE_SHOW_TAGS_STMT, yymsp[0].minor.yy272, yymsp[-1].minor.yy272, OP_TYPE_EQUAL); } break; case 227: /* db_name_cond_opt ::= */ case 232: /* from_db_opt ::= */ yytestcase(yyruleno==232); -{ yymsp[1].minor.yy840 = createDefaultDatabaseCondValue(pCxt); } +{ yymsp[1].minor.yy272 = createDefaultDatabaseCondValue(pCxt); } break; case 228: /* db_name_cond_opt ::= db_name NK_DOT */ -{ yylhsminor.yy840 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[-1].minor.yy617); } - yymsp[-1].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[-1].minor.yy209); } + yymsp[-1].minor.yy272 = yylhsminor.yy272; break; case 229: /* like_pattern_opt ::= */ - case 268: /* into_opt ::= */ yytestcase(yyruleno==268); - case 406: /* from_clause_opt ::= */ yytestcase(yyruleno==406); - case 435: /* where_clause_opt ::= */ yytestcase(yyruleno==435); - case 439: /* twindow_clause_opt ::= */ yytestcase(yyruleno==439); - case 444: /* sliding_opt ::= */ yytestcase(yyruleno==444); - case 446: /* fill_opt ::= */ yytestcase(yyruleno==446); - case 458: /* having_clause_opt ::= */ yytestcase(yyruleno==458); - case 460: /* range_opt ::= */ yytestcase(yyruleno==460); - case 462: /* every_opt ::= */ yytestcase(yyruleno==462); - case 472: /* slimit_clause_opt ::= */ yytestcase(yyruleno==472); - case 476: /* limit_clause_opt ::= */ yytestcase(yyruleno==476); -{ yymsp[1].minor.yy840 = NULL; } + case 404: /* from_clause_opt ::= */ yytestcase(yyruleno==404); + case 433: /* where_clause_opt ::= */ yytestcase(yyruleno==433); + case 437: /* twindow_clause_opt ::= */ yytestcase(yyruleno==437); + case 442: /* sliding_opt ::= */ yytestcase(yyruleno==442); + case 444: /* fill_opt ::= */ yytestcase(yyruleno==444); + case 456: /* having_clause_opt ::= */ yytestcase(yyruleno==456); + case 458: /* range_opt ::= */ yytestcase(yyruleno==458); + case 460: /* every_opt ::= */ yytestcase(yyruleno==460); + case 470: /* slimit_clause_opt ::= */ yytestcase(yyruleno==470); + case 474: /* limit_clause_opt ::= */ yytestcase(yyruleno==474); +{ yymsp[1].minor.yy272 = NULL; } break; case 230: /* like_pattern_opt ::= LIKE NK_STRING */ -{ yymsp[-1].minor.yy840 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0); } +{ yymsp[-1].minor.yy272 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0); } break; case 231: /* table_name_cond ::= table_name */ -{ yylhsminor.yy840 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy617); } - yymsp[0].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy209); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; case 233: /* from_db_opt ::= FROM db_name */ -{ yymsp[-1].minor.yy840 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy617); } +{ yymsp[-1].minor.yy272 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy209); } break; case 234: /* cmd ::= CREATE SMA INDEX not_exists_opt full_table_name ON full_table_name index_options */ -{ pCxt->pRootNode = createCreateIndexStmt(pCxt, INDEX_TYPE_SMA, yymsp[-4].minor.yy313, yymsp[-3].minor.yy840, yymsp[-1].minor.yy840, NULL, yymsp[0].minor.yy840); } +{ pCxt->pRootNode = createCreateIndexStmt(pCxt, INDEX_TYPE_SMA, yymsp[-4].minor.yy293, yymsp[-3].minor.yy272, yymsp[-1].minor.yy272, NULL, yymsp[0].minor.yy272); } break; case 235: /* cmd ::= DROP INDEX exists_opt full_table_name */ -{ pCxt->pRootNode = createDropIndexStmt(pCxt, yymsp[-1].minor.yy313, yymsp[0].minor.yy840); } +{ pCxt->pRootNode = createDropIndexStmt(pCxt, yymsp[-1].minor.yy293, yymsp[0].minor.yy272); } break; case 236: /* index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_RP sliding_opt sma_stream_opt */ -{ yymsp[-9].minor.yy840 = createIndexOption(pCxt, yymsp[-7].minor.yy544, releaseRawExprNode(pCxt, yymsp[-3].minor.yy840), NULL, yymsp[-1].minor.yy840, yymsp[0].minor.yy840); } +{ yymsp[-9].minor.yy272 = createIndexOption(pCxt, yymsp[-7].minor.yy172, releaseRawExprNode(pCxt, yymsp[-3].minor.yy272), NULL, yymsp[-1].minor.yy272, yymsp[0].minor.yy272); } break; case 237: /* index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt sma_stream_opt */ -{ yymsp[-11].minor.yy840 = createIndexOption(pCxt, yymsp[-9].minor.yy544, releaseRawExprNode(pCxt, yymsp[-5].minor.yy840), releaseRawExprNode(pCxt, yymsp[-3].minor.yy840), yymsp[-1].minor.yy840, yymsp[0].minor.yy840); } +{ yymsp[-11].minor.yy272 = createIndexOption(pCxt, yymsp[-9].minor.yy172, releaseRawExprNode(pCxt, yymsp[-5].minor.yy272), releaseRawExprNode(pCxt, yymsp[-3].minor.yy272), yymsp[-1].minor.yy272, yymsp[0].minor.yy272); } break; case 240: /* func ::= function_name NK_LP expression_list NK_RP */ -{ yylhsminor.yy840 = createFunctionNode(pCxt, &yymsp[-3].minor.yy617, yymsp[-1].minor.yy544); } - yymsp[-3].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = createFunctionNode(pCxt, &yymsp[-3].minor.yy209, yymsp[-1].minor.yy172); } + yymsp[-3].minor.yy272 = yylhsminor.yy272; break; case 241: /* sma_stream_opt ::= */ - case 270: /* stream_options ::= */ yytestcase(yyruleno==270); -{ yymsp[1].minor.yy840 = createStreamOptions(pCxt); } + case 268: /* stream_options ::= */ yytestcase(yyruleno==268); +{ yymsp[1].minor.yy272 = createStreamOptions(pCxt); } break; case 242: /* sma_stream_opt ::= stream_options WATERMARK duration_literal */ - case 274: /* stream_options ::= stream_options WATERMARK duration_literal */ yytestcase(yyruleno==274); -{ ((SStreamOptions*)yymsp[-2].minor.yy840)->pWatermark = releaseRawExprNode(pCxt, yymsp[0].minor.yy840); yylhsminor.yy840 = yymsp[-2].minor.yy840; } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + case 272: /* stream_options ::= stream_options WATERMARK duration_literal */ yytestcase(yyruleno==272); +{ ((SStreamOptions*)yymsp[-2].minor.yy272)->pWatermark = releaseRawExprNode(pCxt, yymsp[0].minor.yy272); yylhsminor.yy272 = yymsp[-2].minor.yy272; } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 243: /* sma_stream_opt ::= stream_options MAX_DELAY duration_literal */ -{ ((SStreamOptions*)yymsp[-2].minor.yy840)->pDelay = releaseRawExprNode(pCxt, yymsp[0].minor.yy840); yylhsminor.yy840 = yymsp[-2].minor.yy840; } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ ((SStreamOptions*)yymsp[-2].minor.yy272)->pDelay = releaseRawExprNode(pCxt, yymsp[0].minor.yy272); yylhsminor.yy272 = yymsp[-2].minor.yy272; } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 244: /* cmd ::= CREATE TOPIC not_exists_opt topic_name AS query_expression */ -{ pCxt->pRootNode = createCreateTopicStmtUseQuery(pCxt, yymsp[-3].minor.yy313, &yymsp[-2].minor.yy617, yymsp[0].minor.yy840); } +{ pCxt->pRootNode = createCreateTopicStmtUseQuery(pCxt, yymsp[-3].minor.yy293, &yymsp[-2].minor.yy209, yymsp[0].minor.yy272); } break; case 245: /* cmd ::= CREATE TOPIC not_exists_opt topic_name AS DATABASE db_name */ -{ pCxt->pRootNode = createCreateTopicStmtUseDb(pCxt, yymsp[-4].minor.yy313, &yymsp[-3].minor.yy617, &yymsp[0].minor.yy617, false); } +{ pCxt->pRootNode = createCreateTopicStmtUseDb(pCxt, yymsp[-4].minor.yy293, &yymsp[-3].minor.yy209, &yymsp[0].minor.yy209, false); } break; case 246: /* cmd ::= CREATE TOPIC not_exists_opt topic_name WITH META AS DATABASE db_name */ -{ pCxt->pRootNode = createCreateTopicStmtUseDb(pCxt, yymsp[-6].minor.yy313, &yymsp[-5].minor.yy617, &yymsp[0].minor.yy617, true); } +{ pCxt->pRootNode = createCreateTopicStmtUseDb(pCxt, yymsp[-6].minor.yy293, &yymsp[-5].minor.yy209, &yymsp[0].minor.yy209, true); } break; case 247: /* cmd ::= CREATE TOPIC not_exists_opt topic_name AS STABLE full_table_name */ -{ pCxt->pRootNode = createCreateTopicStmtUseTable(pCxt, yymsp[-4].minor.yy313, &yymsp[-3].minor.yy617, yymsp[0].minor.yy840, false); } +{ pCxt->pRootNode = createCreateTopicStmtUseTable(pCxt, yymsp[-4].minor.yy293, &yymsp[-3].minor.yy209, yymsp[0].minor.yy272, false); } break; case 248: /* cmd ::= CREATE TOPIC not_exists_opt topic_name WITH META AS STABLE full_table_name */ -{ pCxt->pRootNode = createCreateTopicStmtUseTable(pCxt, yymsp[-6].minor.yy313, &yymsp[-5].minor.yy617, yymsp[0].minor.yy840, true); } +{ pCxt->pRootNode = createCreateTopicStmtUseTable(pCxt, yymsp[-6].minor.yy293, &yymsp[-5].minor.yy209, yymsp[0].minor.yy272, true); } break; case 249: /* cmd ::= DROP TOPIC exists_opt topic_name */ -{ pCxt->pRootNode = createDropTopicStmt(pCxt, yymsp[-1].minor.yy313, &yymsp[0].minor.yy617); } +{ pCxt->pRootNode = createDropTopicStmt(pCxt, yymsp[-1].minor.yy293, &yymsp[0].minor.yy209); } break; case 250: /* cmd ::= DROP CONSUMER GROUP exists_opt cgroup_name ON topic_name */ -{ pCxt->pRootNode = createDropCGroupStmt(pCxt, yymsp[-3].minor.yy313, &yymsp[-2].minor.yy617, &yymsp[0].minor.yy617); } +{ pCxt->pRootNode = createDropCGroupStmt(pCxt, yymsp[-3].minor.yy293, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209); } break; case 251: /* cmd ::= DESC full_table_name */ case 252: /* cmd ::= DESCRIBE full_table_name */ yytestcase(yyruleno==252); -{ pCxt->pRootNode = createDescribeStmt(pCxt, yymsp[0].minor.yy840); } +{ pCxt->pRootNode = createDescribeStmt(pCxt, yymsp[0].minor.yy272); } break; case 253: /* cmd ::= RESET QUERY CACHE */ { pCxt->pRootNode = createResetQueryCacheStmt(pCxt); } break; case 254: /* cmd ::= EXPLAIN analyze_opt explain_options query_expression */ -{ pCxt->pRootNode = createExplainStmt(pCxt, yymsp[-2].minor.yy313, yymsp[-1].minor.yy840, yymsp[0].minor.yy840); } +{ pCxt->pRootNode = createExplainStmt(pCxt, yymsp[-2].minor.yy293, yymsp[-1].minor.yy272, yymsp[0].minor.yy272); } break; case 256: /* analyze_opt ::= ANALYZE */ case 263: /* agg_func_opt ::= AGGREGATE */ yytestcase(yyruleno==263); - case 426: /* set_quantifier_opt ::= DISTINCT */ yytestcase(yyruleno==426); -{ yymsp[0].minor.yy313 = true; } + case 424: /* set_quantifier_opt ::= DISTINCT */ yytestcase(yyruleno==424); +{ yymsp[0].minor.yy293 = true; } break; case 257: /* explain_options ::= */ -{ yymsp[1].minor.yy840 = createDefaultExplainOptions(pCxt); } +{ yymsp[1].minor.yy272 = createDefaultExplainOptions(pCxt); } break; case 258: /* explain_options ::= explain_options VERBOSE NK_BOOL */ -{ yylhsminor.yy840 = setExplainVerbose(pCxt, yymsp[-2].minor.yy840, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setExplainVerbose(pCxt, yymsp[-2].minor.yy272, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 259: /* explain_options ::= explain_options RATIO NK_FLOAT */ -{ yylhsminor.yy840 = setExplainRatio(pCxt, yymsp[-2].minor.yy840, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; +{ yylhsminor.yy272 = setExplainRatio(pCxt, yymsp[-2].minor.yy272, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; case 260: /* cmd ::= CREATE agg_func_opt FUNCTION not_exists_opt function_name AS NK_STRING OUTPUTTYPE type_name bufsize_opt */ -{ pCxt->pRootNode = createCreateFunctionStmt(pCxt, yymsp[-6].minor.yy313, yymsp[-8].minor.yy313, &yymsp[-5].minor.yy617, &yymsp[-3].minor.yy0, yymsp[-1].minor.yy784, yymsp[0].minor.yy844); } +{ pCxt->pRootNode = createCreateFunctionStmt(pCxt, yymsp[-6].minor.yy293, yymsp[-8].minor.yy293, &yymsp[-5].minor.yy209, &yymsp[-3].minor.yy0, yymsp[-1].minor.yy616, yymsp[0].minor.yy232); } break; case 261: /* cmd ::= DROP FUNCTION exists_opt function_name */ -{ pCxt->pRootNode = createDropFunctionStmt(pCxt, yymsp[-1].minor.yy313, &yymsp[0].minor.yy617); } +{ pCxt->pRootNode = createDropFunctionStmt(pCxt, yymsp[-1].minor.yy293, &yymsp[0].minor.yy209); } break; case 264: /* bufsize_opt ::= */ -{ yymsp[1].minor.yy844 = 0; } +{ yymsp[1].minor.yy232 = 0; } break; case 265: /* bufsize_opt ::= BUFSIZE NK_INTEGER */ -{ yymsp[-1].minor.yy844 = taosStr2Int32(yymsp[0].minor.yy0.z, NULL, 10); } +{ yymsp[-1].minor.yy232 = taosStr2Int32(yymsp[0].minor.yy0.z, NULL, 10); } break; - case 266: /* cmd ::= CREATE STREAM not_exists_opt stream_name stream_options into_opt AS query_expression */ -{ pCxt->pRootNode = createCreateStreamStmt(pCxt, yymsp[-5].minor.yy313, &yymsp[-4].minor.yy617, yymsp[-2].minor.yy840, yymsp[-3].minor.yy840, yymsp[0].minor.yy840); } + case 266: /* cmd ::= CREATE STREAM not_exists_opt stream_name stream_options INTO full_table_name AS query_expression */ +{ pCxt->pRootNode = createCreateStreamStmt(pCxt, yymsp[-6].minor.yy293, &yymsp[-5].minor.yy209, yymsp[-2].minor.yy272, yymsp[-4].minor.yy272, yymsp[0].minor.yy272); } break; case 267: /* cmd ::= DROP STREAM exists_opt stream_name */ -{ pCxt->pRootNode = createDropStreamStmt(pCxt, yymsp[-1].minor.yy313, &yymsp[0].minor.yy617); } +{ pCxt->pRootNode = createDropStreamStmt(pCxt, yymsp[-1].minor.yy293, &yymsp[0].minor.yy209); } break; - case 269: /* into_opt ::= INTO full_table_name */ - case 407: /* from_clause_opt ::= FROM table_reference_list */ yytestcase(yyruleno==407); - case 436: /* where_clause_opt ::= WHERE search_condition */ yytestcase(yyruleno==436); - case 459: /* having_clause_opt ::= HAVING search_condition */ yytestcase(yyruleno==459); -{ yymsp[-1].minor.yy840 = yymsp[0].minor.yy840; } + case 269: /* stream_options ::= stream_options TRIGGER AT_ONCE */ +{ ((SStreamOptions*)yymsp[-2].minor.yy272)->triggerType = STREAM_TRIGGER_AT_ONCE; yylhsminor.yy272 = yymsp[-2].minor.yy272; } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 271: /* stream_options ::= stream_options TRIGGER AT_ONCE */ -{ ((SStreamOptions*)yymsp[-2].minor.yy840)->triggerType = STREAM_TRIGGER_AT_ONCE; yylhsminor.yy840 = yymsp[-2].minor.yy840; } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + case 270: /* stream_options ::= stream_options TRIGGER WINDOW_CLOSE */ +{ ((SStreamOptions*)yymsp[-2].minor.yy272)->triggerType = STREAM_TRIGGER_WINDOW_CLOSE; yylhsminor.yy272 = yymsp[-2].minor.yy272; } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 272: /* stream_options ::= stream_options TRIGGER WINDOW_CLOSE */ -{ ((SStreamOptions*)yymsp[-2].minor.yy840)->triggerType = STREAM_TRIGGER_WINDOW_CLOSE; yylhsminor.yy840 = yymsp[-2].minor.yy840; } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + case 271: /* stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal */ +{ ((SStreamOptions*)yymsp[-3].minor.yy272)->triggerType = STREAM_TRIGGER_MAX_DELAY; ((SStreamOptions*)yymsp[-3].minor.yy272)->pDelay = releaseRawExprNode(pCxt, yymsp[0].minor.yy272); yylhsminor.yy272 = yymsp[-3].minor.yy272; } + yymsp[-3].minor.yy272 = yylhsminor.yy272; break; - case 273: /* stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal */ -{ ((SStreamOptions*)yymsp[-3].minor.yy840)->triggerType = STREAM_TRIGGER_MAX_DELAY; ((SStreamOptions*)yymsp[-3].minor.yy840)->pDelay = releaseRawExprNode(pCxt, yymsp[0].minor.yy840); yylhsminor.yy840 = yymsp[-3].minor.yy840; } - yymsp[-3].minor.yy840 = yylhsminor.yy840; + case 273: /* stream_options ::= stream_options IGNORE EXPIRED NK_INTEGER */ +{ ((SStreamOptions*)yymsp[-3].minor.yy272)->ignoreExpired = taosStr2Int8(yymsp[0].minor.yy0.z, NULL, 10); yylhsminor.yy272 = yymsp[-3].minor.yy272; } + yymsp[-3].minor.yy272 = yylhsminor.yy272; break; - case 275: /* stream_options ::= stream_options IGNORE EXPIRED NK_INTEGER */ -{ ((SStreamOptions*)yymsp[-3].minor.yy840)->ignoreExpired = taosStr2Int8(yymsp[0].minor.yy0.z, NULL, 10); yylhsminor.yy840 = yymsp[-3].minor.yy840; } - yymsp[-3].minor.yy840 = yylhsminor.yy840; - break; - case 276: /* cmd ::= KILL CONNECTION NK_INTEGER */ + case 274: /* cmd ::= KILL CONNECTION NK_INTEGER */ { pCxt->pRootNode = createKillStmt(pCxt, QUERY_NODE_KILL_CONNECTION_STMT, &yymsp[0].minor.yy0); } break; - case 277: /* cmd ::= KILL QUERY NK_STRING */ + case 275: /* cmd ::= KILL QUERY NK_STRING */ { pCxt->pRootNode = createKillQueryStmt(pCxt, &yymsp[0].minor.yy0); } break; - case 278: /* cmd ::= KILL TRANSACTION NK_INTEGER */ + case 276: /* cmd ::= KILL TRANSACTION NK_INTEGER */ { pCxt->pRootNode = createKillStmt(pCxt, QUERY_NODE_KILL_TRANSACTION_STMT, &yymsp[0].minor.yy0); } break; - case 279: /* cmd ::= BALANCE VGROUP */ + case 277: /* cmd ::= BALANCE VGROUP */ { pCxt->pRootNode = createBalanceVgroupStmt(pCxt); } break; - case 280: /* cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER */ + case 278: /* cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER */ { pCxt->pRootNode = createMergeVgroupStmt(pCxt, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy0); } break; - case 281: /* cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list */ -{ pCxt->pRootNode = createRedistributeVgroupStmt(pCxt, &yymsp[-1].minor.yy0, yymsp[0].minor.yy544); } + case 279: /* cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list */ +{ pCxt->pRootNode = createRedistributeVgroupStmt(pCxt, &yymsp[-1].minor.yy0, yymsp[0].minor.yy172); } break; - case 282: /* cmd ::= SPLIT VGROUP NK_INTEGER */ + case 280: /* cmd ::= SPLIT VGROUP NK_INTEGER */ { pCxt->pRootNode = createSplitVgroupStmt(pCxt, &yymsp[0].minor.yy0); } break; - case 283: /* dnode_list ::= DNODE NK_INTEGER */ -{ yymsp[-1].minor.yy544 = createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } + case 281: /* dnode_list ::= DNODE NK_INTEGER */ +{ yymsp[-1].minor.yy172 = createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } break; - case 285: /* cmd ::= DELETE FROM full_table_name where_clause_opt */ -{ pCxt->pRootNode = createDeleteStmt(pCxt, yymsp[-1].minor.yy840, yymsp[0].minor.yy840); } + case 283: /* cmd ::= DELETE FROM full_table_name where_clause_opt */ +{ pCxt->pRootNode = createDeleteStmt(pCxt, yymsp[-1].minor.yy272, yymsp[0].minor.yy272); } break; - case 287: /* cmd ::= INSERT INTO full_table_name NK_LP col_name_list NK_RP query_expression */ -{ pCxt->pRootNode = createInsertStmt(pCxt, yymsp[-4].minor.yy840, yymsp[-2].minor.yy544, yymsp[0].minor.yy840); } + case 285: /* cmd ::= INSERT INTO full_table_name NK_LP col_name_list NK_RP query_expression */ +{ pCxt->pRootNode = createInsertStmt(pCxt, yymsp[-4].minor.yy272, yymsp[-2].minor.yy172, yymsp[0].minor.yy272); } break; - case 288: /* cmd ::= INSERT INTO full_table_name query_expression */ -{ pCxt->pRootNode = createInsertStmt(pCxt, yymsp[-1].minor.yy840, NULL, yymsp[0].minor.yy840); } + case 286: /* cmd ::= INSERT INTO full_table_name query_expression */ +{ pCxt->pRootNode = createInsertStmt(pCxt, yymsp[-1].minor.yy272, NULL, yymsp[0].minor.yy272); } break; - case 289: /* literal ::= NK_INTEGER */ -{ yylhsminor.yy840 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_UBIGINT, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 287: /* literal ::= NK_INTEGER */ +{ yylhsminor.yy272 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_UBIGINT, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 290: /* literal ::= NK_FLOAT */ -{ yylhsminor.yy840 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 288: /* literal ::= NK_FLOAT */ +{ yylhsminor.yy272 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 291: /* literal ::= NK_STRING */ -{ yylhsminor.yy840 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 289: /* literal ::= NK_STRING */ +{ yylhsminor.yy272 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 292: /* literal ::= NK_BOOL */ -{ yylhsminor.yy840 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_BOOL, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 290: /* literal ::= NK_BOOL */ +{ yylhsminor.yy272 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_BOOL, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 293: /* literal ::= TIMESTAMP NK_STRING */ -{ yylhsminor.yy840 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_TIMESTAMP, &yymsp[0].minor.yy0)); } - yymsp[-1].minor.yy840 = yylhsminor.yy840; + case 291: /* literal ::= TIMESTAMP NK_STRING */ +{ yylhsminor.yy272 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_TIMESTAMP, &yymsp[0].minor.yy0)); } + yymsp[-1].minor.yy272 = yylhsminor.yy272; break; - case 294: /* literal ::= duration_literal */ - case 304: /* signed_literal ::= signed */ yytestcase(yyruleno==304); - case 324: /* expression ::= literal */ yytestcase(yyruleno==324); - case 325: /* expression ::= pseudo_column */ yytestcase(yyruleno==325); - case 326: /* expression ::= column_reference */ yytestcase(yyruleno==326); - case 327: /* expression ::= function_expression */ yytestcase(yyruleno==327); - case 328: /* expression ::= subquery */ yytestcase(yyruleno==328); - case 356: /* function_expression ::= literal_func */ yytestcase(yyruleno==356); - case 398: /* boolean_value_expression ::= boolean_primary */ yytestcase(yyruleno==398); - case 402: /* boolean_primary ::= predicate */ yytestcase(yyruleno==402); - case 404: /* common_expression ::= expression */ yytestcase(yyruleno==404); - case 405: /* common_expression ::= boolean_value_expression */ yytestcase(yyruleno==405); - case 408: /* table_reference_list ::= table_reference */ yytestcase(yyruleno==408); - case 410: /* table_reference ::= table_primary */ yytestcase(yyruleno==410); - case 411: /* table_reference ::= joined_table */ yytestcase(yyruleno==411); - case 415: /* table_primary ::= parenthesized_joined_table */ yytestcase(yyruleno==415); - case 465: /* query_expression_body ::= query_primary */ yytestcase(yyruleno==465); - case 468: /* query_primary ::= query_specification */ yytestcase(yyruleno==468); -{ yylhsminor.yy840 = yymsp[0].minor.yy840; } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 292: /* literal ::= duration_literal */ + case 302: /* signed_literal ::= signed */ yytestcase(yyruleno==302); + case 322: /* expression ::= literal */ yytestcase(yyruleno==322); + case 323: /* expression ::= pseudo_column */ yytestcase(yyruleno==323); + case 324: /* expression ::= column_reference */ yytestcase(yyruleno==324); + case 325: /* expression ::= function_expression */ yytestcase(yyruleno==325); + case 326: /* expression ::= subquery */ yytestcase(yyruleno==326); + case 354: /* function_expression ::= literal_func */ yytestcase(yyruleno==354); + case 396: /* boolean_value_expression ::= boolean_primary */ yytestcase(yyruleno==396); + case 400: /* boolean_primary ::= predicate */ yytestcase(yyruleno==400); + case 402: /* common_expression ::= expression */ yytestcase(yyruleno==402); + case 403: /* common_expression ::= boolean_value_expression */ yytestcase(yyruleno==403); + case 406: /* table_reference_list ::= table_reference */ yytestcase(yyruleno==406); + case 408: /* table_reference ::= table_primary */ yytestcase(yyruleno==408); + case 409: /* table_reference ::= joined_table */ yytestcase(yyruleno==409); + case 413: /* table_primary ::= parenthesized_joined_table */ yytestcase(yyruleno==413); + case 463: /* query_expression_body ::= query_primary */ yytestcase(yyruleno==463); + case 466: /* query_primary ::= query_specification */ yytestcase(yyruleno==466); +{ yylhsminor.yy272 = yymsp[0].minor.yy272; } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 295: /* literal ::= NULL */ -{ yylhsminor.yy840 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_NULL, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 293: /* literal ::= NULL */ +{ yylhsminor.yy272 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_NULL, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 296: /* literal ::= NK_QUESTION */ -{ yylhsminor.yy840 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createPlaceholderValueNode(pCxt, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 294: /* literal ::= NK_QUESTION */ +{ yylhsminor.yy272 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createPlaceholderValueNode(pCxt, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 297: /* duration_literal ::= NK_VARIABLE */ -{ yylhsminor.yy840 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 295: /* duration_literal ::= NK_VARIABLE */ +{ yylhsminor.yy272 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 298: /* signed ::= NK_INTEGER */ -{ yylhsminor.yy840 = createValueNode(pCxt, TSDB_DATA_TYPE_UBIGINT, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 296: /* signed ::= NK_INTEGER */ +{ yylhsminor.yy272 = createValueNode(pCxt, TSDB_DATA_TYPE_UBIGINT, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 299: /* signed ::= NK_PLUS NK_INTEGER */ -{ yymsp[-1].minor.yy840 = createValueNode(pCxt, TSDB_DATA_TYPE_UBIGINT, &yymsp[0].minor.yy0); } + case 297: /* signed ::= NK_PLUS NK_INTEGER */ +{ yymsp[-1].minor.yy272 = createValueNode(pCxt, TSDB_DATA_TYPE_UBIGINT, &yymsp[0].minor.yy0); } break; - case 300: /* signed ::= NK_MINUS NK_INTEGER */ + case 298: /* signed ::= NK_MINUS NK_INTEGER */ { SToken t = yymsp[-1].minor.yy0; t.n = (yymsp[0].minor.yy0.z + yymsp[0].minor.yy0.n) - yymsp[-1].minor.yy0.z; - yylhsminor.yy840 = createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &t); + yylhsminor.yy272 = createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &t); } - yymsp[-1].minor.yy840 = yylhsminor.yy840; + yymsp[-1].minor.yy272 = yylhsminor.yy272; break; - case 301: /* signed ::= NK_FLOAT */ -{ yylhsminor.yy840 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 299: /* signed ::= NK_FLOAT */ +{ yylhsminor.yy272 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 302: /* signed ::= NK_PLUS NK_FLOAT */ -{ yymsp[-1].minor.yy840 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0); } + case 300: /* signed ::= NK_PLUS NK_FLOAT */ +{ yymsp[-1].minor.yy272 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0); } break; - case 303: /* signed ::= NK_MINUS NK_FLOAT */ + case 301: /* signed ::= NK_MINUS NK_FLOAT */ { SToken t = yymsp[-1].minor.yy0; t.n = (yymsp[0].minor.yy0.z + yymsp[0].minor.yy0.n) - yymsp[-1].minor.yy0.z; - yylhsminor.yy840 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &t); + yylhsminor.yy272 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &t); } - yymsp[-1].minor.yy840 = yylhsminor.yy840; + yymsp[-1].minor.yy272 = yylhsminor.yy272; break; - case 305: /* signed_literal ::= NK_STRING */ -{ yylhsminor.yy840 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 303: /* signed_literal ::= NK_STRING */ +{ yylhsminor.yy272 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 306: /* signed_literal ::= NK_BOOL */ -{ yylhsminor.yy840 = createValueNode(pCxt, TSDB_DATA_TYPE_BOOL, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 304: /* signed_literal ::= NK_BOOL */ +{ yylhsminor.yy272 = createValueNode(pCxt, TSDB_DATA_TYPE_BOOL, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 307: /* signed_literal ::= TIMESTAMP NK_STRING */ -{ yymsp[-1].minor.yy840 = createValueNode(pCxt, TSDB_DATA_TYPE_TIMESTAMP, &yymsp[0].minor.yy0); } + case 305: /* signed_literal ::= TIMESTAMP NK_STRING */ +{ yymsp[-1].minor.yy272 = createValueNode(pCxt, TSDB_DATA_TYPE_TIMESTAMP, &yymsp[0].minor.yy0); } break; - case 308: /* signed_literal ::= duration_literal */ - case 310: /* signed_literal ::= literal_func */ yytestcase(yyruleno==310); - case 376: /* star_func_para ::= expression */ yytestcase(yyruleno==376); - case 431: /* select_item ::= common_expression */ yytestcase(yyruleno==431); - case 481: /* search_condition ::= common_expression */ yytestcase(yyruleno==481); -{ yylhsminor.yy840 = releaseRawExprNode(pCxt, yymsp[0].minor.yy840); } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 306: /* signed_literal ::= duration_literal */ + case 308: /* signed_literal ::= literal_func */ yytestcase(yyruleno==308); + case 374: /* star_func_para ::= expression */ yytestcase(yyruleno==374); + case 429: /* select_item ::= common_expression */ yytestcase(yyruleno==429); + case 479: /* search_condition ::= common_expression */ yytestcase(yyruleno==479); +{ yylhsminor.yy272 = releaseRawExprNode(pCxt, yymsp[0].minor.yy272); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 309: /* signed_literal ::= NULL */ -{ yylhsminor.yy840 = createValueNode(pCxt, TSDB_DATA_TYPE_NULL, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 307: /* signed_literal ::= NULL */ +{ yylhsminor.yy272 = createValueNode(pCxt, TSDB_DATA_TYPE_NULL, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 311: /* signed_literal ::= NK_QUESTION */ -{ yylhsminor.yy840 = createPlaceholderValueNode(pCxt, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 309: /* signed_literal ::= NK_QUESTION */ +{ yylhsminor.yy272 = createPlaceholderValueNode(pCxt, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 329: /* expression ::= NK_LP expression NK_RP */ - case 403: /* boolean_primary ::= NK_LP boolean_value_expression NK_RP */ yytestcase(yyruleno==403); -{ yylhsminor.yy840 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, releaseRawExprNode(pCxt, yymsp[-1].minor.yy840)); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + case 327: /* expression ::= NK_LP expression NK_RP */ + case 401: /* boolean_primary ::= NK_LP boolean_value_expression NK_RP */ yytestcase(yyruleno==401); +{ yylhsminor.yy272 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, releaseRawExprNode(pCxt, yymsp[-1].minor.yy272)); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 330: /* expression ::= NK_PLUS expression */ + case 328: /* expression ::= NK_PLUS expression */ { - SToken t = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &t, releaseRawExprNode(pCxt, yymsp[0].minor.yy840)); + SToken t = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &t, releaseRawExprNode(pCxt, yymsp[0].minor.yy272)); } - yymsp[-1].minor.yy840 = yylhsminor.yy840; + yymsp[-1].minor.yy272 = yylhsminor.yy272; break; - case 331: /* expression ::= NK_MINUS expression */ + case 329: /* expression ::= NK_MINUS expression */ { - SToken t = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &t, createOperatorNode(pCxt, OP_TYPE_MINUS, releaseRawExprNode(pCxt, yymsp[0].minor.yy840), NULL)); + SToken t = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &t, createOperatorNode(pCxt, OP_TYPE_MINUS, releaseRawExprNode(pCxt, yymsp[0].minor.yy272), NULL)); } - yymsp[-1].minor.yy840 = yylhsminor.yy840; + yymsp[-1].minor.yy272 = yylhsminor.yy272; break; - case 332: /* expression ::= expression NK_PLUS expression */ + case 330: /* expression ::= expression NK_PLUS expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy840); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_ADD, releaseRawExprNode(pCxt, yymsp[-2].minor.yy840), releaseRawExprNode(pCxt, yymsp[0].minor.yy840))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy272); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_ADD, releaseRawExprNode(pCxt, yymsp[-2].minor.yy272), releaseRawExprNode(pCxt, yymsp[0].minor.yy272))); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 333: /* expression ::= expression NK_MINUS expression */ + case 331: /* expression ::= expression NK_MINUS expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy840); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_SUB, releaseRawExprNode(pCxt, yymsp[-2].minor.yy840), releaseRawExprNode(pCxt, yymsp[0].minor.yy840))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy272); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_SUB, releaseRawExprNode(pCxt, yymsp[-2].minor.yy272), releaseRawExprNode(pCxt, yymsp[0].minor.yy272))); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 334: /* expression ::= expression NK_STAR expression */ + case 332: /* expression ::= expression NK_STAR expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy840); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_MULTI, releaseRawExprNode(pCxt, yymsp[-2].minor.yy840), releaseRawExprNode(pCxt, yymsp[0].minor.yy840))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy272); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_MULTI, releaseRawExprNode(pCxt, yymsp[-2].minor.yy272), releaseRawExprNode(pCxt, yymsp[0].minor.yy272))); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 335: /* expression ::= expression NK_SLASH expression */ + case 333: /* expression ::= expression NK_SLASH expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy840); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_DIV, releaseRawExprNode(pCxt, yymsp[-2].minor.yy840), releaseRawExprNode(pCxt, yymsp[0].minor.yy840))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy272); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_DIV, releaseRawExprNode(pCxt, yymsp[-2].minor.yy272), releaseRawExprNode(pCxt, yymsp[0].minor.yy272))); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 336: /* expression ::= expression NK_REM expression */ + case 334: /* expression ::= expression NK_REM expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy840); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_REM, releaseRawExprNode(pCxt, yymsp[-2].minor.yy840), releaseRawExprNode(pCxt, yymsp[0].minor.yy840))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy272); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_REM, releaseRawExprNode(pCxt, yymsp[-2].minor.yy272), releaseRawExprNode(pCxt, yymsp[0].minor.yy272))); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 337: /* expression ::= column_reference NK_ARROW NK_STRING */ + case 335: /* expression ::= column_reference NK_ARROW NK_STRING */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_JSON_GET_VALUE, releaseRawExprNode(pCxt, yymsp[-2].minor.yy840), createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_JSON_GET_VALUE, releaseRawExprNode(pCxt, yymsp[-2].minor.yy272), createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0))); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 338: /* expression ::= expression NK_BITAND expression */ + case 336: /* expression ::= expression NK_BITAND expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy840); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_BIT_AND, releaseRawExprNode(pCxt, yymsp[-2].minor.yy840), releaseRawExprNode(pCxt, yymsp[0].minor.yy840))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy272); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_BIT_AND, releaseRawExprNode(pCxt, yymsp[-2].minor.yy272), releaseRawExprNode(pCxt, yymsp[0].minor.yy272))); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 339: /* expression ::= expression NK_BITOR expression */ + case 337: /* expression ::= expression NK_BITOR expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy840); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_BIT_OR, releaseRawExprNode(pCxt, yymsp[-2].minor.yy840), releaseRawExprNode(pCxt, yymsp[0].minor.yy840))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy272); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_BIT_OR, releaseRawExprNode(pCxt, yymsp[-2].minor.yy272), releaseRawExprNode(pCxt, yymsp[0].minor.yy272))); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 342: /* column_reference ::= column_name */ -{ yylhsminor.yy840 = createRawExprNode(pCxt, &yymsp[0].minor.yy617, createColumnNode(pCxt, NULL, &yymsp[0].minor.yy617)); } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 340: /* column_reference ::= column_name */ +{ yylhsminor.yy272 = createRawExprNode(pCxt, &yymsp[0].minor.yy209, createColumnNode(pCxt, NULL, &yymsp[0].minor.yy209)); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 343: /* column_reference ::= table_name NK_DOT column_name */ -{ yylhsminor.yy840 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy617, &yymsp[0].minor.yy617, createColumnNode(pCxt, &yymsp[-2].minor.yy617, &yymsp[0].minor.yy617)); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + case 341: /* column_reference ::= table_name NK_DOT column_name */ +{ yylhsminor.yy272 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209, createColumnNode(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209)); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 344: /* pseudo_column ::= ROWTS */ - case 345: /* pseudo_column ::= TBNAME */ yytestcase(yyruleno==345); - case 347: /* pseudo_column ::= QSTART */ yytestcase(yyruleno==347); - case 348: /* pseudo_column ::= QEND */ yytestcase(yyruleno==348); - case 349: /* pseudo_column ::= QDURATION */ yytestcase(yyruleno==349); - case 350: /* pseudo_column ::= WSTART */ yytestcase(yyruleno==350); - case 351: /* pseudo_column ::= WEND */ yytestcase(yyruleno==351); - case 352: /* pseudo_column ::= WDURATION */ yytestcase(yyruleno==352); - case 358: /* literal_func ::= NOW */ yytestcase(yyruleno==358); -{ yylhsminor.yy840 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[0].minor.yy0, NULL)); } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 342: /* pseudo_column ::= ROWTS */ + case 343: /* pseudo_column ::= TBNAME */ yytestcase(yyruleno==343); + case 345: /* pseudo_column ::= QSTART */ yytestcase(yyruleno==345); + case 346: /* pseudo_column ::= QEND */ yytestcase(yyruleno==346); + case 347: /* pseudo_column ::= QDURATION */ yytestcase(yyruleno==347); + case 348: /* pseudo_column ::= WSTART */ yytestcase(yyruleno==348); + case 349: /* pseudo_column ::= WEND */ yytestcase(yyruleno==349); + case 350: /* pseudo_column ::= WDURATION */ yytestcase(yyruleno==350); + case 356: /* literal_func ::= NOW */ yytestcase(yyruleno==356); +{ yylhsminor.yy272 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[0].minor.yy0, NULL)); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 346: /* pseudo_column ::= table_name NK_DOT TBNAME */ -{ yylhsminor.yy840 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy617, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[0].minor.yy0, createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[-2].minor.yy617)))); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + case 344: /* pseudo_column ::= table_name NK_DOT TBNAME */ +{ yylhsminor.yy272 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[0].minor.yy0, createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[-2].minor.yy209)))); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 353: /* function_expression ::= function_name NK_LP expression_list NK_RP */ - case 354: /* function_expression ::= star_func NK_LP star_func_para_list NK_RP */ yytestcase(yyruleno==354); -{ yylhsminor.yy840 = createRawExprNodeExt(pCxt, &yymsp[-3].minor.yy617, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[-3].minor.yy617, yymsp[-1].minor.yy544)); } - yymsp[-3].minor.yy840 = yylhsminor.yy840; + case 351: /* function_expression ::= function_name NK_LP expression_list NK_RP */ + case 352: /* function_expression ::= star_func NK_LP star_func_para_list NK_RP */ yytestcase(yyruleno==352); +{ yylhsminor.yy272 = createRawExprNodeExt(pCxt, &yymsp[-3].minor.yy209, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[-3].minor.yy209, yymsp[-1].minor.yy172)); } + yymsp[-3].minor.yy272 = yylhsminor.yy272; break; - case 355: /* function_expression ::= CAST NK_LP expression AS type_name NK_RP */ -{ yylhsminor.yy840 = createRawExprNodeExt(pCxt, &yymsp[-5].minor.yy0, &yymsp[0].minor.yy0, createCastFunctionNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy840), yymsp[-1].minor.yy784)); } - yymsp[-5].minor.yy840 = yylhsminor.yy840; + case 353: /* function_expression ::= CAST NK_LP expression AS type_name NK_RP */ +{ yylhsminor.yy272 = createRawExprNodeExt(pCxt, &yymsp[-5].minor.yy0, &yymsp[0].minor.yy0, createCastFunctionNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy272), yymsp[-1].minor.yy616)); } + yymsp[-5].minor.yy272 = yylhsminor.yy272; break; - case 357: /* literal_func ::= noarg_func NK_LP NK_RP */ -{ yylhsminor.yy840 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy617, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[-2].minor.yy617, NULL)); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + case 355: /* literal_func ::= noarg_func NK_LP NK_RP */ +{ yylhsminor.yy272 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[-2].minor.yy209, NULL)); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 372: /* star_func_para_list ::= NK_STAR */ -{ yylhsminor.yy544 = createNodeList(pCxt, createColumnNode(pCxt, NULL, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy544 = yylhsminor.yy544; + case 370: /* star_func_para_list ::= NK_STAR */ +{ yylhsminor.yy172 = createNodeList(pCxt, createColumnNode(pCxt, NULL, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy172 = yylhsminor.yy172; break; - case 377: /* star_func_para ::= table_name NK_DOT NK_STAR */ - case 434: /* select_item ::= table_name NK_DOT NK_STAR */ yytestcase(yyruleno==434); -{ yylhsminor.yy840 = createColumnNode(pCxt, &yymsp[-2].minor.yy617, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + case 375: /* star_func_para ::= table_name NK_DOT NK_STAR */ + case 432: /* select_item ::= table_name NK_DOT NK_STAR */ yytestcase(yyruleno==432); +{ yylhsminor.yy272 = createColumnNode(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 378: /* predicate ::= expression compare_op expression */ - case 383: /* predicate ::= expression in_op in_predicate_value */ yytestcase(yyruleno==383); + case 376: /* predicate ::= expression compare_op expression */ + case 381: /* predicate ::= expression in_op in_predicate_value */ yytestcase(yyruleno==381); { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy840); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, yymsp[-1].minor.yy198, releaseRawExprNode(pCxt, yymsp[-2].minor.yy840), releaseRawExprNode(pCxt, yymsp[0].minor.yy840))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy272); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, yymsp[-1].minor.yy392, releaseRawExprNode(pCxt, yymsp[-2].minor.yy272), releaseRawExprNode(pCxt, yymsp[0].minor.yy272))); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 379: /* predicate ::= expression BETWEEN expression AND expression */ + case 377: /* predicate ::= expression BETWEEN expression AND expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-4].minor.yy840); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &s, &e, createBetweenAnd(pCxt, releaseRawExprNode(pCxt, yymsp[-4].minor.yy840), releaseRawExprNode(pCxt, yymsp[-2].minor.yy840), releaseRawExprNode(pCxt, yymsp[0].minor.yy840))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-4].minor.yy272); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &s, &e, createBetweenAnd(pCxt, releaseRawExprNode(pCxt, yymsp[-4].minor.yy272), releaseRawExprNode(pCxt, yymsp[-2].minor.yy272), releaseRawExprNode(pCxt, yymsp[0].minor.yy272))); } - yymsp[-4].minor.yy840 = yylhsminor.yy840; + yymsp[-4].minor.yy272 = yylhsminor.yy272; break; - case 380: /* predicate ::= expression NOT BETWEEN expression AND expression */ + case 378: /* predicate ::= expression NOT BETWEEN expression AND expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-5].minor.yy840); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &s, &e, createNotBetweenAnd(pCxt, releaseRawExprNode(pCxt, yymsp[-5].minor.yy840), releaseRawExprNode(pCxt, yymsp[-2].minor.yy840), releaseRawExprNode(pCxt, yymsp[0].minor.yy840))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-5].minor.yy272); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &s, &e, createNotBetweenAnd(pCxt, releaseRawExprNode(pCxt, yymsp[-5].minor.yy272), releaseRawExprNode(pCxt, yymsp[-2].minor.yy272), releaseRawExprNode(pCxt, yymsp[0].minor.yy272))); } - yymsp[-5].minor.yy840 = yylhsminor.yy840; + yymsp[-5].minor.yy272 = yylhsminor.yy272; break; - case 381: /* predicate ::= expression IS NULL */ + case 379: /* predicate ::= expression IS NULL */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_IS_NULL, releaseRawExprNode(pCxt, yymsp[-2].minor.yy840), NULL)); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_IS_NULL, releaseRawExprNode(pCxt, yymsp[-2].minor.yy272), NULL)); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 382: /* predicate ::= expression IS NOT NULL */ + case 380: /* predicate ::= expression IS NOT NULL */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-3].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_IS_NOT_NULL, releaseRawExprNode(pCxt, yymsp[-3].minor.yy840), NULL)); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-3].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_IS_NOT_NULL, releaseRawExprNode(pCxt, yymsp[-3].minor.yy272), NULL)); } - yymsp[-3].minor.yy840 = yylhsminor.yy840; + yymsp[-3].minor.yy272 = yylhsminor.yy272; break; - case 384: /* compare_op ::= NK_LT */ -{ yymsp[0].minor.yy198 = OP_TYPE_LOWER_THAN; } + case 382: /* compare_op ::= NK_LT */ +{ yymsp[0].minor.yy392 = OP_TYPE_LOWER_THAN; } break; - case 385: /* compare_op ::= NK_GT */ -{ yymsp[0].minor.yy198 = OP_TYPE_GREATER_THAN; } + case 383: /* compare_op ::= NK_GT */ +{ yymsp[0].minor.yy392 = OP_TYPE_GREATER_THAN; } break; - case 386: /* compare_op ::= NK_LE */ -{ yymsp[0].minor.yy198 = OP_TYPE_LOWER_EQUAL; } + case 384: /* compare_op ::= NK_LE */ +{ yymsp[0].minor.yy392 = OP_TYPE_LOWER_EQUAL; } break; - case 387: /* compare_op ::= NK_GE */ -{ yymsp[0].minor.yy198 = OP_TYPE_GREATER_EQUAL; } + case 385: /* compare_op ::= NK_GE */ +{ yymsp[0].minor.yy392 = OP_TYPE_GREATER_EQUAL; } break; - case 388: /* compare_op ::= NK_NE */ -{ yymsp[0].minor.yy198 = OP_TYPE_NOT_EQUAL; } + case 386: /* compare_op ::= NK_NE */ +{ yymsp[0].minor.yy392 = OP_TYPE_NOT_EQUAL; } break; - case 389: /* compare_op ::= NK_EQ */ -{ yymsp[0].minor.yy198 = OP_TYPE_EQUAL; } + case 387: /* compare_op ::= NK_EQ */ +{ yymsp[0].minor.yy392 = OP_TYPE_EQUAL; } break; - case 390: /* compare_op ::= LIKE */ -{ yymsp[0].minor.yy198 = OP_TYPE_LIKE; } + case 388: /* compare_op ::= LIKE */ +{ yymsp[0].minor.yy392 = OP_TYPE_LIKE; } break; - case 391: /* compare_op ::= NOT LIKE */ -{ yymsp[-1].minor.yy198 = OP_TYPE_NOT_LIKE; } + case 389: /* compare_op ::= NOT LIKE */ +{ yymsp[-1].minor.yy392 = OP_TYPE_NOT_LIKE; } break; - case 392: /* compare_op ::= MATCH */ -{ yymsp[0].minor.yy198 = OP_TYPE_MATCH; } + case 390: /* compare_op ::= MATCH */ +{ yymsp[0].minor.yy392 = OP_TYPE_MATCH; } break; - case 393: /* compare_op ::= NMATCH */ -{ yymsp[0].minor.yy198 = OP_TYPE_NMATCH; } + case 391: /* compare_op ::= NMATCH */ +{ yymsp[0].minor.yy392 = OP_TYPE_NMATCH; } break; - case 394: /* compare_op ::= CONTAINS */ -{ yymsp[0].minor.yy198 = OP_TYPE_JSON_CONTAINS; } + case 392: /* compare_op ::= CONTAINS */ +{ yymsp[0].minor.yy392 = OP_TYPE_JSON_CONTAINS; } break; - case 395: /* in_op ::= IN */ -{ yymsp[0].minor.yy198 = OP_TYPE_IN; } + case 393: /* in_op ::= IN */ +{ yymsp[0].minor.yy392 = OP_TYPE_IN; } break; - case 396: /* in_op ::= NOT IN */ -{ yymsp[-1].minor.yy198 = OP_TYPE_NOT_IN; } + case 394: /* in_op ::= NOT IN */ +{ yymsp[-1].minor.yy392 = OP_TYPE_NOT_IN; } break; - case 397: /* in_predicate_value ::= NK_LP literal_list NK_RP */ -{ yylhsminor.yy840 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, createNodeListNode(pCxt, yymsp[-1].minor.yy544)); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + case 395: /* in_predicate_value ::= NK_LP literal_list NK_RP */ +{ yylhsminor.yy272 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, createNodeListNode(pCxt, yymsp[-1].minor.yy172)); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 399: /* boolean_value_expression ::= NOT boolean_primary */ + case 397: /* boolean_value_expression ::= NOT boolean_primary */ { - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_NOT, releaseRawExprNode(pCxt, yymsp[0].minor.yy840), NULL)); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_NOT, releaseRawExprNode(pCxt, yymsp[0].minor.yy272), NULL)); } - yymsp[-1].minor.yy840 = yylhsminor.yy840; + yymsp[-1].minor.yy272 = yylhsminor.yy272; break; - case 400: /* boolean_value_expression ::= boolean_value_expression OR boolean_value_expression */ + case 398: /* boolean_value_expression ::= boolean_value_expression OR boolean_value_expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy840); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &s, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_OR, releaseRawExprNode(pCxt, yymsp[-2].minor.yy840), releaseRawExprNode(pCxt, yymsp[0].minor.yy840))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy272); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &s, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_OR, releaseRawExprNode(pCxt, yymsp[-2].minor.yy272), releaseRawExprNode(pCxt, yymsp[0].minor.yy272))); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 401: /* boolean_value_expression ::= boolean_value_expression AND boolean_value_expression */ + case 399: /* boolean_value_expression ::= boolean_value_expression AND boolean_value_expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy840); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy840); - yylhsminor.yy840 = createRawExprNodeExt(pCxt, &s, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_AND, releaseRawExprNode(pCxt, yymsp[-2].minor.yy840), releaseRawExprNode(pCxt, yymsp[0].minor.yy840))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy272); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy272); + yylhsminor.yy272 = createRawExprNodeExt(pCxt, &s, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_AND, releaseRawExprNode(pCxt, yymsp[-2].minor.yy272), releaseRawExprNode(pCxt, yymsp[0].minor.yy272))); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 409: /* table_reference_list ::= table_reference_list NK_COMMA table_reference */ -{ yylhsminor.yy840 = createJoinTableNode(pCxt, JOIN_TYPE_INNER, yymsp[-2].minor.yy840, yymsp[0].minor.yy840, NULL); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + case 405: /* from_clause_opt ::= FROM table_reference_list */ + case 434: /* where_clause_opt ::= WHERE search_condition */ yytestcase(yyruleno==434); + case 457: /* having_clause_opt ::= HAVING search_condition */ yytestcase(yyruleno==457); +{ yymsp[-1].minor.yy272 = yymsp[0].minor.yy272; } break; - case 412: /* table_primary ::= table_name alias_opt */ -{ yylhsminor.yy840 = createRealTableNode(pCxt, NULL, &yymsp[-1].minor.yy617, &yymsp[0].minor.yy617); } - yymsp[-1].minor.yy840 = yylhsminor.yy840; + case 407: /* table_reference_list ::= table_reference_list NK_COMMA table_reference */ +{ yylhsminor.yy272 = createJoinTableNode(pCxt, JOIN_TYPE_INNER, yymsp[-2].minor.yy272, yymsp[0].minor.yy272, NULL); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 413: /* table_primary ::= db_name NK_DOT table_name alias_opt */ -{ yylhsminor.yy840 = createRealTableNode(pCxt, &yymsp[-3].minor.yy617, &yymsp[-1].minor.yy617, &yymsp[0].minor.yy617); } - yymsp[-3].minor.yy840 = yylhsminor.yy840; + case 410: /* table_primary ::= table_name alias_opt */ +{ yylhsminor.yy272 = createRealTableNode(pCxt, NULL, &yymsp[-1].minor.yy209, &yymsp[0].minor.yy209); } + yymsp[-1].minor.yy272 = yylhsminor.yy272; break; - case 414: /* table_primary ::= subquery alias_opt */ -{ yylhsminor.yy840 = createTempTableNode(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy840), &yymsp[0].minor.yy617); } - yymsp[-1].minor.yy840 = yylhsminor.yy840; + case 411: /* table_primary ::= db_name NK_DOT table_name alias_opt */ +{ yylhsminor.yy272 = createRealTableNode(pCxt, &yymsp[-3].minor.yy209, &yymsp[-1].minor.yy209, &yymsp[0].minor.yy209); } + yymsp[-3].minor.yy272 = yylhsminor.yy272; break; - case 416: /* alias_opt ::= */ -{ yymsp[1].minor.yy617 = nil_token; } + case 412: /* table_primary ::= subquery alias_opt */ +{ yylhsminor.yy272 = createTempTableNode(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy272), &yymsp[0].minor.yy209); } + yymsp[-1].minor.yy272 = yylhsminor.yy272; break; - case 417: /* alias_opt ::= table_alias */ -{ yylhsminor.yy617 = yymsp[0].minor.yy617; } - yymsp[0].minor.yy617 = yylhsminor.yy617; + case 414: /* alias_opt ::= */ +{ yymsp[1].minor.yy209 = nil_token; } break; - case 418: /* alias_opt ::= AS table_alias */ -{ yymsp[-1].minor.yy617 = yymsp[0].minor.yy617; } + case 415: /* alias_opt ::= table_alias */ +{ yylhsminor.yy209 = yymsp[0].minor.yy209; } + yymsp[0].minor.yy209 = yylhsminor.yy209; break; - case 419: /* parenthesized_joined_table ::= NK_LP joined_table NK_RP */ - case 420: /* parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP */ yytestcase(yyruleno==420); -{ yymsp[-2].minor.yy840 = yymsp[-1].minor.yy840; } + case 416: /* alias_opt ::= AS table_alias */ +{ yymsp[-1].minor.yy209 = yymsp[0].minor.yy209; } break; - case 421: /* joined_table ::= table_reference join_type JOIN table_reference ON search_condition */ -{ yylhsminor.yy840 = createJoinTableNode(pCxt, yymsp[-4].minor.yy708, yymsp[-5].minor.yy840, yymsp[-2].minor.yy840, yymsp[0].minor.yy840); } - yymsp[-5].minor.yy840 = yylhsminor.yy840; + case 417: /* parenthesized_joined_table ::= NK_LP joined_table NK_RP */ + case 418: /* parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP */ yytestcase(yyruleno==418); +{ yymsp[-2].minor.yy272 = yymsp[-1].minor.yy272; } break; - case 422: /* join_type ::= */ -{ yymsp[1].minor.yy708 = JOIN_TYPE_INNER; } + case 419: /* joined_table ::= table_reference join_type JOIN table_reference ON search_condition */ +{ yylhsminor.yy272 = createJoinTableNode(pCxt, yymsp[-4].minor.yy156, yymsp[-5].minor.yy272, yymsp[-2].minor.yy272, yymsp[0].minor.yy272); } + yymsp[-5].minor.yy272 = yylhsminor.yy272; break; - case 423: /* join_type ::= INNER */ -{ yymsp[0].minor.yy708 = JOIN_TYPE_INNER; } + case 420: /* join_type ::= */ +{ yymsp[1].minor.yy156 = JOIN_TYPE_INNER; } break; - case 424: /* query_specification ::= SELECT set_quantifier_opt select_list from_clause_opt where_clause_opt partition_by_clause_opt range_opt every_opt fill_opt twindow_clause_opt group_by_clause_opt having_clause_opt */ + case 421: /* join_type ::= INNER */ +{ yymsp[0].minor.yy156 = JOIN_TYPE_INNER; } + break; + case 422: /* query_specification ::= SELECT set_quantifier_opt select_list from_clause_opt where_clause_opt partition_by_clause_opt range_opt every_opt fill_opt twindow_clause_opt group_by_clause_opt having_clause_opt */ { - yymsp[-11].minor.yy840 = createSelectStmt(pCxt, yymsp[-10].minor.yy313, yymsp[-9].minor.yy544, yymsp[-8].minor.yy840); - yymsp[-11].minor.yy840 = addWhereClause(pCxt, yymsp[-11].minor.yy840, yymsp[-7].minor.yy840); - yymsp[-11].minor.yy840 = addPartitionByClause(pCxt, yymsp[-11].minor.yy840, yymsp[-6].minor.yy544); - yymsp[-11].minor.yy840 = addWindowClauseClause(pCxt, yymsp[-11].minor.yy840, yymsp[-2].minor.yy840); - yymsp[-11].minor.yy840 = addGroupByClause(pCxt, yymsp[-11].minor.yy840, yymsp[-1].minor.yy544); - yymsp[-11].minor.yy840 = addHavingClause(pCxt, yymsp[-11].minor.yy840, yymsp[0].minor.yy840); - yymsp[-11].minor.yy840 = addRangeClause(pCxt, yymsp[-11].minor.yy840, yymsp[-5].minor.yy840); - yymsp[-11].minor.yy840 = addEveryClause(pCxt, yymsp[-11].minor.yy840, yymsp[-4].minor.yy840); - yymsp[-11].minor.yy840 = addFillClause(pCxt, yymsp[-11].minor.yy840, yymsp[-3].minor.yy840); + yymsp[-11].minor.yy272 = createSelectStmt(pCxt, yymsp[-10].minor.yy293, yymsp[-9].minor.yy172, yymsp[-8].minor.yy272); + yymsp[-11].minor.yy272 = addWhereClause(pCxt, yymsp[-11].minor.yy272, yymsp[-7].minor.yy272); + yymsp[-11].minor.yy272 = addPartitionByClause(pCxt, yymsp[-11].minor.yy272, yymsp[-6].minor.yy172); + yymsp[-11].minor.yy272 = addWindowClauseClause(pCxt, yymsp[-11].minor.yy272, yymsp[-2].minor.yy272); + yymsp[-11].minor.yy272 = addGroupByClause(pCxt, yymsp[-11].minor.yy272, yymsp[-1].minor.yy172); + yymsp[-11].minor.yy272 = addHavingClause(pCxt, yymsp[-11].minor.yy272, yymsp[0].minor.yy272); + yymsp[-11].minor.yy272 = addRangeClause(pCxt, yymsp[-11].minor.yy272, yymsp[-5].minor.yy272); + yymsp[-11].minor.yy272 = addEveryClause(pCxt, yymsp[-11].minor.yy272, yymsp[-4].minor.yy272); + yymsp[-11].minor.yy272 = addFillClause(pCxt, yymsp[-11].minor.yy272, yymsp[-3].minor.yy272); } break; - case 427: /* set_quantifier_opt ::= ALL */ -{ yymsp[0].minor.yy313 = false; } + case 425: /* set_quantifier_opt ::= ALL */ +{ yymsp[0].minor.yy293 = false; } break; - case 430: /* select_item ::= NK_STAR */ -{ yylhsminor.yy840 = createColumnNode(pCxt, NULL, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy840 = yylhsminor.yy840; + case 428: /* select_item ::= NK_STAR */ +{ yylhsminor.yy272 = createColumnNode(pCxt, NULL, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy272 = yylhsminor.yy272; break; - case 432: /* select_item ::= common_expression column_alias */ -{ yylhsminor.yy840 = setProjectionAlias(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy840), &yymsp[0].minor.yy617); } - yymsp[-1].minor.yy840 = yylhsminor.yy840; + case 430: /* select_item ::= common_expression column_alias */ +{ yylhsminor.yy272 = setProjectionAlias(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy272), &yymsp[0].minor.yy209); } + yymsp[-1].minor.yy272 = yylhsminor.yy272; break; - case 433: /* select_item ::= common_expression AS column_alias */ -{ yylhsminor.yy840 = setProjectionAlias(pCxt, releaseRawExprNode(pCxt, yymsp[-2].minor.yy840), &yymsp[0].minor.yy617); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + case 431: /* select_item ::= common_expression AS column_alias */ +{ yylhsminor.yy272 = setProjectionAlias(pCxt, releaseRawExprNode(pCxt, yymsp[-2].minor.yy272), &yymsp[0].minor.yy209); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 438: /* partition_by_clause_opt ::= PARTITION BY expression_list */ - case 455: /* group_by_clause_opt ::= GROUP BY group_by_list */ yytestcase(yyruleno==455); - case 471: /* order_by_clause_opt ::= ORDER BY sort_specification_list */ yytestcase(yyruleno==471); -{ yymsp[-2].minor.yy544 = yymsp[0].minor.yy544; } + case 436: /* partition_by_clause_opt ::= PARTITION BY expression_list */ + case 453: /* group_by_clause_opt ::= GROUP BY group_by_list */ yytestcase(yyruleno==453); + case 469: /* order_by_clause_opt ::= ORDER BY sort_specification_list */ yytestcase(yyruleno==469); +{ yymsp[-2].minor.yy172 = yymsp[0].minor.yy172; } break; - case 440: /* twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP */ -{ yymsp[-5].minor.yy840 = createSessionWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy840), releaseRawExprNode(pCxt, yymsp[-1].minor.yy840)); } + case 438: /* twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP */ +{ yymsp[-5].minor.yy272 = createSessionWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy272), releaseRawExprNode(pCxt, yymsp[-1].minor.yy272)); } break; - case 441: /* twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP */ -{ yymsp[-3].minor.yy840 = createStateWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy840)); } + case 439: /* twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP */ +{ yymsp[-3].minor.yy272 = createStateWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy272)); } break; - case 442: /* twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt */ -{ yymsp[-5].minor.yy840 = createIntervalWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy840), NULL, yymsp[-1].minor.yy840, yymsp[0].minor.yy840); } + case 440: /* twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt */ +{ yymsp[-5].minor.yy272 = createIntervalWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy272), NULL, yymsp[-1].minor.yy272, yymsp[0].minor.yy272); } break; - case 443: /* twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt */ -{ yymsp[-7].minor.yy840 = createIntervalWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-5].minor.yy840), releaseRawExprNode(pCxt, yymsp[-3].minor.yy840), yymsp[-1].minor.yy840, yymsp[0].minor.yy840); } + case 441: /* twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt */ +{ yymsp[-7].minor.yy272 = createIntervalWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-5].minor.yy272), releaseRawExprNode(pCxt, yymsp[-3].minor.yy272), yymsp[-1].minor.yy272, yymsp[0].minor.yy272); } break; - case 445: /* sliding_opt ::= SLIDING NK_LP duration_literal NK_RP */ - case 463: /* every_opt ::= EVERY NK_LP duration_literal NK_RP */ yytestcase(yyruleno==463); -{ yymsp[-3].minor.yy840 = releaseRawExprNode(pCxt, yymsp[-1].minor.yy840); } + case 443: /* sliding_opt ::= SLIDING NK_LP duration_literal NK_RP */ + case 461: /* every_opt ::= EVERY NK_LP duration_literal NK_RP */ yytestcase(yyruleno==461); +{ yymsp[-3].minor.yy272 = releaseRawExprNode(pCxt, yymsp[-1].minor.yy272); } break; - case 447: /* fill_opt ::= FILL NK_LP fill_mode NK_RP */ -{ yymsp[-3].minor.yy840 = createFillNode(pCxt, yymsp[-1].minor.yy816, NULL); } + case 445: /* fill_opt ::= FILL NK_LP fill_mode NK_RP */ +{ yymsp[-3].minor.yy272 = createFillNode(pCxt, yymsp[-1].minor.yy186, NULL); } break; - case 448: /* fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP */ -{ yymsp[-5].minor.yy840 = createFillNode(pCxt, FILL_MODE_VALUE, createNodeListNode(pCxt, yymsp[-1].minor.yy544)); } + case 446: /* fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP */ +{ yymsp[-5].minor.yy272 = createFillNode(pCxt, FILL_MODE_VALUE, createNodeListNode(pCxt, yymsp[-1].minor.yy172)); } break; - case 449: /* fill_mode ::= NONE */ -{ yymsp[0].minor.yy816 = FILL_MODE_NONE; } + case 447: /* fill_mode ::= NONE */ +{ yymsp[0].minor.yy186 = FILL_MODE_NONE; } break; - case 450: /* fill_mode ::= PREV */ -{ yymsp[0].minor.yy816 = FILL_MODE_PREV; } + case 448: /* fill_mode ::= PREV */ +{ yymsp[0].minor.yy186 = FILL_MODE_PREV; } break; - case 451: /* fill_mode ::= NULL */ -{ yymsp[0].minor.yy816 = FILL_MODE_NULL; } + case 449: /* fill_mode ::= NULL */ +{ yymsp[0].minor.yy186 = FILL_MODE_NULL; } break; - case 452: /* fill_mode ::= LINEAR */ -{ yymsp[0].minor.yy816 = FILL_MODE_LINEAR; } + case 450: /* fill_mode ::= LINEAR */ +{ yymsp[0].minor.yy186 = FILL_MODE_LINEAR; } break; - case 453: /* fill_mode ::= NEXT */ -{ yymsp[0].minor.yy816 = FILL_MODE_NEXT; } + case 451: /* fill_mode ::= NEXT */ +{ yymsp[0].minor.yy186 = FILL_MODE_NEXT; } break; - case 456: /* group_by_list ::= expression */ -{ yylhsminor.yy544 = createNodeList(pCxt, createGroupingSetNode(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy840))); } - yymsp[0].minor.yy544 = yylhsminor.yy544; + case 454: /* group_by_list ::= expression */ +{ yylhsminor.yy172 = createNodeList(pCxt, createGroupingSetNode(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy272))); } + yymsp[0].minor.yy172 = yylhsminor.yy172; break; - case 457: /* group_by_list ::= group_by_list NK_COMMA expression */ -{ yylhsminor.yy544 = addNodeToList(pCxt, yymsp[-2].minor.yy544, createGroupingSetNode(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy840))); } - yymsp[-2].minor.yy544 = yylhsminor.yy544; + case 455: /* group_by_list ::= group_by_list NK_COMMA expression */ +{ yylhsminor.yy172 = addNodeToList(pCxt, yymsp[-2].minor.yy172, createGroupingSetNode(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy272))); } + yymsp[-2].minor.yy172 = yylhsminor.yy172; break; - case 461: /* range_opt ::= RANGE NK_LP expression NK_COMMA expression NK_RP */ -{ yymsp[-5].minor.yy840 = createInterpTimeRange(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy840), releaseRawExprNode(pCxt, yymsp[-1].minor.yy840)); } + case 459: /* range_opt ::= RANGE NK_LP expression NK_COMMA expression NK_RP */ +{ yymsp[-5].minor.yy272 = createInterpTimeRange(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy272), releaseRawExprNode(pCxt, yymsp[-1].minor.yy272)); } break; - case 464: /* query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt */ + case 462: /* query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt */ { - yylhsminor.yy840 = addOrderByClause(pCxt, yymsp[-3].minor.yy840, yymsp[-2].minor.yy544); - yylhsminor.yy840 = addSlimitClause(pCxt, yylhsminor.yy840, yymsp[-1].minor.yy840); - yylhsminor.yy840 = addLimitClause(pCxt, yylhsminor.yy840, yymsp[0].minor.yy840); + yylhsminor.yy272 = addOrderByClause(pCxt, yymsp[-3].minor.yy272, yymsp[-2].minor.yy172); + yylhsminor.yy272 = addSlimitClause(pCxt, yylhsminor.yy272, yymsp[-1].minor.yy272); + yylhsminor.yy272 = addLimitClause(pCxt, yylhsminor.yy272, yymsp[0].minor.yy272); } - yymsp[-3].minor.yy840 = yylhsminor.yy840; + yymsp[-3].minor.yy272 = yylhsminor.yy272; break; - case 466: /* query_expression_body ::= query_expression_body UNION ALL query_expression_body */ -{ yylhsminor.yy840 = createSetOperator(pCxt, SET_OP_TYPE_UNION_ALL, yymsp[-3].minor.yy840, yymsp[0].minor.yy840); } - yymsp[-3].minor.yy840 = yylhsminor.yy840; + case 464: /* query_expression_body ::= query_expression_body UNION ALL query_expression_body */ +{ yylhsminor.yy272 = createSetOperator(pCxt, SET_OP_TYPE_UNION_ALL, yymsp[-3].minor.yy272, yymsp[0].minor.yy272); } + yymsp[-3].minor.yy272 = yylhsminor.yy272; break; - case 467: /* query_expression_body ::= query_expression_body UNION query_expression_body */ -{ yylhsminor.yy840 = createSetOperator(pCxt, SET_OP_TYPE_UNION, yymsp[-2].minor.yy840, yymsp[0].minor.yy840); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + case 465: /* query_expression_body ::= query_expression_body UNION query_expression_body */ +{ yylhsminor.yy272 = createSetOperator(pCxt, SET_OP_TYPE_UNION, yymsp[-2].minor.yy272, yymsp[0].minor.yy272); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 469: /* query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP */ + case 467: /* query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP */ { - yymsp[-5].minor.yy840 = addOrderByClause(pCxt, yymsp[-4].minor.yy840, yymsp[-3].minor.yy544); - yymsp[-5].minor.yy840 = addSlimitClause(pCxt, yymsp[-5].minor.yy840, yymsp[-2].minor.yy840); - yymsp[-5].minor.yy840 = addLimitClause(pCxt, yymsp[-5].minor.yy840, yymsp[-1].minor.yy840); + yymsp[-5].minor.yy272 = addOrderByClause(pCxt, yymsp[-4].minor.yy272, yymsp[-3].minor.yy172); + yymsp[-5].minor.yy272 = addSlimitClause(pCxt, yymsp[-5].minor.yy272, yymsp[-2].minor.yy272); + yymsp[-5].minor.yy272 = addLimitClause(pCxt, yymsp[-5].minor.yy272, yymsp[-1].minor.yy272); } break; - case 473: /* slimit_clause_opt ::= SLIMIT NK_INTEGER */ - case 477: /* limit_clause_opt ::= LIMIT NK_INTEGER */ yytestcase(yyruleno==477); -{ yymsp[-1].minor.yy840 = createLimitNode(pCxt, &yymsp[0].minor.yy0, NULL); } + case 471: /* slimit_clause_opt ::= SLIMIT NK_INTEGER */ + case 475: /* limit_clause_opt ::= LIMIT NK_INTEGER */ yytestcase(yyruleno==475); +{ yymsp[-1].minor.yy272 = createLimitNode(pCxt, &yymsp[0].minor.yy0, NULL); } break; - case 474: /* slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER */ - case 478: /* limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER */ yytestcase(yyruleno==478); -{ yymsp[-3].minor.yy840 = createLimitNode(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0); } + case 472: /* slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER */ + case 476: /* limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER */ yytestcase(yyruleno==476); +{ yymsp[-3].minor.yy272 = createLimitNode(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0); } break; - case 475: /* slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER */ - case 479: /* limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER */ yytestcase(yyruleno==479); -{ yymsp[-3].minor.yy840 = createLimitNode(pCxt, &yymsp[0].minor.yy0, &yymsp[-2].minor.yy0); } + case 473: /* slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER */ + case 477: /* limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER */ yytestcase(yyruleno==477); +{ yymsp[-3].minor.yy272 = createLimitNode(pCxt, &yymsp[0].minor.yy0, &yymsp[-2].minor.yy0); } break; - case 480: /* subquery ::= NK_LP query_expression NK_RP */ -{ yylhsminor.yy840 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, yymsp[-1].minor.yy840); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + case 478: /* subquery ::= NK_LP query_expression NK_RP */ +{ yylhsminor.yy272 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, yymsp[-1].minor.yy272); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 484: /* sort_specification ::= expression ordering_specification_opt null_ordering_opt */ -{ yylhsminor.yy840 = createOrderByExprNode(pCxt, releaseRawExprNode(pCxt, yymsp[-2].minor.yy840), yymsp[-1].minor.yy204, yymsp[0].minor.yy277); } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + case 482: /* sort_specification ::= expression ordering_specification_opt null_ordering_opt */ +{ yylhsminor.yy272 = createOrderByExprNode(pCxt, releaseRawExprNode(pCxt, yymsp[-2].minor.yy272), yymsp[-1].minor.yy818, yymsp[0].minor.yy493); } + yymsp[-2].minor.yy272 = yylhsminor.yy272; break; - case 485: /* ordering_specification_opt ::= */ -{ yymsp[1].minor.yy204 = ORDER_ASC; } + case 483: /* ordering_specification_opt ::= */ +{ yymsp[1].minor.yy818 = ORDER_ASC; } break; - case 486: /* ordering_specification_opt ::= ASC */ -{ yymsp[0].minor.yy204 = ORDER_ASC; } + case 484: /* ordering_specification_opt ::= ASC */ +{ yymsp[0].minor.yy818 = ORDER_ASC; } break; - case 487: /* ordering_specification_opt ::= DESC */ -{ yymsp[0].minor.yy204 = ORDER_DESC; } + case 485: /* ordering_specification_opt ::= DESC */ +{ yymsp[0].minor.yy818 = ORDER_DESC; } break; - case 488: /* null_ordering_opt ::= */ -{ yymsp[1].minor.yy277 = NULL_ORDER_DEFAULT; } + case 486: /* null_ordering_opt ::= */ +{ yymsp[1].minor.yy493 = NULL_ORDER_DEFAULT; } break; - case 489: /* null_ordering_opt ::= NULLS FIRST */ -{ yymsp[-1].minor.yy277 = NULL_ORDER_FIRST; } + case 487: /* null_ordering_opt ::= NULLS FIRST */ +{ yymsp[-1].minor.yy493 = NULL_ORDER_FIRST; } break; - case 490: /* null_ordering_opt ::= NULLS LAST */ -{ yymsp[-1].minor.yy277 = NULL_ORDER_LAST; } + case 488: /* null_ordering_opt ::= NULLS LAST */ +{ yymsp[-1].minor.yy493 = NULL_ORDER_LAST; } break; default: break; diff --git a/source/libs/parser/test/parInitialCTest.cpp b/source/libs/parser/test/parInitialCTest.cpp index 9bca6cae0a..68c4ac3706 100644 --- a/source/libs/parser/test/parInitialCTest.cpp +++ b/source/libs/parser/test/parInitialCTest.cpp @@ -568,15 +568,13 @@ TEST_F(ParserInitialCTest, createStream) { memset(&expect, 0, sizeof(SCMCreateStreamReq)); }; - auto setCreateStreamReqFunc = [&](const char* pStream, const char* pSrcDb, const char* pSql, - const char* pDstStb = nullptr, int8_t igExists = 0, - int8_t triggerType = STREAM_TRIGGER_AT_ONCE, int64_t maxDelay = 0, - int64_t watermark = 0, int8_t igExpired = STREAM_DEFAULT_IGNORE_EXPIRED) { + auto setCreateStreamReqFunc = [&](const char* pStream, const char* pSrcDb, const char* pSql, const char* pDstStb, + int8_t igExists = 0, int8_t triggerType = STREAM_TRIGGER_AT_ONCE, + int64_t maxDelay = 0, int64_t watermark = 0, + int8_t igExpired = STREAM_DEFAULT_IGNORE_EXPIRED) { snprintf(expect.name, sizeof(expect.name), "0.%s", pStream); snprintf(expect.sourceDB, sizeof(expect.sourceDB), "0.%s", pSrcDb); - if (NULL != pDstStb) { - snprintf(expect.targetStbFullName, sizeof(expect.targetStbFullName), "0.test.%s", pDstStb); - } + snprintf(expect.targetStbFullName, sizeof(expect.targetStbFullName), "0.test.%s", pDstStb); expect.igExists = igExists; expect.sql = strdup(pSql); expect.triggerType = triggerType; @@ -603,15 +601,6 @@ TEST_F(ParserInitialCTest, createStream) { tFreeSCMCreateStreamReq(&req); }); - setCreateStreamReqFunc("s1", "test", "create stream s1 as select count(*) from t1 interval(10s)"); - run("CREATE STREAM s1 AS SELECT COUNT(*) FROM t1 INTERVAL(10S)"); - clearCreateStreamReq(); - - setCreateStreamReqFunc("s1", "test", "create stream if not exists s1 as select count(*) from t1 interval(10s)", - nullptr, 1); - run("CREATE STREAM IF NOT EXISTS s1 AS SELECT COUNT(*) FROM t1 INTERVAL(10S)"); - clearCreateStreamReq(); - setCreateStreamReqFunc("s1", "test", "create stream s1 into st1 as select count(*) from t1 interval(10s)", "st1"); run("CREATE STREAM s1 INTO st1 AS SELECT COUNT(*) FROM t1 INTERVAL(10S)"); clearCreateStreamReq(); @@ -629,7 +618,8 @@ TEST_F(ParserInitialCTest, createStream) { TEST_F(ParserInitialCTest, createStreamSemanticCheck) { useDb("root", "test"); - run("CREATE STREAM s1 AS SELECT PERCENTILE(c1, 30) FROM t1 INTERVAL(10S)", TSDB_CODE_PAR_STREAM_NOT_ALLOWED_FUNC); + run("CREATE STREAM s1 INTO st1 AS SELECT PERCENTILE(c1, 30) FROM t1 INTERVAL(10S)", + TSDB_CODE_PAR_STREAM_NOT_ALLOWED_FUNC); } TEST_F(ParserInitialCTest, createTable) { diff --git a/source/libs/parser/test/parTestUtil.cpp b/source/libs/parser/test/parTestUtil.cpp index 98281b7bf0..360b904c17 100644 --- a/source/libs/parser/test/parTestUtil.cpp +++ b/source/libs/parser/test/parTestUtil.cpp @@ -207,6 +207,7 @@ class ParserTestBaseImpl { pCxt->db = caseEnv_.db_.c_str(); pCxt->pUser = caseEnv_.user_.c_str(); pCxt->isSuperUser = caseEnv_.user_ == "root"; + pCxt->enableSysInfo = true; pCxt->pSql = stmtEnv_.sql_.c_str(); pCxt->sqlLen = stmtEnv_.sql_.length(); pCxt->pMsg = stmtEnv_.msgBuf_.data(); diff --git a/source/libs/planner/src/planOptimizer.c b/source/libs/planner/src/planOptimizer.c index 45ab3903a9..653291d3a4 100644 --- a/source/libs/planner/src/planOptimizer.c +++ b/source/libs/planner/src/planOptimizer.c @@ -1084,7 +1084,7 @@ static int32_t sortPriKeyOptGetSequencingNodesImpl(SLogicNode* pNode, bool* pNot switch (nodeType(pNode)) { case QUERY_NODE_LOGIC_PLAN_SCAN: { SScanLogicNode* pScan = (SScanLogicNode*)pNode; - if (NULL != pScan->pGroupTags) { + if (NULL != pScan->pGroupTags || TSDB_SYSTEM_TABLE == pScan->tableType) { *pNotOptimize = true; return TSDB_CODE_SUCCESS; } diff --git a/source/libs/planner/src/planPhysiCreater.c b/source/libs/planner/src/planPhysiCreater.c index c7eb6f7b5e..cafae18dbe 100644 --- a/source/libs/planner/src/planPhysiCreater.c +++ b/source/libs/planner/src/planPhysiCreater.c @@ -576,6 +576,7 @@ static int32_t createSystemTableScanPhysiNode(SPhysiPlanContext* pCxt, SSubplan* pScan->showRewrite = pScanLogicNode->showRewrite; pScan->accountId = pCxt->pPlanCxt->acctId; + pScan->sysInfo = pCxt->pPlanCxt->sysInfo; if (0 == strcmp(pScanLogicNode->tableName.tname, TSDB_INS_TABLE_TABLES) || 0 == strcmp(pScanLogicNode->tableName.tname, TSDB_INS_TABLE_TABLE_DISTRIBUTED) || 0 == strcmp(pScanLogicNode->tableName.tname, TSDB_INS_TABLE_TAGS)) { diff --git a/source/libs/planner/test/planOtherTest.cpp b/source/libs/planner/test/planOtherTest.cpp index 7107f8b3c9..350ccd0d92 100644 --- a/source/libs/planner/test/planOtherTest.cpp +++ b/source/libs/planner/test/planOtherTest.cpp @@ -37,9 +37,9 @@ TEST_F(PlanOtherTest, createStream) { TEST_F(PlanOtherTest, createStreamUseSTable) { useDb("root", "test"); - run("CREATE STREAM IF NOT EXISTS s1 as SELECT COUNT(*) FROM st1 INTERVAL(10s)"); + run("CREATE STREAM IF NOT EXISTS s1 into st1 as SELECT COUNT(*) FROM st1 INTERVAL(10s)"); - run("CREATE STREAM IF NOT EXISTS s1 as SELECT COUNT(*) FROM st1 PARTITION BY TBNAME INTERVAL(10s)"); + run("CREATE STREAM IF NOT EXISTS s1 into st1 as SELECT COUNT(*) FROM st1 PARTITION BY TBNAME INTERVAL(10s)"); } TEST_F(PlanOtherTest, createSmaIndex) { diff --git a/source/libs/planner/test/planTestUtil.cpp b/source/libs/planner/test/planTestUtil.cpp index 5fc8b3cf30..f904643be9 100644 --- a/source/libs/planner/test/planTestUtil.cpp +++ b/source/libs/planner/test/planTestUtil.cpp @@ -343,6 +343,7 @@ class PlannerTestBaseImpl { cxt.pMsg = stmtEnv_.msgBuf_.data(); cxt.msgLen = stmtEnv_.msgBuf_.max_size(); cxt.svrVer = "3.0.0.0"; + cxt.enableSysInfo = true; if (prepare) { SStmtCallback stmtCb = {0}; cxt.pStmtCb = &stmtCb; diff --git a/source/libs/qcom/src/queryUtil.c b/source/libs/qcom/src/queryUtil.c index 5143aa4af1..d848016e46 100644 --- a/source/libs/qcom/src/queryUtil.c +++ b/source/libs/qcom/src/queryUtil.c @@ -213,15 +213,25 @@ SSchema createSchema(int8_t type, int32_t bytes, col_id_t colId, const char* nam return s; } +void freeSTableMetaRspPointer(void *p) { + tFreeSTableMetaRsp(*(void**)p); + taosMemoryFreeClear(*(void**)p); +} + void destroyQueryExecRes(SExecResult* pRes) { if (NULL == pRes || NULL == pRes->res) { return; } switch (pRes->msgType) { + case TDMT_VND_CREATE_TABLE: { + taosArrayDestroyEx((SArray*)pRes->res, freeSTableMetaRspPointer); + break; + } + case TDMT_MND_CREATE_STB: case TDMT_VND_ALTER_TABLE: case TDMT_MND_ALTER_STB: { - tFreeSTableMetaRsp((STableMetaRsp*)pRes->res); + tFreeSTableMetaRsp(pRes->res); taosMemoryFreeClear(pRes->res); break; } diff --git a/source/libs/qcom/src/querymsg.c b/source/libs/qcom/src/querymsg.c index ed8786170d..e2d3ac1583 100644 --- a/source/libs/qcom/src/querymsg.c +++ b/source/libs/qcom/src/querymsg.c @@ -354,6 +354,19 @@ static int32_t queryConvertTableMetaMsg(STableMetaRsp *pMetaMsg) { return TSDB_CODE_SUCCESS; } +int32_t queryCreateCTableMetaFromMsg(STableMetaRsp *msg, SCTableMeta *pMeta) { + pMeta->vgId = msg->vgId; + pMeta->tableType = msg->tableType; + pMeta->uid = msg->tuid; + pMeta->suid = msg->suid; + + qDebug("ctable %s uid %" PRIx64 " meta returned, type %d vgId:%d db %s suid %" PRIx64 , + msg->tbName, pMeta->uid, pMeta->tableType, pMeta->vgId, msg->dbFName, pMeta->suid); + + return TSDB_CODE_SUCCESS; +} + + int32_t queryCreateTableMetaFromMsg(STableMetaRsp *msg, bool isStb, STableMeta **pMeta) { int32_t total = msg->numOfColumns + msg->numOfTags; int32_t metaSize = sizeof(STableMeta) + sizeof(SSchema) * total; diff --git a/source/libs/scalar/inc/filterInt.h b/source/libs/scalar/inc/filterInt.h index 23693c785a..87327a3657 100644 --- a/source/libs/scalar/inc/filterInt.h +++ b/source/libs/scalar/inc/filterInt.h @@ -350,6 +350,7 @@ struct SFilterInfo { extern bool filterDoCompare(__compar_fn_t func, uint8_t optr, void *left, void *right); extern __compar_fn_t filterGetCompFunc(int32_t type, int32_t optr); +extern __compar_fn_t filterGetCompFuncEx(int32_t lType, int32_t rType, int32_t optr); #ifdef __cplusplus } diff --git a/source/libs/scalar/inc/sclInt.h b/source/libs/scalar/inc/sclInt.h index d423b92da7..15e9026ddb 100644 --- a/source/libs/scalar/inc/sclInt.h +++ b/source/libs/scalar/inc/sclInt.h @@ -45,6 +45,9 @@ typedef struct SScalarCtx { #define SCL_IS_CONST_CALC(_ctx) (NULL == (_ctx)->pBlockList) //#define SCL_IS_NULL_VALUE_NODE(_node) ((QUERY_NODE_VALUE == nodeType(_node)) && (TSDB_DATA_TYPE_NULL == ((SValueNode *)_node)->node.resType.type) && (((SValueNode *)_node)->placeholderNo <= 0)) #define SCL_IS_NULL_VALUE_NODE(_node) ((QUERY_NODE_VALUE == nodeType(_node)) && (TSDB_DATA_TYPE_NULL == ((SValueNode *)_node)->node.resType.type)) +#define SCL_IS_COMPARISON_OPERATOR(_opType) ((_opType) >= OP_TYPE_GREATER_THAN && (_opType) < OP_TYPE_IS_NOT_UNKNOWN) +#define SCL_DOWNGRADE_DATETYPE(_type) ((_type) == TSDB_DATA_TYPE_BIGINT || TSDB_DATA_TYPE_DOUBLE == (_type) || (_type) == TSDB_DATA_TYPE_UBIGINT) +#define SCL_NO_NEED_CONVERT_COMPARISION(_ltype, _rtype, _optr) (IS_NUMERIC_TYPE(_ltype) && IS_NUMERIC_TYPE(_rtype) && ((_optr) >= OP_TYPE_GREATER_THAN && (_optr) <= OP_TYPE_NOT_EQUAL)) #define sclFatal(...) qFatal(__VA_ARGS__) #define sclError(...) qError(__VA_ARGS__) diff --git a/source/libs/scalar/src/filter.c b/source/libs/scalar/src/filter.c index 4377dbf14e..5555a52c8e 100644 --- a/source/libs/scalar/src/filter.c +++ b/source/libs/scalar/src/filter.c @@ -132,6 +132,77 @@ __compar_fn_t gDataCompare[] = {compareInt32Val, compareInt8Val, compareInt16Val compareChkNotInString, compareStrPatternNotMatch, compareWStrPatternNotMatch }; +__compar_fn_t gInt8SignCompare[] = { + compareInt8Val, compareInt8Int16, compareInt8Int32, compareInt8Int64, compareInt8Float, compareInt8Double +}; +__compar_fn_t gInt8UsignCompare[] = { + compareInt8Uint8, compareInt8Uint16, compareInt8Uint32, compareInt8Uint64 +}; + +__compar_fn_t gInt16SignCompare[] = { + compareInt16Int8, compareInt16Val, compareInt16Int32, compareInt16Int64, compareInt16Float, compareInt16Double +}; +__compar_fn_t gInt16UsignCompare[] = { + compareInt16Uint8, compareInt16Uint16, compareInt16Uint32, compareInt16Uint64 +}; + +__compar_fn_t gInt32SignCompare[] = { + compareInt32Int8, compareInt32Int16, compareInt32Val, compareInt32Int64, compareInt32Float, compareInt32Double +}; +__compar_fn_t gInt32UsignCompare[] = { + compareInt32Uint8, compareInt32Uint16, compareInt32Uint32, compareInt32Uint64 +}; + +__compar_fn_t gInt64SignCompare[] = { + compareInt64Int8, compareInt64Int16, compareInt64Int32, compareInt64Val, compareInt64Float, compareInt64Double +}; +__compar_fn_t gInt64UsignCompare[] = { + compareInt64Uint8, compareInt64Uint16, compareInt64Uint32, compareInt64Uint64 +}; + +__compar_fn_t gFloatSignCompare[] = { + compareFloatInt8, compareFloatInt16, compareFloatInt32, compareFloatInt64, compareFloatVal, compareFloatDouble +}; +__compar_fn_t gFloatUsignCompare[] = { + compareFloatUint8, compareFloatUint16, compareFloatUint32, compareFloatUint64 +}; + +__compar_fn_t gDoubleSignCompare[] = { + compareDoubleInt8, compareDoubleInt16, compareDoubleInt32, compareDoubleInt64, compareDoubleFloat, compareDoubleVal +}; +__compar_fn_t gDoubleUsignCompare[] = { + compareDoubleUint8, compareDoubleUint16, compareDoubleUint32, compareDoubleUint64 +}; + +__compar_fn_t gUint8SignCompare[] = { + compareUint8Int8, compareUint8Int16, compareUint8Int32, compareUint8Int64, compareUint8Float, compareUint8Double +}; +__compar_fn_t gUint8UsignCompare[] = { + compareUint8Val, compareUint8Uint16, compareUint8Uint32, compareUint8Uint64 +}; + +__compar_fn_t gUint16SignCompare[] = { + compareUint16Int8, compareUint16Int16, compareUint16Int32, compareUint16Int64, compareUint16Float, compareUint16Double +}; +__compar_fn_t gUint16UsignCompare[] = { + compareUint16Uint8, compareUint16Val, compareUint16Uint32, compareUint16Uint64 +}; + +__compar_fn_t gUint32SignCompare[] = { + compareUint32Int8, compareUint32Int16, compareUint32Int32, compareUint32Int64, compareUint32Float, compareUint32Double +}; +__compar_fn_t gUint32UsignCompare[] = { + compareUint32Uint8, compareUint32Uint16, compareUint32Val, compareUint32Uint64 +}; + +__compar_fn_t gUint64SignCompare[] = { + compareUint64Int8, compareUint64Int16, compareUint64Int32, compareUint64Int64, compareUint64Float, compareUint64Double +}; +__compar_fn_t gUint64UsignCompare[] = { + compareUint64Uint8, compareUint64Uint16, compareUint64Uint32, compareUint64Val +}; + + int8_t filterGetCompFuncIdx(int32_t type, int32_t optr) { int8_t comparFn = 0; @@ -257,6 +328,93 @@ __compar_fn_t filterGetCompFunc(int32_t type, int32_t optr) { return gDataCompare[filterGetCompFuncIdx(type, optr)]; } +__compar_fn_t filterGetCompFuncEx(int32_t lType, int32_t rType, int32_t optr) { + switch (lType) { + case TSDB_DATA_TYPE_TINYINT: { + if (IS_SIGNED_NUMERIC_TYPE(rType) || IS_FLOAT_TYPE(rType)) { + return gInt8SignCompare[rType - TSDB_DATA_TYPE_TINYINT]; + } else { + return gInt8UsignCompare[rType - TSDB_DATA_TYPE_UTINYINT]; + } + break; + } + case TSDB_DATA_TYPE_SMALLINT: { + if (IS_SIGNED_NUMERIC_TYPE(rType) || IS_FLOAT_TYPE(rType)) { + return gInt16SignCompare[rType - TSDB_DATA_TYPE_TINYINT]; + } else { + return gInt16UsignCompare[rType - TSDB_DATA_TYPE_UTINYINT]; + } + break; + } + case TSDB_DATA_TYPE_INT: { + if (IS_SIGNED_NUMERIC_TYPE(rType) || IS_FLOAT_TYPE(rType)) { + return gInt32SignCompare[rType - TSDB_DATA_TYPE_TINYINT]; + } else { + return gInt32UsignCompare[rType - TSDB_DATA_TYPE_UTINYINT]; + } + break; + } + case TSDB_DATA_TYPE_BIGINT: { + if (IS_SIGNED_NUMERIC_TYPE(rType) || IS_FLOAT_TYPE(rType)) { + return gInt64SignCompare[rType - TSDB_DATA_TYPE_TINYINT]; + } else { + return gInt64UsignCompare[rType - TSDB_DATA_TYPE_UTINYINT]; + } + break; + } + case TSDB_DATA_TYPE_FLOAT: { + if (IS_SIGNED_NUMERIC_TYPE(rType) || IS_FLOAT_TYPE(rType)) { + return gFloatSignCompare[rType - TSDB_DATA_TYPE_TINYINT]; + } else { + return gFloatUsignCompare[rType - TSDB_DATA_TYPE_UTINYINT]; + } + break; + } + case TSDB_DATA_TYPE_DOUBLE: { + if (IS_SIGNED_NUMERIC_TYPE(rType) || IS_FLOAT_TYPE(rType)) { + return gDoubleSignCompare[rType - TSDB_DATA_TYPE_TINYINT]; + } else { + return gDoubleUsignCompare[rType - TSDB_DATA_TYPE_UTINYINT]; + } + break; + } + case TSDB_DATA_TYPE_UTINYINT: { + if (IS_SIGNED_NUMERIC_TYPE(rType) || IS_FLOAT_TYPE(rType)) { + return gUint8SignCompare[rType - TSDB_DATA_TYPE_TINYINT]; + } else { + return gUint8UsignCompare[rType - TSDB_DATA_TYPE_UTINYINT]; + } + break; + } + case TSDB_DATA_TYPE_USMALLINT: { + if (IS_SIGNED_NUMERIC_TYPE(rType) || IS_FLOAT_TYPE(rType)) { + return gUint16SignCompare[rType - TSDB_DATA_TYPE_TINYINT]; + } else { + return gUint16UsignCompare[rType - TSDB_DATA_TYPE_UTINYINT]; + } + break; + } + case TSDB_DATA_TYPE_UINT: { + if (IS_SIGNED_NUMERIC_TYPE(rType) || IS_FLOAT_TYPE(rType)) { + return gUint32SignCompare[rType - TSDB_DATA_TYPE_TINYINT]; + } else { + return gUint32UsignCompare[rType - TSDB_DATA_TYPE_UTINYINT]; + } + break; + } + case TSDB_DATA_TYPE_UBIGINT: { + if (IS_SIGNED_NUMERIC_TYPE(rType) || IS_FLOAT_TYPE(rType)) { + return gUint64SignCompare[rType - TSDB_DATA_TYPE_TINYINT]; + } else { + return gUint64UsignCompare[rType - TSDB_DATA_TYPE_UTINYINT]; + } + break; + } + default: + break; + } + return NULL; +} static FORCE_INLINE int32_t filterCompareGroupCtx(const void *pLeft, const void *pRight) { SFilterGroupCtx *left = *((SFilterGroupCtx**)pLeft), *right = *((SFilterGroupCtx**)pRight); diff --git a/source/libs/scalar/src/scalar.c b/source/libs/scalar/src/scalar.c index 6634a29f40..cd1f6624bd 100644 --- a/source/libs/scalar/src/scalar.c +++ b/source/libs/scalar/src/scalar.c @@ -9,6 +9,7 @@ #include "scalar.h" #include "tudf.h" #include "ttime.h" +#include "tcompare.h" int32_t scalarGetOperatorParamNum(EOperatorType type) { if (OP_TYPE_IS_NULL == type || OP_TYPE_IS_NOT_NULL == type || OP_TYPE_IS_TRUE == type || OP_TYPE_IS_NOT_TRUE == type @@ -219,6 +220,82 @@ void sclFreeParamList(SScalarParam *param, int32_t paramNum) { taosMemoryFree(param); } +void sclDowngradeValueType(SValueNode *valueNode) { + switch (valueNode->node.resType.type) { + case TSDB_DATA_TYPE_BIGINT: { + int8_t i8 = valueNode->datum.i; + if (i8 == valueNode->datum.i) { + valueNode->node.resType.type = TSDB_DATA_TYPE_TINYINT; + *(int8_t*)&valueNode->typeData = i8; + break; + } + int16_t i16 = valueNode->datum.i; + if (i16 == valueNode->datum.i) { + valueNode->node.resType.type = TSDB_DATA_TYPE_SMALLINT; + *(int16_t*)&valueNode->typeData = i16; + break; + } + int32_t i32 = valueNode->datum.i; + if (i32 == valueNode->datum.i) { + valueNode->node.resType.type = TSDB_DATA_TYPE_INT; + *(int32_t*)&valueNode->typeData = i32; + break; + } + break; + } + case TSDB_DATA_TYPE_UBIGINT:{ + uint8_t u8 = valueNode->datum.i; + if (u8 == valueNode->datum.i) { + int8_t i8 = valueNode->datum.i; + if (i8 == valueNode->datum.i) { + valueNode->node.resType.type = TSDB_DATA_TYPE_TINYINT; + *(int8_t*)&valueNode->typeData = i8; + } else { + valueNode->node.resType.type = TSDB_DATA_TYPE_UTINYINT; + *(uint8_t*)&valueNode->typeData = u8; + } + break; + } + uint16_t u16 = valueNode->datum.i; + if (u16 == valueNode->datum.i) { + int16_t i16 = valueNode->datum.i; + if (i16 == valueNode->datum.i) { + valueNode->node.resType.type = TSDB_DATA_TYPE_SMALLINT; + *(int16_t*)&valueNode->typeData = i16; + } else { + valueNode->node.resType.type = TSDB_DATA_TYPE_USMALLINT; + *(uint16_t*)&valueNode->typeData = u16; + } + break; + } + uint32_t u32 = valueNode->datum.i; + if (u32 == valueNode->datum.i) { + int32_t i32 = valueNode->datum.i; + if (i32 == valueNode->datum.i) { + valueNode->node.resType.type = TSDB_DATA_TYPE_INT; + *(int32_t*)&valueNode->typeData = i32; + } else { + valueNode->node.resType.type = TSDB_DATA_TYPE_UINT; + *(uint32_t*)&valueNode->typeData = u32; + } + break; + } + break; + } + case TSDB_DATA_TYPE_DOUBLE: { + float f = valueNode->datum.d; + if (FLT_EQUAL(f, valueNode->datum.d)) { + valueNode->node.resType.type = TSDB_DATA_TYPE_FLOAT; + *(float*)&valueNode->typeData = f; + break; + } + break; + } + default: + break; + } +} + int32_t sclInitParam(SNode* node, SScalarParam *param, SScalarCtx *ctx, int32_t *rowNum) { switch (nodeType(node)) { case QUERY_NODE_LEFT_VALUE: { @@ -675,6 +752,10 @@ EDealRes sclRewriteNonConstOperator(SNode** pNode, SScalarCtx *ctx) { return DEAL_RES_ERROR; } } + + if (SCL_IS_COMPARISON_OPERATOR(node->opType) && SCL_DOWNGRADE_DATETYPE(valueNode->node.resType.type)) { + sclDowngradeValueType(valueNode); + } } if (node->pRight && (QUERY_NODE_VALUE == nodeType(node->pRight))) { @@ -692,6 +773,10 @@ EDealRes sclRewriteNonConstOperator(SNode** pNode, SScalarCtx *ctx) { return DEAL_RES_ERROR; } } + + if (SCL_IS_COMPARISON_OPERATOR(node->opType) && SCL_DOWNGRADE_DATETYPE(valueNode->node.resType.type)) { + sclDowngradeValueType(valueNode); + } } if (node->pRight && (QUERY_NODE_NODE_LIST == nodeType(node->pRight))) { diff --git a/source/libs/scalar/src/sclvector.c b/source/libs/scalar/src/sclvector.c index aaa70ef5ae..a003315fca 100644 --- a/source/libs/scalar/src/sclvector.c +++ b/source/libs/scalar/src/sclvector.c @@ -909,11 +909,11 @@ int32_t vectorConvertImpl(const SScalarParam* pIn, SScalarParam* pOut, int32_t* int8_t gConvertTypes[TSDB_DATA_TYPE_BLOB+1][TSDB_DATA_TYPE_BLOB+1] = { /* NULL BOOL TINY SMAL INT BIG FLOA DOUB VARC TIME NCHA UTIN USMA UINT UBIG JSON VARB DECI BLOB */ /*NULL*/ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -/*BOOL*/ 0, 0, 0, 3, 4, 5, 6, 7, 7, 9, 7, 0, 12, 13, 14, 0, 7, 0, 0, +/*BOOL*/ 0, 0, 2, 3, 4, 5, 6, 7, 7, 9, 7, 11, 12, 13, 14, 0, 7, 0, 0, /*TINY*/ 0, 0, 0, 3, 4, 5, 6, 7, 7, 9, 7, 3, 4, 5, 7, 0, 7, 0, 0, /*SMAL*/ 0, 0, 0, 0, 4, 5, 6, 7, 7, 9, 7, 3, 4, 5, 7, 0, 7, 0, 0, /*INT */ 0, 0, 0, 0, 0, 5, 6, 7, 7, 9, 7, 4, 4, 5, 7, 0, 7, 0, 0, -/*BIGI*/ 0, 0, 0, 0, 0, 0, 6, 7, 7, 0, 7, 5, 5, 5, 7, 0, 7, 0, 0, +/*BIGI*/ 0, 0, 0, 0, 0, 0, 6, 7, 7, 9, 7, 5, 5, 5, 7, 0, 7, 0, 0, /*FLOA*/ 0, 0, 0, 0, 0, 0, 0, 7, 7, 6, 7, 6, 6, 6, 6, 0, 7, 0, 0, /*DOUB*/ 0, 0, 0, 0, 0, 0, 0, 0, 7, 7, 7, 7, 7, 7, 7, 0, 7, 0, 0, /*VARC*/ 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, 8, 7, 7, 7, 7, 0, 0, 0, 0, @@ -1681,10 +1681,14 @@ void vectorBitOr(SScalarParam* pLeft, SScalarParam* pRight, SScalarParam *pOut, void vectorCompareImpl(SScalarParam* pLeft, SScalarParam* pRight, SScalarParam *pOut, int32_t _ord, int32_t optr) { int32_t i = ((_ord) == TSDB_ORDER_ASC) ? 0 : TMAX(pLeft->numOfRows, pRight->numOfRows) - 1; int32_t step = ((_ord) == TSDB_ORDER_ASC) ? 1 : -1; - - __compar_fn_t fp = filterGetCompFunc(GET_PARAM_TYPE(pLeft), optr); - if(terrno != TSDB_CODE_SUCCESS){ - return; + int32_t lType = GET_PARAM_TYPE(pLeft); + int32_t rType = GET_PARAM_TYPE(pRight); + __compar_fn_t fp = NULL; + + if (lType == rType) { + fp = filterGetCompFunc(lType, optr); + } else { + fp = filterGetCompFuncEx(lType, rType, optr); } pOut->numOfRows = TMAX(pLeft->numOfRows, pRight->numOfRows); @@ -1716,22 +1720,26 @@ void vectorCompareImpl(SScalarParam* pLeft, SScalarParam* pRight, SScalarParam * void vectorCompare(SScalarParam* pLeft, SScalarParam* pRight, SScalarParam *pOut, int32_t _ord, int32_t optr) { SScalarParam pLeftOut = {0}; SScalarParam pRightOut = {0}; - - vectorConvert(pLeft, pRight, &pLeftOut, &pRightOut); - SScalarParam *param1 = NULL; SScalarParam *param2 = NULL; - if (pLeftOut.columnData != NULL) { - param1 = &pLeftOut; - } else { + if (SCL_NO_NEED_CONVERT_COMPARISION(GET_PARAM_TYPE(pLeft), GET_PARAM_TYPE(pRight), optr)) { param1 = pLeft; - } - - if (pRightOut.columnData != NULL) { - param2 = &pRightOut; - } else { param2 = pRight; + } else { + vectorConvert(pLeft, pRight, &pLeftOut, &pRightOut); + + if (pLeftOut.columnData != NULL) { + param1 = &pLeftOut; + } else { + param1 = pLeft; + } + + if (pRightOut.columnData != NULL) { + param2 = &pRightOut; + } else { + param2 = pRight; + } } vectorCompareImpl(param1, param2, pOut, _ord, optr); diff --git a/source/libs/scheduler/src/schRemote.c b/source/libs/scheduler/src/schRemote.c index ecd9daf1bc..5a64aaaebb 100644 --- a/source/libs/scheduler/src/schRemote.c +++ b/source/libs/scheduler/src/schRemote.c @@ -102,15 +102,30 @@ int32_t schHandleResponseMsg(SSchJob *pJob, SSchTask *pTask, int32_t execId, SDa tDecoderInit(&coder, msg, msgSize); code = tDecodeSVCreateTbBatchRsp(&coder, &batchRsp); if (TSDB_CODE_SUCCESS == code && batchRsp.nRsps > 0) { + SCH_LOCK(SCH_WRITE, &pJob->resLock); + if (NULL == pJob->execRes.res) { + pJob->execRes.res = taosArrayInit(batchRsp.nRsps, POINTER_BYTES); + pJob->execRes.msgType = TDMT_VND_CREATE_TABLE; + } + for (int32_t i = 0; i < batchRsp.nRsps; ++i) { SVCreateTbRsp *rsp = batchRsp.pRsps + i; + if (rsp->pMeta) { + taosArrayPush((SArray*)pJob->execRes.res, &rsp->pMeta); + } + if (TSDB_CODE_SUCCESS != rsp->code) { code = rsp->code; - tDecoderClear(&coder); - SCH_ERR_JRET(code); } } + SCH_UNLOCK(SCH_WRITE, &pJob->resLock); + + if (taosArrayGetSize((SArray*)pJob->execRes.res) <= 0) { + taosArrayDestroy((SArray*)pJob->execRes.res); + pJob->execRes.res = NULL; + } } + tDecoderClear(&coder); SCH_ERR_JRET(code); } diff --git a/source/libs/stream/src/streamMeta.c b/source/libs/stream/src/streamMeta.c index 20a2f7d332..1442ed2e05 100644 --- a/source/libs/stream/src/streamMeta.c +++ b/source/libs/stream/src/streamMeta.c @@ -265,6 +265,8 @@ int32_t streamLoadTasks(SStreamMeta* pMeta) { } } + tdbFree(pKey); + tdbFree(pVal); if (tdbTbcClose(pCur) < 0) { return -1; } diff --git a/source/libs/tdb/src/db/tdbBtree.c b/source/libs/tdb/src/db/tdbBtree.c index 5430acb972..1480920f90 100644 --- a/source/libs/tdb/src/db/tdbBtree.c +++ b/source/libs/tdb/src/db/tdbBtree.c @@ -509,7 +509,7 @@ static int tdbBtreeBalanceDeeper(SBTree *pBt, SPage *pRoot, SPage **ppChild, TXN static int tdbBtreeBalanceNonRoot(SBTree *pBt, SPage *pParent, int idx, TXN *pTxn) { int ret; - int nOlds; + int nOlds, pageIdx; SPage *pOlds[3] = {0}; SCell *pDivCell[3] = {0}; int szDivCell[3]; @@ -782,6 +782,11 @@ static int tdbBtreeBalanceNonRoot(SBTree *pBt, SPage *pParent, int idx, TXN *pTx pBt); tdbPageInsertCell(pParent, sIdx++, pNewCell, szNewCell, 0); tdbOsFree(pNewCell); + + if (TDB_CELLDECODER_FREE_VAL(&cd)) { + tdbFree(cd.pVal); + cd.pVal = NULL; + } } // move to next new page @@ -844,13 +849,11 @@ static int tdbBtreeBalanceNonRoot(SBTree *pBt, SPage *pParent, int idx, TXN *pTx } } - // TODO: here is not corrent for drop case - for (int i = 0; i < nNews; i++) { - if (i < nOlds) { - tdbPagerReturnPage(pBt->pPager, pOlds[i], pTxn); - } else { - tdbPagerReturnPage(pBt->pPager, pNews[i], pTxn); - } + for (pageIdx = 0; pageIdx < nOlds; ++pageIdx) { + tdbPagerReturnPage(pBt->pPager, pOlds[pageIdx], pTxn); + } + for (; pageIdx < nNews; ++pageIdx) { + tdbPagerReturnPage(pBt->pPager, pNews[pageIdx], pTxn); } return 0; @@ -934,6 +937,8 @@ static int tdbFetchOvflPage(SPgno *pPgno, SPage **ppOfp, TXN *pTxn, SBTree *pBt) return -1; } + tdbPCacheRelease(pBt->pPager->pCache, *ppOfp, pTxn); + return ret; } @@ -1277,6 +1282,8 @@ static int tdbBtreeDecodePayload(SPage *pPage, const SCell *pCell, int nHeader, nLeft -= bytes; memcpy(&pgno, ofpCell + bytes, sizeof(pgno)); + + tdbPCacheRelease(pBt->pPager->pCache, ofp, pTxn); } } else { int nLeftKey = kLen; @@ -1336,6 +1343,8 @@ static int tdbBtreeDecodePayload(SPage *pPage, const SCell *pCell, int nHeader, memcpy(&pgno, ofpCell + bytes, sizeof(pgno)); + tdbPCacheRelease(pBt->pPager->pCache, ofp, pTxn); + nLeftKey -= bytes; nLeft -= bytes; } @@ -1374,6 +1383,8 @@ static int tdbBtreeDecodePayload(SPage *pPage, const SCell *pCell, int nHeader, memcpy(&pgno, ofpCell + vLen - nLeft + bytes, sizeof(pgno)); + tdbPCacheRelease(pBt->pPager->pCache, ofp, pTxn); + nLeft -= bytes; } } diff --git a/source/libs/tdb/src/db/tdbPCache.c b/source/libs/tdb/src/db/tdbPCache.c index 527ad200d4..6254158591 100644 --- a/source/libs/tdb/src/db/tdbPCache.c +++ b/source/libs/tdb/src/db/tdbPCache.c @@ -98,6 +98,7 @@ SPage *tdbPCacheFetch(SPCache *pCache, const SPgid *pPgid, TXN *pTxn) { // printf("thread %" PRId64 " fetch page %d pgno %d pPage %p nRef %d\n", taosGetSelfPthreadId(), pPage->id, // TDB_PAGE_PGNO(pPage), pPage, nRef); + tdbDebug("pcache/fetch page %p/%d/%d/%d", pPage, TDB_PAGE_PGNO(pPage), pPage->id, nRef); return pPage; } @@ -111,6 +112,7 @@ void tdbPCacheRelease(SPCache *pCache, SPage *pPage, TXN *pTxn) { tdbPCacheLock(pCache); nRef = tdbUnrefPage(pPage); + tdbDebug("pcache/release page %p/%d/%d/%d", pPage, TDB_PAGE_PGNO(pPage), pPage->id, nRef); if (nRef == 0) { // test the nRef again to make sure // it is safe th handle the page @@ -212,7 +214,8 @@ static SPage *tdbPCacheFetchImpl(SPCache *pCache, const SPgid *pPgid, TXN *pTxn) pPage->pPager = pPageH->pPager; memcpy(pPage->pData, pPageH->pData, pPage->pageSize); - tdbDebug("pcache/pPageH: %p %d %p %p", pPageH, pPageH->pPageHdr - pPageH->pData, pPageH->xCellSize, pPage); + tdbDebug("pcache/pPageH: %p %d %p %p %d", pPageH, pPageH->pPageHdr - pPageH->pData, pPageH->xCellSize, pPage, + TDB_PAGE_PGNO(pPageH)); tdbPageInit(pPage, pPageH->pPageHdr - pPageH->pData, pPageH->xCellSize); pPage->kLen = pPageH->kLen; pPage->vLen = pPageH->vLen; @@ -243,7 +246,7 @@ static void tdbPCachePinPage(SPCache *pCache, SPage *pPage) { pCache->nRecyclable--; // printf("pin page %d pgno %d pPage %p\n", pPage->id, TDB_PAGE_PGNO(pPage), pPage); - tdbDebug("pin page %d", pPage->id); + tdbDebug("pcache/pin page %p/%d/%d", pPage, TDB_PAGE_PGNO(pPage), pPage->id); } } @@ -264,29 +267,23 @@ static void tdbPCacheUnpinPage(SPCache *pCache, SPage *pPage) { pCache->nRecyclable++; // printf("unpin page %d pgno %d pPage %p\n", pPage->id, TDB_PAGE_PGNO(pPage), pPage); - tdbDebug("unpin page %d", pPage->id); + tdbDebug("pcache/unpin page %p/%d/%d", pPage, TDB_PAGE_PGNO(pPage), pPage->id); } static void tdbPCacheRemovePageFromHash(SPCache *pCache, SPage *pPage) { uint32_t h = tdbPCachePageHash(&(pPage->pgid)) % pCache->nHash; SPage **ppPage = &(pCache->pgHash[h]); - if (*ppPage == pPage) { - pCache->pgHash[h] = pPage->pHashNext; - } else { - for (; (*ppPage) && (*ppPage)->pHashNext != pPage; ppPage = &((*ppPage)->pHashNext)) - ; - if (*ppPage) { - (*ppPage)->pHashNext = pPage->pHashNext; - } - } + for (; (*ppPage) && *ppPage != pPage; ppPage = &((*ppPage)->pHashNext)) + ; + if (*ppPage) { - pPage->pHashNext = NULL; - --pCache->nPage; + *ppPage = pPage->pHashNext; + pCache->nPage--; // printf("rmv page %d to hash, pgno %d, pPage %p\n", pPage->id, TDB_PAGE_PGNO(pPage), pPage); } - tdbDebug("remove page %p/%d from hash", pPage, pPage->id); + tdbDebug("pcache/remove page %p/%d/%d from hash %" PRIu32, pPage, TDB_PAGE_PGNO(pPage), pPage->id, h); } static void tdbPCacheAddPageToHash(SPCache *pCache, SPage *pPage) { @@ -298,7 +295,7 @@ static void tdbPCacheAddPageToHash(SPCache *pCache, SPage *pPage) { pCache->nPage++; // printf("add page %d to hash, pgno %d, pPage %p\n", pPage->id, TDB_PAGE_PGNO(pPage), pPage); - tdbDebug("add page %p/%d to hash", pPage, pPage->id); + tdbDebug("pcache/add page %p/%d/%d to hash %" PRIu32, pPage, TDB_PAGE_PGNO(pPage), pPage->id, h); } static int tdbPCacheOpenImpl(SPCache *pCache) { diff --git a/source/libs/tdb/src/db/tdbPage.c b/source/libs/tdb/src/db/tdbPage.c index 9e0cd76573..a3f376b929 100644 --- a/source/libs/tdb/src/db/tdbPage.c +++ b/source/libs/tdb/src/db/tdbPage.c @@ -68,6 +68,8 @@ int tdbPageCreate(int pageSize, SPage **ppPage, void *(*xMalloc)(void *, size_t) } *ppPage = pPage; + + tdbDebug("page/create: %p %p", pPage, xMalloc); return 0; } diff --git a/source/libs/tdb/src/db/tdbPager.c b/source/libs/tdb/src/db/tdbPager.c index 4de99e8b1b..f90c392788 100644 --- a/source/libs/tdb/src/db/tdbPager.c +++ b/source/libs/tdb/src/db/tdbPager.c @@ -166,6 +166,7 @@ int tdbPagerWrite(SPager *pPager, SPage *pPage) { // ref page one more time so the page will not be release tdbRefPage(pPage); + tdbDebug("pcache/mdirty page %p/%d/%d", pPage, TDB_PAGE_PGNO(pPage), pPage->id); // Set page as dirty pPage->isDirty = 1; diff --git a/source/libs/tdb/src/inc/tdbInt.h b/source/libs/tdb/src/inc/tdbInt.h index 49126b80b6..6a694cf8f1 100644 --- a/source/libs/tdb/src/inc/tdbInt.h +++ b/source/libs/tdb/src/inc/tdbInt.h @@ -280,13 +280,13 @@ struct SPage { static inline i32 tdbRefPage(SPage *pPage) { i32 nRef = atomic_add_fetch_32(&((pPage)->nRef), 1); - tdbTrace("ref page %d, nRef %d", pPage->id, nRef); + tdbTrace("ref page %p/%d, nRef %d", pPage, pPage->id, nRef); return nRef; } static inline i32 tdbUnrefPage(SPage *pPage) { i32 nRef = atomic_sub_fetch_32(&((pPage)->nRef), 1); - tdbTrace("unref page %d, nRef %d", pPage->id, nRef); + tdbTrace("unref page %p/%d, nRef %d", pPage, pPage->id, nRef); return nRef; } diff --git a/source/libs/transport/src/thttp.c b/source/libs/transport/src/thttp.c index 98e0b8f9c9..386ea95dd7 100644 --- a/source/libs/transport/src/thttp.c +++ b/source/libs/transport/src/thttp.c @@ -175,7 +175,7 @@ static int32_t taosBuildDstAddr(const char* server, uint16_t port, struct sockad uint32_t ip = taosGetIpv4FromFqdn(server); if (ip == 0xffffffff) { terrno = TAOS_SYSTEM_ERROR(errno); - uError("http-report failed to get http server:%s ip since %s", server, terrstr()); + uError("http-report failed to get http server:%s since %s", server, errno == 0 ? "invalid http server" : terrstr()); return -1; } char buf[128] = {0}; @@ -224,6 +224,7 @@ int32_t taosSendHttpReport(const char* server, uint16_t port, char* pCont, int32 if (ret != 0) { uError("http-report failed to connect to server, reason:%s, dst:%s:%d", uv_strerror(ret), cli->addr, cli->port); destroyHttpClient(cli); + uv_stop(loop); } uv_run(loop, UV_RUN_DEFAULT); diff --git a/source/os/src/osDir.c b/source/os/src/osDir.c index b755a35815..30aaa01dae 100644 --- a/source/os/src/osDir.c +++ b/source/os/src/osDir.c @@ -133,6 +133,7 @@ int32_t taosMulMkDir(const char *dirname) { code = mkdir(temp, 0755); #endif if (code < 0 && errno != EEXIST) { + terrno = TAOS_SYSTEM_ERROR(errno); return code; } *pos = TD_DIRSEP[0]; @@ -146,6 +147,7 @@ int32_t taosMulMkDir(const char *dirname) { code = mkdir(temp, 0755); #endif if (code < 0 && errno != EEXIST) { + terrno = TAOS_SYSTEM_ERROR(errno); return code; } } diff --git a/source/os/src/osFile.c b/source/os/src/osFile.c index f9797f6319..fab933755a 100644 --- a/source/os/src/osFile.c +++ b/source/os/src/osFile.c @@ -313,6 +313,7 @@ TdFilePtr taosOpenFile(const char *path, int32_t tdFileOptions) { assert(!(tdFileOptions & TD_FILE_EXCL)); fp = fopen(path, mode); if (fp == NULL) { + terrno = TAOS_SYSTEM_ERROR(errno); return NULL; } } else { @@ -335,6 +336,7 @@ TdFilePtr taosOpenFile(const char *path, int32_t tdFileOptions) { fd = open(path, access, S_IRWXU | S_IRWXG | S_IRWXO); #endif if (fd == -1) { + terrno = TAOS_SYSTEM_ERROR(errno); return NULL; } } diff --git a/source/os/src/osSemaphore.c b/source/os/src/osSemaphore.c index a7d2ba8531..8cc6f0ef2e 100644 --- a/source/os/src/osSemaphore.c +++ b/source/os/src/osSemaphore.c @@ -392,179 +392,32 @@ int32_t tsem_timewait(tsem_t* sem, int64_t nanosecs) { // *sem = NULL; // return 0; // } -typedef struct { - pthread_mutex_t count_lock; - pthread_cond_t count_bump; - unsigned int count; -} bosal_sem_t; int tsem_init(tsem_t *psem, int flags, unsigned int count) { - bosal_sem_t *pnewsem; - int result; - - pnewsem = (bosal_sem_t *)malloc(sizeof(bosal_sem_t)); - if (!pnewsem) { - return -1; - } - result = pthread_mutex_init(&pnewsem->count_lock, NULL); - if (result) { - free(pnewsem); - return result; - } - result = pthread_cond_init(&pnewsem->count_bump, NULL); - if (result) { - pthread_mutex_destroy(&pnewsem->count_lock); - free(pnewsem); - return result; - } - pnewsem->count = count; - *psem = (tsem_t)pnewsem; + *psem = dispatch_semaphore_create(count); + if (*psem == NULL) return -1; return 0; } int tsem_destroy(tsem_t *psem) { - bosal_sem_t *poldsem; - - if (!psem) { - return EINVAL; - } - poldsem = (bosal_sem_t *)*psem; - - pthread_mutex_destroy(&poldsem->count_lock); - pthread_cond_destroy(&poldsem->count_bump); - free(poldsem); return 0; } int tsem_post(tsem_t *psem) { - bosal_sem_t *pxsem; - int result, xresult; - - if (!psem) { - return EINVAL; - } - pxsem = (bosal_sem_t *)*psem; - - result = pthread_mutex_lock(&pxsem->count_lock); - if (result) { - return result; - } - pxsem->count = pxsem->count + 1; - - xresult = pthread_cond_signal(&pxsem->count_bump); - - result = pthread_mutex_unlock(&pxsem->count_lock); - if (result) { - return result; - } - if (xresult) { - errno = xresult; - return -1; - } - return 0; -} - -int tsem_trywait(tsem_t *psem) { - bosal_sem_t *pxsem; - int result, xresult; - - if (!psem) { - return EINVAL; - } - pxsem = (bosal_sem_t *)*psem; - - result = pthread_mutex_lock(&pxsem->count_lock); - if (result) { - return result; - } - xresult = 0; - - if (pxsem->count > 0) { - pxsem->count--; - } else { - xresult = EAGAIN; - } - result = pthread_mutex_unlock(&pxsem->count_lock); - if (result) { - return result; - } - if (xresult) { - errno = xresult; - return -1; - } + if (psem == NULL || *psem == NULL) return -1; + dispatch_semaphore_signal(*psem); return 0; } int tsem_wait(tsem_t *psem) { - bosal_sem_t *pxsem; - int result, xresult; - - if (!psem) { - return EINVAL; - } - pxsem = (bosal_sem_t *)*psem; - - result = pthread_mutex_lock(&pxsem->count_lock); - if (result) { - return result; - } - xresult = 0; - - if (pxsem->count == 0) { - xresult = pthread_cond_wait(&pxsem->count_bump, &pxsem->count_lock); - } - if (!xresult) { - if (pxsem->count > 0) { - pxsem->count--; - } - } - result = pthread_mutex_unlock(&pxsem->count_lock); - if (result) { - return result; - } - if (xresult) { - errno = xresult; - return -1; - } + if (psem == NULL || *psem == NULL) return -1; + dispatch_semaphore_wait(*psem, DISPATCH_TIME_FOREVER); return 0; } int tsem_timewait(tsem_t *psem, int64_t nanosecs) { - struct timespec abstim = { - .tv_sec = 0, - .tv_nsec = nanosecs, - }; - - bosal_sem_t *pxsem; - int result, xresult; - - if (!psem) { - return EINVAL; - } - pxsem = (bosal_sem_t *)*psem; - - result = pthread_mutex_lock(&pxsem->count_lock); - if (result) { - return result; - } - xresult = 0; - - if (pxsem->count == 0) { - xresult = pthread_cond_timedwait(&pxsem->count_bump, &pxsem->count_lock, &abstim); - } - if (!xresult) { - if (pxsem->count > 0) { - pxsem->count--; - } - } - result = pthread_mutex_unlock(&pxsem->count_lock); - if (result) { - return result; - } - if (xresult) { - errno = xresult; - return -1; - } + if (psem == NULL || *psem == NULL) return -1; + dispatch_semaphore_wait(*psem, nanosecs); return 0; } diff --git a/source/os/src/osSysinfo.c b/source/os/src/osSysinfo.c index 3aa3f4f29e..19e9568bbe 100644 --- a/source/os/src/osSysinfo.c +++ b/source/os/src/osSysinfo.c @@ -595,6 +595,7 @@ int32_t taosGetDiskSize(char *dataDir, SDiskSize *diskSize) { #else struct statvfs info; if (statvfs(dataDir, &info)) { + terrno = TAOS_SYSTEM_ERROR(errno); return -1; } else { diskSize->total = info.f_blocks * info.f_frsize; diff --git a/source/util/src/tcompare.c b/source/util/src/tcompare.c index 7d7a14483a..7032f39744 100644 --- a/source/util/src/tcompare.c +++ b/source/util/src/tcompare.c @@ -247,6 +247,756 @@ int32_t compareJsonVal(const void *pLeft, const void *pRight) { } } +int32_t compareInt8Int16(const void *pLeft, const void *pRight) { + int8_t left = GET_INT32_VAL(pLeft); + int16_t right = GET_INT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt8Int32(const void *pLeft, const void *pRight) { + int8_t left = GET_INT32_VAL(pLeft); + int32_t right = GET_INT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt8Int64(const void *pLeft, const void *pRight) { + int8_t left = GET_INT32_VAL(pLeft); + int64_t right = GET_INT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt8Float(const void *pLeft, const void *pRight) { + int8_t left = GET_INT32_VAL(pLeft); + float right = GET_FLOAT_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt8Double(const void *pLeft, const void *pRight) { + int8_t left = GET_INT32_VAL(pLeft); + double right = GET_DOUBLE_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt8Uint8(const void *pLeft, const void *pRight) { + int8_t left = GET_INT32_VAL(pLeft); + uint8_t right = GET_UINT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt8Uint16(const void *pLeft, const void *pRight) { + int8_t left = GET_INT32_VAL(pLeft); + uint16_t right = GET_UINT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt8Uint32(const void *pLeft, const void *pRight) { + int8_t left = GET_INT32_VAL(pLeft); + uint32_t right = GET_UINT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt8Uint64(const void *pLeft, const void *pRight) { + int8_t left = GET_INT32_VAL(pLeft); + uint64_t right = GET_UINT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt16Int8(const void *pLeft, const void *pRight) { + int16_t left = GET_INT32_VAL(pLeft); + int8_t right = GET_INT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt16Int32(const void *pLeft, const void *pRight) { + int16_t left = GET_INT32_VAL(pLeft); + int32_t right = GET_INT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt16Int64(const void *pLeft, const void *pRight) { + int16_t left = GET_INT32_VAL(pLeft); + int64_t right = GET_INT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt16Float(const void *pLeft, const void *pRight) { + int16_t left = GET_INT32_VAL(pLeft); + float right = GET_FLOAT_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt16Double(const void *pLeft, const void *pRight) { + int16_t left = GET_INT32_VAL(pLeft); + double right = GET_DOUBLE_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt16Uint8(const void *pLeft, const void *pRight) { + int16_t left = GET_INT32_VAL(pLeft); + uint8_t right = GET_UINT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt16Uint16(const void *pLeft, const void *pRight) { + int16_t left = GET_INT32_VAL(pLeft); + uint16_t right = GET_UINT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt16Uint32(const void *pLeft, const void *pRight) { + int16_t left = GET_INT32_VAL(pLeft); + uint32_t right = GET_UINT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt16Uint64(const void *pLeft, const void *pRight) { + int16_t left = GET_INT32_VAL(pLeft); + uint64_t right = GET_UINT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + + +int32_t compareInt32Int8(const void *pLeft, const void *pRight) { + int32_t left = GET_INT32_VAL(pLeft); + int8_t right = GET_INT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt32Int16(const void *pLeft, const void *pRight) { + int32_t left = GET_INT32_VAL(pLeft); + int16_t right = GET_INT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt32Int64(const void *pLeft, const void *pRight) { + int32_t left = GET_INT32_VAL(pLeft); + int64_t right = GET_INT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt32Float(const void *pLeft, const void *pRight) { + int32_t left = GET_INT32_VAL(pLeft); + float right = GET_FLOAT_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt32Double(const void *pLeft, const void *pRight) { + int32_t left = GET_INT32_VAL(pLeft); + double right = GET_DOUBLE_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt32Uint8(const void *pLeft, const void *pRight) { + int32_t left = GET_INT32_VAL(pLeft); + uint8_t right = GET_UINT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt32Uint16(const void *pLeft, const void *pRight) { + int32_t left = GET_INT32_VAL(pLeft); + uint16_t right = GET_UINT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt32Uint32(const void *pLeft, const void *pRight) { + int32_t left = GET_INT32_VAL(pLeft); + uint32_t right = GET_UINT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt32Uint64(const void *pLeft, const void *pRight) { + int32_t left = GET_INT32_VAL(pLeft); + uint64_t right = GET_UINT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt64Int8(const void *pLeft, const void *pRight) { + int64_t left = GET_INT64_VAL(pLeft); + int8_t right = GET_INT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt64Int16(const void *pLeft, const void *pRight) { + int64_t left = GET_INT64_VAL(pLeft); + int16_t right = GET_INT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt64Int32(const void *pLeft, const void *pRight) { + int64_t left = GET_INT64_VAL(pLeft); + int32_t right = GET_INT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt64Float(const void *pLeft, const void *pRight) { + int64_t left = GET_INT64_VAL(pLeft); + float right = GET_FLOAT_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt64Double(const void *pLeft, const void *pRight) { + int64_t left = GET_INT64_VAL(pLeft); + double right = GET_DOUBLE_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt64Uint8(const void *pLeft, const void *pRight) { + int64_t left = GET_INT64_VAL(pLeft); + uint8_t right = GET_UINT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt64Uint16(const void *pLeft, const void *pRight) { + int64_t left = GET_INT64_VAL(pLeft); + uint16_t right = GET_UINT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt64Uint32(const void *pLeft, const void *pRight) { + int64_t left = GET_INT64_VAL(pLeft); + uint32_t right = GET_UINT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareInt64Uint64(const void *pLeft, const void *pRight) { + int64_t left = GET_INT64_VAL(pLeft); + uint64_t right = GET_UINT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareFloatInt8(const void *pLeft, const void *pRight) { + float left = GET_FLOAT_VAL(pLeft); + int8_t right = GET_INT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareFloatInt16(const void *pLeft, const void *pRight) { + float left = GET_FLOAT_VAL(pLeft); + int16_t right = GET_INT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareFloatInt32(const void *pLeft, const void *pRight) { + float left = GET_FLOAT_VAL(pLeft); + int32_t right = GET_INT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareFloatInt64(const void *pLeft, const void *pRight) { + float left = GET_FLOAT_VAL(pLeft); + int64_t right = GET_INT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareFloatDouble(const void *pLeft, const void *pRight) { + float left = GET_FLOAT_VAL(pLeft); + double right = GET_DOUBLE_VAL(pRight); + + if (isnan(left) && isnan(right)) { + return 0; + } + + if (isnan(left)) { + return -1; + } + + if (isnan(right)) { + return 1; + } + + if (FLT_EQUAL(left, right)) { + return 0; + } + return FLT_GREATER(left, right) ? 1 : -1; +} + +int32_t compareFloatUint8(const void *pLeft, const void *pRight) { + float left = GET_FLOAT_VAL(pLeft); + uint8_t right = GET_UINT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareFloatUint16(const void *pLeft, const void *pRight) { + float left = GET_FLOAT_VAL(pLeft); + uint16_t right = GET_UINT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareFloatUint32(const void *pLeft, const void *pRight) { + float left = GET_FLOAT_VAL(pLeft); + uint32_t right = GET_UINT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareFloatUint64(const void *pLeft, const void *pRight) { + float left = GET_FLOAT_VAL(pLeft); + uint64_t right = GET_UINT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareDoubleInt8(const void *pLeft, const void *pRight) { + double left = GET_DOUBLE_VAL(pLeft); + int8_t right = GET_INT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareDoubleInt16(const void *pLeft, const void *pRight) { + double left = GET_DOUBLE_VAL(pLeft); + int16_t right = GET_INT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareDoubleInt32(const void *pLeft, const void *pRight) { + double left = GET_DOUBLE_VAL(pLeft); + int32_t right = GET_INT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareDoubleInt64(const void *pLeft, const void *pRight) { + double left = GET_DOUBLE_VAL(pLeft); + int64_t right = GET_INT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareDoubleFloat(const void *pLeft, const void *pRight) { + double left = GET_DOUBLE_VAL(pLeft); + float right = GET_FLOAT_VAL(pRight); + + if (isnan(left) && isnan(right)) { + return 0; + } + + if (isnan(left)) { + return -1; + } + + if (isnan(right)) { + return 1; + } + + if (FLT_EQUAL(left, right)) { + return 0; + } + return FLT_GREATER(left, right) ? 1 : -1; +} + +int32_t compareDoubleUint8(const void *pLeft, const void *pRight) { + double left = GET_DOUBLE_VAL(pLeft); + uint8_t right = GET_UINT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareDoubleUint16(const void *pLeft, const void *pRight) { + double left = GET_DOUBLE_VAL(pLeft); + uint16_t right = GET_UINT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareDoubleUint32(const void *pLeft, const void *pRight) { + double left = GET_DOUBLE_VAL(pLeft); + uint32_t right = GET_UINT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareDoubleUint64(const void *pLeft, const void *pRight) { + double left = GET_DOUBLE_VAL(pLeft); + uint64_t right = GET_UINT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint8Int8(const void *pLeft, const void *pRight) { + uint8_t left = GET_UINT8_VAL(pLeft); + int8_t right = GET_INT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint8Int16(const void *pLeft, const void *pRight) { + uint8_t left = GET_UINT8_VAL(pLeft); + int16_t right = GET_INT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint8Int32(const void *pLeft, const void *pRight) { + uint8_t left = GET_UINT8_VAL(pLeft); + int32_t right = GET_INT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint8Int64(const void *pLeft, const void *pRight) { + uint8_t left = GET_UINT8_VAL(pLeft); + int64_t right = GET_INT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint8Float(const void *pLeft, const void *pRight) { + uint8_t left = GET_UINT8_VAL(pLeft); + float right = GET_FLOAT_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint8Double(const void *pLeft, const void *pRight) { + uint8_t left = GET_UINT8_VAL(pLeft); + double right = GET_DOUBLE_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint8Uint16(const void *pLeft, const void *pRight) { + uint8_t left = GET_UINT8_VAL(pLeft); + uint16_t right = GET_UINT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint8Uint32(const void *pLeft, const void *pRight) { + uint8_t left = GET_UINT8_VAL(pLeft); + uint32_t right = GET_UINT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint8Uint64(const void *pLeft, const void *pRight) { + uint8_t left = GET_UINT8_VAL(pLeft); + uint64_t right = GET_UINT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint16Int8(const void *pLeft, const void *pRight) { + uint16_t left = GET_UINT16_VAL(pLeft); + int8_t right = GET_INT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint16Int16(const void *pLeft, const void *pRight) { + uint16_t left = GET_UINT16_VAL(pLeft); + int16_t right = GET_INT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint16Int32(const void *pLeft, const void *pRight) { + uint16_t left = GET_UINT16_VAL(pLeft); + int32_t right = GET_INT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint16Int64(const void *pLeft, const void *pRight) { + uint16_t left = GET_UINT16_VAL(pLeft); + int64_t right = GET_INT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint16Float(const void *pLeft, const void *pRight) { + uint16_t left = GET_UINT16_VAL(pLeft); + float right = GET_FLOAT_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint16Double(const void *pLeft, const void *pRight) { + uint16_t left = GET_UINT16_VAL(pLeft); + double right = GET_DOUBLE_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint16Uint8(const void *pLeft, const void *pRight) { + uint16_t left = GET_UINT16_VAL(pLeft); + uint8_t right = GET_UINT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint16Uint32(const void *pLeft, const void *pRight) { + uint16_t left = GET_UINT16_VAL(pLeft); + uint32_t right = GET_UINT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint16Uint64(const void *pLeft, const void *pRight) { + uint16_t left = GET_UINT16_VAL(pLeft); + uint64_t right = GET_UINT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint32Int8(const void *pLeft, const void *pRight) { + uint32_t left = GET_UINT32_VAL(pLeft); + int8_t right = GET_INT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint32Int16(const void *pLeft, const void *pRight) { + uint32_t left = GET_UINT32_VAL(pLeft); + int16_t right = GET_INT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint32Int32(const void *pLeft, const void *pRight) { + uint32_t left = GET_UINT32_VAL(pLeft); + int32_t right = GET_INT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint32Int64(const void *pLeft, const void *pRight) { + uint32_t left = GET_UINT32_VAL(pLeft); + int64_t right = GET_INT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint32Float(const void *pLeft, const void *pRight) { + uint32_t left = GET_UINT32_VAL(pLeft); + float right = GET_FLOAT_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint32Double(const void *pLeft, const void *pRight) { + uint32_t left = GET_UINT32_VAL(pLeft); + double right = GET_DOUBLE_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint32Uint8(const void *pLeft, const void *pRight) { + uint32_t left = GET_UINT32_VAL(pLeft); + uint8_t right = GET_UINT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint32Uint16(const void *pLeft, const void *pRight) { + uint32_t left = GET_UINT32_VAL(pLeft); + uint16_t right = GET_UINT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint32Uint64(const void *pLeft, const void *pRight) { + uint32_t left = GET_UINT32_VAL(pLeft); + uint64_t right = GET_UINT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint64Int8(const void *pLeft, const void *pRight) { + uint64_t left = GET_UINT64_VAL(pLeft); + int8_t right = GET_INT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint64Int16(const void *pLeft, const void *pRight) { + uint64_t left = GET_UINT64_VAL(pLeft); + int16_t right = GET_INT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint64Int32(const void *pLeft, const void *pRight) { + uint64_t left = GET_UINT64_VAL(pLeft); + int32_t right = GET_INT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint64Int64(const void *pLeft, const void *pRight) { + uint64_t left = GET_UINT64_VAL(pLeft); + int64_t right = GET_INT64_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint64Float(const void *pLeft, const void *pRight) { + uint64_t left = GET_UINT64_VAL(pLeft); + float right = GET_FLOAT_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint64Double(const void *pLeft, const void *pRight) { + uint64_t left = GET_UINT64_VAL(pLeft); + double right = GET_DOUBLE_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint64Uint8(const void *pLeft, const void *pRight) { + uint64_t left = GET_UINT64_VAL(pLeft); + uint8_t right = GET_UINT8_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint64Uint16(const void *pLeft, const void *pRight) { + uint64_t left = GET_UINT64_VAL(pLeft); + uint16_t right = GET_UINT16_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + +int32_t compareUint64Uint32(const void *pLeft, const void *pRight) { + uint64_t left = GET_UINT64_VAL(pLeft); + uint32_t right = GET_UINT32_VAL(pRight); + if (left > right) return 1; + if (left < right) return -1; + return 0; +} + + int32_t compareJsonValDesc(const void *pLeft, const void *pRight) { return compareJsonVal(pRight, pLeft); } diff --git a/source/util/src/thash.c b/source/util/src/thash.c index aee84a0d55..b69d8ea528 100644 --- a/source/util/src/thash.c +++ b/source/util/src/thash.c @@ -21,7 +21,7 @@ // the add ref count operation may trigger the warning if the reference count is greater than the MAX_WARNING_REF_COUNT #define MAX_WARNING_REF_COUNT 10000 -#define HASH_MAX_CAPACITY (1024 * 1024 * 16) +#define HASH_MAX_CAPACITY (1024 * 1024 * 1024) #define HASH_DEFAULT_LOAD_FACTOR (0.75) #define HASH_INDEX(v, c) ((v) & ((c)-1)) @@ -67,6 +67,7 @@ struct SHashObj { bool enableUpdate; // enable update SArray *pMemBlock; // memory block allocated for SHashEntry _hash_before_fn_t callbackFp; // function invoked before return the value to caller + int64_t compTimes; }; /* @@ -146,6 +147,7 @@ static FORCE_INLINE SHashNode *doSearchInEntryList(SHashObj *pHashObj, SHashEntr uint32_t hashVal) { SHashNode *pNode = pe->next; while (pNode) { + atomic_add_fetch_64(&pHashObj->compTimes, 1); if ((pNode->keyLen == keyLen) && ((*(pHashObj->equalFp))(GET_HASH_NODE_KEY(pNode), key, keyLen) == 0) && pNode->removed == 0) { assert(pNode->hashVal == hashVal); @@ -250,11 +252,15 @@ SHashObj *taosHashInit(size_t capacity, _hash_fn_t fn, bool update, SHashLockTyp // the max slots is not defined by user pHashObj->capacity = taosHashCapacity((int32_t)capacity); + pHashObj->size = 0; pHashObj->equalFp = memcmp; pHashObj->hashFp = fn; pHashObj->type = type; + pHashObj->lock = 0; pHashObj->enableUpdate = update; + pHashObj->freeFp = NULL; + pHashObj->callbackFp = NULL; ASSERT((pHashObj->capacity & (pHashObj->capacity - 1)) == 0); @@ -327,7 +333,7 @@ int32_t taosHashPut(SHashObj *pHashObj, const void *key, size_t keyLen, const vo // disable resize taosHashRLock(pHashObj); - int32_t slot = HASH_INDEX(hashVal, pHashObj->capacity); + uint32_t slot = HASH_INDEX(hashVal, pHashObj->capacity); SHashEntry *pe = pHashObj->hashList[slot]; taosHashEntryWLock(pHashObj, pe); @@ -882,3 +888,7 @@ void *taosHashAcquire(SHashObj *pHashObj, const void *key, size_t keyLen) { } void taosHashRelease(SHashObj *pHashObj, void *p) { taosHashCancelIterate(pHashObj, p); } + +int64_t taosHashGetCompTimes(SHashObj *pHashObj) { return atomic_load_64(&pHashObj->compTimes); } + + diff --git a/source/util/src/tlog.c b/source/util/src/tlog.c index 2e8239c68f..a2d65d6a54 100644 --- a/source/util/src/tlog.c +++ b/source/util/src/tlog.c @@ -429,7 +429,7 @@ static inline int32_t taosBuildLogHead(char *buffer, const char *flags) { } static inline void taosPrintLogImp(ELogLevel level, int32_t dflag, const char *buffer, int32_t len) { - if ((dflag & DEBUG_FILE) && tsLogObj.logHandle && tsLogObj.logHandle->pFile != NULL) { + if ((dflag & DEBUG_FILE) && tsLogObj.logHandle && tsLogObj.logHandle->pFile != NULL && osLogSpaceAvailable()) { taosUpdateLogNums(level); if (tsAsyncLog) { taosPushLogBuffer(tsLogObj.logHandle, buffer, len); @@ -451,7 +451,6 @@ static inline void taosPrintLogImp(ELogLevel level, int32_t dflag, const char *b } void taosPrintLog(const char *flags, ELogLevel level, int32_t dflag, const char *format, ...) { - if (!osLogSpaceAvailable()) return; if (!(dflag & DEBUG_FILE) && !(dflag & DEBUG_SCREEN)) return; char buffer[LOG_MAX_LINE_BUFFER_SIZE]; diff --git a/source/util/test/hashTest.cpp b/source/util/test/hashTest.cpp index 99f5a761c5..97e67ea36e 100644 --- a/source/util/test/hashTest.cpp +++ b/source/util/test/hashTest.cpp @@ -197,6 +197,201 @@ void acquireRleaseTest() { taosMemoryFreeClear(data.p); } +void perfTest() { + SHashObj* hash1h = (SHashObj*) taosHashInit(100, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_NO_LOCK); + SHashObj* hash1s = (SHashObj*) taosHashInit(1000, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_NO_LOCK); + SHashObj* hash10s = (SHashObj*) taosHashInit(10000, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_NO_LOCK); + SHashObj* hash100s = (SHashObj*) taosHashInit(100000, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_NO_LOCK); + SHashObj* hash1m = (SHashObj*) taosHashInit(1000000, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_NO_LOCK); + SHashObj* hash10m = (SHashObj*) taosHashInit(10000000, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_NO_LOCK); + SHashObj* hash100m = (SHashObj*) taosHashInit(100000000, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_NO_LOCK); + + char *name = (char*)taosMemoryCalloc(50000000, 9); + for (int64_t i = 0; i < 50000000; ++i) { + sprintf(name + i * 9, "t%08d", i); + } + + for (int64_t i = 0; i < 50; ++i) { + taosHashPut(hash1h, name + i * 9, 9, &i, sizeof(i)); + } + + for (int64_t i = 0; i < 500; ++i) { + taosHashPut(hash1s, name + i * 9, 9, &i, sizeof(i)); + } + + for (int64_t i = 0; i < 5000; ++i) { + taosHashPut(hash10s, name + i * 9, 9, &i, sizeof(i)); + } + + for (int64_t i = 0; i < 50000; ++i) { + taosHashPut(hash100s, name + i * 9, 9, &i, sizeof(i)); + } + + for (int64_t i = 0; i < 500000; ++i) { + taosHashPut(hash1m, name + i * 9, 9, &i, sizeof(i)); + } + + for (int64_t i = 0; i < 5000000; ++i) { + taosHashPut(hash10m, name + i * 9, 9, &i, sizeof(i)); + } + + for (int64_t i = 0; i < 50000000; ++i) { + taosHashPut(hash100m, name + i * 9, 9, &i, sizeof(i)); + } + + int64_t start1h = taosGetTimestampMs(); + int64_t start1hCt = taosHashGetCompTimes(hash1h); + for (int64_t i = 0; i < 10000000; ++i) { + ASSERT(taosHashGet(hash1h, name + (i % 50) * 9, 9)); + } + int64_t end1h = taosGetTimestampMs(); + int64_t end1hCt = taosHashGetCompTimes(hash1h); + + int64_t start1s = taosGetTimestampMs(); + int64_t start1sCt = taosHashGetCompTimes(hash1s); + for (int64_t i = 0; i < 10000000; ++i) { + ASSERT(taosHashGet(hash1s, name + (i % 500) * 9, 9)); + } + int64_t end1s = taosGetTimestampMs(); + int64_t end1sCt = taosHashGetCompTimes(hash1s); + + int64_t start10s = taosGetTimestampMs(); + int64_t start10sCt = taosHashGetCompTimes(hash10s); + for (int64_t i = 0; i < 10000000; ++i) { + ASSERT(taosHashGet(hash10s, name + (i % 5000) * 9, 9)); + } + int64_t end10s = taosGetTimestampMs(); + int64_t end10sCt = taosHashGetCompTimes(hash10s); + + int64_t start100s = taosGetTimestampMs(); + int64_t start100sCt = taosHashGetCompTimes(hash100s); + for (int64_t i = 0; i < 10000000; ++i) { + ASSERT(taosHashGet(hash100s, name + (i % 50000) * 9, 9)); + } + int64_t end100s = taosGetTimestampMs(); + int64_t end100sCt = taosHashGetCompTimes(hash100s); + + int64_t start1m = taosGetTimestampMs(); + int64_t start1mCt = taosHashGetCompTimes(hash1m); + for (int64_t i = 0; i < 10000000; ++i) { + ASSERT(taosHashGet(hash1m, name + (i % 500000) * 9, 9)); + } + int64_t end1m = taosGetTimestampMs(); + int64_t end1mCt = taosHashGetCompTimes(hash1m); + + int64_t start10m = taosGetTimestampMs(); + int64_t start10mCt = taosHashGetCompTimes(hash10m); + for (int64_t i = 0; i < 10000000; ++i) { + ASSERT(taosHashGet(hash10m, name + (i % 5000000) * 9, 9)); + } + int64_t end10m = taosGetTimestampMs(); + int64_t end10mCt = taosHashGetCompTimes(hash10m); + + int64_t start100m = taosGetTimestampMs(); + int64_t start100mCt = taosHashGetCompTimes(hash100m); + for (int64_t i = 0; i < 10000000; ++i) { + ASSERT(taosHashGet(hash100m, name + (i % 50000000) * 9, 9)); + } + int64_t end100m = taosGetTimestampMs(); + int64_t end100mCt = taosHashGetCompTimes(hash100m); + + + SArray *sArray[1000] = {0}; + for (int64_t i = 0; i < 1000; ++i) { + sArray[i] = taosArrayInit(100000, 9); + } + int64_t cap = 4; + while (cap < 100000000) cap = (cap << 1u); + + _hash_fn_t hashFp = taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY); + int32_t slotR = cap / 1000 + 1; + for (int64_t i = 0; i < 10000000; ++i) { + char* p = name + (i % 50000000) * 9; + uint32_t v = (*hashFp)(p, 9); + taosArrayPush(sArray[(v%cap)/slotR], p); + } + SArray *slArray = taosArrayInit(100000000, 9); + for (int64_t i = 0; i < 1000; ++i) { + int32_t num = taosArrayGetSize(sArray[i]); + SArray* pArray = sArray[i]; + for (int64_t m = 0; m < num; ++m) { + char* p = (char*)taosArrayGet(pArray, m); + ASSERT(taosArrayPush(slArray, p)); + } + } + int64_t start100mS = taosGetTimestampMs(); + int64_t start100mSCt = taosHashGetCompTimes(hash100m); + int32_t num = taosArrayGetSize(slArray); + for (int64_t i = 0; i < num; ++i) { + ASSERT(taosHashGet(hash100m, (char*)TARRAY_GET_ELEM(slArray, i), 9)); + } + int64_t end100mS = taosGetTimestampMs(); + int64_t end100mSCt = taosHashGetCompTimes(hash100m); + for (int64_t i = 0; i < 1000; ++i) { + taosArrayDestroy(sArray[i]); + } + taosArrayDestroy(slArray); + + printf("1h \t %" PRId64 "ms,%" PRId64 "\n", end1h - start1h, end1hCt - start1hCt); + printf("1s \t %" PRId64 "ms,%" PRId64 "\n", end1s - start1s, end1sCt - start1sCt); + printf("10s \t %" PRId64 "ms,%" PRId64 "\n", end10s - start10s, end10sCt - start10sCt); + printf("100s \t %" PRId64 "ms,%" PRId64 "\n", end100s - start100s, end100sCt - start100sCt); + printf("1m \t %" PRId64 "ms,%" PRId64 "\n", end1m - start1m, end1mCt - start1mCt); + printf("10m \t %" PRId64 "ms,%" PRId64 "\n", end10m - start10m, end10mCt - start10mCt); + printf("100m \t %" PRId64 "ms,%" PRId64 "\n", end100m - start100m, end100mCt - start100mCt); + printf("100mS \t %" PRId64 "ms,%" PRId64 "\n", end100mS - start100mS, end100mSCt - start100mSCt); + + taosHashCleanup(hash1h); + taosHashCleanup(hash1s); + taosHashCleanup(hash10s); + taosHashCleanup(hash100s); + taosHashCleanup(hash1m); + taosHashCleanup(hash10m); + taosHashCleanup(hash100m); + + SHashObj *mhash[1000] = {0}; + for (int64_t i = 0; i < 1000; ++i) { + mhash[i] = (SHashObj*) taosHashInit(100000, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_NO_LOCK); + } + + for (int64_t i = 0; i < 50000000; ++i) { +#if 0 + taosHashPut(mhash[i%1000], name + i * 9, 9, &i, sizeof(i)); +#else + taosHashPut(mhash[i/50000], name + i * 9, 9, &i, sizeof(i)); +#endif + } + + int64_t startMhashCt = 0; + for (int64_t i = 0; i < 1000; ++i) { + startMhashCt += taosHashGetCompTimes(mhash[i]); + } + + int64_t startMhash = taosGetTimestampMs(); +#if 0 + for (int32_t i = 0; i < 10000000; ++i) { + ASSERT(taosHashGet(mhash[i%1000], name + i * 9, 9)); + } +#else +// for (int64_t i = 0; i < 10000000; ++i) { + for (int64_t i = 0; i < 50000000; i+=5) { + ASSERT(taosHashGet(mhash[i/50000], name + i * 9, 9)); + } +#endif + int64_t endMhash = taosGetTimestampMs(); + int64_t endMhashCt = 0; + for (int64_t i = 0; i < 1000; ++i) { + printf(" %" PRId64 , taosHashGetCompTimes(mhash[i])); + endMhashCt += taosHashGetCompTimes(mhash[i]); + } + printf("\n100m \t %" PRId64 "ms,%" PRId64 "\n", endMhash - startMhash, endMhashCt - startMhashCt); + + for (int64_t i = 0; i < 1000; ++i) { + taosHashCleanup(mhash[i]); + } +} + + } int main(int argc, char** argv) { @@ -210,4 +405,5 @@ TEST(testCase, hashTest) { noLockPerformanceTest(); multithreadsTest(); acquireRleaseTest(); + //perfTest(); } diff --git a/tests/docs-examples-test/jdbc.sh b/tests/docs-examples-test/jdbc.sh new file mode 100644 index 0000000000..d71085a403 --- /dev/null +++ b/tests/docs-examples-test/jdbc.sh @@ -0,0 +1,27 @@ +#!/bin/bash + +pgrep taosd || taosd >> /dev/null 2>&1 & +pgrep taosadapter || taosadapter >> /dev/null 2>&1 & +cd ../../docs/examples/java + +mvn clean test > jdbc-out.log 2>&1 +tail -n 20 jdbc-out.log + +cases=`grep 'Tests run' jdbc-out.log | awk 'END{print $3}'` +totalJDBCCases=`echo ${cases/%,}` +failed=`grep 'Tests run' jdbc-out.log | awk 'END{print $5}'` +JDBCFailed=`echo ${failed/%,}` +error=`grep 'Tests run' jdbc-out.log | awk 'END{print $7}'` +JDBCError=`echo ${error/%,}` + +totalJDBCFailed=`expr $JDBCFailed + $JDBCError` +totalJDBCSuccess=`expr $totalJDBCCases - $totalJDBCFailed` + +if [ "$totalJDBCSuccess" -gt "0" ]; then + echo -e "\n${GREEN} ### Total $totalJDBCSuccess JDBC case(s) succeed! ### ${NC}" +fi + +if [ "$totalJDBCFailed" -ne "0" ]; then + echo -e "\n${RED} ### Total $totalJDBCFailed JDBC case(s) failed! ### ${NC}" + exit 8 +fi \ No newline at end of file diff --git a/tests/script/api/batchprepare.c b/tests/script/api/batchprepare.c index ada2039460..f39d5e6528 100644 --- a/tests/script/api/batchprepare.c +++ b/tests/script/api/batchprepare.c @@ -2598,7 +2598,6 @@ void runAll(TAOS *taos) { printf("%s Begin\n", gCaseCtrl.caseCatalog); runCaseList(taos); -#if 0 strcpy(gCaseCtrl.caseCatalog, "Micro DB precision Test"); printf("%s Begin\n", gCaseCtrl.caseCatalog); gCaseCtrl.precision = TIME_PRECISION_MICRO; @@ -2654,7 +2653,6 @@ void runAll(TAOS *taos) { gCaseCtrl.bindColNum = 6; runCaseList(taos); gCaseCtrl.bindColNum = 0; -#endif /* strcpy(gCaseCtrl.caseCatalog, "Bind Col Type Test"); diff --git a/tests/script/tmp/monitor.sim b/tests/script/tmp/monitor.sim index c0c1da567c..b410e1b6ad 100644 --- a/tests/script/tmp/monitor.sim +++ b/tests/script/tmp/monitor.sim @@ -4,6 +4,7 @@ system sh/cfg.sh -n dnode1 -c monitorfqdn -v localhost system sh/cfg.sh -n dnode1 -c monitorport -v 80 system sh/cfg.sh -n dnode1 -c monitorInterval -v 1 system sh/cfg.sh -n dnode1 -c monitorComp -v 1 +system sh/cfg.sh -n dnode1 -c uptimeInterval -v 3 #system sh/cfg.sh -n dnode1 -c supportVnodes -v 128 #system sh/cfg.sh -n dnode1 -c telemetryReporting -v 1 @@ -14,7 +15,7 @@ system sh/cfg.sh -n dnode1 -c monitorComp -v 1 system sh/exec.sh -n dnode1 -s start sql connect -print =============== select * from information_schema.ins_dnodes +print =============== create database sql create database db vgroups 2; sql use db; sql create table db.stb (ts timestamp, c1 int, c2 binary(4)) tags(t1 int, t2 binary(16)) comment "abd"; diff --git a/tests/script/tsim/parser/alter1.sim b/tests/script/tsim/parser/alter1.sim index 9d0049e45e..369419dcd9 100644 --- a/tests/script/tsim/parser/alter1.sim +++ b/tests/script/tsim/parser/alter1.sim @@ -130,4 +130,4 @@ endi # return -1 #endi -system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/parser/binary_escapeCharacter.sim b/tests/script/tsim/parser/binary_escapeCharacter.sim index 0b437d8b04..5a9c0e7bb1 100644 --- a/tests/script/tsim/parser/binary_escapeCharacter.sim +++ b/tests/script/tsim/parser/binary_escapeCharacter.sim @@ -101,4 +101,4 @@ sql_error insert into tb values(now, '\'); #sql_error insert into tb values(now, '\\\n'); sql insert into tb values(now, '\n'); -system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/parser/col_arithmetic_operation.sim b/tests/script/tsim/parser/col_arithmetic_operation.sim index f22beefdf8..9a2ba34c85 100644 --- a/tests/script/tsim/parser/col_arithmetic_operation.sim +++ b/tests/script/tsim/parser/col_arithmetic_operation.sim @@ -132,4 +132,4 @@ sql_error select max(c1-c2) from $tb print =====================> td-1764 sql select sum(c1)/count(*), sum(c1) as b, count(*) as b from $stb interval(1y) -system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/parser/columnValue_unsign.sim b/tests/script/tsim/parser/columnValue_unsign.sim index 85ff490bf4..7ae1b20eca 100644 --- a/tests/script/tsim/parser/columnValue_unsign.sim +++ b/tests/script/tsim/parser/columnValue_unsign.sim @@ -129,4 +129,4 @@ if $rows != 1 then return -1 endi -system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/parser/fill_stb.sim b/tests/script/tsim/parser/fill_stb.sim index 656b1ac94e..6c61631aa8 100644 --- a/tests/script/tsim/parser/fill_stb.sim +++ b/tests/script/tsim/parser/fill_stb.sim @@ -279,7 +279,7 @@ endi #endi ## linear fill -sql select _wstart, max(c1), min(c2), avg(c3), sum(c4), first(c7), last(c8), first(c9) from $stb where ts >= $ts0 and ts <= $tsu partition by t1 interval(5m) fill(linear) +sql select _wstart, max(c1), min(c2), avg(c3), sum(c4), first(c7), last(c8), first(c9) from $stb where ts >= $ts0 and ts <= $tsu partition by t1 interval(5m) fill(linear) $val = $rowNum * 2 $val = $val - 1 $val = $val * $tbNum diff --git a/tests/script/tsim/parser/import_file.sim b/tests/script/tsim/parser/import_file.sim index e031e0249d..37dc0c4476 100644 --- a/tests/script/tsim/parser/import_file.sim +++ b/tests/script/tsim/parser/import_file.sim @@ -69,4 +69,4 @@ endi system rm -f $inFileName -system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/parser/repeatAlter.sim b/tests/script/tsim/parser/repeatAlter.sim index d28a03e193..b4012048cc 100644 --- a/tests/script/tsim/parser/repeatAlter.sim +++ b/tests/script/tsim/parser/repeatAlter.sim @@ -6,4 +6,4 @@ while $i <= $loops $i = $i + 1 endw -system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/parser/select_from_cache_disk.sim b/tests/script/tsim/parser/select_from_cache_disk.sim index 0983e36a3a..3c0b13c638 100644 --- a/tests/script/tsim/parser/select_from_cache_disk.sim +++ b/tests/script/tsim/parser/select_from_cache_disk.sim @@ -60,4 +60,4 @@ if $data12 != 1 then return -1 endi -system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/parser/single_row_in_tb.sim b/tests/script/tsim/parser/single_row_in_tb.sim index 1bd53ad24e..e7b4c9a871 100644 --- a/tests/script/tsim/parser/single_row_in_tb.sim +++ b/tests/script/tsim/parser/single_row_in_tb.sim @@ -33,4 +33,4 @@ print ================== server restart completed run tsim/parser/single_row_in_tb_query.sim -system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/parser/single_row_in_tb_query.sim b/tests/script/tsim/parser/single_row_in_tb_query.sim index 422756b798..37e193f9d2 100644 --- a/tests/script/tsim/parser/single_row_in_tb_query.sim +++ b/tests/script/tsim/parser/single_row_in_tb_query.sim @@ -195,4 +195,4 @@ endi print ===============>safty check TD-4927 sql select first(ts, c1) from sr_stb where ts<1 group by t1; -sql select first(ts, c1) from sr_stb where ts>0 and ts<1; \ No newline at end of file +sql select first(ts, c1) from sr_stb where ts>0 and ts<1; diff --git a/tests/script/tsim/query/complex_group.sim b/tests/script/tsim/query/complex_group.sim index 3dad8059cd..d7d14c0ee8 100644 --- a/tests/script/tsim/query/complex_group.sim +++ b/tests/script/tsim/query/complex_group.sim @@ -454,4 +454,4 @@ if $rows != 1 then return -1 endi -system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/query/complex_having.sim b/tests/script/tsim/query/complex_having.sim index 9e28c3803e..4c0af6d10c 100644 --- a/tests/script/tsim/query/complex_having.sim +++ b/tests/script/tsim/query/complex_having.sim @@ -365,4 +365,4 @@ if $rows != 1 then return -1 endi -system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/query/complex_limit.sim b/tests/script/tsim/query/complex_limit.sim index 2a90e7ff1d..acb133f650 100644 --- a/tests/script/tsim/query/complex_limit.sim +++ b/tests/script/tsim/query/complex_limit.sim @@ -508,4 +508,4 @@ if $rows != 1 then return -1 endi -system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/query/complex_select.sim b/tests/script/tsim/query/complex_select.sim index f4c9877bfd..b7697e5cab 100644 --- a/tests/script/tsim/query/complex_select.sim +++ b/tests/script/tsim/query/complex_select.sim @@ -558,4 +558,4 @@ if $data00 != 33 then endi -system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/query/complex_where.sim b/tests/script/tsim/query/complex_where.sim index bda1c036f0..847f67ed34 100644 --- a/tests/script/tsim/query/complex_where.sim +++ b/tests/script/tsim/query/complex_where.sim @@ -669,4 +669,4 @@ if $rows != 1 then return -1 endi -system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/query/crash_sql.sim b/tests/script/tsim/query/crash_sql.sim index 1d20491869..79a9165e66 100644 --- a/tests/script/tsim/query/crash_sql.sim +++ b/tests/script/tsim/query/crash_sql.sim @@ -76,7 +76,7 @@ sql insert into ct4 values ( '2022-05-21 01:01:01.000', NULL, NULL, NULL, NULL, print ================ start query ====================== -print ================ SQL used to cause taosd or taos shell crash +print ================ SQL used to cause taosd or TDengine CLI crash sql_error select sum(c1) ,count(c1) from ct4 group by c1 having sum(c10) between 0 and 1 ; -#system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +#system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/query/diff.sim b/tests/script/tsim/query/diff.sim index f0d82b01e9..badd139a9f 100644 --- a/tests/script/tsim/query/diff.sim +++ b/tests/script/tsim/query/diff.sim @@ -25,17 +25,17 @@ $i = 0 while $i < $tbNum $tb = $tbPrefix . $i sql create table $tb using $mt tags( $i ) - + $x = 0 while $x < $rowNum $cc = $x * 60000 $ms = 1601481600000 + $cc - sql insert into $tb values ($ms , $x ) + sql insert into $tb values ($ms , $x ) $x = $x + 1 - endw - + endw + $i = $i + 1 -endw +endw sleep 100 @@ -61,7 +61,7 @@ sql select _rowts, diff(tbcol) from $tb where ts > $ms print ===> rows: $rows print ===> $data00 $data01 $data02 $data03 $data04 $data05 print ===> $data10 $data11 $data12 $data13 $data14 $data15 -if $data11 != 1 then +if $data11 != 1 then return -1 endi @@ -72,7 +72,7 @@ sql select _rowts, diff(tbcol) from $tb where ts <= $ms print ===> rows: $rows print ===> $data00 $data01 $data02 $data03 $data04 $data05 print ===> $data10 $data11 $data12 $data13 $data14 $data15 -if $data11 != 1 then +if $data11 != 1 then return -1 endi @@ -82,7 +82,7 @@ sql select _rowts, diff(tbcol) as b from $tb print ===> rows: $rows print ===> $data00 $data01 $data02 $data03 $data04 $data05 print ===> $data10 $data11 $data12 $data13 $data14 $data15 -if $data11 != 1 then +if $data11 != 1 then return -1 endi @@ -107,4 +107,4 @@ if $rows != 2 then return -1 endi -system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/query/explain.sim b/tests/script/tsim/query/explain.sim index 30a857815c..2871252d91 100644 --- a/tests/script/tsim/query/explain.sim +++ b/tests/script/tsim/query/explain.sim @@ -3,7 +3,7 @@ system sh/deploy.sh -n dnode1 -i 1 system sh/exec.sh -n dnode1 -s start sql connect -print ======== step1 +print ======== step1 sql create database db1 vgroups 3; sql use db1; sql select * from information_schema.ins_databases; @@ -30,7 +30,7 @@ sql insert into tb4 values (now, 4, "Bitmap Heap Scan on tenk1 t1 (cost=5.07..2 #sql insert into tb4 values (now, 4, "Bitmap Heap Scan on tenk1 t1 (cost=5.07..229.20 rows=101 width=244) (actual time=0.080..0.526 rows=100 loops=1)"); -print ======== step2 +print ======== step2 sql explain select * from st1 where -2; sql explain select ts from tb1; sql explain select * from st1; @@ -41,14 +41,14 @@ sql explain select count(*),sum(f1) from st1; sql explain select count(*),sum(f1) from st1 group by f1; #sql explain select count(f1) from tb1 interval(10s, 2s) sliding(3s) fill(prev); -print ======== step3 +print ======== step3 sql explain verbose true select * from st1 where -2; sql explain verbose true select ts from tb1 where f1 > 0; sql explain verbose true select * from st1 where f1 > 0 and ts > '2020-10-31 00:00:00' and ts < '2021-10-31 00:00:00'; sql explain verbose true select count(*) from st1 partition by tbname slimit 1 soffset 2 limit 2 offset 1; sql explain verbose true select * from information_schema.ins_stables where db_name='db2'; -print ======== step4 +print ======== step4 sql explain analyze select ts from st1 where -2; sql explain analyze select ts from tb1; sql explain analyze select ts from st1; @@ -59,7 +59,7 @@ sql explain analyze select count(*),sum(f1) from tb1; sql explain analyze select count(*),sum(f1) from st1; sql explain analyze select count(*),sum(f1) from st1 group by f1; -print ======== step5 +print ======== step5 sql explain analyze verbose true select ts from st1 where -2; sql explain analyze verbose true select ts from tb1; sql explain analyze verbose true select ts from st1; @@ -87,12 +87,12 @@ sql explain analyze verbose true select count(f1) from st1 group by tbname; #sql explain select * from tb1, tb2 where tb1.ts=tb2.ts; #sql explain select * from st1, st2 where tb1.ts=tb2.ts; #sql explain analyze verbose true select sum(a+b) from (select _rowts, min(f1) b,count(*) a from st1 where f1 > 0 interval(1a)) where a < 0 interval(1s); -#sql explain select min(f1) from st1 interval(1m, 2a) sliding(30s); +#sql explain select min(f1) from st1 interval(1m, 2a) sliding(30s); #sql explain verbose true select count(*),sum(f1) from st1 where f1 > 0 and ts > '2021-10-31 00:00:00' group by f1 having sum(f1) > 0; -#sql explain analyze select min(f1) from st1 interval(3m, 2a) sliding(1m); +#sql explain analyze select min(f1) from st1 interval(3m, 2a) sliding(1m); #sql explain analyze select count(f1) from tb1 interval(10s, 2s) sliding(3s) fill(prev); #sql explain analyze verbose true select count(*),sum(f1) from st1 where f1 > 0 and ts > '2021-10-31 00:00:00' group by f1 having sum(f1) > 0; -#sql explain analyze verbose true select min(f1) from st1 interval(3m, 2a) sliding(1m); +#sql explain analyze verbose true select min(f1) from st1 interval(3m, 2a) sliding(1m); system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/query/interval.sim b/tests/script/tsim/query/interval.sim index cc8a73daec..833da4a8ba 100644 --- a/tests/script/tsim/query/interval.sim +++ b/tests/script/tsim/query/interval.sim @@ -177,4 +177,4 @@ print =============== clear # return -1 #endi -system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/query/scalarFunction.sim b/tests/script/tsim/query/scalarFunction.sim index 103e66e54e..1b8115fec6 100644 --- a/tests/script/tsim/query/scalarFunction.sim +++ b/tests/script/tsim/query/scalarFunction.sim @@ -33,7 +33,7 @@ print =============== create normal table sql create table ntb (ts timestamp, c1 int, c2 float, c3 double) sql show tables -if $rows != 101 then +if $rows != 101 then return -1 endi @@ -444,7 +444,7 @@ if $loop_test == 0 then print =============== stop and restart taosd system sh/exec.sh -n dnode1 -s stop -x SIGINT system sh/exec.sh -n dnode1 -s start - + $loop_cnt = 0 check_dnode_ready_0: $loop_cnt = $loop_cnt + 1 @@ -462,7 +462,7 @@ if $loop_test == 0 then goto check_dnode_ready_0 endi - $loop_test = 1 + $loop_test = 1 goto loop_test_pos endi diff --git a/tests/script/tsim/query/scalarNull.sim b/tests/script/tsim/query/scalarNull.sim index ec95c94f23..6abe3d62d9 100644 --- a/tests/script/tsim/query/scalarNull.sim +++ b/tests/script/tsim/query/scalarNull.sim @@ -3,7 +3,7 @@ system sh/deploy.sh -n dnode1 -i 1 system sh/exec.sh -n dnode1 -s start sql connect -print ======== step1 +print ======== step1 sql create database db1 vgroups 3; sql use db1; sql select * from information_schema.ins_databases; diff --git a/tests/script/tsim/query/session.sim b/tests/script/tsim/query/session.sim index 158448d765..b6eb4ed3aa 100644 --- a/tests/script/tsim/query/session.sim +++ b/tests/script/tsim/query/session.sim @@ -35,8 +35,8 @@ sql INSERT INTO dev_001 VALUES('2020-05-13 13:00:00.001', 12) sql INSERT INTO dev_001 VALUES('2020-05-14 13:00:00.001', 13) sql INSERT INTO dev_001 VALUES('2020-05-15 14:00:00.000', 14) sql INSERT INTO dev_001 VALUES('2020-05-20 10:00:00.000', 15) -sql INSERT INTO dev_001 VALUES('2020-05-27 10:00:00.001', 16) - +sql INSERT INTO dev_001 VALUES('2020-05-27 10:00:00.001', 16) + sql INSERT INTO dev_002 VALUES('2020-05-13 10:00:00.000', 1) sql INSERT INTO dev_002 VALUES('2020-05-13 10:00:00.005', 2) sql INSERT INTO dev_002 VALUES('2020-05-13 10:00:00.009', 3) @@ -46,7 +46,7 @@ sql INSERT INTO dev_002 VALUES('2020-05-13 10:00:00.036', 6) sql INSERT INTO dev_002 VALUES('2020-05-13 10:00:00.51', 7) # vnode does not return the precision of the table -print ====> create database d1 precision 'us' +print ====> create database d1 precision 'us' sql create database d1 precision 'us' sql use d1 sql create table dev_001 (ts timestamp ,i timestamp ,j int) @@ -54,7 +54,7 @@ sql insert into dev_001 values(1623046993681000,now,1)(1623046993681001,now+1s,2 sql create table secondts(ts timestamp,t2 timestamp,i int) sql insert into secondts values(1623046993681000,now,1)(1623046993681001,now+1s,2)(1623046993681002,now+2s,3)(1623046993681004,now+5s,4) -$loop_test = 0 +$loop_test = 0 loop_test_pos: sql use $dbNamme @@ -299,7 +299,7 @@ if $loop_test == 0 then print =============== stop and restart taosd system sh/exec.sh -n dnode1 -s stop -x SIGINT system sh/exec.sh -n dnode1 -s start - + $loop_cnt = 0 check_dnode_ready_0: $loop_cnt = $loop_cnt + 1 @@ -317,7 +317,7 @@ if $loop_test == 0 then goto check_dnode_ready_0 endi - $loop_test = 1 + $loop_test = 1 goto loop_test_pos endi diff --git a/tests/script/tsim/query/stddev.sim b/tests/script/tsim/query/stddev.sim index d61c7273e1..b45c7d80a3 100644 --- a/tests/script/tsim/query/stddev.sim +++ b/tests/script/tsim/query/stddev.sim @@ -409,4 +409,4 @@ if $rows != 2 then return -1 endi -system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/query/time_process.sim b/tests/script/tsim/query/time_process.sim index b3c0e9561f..83a6445846 100644 --- a/tests/script/tsim/query/time_process.sim +++ b/tests/script/tsim/query/time_process.sim @@ -111,4 +111,4 @@ if $rows != 2 then return -1 endi -system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/query/udf.sim b/tests/script/tsim/query/udf.sim index 7cc1403bcb..7f8b1044ef 100644 --- a/tests/script/tsim/query/udf.sim +++ b/tests/script/tsim/query/udf.sim @@ -9,7 +9,7 @@ system sh/cfg.sh -n dnode1 -c udf -v 1 system sh/exec.sh -n dnode1 -s start sql connect -print ======== step1 udf +print ======== step1 udf system sh/compile_udf.sh sql create database udf vgroups 3; sql use udf; diff --git a/tests/script/tsim/user/privilege_sysinfo.sim b/tests/script/tsim/user/privilege_sysinfo.sim index 25c1a84db6..e8348d92d4 100644 --- a/tests/script/tsim/user/privilege_sysinfo.sim +++ b/tests/script/tsim/user/privilege_sysinfo.sim @@ -8,7 +8,20 @@ sql create user sysinfo0 pass 'taosdata' sql create user sysinfo1 pass 'taosdata' sql alter user sysinfo0 sysinfo 0 sql alter user sysinfo1 sysinfo 1 + sql create database db +sql use db +sql create table db.stb (ts timestamp, i int) tags (t int) +sql create table db.ctb using db.stb tags (1) +sql create table db.ntb (ts timestamp, i int) +sql insert into db.ctb values (now, 1); +sql insert into db.ntb values (now, 1); +sql select * from db.stb +sql select * from db.ctb +sql select * from db.ntb + +sql create database d2 +sql GRANT all ON d2.* to sysinfo0; print user sysinfo0 login sql close @@ -17,11 +30,31 @@ sql connect sysinfo0 print =============== check oper sql_error create user u1 pass 'u1' sql_error drop user sysinfo1 -sql_error alter user sysinfo1 pass '1' sql_error alter user sysinfo0 pass '1' +sql_error alter user sysinfo0 enable 0 +sql_error alter user sysinfo0 enable 1 +sql_error alter user sysinfo1 pass '1' +sql_error alter user sysinfo1 enable 1 +sql_error alter user sysinfo1 enable 1 +sql_error GRANT read ON db.* to sysinfo0; +sql_error GRANT read ON *.* to sysinfo0; +sql_error REVOKE read ON db.* from sysinfo0; +sql_error REVOKE read ON *.* from sysinfo0; +sql_error GRANT write ON db.* to sysinfo0; +sql_error GRANT write ON *.* to sysinfo0; +sql_error REVOKE write ON db.* from sysinfo0; +sql_error REVOKE write ON *.* from sysinfo0; +sql_error REVOKE write ON *.* from sysinfo0; sql_error create dnode $hostname port 7200 sql_error drop dnode 1 +sql_error alter dnode 1 'debugFlag 135' +sql_error alter dnode 1 'dDebugFlag 131' +sql_error alter dnode 1 'resetlog' +sql_error alter dnode 1 'monitor' '1' +sql_error alter dnode 1 'monitor' '0' +sql_error alter dnode 1 'monitor 1' +sql_error alter dnode 1 'monitor 0' sql_error create qnode on dnode 1 sql_error drop qnode on dnode 1 @@ -44,20 +77,106 @@ sql_error create database d1 sql_error drop database db sql_error use db sql_error alter database db replica 1; +sql_error alter database db keep 21 sql_error show db.vgroups -sql select * from information_schema.ins_stables where db_name = 'db' -sql select * from information_schema.ins_tables where db_name = 'db' + +sql_error create table db.stb1 (ts timestamp, i int) tags (t int) +sql_error create table db.ctb1 using db.stb1 tags (1) +sql_error create table db.ntb1 (ts timestamp, i int) +sql_error insert into db.ctb values (now, 1); +sql_error insert into db.ntb values (now, 1); +sql_error select * from db.stb +sql_error select * from db.ctb +sql_error select * from db.ntb + +sql use d2 +sql create table d2.stb2 (ts timestamp, i int) tags (t int) +sql create table d2.ctb2 using d2.stb2 tags (1) +sql create table d2.ntb2 (ts timestamp, i int) +sql insert into d2.ctb2 values (now, 1); +sql insert into d2.ntb2 values (now, 1); +sql select * from d2.stb2 +sql select * from d2.ctb2 +sql select * from d2.ntb2 print =============== check show -sql select * from information_schema.ins_users +sql_error show users sql_error show cluster -sql select * from information_schema.ins_dnodes -sql select * from information_schema.ins_mnodes +sql_error select * from information_schema.ins_dnodes +sql_error select * from information_schema.ins_mnodes sql_error show snodes -sql select * from information_schema.ins_qnodes +sql_error select * from information_schema.ins_qnodes +sql_error show dnodes +sql_error show snodes +sql_error show qnodes +sql_error show mnodes sql_error show bnodes +sql_error show db.vgroups +sql_error show db.stables +sql_error show db.tables +sql_error show indexes from stb from db +sql show databases +sql_error show d2.vgroups +sql show d2.stables +sql show d2.tables +sql show indexes from stb2 from d2 +#sql_error show create database db +sql_error show create table db.stb; +sql_error show create table db.ctb; +sql_error show create table db.ntb; +sql show streams +sql show consumers +sql show topics +sql show subscriptions +sql show functions sql_error show grants +sql show queries +sql show connections +sql show apps +sql show transactions +sql_error show create database d2 +sql show create table d2.stb2; +sql show create table d2.ctb2; +sql show create table d2.ntb2; +sql_error show variables; +sql show local variables; sql_error show dnode 1 variables; -sql show variables; +sql_error show variables; -system sh/exec.sh -n dnode1 -s stop -x SIGINT + +print =============== check information_schema +sql show databases +if $rows != 3 then + return -1 +endi + +sql use information_schema; +sql_error select * from information_schema.ins_dnodes +sql_error select * from information_schema.ins_mnodes +sql_error select * from information_schema.ins_modules +sql_error select * from information_schema.ins_qnodes +sql_error select * from information_schema.ins_cluster +sql select * from information_schema.ins_databases +sql select * from information_schema.ins_functions +sql select * from information_schema.ins_indexes +sql select * from information_schema.ins_stables +sql select * from information_schema.ins_tables +sql select * from information_schema.ins_tags +sql select * from information_schema.ins_users +sql_error select * from information_schema.ins_grants +sql_error select * from information_schema.ins_vgroups +sql_error select * from information_schema.ins_configs +sql_error select * from information_schema.ins_dnode_variables + +print =============== check performance_schema +sql use performance_schema; +sql select * from performance_schema.perf_connections +sql select * from performance_schema.perf_queries +sql select * from performance_schema.perf_topics +sql select * from performance_schema.perf_consumers +sql select * from performance_schema.perf_subscriptions +sql select * from performance_schema.perf_trans +sql select * from performance_schema.perf_streams +sql select * from performance_schema.perf_apps + +#system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/system-test/2-query/interp.py b/tests/system-test/2-query/interp.py index 934ba9e161..5550519e05 100644 --- a/tests/system-test/2-query/interp.py +++ b/tests/system-test/2-query/interp.py @@ -551,7 +551,57 @@ class TDTestCase: tdSql.checkData(0, 0, 15) tdSql.checkData(1, 0, 15) - tdLog.printNoPrefix("==========step9:test error cases") + tdLog.printNoPrefix("==========step9:test multi-interp cases") + tdSql.query(f"select interp(c0),interp(c1),interp(c2),interp(c3) from {dbname}.{tbname} range('2020-02-09 00:00:05', '2020-02-13 00:00:05') every(1d) fill(null)") + tdSql.checkRows(5) + tdSql.checkCols(4) + + for i in range (tdSql.queryCols): + tdSql.checkData(0, i, None) + tdSql.checkData(1, i, None) + tdSql.checkData(2, i, 15) + tdSql.checkData(3, i, None) + tdSql.checkData(4, i, None) + + tdSql.query(f"select interp(c0),interp(c1),interp(c2),interp(c3) from {dbname}.{tbname} range('2020-02-09 00:00:05', '2020-02-13 00:00:05') every(1d) fill(value, 1)") + tdSql.checkRows(5) + tdSql.checkCols(4) + + for i in range (tdSql.queryCols): + tdSql.checkData(0, i, 1) + tdSql.checkData(1, i, 1) + tdSql.checkData(2, i, 15) + tdSql.checkData(3, i, 1) + tdSql.checkData(4, i, 1) + + tdSql.query(f"select interp(c0),interp(c1),interp(c2),interp(c3) from {dbname}.{tbname} range('2020-02-09 00:00:05', '2020-02-13 00:00:05') every(1d) fill(prev)") + tdSql.checkRows(5) + tdSql.checkCols(4) + + for i in range (tdSql.queryCols): + tdSql.checkData(0, i, 5) + tdSql.checkData(1, i, 5) + tdSql.checkData(2, i, 15) + tdSql.checkData(3, i, 15) + tdSql.checkData(4, i, 15) + + tdSql.query(f"select interp(c0),interp(c1),interp(c2),interp(c3) from {dbname}.{tbname} range('2020-02-09 00:00:05', '2020-02-13 00:00:05') every(1d) fill(next)") + tdSql.checkRows(3) + tdSql.checkCols(4) + + for i in range (tdSql.queryCols): + tdSql.checkData(0, i, 15) + tdSql.checkData(1, i, 15) + tdSql.checkData(2, i, 15) + + tdSql.query(f"select interp(c0),interp(c1),interp(c2),interp(c3) from {dbname}.{tbname} range('2020-02-09 00:00:05', '2020-02-13 00:00:05') every(1d) fill(linear)") + tdSql.checkRows(1) + tdSql.checkCols(4) + + for i in range (tdSql.queryCols): + tdSql.checkData(0, i, 15) + + tdLog.printNoPrefix("==========step10:test error cases") tdSql.error(f"select interp(c0) from {dbname}.{tbname}") tdSql.error(f"select interp(c0) from {dbname}.{tbname} range('2020-02-10 00:00:05', '2020-02-15 00:00:05')") diff --git a/tests/system-test/2-query/sml.py b/tests/system-test/2-query/sml.py index 6cfb9a1dad..5d74c568ce 100644 --- a/tests/system-test/2-query/sml.py +++ b/tests/system-test/2-query/sml.py @@ -85,6 +85,9 @@ class TDTestCase: tdSql.query("select * from macylr") tdSql.checkRows(2) + + tdSql.query("desc macylr") + tdSql.checkRows(25) return def run(self): diff --git a/tests/system-test/7-tmq/stbTagFilter-1ctb.py b/tests/system-test/7-tmq/stbTagFilter-1ctb.py index 6a26d2ce1f..6cb152342b 100644 --- a/tests/system-test/7-tmq/stbTagFilter-1ctb.py +++ b/tests/system-test/7-tmq/stbTagFilter-1ctb.py @@ -259,7 +259,6 @@ class TDTestCase: self.tmqCase1() self.tmqCase2() - def stop(self): tdSql.close() tdLog.success(f"{__file__} successfully executed") diff --git a/tests/system-test/7-tmq/tmqDropNtb-snapshot1.py b/tests/system-test/7-tmq/tmqDropNtb-snapshot1.py index 20e363341f..4cb208b616 100644 --- a/tests/system-test/7-tmq/tmqDropNtb-snapshot1.py +++ b/tests/system-test/7-tmq/tmqDropNtb-snapshot1.py @@ -99,8 +99,8 @@ class TDTestCase: tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) - if not ((totalConsumeRows >= expectrowcnt * 3/4) and (totalConsumeRows < expectrowcnt)): - tdLog.exit("tmq consume rows error with snapshot = 0!") + # if not ((totalConsumeRows >= expectrowcnt * 3/4) and (totalConsumeRows < expectrowcnt)): + # tdLog.exit("tmq consume rows error with snapshot = 0!") tdLog.info("wait subscriptions exit ....") tmqCom.waitSubscriptionExit(tdSql, topicFromDb) @@ -192,8 +192,8 @@ class TDTestCase: tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) - if not ((totalConsumeRows >= expectrowcnt / 2 * (1 + 3/4)) and (totalConsumeRows < expectrowcnt)): - tdLog.exit("tmq consume rows error with snapshot = 0!") + # if not ((totalConsumeRows >= expectrowcnt / 2 * (1 + 3/4)) and (totalConsumeRows < expectrowcnt)): + # tdLog.exit("tmq consume rows error with snapshot = 0!") tdLog.info("wait subscriptions exit ....") tmqCom.waitSubscriptionExit(tdSql, topicFromDb) diff --git a/tests/system-test/7-tmq/tmqDropStbCtb.py b/tests/system-test/7-tmq/tmqDropStbCtb.py index 992a128ac0..704811d083 100644 --- a/tests/system-test/7-tmq/tmqDropStbCtb.py +++ b/tests/system-test/7-tmq/tmqDropStbCtb.py @@ -155,8 +155,9 @@ class TDTestCase: tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) - if not ((totalConsumeRows > expectrowcnt / 2) and (totalConsumeRows < expectrowcnt)): - tdLog.exit("tmq consume rows error with snapshot = 0!") + if self.snapshot == 0: + if not ((totalConsumeRows > expectrowcnt / 2) and (totalConsumeRows < expectrowcnt)): + tdLog.exit("tmq consume rows error with snapshot = 0!") tdLog.info("wait subscriptions exit ....") tmqCom.waitSubscriptionExit(tdSql, topicFromDb) @@ -246,8 +247,9 @@ class TDTestCase: tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) - if not ((totalConsumeRows > expectrowcnt / 2) and (totalConsumeRows < expectrowcnt)): - tdLog.exit("tmq consume rows error with snapshot = 0!") + if self.snapshot == 0: + if not ((totalConsumeRows > expectrowcnt / 2) and (totalConsumeRows < expectrowcnt)): + tdLog.exit("tmq consume rows error with snapshot = 0!") tdLog.info("wait subscriptions exit ....") tmqCom.waitSubscriptionExit(tdSql, topicFromDb) diff --git a/tests/system-test/7-tmq/tmq_taosx.py b/tests/system-test/7-tmq/tmq_taosx.py index cd13535684..d38e509d26 100644 --- a/tests/system-test/7-tmq/tmq_taosx.py +++ b/tests/system-test/7-tmq/tmq_taosx.py @@ -52,6 +52,148 @@ class TDTestCase: tdSql.checkData(1, 1, 23) tdSql.checkData(1, 4, None) + tdSql.query("select * from st1 order by ts") + tdSql.checkRows(8) + tdSql.checkData(0, 1, 1) + tdSql.checkData(1, 1, 3) + tdSql.checkData(4, 1, 4) + tdSql.checkData(6, 1, 23) + + tdSql.checkData(0, 2, 2) + tdSql.checkData(1, 2, 4) + tdSql.checkData(4, 2, 3) + tdSql.checkData(6, 2, 32) + + tdSql.checkData(0, 3, 'a') + tdSql.checkData(1, 3, 'b') + tdSql.checkData(4, 3, 'hwj') + tdSql.checkData(6, 3, 's21ds') + + tdSql.checkData(0, 4, None) + tdSql.checkData(1, 4, None) + tdSql.checkData(5, 4, 940) + tdSql.checkData(6, 4, None) + + tdSql.checkData(0, 5, 1000) + tdSql.checkData(1, 5, 2000) + tdSql.checkData(4, 5, 1000) + tdSql.checkData(6, 5, 5000) + + tdSql.checkData(0, 6, 'ttt') + tdSql.checkData(1, 6, None) + tdSql.checkData(4, 6, 'ttt') + tdSql.checkData(6, 6, None) + + tdSql.checkData(0, 7, True) + tdSql.checkData(1, 7, None) + tdSql.checkData(4, 7, True) + tdSql.checkData(6, 7, None) + + tdSql.checkData(0, 8, None) + tdSql.checkData(1, 8, None) + tdSql.checkData(4, 8, None) + tdSql.checkData(6, 8, None) + + tdSql.query("select * from ct1") + tdSql.checkRows(4) + + tdSql.query("select * from ct2") + tdSql.checkRows(0) + + tdSql.query("select * from ct0 order by c1") + tdSql.checkRows(2) + tdSql.checkData(0, 3, "a") + tdSql.checkData(1, 4, None) + + tdSql.query("select * from n1 order by cc3 desc") + tdSql.checkRows(2) + tdSql.checkData(0, 1, "eeee") + tdSql.checkData(1, 2, 940) + + tdSql.query("select * from jt order by i desc") + tdSql.checkRows(2) + tdSql.checkData(0, 1, 11) + tdSql.checkData(0, 2, None) + tdSql.checkData(1, 1, 1) + tdSql.checkData(1, 2, '{"k1":1,"k2":"hello"}') + + tdSql.execute('drop topic if exists topic_ctb_column') + return + + def checkFileContentSnapshot(self): + buildPath = tdCom.getBuildPath() + cfgPath = tdCom.getClientCfgPath() + cmdStr = '%s/build/bin/tmq_taosx_snapshot_ci -c %s'%(buildPath, cfgPath) + tdLog.info(cmdStr) + os.system(cmdStr) + + srcFile = '%s/../log/tmq_taosx_tmp_snapshot.source'%(cfgPath) + dstFile = '%s/../log/tmq_taosx_tmp_snapshot.result'%(cfgPath) + tdLog.info("compare file: %s, %s"%(srcFile, dstFile)) + + consumeFile = open(srcFile, mode='r') + queryFile = open(dstFile, mode='r') + + while True: + dst = queryFile.readline() + src = consumeFile.readline() + + if dst: + if dst != src: + tdLog.exit("compare error: %s != %s"%src, dst) + else: + break + + tdSql.execute('use db_taosx') + tdSql.query("select * from ct3 order by c1 desc") + tdSql.checkRows(2) + tdSql.checkData(0, 1, 51) + tdSql.checkData(0, 4, 940) + tdSql.checkData(1, 1, 23) + tdSql.checkData(1, 4, None) + + tdSql.query("select * from st1 order by ts") + tdSql.checkRows(8) + tdSql.checkData(0, 1, 1) + tdSql.checkData(1, 1, 3) + tdSql.checkData(4, 1, 4) + tdSql.checkData(6, 1, 23) + + tdSql.checkData(0, 2, 2) + tdSql.checkData(1, 2, 4) + tdSql.checkData(4, 2, 3) + tdSql.checkData(6, 2, 32) + + tdSql.checkData(0, 3, 'a') + tdSql.checkData(1, 3, 'b') + tdSql.checkData(4, 3, 'hwj') + tdSql.checkData(6, 3, 's21ds') + + tdSql.checkData(0, 4, None) + tdSql.checkData(1, 4, None) + tdSql.checkData(5, 4, 940) + tdSql.checkData(6, 4, None) + + tdSql.checkData(0, 5, 1000) + tdSql.checkData(1, 5, 2000) + tdSql.checkData(4, 5, 1000) + tdSql.checkData(6, 5, 5000) + + tdSql.checkData(0, 6, 'ttt') + tdSql.checkData(1, 6, None) + tdSql.checkData(4, 6, 'ttt') + tdSql.checkData(6, 6, None) + + tdSql.checkData(0, 7, True) + tdSql.checkData(1, 7, None) + tdSql.checkData(4, 7, True) + tdSql.checkData(6, 7, None) + + tdSql.checkData(0, 8, None) + tdSql.checkData(1, 8, None) + tdSql.checkData(4, 8, None) + tdSql.checkData(6, 8, None) + tdSql.query("select * from ct1") tdSql.checkRows(4) @@ -80,6 +222,7 @@ class TDTestCase: def run(self): tdSql.prepare() self.checkFileContent() + self.checkFileContentSnapshot() def stop(self): tdSql.close() diff --git a/tests/test/c/CMakeLists.txt b/tests/test/c/CMakeLists.txt index 31331b5265..0fb80e69c2 100644 --- a/tests/test/c/CMakeLists.txt +++ b/tests/test/c/CMakeLists.txt @@ -2,6 +2,7 @@ add_executable(tmq_demo tmqDemo.c) add_executable(tmq_sim tmqSim.c) add_executable(create_table createTable.c) add_executable(tmq_taosx_ci tmq_taosx_ci.c) +add_executable(tmq_taosx_snapshot_ci tmq_taosx_snapshot_ci.c) add_executable(sml_test sml_test.c) target_link_libraries( create_table @@ -31,6 +32,13 @@ target_link_libraries( PUBLIC common PUBLIC os ) +target_link_libraries( + tmq_taosx_snapshot_ci + PUBLIC taos_static + PUBLIC util + PUBLIC common + PUBLIC os +) target_link_libraries( sml_test diff --git a/tests/test/c/sml_test.c b/tests/test/c/sml_test.c index 50249a5c56..18181d2073 100644 --- a/tests/test/c/sml_test.c +++ b/tests/test/c/sml_test.c @@ -1089,7 +1089,7 @@ int sml_add_tag_col_Test() { if (code) return code; const char *sql1[] = { - "macylr,id=macylr_17875_1804,t0=f,t1=127i8,t2=32767i16,t3=2147483647i32,t4=9223372036854775807i64,t5=11.12345f32,t6=22.123456789f64,t7=\"binaryTagValue\",t8=L\"ncharTagValue\",t11=127i8,t10=L\"ncharTagValue\" c0=f,c1=127i8,c2=32767i16,c3=2147483647i32,c4=9223372036854775807i64,c5=11.12345f32,c6=22.123456789f64,c7=\"binaryColValue\",c8=L\"ncharColValue\",c9=7u64,c11=L\"ncharColValue\",c10=f 1626006833639000000" + "macylr,id=macylr_17875_1804,t1=127i8,t2=32767i16,t3=2147483647i32,t4=9223372036854775807i64,t5=11.12345f32,t6=22.123456789f64,t7=\"binaryTagValue\",t8=L\"ncharTagValue\",t11=127i8,t10=L\"ncharTagValue\" c0=f,c1=127i8,c2=32767i16,c3=2147483647i32,c4=9223372036854775807i64,c5=11.12345f32,c6=22.123456789f64,c8=L\"ncharColValue\",c9=7u64,c11=L\"ncharColValue\",c10=f 1626006833639000000" }; pRes = taos_schemaless_insert(taos, (char **)sql1, sizeof(sql1) / sizeof(sql1[0]), TSDB_SML_LINE_PROTOCOL, 0); diff --git a/tests/test/c/tmq_taosx_ci.c b/tests/test/c/tmq_taosx_ci.c index ece7ad4819..ee5af03f05 100644 --- a/tests/test/c/tmq_taosx_ci.c +++ b/tests/test/c/tmq_taosx_ci.c @@ -501,7 +501,8 @@ int main(int argc, char* argv[]) { if(argc == 3 && strcmp(argv[1], "-c") == 0) { strcpy(dir, argv[2]); }else{ - strcpy(dir, "../../../sim/psim/cfg"); +// strcpy(dir, "../../../sim/psim/cfg"); + strcpy(dir, "/var/log"); } printf("env init\n"); diff --git a/tests/test/c/tmq_taosx_snapshot_ci.c b/tests/test/c/tmq_taosx_snapshot_ci.c new file mode 100644 index 0000000000..e3a52f7cad --- /dev/null +++ b/tests/test/c/tmq_taosx_snapshot_ci.c @@ -0,0 +1,512 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#include +#include +#include +#include +#include +#include "taos.h" +#include "types.h" + +static int running = 1; +TdFilePtr g_fp = NULL; +char dir[64]={0}; + +static TAOS* use_db(){ + TAOS* pConn = taos_connect("localhost", "root", "taosdata", NULL, 0); + if (pConn == NULL) { + return NULL; + } + + TAOS_RES* pRes = taos_query(pConn, "use db_taosx"); + if (taos_errno(pRes) != 0) { + printf("error in use db_taosx, reason:%s\n", taos_errstr(pRes)); + return NULL; + } + taos_free_result(pRes); + return pConn; +} + +static void msg_process(TAOS_RES* msg) { + /*memset(buf, 0, 1024);*/ + printf("-----------topic-------------: %s\n", tmq_get_topic_name(msg)); + printf("db: %s\n", tmq_get_db_name(msg)); + printf("vg: %d\n", tmq_get_vgroup_id(msg)); + TAOS *pConn = use_db(); + if (tmq_get_res_type(msg) == TMQ_RES_TABLE_META) { + char* result = tmq_get_json_meta(msg); + if (result) { + printf("meta result: %s\n", result); + } + taosFprintfFile(g_fp, result); + taosFprintfFile(g_fp, "\n"); + tmq_free_json_meta(result); + } + + tmq_raw_data raw = {0}; + tmq_get_raw(msg, &raw); + int32_t ret = tmq_write_raw(pConn, raw); + printf("write raw data: %s\n", tmq_err2str(ret)); + +// else{ +// while(1){ +// int numOfRows = 0; +// void *pData = NULL; +// taos_fetch_raw_block(msg, &numOfRows, &pData); +// if(numOfRows == 0) break; +// printf("write data: tbname:%s, numOfRows:%d\n", tmq_get_table_name(msg), numOfRows); +// int ret = taos_write_raw_block(pConn, numOfRows, pData, tmq_get_table_name(msg)); +// printf("write raw data: %s\n", tmq_err2str(ret)); +// } +// } + + taos_close(pConn); +} + +int32_t init_env() { + TAOS* pConn = taos_connect("localhost", "root", "taosdata", NULL, 0); + if (pConn == NULL) { + return -1; + } + + TAOS_RES* pRes = taos_query(pConn, "drop database if exists db_taosx"); + if (taos_errno(pRes) != 0) { + printf("error in drop db_taosx, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "create database if not exists db_taosx vgroups 1"); + if (taos_errno(pRes) != 0) { + printf("error in create db_taosx, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "drop database if exists abc1"); + if (taos_errno(pRes) != 0) { + printf("error in drop db, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "create database if not exists abc1 vgroups 1"); + if (taos_errno(pRes) != 0) { + printf("error in create db, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "use abc1"); + if (taos_errno(pRes) != 0) { + printf("error in use db, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, + "create stable if not exists st1 (ts timestamp, c1 int, c2 float, c3 binary(16)) tags(t1 int, t3 " + "nchar(8), t4 bool)"); + if (taos_errno(pRes) != 0) { + printf("failed to create super table st1, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "create table if not exists ct0 using st1 tags(1000, \"ttt\", true)"); + if (taos_errno(pRes) != 0) { + printf("failed to create child table tu1, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "insert into ct0 values(1626006833600, 1, 2, 'a')"); + if (taos_errno(pRes) != 0) { + printf("failed to insert into ct0, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "create table if not exists ct1 using st1(t1) tags(2000)"); + if (taos_errno(pRes) != 0) { + printf("failed to create child table ct1, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "create table if not exists ct2 using st1(t1) tags(NULL)"); + if (taos_errno(pRes) != 0) { + printf("failed to create child table ct2, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "insert into ct1 values(1626006833600, 3, 4, 'b')"); + if (taos_errno(pRes) != 0) { + printf("failed to insert into ct1, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "create table if not exists ct3 using st1(t1) tags(3000)"); + if (taos_errno(pRes) != 0) { + printf("failed to create child table ct3, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "insert into ct3 values(1626006833600, 5, 6, 'c') ct1 values(1626006833601, 2, 3, 'sds') (1626006833602, 4, 5, 'ddd') ct0 values(1626006833602, 4, 3, 'hwj') ct1 values(now+5s, 23, 32, 's21ds')"); + if (taos_errno(pRes) != 0) { + printf("failed to insert into ct3, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "alter table st1 add column c4 bigint"); + if (taos_errno(pRes) != 0) { + printf("failed to alter super table st1, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "alter table st1 modify column c3 binary(64)"); + if (taos_errno(pRes) != 0) { + printf("failed to alter super table st1, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "insert into ct3 values(1626006833605, 53, 63, 'cffffffffffffffffffffffffffff', 8989898899999) (1626006833609, 51, 62, 'c333', 940)"); + if (taos_errno(pRes) != 0) { + printf("failed to insert into ct3, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "insert into ct3 select * from ct1"); + if (taos_errno(pRes) != 0) { + printf("failed to insert into ct3, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "alter table st1 add tag t2 binary(64)"); + if (taos_errno(pRes) != 0) { + printf("failed to alter super table st1, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "alter table ct3 set tag t1=5000"); + if (taos_errno(pRes) != 0) { + printf("failed to slter child table ct3, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "delete from abc1 .ct3 where ts < 1626006833606"); + if (taos_errno(pRes) != 0) { + printf("failed to insert into ct3, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "create table if not exists n1(ts timestamp, c1 int, c2 nchar(4))"); + if (taos_errno(pRes) != 0) { + printf("failed to create normal table n1, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "alter table n1 add column c3 bigint"); + if (taos_errno(pRes) != 0) { + printf("failed to alter normal table n1, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "alter table n1 modify column c2 nchar(8)"); + if (taos_errno(pRes) != 0) { + printf("failed to alter normal table n1, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "alter table n1 rename column c3 cc3"); + if (taos_errno(pRes) != 0) { + printf("failed to alter normal table n1, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "alter table n1 comment 'hello'"); + if (taos_errno(pRes) != 0) { + printf("failed to alter normal table n1, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "alter table n1 drop column c1"); + if (taos_errno(pRes) != 0) { + printf("failed to alter normal table n1, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "insert into n1 values(now, 'eeee', 8989898899999) (now+9s, 'c333', 940)"); + if (taos_errno(pRes) != 0) { + printf("failed to insert into n1, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "create table jt(ts timestamp, i int) tags(t json)"); + if (taos_errno(pRes) != 0) { + printf("failed to create super table jt, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "create table jt1 using jt tags('{\"k1\":1, \"k2\":\"hello\"}')"); + if (taos_errno(pRes) != 0) { + printf("failed to create super table jt, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "create table jt2 using jt tags('')"); + if (taos_errno(pRes) != 0) { + printf("failed to create super table jt2, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "insert into jt1 values(now, 1)"); + if (taos_errno(pRes) != 0) { + printf("failed to create super table jt1, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "insert into jt2 values(now, 11)"); + if (taos_errno(pRes) != 0) { + printf("failed to create super table jt2, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + taos_close(pConn); + return 0; +} + +int32_t create_topic() { + printf("create topic\n"); + TAOS_RES* pRes; + TAOS* pConn = taos_connect("localhost", "root", "taosdata", NULL, 0); + if (pConn == NULL) { + return -1; + } + + pRes = taos_query(pConn, "use abc1"); + if (taos_errno(pRes) != 0) { + printf("error in use db, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + pRes = taos_query(pConn, "create topic topic_ctb_column with meta as database abc1"); + if (taos_errno(pRes) != 0) { + printf("failed to create topic topic_ctb_column, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + + taos_close(pConn); + return 0; +} + +void tmq_commit_cb_print(tmq_t* tmq, int32_t code, void* param) { + printf("commit %d tmq %p param %p\n", code, tmq, param); +} + +tmq_t* build_consumer() { +#if 0 + TAOS* pConn = taos_connect("localhost", "root", "taosdata", NULL, 0); + assert(pConn != NULL); + + TAOS_RES* pRes = taos_query(pConn, "use abc1"); + if (taos_errno(pRes) != 0) { + printf("error in use db, reason:%s\n", taos_errstr(pRes)); + } + taos_free_result(pRes); +#endif + + tmq_conf_t* conf = tmq_conf_new(); + tmq_conf_set(conf, "group.id", "tg2"); + tmq_conf_set(conf, "client.id", "my app 1"); + tmq_conf_set(conf, "td.connect.user", "root"); + tmq_conf_set(conf, "td.connect.pass", "taosdata"); + tmq_conf_set(conf, "msg.with.table.name", "true"); + tmq_conf_set(conf, "enable.auto.commit", "true"); + tmq_conf_set(conf, "enable.heartbeat.background", "true"); + tmq_conf_set(conf, "experimental.snapshot.enable", "true"); + /*tmq_conf_set(conf, "experimental.snapshot.enable", "true");*/ + + tmq_conf_set_auto_commit_cb(conf, tmq_commit_cb_print, NULL); + tmq_t* tmq = tmq_consumer_new(conf, NULL, 0); + assert(tmq); + tmq_conf_destroy(conf); + return tmq; +} + +tmq_list_t* build_topic_list() { + tmq_list_t* topic_list = tmq_list_new(); + tmq_list_append(topic_list, "topic_ctb_column"); + /*tmq_list_append(topic_list, "tmq_test_db_multi_insert_topic");*/ + return topic_list; +} + +void basic_consume_loop(tmq_t* tmq, tmq_list_t* topics) { + int32_t code; + + if ((code = tmq_subscribe(tmq, topics))) { + fprintf(stderr, "%% Failed to start consuming topics: %s\n", tmq_err2str(code)); + printf("subscribe err\n"); + return; + } + int32_t cnt = 0; + while (running) { + TAOS_RES* tmqmessage = tmq_consumer_poll(tmq, 1000); + if (tmqmessage) { + cnt++; + msg_process(tmqmessage); + /*if (cnt >= 2) break;*/ + /*printf("get data\n");*/ + taos_free_result(tmqmessage); + /*} else {*/ + /*break;*/ + /*tmq_commit_sync(tmq, NULL);*/ + }else{ + break; + } + } + + code = tmq_consumer_close(tmq); + if (code) + fprintf(stderr, "%% Failed to close consumer: %s\n", tmq_err2str(code)); + else + fprintf(stderr, "%% Consumer closed\n"); +} + +void sync_consume_loop(tmq_t* tmq, tmq_list_t* topics) { + static const int MIN_COMMIT_COUNT = 1; + + int msg_count = 0; + int32_t code; + + if ((code = tmq_subscribe(tmq, topics))) { + fprintf(stderr, "%% Failed to start consuming topics: %s\n", tmq_err2str(code)); + return; + } + + tmq_list_t* subList = NULL; + tmq_subscription(tmq, &subList); + char** subTopics = tmq_list_to_c_array(subList); + int32_t sz = tmq_list_get_size(subList); + printf("subscribed topics: "); + for (int32_t i = 0; i < sz; i++) { + printf("%s, ", subTopics[i]); + } + printf("\n"); + tmq_list_destroy(subList); + + while (running) { + TAOS_RES* tmqmessage = tmq_consumer_poll(tmq, 1000); + if (tmqmessage) { + msg_process(tmqmessage); + taos_free_result(tmqmessage); + + /*tmq_commit_sync(tmq, NULL);*/ + /*if ((++msg_count % MIN_COMMIT_COUNT) == 0) tmq_commit(tmq, NULL, 0);*/ + } + } + + code = tmq_consumer_close(tmq); + if (code) + fprintf(stderr, "%% Failed to close consumer: %s\n", tmq_err2str(code)); + else + fprintf(stderr, "%% Consumer closed\n"); +} + +void initLogFile() { + char f1[256] = {0}; + char f2[256] = {0}; + + sprintf(f1, "%s/../log/tmq_taosx_tmp_snapshot.source", dir); + sprintf(f2, "%s/../log/tmq_taosx_tmp_snapshot.result", dir); + TdFilePtr pFile = taosOpenFile(f1, TD_FILE_TEXT | TD_FILE_TRUNC | TD_FILE_STREAM); + if (NULL == pFile) { + fprintf(stderr, "Failed to open %s for save result\n", f1); + exit(-1); + } + g_fp = pFile; + + TdFilePtr pFile2 = taosOpenFile(f2, TD_FILE_TEXT | TD_FILE_TRUNC | TD_FILE_STREAM); + if (NULL == pFile2) { + fprintf(stderr, "Failed to open %s for save result\n", f2); + exit(-1); + } + char *result[] = { + "{\"type\":\"create\",\"tableName\":\"st1\",\"tableType\":\"super\",\"columns\":[{\"name\":\"ts\",\"type\":9},{\"name\":\"c1\",\"type\":4},{\"name\":\"c2\",\"type\":6},{\"name\":\"c3\",\"type\":8,\"length\":64},{\"name\":\"c4\",\"type\":5}],\"tags\":[{\"name\":\"t1\",\"type\":4},{\"name\":\"t3\",\"type\":10,\"length\":8},{\"name\":\"t4\",\"type\":1},{\"name\":\"t2\",\"type\":8,\"length\":64}]}", + "{\"type\":\"create\",\"tableName\":\"ct0\",\"tableType\":\"child\",\"using\":\"st1\",\"tagNum\":4,\"tags\":[{\"name\":\"t1\",\"type\":4,\"value\":1000},{\"name\":\"t3\",\"type\":10,\"value\":\"\\\"ttt\\\"\"},{\"name\":\"t4\",\"type\":1,\"value\":1}]}", + "{\"type\":\"create\",\"tableName\":\"ct1\",\"tableType\":\"child\",\"using\":\"st1\",\"tagNum\":4,\"tags\":[{\"name\":\"t1\",\"type\":4,\"value\":2000}]}", + "{\"type\":\"create\",\"tableName\":\"ct2\",\"tableType\":\"child\",\"using\":\"st1\",\"tagNum\":4,\"tags\":[]}", + "{\"type\":\"create\",\"tableName\":\"ct3\",\"tableType\":\"child\",\"using\":\"st1\",\"tagNum\":4,\"tags\":[{\"name\":\"t1\",\"type\":4,\"value\":5000}]}", + "{\"type\":\"create\",\"tableName\":\"n1\",\"tableType\":\"normal\",\"columns\":[{\"name\":\"ts\",\"type\":9},{\"name\":\"c2\",\"type\":10,\"length\":8},{\"name\":\"cc3\",\"type\":5}],\"tags\":[]}", + "{\"type\":\"create\",\"tableName\":\"jt\",\"tableType\":\"super\",\"columns\":[{\"name\":\"ts\",\"type\":9},{\"name\":\"i\",\"type\":4}],\"tags\":[{\"name\":\"t\",\"type\":15}]}", + "{\"type\":\"create\",\"tableName\":\"jt1\",\"tableType\":\"child\",\"using\":\"jt\",\"tagNum\":1,\"tags\":[{\"name\":\"t\",\"type\":15,\"value\":\"{\\\"k1\\\":1,\\\"k2\\\":\\\"hello\\\"}\"}]}", + "{\"type\":\"create\",\"tableName\":\"jt2\",\"tableType\":\"child\",\"using\":\"jt\",\"tagNum\":1,\"tags\":[]}", + }; + + for(int i = 0; i < sizeof(result)/sizeof(result[0]); i++){ + taosFprintfFile(pFile2, result[i]); + taosFprintfFile(pFile2, "\n"); + } + taosCloseFile(&pFile2); +} + +int main(int argc, char* argv[]) { + if(argc == 3 && strcmp(argv[1], "-c") == 0) { + strcpy(dir, argv[2]); + }else{ +// strcpy(dir, "../../../sim/psim/cfg"); + strcpy(dir, "/var/log"); + } + + printf("env init\n"); + initLogFile(); + + if (init_env() < 0) { + return -1; + } + create_topic(); + + tmq_t* tmq = build_consumer(); + tmq_list_t* topic_list = build_topic_list(); + basic_consume_loop(tmq, topic_list); + /*sync_consume_loop(tmq, topic_list);*/ + taosCloseFile(&g_fp); +} diff --git a/tools/CMakeLists.txt b/tools/CMakeLists.txt index d27fb99ef7..03097e31b9 100644 --- a/tools/CMakeLists.txt +++ b/tools/CMakeLists.txt @@ -82,6 +82,7 @@ ELSE () COMMAND cd ${CMAKE_CURRENT_SOURCE_DIR}/taosadapter ) EXECUTE_PROCESS( + WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/taosadapter COMMAND git rev-parse --short HEAD RESULT_VARIABLE commit_sha1 OUTPUT_VARIABLE taosadapter_commit_sha1 @@ -118,8 +119,8 @@ ELSE () PATCH_COMMAND COMMAND git clean -f -d BUILD_COMMAND - COMMAND CGO_CFLAGS=-I${CMAKE_CURRENT_SOURCE_DIR}/../include/client CGO_LDFLAGS=-L${CMAKE_BINARY_DIR}/build/lib go build -a -ldflags "-s -w -X github.com/taosdata/taosadapter/version.Version=${taos_version} -X github.com/taosdata/taosadapter/version.CommitID=${taosadapter_commit_sha1}" - COMMAND CGO_CFLAGS=-I${CMAKE_CURRENT_SOURCE_DIR}/../include/client CGO_LDFLAGS=-L${CMAKE_BINARY_DIR}/build/lib go build -a -o taosadapter-debug -ldflags "-X github.com/taosdata/taosadapter/version.Version=${taos_version} -X github.com/taosdata/taosadapter/version.CommitID=${taosadapter_commit_sha1}" + COMMAND CGO_CFLAGS=-I${CMAKE_CURRENT_SOURCE_DIR}/../include/client CGO_LDFLAGS=-L${CMAKE_BINARY_DIR}/build/lib go build -a -ldflags "-s -w -X github.com/taosdata/taosadapter/v3/version.Version=${taos_version} -X github.com/taosdata/taosadapter/v3/version.CommitID=${taosadapter_commit_sha1}" + COMMAND CGO_CFLAGS=-I${CMAKE_CURRENT_SOURCE_DIR}/../include/client CGO_LDFLAGS=-L${CMAKE_BINARY_DIR}/build/lib go build -a -o taosadapter-debug -ldflags "-X github.com/taosdata/taosadapter/v3/version.Version=${taos_version} -X github.com/taosdata/taosadapter/v3/version.CommitID=${taosadapter_commit_sha1}" INSTALL_COMMAND COMMAND ${_upx_prefix}/src/upx/upx taosadapter COMMAND cmake -E copy taosadapter ${CMAKE_BINARY_DIR}/build/bin @@ -141,8 +142,8 @@ ELSE () PATCH_COMMAND COMMAND git clean -f -d BUILD_COMMAND - COMMAND CGO_CFLAGS=-I${CMAKE_CURRENT_SOURCE_DIR}/../include/client CGO_LDFLAGS=-L${CMAKE_BINARY_DIR}/build/lib go build -a -ldflags "-s -w -X github.com/taosdata/taosadapter/version.Version=${taos_version} -X github.com/taosdata/taosadapter/version.CommitID=${taosadapter_commit_sha1}" - COMMAND CGO_CFLAGS=-I${CMAKE_CURRENT_SOURCE_DIR}/../include/client CGO_LDFLAGS=-L${CMAKE_BINARY_DIR}/build/lib go build -a -o taosadapter-debug -ldflags "-X github.com/taosdata/taosadapter/version.Version=${taos_version} -X github.com/taosdata/taosadapter/version.CommitID=${taosadapter_commit_sha1}" + COMMAND CGO_CFLAGS=-I${CMAKE_CURRENT_SOURCE_DIR}/../include/client CGO_LDFLAGS=-L${CMAKE_BINARY_DIR}/build/lib go build -a -ldflags "-s -w -X github.com/taosdata/taosadapter/v3/version.Version=${taos_version} -X github.com/taosdata/taosadapter/v3/version.CommitID=${taosadapter_commit_sha1}" + COMMAND CGO_CFLAGS=-I${CMAKE_CURRENT_SOURCE_DIR}/../include/client CGO_LDFLAGS=-L${CMAKE_BINARY_DIR}/build/lib go build -a -o taosadapter-debug -ldflags "-X github.com/taosdata/taosadapter/v3/version.Version=${taos_version} -X github.com/taosdata/taosadapter/v3/version.CommitID=${taosadapter_commit_sha1}" INSTALL_COMMAND COMMAND cmake -E copy taosadapter ${CMAKE_BINARY_DIR}/build/bin COMMAND cmake -E make_directory ${CMAKE_BINARY_DIR}/test/cfg/ @@ -174,8 +175,8 @@ ELSE () BUILD_COMMAND COMMAND set CGO_CFLAGS=-I${CMAKE_CURRENT_SOURCE_DIR}/../include/client COMMAND set CGO_LDFLAGS=-L${CMAKE_BINARY_DIR}/build/lib - COMMAND go build -a -o taosadapter.exe -ldflags "-s -w -X github.com/taosdata/taosadapter/version.Version=${taos_version} -X github.com/taosdata/taosadapter/version.CommitID=${taosadapter_commit_sha1}" - COMMAND go build -a -o taosadapter-debug.exe -ldflags "-X github.com/taosdata/taosadapter/version.Version=${taos_version} -X github.com/taosdata/taosadapter/version.CommitID=${taosadapter_commit_sha1}" + COMMAND go build -a -o taosadapter.exe -ldflags "-s -w -X github.com/taosdata/taosadapter/v3/version.Version=${taos_version} -X github.com/taosdata/taosadapter/v3/version.CommitID=${taosadapter_commit_sha1}" + COMMAND go build -a -o taosadapter-debug.exe -ldflags "-X github.com/taosdata/taosadapter/v3/version.Version=${taos_version} -X github.com/taosdata/taosadapter/v3/version.CommitID=${taosadapter_commit_sha1}" INSTALL_COMMAND COMMAND ${_upx_prefix}/src/upx/upx taosadapter.exe COMMAND cmake -E copy taosadapter.exe ${CMAKE_BINARY_DIR}/build/bin