diff --git a/README-CN.md b/README-CN.md index 37bde323c8..7df2733a2e 100644 --- a/README-CN.md +++ b/README-CN.md @@ -35,16 +35,18 @@ TDengine 是一款开源、高性能、云原生的时序数据库 (Time-Series # 文档 -关于完整的使用手册，系统架构和更多细节，请参考 [TDengine 文档](https://docs.taosdata.com) 或者 [English Documents](https://docs.tdengine.com)。 +关于完整的使用手册，系统架构和更多细节，请参考 [TDengine 文档](https://docs.taosdata.com) 或者 [TDengine Documentation](https://docs.tdengine.com)。 # 构建 TDengine 目前可以在 Linux、 Windows 等平台上安装和运行。任何 OS 的应用也可以选择 taosAdapter 的 RESTful 接口连接服务端 taosd。CPU 支持 X64/ARM64，后续会支持 MIPS64、Alpha64、ARM32、RISC-V 等 CPU 架构。 -用户可根据需求选择通过源码、[容器](https://docs.taosdata.com/3.0/get-started/docker/)、[安装包](https://docs.taosdata.com/3.0/get-started/package/)或[Kubenetes](https://docs.taosdata.com/3.0/deployment/k8s/)来安装。本快速指南仅适用于通过源码安装。 +用户可根据需求选择通过源码、[容器](https://docs.taosdata.com/get-started/docker/)、[安装包](https://docs.taosdata.com/get-started/package/)或[Kubenetes](https://docs.taosdata.com/deployment/k8s/)来安装。本快速指南仅适用于通过源码安装。 TDengine 还提供一组辅助工具软件 taosTools，目前它包含 taosBenchmark（曾命名为 taosdemo）和 taosdump 两个软件。默认 TDengine 编译不包含 taosTools, 您可以在编译 TDengine 时使用`cmake .. -DBUILD_TOOLS=true` 来同时编译 taosTools。 +为了构建TDengine, 请使用 [CMake](https://cmake.org/) 3.0.2 或者更高版本。 + ## 安装工具 ### Ubuntu 18.04 及以上版本 & Debian： @@ -61,7 +63,7 @@ sudo apt-get install -y gcc cmake build-essential git libssl-dev sudo apt install build-essential libjansson-dev libsnappy-dev liblzma-dev libz-dev pkg-config ``` -### CentOS 7.9： +### CentOS 7.9 ```bash sudo yum install epel-release @@ -78,13 +80,15 @@ sudo dnf install -y gcc gcc-c++ make cmake epel-release git openssl-devel #### 在 CentOS 上构建 taosTools 安装依赖软件 -#### For CentOS 7/RHEL + +#### CentOS 7.9 + ``` sudo yum install -y zlib-devel xz-devel snappy-devel jansson jansson-devel pkgconfig libatomic libstdc++-static openssl-devel ``` -#### For CentOS 8/Rocky Linux +#### CentOS 8/Rocky Linux ``` sudo yum install -y epel-release @@ -129,14 +133,16 @@ TDengine 包含数个使用 Rust 语言开发的组件. 请参考 rust-lang.org git clone https://github.com/taosdata/TDengine.git cd TDengine ``` - -Go 连接器和 Grafana 插件已移到其他独立仓库。 如果使用 https 协议下载比较慢，可以通过修改 ~/.gitconfig 文件添加以下两行设置使用 ssh 协议下载。需要首先上传 ssh 密钥到 GitHub，详细方法请参考 GitHub 官方文档。 ``` [url "git@github.com:"] insteadOf = https://github.com/ ``` +## 特别说明 + +[JDBC 连接器](https://github.com/taosdata/taos-connector-jdbc)， [Go 连接器](https://github.com/taosdata/driver-go)，[Python 连接器](https://github.com/taosdata/taos-connector-python)，[Node.js 连接器](https://github.com/taosdata/taos-connector-node)，[C# 连接器](https://github.com/taosdata/taos-connector-dotnet) ，[Rust 连接器](https://github.com/taosdata/taos-connector-rust) 和 [Grafana 插件](https://github.com/taosdata/grafanaplugin)已移到独立仓库。 + ## 构建 TDengine @@ -223,9 +229,9 @@ cmake .. && cmake --build . sudo make install ``` -用户可以在[文件目录结构](https://www.taosdata.com/cn/documentation/administrator#directories)中了解更多在操作系统中生成的目录或文件。 -从 2.0 版本开始, 从源代码安装也会为 TDengine 配置服务管理。 -用户也可以选择[从安装包中安装](https://www.taosdata.com/en/getting-started/#Install-from-Package)。 +用户可以在[文件目录结构](https://docs.taosdata.com/reference/directory/)中了解更多在操作系统中生成的目录或文件。 + +从源代码安装也会为 TDengine 配置服务管理 ，用户也可以选择[从安装包中安装](https://docs.taosdata.com/get-started/package/)。 安装成功后，在终端中启动 TDengine 服务： @@ -233,13 +239,13 @@ sudo make install sudo systemctl start taosd ``` -用户可以使用 TDengine Shell 来连接 TDengine 服务，在终端中，输入： +用户可以使用 TDengine CLI 来连接 TDengine 服务，在终端中，输入： ```bash taos ``` -如果 TDengine Shell 连接服务成功，将会打印出欢迎消息和版本信息。如果失败，则会打印出错误消息。 +如果 TDengine CLI 连接服务成功，将会打印出欢迎消息和版本信息。如果失败，则会打印出错误消息。 ## Windows 系统 @@ -265,7 +271,7 @@ sudo make install ./build/bin/taosd -c test/cfg ``` -在另一个终端，使用 TDengine shell 连接服务器： +在另一个终端，使用 TDengine CLI 连接服务器： ```bash ./build/bin/taos -c test/cfg diff --git a/README.md b/README.md index e34dc583c8..c915fe3aef 100644 --- a/README.md +++ b/README.md @@ -14,13 +14,17 @@ [![Build status](https://ci.appveyor.com/api/projects/status/kf3pwh2or5afsgl9/branch/master?svg=true)](https://ci.appveyor.com/project/sangshuduo/tdengine-2n8ge/branch/master) [![Coverage Status](https://coveralls.io/repos/github/taosdata/TDengine/badge.svg?branch=develop)](https://coveralls.io/github/taosdata/TDengine?branch=develop) [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/4201/badge)](https://bestpractices.coreinfrastructure.org/projects/4201) -[![tdengine](https://snapcraft.io//tdengine/badge.svg)](https://snapcraft.io/tdengine) + English | [简体中文](README-CN.md) | We are hiring, check [here](https://tdengine.com/careers) # What is TDengine？ -TDengine is an open source, high performance, cloud native time-series database optimized for Internet of Things (IoT), Connected Cars, and Industrial IoT. It enables efficient, real-time data ingestion, processing, and monitoring of TB and even PB scale data per day, generated by billions of sensors and data collectors. TDengine differentiates itself from other TSDBs with the following advantages.: + +TDengine is an open source, high performance , cloud native time-series database (Time-Series Database, TSDB). + +TDengine can be optimized for Internet of Things (IoT), Connected Cars, and Industrial IoT, IT operation and maintenance, finance and other fields. In addition to the core time series database functions, TDengine also provides functions such as caching, data subscription, and streaming computing. It is a minimalist time series data processing platform that minimizes the complexity of system design and reduces R&D and operating costs. Compared with other time series databases, the main advantages of TDengine are as follows: + - High-Performance: TDengine is the only time-series database to solve the high cardinality issue to support billions of data collection points while out performing other time-series databases for data ingestion, querying and data compression. @@ -36,17 +40,24 @@ TDengine is an open source, high performance, cloud native time-series database # Documentation -For user manual, system design and architecture, please refer to [TDengine Documentation](https://docs.tdengine.com) ([中文版](https://docs.taosdata.com)) +For user manual, system design and architecture, please refer to [TDengine Documentation](https://docs.taosdata.com) ([TDengine 文档](https://docs.taosdata.com)) # Building -At the moment, TDengine server supports running on Linux, Windows, and macOS systems. You can choose to [install from packages](https://www.tdengine.com/getting-started/#Install-from-Package) or build it from the source code. This quick guide is for installation from the source only. -We provide a few useful tools such as taosBenchmark (was named taosdemo) and taosdump. They were part of TDengine. By default, TDengine compiling does not include taosTools. You can use 'cmake .. -DBUILD_TOOLS=true' to make them be compiled with TDengine. +At the moment, TDengine server supports running on Linux, Windows systems.Any OS application can also choose the RESTful interface of taosAdapter to connect the taosd service . TDengine supports X64/ARM64 CPU , and it will support MIPS64, Alpha64, ARM32, RISC-V and other CPU architectures in the future. + + + +You can choose to install through source code according to your needs, [container](https://docs.taosdata.com/get-started/docker/), [installation package](https://docs.taosdata.com/get-started/package/) or [Kubenetes](https://docs.taosdata.com/deployment/k8s/) to install. This quick guide only applies to installing from source. + + + +TDengine provide a few useful tools such as taosBenchmark (was named taosdemo) and taosdump. They were part of TDengine. By default, TDengine compiling does not include taosTools. You can use `cmake .. -DBUILD_TOOLS=true` to make them be compiled with TDengine. To build TDengine, use [CMake](https://cmake.org/) 3.0.2 or higher versions in the project directory. -## Install build dependencies +## Install build tools ### Ubuntu 18.04 and above or Debian @@ -56,6 +67,7 @@ sudo apt-get install -y gcc cmake build-essential git libssl-dev #### Install build dependencies for taosTools + To build the [taosTools](https://github.com/taosdata/taos-tools) on Ubuntu/Debian, the following packages need to be installed. ```bash @@ -79,16 +91,32 @@ sudo dnf install -y gcc gcc-c++ make cmake epel-release git openssl-devel #### Install build dependencies for taosTools on CentOS -To build the [taosTools](https://github.com/taosdata/taos-tools) on CentOS, the following packages need to be installed. -```bash -sudo yum install zlib-devel xz-devel snappy-devel jansson jansson-devel pkgconfig libatomic libstdc++-static openssl-devel +#### CentOS 7.9 + +``` +sudo yum install -y zlib-devel xz-devel snappy-devel jansson jansson-devel pkgconfig libatomic libstdc++-static openssl-devel +``` + +#### CentOS 8/Rocky Linux + +``` +sudo yum install -y epel-release +sudo yum install -y dnf-plugins-core +sudo yum config-manager --set-enabled powertools +sudo yum install -y zlib-devel xz-devel snappy-devel jansson jansson-devel pkgconfig libatomic libstdc++-static openssl-devel ``` Note: Since snappy lacks pkg-config support (refer to [link](https://github.com/google/snappy/pull/86)), it leads a cmake prompt libsnappy not found. But snappy still works well. +If the powertools installation fails, you can try to use: +``` +sudo yum config-manager --set-enabled Powertools +``` + ### Setup golang environment + TDengine includes a few components like taosAdapter developed by Go language. Please refer to golang.org official documentation for golang environment setup. Please use version 1.14+. For the user in China, we recommend using a proxy to accelerate package downloading. @@ -98,6 +126,12 @@ go env -w GO111MODULE=on go env -w GOPROXY=https://goproxy.cn,direct ``` +The default will not build taosAdapter, but you can use the following command to build taosAdapter as the service for RESTful interface. + +``` +cmake .. -DBUILD_HTTP=false +``` + ### Setup rust environment TDengine includes a few compoments developed by Rust language. Please refer to rust-lang.org official documentation for rust environment setup. @@ -111,7 +145,6 @@ git clone https://github.com/taosdata/TDengine.git cd TDengine ``` -The connectors for go & Grafana and some tools have been moved to separated repositories. You can modify the file ~/.gitconfig to use ssh protocol instead of https for better download speed. You will need to upload ssh public key to GitHub first. Please refer to GitHub official documentation for detail. @@ -120,10 +153,16 @@ You can modify the file ~/.gitconfig to use ssh protocol instead of https for be insteadOf = https://github.com/ ``` +## Special Note + + +[JDBC Connector](https://github.com/taosdata/taos-connector-jdbc)， [Go Connector](https://github.com/taosdata/driver-go)，[Python Connector](https://github.com/taosdata/taos-connector-python)，[Node.js Connector](https://github.com/taosdata/taos-connector-node)，[C# Connector](https://github.com/taosdata/taos-connector-dotnet) ，[Rust Connector](https://github.com/taosdata/taos-connector-rust) and [Grafana plugin](https://github.com/taosdata/grafanaplugin) has been moved to standalone repository. + ## Build TDengine ### On Linux platform + You can run the bash script `build.sh` to build both TDengine and taosTools including taosBenchmark and taosdump as below: ```bash @@ -139,11 +178,6 @@ cmake .. -DBUILD_TOOLS=true make ``` -Note TDengine 2.3.x.0 and later use a component named 'taosAdapter' to play http daemon role. If you pull TDengine source code to the latest from an existing codebase, please execute 'git submodule update --init --recursive' to pull taosAdapter source code, and use the following command to choose to build taosAdapter. - -``` -cmake .. -DBUILD_HTTP=false -``` You can use Jemalloc as memory allocator instead of glibc: @@ -212,8 +246,9 @@ After building successfully, TDengine can be installed by sudo make install ``` -Users can find more information about directories installed on the system in the [directory and files](https://www.taosdata.com/en/documentation/administrator/#Directory-and-Files) section. Since version 2.0, installing from source code will also configure service management for TDengine. -Users can also choose to [install from packages](https://www.taosdata.com/en/getting-started/#Install-from-Package) for it. +Users can find more information about directories installed on the system in the [directory and files](https://docs.taosdata.com/reference/directory/) section. + +Installing from source code will also configure service management for TDengine.Users can also choose to [install from packages](https://docs.taosdata.com/get-started/package/) for it. To start the service after installation, in a terminal, use: @@ -221,13 +256,13 @@ To start the service after installation, in a terminal, use: sudo systemctl start taosd ``` -Then users can use the [TDengine shell](https://www.taosdata.com/en/getting-started/#TDengine-Shell) to connect the TDengine server. In a terminal, use: +Then users can use the TDengine CLI to connect the TDengine server. In a terminal, use: ```bash taos ``` -If TDengine shell connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown. +If TDengine CLI connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown. ## On Windows platform @@ -253,7 +288,7 @@ If you don't want to run TDengine as a service, you can run it in current shell. ./build/bin/taosd -c test/cfg ``` -In another terminal, use the TDengine shell to connect the server: +In another terminal, use the TDengine CLI to connect the server: ```bash ./build/bin/taos -c test/cfg @@ -263,7 +298,7 @@ option "-c test/cfg" specifies the system configuration file directory. # Try TDengine -It is easy to run SQL commands from TDengine shell which is the same as other SQL databases. +It is easy to run SQL commands from TDengine CLI which is the same as other SQL databases. ```sql CREATE DATABASE demo; @@ -283,7 +318,7 @@ Query OK, 2 row(s) in set (0.001700s) ## Official Connectors -TDengine provides abundant developing tools for users to develop on TDengine. Follow the links below to find your desired connectors and relevant documentation. +TDengine provides abundant developing tools for users to develop on TDengine. include C/C++、Java、Python、Go、Node.js、C# 、RESTful ,Follow the links below to find your desired connectors and relevant documentation. - [Java](https://docs.taosdata.com/reference/connector/java/) - [C/C++](https://docs.taosdata.com/reference/connector/cpp/) @@ -294,11 +329,6 @@ TDengine provides abundant developing tools for users to develop on TDengine. Fo - [C#](https://docs.taosdata.com/reference/connector/csharp/) - [RESTful API](https://docs.taosdata.com/reference/rest-api/) -# How to run the test cases and how to add a new test case - -TDengine's test framework and all test cases are fully open source. -Please refer to [this document](https://github.com/taosdata/TDengine/blob/develop/tests/How-To-Run-Test-And-How-To-Add-New-Test-Case.md) for how to run test and develop new test case. - # Contribute to TDengine Please follow the [contribution guidelines](CONTRIBUTING.md) to contribute to the project. @@ -306,7 +336,3 @@ Please follow the [contribution guidelines](CONTRIBUTING.md) to contribute to th # Join TDengine WeChat Group Add WeChat “tdengine” to join the group，you can communicate with other users. - -# [User List](https://github.com/taosdata/TDengine/issues/2432) - -If you are using TDengine and feel it helps or you'd like to do some contributions, please add your company to [user list](https://github.com/taosdata/TDengine/issues/2432) and let us know your needs. diff --git a/cmake/taostools_CMakeLists.txt.in b/cmake/taostools_CMakeLists.txt.in index 3a6eb3c25a..1751549680 100644 --- a/cmake/taostools_CMakeLists.txt.in +++ b/cmake/taostools_CMakeLists.txt.in @@ -2,7 +2,7 @@ # taos-tools ExternalProject_Add(taos-tools GIT_REPOSITORY https://github.com/taosdata/taos-tools.git - GIT_TAG 53a0103 + GIT_TAG d237772 SOURCE_DIR "${TD_SOURCE_DIR}/tools/taos-tools" BINARY_DIR "" #BUILD_IN_SOURCE TRUE diff --git a/contrib/CMakeLists.txt b/contrib/CMakeLists.txt index de7b75a245..b4e8825431 100644 --- a/contrib/CMakeLists.txt +++ b/contrib/CMakeLists.txt @@ -27,10 +27,6 @@ else () cat("${TD_SUPPORT_DIR}/taosadapter_CMakeLists.txt.in" ${CONTRIB_TMP_FILE}) endif() -if(TD_LINUX_64 AND JEMALLOC_ENABLED) - cat("${TD_SUPPORT_DIR}/jemalloc_CMakeLists.txt.in" ${CONTRIB_TMP_FILE}) -endif() - # pthread if(${BUILD_PTHREAD}) cat("${TD_SUPPORT_DIR}/pthread_CMakeLists.txt.in" ${CONTRIB_TMP_FILE}) @@ -396,19 +392,6 @@ if(${BUILD_WITH_SQLITE}) endif(NOT TD_WINDOWS) endif(${BUILD_WITH_SQLITE}) -# jemalloc -IF (TD_LINUX_64 AND JEMALLOC_ENABLED) - include(ExternalProject) - ExternalProject_Add(jemalloc - PREFIX "jemalloc" - SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/jemalloc - BUILD_IN_SOURCE 1 - CONFIGURE_COMMAND ./autogen.sh COMMAND ./configure --prefix=${CMAKE_BINARY_DIR}/build/ - BUILD_COMMAND ${MAKE} - ) - INCLUDE_DIRECTORIES(${CMAKE_BINARY_DIR}/build/include) -ENDIF () - # addr2line if(${BUILD_ADDR2LINE}) if(NOT ${TD_WINDOWS}) diff --git a/docs/en/07-develop/_sub_java.mdx b/docs/en/07-develop/_sub_java.mdx index ab77f61348..e7de158cc8 100644 --- a/docs/en/07-develop/_sub_java.mdx +++ b/docs/en/07-develop/_sub_java.mdx @@ -1,7 +1,9 @@ ```java {{#include docs/examples/java/src/main/java/com/taos/example/SubscribeDemo.java}} ``` -:::note -For now Java connector doesn't provide asynchronous subscription, but `TimerTask` can be used to achieve similar purpose. - -::: \ No newline at end of file +```java +{{#include docs/examples/java/src/main/java/com/taos/example/MetersDeserializer.java}} +``` +```java +{{#include docs/examples/java/src/main/java/com/taos/example/Meters.java}} +``` \ No newline at end of file diff --git a/docs/en/07-develop/_sub_rust.mdx b/docs/en/07-develop/_sub_rust.mdx index afb8d79daa..0021666a70 100644 --- a/docs/en/07-develop/_sub_rust.mdx +++ b/docs/en/07-develop/_sub_rust.mdx @@ -1,3 +1,3 @@ -```rs +```rust {{#include docs/examples/rust/nativeexample/examples/subscribe_demo.rs}} -``` \ No newline at end of file +``` diff --git a/docs/en/14-reference/03-connector/03-connector.mdx b/docs/en/14-reference/03-connector/03-connector.mdx index baec6abe38..c3f4530023 100644 --- a/docs/en/14-reference/03-connector/03-connector.mdx +++ b/docs/en/14-reference/03-connector/03-connector.mdx @@ -8,16 +8,13 @@ TDengine provides a rich set of APIs (application development interface). To fac ## Supported platforms -Currently, TDengine's native interface connectors can support platforms such as X64/X86/ARM64/ARM32/MIPS/Alpha hardware platforms and Linux/Win64/Win32 development environments. The comparison matrix is as follows. +Currently, TDengine's native interface connectors can support platforms such as X64/ARM64 hardware platforms and Linux/Win64 development environments. The comparison matrix is as follows. | **CPU** | **OS** | **JDBC** | **Python** | **Go** | **Node.js** | **C#** | **Rust** | C/C++ | | ------- | ------ | -------- | ---------- | ------ | ----------- | ------ | -------- | ----- | | **X86 64bit** | **Linux** | ● | ● | ● | ● | ● | ● | ● | | **X86 64bit** | **Win64** | ● | ● | ● | ● | ● | ● | ● | -| **X86 64bit** | **Win32** | ● | ● | ● | ● | ○ | ○ | ● | -| **X86 32bit** | **Win32** | ○ | ○ | ○ | ○ | ○ | ○ | ● | | **ARM64** | **Linux** | ● | ● | ● | ● | ○ | ○ | ● | -| **MIPS** | **Linux** | ○ | ○ | ○ | ○ | ○ | ○ | ○ | Where ● means the official test verification passed, ○ means the unofficial test verification passed, -- means no assurance. diff --git a/docs/en/14-reference/03-connector/java.mdx b/docs/en/14-reference/03-connector/java.mdx index 22f99bb9ae..cbf7daa879 100644 --- a/docs/en/14-reference/03-connector/java.mdx +++ b/docs/en/14-reference/03-connector/java.mdx @@ -41,19 +41,20 @@ Please refer to [Version Support List](/reference/connector#version-support). TDengine currently supports timestamp, number, character, Boolean type, and the corresponding type conversion with Java is as follows: -| TDengine DataType | JDBCType (driver version < 2.0.24) | JDBCType (driver version > = 2.0.24) | -| ----------------- | ---------------------------------- | ------------------------------------ | -| TIMESTAMP | java.lang.Long | java.sql.Timestamp | -| INT | java.lang.Integer | java.lang.Integer | -| BIGINT | java.lang.Long | java.lang.Long | -| FLOAT | java.lang.Float | java.lang.Float | -| DOUBLE | java.lang.Double | java.lang.Double | -| SMALLINT | java.lang.Short | java.lang.Short | -| TINYINT | java.lang.Byte | java.lang.Byte | -| BOOL | java.lang.Boolean | java.lang.Boolean | -| BINARY | java.lang.String | byte array | -| NCHAR | java.lang.String | java.lang.String | -| JSON | - | java.lang.String | + +| TDengine DataType | JDBCType | +| ----------------- | ---------------------------------- | +| TIMESTAMP | java.sql.Timestamp | +| INT | java.lang.Integer | +| BIGINT | java.lang.Long | +| FLOAT | java.lang.Float | +| DOUBLE | java.lang.Double | +| SMALLINT | java.lang.Short | +| TINYINT | java.lang.Byte | +| BOOL | java.lang.Boolean | +| BINARY | byte array | +| NCHAR | java.lang.String | +| JSON | java.lang.String | **Note**: Only TAG supports JSON types @@ -81,7 +82,7 @@ Add following dependency in the `pom.xml` file of your Maven project: com.taosdata.jdbc taos-jdbcdriver - 2.0.** + 3.0.0 ``` @@ -845,7 +846,13 @@ Please refer to: [JDBC example](https://github.com/taosdata/TDengine/tree/develo **Cause**: Currently, TDengine only supports 64-bit JDK. - **Solution**: Reinstall the 64-bit JDK. 4. + **Solution**: Reinstall the 64-bit JDK. + +4. java.lang.NoSuchMethodError: setByteArray + + **Cause**: taos-jdbcdriver version 3.* only supports TDengine 3.0 or above. + + **Solution**: connect TDengine 2.* using taos-jdbcdriver 2.* version. For other questions, please refer to [FAQ](/train-faq/faq) diff --git a/docs/en/14-reference/09-support-platform/index.md b/docs/en/14-reference/09-support-platform/index.md index ee2479e817..0bb269f232 100644 --- a/docs/en/14-reference/09-support-platform/index.md +++ b/docs/en/14-reference/09-support-platform/index.md @@ -5,11 +5,11 @@ description: "List of platforms supported by TDengine server, client, and connec ## List of supported platforms for TDengine server -| | **Windows 10/11** | **CentOS 7.9/8** | **Ubuntu 18/20** | **Other Linux** | **UOS** | **Kylin** | **Ningsi V60/V80** | **HUAWEI EulerOS** | -| ------------------ | ----------------- | ---------------- | ---------------- | --------------- | ------- | --------- | ------------------ | ------------------ | -| X64 | ● | ● | ● | | ● | ● | ● | | -| Raspberry Pi ARM64 | | | | ● | | | | | -| HUAWEI cloud ARM64 | | | | | | | | ● | +| | **Windows server 2016/2019** | **Windows 10/11** | **CentOS 7.9/8** | **Ubuntu 18/20** | **UOS** | **kylin** | **Ningsi V60/V80** | +| ------------------ | ---------------------------- | ----------------- | ---------------- | ---------------- | ------- | --------- | ------------------ | +| X64 | ● | ● | ● | ● | ● | ● | ● | +| Raspberry Pi ARM64 | | | ● | | | | | +| HUAWEI Cloud ARM64 | | | | ● | | | | Note: ● means officially tested and verified, ○ means unofficially tested and verified. @@ -19,15 +19,15 @@ TDengine's connector can support a wide range of platforms, including X64/X86/AR The comparison matrix is as follows. -| **CPU** | **X64 64bit** | | | **X86 32bit** | **ARM64** | **MIPS** | **Alpha** | -| ----------- | ------------- | --------- | --------- | ------------- | --------- | --------- | --------- | -| **OS** | **Linux** | **Win64** | **Win32** | **Win32** | **Linux** | **Linux** | **Linux** | -| **C/C++** | ● | ● | ● | ○ | ● | ● | ● | -| **JDBC** | ● | ● | ● | ○ | ● | ● | ● | -| **Python** | ● | ● | ● | ○ | ● | ● | -- | -| **Go** | ● | ● | ● | ○ | ● | ○ | -- | -| **NodeJs** | ● | ● | ○ | ○ | ● | ○ | -- | -| **C#** | ● | ● | ○ | ○ | ○ | ○ | -- | -| **RESTful** | ● | ● | ● | ● | ● | ● | ● | +| **CPU** | **X64 64bit** | **X64 64bit** | **ARM64** | +| ----------- | ------------- | ------------- | --------- | +| **OS** | **Linux** | **Win64** | **Linux** | +| **C/C++** | ● | ● | ● | +| **JDBC** | ● | ● | ● | +| **Python** | ● | ● | ● | +| **Go** | ● | ● | ● | +| **NodeJs** | ● | ● | ● | +| **C#** | ● | ● | ○ | +| **RESTful** | ● | ● | ● | Note: ● means the official test is verified, ○ means the unofficial test is verified, -- means not verified. diff --git a/docs/examples/csharp/QueryExample.cs b/docs/examples/csharp/QueryExample.cs index c90a8cd0b7..d75bb8d661 100644 --- a/docs/examples/csharp/QueryExample.cs +++ b/docs/examples/csharp/QueryExample.cs @@ -10,7 +10,7 @@ namespace TDengineExample { IntPtr conn = GetConnection(); // run query - IntPtr res = TDengine.Query(conn, "SELECT * FROM test.meters LIMIT 2"); + IntPtr res = TDengine.Query(conn, "SELECT * FROM meters LIMIT 2"); if (TDengine.ErrorNo(res) != 0) { Console.WriteLine("Failed to query since: " + TDengine.Error(res)); diff --git a/docs/examples/java/pom.xml b/docs/examples/java/pom.xml index a48ba398da..634c3f75a8 100644 --- a/docs/examples/java/pom.xml +++ b/docs/examples/java/pom.xml @@ -21,7 +21,7 @@ com.taosdata.jdbc taos-jdbcdriver - 2.0.38 + 3.0.0 diff --git a/docs/examples/java/src/main/java/com/taos/example/Meters.java b/docs/examples/java/src/main/java/com/taos/example/Meters.java new file mode 100644 index 0000000000..0f1eadd55b --- /dev/null +++ b/docs/examples/java/src/main/java/com/taos/example/Meters.java @@ -0,0 +1,62 @@ +package com.taos.example; + +import java.sql.Timestamp; + +public class Meters { + private Timestamp ts; + private float current; + private int voltage; + private int groupid; + private String location; + + public Timestamp getTs() { + return ts; + } + + public void setTs(Timestamp ts) { + this.ts = ts; + } + + public float getCurrent() { + return current; + } + + public void setCurrent(float current) { + this.current = current; + } + + public int getVoltage() { + return voltage; + } + + public void setVoltage(int voltage) { + this.voltage = voltage; + } + + public int getGroupid() { + return groupid; + } + + public void setGroupid(int groupid) { + this.groupid = groupid; + } + + public String getLocation() { + return location; + } + + public void setLocation(String location) { + this.location = location; + } + + @Override + public String toString() { + return "Meters{" + + "ts=" + ts + + ", current=" + current + + ", voltage=" + voltage + + ", groupid=" + groupid + + ", location='" + location + '\'' + + '}'; + } +} diff --git a/docs/examples/java/src/main/java/com/taos/example/MetersDeserializer.java b/docs/examples/java/src/main/java/com/taos/example/MetersDeserializer.java new file mode 100644 index 0000000000..9b7fa35e90 --- /dev/null +++ b/docs/examples/java/src/main/java/com/taos/example/MetersDeserializer.java @@ -0,0 +1,6 @@ +package com.taos.example; + +import com.taosdata.jdbc.tmq.ReferenceDeserializer; + +public class MetersDeserializer extends ReferenceDeserializer { +} \ No newline at end of file diff --git a/docs/examples/java/src/main/java/com/taos/example/SubscribeDemo.java b/docs/examples/java/src/main/java/com/taos/example/SubscribeDemo.java index d82d03b9de..b1e675cdf6 100644 --- a/docs/examples/java/src/main/java/com/taos/example/SubscribeDemo.java +++ b/docs/examples/java/src/main/java/com/taos/example/SubscribeDemo.java @@ -1,65 +1,77 @@ package com.taos.example; -import com.taosdata.jdbc.TSDBConnection; -import com.taosdata.jdbc.TSDBDriver; -import com.taosdata.jdbc.TSDBResultSet; -import com.taosdata.jdbc.TSDBSubscribe; +import com.taosdata.jdbc.tmq.ConsumerRecords; +import com.taosdata.jdbc.tmq.TMQConstants; +import com.taosdata.jdbc.tmq.TaosConsumer; import java.sql.Connection; import java.sql.DriverManager; -import java.sql.ResultSetMetaData; import java.sql.SQLException; +import java.sql.Statement; +import java.time.Duration; +import java.util.Collections; import java.util.Properties; -import java.util.concurrent.TimeUnit; +import java.util.Timer; +import java.util.TimerTask; +import java.util.concurrent.atomic.AtomicBoolean; public class SubscribeDemo { - private static final String topic = "topic-meter-current-bg-10"; - private static final String sql = "select * from meters where current > 10"; + private static final String TOPIC = "tmq_topic"; + private static final String DB_NAME = "meters"; + private static final AtomicBoolean shutdown = new AtomicBoolean(false); public static void main(String[] args) { - Connection connection = null; - TSDBSubscribe subscribe = null; - + Timer timer = new Timer(); + timer.schedule(new TimerTask() { + public void run() { + shutdown.set(true); + } + }, 3_000); try { + // prepare Class.forName("com.taosdata.jdbc.TSDBDriver"); + String jdbcUrl = "jdbc:TAOS://127.0.0.1:6030/?user=root&password=taosdata"; + Connection connection = DriverManager.getConnection(jdbcUrl); + try (Statement statement = connection.createStatement()) { + statement.executeUpdate("drop topic if exists " + TOPIC); + statement.executeUpdate("drop database if exists " + DB_NAME); + statement.executeUpdate("create database " + DB_NAME); + statement.executeUpdate("use " + DB_NAME); + statement.executeUpdate( + "CREATE TABLE `meters` (`ts` TIMESTAMP, `current` FLOAT, `voltage` INT) TAGS (`groupid` INT, `location` BINARY(16))"); + statement.executeUpdate("CREATE TABLE `d0` USING `meters` TAGS(0, 'Los Angles')"); + statement.executeUpdate("INSERT INTO `d0` values(now - 10s, 0.32, 116)"); + statement.executeUpdate("INSERT INTO `d0` values(now - 8s, NULL, NULL)"); + statement.executeUpdate( + "INSERT INTO `d1` USING `meters` TAGS(1, 'San Francisco') values(now - 9s, 10.1, 119)"); + statement.executeUpdate( + "INSERT INTO `d1` values (now-8s, 10, 120) (now - 6s, 10, 119) (now - 4s, 11.2, 118)"); + // create topic + statement.executeUpdate("create topic " + TOPIC + " as select * from meters"); + } + + // create consumer Properties properties = new Properties(); - properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8"); - properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8"); - String jdbcUrl = "jdbc:TAOS://127.0.0.1:6030/power?user=root&password=taosdata"; - connection = DriverManager.getConnection(jdbcUrl, properties); - // create subscribe - subscribe = ((TSDBConnection) connection).subscribe(topic, sql, true); - int count = 0; - while (count < 10) { - // wait 1 second to avoid frequent calls to consume - TimeUnit.SECONDS.sleep(1); - // consume - TSDBResultSet resultSet = subscribe.consume(); - if (resultSet == null) { - continue; - } - ResultSetMetaData metaData = resultSet.getMetaData(); - while (resultSet.next()) { - int columnCount = metaData.getColumnCount(); - for (int i = 1; i <= columnCount; i++) { - System.out.print(metaData.getColumnLabel(i) + ": " + resultSet.getString(i) + "\t"); + properties.setProperty(TMQConstants.BOOTSTRAP_SERVERS, "127.0.0.1:6030"); + properties.setProperty(TMQConstants.MSG_WITH_TABLE_NAME, "true"); + properties.setProperty(TMQConstants.ENABLE_AUTO_COMMIT, "true"); + properties.setProperty(TMQConstants.GROUP_ID, "test"); + properties.setProperty(TMQConstants.VALUE_DESERIALIZER, + "com.taosdata.jdbc.MetersDeserializer"); + + // poll data + try (TaosConsumer consumer = new TaosConsumer<>(properties)) { + consumer.subscribe(Collections.singletonList(TOPIC)); + while (!shutdown.get()) { + ConsumerRecords meters = consumer.poll(Duration.ofMillis(100)); + for (Meters meter : meters) { + System.out.println(meter); } - System.out.println(); - count++; } } - } catch (Exception e) { + } catch (ClassNotFoundException | SQLException e) { e.printStackTrace(); - } finally { - try { - if (null != subscribe) - // close subscribe - subscribe.close(true); - if (connection != null) - connection.close(); - } catch (SQLException throwable) { - throwable.printStackTrace(); - } } + timer.cancel(); } } \ No newline at end of file diff --git a/docs/examples/node/nativeexample/connect.js b/docs/examples/node/nativeexample/connect.js index bb027d4fe8..7971cc2348 100644 --- a/docs/examples/node/nativeexample/connect.js +++ b/docs/examples/node/nativeexample/connect.js @@ -1,20 +1,13 @@ -const { options, connect } = require("@tdengine/rest"); +//A cursor also needs to be initialized in order to interact with TDengine from Node.js. +const taos = require("@tdengine/client"); +var conn = taos.connect({ + host: "127.0.0.1", + user: "root", + password: "taosdata", + config: "/etc/taos", + port: 0, +}); +var cursor = conn.cursor(); // Initializing a new cursor -async function test() { - options.path = "/rest/sql"; - options.host = "localhost"; - let conn = connect(options); - let cursor = conn.cursor(); - try { - let res = await cursor.query("SELECT server_version()"); - res.toString(); - } catch (err) { - console.log(err); - } -} -test(); - -// output: -// server_version() | -// =================== -// 3.0.0.0 | +//Close a connection +conn.close(); \ No newline at end of file diff --git a/docs/examples/node/restexample/connect.js b/docs/examples/node/restexample/connect.js index 132e284ce9..bb027d4fe8 100644 --- a/docs/examples/node/restexample/connect.js +++ b/docs/examples/node/restexample/connect.js @@ -1,7 +1,7 @@ const { options, connect } = require("@tdengine/rest"); async function test() { - options.path = "/rest/sqlt"; + options.path = "/rest/sql"; options.host = "localhost"; let conn = connect(options); let cursor = conn.cursor(); diff --git a/docs/zh/01-index.md b/docs/zh/01-index.md index 64a7d419e1..79d5424ac2 100644 --- a/docs/zh/01-index.md +++ b/docs/zh/01-index.md @@ -4,7 +4,7 @@ sidebar_label: 文档首页 slug: / --- -TDengine是一款[开源](https://www.taosdata.com/tdengine/open_source_time-series_database)、[高性能](https://www.taosdata.com/fast)、[云原生](https://www.taosdata.com/tdengine/cloud_native_time-series_database)的时序数据库(Time-Series Database, TSDB), 它专为物联网、工业互联网、金融等场景优化设计。同时它还带有内建的缓存、流式计算、数据订阅等系统功能，能大幅减少系统设计的复杂度，降低研发和运营成本，是一极简的时序数据处理平台。本文档是 TDengine 用户手册，主要是介绍 TDengine 的基本概念、安装、使用、功能、开发接口、运营维护、TDengine 内核设计等等，它主要是面向架构师、开发者与系统管理员的。 +TDengine是一款[开源](https://www.taosdata.com/tdengine/open_source_time-series_database)、[高性能](https://www.taosdata.com/fast)、[云原生](https://www.taosdata.com/tdengine/cloud_native_time-series_database)的时序数据库（Time Series Database, TSDB）, 它专为物联网、工业互联网、金融等场景优化设计。同时它还带有内建的缓存、流式计算、数据订阅等系统功能，能大幅减少系统设计的复杂度，降低研发和运营成本，是一极简的时序数据处理平台。本文档是 TDengine 用户手册，主要是介绍 TDengine 的基本概念、安装、使用、功能、开发接口、运营维护、TDengine 内核设计等等，它主要是面向架构师、开发者与系统管理员的。 TDengine 充分利用了时序数据的特点，提出了“一个数据采集点一张表”与“超级表”的概念，设计了创新的存储引擎，让数据的写入、查询和存储效率都得到极大的提升。为正确理解并使用TDengine, 无论如何，请您仔细阅读[基本概念](./concept)一章。 diff --git a/docs/zh/02-intro.md b/docs/zh/02-intro.md index 97322c68a2..a6ef2b94b6 100644 --- a/docs/zh/02-intro.md +++ b/docs/zh/02-intro.md @@ -3,7 +3,7 @@ title: 产品简介 toc_max_heading_level: 2 --- -TDengine 是一款[开源](https://www.taosdata.com/tdengine/open_source_time-series_database)、[高性能](https://www.taosdata.com/tdengine/fast)、[云原生](https://www.taosdata.com/tdengine/cloud_native_time-series_database)的时序数据库 (Time-Series Database, TSDB)。TDengine 能被广泛运用于物联网、工业互联网、车联网、IT 运维、金融等领域。除核心的时序数据库功能外，TDengine 还提供[缓存](../develop/cache/)、[数据订阅](../develop/tmq)、[流式计算](../develop/stream)等功能，是一极简的时序数据处理平台，最大程度的减小系统设计的复杂度，降低研发和运营成本。 +TDengine 是一款[开源](https://www.taosdata.com/tdengine/open_source_time-series_database)、[高性能](https://www.taosdata.com/tdengine/fast)、[云原生](https://www.taosdata.com/tdengine/cloud_native_time-series_database)的时序数据库（Time Series Database, TSDB）。TDengine 能被广泛运用于物联网、工业互联网、车联网、IT 运维、金融等领域。除核心的时序数据库功能外，TDengine 还提供[缓存](../develop/cache/)、[数据订阅](../develop/tmq)、[流式计算](../develop/stream)等功能，是一极简的时序数据处理平台，最大程度的减小系统设计的复杂度，降低研发和运营成本。 本章节介绍TDengine的主要功能、竞争优势、适用场景、与其他数据库的对比测试等等，让大家对TDengine有个整体的了解。 diff --git a/docs/zh/05-get-started/03-package.md b/docs/zh/05-get-started/03-package.md index 63698aab50..0c178e5962 100644 --- a/docs/zh/05-get-started/03-package.md +++ b/docs/zh/05-get-started/03-package.md @@ -29,6 +29,7 @@ echo "deb [arch=amd64] http://repos.taosdata.com/tdengine-stable stable main" | 如果安装 Beta 版需要安装包仓库 ```bash +wget -qO - http://repos.taosdata.com/tdengine.key | sudo apt-key add - echo "deb [arch=amd64] http://repos.taosdata.com/tdengine-beta beta main" | sudo tee /etc/apt/sources.list.d/tdengine-beta.list ``` diff --git a/docs/zh/07-develop/04-query-data/index.mdx b/docs/zh/07-develop/04-query-data/index.mdx index eecda92744..68f49d9f2b 100644 --- a/docs/zh/07-develop/04-query-data/index.mdx +++ b/docs/zh/07-develop/04-query-data/index.mdx @@ -43,7 +43,7 @@ Query OK, 2 row(s) in set (0.001100s) 为满足物联网场景的需求，TDengine 支持几个特殊的函数，比如 twa(时间加权平均)，spread (最大值与最小值的差)，last_row(最后一条记录)等，更多与物联网场景相关的函数将添加进来。 -具体的查询语法请看 [TAOS SQL 的数据查询](/taos-sql/select) 章节。 +具体的查询语法请看 [TAOS SQL 的数据查询](../../taos-sql/select) 章节。 ## 多表聚合查询 @@ -74,7 +74,7 @@ taos> SELECT count(*), max(current) FROM meters where groupId = 2 and ts > now - Query OK, 1 row(s) in set (0.002136s) ``` -在 [TAOS SQL 的数据查询](/taos-sql/select) 一章，查询类操作都会注明是否支持超级表。 +在 [TAOS SQL 的数据查询](../../taos-sql/select) 一章，查询类操作都会注明是否支持超级表。 ## 降采样查询、插值 @@ -121,7 +121,7 @@ Query OK, 5 row(s) in set (0.001521s) 如果一个时间间隔里，没有采集的数据，TDengine 还提供插值计算的功能。 -语法规则细节请见 [TAOS SQL 的按时间窗口切分聚合](/taos-sql/interval) 章节。 +语法规则细节请见 [TAOS SQL 的按时间窗口切分聚合](../../taos-sql/distinguished) 章节。 ## 示例代码 diff --git a/docs/zh/07-develop/06-stream.md b/docs/zh/07-develop/06-stream.md index dbf8684008..d5296582d5 100644 --- a/docs/zh/07-develop/06-stream.md +++ b/docs/zh/07-develop/06-stream.md @@ -4,8 +4,16 @@ description: "TDengine 流式计算将数据的写入、预处理、复杂分析 title: 流式计算 --- -在时序数据的处理中，经常要对原始数据进行清洗、预处理，再使用时序数据库进行长久的储存。用户通常需要在时序数据库之外再搭建 Kafka、Flink、Spark 等流计算处理引擎，增加了用户的开发成本和维护成本。 -使用 TDengine 3.0 的流式计算引擎能够最大限度的减少对这些额外中间件的依赖，真正将数据的写入、预处理、长期存储、复杂分析、实时计算、实时报警触发等功能融为一体，并且，所有这些任务只需要使用 SQL 完成，极大降低了用户的学习成本、使用成本。 +在时序数据的处理中，经常要对原始数据进行清洗、预处理，再使用时序数据库进行长久的储存。在传统的时序数据解决方案中，常常需要部署 Kafka、Flink 等流处理系统。而流处理系统的复杂性，带来了高昂的开发与运维成本。 + +TDengine 3.0 的流式计算引擎提供了实时处理写入的数据流的能力，使用 SQL 定义实时流变换，当数据被写入流的源表后，数据会被以定义的方式自动处理，并根据定义的触发模式向目的表推送结果。它提供了替代复杂流处理系统的轻量级解决方案，并能够在高吞吐的数据写入的情况下，提供毫秒级的计算结果延迟。 + +流式计算可以包含数据过滤，标量函数计算（含UDF），以及窗口聚合（支持滑动窗口、会话窗口与状态窗口），可以以超级表、子表、普通表为源表，写入到目的超级表。在创建流时，目的超级表将被自动创建，随后新插入的数据会被流定义的方式处理并写入其中，通过 partition by 子句，可以以表名或标签划分 partition，不同的 partition 将写入到目的超级表的不同子表。 + +TDengine 的流式计算能够支持分布在多个 vnode 中的超级表聚合；还能够处理乱序数据的写入：它提供了 watermark 机制以度量容忍数据乱序的程度，并提供了 ignore expired 配置项以决定乱序数据的处理策略——丢弃或者重新计算。 + +详见 [流式计算](../../taos-sql/stream) + ## 流式计算的创建 @@ -14,7 +22,7 @@ CREATE STREAM [IF NOT EXISTS] stream_name [stream_options] INTO stb_name AS subq stream_options: { TRIGGER [AT_ONCE | WINDOW_CLOSE | MAX_DELAY time] WATERMARK time - IGNORE EXPIRED + IGNORE EXPIRED [0 | 1] } ``` @@ -22,107 +30,84 @@ stream_options: { ## 示例一 -企业电表的数据经常都是成百上千亿条的，那么想要将这些分散、凌乱的数据清洗或转换都需要比较长的时间，很难做到高效性和实时性，以下例子中，通过流计算可以将过去 12 小时电表电压大于 220V 的数据清洗掉，然后以小时为窗口整合并计算出每个窗口中电流的最大值，并将结果输出到指定的数据表中。 +企业电表的数据经常都是成百上千亿条的，那么想要将这些分散、凌乱的数据清洗或转换都需要比较长的时间，很难做到高效性和实时性，以下例子中，通过流计算可以将电表电压大于 220V 的数据清洗掉，然后以 5 秒为窗口整合并计算出每个窗口中电流的最大值，最后将结果输出到指定的数据表中。 ### 创建 DB 和原始数据表 首先准备数据，完成建库、建一张超级表和多张子表操作 ```sql -drop database if exists stream_db; -create database stream_db; +DROP DATABASE IF EXISTS power; +CREATE DATABASE power; +USE power; -create stable stream_db.meters (ts timestamp, current float, voltage int) TAGS (location varchar(64), groupId int); +CREATE STABLE meters (ts timestamp, current float, voltage int, phase float) TAGS (location binary(64), groupId int); -create table stream_db.d1001 using stream_db.meters tags("beijing", 1); -create table stream_db.d1002 using stream_db.meters tags("guangzhou", 2); -create table stream_db.d1003 using stream_db.meters tags("shanghai", 3); +CREATE TABLE d1001 USING meters TAGS ("California.SanFrancisco", 2); +CREATE TABLE d1002 USING meters TAGS ("California.SanFrancisco", 3); +CREATE TABLE d1003 USING meters TAGS ("California.LosAngeles", 2); +CREATE TABLE d1004 USING meters TAGS ("California.LosAngeles", 3); ``` ### 创建流 ```sql -create stream stream1 into stream_db.stream1_output_stb as select _wstart as start, _wend as end, max(current) as max_current from stream_db.meters where voltage <= 220 and ts > now - 12h interval (1h); +create stream current_stream into current_stream_output_stb as select _wstart as start, _wend as end, max(current) as max_current from meters where voltage <= 220 interval (5s); ``` ### 写入数据 ```sql -insert into stream_db.d1001 values(now-14h, 10.3, 210); -insert into stream_db.d1001 values(now-13h, 13.5, 216); -insert into stream_db.d1001 values(now-12h, 12.5, 219); -insert into stream_db.d1002 values(now-11h, 14.7, 221); -insert into stream_db.d1002 values(now-10h, 10.5, 218); -insert into stream_db.d1002 values(now-9h, 11.2, 220); -insert into stream_db.d1003 values(now-8h, 11.5, 217); -insert into stream_db.d1003 values(now-7h, 12.3, 227); -insert into stream_db.d1003 values(now-6h, 12.3, 215); +insert into d1001 values("2018-10-03 14:38:05.000", 10.30000, 219, 0.31000); +insert into d1001 values("2018-10-03 14:38:15.000", 12.60000, 218, 0.33000); +insert into d1001 values("2018-10-03 14:38:16.800", 12.30000, 221, 0.31000); +insert into d1002 values("2018-10-03 14:38:16.650", 10.30000, 218, 0.25000); +insert into d1003 values("2018-10-03 14:38:05.500", 11.80000, 221, 0.28000); +insert into d1003 values("2018-10-03 14:38:16.600", 13.40000, 223, 0.29000); +insert into d1004 values("2018-10-03 14:38:05.000", 10.80000, 223, 0.29000); +insert into d1004 values("2018-10-03 14:38:06.500", 11.50000, 221, 0.35000); ``` -### 查询以观查结果 +### 查询以观察结果 + ```sql -taos> select * from stream_db.stream1_output_stb; - start | end | max_current | group_id | -=================================================================================================== - 2022-08-09 14:00:00.000 | 2022-08-09 15:00:00.000 | 10.50000 | 0 | - 2022-08-09 15:00:00.000 | 2022-08-09 16:00:00.000 | 11.20000 | 0 | - 2022-08-09 16:00:00.000 | 2022-08-09 17:00:00.000 | 11.50000 | 0 | - 2022-08-09 18:00:00.000 | 2022-08-09 19:00:00.000 | 12.30000 | 0 | -Query OK, 4 rows in database (0.012033s) +taos> select start, end, max_current from current_stream_output_stb; + start | end | max_current | +=========================================================================== + 2018-10-03 14:38:05.000 | 2018-10-03 14:38:10.000 | 10.30000 | + 2018-10-03 14:38:15.000 | 2018-10-03 14:38:20.000 | 12.60000 | +Query OK, 2 rows in database (0.018762s) ``` ## 示例二 -某运营商平台要采集机房所有服务器的系统资源指标，包含 cpu、内存、网络延迟等，采集后需要对数据进行四舍五入运算，将地域和服务器名以下划线拼接，然后将结果按时间排序并以服务器名分组输出到新的数据表中。 + +依然以示例一中的数据为基础，我们已经采集到了每个智能电表的电流和电压数据，现在需要求出有功功率和无功功率，并将地域和电表名以符号 "." 拼接，然后以电表名称分组输出到新的数据表中。 ### 创建 DB 和原始数据表 -首先准备数据，完成建库、建一张超级表和多张子表操作 -```sql -drop database if exists stream_db; -create database stream_db; - -create stable stream_db.idc (ts timestamp, cpu float, mem float, latency float) TAGS (location varchar(64), groupId int); - -create table stream_db.server01 using stream_db.idc tags("beijing", 1); -create table stream_db.server02 using stream_db.idc tags("shanghai", 2); -create table stream_db.server03 using stream_db.idc tags("beijing", 2); -create table stream_db.server04 using stream_db.idc tags("tianjin", 3); -create table stream_db.server05 using stream_db.idc tags("shanghai", 1); -``` +参考示例一 [创建 DB 和原始数据表](#创建-db-和原始数据表) ### 创建流 ```sql -create stream stream2 into stream_db.stream2_output_stb as select ts, concat_ws("_", location, tbname) as server_location, round(cpu) as cpu, round(mem) as mem, round(latency) as latency from stream_db.idc partition by tbname order by ts; +create stream power_stream into power_stream_output_stb as select ts, concat_ws(".", location, tbname) as meter_location, current*voltage*cos(phase) as active_power, current*voltage*sin(phase) as reactive_power from meters partition by tbname; ``` ### 写入数据 -```sql -insert into stream_db.server01 values(now-14h, 50.9, 654.8, 23.11); -insert into stream_db.server01 values(now-13h, 13.5, 221.2, 11.22); -insert into stream_db.server02 values(now-12h, 154.7, 218.3, 22.33); -insert into stream_db.server02 values(now-11h, 120.5, 111.5, 5.55); -insert into stream_db.server03 values(now-10h, 101.5, 125.6, 5.99); -insert into stream_db.server03 values(now-9h, 12.3, 165.6, 6.02); -insert into stream_db.server04 values(now-8h, 160.9, 120.7, 43.51); -insert into stream_db.server04 values(now-7h, 240.9, 520.7, 54.55); -insert into stream_db.server05 values(now-6h, 190.9, 320.7, 55.43); -insert into stream_db.server05 values(now-5h, 110.9, 600.7, 35.54); -``` -### 查询以观查结果 -```sql -taos> select ts, server_location, cpu, mem, latency from stream_db.stream2_output_stb; - ts | server_location | cpu | mem | latency | -================================================================================================================================ - 2022-08-09 21:24:56.785 | beijing_server01 | 51.00000 | 655.00000 | 23.00000 | - 2022-08-09 22:24:56.795 | beijing_server01 | 14.00000 | 221.00000 | 11.00000 | - 2022-08-09 23:24:56.806 | shanghai_server02 | 155.00000 | 218.00000 | 22.00000 | - 2022-08-10 00:24:56.815 | shanghai_server02 | 121.00000 | 112.00000 | 6.00000 | - 2022-08-10 01:24:56.826 | beijing_server03 | 102.00000 | 126.00000 | 6.00000 | - 2022-08-10 02:24:56.838 | beijing_server03 | 12.00000 | 166.00000 | 6.00000 | - 2022-08-10 03:24:56.846 | tianjin_server04 | 161.00000 | 121.00000 | 44.00000 | - 2022-08-10 04:24:56.853 | tianjin_server04 | 241.00000 | 521.00000 | 55.00000 | - 2022-08-10 05:24:56.866 | shanghai_server05 | 191.00000 | 321.00000 | 55.00000 | - 2022-08-10 06:24:57.301 | shanghai_server05 | 111.00000 | 601.00000 | 36.00000 | -Query OK, 10 rows in database (0.022950s) -``` +参考示例一 [写入数据](#写入数据) + +### 查询以观察结果 +```sql +taos> select ts, meter_location, active_power, reactive_power from power_stream_output_stb; + ts | meter_location | active_power | reactive_power | +=================================================================================================================== + 2018-10-03 14:38:05.000 | California.LosAngeles.d1004 | 2307.834596289 | 688.687331847 | + 2018-10-03 14:38:06.500 | California.LosAngeles.d1004 | 2387.415754896 | 871.474763418 | + 2018-10-03 14:38:05.500 | California.LosAngeles.d1003 | 2506.240411679 | 720.680274962 | + 2018-10-03 14:38:16.600 | California.LosAngeles.d1003 | 2863.424274422 | 854.482390839 | + 2018-10-03 14:38:05.000 | California.SanFrancisco.d1001 | 2148.178871730 | 688.120784090 | + 2018-10-03 14:38:15.000 | California.SanFrancisco.d1001 | 2598.589176205 | 890.081451418 | + 2018-10-03 14:38:16.800 | California.SanFrancisco.d1001 | 2588.728381186 | 829.240910475 | + 2018-10-03 14:38:16.650 | California.SanFrancisco.d1002 | 2175.595991997 | 555.520860397 | +Query OK, 8 rows in database (0.014753s) +``` diff --git a/docs/zh/07-develop/07-tmq.md b/docs/zh/07-develop/07-tmq.md index 372d458c36..25d468cad3 100644 --- a/docs/zh/07-develop/07-tmq.md +++ b/docs/zh/07-develop/07-tmq.md @@ -6,6 +6,14 @@ title: 数据订阅 import Tabs from "@theme/Tabs"; import TabItem from "@theme/TabItem"; +import Java from "./_sub_java.mdx"; +import Python from "./_sub_python.mdx"; +import Go from "./_sub_go.mdx"; +import Rust from "./_sub_rust.mdx"; +import Node from "./_sub_node.mdx"; +import CSharp from "./_sub_cs.mdx"; +import CDemo from "./_sub_c.mdx"; + 为了帮助应用实时获取写入 TDengine 的数据，或者以事件到达顺序处理数据，TDengine提供了类似消息队列产品的数据订阅、消费接口。这样在很多场景下，采用 TDengine 的时序数据处理系统不再需要集成消息队列产品，比如 kafka, 从而简化系统设计的复杂度，降低运营维护成本。 @@ -211,16 +219,15 @@ TMQ支持多种订阅类型： - -TODO + -TODO + -TODO + @@ -232,11 +239,11 @@ TODO -TODO + -TODO + diff --git a/docs/zh/07-develop/_sub_java.mdx b/docs/zh/07-develop/_sub_java.mdx index 52df23f7dd..e7de158cc8 100644 --- a/docs/zh/07-develop/_sub_java.mdx +++ b/docs/zh/07-develop/_sub_java.mdx @@ -1,7 +1,9 @@ ```java {{#include docs/examples/java/src/main/java/com/taos/example/SubscribeDemo.java}} ``` -:::note -目前 Java 接口没有提供异步订阅模式，但用户程序可以通过创建 `TimerTask` 等方式达到同样的效果。 - -::: \ No newline at end of file +```java +{{#include docs/examples/java/src/main/java/com/taos/example/MetersDeserializer.java}} +``` +```java +{{#include docs/examples/java/src/main/java/com/taos/example/Meters.java}} +``` \ No newline at end of file diff --git a/docs/zh/07-develop/_sub_rust.mdx b/docs/zh/07-develop/_sub_rust.mdx index afb8d79daa..0021666a70 100644 --- a/docs/zh/07-develop/_sub_rust.mdx +++ b/docs/zh/07-develop/_sub_rust.mdx @@ -1,3 +1,3 @@ -```rs +```rust {{#include docs/examples/rust/nativeexample/examples/subscribe_demo.rs}} -``` \ No newline at end of file +``` diff --git a/docs/zh/07-develop/index.md b/docs/zh/07-develop/index.md index 0393a87ab2..4d0f3c3cea 100644 --- a/docs/zh/07-develop/index.md +++ b/docs/zh/07-develop/index.md @@ -7,7 +7,7 @@ title: 开发指南 2. 根据自己的应用场景，确定数据模型。根据数据特征，决定建立一个还是多个库；分清静态标签、采集量，建立正确的超级表，建立子表。 3. 决定插入数据的方式。TDengine支持使用标准的SQL写入，但同时也支持schemaless模式写入，这样不用手工建表，可以将数据直接写入。 4. 根据业务要求，看需要撰写哪些SQL查询语句。 -5. 如果你要基于时序数据做实时的统计分析，包括各种监测看板，那么建议你采用TDengine的连续查询功能，而不用上线Spark, Flink等复杂的流式计算系统。 +5. 如果你要基于时序数据做轻量级的实时统计分析，包括各种监测看板，那么建议你采用 TDengine 3.0 的流式计算功能，而不用额外部署 Spark, Flink 等复杂的流式计算系统。 6. 如果你的应用有模块需要消费插入的数据，希望有新的数据插入时，就能获取通知，那么建议你采用TDengine提供的数据订阅功能，而无需专门部署Kafka或其他消息队列软件。 7. 在很多场景下(如车辆管理)，应用需要获取每个数据采集点的最新状态，那么建议你采用TDengine的cache功能，而不用单独部署Redis等缓存软件。 8. 如果你发现TDengine的函数无法满足你的要求，那么你可以使用用户自定义函数来解决问题。 diff --git a/docs/zh/12-taos-sql/01-data-type.md b/docs/zh/12-taos-sql/01-data-type.md index 8ac6ee3b87..628086f5a9 100644 --- a/docs/zh/12-taos-sql/01-data-type.md +++ b/docs/zh/12-taos-sql/01-data-type.md @@ -34,7 +34,7 @@ CREATE DATABASE db_name PRECISION 'ns'; | 7 | DOUBLE | 8 | 双精度浮点型，有效位数 15-16，范围 [-1.7E308, 1.7E308] | | 8 | BINARY | 自定义 | 记录单字节字符串，建议只用于处理 ASCII 可见字符，中文等多字节字符需使用 nchar。 | | 9 | SMALLINT | 2 | 短整型， 范围 [-32768, 32767] | -| 10 | SMALLINT UNSIGNED | 2| 无符号短整型，范围 [0, 655357] | +| 10 | SMALLINT UNSIGNED | 2| 无符号短整型，范围 [0, 65535] | | 11 | TINYINT | 1 | 单字节整型，范围 [-128, 127] | | 12 | TINYINT UNSIGNED | 1 | 无符号单字节整型，范围 [0, 255] | | 13 | BOOL | 1 | 布尔型，{true, false} | diff --git a/docs/zh/12-taos-sql/14-stream.md b/docs/zh/12-taos-sql/14-stream.md index 024742ed8b..1a056e278c 100644 --- a/docs/zh/12-taos-sql/14-stream.md +++ b/docs/zh/12-taos-sql/14-stream.md @@ -26,10 +26,19 @@ subquery: SELECT [DISTINCT] select_list [WHERE condition] [PARTITION BY tag_list] [window_clause] - [group_by_clause] ``` -不支持 order_by，limit，slimit，fill 语句 +支持会话窗口、状态窗口与滑动窗口，其中，会话窗口与状态窗口搭配超级表时必须与partition by tbname一起使用 + +```sql +window_clause: { + SESSION(ts_col, tol_val) + | STATE_WINDOW(col) + | INTERVAL(interval_val [, interval_offset]) [SLIDING (sliding_val)] +} +``` + +其中，SESSION 是会话窗口，tol_val 是时间间隔的最大范围。在 tol_val 时间间隔范围内的数据都属于同一个窗口，如果连续的两条数据的时间超过 tol_val，则自动开启下一个窗口。 例如，如下语句创建流式计算，同时自动创建名为 avg_vol 的超级表，此流计算以一分钟为时间窗口、30 秒为前向增量统计这些电表的平均电压，并将来自 meters 表的数据的计算结果写入 avg_vol 表，不同 partition 的数据会分别创建子表并写入不同子表。 @@ -88,23 +97,3 @@ T = 最新事件时间 - watermark 2. 重新计算：从 TSDB 中重新查找对应窗口的所有数据并重新计算得到最新结果 无论在哪种模式下，watermark 都应该被妥善设置，来得到正确结果（直接丢弃模式）或避免频繁触发重算带来的性能开销（重新计算模式）。 - -## 流式计算与会话窗口（session window） - -```sql -window_clause: { - SESSION(ts_col, tol_val) - | STATE_WINDOW(col) - | INTERVAL(interval_val [, interval_offset]) [SLIDING (sliding_val)] [FILL(fill_mod_and_val)] -} -``` - -其中，SESSION 是会话窗口，tol_val 是时间间隔的最大范围。在 tol_val 时间间隔范围内的数据都属于同一个窗口，如果连续的两条数据的时间超过 tol_val，则自动开启下一个窗口。 - -## 流式计算的暂停与恢复 - -```sql -STOP STREAM stream_name; - -RESUME STREAM stream_name; -``` diff --git a/docs/zh/14-reference/03-connector/03-connector.mdx b/docs/zh/14-reference/03-connector/03-connector.mdx index d681903aed..bdad0b7e25 100644 --- a/docs/zh/14-reference/03-connector/03-connector.mdx +++ b/docs/zh/14-reference/03-connector/03-connector.mdx @@ -8,18 +8,13 @@ TDengine 提供了丰富的应用程序开发接口，为了便于用户快速 ## 支持的平台 -目前 TDengine 的原生接口连接器可支持的平台包括：X64/X86/ARM64/ARM32/MIPS/Alpha 等硬件平台，以及 Linux/Win64/Win32 等开发环境。对照矩阵如下： +目前 TDengine 的原生接口连接器可支持的平台包括：X64/ARM64 等硬件平台，以及 Linux/Win64 等开发环境。对照矩阵如下： | **CPU** | **OS** | **Java** | **Python** | **Go** | **Node.js** | **C#** | **Rust** | C/C++ | | -------------- | --------- | -------- | ---------- | ------ | ----------- | ------ | -------- | ----- | | **X86 64bit** | **Linux** | ● | ● | ● | ● | ● | ● | ● | | **X86 64bit** | **Win64** | ● | ● | ● | ● | ● | ● | ● | -| **X86 64bit** | **Win32** | ● | ● | ● | ● | ○ | ○ | ● | -| **X86 32bit** | **Win32** | ○ | ○ | ○ | ○ | ○ | ○ | ● | | **ARM64** | **Linux** | ● | ● | ● | ● | ○ | ○ | ● | -| **MIPS 龙芯** | **Linux** | ○ | ○ | ○ | ○ | ○ | ○ | ○ | -| **Alpha 申威** | **Linux** | ○ | ○ | -- | -- | -- | -- | ○ | -| **X86 海光** | **Linux** | ○ | ○ | ○ | -- | -- | -- | ○ | 其中 ● 表示官方测试验证通过，○ 表示非官方测试验证通过，-- 表示未经验证。 diff --git a/docs/zh/14-reference/03-connector/java.mdx b/docs/zh/14-reference/03-connector/java.mdx index e33d09c1ce..c9d74dcaeb 100644 --- a/docs/zh/14-reference/03-connector/java.mdx +++ b/docs/zh/14-reference/03-connector/java.mdx @@ -83,7 +83,7 @@ Maven 项目中，在 pom.xml 中添加以下依赖： com.taosdata.jdbc taos-jdbcdriver - 2.0.** + 3.0.0 ``` @@ -712,7 +712,7 @@ while(true) { } ``` -`poll` 方法返回一个结果集，其中包含从上次 `poll` 到目前为止的所有新数据。请务必按需选择合理的调用 `poll` 的频率（如例子中的 `Duration.ofMillis(100)`），否则会给服务端造成不必要的压力。 +`poll` 每次调用获取一个消息。请按需选择合理的调用 `poll` 的频率（如例子中的 `Duration.ofMillis(100)`），否则会给服务端造成不必要的压力。 #### 关闭订阅 @@ -900,7 +900,13 @@ public static void main(String[] args) throws Exception { **解决方法**：重新安装 64 位 JDK。 -4. 其它问题请参考 [FAQ](../../../train-faq/faq) +4. java.lang.NoSuchMethodError: setByteArray + + **原因**：taos-jdbcdriver 3.* 版本仅支持 TDengine 3.0 及以上版本。 + + **解决方法**： 使用 taos-jdbcdriver 2.* 版本连接 TDengine 2.* 版本。 + +其它问题请参考 [FAQ](../../../train-faq/faq) ## API 参考 diff --git a/docs/zh/14-reference/09-support-platform/index.md b/docs/zh/14-reference/09-support-platform/index.md index 02a3dbdd15..ddacdd369a 100644 --- a/docs/zh/14-reference/09-support-platform/index.md +++ b/docs/zh/14-reference/09-support-platform/index.md @@ -5,11 +5,11 @@ description: "TDengine 服务端、客户端和连接器支持的平台列表" ## TDengine 服务端支持的平台列表 -| | **Windows 10/11** | **CentOS 7.9/8** | **Ubuntu 18/20** | **Other Linux** | **统信 UOS** | **银河/中标麒麟** | **凝思 V60/V80** | **华为 EulerOS** | -| ------------ | ----------------- | ---------------- | ---------------- | --------------- | ------------ | ----------------- | ---------------- | ---------------- | -| X64 | ● | ● | ● | | ● | ● | ● | | -| 树莓派 ARM64 | | | | ● | | | | | -| 华为云 ARM64 | | | | | | | | ● | +| | **Windows server 2016/2019** | **Windows 10/11** | **CentOS 7.9/8** | **Ubuntu 18/20** | **统信 UOS** | **银河/中标麒麟** | **凝思 V60/V80** | +| ------------ | ---------------------------- | ----------------- | ---------------- | ---------------- | ------------ | ----------------- | ---------------- | +| X64 | ● | ● | ● | ● | ● | ● | ● | +| 树莓派 ARM64 | | | ● | | | | | +| 华为云 ARM64 | | | | ● | | | | 注： ● 表示经过官方测试验证， ○ 表示非官方测试验证。 @@ -19,15 +19,15 @@ description: "TDengine 服务端、客户端和连接器支持的平台列表" 对照矩阵如下： -| **CPU** | **X64 64bit** | | | **X86 32bit** | **ARM64** | **MIPS 龙芯** | **Alpha 申威** | **X64 海光** | -| ----------- | ------------- | --------- | --------- | ------------- | --------- | ------------- | -------------- | ------------ | -| **OS** | **Linux** | **Win64** | **Win32** | **Win32** | **Linux** | **Linux** | **Linux** | **Linux** | -| **C/C++** | ● | ● | ● | ○ | ● | ● | ● | ● | -| **JDBC** | ● | ● | ● | ○ | ● | ● | ● | ● | -| **Python** | ● | ● | ● | ○ | ● | ● | -- | ● | -| **Go** | ● | ● | ● | ○ | ● | ○ | -- | -- | -| **NodeJs** | ● | ● | ○ | ○ | ● | ○ | -- | -- | -| **C#** | ● | ● | ○ | ○ | ○ | ○ | -- | -- | -| **RESTful** | ● | ● | ● | ● | ● | ● | ● | ● | +| **CPU** | **X64 64bit** | **X64 64bit** | **ARM64** | +| ----------- | ------------- | ------------- | --------- | +| **OS** | **Linux** | **Win64** | **Linux** | +| **C/C++** | ● | ● | ● | +| **JDBC** | ● | ● | ● | +| **Python** | ● | ● | ● | +| **Go** | ● | ● | ● | +| **NodeJs** | ● | ● | ● | +| **C#** | ● | ● | ○ | +| **RESTful** | ● | ● | ● | 注：● 表示官方测试验证通过，○ 表示非官方测试验证通过，-- 表示未经验证。 diff --git a/docs/zh/21-tdinternal/01-arch.md b/docs/zh/21-tdinternal/01-arch.md index e60debc87f..a910c584d6 100644 --- a/docs/zh/21-tdinternal/01-arch.md +++ b/docs/zh/21-tdinternal/01-arch.md @@ -162,8 +162,6 @@ Vnode 会保持一个数据版本号（version），对内存数据进行持久 一个 vnode 启动时，角色（leader、follower）是不定的，数据是处于未同步状态，它需要与虚拟节点组内其他节点建立 TCP 连接，并互相交换 status，按照标准的 raft 一致性算法完成选主。 -更多的关于数据复制的流程，请见[《TDengine 3.0 数据复制模块设计》](/tdinternal/replica/)。 - ### 同步复制 对于数据一致性要求更高的场景，异步数据复制提供的最终一致性无法满足要求。因此 TDengine 提供同步复制的机制供用户选择。在创建数据库时，除指定副本数 replica 之外，用户还需要指定新的参数 strict。如果 strict 等于 1，它表示每次 leader 转发给副本时，需要等待半数以上副本达成一致后，才能通知应用，数据在 follower 已经写入成功。如果在一定的时间内，得不到半数以上副本的确认，leader vnode 将返回错误给应用。 @@ -241,15 +239,16 @@ dataDir /mnt/data6 2 0 ## 数据查询 -TDengine 提供了多种多样针对表和超级表的查询处理功能，除了常规的聚合查询之外，还提供针对时序数据的窗口查询、统计聚合等功能。TDengine 的查询处理需要客户端、vnode、mnode 节点协同完成。 +TDengine 提供了多种多样针对表和超级表的查询处理功能，除了常规的聚合查询之外，还提供针对时序数据的窗口查询、统计聚合等功能。TDengine 的查询处理需要客户端、vnode、qnode、mnode 节点协同完成，一个复杂的超级表聚合查询可能需要多个 vnode 和 qnode 节点公共分担查询和计算任务。 -### 单表查询 +### 查询基本流程 -SQL 语句的解析和校验工作在客户端完成。解析 SQL 语句并生成抽象语法树（Abstract Syntax Tree，AST），然后对其进行校验和检查。以及向管理节点（mnode）请求查询中指定表的元数据信息（table metadata）。 - -根据元数据信息中的 End Point 信息，将查询请求序列化后发送到该表所在的数据节点（dnode）。dnode 接收到查询请求后，识别出该查询请求指向的虚拟节点（vnode），将消息转发到 vnode 的查询执行队列。vnode 的查询执行线程建立基础的查询执行环境，并立即返回该查询请求，同时开始执行该查询。 - -客户端在获取查询结果的时候，dnode 的查询执行队列中的工作线程会等待 vnode 执行线程执行完成，才能将查询结果返回到请求的客户端。 +1. 客户端解析输入 SQL 语句并生成抽象语法树（Abstract Syntax Tree，AST），然后根据元数据信息对其进行校验和检查。在此期间，元数据管理模块（Catalog）会向管理节点（mnode）或 vnode 请求查询中指定库和表的元数据信息（table metadata）。 +2. 在通过校验检查后，客户端将生成分布式的查询计划并对查询计划进行优化处理。 +3. 客户端根据配置的查询策略进行任务调度处理，一个查询子任务会根据其数据亲缘关系或负载信息调度到某个 vnode 或 qnode 所属的数据节点（dnode）进行处理。 +4. dnode 接收到查询请求后，识别出该查询请求指向的虚拟节点（vnode）或查询节点（qnode），将消息转发到 vnode 或 qnode 的查询执行队列。 +5. vnode 或 qnode 的查询执行线程建立基础的查询执行环境，并立即执行该查询，在得到部分可获取查询结果后通知客户端。 +6. 客户端将启动下级查询任务或直接获取查询结果。 ### 按时间轴聚合、降采样、插值 @@ -279,12 +278,14 @@ TDengine 对每个数据采集点单独建表，但在实际应用中经常需
图 5 多表聚合查询原理图
-1. 应用将一个查询条件发往系统； -2. taosc 将超级表的名字发往 meta node（管理节点）； -3. 管理节点将超级表所拥有的 vnode 列表发回 taosc； -4. taosc 将计算的请求连同标签过滤条件发往这些 vnode 对应的多个数据节点； -5. 每个 vnode 先在内存里查找出自己节点里符合标签过滤条件的表的集合，然后扫描存储的时序数据，完成相应的聚合计算，将结果返回给 taosc； -6. taosc 将多个数据节点返回的结果做最后的聚合，将其返回给应用。 +1. 客户端从 mnode 获取库和表的元数据信息； +2. mnode 返回请求的元数据信息； +3. 客户端向超级表所属的每个 vnode 发送查询请求； +4. vnode 启动本地查询，在获得查询结果后返回查询响应； +5. 客户端向聚合节点 （在本例中为 qnode）发送查询请求； +6. qnode 向每个 vnode 节点发送数据请求消息来拉取数据； +7. vnode 返回本节点的查询计算结果； +8. qnode 完成多节点数据聚合后将最终查询结果返回给客户端； 由于 TDengine 在 vnode 内将标签数据与时序数据分离存储，通过在内存里过滤标签数据，先找到需要参与聚合操作的表的集合，将需要扫描的数据集大幅减少，大幅提升聚合计算速度。同时，由于数据分布在多个 vnode/dnode，聚合计算操作在多个 vnode 里并发进行，又进一步提升了聚合的速度。 对普通表的聚合函数以及绝大部分操作都适用于超级表，语法完全一样，细节请看 TAOS SQL。 diff --git a/docs/zh/21-tdinternal/multi_tables.webp b/docs/zh/21-tdinternal/multi_tables.webp index 8f649e34a3..481020a5f7 100644 Binary files a/docs/zh/21-tdinternal/multi_tables.webp and b/docs/zh/21-tdinternal/multi_tables.webp differ diff --git a/include/common/tmsg.h b/include/common/tmsg.h index df127af256..cc15d4ed6b 100644 --- a/include/common/tmsg.h +++ b/include/common/tmsg.h @@ -1678,9 +1678,10 @@ typedef struct { int32_t code; } STaskDropRsp; -#define STREAM_TRIGGER_AT_ONCE 1 -#define STREAM_TRIGGER_WINDOW_CLOSE 2 -#define STREAM_TRIGGER_MAX_DELAY 3 +#define STREAM_TRIGGER_AT_ONCE 1 +#define STREAM_TRIGGER_WINDOW_CLOSE 2 +#define STREAM_TRIGGER_MAX_DELAY 3 +#define STREAM_DEFAULT_IGNORE_EXPIRED 0 typedef struct { char name[TSDB_STREAM_FNAME_LEN]; diff --git a/include/libs/nodes/cmdnodes.h b/include/libs/nodes/cmdnodes.h index 21d3fa92a8..3223d4cdb8 100644 --- a/include/libs/nodes/cmdnodes.h +++ b/include/libs/nodes/cmdnodes.h @@ -359,7 +359,7 @@ typedef struct SStreamOptions { int8_t triggerType; SNode* pDelay; SNode* pWatermark; - bool ignoreExpired; + int8_t ignoreExpired; } SStreamOptions; typedef struct SCreateStreamStmt { diff --git a/include/libs/nodes/plannodes.h b/include/libs/nodes/plannodes.h index 80b23e555c..8661baceb2 100644 --- a/include/libs/nodes/plannodes.h +++ b/include/libs/nodes/plannodes.h @@ -213,6 +213,8 @@ typedef struct SWindowLogicNode { typedef struct SFillLogicNode { SLogicNode node; EFillMode mode; + SNodeList* pFillExprs; + SNodeList* pNotFillExprs; SNode* pWStartTs; SNode* pValues; // SNodeListNode STimeWindow timeRange; @@ -440,9 +442,10 @@ typedef SIntervalPhysiNode SStreamSemiIntervalPhysiNode; typedef struct SFillPhysiNode { SPhysiNode node; EFillMode mode; + SNodeList* pFillExprs; + SNodeList* pNotFillExprs; SNode* pWStartTs; // SColumnNode SNode* pValues; // SNodeListNode - SNodeList* pTargets; STimeWindow timeRange; EOrder inputTsOrder; } SFillPhysiNode; diff --git a/include/libs/nodes/querynodes.h b/include/libs/nodes/querynodes.h index 38c5055059..088da73a1a 100644 --- a/include/libs/nodes/querynodes.h +++ b/include/libs/nodes/querynodes.h @@ -53,7 +53,13 @@ typedef struct SExprNode { bool orderAlias; } SExprNode; -typedef enum EColumnType { COLUMN_TYPE_COLUMN = 1, COLUMN_TYPE_TAG, COLUMN_TYPE_TBNAME } EColumnType; +typedef enum EColumnType { + COLUMN_TYPE_COLUMN = 1, + COLUMN_TYPE_TAG, + COLUMN_TYPE_TBNAME, + COLUMN_TYPE_WINDOW_PC, + COLUMN_TYPE_GROUP_KEY +} EColumnType; typedef struct SColumnNode { SExprNode node; // QUERY_NODE_COLUMN @@ -293,6 +299,7 @@ typedef enum ESqlClause { SQL_CLAUSE_WHERE, SQL_CLAUSE_PARTITION_BY, SQL_CLAUSE_WINDOW, + SQL_CLAUSE_FILL, SQL_CLAUSE_GROUP_BY, SQL_CLAUSE_HAVING, SQL_CLAUSE_DISTINCT, diff --git a/include/libs/stream/tstream.h b/include/libs/stream/tstream.h index f51c37ed47..eac92d76ba 100644 --- a/include/libs/stream/tstream.h +++ b/include/libs/stream/tstream.h @@ -275,12 +275,8 @@ typedef struct SStreamTask { int32_t nodeId; SEpSet epSet; - // used for task source and sink, - // while task agg should have processedVer for each child int64_t recoverSnapVer; int64_t startVer; - int64_t checkpointVer; - int64_t processedVer; // children info SArray* childEpInfo; // SArray diff --git a/source/client/src/tmq.c b/source/client/src/tmq.c index ea7f03a416..436a9c2cfd 100644 --- a/source/client/src/tmq.c +++ b/source/client/src/tmq.c @@ -381,6 +381,9 @@ int32_t tmqCommitCb(void* param, SDataBuf* pBuf, int32_t code) { } #endif + taosMemoryFree(pParam->pOffset); + if (pBuf->pData) taosMemoryFree(pBuf->pData); + /*tscDebug("receive offset commit cb of %s on vgId:%d, offset is %" PRId64, pParam->pOffset->subKey, pParam->->vgId, * pOffset->version);*/ @@ -402,6 +405,8 @@ int32_t tmqCommitCb(void* param, SDataBuf* pBuf, int32_t code) { tsem_post(&pParamSet->rspSem); } + taosMemoryFree(pParamSet); + #if 0 taosArrayDestroyP(pParamSet->successfulOffsets, taosMemoryFree); taosArrayDestroyP(pParamSet->failedOffsets, taosMemoryFree); @@ -611,12 +616,12 @@ int32_t tmqCommitInner(tmq_t* tmq, const TAOS_RES* msg, int8_t automatic, int8_t } } -#if 0 if (!async) { +#if 0 taosArrayDestroyP(pParamSet->successfulOffsets, taosMemoryFree); taosArrayDestroyP(pParamSet->failedOffsets, taosMemoryFree); - } #endif + } return 0; } @@ -1216,6 +1221,7 @@ END: } else { taosMemoryFree(pParam); } + taosMemoryFree(pMsg->pData); return code; } diff --git a/source/client/test/CMakeLists.txt b/source/client/test/CMakeLists.txt index ab9d970dd6..34c377c6ea 100644 --- a/source/client/test/CMakeLists.txt +++ b/source/client/test/CMakeLists.txt @@ -8,7 +8,7 @@ AUX_SOURCE_DIRECTORY(${CMAKE_CURRENT_SOURCE_DIR} SOURCE_LIST) ADD_EXECUTABLE(clientTest clientTests.cpp) TARGET_LINK_LIBRARIES( clientTest - PUBLIC os util common transport parser catalog scheduler function gtest taos_static qcom executor + os util common transport parser catalog scheduler gtest taos_static qcom executor function ) ADD_EXECUTABLE(tmqTest tmqTest.cpp) diff --git a/source/dnode/mnode/impl/src/mndMnode.c b/source/dnode/mnode/impl/src/mndMnode.c index 13655ac21f..4f07d9e014 100644 --- a/source/dnode/mnode/impl/src/mndMnode.c +++ b/source/dnode/mnode/impl/src/mndMnode.c @@ -742,7 +742,9 @@ static int32_t mndProcessAlterMnodeReq(SRpcMsg *pReq) { return code; } else { pMgmt->errCode = 0; + taosWLockLatch(&pMgmt->lock); pMgmt->transId = -1; + taosWUnLockLatch(&pMgmt->lock); tsem_wait(&pMgmt->syncSem); mInfo("alter mnode sync result:0x%x %s", pMgmt->errCode, tstrerror(pMgmt->errCode)); terrno = pMgmt->errCode; diff --git a/source/dnode/mnode/impl/src/mndSync.c b/source/dnode/mnode/impl/src/mndSync.c index a1cd5c96d4..8e8cff853c 100644 --- a/source/dnode/mnode/impl/src/mndSync.c +++ b/source/dnode/mnode/impl/src/mndSync.c @@ -60,22 +60,24 @@ void mndSyncCommitMsg(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta cbM sdbSetApplyInfo(pMnode->pSdb, cbMeta.index, cbMeta.term, cbMeta.lastConfigIndex); } - taosRLockLatch(&pMgmt->lock); + taosWLockLatch(&pMgmt->lock); if (transId <= 0) { - taosRUnLockLatch(&pMgmt->lock); + taosWUnLockLatch(&pMgmt->lock); mError("trans:%d, invalid commit msg", transId); } else if (transId == pMgmt->transId) { - taosRUnLockLatch(&pMgmt->lock); if (pMgmt->errCode != 0) { - mError("trans:%d, failed to propose since %s", transId, tstrerror(pMgmt->errCode)); + mError("trans:%d, failed to propose since %s, post sem", transId, tstrerror(pMgmt->errCode)); + } else { + mInfo("trans:%d, is proposed and post sem", transId, tstrerror(pMgmt->errCode)); } pMgmt->transId = 0; + taosWUnLockLatch(&pMgmt->lock); tsem_post(&pMgmt->syncSem); } else { - taosRUnLockLatch(&pMgmt->lock); + taosWUnLockLatch(&pMgmt->lock); STrans *pTrans = mndAcquireTrans(pMnode, transId); if (pTrans != NULL) { - mDebug("trans:%d, execute in mnode which not leader", transId); + mInfo("trans:%d, execute in mnode which not leader", transId); mndTransExecute(pMnode, pTrans); mndReleaseTrans(pMnode, pTrans); // sdbWriteFile(pMnode->pSdb, SDB_WRITE_DELTA); @@ -122,7 +124,10 @@ void mndReConfig(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SReConfigCbMeta cbM taosWLockLatch(&pMgmt->lock); if (pMgmt->transId == -1) { if (pMgmt->errCode != 0) { - mError("trans:-1, failed to propose sync reconfig since %s", tstrerror(pMgmt->errCode)); + mError("trans:-1, failed to propose sync reconfig since %s, post sem", tstrerror(pMgmt->errCode)); + } else { + mInfo("trans:-1, sync reconfig is proposed, saved:%d code:0x%x, index:%" PRId64 " term:%" PRId64 " post sem", + pMgmt->transId, cbMeta.code, cbMeta.index, cbMeta.term); } pMgmt->transId = 0; tsem_post(&pMgmt->syncSem); @@ -174,7 +179,7 @@ void mndLeaderTransfer(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta cb static void mndBecomeFollower(struct SSyncFSM *pFsm) { SMnode *pMnode = pFsm->data; - mDebug("vgId:1, become follower"); + mDebug("vgId:1, become follower and post sem"); taosWLockLatch(&pMnode->syncMgmt.lock); if (pMnode->syncMgmt.transId != 0) { @@ -187,13 +192,6 @@ static void mndBecomeFollower(struct SSyncFSM *pFsm) { static void mndBecomeLeader(struct SSyncFSM *pFsm) { mDebug("vgId:1, become leader"); SMnode *pMnode = pFsm->data; - - taosWLockLatch(&pMnode->syncMgmt.lock); - if (pMnode->syncMgmt.transId != 0) { - pMnode->syncMgmt.transId = 0; - tsem_post(&pMnode->syncMgmt.syncSem); - } - taosWUnLockLatch(&pMnode->syncMgmt.lock); } SSyncFSM *mndSyncMakeFsm(SMnode *pMnode) { @@ -279,9 +277,16 @@ int32_t mndSyncPropose(SMnode *pMnode, SSdbRaw *pRaw, int32_t transId) { pMgmt->errCode = 0; taosWLockLatch(&pMgmt->lock); - pMgmt->transId = transId; - taosWUnLockLatch(&pMgmt->lock); - mTrace("trans:%d, will be proposed", pMgmt->transId); + if (pMgmt->transId != 0) { + mError("trans:%d, can't be proposed since trans:%s alrady waiting for confirm", transId, pMgmt->transId); + taosWUnLockLatch(&pMgmt->lock); + terrno = TSDB_CODE_APP_NOT_READY; + return -1; + } else { + pMgmt->transId = transId; + mDebug("trans:%d, will be proposed", pMgmt->transId); + taosWUnLockLatch(&pMgmt->lock); + } const bool isWeak = false; int32_t code = syncPropose(pMgmt->sync, &req, isWeak); diff --git a/source/dnode/vnode/inc/vnode.h b/source/dnode/vnode/inc/vnode.h index 18a7583f4c..19cafcbbbb 100644 --- a/source/dnode/vnode/inc/vnode.h +++ b/source/dnode/vnode/inc/vnode.h @@ -128,19 +128,19 @@ typedef struct STsdbReader STsdbReader; #define LASTROW_RETRIEVE_TYPE_ALL 0x1 #define LASTROW_RETRIEVE_TYPE_SINGLE 0x2 -int32_t tsdbSetTableId(STsdbReader *pReader, int64_t uid); -int32_t tsdbReaderOpen(SVnode *pVnode, SQueryTableDataCond *pCond, SArray *pTableList, STsdbReader **ppReader, - const char *idstr); -void tsdbReaderClose(STsdbReader *pReader); -bool tsdbNextDataBlock(STsdbReader *pReader); -void tsdbRetrieveDataBlockInfo(STsdbReader *pReader, SDataBlockInfo *pDataBlockInfo); -int32_t tsdbRetrieveDatablockSMA(STsdbReader *pReader, SColumnDataAgg ***pBlockStatis, bool *allHave); -SArray *tsdbRetrieveDataBlock(STsdbReader *pTsdbReadHandle, SArray *pColumnIdList); -int32_t tsdbReaderReset(STsdbReader *pReader, SQueryTableDataCond *pCond); -int32_t tsdbGetFileBlocksDistInfo(STsdbReader *pReader, STableBlockDistInfo *pTableBlockInfo); -int64_t tsdbGetNumOfRowsInMemTable(STsdbReader *pHandle); -void *tsdbGetIdx(SMeta *pMeta); -void *tsdbGetIvtIdx(SMeta *pMeta); +int32_t tsdbSetTableId(STsdbReader *pReader, int64_t uid); +int32_t tsdbReaderOpen(SVnode *pVnode, SQueryTableDataCond *pCond, SArray *pTableList, STsdbReader **ppReader, + const char *idstr); +void tsdbReaderClose(STsdbReader *pReader); +bool tsdbNextDataBlock(STsdbReader *pReader); +void tsdbRetrieveDataBlockInfo(STsdbReader *pReader, SDataBlockInfo *pDataBlockInfo); +int32_t tsdbRetrieveDatablockSMA(STsdbReader *pReader, SColumnDataAgg ***pBlockStatis, bool *allHave); +SArray *tsdbRetrieveDataBlock(STsdbReader *pTsdbReadHandle, SArray *pColumnIdList); +int32_t tsdbReaderReset(STsdbReader *pReader, SQueryTableDataCond *pCond); +int32_t tsdbGetFileBlocksDistInfo(STsdbReader *pReader, STableBlockDistInfo *pTableBlockInfo); +int64_t tsdbGetNumOfRowsInMemTable(STsdbReader *pHandle); +void *tsdbGetIdx(SMeta *pMeta); +void *tsdbGetIvtIdx(SMeta *pMeta); uint64_t getReaderMaxVersion(STsdbReader *pReader); int32_t tsdbLastRowReaderOpen(void *pVnode, int32_t type, SArray *pTableIdList, int32_t numOfCols, void **pReader); diff --git a/source/dnode/vnode/src/inc/vnodeInt.h b/source/dnode/vnode/src/inc/vnodeInt.h index 700c6cf8a3..35c26eac44 100644 --- a/source/dnode/vnode/src/inc/vnodeInt.h +++ b/source/dnode/vnode/src/inc/vnodeInt.h @@ -144,6 +144,7 @@ int32_t tsdbDeleteTableData(STsdb* pTsdb, int64_t version, tb_uid_t suid, tb STsdbReader tsdbQueryCacheLastT(STsdb* tsdb, SQueryTableDataCond* pCond, STableListInfo* tableList, uint64_t qId, void* pMemRef); int32_t tsdbSetKeepCfg(STsdb* pTsdb, STsdbCfg* pCfg); +int32_t tsdbGetStbIdList(SMeta* pMeta, int64_t suid, SArray* list); // tq int tqInit(); @@ -169,10 +170,9 @@ int32_t tqProcessTaskDispatchRsp(STQ* pTq, SRpcMsg* pMsg); int32_t tqProcessTaskRecoverRsp(STQ* pTq, SRpcMsg* pMsg); int32_t tqProcessTaskRetrieveReq(STQ* pTq, SRpcMsg* pMsg); int32_t tqProcessTaskRetrieveRsp(STQ* pTq, SRpcMsg* pMsg); -int32_t tsdbGetStbIdList(SMeta* pMeta, int64_t suid, SArray* list); -SSubmitReq* tdBlockToSubmit(SVnode* pVnode, const SArray* pBlocks, const STSchema* pSchema, bool createTb, int64_t suid, - const char* stbFullName, int32_t vgId, SBatchDeleteReq* pDeleteReq); +SSubmitReq* tqBlockToSubmit(SVnode* pVnode, const SArray* pBlocks, const STSchema* pSchema, bool createTb, int64_t suid, + const char* stbFullName, SBatchDeleteReq* pDeleteReq); // sma int32_t smaInit(); @@ -308,7 +308,7 @@ struct SVnode { SSink* pSink; tsem_t canCommit; int64_t sync; - SRWLatch lock; + TdThreadMutex lock; bool blocked; bool restored; tsem_t syncSem; diff --git a/source/dnode/vnode/src/meta/metaTable.c b/source/dnode/vnode/src/meta/metaTable.c index 3e7fd9df2b..e56b8ad939 100644 --- a/source/dnode/vnode/src/meta/metaTable.c +++ b/source/dnode/vnode/src/meta/metaTable.c @@ -298,14 +298,14 @@ int metaAlterSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) { tdbTbcClose(pUidIdxc); terrno = TSDB_CODE_TDB_STB_NOT_EXIST; - // ASSERT(0); return -1; } ret = tdbTbcGet(pUidIdxc, NULL, NULL, &pData, &nData); if (ret < 0) { + tdbTbcClose(pUidIdxc); + terrno = TSDB_CODE_TDB_STB_NOT_EXIST; - // ASSERT(0); return -1; } diff --git a/source/dnode/vnode/src/sma/smaTimeRange.c b/source/dnode/vnode/src/sma/smaTimeRange.c index f46d9dc29c..b09d7e3c23 100644 --- a/source/dnode/vnode/src/sma/smaTimeRange.c +++ b/source/dnode/vnode/src/sma/smaTimeRange.c @@ -201,9 +201,8 @@ int32_t tdProcessTSmaInsertImpl(SSma *pSma, int64_t indexUid, const char *msg) { } SBatchDeleteReq deleteReq; - SSubmitReq *pSubmitReq = - tdBlockToSubmit(pSma->pVnode, (const SArray *)msg, pTsmaStat->pTSchema, true, pTsmaStat->pTSma->dstTbUid, - pTsmaStat->pTSma->dstTbName, pTsmaStat->pTSma->dstVgId, &deleteReq); + SSubmitReq *pSubmitReq = tqBlockToSubmit(pSma->pVnode, (const SArray *)msg, pTsmaStat->pTSchema, true, + pTsmaStat->pTSma->dstTbUid, pTsmaStat->pTSma->dstTbName, &deleteReq); if (!pSubmitReq) { smaError("vgId:%d, failed to gen submit blk while tsma insert for smaIndex %" PRIi64 " since %s", SMA_VID(pSma), diff --git a/source/dnode/vnode/src/tq/tqPush.c b/source/dnode/vnode/src/tq/tqPush.c index ae3fef9b4b..ed7fa80c47 100644 --- a/source/dnode/vnode/src/tq/tqPush.c +++ b/source/dnode/vnode/src/tq/tqPush.c @@ -14,6 +14,7 @@ */ #include "tq.h" +#include "vnd.h" #if 0 void tqTmrRspFunc(void* param, void* tmrId) { @@ -212,9 +213,7 @@ int32_t tqPushMsgNew(STQ* pTq, void* msg, int32_t msgLen, tmsg_t msgType, int64_ #endif int tqPushMsg(STQ* pTq, void* msg, int32_t msgLen, tmsg_t msgType, int64_t ver) { - walApplyVer(pTq->pVnode->pWal, ver); - - if (msgType == TDMT_VND_SUBMIT) { + if (vnodeIsRoleLeader(pTq->pVnode) && msgType == TDMT_VND_SUBMIT) { if (taosHashGetSize(pTq->pStreamMeta->pTasks) == 0) return 0; void* data = taosMemoryMalloc(msgLen); diff --git a/source/dnode/vnode/src/tq/tqSink.c b/source/dnode/vnode/src/tq/tqSink.c index 42fb5c329d..8a249eb105 100644 --- a/source/dnode/vnode/src/tq/tqSink.c +++ b/source/dnode/vnode/src/tq/tqSink.c @@ -25,8 +25,7 @@ int32_t tdBuildDeleteReq(SVnode* pVnode, const char* stbFullName, const SSDataBl SColumnInfoData* pGidCol = taosArrayGet(pDataBlock->pDataBlock, GROUPID_COLUMN_INDEX); for (int32_t row = 0; row < totRow; row++) { int64_t ts = *(int64_t*)colDataGetData(pTsCol, row); - /*int64_t groupId = *(int64_t*)colDataGetData(pGidCol, row);*/ - int64_t groupId = 0; + int64_t groupId = *(int64_t*)colDataGetData(pGidCol, row); char* name = buildCtbNameByGroupId(stbFullName, groupId); tqDebug("stream delete msg: groupId :%ld, name: %s", groupId, name); SMetaReader mr = {0}; @@ -49,8 +48,8 @@ int32_t tdBuildDeleteReq(SVnode* pVnode, const char* stbFullName, const SSDataBl return 0; } -SSubmitReq* tdBlockToSubmit(SVnode* pVnode, const SArray* pBlocks, const STSchema* pTSchema, bool createTb, - int64_t suid, const char* stbFullName, int32_t vgId, SBatchDeleteReq* pDeleteReq) { +SSubmitReq* tqBlockToSubmit(SVnode* pVnode, const SArray* pBlocks, const STSchema* pTSchema, bool createTb, + int64_t suid, const char* stbFullName, SBatchDeleteReq* pDeleteReq) { SSubmitReq* ret = NULL; SArray* schemaReqs = NULL; SArray* schemaReqSz = NULL; @@ -153,7 +152,7 @@ SSubmitReq* tdBlockToSubmit(SVnode* pVnode, const SArray* pBlocks, const STSchem // assign data // TODO ret = rpcMallocCont(cap); - ret->header.vgId = vgId; + ret->header.vgId = pVnode->config.vgId; ret->length = sizeof(SSubmitReq); ret->numOfBlocks = htonl(sz); @@ -234,8 +233,8 @@ void tqTableSink(SStreamTask* pTask, void* vnode, int64_t ver, void* data) { ASSERT(pTask->tbSink.pTSchema); deleteReq.deleteReqs = taosArrayInit(0, sizeof(SSingleDeleteReq)); - SSubmitReq* pReq = tdBlockToSubmit(pVnode, pRes, pTask->tbSink.pTSchema, true, pTask->tbSink.stbUid, - pTask->tbSink.stbFullName, pVnode->config.vgId, &deleteReq); + SSubmitReq* pReq = tqBlockToSubmit(pVnode, pRes, pTask->tbSink.pTSchema, true, pTask->tbSink.stbUid, + pTask->tbSink.stbFullName, &deleteReq); tqDebug("vgId:%d, task %d convert blocks over, put into write-queue", TD_VID(pVnode), pTask->taskId); diff --git a/source/dnode/vnode/src/vnd/vnodeOpen.c b/source/dnode/vnode/src/vnd/vnodeOpen.c index 4ee5c4760c..a4fd984fb7 100644 --- a/source/dnode/vnode/src/vnd/vnodeOpen.c +++ b/source/dnode/vnode/src/vnd/vnodeOpen.c @@ -85,7 +85,7 @@ SVnode *vnodeOpen(const char *path, STfs *pTfs, SMsgCb msgCb) { pVnode->state.commitTerm = info.state.commitTerm; pVnode->pTfs = pTfs; pVnode->msgCb = msgCb; - taosInitRWLatch(&pVnode->lock); + taosThreadMutexInit(&pVnode->lock, NULL); pVnode->blocked = false; tsem_init(&pVnode->syncSem, 0, 0); @@ -200,6 +200,7 @@ void vnodeClose(SVnode *pVnode) { tsem_destroy(&pVnode->syncSem); taosThreadCondDestroy(&pVnode->poolNotEmpty); taosThreadMutexDestroy(&pVnode->mutex); + taosThreadMutexDestroy(&pVnode->lock); taosMemoryFree(pVnode); } } diff --git a/source/dnode/vnode/src/vnd/vnodeSvr.c b/source/dnode/vnode/src/vnd/vnodeSvr.c index e17d4a0b6d..43c9b4c09f 100644 --- a/source/dnode/vnode/src/vnd/vnodeSvr.c +++ b/source/dnode/vnode/src/vnd/vnodeSvr.c @@ -247,6 +247,8 @@ int32_t vnodeProcessWriteMsg(SVnode *pVnode, SRpcMsg *pMsg, int64_t version, SRp vTrace("vgId:%d, process %s request success, index:%" PRId64, TD_VID(pVnode), TMSG_INFO(pMsg->msgType), version); + walApplyVer(pVnode->pWal, version); + if (tqPushMsg(pVnode->pTq, pMsg->pCont, pMsg->contLen, pMsg->msgType, version) < 0) { vError("vgId:%d, failed to push msg to TQ since %s", TD_VID(pVnode), tstrerror(terrno)); return -1; diff --git a/source/dnode/vnode/src/vnd/vnodeSync.c b/source/dnode/vnode/src/vnd/vnodeSync.c index 2779ab724f..65d4e9aaf1 100644 --- a/source/dnode/vnode/src/vnd/vnodeSync.c +++ b/source/dnode/vnode/src/vnd/vnodeSync.c @@ -28,14 +28,14 @@ static inline bool vnodeIsMsgWeak(tmsg_t type) { return false; } static inline void vnodeWaitBlockMsg(SVnode *pVnode, const SRpcMsg *pMsg) { if (vnodeIsMsgBlock(pMsg->msgType)) { const STraceId *trace = &pMsg->info.traceId; - taosWLockLatch(&pVnode->lock); + taosThreadMutexLock(&pVnode->lock); if (!pVnode->blocked) { vGTrace("vgId:%d, msg:%p wait block, type:%s", pVnode->config.vgId, pMsg, TMSG_INFO(pMsg->msgType)); pVnode->blocked = true; - taosWUnLockLatch(&pVnode->lock); + taosThreadMutexUnlock(&pVnode->lock); tsem_wait(&pVnode->syncSem); } else { - taosWUnLockLatch(&pVnode->lock); + taosThreadMutexUnlock(&pVnode->lock); } } } @@ -43,13 +43,13 @@ static inline void vnodeWaitBlockMsg(SVnode *pVnode, const SRpcMsg *pMsg) { static inline void vnodePostBlockMsg(SVnode *pVnode, const SRpcMsg *pMsg) { if (vnodeIsMsgBlock(pMsg->msgType)) { const STraceId *trace = &pMsg->info.traceId; - taosWLockLatch(&pVnode->lock); + taosThreadMutexLock(&pVnode->lock); if (pVnode->blocked) { vGTrace("vgId:%d, msg:%p post block, type:%s", pVnode->config.vgId, pMsg, TMSG_INFO(pMsg->msgType)); pVnode->blocked = false; tsem_post(&pVnode->syncSem); } - taosWUnLockLatch(&pVnode->lock); + taosThreadMutexUnlock(&pVnode->lock); } } @@ -685,17 +685,25 @@ static void vnodeBecomeFollower(struct SSyncFSM *pFsm) { vDebug("vgId:%d, become follower", pVnode->config.vgId); // clear old leader resource - taosWLockLatch(&pVnode->lock); + taosThreadMutexLock(&pVnode->lock); if (pVnode->blocked) { pVnode->blocked = false; + vDebug("vgId:%d, become follower and post block", pVnode->config.vgId); tsem_post(&pVnode->syncSem); } - taosWUnLockLatch(&pVnode->lock); + taosThreadMutexUnlock(&pVnode->lock); } static void vnodeBecomeLeader(struct SSyncFSM *pFsm) { SVnode *pVnode = pFsm->data; vDebug("vgId:%d, become leader", pVnode->config.vgId); + + // taosThreadMutexLock(&pVnode->lock); + // if (pVnode->blocked) { + // pVnode->blocked = false; + // tsem_post(&pVnode->syncSem); + // } + // taosThreadMutexUnlock(&pVnode->lock); } static SSyncFSM *vnodeSyncMakeFsm(SVnode *pVnode) { @@ -756,6 +764,8 @@ void vnodeSyncStart(SVnode *pVnode) { void vnodeSyncClose(SVnode *pVnode) { syncStop(pVnode->sync); } +bool vnodeIsRoleLeader(SVnode *pVnode) { return syncGetMyRole(pVnode->sync) == TAOS_SYNC_STATE_LEADER; } + bool vnodeIsLeader(SVnode *pVnode) { if (!syncIsReady(pVnode->sync)) { vDebug("vgId:%d, vnode not ready, state:%s, restore:%d", pVnode->config.vgId, syncGetMyRoleStr(pVnode->sync), diff --git a/source/libs/catalog/src/ctgAsync.c b/source/libs/catalog/src/ctgAsync.c index f279d87815..64ca85edf4 100644 --- a/source/libs/catalog/src/ctgAsync.c +++ b/source/libs/catalog/src/ctgAsync.c @@ -1105,6 +1105,7 @@ int32_t ctgHandleGetTbMetasRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBu SName* pName = ctgGetFetchName(ctx->pNames, pFetch); int32_t flag = pFetch->flag; int32_t* vgId = &pFetch->vgId; + bool taskDone = false; CTG_ERR_JRET(ctgProcessRspMsg(pMsgCtx->out, reqType, pMsg->pData, pMsg->len, rspCode, pMsgCtx->target)); @@ -1250,6 +1251,7 @@ int32_t ctgHandleGetTbMetasRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBu pOut->tbMeta = NULL; if (0 == atomic_sub_fetch_32(&ctx->fetchNum, 1)) { TSWAP(pTask->res, ctx->pResList); + taskDone = true; } _return: @@ -1264,10 +1266,11 @@ _return: pRes->pRes = NULL; if (0 == atomic_sub_fetch_32(&ctx->fetchNum, 1)) { TSWAP(pTask->res, ctx->pResList); + taskDone = true; } } - if (pTask->res) { + if (pTask->res && taskDone) { ctgHandleTaskEnd(pTask, code); } @@ -1354,6 +1357,7 @@ int32_t ctgHandleGetTbHashsRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBu SCatalog* pCtg = pTask->pJob->pCtg; SCtgMsgCtx* pMsgCtx = CTG_GET_TASK_MSGCTX(pTask, tReq->msgIdx); SCtgFetch* pFetch = taosArrayGet(ctx->pFetchs, tReq->msgIdx); + bool taskDone = false; CTG_ERR_JRET(ctgProcessRspMsg(pMsgCtx->out, reqType, pMsg->pData, pMsg->len, rspCode, pMsgCtx->target)); @@ -1377,6 +1381,7 @@ int32_t ctgHandleGetTbHashsRsp(SCtgTaskReq* tReq, int32_t reqType, const SDataBu if (0 == atomic_sub_fetch_32(&ctx->fetchNum, 1)) { TSWAP(pTask->res, ctx->pResList); + taskDone = true; } _return: @@ -1392,10 +1397,11 @@ _return: if (0 == atomic_sub_fetch_32(&ctx->fetchNum, 1)) { TSWAP(pTask->res, ctx->pResList); + taskDone = true; } } - if (pTask->res) { + if (pTask->res && taskDone) { ctgHandleTaskEnd(pTask, code); } diff --git a/source/libs/executor/inc/executorimpl.h b/source/libs/executor/inc/executorimpl.h index 410bab341c..311d82c8a2 100644 --- a/source/libs/executor/inc/executorimpl.h +++ b/source/libs/executor/inc/executorimpl.h @@ -619,15 +619,20 @@ typedef struct SIndefOperatorInfo { typedef struct SFillOperatorInfo { struct SFillInfo* pFillInfo; SSDataBlock* pRes; + SSDataBlock* pFinalRes; int64_t totalInputRows; void** p; SSDataBlock* existNewGroupBlock; - bool multigroupResult; STimeWindow win; SNode* pCondition; SArray* pColMatchColInfo; int32_t primaryTsCol; + int32_t primarySrcSlotId; uint64_t curGroupId; // current handled group id + SExprInfo* pExprInfo; + int32_t numOfExpr; + SExprInfo* pNotFillExprInfo; + int32_t numOfNotFillExpr; } SFillOperatorInfo; typedef struct SGroupbyOperatorInfo { diff --git a/source/libs/executor/inc/tfill.h b/source/libs/executor/inc/tfill.h index 1e95f4a04f..63abfc019d 100644 --- a/source/libs/executor/inc/tfill.h +++ b/source/libs/executor/inc/tfill.h @@ -28,8 +28,7 @@ struct SSDataBlock; typedef struct SFillColInfo { SExprInfo *pExpr; - int16_t flag; // column flag: TAG COLUMN|NORMAL COLUMN - int16_t tagIndex; // index of current tag in SFillTagColInfo array list + bool notFillCol; // denote if this column needs fill operation SVariant fillVal; } SFillColInfo; @@ -46,25 +45,27 @@ typedef struct { char* tagVal; } SFillTagColInfo; +typedef struct { + int64_t key; + SArray* pRowVal; +} SRowVal; + typedef struct SFillInfo { TSKEY start; // start timestamp TSKEY end; // endKey for fill TSKEY currentKey; // current active timestamp, the value may be changed during the fill procedure. int32_t tsSlotId; // primary time stamp slot id + int32_t srcTsSlotId; // timestamp column id in the source data block. int32_t order; // order [TSDB_ORDER_ASC|TSDB_ORDER_DESC] int32_t type; // fill type int32_t numOfRows; // number of rows in the input data block int32_t index; // active row index int32_t numOfTotal; // number of filled rows in one round int32_t numOfCurrent; // number of filled rows in current results - - int32_t numOfTags; // number of tags int32_t numOfCols; // number of columns, including the tags columns - int32_t rowSize; // size of each row SInterval interval; - - SArray *prev; - SArray *next; + SRowVal prev; + SRowVal next; SSDataBlock *pSrcBlock; int32_t alloc; // data buffer size in rows @@ -79,10 +80,10 @@ int64_t getNumOfResultsAfterFillGap(SFillInfo* pFillInfo, int64_t ekey, int32_t void taosFillSetStartInfo(struct SFillInfo* pFillInfo, int32_t numOfRows, TSKEY endKey); void taosResetFillInfo(struct SFillInfo* pFillInfo, TSKEY startTimestamp); void taosFillSetInputDataBlock(struct SFillInfo* pFillInfo, const struct SSDataBlock* pInput); -struct SFillColInfo* createFillColInfo(SExprInfo* pExpr, int32_t numOfOutput, const struct SNodeListNode* val); +struct SFillColInfo* createFillColInfo(SExprInfo* pExpr, int32_t numOfFillExpr, SExprInfo* pNotFillExpr, int32_t numOfNotFillCols, const struct SNodeListNode* val); bool taosFillHasMoreResults(struct SFillInfo* pFillInfo); -SFillInfo* taosCreateFillInfo(TSKEY skey, int32_t numOfTags, int32_t capacity, int32_t numOfCols, +SFillInfo* taosCreateFillInfo(TSKEY skey, int32_t numOfFillCols, int32_t numOfNotFillCols, int32_t capacity, SInterval* pInterval, int32_t fillType, struct SFillColInfo* pCol, int32_t slotId, int32_t order, const char* id); diff --git a/source/libs/executor/src/executorimpl.c b/source/libs/executor/src/executorimpl.c index 12fed5a118..86b9036454 100644 --- a/source/libs/executor/src/executorimpl.c +++ b/source/libs/executor/src/executorimpl.c @@ -716,7 +716,7 @@ int32_t projectApplyFunctions(SExprInfo* pExpr, SSDataBlock* pResult, SSDataBloc taosArrayDestroy(pBlockList); } } else { - ASSERT(0); + return TSDB_CODE_OPS_NOT_SUPPORT; } } @@ -3216,36 +3216,71 @@ int32_t handleLimitOffset(SOperatorInfo* pOperator, SLimitInfo* pLimitInfo, SSDa } } -static void doHandleRemainBlockForNewGroupImpl(SFillOperatorInfo* pInfo, SResultInfo* pResultInfo, +static void doApplyScalarCalculation(SOperatorInfo* pOperator, SSDataBlock* pBlock, int32_t order, int32_t scanFlag); +static void doHandleRemainBlockForNewGroupImpl(SOperatorInfo *pOperator, SFillOperatorInfo* pInfo, SResultInfo* pResultInfo, SExecTaskInfo* pTaskInfo) { pInfo->totalInputRows = pInfo->existNewGroupBlock->info.rows; + SSDataBlock* pResBlock = pInfo->pFinalRes; + + int32_t order = TSDB_ORDER_ASC; + int32_t scanFlag = MAIN_SCAN; + getTableScanInfo(pOperator, &order, &scanFlag); int64_t ekey = Q_STATUS_EQUAL(pTaskInfo->status, TASK_COMPLETED) ? pInfo->win.ekey : pInfo->existNewGroupBlock->info.window.ekey; taosResetFillInfo(pInfo->pFillInfo, getFillInfoStart(pInfo->pFillInfo)); - taosFillSetStartInfo(pInfo->pFillInfo, pInfo->existNewGroupBlock->info.rows, ekey); - taosFillSetInputDataBlock(pInfo->pFillInfo, pInfo->existNewGroupBlock); + doApplyScalarCalculation(pOperator, pInfo->existNewGroupBlock, order, scanFlag); - int32_t numOfResultRows = pResultInfo->capacity - pInfo->pRes->info.rows; - taosFillResultDataBlock(pInfo->pFillInfo, pInfo->pRes, numOfResultRows); + taosFillSetStartInfo(pInfo->pFillInfo, pInfo->pRes->info.rows, ekey); + taosFillSetInputDataBlock(pInfo->pFillInfo, pInfo->pRes); + + int32_t numOfResultRows = pResultInfo->capacity - pResBlock->info.rows; + taosFillResultDataBlock(pInfo->pFillInfo, pResBlock, numOfResultRows); pInfo->curGroupId = pInfo->existNewGroupBlock->info.groupId; pInfo->existNewGroupBlock = NULL; } -static void doHandleRemainBlockFromNewGroup(SFillOperatorInfo* pInfo, SResultInfo* pResultInfo, +static void doHandleRemainBlockFromNewGroup(SOperatorInfo* pOperator, SFillOperatorInfo* pInfo, SResultInfo* pResultInfo, SExecTaskInfo* pTaskInfo) { if (taosFillHasMoreResults(pInfo->pFillInfo)) { - int32_t numOfResultRows = pResultInfo->capacity - pInfo->pRes->info.rows; - taosFillResultDataBlock(pInfo->pFillInfo, pInfo->pRes, numOfResultRows); + int32_t numOfResultRows = pResultInfo->capacity - pInfo->pFinalRes->info.rows; + taosFillResultDataBlock(pInfo->pFillInfo, pInfo->pFinalRes, numOfResultRows); pInfo->pRes->info.groupId = pInfo->curGroupId; return; } // handle the cached new group data block if (pInfo->existNewGroupBlock) { - doHandleRemainBlockForNewGroupImpl(pInfo, pResultInfo, pTaskInfo); + doHandleRemainBlockForNewGroupImpl(pOperator, pInfo, pResultInfo, pTaskInfo); + } +} + +static void doApplyScalarCalculation(SOperatorInfo* pOperator, SSDataBlock* pBlock, int32_t order, int32_t scanFlag) { + SFillOperatorInfo* pInfo = pOperator->info; + SExprSupp* pSup = &pOperator->exprSupp; + SSDataBlock* pResBlock = pInfo->pFinalRes; + + setInputDataBlock(pOperator, pSup->pCtx, pBlock, order, scanFlag, false); + projectApplyFunctions(pSup->pExprInfo, pInfo->pRes, pBlock, pSup->pCtx, pSup->numOfExprs, NULL); + pInfo->pRes->info.groupId = pBlock->info.groupId; + + SColumnInfoData* pDst = taosArrayGet(pInfo->pRes->pDataBlock, pInfo->primaryTsCol); + SColumnInfoData* pSrc = taosArrayGet(pBlock->pDataBlock, pInfo->primarySrcSlotId); + colDataAssign(pDst, pSrc, pInfo->pRes->info.rows, &pResBlock->info); + + for(int32_t i = 0; i < pInfo->numOfNotFillExpr; ++i) { + SFillColInfo* pCol = &pInfo->pFillInfo->pFillCol[i + pInfo->numOfExpr]; + ASSERT(pCol->notFillCol); + + SExprInfo* pExpr = pCol->pExpr; + int32_t srcSlotId = pExpr->base.pParam[0].pCol->slotId; + int32_t dstSlotId = pExpr->base.resSchema.slotId; + + SColumnInfoData* pDst1 = taosArrayGet(pInfo->pRes->pDataBlock, dstSlotId); + SColumnInfoData* pSrc1 = taosArrayGet(pBlock->pDataBlock, srcSlotId); + colDataAssign(pDst1, pSrc1, pInfo->pRes->info.rows, &pResBlock->info); } } @@ -3254,11 +3289,16 @@ static SSDataBlock* doFillImpl(SOperatorInfo* pOperator) { SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; SResultInfo* pResultInfo = &pOperator->resultInfo; - SSDataBlock* pResBlock = pInfo->pRes; + SSDataBlock* pResBlock = pInfo->pFinalRes; blockDataCleanup(pResBlock); + blockDataCleanup(pInfo->pRes); - doHandleRemainBlockFromNewGroup(pInfo, pResultInfo, pTaskInfo); + int32_t order = TSDB_ORDER_ASC; + int32_t scanFlag = MAIN_SCAN; + getTableScanInfo(pOperator, &order, &scanFlag); + + doHandleRemainBlockFromNewGroup(pOperator, pInfo, pResultInfo, pTaskInfo); if (pResBlock->info.rows > 0) { pResBlock->info.groupId = pInfo->curGroupId; return pResBlock; @@ -3269,21 +3309,21 @@ static SSDataBlock* doFillImpl(SOperatorInfo* pOperator) { SSDataBlock* pBlock = pDownstream->fpSet.getNextFn(pDownstream); if (pBlock == NULL) { if (pInfo->totalInputRows == 0) { - pOperator->status = OP_EXEC_DONE; + doSetOperatorCompleted(pOperator); return NULL; } taosFillSetStartInfo(pInfo->pFillInfo, 0, pInfo->win.ekey); } else { blockDataUpdateTsWindow(pBlock, pInfo->primaryTsCol); + doApplyScalarCalculation(pOperator, pBlock, order, scanFlag); - if (pInfo->curGroupId == 0 || pInfo->curGroupId == pBlock->info.groupId) { - pInfo->curGroupId = pBlock->info.groupId; // the first data block + if (pInfo->curGroupId == 0 || pInfo->curGroupId == pInfo->pRes->info.groupId) { + pInfo->curGroupId = pInfo->pRes->info.groupId; // the first data block + pInfo->totalInputRows += pInfo->pRes->info.rows; - pInfo->totalInputRows += pBlock->info.rows; - - taosFillSetStartInfo(pInfo->pFillInfo, pBlock->info.rows, pBlock->info.window.ekey); - taosFillSetInputDataBlock(pInfo->pFillInfo, pBlock); + taosFillSetStartInfo(pInfo->pFillInfo, pInfo->pRes->info.rows, pBlock->info.window.ekey); + taosFillSetInputDataBlock(pInfo->pFillInfo, pInfo->pRes); } else if (pInfo->curGroupId != pBlock->info.groupId) { // the new group data block pInfo->existNewGroupBlock = pBlock; @@ -3293,8 +3333,6 @@ static SSDataBlock* doFillImpl(SOperatorInfo* pOperator) { } } - blockDataEnsureCapacity(pResBlock, pOperator->resultInfo.capacity); - int32_t numOfResultRows = pOperator->resultInfo.capacity - pResBlock->info.rows; taosFillResultDataBlock(pInfo->pFillInfo, pResBlock, numOfResultRows); @@ -3307,14 +3345,18 @@ static SSDataBlock* doFillImpl(SOperatorInfo* pOperator) { return pResBlock; } - doHandleRemainBlockFromNewGroup(pInfo, pResultInfo, pTaskInfo); + doHandleRemainBlockFromNewGroup(pOperator, pInfo, pResultInfo, pTaskInfo); if (pResBlock->info.rows >= pOperator->resultInfo.threshold || pBlock == NULL) { pResBlock->info.groupId = pInfo->curGroupId; return pResBlock; } } else if (pInfo->existNewGroupBlock) { // try next group assert(pBlock != NULL); - doHandleRemainBlockForNewGroupImpl(pInfo, pResultInfo, pTaskInfo); + + blockDataCleanup(pResBlock); + blockDataCleanup(pInfo->pRes); + + doHandleRemainBlockForNewGroupImpl(pOperator, pInfo, pResultInfo, pTaskInfo); if (pResBlock->info.rows > pResultInfo->threshold) { pResBlock->info.groupId = pInfo->curGroupId; return pResBlock; @@ -3605,6 +3647,13 @@ void destroyFillOperatorInfo(void* param, int32_t numOfOutput) { SFillOperatorInfo* pInfo = (SFillOperatorInfo*)param; pInfo->pFillInfo = taosDestroyFillInfo(pInfo->pFillInfo); pInfo->pRes = blockDataDestroy(pInfo->pRes); + pInfo->pFinalRes = blockDataDestroy(pInfo->pFinalRes); + + if (pInfo->pNotFillExprInfo != NULL) { + destroyExprInfo(pInfo->pNotFillExprInfo, pInfo->numOfNotFillExpr); + taosMemoryFree(pInfo->pNotFillExprInfo); + } + taosMemoryFreeClear(pInfo->p); taosArrayDestroy(pInfo->pColMatchColInfo); taosMemoryFreeClear(param); @@ -3637,16 +3686,16 @@ void doDestroyExchangeOperatorInfo(void* param) { taosMemoryFreeClear(param); } -static int32_t initFillInfo(SFillOperatorInfo* pInfo, SExprInfo* pExpr, int32_t numOfCols, SNodeListNode* pValNode, - STimeWindow win, int32_t capacity, const char* id, SInterval* pInterval, int32_t fillType, - int32_t order) { - SFillColInfo* pColInfo = createFillColInfo(pExpr, numOfCols, pValNode); +static int32_t initFillInfo(SFillOperatorInfo* pInfo, SExprInfo* pExpr, int32_t numOfCols, SExprInfo* pNotFillExpr, + int32_t numOfNotFillCols, SNodeListNode* pValNode, STimeWindow win, int32_t capacity, + const char* id, SInterval* pInterval, int32_t fillType, int32_t order) { + SFillColInfo* pColInfo = createFillColInfo(pExpr, numOfCols, pNotFillExpr, numOfNotFillCols, pValNode); STimeWindow w = getAlignQueryTimeWindow(pInterval, pInterval->precision, win.skey); w = getFirstQualifiedTimeWindow(win.skey, &w, pInterval, TSDB_ORDER_ASC); pInfo->pFillInfo = - taosCreateFillInfo(w.skey, 0, capacity, numOfCols, pInterval, fillType, pColInfo, pInfo->primaryTsCol, order, id); + taosCreateFillInfo(w.skey, numOfCols, numOfNotFillCols, capacity, pInterval, fillType, pColInfo, pInfo->primaryTsCol, order, id); pInfo->win = win; pInfo->p = taosMemoryCalloc(numOfCols, POINTER_BYTES); @@ -3668,9 +3717,10 @@ SOperatorInfo* createFillOperatorInfo(SOperatorInfo* downstream, SFillPhysiNode* goto _error; } - int32_t num = 0; SSDataBlock* pResBlock = createResDataBlock(pPhyFillNode->node.pOutputDataBlockDesc); - SExprInfo* pExprInfo = createExprInfo(pPhyFillNode->pTargets, NULL, &num); + SExprInfo* pExprInfo = createExprInfo(pPhyFillNode->pFillExprs, NULL, &pInfo->numOfExpr); + pInfo->pNotFillExprInfo = createExprInfo(pPhyFillNode->pNotFillExprs, NULL, &pInfo->numOfNotFillExpr); + SInterval* pInterval = QUERY_NODE_PHYSICAL_PLAN_MERGE_ALIGNED_INTERVAL == downstream->operatorType ? &((SMergeAlignedIntervalAggOperatorInfo*)downstream->info)->intervalAggOperatorInfo->interval @@ -3681,19 +3731,27 @@ SOperatorInfo* createFillOperatorInfo(SOperatorInfo* downstream, SFillPhysiNode* SResultInfo* pResultInfo = &pOperator->resultInfo; initResultSizeInfo(&pOperator->resultInfo, 4096); - pInfo->primaryTsCol = ((SColumnNode*)pPhyFillNode->pWStartTs)->slotId; + blockDataEnsureCapacity(pResBlock, pOperator->resultInfo.capacity); + initExprSupp(&pOperator->exprSupp, pExprInfo, pInfo->numOfExpr); + + pInfo->primaryTsCol = ((STargetNode*)pPhyFillNode->pWStartTs)->slotId; + pInfo->primarySrcSlotId = ((SColumnNode*)((STargetNode*)pPhyFillNode->pWStartTs)->pExpr)->slotId; int32_t numOfOutputCols = 0; - SArray* pColMatchColInfo = extractColMatchInfo(pPhyFillNode->pTargets, pPhyFillNode->node.pOutputDataBlockDesc, + SArray* pColMatchColInfo = extractColMatchInfo(pPhyFillNode->pFillExprs, pPhyFillNode->node.pOutputDataBlockDesc, &numOfOutputCols, COL_MATCH_FROM_SLOT_ID); - int32_t code = initFillInfo(pInfo, pExprInfo, num, (SNodeListNode*)pPhyFillNode->pValues, pPhyFillNode->timeRange, - pResultInfo->capacity, pTaskInfo->id.str, pInterval, type, order); + int32_t code = + initFillInfo(pInfo, pExprInfo, pInfo->numOfExpr, pInfo->pNotFillExprInfo, pInfo->numOfNotFillExpr, (SNodeListNode*)pPhyFillNode->pValues, + pPhyFillNode->timeRange, pResultInfo->capacity, pTaskInfo->id.str, pInterval, type, order); if (code != TSDB_CODE_SUCCESS) { goto _error; } pInfo->pRes = pResBlock; + pInfo->pFinalRes = createOneDataBlock(pResBlock, false); + blockDataEnsureCapacity(pInfo->pFinalRes, pOperator->resultInfo.capacity); + pInfo->pCondition = pPhyFillNode->node.pConditions; pInfo->pColMatchColInfo = pColMatchColInfo; pOperator->name = "FillOperator"; @@ -3701,7 +3759,7 @@ SOperatorInfo* createFillOperatorInfo(SOperatorInfo* downstream, SFillPhysiNode* pOperator->status = OP_NOT_OPENED; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_FILL; pOperator->exprSupp.pExprInfo = pExprInfo; - pOperator->exprSupp.numOfExprs = num; + pOperator->exprSupp.numOfExprs = pInfo->numOfExpr; pOperator->info = pInfo; pOperator->pTaskInfo = pTaskInfo; diff --git a/source/libs/executor/src/projectoperator.c b/source/libs/executor/src/projectoperator.c index 9a7c3cf7fb..f2b79bf703 100644 --- a/source/libs/executor/src/projectoperator.c +++ b/source/libs/executor/src/projectoperator.c @@ -174,7 +174,8 @@ static int32_t doIngroupLimitOffset(SLimitInfo* pLimitInfo, uint64_t groupId, SS if (pLimitInfo->limit.limit >= 0 && pLimitInfo->numOfOutputRows + pBlock->info.rows >= pLimitInfo->limit.limit) { int32_t keepRows = (int32_t)(pLimitInfo->limit.limit - pLimitInfo->numOfOutputRows); blockDataKeepFirstNRows(pBlock, keepRows); - if (pLimitInfo->slimit.limit > 0 && pLimitInfo->slimit.limit <= pLimitInfo->numOfOutputGroups) { + //TODO: optimize it later when partition by + limit + if ((pLimitInfo->slimit.limit == -1 && pLimitInfo->currentGroupId == 0) || pLimitInfo->slimit.limit > 0 && pLimitInfo->slimit.limit <= pLimitInfo->numOfOutputGroups) { doSetOperatorCompleted(pOperator); } } @@ -240,7 +241,8 @@ SSDataBlock* doProjectOperation(SOperatorInfo* pOperator) { } // for stream interval - if (pBlock->info.type == STREAM_RETRIEVE) { + if (pBlock->info.type == STREAM_RETRIEVE || pBlock->info.type == STREAM_DELETE_RESULT || + pBlock->info.type == STREAM_DELETE_DATA) { // printDataBlock1(pBlock, "project1"); return pBlock; } diff --git a/source/libs/executor/src/tfill.c b/source/libs/executor/src/tfill.c index bc266cc33e..6d7cd727b9 100644 --- a/source/libs/executor/src/tfill.c +++ b/source/libs/executor/src/tfill.c @@ -33,39 +33,29 @@ #define DO_INTERPOLATION(_v1, _v2, _k1, _k2, _k) \ ((_v1) + ((_v2) - (_v1)) * (((double)(_k)) - ((double)(_k1))) / (((double)(_k2)) - ((double)(_k1)))) -static void setTagsValue(SFillInfo* pFillInfo, void** data, int32_t genRows) { - for (int32_t j = 0; j < pFillInfo->numOfCols; ++j) { - SFillColInfo* pCol = &pFillInfo->pFillCol[j]; - if (TSDB_COL_IS_NORMAL_COL(pCol->flag) || TSDB_COL_IS_UD_COL(pCol->flag)) { - continue; - } - - SResSchema* pSchema = &pCol->pExpr->base.resSchema; - char* val1 = elePtrAt(data[j], pSchema->bytes, genRows); - - assert(pCol->tagIndex >= 0 && pCol->tagIndex < pFillInfo->numOfTags); - SFillTagColInfo* pTag = &pFillInfo->pTags[pCol->tagIndex]; - assignVal(val1, pTag->tagVal, pSchema->bytes, pSchema->type); - } -} - -static void setNullRow(SSDataBlock* pBlock, int64_t ts, int32_t rowIndex) { - // the first are always the timestamp column, so start from the second column. - for (int32_t i = 0; i < taosArrayGetSize(pBlock->pDataBlock); ++i) { - SColumnInfoData* p = taosArrayGet(pBlock->pDataBlock, i); - if (p->info.type == TSDB_DATA_TYPE_TIMESTAMP) { // handle timestamp - colDataAppend(p, rowIndex, (const char*)&ts, false); - } else { - colDataAppendNULL(p, rowIndex); - } - } -} - #define GET_DEST_SLOT_ID(_p) ((_p)->pExpr->base.resSchema.slotId) -#define GET_SRC_SLOT_ID(_p) ((_p)->pExpr->base.pParam[0].pCol->slotId) static void doSetVal(SColumnInfoData* pDstColInfoData, int32_t rowIndex, const SGroupKeys* pKey); +static void setNullRow(SSDataBlock* pBlock, SFillInfo* pFillInfo, int32_t rowIndex) { + for(int32_t i = 0; i < pFillInfo->numOfCols; ++i) { + SFillColInfo* pCol = &pFillInfo->pFillCol[i]; + int32_t dstSlotId = GET_DEST_SLOT_ID(pCol); + SColumnInfoData* pDstColInfo = taosArrayGet(pBlock->pDataBlock, dstSlotId); + if (pCol->notFillCol) { + if (pDstColInfo->info.type == TSDB_DATA_TYPE_TIMESTAMP) { + colDataAppend(pDstColInfo, rowIndex, (const char*)&pFillInfo->currentKey, false); + } else { + SArray* p = FILL_IS_ASC_FILL(pFillInfo) ? pFillInfo->prev.pRowVal : pFillInfo->next.pRowVal; + SGroupKeys* pKey = taosArrayGet(p, i); + doSetVal(pDstColInfo, rowIndex, pKey); + } + } else { + colDataAppendNULL(pDstColInfo, rowIndex); + } + } +} + static void doSetUserSpecifiedValue(SColumnInfoData* pDst, SVariant* pVar, int32_t rowIndex, int64_t currentKey) { if (pDst->info.type == TSDB_DATA_TYPE_FLOAT) { float v = 0; @@ -96,13 +86,10 @@ static void doFillOneRow(SFillInfo* pFillInfo, SSDataBlock* pBlock, SSDataBlock* // set the other values if (pFillInfo->type == TSDB_FILL_PREV) { - SArray* p = FILL_IS_ASC_FILL(pFillInfo) ? pFillInfo->prev : pFillInfo->next; + SArray* p = FILL_IS_ASC_FILL(pFillInfo)? pFillInfo->prev.pRowVal : pFillInfo->next.pRowVal; for (int32_t i = 0; i < pFillInfo->numOfCols; ++i) { SFillColInfo* pCol = &pFillInfo->pFillCol[i]; - if (TSDB_COL_IS_TAG(pCol->flag)) { - continue; - } SColumnInfoData* pDstColInfoData = taosArrayGet(pBlock->pDataBlock, GET_DEST_SLOT_ID(pCol)); @@ -114,14 +101,10 @@ static void doFillOneRow(SFillInfo* pFillInfo, SSDataBlock* pBlock, SSDataBlock* } } } else if (pFillInfo->type == TSDB_FILL_NEXT) { - SArray* p = FILL_IS_ASC_FILL(pFillInfo) ? pFillInfo->next : pFillInfo->prev; + SArray* p = FILL_IS_ASC_FILL(pFillInfo) ? pFillInfo->next.pRowVal : pFillInfo->prev.pRowVal; // todo refactor: start from 0 not 1 for (int32_t i = 0; i < pFillInfo->numOfCols; ++i) { SFillColInfo* pCol = &pFillInfo->pFillCol[i]; - if (TSDB_COL_IS_TAG(pCol->flag)) { - continue; - } - SColumnInfoData* pDstColInfoData = taosArrayGet(pBlock->pDataBlock, GET_DEST_SLOT_ID(pCol)); if (pDstColInfoData->info.type == TSDB_DATA_TYPE_TIMESTAMP) { @@ -134,59 +117,70 @@ static void doFillOneRow(SFillInfo* pFillInfo, SSDataBlock* pBlock, SSDataBlock* } else if (pFillInfo->type == TSDB_FILL_LINEAR) { // TODO : linear interpolation supports NULL value if (outOfBound) { - setNullRow(pBlock, pFillInfo->currentKey, index); + setNullRow(pBlock, pFillInfo, index); } else { for (int32_t i = 0; i < pFillInfo->numOfCols; ++i) { SFillColInfo* pCol = &pFillInfo->pFillCol[i]; - if (TSDB_COL_IS_TAG(pCol->flag)) { - continue; - } int32_t dstSlotId = GET_DEST_SLOT_ID(pCol); SColumnInfoData* pDstCol = taosArrayGet(pBlock->pDataBlock, dstSlotId); + int16_t type = pDstCol->info.type; - int16_t type = pDstCol->info.type; - if (type == TSDB_DATA_TYPE_TIMESTAMP) { - colDataAppend(pDstCol, index, (const char*)&pFillInfo->currentKey, false); - continue; + if (pCol->notFillCol) { + if (type == TSDB_DATA_TYPE_TIMESTAMP) { + colDataAppend(pDstCol, index, (const char*)&pFillInfo->currentKey, false); + } else { + SArray* p = FILL_IS_ASC_FILL(pFillInfo) ? pFillInfo->prev.pRowVal : pFillInfo->next.pRowVal; + SGroupKeys* pKey = taosArrayGet(p, i); + doSetVal(pDstCol, index, pKey); + } + } else { + SGroupKeys* pKey = taosArrayGet(pFillInfo->prev.pRowVal, i); + if (IS_VAR_DATA_TYPE(type) || type == TSDB_DATA_TYPE_BOOL || pKey->isNull) { + colDataAppendNULL(pDstCol, index); + continue; + } + + SGroupKeys* pKey1 = taosArrayGet(pFillInfo->prev.pRowVal, pFillInfo->tsSlotId); + + int64_t prevTs = *(int64_t*)pKey1->pData; + int32_t srcSlotId = GET_DEST_SLOT_ID(pCol); + + SColumnInfoData* pSrcCol = taosArrayGet(pSrcBlock->pDataBlock, srcSlotId); + char* data = colDataGetData(pSrcCol, pFillInfo->index); + + point1 = (SPoint){.key = prevTs, .val = pKey->pData}; + point2 = (SPoint){.key = ts, .val = data}; + + int64_t out = 0; + point = (SPoint){.key = pFillInfo->currentKey, .val = &out}; + taosGetLinearInterpolationVal(&point, type, &point1, &point2, type); + + colDataAppend(pDstCol, index, (const char*)&out, false); } - - SGroupKeys* pKey = taosArrayGet(pFillInfo->prev, i); - if (IS_VAR_DATA_TYPE(type) || type == TSDB_DATA_TYPE_BOOL || pKey->isNull) { - colDataAppendNULL(pDstCol, index); - continue; - } - - SGroupKeys* pKey1 = taosArrayGet(pFillInfo->prev, pFillInfo->tsSlotId); - - int64_t prevTs = *(int64_t*)pKey1->pData; - int32_t srcSlotId = GET_SRC_SLOT_ID(pCol); - - SColumnInfoData* pSrcCol = taosArrayGet(pSrcBlock->pDataBlock, srcSlotId); - char* data = colDataGetData(pSrcCol, pFillInfo->index); - - point1 = (SPoint){.key = prevTs, .val = pKey->pData}; - point2 = (SPoint){.key = ts, .val = data}; - - int64_t out = 0; - point = (SPoint){.key = pFillInfo->currentKey, .val = &out}; - taosGetLinearInterpolationVal(&point, type, &point1, &point2, type); - - colDataAppend(pDstCol, index, (const char*)&out, false); } } } else if (pFillInfo->type == TSDB_FILL_NULL) { // fill with NULL - setNullRow(pBlock, pFillInfo->currentKey, index); + setNullRow(pBlock, pFillInfo, index); } else { // fill with user specified value for each column for (int32_t i = 0; i < pFillInfo->numOfCols; ++i) { SFillColInfo* pCol = &pFillInfo->pFillCol[i]; - if (TSDB_COL_IS_TAG(pCol->flag)) { - continue; - } - SVariant* pVar = &pFillInfo->pFillCol[i].fillVal; - SColumnInfoData* pDst = taosArrayGet(pBlock->pDataBlock, i); - doSetUserSpecifiedValue(pDst, pVar, index, pFillInfo->currentKey); + int32_t slotId = GET_DEST_SLOT_ID(pCol); + SColumnInfoData* pDst = taosArrayGet(pBlock->pDataBlock, slotId); + + if (pCol->notFillCol) { + if (pDst->info.type == TSDB_DATA_TYPE_TIMESTAMP) { + colDataAppend(pDst, index, (const char*)&pFillInfo->currentKey, false); + } else { + SArray* p = FILL_IS_ASC_FILL(pFillInfo) ? pFillInfo->prev.pRowVal : pFillInfo->next.pRowVal; + SGroupKeys* pKey = taosArrayGet(p, i); + doSetVal(pDst, index, pKey); + } + } else { + SVariant* pVar = &pFillInfo->pFillCol[i].fillVal; + doSetUserSpecifiedValue(pDst, pVar, index, pFillInfo->currentKey); + } } } @@ -207,7 +201,7 @@ void doSetVal(SColumnInfoData* pDstCol, int32_t rowIndex, const SGroupKeys* pKey } static void initBeforeAfterDataBuf(SFillInfo* pFillInfo) { - if (taosArrayGetSize(pFillInfo->next) > 0) { + if (taosArrayGetSize(pFillInfo->next.pRowVal) > 0) { return; } @@ -221,10 +215,10 @@ static void initBeforeAfterDataBuf(SFillInfo* pFillInfo) { key.bytes = pSchema->bytes; key.type = pSchema->type; - taosArrayPush(pFillInfo->next, &key); + taosArrayPush(pFillInfo->next.pRowVal, &key); key.pData = taosMemoryMalloc(pSchema->bytes); - taosArrayPush(pFillInfo->prev, &key); + taosArrayPush(pFillInfo->prev.pRowVal, &key); } } @@ -232,20 +226,31 @@ static void saveColData(SArray* rowBuf, int32_t columnIndex, const char* src, bo static void copyCurrentRowIntoBuf(SFillInfo* pFillInfo, int32_t rowIndex, SArray* pRow) { for (int32_t i = 0; i < pFillInfo->numOfCols; ++i) { - int32_t srcSlotId = GET_SRC_SLOT_ID(&pFillInfo->pFillCol[i]); + int32_t type = pFillInfo->pFillCol[i].pExpr->pExpr->nodeType; + if (type == QUERY_NODE_COLUMN) { + int32_t srcSlotId = GET_DEST_SLOT_ID(&pFillInfo->pFillCol[i]); - SColumnInfoData* pSrcCol = taosArrayGet(pFillInfo->pSrcBlock->pDataBlock, srcSlotId); + SColumnInfoData* pSrcCol = taosArrayGet(pFillInfo->pSrcBlock->pDataBlock, srcSlotId); - bool isNull = colDataIsNull_s(pSrcCol, rowIndex); - char* p = colDataGetData(pSrcCol, rowIndex); - saveColData(pRow, i, p, isNull); + bool isNull = colDataIsNull_s(pSrcCol, rowIndex); + char* p = colDataGetData(pSrcCol, rowIndex); + saveColData(pRow, i, p, isNull); + } else if (type == QUERY_NODE_OPERATOR) { + SColumnInfoData* pSrcCol = taosArrayGet(pFillInfo->pSrcBlock->pDataBlock, i); + + bool isNull = colDataIsNull_s(pSrcCol, rowIndex); + char* p = colDataGetData(pSrcCol, rowIndex); + saveColData(pRow, i, p, isNull); + } else { + ASSERT(0); + } } } static int32_t fillResultImpl(SFillInfo* pFillInfo, SSDataBlock* pBlock, int32_t outputRows) { pFillInfo->numOfCurrent = 0; - SColumnInfoData* pTsCol = taosArrayGet(pFillInfo->pSrcBlock->pDataBlock, pFillInfo->tsSlotId); + SColumnInfoData* pTsCol = taosArrayGet(pFillInfo->pSrcBlock->pDataBlock, pFillInfo->srcTsSlotId); int32_t step = GET_FORWARD_DIRECTION_FACTOR(pFillInfo->order); bool ascFill = FILL_IS_ASC_FILL(pFillInfo); @@ -259,7 +264,7 @@ static int32_t fillResultImpl(SFillInfo* pFillInfo, SSDataBlock* pBlock, int32_t // set the next value for interpolation if ((pFillInfo->currentKey < ts && ascFill) || (pFillInfo->currentKey > ts && !ascFill)) { - copyCurrentRowIntoBuf(pFillInfo, pFillInfo->index, pFillInfo->next); + copyCurrentRowIntoBuf(pFillInfo, pFillInfo->index, pFillInfo->next.pRowVal); } if (((pFillInfo->currentKey < ts && ascFill) || (pFillInfo->currentKey > ts && !ascFill)) && @@ -281,43 +286,38 @@ static int32_t fillResultImpl(SFillInfo* pFillInfo, SSDataBlock* pBlock, int32_t if (pFillInfo->type == TSDB_FILL_NEXT && (pFillInfo->index + 1) < pFillInfo->numOfRows) { int32_t nextRowIndex = pFillInfo->index + 1; - copyCurrentRowIntoBuf(pFillInfo, nextRowIndex, pFillInfo->next); + copyCurrentRowIntoBuf(pFillInfo, nextRowIndex, pFillInfo->next.pRowVal); } - // assign rows to dst buffer + // copy rows to dst buffer for (int32_t i = 0; i < pFillInfo->numOfCols; ++i) { SFillColInfo* pCol = &pFillInfo->pFillCol[i]; - if (TSDB_COL_IS_TAG(pCol->flag) /* || IS_VAR_DATA_TYPE(pCol->schema.type)*/) { - continue; - } - int32_t srcSlotId = GET_SRC_SLOT_ID(pCol); int32_t dstSlotId = GET_DEST_SLOT_ID(pCol); SColumnInfoData* pDst = taosArrayGet(pBlock->pDataBlock, dstSlotId); - SColumnInfoData* pSrc = taosArrayGet(pFillInfo->pSrcBlock->pDataBlock, srcSlotId); + SColumnInfoData* pSrc = taosArrayGet(pFillInfo->pSrcBlock->pDataBlock, dstSlotId); char* src = colDataGetData(pSrc, pFillInfo->index); - if (/*i == 0 || (*/ !colDataIsNull_s(pSrc, pFillInfo->index)) { - bool isNull = colDataIsNull_s(pSrc, pFillInfo->index); - colDataAppend(pDst, index, src, isNull); - saveColData(pFillInfo->prev, i, src, isNull); - } else { + if (!colDataIsNull_s(pSrc, pFillInfo->index)) { + colDataAppend(pDst, index, src, false); + saveColData(pFillInfo->prev.pRowVal, i, src, false); + } else { // the value is null if (pDst->info.type == TSDB_DATA_TYPE_TIMESTAMP) { colDataAppend(pDst, index, (const char*)&pFillInfo->currentKey, false); } else { // i > 0 and data is null , do interpolation if (pFillInfo->type == TSDB_FILL_PREV) { - SArray* p = FILL_IS_ASC_FILL(pFillInfo) ? pFillInfo->prev : pFillInfo->next; + SArray* p = FILL_IS_ASC_FILL(pFillInfo) ? pFillInfo->prev.pRowVal : pFillInfo->next.pRowVal; SGroupKeys* pKey = taosArrayGet(p, i); doSetVal(pDst, index, pKey); } else if (pFillInfo->type == TSDB_FILL_LINEAR) { bool isNull = colDataIsNull_s(pSrc, pFillInfo->index); colDataAppend(pDst, index, src, isNull); - saveColData(pFillInfo->prev, i, src, isNull); // todo: + saveColData(pFillInfo->prev.pRowVal, i, src, isNull); // todo: } else if (pFillInfo->type == TSDB_FILL_NULL) { colDataAppendNULL(pDst, index); } else if (pFillInfo->type == TSDB_FILL_NEXT) { - SArray* p = FILL_IS_ASC_FILL(pFillInfo) ? pFillInfo->next : pFillInfo->prev; + SArray* p = FILL_IS_ASC_FILL(pFillInfo) ? pFillInfo->next.pRowVal : pFillInfo->prev.pRowVal; SGroupKeys* pKey = taosArrayGet(p, i); doSetVal(pDst, index, pKey); } else { @@ -340,10 +340,6 @@ static int32_t fillResultImpl(SFillInfo* pFillInfo, SSDataBlock* pBlock, int32_t } if (pFillInfo->index >= pFillInfo->numOfRows || pFillInfo->numOfCurrent >= outputRows) { - /* the raw data block is exhausted, next value does not exists */ - // if (pFillInfo->index >= pFillInfo->numOfRows) { - // taosMemoryFreeClear(*next); - // } pFillInfo->numOfTotal += pFillInfo->numOfCurrent; return pFillInfo->numOfCurrent; } @@ -357,7 +353,11 @@ static void saveColData(SArray* rowBuf, int32_t columnIndex, const char* src, bo if (isNull) { pKey->isNull = true; } else { - memcpy(pKey->pData, src, pKey->bytes); + if (IS_VAR_DATA_TYPE(pKey->type)) { + memcpy(pKey->pData, src, varDataTLen(src)); + } else { + memcpy(pKey->pData, src, pKey->bytes); + } pKey->isNull = false; } } @@ -378,53 +378,6 @@ static int64_t appendFilledResult(SFillInfo* pFillInfo, SSDataBlock* pBlock, int return resultCapacity; } -// there are no duplicated tags in the SFillTagColInfo list -static int32_t setTagColumnInfo(SFillInfo* pFillInfo, int32_t numOfCols, int32_t capacity) { - int32_t rowsize = 0; - int32_t numOfTags = 0; - - int32_t k = 0; - for (int32_t i = 0; i < numOfCols; ++i) { - SFillColInfo* pColInfo = &pFillInfo->pFillCol[i]; - SResSchema* pSchema = &pColInfo->pExpr->base.resSchema; - - if (TSDB_COL_IS_TAG(pColInfo->flag) || pSchema->type == TSDB_DATA_TYPE_BINARY) { - numOfTags += 1; - - bool exists = false; - int32_t index = -1; - for (int32_t j = 0; j < k; ++j) { - if (pFillInfo->pTags[j].col.colId == pSchema->slotId) { - exists = true; - index = j; - break; - } - } - - if (!exists) { - SSchema* pSchema1 = &pFillInfo->pTags[k].col; - pSchema1->colId = pSchema->slotId; - pSchema1->type = pSchema->type; - pSchema1->bytes = pSchema->bytes; - - pFillInfo->pTags[k].tagVal = taosMemoryCalloc(1, pSchema->bytes); - pColInfo->tagIndex = k; - - k += 1; - } else { - pColInfo->tagIndex = index; - } - } - - rowsize += pSchema->bytes; - } - - pFillInfo->numOfTags = numOfTags; - - assert(k <= pFillInfo->numOfTags); - return rowsize; -} - static int32_t taosNumOfRemainRows(SFillInfo* pFillInfo) { if (pFillInfo->numOfRows == 0 || (pFillInfo->numOfRows > 0 && pFillInfo->index >= pFillInfo->numOfRows)) { return 0; @@ -433,7 +386,7 @@ static int32_t taosNumOfRemainRows(SFillInfo* pFillInfo) { return pFillInfo->numOfRows - pFillInfo->index; } -struct SFillInfo* taosCreateFillInfo(TSKEY skey, int32_t numOfTags, int32_t capacity, int32_t numOfCols, +struct SFillInfo* taosCreateFillInfo(TSKEY skey, int32_t numOfFillCols, int32_t numOfNotFillCols, int32_t capacity, SInterval* pInterval, int32_t fillType, struct SFillColInfo* pCol, int32_t primaryTsSlotId, int32_t order, const char* id) { if (fillType == TSDB_FILL_NONE) { @@ -447,7 +400,17 @@ struct SFillInfo* taosCreateFillInfo(TSKEY skey, int32_t numOfTags, int32_t capa } pFillInfo->order = order; - pFillInfo->tsSlotId = primaryTsSlotId; + pFillInfo->srcTsSlotId = primaryTsSlotId; + + for(int32_t i = 0; i < numOfNotFillCols; ++i) { + SFillColInfo* p = &pCol[i + numOfFillCols]; + int32_t srcSlotId = GET_DEST_SLOT_ID(p); + if (srcSlotId == primaryTsSlotId) { + pFillInfo->tsSlotId = i + numOfFillCols; + break; + } + } + taosResetFillInfo(pFillInfo, skey); switch (fillType) { @@ -476,26 +439,15 @@ struct SFillInfo* taosCreateFillInfo(TSKEY skey, int32_t numOfTags, int32_t capa pFillInfo->type = fillType; pFillInfo->pFillCol = pCol; - pFillInfo->numOfTags = numOfTags; - pFillInfo->numOfCols = numOfCols; + pFillInfo->numOfCols = numOfFillCols + numOfNotFillCols; pFillInfo->alloc = capacity; pFillInfo->id = id; pFillInfo->interval = *pInterval; - // if (numOfTags > 0) { - pFillInfo->pTags = taosMemoryCalloc(numOfCols, sizeof(SFillTagColInfo)); - for (int32_t i = 0; i < numOfCols; ++i) { - pFillInfo->pTags[i].col.colId = -2; // TODO - } - // } - - pFillInfo->next = taosArrayInit(numOfCols, sizeof(SGroupKeys)); - pFillInfo->prev = taosArrayInit(numOfCols, sizeof(SGroupKeys)); + pFillInfo->next.pRowVal = taosArrayInit(pFillInfo->numOfCols, sizeof(SGroupKeys)); + pFillInfo->prev.pRowVal = taosArrayInit(pFillInfo->numOfCols, sizeof(SGroupKeys)); initBeforeAfterDataBuf(pFillInfo); - - pFillInfo->rowSize = setTagColumnInfo(pFillInfo, pFillInfo->numOfCols, pFillInfo->alloc); - assert(pFillInfo->rowSize > 0); return pFillInfo; } @@ -513,20 +465,20 @@ void* taosDestroyFillInfo(SFillInfo* pFillInfo) { if (pFillInfo == NULL) { return NULL; } - for (int32_t i = 0; i < taosArrayGetSize(pFillInfo->prev); ++i) { - SGroupKeys* pKey = taosArrayGet(pFillInfo->prev, i); + for (int32_t i = 0; i < taosArrayGetSize(pFillInfo->prev.pRowVal); ++i) { + SGroupKeys* pKey = taosArrayGet(pFillInfo->prev.pRowVal, i); taosMemoryFree(pKey->pData); } - taosArrayDestroy(pFillInfo->prev); - for (int32_t i = 0; i < taosArrayGetSize(pFillInfo->next); ++i) { - SGroupKeys* pKey = taosArrayGet(pFillInfo->next, i); + taosArrayDestroy(pFillInfo->prev.pRowVal); + for (int32_t i = 0; i < taosArrayGetSize(pFillInfo->next.pRowVal); ++i) { + SGroupKeys* pKey = taosArrayGet(pFillInfo->next.pRowVal, i); taosMemoryFree(pKey->pData); } - taosArrayDestroy(pFillInfo->next); + taosArrayDestroy(pFillInfo->next.pRowVal); - for (int32_t i = 0; i < pFillInfo->numOfTags; ++i) { - taosMemoryFreeClear(pFillInfo->pTags[i].tagVal); - } +// for (int32_t i = 0; i < pFillInfo->numOfTags; ++i) { +// taosMemoryFreeClear(pFillInfo->pTags[i].tagVal); +// } taosMemoryFreeClear(pFillInfo->pTags); taosMemoryFreeClear(pFillInfo->pFillCol); @@ -576,7 +528,7 @@ bool taosFillHasMoreResults(SFillInfo* pFillInfo) { } int64_t getNumOfResultsAfterFillGap(SFillInfo* pFillInfo, TSKEY ekey, int32_t maxNumOfRows) { - SColumnInfoData* pCol = taosArrayGet(pFillInfo->pSrcBlock->pDataBlock, 0); + SColumnInfoData* pCol = taosArrayGet(pFillInfo->pSrcBlock->pDataBlock, pFillInfo->srcTsSlotId); int64_t* tsList = (int64_t*)pCol->pData; int32_t numOfRows = taosNumOfRemainRows(pFillInfo); @@ -642,17 +594,18 @@ int64_t taosFillResultDataBlock(SFillInfo* pFillInfo, SSDataBlock* p, int32_t ca int64_t getFillInfoStart(struct SFillInfo* pFillInfo) { return pFillInfo->start; } -SFillColInfo* createFillColInfo(SExprInfo* pExpr, int32_t numOfOutput, const struct SNodeListNode* pValNode) { - SFillColInfo* pFillCol = taosMemoryCalloc(numOfOutput, sizeof(SFillColInfo)); +SFillColInfo* createFillColInfo(SExprInfo* pExpr, int32_t numOfFillExpr, SExprInfo* pNotFillExpr, + int32_t numOfNotFillExpr, const struct SNodeListNode* pValNode) { + SFillColInfo* pFillCol = taosMemoryCalloc(numOfFillExpr + numOfNotFillExpr, sizeof(SFillColInfo)); if (pFillCol == NULL) { return NULL; } size_t len = (pValNode != NULL) ? LIST_LENGTH(pValNode->pNodeList) : 0; - for (int32_t i = 0; i < numOfOutput; ++i) { + for (int32_t i = 0; i < numOfFillExpr; ++i) { SExprInfo* pExprInfo = &pExpr[i]; pFillCol[i].pExpr = pExprInfo; - pFillCol[i].tagIndex = -2; + pFillCol[i].notFillCol = false; // todo refactor if (len > 0) { @@ -662,10 +615,12 @@ SFillColInfo* createFillColInfo(SExprInfo* pExpr, int32_t numOfOutput, const str SValueNode* pv = (SValueNode*)nodesListGetNode(pValNode->pNodeList, index); nodesValueNodeToVariant(pv, &pFillCol[i].fillVal); } + } - if (pExprInfo->base.numOfParams > 0) { - pFillCol[i].flag = pExprInfo->base.pParam[0].pCol->flag; // always be the normal column for table query - } + for(int32_t i = 0; i < numOfNotFillExpr; ++i) { + SExprInfo* pExprInfo = &pNotFillExpr[i]; + pFillCol[i + numOfFillExpr].pExpr = pExprInfo; + pFillCol[i + numOfFillExpr].notFillCol = true; } return pFillCol; diff --git a/source/libs/executor/src/timewindowoperator.c b/source/libs/executor/src/timewindowoperator.c index 481215b502..6418f5305c 100644 --- a/source/libs/executor/src/timewindowoperator.c +++ b/source/libs/executor/src/timewindowoperator.c @@ -2675,10 +2675,8 @@ SOperatorInfo* createTimeSliceOperatorInfo(SOperatorInfo* downstream, SPhysiNode pInfo->fillType = convertFillType(pInterpPhyNode->fillMode); initResultSizeInfo(&pOperator->resultInfo, 4096); - pInfo->pPrevRow = NULL; - pInfo->pNextRow = NULL; + pInfo->pFillColInfo = createFillColInfo(pExprInfo, numOfExprs, NULL, 0, (SNodeListNode*)pInterpPhyNode->pFillValues); pInfo->pLinearInfo = NULL; - pInfo->pFillColInfo = createFillColInfo(pExprInfo, numOfExprs, (SNodeListNode*)pInterpPhyNode->pFillValues); pInfo->pRes = createResDataBlock(pPhyNode->pOutputDataBlockDesc); pInfo->win = pInterpPhyNode->timeRange; pInfo->interval.interval = pInterpPhyNode->interval; diff --git a/source/libs/nodes/src/nodesCloneFuncs.c b/source/libs/nodes/src/nodesCloneFuncs.c index c4f3dbb9bd..9390d129df 100644 --- a/source/libs/nodes/src/nodesCloneFuncs.c +++ b/source/libs/nodes/src/nodesCloneFuncs.c @@ -451,6 +451,8 @@ static int32_t logicWindowCopy(const SWindowLogicNode* pSrc, SWindowLogicNode* p static int32_t logicFillCopy(const SFillLogicNode* pSrc, SFillLogicNode* pDst) { COPY_BASE_OBJECT_FIELD(node, logicNodeCopy); COPY_SCALAR_FIELD(mode); + CLONE_NODE_LIST_FIELD(pFillExprs); + CLONE_NODE_LIST_FIELD(pNotFillExprs); CLONE_NODE_FIELD(pWStartTs); CLONE_NODE_FIELD(pValues); COPY_OBJECT_FIELD(timeRange, sizeof(STimeWindow)); diff --git a/source/libs/nodes/src/nodesCodeFuncs.c b/source/libs/nodes/src/nodesCodeFuncs.c index ab76eb21b8..a6546f3299 100644 --- a/source/libs/nodes/src/nodesCodeFuncs.c +++ b/source/libs/nodes/src/nodesCodeFuncs.c @@ -2086,9 +2086,10 @@ static int32_t jsonToPhysiIntervalNode(const SJson* pJson, void* pObj) { } static const char* jkFillPhysiPlanMode = "Mode"; +static const char* jkFillPhysiPlanFillExprs = "FillExprs"; +static const char* jkFillPhysiPlanNotFillExprs = "NotFillExprs"; static const char* jkFillPhysiPlanWStartTs = "WStartTs"; static const char* jkFillPhysiPlanValues = "Values"; -static const char* jkFillPhysiPlanTargets = "Targets"; static const char* jkFillPhysiPlanStartTime = "StartTime"; static const char* jkFillPhysiPlanEndTime = "EndTime"; static const char* jkFillPhysiPlanInputTsOrder = "inputTsOrder"; @@ -2100,15 +2101,18 @@ static int32_t physiFillNodeToJson(const void* pObj, SJson* pJson) { if (TSDB_CODE_SUCCESS == code) { code = tjsonAddIntegerToObject(pJson, jkFillPhysiPlanMode, pNode->mode); } + if (TSDB_CODE_SUCCESS == code) { + code = nodeListToJson(pJson, jkFillPhysiPlanFillExprs, pNode->pFillExprs); + } + if (TSDB_CODE_SUCCESS == code) { + code = nodeListToJson(pJson, jkFillPhysiPlanNotFillExprs, pNode->pNotFillExprs); + } if (TSDB_CODE_SUCCESS == code) { code = tjsonAddObject(pJson, jkFillPhysiPlanWStartTs, nodeToJson, pNode->pWStartTs); } if (TSDB_CODE_SUCCESS == code) { code = tjsonAddObject(pJson, jkFillPhysiPlanValues, nodeToJson, pNode->pValues); } - if (TSDB_CODE_SUCCESS == code) { - code = nodeListToJson(pJson, jkFillPhysiPlanTargets, pNode->pTargets); - } if (TSDB_CODE_SUCCESS == code) { code = tjsonAddIntegerToObject(pJson, jkFillPhysiPlanStartTime, pNode->timeRange.skey); } @@ -2128,7 +2132,12 @@ static int32_t jsonToPhysiFillNode(const SJson* pJson, void* pObj) { int32_t code = jsonToPhysicPlanNode(pJson, pObj); if (TSDB_CODE_SUCCESS == code) { tjsonGetNumberValue(pJson, jkFillPhysiPlanMode, pNode->mode, code); - ; + } + if (TSDB_CODE_SUCCESS == code) { + code = jsonToNodeList(pJson, jkFillPhysiPlanFillExprs, &pNode->pFillExprs); + } + if (TSDB_CODE_SUCCESS == code) { + code = jsonToNodeList(pJson, jkFillPhysiPlanNotFillExprs, &pNode->pNotFillExprs); } if (TSDB_CODE_SUCCESS == code) { code = jsonToNodeObject(pJson, jkFillPhysiPlanWStartTs, &pNode->pWStartTs); @@ -2136,9 +2145,6 @@ static int32_t jsonToPhysiFillNode(const SJson* pJson, void* pObj) { if (TSDB_CODE_SUCCESS == code) { code = jsonToNodeObject(pJson, jkFillPhysiPlanValues, &pNode->pValues); } - if (TSDB_CODE_SUCCESS == code) { - code = jsonToNodeList(pJson, jkFillPhysiPlanTargets, &pNode->pTargets); - } if (TSDB_CODE_SUCCESS == code) { code = tjsonGetBigIntValue(pJson, jkFillPhysiPlanStartTime, &pNode->timeRange.skey); } diff --git a/source/libs/nodes/src/nodesTraverseFuncs.c b/source/libs/nodes/src/nodesTraverseFuncs.c index 77681af1bc..2e23998aad 100644 --- a/source/libs/nodes/src/nodesTraverseFuncs.c +++ b/source/libs/nodes/src/nodesTraverseFuncs.c @@ -346,6 +346,7 @@ void nodesWalkSelectStmt(SSelectStmt* pSelect, ESqlClause clause, FNodeWalker wa if (NULL != pSelect->pWindow && QUERY_NODE_INTERVAL_WINDOW == nodeType(pSelect->pWindow)) { nodesWalkExpr(((SIntervalWindowNode*)pSelect->pWindow)->pFill, walker, pContext); } + case SQL_CLAUSE_FILL: nodesWalkExprs(pSelect->pGroupByList, walker, pContext); case SQL_CLAUSE_GROUP_BY: nodesWalkExpr(pSelect->pHaving, walker, pContext); @@ -379,6 +380,7 @@ void nodesRewriteSelectStmt(SSelectStmt* pSelect, ESqlClause clause, FNodeRewrit if (NULL != pSelect->pWindow && QUERY_NODE_INTERVAL_WINDOW == nodeType(pSelect->pWindow)) { nodesRewriteExpr(&(((SIntervalWindowNode*)pSelect->pWindow)->pFill), rewriter, pContext); } + case SQL_CLAUSE_FILL: nodesRewriteExprs(pSelect->pGroupByList, rewriter, pContext); case SQL_CLAUSE_GROUP_BY: nodesRewriteExpr(&(pSelect->pHaving), rewriter, pContext); diff --git a/source/libs/nodes/src/nodesUtilFuncs.c b/source/libs/nodes/src/nodesUtilFuncs.c index d48fd2c4c0..f8ba6e6901 100644 --- a/source/libs/nodes/src/nodesUtilFuncs.c +++ b/source/libs/nodes/src/nodesUtilFuncs.c @@ -931,9 +931,10 @@ void nodesDestroyNode(SNode* pNode) { case QUERY_NODE_PHYSICAL_PLAN_FILL: { SFillPhysiNode* pPhyNode = (SFillPhysiNode*)pNode; destroyPhysiNode((SPhysiNode*)pPhyNode); + nodesDestroyList(pPhyNode->pFillExprs); + nodesDestroyList(pPhyNode->pNotFillExprs); nodesDestroyNode(pPhyNode->pWStartTs); nodesDestroyNode(pPhyNode->pValues); - nodesDestroyList(pPhyNode->pTargets); break; } case QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION: diff --git a/source/libs/parser/inc/sql.y b/source/libs/parser/inc/sql.y index 7afafd34b6..56e68d8374 100644 --- a/source/libs/parser/inc/sql.y +++ b/source/libs/parser/inc/sql.y @@ -506,7 +506,7 @@ stream_options(A) ::= stream_options(B) TRIGGER AT_ONCE. stream_options(A) ::= stream_options(B) TRIGGER WINDOW_CLOSE. { ((SStreamOptions*)B)->triggerType = STREAM_TRIGGER_WINDOW_CLOSE; A = B; } stream_options(A) ::= stream_options(B) TRIGGER MAX_DELAY duration_literal(C). { ((SStreamOptions*)B)->triggerType = STREAM_TRIGGER_MAX_DELAY; ((SStreamOptions*)B)->pDelay = releaseRawExprNode(pCxt, C); A = B; } stream_options(A) ::= stream_options(B) WATERMARK duration_literal(C). { ((SStreamOptions*)B)->pWatermark = releaseRawExprNode(pCxt, C); A = B; } -stream_options(A) ::= stream_options(B) IGNORE EXPIRED. { ((SStreamOptions*)B)->ignoreExpired = true; A = B; } +stream_options(A) ::= stream_options(B) IGNORE EXPIRED NK_INTEGER(C). { ((SStreamOptions*)B)->ignoreExpired = taosStr2Int8(C.z, NULL, 10); A = B; } /************************************************ kill connection/query ***********************************************/ cmd ::= KILL CONNECTION NK_INTEGER(A). { pCxt->pRootNode = createKillStmt(pCxt, QUERY_NODE_KILL_CONNECTION_STMT, &A); } diff --git a/source/libs/parser/src/parAstCreater.c b/source/libs/parser/src/parAstCreater.c index aa526dd440..4d0b0bbb25 100644 --- a/source/libs/parser/src/parAstCreater.c +++ b/source/libs/parser/src/parAstCreater.c @@ -1628,6 +1628,7 @@ SNode* createStreamOptions(SAstCreateContext* pCxt) { SStreamOptions* pOptions = (SStreamOptions*)nodesMakeNode(QUERY_NODE_STREAM_OPTIONS); CHECK_OUT_OF_MEM(pOptions); pOptions->triggerType = STREAM_TRIGGER_AT_ONCE; + pOptions->ignoreExpired = STREAM_DEFAULT_IGNORE_EXPIRED; return (SNode*)pOptions; } diff --git a/source/libs/parser/src/sql.c b/source/libs/parser/src/sql.c index 4756a78be6..c820e955d7 100644 --- a/source/libs/parser/src/sql.c +++ b/source/libs/parser/src/sql.c @@ -139,17 +139,17 @@ typedef union { #define ParseCTX_FETCH #define ParseCTX_STORE #define YYFALLBACK 1 -#define YYNSTATE 666 +#define YYNSTATE 667 #define YYNRULE 491 #define YYNTOKEN 305 -#define YY_MAX_SHIFT 665 -#define YY_MIN_SHIFTREDUCE 972 -#define YY_MAX_SHIFTREDUCE 1462 -#define YY_ERROR_ACTION 1463 -#define YY_ACCEPT_ACTION 1464 -#define YY_NO_ACTION 1465 -#define YY_MIN_REDUCE 1466 -#define YY_MAX_REDUCE 1956 +#define YY_MAX_SHIFT 666 +#define YY_MIN_SHIFTREDUCE 973 +#define YY_MAX_SHIFTREDUCE 1463 +#define YY_ERROR_ACTION 1464 +#define YY_ACCEPT_ACTION 1465 +#define YY_NO_ACTION 1466 +#define YY_MIN_REDUCE 1467 +#define YY_MAX_REDUCE 1957 /************* End control #defines *******************************************/ #define YY_NLOOKAHEAD ((int)(sizeof(yy_lookahead)/sizeof(yy_lookahead[0]))) @@ -218,261 +218,261 @@ typedef union { *********** Begin parsing tables **********************************************/ #define YY_ACTTAB_COUNT (2548) static const YYACTIONTYPE yy_action[] = { - /* 0 */ 525, 30, 261, 525, 548, 433, 525, 434, 1501, 11, - /* 10 */ 10, 117, 39, 37, 55, 1652, 1653, 117, 471, 378, - /* 20 */ 339, 1467, 1263, 1005, 476, 1022, 1289, 1021, 1606, 1790, - /* 30 */ 1597, 1606, 127, 1339, 1606, 1261, 441, 551, 434, 1501, - /* 40 */ 469, 1774, 107, 1778, 1289, 106, 105, 104, 103, 102, - /* 50 */ 101, 100, 99, 98, 1774, 1023, 1334, 1808, 150, 64, - /* 60 */ 1934, 14, 1566, 1009, 1010, 552, 1770, 1776, 1269, 450, - /* 70 */ 1760, 125, 576, 165, 39, 37, 1402, 1931, 570, 1770, - /* 80 */ 1776, 328, 339, 1528, 1263, 550, 161, 1876, 1877, 1, - /* 90 */ 1881, 570, 1658, 479, 478, 1339, 1822, 1261, 1375, 327, - /* 100 */ 95, 1791, 579, 1793, 1794, 575, 496, 570, 1656, 344, - /* 110 */ 1868, 662, 1651, 1653, 330, 1864, 160, 513, 1334, 494, - /* 120 */ 1934, 492, 1288, 14, 325, 1341, 1342, 1704, 164, 542, - /* 130 */ 1269, 1160, 1161, 1933, 33, 32, 1894, 1931, 40, 38, - /* 140 */ 36, 35, 34, 148, 63, 1478, 639, 638, 637, 636, - /* 150 */ 349, 2, 635, 634, 128, 629, 628, 627, 626, 625, - /* 160 */ 624, 623, 139, 619, 618, 617, 348, 347, 614, 613, - /* 170 */ 1264, 107, 1262, 662, 106, 105, 104, 103, 102, 101, - /* 180 */ 100, 99, 98, 1808, 36, 35, 34, 1341, 1342, 224, - /* 190 */ 225, 541, 384, 1267, 1268, 612, 1316, 1317, 1319, 1320, - /* 200 */ 1321, 1322, 1323, 1324, 572, 568, 1332, 1333, 1335, 1336, - /* 210 */ 1337, 1338, 1340, 1343, 1466, 1287, 1433, 33, 32, 482, - /* 220 */ 481, 40, 38, 36, 35, 34, 123, 168, 540, 303, - /* 230 */ 1464, 223, 1264, 84, 1262, 1263, 477, 480, 116, 115, - /* 240 */ 114, 113, 112, 111, 110, 109, 108, 305, 1261, 1022, - /* 250 */ 515, 1021, 22, 174, 1599, 1267, 1268, 1489, 1316, 1317, - /* 260 */ 1319, 1320, 1321, 1322, 1323, 1324, 572, 568, 1332, 1333, - /* 270 */ 1335, 1336, 1337, 1338, 1340, 1343, 39, 37, 1488, 1023, - /* 280 */ 537, 1269, 168, 525, 339, 71, 1263, 1487, 70, 354, - /* 290 */ 1243, 1244, 1707, 1790, 170, 211, 512, 1339, 1760, 1261, - /* 300 */ 1118, 601, 600, 599, 1122, 598, 1124, 1125, 597, 1127, - /* 310 */ 594, 1606, 1133, 591, 1135, 1136, 588, 585, 1934, 1760, - /* 320 */ 1334, 1808, 1583, 1269, 662, 14, 1658, 1934, 1760, 552, - /* 330 */ 1934, 166, 1269, 343, 1760, 1931, 576, 1934, 39, 37, - /* 340 */ 1932, 487, 1656, 165, 1931, 551, 339, 1931, 1263, 548, - /* 350 */ 165, 76, 305, 2, 1931, 515, 497, 543, 538, 1339, - /* 360 */ 1822, 1261, 1697, 159, 95, 1791, 579, 1793, 1794, 575, - /* 370 */ 210, 570, 63, 173, 1868, 662, 1645, 127, 330, 1864, - /* 380 */ 160, 551, 1334, 1264, 490, 1262, 419, 604, 484, 1341, - /* 390 */ 1342, 33, 32, 209, 1269, 40, 38, 36, 35, 34, - /* 400 */ 1895, 633, 631, 39, 37, 1344, 1267, 1268, 1486, 91, - /* 410 */ 621, 339, 1790, 1263, 42, 8, 125, 40, 38, 36, - /* 420 */ 35, 34, 124, 610, 1339, 58, 1261, 1595, 57, 49, - /* 430 */ 1598, 162, 1876, 1877, 1264, 1881, 1262, 662, 178, 177, - /* 440 */ 1808, 352, 137, 136, 607, 606, 605, 1334, 574, 1760, - /* 450 */ 43, 1341, 1342, 1760, 316, 576, 1485, 1267, 1268, 1269, - /* 460 */ 1316, 1317, 1319, 1320, 1321, 1322, 1323, 1324, 572, 568, - /* 470 */ 1332, 1333, 1335, 1336, 1337, 1338, 1340, 1343, 63, 1822, - /* 480 */ 9, 74, 1934, 294, 1791, 579, 1793, 1794, 575, 573, - /* 490 */ 570, 567, 1840, 1288, 122, 165, 1264, 1760, 1262, 1931, - /* 500 */ 33, 32, 662, 1601, 40, 38, 36, 35, 34, 317, - /* 510 */ 168, 315, 314, 1484, 473, 351, 1341, 1342, 475, 1267, - /* 520 */ 1268, 1290, 1316, 1317, 1319, 1320, 1321, 1322, 1323, 1324, - /* 530 */ 572, 568, 1332, 1333, 1335, 1336, 1337, 1338, 1340, 1343, - /* 540 */ 474, 1009, 1010, 33, 32, 1459, 1363, 40, 38, 36, - /* 550 */ 35, 34, 168, 168, 1760, 525, 1934, 1591, 377, 146, - /* 560 */ 376, 1264, 63, 1262, 26, 1531, 382, 168, 1609, 165, - /* 570 */ 33, 32, 217, 1931, 40, 38, 36, 35, 34, 218, - /* 580 */ 1483, 1790, 1413, 1606, 1267, 1268, 1593, 1316, 1317, 1319, - /* 590 */ 1320, 1321, 1322, 1323, 1324, 572, 568, 1332, 1333, 1335, - /* 600 */ 1336, 1337, 1338, 1340, 1343, 39, 37, 77, 27, 1808, - /* 610 */ 498, 1883, 63, 339, 78, 1263, 168, 577, 1368, 1482, - /* 620 */ 505, 1760, 1760, 373, 576, 1301, 1339, 28, 1261, 482, - /* 630 */ 481, 1481, 1458, 33, 32, 1880, 123, 40, 38, 36, - /* 640 */ 35, 34, 375, 371, 438, 1589, 477, 480, 1822, 1334, - /* 650 */ 1286, 1934, 96, 1791, 579, 1793, 1794, 575, 253, 570, - /* 660 */ 1760, 1269, 1868, 513, 165, 1480, 1867, 1864, 1931, 1080, - /* 670 */ 33, 32, 1760, 1705, 40, 38, 36, 35, 34, 665, - /* 680 */ 33, 32, 9, 525, 40, 38, 36, 35, 34, 1477, - /* 690 */ 1476, 33, 32, 268, 383, 40, 38, 36, 35, 34, - /* 700 */ 168, 1703, 1082, 300, 662, 432, 1760, 157, 436, 1697, - /* 710 */ 214, 1606, 655, 651, 647, 643, 266, 1581, 1341, 1342, - /* 720 */ 176, 33, 32, 307, 571, 40, 38, 36, 35, 34, - /* 730 */ 1760, 1760, 39, 37, 525, 603, 525, 302, 1475, 1286, - /* 740 */ 339, 548, 1263, 525, 307, 389, 412, 404, 92, 424, - /* 750 */ 168, 231, 1301, 1339, 405, 1261, 440, 1584, 74, 436, - /* 760 */ 1361, 1406, 1606, 1264, 1606, 1262, 397, 1288, 425, 127, - /* 770 */ 399, 1606, 1474, 1702, 1778, 300, 1334, 1888, 1395, 1760, - /* 780 */ 1602, 1361, 44, 4, 522, 1774, 1267, 1268, 1269, 1316, - /* 790 */ 1317, 1319, 1320, 1321, 1322, 1323, 1324, 572, 568, 1332, - /* 800 */ 1333, 1335, 1336, 1337, 1338, 1340, 1343, 390, 125, 2, - /* 810 */ 1770, 1776, 334, 1760, 1362, 7, 220, 450, 610, 386, - /* 820 */ 90, 525, 570, 163, 1876, 1877, 1658, 1881, 1423, 145, - /* 830 */ 87, 662, 448, 312, 1235, 1362, 213, 137, 136, 607, - /* 840 */ 606, 605, 1656, 1479, 1883, 1341, 1342, 423, 1473, 1606, + /* 0 */ 526, 30, 261, 526, 549, 433, 526, 434, 1502, 11, + /* 10 */ 10, 117, 39, 37, 55, 1653, 1654, 117, 471, 378, + /* 20 */ 339, 1468, 1264, 1006, 476, 1023, 1290, 1022, 1607, 1791, + /* 30 */ 1598, 1607, 127, 1340, 1607, 1262, 441, 552, 434, 1502, + /* 40 */ 469, 1775, 107, 1779, 1290, 106, 105, 104, 103, 102, + /* 50 */ 101, 100, 99, 98, 1775, 1024, 1335, 1809, 150, 64, + /* 60 */ 1935, 14, 1567, 1010, 1011, 553, 1771, 1777, 1270, 450, + /* 70 */ 1761, 125, 577, 165, 39, 37, 1403, 1932, 571, 1771, + /* 80 */ 1777, 328, 339, 1529, 1264, 551, 161, 1877, 1878, 1, + /* 90 */ 1882, 571, 1659, 479, 478, 1340, 1823, 1262, 1376, 327, + /* 100 */ 95, 1792, 580, 1794, 1795, 576, 496, 571, 1657, 344, + /* 110 */ 1869, 663, 1652, 1654, 330, 1865, 160, 513, 1335, 494, + /* 120 */ 1935, 492, 1289, 14, 325, 1342, 1343, 1705, 164, 543, + /* 130 */ 1270, 1161, 1162, 1934, 33, 32, 1895, 1932, 40, 38, + /* 140 */ 36, 35, 34, 148, 63, 1479, 640, 639, 638, 637, + /* 150 */ 349, 2, 636, 635, 128, 630, 629, 628, 627, 626, + /* 160 */ 625, 624, 139, 620, 619, 618, 348, 347, 615, 614, + /* 170 */ 1265, 107, 1263, 663, 106, 105, 104, 103, 102, 101, + /* 180 */ 100, 99, 98, 1809, 36, 35, 34, 1342, 1343, 224, + /* 190 */ 225, 542, 384, 1268, 1269, 613, 1317, 1318, 1320, 1321, + /* 200 */ 1322, 1323, 1324, 1325, 573, 569, 1333, 1334, 1336, 1337, + /* 210 */ 1338, 1339, 1341, 1344, 1467, 1288, 1434, 33, 32, 482, + /* 220 */ 481, 40, 38, 36, 35, 34, 123, 168, 541, 303, + /* 230 */ 1465, 223, 1265, 84, 1263, 1264, 477, 480, 116, 115, + /* 240 */ 114, 113, 112, 111, 110, 109, 108, 305, 1262, 1023, + /* 250 */ 516, 1022, 22, 174, 1600, 1268, 1269, 1490, 1317, 1318, + /* 260 */ 1320, 1321, 1322, 1323, 1324, 1325, 573, 569, 1333, 1334, + /* 270 */ 1336, 1337, 1338, 1339, 1341, 1344, 39, 37, 1489, 1024, + /* 280 */ 538, 1270, 168, 526, 339, 71, 1264, 1488, 70, 354, + /* 290 */ 1244, 1245, 1708, 1791, 170, 211, 512, 1340, 1761, 1262, + /* 300 */ 1119, 602, 601, 600, 1123, 599, 1125, 1126, 598, 1128, + /* 310 */ 595, 1607, 1134, 592, 1136, 1137, 589, 586, 1935, 1761, + /* 320 */ 1335, 1809, 1584, 1270, 663, 14, 1659, 1935, 1761, 553, + /* 330 */ 1935, 166, 1270, 343, 1761, 1932, 577, 1935, 39, 37, + /* 340 */ 1933, 487, 1657, 165, 1932, 552, 339, 1932, 1264, 549, + /* 350 */ 165, 76, 305, 2, 1932, 516, 497, 544, 539, 1340, + /* 360 */ 1823, 1262, 1698, 159, 95, 1792, 580, 1794, 1795, 576, + /* 370 */ 210, 571, 63, 173, 1869, 663, 1646, 127, 330, 1865, + /* 380 */ 160, 552, 1335, 1265, 490, 1263, 419, 605, 484, 1342, + /* 390 */ 1343, 33, 32, 209, 1270, 40, 38, 36, 35, 34, + /* 400 */ 1896, 634, 632, 39, 37, 1345, 1268, 1269, 1487, 91, + /* 410 */ 622, 339, 1791, 1264, 42, 8, 125, 40, 38, 36, + /* 420 */ 35, 34, 124, 611, 1340, 58, 1262, 1596, 57, 49, + /* 430 */ 1599, 162, 1877, 1878, 1265, 1882, 1263, 663, 178, 177, + /* 440 */ 1809, 352, 137, 136, 608, 607, 606, 1335, 575, 1761, + /* 450 */ 43, 1342, 1343, 1761, 316, 577, 1486, 1268, 1269, 1270, + /* 460 */ 1317, 1318, 1320, 1321, 1322, 1323, 1324, 1325, 573, 569, + /* 470 */ 1333, 1334, 1336, 1337, 1338, 1339, 1341, 1344, 63, 1823, + /* 480 */ 9, 74, 1935, 294, 1792, 580, 1794, 1795, 576, 574, + /* 490 */ 571, 568, 1841, 1289, 122, 165, 1265, 1761, 1263, 1932, + /* 500 */ 33, 32, 663, 1602, 40, 38, 36, 35, 34, 317, + /* 510 */ 168, 315, 314, 1485, 473, 351, 1342, 1343, 475, 1268, + /* 520 */ 1269, 1291, 1317, 1318, 1320, 1321, 1322, 1323, 1324, 1325, + /* 530 */ 573, 569, 1333, 1334, 1336, 1337, 1338, 1339, 1341, 1344, + /* 540 */ 474, 1010, 1011, 33, 32, 1460, 1364, 40, 38, 36, + /* 550 */ 35, 34, 168, 168, 1761, 526, 1935, 1592, 377, 146, + /* 560 */ 376, 1265, 63, 1263, 26, 1532, 382, 168, 1610, 165, + /* 570 */ 33, 32, 217, 1932, 40, 38, 36, 35, 34, 218, + /* 580 */ 1484, 1791, 1414, 1607, 1268, 1269, 1594, 1317, 1318, 1320, + /* 590 */ 1321, 1322, 1323, 1324, 1325, 573, 569, 1333, 1334, 1336, + /* 600 */ 1337, 1338, 1339, 1341, 1344, 39, 37, 77, 27, 1809, + /* 610 */ 498, 1884, 63, 339, 78, 1264, 168, 578, 1369, 1483, + /* 620 */ 505, 1761, 1761, 373, 577, 1302, 1340, 28, 1262, 482, + /* 630 */ 481, 1482, 1459, 33, 32, 1881, 123, 40, 38, 36, + /* 640 */ 35, 34, 375, 371, 438, 1590, 477, 480, 1823, 1335, + /* 650 */ 1287, 1935, 96, 1792, 580, 1794, 1795, 576, 253, 571, + /* 660 */ 1761, 1270, 1869, 513, 165, 1481, 1868, 1865, 1932, 1081, + /* 670 */ 33, 32, 1761, 1706, 40, 38, 36, 35, 34, 666, + /* 680 */ 33, 32, 9, 526, 40, 38, 36, 35, 34, 1478, + /* 690 */ 1477, 33, 32, 268, 383, 40, 38, 36, 35, 34, + /* 700 */ 168, 1704, 1083, 300, 663, 432, 1761, 157, 436, 1698, + /* 710 */ 214, 1607, 656, 652, 648, 644, 266, 1582, 1342, 1343, + /* 720 */ 176, 33, 32, 307, 572, 40, 38, 36, 35, 34, + /* 730 */ 1761, 1761, 39, 37, 526, 604, 526, 302, 1476, 1287, + /* 740 */ 339, 549, 1264, 526, 307, 389, 412, 404, 92, 424, + /* 750 */ 168, 231, 1302, 1340, 405, 1262, 440, 1585, 74, 436, + /* 760 */ 1362, 1407, 1607, 1265, 1607, 1263, 397, 1289, 425, 127, + /* 770 */ 399, 1607, 1475, 1703, 1779, 300, 1335, 1889, 1396, 1761, + /* 780 */ 1603, 1362, 44, 4, 523, 1775, 1268, 1269, 1270, 1317, + /* 790 */ 1318, 1320, 1321, 1322, 1323, 1324, 1325, 573, 569, 1333, + /* 800 */ 1334, 1336, 1337, 1338, 1339, 1341, 1344, 390, 125, 2, + /* 810 */ 1771, 1777, 334, 1761, 1363, 7, 220, 450, 611, 386, + /* 820 */ 90, 526, 571, 163, 1877, 1878, 1659, 1882, 1424, 145, + /* 830 */ 87, 663, 448, 312, 1236, 1363, 213, 137, 136, 608, + /* 840 */ 607, 606, 1657, 1480, 1884, 1342, 1343, 423, 1474, 1607, /* 850 */ 418, 417, 416, 415, 414, 411, 410, 409, 408, 407, - /* 860 */ 403, 402, 401, 400, 394, 393, 392, 391, 1879, 388, - /* 870 */ 387, 534, 1421, 1422, 1424, 1425, 29, 337, 1356, 1357, - /* 880 */ 1358, 1359, 1360, 1364, 1365, 1366, 1367, 1349, 61, 1760, - /* 890 */ 1264, 608, 1262, 1288, 1649, 1934, 1399, 29, 337, 1356, - /* 900 */ 1357, 1358, 1359, 1360, 1364, 1365, 1366, 1367, 166, 1582, - /* 910 */ 1790, 1472, 1931, 1267, 1268, 1471, 1316, 1317, 1319, 1320, - /* 920 */ 1321, 1322, 1323, 1324, 572, 568, 1332, 1333, 1335, 1336, - /* 930 */ 1337, 1338, 1340, 1343, 622, 147, 1578, 1790, 1808, 525, - /* 940 */ 279, 610, 609, 256, 1318, 1649, 577, 1883, 1470, 1469, - /* 950 */ 449, 1760, 1760, 576, 277, 60, 1760, 475, 59, 1291, - /* 960 */ 137, 136, 607, 606, 605, 1808, 553, 1606, 1288, 612, - /* 970 */ 1567, 1878, 135, 577, 181, 429, 427, 1822, 1760, 474, - /* 980 */ 576, 94, 1791, 579, 1793, 1794, 575, 535, 570, 1760, - /* 990 */ 1760, 1868, 1779, 553, 468, 306, 1864, 273, 53, 509, - /* 1000 */ 1636, 1658, 1395, 1774, 1822, 525, 63, 1934, 94, 1791, - /* 1010 */ 579, 1793, 1794, 575, 525, 570, 1603, 1657, 1868, 54, - /* 1020 */ 167, 1747, 306, 1864, 1931, 1735, 1518, 202, 1770, 1776, - /* 1030 */ 200, 336, 335, 1606, 1934, 1461, 1462, 557, 525, 525, - /* 1040 */ 570, 1277, 1606, 1272, 93, 525, 525, 165, 483, 506, - /* 1050 */ 510, 1931, 1339, 560, 1270, 326, 228, 521, 525, 204, - /* 1060 */ 525, 1790, 203, 146, 499, 525, 1606, 1606, 361, 523, - /* 1070 */ 1318, 524, 1608, 1606, 1606, 1334, 262, 41, 222, 68, - /* 1080 */ 67, 381, 342, 525, 172, 1271, 1606, 1269, 1606, 1808, - /* 1090 */ 146, 131, 245, 1606, 346, 206, 233, 577, 205, 1608, - /* 1100 */ 301, 566, 1760, 369, 576, 367, 363, 359, 356, 353, - /* 1110 */ 345, 1606, 1781, 208, 134, 135, 207, 1809, 146, 1513, - /* 1120 */ 1398, 1511, 51, 1790, 1212, 226, 237, 1608, 1822, 555, - /* 1130 */ 565, 51, 95, 1791, 579, 1793, 1794, 575, 518, 570, - /* 1140 */ 41, 485, 1868, 488, 168, 1318, 330, 1864, 1947, 11, - /* 1150 */ 10, 1808, 615, 41, 616, 1783, 350, 1902, 583, 577, - /* 1160 */ 134, 230, 1111, 1502, 1760, 1646, 576, 135, 119, 1420, - /* 1170 */ 134, 1898, 549, 240, 1068, 1790, 1066, 255, 1369, 250, - /* 1180 */ 1275, 258, 260, 3, 5, 355, 313, 1325, 1049, 1278, - /* 1190 */ 1822, 1273, 360, 1228, 95, 1791, 579, 1793, 1794, 575, - /* 1200 */ 272, 570, 269, 1808, 1868, 1139, 1507, 1143, 330, 1864, - /* 1210 */ 1947, 577, 1281, 1283, 1150, 1148, 1760, 138, 576, 1925, - /* 1220 */ 175, 1050, 1274, 1286, 568, 1332, 1333, 1335, 1336, 1337, - /* 1230 */ 1338, 1790, 385, 1353, 406, 1699, 413, 421, 420, 1292, - /* 1240 */ 558, 1790, 1822, 422, 426, 431, 95, 1791, 579, 1793, - /* 1250 */ 1794, 575, 428, 570, 657, 439, 1868, 430, 561, 1808, - /* 1260 */ 330, 1864, 1947, 1294, 442, 443, 184, 577, 1293, 1808, - /* 1270 */ 186, 1887, 1760, 1295, 576, 444, 445, 577, 189, 447, - /* 1280 */ 191, 72, 1760, 73, 576, 451, 470, 553, 195, 472, - /* 1290 */ 1790, 304, 1596, 199, 118, 1592, 1740, 553, 1822, 501, - /* 1300 */ 201, 140, 286, 1791, 579, 1793, 1794, 575, 1822, 570, - /* 1310 */ 141, 1594, 286, 1791, 579, 1793, 1794, 575, 1808, 570, - /* 1320 */ 1590, 142, 143, 212, 270, 500, 577, 215, 1934, 507, - /* 1330 */ 504, 1760, 511, 576, 322, 219, 533, 514, 1934, 132, - /* 1340 */ 1739, 167, 1709, 519, 516, 1931, 133, 324, 81, 520, - /* 1350 */ 1790, 165, 1291, 529, 271, 1931, 83, 1822, 1607, 235, - /* 1360 */ 1790, 96, 1791, 579, 1793, 1794, 575, 1899, 570, 536, - /* 1370 */ 239, 1868, 531, 1909, 6, 564, 1864, 532, 1808, 545, - /* 1380 */ 329, 1908, 539, 530, 528, 244, 577, 1890, 1808, 527, - /* 1390 */ 1395, 1760, 1290, 576, 154, 126, 577, 249, 562, 559, - /* 1400 */ 246, 1760, 48, 576, 1884, 247, 331, 248, 85, 1790, - /* 1410 */ 581, 1650, 1579, 265, 274, 658, 659, 1822, 1930, 661, - /* 1420 */ 52, 149, 1791, 579, 1793, 1794, 575, 1822, 570, 1950, - /* 1430 */ 153, 96, 1791, 579, 1793, 1794, 575, 1808, 570, 556, - /* 1440 */ 1754, 1868, 323, 287, 297, 577, 1865, 1849, 296, 254, - /* 1450 */ 1760, 276, 576, 563, 1753, 278, 257, 259, 65, 1752, - /* 1460 */ 1790, 1751, 66, 1748, 357, 554, 1948, 358, 1255, 1256, - /* 1470 */ 171, 362, 1746, 364, 365, 366, 1822, 1745, 1744, 368, - /* 1480 */ 295, 1791, 579, 1793, 1794, 575, 370, 570, 1808, 1743, - /* 1490 */ 372, 1742, 374, 526, 1231, 1230, 577, 1720, 1719, 379, - /* 1500 */ 380, 1760, 1200, 576, 1718, 1717, 1692, 129, 1691, 1690, - /* 1510 */ 1689, 69, 1790, 1688, 1687, 1686, 1685, 1684, 395, 396, - /* 1520 */ 1683, 398, 1790, 130, 1668, 1667, 1666, 1822, 1682, 1681, - /* 1530 */ 1680, 295, 1791, 579, 1793, 1794, 575, 1679, 570, 1790, - /* 1540 */ 1808, 1678, 1677, 1676, 1675, 1674, 1673, 1672, 577, 1671, - /* 1550 */ 1808, 1670, 1669, 1760, 1665, 576, 1664, 1663, 577, 1662, - /* 1560 */ 1202, 1661, 1660, 1760, 1659, 576, 1533, 1808, 179, 1532, - /* 1570 */ 1530, 1498, 120, 182, 180, 574, 1497, 158, 435, 1822, - /* 1580 */ 1760, 1012, 576, 290, 1791, 579, 1793, 1794, 575, 1822, - /* 1590 */ 570, 190, 1011, 149, 1791, 579, 1793, 1794, 575, 1790, - /* 1600 */ 570, 437, 1733, 183, 121, 1727, 1822, 1716, 1715, 1701, - /* 1610 */ 294, 1791, 579, 1793, 1794, 575, 1790, 570, 188, 1841, - /* 1620 */ 1585, 544, 1042, 1529, 1527, 452, 454, 1808, 1525, 453, - /* 1630 */ 456, 457, 338, 458, 1523, 577, 460, 462, 1949, 461, - /* 1640 */ 1760, 1521, 576, 465, 1808, 464, 1510, 1509, 1494, 340, - /* 1650 */ 466, 1587, 577, 1154, 1153, 1586, 50, 1760, 630, 576, - /* 1660 */ 1079, 1076, 632, 1519, 198, 1075, 1822, 1074, 1514, 1512, - /* 1670 */ 295, 1791, 579, 1793, 1794, 575, 318, 570, 319, 320, - /* 1680 */ 486, 1493, 1492, 1822, 1790, 489, 197, 295, 1791, 579, - /* 1690 */ 1793, 1794, 575, 491, 570, 1491, 493, 495, 97, 1732, - /* 1700 */ 152, 1237, 1790, 1726, 216, 467, 463, 459, 455, 196, - /* 1710 */ 56, 502, 1808, 144, 1714, 1712, 1713, 1711, 1710, 221, - /* 1720 */ 577, 1247, 15, 1708, 227, 1760, 79, 576, 1700, 503, - /* 1730 */ 1808, 321, 508, 80, 232, 517, 229, 87, 577, 41, - /* 1740 */ 47, 75, 16, 1760, 194, 576, 243, 242, 82, 25, - /* 1750 */ 17, 1822, 1435, 23, 234, 280, 1791, 579, 1793, 1794, - /* 1760 */ 575, 1790, 570, 236, 1417, 238, 1781, 1419, 151, 1822, - /* 1770 */ 1412, 252, 241, 281, 1791, 579, 1793, 1794, 575, 24, - /* 1780 */ 570, 86, 46, 1392, 1780, 18, 155, 1447, 1391, 1808, - /* 1790 */ 1446, 1452, 1441, 332, 1451, 1450, 333, 577, 10, 1279, - /* 1800 */ 45, 1825, 1760, 1329, 576, 1354, 193, 187, 13, 192, - /* 1810 */ 1790, 19, 1327, 446, 1326, 156, 569, 169, 31, 12, - /* 1820 */ 20, 1309, 578, 21, 582, 1140, 341, 1137, 1822, 185, - /* 1830 */ 586, 1790, 282, 1791, 579, 1793, 1794, 575, 1808, 570, - /* 1840 */ 584, 580, 587, 589, 1134, 590, 577, 1128, 592, 595, - /* 1850 */ 1117, 1760, 593, 576, 1126, 596, 1132, 1131, 1130, 1808, - /* 1860 */ 1129, 88, 89, 602, 263, 1149, 1145, 577, 62, 1040, - /* 1870 */ 611, 1071, 1760, 1070, 576, 1069, 1067, 1822, 1065, 1086, - /* 1880 */ 1064, 289, 1791, 579, 1793, 1794, 575, 1063, 570, 1790, - /* 1890 */ 620, 264, 1061, 1060, 1059, 1058, 1057, 1056, 1822, 1790, - /* 1900 */ 1055, 1083, 291, 1791, 579, 1793, 1794, 575, 1081, 570, - /* 1910 */ 1052, 1051, 1048, 1047, 1046, 1045, 1526, 1808, 640, 1524, - /* 1920 */ 642, 644, 1522, 646, 648, 577, 641, 1808, 1520, 645, - /* 1930 */ 1760, 652, 576, 650, 649, 577, 654, 1508, 653, 656, - /* 1940 */ 1760, 1002, 576, 1490, 664, 267, 660, 1465, 1265, 275, - /* 1950 */ 663, 1790, 1465, 1465, 1465, 1465, 1822, 1465, 1465, 1465, - /* 1960 */ 283, 1791, 579, 1793, 1794, 575, 1822, 570, 1790, 1465, - /* 1970 */ 292, 1791, 579, 1793, 1794, 575, 1465, 570, 1465, 1808, - /* 1980 */ 1465, 1465, 1465, 1465, 1465, 1465, 1465, 577, 1465, 1465, - /* 1990 */ 1465, 1465, 1760, 1465, 576, 1465, 1808, 1465, 1465, 1465, - /* 2000 */ 1465, 1465, 1465, 1465, 577, 1465, 1465, 1465, 1465, 1760, - /* 2010 */ 1465, 576, 1465, 1465, 1465, 1465, 1465, 1790, 1822, 1465, - /* 2020 */ 1465, 1465, 284, 1791, 579, 1793, 1794, 575, 1465, 570, - /* 2030 */ 1465, 1465, 1465, 1465, 1790, 1822, 1465, 1465, 1465, 293, - /* 2040 */ 1791, 579, 1793, 1794, 575, 1808, 570, 1465, 1465, 1465, - /* 2050 */ 1465, 1465, 1465, 577, 1465, 1465, 1465, 1465, 1760, 1465, - /* 2060 */ 576, 1465, 1808, 1465, 1465, 1465, 1465, 1465, 1465, 1465, - /* 2070 */ 577, 1465, 1465, 1465, 1465, 1760, 1465, 576, 1465, 1465, - /* 2080 */ 1465, 1465, 1465, 1790, 1822, 1465, 1465, 1465, 285, 1791, - /* 2090 */ 579, 1793, 1794, 575, 1465, 570, 1465, 1465, 1465, 1465, - /* 2100 */ 1465, 1822, 1465, 1465, 1465, 298, 1791, 579, 1793, 1794, - /* 2110 */ 575, 1808, 570, 1465, 1465, 1465, 1465, 1465, 1465, 577, - /* 2120 */ 1465, 1465, 1465, 1465, 1760, 1465, 576, 1465, 1465, 1465, - /* 2130 */ 1465, 1465, 1465, 1465, 1465, 1465, 1465, 1790, 1465, 1465, - /* 2140 */ 1465, 1465, 1465, 1465, 1465, 1465, 1465, 1790, 1465, 1465, - /* 2150 */ 1822, 1465, 1465, 1465, 299, 1791, 579, 1793, 1794, 575, - /* 2160 */ 1465, 570, 1465, 1465, 1465, 1808, 1465, 1465, 1465, 1465, - /* 2170 */ 1465, 1465, 1465, 577, 1465, 1808, 1465, 1465, 1760, 1465, - /* 2180 */ 576, 1465, 1465, 577, 1465, 1465, 1465, 1465, 1760, 1465, - /* 2190 */ 576, 1465, 1465, 1465, 1465, 1465, 1465, 1465, 1790, 1465, - /* 2200 */ 1465, 1465, 1465, 1465, 1822, 1465, 1465, 1465, 1802, 1791, - /* 2210 */ 579, 1793, 1794, 575, 1822, 570, 1790, 1465, 1801, 1791, - /* 2220 */ 579, 1793, 1794, 575, 1465, 570, 1808, 1465, 1465, 1465, - /* 2230 */ 1465, 1465, 1465, 1465, 577, 1465, 1465, 1465, 1465, 1760, - /* 2240 */ 1465, 576, 1465, 1465, 1808, 1465, 1465, 1465, 1465, 1465, - /* 2250 */ 1465, 1465, 577, 1465, 1465, 1465, 1465, 1760, 1465, 576, - /* 2260 */ 1465, 1465, 1465, 1465, 1465, 1822, 1465, 1465, 1465, 1800, - /* 2270 */ 1791, 579, 1793, 1794, 575, 1790, 570, 1465, 1465, 1465, - /* 2280 */ 1465, 1465, 1465, 1822, 1465, 1465, 1465, 310, 1791, 579, - /* 2290 */ 1793, 1794, 575, 1465, 570, 1465, 1790, 1465, 1465, 1465, - /* 2300 */ 1465, 1465, 1465, 1808, 1465, 1465, 1465, 1465, 1465, 1465, - /* 2310 */ 1465, 577, 1465, 1465, 1465, 1465, 1760, 1465, 576, 1465, - /* 2320 */ 1465, 1465, 1465, 1465, 1808, 1465, 1465, 1465, 1465, 1465, - /* 2330 */ 1465, 1465, 577, 1465, 1465, 1465, 1465, 1760, 1465, 576, - /* 2340 */ 1465, 1465, 1822, 1465, 1465, 1465, 309, 1791, 579, 1793, - /* 2350 */ 1794, 575, 1790, 570, 1465, 1465, 1465, 1465, 1465, 1465, - /* 2360 */ 1465, 1465, 1790, 1822, 1465, 1465, 1465, 311, 1791, 579, - /* 2370 */ 1793, 1794, 575, 1465, 570, 1465, 1465, 1465, 1465, 1465, - /* 2380 */ 1808, 1465, 1465, 1465, 1465, 1465, 1465, 1465, 577, 1465, - /* 2390 */ 1808, 1465, 1465, 1760, 548, 576, 1465, 1465, 577, 1465, - /* 2400 */ 1465, 1465, 1465, 1760, 1465, 576, 1465, 1465, 1465, 1465, - /* 2410 */ 1465, 1465, 1465, 1465, 1465, 1465, 1465, 1465, 1465, 1822, - /* 2420 */ 1465, 1465, 127, 308, 1791, 579, 1793, 1794, 575, 1822, - /* 2430 */ 570, 1465, 1465, 288, 1791, 579, 1793, 1794, 575, 1465, - /* 2440 */ 570, 548, 553, 1465, 1465, 1465, 1465, 1465, 1465, 1465, - /* 2450 */ 1465, 1465, 1465, 1465, 1465, 1465, 1465, 1465, 1465, 1465, - /* 2460 */ 1465, 125, 1465, 1465, 1465, 1465, 1465, 1465, 1465, 127, - /* 2470 */ 1465, 1465, 1465, 1465, 1465, 1465, 251, 1876, 547, 1465, - /* 2480 */ 546, 1465, 1465, 1934, 1465, 1465, 1465, 1465, 1465, 553, - /* 2490 */ 1465, 1465, 1465, 1465, 1465, 1465, 167, 1465, 1465, 1465, - /* 2500 */ 1931, 1465, 1465, 1465, 1465, 1465, 1465, 1465, 125, 1465, - /* 2510 */ 1465, 1465, 1465, 1465, 1465, 1465, 1465, 1465, 1465, 1465, - /* 2520 */ 1465, 1465, 1465, 251, 1876, 547, 1465, 546, 1465, 1465, - /* 2530 */ 1934, 1465, 1465, 1465, 1465, 1465, 1465, 1465, 1465, 1465, - /* 2540 */ 1465, 1465, 1465, 165, 1465, 1465, 1465, 1931, + /* 860 */ 403, 402, 401, 400, 394, 393, 392, 391, 1880, 388, + /* 870 */ 387, 535, 1422, 1423, 1425, 1426, 29, 337, 1357, 1358, + /* 880 */ 1359, 1360, 1361, 1365, 1366, 1367, 1368, 1350, 61, 1761, + /* 890 */ 1265, 609, 1263, 1289, 1650, 1935, 1400, 29, 337, 1357, + /* 900 */ 1358, 1359, 1360, 1361, 1365, 1366, 1367, 1368, 166, 1583, + /* 910 */ 1791, 1473, 1932, 1268, 1269, 1472, 1317, 1318, 1320, 1321, + /* 920 */ 1322, 1323, 1324, 1325, 573, 569, 1333, 1334, 1336, 1337, + /* 930 */ 1338, 1339, 1341, 1344, 623, 147, 1579, 1791, 1809, 526, + /* 940 */ 279, 611, 610, 256, 1319, 1650, 578, 1884, 1471, 1470, + /* 950 */ 449, 1761, 1761, 577, 277, 60, 1761, 475, 59, 1292, + /* 960 */ 137, 136, 608, 607, 606, 1809, 554, 1607, 1289, 613, + /* 970 */ 1568, 1879, 135, 578, 181, 429, 427, 1823, 1761, 474, + /* 980 */ 577, 94, 1792, 580, 1794, 1795, 576, 536, 571, 1761, + /* 990 */ 1761, 1869, 1780, 554, 468, 306, 1865, 273, 53, 509, + /* 1000 */ 1637, 1659, 1396, 1775, 1823, 526, 63, 1935, 94, 1792, + /* 1010 */ 580, 1794, 1795, 576, 526, 571, 1604, 1658, 1869, 54, + /* 1020 */ 167, 1748, 306, 1865, 1932, 1736, 1519, 202, 1771, 1777, + /* 1030 */ 200, 336, 335, 1607, 1935, 1462, 1463, 558, 526, 526, + /* 1040 */ 571, 1278, 1607, 1273, 93, 526, 526, 165, 483, 506, + /* 1050 */ 510, 1932, 1340, 561, 1271, 326, 228, 522, 526, 204, + /* 1060 */ 526, 1791, 203, 146, 499, 526, 1607, 1607, 361, 524, + /* 1070 */ 1319, 525, 1609, 1607, 1607, 1335, 262, 41, 222, 68, + /* 1080 */ 67, 381, 342, 526, 172, 1272, 1607, 1270, 1607, 1809, + /* 1090 */ 146, 131, 245, 1607, 346, 206, 233, 578, 205, 1609, + /* 1100 */ 301, 567, 1761, 369, 577, 367, 363, 359, 356, 353, + /* 1110 */ 345, 1607, 1782, 208, 134, 135, 207, 1810, 146, 1514, + /* 1120 */ 1399, 1512, 51, 1791, 1213, 226, 237, 1609, 1823, 556, + /* 1130 */ 566, 51, 95, 1792, 580, 1794, 1795, 576, 519, 571, + /* 1140 */ 41, 485, 1869, 488, 168, 1319, 330, 1865, 1948, 11, + /* 1150 */ 10, 1809, 616, 41, 617, 1784, 350, 1903, 584, 578, + /* 1160 */ 134, 230, 1112, 1503, 1761, 1647, 577, 135, 119, 1421, + /* 1170 */ 134, 1899, 550, 240, 1069, 1791, 1067, 255, 1370, 250, + /* 1180 */ 1276, 258, 260, 3, 5, 355, 313, 1326, 1050, 1279, + /* 1190 */ 1823, 1274, 360, 1229, 95, 1792, 580, 1794, 1795, 576, + /* 1200 */ 272, 571, 269, 1809, 1869, 1140, 1508, 1144, 330, 1865, + /* 1210 */ 1948, 578, 1282, 1284, 1151, 1149, 1761, 138, 577, 1926, + /* 1220 */ 175, 1051, 1275, 1287, 569, 1333, 1334, 1336, 1337, 1338, + /* 1230 */ 1339, 1791, 385, 1354, 406, 1700, 413, 421, 420, 1293, + /* 1240 */ 559, 1791, 1823, 422, 426, 431, 95, 1792, 580, 1794, + /* 1250 */ 1795, 576, 428, 571, 658, 439, 1869, 430, 562, 1809, + /* 1260 */ 330, 1865, 1948, 1295, 442, 443, 184, 578, 1294, 1809, + /* 1270 */ 186, 1888, 1761, 1296, 577, 444, 445, 578, 189, 447, + /* 1280 */ 191, 72, 1761, 73, 577, 451, 470, 554, 195, 472, + /* 1290 */ 1791, 304, 1597, 199, 118, 1593, 1741, 554, 1823, 501, + /* 1300 */ 201, 140, 286, 1792, 580, 1794, 1795, 576, 1823, 571, + /* 1310 */ 141, 1595, 286, 1792, 580, 1794, 1795, 576, 1809, 571, + /* 1320 */ 1591, 142, 143, 212, 270, 500, 578, 215, 1935, 507, + /* 1330 */ 504, 1761, 511, 577, 322, 219, 534, 514, 1935, 132, + /* 1340 */ 1740, 167, 1710, 520, 517, 1932, 133, 324, 81, 521, + /* 1350 */ 1791, 165, 1292, 530, 271, 1932, 83, 1823, 1608, 235, + /* 1360 */ 1791, 96, 1792, 580, 1794, 1795, 576, 1900, 571, 537, + /* 1370 */ 239, 1869, 532, 1910, 6, 565, 1865, 533, 1809, 546, + /* 1380 */ 329, 1909, 540, 531, 529, 244, 578, 1891, 1809, 528, + /* 1390 */ 1396, 1761, 1291, 577, 154, 126, 578, 249, 563, 560, + /* 1400 */ 246, 1761, 48, 577, 1885, 247, 331, 248, 85, 1791, + /* 1410 */ 582, 1651, 1580, 265, 274, 659, 660, 1823, 1931, 662, + /* 1420 */ 52, 149, 1792, 580, 1794, 1795, 576, 1823, 571, 1951, + /* 1430 */ 153, 96, 1792, 580, 1794, 1795, 576, 1809, 571, 557, + /* 1440 */ 1755, 1869, 323, 287, 297, 578, 1866, 1850, 296, 254, + /* 1450 */ 1761, 276, 577, 564, 1754, 278, 257, 259, 65, 1753, + /* 1460 */ 1791, 1752, 66, 1749, 357, 555, 1949, 358, 1256, 1257, + /* 1470 */ 171, 362, 1747, 364, 365, 366, 1823, 1746, 1745, 368, + /* 1480 */ 295, 1792, 580, 1794, 1795, 576, 370, 571, 1809, 1744, + /* 1490 */ 372, 1743, 374, 527, 1232, 1231, 578, 1721, 1720, 379, + /* 1500 */ 380, 1761, 1201, 577, 1719, 1718, 1693, 129, 1692, 1691, + /* 1510 */ 1690, 69, 1791, 1689, 1688, 1687, 1686, 1685, 395, 396, + /* 1520 */ 1684, 398, 1791, 130, 1669, 1668, 1667, 1823, 1683, 1682, + /* 1530 */ 1681, 295, 1792, 580, 1794, 1795, 576, 1680, 571, 1791, + /* 1540 */ 1809, 1679, 1678, 1677, 1676, 1675, 1674, 1673, 578, 1672, + /* 1550 */ 1809, 1671, 1670, 1761, 1666, 577, 1665, 1664, 578, 1663, + /* 1560 */ 1203, 1662, 1661, 1761, 1660, 577, 1534, 1809, 179, 1533, + /* 1570 */ 1531, 1499, 120, 182, 180, 575, 1498, 158, 435, 1823, + /* 1580 */ 1761, 1013, 577, 290, 1792, 580, 1794, 1795, 576, 1823, + /* 1590 */ 571, 190, 1012, 149, 1792, 580, 1794, 1795, 576, 1791, + /* 1600 */ 571, 437, 1734, 183, 121, 1728, 1823, 1717, 1716, 1702, + /* 1610 */ 294, 1792, 580, 1794, 1795, 576, 1791, 571, 188, 1842, + /* 1620 */ 1586, 545, 1043, 1530, 1528, 452, 454, 1809, 1526, 453, + /* 1630 */ 456, 457, 338, 458, 1524, 578, 460, 462, 1950, 461, + /* 1640 */ 1761, 1522, 577, 465, 1809, 464, 1511, 1510, 1495, 340, + /* 1650 */ 466, 1588, 578, 1155, 1154, 1587, 50, 1761, 631, 577, + /* 1660 */ 1080, 1077, 633, 1520, 198, 1076, 1823, 1075, 1515, 1513, + /* 1670 */ 295, 1792, 580, 1794, 1795, 576, 318, 571, 319, 320, + /* 1680 */ 486, 1494, 1493, 1823, 1791, 489, 197, 295, 1792, 580, + /* 1690 */ 1794, 1795, 576, 491, 571, 1492, 493, 495, 97, 1733, + /* 1700 */ 152, 1238, 1791, 1727, 216, 467, 463, 459, 455, 196, + /* 1710 */ 56, 502, 1809, 144, 1715, 1713, 1714, 1712, 1711, 221, + /* 1720 */ 578, 1248, 15, 1709, 227, 1761, 79, 577, 1701, 503, + /* 1730 */ 1809, 321, 508, 80, 232, 518, 41, 87, 578, 229, + /* 1740 */ 47, 75, 16, 1761, 194, 577, 243, 242, 82, 25, + /* 1750 */ 17, 1823, 1436, 23, 234, 280, 1792, 580, 1794, 1795, + /* 1760 */ 576, 1791, 571, 236, 1418, 515, 238, 1782, 151, 1823, + /* 1770 */ 1420, 252, 241, 281, 1792, 580, 1794, 1795, 576, 24, + /* 1780 */ 571, 1413, 1393, 46, 1781, 86, 18, 155, 1392, 1809, + /* 1790 */ 1448, 1453, 1442, 1447, 332, 1452, 1451, 578, 333, 10, + /* 1800 */ 45, 1280, 1761, 1330, 577, 1355, 193, 187, 13, 192, + /* 1810 */ 1791, 19, 1328, 446, 1327, 156, 1826, 169, 570, 31, + /* 1820 */ 12, 20, 1310, 21, 583, 1141, 341, 1138, 1823, 185, + /* 1830 */ 587, 1791, 282, 1792, 580, 1794, 1795, 576, 1809, 571, + /* 1840 */ 585, 588, 581, 1135, 579, 590, 578, 1129, 593, 596, + /* 1850 */ 1118, 1761, 1127, 577, 591, 594, 597, 1133, 1132, 1809, + /* 1860 */ 1131, 1130, 88, 89, 263, 603, 1150, 578, 1146, 62, + /* 1870 */ 1041, 1072, 1761, 612, 577, 1071, 1070, 1823, 1068, 1066, + /* 1880 */ 1065, 289, 1792, 580, 1794, 1795, 576, 1064, 571, 1791, + /* 1890 */ 1087, 621, 264, 1062, 1061, 1060, 1059, 1058, 1823, 1791, + /* 1900 */ 1057, 1056, 291, 1792, 580, 1794, 1795, 576, 1047, 571, + /* 1910 */ 1084, 1082, 1053, 1052, 1049, 1048, 1046, 1809, 1527, 641, + /* 1920 */ 1525, 642, 643, 645, 647, 578, 1523, 1809, 649, 646, + /* 1930 */ 1761, 651, 577, 1521, 650, 578, 653, 655, 654, 1509, + /* 1940 */ 1761, 657, 577, 1491, 1003, 267, 661, 1466, 1466, 1266, + /* 1950 */ 275, 1791, 664, 1466, 665, 1466, 1823, 1466, 1466, 1466, + /* 1960 */ 283, 1792, 580, 1794, 1795, 576, 1823, 571, 1791, 1466, + /* 1970 */ 292, 1792, 580, 1794, 1795, 576, 1466, 571, 1466, 1809, + /* 1980 */ 1466, 1466, 1466, 1466, 1466, 1466, 1466, 578, 1466, 1466, + /* 1990 */ 1466, 1466, 1761, 1466, 577, 1466, 1809, 1466, 1466, 1466, + /* 2000 */ 1466, 1466, 1466, 1466, 578, 1466, 1466, 1466, 1466, 1761, + /* 2010 */ 1466, 577, 1466, 1466, 1466, 1466, 1466, 1791, 1823, 1466, + /* 2020 */ 1466, 1466, 284, 1792, 580, 1794, 1795, 576, 1466, 571, + /* 2030 */ 1466, 1466, 1466, 1466, 1791, 1823, 1466, 1466, 1466, 293, + /* 2040 */ 1792, 580, 1794, 1795, 576, 1809, 571, 1466, 1466, 1466, + /* 2050 */ 1466, 1466, 1466, 578, 1466, 1466, 1466, 1466, 1761, 1466, + /* 2060 */ 577, 1466, 1809, 1466, 1466, 1466, 1466, 1466, 1466, 1466, + /* 2070 */ 578, 1466, 1466, 1466, 1466, 1761, 1466, 577, 1466, 1466, + /* 2080 */ 1466, 1466, 1466, 1791, 1823, 1466, 1466, 1466, 285, 1792, + /* 2090 */ 580, 1794, 1795, 576, 1466, 571, 1466, 1466, 1466, 1466, + /* 2100 */ 1466, 1823, 1466, 1466, 1466, 298, 1792, 580, 1794, 1795, + /* 2110 */ 576, 1809, 571, 1466, 1466, 1466, 1466, 1466, 1466, 578, + /* 2120 */ 1466, 1466, 1466, 1466, 1761, 1466, 577, 1466, 1466, 1466, + /* 2130 */ 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1791, 1466, 1466, + /* 2140 */ 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1791, 1466, 1466, + /* 2150 */ 1823, 1466, 1466, 1466, 299, 1792, 580, 1794, 1795, 576, + /* 2160 */ 1466, 571, 1466, 1466, 1466, 1809, 1466, 1466, 1466, 1466, + /* 2170 */ 1466, 1466, 1466, 578, 1466, 1809, 1466, 1466, 1761, 1466, + /* 2180 */ 577, 1466, 1466, 578, 1466, 1466, 1466, 1466, 1761, 1466, + /* 2190 */ 577, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1791, 1466, + /* 2200 */ 1466, 1466, 1466, 1466, 1823, 1466, 1466, 1466, 1803, 1792, + /* 2210 */ 580, 1794, 1795, 576, 1823, 571, 1791, 1466, 1802, 1792, + /* 2220 */ 580, 1794, 1795, 576, 1466, 571, 1809, 1466, 1466, 1466, + /* 2230 */ 1466, 1466, 1466, 1466, 578, 1466, 1466, 1466, 1466, 1761, + /* 2240 */ 1466, 577, 1466, 1466, 1809, 1466, 1466, 1466, 1466, 1466, + /* 2250 */ 1466, 1466, 578, 1466, 1466, 1466, 1466, 1761, 1466, 577, + /* 2260 */ 1466, 1466, 1466, 1466, 1466, 1823, 1466, 1466, 1466, 1801, + /* 2270 */ 1792, 580, 1794, 1795, 576, 1791, 571, 1466, 1466, 1466, + /* 2280 */ 1466, 1466, 1466, 1823, 1466, 1466, 1466, 310, 1792, 580, + /* 2290 */ 1794, 1795, 576, 1466, 571, 1466, 1791, 1466, 1466, 1466, + /* 2300 */ 1466, 1466, 1466, 1809, 1466, 1466, 1466, 1466, 1466, 1466, + /* 2310 */ 1466, 578, 1466, 1466, 1466, 1466, 1761, 1466, 577, 1466, + /* 2320 */ 1466, 1466, 1466, 1466, 1809, 1466, 1466, 1466, 1466, 1466, + /* 2330 */ 1466, 1466, 578, 1466, 1466, 1466, 1466, 1761, 1466, 577, + /* 2340 */ 1466, 1466, 1823, 1466, 1466, 1466, 309, 1792, 580, 1794, + /* 2350 */ 1795, 576, 1791, 571, 1466, 1466, 1466, 1466, 1466, 1466, + /* 2360 */ 1466, 1466, 1791, 1823, 1466, 1466, 1466, 311, 1792, 580, + /* 2370 */ 1794, 1795, 576, 1466, 571, 1466, 1466, 1466, 1466, 1466, + /* 2380 */ 1809, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 578, 1466, + /* 2390 */ 1809, 1466, 1466, 1761, 549, 577, 1466, 1466, 578, 1466, + /* 2400 */ 1466, 1466, 1466, 1761, 1466, 577, 1466, 1466, 1466, 1466, + /* 2410 */ 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1823, + /* 2420 */ 1466, 1466, 127, 308, 1792, 580, 1794, 1795, 576, 1823, + /* 2430 */ 571, 1466, 1466, 288, 1792, 580, 1794, 1795, 576, 1466, + /* 2440 */ 571, 549, 554, 1466, 1466, 1466, 1466, 1466, 1466, 1466, + /* 2450 */ 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, + /* 2460 */ 1466, 125, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 127, + /* 2470 */ 1466, 1466, 1466, 1466, 1466, 1466, 251, 1877, 548, 1466, + /* 2480 */ 547, 1466, 1466, 1935, 1466, 1466, 1466, 1466, 1466, 554, + /* 2490 */ 1466, 1466, 1466, 1466, 1466, 1466, 167, 1466, 1466, 1466, + /* 2500 */ 1932, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 125, 1466, + /* 2510 */ 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, + /* 2520 */ 1466, 1466, 1466, 251, 1877, 548, 1466, 547, 1466, 1466, + /* 2530 */ 1935, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, 1466, + /* 2540 */ 1466, 1466, 1466, 165, 1466, 1466, 1466, 1932, }; static const YYCODETYPE yy_lookahead[] = { /* 0 */ 316, 390, 391, 316, 316, 312, 316, 314, 315, 1, @@ -647,30 +647,30 @@ static const YYCODETYPE yy_lookahead[] = { /* 1690 */ 382, 383, 384, 35, 386, 0, 35, 22, 20, 0, /* 1700 */ 47, 35, 308, 0, 154, 52, 53, 54, 55, 56, /* 1710 */ 157, 22, 336, 173, 0, 0, 0, 0, 0, 90, - /* 1720 */ 344, 182, 89, 0, 89, 349, 89, 351, 0, 157, - /* 1730 */ 336, 157, 159, 39, 46, 155, 153, 99, 344, 43, + /* 1720 */ 344, 35, 89, 0, 89, 349, 89, 351, 0, 157, + /* 1730 */ 336, 157, 159, 39, 46, 155, 43, 99, 344, 153, /* 1740 */ 43, 88, 231, 349, 91, 351, 46, 43, 89, 43, /* 1750 */ 231, 375, 90, 89, 89, 379, 380, 381, 382, 383, - /* 1760 */ 384, 308, 386, 90, 90, 89, 46, 90, 89, 375, + /* 1760 */ 384, 308, 386, 90, 90, 182, 89, 46, 89, 375, /* 1770 */ 90, 46, 89, 379, 380, 381, 382, 383, 384, 89, - /* 1780 */ 386, 89, 43, 90, 46, 43, 46, 35, 90, 336, - /* 1790 */ 35, 90, 90, 35, 35, 35, 35, 344, 2, 22, - /* 1800 */ 225, 89, 349, 90, 351, 193, 153, 154, 231, 156, + /* 1780 */ 386, 90, 90, 43, 46, 89, 43, 46, 90, 336, + /* 1790 */ 35, 90, 90, 35, 35, 35, 35, 344, 35, 2, + /* 1800 */ 225, 22, 349, 90, 351, 193, 153, 154, 231, 156, /* 1810 */ 308, 43, 90, 160, 90, 46, 89, 46, 89, 89, - /* 1820 */ 89, 22, 195, 89, 35, 90, 35, 90, 375, 176, + /* 1820 */ 89, 89, 22, 89, 35, 90, 35, 90, 375, 176, /* 1830 */ 35, 308, 379, 380, 381, 382, 383, 384, 336, 386, - /* 1840 */ 89, 100, 89, 35, 90, 89, 344, 90, 35, 35, - /* 1850 */ 22, 349, 89, 351, 90, 89, 113, 113, 113, 336, - /* 1860 */ 113, 89, 89, 101, 43, 35, 22, 344, 89, 62, - /* 1870 */ 61, 35, 349, 35, 351, 35, 35, 375, 35, 68, + /* 1840 */ 89, 89, 100, 90, 195, 35, 344, 90, 35, 35, + /* 1850 */ 22, 349, 90, 351, 89, 89, 89, 113, 113, 336, + /* 1860 */ 113, 113, 89, 89, 43, 101, 35, 344, 22, 89, + /* 1870 */ 62, 35, 349, 61, 351, 35, 35, 375, 35, 35, /* 1880 */ 35, 379, 380, 381, 382, 383, 384, 35, 386, 308, - /* 1890 */ 87, 43, 35, 35, 22, 35, 22, 35, 375, 308, - /* 1900 */ 35, 68, 379, 380, 381, 382, 383, 384, 35, 386, - /* 1910 */ 35, 35, 35, 35, 22, 35, 0, 336, 35, 0, - /* 1920 */ 39, 35, 0, 39, 35, 344, 47, 336, 0, 47, - /* 1930 */ 349, 35, 351, 39, 47, 344, 39, 0, 47, 35, - /* 1940 */ 349, 35, 351, 0, 20, 22, 21, 427, 22, 22, - /* 1950 */ 21, 308, 427, 427, 427, 427, 375, 427, 427, 427, + /* 1890 */ 68, 87, 43, 35, 35, 22, 35, 22, 375, 308, + /* 1900 */ 35, 35, 379, 380, 381, 382, 383, 384, 22, 386, + /* 1910 */ 68, 35, 35, 35, 35, 35, 35, 336, 0, 35, + /* 1920 */ 0, 47, 39, 35, 39, 344, 0, 336, 35, 47, + /* 1930 */ 349, 39, 351, 0, 47, 344, 35, 39, 47, 0, + /* 1940 */ 349, 35, 351, 0, 35, 22, 21, 427, 427, 22, + /* 1950 */ 22, 308, 21, 427, 20, 427, 375, 427, 427, 427, /* 1960 */ 379, 380, 381, 382, 383, 384, 375, 386, 308, 427, /* 1970 */ 379, 380, 381, 382, 383, 384, 427, 386, 427, 336, /* 1980 */ 427, 427, 427, 427, 427, 427, 427, 344, 427, 427, @@ -731,7 +731,7 @@ static const YYCODETYPE yy_lookahead[] = { /* 2530 */ 405, 427, 427, 427, 427, 427, 427, 427, 427, 427, /* 2540 */ 427, 427, 427, 418, 427, 427, 427, 422, }; -#define YY_SHIFT_COUNT (665) +#define YY_SHIFT_COUNT (666) #define YY_SHIFT_MIN (0) #define YY_SHIFT_MAX (1943) static const unsigned short int yy_shift_ofst[] = { @@ -786,22 +786,22 @@ static const unsigned short int yy_shift_ofst[] = { /* 480 */ 1626, 1630, 1645, 1663, 1654, 1668, 1656, 1631, 1669, 1657, /* 490 */ 1650, 1681, 1658, 1682, 1661, 1695, 1675, 1678, 1699, 1553, /* 500 */ 1666, 1703, 1540, 1689, 1572, 1550, 1714, 1715, 1574, 1573, - /* 510 */ 1716, 1717, 1718, 1633, 1629, 1539, 1723, 1635, 1580, 1637, - /* 520 */ 1728, 1694, 1583, 1659, 1638, 1688, 1696, 1511, 1664, 1662, - /* 530 */ 1665, 1673, 1674, 1676, 1697, 1677, 1679, 1683, 1690, 1680, - /* 540 */ 1704, 1700, 1720, 1692, 1706, 1519, 1693, 1698, 1725, 1575, - /* 550 */ 1739, 1738, 1740, 1701, 1742, 1577, 1702, 1752, 1755, 1758, - /* 560 */ 1759, 1760, 1761, 1702, 1796, 1777, 1612, 1768, 1712, 1713, - /* 570 */ 1727, 1722, 1729, 1724, 1769, 1730, 1731, 1771, 1799, 1627, - /* 580 */ 1734, 1741, 1735, 1789, 1791, 1751, 1737, 1795, 1753, 1754, - /* 590 */ 1808, 1756, 1757, 1813, 1763, 1764, 1814, 1766, 1743, 1744, - /* 600 */ 1745, 1747, 1828, 1762, 1772, 1773, 1830, 1779, 1821, 1821, - /* 610 */ 1844, 1807, 1809, 1836, 1838, 1840, 1841, 1843, 1845, 1852, - /* 620 */ 1811, 1803, 1848, 1857, 1858, 1872, 1860, 1874, 1862, 1865, - /* 630 */ 1833, 1615, 1873, 1619, 1875, 1876, 1877, 1878, 1892, 1880, - /* 640 */ 1916, 1883, 1879, 1881, 1919, 1886, 1882, 1884, 1922, 1889, - /* 650 */ 1887, 1894, 1928, 1896, 1891, 1897, 1937, 1904, 1906, 1943, - /* 660 */ 1923, 1925, 1926, 1927, 1929, 1924, + /* 510 */ 1716, 1717, 1718, 1633, 1629, 1686, 1583, 1723, 1635, 1580, + /* 520 */ 1637, 1728, 1694, 1586, 1659, 1638, 1688, 1693, 1511, 1664, + /* 530 */ 1662, 1665, 1673, 1674, 1677, 1697, 1680, 1679, 1683, 1690, + /* 540 */ 1691, 1704, 1700, 1721, 1696, 1706, 1519, 1692, 1698, 1725, + /* 550 */ 1575, 1740, 1738, 1741, 1701, 1743, 1577, 1702, 1755, 1758, + /* 560 */ 1759, 1760, 1761, 1763, 1702, 1797, 1779, 1612, 1768, 1727, + /* 570 */ 1713, 1729, 1722, 1730, 1724, 1769, 1731, 1732, 1771, 1800, + /* 580 */ 1649, 1734, 1742, 1735, 1789, 1791, 1751, 1737, 1795, 1752, + /* 590 */ 1753, 1810, 1765, 1757, 1813, 1766, 1762, 1814, 1767, 1744, + /* 600 */ 1745, 1747, 1748, 1828, 1764, 1773, 1774, 1831, 1780, 1821, + /* 610 */ 1821, 1846, 1808, 1812, 1836, 1840, 1841, 1843, 1844, 1845, + /* 620 */ 1852, 1822, 1804, 1849, 1858, 1859, 1873, 1861, 1875, 1865, + /* 630 */ 1866, 1842, 1615, 1876, 1619, 1877, 1878, 1879, 1880, 1886, + /* 640 */ 1881, 1918, 1884, 1874, 1883, 1920, 1888, 1882, 1885, 1926, + /* 650 */ 1893, 1887, 1892, 1933, 1901, 1891, 1898, 1939, 1906, 1909, + /* 660 */ 1943, 1923, 1925, 1927, 1928, 1931, 1934, }; #define YY_REDUCE_COUNT (275) #define YY_REDUCE_MIN (-389) @@ -837,73 +837,73 @@ static const short yy_reduce_ofst[] = { /* 270 */ 1068, 1113, 1114, 1118, 1132, 1149, }; static const YYACTIONTYPE yy_default[] = { - /* 0 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 10 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 20 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 30 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 40 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 50 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 60 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 70 */ 1463, 1463, 1463, 1463, 1463, 1537, 1463, 1463, 1463, 1463, - /* 80 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 90 */ 1463, 1463, 1535, 1693, 1463, 1870, 1463, 1463, 1463, 1463, - /* 100 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 110 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 120 */ 1463, 1463, 1537, 1463, 1535, 1882, 1882, 1882, 1463, 1463, - /* 130 */ 1463, 1463, 1736, 1736, 1463, 1463, 1463, 1463, 1635, 1463, - /* 140 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1728, 1463, 1951, - /* 150 */ 1463, 1463, 1463, 1734, 1905, 1463, 1463, 1463, 1463, 1588, - /* 160 */ 1897, 1874, 1888, 1875, 1872, 1936, 1936, 1936, 1891, 1463, - /* 170 */ 1901, 1463, 1721, 1698, 1463, 1463, 1698, 1695, 1695, 1463, - /* 180 */ 1463, 1463, 1463, 1463, 1463, 1537, 1463, 1537, 1463, 1463, - /* 190 */ 1537, 1463, 1537, 1537, 1537, 1463, 1537, 1463, 1463, 1463, - /* 200 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 210 */ 1463, 1463, 1463, 1535, 1730, 1463, 1535, 1463, 1463, 1463, - /* 220 */ 1535, 1910, 1463, 1463, 1463, 1463, 1910, 1463, 1463, 1535, - /* 230 */ 1463, 1535, 1463, 1463, 1463, 1912, 1910, 1463, 1463, 1912, - /* 240 */ 1910, 1463, 1463, 1463, 1924, 1920, 1912, 1928, 1926, 1903, - /* 250 */ 1901, 1888, 1463, 1463, 1942, 1938, 1954, 1942, 1938, 1942, - /* 260 */ 1938, 1463, 1604, 1463, 1463, 1463, 1535, 1495, 1463, 1723, - /* 270 */ 1736, 1638, 1638, 1638, 1538, 1468, 1463, 1463, 1463, 1463, - /* 280 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1807, 1923, - /* 290 */ 1922, 1846, 1845, 1844, 1842, 1806, 1463, 1600, 1805, 1804, - /* 300 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1798, 1799, - /* 310 */ 1797, 1796, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 320 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 330 */ 1871, 1463, 1939, 1943, 1463, 1463, 1463, 1463, 1463, 1782, - /* 340 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 350 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 360 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 370 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 380 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 390 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 400 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 410 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 420 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 430 */ 1463, 1463, 1463, 1463, 1500, 1463, 1463, 1463, 1463, 1463, - /* 440 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 450 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 460 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 470 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1572, 1571, - /* 480 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 490 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 500 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 510 */ 1463, 1463, 1463, 1463, 1463, 1463, 1740, 1463, 1463, 1463, - /* 520 */ 1463, 1463, 1463, 1463, 1463, 1463, 1904, 1463, 1463, 1463, - /* 530 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 540 */ 1463, 1463, 1782, 1463, 1921, 1463, 1881, 1877, 1463, 1463, - /* 550 */ 1873, 1781, 1463, 1463, 1937, 1463, 1463, 1463, 1463, 1463, - /* 560 */ 1463, 1463, 1463, 1463, 1866, 1463, 1463, 1839, 1824, 1463, - /* 570 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1792, - /* 580 */ 1463, 1463, 1463, 1463, 1463, 1632, 1463, 1463, 1463, 1463, - /* 590 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1617, 1615, - /* 600 */ 1614, 1613, 1463, 1610, 1463, 1463, 1463, 1463, 1641, 1640, - /* 610 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 620 */ 1463, 1463, 1556, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 630 */ 1463, 1548, 1463, 1547, 1463, 1463, 1463, 1463, 1463, 1463, - /* 640 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 650 */ 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, 1463, - /* 660 */ 1463, 1463, 1463, 1463, 1463, 1463, + /* 0 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 10 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 20 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 30 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 40 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 50 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 60 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 70 */ 1464, 1464, 1464, 1464, 1464, 1538, 1464, 1464, 1464, 1464, + /* 80 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 90 */ 1464, 1464, 1536, 1694, 1464, 1871, 1464, 1464, 1464, 1464, + /* 100 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 110 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 120 */ 1464, 1464, 1538, 1464, 1536, 1883, 1883, 1883, 1464, 1464, + /* 130 */ 1464, 1464, 1737, 1737, 1464, 1464, 1464, 1464, 1636, 1464, + /* 140 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1729, 1464, 1952, + /* 150 */ 1464, 1464, 1464, 1735, 1906, 1464, 1464, 1464, 1464, 1589, + /* 160 */ 1898, 1875, 1889, 1876, 1873, 1937, 1937, 1937, 1892, 1464, + /* 170 */ 1902, 1464, 1722, 1699, 1464, 1464, 1699, 1696, 1696, 1464, + /* 180 */ 1464, 1464, 1464, 1464, 1464, 1538, 1464, 1538, 1464, 1464, + /* 190 */ 1538, 1464, 1538, 1538, 1538, 1464, 1538, 1464, 1464, 1464, + /* 200 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 210 */ 1464, 1464, 1464, 1536, 1731, 1464, 1536, 1464, 1464, 1464, + /* 220 */ 1536, 1911, 1464, 1464, 1464, 1464, 1911, 1464, 1464, 1536, + /* 230 */ 1464, 1536, 1464, 1464, 1464, 1913, 1911, 1464, 1464, 1913, + /* 240 */ 1911, 1464, 1464, 1464, 1925, 1921, 1913, 1929, 1927, 1904, + /* 250 */ 1902, 1889, 1464, 1464, 1943, 1939, 1955, 1943, 1939, 1943, + /* 260 */ 1939, 1464, 1605, 1464, 1464, 1464, 1536, 1496, 1464, 1724, + /* 270 */ 1737, 1639, 1639, 1639, 1539, 1469, 1464, 1464, 1464, 1464, + /* 280 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1808, 1924, + /* 290 */ 1923, 1847, 1846, 1845, 1843, 1807, 1464, 1601, 1806, 1805, + /* 300 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1799, 1800, + /* 310 */ 1798, 1797, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 320 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 330 */ 1872, 1464, 1940, 1944, 1464, 1464, 1464, 1464, 1464, 1783, + /* 340 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 350 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 360 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 370 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 380 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 390 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 400 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 410 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 420 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 430 */ 1464, 1464, 1464, 1464, 1501, 1464, 1464, 1464, 1464, 1464, + /* 440 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 450 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 460 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 470 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1573, 1572, + /* 480 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 490 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 500 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 510 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1741, 1464, 1464, + /* 520 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1905, 1464, 1464, + /* 530 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 540 */ 1464, 1464, 1464, 1783, 1464, 1922, 1464, 1882, 1878, 1464, + /* 550 */ 1464, 1874, 1782, 1464, 1464, 1938, 1464, 1464, 1464, 1464, + /* 560 */ 1464, 1464, 1464, 1464, 1464, 1867, 1464, 1464, 1840, 1825, + /* 570 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 580 */ 1793, 1464, 1464, 1464, 1464, 1464, 1633, 1464, 1464, 1464, + /* 590 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1618, + /* 600 */ 1616, 1615, 1614, 1464, 1611, 1464, 1464, 1464, 1464, 1642, + /* 610 */ 1641, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 620 */ 1464, 1464, 1464, 1557, 1464, 1464, 1464, 1464, 1464, 1464, + /* 630 */ 1464, 1464, 1549, 1464, 1548, 1464, 1464, 1464, 1464, 1464, + /* 640 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 650 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, 1464, + /* 660 */ 1464, 1464, 1464, 1464, 1464, 1464, 1464, }; /********** End of lemon-generated parsing tables *****************************/ @@ -2024,7 +2024,7 @@ static const char *const yyRuleName[] = { /* 272 */ "stream_options ::= stream_options TRIGGER WINDOW_CLOSE", /* 273 */ "stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal", /* 274 */ "stream_options ::= stream_options WATERMARK duration_literal", - /* 275 */ "stream_options ::= stream_options IGNORE EXPIRED", + /* 275 */ "stream_options ::= stream_options IGNORE EXPIRED NK_INTEGER", /* 276 */ "cmd ::= KILL CONNECTION NK_INTEGER", /* 277 */ "cmd ::= KILL QUERY NK_STRING", /* 278 */ "cmd ::= KILL TRANSACTION NK_INTEGER", @@ -3113,7 +3113,7 @@ static const struct { { 362, -3 }, /* (272) stream_options ::= stream_options TRIGGER WINDOW_CLOSE */ { 362, -4 }, /* (273) stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal */ { 362, -3 }, /* (274) stream_options ::= stream_options WATERMARK duration_literal */ - { 362, -3 }, /* (275) stream_options ::= stream_options IGNORE EXPIRED */ + { 362, -4 }, /* (275) stream_options ::= stream_options IGNORE EXPIRED NK_INTEGER */ { 305, -3 }, /* (276) cmd ::= KILL CONNECTION NK_INTEGER */ { 305, -3 }, /* (277) cmd ::= KILL QUERY NK_STRING */ { 305, -3 }, /* (278) cmd ::= KILL TRANSACTION NK_INTEGER */ @@ -4297,9 +4297,9 @@ static YYACTIONTYPE yy_reduce( { ((SStreamOptions*)yymsp[-3].minor.yy840)->triggerType = STREAM_TRIGGER_MAX_DELAY; ((SStreamOptions*)yymsp[-3].minor.yy840)->pDelay = releaseRawExprNode(pCxt, yymsp[0].minor.yy840); yylhsminor.yy840 = yymsp[-3].minor.yy840; } yymsp[-3].minor.yy840 = yylhsminor.yy840; break; - case 275: /* stream_options ::= stream_options IGNORE EXPIRED */ -{ ((SStreamOptions*)yymsp[-2].minor.yy840)->ignoreExpired = true; yylhsminor.yy840 = yymsp[-2].minor.yy840; } - yymsp[-2].minor.yy840 = yylhsminor.yy840; + case 275: /* stream_options ::= stream_options IGNORE EXPIRED NK_INTEGER */ +{ ((SStreamOptions*)yymsp[-3].minor.yy840)->ignoreExpired = taosStr2Int8(yymsp[0].minor.yy0.z, NULL, 10); yylhsminor.yy840 = yymsp[-3].minor.yy840; } + yymsp[-3].minor.yy840 = yylhsminor.yy840; break; case 276: /* cmd ::= KILL CONNECTION NK_INTEGER */ { pCxt->pRootNode = createKillStmt(pCxt, QUERY_NODE_KILL_CONNECTION_STMT, &yymsp[0].minor.yy0); } diff --git a/source/libs/parser/test/parInitialCTest.cpp b/source/libs/parser/test/parInitialCTest.cpp index 5cf6c30b31..9bca6cae0a 100644 --- a/source/libs/parser/test/parInitialCTest.cpp +++ b/source/libs/parser/test/parInitialCTest.cpp @@ -571,7 +571,7 @@ TEST_F(ParserInitialCTest, createStream) { auto setCreateStreamReqFunc = [&](const char* pStream, const char* pSrcDb, const char* pSql, const char* pDstStb = nullptr, int8_t igExists = 0, int8_t triggerType = STREAM_TRIGGER_AT_ONCE, int64_t maxDelay = 0, - int64_t watermark = 0, int8_t igExpired = 0) { + int64_t watermark = 0, int8_t igExpired = STREAM_DEFAULT_IGNORE_EXPIRED) { snprintf(expect.name, sizeof(expect.name), "0.%s", pStream); snprintf(expect.sourceDB, sizeof(expect.sourceDB), "0.%s", pSrcDb); if (NULL != pDstStb) { @@ -617,11 +617,11 @@ TEST_F(ParserInitialCTest, createStream) { clearCreateStreamReq(); setCreateStreamReqFunc("s1", "test", - "create stream if not exists s1 trigger max_delay 20s watermark 10s ignore expired into st1 " + "create stream if not exists s1 trigger max_delay 20s watermark 10s ignore expired 0 into st1 " "as select count(*) from t1 interval(10s)", "st1", 1, STREAM_TRIGGER_MAX_DELAY, 20 * MILLISECOND_PER_SECOND, 10 * MILLISECOND_PER_SECOND, - 1); - run("CREATE STREAM IF NOT EXISTS s1 TRIGGER MAX_DELAY 20s WATERMARK 10s IGNORE EXPIRED INTO st1 AS SELECT COUNT(*) " + 0); + run("CREATE STREAM IF NOT EXISTS s1 TRIGGER MAX_DELAY 20s WATERMARK 10s IGNORE EXPIRED 0 INTO st1 AS SELECT COUNT(*) " "FROM t1 INTERVAL(10S)"); clearCreateStreamReq(); } diff --git a/source/libs/planner/src/planLogicCreater.c b/source/libs/planner/src/planLogicCreater.c index 875ec21b24..c843dd0a67 100644 --- a/source/libs/planner/src/planLogicCreater.c +++ b/source/libs/planner/src/planLogicCreater.c @@ -38,6 +38,27 @@ typedef struct SRewriteExprCxt { SNodeList* pExprs; } SRewriteExprCxt; +static void setColumnInfo(SFunctionNode* pFunc, SColumnNode* pCol) { + switch (pFunc->funcType) { + case FUNCTION_TYPE_TBNAME: + pCol->colType = COLUMN_TYPE_TBNAME; + break; + case FUNCTION_TYPE_WSTART: + case FUNCTION_TYPE_WEND: + pCol->colId = PRIMARYKEY_TIMESTAMP_COL_ID; + pCol->colType = COLUMN_TYPE_WINDOW_PC; + break; + case FUNCTION_TYPE_WDURATION: + pCol->colType = COLUMN_TYPE_WINDOW_PC; + break; + case FUNCTION_TYPE_GROUP_KEY: + pCol->colType = COLUMN_TYPE_GROUP_KEY; + break; + default: + break; + } +} + static EDealRes doRewriteExpr(SNode** pNode, void* pContext) { switch (nodeType(*pNode)) { case QUERY_NODE_OPERATOR: @@ -60,11 +81,7 @@ static EDealRes doRewriteExpr(SNode** pNode, void* pContext) { strcpy(pCol->node.aliasName, pToBeRewrittenExpr->aliasName); strcpy(pCol->colName, ((SExprNode*)pExpr)->aliasName); if (QUERY_NODE_FUNCTION == nodeType(pExpr)) { - if (FUNCTION_TYPE_WSTART == ((SFunctionNode*)pExpr)->funcType) { - pCol->colId = PRIMARYKEY_TIMESTAMP_COL_ID; - } else if (FUNCTION_TYPE_TBNAME == ((SFunctionNode*)pExpr)->funcType) { - pCol->colType = COLUMN_TYPE_TBNAME; - } + setColumnInfo((SFunctionNode*)pExpr, pCol); } nodesDestroyNode(*pNode); *pNode = (SNode*)pCol; @@ -764,6 +781,57 @@ static int32_t createWindowLogicNode(SLogicPlanContext* pCxt, SSelectStmt* pSele return TSDB_CODE_FAILED; } +static EDealRes needFillValueImpl(SNode* pNode, void* pContext) { + if (QUERY_NODE_COLUMN == nodeType(pNode)) { + SColumnNode* pCol = (SColumnNode*)pNode; + if (COLUMN_TYPE_WINDOW_PC != pCol->colType && COLUMN_TYPE_GROUP_KEY != pCol->colType) { + *(bool*)pContext = true; + return DEAL_RES_END; + } + } + return DEAL_RES_CONTINUE; +} + +static bool needFillValue(SNode* pNode) { + bool hasFillCol = false; + nodesWalkExpr(pNode, needFillValueImpl, &hasFillCol); + return hasFillCol; +} + +static int32_t partFillExprs(SSelectStmt* pSelect, SNodeList** pFillExprs, SNodeList** pNotFillExprs) { + int32_t code = TSDB_CODE_SUCCESS; + SNode* pProject = NULL; + FOREACH(pProject, pSelect->pProjectionList) { + if (needFillValue(pProject)) { + code = nodesListMakeStrictAppend(pFillExprs, nodesCloneNode(pProject)); + } else if (QUERY_NODE_VALUE != nodeType(pProject)) { + code = nodesListMakeStrictAppend(pNotFillExprs, nodesCloneNode(pProject)); + } + if (TSDB_CODE_SUCCESS != code) { + NODES_DESTORY_LIST(*pFillExprs); + NODES_DESTORY_LIST(*pNotFillExprs); + break; + } + } + if (!pSelect->isDistinct) { + SNode* pOrderExpr = NULL; + FOREACH(pOrderExpr, pSelect->pOrderByList) { + SNode* pExpr = ((SOrderByExprNode*)pOrderExpr)->pExpr; + if (needFillValue(pExpr)) { + code = nodesListMakeStrictAppend(pFillExprs, nodesCloneNode(pExpr)); + } else if (QUERY_NODE_VALUE != nodeType(pExpr)) { + code = nodesListMakeStrictAppend(pNotFillExprs, nodesCloneNode(pExpr)); + } + if (TSDB_CODE_SUCCESS != code) { + NODES_DESTORY_LIST(*pFillExprs); + NODES_DESTORY_LIST(*pNotFillExprs); + break; + } + } + } + return code; +} + static int32_t createFillLogicNode(SLogicPlanContext* pCxt, SSelectStmt* pSelect, SLogicNode** pLogicNode) { if (NULL == pSelect->pWindow || QUERY_NODE_INTERVAL_WINDOW != nodeType(pSelect->pWindow) || NULL == ((SIntervalWindowNode*)pSelect->pWindow)->pFill) { @@ -785,10 +853,18 @@ static int32_t createFillLogicNode(SLogicPlanContext* pCxt, SSelectStmt* pSelect pFill->node.resultDataOrder = pFill->node.requireDataOrder; pFill->inputTsOrder = ORDER_ASC; - int32_t code = nodesCollectColumns(pSelect, SQL_CLAUSE_WINDOW, NULL, COLLECT_COL_TYPE_ALL, &pFill->node.pTargets); - if (TSDB_CODE_SUCCESS == code && NULL == pFill->node.pTargets) { - code = nodesListMakeStrictAppend(&pFill->node.pTargets, - nodesCloneNode(nodesListGetNode(pCxt->pCurrRoot->pTargets, 0))); + int32_t code = partFillExprs(pSelect, &pFill->pFillExprs, &pFill->pNotFillExprs); + if (TSDB_CODE_SUCCESS == code) { + code = rewriteExprsForSelect(pFill->pFillExprs, pSelect, SQL_CLAUSE_FILL); + } + if (TSDB_CODE_SUCCESS == code) { + code = rewriteExprsForSelect(pFill->pNotFillExprs, pSelect, SQL_CLAUSE_FILL); + } + if (TSDB_CODE_SUCCESS == code) { + code = createColumnByRewriteExprs(pFill->pFillExprs, &pFill->node.pTargets); + } + if (TSDB_CODE_SUCCESS == code) { + code = createColumnByRewriteExprs(pFill->pNotFillExprs, &pFill->node.pTargets); } pFill->mode = pFillNode->mode; diff --git a/source/libs/planner/src/planPhysiCreater.c b/source/libs/planner/src/planPhysiCreater.c index bb9ef5bd4b..c7eb6f7b5e 100644 --- a/source/libs/planner/src/planPhysiCreater.c +++ b/source/libs/planner/src/planPhysiCreater.c @@ -195,7 +195,7 @@ static int32_t addDataBlockSlotsImpl(SPhysiPlanContext* pCxt, SNodeList* pList, int32_t code = TSDB_CODE_SUCCESS; SHashObj* pHash = taosArrayGetP(pCxt->pLocationHelper, pDataBlockDesc->dataBlockId); - int16_t nextSlotId = taosHashGetSize(pHash), slotId = 0; + int16_t nextSlotId = LIST_LENGTH(pDataBlockDesc->pSlots), slotId = 0; SNode* pNode = NULL; FOREACH(pNode, pList) { SNode* pExpr = QUERY_NODE_ORDER_BY_EXPR == nodeType(pNode) ? ((SOrderByExprNode*)pNode)->pExpr : pNode; @@ -311,6 +311,10 @@ static EDealRes doSetSlotId(SNode* pNode, void* pContext) { static int32_t setNodeSlotId(SPhysiPlanContext* pCxt, int16_t leftDataBlockId, int16_t rightDataBlockId, SNode* pNode, SNode** pOutput) { + if (NULL == pNode) { + return TSDB_CODE_SUCCESS; + } + SNode* pRes = nodesCloneNode(pNode); if (NULL == pRes) { return TSDB_CODE_OUT_OF_MEMORY; @@ -332,6 +336,10 @@ static int32_t setNodeSlotId(SPhysiPlanContext* pCxt, int16_t leftDataBlockId, i static int32_t setListSlotId(SPhysiPlanContext* pCxt, int16_t leftDataBlockId, int16_t rightDataBlockId, const SNodeList* pList, SNodeList** pOutput) { + if (NULL == pList) { + return TSDB_CODE_SUCCESS; + } + SNodeList* pRes = nodesCloneList(pList); if (NULL == pRes) { return TSDB_CODE_OUT_OF_MEMORY; @@ -1372,14 +1380,23 @@ static int32_t createFillPhysiNode(SPhysiPlanContext* pCxt, SNodeList* pChildren pFill->inputTsOrder = pFillNode->inputTsOrder; SDataBlockDescNode* pChildTupe = (((SPhysiNode*)nodesListGetNode(pChildren, 0))->pOutputDataBlockDesc); - int32_t code = setListSlotId(pCxt, pChildTupe->dataBlockId, -1, pFillNode->node.pTargets, &pFill->pTargets); + int32_t code = setListSlotId(pCxt, pChildTupe->dataBlockId, -1, pFillNode->pFillExprs, &pFill->pFillExprs); if (TSDB_CODE_SUCCESS == code) { - code = addDataBlockSlots(pCxt, pFill->pTargets, pFill->node.pOutputDataBlockDesc); + code = addDataBlockSlots(pCxt, pFill->pFillExprs, pFill->node.pOutputDataBlockDesc); + } + if (TSDB_CODE_SUCCESS == code) { + code = setListSlotId(pCxt, pChildTupe->dataBlockId, -1, pFillNode->pNotFillExprs, &pFill->pNotFillExprs); + } + if (TSDB_CODE_SUCCESS == code) { + code = addDataBlockSlots(pCxt, pFill->pNotFillExprs, pFill->node.pOutputDataBlockDesc); } if (TSDB_CODE_SUCCESS == code) { code = setNodeSlotId(pCxt, pChildTupe->dataBlockId, -1, pFillNode->pWStartTs, &pFill->pWStartTs); } + if (TSDB_CODE_SUCCESS == code) { + code = addDataBlockSlot(pCxt, &pFill->pWStartTs, pFill->node.pOutputDataBlockDesc); + } if (TSDB_CODE_SUCCESS == code && NULL != pFillNode->pValues) { pFill->pValues = nodesCloneNode(pFillNode->pValues); diff --git a/source/libs/planner/test/planIntervalTest.cpp b/source/libs/planner/test/planIntervalTest.cpp index 30f7051722..dddd38fc95 100644 --- a/source/libs/planner/test/planIntervalTest.cpp +++ b/source/libs/planner/test/planIntervalTest.cpp @@ -45,8 +45,15 @@ TEST_F(PlanIntervalTest, fill) { "WHERE ts > TIMESTAMP '2022-04-01 00:00:00' and ts < TIMESTAMP '2022-04-30 23:59:59' " "INTERVAL(10s) FILL(VALUE, 10, 20)"); - run("SELECT COUNT(*) FROM st1 WHERE ts > TIMESTAMP '2022-04-01 00:00:00' and ts < TIMESTAMP '2022-04-30 23:59:59' " - "PARTITION BY TBNAME interval(10s) fill(prev)"); + run("SELECT _WSTART, TBNAME, COUNT(*) FROM st1 " + "WHERE ts > '2022-04-01 00:00:00' and ts < '2022-04-30 23:59:59' " + "PARTITION BY TBNAME INTERVAL(10s) FILL(PREV)"); + + run("SELECT COUNT(c1), MAX(c3), COUNT(c1) FROM t1 " + "WHERE ts > '2022-04-01 00:00:00' and ts < '2022-04-30 23:59:59' INTERVAL(10s) FILL(PREV)"); + + run("SELECT COUNT(c1) FROM t1 WHERE ts > '2022-04-01 00:00:00' and ts < '2022-04-30 23:59:59' " + "PARTITION BY c2 INTERVAL(10s) FILL(PREV) ORDER BY c2"); } TEST_F(PlanIntervalTest, selectFunc) { diff --git a/source/libs/qworker/src/qworker.c b/source/libs/qworker/src/qworker.c index e06b752862..862d142100 100644 --- a/source/libs/qworker/src/qworker.c +++ b/source/libs/qworker/src/qworker.c @@ -1,3 +1,5 @@ +#include "qworker.h" + #include "dataSinkMgt.h" #include "executor.h" #include "planner.h" @@ -7,7 +9,6 @@ #include "tcommon.h" #include "tmsg.h" #include "tname.h" -#include "qworker.h" SQWorkerMgmt gQwMgmt = { .lock = 0, @@ -15,7 +16,6 @@ SQWorkerMgmt gQwMgmt = { .qwNum = 0, }; - int32_t qwProcessHbLinkBroken(SQWorker *mgmt, SQWMsg *qwMsg, SSchedulerHbReq *req) { int32_t code = 0; SSchedulerHbRsp rsp = {0}; @@ -26,7 +26,7 @@ int32_t qwProcessHbLinkBroken(SQWorker *mgmt, SQWMsg *qwMsg, SSchedulerHbReq *re QW_LOCK(QW_WRITE, &sch->hbConnLock); sch->hbBrokenTs = taosGetTimestampMs(); - + if (qwMsg->connInfo.handle == sch->hbConnInfo.handle) { tmsgReleaseHandle(&sch->hbConnInfo, TAOS_CONN_SERVER); sch->hbConnInfo.handle = NULL; @@ -44,8 +44,8 @@ int32_t qwProcessHbLinkBroken(SQWorker *mgmt, SQWMsg *qwMsg, SSchedulerHbReq *re QW_RET(TSDB_CODE_SUCCESS); } -static void freeItem(void* param) { - SExplainExecInfo* pInfo = param; +static void freeItem(void *param) { + SExplainExecInfo *pInfo = param; taosMemoryFree(pInfo->verboseInfo); } @@ -54,7 +54,7 @@ int32_t qwHandleTaskComplete(QW_FPARAMS_DEF, SQWTaskCtx *ctx) { if (TASK_TYPE_TEMP == ctx->taskType && taskHandle) { if (ctx->explain) { - SArray* execInfoList = taosArrayInit(4, sizeof(SExplainExecInfo)); + SArray *execInfoList = taosArrayInit(4, sizeof(SExplainExecInfo)); QW_ERR_RET(qGetExplainExecInfo(taskHandle, execInfoList)); SRpcHandleInfo connInfo = ctx->ctrlConnInfo; @@ -81,7 +81,7 @@ int32_t qwExecTask(QW_FPARAMS_DEF, SQWTaskCtx *ctx, bool *queryStop) { qTaskInfo_t taskHandle = ctx->taskHandle; DataSinkHandle sinkHandle = ctx->sinkHandle; - SArray* pResList = taosArrayInit(4, POINTER_BYTES); + SArray *pResList = taosArrayInit(4, POINTER_BYTES); while (true) { QW_TASK_DLOG("start to execTask, loopIdx:%d", i++); @@ -95,7 +95,7 @@ int32_t qwExecTask(QW_FPARAMS_DEF, SQWTaskCtx *ctx, bool *queryStop) { } else { QW_TASK_DLOG("qExecTask failed, code:%x - %s", code, tstrerror(code)); } - QW_ERR_RET(code); + QW_ERR_JRET(code); } } @@ -105,7 +105,7 @@ int32_t qwExecTask(QW_FPARAMS_DEF, SQWTaskCtx *ctx, bool *queryStop) { QW_TASK_DLOG("qExecTask end with empty res, useconds:%" PRIu64, useconds); dsEndPut(sinkHandle, useconds); - QW_ERR_RET(qwHandleTaskComplete(QW_FPARAMS(), ctx)); + QW_ERR_JRET(qwHandleTaskComplete(QW_FPARAMS(), ctx)); if (queryStop) { *queryStop = true; @@ -114,7 +114,7 @@ int32_t qwExecTask(QW_FPARAMS_DEF, SQWTaskCtx *ctx, bool *queryStop) { break; } - for(int32_t j = 0; j < taosArrayGetSize(pResList); ++j) { + for (int32_t j = 0; j < taosArrayGetSize(pResList); ++j) { SSDataBlock *pRes = taosArrayGetP(pResList, j); ASSERT(pRes->info.rows > 0); @@ -122,7 +122,7 @@ int32_t qwExecTask(QW_FPARAMS_DEF, SQWTaskCtx *ctx, bool *queryStop) { code = dsPutDataBlock(sinkHandle, &inputData, &qcontinue); if (code) { QW_TASK_ELOG("dsPutDataBlock failed, code:%x - %s", code, tstrerror(code)); - QW_ERR_RET(code); + QW_ERR_JRET(code); } QW_TASK_DLOG("data put into sink, rows:%d, continueExecTask:%d", pRes->info.rows, qcontinue); @@ -132,7 +132,7 @@ int32_t qwExecTask(QW_FPARAMS_DEF, SQWTaskCtx *ctx, bool *queryStop) { if (queryStop) { *queryStop = true; } - + break; } @@ -151,6 +151,11 @@ int32_t qwExecTask(QW_FPARAMS_DEF, SQWTaskCtx *ctx, bool *queryStop) { taosArrayDestroy(pResList); QW_RET(code); + +_return: + taosArrayDestroy(pResList); + + return code; } int32_t qwGenerateSchHbRsp(SQWorker *mgmt, SQWSchStatus *sch, SQWHbInfo *hbInfo) { @@ -222,7 +227,8 @@ int32_t qwGetQueryResFromSink(QW_FPARAMS_DEF, SQWTaskCtx *ctx, int32_t *dataLen, QW_ERR_RET(code); } - QW_TASK_DLOG("no more data in sink and query end, fetched blocks %d rows %d", pOutput->numOfBlocks, pOutput->numOfRows); + QW_TASK_DLOG("no more data in sink and query end, fetched blocks %d rows %d", pOutput->numOfBlocks, + pOutput->numOfRows); qwUpdateTaskStatus(QW_FPARAMS(), JOB_TASK_STATUS_SUCC); if (NULL == rsp) { @@ -266,7 +272,8 @@ int32_t qwGetQueryResFromSink(QW_FPARAMS_DEF, SQWTaskCtx *ctx, int32_t *dataLen, pOutput->numOfBlocks++; if (DS_BUF_EMPTY == pOutput->bufStatus && pOutput->queryEnd) { - QW_TASK_DLOG("task all data fetched and done, fetched blocks %d rows %d", pOutput->numOfBlocks, pOutput->numOfRows); + QW_TASK_DLOG("task all data fetched and done, fetched blocks %d rows %d", pOutput->numOfBlocks, + pOutput->numOfRows); qwUpdateTaskStatus(QW_FPARAMS(), JOB_TASK_STATUS_SUCC); break; } @@ -288,10 +295,10 @@ int32_t qwGetQueryResFromSink(QW_FPARAMS_DEF, SQWTaskCtx *ctx, int32_t *dataLen, } int32_t qwGetDeleteResFromSink(QW_FPARAMS_DEF, SQWTaskCtx *ctx, SDeleteRes *pRes) { - int64_t len = 0; - bool queryEnd = false; - int32_t code = 0; - SOutputData output = {0}; + int64_t len = 0; + bool queryEnd = false; + int32_t code = 0; + SOutputData output = {0}; dsGetDataLength(ctx->sinkHandle, &len, &queryEnd); @@ -304,7 +311,7 @@ int32_t qwGetDeleteResFromSink(QW_FPARAMS_DEF, SQWTaskCtx *ctx, SDeleteRes *pRes if (NULL == output.pData) { QW_ERR_RET(TSDB_CODE_OUT_OF_MEMORY); } - + code = dsGetDataBlock(ctx->sinkHandle, &output); if (code) { QW_TASK_ELOG("dsGetDataBlock failed, code:%x - %s", code, tstrerror(code)); @@ -312,8 +319,8 @@ int32_t qwGetDeleteResFromSink(QW_FPARAMS_DEF, SQWTaskCtx *ctx, SDeleteRes *pRes QW_ERR_RET(code); } - SDeleterRes* pDelRes = (SDeleterRes*)output.pData; - + SDeleterRes *pDelRes = (SDeleterRes *)output.pData; + pRes->suid = pDelRes->suid; pRes->uidList = pDelRes->uidList; pRes->skey = pDelRes->skey; @@ -322,14 +329,13 @@ int32_t qwGetDeleteResFromSink(QW_FPARAMS_DEF, SQWTaskCtx *ctx, SDeleteRes *pRes strcpy(pRes->tableFName, pDelRes->tableName); strcpy(pRes->tsColName, pDelRes->tsColName); taosMemoryFree(output.pData); - + return TSDB_CODE_SUCCESS; } - int32_t qwHandlePrePhaseEvents(QW_FPARAMS_DEF, int8_t phase, SQWPhaseInput *input, SQWPhaseOutput *output) { - int32_t code = 0; - SQWTaskCtx *ctx = NULL; + int32_t code = 0; + SQWTaskCtx *ctx = NULL; QW_TASK_DLOG("start to handle event at phase %s", qwPhaseStr(phase)); @@ -355,8 +361,8 @@ int32_t qwHandlePrePhaseEvents(QW_FPARAMS_DEF, int8_t phase, SQWPhaseInput *inpu if (QW_EVENT_RECEIVED(ctx, QW_EVENT_DROP)) { QW_ERR_JRET(qwDropTask(QW_FPARAMS())); - //qwBuildAndSendDropRsp(&ctx->ctrlConnInfo, code); - //QW_TASK_DLOG("drop rsp send, handle:%p, code:%x - %s", ctx->ctrlConnInfo.handle, code, tstrerror(code)); + // qwBuildAndSendDropRsp(&ctx->ctrlConnInfo, code); + // QW_TASK_DLOG("drop rsp send, handle:%p, code:%x - %s", ctx->ctrlConnInfo.handle, code, tstrerror(code)); QW_ERR_JRET(TSDB_CODE_QRY_TASK_DROPPED); break; @@ -391,8 +397,8 @@ int32_t qwHandlePrePhaseEvents(QW_FPARAMS_DEF, int8_t phase, SQWPhaseInput *inpu if (QW_EVENT_RECEIVED(ctx, QW_EVENT_DROP)) { QW_ERR_JRET(qwDropTask(QW_FPARAMS())); - //qwBuildAndSendDropRsp(&ctx->ctrlConnInfo, code); - //QW_TASK_DLOG("drop rsp send, handle:%p, code:%x - %s", ctx->ctrlConnInfo.handle, code, tstrerror(code)); + // qwBuildAndSendDropRsp(&ctx->ctrlConnInfo, code); + // QW_TASK_DLOG("drop rsp send, handle:%p, code:%x - %s", ctx->ctrlConnInfo.handle, code, tstrerror(code)); QW_ERR_JRET(TSDB_CODE_QRY_TASK_DROPPED); } @@ -428,9 +434,9 @@ _return: } int32_t qwHandlePostPhaseEvents(QW_FPARAMS_DEF, int8_t phase, SQWPhaseInput *input, SQWPhaseOutput *output) { - int32_t code = 0; - SQWTaskCtx *ctx = NULL; - SRpcHandleInfo connInfo = {0}; + int32_t code = 0; + SQWTaskCtx *ctx = NULL; + SRpcHandleInfo connInfo = {0}; QW_TASK_DLOG("start to handle event at phase %s", qwPhaseStr(phase)); @@ -449,8 +455,8 @@ int32_t qwHandlePostPhaseEvents(QW_FPARAMS_DEF, int8_t phase, SQWPhaseInput *inp QW_ERR_JRET(TSDB_CODE_QRY_APP_ERROR); } - //qwBuildAndSendDropRsp(&ctx->ctrlConnInfo, code); - //QW_TASK_DLOG("drop rsp send, handle:%p, code:%x - %s", ctx->ctrlConnInfo.handle, code, tstrerror(code)); + // qwBuildAndSendDropRsp(&ctx->ctrlConnInfo, code); + // QW_TASK_DLOG("drop rsp send, handle:%p, code:%x - %s", ctx->ctrlConnInfo.handle, code, tstrerror(code)); QW_ERR_JRET(qwDropTask(QW_FPARAMS())); QW_ERR_JRET(TSDB_CODE_QRY_TASK_DROPPED); @@ -473,14 +479,14 @@ _return: if (QW_PHASE_POST_QUERY == phase && ctx) { ctx->queryRsped = true; - bool rsped = false; + bool rsped = false; SQWMsg qwMsg = {.msgType = ctx->msgType, .connInfo = ctx->ctrlConnInfo}; qwDbgSimulateRedirect(&qwMsg, ctx, &rsped); qwDbgSimulateDead(QW_FPARAMS(), ctx, &rsped); if (!rsped) { qwBuildAndSendQueryRsp(input->msgType + 1, &ctx->ctrlConnInfo, code, ctx); QW_TASK_DLOG("query msg rsped, handle:%p, code:%x - %s", ctx->ctrlConnInfo.handle, code, tstrerror(code)); - } + } } if (ctx) { @@ -507,7 +513,6 @@ int32_t qwAbortPrerocessQuery(QW_FPARAMS_DEF) { QW_RET(TSDB_CODE_SUCCESS); } - int32_t qwPreprocessQuery(QW_FPARAMS_DEF, SQWMsg *qwMsg) { int32_t code = 0; bool queryRsped = false; @@ -537,8 +542,7 @@ _return: QW_RET(TSDB_CODE_SUCCESS); } - -int32_t qwProcessQuery(QW_FPARAMS_DEF, SQWMsg *qwMsg, char* sql) { +int32_t qwProcessQuery(QW_FPARAMS_DEF, SQWMsg *qwMsg, char *sql) { int32_t code = 0; bool queryRsped = false; SSubplan *plan = NULL; @@ -556,7 +560,7 @@ int32_t qwProcessQuery(QW_FPARAMS_DEF, SQWMsg *qwMsg, char* sql) { ctx->needFetch = qwMsg->msgInfo.needFetch; ctx->msgType = qwMsg->msgType; - //QW_TASK_DLOGL("subplan json string, len:%d, %s", qwMsg->msgLen, qwMsg->msg); + // QW_TASK_DLOGL("subplan json string, len:%d, %s", qwMsg->msgLen, qwMsg->msg); code = qStringToSubplan(qwMsg->msg, &plan); if (TSDB_CODE_SUCCESS != code) { @@ -594,7 +598,7 @@ int32_t qwProcessQuery(QW_FPARAMS_DEF, SQWMsg *qwMsg, char* sql) { _return: taosMemoryFree(sql); - + input.code = code; input.msgType = qwMsg->msgType; code = qwHandlePostPhaseEvents(QW_FPARAMS(), QW_PHASE_POST_QUERY, &input, NULL); @@ -648,7 +652,7 @@ int32_t qwProcessCQuery(QW_FPARAMS_DEF, SQWMsg *qwMsg) { qwBuildAndSendFetchRsp(ctx->fetchType, &qwMsg->connInfo, rsp, dataLen, code); rsp = NULL; - + QW_TASK_DLOG("fetch rsp send, handle:%p, code:%x - %s, dataLen:%d", qwMsg->connInfo.handle, code, tstrerror(code), dataLen); } else { @@ -754,13 +758,13 @@ _return: if (code || rsp) { bool rsped = false; if (ctx) { - qwDbgSimulateRedirect(qwMsg, ctx, &rsped); + qwDbgSimulateRedirect(qwMsg, ctx, &rsped); qwDbgSimulateDead(QW_FPARAMS(), ctx, &rsped); } if (!rsped) { qwBuildAndSendFetchRsp(qwMsg->msgType + 1, &qwMsg->connInfo, rsp, dataLen, code); - QW_TASK_DLOG("%s send, handle:%p, code:%x - %s, dataLen:%d", TMSG_INFO(qwMsg->msgType + 1), qwMsg->connInfo.handle, code, tstrerror(code), - dataLen); + QW_TASK_DLOG("%s send, handle:%p, code:%x - %s, dataLen:%d", TMSG_INFO(qwMsg->msgType + 1), + qwMsg->connInfo.handle, code, tstrerror(code), dataLen); } } @@ -919,10 +923,11 @@ void qwProcessHbTimerEvent(void *param, void *tmrId) { uint64_t *sId = taosHashGetKey(pIter, NULL); QW_TLOG("cancel send hb to sch %" PRIx64 " cause of no connection handle", *sId); - if (sch->hbBrokenTs > 0 && ((currentMs - sch->hbBrokenTs) > QW_SCH_TIMEOUT_MSEC) && taosHashGetSize(sch->tasksHash) <= 0) { + if (sch->hbBrokenTs > 0 && ((currentMs - sch->hbBrokenTs) > QW_SCH_TIMEOUT_MSEC) && + taosHashGetSize(sch->tasksHash) <= 0) { taosArrayPush(pExpiredSch, sId); } - + pIter = taosHashIterate(mgmt->schHash, pIter); continue; } @@ -998,7 +1003,6 @@ _return: QW_RET(TSDB_CODE_SUCCESS); } - int32_t qWorkerInit(int8_t nodeType, int32_t nodeId, SQWorkerCfg *cfg, void **qWorkerMgmt, const SMsgCb *pMsgCb) { if (NULL == qWorkerMgmt || pMsgCb->mgmt == NULL) { qError("invalid param to init qworker"); @@ -1119,12 +1123,12 @@ int32_t qWorkerGetStat(SReadHandle *handle, void *qWorkerMgmt, SQWorkerStat *pSt QW_RET(TSDB_CODE_QRY_INVALID_INPUT); } - SQWorker *mgmt = (SQWorker *)qWorkerMgmt; + SQWorker *mgmt = (SQWorker *)qWorkerMgmt; SDataSinkStat sinkStat = {0}; - + dsDataSinkGetCacheSize(&sinkStat); pStat->cacheDataSize = sinkStat.cachedSize; - + pStat->queryProcessed = QW_STAT_GET(mgmt->stat.msgStat.queryProcessed); pStat->cqueryProcessed = QW_STAT_GET(mgmt->stat.msgStat.cqueryProcessed); pStat->fetchProcessed = QW_STAT_GET(mgmt->stat.msgStat.fetchProcessed); @@ -1139,6 +1143,3 @@ int32_t qWorkerGetStat(SReadHandle *handle, void *qWorkerMgmt, SQWorkerStat *pSt return TSDB_CODE_SUCCESS; } - - - diff --git a/source/libs/transport/inc/transComm.h b/source/libs/transport/inc/transComm.h index 04b58da570..6b52c74271 100644 --- a/source/libs/transport/inc/transComm.h +++ b/source/libs/transport/inc/transComm.h @@ -293,7 +293,7 @@ int transSendResponse(const STransMsg* msg); int transRegisterMsg(const STransMsg* msg); int transSetDefaultAddr(void* shandle, const char* ip, const char* fqdn); -int transGetSockDebugInfo(struct sockaddr* sockname, char* dst); +int transSockInfo2Str(struct sockaddr* sockname, char* dst); int64_t transAllocHandle(); diff --git a/source/libs/transport/src/transCli.c b/source/libs/transport/src/transCli.c index 9eea43be23..ebad365ce0 100644 --- a/source/libs/transport/src/transCli.c +++ b/source/libs/transport/src/transCli.c @@ -103,14 +103,6 @@ static SCliConn* getConnFromPool(void* pool, char* ip, uint32_t port); static void addConnToPool(void* pool, SCliConn* conn); static void doCloseIdleConn(void* param); -static int sockDebugInfo(struct sockaddr* sockname, char* dst) { - struct sockaddr_in addr = *(struct sockaddr_in*)sockname; - - char buf[16] = {0}; - int r = uv_ip4_name(&addr, (char*)buf, sizeof(buf)); - sprintf(dst, "%s:%d", buf, ntohs(addr.sin_port)); - return r; -} // register timer for read static void cliReadTimeoutCb(uv_timer_t* handle); // register timer in each thread to clear expire conn @@ -121,12 +113,14 @@ static void cliAllocRecvBufferCb(uv_handle_t* handle, size_t suggested_size, uv_ static void cliRecvCb(uv_stream_t* cli, ssize_t nread, const uv_buf_t* buf); // callback after write data to socket static void cliSendCb(uv_write_t* req, int status); -// callback after conn to server +// callback after conn to server static void cliConnCb(uv_connect_t* req, int status); static void cliAsyncCb(uv_async_t* handle); static void cliIdleCb(uv_idle_t* handle); static void cliPrepareCb(uv_prepare_t* handle); +static bool cliRecvReleaseReq(SCliConn* conn, STransMsgHead* pHead); + static int32_t allocConnRef(SCliConn* conn, bool update); static int cliAppCb(SCliConn* pConn, STransMsg* pResp, SCliMsg* pMsg); @@ -361,6 +355,9 @@ void cliHandleResp(SCliConn* conn) { SCliMsg* pMsg = NULL; STransConnCtx* pCtx = NULL; + if (cliRecvReleaseReq(conn, pHead)) { + return; + } CONN_SHOULD_RELEASE(conn, pHead); if (CONN_NO_PERSIST_BY_APP(conn)) { @@ -383,7 +380,7 @@ void cliHandleResp(SCliConn* conn) { transMsg.info.ahandle); } } else { - pCtx = pMsg ? pMsg->ctx : NULL; + pCtx = pMsg->ctx; transMsg.info.ahandle = pCtx ? pCtx->ahandle : NULL; tDebug("%s conn %p get ahandle %p, persist: 1", CONN_GET_INST_LABEL(conn), conn, transMsg.info.ahandle); } @@ -395,7 +392,6 @@ void cliHandleResp(SCliConn* conn) { } STraceId* trace = &transMsg.info.traceId; - tGDebug("%s conn %p %s received from %s, local info:%s, len:%d, code str:%s", CONN_GET_INST_LABEL(conn), conn, TMSG_INFO(pHead->msgType), conn->dst, conn->src, transMsg.contLen, tstrerror(transMsg.code)); @@ -830,11 +826,11 @@ void cliConnCb(uv_connect_t* req, int status) { int addrlen = sizeof(peername); uv_tcp_getpeername((uv_tcp_t*)pConn->stream, &peername, &addrlen); - transGetSockDebugInfo(&peername, pConn->dst); + transSockInfo2Str(&peername, pConn->dst); addrlen = sizeof(sockname); uv_tcp_getsockname((uv_tcp_t*)pConn->stream, &sockname, &addrlen); - transGetSockDebugInfo(&sockname, pConn->src); + transSockInfo2Str(&sockname, pConn->src); tTrace("%s conn %p connect to server successfully", CONN_GET_INST_LABEL(pConn), pConn); assert(pConn->stream == req->handle); @@ -1053,6 +1049,30 @@ static void cliPrepareCb(uv_prepare_t* handle) { if (thrd->stopMsg != NULL) cliHandleQuit(thrd->stopMsg, thrd); } +bool cliRecvReleaseReq(SCliConn* conn, STransMsgHead* pHead) { + if (pHead->release == 1 && (pHead->msgLen) == sizeof(*pHead)) { + uint64_t ahandle = pHead->ahandle; + SCliMsg* pMsg = NULL; + CONN_GET_MSGCTX_BY_AHANDLE(conn, ahandle); + transClearBuffer(&conn->readBuf); + transFreeMsg(transContFromHead((char*)pHead)); + if (transQueueSize(&conn->cliMsgs) > 0 && ahandle == 0) { + SCliMsg* cliMsg = transQueueGet(&conn->cliMsgs, 0); + if (cliMsg->type == Release) return true; + } + tDebug("%s conn %p receive release request, refId:%" PRId64 "", CONN_GET_INST_LABEL(conn), conn, conn->refId); + if (T_REF_VAL_GET(conn) > 1) { + transUnrefCliHandle(conn); + } + destroyCmsg(pMsg); + cliReleaseUnfinishedMsg(conn); + transQueueClear(&conn->cliMsgs); + addConnToPool(((SCliThrd*)conn->hostThrd)->pool, conn); + return true; + } + return false; +} + static void* cliWorkThread(void* arg) { SCliThrd* pThrd = (SCliThrd*)arg; pThrd->pid = taosGetSelfPthreadId(); diff --git a/source/libs/transport/src/transComm.c b/source/libs/transport/src/transComm.c index b568163e23..4272ec0b1c 100644 --- a/source/libs/transport/src/transComm.c +++ b/source/libs/transport/src/transComm.c @@ -77,7 +77,7 @@ void transFreeMsg(void* msg) { } taosMemoryFree((char*)msg - sizeof(STransMsgHead)); } -int transGetSockDebugInfo(struct sockaddr* sockname, char* dst) { +int transSockInfo2Str(struct sockaddr* sockname, char* dst) { struct sockaddr_in addr = *(struct sockaddr_in*)sockname; char buf[20] = {0}; diff --git a/source/libs/transport/src/transSvr.c b/source/libs/transport/src/transSvr.c index 4d35e346b1..3512b27bf8 100644 --- a/source/libs/transport/src/transSvr.c +++ b/source/libs/transport/src/transSvr.c @@ -114,6 +114,8 @@ static void uvAcceptAsyncCb(uv_async_t* handle); static void uvShutDownCb(uv_shutdown_t* req, int status); static void uvPrepareCb(uv_prepare_t* handle); +static bool uvRecvReleaseReq(SSvrConn* conn, STransMsgHead* pHead); + /* * time-consuming task throwed into BG work thread */ @@ -123,7 +125,7 @@ static void uvWorkAfterTask(uv_work_t* req, int status); static void uvWalkCb(uv_handle_t* handle, void* arg); static void uvFreeCb(uv_handle_t* handle); -static void uvStartSendRespInternal(SSvrMsg* smsg); +static void uvStartSendRespImpl(SSvrMsg* smsg); static void uvPrepareSendData(SSvrMsg* msg, uv_buf_t* wb); static void uvStartSendResp(SSvrMsg* msg); @@ -154,37 +156,6 @@ static void* transAcceptThread(void* arg); static bool addHandleToWorkloop(SWorkThrd* pThrd, char* pipeName); static bool addHandleToAcceptloop(void* arg); -#define CONN_SHOULD_RELEASE(conn, head) \ - do { \ - if ((head)->release == 1 && (head->msgLen) == sizeof(*head)) { \ - reallocConnRef(conn); \ - tTrace("conn %p received release request", conn); \ - \ - STraceId traceId = head->traceId; \ - conn->status = ConnRelease; \ - transClearBuffer(&conn->readBuf); \ - transFreeMsg(transContFromHead((char*)head)); \ - \ - STransMsg tmsg = { \ - .code = 0, .info.handle = (void*)conn, .info.traceId = traceId, .info.ahandle = (void*)0x9527}; \ - SSvrMsg* srvMsg = taosMemoryCalloc(1, sizeof(SSvrMsg)); \ - srvMsg->msg = tmsg; \ - srvMsg->type = Release; \ - srvMsg->pConn = conn; \ - if (!transQueuePush(&conn->srvMsgs, srvMsg)) { \ - return; \ - } \ - if (conn->regArg.init) { \ - tTrace("conn %p release, notify server app", conn); \ - STrans* pTransInst = conn->pTransInst; \ - (*pTransInst->cfp)(pTransInst->parent, &(conn->regArg.msg), NULL); \ - memset(&conn->regArg, 0, sizeof(conn->regArg)); \ - } \ - uvStartSendRespInternal(srvMsg); \ - return; \ - } \ - } while (0) - #define SRV_RELEASE_UV(loop) \ do { \ uv_walk(loop, uvWalkCb, NULL); \ @@ -230,7 +201,9 @@ static void uvHandleReq(SSvrConn* pConn) { // transRefSrvHandle(pConn); // uv_queue_work(((SWorkThrd*)pConn->hostThrd)->loop, wreq, uvWorkDoTask, uvWorkAfterTask); - CONN_SHOULD_RELEASE(pConn, pHead); + if (uvRecvReleaseReq(pConn, pHead)) { + return; + } STransMsg transMsg; memset(&transMsg, 0, sizeof(transMsg)); @@ -356,10 +329,10 @@ void uvOnSendCb(uv_write_t* req, int status) { msg = (SSvrMsg*)transQueueGet(&conn->srvMsgs, 0); if (msg != NULL) { - uvStartSendRespInternal(msg); + uvStartSendRespImpl(msg); } } else { - uvStartSendRespInternal(msg); + uvStartSendRespImpl(msg); } } } @@ -423,7 +396,7 @@ static void uvPrepareSendData(SSvrMsg* smsg, uv_buf_t* wb) { wb->len = len; } -static void uvStartSendRespInternal(SSvrMsg* smsg) { +static void uvStartSendRespImpl(SSvrMsg* smsg) { SSvrConn* pConn = smsg->pConn; if (pConn->broken) { return; @@ -453,7 +426,7 @@ static void uvStartSendResp(SSvrMsg* smsg) { if (!transQueuePush(&pConn->srvMsgs, smsg)) { return; } - uvStartSendRespInternal(smsg); + uvStartSendRespImpl(smsg); return; } @@ -544,6 +517,35 @@ static void uvShutDownCb(uv_shutdown_t* req, int status) { uv_close((uv_handle_t*)req->handle, uvDestroyConn); taosMemoryFree(req); } +static bool uvRecvReleaseReq(SSvrConn* pConn, STransMsgHead* pHead) { + if ((pHead)->release == 1 && (pHead->msgLen) == sizeof(*pHead)) { + reallocConnRef(pConn); + tTrace("conn %p received release request", pConn); + + STraceId traceId = pHead->traceId; + pConn->status = ConnRelease; + transClearBuffer(&pConn->readBuf); + transFreeMsg(transContFromHead((char*)pHead)); + + STransMsg tmsg = {.code = 0, .info.handle = (void*)pConn, .info.traceId = traceId, .info.ahandle = (void*)0x9527}; + SSvrMsg* srvMsg = taosMemoryCalloc(1, sizeof(SSvrMsg)); + srvMsg->msg = tmsg; + srvMsg->type = Release; + srvMsg->pConn = pConn; + if (!transQueuePush(&pConn->srvMsgs, srvMsg)) { + return true; + } + if (pConn->regArg.init) { + tTrace("conn %p release, notify server app", pConn); + STrans* pTransInst = pConn->pTransInst; + (*pTransInst->cfp)(pTransInst->parent, &(pConn->regArg.msg), NULL); + memset(&pConn->regArg, 0, sizeof(pConn->regArg)); + } + uvStartSendRespImpl(srvMsg); + return true; + } + return false; +} static void uvPrepareCb(uv_prepare_t* handle) { // prepare callback SWorkThrd* pThrd = handle->data; @@ -696,7 +698,7 @@ void uvOnConnectionCb(uv_stream_t* q, ssize_t nread, const uv_buf_t* buf) { transUnrefSrvHandle(pConn); return; } - transGetSockDebugInfo(&peername, pConn->dst); + transSockInfo2Str(&peername, pConn->dst); addrlen = sizeof(sockname); if (0 != uv_tcp_getsockname(pConn->pTcp, (struct sockaddr*)&sockname, &addrlen)) { @@ -704,7 +706,7 @@ void uvOnConnectionCb(uv_stream_t* q, ssize_t nread, const uv_buf_t* buf) { transUnrefSrvHandle(pConn); return; } - transGetSockDebugInfo(&sockname, pConn->src); + transSockInfo2Str(&sockname, pConn->src); struct sockaddr_in addr = *(struct sockaddr_in*)&sockname; pConn->clientIp = addr.sin_addr.s_addr; @@ -992,7 +994,7 @@ void uvHandleRelease(SSvrMsg* msg, SWorkThrd* thrd) { if (!transQueuePush(&conn->srvMsgs, msg)) { return; } - uvStartSendRespInternal(msg); + uvStartSendRespImpl(msg); return; } else if (conn->status == ConnRelease || conn->status == ConnNormal) { tDebug("%s conn %p already released, ignore release-msg", transLabel(thrd->pTransInst), conn); diff --git a/source/libs/transport/test/CMakeLists.txt b/source/libs/transport/test/CMakeLists.txt index 51be28ba0e..b0548149d0 100644 --- a/source/libs/transport/test/CMakeLists.txt +++ b/source/libs/transport/test/CMakeLists.txt @@ -33,7 +33,8 @@ target_link_libraries (transportTest util common gtest_main - transport + transport + function ) target_link_libraries (transUT diff --git a/source/util/src/terror.c b/source/util/src/terror.c index 8a3d0620ed..6e3067d44e 100644 --- a/source/util/src/terror.c +++ b/source/util/src/terror.c @@ -467,7 +467,6 @@ TAOS_DEFINE_ERROR(TSDB_CODE_FS_APP_ERROR, "tfs out of memory") TAOS_DEFINE_ERROR(TSDB_CODE_CTG_INTERNAL_ERROR, "catalog internal error") TAOS_DEFINE_ERROR(TSDB_CODE_CTG_INVALID_INPUT, "invalid catalog input parameters") TAOS_DEFINE_ERROR(TSDB_CODE_CTG_NOT_READY, "catalog is not ready") -TAOS_DEFINE_ERROR(TSDB_CODE_OUT_OF_MEMORY, "catalog memory error") TAOS_DEFINE_ERROR(TSDB_CODE_CTG_SYS_ERROR, "catalog system error") TAOS_DEFINE_ERROR(TSDB_CODE_CTG_DB_DROPPED, "Database is dropped") TAOS_DEFINE_ERROR(TSDB_CODE_CTG_OUT_OF_SERVICE, "catalog is out of service") diff --git a/tests/docs-examples-test/csharp.sh b/tests/docs-examples-test/csharp.sh new file mode 100644 index 0000000000..a8f1ce4119 --- /dev/null +++ b/tests/docs-examples-test/csharp.sh @@ -0,0 +1,28 @@ +#!/bin/bash + +set -e + +pgrep taosd || taosd >> /dev/null 2>&1 & +pgrep taosadapter || taosadapter >> /dev/null 2>&1 & +cd ../../docs/examples/csharp + +dotnet run --project connect.csproj + +taos -s "drop database if exists power" +dotnet run --project sqlinsert.csproj +dotnet run --project query.csproj +dotnet run --project asyncquery.csproj +dotnet run --project subscribe.csproj + +taos -s "drop topic if exists topic_example" +taos -s "drop database if exists power" +dotnet run --project stmtinsert.csproj + +taos -s "drop database if exists test" +dotnet run --project influxdbline.csproj + +taos -s "drop database if exists test" +dotnet run --project optstelnet.csproj + +taos -s "drop database if exists test" +dotnet run --project optsjson.csproj \ No newline at end of file diff --git a/tests/docs-examples-test/node.sh b/tests/docs-examples-test/node.sh new file mode 100644 index 0000000000..0283904815 --- /dev/null +++ b/tests/docs-examples-test/node.sh @@ -0,0 +1,42 @@ +#!/bin/bash + +set -e + +pgrep taosd || taosd >> /dev/null 2>&1 & +pgrep taosadapter || taosadapter >> /dev/null 2>&1 & + +cd ../../docs/examples/node + +npm install +cd restexample; + +node connect.js + +cd ../nativeexample + +node connect.js + +taos -s "drop database if exists power" +node insert_example.js + +node query_example.js + +node async_query_example.js + +node subscribe_demo.js + +taos -s "drop topic if exists topic_name_example" +taos -s "drop database if exists power" +node param_bind_example.js + +taos -s "drop database if exists power" +node multi_bind_example.js + +taos -s "drop database if exists test" +node influxdb_line_example.js + +taos -s "drop database if exists test" +node opentsdb_telnet_example.js + +taos -s "drop database if exists test" +node opentsdb_json_example.js \ No newline at end of file diff --git a/tests/script/tsim/parser/fill.sim b/tests/script/tsim/parser/fill.sim index 3b8d6b136e..f688d815e7 100644 --- a/tests/script/tsim/parser/fill.sim +++ b/tests/script/tsim/parser/fill.sim @@ -885,15 +885,15 @@ if $data10 != @20-01-01 01:01:10.000@ then return -1 endi -if $data11 != 1.000000000 then +if $data11 != 99.000000000 then return -1 endi -if $data12 != 1.000000000 then +if $data12 != 91.000000000 then return -1 endi -if $data13 != -87.000000000 then +if $data13 != 90.000000000 then return -1 endi @@ -917,15 +917,15 @@ if $data70 != @20-01-01 01:02:10.000@ then return -1 endi -if $data71 != 1.000000000 then +if $data71 != 99.000000000 then return -1 endi -if $data72 != 1.000000000 then +if $data72 != 91.000000000 then return -1 endi -if $data73 != -87.000000000 then +if $data73 != 90.000000000 then return -1 endi @@ -994,19 +994,19 @@ if $data10 != @20-01-01 01:01:10.000@ then return -1 endi -if $data11 != 1.000000000 then +if $data11 != 99.000000000 then return -1 endi -if $data12 != 1.000000000 then +if $data12 != 91.000000000 then return -1 endi -if $data13 != -87.000000000 then +if $data13 != 90.000000000 then return -1 endi -if $data14 != 86 then +if $data14 != 89 then return -1 endi diff --git a/tests/script/tsim/parser/fill_stb.sim b/tests/script/tsim/parser/fill_stb.sim index 195bd2e1c3..51ae6f4b41 100644 --- a/tests/script/tsim/parser/fill_stb.sim +++ b/tests/script/tsim/parser/fill_stb.sim @@ -111,13 +111,15 @@ endi if $data12 != -2 then return -1 endi -if $data13 != -3.00000 then +if $data13 != -3 then return -1 endi -if $data14 != -4.000000000 then +if $data14 != -4.00000 then + print expect -4.00000, actual: $data14 return -1 endi -if $data15 != -5 then +if $data15 != -5.000000000 then + print expect -5.000000000, actual: $data15 return -1 endi if $data31 != -1 then @@ -126,10 +128,10 @@ endi if $data52 != -2 then return -1 endi -if $data73 != -3.00000 then +if $data73 != -3 then return -1 endi -if $data74 != -4.000000000 then +if $data74 != -4.00000 then return -1 endi diff --git a/tests/script/tsim/parser/fill_us.sim b/tests/script/tsim/parser/fill_us.sim index 03231d2241..0a45c02f58 100644 --- a/tests/script/tsim/parser/fill_us.sim +++ b/tests/script/tsim/parser/fill_us.sim @@ -1010,6 +1010,7 @@ if $data31 != 9.000000000 then return -1 endi if $data41 != 12.500000000 then + print expect 12.500000000, actual: $data41 return -1 endi if $data51 != 16.000000000 then diff --git a/tests/script/tsim/stream/ignoreExpiredData.sim b/tests/script/tsim/stream/ignoreExpiredData.sim index a789416f5b..03f574bc52 100644 --- a/tests/script/tsim/stream/ignoreExpiredData.sim +++ b/tests/script/tsim/stream/ignoreExpiredData.sim @@ -45,9 +45,9 @@ print $data00 $data01 $data02 sql use test sql create table t1(ts timestamp, a int, b int , c int, d double); -sql create stream streams1 trigger at_once IGNORE EXPIRED into streamt1 as select _wstart, count(*) c1, sum(a) c3 from t1 interval(10s); -sql create stream streams2 trigger at_once IGNORE EXPIRED into streamt2 as select _wstart, count(*) c1, sum(a) c3 from t1 session(ts,10s); -sql create stream streams3 trigger at_once IGNORE EXPIRED into streamt3 as select _wstart, count(*) c1, sum(a) c3 from t1 state_window(a); +sql create stream streams1 trigger at_once IGNORE EXPIRED 1 into streamt1 as select _wstart, count(*) c1, sum(a) c3 from t1 interval(10s); +sql create stream streams2 trigger at_once IGNORE EXPIRED 1 into streamt2 as select _wstart, count(*) c1, sum(a) c3 from t1 session(ts,10s); +sql create stream streams3 trigger at_once IGNORE EXPIRED 1 into streamt3 as select _wstart, count(*) c1, sum(a) c3 from t1 state_window(a); sql insert into t1 values(1648791213000,1,2,3,1.0); sql insert into t1 values(1648791223001,1,2,3,1.1); sql insert into t1 values(1648791233002,2,2,3,2.1); @@ -111,8 +111,8 @@ sql use test1 sql create stable st(ts timestamp,a int,b int,c int) tags(ta int,tb int,tc int); sql create table ts1 using st tags(1,1,1); sql create table ts2 using st tags(2,2,2); -sql create stream stream_t1 trigger at_once IGNORE EXPIRED into streamtST1 as select _wstart, count(*) c1, count(a) c2 , sum(a) c3 , max(b) c5, min(c) c6 from st interval(10s) ; -sql create stream stream_t2 trigger at_once IGNORE EXPIRED into streamtST2 as select _wstart, count(*) c1, count(a) c2 , sum(a) c3 , max(b) c5, min(c) c6 from st session(ts, 10s) ; +sql create stream stream_t1 trigger at_once IGNORE EXPIRED 1 into streamtST1 as select _wstart, count(*) c1, count(a) c2 , sum(a) c3 , max(b) c5, min(c) c6 from st interval(10s) ; +sql create stream stream_t2 trigger at_once IGNORE EXPIRED 1 into streamtST2 as select _wstart, count(*) c1, count(a) c2 , sum(a) c3 , max(b) c5, min(c) c6 from st session(ts, 10s) ; sql insert into ts1 values(1648791211000,1,2,3); sql insert into ts1 values(1648791222001,2,2,3); sql insert into ts2 values(1648791211000,1,2,3); diff --git a/tests/system-test/6-cluster/vnode/4dnode1mnode_basic_replica3_vgroups.py b/tests/system-test/6-cluster/vnode/4dnode1mnode_basic_replica3_vgroups.py index 31f380f29f..8a6412dfcc 100644 --- a/tests/system-test/6-cluster/vnode/4dnode1mnode_basic_replica3_vgroups.py +++ b/tests/system-test/6-cluster/vnode/4dnode1mnode_basic_replica3_vgroups.py @@ -29,7 +29,7 @@ class TDTestCase: self.vgroups = 2 self.tb_nums = 10 self.row_nums = 100 - self.max_vote_time_cost = 10 # seconds + self.max_vote_time_cost = 30 # seconds def getBuildPath(self): selfPath = os.path.dirname(os.path.realpath(__file__)) diff --git a/tests/system-test/fulltest.sh b/tests/system-test/fulltest.sh index 6d2afcca6e..e9331de6bf 100755 --- a/tests/system-test/fulltest.sh +++ b/tests/system-test/fulltest.sh @@ -230,7 +230,7 @@ python3 test.py -f 6-cluster/vnode/4dnode1mnode_basic_replica3_insertdatas_stop_ # python3 test.py -f 6-cluster/vnode/4dnode1mnode_basic_replica3_querydatas_stop_follower.py -N 4 -M 1 # python3 test.py -f 6-cluster/vnode/4dnode1mnode_basic_replica3_querydatas_stop_leader_force_stop.py -N 4 -M 1 # python3 test.py -f 6-cluster/vnode/4dnode1mnode_basic_replica3_querydatas_stop_leader.py -N 4 -M 1 -# python3 test.py -f 6-cluster/vnode/4dnode1mnode_basic_replica3_vgroups.py -N 4 -M 1 +python3 test.py -f 6-cluster/vnode/4dnode1mnode_basic_replica3_vgroups.py -N 4 -M 1 # python3 test.py -f 6-cluster/vnode/4dnode1mnode_basic_replica3_vgroups_stopOne.py -N 4 -M 1 diff --git a/tools/CMakeLists.txt b/tools/CMakeLists.txt index 5751c347e3..25d6e33175 100644 --- a/tools/CMakeLists.txt +++ b/tools/CMakeLists.txt @@ -99,11 +99,20 @@ ELSE () MESSAGE("CURRENT SOURCE DIR ${CMAKE_CURRENT_SOURCE_DIR}") IF (TD_LINUX) include(ExternalProject) + set(_upx_prefix "$ENV{HOME}/.taos/externals/upx") + ExternalProject_Add(upx + PREFIX "${_upx_prefix}" + URL https://github.com/upx/upx/releases/download/v3.96/upx-3.96-${PLATFORM_ARCH_STR}_linux.tar.xz + CONFIGURE_COMMAND cmake -E true + BUILD_COMMAND cmake -E true + INSTALL_COMMAND cmake -E true + ) + ExternalProject_Add(taosadapter PREFIX "taosadapter" SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/taosadapter BUILD_ALWAYS off - DEPENDS taos + DEPENDS taos upx BUILD_IN_SOURCE 1 CONFIGURE_COMMAND cmake -E echo "taosadapter no need cmake to config" PATCH_COMMAND @@ -112,7 +121,7 @@ ELSE () COMMAND CGO_CFLAGS=-I${CMAKE_CURRENT_SOURCE_DIR}/../include/client CGO_LDFLAGS=-L${CMAKE_BINARY_DIR}/build/lib go build -a -ldflags "-s -w -X github.com/taosdata/taosadapter/version.Version=${taos_version} -X github.com/taosdata/taosadapter/version.CommitID=${taosadapter_commit_sha1}" COMMAND CGO_CFLAGS=-I${CMAKE_CURRENT_SOURCE_DIR}/../include/client CGO_LDFLAGS=-L${CMAKE_BINARY_DIR}/build/lib go build -a -o taosadapter-debug -ldflags "-X github.com/taosdata/taosadapter/version.Version=${taos_version} -X github.com/taosdata/taosadapter/version.CommitID=${taosadapter_commit_sha1}" INSTALL_COMMAND - COMMAND wget -nc https://github.com/upx/upx/releases/download/v3.96/upx-3.96-${PLATFORM_ARCH_STR}_linux.tar.xz -O $ENV{HOME}/upx.tar.xz && tar -xvJf $ENV{HOME}/upx.tar.xz -C $ENV{HOME}/ --strip-components 1 > /dev/null && $ENV{HOME}/upx taosadapter || : + COMMAND ${_upx_prefix}/src/upx/upx taosadapter COMMAND cmake -E copy taosadapter ${CMAKE_BINARY_DIR}/build/bin COMMAND cmake -E make_directory ${CMAKE_BINARY_DIR}/test/cfg/ COMMAND cmake -E copy ./example/config/taosadapter.toml ${CMAKE_BINARY_DIR}/test/cfg/ @@ -140,6 +149,7 @@ ELSE () COMMAND cmake -E copy ./taosadapter.service ${CMAKE_BINARY_DIR}/test/cfg/ COMMAND cmake -E copy taosadapter-debug ${CMAKE_BINARY_DIR}/build/bin ) + unset(_upx_prefix) ELSE () MESSAGE("${Yellow} Windows system still use original embedded httpd ${ColourReset}") ENDIF ()