xxq250
/
homework-jianmu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Merge branch '3.0' into merge/mainto3.0

This commit is contained in:
Simon Guan 2025-03-10 11:07:48 +08:00
parent 1170955b95 f60219e2f0
commit abaf48ae15
149 changed files with 6978 additions and 1364 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
CMakeLists.txt
View File

@ -11,36 +11,29 @@ if(NOT DEFINED TD_SOURCE_DIR)
endif()
SET(TD_COMMUNITY_DIR ${PROJECT_SOURCE_DIR})
set(TD_SUPPORT_DIR "${TD_SOURCE_DIR}/cmake")
set(TD_CONTRIB_DIR "${TD_SOURCE_DIR}/contrib")
include(${TD_SUPPORT_DIR}/cmake.platform)
include(${TD_SUPPORT_DIR}/cmake.define)
include(${TD_SUPPORT_DIR}/cmake.options)
include(${TD_SUPPORT_DIR}/cmake.define)
include(${TD_SUPPORT_DIR}/cmake.version)
# contrib
add_subdirectory(contrib)
include(${TD_SUPPORT_DIR}/cmake.install)
set_property(GLOBAL PROPERTY GLOBAL_DEPENDS_NO_CYCLES OFF)
# api
add_library(api INTERFACE)
target_include_directories(api INTERFACE "include/client")
# src
if(${BUILD_TEST})
include(CTest)
enable_testing()
endif(${BUILD_TEST})
add_subdirectory(contrib)
add_subdirectory(source)
add_subdirectory(tools)
add_subdirectory(utils)
add_subdirectory(examples/c)
add_subdirectory(tests)
include(${TD_SUPPORT_DIR}/cmake.install)
# docs
add_subdirectory(docs/doxgen)
if(${BUILD_TEST})
include(CTest)
enable_testing()
add_subdirectory(examples/c)
endif(${BUILD_TEST})

170
README-CN.md
View File

@ -8,30 +8,30 @@
</a>
</p>
简体中文 | [English](README.md) | [TDengine 云服务](https://cloud.taosdata.com/?utm_medium=cn&utm_source=github) | 很多职位正在热招中，请看[这里](https://www.taosdata.com/careers/)
简体中文 | [English](README.md) | [TDengine 云服务](https://cloud.taosdata.com/?utm_medium=cn&utm_source=github) | 很多职位正在热招中，请看 [这里](https://www.taosdata.com/careers/)
# 目录
1. [TDengine 简介](#1-tdengine-简介)
1. [文档](#2-文档)
1. [必备工具](#3-必备工具)
- [3.1 Linux预备](#31-linux系统)
- [3.2 macOS预备](#32-macos系统)
- [3.3 Windows预备](#33-windows系统)
- [3.1 Linux 预备](#31-Linux系统)
- [3.2 macOS 预备](#32-macOS系统)
- [3.3 Windows 预备](#3.3-Windows系统)
- [3.4 克隆仓库](#34-克隆仓库)
1. [构建](#4-构建)
- [4.1 Linux系统上构建](#41-linux系统上构建)
- [4.2 macOS系统上构建](#42-macos系统上构建)
- [4.3 Windows系统上构建](#43-windows系统上构建)
- [4.1 Linux 系统上构建](#41-Linux系统上构建)
- [4.2 macOS 系统上构建](#42-macOS系统上构建)
- [4.3 Windows 系统上构建](#43-Windows系统上构建)
1. [打包](#5-打包)
1. [安装](#6-安装)
- [6.1 Linux系统上安装](#61-linux系统上安装)
- [6.2 macOS系统上安装](#62-macos系统上安装)
- [6.3 Windows系统上安装](#63-windows系统上安装)
- [6.1 Linux 系统上安装](#61-Linux系统上安装)
- [6.2 macOS 系统上安装](#62-macOS系统上安装)
- [6.3 Windows 系统上安装](#63-Windows系统上安装)
1. [快速运行](#7-快速运行)
- [7.1 Linux系统上运行](#71-linux系统上运行)
- [7.2 macOS系统上运行](#72-macos系统上运行)
- [7.3 Windows系统上运行](#73-windows系统上运行)
- [7.1 Linux 系统上运行](#71-Linux系统上运行)
- [7.2 macOS 系统上运行](#72-macOS系统上运行)
- [7.3 Windows 系统上运行](#73-Windows系统上运行)
1. [测试](#8-测试)
1. [版本发布](#9-版本发布)
1. [工作流](#10-工作流)
@ -43,9 +43,9 @@
TDengine 是一款开源、高性能、云原生的时序数据库 (Time-Series Database, TSDB)。TDengine 能被广泛运用于物联网、工业互联网、车联网、IT 运维、金融等领域。除核心的时序数据库功能外，TDengine 还提供缓存、数据订阅、流式计算等功能，是一极简的时序数据处理平台，最大程度的减小系统设计的复杂度，降低研发和运营成本。与其他时序数据库相比，TDengine 的主要优势如下：
- **高性能**：通过创新的存储引擎设计，无论是数据写入还是查询，TDengine 的性能比通用数据库快 10 倍以上，也远超其他时序数据库，存储空间不及通用数据库的1/10。
- **高性能**：通过创新的存储引擎设计，无论是数据写入还是查询，TDengine 的性能比通用数据库快 10 倍以上，也远超其他时序数据库，存储空间不及通用数据库的 1/10。
- **云原生**：通过原生分布式的设计，充分利用云平台的优势，TDengine 提供了水平扩展能力，具备弹性、韧性和可观测性，支持k8s部署，可运行在公有云、私有云和混合云上。
- **云原生**：通过原生分布式的设计，充分利用云平台的优势，TDengine 提供了水平扩展能力，具备弹性、韧性和可观测性，支持 k8s 部署，可运行在公有云、私有云和混合云上。
- **极简时序数据平台**：TDengine 内建消息队列、缓存、流式计算等功能，应用无需再集成 Kafka/Redis/HBase/Spark 等软件，大幅降低系统的复杂度，降低应用开发和运营成本。
@ -53,29 +53,29 @@ TDengine 是一款开源、高性能、云原生的时序数据库 (Time-Series
- **简单易用**：无任何依赖，安装、集群几秒搞定；提供REST以及各种语言连接器，与众多第三方工具无缝集成；提供命令行程序，便于管理和即席查询；提供各种运维工具。
- **核心开源**：TDengine 的核心代码包括集群功能全部开源，截止到2022年8月1日，全球超过 135.9k 个运行实例，GitHub Star 18.7k，Fork 4.4k，社区活跃。
- **核心开源**：TDengine 的核心代码包括集群功能全部开源，截止到 2022 年 8 月 1 日，全球超过 135.9k 个运行实例，GitHub Star 18.7k，Fork 4.4k，社区活跃。
了解TDengine高级功能的完整列表，请 [点击](https://tdengine.com/tdengine/)。体验TDengine最简单的方式是通过[TDengine云平台](https://cloud.tdengine.com)。
了解TDengine高级功能的完整列表，请 [点击](https://tdengine.com/tdengine/)。体验 TDengine 最简单的方式是通过 [TDengine云平台](https://cloud.tdengine.com)。
# 2. 文档
关于完整的使用手册，系统架构和更多细节，请参考 [TDengine](https://www.taosdata.com/) 或者 [TDengine 官方文档](https://docs.taosdata.com)。
用户可根据需求选择通过[容器](https://docs.taosdata.com/get-started/docker/)、[安装包](https://docs.taosdata.com/get-started/package/)、[Kubernetes](https://docs.taosdata.com/deployment/k8s/)来安装或直接使用无需安装部署的[云服务](https://cloud.taosdata.com/)。本快速指南是面向想自己编译、打包、测试的开发者的。
用户可根据需求选择通过 [容器](https://docs.taosdata.com/get-started/docker/)、[安装包](https://docs.taosdata.com/get-started/package/)、[Kubernetes](https://docs.taosdata.com/deployment/k8s/) 来安装或直接使用无需安装部署的 [云服务](https://cloud.taosdata.com/)。本快速指南是面向想自己编译、打包、测试的开发者的。
如果想编译或测试TDengine连接器，请访问以下仓库: [JDBC连接器](https://github.com/taosdata/taos-connector-jdbc), [Go连接器](https://github.com/taosdata/driver-go), [Python连接器](https://github.com/taosdata/taos-connector-python), [Node.js连接器](https://github.com/taosdata/taos-connector-node), [C#连接器](https://github.com/taosdata/taos-connector-dotnet), [Rust连接器](https://github.com/taosdata/taos-connector-rust).
如果想编译或测试 TDengine 连接器，请访问以下仓库：[JDBC连接器](https://github.com/taosdata/taos-connector-jdbc)、[Go连接器](https://github.com/taosdata/driver-go)、[Python连接器](https://github.com/taosdata/taos-connector-python)、[Node.js连接器](https://github.com/taosdata/taos-connector-node)、[C#连接器](https://github.com/taosdata/taos-connector-dotnet)、[Rust连接器](https://github.com/taosdata/taos-connector-rust)。
# 3. 前置条件
TDengine 目前可以在 Linux、 Windows、macOS 等平台上安装和运行。任何 OS 的应用也可以选择 taosAdapter 的 RESTful 接口连接服务端 taosd。CPU 支持 X64/ARM64，后续会支持 MIPS64、Alpha64、ARM32、RISC-V 等 CPU 架构。目前不支持使用交叉编译器构建。
TDengine 目前可以在 Linux、 Windows、macOS 等平台上安装和运行。任何 OS 的应用也可以选择 taosAdapter 的 RESTful 接口连接服务端 taosd。CPU 支持 X64、ARM64，后续会支持 MIPS64、Alpha64、ARM32、RISC-V 等 CPU 架构。目前不支持使用交叉编译器构建。
如果你想要编译 taosAdapter 或者 taosKeeper，需要安装 Go 1.18 及以上版本。
## 3.1 Linux系统
## 3.1 Linux 系统
<details>
<summary>安装Linux必备工具</summary>
<summary>安装 Linux 必备工具</summary>
### Ubuntu 18.04、20.04、22.04
@ -96,13 +96,13 @@ yum install -y zlib-static xz-devel snappy-devel jansson-devel pkgconfig libatom
</details>
## 3.2 macOS系统
## 3.2 macOS 系统
<details>
<summary>安装macOS必备工具</summary>
<summary>安装 macOS 必备工具</summary>
根据提示安装依赖工具 [brew](https://brew.sh/).
根据提示安装依赖工具 [brew](https://brew.sh/)
```bash
brew install argp-standalone gflags pkgconfig
@ -110,11 +110,11 @@ brew install argp-standalone gflags pkgconfig
</details>
## 3.3 Windows系统
## 3.3 Windows 系统
<details>
<summary>安装Windows必备工具</summary>
<summary>安装 Windows 必备工具</summary>
进行中。
@ -122,7 +122,7 @@ brew install argp-standalone gflags pkgconfig
## 3.4 克隆仓库
通过如下命令将TDengine仓库克隆到指定计算机:
通过如下命令将 TDengine 仓库克隆到指定计算机：
```bash
git clone https://github.com/taosdata/TDengine.git
@ -131,23 +131,23 @@ cd TDengine
# 4. 构建
TDengine 还提供一组辅助工具软件 taosTools，目前它包含 taosBenchmark（曾命名为 taosdemo）和 taosdump 两个软件。默认 TDengine 编译不包含 taosTools, 您可以在编译 TDengine 时使用`cmake .. -DBUILD_TOOLS=true` 来同时编译 taosTools。
TDengine 还提供一组辅助工具软件 taosTools，目前它包含 taosBenchmark（曾命名为 taosdemo）和 taosdump 两个软件。默认 TDengine 编译不包含 taosTools，您可以在编译 TDengine 时使用 `cmake .. -DBUILD_TOOLS=true` 来同时编译 taosTools。
为了构建TDengine, 请使用 [CMake](https://cmake.org/) 3.13.0 或者更高版本。
为了构建 TDengine，请使用 [CMake](https://cmake.org/) 3.13.0 或者更高版本。
## 4.1 Linux系统上构建
## 4.1 Linux 系统上构建
<details>
<summary>Linux系统上构建步骤</summary>
<summary>Linux 系统上构建步骤</summary>
可以通过以下命令使用脚本 `build.sh` 编译TDengine和taosTools，包括taosBenchmark和taosdump:
可以通过以下命令使用脚本 `build.sh` 编译 TDengine 和 taosTools，包括 taosBenchmark 和 taosdump。
```bash
./build.sh
```
也可以通过以下命令进行构建:
也可以通过以下命令进行构建：
```bash
mkdir debug && cd debug
@ -157,15 +157,15 @@ make
如果你想要编译 taosAdapter，需要添加 `-DBUILD_HTTP=false` 选项。
如果你想要编译 taosKeeper，需要添加 `--DBUILD_KEEPER=true` 选项。
如果你想要编译 taosKeeper，需要添加 `-DBUILD_KEEPER=true` 选项。
可以使用Jemalloc作为内存分配器，而不是使用glibc:
可以使用 Jemalloc 作为内存分配器，而不是使用 glibc：
```bash
cmake .. -DJEMALLOC_ENABLED=true
cmake .. -DJEMALLOC_ENABLED=ON
```
TDengine构建脚本可以自动检测x86、x86-64、arm64平台上主机的体系结构。
您也可以通过CPUTYPE选项手动指定架构:
TDengine 构建脚本可以自动检测 x86、x86-64、arm64 平台上主机的体系结构。
您也可以通过 CPUTYPE 选项手动指定架构：
```bash
cmake .. -DCPUTYPE=aarch64 && cmake --build .
@ -173,13 +173,13 @@ cmake .. -DCPUTYPE=aarch64 && cmake --build .
</details>
## 4.2 macOS系统上构建
## 4.2 macOS 系统上构建
<details>
<summary>macOS系统上构建步骤</summary>
<summary>macOS 系统上构建步骤</summary>
请安装XCode命令行工具和cmake。使用XCode 11.4+在Catalina和Big Sur上完成验证。
请安装 XCode 命令行工具和 cmake。使用 XCode 11.4+ 在 Catalina 和 Big Sur 上完成验证。
```shell
mkdir debug && cd debug
@ -192,14 +192,14 @@ cmake .. && cmake --build .
</details>
## 4.3 Windows系统上构建
## 4.3 Windows 系统上构建
<details>
<summary>Windows系统上构建步骤</summary>
<summary>Windows 系统上构建步骤</summary>
如果您使用的是Visual Studio 2013，请执行“cmd.exe”打开命令窗口执行如下命令。
执行vcvarsall.bat时，64位的Windows请指定“amd64”，32位的Windows请指定“x86”。
如果您使用的是 Visual Studio 2013，请执行 “cmd.exe” 打开命令窗口执行如下命令。
执行 vcvarsall.bat 时，64 位的 Windows 请指定 “amd64”，32 位的 Windows 请指定 “x86”。
```cmd
mkdir debug && cd debug
@ -208,19 +208,19 @@ cmake .. -G "NMake Makefiles"
nmake
```
如果您使用Visual Studio 2019或2017:
如果您使用 Visual Studio 2019 或 2017：
请执行“cmd.exe”打开命令窗口执行如下命令。
执行vcvarsall.bat时，64位的Windows请指定“x64”，32位的Windows请指定“x86”。
请执行 “cmd.exe” 打开命令窗口执行如下命令。
执行 vcvarsall.bat 时，64 位的 Windows 请指定 “x64”，32 位的 Windows 请指定 “x86”。
```cmd
mkdir debug && cd debug
"c:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" < x64 | x86 >
"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" < x64 | x86 >
cmake .. -G "NMake Makefiles"
nmake
```
或者，您可以通过点击Windows开始菜单打开命令窗口->“Visual Studio < 2019 | 2017 >”文件夹->“x64原生工具命令提示符VS < 2019 | 2017 >”或“x86原生工具命令提示符VS < 2019 | 2017 >”取决于你的Windows是什么架构，然后执行命令如下：
或者，您可以通过点击 Windows 开始菜单打开命令窗口 -> `Visual Studio < 2019 | 2017 >` 文件夹 -> `x64 原生工具命令提示符 VS < 2019 | 2017 >` 或 `x86 原生工具命令提示符 < 2019 | 2017 >` 取决于你的 Windows 是什么架构，然后执行命令如下：
```cmd
mkdir debug && cd debug
@ -231,33 +231,33 @@ nmake
# 5. 打包
由于一些组件依赖关系，TDengine社区安装程序不能仅由该存储库创建。我们仍在努力改进。
由于一些组件依赖关系，TDengine 社区安装程序不能仅由该存储库创建。我们仍在努力改进。
# 6. 安装
## 6.1 Linux系统上安装
## 6.1 Linux 系统上安装
<details>
<summary>Linux系统上安装详细步骤</summary>
<summary>Linux 系统上安装详细步骤</summary>
构建成功后，TDengine可以通过以下命令进行安装:
构建成功后，TDengine 可以通过以下命令进行安装：
```bash
sudo make install
```
从源代码安装还将为TDengine配置服务管理。用户也可以使用[TDengine安装包](https://docs.taosdata.com/get-started/package/)进行安装。
从源代码安装还将为 TDengine 配置服务管理。用户也可以使用 [TDengine安装包](https://docs.taosdata.com/get-started/package/)进行安装。
</details>
## 6.2 macOS系统上安装
## 6.2 macOS 系统上安装
<details>
<summary>macOS系统上安装详细步骤</summary>
<summary>macOS 系统上安装详细步骤</summary>
构建成功后，TDengine可以通过以下命令进行安装:
构建成功后，TDengine可以通过以下命令进行安装：
```bash
sudo make install
@ -265,13 +265,13 @@ sudo make install
</details>
## 6.3 Windows系统上安装
## 6.3 Windows 系统上安装
<details>
<summary>Windows系统上安装详细步骤</summary>
<summary>Windows 系统上安装详细步骤</summary>
构建成功后，TDengine可以通过以下命令进行安装:
构建成功后，TDengine 可以通过以下命令进行安装：
```cmd
nmake install
@ -281,32 +281,32 @@ nmake install
# 7. 快速运行
## 7.1 Linux系统上运行
## 7.1 Linux 系统上运行
<details>
<summary>Linux系统上运行详细步骤</summary>
<summary>Linux 系统上运行详细步骤</summary>
在Linux系统上安装TDengine完成后，在终端运行如下命令启动服务:
在Linux 系统上安装 TDengine 完成后，在终端运行如下命令启动服务：
```bash
sudo systemctl start taosd
```
然后用户可以通过如下命令使用TDengine命令行连接TDengine服务:
然后用户可以通过如下命令使用 TDengine 命令行连接 TDengine 服务：
```bash
taos
```
如果TDengine 命令行连接服务器成功，系统将打印欢迎信息和版本信息。否则，将显示连接错误信息。
如果 TDengine 命令行连接服务器成功，系统将打印欢迎信息和版本信息。否则，将显示连接错误信息。
如果您不想将TDengine作为服务运行，您可以在当前终端中运行它。例如，要在构建完成后快速启动TDengine服务器，在终端中运行以下命令：（我们以Linux为例，Windows上的命令为 `taosd.exe`）
如果您不想将 TDengine 作为服务运行，您可以在当前终端中运行它。例如，要在构建完成后快速启动 TDengine 服务器，在终端中运行以下命令：（以 Linux 为例，Windows 上的命令为 `taosd.exe`）
```bash
./build/bin/taosd -c test/cfg
```
在另一个终端上，使用TDengine命令行连接服务器:
在另一个终端上，使用 TDengine 命令行连接服务器：
```bash
./build/bin/taos -c test/cfg
@ -316,42 +316,42 @@ taos
</details>
## 7.2 macOS系统上运行
## 7.2 macOS 系统上运行
<details>
<summary>macOS系统上运行详细步骤</summary>
<summary>macOS 系统上运行详细步骤</summary>
在macOS上安装完成后启动服务，双击/applications/TDengine启动程序，或者在终端中执行如下命令：
在 macOS 上安装完成后启动服务，双击 `/applications/TDengine` 启动程序，或者在终端中执行如下命令：
```bash
sudo launchctl start com.tdengine.taosd
```
然后在终端中使用如下命令通过TDengine命令行连接TDengine服务器:
然后在终端中使用如下命令通过 TDengine 命令行连接 TDengine 服务器：
```bash
taos
```
如果TDengine命令行连接服务器成功，系统将打印欢迎信息和版本信息。否则，将显示错误信息。
如果 TDengine 命令行连接服务器成功，系统将打印欢迎信息和版本信息。否则，将显示错误信息。
</details>
## 7.3 Windows系统上运行
## 7.3 Windows 系统上运行
<details>
<summary>Windows系统上运行详细步骤</summary>
<summary>Windows 系统上运行详细步骤</summary>
您可以使用以下命令在Windows平台上启动TDengine服务器:
您可以使用以下命令在 Windows 平台上启动 TDengine 服务器：
```cmd
.\build\bin\taosd.exe -c test\cfg
```
在另一个终端上，使用TDengine命令行连接服务器:
在另一个终端上，使用 TDengine 命令行连接服务器：
```cmd
.\build\bin\taos.exe -c test\cfg
@ -363,25 +363,25 @@ taos
# 8. 测试
有关如何在TDengine上运行不同类型的测试，请参考 [TDengine测试](./tests/README-CN.md)
有关如何在 TDengine 上运行不同类型的测试，请参考 [TDengine测试](./tests/README-CN.md)
# 9. 版本发布
TDengine发布版本的完整列表，请参考 [版本列表](https://github.com/taosdata/TDengine/releases)
TDengine 发布版本的完整列表，请参考 [版本列表](https://github.com/taosdata/TDengine/releases)
# 10. 工作流
TDengine构建检查工作流可以在参考 [Github Action](https://github.com/taosdata/TDengine/actions/workflows/taosd-ci-build.yml), 更多的工作流正在创建中，将很快可用。
TDengine 构建检查工作流可以在参考 [Github Action](https://github.com/taosdata/TDengine/actions/workflows/taosd-ci-build.yml)，更多的工作流正在创建中，将很快可用。
# 11. 覆盖率
最新的TDengine测试覆盖率报告可参考 [coveralls.io](https://coveralls.io/github/taosdata/TDengine)
最新的 TDengine 测试覆盖率报告可参考 [coveralls.io](https://coveralls.io/github/taosdata/TDengine)
<details>
<summary>如何在本地运行测试覆盖率报告？</summary>
在本地创建测试覆盖率报告（HTML格式），请运行以下命令:
在本地创建测试覆盖率报告（HTML 格式），请运行以下命令：
```bash
cd tests
@ -389,8 +389,8 @@ bash setup-lcov.sh -v 1.16 && ./run_local_coverage.sh -b main -c task
# on main branch and run cases in longtimeruning_cases.task
# for more infomation about options please refer to ./run_local_coverage.sh -h
```
> **注意:**
> 请注意，-b和-i选项将使用-DCOVER=true选项重新编译TDengine，这可能需要花费一些时间。
> **注意**：
> 请注意，-b 和 -i 选项将使用 -DCOVER=true 选项重新编译 TDengine，这可能需要花费一些时间。
</details>

2
README.md
View File

@ -177,7 +177,7 @@ If you want to compile taosKeeper, you need to add the `--DBUILD_KEEPER=true` op
You can use Jemalloc as memory allocator instead of glibc:
```bash
cmake .. -DJEMALLOC_ENABLED=true
cmake .. -DJEMALLOC_ENABLED=ON
```
TDengine build script can auto-detect the host machine's architecture on x86, x86-64, arm64 platform.

12
cmake/cmake.define
View File

@ -116,9 +116,6 @@ ELSE()
set(VAR_TSZ "TSZ" CACHE INTERNAL "global variant tsz")
ENDIF()
# force set all platform to JEMALLOC_ENABLED = false
# SET(JEMALLOC_ENABLED OFF)
IF(TD_WINDOWS)
MESSAGE("${Yellow} set compiler flag for Windows! ${ColourReset}")
@ -260,12 +257,15 @@ ELSE()
ENDIF()
IF(TD_LINUX)
IF(TD_LINUX_64)
IF(${JEMALLOC_ENABLED})
MESSAGE(STATUS "JEMALLOC_ENABLED Enabled")
MESSAGE(STATUS "JEMALLOC Enabled")
SET(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-error=attributes")
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-error=attributes")
SET(LINK_JEMALLOC "-L${CMAKE_BINARY_DIR}/build/lib -ljemalloc")
ADD_DEFINITIONS(-DTD_JEMALLOC_ENABLED -I${CMAKE_BINARY_DIR}/build/include -L${CMAKE_BINARY_DIR}/build/lib -Wl,-rpath,${CMAKE_BINARY_DIR}/build/lib)
ELSE()
MESSAGE(STATUS "JEMALLOC_ENABLED Disabled")
MESSAGE(STATUS "JEMALLOC Disabled")
SET(LINK_JEMALLOC "")
ENDIF()
ENDIF()

4
docs/en/07-develop/09-udf.md
View File

@ -495,10 +495,10 @@ taos> select myfun(v1, v2) from t;
DB error: udf function execution failure (0.011088s)
```
Unfortunately, the execution failed. What could be the reason? Check the udfd process logs.
Unfortunately, the execution failed. What could be the reason? Check the taosudf process logs.
```shell
tail -10 /var/log/taos/udfd.log
tail -10 /var/log/taos/taosudf.log
```
Found the following error messages.

5
docs/en/08-operation/14-user.md
View File

@ -18,7 +18,10 @@ create user user_name pass'password' [sysinfo {1|0}] [createdb {1|0}]
The parameters are explained as follows.
- user_name: Up to 23 B long.
- password: The password must be between 8 and 16 characters long and include at least three types of characters from the following: uppercase letters, lowercase letters, numbers, and special characters. Special characters include `! @ # $ % ^ & * ( ) - _ + = [ ] { } : ; > < ? | ~ , .`.
- password: The password must be between 8 and 255 characters long. The password include at least three types of characters from the following: uppercase letters, lowercase letters, numbers, and special characters, special characters include `! @ # $ % ^ & * ( ) - _ + = [ ] { } : ; > < ? | ~ , .`, and this reqirement is able to be closed by adding enableStrongPassword 0 in taos.cfg, or by the following SQL:
```sql
alter all dnode 'EnableStrongPassword' '0'
```
- sysinfo: Whether the user can view system information. 1 means they can view it, 0 means they cannot. System information includes server configuration information, various node information such as dnode, query node (qnode), etc., as well as storage-related information, etc. The default is to view system information.
- createdb: Whether the user can create databases. 1 means they can create databases, 0 means they cannot. The default value is 0. // Supported starting from TDengine Enterprise version 3.3.2.0

1
docs/en/14-reference/01-components/01-taosd.md
View File

@ -231,6 +231,7 @@ The effective value of charset is UTF-8.
|udf | |Supported, effective after restart|Whether to start UDF service; 0: do not start, 1: start; default value 0 |
|udfdResFuncs | |Supported, effective after restart|Internal parameter, for setting UDF result sets|
|udfdLdLibPath | |Supported, effective after restart|Internal parameter, indicates the library path for loading UDF|
|enableStrongPassword | After 3.3.5.0 |Supported, effective after restart|The password include at least three types of characters from the following: uppercase letters, lowercase letters, numbers, and special characters, special characters include `! @ # $ % ^ & * ( ) - _ + = [ ] { } : ; > < ? \| ~ , .`; 0: disable, 1: enable; default value 1 |
### Stream Computing Parameters

37
docs/en/14-reference/02-tools/10-taosbenchmark.md
View File

@ -188,9 +188,12 @@ taosBenchmark -A INT,DOUBLE,NCHAR,BINARY\(16\)
The parameters listed in this section apply to all functional modes.
- **filetype**: The function to test, possible values are `insert`, `query`, and `subscribe`. Corresponding to insert, query, and subscribe functions. Only one can be specified in each configuration file.
- **filetype**: The function to test, possible values are `insert`, `query`, `subscribe` and `csvfile`. Corresponding to insert, query, subscribe and generate csv file functions. Only one can be specified in each configuration file.
- **cfgdir**: Directory where the TDengine client configuration file is located, default path is /etc/taos.
- **output_dir**: The directory specified for output files. When the feature category is csvfile, it refers to the directory where the generated csv files will be saved. The default value is ./output/.
- **host**: Specifies the FQDN of the TDengine server to connect to, default value is localhost.
- **port**: The port number of the TDengine server to connect to, default value is 6030.
@ -283,6 +286,27 @@ Parameters related to supertable creation are configured in the `super_tables` s
- **repeat_ts_max** : Numeric type, when composite primary key is enabled, specifies the maximum number of records with the same timestamp to be generated
- **sqls** : Array of strings type, specifies the array of sql to be executed after the supertable is successfully created, the table name specified in sql must be prefixed with the database name, otherwise an unspecified database error will occur
- **csv_file_prefix**: String type, sets the prefix for the names of the generated csv files. Default value is "data".
- **csv_ts_format**: String type, sets the format of the time string in the names of the generated csv files, following the `strftime` format standard. If not set, files will not be split by time intervals. Supported patterns include:
- %Y: Year as a four-digit number (e.g., 2025)
- %m: Month as a two-digit number (01 to 12)
- %d: Day of the month as a two-digit number (01 to 31)
- %H: Hour in 24-hour format as a two-digit number (00 to 23)
- %M: Minute as a two-digit number (00 to 59)
- %S: Second as a two-digit number (00 to 59)
- **csv_ts_interval**: String type, sets the time interval for splitting generated csv file names. Supports daily, hourly, minute, and second intervals such as 1d/2h/30m/40s. The default value is "1d".
- **csv_output_header**: String type, sets whether the generated csv files should contain column header descriptions. The default value is "yes".
- **csv_tbname_alias**: String type, sets the alias for the tbname field in the column header descriptions of csv files. The default value is "device_id".
- **csv_compress_level**: String type, sets the compression level for generating csv-encoded data and automatically compressing it into gzip file. This process directly encodes and compresses the data, rather than first generating a csv file and then compressing it. Possible values are:
- none: No compression
- fast: gzip level 1 compression
- balance: gzip level 6 compression
- best: gzip level 9 compression
#### Tag and Data Columns
@ -478,6 +502,17 @@ Note: Data types in the taosBenchmark configuration file must be in lowercase to
</details>
### Export CSV File Example
<details>
<summary>csv-export.json</summary>
```json
{{#include /TDengine/tools/taos-tools/example/csv-export.json}}
```
</details>
Other json examples see [here](https://github.com/taosdata/TDengine/tree/main/tools/taos-tools/example)
## Output Performance Indicators

2
docs/en/14-reference/03-taos-sql/19-limit.md
View File

@ -37,6 +37,6 @@ Removed `` ‘“`\ `` (single and double quotes, apostrophe, backslash, space)
- Number of databases, supertables, and tables are not limited by the system, only by system resources
- Number of replicas for a database can only be set to 1 or 3
- Maximum length of username is 23 bytes
- Maximum length of user password is 31 bytes
- Maximum length of user password is 255 bytes
- Total number of data rows depends on available resources
- Maximum number of virtual nodes for a single database is 1024

9
docs/en/14-reference/03-taos-sql/24-show.md
View File

@ -127,10 +127,11 @@ Displays created indexes.
## SHOW LOCAL VARIABLES
```sql
SHOW LOCAL VARIABLES;
SHOW LOCAL VARIABLES [like pattern];
```
Displays the runtime values of configuration parameters for the current client.
You can use the like pattern to filter by name.
## SHOW MNODES
@ -320,11 +321,11 @@ Displays information about all users in the current system, including user-defin
## SHOW CLUSTER VARIABLES (before version 3.0.1.6 it was SHOW VARIABLES)
```sql
SHOW CLUSTER VARIABLES;
SHOW DNODE dnode_id VARIABLES;
SHOW CLUSTER VARIABLES [like pattern];
SHOW DNODE dnode_id VARIABLES [like pattern];
```
Displays the runtime values of configuration parameters that need to be the same across nodes in the current system, or you can specify a DNODE to view its configuration parameters.
Displays the runtime values of configuration parameters that need to be the same across nodes in the current system, or you can specify a DNODE to view its configuration parameters. And you can use the like pattern to filter by name.
## SHOW VGROUPS

8
docs/en/14-reference/09-error-code.md
View File

@ -485,10 +485,10 @@ This document details the server error codes that may be encountered when using
| Error Code | Description | Possible Scenarios or Reasons | Recommended Actions |
| ---------- | ---------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| 0x80002901 | udf is stopping | udf call received when dnode exits | Stop executing udf queries |
| 0x80002902 | udf pipe read error | Error occurred when taosd reads from udfd pipe | udfd unexpectedly exits, 1) C udf crash 2) udfd crash |
| 0x80002903 | udf pipe connect error | Error establishing pipe connection to udfd in taosd | 1) Corresponding udfd not started in taosd. Restart taosd |
| 0x80002904 | udf pipe not exist | Connection error occurs between two phases of udf setup, call, and teardown, causing the connection to disappear, subsequent phases continue | udfd unexpectedly exits, 1) C udf crash 2) udfd crash |
| 0x80002905 | udf load failure | Error loading udf in udfd | 1) udf does not exist in mnode 2) Error in udf loading. Check logs |
| 0x80002902 | udf pipe read error | Error occurred when taosd reads from taosudf pipe | taosudf unexpectedly exits, 1) C udf crash 2) taosudf crash |
| 0x80002903 | udf pipe connect error | Error establishing pipe connection to taosudf in taosd | 1) Corresponding taosudf not started in taosd. Restart taosd |
| 0x80002904 | udf pipe not exist | Connection error occurs between two phases of udf setup, call, and teardown, causing the connection to disappear, subsequent phases continue | taosudf unexpectedly exits, 1) C udf crash 2) taosudf crash |
| 0x80002905 | udf load failure | Error loading udf in taosudf | 1) udf does not exist in mnode 2) Error in udf loading. Check logs |
| 0x80002906 | udf invalid function input | udf input check | udf function does not accept input, such as wrong column type |
| 0x80002907 | udf invalid bufsize | Intermediate result in udf aggregation function exceeds specified bufsize | Increase bufsize, or reduce intermediate result size |
| 0x80002908 | udf invalid output type | udf output type differs from the type specified when creating udf | Modify udf, or the type when creating udf, to match the result |

4
docs/zh/06-advanced/05-data-in/index.md
View File

@ -154,8 +154,8 @@ let v3 = data["voltage"].split(",");
如下图所示
* 对字段`ts`使用 split 规则拆分成日期和时间。split 规则需要设置**分隔符**和**拆分数量**，拆分后的字段命名规则为`{原字段名}_{顺序号}`。
* 对字段`voltage`使用正则表达式 `^(?<voltage>[0-9]+)(?<voltage_unit>[a-zA-Z]+)$` 提取出电压值和电压单位，Regex 规则同解析过程中的一样，使用**命名捕获组**命名提取字段。
* 对字段 `ts` 使用 split 规则拆分成日期和时间。split 规则需要设置 **分隔符** 和 **拆分数量**，拆分后的字段命名规则为 `{原字段名}_{顺序号}`。
* 对字段 `voltage` 使用正则表达式 `^(?<voltage>[0-9]+)(?<voltage_unit>[a-zA-Z]+)$` 提取出电压值和电压单位，Regex 规则同解析过程中的一样，使用 **命名捕获组** 命名提取字段。
* 对字段 `location` 使用 convert 转换，填写一个 JSON map 对象，其中 key 为字段 `current` 的值，`value` 为转换后的值。如图，`location` 字段的值 `"beijing.chaoyang.datun"` 被转换为 `"beijing.chaoyang.datunludong"`。
![拆分和提取](./pic/transform-04.png)

4
docs/zh/07-develop/09-udf.md
View File

@ -472,10 +472,10 @@ taos> select myfun(v1, v2) from t;
DB error: udf function execution failure (0.011088s)
```
不幸的是执行失败了，什么原因呢？查看 udfd 进程的日志。
不幸的是执行失败了，什么原因呢？查看 taosudf 进程的日志。
```shell
tail -10 /var/log/taos/udfd.log
tail -10 /var/log/taos/taosudf.log
```
发现以下错误信息。

4
docs/zh/08-operation/12-multi.md
View File

@ -127,7 +127,7 @@ s3migrate database <db_name>;
当 TSDB 时序数据超过 `s3_keeplocal` 参数指定的时间，相关的数据文件会被切分成多个文件块，每个文件块的默认大小是 512M 字节 (`s3_chunkpages * tsdb_pagesize`)。除了最后一个文件块保留在本地文件系统外，其余的文件块会被上传到对象存储服务。
```math
```text
上传次数 = 数据文件大小 / (s3_chunkpages * tsdb_pagesize) - 1
```
@ -141,7 +141,7 @@ s3migrate database <db_name>;
相邻的多个数据页会作为一个数据块从对象存储下载一次，以减少从对象存储下载的次数。每个数据页的大小，在创建数据库时，通过 `tsdb_pagesize` 参数指定，默认 4K 字节。
```math
```text
下载次数 = 查询需要的数据块数量 - 已缓存的数据块数量
```

15
docs/zh/08-operation/14-user.md
View File

@ -17,11 +17,14 @@ create user user_name pass'password' [sysinfo {1|0}] [createdb {1|0}]
相关参数说明如下。
- user_name：用户名最长不超过 23 个字节。
- password：密码长度必须为 8 到 16 位，并且至少包含大写字母、小写字母、数字、特殊字符中的三类。特殊字符包括 `! @ # $ % ^ & * ( ) - _ + = [ ] { } : ; > < ? | ~ , .`。（始自 3.3.5.0 版本）
- sysinfo ：用户是否可以查看系统信息。1 表示可以查看，0 表示不可以查看。系统信息包括服务端配置信息、服务端各种节点信息，如 dnode、查询节点（qnode）等，以及与存储相关的信息等。默认为可以查看系统信息。
- createdb：用户是否可以创建数据库。1 表示可以创建，0 表示不可以创建。缺省值为 0。// 从 TDengine 企业版 3.3.2.0 开始支持
- password：密码长度必须为 8 到 255 个字节。密码至少包含大写字母、小写字母、数字、特殊字符中的三类。特殊字符包括 `! @ # $ % ^ & * ( ) - _ + = [ ] { } : ; > < ? | ~ , .`（始自 v3.3.5.0），可以通过在 taos.cfg 中添加参数 `enableStrongPassword 0` 关闭此强制要求，或者通过如下 SQL 关闭（始自 v3.3.6.0）。
```sql
alter all dnode 'EnableStrongPassword' '0'
```
- sysinfo：用户是否可以查看系统信息。1 表示可以查看，0 表示不可以查看。系统信息包括服务端配置信息、服务端各种节点信息，如 dnode、查询节点（qnode）等，以及与存储相关的信息等。默认为可以查看系统信息。
- createdb：用户是否可以创建数据库。1 表示可以创建，0 表示不可以创建。缺省值为 0。从企业版 v3.3.2.0 开始支持。
如下 SQL 可以创建密码为 abc123!@# 且可以查看系统信息的用户 test。
如下 SQL 可以创建密码为 `abc123!@#` 且可以查看系统信息的用户 test。
```sql
create user test pass 'abc123!@#' sysinfo 1
@ -55,8 +58,8 @@ alter_user_clause: {
相关参数说明如下。
- pass：修改用户密码。
- enable：是否启用用户。1 表示启用此用户，0 表示禁用此用户。
- sysinfo ：用户是否可查看系统信息。1 表示可以查看系统信息，0 表示不可以查看系统信息
- createdb：用户是否可创建数据库。1 表示可以创建数据库，0 表示不可以创建数据库。从 TDengine 企业版 3.3.2.0 开始支持。
- sysinfo：用户是否可查看系统信息。1 表示可以查看系统信息，0 表示不可以查看系统信息
- createdb：用户是否可创建数据库。1 表示可以创建数据库，0 表示不可以创建数据库。从企业版 v3.3.2.0 开始支持。
如下 SQL 禁用 test 用户。
```sql

351
docs/zh/14-reference/01-components/01-taosd.md
View File

File diff suppressed because it is too large Load Diff

42
docs/zh/14-reference/02-tools/10-taosbenchmark.md
View File

@ -93,14 +93,17 @@ taosBenchmark -f <json file>
本节所列参数适用于所有功能模式。
- **filetype**：功能分类，可选值为 `insert`、`query` 和 `subscribe`。分别对应插入、查询和订阅功能。每个配置文件中只能指定其中之一。
- **filetype**：功能分类，可选值为 `insert`、`query`、`subscribe` 和 `csvfile`。分别对应插入、查询、订阅和生成 csv 文件功能。每个配置文件中只能指定其中之一。
- **cfgdir**：TDengine 客户端配置文件所在的目录，默认路径是 /etc/taos 。
- **output_dir**：指定输出文件的目录，当功能分类是 `csvfile` 时，指生成的 csv 文件的保存目录，默认值为 ./output/ 。
- **host**：指定要连接的 TDengine 服务端的 FQDN，默认值为 localhost 。
- **port**：要连接的 TDengine 服务器的端口号，默认值为 6030 。
- **user**：用于连接 TDengine 服务端的用户名，默认为 root 。
- **user**：用于连接 TDengine 服务端的用户名，默认值为 root 。
- **password**：用于连接 TDengine 服务端的密码，默认值为 taosdata。
@ -184,10 +187,34 @@ taosBenchmark -f <json file>
- **tags_file**：仅当 insert_mode 为 taosc，rest 的模式下生效。最终的 tag 的数值与 childtable_count 有关，如果 csv 文件内的 tag 数据行小于给定的子表数量，那么会循环读取 csv 文件数据直到生成 childtable_count 指定的子表数量；否则则只会读取 childtable_count 行 tag 数据。也即最终生成的子表数量为二者取小。
- **primary_key**：指定超级表是否有复合主键，取值 1 和 0，复合主键列只能是超级表的第二列，指定生成复合主键后要确保第二列符合复合主键的数据类型，否则会报错。
- **repeat_ts_min**：数值类型，复合主键开启情况下指定生成相同时间戳记录的最小个数，生成相同时间戳记录的个数是在范围[repeat_ts_min, repeat_ts_max] 内的随机值，最小值等于最大值时为固定个数。
- **repeat_ts_max**：数值类型，复合主键开启情况下指定生成相同时间戳记录的最大个数。
- **sqls**：字符串数组类型，指定超级表创建成功后要执行的 sql 数组，sql 中指定表名前面要带数据库名，否则会报未指定数据库错误。
- **csv_file_prefix**：字符串类型，设置生成的 csv 文件名称的前缀，默认值为 data 。
- **csv_ts_format**：字符串类型，设置生成的 csv 文件名称中时间字符串的格式，格式遵循 `strftime` 格式标准，如果没有设置表示不按照时间段切分文件。支持的模式有：
- %Y: 年份，四位数表示（例如：2025）
- %m: 月份，两位数表示（01到12）
- %d: 一个月中的日子，两位数表示（01到31）
- %H: 小时，24小时制，两位数表示（00到23）
- %M: 分钟，两位数表示（00到59）
- %S: 秒，两位数表示（00到59）
- **csv_ts_interval**：字符串类型，设置生成的 csv 文件名称中时间段间隔，支持天、小时、分钟、秒级间隔，如 1d/2h/30m/40s，默认值为 1d 。
- **csv_output_header**：字符串类型，设置生成的 csv 文件是否包含列头描述，默认值为 yes 。
- **csv_tbname_alias**：字符串类型，设置 csv 文件列头描述中 tbname 字段的别名，默认值为 device_id 。
- **csv_compress_level**：字符串类型，设置生成 csv 编码数据并自动压缩成 gzip 格式文件的压缩等级。此过程直接编码并压缩，而非先生成 csv 文件再压缩。可选值为：
- none：不压缩
- fast：gzip 1级压缩
- balance：gzip 6级压缩
- best：gzip 9级压缩
#### 标签列与数据列
@ -383,6 +410,17 @@ interval 控制休眠时间，避免持续查询慢查询消耗 CPU，单位为
</details>
### 生成 CSV 文件 JSON 示例
<details>
<summary>csv-export.json</summary>
```json
{{#include /TDengine/tools/taos-tools/example/csv-export.json}}
```
</details>
查看更多 json 配置文件示例可 [点击这里](https://github.com/taosdata/TDengine/tree/main/tools/taos-tools/example)
## 输出性能指标

50
docs/zh/14-reference/03-taos-sql/10-function.md
View File

@ -1091,9 +1091,9 @@ CAST(expr AS type_name)
- 对于不能支持的类型转换会直接报错。
- 对于类型支持但某些值无法正确转换的情况，对应的转换后的值以转换函数输出为准。目前可能遇到的几种情况：
1）字符串类型转换数值类型时可能出现的无效字符情况，例如 "a" 可能转为 0，但不会报错。
2）转换到数值类型时，数值大于 type_name 可表示的范围时，则会溢出，但不会报错。
3）转换到字符串类型时，如果转换后长度超过 type_name 中指定的长度，则会截断，但不会报错。
- 字符串类型转换数值类型时可能出现的无效字符情况，例如 "a" 可能转为 0，但不会报错。
- 转换到数值类型时，数值大于 type_name 可表示的范围时，则会溢出，但不会报错。
- 转换到字符串类型时，如果转换后长度超过 type_name 中指定的长度，则会截断，但不会报错。
#### TO_ISO8601
@ -1230,7 +1230,7 @@ TO_TIMESTAMP(ts_str_literal, format_str_literal)
**功能说明**：将字符串按照指定格式转化为时间戳。
**使用说明**：ver-3.2.2.0
**使用说明**：v3.2.2.0
**返回结果数据类型**：TIMESTAMP。
@ -1568,7 +1568,7 @@ algo_type: {
**适用于**：表和超级表。
**说明**：
- p 值范围是 [0,100]，当为 0 时等同 于MIN，为 100 时等同于 MAX。
- p 值范围是 [0,100]，当为 0 时等同 于 MIN，为 100 时等同于 MAX。
- algo_type 取值为 "default" 或 "t-digest"。输入为 "default" 时函数使用基于直方图算法进行计算。输入为 "t-digest" 时使用 t-digest 算法计算分位数的近似结果。如果不指定 algo_type 则使用 "default" 算法。
- t-digest 算法的近似结果对于输入数据顺序敏感，对超级表查询时不同的输入排序结果可能会有微小的误差。
@ -1884,12 +1884,12 @@ ignore_null_values: {
- INTERP 根据 FILL 字段来决定在每个符合输出条件的时刻如何进行插值。关于 FILL 子句如何使用请参考 [FILL 子句](../distinguished/#fill-子句)
- INTERP 可以在 RANGE 字段中只指定唯一的时间戳对单个时间点进行插值，在这种情况下，EVERY 字段可以省略。例如 SELECT INTERP(col) FROM tb RANGE('2023-01-01 00:00:00') FILL(linear)。
- INTERP 作用于超级表时，会将该超级表下的所有子表数据按照主键列排序后进行插值计算，也可以搭配 PARTITION BY tbname 使用，将结果强制规约到单个时间线。
- INTERP 可以与伪列 _irowts 一起使用，返回插值点所对应的时间戳(3.0.2.0 版本以后支持)。
- INTERP 可以与伪列 _isfilled 一起使用，显示返回结果是否为原始记录或插值算法产生的数据(3.0.3.0 版本以后支持)。
- INTERP 可以与伪列 _irowts 一起使用，返回插值点所对应的时间戳(v3.0.2.0 以后支持)。
- INTERP 可以与伪列 _isfilled 一起使用，显示返回结果是否为原始记录或插值算法产生的数据(v3.0.3.0 以后支持)。
- INTERP 对于带复合主键的表的查询，若存在相同时间戳的数据，则只有对应的复合主键最小的数据参与运算。
- INTERP 查询支持 NEAR FILL 模式，即当需要 FILL 时，使用距离当前时间点最近的数据进行插值，当前后时间戳与当前时间断面一样近时，FILL 前一行的值。此模式在流计算中和窗口查询中不支持。例如 SELECT INTERP(col) FROM tb RANGE('2023-01-01 00:00:00', '2023-01-01 00:10:00') FILL(NEAR)(3.3.4.9 版本及以后支持)。
- INTERP 只有在使用 FILL PREV/NEXT/NEAR 模式时才可以使用伪列 `_irowts_origin`。`_irowts_origin` 在 3.3.4.9 版本及以后支持。
- INTERP `RANGE`子句从 3.3.4.9 版本开始支持时间范围的扩展，如 `RANGE('2023-01-01 00:00:00', 10s)` 表示只能使用时间点 '2023-01-01 00:00:00' 周边 10s 内的数据进行插值，FILL PREV/NEXT/NEAR 分别表示从时间点开始向前/向后/前后在时间范围内查找数据，若时间点周边在指定时间范围内没有数据，则使用 FILL 指定的默认值进行插值，因此此时 FILL 子句必须同时指定默认值。例如 SELECT INTERP(col) FROM tb RANGE('2023-01-01 00:00:00', 10s) FILL(PREV, 1)。从 3.3.6.0 版本开始支持时间区间和时间范围的组合，对于时间区间内的每个断面进行插值时都需要满足时间范围的要求，在此之前的版本仅支持时间点和时间范围的组合。时间范围的值域规则与 EVERY 类似，单位不能是年或月，值必须大于 0，不能带引号。使用该扩展时，不支持除 FILL PREV/NEXT/NEAR 外的其他 FILL 模式。
- INTERP 查询支持 NEAR FILL 模式，即当需要 FILL 时，使用距离当前时间点最近的数据进行插值，当前后时间戳与当前时间断面一样近时，FILL 前一行的值。此模式在流计算中和窗口查询中不支持。例如 SELECT INTERP(col) FROM tb RANGE('2023-01-01 00:00:00', '2023-01-01 00:10:00') FILL(NEAR)(v3.3.4.9 及以后支持)。
- INTERP 只有在使用 FILL PREV/NEXT/NEAR 模式时才可以使用伪列 `_irowts_origin`。`_irowts_origin` 在 v3.3.4.9 以后支持。
- INTERP `RANGE`子句从 v3.3.4.9 开始支持时间范围的扩展，如 `RANGE('2023-01-01 00:00:00', 10s)` 表示只能使用时间点 '2023-01-01 00:00:00' 周边 10s 内的数据进行插值，FILL PREV/NEXT/NEAR 分别表示从时间点开始向前/向后/前后在时间范围内查找数据，若时间点周边在指定时间范围内没有数据，则使用 FILL 指定的默认值进行插值，因此此时 FILL 子句必须同时指定默认值。例如 SELECT INTERP(col) FROM tb RANGE('2023-01-01 00:00:00', 10s) FILL(PREV, 1)。从 v3.3.6.0 开始支持时间区间和时间范围的组合，对于时间区间内的每个断面进行插值时都需要满足时间范围的要求，在此之前的版本仅支持时间点和时间范围的组合。时间范围的值域规则与 EVERY 类似，单位不能是年或月，值必须大于 0，不能带引号。使用该扩展时，不支持除 `FILL PREV/NEXT/NEAR` 外的其他 FILL 模式。
### LAST
@ -2032,8 +2032,8 @@ TOP(expr, k)
**使用说明**：
- *k* 值取值范围 1≤*k*≤100；
- 系统同时返回该记录关联的时间戳列；
- *k* 值取值范围 1≤*k*≤100。
- 系统同时返回该记录关联的时间戳列。
- 限制：TOP 函数不支持 FILL 子句。
### UNIQUE
@ -2056,7 +2056,7 @@ UNIQUE(expr)
COLS(func(expr), output_expr1, [, output_expr2] ... )
```
**功能说明**：在选择函数 func(expr) 执行结果所在数据行上，执行表达式 output_expr1, [, output_expr2]，返回其结果，func(expr)结果不输出。
**功能说明**：在选择函数 func(expr) 执行结果所在数据行上，执行表达式 output_expr1, [, output_expr2]，返回其结果，func(expr) 结果不输出。
**返回数据类型**：返回多列数据，每列数据类型为对应表达式返回结果的类型。
@ -2134,11 +2134,11 @@ ignore_option: {
}
```
**功能说明**：统计表中特定列与之前行的当前列有效值之差。ignore_option 取值为 0|1|2|3，可以不填，默认值为 0.
- `0` 表示不忽略(diff结果)负值不忽略 null 值
- `1` 表示(diff结果)负值作为 null 值
- `2` 表示不忽略(diff结果)负值但忽略 null 值
- `3` 表示忽略(diff结果)负值且忽略 null 值
**功能说明**：统计表中特定列与之前行的当前列有效值之差。ignore_option 取值为 0|1|2|3，可以不填，默认值为 0。
- `0` 表示 diff 结果不忽略负值不忽略 null 值
- `1` 表示 diff 结果的负值作为 null 值
- `2` 表示 diff 结果不忽略负值但忽略 null 值
- `3` 表示 diff 结果忽略负值且忽略 null 值
- 对于存在复合主键的表的查询，若时间戳相同的数据存在多条，则只有对应的复合主键最小的数据参与运算。
**返回数据类型**：bool、时间戳及整型数值类型均返回 bigint，浮点类型返回 double，若 diff 结果溢出则返回溢出后的值。
@ -2150,13 +2150,13 @@ ignore_option: {
**使用说明**：
- diff 是计算本行特定列与同列的前一个有效数据的差值，同列的前一个有效数据：指的是同一列中时间戳较小的最临近的非空值。
- 数值类型 diff 结果为对应的算术差值；时间戳类型根据数据库的时间戳精度进行差值计算；bool 类型计算差值时 true 视为 1，false 视为 0
- 如当前行数据为 null 或者没有找到同列前一个有效数据时，diff 结果为 null
- 忽略负值时（ignore_option 设置为 1 或 3 ），如果 diff 结果为负值，则结果设置为 null，然后根据 null 值过滤规则进行过滤
- 当 diff 结果发生溢出时，结果是否是 `应该忽略的负值` 取决于逻辑运算结果是正数还是负数，例如 9223372036854775800 - (-9223372036854775806) 的值超出 BIGINT 的范围，diff 结果会显示溢出值 -10，但并不会被作为负值忽略
- 单个语句中可以使用单个或者多个 diff，并且每个 diff 可以指定相同或不同的 ignore_option，当单个语句中存在多个 diff 时当且仅当某行所有 diff 的结果都为 null，并且 ignore_option 都设置为忽略 null 值，该行才从结果集中剔除
- 数值类型 diff 结果为对应的算术差值；时间戳类型根据数据库的时间戳精度进行差值计算；bool 类型计算差值时 true 视为 1，false 视为 0。
- 如当前行数据为 null 或者没有找到同列前一个有效数据时，diff 结果为 null。
- 忽略负值时（ignore_option 设置为 1 或 3 ），如果 diff 结果为负值，则结果设置为 null，然后根据 null 值过滤规则进行过滤。
- 当 diff 结果发生溢出时，结果是否是 `应该忽略的负值` 取决于逻辑运算结果是正数还是负数，例如 9223372036854775800 - (-9223372036854775806) 的值超出 BIGINT 的范围，diff 结果会显示溢出值 -10，但并不会被作为负值忽略。
- 单个语句中可以使用单个或者多个 diff，并且每个 diff 可以指定相同或不同的 ignore_option，当单个语句中存在多个 diff 时当且仅当某行所有 diff 的结果都为 null，并且 ignore_option 都设置为忽略 null 值，该行才从结果集中剔除。
- 可以选择与相关联的列一起使用。例如 `select _rowts, DIFF() from`。
- 当没有复合主键时，如果不同的子表有相同时间戳的数据，会提示 "Duplicate timestamps not allowed"
- 当没有复合主键时，如果不同的子表有相同时间戳的数据，会提示 "Duplicate timestamps not allowed"。
- 当使用复合主键时，不同子表的时间戳和主键组合可能相同，使用哪一行取决于先找到哪一行，这意味着在这种情况下多次运行 diff() 的结果可能会不同。
### IRATE
@ -2232,7 +2232,7 @@ STATEDURATION(expr, oper, val, unit)
**参数范围**：
- oper：`'LT'` (小于)、`'GT'`（大于）、`'LE'`（小于等于）、`'GE'`（大于等于）、`'NE'`（不等于）、`'EQ'`（等于），不区分大小写，但需要用`''`包括。
- oper：`'LT'` (小于)、`'GT'`（大于）、`'LE'`（小于等于）、`'GE'`（大于等于）、`'NE'`（不等于）、`'EQ'`（等于），不区分大小写，但需要用 `''` 包括。
- val：数值型
- unit：时间长度的单位，可取值时间单位：1b(纳秒)、1u(微秒)、1a(毫秒)、1s(秒)、1m(分)、1h(小时)、1d(天)、1w(周)。如果省略，默认为当前数据库精度。

8
docs/zh/14-reference/03-taos-sql/12-distinguished.md
View File

@ -76,7 +76,7 @@ window_clause: {
FILL 语句指定某一窗口区间数据缺失的情况下的填充模式。填充模式包括以下几种：
1. 不进行填充：NONE（默认填充模式）。
2. VALUE 填充：固定值填充，此时需要指定填充的数值。例如 FILL(VALUE, 1.23)。这里需要注意，最终填充的值受由相应列的类型决定，如 FILL(VALUE, 1.23)，相应列为 INT 类型，则填充值为 1，若查询列表中有多列需要 FILL，则需要给每一个 FILL 列指定 VALUE，如 `SELECT _wstart, min(c1), max(c1) FROM ... FILL(VALUE, 0, 0)`，注意，SELECT 表达式中只有包含普通列时才需要指定 FILL VALUE，如 `_wstart`、`_wstart+1a`、`now`、`1+1` 以及使用 partition by 时的 partition key (如 tbname)都不需要指定 VALUE，如 `timediff(last(ts), _wstart)` 则需要指定 VALUE。
2. VALUE 填充：固定值填充，此时需要指定填充的数值。例如 `FILL(VALUE, 1.23)`。这里需要注意，最终填充的值受由相应列的类型决定，如 `FILL(VALUE, 1.23)`，相应列为 INT 类型，则填充值为 1，若查询列表中有多列需要 FILL，则需要给每一个 FILL 列指定 VALUE，如 `SELECT _wstart, min(c1), max(c1) FROM ... FILL(VALUE, 0, 0)`，注意，SELECT 表达式中只有包含普通列时才需要指定 FILL VALUE，如 `_wstart`、`_wstart+1a`、`now`、`1+1` 以及使用 `partition by` 时的 `partition key` (如 tbname)都不需要指定 VALUE，如 `timediff(last(ts), _wstart)` 则需要指定 VALUE。
3. PREV 填充：使用前一个非 NULL 值填充数据。例如 FILL(PREV)。
4. NULL 填充：使用 NULL 填充数据。例如 FILL(NULL)。
5. LINEAR 填充：根据前后距离最近的非 NULL 值做线性插值填充。例如 FILL(LINEAR)。
@ -165,7 +165,7 @@ TDengine 还支持将 CASE 表达式用在状态量，可以表达某个状态
SELECT tbname, _wstart, CASE WHEN voltage >= 205 and voltage <= 235 THEN 1 ELSE 0 END status FROM meters PARTITION BY tbname STATE_WINDOW(CASE WHEN voltage >= 205 and voltage <= 235 THEN 1 ELSE 0 END);
```
状态窗口支持使用 TRUE_FOR 参数来设定窗口的最小持续时长。如果某个状态窗口的宽度低于该设定值，则会自动舍弃，不返回任何计算结果。例如，设置最短持续时长为 3s:
状态窗口支持使用 TRUE_FOR 参数来设定窗口的最小持续时长。如果某个状态窗口的宽度低于该设定值，则会自动舍弃，不返回任何计算结果。例如，设置最短持续时长为 3s。
```
SELECT COUNT(*), FIRST(ts), status FROM temp_tb_1 STATE_WINDOW(status) TRUE_FOR (3s);
@ -202,7 +202,7 @@ select _wstart, _wend, count(*) from t event_window start with c1 > 0 end with c
![TDengine Database 事件窗口示意图](./event_window.webp)
事件窗口支持使用 TRUE_FOR 参数来设定窗口的最小持续时长。如果某个事件窗口的宽度低于该设定值，则会自动舍弃，不返回任何计算结果。例如，设置最短持续时长为 3s:
事件窗口支持使用 TRUE_FOR 参数来设定窗口的最小持续时长。如果某个事件窗口的宽度低于该设定值，则会自动舍弃，不返回任何计算结果。例如，设置最短持续时长为 3s。
```
select _wstart, _wend, count(*) from t event_window start with c1 > 0 end with c2 < 10 true_for (3s);
@ -223,7 +223,7 @@ select _wstart, _wend, count(*) from t count_window(4);
### 时间戳伪列
窗口聚合查询结果中，如果 SQL 语句中没有指定输出查询结果中的时间戳列，那么最终结果中不会自动包含窗口的时间列信息。如果需要在结果中输出聚合结果所对应的时间窗口信息，需要在 SELECT 子句中使用时间戳相关的伪列: 时间窗口起始时间 (\_WSTART), 时间窗口结束时间 (\_WEND), 时间窗口持续时间 (\_WDURATION), 以及查询整体窗口相关的伪列：查询窗口起始时间(\_QSTART) 和查询窗口结束时间(\_QEND)。需要注意的是时间窗口起始时间和结束时间均是闭区间，时间窗口持续时间是数据当前时间分辨率下的数值。例如，如果当前数据库的时间分辨率是毫秒，那么结果中 500 就表示当前时间窗口的持续时间是 500毫秒 (500 ms)。
窗口聚合查询结果中，如果 SQL 语句中没有指定输出查询结果中的时间戳列，那么最终结果中不会自动包含窗口的时间列信息。如果需要在结果中输出聚合结果所对应的时间窗口信息，需要在 SELECT 子句中使用时间戳相关的伪列：时间窗口起始时间 (\_WSTART)，时间窗口结束时间 (\_WEND)，时间窗口持续时间 (\_WDURATION)，以及查询整体窗口相关的伪列：查询窗口起始时间(\_QSTART) 和查询窗口结束时间(\_QEND)。需要注意的是时间窗口起始时间和结束时间均是闭区间，时间窗口持续时间是数据当前时间分辨率下的数值。例如，如果当前数据库的时间分辨率是毫秒，那么结果中 500 就表示当前时间窗口的持续时间是 500毫秒 (500 ms)。
### 示例

2
docs/zh/14-reference/03-taos-sql/14-stream.md
View File

@ -300,7 +300,7 @@ RESUME STREAM [IF EXISTS] [IGNORE UNTREATED] stream_name;
CREATE SNODE ON DNODE [id]
```
其中的 id 是集群中的 dnode 的序号。请注意选择的dnode，流计算的中间状态将自动在其上进行备份。
从 3.3.4.0 版本开始，在多副本环境中创建流会进行 snode 的**存在性检查**，要求首先创建 snode。如果 snode 不存在，无法创建流。
从 v3.3.4.0 开始，在多副本环境中创建流会进行 snode 的**存在性检查**，要求首先创建 snode。如果 snode 不存在，无法创建流。
## 流式计算的事件通知

2
docs/zh/14-reference/03-taos-sql/19-limit.md
View File

@ -37,6 +37,6 @@ description: 合法字符集和命名中的限制规则
- 库的数目，超级表的数目、表的数目，系统不做限制，仅受系统资源限制
- 数据库的副本数只能设置为 1 或 3
- 用户名的最大长度是 23 字节
- 用户密码的长度范围是 8-16 字节
- 用户密码的长度范围是 8-255 字节
- 总数据行数取决于可用资源
- 单个数据库的虚拟结点数上限为 1024

9
docs/zh/14-reference/03-taos-sql/24-show.md
View File

@ -134,10 +134,10 @@ SHOW INDEXES FROM [db_name.]tbl_name;
## SHOW LOCAL VARIABLES
```sql
SHOW LOCAL VARIABLES;
SHOW LOCAL VARIABLES [like pattern];
```
显示当前客户端配置参数的运行值。
显示当前客户端配置参数的运行值，可使用 like pattern 根据 name 进行过滤。
## SHOW MNODES
@ -329,11 +329,12 @@ SHOW USERS;
## SHOW CLUSTER VARIABLES(3.0.1.6 之前为 SHOW VARIABLES)
```sql
SHOW CLUSTER VARIABLES;
SHOW DNODE dnode_id VARIABLES;
SHOW CLUSTER VARIABLES [like pattern];;
SHOW DNODE dnode_id VARIABLES [like pattern];;
```
显示当前系统中各节点需要相同的配置参数的运行值，也可以指定 DNODE 来查看其的配置参数。
可使用 like pattern 根据 name 进行过滤。
## SHOW VGROUPS

4
include/common/tcommon.h
View File

@ -124,7 +124,7 @@ enum {
TMQ_MSG_TYPE__POLL_RAW_DATA_RSP,
};
static char* tmqMsgTypeStr[] = {
static const char* tmqMsgTypeStr[] = {
"data", "meta", "ask ep", "meta data", "wal info", "batch meta", "raw data"
};
@ -414,7 +414,7 @@ typedef struct STUidTagInfo {
#define NOTIFY_EVENT_STR_COLUMN_INDEX 0
int32_t taosGenCrashJsonMsg(int signum, char** pMsg, int64_t clusterId, int64_t startTime);
int32_t dumpConfToDataBlock(SSDataBlock* pBlock, int32_t startCol);
int32_t dumpConfToDataBlock(SSDataBlock* pBlock, int32_t startCol, char* likePattern);
#define TSMA_RES_STB_POSTFIX "_tsma_res_stb_"
#define MD5_OUTPUT_LEN 32

1
include/common/tglobal.h
View File

@ -69,6 +69,7 @@ extern EEncryptAlgor tsiEncryptAlgorithm;
extern EEncryptScope tsiEncryptScope;
// extern char tsAuthCode[];
extern char tsEncryptKey[];
extern int8_t tsEnableStrongPassword;
// common
extern int32_t tsMaxShellConns;

12
include/common/tmsg.h
View File

@ -490,6 +490,7 @@ typedef enum ENodeType {
typedef struct {
int32_t vgId;
uint8_t option; // 0x0 REQ_OPT_TBNAME, 0x01 REQ_OPT_TBUID
uint8_t autoCreateCtb; // 0x0 not auto create, 0x01 auto create
const char* dbFName;
const char* tbName;
} SBuildTableInput;
@ -1090,6 +1091,7 @@ typedef struct {
char* sql;
int8_t isImport;
int8_t createDb;
int8_t passIsMd5;
} SCreateUserReq;
int32_t tSerializeSCreateUserReq(void* buf, int32_t bufLen, SCreateUserReq* pReq);
@ -1160,6 +1162,7 @@ typedef struct {
int64_t privileges;
int32_t sqlLen;
char* sql;
int8_t passIsMd5;
} SAlterUserReq;
int32_t tSerializeSAlterUserReq(void* buf, int32_t bufLen, SAlterUserReq* pReq);
@ -2173,6 +2176,7 @@ typedef struct {
char dbFName[TSDB_DB_FNAME_LEN];
char tbName[TSDB_TABLE_NAME_LEN];
uint8_t option;
uint8_t autoCreateCtb;
} STableInfoReq;
int32_t tSerializeSTableInfoReq(void* buf, int32_t bufLen, STableInfoReq* pReq);
@ -2251,11 +2255,14 @@ typedef struct {
} STagData;
typedef struct {
int32_t useless; // useless
int32_t opType;
uint32_t valLen;
char* val;
} SShowVariablesReq;
int32_t tSerializeSShowVariablesReq(void* buf, int32_t bufLen, SShowVariablesReq* pReq);
// int32_t tDeserializeSShowVariablesReq(void* buf, int32_t bufLen, SShowVariablesReq* pReq);
int32_t tDeserializeSShowVariablesReq(void* buf, int32_t bufLen, SShowVariablesReq* pReq);
void tFreeSShowVariablesReq(SShowVariablesReq* pReq);
typedef struct {
char name[TSDB_CONFIG_OPTION_LEN + 1];
@ -3569,6 +3576,7 @@ typedef struct {
SArray* rsps; // SArray<SClientHbRsp>
SMonitorParas monitorParas;
int8_t enableAuditDelete;
int8_t enableStrongPass;
} SClientHbBatchRsp;
static FORCE_INLINE uint32_t hbKeyHashFunc(const char* key, uint32_t keyLen) { return taosIntHash_64(key, keyLen); }

1
include/libs/catalog/catalog.h
View File

@ -79,6 +79,7 @@ typedef struct SDbInfo {
typedef struct STablesReq {
char dbFName[TSDB_DB_FNAME_LEN];
SArray* pTables;
uint8_t autoCreate; // 0x0 not auto create, 0x01 auto create
} STablesReq;
typedef struct SCatalogReq {

18
include/libs/function/tudf.h
View File

@ -37,9 +37,9 @@ extern "C" {
#define UDF_LISTEN_PIPE_NAME_LEN 32
#ifdef _WIN32
#define UDF_LISTEN_PIPE_NAME_PREFIX "\\\\?\\pipe\\udfd.sock"
#define UDF_LISTEN_PIPE_NAME_PREFIX "\\\\?\\pipe\\taosudf.sock"
#else
#define UDF_LISTEN_PIPE_NAME_PREFIX ".udfd.sock."
#define UDF_LISTEN_PIPE_NAME_PREFIX ".taosudf.sock."
#endif
#define UDF_DNODE_ID_ENV_NAME "DNODE_ID"
@ -66,7 +66,7 @@ extern "C" {
const void *ptrs[] = {__VA_ARGS__}; \
for (int i = 0; i < sizeof(ptrs) / sizeof(ptrs[0]); ++i) { \
if (ptrs[i] == NULL) { \
fnError("udfd %dth parameter invalid, NULL PTR.line:%d", i, __LINE__); \
fnError("taosudf %dth parameter invalid, NULL PTR.line:%d", i, __LINE__); \
return TSDB_CODE_INVALID_PARA; \
} \
} \
@ -77,7 +77,7 @@ extern "C" {
const void *ptrs[] = {__VA_ARGS__}; \
for (int i = 0; i < sizeof(ptrs) / sizeof(ptrs[0]); ++i) { \
if (ptrs[i] == NULL) { \
fnError("udfd %dth parameter invalid, NULL PTR.line:%d", i, __LINE__); \
fnError("taosudf %dth parameter invalid, NULL PTR.line:%d", i, __LINE__); \
return; \
} \
} \
@ -137,31 +137,31 @@ int32_t cleanUpUdfs();
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// udf api
/**
* create udfd proxy, called once in process that call doSetupUdf/callUdfxxx/doTeardownUdf
* create taosudf proxy, called once in process that call doSetupUdf/callUdfxxx/doTeardownUdf
* @return error code
*/
int32_t udfcOpen();
/**
* destroy udfd proxy
* destroy taosudf proxy
* @return error code
*/
int32_t udfcClose();
/**
* start udfd that serves udf function invocation under dnode startDnodeId
* start taosudf that serves udf function invocation under dnode startDnodeId
* @param startDnodeId
* @return
*/
int32_t udfStartUdfd(int32_t startDnodeId);
/**
* stop udfd
* stop taosudf
* @return
*/
void udfStopUdfd();
/**
* get udfd pid
* get taosudf pid
*
*/
// int32_t udfGetUdfdPid(int32_t* pUdfdPid);

4
include/libs/nodes/cmdnodes.h
View File

@ -301,7 +301,7 @@ typedef struct SAlterTableMultiStmt {
typedef struct SCreateUserStmt {
ENodeType type;
char userName[TSDB_USER_LEN];
char password[TSDB_USET_PASSWORD_LEN];
char password[TSDB_USET_PASSWORD_LONGLEN];
int8_t sysinfo;
int8_t createDb;
int8_t isImport;
@ -315,7 +315,7 @@ typedef struct SAlterUserStmt {
ENodeType type;
char userName[TSDB_USER_LEN];
int8_t alterType;
char password[TSDB_USET_PASSWORD_LEN];
char password[TSDB_USET_PASSWORD_LONGLEN];
int8_t enable;
int8_t sysinfo;
int8_t createdb;

1
include/libs/nodes/querynodes.h
View File

@ -571,6 +571,7 @@ typedef struct SVnodeModifyOpStmt {
SHashObj* pVgroupsHashObj; // SHashObj<vgId, SVgInfo>
SHashObj* pTableBlockHashObj; // SHashObj<tuid, STableDataCxt*>
SHashObj* pSubTableHashObj; // SHashObj<table_name, STableMeta*>
SHashObj* pSuperTableHashObj; // SHashObj<table_name, STableMeta*>
SHashObj* pTableNameHashObj; // set of table names for refreshing meta, sync mode
SHashObj* pDbFNameHashObj; // set of db names for refreshing meta, sync mode
SHashObj* pTableCxtHashObj; // temp SHashObj<tuid, STableDataCxt*> for single request

1
include/util/tcompare.h
View File

@ -47,6 +47,7 @@ typedef struct SPatternCompareInfo {
int32_t InitRegexCache();
void DestroyRegexCache();
int32_t rawStrPatternMatch(const char *pattern, const char *str);
int32_t patternMatch(const char *pattern, size_t psize, const char *str, size_t ssize, const SPatternCompareInfo *pInfo);
int32_t checkRegexPattern(const char *pPattern);
void DestoryThreadLocalRegComp();

3
include/util/tdef.h
View File

@ -292,9 +292,10 @@ typedef enum ELogicConditionType {
#define TSDB_AUTH_LEN 16
#define TSDB_PASSWORD_MIN_LEN 8
#define TSDB_PASSWORD_MAX_LEN 16
#define TSDB_PASSWORD_MAX_LEN 255
#define TSDB_PASSWORD_LEN 32
#define TSDB_USET_PASSWORD_LEN 129
#define TSDB_USET_PASSWORD_LONGLEN 256
#define TSDB_VERSION_LEN 32
#define TSDB_LABEL_LEN 16
#define TSDB_JOB_STATUS_LEN 32

2
packaging/deb/DEBIAN/prerm
View File

@ -30,7 +30,7 @@ else
# Remove all links
${csudo}rm -f ${bin_link_dir}/taos || :
${csudo}rm -f ${bin_link_dir}/taosd || :
${csudo}rm -f ${bin_link_dir}/udfd || :
${csudo}rm -f ${bin_link_dir}/taosudf || :
${csudo}rm -f ${bin_link_dir}/taosadapter || :
${csudo}rm -f ${bin_link_dir}/taosdemo || :
${csudo}rm -f ${bin_link_dir}/taoskeeper || :

2
packaging/deb/makedeb.sh
View File

@ -103,7 +103,7 @@ sed -i "s/versionType=\"enterprise\"/versionType=\"community\"/g" ${pkg_dir}${in
cp ${compile_dir}/build/bin/taosd ${pkg_dir}${install_home_path}/bin
cp ${compile_dir}/build/bin/udfd ${pkg_dir}${install_home_path}/bin
cp ${compile_dir}/build/bin/taosudf ${pkg_dir}${install_home_path}/bin
cp ${compile_dir}/build/bin/taosBenchmark ${pkg_dir}${install_home_path}/bin
cp ${compile_dir}/build/bin/taosdump ${pkg_dir}${install_home_path}/bin

4
packaging/rpm/tdengine.spec
View File

@ -92,7 +92,7 @@ cp %{_compiledir}/../packaging/tools/set_core.sh %{buildroot}%{homepath}/bin
cp %{_compiledir}/../packaging/tools/taosd-dump-cfg.gdb %{buildroot}%{homepath}/bin
cp %{_compiledir}/build/bin/taos %{buildroot}%{homepath}/bin
cp %{_compiledir}/build/bin/taosd %{buildroot}%{homepath}/bin
cp %{_compiledir}/build/bin/udfd %{buildroot}%{homepath}/bin
cp %{_compiledir}/build/bin/taosudf %{buildroot}%{homepath}/bin
cp %{_compiledir}/build/bin/taosBenchmark %{buildroot}%{homepath}/bin
cp %{_compiledir}/build/bin/taosdump %{buildroot}%{homepath}/bin
cp %{_compiledir}/../../enterprise/packaging/start-all.sh %{buildroot}%{homepath}/bin
@ -233,7 +233,7 @@ if [ $1 -eq 0 ];then
# Remove all links
${csudo}rm -f ${bin_link_dir}/taos || :
${csudo}rm -f ${bin_link_dir}/taosd || :
${csudo}rm -f ${bin_link_dir}/udfd || :
${csudo}rm -f ${bin_link_dir}/taosudf || :
${csudo}rm -f ${bin_link_dir}/taosadapter || :
${csudo}rm -f ${bin_link_dir}/taoskeeper || :
${csudo}rm -f ${bin_link_dir}/taosdump || :

4
packaging/tools/install.sh
View File

@ -19,7 +19,7 @@ script_dir=$(dirname $(readlink -f "$0"))
PREFIX="taos"
clientName="${PREFIX}"
serverName="${PREFIX}d"
udfdName="udfd"
udfdName="taosudf"
configFile="${PREFIX}.cfg"
productName="TDengine"
emailName="taosdata.com"
@ -156,7 +156,7 @@ done
#echo "verType=${verType} interactiveFqdn=${interactiveFqdn}"
tools=(${clientName} ${benchmarkName} ${dumpName} ${demoName} remove.sh udfd set_core.sh TDinsight.sh start_pre.sh start-all.sh stop-all.sh)
tools=(${clientName} ${benchmarkName} ${dumpName} ${demoName} remove.sh taosudf set_core.sh TDinsight.sh start_pre.sh start-all.sh stop-all.sh)
if [ "${verMode}" == "cluster" ]; then
services=(${serverName} ${adapterName} ${xname} ${explorerName} ${keeperName})
elif [ "${verMode}" == "edge" ]; then

2
packaging/tools/make_install.bat
View File

@ -114,7 +114,7 @@ if %Enterprise% == TRUE (
)
copy %binary_dir%\\build\\bin\\taosd.exe %target_dir% > nul
copy %binary_dir%\\build\\bin\\udfd.exe %target_dir% > nul
copy %binary_dir%\\build\\bin\\taosudf.exe %target_dir% > nul
if exist %binary_dir%\\build\\bin\\taosadapter.exe (
copy %binary_dir%\\build\\bin\\taosadapter.exe %target_dir% > nul
)

10
packaging/tools/make_install.sh
View File

@ -171,7 +171,7 @@ function install_bin() {
${csudo}rm -f ${bin_link_dir}/${serverName} || :
${csudo}rm -f ${bin_link_dir}/taosadapter || :
${csudo}rm -f ${bin_link_dir}/taoskeeper || :
${csudo}rm -f ${bin_link_dir}/udfd || :
${csudo}rm -f ${bin_link_dir}/taosudf || :
${csudo}rm -f ${bin_link_dir}/taosdemo || :
${csudo}rm -f ${bin_link_dir}/taosdump || :
${csudo}rm -f ${bin_link_dir}/${uninstallScript} || :
@ -186,7 +186,7 @@ function install_bin() {
[ -f ${binary_dir}/build/bin/taosdump ] && ${csudo}cp -r ${binary_dir}/build/bin/taosdump ${install_main_dir}/bin || :
[ -f ${binary_dir}/build/bin/taosadapter ] && ${csudo}cp -r ${binary_dir}/build/bin/taosadapter ${install_main_dir}/bin || :
[ -f ${binary_dir}/build/bin/taoskeeper ] && ${csudo}cp -r ${binary_dir}/build/bin/taoskeeper ${install_main_dir}/bin || :
[ -f ${binary_dir}/build/bin/udfd ] && ${csudo}cp -r ${binary_dir}/build/bin/udfd ${install_main_dir}/bin || :
[ -f ${binary_dir}/build/bin/taosudf ] && ${csudo}cp -r ${binary_dir}/build/bin/taosudf ${install_main_dir}/bin || :
[ -f ${binary_dir}/build/bin/taosx ] && ${csudo}cp -r ${binary_dir}/build/bin/taosx ${install_main_dir}/bin || :
${csudo}cp -r ${binary_dir}/build/bin/${serverName} ${install_main_dir}/bin || :
@ -201,7 +201,7 @@ function install_bin() {
[ -x ${install_main_dir}/bin/${serverName} ] && ${csudo}ln -s ${install_main_dir}/bin/${serverName} ${bin_link_dir}/${serverName} > /dev/null 2>&1 || :
[ -x ${install_main_dir}/bin/taosadapter ] && ${csudo}ln -s ${install_main_dir}/bin/taosadapter ${bin_link_dir}/taosadapter > /dev/null 2>&1 || :
[ -x ${install_main_dir}/bin/taoskeeper ] && ${csudo}ln -s ${install_main_dir}/bin/taoskeeper ${bin_link_dir}/taoskeeper > /dev/null 2>&1 || :
[ -x ${install_main_dir}/bin/udfd ] && ${csudo}ln -s ${install_main_dir}/bin/udfd ${bin_link_dir}/udfd > /dev/null 2>&1 || :
[ -x ${install_main_dir}/bin/taosudf ] && ${csudo}ln -s ${install_main_dir}/bin/taosudf ${bin_link_dir}/taosudf > /dev/null 2>&1 || :
[ -x ${install_main_dir}/bin/taosdump ] && ${csudo}ln -s ${install_main_dir}/bin/taosdump ${bin_link_dir}/taosdump > /dev/null 2>&1 || :
[ -x ${install_main_dir}/bin/taosdemo ] && ${csudo}ln -s ${install_main_dir}/bin/taosdemo ${bin_link_dir}/taosdemo > /dev/null 2>&1 || :
[ -x ${install_main_dir}/bin/taosx ] && ${csudo}ln -s ${install_main_dir}/bin/taosx ${bin_link_dir}/taosx > /dev/null 2>&1 || :
@ -216,7 +216,7 @@ function install_bin() {
[ -f ${binary_dir}/build/bin/taosdump ] && ${csudo}cp -r ${binary_dir}/build/bin/taosdump ${install_main_dir}/bin || :
[ -f ${binary_dir}/build/bin/taosadapter ] && ${csudo}cp -r ${binary_dir}/build/bin/taosadapter ${install_main_dir}/bin || :
[ -f ${binary_dir}/build/bin/taoskeeper ] && ${csudo}cp -r ${binary_dir}/build/bin/taoskeeper ${install_main_dir}/bin || :
[ -f ${binary_dir}/build/bin/udfd ] && ${csudo}cp -r ${binary_dir}/build/bin/udfd ${install_main_dir}/bin || :
[ -f ${binary_dir}/build/bin/taosudf ] && ${csudo}cp -r ${binary_dir}/build/bin/taosudf ${install_main_dir}/bin || :
[ -f ${binary_dir}/build/bin/taosx ] && ${csudo}cp -r ${binary_dir}/build/bin/taosx ${install_main_dir}/bin || :
[ -f ${binary_dir}/build/bin/*explorer ] && ${csudo}cp -r ${binary_dir}/build/bin/*explorer ${install_main_dir}/bin || :
${csudo}cp -r ${binary_dir}/build/bin/${serverName} ${install_main_dir}/bin || :
@ -228,7 +228,7 @@ function install_bin() {
[ -x ${install_main_dir}/bin/${serverName} ] && ${csudo}ln -s ${install_main_dir}/bin/${serverName} ${bin_link_dir}/${serverName} > /dev/null 2>&1 || :
[ -x ${install_main_dir}/bin/taosadapter ] && ${csudo}ln -s ${install_main_dir}/bin/taosadapter ${bin_link_dir}/taosadapter > /dev/null 2>&1 || :
[ -x ${install_main_dir}/bin/taoskeeper ] && ${csudo}ln -s ${install_main_dir}/bin/taoskeeper ${bin_link_dir}/taoskeeper > /dev/null 2>&1 || :
[ -x ${install_main_dir}/bin/udfd ] && ${csudo}ln -s ${install_main_dir}/bin/udfd ${bin_link_dir}/udfd > /dev/null 2>&1 || :
[ -x ${install_main_dir}/bin/taosudf ] && ${csudo}ln -s ${install_main_dir}/bin/taosudf ${bin_link_dir}/taosudf > /dev/null 2>&1 || :
[ -x ${install_main_dir}/bin/taosdump ] && ${csudo}ln -s ${install_main_dir}/bin/taosdump ${bin_link_dir}/taosdump > /dev/null 2>&1 || :
[ -f ${install_main_dir}/bin/taosBenchmark ] && ${csudo}ln -sf ${install_main_dir}/bin/taosBenchmark ${install_main_dir}/bin/taosdemo > /dev/null 2>&1 || :
[ -x ${install_main_dir}/bin/taosx ] && ${csudo}ln -s ${install_main_dir}/bin/taosx ${bin_link_dir}/taosx > /dev/null 2>&1 || :

2
packaging/tools/makepkg.sh
View File

@ -98,7 +98,7 @@ else
${taostools_bin_files} \
${build_dir}/bin/${clientName}adapter \
${build_dir}/bin/${clientName}keeper \
${build_dir}/bin/udfd \
${build_dir}/bin/taosudf \
${script_dir}/remove.sh \
${script_dir}/set_core.sh \
${script_dir}/startPre.sh \

6
packaging/tools/post.sh
View File

@ -233,7 +233,7 @@ function install_bin() {
log_print "start install bin from ${bin_dir} to ${bin_link_dir}"
${csudo}rm -f ${bin_link_dir}/taos || :
${csudo}rm -f ${bin_link_dir}/taosd || :
${csudo}rm -f ${bin_link_dir}/udfd || :
${csudo}rm -f ${bin_link_dir}/taosudf || :
${csudo}rm -f ${bin_link_dir}/taosadapter || :
${csudo}rm -f ${bin_link_dir}/taosBenchmark || :
${csudo}rm -f ${bin_link_dir}/taoskeeper || :
@ -255,8 +255,8 @@ function install_bin() {
if [ -x ${bin_dir}/taosd ]; then
${csudo}ln -s ${bin_dir}/taosd ${bin_link_dir}/taosd 2>>${install_log_path} || return 1
fi
if [ -x ${bin_dir}/udfd ]; then
${csudo}ln -s ${bin_dir}/udfd ${bin_link_dir}/udfd 2>>${install_log_path} || return 1
if [ -x ${bin_dir}/taosudf ]; then
${csudo}ln -s ${bin_dir}/taosudf ${bin_link_dir}/taosudf 2>>${install_log_path} || return 1
fi
if [ -x ${bin_dir}/taosadapter ]; then
${csudo}ln -s ${bin_dir}/taosadapter ${bin_link_dir}/taosadapter 2>>${install_log_path} || return 1

2
packaging/tools/remove.sh
View File

@ -61,7 +61,7 @@ if [ "${verMode}" == "cluster" ]; then
else
services=(${PREFIX}"d" ${PREFIX}"adapter" ${PREFIX}"keeper" ${PREFIX}"-explorer")
fi
tools=(${PREFIX} ${PREFIX}"Benchmark" ${PREFIX}"dump" ${PREFIX}"demo" udfd set_core.sh TDinsight.sh $uninstallScript start-all.sh stop-all.sh)
tools=(${PREFIX} ${PREFIX}"Benchmark" ${PREFIX}"dump" ${PREFIX}"demo" taosudf set_core.sh TDinsight.sh $uninstallScript start-all.sh stop-all.sh)
csudo=""
if command -v sudo >/dev/null; then

1
source/client/inc/clientInt.h
View File

@ -115,6 +115,7 @@ typedef struct SQueryExecMetric {
typedef struct {
SMonitorParas monitorParas;
int8_t enableAuditDelete;
int8_t enableStrongPass;
} SAppInstServerCFG;
struct SAppInstInfo {
int64_t numOfConns;

2
source/client/src/clientHb.c
View File

@ -608,6 +608,8 @@ static int32_t hbAsyncCallBack(void *param, SDataBuf *pMsg, int32_t code) {
pInst->serverCfg.monitorParas = pRsp.monitorParas;
pInst->serverCfg.enableAuditDelete = pRsp.enableAuditDelete;
pInst->serverCfg.enableStrongPass = pRsp.enableStrongPass;
tsEnableStrongPassword = pInst->serverCfg.enableStrongPass;
tscDebug("monitor paras from hb, clusterId:0x%" PRIx64 ", threshold:%d scope:%d", pInst->clusterId,
pRsp.monitorParas.tsSlowLogThreshold, pRsp.monitorParas.tsSlowLogScope);

2
source/client/src/clientImpl.c
View File

@ -54,7 +54,7 @@ static bool stringLengthCheck(const char* str, size_t maxsize) {
static bool validateUserName(const char* user) { return stringLengthCheck(user, TSDB_USER_LEN - 1); }
static bool validatePassword(const char* passwd) { return stringLengthCheck(passwd, TSDB_PASSWORD_LEN - 1); }
static bool validatePassword(const char* passwd) { return stringLengthCheck(passwd, TSDB_PASSWORD_MAX_LEN); }
static bool validateDbName(const char* db) { return stringLengthCheck(db, TSDB_DB_NAME_LEN - 1); }

10
source/client/src/clientMsgHandler.c
View File

@ -630,14 +630,14 @@ static int32_t buildShowVariablesRsp(SArray* pVars, SRetrieveTableRsp** pRsp) {
(*pRsp)->numOfRows = htobe64((int64_t)pBlock->info.rows);
(*pRsp)->numOfCols = htonl(SHOW_VARIABLES_RESULT_COLS);
int32_t len = blockEncode(pBlock, (*pRsp)->data + PAYLOAD_PREFIX_LEN, dataEncodeBufSize, SHOW_VARIABLES_RESULT_COLS);
int32_t len = 0;
if ((*pRsp)->numOfRows > 0) {
len = blockEncode(pBlock, (*pRsp)->data + PAYLOAD_PREFIX_LEN, dataEncodeBufSize, SHOW_VARIABLES_RESULT_COLS);
if (len < 0) {
uError("buildShowVariablesRsp error, len:%d", len);
code = terrno;
goto _exit;
}
blockDataDestroy(pBlock);
SET_PAYLOAD_LEN((*pRsp)->data, len, len);
int32_t payloadLen = len + PAYLOAD_PREFIX_LEN;
@ -650,6 +650,10 @@ static int32_t buildShowVariablesRsp(SArray* pVars, SRetrieveTableRsp** pRsp) {
code = TSDB_CODE_TSC_INVALID_INPUT;
goto _exit;
}
}
blockDataDestroy(pBlock);
pBlock = NULL;
return TSDB_CODE_SUCCESS;
_exit:

4
source/client/test/stmt2Test.cpp
View File

@ -1927,8 +1927,10 @@ TEST(stmt2Case, async_order) {
while (!stop_task) {
auto elapsed_time = std::chrono::steady_clock::now() - start_time;
if (std::chrono::duration_cast<std::chrono::seconds>(elapsed_time).count() > 100) {
FAIL() << "Test[stmt2_async_test] timed out";
if (t.joinable()) {
t.detach();
}
FAIL() << "Test[stmt2_async_test] timed out";
break;
}
std::this_thread::sleep_for(std::chrono::seconds(1)); // 每 1s 检查一次

64
source/common/src/msg/tmsg.c
View File

@ -575,6 +575,7 @@ int32_t tSerializeSClientHbBatchRsp(void *buf, int32_t bufLen, const SClientHbBa
}
TAOS_CHECK_EXIT(tSerializeSMonitorParas(&encoder, &pBatchRsp->monitorParas));
TAOS_CHECK_EXIT(tEncodeI8(&encoder, pBatchRsp->enableAuditDelete));
TAOS_CHECK_EXIT(tEncodeI8(&encoder, pBatchRsp->enableStrongPass));
tEndEncode(&encoder);
_exit:
@ -623,6 +624,12 @@ int32_t tDeserializeSClientHbBatchRsp(void *buf, int32_t bufLen, SClientHbBatchR
pBatchRsp->enableAuditDelete = 0;
}
if (!tDecodeIsEnd(&decoder)) {
TAOS_CHECK_EXIT(tDecodeI8(&decoder, &pBatchRsp->enableStrongPass));
} else {
pBatchRsp->enableStrongPass = 0;
}
tEndDecode(&decoder);
_exit:
@ -2028,6 +2035,7 @@ int32_t tSerializeSCreateUserReq(void *buf, int32_t bufLen, SCreateUserReq *pReq
ENCODESQL();
TAOS_CHECK_EXIT(tEncodeI8(&encoder, pReq->isImport));
TAOS_CHECK_EXIT(tEncodeI8(&encoder, pReq->createDb));
TAOS_CHECK_EXIT(tEncodeI8(&encoder, pReq->passIsMd5));
tEndEncode(&encoder);
@ -2068,6 +2076,9 @@ int32_t tDeserializeSCreateUserReq(void *buf, int32_t bufLen, SCreateUserReq *pR
TAOS_CHECK_EXIT(tDecodeI8(&decoder, &pReq->createDb));
TAOS_CHECK_EXIT(tDecodeI8(&decoder, &pReq->isImport));
}
if (!tDecodeIsEnd(&decoder)) {
TAOS_CHECK_EXIT(tDecodeI8(&decoder, &pReq->passIsMd5));
}
tEndDecode(&decoder);
@ -2423,6 +2434,7 @@ int32_t tSerializeSAlterUserReq(void *buf, int32_t bufLen, SAlterUserReq *pReq)
TAOS_CHECK_EXIT(tEncodeI64(&encoder, pReq->privileges));
ENCODESQL();
TAOS_CHECK_EXIT(tEncodeU8(&encoder, pReq->flag));
TAOS_CHECK_EXIT(tEncodeU8(&encoder, pReq->passIsMd5));
tEndEncode(&encoder);
_exit:
@ -2474,6 +2486,9 @@ int32_t tDeserializeSAlterUserReq(void *buf, int32_t bufLen, SAlterUserReq *pReq
if (!tDecodeIsEnd(&decoder)) {
TAOS_CHECK_EXIT(tDecodeU8(&decoder, &pReq->flag));
}
if (!tDecodeIsEnd(&decoder)) {
TAOS_CHECK_EXIT(tDecodeU8(&decoder, &pReq->passIsMd5));
}
tEndDecode(&decoder);
_exit:
@ -5822,7 +5837,11 @@ int32_t tSerializeSShowVariablesReq(void *buf, int32_t bufLen, SShowVariablesReq
tEncoderInit(&encoder, buf, bufLen);
TAOS_CHECK_EXIT(tStartEncode(&encoder));
TAOS_CHECK_EXIT(tEncodeI32(&encoder, pReq->useless));
TAOS_CHECK_EXIT(tEncodeI32(&encoder, pReq->opType));
TAOS_CHECK_EXIT(tEncodeI32(&encoder, pReq->valLen));
if (pReq->valLen > 0) {
TAOS_CHECK_EXIT(tEncodeBinary(&encoder, (const uint8_t *)pReq->val, pReq->valLen));
}
tEndEncode(&encoder);
_exit:
@ -5835,6 +5854,37 @@ _exit:
return tlen;
}
int32_t tDeserializeSShowVariablesReq(void *buf, int32_t bufLen, SShowVariablesReq *pReq) {
SDecoder decoder = {0};
int32_t code = 0;
int32_t lino;
tDecoderInit(&decoder, buf, bufLen);
TAOS_CHECK_EXIT(tStartDecode(&decoder));
TAOS_CHECK_EXIT(tDecodeI32(&decoder, &pReq->opType));
TAOS_CHECK_EXIT(tDecodeU32(&decoder, &pReq->valLen));
if (pReq->valLen > 0) {
pReq->val = taosMemoryCalloc(1, pReq->valLen + 1);
if (pReq->val == NULL) {
TAOS_CHECK_EXIT(terrno);
}
TAOS_CHECK_EXIT(tDecodeCStrTo(&decoder, pReq->val));
}
tEndDecode(&decoder);
_exit:
tDecoderClear(&decoder);
return code;
}
void tFreeSShowVariablesReq(SShowVariablesReq *pReq) {
if (NULL != pReq && NULL != pReq->val) {
taosMemoryFree(pReq->val);
pReq->val = NULL;
}
}
int32_t tEncodeSVariablesInfo(SEncoder *pEncoder, SVariablesInfo *pInfo) {
TAOS_CHECK_RETURN(tEncodeCStr(pEncoder, pInfo->name));
TAOS_CHECK_RETURN(tEncodeCStr(pEncoder, pInfo->value));
@ -6293,6 +6343,7 @@ int32_t tSerializeSTableInfoReq(void *buf, int32_t bufLen, STableInfoReq *pReq)
TAOS_CHECK_EXIT(tEncodeCStr(&encoder, pReq->dbFName));
TAOS_CHECK_EXIT(tEncodeCStr(&encoder, pReq->tbName));
TAOS_CHECK_EXIT(tEncodeU8(&encoder, pReq->option));
TAOS_CHECK_EXIT(tEncodeU8(&encoder, pReq->autoCreateCtb));
tEndEncode(&encoder);
_exit:
@ -6332,6 +6383,11 @@ int32_t tDeserializeSTableInfoReq(void *buf, int32_t bufLen, STableInfoReq *pReq
} else {
pReq->option = 0;
}
if (!tDecodeIsEnd(&decoder)) {
TAOS_CHECK_EXIT(tDecodeU8(&decoder, &pReq->autoCreateCtb));
} else {
pReq->autoCreateCtb = 0;
}
tEndDecode(&decoder);
_exit:
@ -11947,7 +12003,11 @@ int32_t tEncodeSubmitReq(SEncoder *pCoder, const SSubmitReq2 *pReq) {
}
} else{
for (uint64_t i = 0; i < taosArrayGetSize(pReq->aSubmitTbData); i++) {
TAOS_CHECK_EXIT(tEncodeSSubmitTbData(pCoder, taosArrayGet(pReq->aSubmitTbData, i)));
SSubmitTbData *pSubmitTbData = taosArrayGet(pReq->aSubmitTbData, i);
if ((pSubmitTbData->flags & SUBMIT_REQ_AUTO_CREATE_TABLE) && pSubmitTbData->pCreateTbReq == NULL) {
pSubmitTbData->flags = 0;
}
TAOS_CHECK_EXIT(tEncodeSSubmitTbData(pCoder, pSubmitTbData));
}
}

10
source/common/src/tglobal.c
View File

@ -54,6 +54,7 @@ EEncryptScope tsiEncryptScope = 0;
// char tsAuthCode[500] = {0};
// char tsEncryptKey[17] = {0};
char tsEncryptKey[17] = {0};
int8_t tsEnableStrongPassword = 1;
// common
int32_t tsMaxShellConns = 50000;
@ -317,7 +318,7 @@ int32_t tsS3MigrateIntervalSec = 60 * 60; // interval of s3migrate db in all vg
bool tsS3MigrateEnabled = 0;
int32_t tsGrantHBInterval = 60;
int32_t tsUptimeInterval = 300; // seconds
char tsUdfdResFuncs[512] = ""; // udfd resident funcs that teardown when udfd exits
char tsUdfdResFuncs[512] = ""; // taosudf resident funcs that teardown when taosudf exits
char tsUdfdLdLibPath[512] = "";
bool tsDisableStream = false;
int64_t tsStreamBufferSize = 128 * 1024 * 1024;
@ -835,6 +836,7 @@ static int32_t taosAddServerCfg(SConfig *pCfg) {
TAOS_CHECK_RETURN(cfgAddString(pCfg, "encryptAlgorithm", tsEncryptAlgorithm, CFG_SCOPE_SERVER, CFG_DYN_NONE, CFG_CATEGORY_GLOBAL));
TAOS_CHECK_RETURN(cfgAddString(pCfg, "encryptScope", tsEncryptScope, CFG_SCOPE_SERVER, CFG_DYN_NONE,CFG_CATEGORY_GLOBAL));
TAOS_CHECK_RETURN(cfgAddBool(pCfg, "enableStrongPassword", tsEnableStrongPassword, CFG_SCOPE_SERVER, CFG_DYN_SERVER,CFG_CATEGORY_GLOBAL));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "statusInterval", tsStatusInterval, 1, 30, CFG_SCOPE_SERVER, CFG_DYN_SERVER_LAZY,CFG_CATEGORY_GLOBAL));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "maxShellConns", tsMaxShellConns, 10, 50000000, CFG_SCOPE_SERVER, CFG_DYN_SERVER_LAZY, CFG_CATEGORY_LOCAL));
@ -1527,6 +1529,9 @@ static int32_t taosSetServerCfg(SConfig *pCfg) {
TAOS_CHECK_RETURN(taosCheckCfgStrValueLen(pItem->name, pItem->str, 100));
tstrncpy(tsEncryptScope, pItem->str, 100);
TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "enableStrongPassword");
tsEnableStrongPassword = pItem->i32;
TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "numOfRpcThreads");
tsNumOfRpcThreads = pItem->i32;
@ -2524,7 +2529,8 @@ static int32_t taosCfgDynamicOptionsForServer(SConfig *pCfg, const char *name) {
{"arbHeartBeatIntervalSec", &tsArbHeartBeatIntervalSec},
{"arbCheckSyncIntervalSec", &tsArbCheckSyncIntervalSec},
{"arbSetAssignedTimeoutSec", &tsArbSetAssignedTimeoutSec},
{"queryNoFetchTimeoutSec", &tsQueryNoFetchTimeoutSec}};
{"queryNoFetchTimeoutSec", &tsQueryNoFetchTimeoutSec},
{"enableStrongPassword", &tsEnableStrongPassword}};
if ((code = taosCfgSetOption(debugOptions, tListLen(debugOptions), pItem, true)) != TSDB_CODE_SUCCESS) {
code = taosCfgSetOption(options, tListLen(options), pItem, false);

6
source/common/src/tmisce.c
View File

@ -18,6 +18,7 @@
#include "tglobal.h"
#include "tjson.h"
#include "tmisce.h"
#include "tcompare.h"
int32_t taosGetFqdnPortFromEp(const char* ep, SEp* pEp) {
pEp->port = 0;
@ -257,7 +258,7 @@ _exit:
TAOS_RETURN(code);
}
int32_t dumpConfToDataBlock(SSDataBlock* pBlock, int32_t startCol) {
int32_t dumpConfToDataBlock(SSDataBlock* pBlock, int32_t startCol, char* likePattern) {
int32_t code = 0;
SConfig* pConf = taosGetCfg();
if (pConf == NULL) {
@ -291,6 +292,9 @@ int32_t dumpConfToDataBlock(SSDataBlock* pBlock, int32_t startCol) {
// GRANT_CFG_SKIP;
char name[TSDB_CONFIG_OPTION_LEN + VARSTR_HEADER_SIZE] = {0};
if (likePattern && rawStrPatternMatch(pItem->name, likePattern) != TSDB_PATTERN_MATCH) {
continue;
}
STR_WITH_MAXSIZE_TO_VARSTR(name, pItem->name, TSDB_CONFIG_OPTION_LEN + VARSTR_HEADER_SIZE);
SColumnInfoData* pColInfo = taosArrayGet(pBlock->pDataBlock, col++);

24
source/dnode/mgmt/CMakeLists.txt
View File

@ -14,24 +14,12 @@ target_include_directories(
PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/node_mgmt/inc"
)
IF(TD_ENTERPRISE)
IF(${BUILD_WITH_S3})
if(TD_ENTERPRISE)
if(${BUILD_WITH_S3})
add_definitions(-DUSE_S3)
ELSEIF(${BUILD_WITH_COS})
elseif(${BUILD_WITH_COS})
add_definitions(-DUSE_COS)
ENDIF()
ENDIF()
endif()
endif()
IF(TD_LINUX_64 AND JEMALLOC_ENABLED)
ADD_DEFINITIONS(-DTD_JEMALLOC_ENABLED -I${CMAKE_BINARY_DIR}/build/include -L${CMAKE_BINARY_DIR}/build/lib -Wl,-rpath,${CMAKE_BINARY_DIR}/build/lib -ljemalloc)
SET(LINK_JEMALLOC "-L${CMAKE_BINARY_DIR}/build/lib -ljemalloc")
ELSE()
SET(LINK_JEMALLOC "")
ENDIF()
IF(TD_LINUX_64 AND JEMALLOC_ENABLED)
ADD_DEPENDENCIES(taosd jemalloc)
target_link_libraries(taosd dnode crypt ${LINK_JEMALLOC})
ELSE()
target_link_libraries(taosd dnode crypt)
ENDIF()
target_link_libraries(taosd dnode crypt)

2
source/dnode/mgmt/mgmt_dnode/src/dmHandle.c
View File

@ -690,7 +690,7 @@ _exit:
}
int32_t dmAppendVariablesToBlock(SSDataBlock *pBlock, int32_t dnodeId) {
int32_t code = dumpConfToDataBlock(pBlock, 1);
int32_t code = dumpConfToDataBlock(pBlock, 1, NULL);
if (code != 0) {
return code;
}

2
source/dnode/mgmt/mgmt_dnode/src/dmInt.c
View File

@ -88,7 +88,7 @@ static int32_t dmOpenMgmt(SMgmtInputOpt *pInput, SMgmtOutputOpt *pOutput) {
}
if ((code = udfStartUdfd(pMgmt->pData->dnodeId)) != 0) {
dError("failed to start udfd since %s", tstrerror(code));
dError("failed to start taosudf since %s", tstrerror(code));
}
if ((code = taosAnalyticsInit()) != 0) {

22
source/dnode/mnode/impl/src/mndConfig.c
View File

@ -23,6 +23,7 @@
#include "mndTrans.h"
#include "mndUser.h"
#include "tutil.h"
#include "tcompare.h"
#define CFG_VER_NUMBER 1
#define CFG_RESERVE_SIZE 63
@ -809,7 +810,7 @@ static void cfgObjArrayCleanUp(SArray *array) {
taosArrayDestroy(array);
}
SArray *initVariablesFromItems(SArray *pItems) {
static SArray *initVariablesFromItems(SArray *pItems, const char* likePattern) {
if (pItems == NULL) {
return NULL;
}
@ -825,6 +826,9 @@ SArray *initVariablesFromItems(SArray *pItems) {
SConfigItem *pItem = taosArrayGet(pItems, i);
SVariablesInfo info = {0};
tstrncpy(info.name, pItem->name, sizeof(info.name));
if (likePattern != NULL && rawStrPatternMatch(pItem->name, likePattern) != TSDB_PATTERN_MATCH) {
continue;
}
// init info value
switch (pItem->dtype) {
@ -891,15 +895,23 @@ SArray *initVariablesFromItems(SArray *pItems) {
static int32_t mndProcessShowVariablesReq(SRpcMsg *pReq) {
SShowVariablesRsp rsp = {0};
int32_t code = -1;
int32_t code = TSDB_CODE_SUCCESS;
SShowVariablesReq req = {0};
SArray *array = NULL;
code = tDeserializeSShowVariablesReq(pReq->pCont, pReq->contLen, &req);
if (code != 0) {
mError("failed to deserialize config req, since %s", terrstr());
goto _OVER;
}
if ((code = mndCheckOperPrivilege(pReq->info.node, pReq->info.conn.user, MND_OPER_SHOW_VARIABLES)) != 0) {
goto _OVER;
}
SVariablesInfo info = {0};
rsp.variables = initVariablesFromItems(taosGetGlobalCfg(tsCfg));
char *likePattern = req.opType == OP_TYPE_LIKE ? req.val : NULL;
rsp.variables = initVariablesFromItems(taosGetGlobalCfg(tsCfg), likePattern);
if (rsp.variables == NULL) {
code = terrno;
goto _OVER;
@ -926,7 +938,7 @@ _OVER:
if (code != 0) {
mError("failed to get show variables info since %s", tstrerror(code));
}
tFreeSShowVariablesReq(&req);
tFreeSShowVariablesRsp(&rsp);
TAOS_RETURN(code);
}

1
source/dnode/mnode/impl/src/mndProfile.c
View File

@ -722,6 +722,7 @@ static int32_t mndProcessHeartBeatReq(SRpcMsg *pReq) {
batchRsp.monitorParas.tsSlowLogMaxLen = tsSlowLogMaxLen;
batchRsp.monitorParas.tsSlowLogScope = tsSlowLogScope;
batchRsp.enableAuditDelete = tsEnableAuditDelete;
batchRsp.enableStrongPass = tsEnableStrongPassword;
int32_t sz = taosArrayGetSize(batchReq.reqs);
for (int i = 0; i < sz; i++) {

22
source/dnode/mnode/impl/src/mndUser.c
View File

@ -1705,12 +1705,18 @@ static int32_t mndCreateUser(SMnode *pMnode, char *acct, SCreateUserReq *pCreate
int32_t code = 0;
int32_t lino = 0;
SUserObj userObj = {0};
if (pCreate->passIsMd5 == 1) {
memcpy(userObj.pass, pCreate->pass, TSDB_PASSWORD_LEN);
} else {
if (pCreate->isImport != 1) {
taosEncryptPass_c((uint8_t *)pCreate->pass, strlen(pCreate->pass), userObj.pass);
} else {
// mInfo("pCreate->pass:%s", pCreate->eass)
memcpy(userObj.pass, pCreate->pass, TSDB_PASSWORD_LEN);
}
}
tstrncpy(userObj.user, pCreate->user, TSDB_USER_LEN);
tstrncpy(userObj.acct, acct, TSDB_USER_LEN);
userObj.createdTime = taosGetTimestampMs();
@ -1884,6 +1890,7 @@ static int32_t mndProcessCreateUserReq(SRpcMsg *pReq) {
TAOS_CHECK_GOTO(TSDB_CODE_MND_INVALID_USER_FORMAT, &lino, _OVER);
}
if(createReq.passIsMd5 == 0){
int32_t len = strlen(createReq.pass);
if (createReq.isImport != 1) {
if (mndCheckPasswordMinLen(createReq.pass, len) != 0) {
@ -1896,6 +1903,7 @@ static int32_t mndProcessCreateUserReq(SRpcMsg *pReq) {
TAOS_CHECK_GOTO(TSDB_CODE_MND_INVALID_PASS_FORMAT, &lino, _OVER);
}
}
}
code = mndAcquireUser(pMnode, createReq.user, &pUser);
if (pUser != NULL) {
@ -2375,7 +2383,7 @@ static int32_t mndProcessAlterUserReq(SRpcMsg *pReq) {
if (alterReq.user[0] == 0) {
TAOS_CHECK_GOTO(TSDB_CODE_MND_INVALID_USER_FORMAT, &lino, _OVER);
}
if(alterReq.passIsMd5 == 0){
if (TSDB_ALTER_USER_PASSWD == alterReq.alterType) {
int32_t len = strlen(alterReq.pass);
if (mndCheckPasswordMinLen(alterReq.pass, len) != 0) {
@ -2388,6 +2396,7 @@ static int32_t mndProcessAlterUserReq(SRpcMsg *pReq) {
TAOS_CHECK_GOTO(TSDB_CODE_MND_INVALID_PASS_FORMAT, &lino, _OVER);
}
}
}
TAOS_CHECK_GOTO(mndAcquireUser(pMnode, alterReq.user, &pUser), &lino, _OVER);
@ -2401,10 +2410,13 @@ static int32_t mndProcessAlterUserReq(SRpcMsg *pReq) {
TAOS_CHECK_GOTO(mndUserDupObj(pUser, &newUser), &lino, _OVER);
if (alterReq.alterType == TSDB_ALTER_USER_PASSWD) {
char pass[TSDB_PASSWORD_LEN + 1] = {0};
taosEncryptPass_c((uint8_t *)alterReq.pass, strlen(alterReq.pass), pass);
(void)memcpy(newUser.pass, pass, TSDB_PASSWORD_LEN);
if (0 != strncmp(pUser->pass, pass, TSDB_PASSWORD_LEN)) {
if (alterReq.passIsMd5 == 1) {
(void)memcpy(newUser.pass, alterReq.pass, TSDB_PASSWORD_LEN);
} else {
taosEncryptPass_c((uint8_t *)alterReq.pass, strlen(alterReq.pass), newUser.pass);
}
if (0 != strncmp(pUser->pass, newUser.pass, TSDB_PASSWORD_LEN)) {
++newUser.passVersion;
}
}

6
source/dnode/vnode/src/vnd/vnodeQuery.c
View File

@ -90,12 +90,14 @@ int32_t vnodeGetTableMeta(SVnode *pVnode, SRpcMsg *pMsg, bool direct) {
void *pRsp = NULL;
SSchemaWrapper schema = {0};
SSchemaWrapper schemaTag = {0};
uint8_t autoCreateCtb = 0;
// decode req
if (tDeserializeSTableInfoReq(pMsg->pCont, pMsg->contLen, &infoReq) != 0) {
code = terrno;
goto _exit4;
}
autoCreateCtb = infoReq.autoCreateCtb;
if (infoReq.option == REQ_OPT_TBUID) reqTbUid = true;
metaRsp.dbId = pVnode->config.dbId;
@ -223,6 +225,10 @@ _exit4:
rpcMsg.code = code;
rpcMsg.msgType = pMsg->msgType;
if (code == TSDB_CODE_PAR_TABLE_NOT_EXIST && autoCreateCtb == 1) {
code = TSDB_CODE_SUCCESS;
}
if (code) {
qError("get table %s meta with %" PRIu8 " failed cause of %s", infoReq.tbName, infoReq.option, tstrerror(code));
}

9
source/dnode/vnode/src/vnd/vnodeSync.c
View File

@ -169,8 +169,9 @@ void vnodeProposeCommitOnNeed(SVnode *pVnode, bool atExit) {
rpcFreeCont(rpcMsg.pCont);
rpcMsg.pCont = NULL;
} else {
if (tmsgPutToQueue(&pVnode->msgCb, WRITE_QUEUE, &rpcMsg) < 0) {
vTrace("vgId:%d, failed to put vnode commit to queue since %s", pVnode->config.vgId, terrstr());
int32_t code = 0;
if ((code = tmsgPutToQueue(&pVnode->msgCb, WRITE_QUEUE, &rpcMsg)) < 0) {
vError("vgId:%d, failed to put vnode commit to write_queue since %s", pVnode->config.vgId, tstrerror(code));
}
}
}
@ -449,7 +450,9 @@ static int32_t vnodeSyncApplyMsg(const SSyncFSM *pFsm, SRpcMsg *pMsg, const SFsm
pVnode->config.vgId, pFsm, pMeta->index, pMeta->term, pMsg->info.conn.applyIndex, pMeta->isWeak, pMeta->code,
pMeta->state, syncStr(pMeta->state), TMSG_INFO(pMsg->msgType), pMsg->code);
return tmsgPutToQueue(&pVnode->msgCb, APPLY_QUEUE, pMsg);
int32_t code = tmsgPutToQueue(&pVnode->msgCb, APPLY_QUEUE, pMsg);
if (code < 0) vError("vgId:%d, failed to put into apply_queue since %s", pVnode->config.vgId, tstrerror(code));
return code;
}
static int32_t vnodeSyncCommitMsg(const SSyncFSM *pFsm, SRpcMsg *pMsg, SFsmCbMeta *pMeta) {

1
source/libs/catalog/inc/catalogInt.h
View File

@ -481,6 +481,7 @@ struct SCtgTask {
typedef struct SCtgTaskReq {
SCtgTask* pTask;
int32_t msgIdx;
uint8_t autoCreateCtb;
} SCtgTaskReq;
typedef int32_t (*ctgInitTaskFp)(SCtgJob*, int32_t, void*);

3
source/libs/catalog/src/ctgAsync.c
View File

@ -3093,6 +3093,7 @@ int32_t ctgLaunchGetTbMetasTask(SCtgTask* pTask) {
SCtgTbMetasCtx* pCtx = (SCtgTbMetasCtx*)pTask->taskCtx;
SCtgJob* pJob = pTask->pJob;
SName* pName = NULL;
bool autoCreate = false;
int32_t dbNum = taosArrayGetSize(pCtx->pNames);
int32_t fetchIdx = 0;
@ -3103,6 +3104,7 @@ int32_t ctgLaunchGetTbMetasTask(SCtgTask* pTask) {
ctgError("fail to get the %dth STablesReq, num:%d", i, dbNum);
CTG_ERR_RET(TSDB_CODE_CTG_INVALID_INPUT);
}
autoCreate = pReq->autoCreate;
ctgDebug("start to check tb metas in db:%s, tbNum:%d", pReq->dbFName, (int32_t)taosArrayGetSize(pReq->pTables));
CTG_ERR_RET(ctgGetTbMetasFromCache(pCtg, pConn, pCtx, i, &fetchIdx, baseResIdx, pReq->pTables));
@ -3143,6 +3145,7 @@ int32_t ctgLaunchGetTbMetasTask(SCtgTask* pTask) {
}
SCtgTaskReq tReq;
tReq.autoCreateCtb = (autoCreate && i == pCtx->fetchNum - 1) ? 1 : 0;
tReq.pTask = pTask;
tReq.msgIdx = pFetch->fetchIdx;
CTG_ERR_RET(ctgAsyncRefreshTbMeta(&tReq, pFetch->flag, pName, &pFetch->vgId));

2
source/libs/catalog/src/ctgRemote.c
View File

@ -1367,6 +1367,7 @@ int32_t ctgGetTbMetaFromMnode(SCatalog* pCtg, SRequestConnInfo* pConn, const SNa
int32_t ctgGetTbMetaFromVnode(SCatalog* pCtg, SRequestConnInfo* pConn, const SName* pTableName, SVgroupInfo* vgroupInfo,
STableMetaOutput* out, SCtgTaskReq* tReq) {
SCtgTask* pTask = tReq ? tReq->pTask : NULL;
uint8_t autoCreateCtb = tReq ? tReq->autoCreateCtb : 0;
char dbFName[TSDB_DB_FNAME_LEN];
(void)tNameGetFullDbName(pTableName, dbFName);
int32_t reqType = (pTask && pTask->type == CTG_TASK_GET_TB_NAME ? TDMT_VND_TABLE_NAME : TDMT_VND_TABLE_META);
@ -1380,6 +1381,7 @@ int32_t ctgGetTbMetaFromVnode(SCatalog* pCtg, SRequestConnInfo* pConn, const SNa
SBuildTableInput bInput = {.vgId = vgroupInfo->vgId,
.option = reqType == TDMT_VND_TABLE_NAME ? REQ_OPT_TBUID : REQ_OPT_TBNAME,
.autoCreateCtb = autoCreateCtb,
.dbFName = dbFName,
.tbName = (char*)tNameGetTableName(pTableName)};
char* msg = NULL;

18
source/libs/command/src/command.c
View File

@ -20,6 +20,7 @@
#include "systable.h"
#include "taosdef.h"
#include "tdatablock.h"
#include "tdataformat.h"
#include "tglobal.h"
#include "tgrant.h"
@ -53,13 +54,16 @@ static int32_t buildRetrieveTableRsp(SSDataBlock* pBlock, int32_t numOfCols, SRe
(*pRsp)->numOfRows = htobe64((int64_t)pBlock->info.rows);
(*pRsp)->numOfCols = htonl(numOfCols);
int32_t len = blockEncode(pBlock, (*pRsp)->data + PAYLOAD_PREFIX_LEN, dataEncodeBufSize, numOfCols);
int32_t len = 0;
if (pBlock->info.rows > 0) {
len = blockEncode(pBlock, (*pRsp)->data + PAYLOAD_PREFIX_LEN, dataEncodeBufSize, numOfCols);
if (len < 0) {
taosMemoryFree(*pRsp);
*pRsp = NULL;
return terrno;
}
SET_PAYLOAD_LEN((*pRsp)->data, len, len);
}
int32_t payloadLen = len + PAYLOAD_PREFIX_LEN;
(*pRsp)->payloadLen = htonl(payloadLen);
@ -985,11 +989,17 @@ _exit:
return terrno;
}
static int32_t execShowLocalVariables(SRetrieveTableRsp** pRsp) {
static int32_t execShowLocalVariables(SShowStmt* pStmt, SRetrieveTableRsp** pRsp) {
SSDataBlock* pBlock = NULL;
char* likePattern = NULL;
int32_t code = buildLocalVariablesResultDataBlock(&pBlock);
if (TSDB_CODE_SUCCESS == code) {
code = dumpConfToDataBlock(pBlock, 0);
if (pStmt->tableCondType == OP_TYPE_LIKE) {
likePattern = ((SValueNode*)pStmt->pTbName)->literal;
}
}
if (TSDB_CODE_SUCCESS == code) {
code = dumpConfToDataBlock(pBlock, 0, likePattern);
}
if (TSDB_CODE_SUCCESS == code) {
code = buildRetrieveTableRsp(pBlock, SHOW_LOCAL_VARIABLES_RESULT_COLS, pRsp);
@ -1091,7 +1101,7 @@ int32_t qExecCommand(int64_t* pConnId, bool sysInfoUser, SNode* pStmt, SRetrieve
case QUERY_NODE_ALTER_LOCAL_STMT:
return execAlterLocal((SAlterLocalStmt*)pStmt);
case QUERY_NODE_SHOW_LOCAL_VARIABLES_STMT:
return execShowLocalVariables(pRsp);
return execShowLocalVariables((SShowStmt*)pStmt, pRsp);
case QUERY_NODE_SELECT_STMT:
return execSelectWithoutFrom((SSelectStmt*)pStmt, pRsp);
default:

182
source/libs/function/CMakeLists.txt
View File

@ -1,11 +1,14 @@
aux_source_directory(src FUNCTION_SRC)
aux_source_directory(src/detail FUNCTION_SRC_DETAIL)
list(REMOVE_ITEM FUNCTION_SRC src/udfd.c)
IF(COMPILER_SUPPORT_AVX2)
if(COMPILER_SUPPORT_AVX2)
MESSAGE(STATUS "AVX2 instructions is ACTIVATED")
set_source_files_properties(src/detail/tminmaxavx.c PROPERTIES COMPILE_FLAGS -mavx2)
ENDIF()
endif()
add_library(function STATIC ${FUNCTION_SRC} ${FUNCTION_SRC_DETAIL})
target_include_directories(
function
PUBLIC
@ -17,17 +20,6 @@ target_include_directories(
PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
)
IF(TD_LINUX_64 AND JEMALLOC_ENABLED)
ADD_DEFINITIONS(-DTD_JEMALLOC_ENABLED -I${CMAKE_BINARY_DIR}/build/include -L${CMAKE_BINARY_DIR}/build/lib -Wl,-rpath,${CMAKE_BINARY_DIR}/build/lib -ljemalloc)
SET(LINK_JEMALLOC "-L${CMAKE_BINARY_DIR}/build/lib -ljemalloc")
ELSE()
SET(LINK_JEMALLOC "")
ENDIF()
IF(TD_LINUX_64 AND JEMALLOC_ENABLED)
ADD_DEPENDENCIES(function jemalloc)
ENDIF()
target_link_libraries(
function
PRIVATE os
@ -41,160 +33,14 @@ target_link_libraries(
PUBLIC uv_a
)
add_executable(runUdf test/runUdf.c)
target_include_directories(
runUdf
PUBLIC
"${TD_SOURCE_DIR}/include/libs/function"
"${TD_SOURCE_DIR}/contrib/libuv/include"
"${TD_SOURCE_DIR}/include/util"
"${TD_SOURCE_DIR}/include/common"
"${TD_SOURCE_DIR}/include/client"
"${TD_SOURCE_DIR}/include/os"
PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
)
IF(TD_LINUX_64 AND JEMALLOC_ENABLED)
ADD_DEPENDENCIES(runUdf jemalloc)
ENDIF()
target_link_libraries(
runUdf
PUBLIC uv_a
PRIVATE os util common nodes function ${LINK_JEMALLOC}
)
add_library(udf1 STATIC MODULE test/udf1.c)
target_include_directories(
udf1
PUBLIC
"${TD_SOURCE_DIR}/include/libs/function"
"${TD_SOURCE_DIR}/include/util"
"${TD_SOURCE_DIR}/include/common"
"${TD_SOURCE_DIR}/include/client"
"${TD_SOURCE_DIR}/include/os"
PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
)
IF(TD_LINUX_64 AND JEMALLOC_ENABLED)
ADD_DEPENDENCIES(udf1 jemalloc)
ENDIF()
target_link_libraries(
udf1 PUBLIC os ${LINK_JEMALLOC})
add_library(udf1_dup STATIC MODULE test/udf1_dup.c)
target_include_directories(
udf1_dup
PUBLIC
"${TD_SOURCE_DIR}/include/libs/function"
"${TD_SOURCE_DIR}/include/util"
"${TD_SOURCE_DIR}/include/common"
"${TD_SOURCE_DIR}/include/client"
"${TD_SOURCE_DIR}/include/os"
PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
)
IF(TD_LINUX_64 AND JEMALLOC_ENABLED)
ADD_DEPENDENCIES(udf1_dup jemalloc)
ENDIF()
target_link_libraries(
udf1_dup PUBLIC os ${LINK_JEMALLOC})
add_library(udf2 STATIC MODULE test/udf2.c)
target_include_directories(
udf2
PUBLIC
"${TD_SOURCE_DIR}/include/libs/function"
"${TD_SOURCE_DIR}/include/util"
"${TD_SOURCE_DIR}/include/common"
"${TD_SOURCE_DIR}/include/client"
"${TD_SOURCE_DIR}/include/os"
PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
)
IF(TD_LINUX_64 AND JEMALLOC_ENABLED)
ADD_DEPENDENCIES(udf2 jemalloc)
ENDIF()
target_link_libraries(
udf2 PUBLIC os ${LINK_JEMALLOC}
)
add_library(udf2_dup STATIC MODULE test/udf2_dup.c)
target_include_directories(
udf2_dup
PUBLIC
"${TD_SOURCE_DIR}/include/libs/function"
"${TD_SOURCE_DIR}/include/util"
"${TD_SOURCE_DIR}/include/common"
"${TD_SOURCE_DIR}/include/client"
"${TD_SOURCE_DIR}/include/os"
PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
)
IF(TD_LINUX_64 AND JEMALLOC_ENABLED)
ADD_DEPENDENCIES(udf2_dup jemalloc)
ENDIF()
target_link_libraries(
udf2_dup PUBLIC os ${LINK_JEMALLOC}
)
set(TARGET_NAMES
change_udf_normal
change_udf_no_init
change_udf_no_process
change_udf_no_destroy
change_udf_init_failed
change_udf_process_failed
change_udf_destory_failed
)
set(COMPILE_DEFINITIONS
CHANGE_UDF_NORMAL
CHANGE_UDF_NO_INIT
CHANGE_UDF_NO_PROCESS
CHANGE_UDF_NO_DESTROY
CHANGE_UDF_INIT_FAILED
CHANGE_UDF_PROCESS_FAILED
CHANGE_UDF_DESTORY_FAILED
)
foreach(index RANGE 0 6)
list(GET TARGET_NAMES ${index} target_name)
list(GET COMPILE_DEFINITIONS ${index} compile_def)
add_library(${target_name} STATIC MODULE test/change_udf.c)
target_include_directories(
${target_name}
PUBLIC
"${TD_SOURCE_DIR}/include/libs/function"
"${TD_SOURCE_DIR}/include/util"
"${TD_SOURCE_DIR}/include/common"
"${TD_SOURCE_DIR}/include/client"
"${TD_SOURCE_DIR}/include/os"
PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
)
target_compile_definitions(${target_name} PRIVATE ${compile_def})
IF(TD_LINUX_64 AND JEMALLOC_ENABLED)
ADD_DEPENDENCIES(${target_name} jemalloc)
ENDIF()
target_link_libraries(
${target_name} PUBLIC os ${LINK_JEMALLOC}
)
endforeach()
# SET(EXECUTABLE_OUTPUT_PATH ${CMAKE_BINARY_DIR}/build/bin)
add_executable(udfd src/udfd.c)
add_executable(taosudf src/udfd.c)
if(${TD_DARWIN})
target_compile_options(udfd PRIVATE -Wno-error=deprecated-non-prototype)
target_compile_options(taosudf PRIVATE -Wno-error=deprecated-non-prototype)
endif()
target_include_directories(
udfd
taosudf
PUBLIC
"${TD_SOURCE_DIR}/include/libs/function"
"${TD_SOURCE_DIR}/contrib/libuv/include"
@ -205,12 +51,12 @@ target_include_directories(
PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
)
IF(TD_LINUX_64 AND JEMALLOC_ENABLED)
ADD_DEPENDENCIES(udfd jemalloc)
ENDIF()
target_link_libraries(
udfd
taosudf
PUBLIC uv_a
PRIVATE os util common nodes function ${LINK_JEMALLOC}
PRIVATE os util common nodes function
)
if(${BUILD_TEST})
add_subdirectory(test)
endif()

64
source/libs/function/src/tudf.c
View File

@ -63,25 +63,25 @@ static void udfWatchUdfd(void *args);
void udfUdfdExit(uv_process_t *process, int64_t exitStatus, int32_t termSignal) {
TAOS_UDF_CHECK_PTR_RVOID(process);
fnInfo("udfd process exited with status %" PRId64 ", signal %d", exitStatus, termSignal);
fnInfo("taosudf process exited with status %" PRId64 ", signal %d", exitStatus, termSignal);
SUdfdData *pData = process->data;
if(pData == NULL) {
fnError("udfd process data is NULL");
fnError("taosudf process data is NULL");
return;
}
if (exitStatus == 0 && termSignal == 0 || atomic_load_32(&pData->stopCalled)) {
fnInfo("udfd process exit due to SIGINT or dnode-mgmt called stop");
fnInfo("taosudf process exit due to SIGINT or dnode-mgmt called stop");
} else {
fnInfo("udfd process restart");
fnInfo("taosudf process restart");
int32_t code = udfSpawnUdfd(pData);
if (code != 0) {
fnError("udfd process restart failed with code:%d", code);
fnError("taosudf process restart failed with code:%d", code);
}
}
}
static int32_t udfSpawnUdfd(SUdfdData *pData) {
fnInfo("start to init udfd");
fnInfo("start to init taosudf");
TAOS_UDF_CHECK_PTR_RCODE(pData);
int32_t err = 0;
@ -106,12 +106,12 @@ static int32_t udfSpawnUdfd(SUdfdData *pData) {
if (strlen(path) == 0) {
TAOS_STRCAT(path, "C:\\TDengine");
}
TAOS_STRCAT(path, "\\udfd.exe");
TAOS_STRCAT(path, "\\taosudf.exe");
#else
if (strlen(path) == 0) {
TAOS_STRCAT(path, "/usr/bin");
}
TAOS_STRCAT(path, "/udfd");
TAOS_STRCAT(path, "/taosudf");
#endif
char *argsUdfd[] = {path, "-c", configDir, NULL};
options.args = argsUdfd;
@ -158,9 +158,9 @@ static int32_t udfSpawnUdfd(SUdfdData *pData) {
udfdPathLdLib[udfdLdLibPathLen] = ':';
tstrncpy(udfdPathLdLib + udfdLdLibPathLen + 1, pathTaosdLdLib, sizeof(udfdPathLdLib) - udfdLdLibPathLen - 1);
if (udfdLdLibPathLen + taosdLdLibPathLen < 1024) {
fnInfo("[UDFD]udfd LD_LIBRARY_PATH: %s", udfdPathLdLib);
fnInfo("[UDFD]taosudf LD_LIBRARY_PATH: %s", udfdPathLdLib);
} else {
fnError("[UDFD]can not set correct udfd LD_LIBRARY_PATH");
fnError("[UDFD]can not set correct taosudf LD_LIBRARY_PATH");
}
char ldLibPathEnvItem[1024 + 32] = {0};
snprintf(ldLibPathEnvItem, 1024 + 32, "%s=%s", "LD_LIBRARY_PATH", udfdPathLdLib);
@ -231,12 +231,12 @@ static int32_t udfSpawnUdfd(SUdfdData *pData) {
pData->process.data = (void *)pData;
#ifdef WINDOWS
// End udfd.exe by Job.
// End taosudf.exe by Job.
if (pData->jobHandle != NULL) CloseHandle(pData->jobHandle);
pData->jobHandle = CreateJobObject(NULL, NULL);
bool add_job_ok = AssignProcessToJobObject(pData->jobHandle, pData->process.process_handle);
if (!add_job_ok) {
fnError("Assign udfd to job failed.");
fnError("Assign taosudf to job failed.");
} else {
JOBOBJECT_EXTENDED_LIMIT_INFORMATION limit_info;
memset(&limit_info, 0x0, sizeof(limit_info));
@ -244,15 +244,15 @@ static int32_t udfSpawnUdfd(SUdfdData *pData) {
bool set_auto_kill_ok =
SetInformationJobObject(pData->jobHandle, JobObjectExtendedLimitInformation, &limit_info, sizeof(limit_info));
if (!set_auto_kill_ok) {
fnError("Set job auto kill udfd failed.");
fnError("Set job auto kill taosudf failed.");
}
}
#endif
if (err != 0) {
fnError("can not spawn udfd. path: %s, error: %s", path, uv_strerror(err));
fnError("can not spawn taosudf. path: %s, error: %s", path, uv_strerror(err));
} else {
fnInfo("udfd is initialized");
fnInfo("taosudf is initialized");
}
_OVER:
@ -295,13 +295,13 @@ static void udfWatchUdfd(void *args) {
atomic_store_32(&pData->spawnErr, 0);
(void)uv_barrier_wait(&pData->barrier);
int32_t num = uv_run(&pData->loop, UV_RUN_DEFAULT);
fnInfo("udfd loop exit with %d active handles, line:%d", num, __LINE__);
fnInfo("taosudf loop exit with %d active handles, line:%d", num, __LINE__);
uv_walk(&pData->loop, udfUdfdCloseWalkCb, NULL);
num = uv_run(&pData->loop, UV_RUN_DEFAULT);
fnInfo("udfd loop exit with %d active handles, line:%d", num, __LINE__);
fnInfo("taosudf loop exit with %d active handles, line:%d", num, __LINE__);
if (uv_loop_close(&pData->loop) != 0) {
fnError("udfd loop close failed, lino:%d", __LINE__);
fnError("taosudf loop close failed, lino:%d", __LINE__);
}
return;
@ -310,9 +310,9 @@ _exit:
(void)uv_barrier_wait(&pData->barrier);
atomic_store_32(&pData->spawnErr, terrno);
if (uv_loop_close(&pData->loop) != 0) {
fnError("udfd loop close failed, lino:%d", __LINE__);
fnError("taosudf loop close failed, lino:%d", __LINE__);
}
fnError("udfd thread exit with code:%d lino:%d", terrno, terrln);
fnError("taosudf thread exit with code:%d lino:%d", terrno, terrln);
terrno = TSDB_CODE_UDF_UV_EXEC_FAILURE;
}
return;
@ -321,11 +321,11 @@ _exit:
int32_t udfStartUdfd(int32_t startDnodeId) {
int32_t code = 0, lino = 0;
if (!tsStartUdfd) {
fnInfo("start udfd is disabled.") return 0;
fnInfo("start taosudf is disabled.") return 0;
}
SUdfdData *pData = &udfdGlobal;
if (pData->startCalled) {
fnInfo("dnode start udfd already called");
fnInfo("dnode start taosudf already called");
return 0;
}
pData->startCalled = true;
@ -341,27 +341,27 @@ int32_t udfStartUdfd(int32_t startDnodeId) {
if (err != 0) {
uv_barrier_destroy(&pData->barrier);
if (uv_async_send(&pData->stopAsync) != 0) {
fnError("start udfd: failed to send stop async");
fnError("start taosudf: failed to send stop async");
}
if (uv_thread_join(&pData->thread) != 0) {
fnError("start udfd: failed to join udfd thread");
fnError("start taosudf: failed to join taosudf thread");
}
pData->needCleanUp = false;
fnInfo("udfd is cleaned up after spawn err");
fnInfo("taosudf is cleaned up after spawn err");
TAOS_CHECK_GOTO(err, &lino, _exit);
} else {
pData->needCleanUp = true;
}
_exit:
if (code != 0) {
fnError("udfd start failed with code:%d, lino:%d", code, lino);
fnError("taosudf start failed with code:%d, lino:%d", code, lino);
}
return code;
}
void udfStopUdfd() {
SUdfdData *pData = &udfdGlobal;
fnInfo("udfd start to stop, need cleanup:%d, spawn err:%d", pData->needCleanUp, pData->spawnErr);
fnInfo("taosudf start to stop, need cleanup:%d, spawn err:%d", pData->needCleanUp, pData->spawnErr);
if (!pData->needCleanUp || atomic_load_32(&pData->stopCalled)) {
return;
}
@ -369,16 +369,16 @@ void udfStopUdfd() {
pData->needCleanUp = false;
uv_barrier_destroy(&pData->barrier);
if (uv_async_send(&pData->stopAsync) != 0) {
fnError("stop udfd: failed to send stop async");
fnError("stop taosudf: failed to send stop async");
}
if (uv_thread_join(&pData->thread) != 0) {
fnError("stop udfd: failed to join udfd thread");
fnError("stop taosudf: failed to join taosudf thread");
}
#ifdef WINDOWS
if (pData->jobHandle != NULL) CloseHandle(pData->jobHandle);
#endif
fnInfo("udfd is cleaned up");
fnInfo("taosudf is cleaned up");
return;
}
@ -2152,7 +2152,7 @@ int32_t callUdf(UdfcFuncHandle handle, int8_t callType, SSDataBlock *input, SUdf
fnDebug("udfc call udf. callType: %d, funcHandle: %p", callType, handle);
SUdfcUvSession *session = (SUdfcUvSession *)handle;
if (session->udfUvPipe == NULL) {
fnError("No pipe to udfd");
fnError("No pipe to taosudf");
return TSDB_CODE_UDF_PIPE_NOT_EXIST;
}
SClientUdfTask *task = taosMemoryCalloc(1, sizeof(SClientUdfTask));
@ -2282,7 +2282,7 @@ int32_t doTeardownUdf(UdfcFuncHandle handle) {
SUdfcUvSession *session = (SUdfcUvSession *)handle;
if (session->udfUvPipe == NULL) {
fnError("tear down udf. pipe to udfd does not exist. udf name: %s", session->udfName);
fnError("tear down udf. pipe to taosudf does not exist. udf name: %s", session->udfName);
taosMemoryFree(session);
return TSDB_CODE_UDF_PIPE_NOT_EXIST;
}

128
source/libs/function/src/udfd.c
View File

@ -166,7 +166,7 @@ int32_t udfdCPluginUdfScalarProc(SUdfDataBlock *block, SUdfColumn *resultCol, vo
if (ctx->scalarProcFunc) {
return ctx->scalarProcFunc(block, resultCol);
} else {
fnError("udfd c plugin scalar proc not implemented");
fnError("taosudf c plugin scalar proc not implemented");
return TSDB_CODE_UDF_FUNC_EXEC_FAILURE;
}
}
@ -177,7 +177,7 @@ int32_t udfdCPluginUdfAggStart(SUdfInterBuf *buf, void *udfCtx) {
if (ctx->aggStartFunc) {
return ctx->aggStartFunc(buf);
} else {
fnError("udfd c plugin aggregation start not implemented");
fnError("taosudf c plugin aggregation start not implemented");
return TSDB_CODE_UDF_FUNC_EXEC_FAILURE;
}
return 0;
@ -189,7 +189,7 @@ int32_t udfdCPluginUdfAggProc(SUdfDataBlock *block, SUdfInterBuf *interBuf, SUdf
if (ctx->aggProcFunc) {
return ctx->aggProcFunc(block, interBuf, newInterBuf);
} else {
fnError("udfd c plugin aggregation process not implemented");
fnError("taosudf c plugin aggregation process not implemented");
return TSDB_CODE_UDF_FUNC_EXEC_FAILURE;
}
}
@ -201,7 +201,7 @@ int32_t udfdCPluginUdfAggProc(SUdfDataBlock *block, SUdfInterBuf *interBuf, SUdf
// if (ctx->aggMergeFunc) {
// return ctx->aggMergeFunc(inputBuf1, inputBuf2, outputBuf);
// } else {
// fnError("udfd c plugin aggregation merge not implemented");
// fnError("taosudf c plugin aggregation merge not implemented");
// return TSDB_CODE_UDF_FUNC_EXEC_FAILURE;
// }
// }
@ -212,7 +212,7 @@ int32_t udfdCPluginUdfAggFinish(SUdfInterBuf *buf, SUdfInterBuf *resultData, voi
if (ctx->aggFinishFunc) {
return ctx->aggFinishFunc(buf, resultData);
} else {
fnError("udfd c plugin aggregation finish not implemented");
fnError("taosudf c plugin aggregation finish not implemented");
return TSDB_CODE_UDF_FUNC_EXEC_FAILURE;
}
return 0;
@ -667,7 +667,7 @@ void udfdFreeUdf(void *pData) {
if (pSudf->scriptPlugin != NULL) {
if(pSudf->scriptPlugin->udfDestroyFunc(pSudf->scriptUdfCtx) != 0) {
fnError("udfdFreeUdf: udfd destroy udf %s failed", pSudf->name);
fnError("udfdFreeUdf: taosudf destroy udf %s failed", pSudf->name);
}
}
@ -688,15 +688,15 @@ int32_t udfdGetOrCreateUdf(SUdf **ppUdf, const char *udfName) {
++(*pUdfHash)->refCount;
*ppUdf = *pUdfHash;
uv_mutex_unlock(&global.udfsMutex);
fnInfo("udfd reuse existing udf. udf %s udf version %d, udf created time %" PRIx64, (*ppUdf)->name, (*ppUdf)->version,
fnInfo("taosudf reuse existing udf. udf %s udf version %d, udf created time %" PRIx64, (*ppUdf)->name, (*ppUdf)->version,
(*ppUdf)->createdTime);
return 0;
} else {
(*pUdfHash)->expired = true;
fnInfo("udfd expired, check for new version. existing udf %s udf version %d, udf created time %" PRIx64,
fnInfo("taosudf expired, check for new version. existing udf %s udf version %d, udf created time %" PRIx64,
(*pUdfHash)->name, (*pUdfHash)->version, (*pUdfHash)->createdTime);
if(taosHashRemove(global.udfsHash, udfName, strlen(udfName)) != 0) {
fnError("udfdGetOrCreateUdf: udfd remove udf %s failed", udfName);
fnError("udfdGetOrCreateUdf: taosudf remove udf %s failed", udfName);
}
}
}
@ -1001,7 +1001,7 @@ void udfdProcessTeardownRequest(SUvUdfWork *uvUdf, SUdfRequest *request) {
uv_cond_destroy(&udf->condReady);
uv_mutex_destroy(&udf->lock);
code = udf->scriptPlugin->udfDestroyFunc(udf->scriptUdfCtx);
fnDebug("udfd destroy function returns %d", code);
fnDebug("taosudf destroy function returns %d", code);
taosMemoryFree(udf);
}
@ -1063,7 +1063,7 @@ int32_t udfdSaveFuncBodyToFile(SFuncInfo *pFuncInfo, SUdf *udf) {
TAOS_UDF_CHECK_PTR_RCODE(pFuncInfo, udf);
if (!osDataSpaceAvailable()) {
terrno = TSDB_CODE_NO_DISKSPACE;
fnError("udfd create shared library failed since %s", terrstr());
fnError("taosudf create shared library failed since %s", terrstr());
return terrno;
}
@ -1072,22 +1072,22 @@ int32_t udfdSaveFuncBodyToFile(SFuncInfo *pFuncInfo, SUdf *udf) {
bool fileExist = !(taosStatFile(path, NULL, NULL, NULL) < 0);
if (fileExist) {
tstrncpy(udf->path, path, PATH_MAX);
fnInfo("udfd func body file. reuse existing file %s", path);
fnInfo("taosudf func body file. reuse existing file %s", path);
return TSDB_CODE_SUCCESS;
}
TdFilePtr file = taosOpenFile(path, TD_FILE_CREATE | TD_FILE_WRITE | TD_FILE_READ | TD_FILE_TRUNC);
if (file == NULL) {
fnError("udfd write udf shared library: %s failed, error: %d %s", path, errno, strerror(terrno));
fnError("taosudf write udf shared library: %s failed, error: %d %s", path, errno, strerror(terrno));
return TSDB_CODE_FILE_CORRUPTED;
}
int64_t count = taosWriteFile(file, pFuncInfo->pCode, pFuncInfo->codeSize);
if (count != pFuncInfo->codeSize) {
fnError("udfd write udf shared library failed");
fnError("taosudf write udf shared library failed");
return TSDB_CODE_FILE_CORRUPTED;
}
if(taosCloseFile(&file) != 0) {
fnError("udfdSaveFuncBodyToFile, udfd close file failed");
fnError("udfdSaveFuncBodyToFile, taosudf close file failed");
return TSDB_CODE_FILE_CORRUPTED;
}
@ -1106,7 +1106,7 @@ void udfdProcessRpcRsp(void *parent, SRpcMsg *pMsg, SEpSet *pEpSet) {
}
if (pMsg->code != TSDB_CODE_SUCCESS) {
fnError("udfd rpc error, code:%s", tstrerror(pMsg->code));
fnError("taosudf rpc error, code:%s", tstrerror(pMsg->code));
msgInfo->code = pMsg->code;
goto _return;
}
@ -1114,7 +1114,7 @@ void udfdProcessRpcRsp(void *parent, SRpcMsg *pMsg, SEpSet *pEpSet) {
if (msgInfo->rpcType == UDFD_RPC_MNODE_CONNECT) {
SConnectRsp connectRsp = {0};
if(tDeserializeSConnectRsp(pMsg->pCont, pMsg->contLen, &connectRsp) < 0){
fnError("udfd deserialize connect response failed");
fnError("taosudf deserialize connect response failed");
goto _return;
}
@ -1137,7 +1137,7 @@ void udfdProcessRpcRsp(void *parent, SRpcMsg *pMsg, SEpSet *pEpSet) {
} else if (msgInfo->rpcType == UDFD_RPC_RETRIVE_FUNC) {
SRetrieveFuncRsp retrieveRsp = {0};
if(tDeserializeSRetrieveFuncRsp(pMsg->pCont, pMsg->contLen, &retrieveRsp) < 0){
fnError("udfd deserialize retrieve func response failed");
fnError("taosudf deserialize retrieve func response failed");
goto _return;
}
@ -1303,16 +1303,16 @@ int32_t udfdOpenClientRpc() {
}
void udfdCloseClientRpc() {
fnInfo("udfd begin closing rpc");
fnInfo("taosudf begin closing rpc");
rpcClose(global.clientRpc);
fnInfo("udfd finish closing rpc");
fnInfo("taosudf finish closing rpc");
}
void udfdOnWrite(uv_write_t *req, int status) {
TAOS_UDF_CHECK_PTR_RVOID(req);
SUvUdfWork *work = (SUvUdfWork *)req->data;
if (status < 0) {
fnError("udfd send response error, length:%zu code:%s", work->output.len, uv_err_name(status));
fnError("taosudf send response error, length:%zu code:%s", work->output.len, uv_err_name(status));
}
// remove work from the connection work list
if (work->conn != NULL) {
@ -1337,14 +1337,14 @@ void udfdSendResponse(uv_work_t *work, int status) {
if (udfWork->conn != NULL) {
uv_write_t *write_req = taosMemoryMalloc(sizeof(uv_write_t));
if(write_req == NULL) {
fnError("udfd send response error, malloc failed");
fnError("taosudf send response error, malloc failed");
taosMemoryFree(work);
return;
}
write_req->data = udfWork;
int32_t code = uv_write(write_req, udfWork->conn->client, &udfWork->output, 1, udfdOnWrite);
if (code != 0) {
fnError("udfd send response error %s", uv_strerror(code));
fnError("taosudf send response error %s", uv_strerror(code));
taosMemoryFree(write_req);
}
}
@ -1365,7 +1365,7 @@ void udfdAllocBuffer(uv_handle_t *handle, size_t suggestedSize, uv_buf_t *buf) {
buf->base = ctx->inputBuf;
buf->len = ctx->inputCap;
} else {
fnError("udfd can not allocate enough memory") buf->base = NULL;
fnError("taosudf can not allocate enough memory") buf->base = NULL;
buf->len = 0;
}
} else if (ctx->inputTotal == -1 && ctx->inputLen < msgHeadSize) {
@ -1379,7 +1379,7 @@ void udfdAllocBuffer(uv_handle_t *handle, size_t suggestedSize, uv_buf_t *buf) {
buf->base = ctx->inputBuf + ctx->inputLen;
buf->len = ctx->inputCap - ctx->inputLen;
} else {
fnError("udfd can not allocate enough memory") buf->base = NULL;
fnError("taosudf can not allocate enough memory") buf->base = NULL;
buf->len = 0;
}
}
@ -1387,7 +1387,7 @@ void udfdAllocBuffer(uv_handle_t *handle, size_t suggestedSize, uv_buf_t *buf) {
bool isUdfdUvMsgComplete(SUdfdUvConn *pipe) {
if (pipe == NULL) {
fnError("udfd pipe is NULL, LINE:%d", __LINE__);
fnError("taosudf pipe is NULL, LINE:%d", __LINE__);
return false;
}
if (pipe->inputTotal == -1 && pipe->inputLen >= sizeof(int32_t)) {
@ -1407,12 +1407,12 @@ void udfdHandleRequest(SUdfdUvConn *conn) {
uv_work_t *work = taosMemoryMalloc(sizeof(uv_work_t));
if(work == NULL) {
fnError("udfd malloc work failed");
fnError("taosudf malloc work failed");
return;
}
SUvUdfWork *udfWork = taosMemoryMalloc(sizeof(SUvUdfWork));
if(udfWork == NULL) {
fnError("udfd malloc udf work failed");
fnError("taosudf malloc udf work failed");
taosMemoryFree(work);
return;
}
@ -1427,7 +1427,7 @@ void udfdHandleRequest(SUdfdUvConn *conn) {
work->data = udfWork;
if(uv_queue_work(global.loop, work, udfdProcessRequest, udfdSendResponse) != 0)
{
fnError("udfd queue work failed");
fnError("taosudf queue work failed");
taosMemoryFree(work);
taosMemoryFree(udfWork);
}
@ -1449,7 +1449,7 @@ void udfdPipeCloseCb(uv_handle_t *pipe) {
void udfdPipeRead(uv_stream_t *client, ssize_t nread, const uv_buf_t *buf) {
TAOS_UDF_CHECK_PTR_RVOID(client, buf);
fnDebug("udfd read %zd bytes from client", nread);
fnDebug("taosudf read %zd bytes from client", nread);
if (nread == 0) return;
SUdfdUvConn *conn = client->data;
@ -1466,7 +1466,7 @@ void udfdPipeRead(uv_stream_t *client, ssize_t nread, const uv_buf_t *buf) {
if (nread < 0) {
if (nread == UV_EOF) {
fnInfo("udfd pipe read EOF");
fnInfo("taosudf pipe read EOF");
} else {
fnError("Receive error %s", uv_err_name(nread));
}
@ -1477,26 +1477,26 @@ void udfdPipeRead(uv_stream_t *client, ssize_t nread, const uv_buf_t *buf) {
void udfdOnNewConnection(uv_stream_t *server, int status) {
TAOS_UDF_CHECK_PTR_RVOID(server);
if (status < 0) {
fnError("udfd new connection error, code:%s", uv_strerror(status));
fnError("taosudf new connection error, code:%s", uv_strerror(status));
return;
}
int32_t code = 0;
uv_pipe_t *client = (uv_pipe_t *)taosMemoryMalloc(sizeof(uv_pipe_t));
if(client == NULL) {
fnError("udfd pipe malloc failed");
fnError("taosudf pipe malloc failed");
return;
}
code = uv_pipe_init(global.loop, client, 0);
if (code) {
fnError("udfd pipe init error %s", uv_strerror(code));
fnError("taosudf pipe init error %s", uv_strerror(code));
taosMemoryFree(client);
return;
}
if (uv_accept(server, (uv_stream_t *)client) == 0) {
SUdfdUvConn *ctx = taosMemoryMalloc(sizeof(SUdfdUvConn));
if(ctx == NULL) {
fnError("udfd conn malloc failed");
fnError("taosudf conn malloc failed");
goto _exit;
}
ctx->pWorkList = NULL;
@ -1508,7 +1508,7 @@ void udfdOnNewConnection(uv_stream_t *server, int status) {
ctx->client = (uv_stream_t *)client;
code = uv_read_start((uv_stream_t *)client, udfdAllocBuffer, udfdPipeRead);
if (code) {
fnError("udfd read start error %s", uv_strerror(code));
fnError("taosudf read start error %s", uv_strerror(code));
udfdUvHandleError(ctx);
taosMemoryFree(ctx);
taosMemoryFree(client);
@ -1522,7 +1522,7 @@ _exit:
void udfdIntrSignalHandler(uv_signal_t *handle, int signum) {
TAOS_UDF_CHECK_PTR_RVOID(handle);
fnInfo("udfd signal received: %d\n", signum);
fnInfo("taosudf signal received: %d\n", signum);
uv_fs_t req;
int32_t code = uv_fs_unlink(global.loop, &req, global.listenPipeName, NULL);
if(code) {
@ -1558,7 +1558,7 @@ static int32_t udfdParseArgs(int32_t argc, char *argv[]) {
}
static void udfdPrintVersion() {
(void)printf("udfd version: %s compatible_version: %s\n", td_version, td_compatible_version);
(void)printf("taosudf version: %s compatible_version: %s\n", td_version, td_compatible_version);
(void)printf("git: %s\n", td_gitinfo);
(void)printf("build: %s\n", td_buildinfo);
}
@ -1573,7 +1573,7 @@ void udfdCtrlAllocBufCb(uv_handle_t *handle, size_t suggested_size, uv_buf_t *bu
TAOS_UDF_CHECK_PTR_RVOID(buf);
buf->base = taosMemoryMalloc(suggested_size);
if (buf->base == NULL) {
fnError("udfd ctrl pipe alloc buffer failed");
fnError("taosudf ctrl pipe alloc buffer failed");
return;
}
buf->len = suggested_size;
@ -1582,13 +1582,13 @@ void udfdCtrlAllocBufCb(uv_handle_t *handle, size_t suggested_size, uv_buf_t *bu
void udfdCtrlReadCb(uv_stream_t *q, ssize_t nread, const uv_buf_t *buf) {
TAOS_UDF_CHECK_PTR_RVOID(q, buf);
if (nread < 0) {
fnError("udfd ctrl pipe read error. %s", uv_err_name(nread));
fnError("taosudf ctrl pipe read error. %s", uv_err_name(nread));
taosMemoryFree(buf->base);
uv_close((uv_handle_t *)q, NULL);
uv_stop(global.loop);
return;
}
fnError("udfd ctrl pipe read %zu bytes", nread);
fnError("taosudf ctrl pipe read %zu bytes", nread);
taosMemoryFree(buf->base);
}
@ -1604,7 +1604,7 @@ static void removeListeningPipe() {
static int32_t udfdUvInit() {
TAOS_CHECK_RETURN(uv_loop_init(global.loop));
if (tsStartUdfd) { // udfd is started by taosd, which shall exit when taosd exit
if (tsStartUdfd) { // taosudf is started by taosd, which shall exit when taosd exit
TAOS_CHECK_RETURN(uv_pipe_init(global.loop, &global.ctrlPipe, 1));
TAOS_CHECK_RETURN(uv_pipe_open(&global.ctrlPipe, 0));
TAOS_CHECK_RETURN(uv_read_start((uv_stream_t *)&global.ctrlPipe, udfdCtrlAllocBufCb, udfdCtrlReadCb));
@ -1642,13 +1642,13 @@ static void udfdCloseWalkCb(uv_handle_t *handle, void *arg) {
static int32_t udfdGlobalDataInit() {
uv_loop_t *loop = taosMemoryMalloc(sizeof(uv_loop_t));
if (loop == NULL) {
fnError("udfd init uv loop failed, mem overflow");
fnError("taosudf init uv loop failed, mem overflow");
return terrno;
}
global.loop = loop;
if (uv_mutex_init(&global.scriptPluginsMutex) != 0) {
fnError("udfd init script plugins mutex failed");
fnError("taosudf init script plugins mutex failed");
return TSDB_CODE_UDF_UV_EXEC_FAILURE;
}
@ -1659,7 +1659,7 @@ static int32_t udfdGlobalDataInit() {
// taosHashSetFreeFp(global.udfsHash, udfdFreeUdf);
if (uv_mutex_init(&global.udfsMutex) != 0) {
fnError("udfd init udfs mutex failed");
fnError("taosudf init udfs mutex failed");
return TSDB_CODE_UDF_UV_EXEC_FAILURE;
}
@ -1670,23 +1670,23 @@ static void udfdGlobalDataDeinit() {
uv_mutex_destroy(&global.udfsMutex);
uv_mutex_destroy(&global.scriptPluginsMutex);
taosMemoryFreeClear(global.loop);
fnInfo("udfd global data deinit");
fnInfo("taosudf global data deinit");
}
static void udfdRun() {
fnInfo("start udfd event loop");
fnInfo("start taosudf event loop");
int32_t code = uv_run(global.loop, UV_RUN_DEFAULT);
if(code != 0) {
fnError("udfd event loop still has active handles or requests.");
fnError("taosudf event loop still has active handles or requests.");
}
fnInfo("udfd event loop stopped.");
fnInfo("taosudf event loop stopped.");
(void)uv_loop_close(global.loop);
uv_walk(global.loop, udfdCloseWalkCb, NULL);
code = uv_run(global.loop, UV_RUN_DEFAULT);
if(code != 0) {
fnError("udfd event loop still has active handles or requests.");
fnError("taosudf event loop still has active handles or requests.");
}
(void)uv_loop_close(global.loop);
}
@ -1702,7 +1702,7 @@ int32_t udfdInitResidentFuncs() {
while ((token = strtok_r(pSave, ",", &pSave)) != NULL) {
char func[TSDB_FUNC_NAME_LEN + 1] = {0};
tstrncpy(func, token, TSDB_FUNC_NAME_LEN);
fnInfo("udfd add resident function %s", func);
fnInfo("taosudf add resident function %s", func);
if(taosArrayPush(global.residentFuncs, func) == NULL)
{
taosArrayDestroy(global.residentFuncs);
@ -1722,18 +1722,18 @@ void udfdDeinitResidentFuncs() {
int32_t code = 0;
if (udf->scriptPlugin->udfDestroyFunc) {
code = udf->scriptPlugin->udfDestroyFunc(udf->scriptUdfCtx);
fnDebug("udfd %s destroy function returns %d", funcName, code);
fnDebug("taosudf %s destroy function returns %d", funcName, code);
}
if(taosHashRemove(global.udfsHash, funcName, strlen(funcName)) != 0)
{
fnError("udfd remove resident function %s failed", funcName);
fnError("taosudf remove resident function %s failed", funcName);
}
taosMemoryFree(udf);
}
}
taosHashCleanup(global.udfsHash);
taosArrayDestroy(global.residentFuncs);
fnInfo("udfd resident functions are deinit");
fnInfo("taosudf resident functions are deinit");
}
int32_t udfdCreateUdfSourceDir() {
@ -1743,7 +1743,7 @@ int32_t udfdCreateUdfSourceDir() {
snprintf(global.udfDataDir, PATH_MAX, "%s/.udf", tsTempDir);
code = taosMkDir(global.udfDataDir);
}
fnInfo("udfd create udf source directory %s. result: %s", global.udfDataDir, tstrerror(code));
fnInfo("taosudf create udf source directory %s. result: %s", global.udfDataDir, tstrerror(code));
return code;
}
@ -1779,7 +1779,7 @@ int main(int argc, char *argv[]) {
if (udfdInitLog() != 0) {
// ignore create log failed, because this error no matter
(void)printf("failed to init udfd log.");
(void)printf("failed to init taosudf log.");
} else {
logInitialized = true; // log is initialized
}
@ -1790,20 +1790,20 @@ int main(int argc, char *argv[]) {
goto _exit;
}
cfgInitialized = true; // cfg is initialized
fnInfo("udfd start with config file %s", configDir);
fnInfo("taosudf start with config file %s", configDir);
if (initEpSetFromCfg(tsFirst, tsSecond, &global.mgmtEp) != 0) {
fnError("init ep set from cfg failed");
code = -3;
goto _exit;
}
fnInfo("udfd start with mnode ep %s", global.mgmtEp.epSet.eps[0].fqdn);
fnInfo("taosudf start with mnode ep %s", global.mgmtEp.epSet.eps[0].fqdn);
if (udfdOpenClientRpc() != 0) {
fnError("open rpc connection to mnode failed");
code = -4;
goto _exit;
}
fnInfo("udfd rpc client is opened");
fnInfo("taosudf rpc client is opened");
openClientRpcFinished = true; // rpc is opened
if (udfdCreateUdfSourceDir() != 0) {
@ -1812,7 +1812,7 @@ int main(int argc, char *argv[]) {
goto _exit;
}
udfSourceDirInited = true; // udf source dir is created
fnInfo("udfd udf source directory is created");
fnInfo("taosudf udf source directory is created");
if (udfdGlobalDataInit() != 0) {
fnError("init global data failed");
@ -1820,14 +1820,14 @@ int main(int argc, char *argv[]) {
goto _exit;
}
globalDataInited = true; // global data is inited
fnInfo("udfd global data is inited");
fnInfo("taosudf global data is inited");
if (udfdUvInit() != 0) {
fnError("uv init failure");
code = -7;
goto _exit;
}
fnInfo("udfd uv is inited");
fnInfo("taosudf uv is inited");
if (udfdInitResidentFuncs() != 0) {
fnError("init resident functions failed");
@ -1835,10 +1835,10 @@ int main(int argc, char *argv[]) {
goto _exit;
}
residentFuncsInited = true; // resident functions are inited
fnInfo("udfd resident functions are inited");
fnInfo("taosudf resident functions are inited");
udfdRun();
fnInfo("udfd exit normally");
fnInfo("taosudf exit normally");
removeListeningPipe();

54
source/libs/function/test/CMakeLists.txt Normal file
View File

@ -0,0 +1,54 @@
include_directories("${TD_SOURCE_DIR}/include/libs/function")
include_directories("${TD_SOURCE_DIR}/contrib/libuv/include")
include_directories("${TD_SOURCE_DIR}/include/util")
include_directories("${TD_SOURCE_DIR}/include/common")
include_directories("${TD_SOURCE_DIR}/include/client")
include_directories("${TD_SOURCE_DIR}/include/os")
include_directories("${CMAKE_CURRENT_SOURCE_DIR}/../inc")
add_executable(runUdf runUdf.c)
target_link_libraries(
runUdf
PUBLIC uv_a
PRIVATE os util common nodes function
)
add_library(udf1 STATIC MODULE udf1.c)
target_link_libraries(udf1 PUBLIC os)
add_library(udf2 STATIC MODULE udf2.c)
target_link_libraries(udf2 PUBLIC os)
add_library(udf1_dup STATIC MODULE udf1_dup.c)
target_link_libraries(udf1_dup PUBLIC os)
add_library(udf2_dup STATIC MODULE udf2_dup.c)
target_link_libraries(udf2_dup PUBLIC os)
set(TARGET_NAMES
change_udf_normal
change_udf_no_init
change_udf_no_process
change_udf_no_destroy
change_udf_init_failed
change_udf_process_failed
change_udf_destory_failed
)
set(COMPILE_DEFINITIONS
CHANGE_UDF_NORMAL
CHANGE_UDF_NO_INIT
CHANGE_UDF_NO_PROCESS
CHANGE_UDF_NO_DESTROY
CHANGE_UDF_INIT_FAILED
CHANGE_UDF_PROCESS_FAILED
CHANGE_UDF_DESTORY_FAILED
)
foreach(index RANGE 0 6)
list(GET TARGET_NAMES ${index} target_name)
list(GET COMPILE_DEFINITIONS ${index} compile_def)
add_library(${target_name} STATIC MODULE change_udf.c)
target_compile_definitions(${target_name} PRIVATE ${compile_def})
target_link_libraries(${target_name} PUBLIC os)
endforeach()

3
source/libs/function/test/runUdf.c
View File

@ -1,6 +1,3 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "uv.h"
#include "fnLog.h"

1
source/libs/nodes/src/nodesUtilFuncs.c
View File

@ -1326,6 +1326,7 @@ void nodesDestroyNode(SNode* pNode) {
taosArrayDestroy(pStmt->pTableTag);
taosHashCleanup(pStmt->pVgroupsHashObj);
taosHashCleanup(pStmt->pSubTableHashObj);
taosHashCleanup(pStmt->pSuperTableHashObj);
taosHashCleanup(pStmt->pTableNameHashObj);
taosHashCleanup(pStmt->pDbFNameHashObj);
taosHashCleanup(pStmt->pTableCxtHashObj);

1
source/libs/parser/inc/parAst.h
View File

@ -241,6 +241,7 @@ SNode* createUseDatabaseStmt(SAstCreateContext* pCxt, SToken* pDbName);
SNode* setShowKind(SAstCreateContext* pCxt, SNode* pStmt, EShowKind showKind);
SNode* createShowStmt(SAstCreateContext* pCxt, ENodeType type);
SNode* createShowStmtWithFull(SAstCreateContext* pCxt, ENodeType type);
SNode* createShowStmtWithLike(SAstCreateContext* pCxt, ENodeType type, SNode* pLikePattern);
SNode* createShowStmtWithCond(SAstCreateContext* pCxt, ENodeType type, SNode* pDbName, SNode* pTbName,
EOperatorType tableCondType);
SNode* createShowTablesStmt(SAstCreateContext* pCxt, SShowTablesOption option, SNode* pTbName,

8
source/libs/parser/inc/sql.y
View File

@ -119,7 +119,7 @@ is_import_opt(A) ::= IS_IMPORT NK_INTEGER(B).
is_createdb_opt(A) ::= . { A = 0; }
is_createdb_opt(A) ::= CREATEDB NK_INTEGER(B). { A = taosStr2Int8(B.z, NULL, 10); }
/************************************************ create/alter/drop user **********************************************/
cmd ::= CREATE USER user_name(A) PASS NK_STRING(B) sysinfo_opt(C) is_createdb_opt(F) is_import_opt(E)
cmd ::= CREATE USER user_name(A) PASS NK_STRING(B) sysinfo_opt(C) is_createdb_opt(E) is_import_opt(F)
white_list_opt(D). {
pCxt->pRootNode = createCreateUserStmt(pCxt, &A, &B, C, E, F);
pCxt->pRootNode = addCreateUserStmtWhiteList(pCxt, pCxt->pRootNode, D);
@ -602,9 +602,9 @@ cmd ::= SHOW ENCRYPTIONS.
cmd ::= SHOW QUERIES. { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_QUERIES_STMT); }
cmd ::= SHOW SCORES. { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_SCORES_STMT); }
cmd ::= SHOW TOPICS. { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_TOPICS_STMT); }
cmd ::= SHOW VARIABLES. { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_VARIABLES_STMT); }
cmd ::= SHOW CLUSTER VARIABLES. { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_VARIABLES_STMT); }
cmd ::= SHOW LOCAL VARIABLES. { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_LOCAL_VARIABLES_STMT); }
cmd ::= SHOW VARIABLES like_pattern_opt(B). { pCxt->pRootNode = createShowStmtWithLike(pCxt, QUERY_NODE_SHOW_VARIABLES_STMT, B); }
cmd ::= SHOW CLUSTER VARIABLES like_pattern_opt(B). { pCxt->pRootNode = createShowStmtWithLike(pCxt, QUERY_NODE_SHOW_VARIABLES_STMT, B); }
cmd ::= SHOW LOCAL VARIABLES like_pattern_opt(B). { pCxt->pRootNode = createShowStmtWithLike(pCxt, QUERY_NODE_SHOW_LOCAL_VARIABLES_STMT, B); }
cmd ::= SHOW DNODE NK_INTEGER(A) VARIABLES like_pattern_opt(B). { pCxt->pRootNode = createShowDnodeVariablesStmt(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &A), B); }
cmd ::= SHOW BNODES. { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_BNODES_STMT); }
cmd ::= SHOW SNODES. { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_SNODES_STMT); }

98
source/libs/parser/src/parAstCreater.c
View File

@ -108,19 +108,74 @@ static bool invalidPassword(const char* pPassword) {
return 0 == res;
}
static bool invalidStrongPassword(const char* pPassword) {
if (strcmp(pPassword, "taosdata") == 0) {
return false;
}
bool charTypes[4] = {0};
for (int32_t i = 0; i < strlen(pPassword); ++i) {
if (taosIsBigChar(pPassword[i])) {
charTypes[0] = true;
} else if (taosIsSmallChar(pPassword[i])) {
charTypes[1] = true;
} else if (taosIsNumberChar(pPassword[i])) {
charTypes[2] = true;
} else if (taosIsSpecialChar(pPassword[i])) {
charTypes[3] = true;
} else {
return true;
}
}
int32_t numOfTypes = 0;
for (int32_t i = 0; i < 4; ++i) {
numOfTypes += charTypes[i];
}
if (numOfTypes < 3) {
return true;
}
return false;
}
static bool checkPassword(SAstCreateContext* pCxt, const SToken* pPasswordToken, char* pPassword) {
if (NULL == pPasswordToken) {
pCxt->errCode = TSDB_CODE_PAR_SYNTAX_ERROR;
} else if (pPasswordToken->n >= (TSDB_USET_PASSWORD_LEN + 2)) {
} else if (pPasswordToken->n >= (TSDB_USET_PASSWORD_LONGLEN + 2)) {
pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NAME_OR_PASSWD_TOO_LONG);
} else {
strncpy(pPassword, pPasswordToken->z, pPasswordToken->n);
(void)strdequote(pPassword);
if (strtrim(pPassword) <= 0) {
if (strtrim(pPassword) < TSDB_PASSWORD_MIN_LEN) {
pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_PASSWD_TOO_SHORT_OR_EMPTY);
} else if (invalidPassword(pPassword)) {
} else {
if (tsEnableStrongPassword) {
if (invalidStrongPassword(pPassword)) {
pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_PASSWD);
}
} else {
if (invalidPassword(pPassword)) {
pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_PASSWD);
}
}
}
}
return TSDB_CODE_SUCCESS == pCxt->errCode;
}
static bool checkImportPassword(SAstCreateContext* pCxt, const SToken* pPasswordToken, char* pPassword) {
if (NULL == pPasswordToken) {
pCxt->errCode = TSDB_CODE_PAR_SYNTAX_ERROR;
} else if (pPasswordToken->n > (32 + 2)) {
pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NAME_OR_PASSWD_TOO_LONG);
} else {
strncpy(pPassword, pPasswordToken->z, pPasswordToken->n);
(void)strdequote(pPassword);
if (strtrim(pPassword) < 32) {
pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_PASSWD_TOO_SHORT_OR_EMPTY);
}
}
return TSDB_CODE_SUCCESS == pCxt->errCode;
}
@ -2749,6 +2804,22 @@ static bool needDbShowStmt(ENodeType type) {
QUERY_NODE_SHOW_VIEWS_STMT == type || QUERY_NODE_SHOW_TSMAS_STMT == type || QUERY_NODE_SHOW_USAGE_STMT == type;
}
SNode* createShowStmtWithLike(SAstCreateContext* pCxt, ENodeType type, SNode* pLikePattern) {
CHECK_PARSER_STATUS(pCxt);
SShowStmt* pStmt = NULL;
pCxt->errCode = nodesMakeNode(type, (SNode**)&pStmt);
CHECK_MAKE_NODE(pStmt);
pStmt->withFull = false;
pStmt->pTbName = pLikePattern;
if (pLikePattern) {
pStmt->tableCondType = OP_TYPE_LIKE;
}
return (SNode*)pStmt;
_err:
nodesDestroyNode(pLikePattern);
return NULL;
}
SNode* createShowStmt(SAstCreateContext* pCxt, ENodeType type) {
CHECK_PARSER_STATUS(pCxt);
SShowStmt* pStmt = NULL;
@ -3051,7 +3122,14 @@ static int32_t fillIpRangesFromWhiteList(SAstCreateContext* pCxt, SNodeList* pIp
}
SNode* addCreateUserStmtWhiteList(SAstCreateContext* pCxt, SNode* pCreateUserStmt, SNodeList* pIpRangesNodeList) {
if (NULL == pCreateUserStmt || NULL == pIpRangesNodeList) {
if (NULL == pCreateUserStmt) {
if (pIpRangesNodeList != NULL) {
nodesDestroyList(pIpRangesNodeList);
}
return NULL;
}
if (NULL == pIpRangesNodeList) {
return pCreateUserStmt;
}
@ -3074,14 +3152,18 @@ _err:
SNode* createCreateUserStmt(SAstCreateContext* pCxt, SToken* pUserName, const SToken* pPassword, int8_t sysinfo,
int8_t createDb, int8_t is_import) {
CHECK_PARSER_STATUS(pCxt);
char password[TSDB_USET_PASSWORD_LEN + 3] = {0};
char password[TSDB_USET_PASSWORD_LONGLEN + 3] = {0};
CHECK_NAME(checkUserName(pCxt, pUserName));
if (is_import == 0) {
CHECK_NAME(checkPassword(pCxt, pPassword, password));
} else {
CHECK_NAME(checkImportPassword(pCxt, pPassword, password));
}
SCreateUserStmt* pStmt = NULL;
pCxt->errCode = nodesMakeNode(QUERY_NODE_CREATE_USER_STMT, (SNode**)&pStmt);
CHECK_MAKE_NODE(pStmt);
COPY_STRING_FORM_ID_TOKEN(pStmt->userName, pUserName);
tstrncpy(pStmt->password, password, TSDB_USET_PASSWORD_LEN);
tstrncpy(pStmt->password, password, TSDB_USET_PASSWORD_LONGLEN);
pStmt->sysinfo = sysinfo;
pStmt->createDb = createDb;
pStmt->isImport = is_import;
@ -3100,10 +3182,10 @@ SNode* createAlterUserStmt(SAstCreateContext* pCxt, SToken* pUserName, int8_t al
pStmt->alterType = alterType;
switch (alterType) {
case TSDB_ALTER_USER_PASSWD: {
char password[TSDB_USET_PASSWORD_LEN] = {0};
char password[TSDB_USET_PASSWORD_LONGLEN] = {0};
SToken* pVal = pAlterInfo;
CHECK_NAME(checkPassword(pCxt, pVal, password));
tstrncpy(pStmt->password, password, TSDB_USET_PASSWORD_LEN);
tstrncpy(pStmt->password, password, TSDB_USET_PASSWORD_LONGLEN);
break;
}
case TSDB_ALTER_USER_ENABLE: {

149
source/libs/parser/src/parInsertSql.c
View File

@ -35,6 +35,8 @@ typedef struct SInsertParseContext {
} SInsertParseContext;
typedef int32_t (*_row_append_fn_t)(SMsgBuf* pMsgBuf, const void* value, int32_t len, void* param);
static int32_t parseBoundTagsClause(SInsertParseContext* pCxt, SVnodeModifyOpStmt* pStmt);
static int32_t parseTagsClause(SInsertParseContext* pCxt, SVnodeModifyOpStmt* pStmt, bool autoCreate);
static uint8_t TRUE_VALUE = (uint8_t)TSDB_TRUE;
static uint8_t FALSE_VALUE = (uint8_t)TSDB_FALSE;
@ -102,7 +104,8 @@ static int32_t skipTableOptions(SInsertParseContext* pCxt, const char** pSql) {
}
// pSql -> stb_name [(tag1_name, ...)] TAGS (tag1_value, ...)
static int32_t ignoreUsingClause(SInsertParseContext* pCxt, const char** pSql) {
static int32_t ignoreUsingClause(SInsertParseContext* pCxt, SVnodeModifyOpStmt* pStmt) {
const char** pSql = &pStmt->pSql;
int32_t code = TSDB_CODE_SUCCESS;
SToken token;
NEXT_TOKEN(*pSql, token);
@ -138,6 +141,26 @@ static int32_t ignoreUsingClause(SInsertParseContext* pCxt, const char** pSql) {
return code;
}
static int32_t ignoreUsingClauseAndCheckTagValues(SInsertParseContext* pCxt, SVnodeModifyOpStmt* pStmt) {
const char** pSql = &pStmt->pSql;
int32_t code = TSDB_CODE_SUCCESS;
code = parseBoundTagsClause(pCxt, pStmt);
if (TSDB_CODE_SUCCESS != code) {
return code;
}
// pSql -> TAGS (tag1_value, ...)
code = parseTagsClause(pCxt, pStmt, true);
if (TSDB_CODE_SUCCESS != code) {
return code;
}
if (TSDB_CODE_SUCCESS == code) {
code = skipTableOptions(pCxt, pSql);
}
return code;
}
static int32_t parseDuplicateUsingClause(SInsertParseContext* pCxt, SVnodeModifyOpStmt* pStmt, bool* pDuplicate) {
int32_t code = TSDB_CODE_SUCCESS;
*pDuplicate = false;
@ -150,10 +173,12 @@ static int32_t parseDuplicateUsingClause(SInsertParseContext* pCxt, SVnodeModify
STableMeta** pMeta = taosHashGet(pStmt->pSubTableHashObj, tbFName, strlen(tbFName));
if (NULL != pMeta) {
*pDuplicate = true;
code = ignoreUsingClause(pCxt, &pStmt->pSql);
if (TSDB_CODE_SUCCESS == code) {
return cloneTableMeta(*pMeta, &pStmt->pTableMeta);
pCxt->missCache = false;
code = cloneTableMeta(*pMeta, &pStmt->pTableMeta);
if (TSDB_CODE_SUCCESS != code) {
return code;
}
return ignoreUsingClause(pCxt, pStmt);
}
return code;
@ -937,7 +962,7 @@ static int32_t checkSubtablePrivilege(SArray* pTagVals, SArray* pTagName, SNode*
}
// pSql -> tag1_value, ...)
static int32_t parseTagsClauseImpl(SInsertParseContext* pCxt, SVnodeModifyOpStmt* pStmt) {
static int32_t parseTagsClauseImpl(SInsertParseContext* pCxt, SVnodeModifyOpStmt* pStmt, bool autoCreate) {
int32_t code = TSDB_CODE_SUCCESS;
SSchema* pSchema = getTableTagSchema(pStmt->pTableMeta);
SArray* pTagVals = NULL;
@ -991,7 +1016,7 @@ static int32_t parseTagsClauseImpl(SInsertParseContext* pCxt, SVnodeModifyOpStmt
code = tTagNew(pTagVals, 1, false, &pTag);
}
if (TSDB_CODE_SUCCESS == code && !isParseBindParam) {
if (TSDB_CODE_SUCCESS == code && !isParseBindParam && !autoCreate) {
code = buildCreateTbReq(pStmt, pTag, pTagName);
pTag = NULL;
}
@ -1011,7 +1036,7 @@ _exit:
// input pStmt->pSql: TAGS (tag1_value, ...) [table_options] ...
// output pStmt->pSql: [table_options] ...
static int32_t parseTagsClause(SInsertParseContext* pCxt, SVnodeModifyOpStmt* pStmt) {
static int32_t parseTagsClause(SInsertParseContext* pCxt, SVnodeModifyOpStmt* pStmt, bool autoCreate) {
SToken token;
NEXT_TOKEN(pStmt->pSql, token);
if (TK_TAGS != token.type) {
@ -1023,7 +1048,7 @@ static int32_t parseTagsClause(SInsertParseContext* pCxt, SVnodeModifyOpStmt* pS
return buildSyntaxErrMsg(&pCxt->msg, "( is expected", token.z);
}
int32_t code = parseTagsClauseImpl(pCxt, pStmt);
int32_t code = parseTagsClauseImpl(pCxt, pStmt, autoCreate);
if (TSDB_CODE_SUCCESS == code) {
NEXT_VALID_TOKEN(pStmt->pSql, token);
if (TK_NK_COMMA == token.type) {
@ -1108,7 +1133,7 @@ static int32_t parseUsingClauseBottom(SInsertParseContext* pCxt, SVnodeModifyOpS
int32_t code = parseBoundTagsClause(pCxt, pStmt);
if (TSDB_CODE_SUCCESS == code) {
code = parseTagsClause(pCxt, pStmt);
code = parseTagsClause(pCxt, pStmt, false);
}
if (TSDB_CODE_SUCCESS == code) {
code = parseTableOptions(pCxt, pStmt);
@ -1288,17 +1313,54 @@ static int32_t preParseUsingTableName(SInsertParseContext* pCxt, SVnodeModifyOpS
return insCreateSName(&pStmt->usingTableName, pTbName, pCxt->pComCxt->acctId, pCxt->pComCxt->db, &pCxt->msg);
}
static int32_t getUsingTableSchema(SInsertParseContext* pCxt, SVnodeModifyOpStmt* pStmt) {
static int32_t getUsingTableSchema(SInsertParseContext* pCxt, SVnodeModifyOpStmt* pStmt, bool* ctbCacheHit) {
int32_t code = TSDB_CODE_SUCCESS;
STableMeta* pStableMeta = NULL;
STableMeta* pCtableMeta = NULL;
if (pCxt->forceUpdate) {
pCxt->missCache = true;
return TSDB_CODE_SUCCESS;
}
int32_t code = checkAuth(pCxt->pComCxt, &pStmt->usingTableName, &pCxt->missCache, &pStmt->pTagCond);
if (TSDB_CODE_SUCCESS == code && !pCxt->missCache) {
bool bUsingTable = true;
code = getTableMeta(pCxt, &pStmt->usingTableName, &pStmt->pTableMeta, &pCxt->missCache, bUsingTable);
if (!pCxt->missCache) {
char tbFName[TSDB_TABLE_FNAME_LEN];
code = tNameExtractFullName(&pStmt->usingTableName, tbFName);
if (TSDB_CODE_SUCCESS != code) {
return code;
}
STableMeta** ppStableMeta = taosHashGet(pStmt->pSuperTableHashObj, tbFName, strlen(tbFName));
if (NULL != ppStableMeta) {
pStableMeta = *ppStableMeta;
}
if (NULL == pStableMeta) {
bool bUsingTable = true;
code = getTableMeta(pCxt, &pStmt->usingTableName, &pStableMeta, &pCxt->missCache, bUsingTable);
if (TSDB_CODE_SUCCESS == code) {
code = taosHashPut(pStmt->pSuperTableHashObj, tbFName, strlen(tbFName), &pStableMeta, POINTER_BYTES);
}
}
}
if (pCxt->isStmtBind) {
goto _no_ctb_cache;
}
if (TSDB_CODE_SUCCESS == code && !pCxt->missCache) {
bool bUsingTable = false;
code = getTableMeta(pCxt, &pStmt->targetTableName, &pCtableMeta, &pCxt->missCache, bUsingTable);
}
if (TSDB_CODE_SUCCESS == code && !pCxt->missCache) {
code = (pStableMeta->suid == pCtableMeta->suid) ? TSDB_CODE_SUCCESS : TSDB_CODE_TDB_TABLE_IN_OTHER_STABLE;
*ctbCacheHit = true;
}
_no_ctb_cache:
if (TSDB_CODE_SUCCESS == code) {
if (*ctbCacheHit) {
code = cloneTableMeta(pCtableMeta, &pStmt->pTableMeta);
} else {
code = cloneTableMeta(pStableMeta, &pStmt->pTableMeta);
}
}
taosMemoryFree(pCtableMeta);
if (TSDB_CODE_SUCCESS == code && !pCxt->missCache) {
code = getTargetTableVgroup(pCxt->pComCxt, pStmt, true, &pCxt->missCache);
}
@ -1314,9 +1376,14 @@ static int32_t getUsingTableSchema(SInsertParseContext* pCxt, SVnodeModifyOpStmt
static int32_t parseUsingTableNameImpl(SInsertParseContext* pCxt, SVnodeModifyOpStmt* pStmt) {
SToken token;
NEXT_TOKEN(pStmt->pSql, token);
bool ctbCacheHit = false;
int32_t code = preParseUsingTableName(pCxt, pStmt, &token);
if (TSDB_CODE_SUCCESS == code) {
code = getUsingTableSchema(pCxt, pStmt);
code = getUsingTableSchema(pCxt, pStmt, &ctbCacheHit);
if (TSDB_CODE_SUCCESS == code && ctbCacheHit) {
pStmt->usingTableProcessing = false;
return ignoreUsingClauseAndCheckTagValues(pCxt, pStmt);
}
}
if (TSDB_CODE_SUCCESS == code && !pCxt->missCache) {
code = storeChildTableMeta(pCxt, pStmt);
@ -1337,7 +1404,6 @@ static int32_t parseUsingTableName(SInsertParseContext* pCxt, SVnodeModifyOpStmt
if (TK_USING != token.type) {
return getTargetTableSchema(pCxt, pStmt);
}
pStmt->usingTableProcessing = true;
// pStmt->pSql -> stb_name [(tag1_name, ...)
pStmt->pSql += index;
@ -2791,6 +2857,7 @@ static int32_t createVnodeModifOpStmt(SInsertParseContext* pCxt, bool reentry, S
}
}
pStmt->pSubTableHashObj = taosHashInit(128, taosGetDefaultHashFunction(TSDB_DATA_TYPE_VARCHAR), true, HASH_NO_LOCK);
pStmt->pSuperTableHashObj = taosHashInit(128, taosGetDefaultHashFunction(TSDB_DATA_TYPE_VARCHAR), true, HASH_NO_LOCK);
pStmt->pTableNameHashObj = taosHashInit(128, taosGetDefaultHashFunction(TSDB_DATA_TYPE_VARCHAR), true, HASH_NO_LOCK);
pStmt->pDbFNameHashObj = taosHashInit(64, taosGetDefaultHashFunction(TSDB_DATA_TYPE_VARCHAR), true, HASH_NO_LOCK);
if ((!reentry && (NULL == pStmt->pVgroupsHashObj || NULL == pStmt->pTableBlockHashObj)) ||
@ -2800,6 +2867,7 @@ static int32_t createVnodeModifOpStmt(SInsertParseContext* pCxt, bool reentry, S
}
taosHashSetFreeFp(pStmt->pSubTableHashObj, destroySubTableHashElem);
taosHashSetFreeFp(pStmt->pSuperTableHashObj, destroySubTableHashElem);
*pOutput = (SNode*)pStmt;
return TSDB_CODE_SUCCESS;
@ -2842,7 +2910,7 @@ static int32_t checkAuthFromMetaData(const SArray* pUsers, SNode** pTagCond) {
}
static int32_t getTableMetaFromMetaData(const SArray* pTables, STableMeta** pMeta) {
if (1 != taosArrayGetSize(pTables)) {
if (1 != taosArrayGetSize(pTables) && 2 != taosArrayGetSize(pTables)) {
return TSDB_CODE_FAILED;
}
@ -3119,6 +3187,29 @@ static int32_t parseInsertSqlImpl(SInsertParseContext* pCxt, SVnodeModifyOpStmt*
return parseInsertSqlFromTable(pCxt, pStmt);
}
static int32_t buildUsingInsertTableReq(SName* pSName, SName* pCName, SArray** pTables) {
if (NULL == *pTables) {
*pTables = taosArrayInit(2, sizeof(SName));
if (NULL == *pTables) {
goto _err;
}
}
if (NULL == taosArrayPush(*pTables, pSName)) {
goto _err;
}
if (NULL == taosArrayPush(*pTables, pCName)) {
goto _err;
}
return TSDB_CODE_SUCCESS;
_err:
if (NULL != *pTables) {
taosArrayDestroy(*pTables);
*pTables = NULL;
}
return terrno;
}
static int32_t buildInsertTableReq(SName* pName, SArray** pTables) {
*pTables = taosArrayInit(1, sizeof(SName));
if (NULL == *pTables) {
@ -3133,6 +3224,26 @@ static int32_t buildInsertTableReq(SName* pName, SArray** pTables) {
return TSDB_CODE_SUCCESS;
}
static int32_t buildInsertUsingDbReq(SName* pSName, SName* pCName, SArray** pDbs) {
if (NULL == *pDbs) {
*pDbs = taosArrayInit(1, sizeof(STablesReq));
if (NULL == *pDbs) {
return terrno;
}
}
STablesReq req = {0};
req.autoCreate = 1;
(void)tNameGetFullDbName(pSName, req.dbFName);
(void)tNameGetFullDbName(pCName, req.dbFName);
int32_t code = buildUsingInsertTableReq(pSName, pCName, &req.pTables);
if (TSDB_CODE_SUCCESS == code && NULL == taosArrayPush(*pDbs, &req)) {
code = TSDB_CODE_OUT_OF_MEMORY;
}
return code;
}
static int32_t buildInsertDbReq(SName* pName, SArray** pDbs) {
if (NULL == *pDbs) {
*pDbs = taosArrayInit(1, sizeof(STablesReq));
@ -3182,7 +3293,7 @@ static int32_t buildInsertCatalogReq(SInsertParseContext* pCxt, SVnodeModifyOpSt
if (0 == pStmt->usingTableName.type) {
code = buildInsertDbReq(&pStmt->targetTableName, &pCatalogReq->pTableMeta);
} else {
code = buildInsertDbReq(&pStmt->usingTableName, &pCatalogReq->pTableMeta);
code = buildInsertUsingDbReq(&pStmt->usingTableName, &pStmt->targetTableName, &pCatalogReq->pTableMeta);
}
}
if (TSDB_CODE_SUCCESS == code) {

29
source/libs/parser/src/parTranslater.c
View File

@ -10503,10 +10503,18 @@ static int32_t translateCreateUser(STranslateContext* pCxt, SCreateUserStmt* pSt
createReq.superUser = 0;
createReq.sysInfo = pStmt->sysinfo;
createReq.enable = 1;
tstrncpy(createReq.pass, pStmt->password, TSDB_USET_PASSWORD_LEN);
createReq.isImport = pStmt->isImport;
createReq.createDb = pStmt->createDb;
if(pStmt->isImport == 1){
tstrncpy(createReq.pass, pStmt->password, TSDB_USET_PASSWORD_LEN);
}
else{
taosEncryptPass_c((uint8_t*)pStmt->password, strlen(pStmt->password), createReq.pass);
}
createReq.passIsMd5 = 1;
createReq.numIpRanges = pStmt->numIpRanges;
if (pStmt->numIpRanges > 0) {
createReq.pIpRanges = taosMemoryMalloc(createReq.numIpRanges * sizeof(SIpV4Range));
@ -10548,7 +10556,13 @@ static int32_t translateAlterUser(STranslateContext* pCxt, SAlterUserStmt* pStmt
alterReq.enable = pStmt->enable;
alterReq.sysInfo = pStmt->sysinfo;
alterReq.createdb = pStmt->createdb ? 1 : 0;
snprintf(alterReq.pass, sizeof(alterReq.pass), "%s", pStmt->password);
int32_t len = strlen(pStmt->password);
if (len > 0) {
taosEncryptPass_c((uint8_t*)pStmt->password, len, alterReq.pass);
alterReq.passIsMd5 = 1;
}
if (NULL != pCxt->pParseCxt->db) {
snprintf(alterReq.objname, sizeof(alterReq.objname), "%s", pCxt->pParseCxt->db);
}
@ -13311,7 +13325,16 @@ static int32_t translateSplitVgroup(STranslateContext* pCxt, SSplitVgroupStmt* p
static int32_t translateShowVariables(STranslateContext* pCxt, SShowStmt* pStmt) {
SShowVariablesReq req = {0};
return buildCmdMsg(pCxt, TDMT_MND_SHOW_VARIABLES, (FSerializeFunc)tSerializeSShowVariablesReq, &req);
req.opType = pStmt->tableCondType;
if (req.opType == OP_TYPE_LIKE && pStmt->pTbName) {
req.valLen = strlen(((SValueNode*)pStmt->pTbName)->literal);
if (req.valLen > 0) {
req.val = taosStrdupi(((SValueNode*)pStmt->pTbName)->literal);
}
}
int32_t code = buildCmdMsg(pCxt, TDMT_MND_SHOW_VARIABLES, (FSerializeFunc)tSerializeSShowVariablesReq, &req);
tFreeSShowVariablesReq(&req);
return code;
}
static int32_t translateShowCreateDatabase(STranslateContext* pCxt, SShowCreateDatabaseStmt* pStmt) {

6
source/libs/parser/test/parAlterToBalanceTest.cpp
View File

@ -817,7 +817,7 @@ TEST_F(ParserInitialATest, alterUser) {
expect.sysInfo = sysInfo;
expect.enable = enable;
if (nullptr != pPass) {
strcpy(expect.pass, pPass);
taosEncryptPass_c((uint8_t*)pPass, strlen(pPass), expect.pass);
}
strcpy(expect.objname, "test");
};
@ -838,8 +838,8 @@ TEST_F(ParserInitialATest, alterUser) {
tFreeSAlterUserReq(&req);
});
setAlterUserReq("wxy", TSDB_ALTER_USER_PASSWD, "123456");
run("ALTER USER wxy PASS '123456'");
setAlterUserReq("wxy", TSDB_ALTER_USER_PASSWD, "12345678@Abc");
run("ALTER USER wxy PASS '12345678@Abc'");
clearAlterUserReq();
setAlterUserReq("wxy", TSDB_ALTER_USER_ENABLE, nullptr, 0, 1);

10
source/libs/parser/test/parInitialCTest.cpp
View File

@ -1345,11 +1345,11 @@ TEST_F(ParserInitialCTest, createUser) {
auto setCreateUserReq = [&](const char* pUser, const char* pPass, int8_t sysInfo = 1) {
strcpy(expect.user, pUser);
strcpy(expect.pass, pPass);
expect.createType = 0;
expect.superUser = 0;
expect.sysInfo = sysInfo;
expect.enable = 1;
taosEncryptPass_c((uint8_t*)pPass, strlen(pPass), expect.pass);
};
setCheckDdlFunc([&](const SQuery* pQuery, ParserStage stage) {
@ -1366,12 +1366,12 @@ TEST_F(ParserInitialCTest, createUser) {
tFreeSCreateUserReq(&req);
});
setCreateUserReq("wxy", "123456");
run("CREATE USER wxy PASS '123456'");
setCreateUserReq("wxy", "12345678@Abc");
run("CREATE USER wxy PASS '12345678@Abc'");
clearCreateUserReq();
setCreateUserReq("wxy1", "a123456", 1);
run("CREATE USER wxy1 PASS 'a123456' SYSINFO 1");
setCreateUserReq("wxy1", "12345678@Abc", 1);
run("CREATE USER wxy1 PASS '12345678@Abc' SYSINFO 1");
clearCreateUserReq();
}

6
source/libs/sync/src/syncPipeline.c
View File

@ -733,9 +733,11 @@ int32_t syncFsmExecute(SSyncNode* pNode, SSyncFSM* pFsm, ESyncState role, SyncTe
if (retry) {
taosMsleep(10);
if (code == TSDB_CODE_OUT_OF_RPC_MEMORY_QUEUE) {
sError("vgId:%d, failed to execute fsm since %s. index:%" PRId64, pNode->vgId, terrstr(), pEntry->index);
sError("vgId:%d, will retry to execute fsm after 10ms, last error is %s. index:%" PRId64, pNode->vgId,
tstrerror(code), pEntry->index);
} else {
sDebug("vgId:%d, retry on fsm commit since %s. index:%" PRId64, pNode->vgId, terrstr(), pEntry->index);
sDebug("vgId:%d, will retry to execute fsm after 10ms, last error is %s. index:%" PRId64, pNode->vgId,
tstrerror(code), pEntry->index);
}
}
} while (retry);

31
source/os/CMakeLists.txt
View File

@ -1,4 +1,5 @@
aux_source_directory(src OS_SRC)
if(NOT ${TD_WINDOWS})
add_definitions(-DTHREAD_SAFE=1)
aux_source_directory(src/timezone OS_TZ)
@ -6,6 +7,7 @@ if(NOT ${TD_WINDOWS})
else()
add_library(os STATIC ${OS_SRC})
endif(NOT ${TD_WINDOWS})
target_include_directories(
os
PUBLIC "${TD_SOURCE_DIR}/include/os"
@ -16,21 +18,26 @@ target_include_directories(
PUBLIC "${TD_SOURCE_DIR}/contrib/msvcregex"
PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
)
# iconv
if(TD_WINDOWS)
find_path(IconvApiIncludes iconv.h "${TD_SOURCE_DIR}/contrib/iconv")
else()
find_path(IconvApiIncludes iconv.h PATHS)
endif(TD_WINDOWS)
if(NOT IconvApiIncludes)
add_definitions(-DDISALLOW_NCHAR_WITHOUT_ICONV)
endif ()
endif()
if(USE_TD_MEMORY)
add_definitions(-DUSE_TD_MEMORY)
endif ()
endif()
if(BUILD_WITH_RAND_ERR)
add_definitions(-DBUILD_WITH_RAND_ERR)
endif ()
endif()
if(BUILD_ADDR2LINE)
if(NOT TD_WINDOWS)
target_include_directories(
@ -41,16 +48,21 @@ if(BUILD_ADDR2LINE)
os PUBLIC addr2line dl z
)
endif()
add_definitions(-DUSE_ADDR2LINE)
endif ()
endif()
if(CHECK_STR2INT_ERROR)
add_definitions(-DTD_CHECK_STR_TO_INT_ERROR)
endif()
target_link_libraries(
os
PUBLIC pthread
PUBLIC zlibstatic
PUBLIC ${LINK_JEMALLOC}
)
if(TD_WINDOWS)
target_link_libraries(
os PUBLIC ws2_32 iconv msvcregex wcwidth winmm crashdump dbghelp version KtmW32
@ -73,15 +85,14 @@ else()
)
endif()
IF (JEMALLOC_ENABLED)
target_link_libraries(os PUBLIC -L${CMAKE_BINARY_DIR}/build/lib -ljemalloc)
ENDIF ()
if(JEMALLOC_ENABLED)
add_dependencies(os jemalloc)
endif()
#if(NOT ${TD_WINDOWS})
# if(NOT ${TD_WINDOWS})
# find_library(tz libtz.a "${TD_SOURCE_DIR}/contrib/tz")
# target_link_libraries(os PUBLIC ${tz})
#endif(NOT ${TD_WINDOWS})
# endif(NOT ${TD_WINDOWS})
if(${BUILD_TEST})
add_subdirectory(test)
endif(${BUILD_TEST})

13
source/util/src/tcompare.c
View File

@ -1123,6 +1123,19 @@ int32_t patternMatch(const char *pattern, size_t psize, const char *str, size_t
return (j >= ssize || str[j] == 0) ? TSDB_PATTERN_MATCH : TSDB_PATTERN_NOMATCH;
}
int32_t rawStrPatternMatch(const char *str, const char *pattern) {
SPatternCompareInfo pInfo = PATTERN_COMPARE_INFO_INITIALIZER;
size_t pLen = strlen(pattern);
size_t sz = strlen(str);
if (pLen > TSDB_MAX_FIELD_LEN) {
return 1;
}
int32_t ret = patternMatch(pattern, pLen, str, sz, &pInfo);
return (ret == TSDB_PATTERN_MATCH) ? 0 : 1;
}
int32_t wcsPatternMatch(const TdUcs4 *pattern, size_t psize, const TdUcs4 *str, size_t ssize,
const SPatternCompareInfo *pInfo) {
TdUcs4 c, c1;

4
source/util/src/terror.c
View File

@ -130,7 +130,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_VERSION, "Invalid client versio
TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_IE, "Invalid client ie")
TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_FQDN, "Invalid host name")
TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_USER_LENGTH, "Invalid user name")
TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_PASS_LENGTH, "Invalid password")
TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_PASS_LENGTH, "Invalid password length")
TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_DB_LENGTH, "Database name too long")
TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH, "Table name too long")
TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_CONNECTION, "Invalid connection")
@ -695,7 +695,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_PAR_PERMISSION_DENIED, "Permission denied o
TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_STREAM_QUERY, "Invalid stream query")
TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_INTERNAL_PK, "Invalid _c0 or _rowts expression")
TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_TIMELINE_FUNC, "Invalid timeline function")
TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_PASSWD, "Invalid password")
TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_PASSWD, "Invalid password 2")
TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_ALTER_TABLE, "Invalid alter table statement")
TAOS_DEFINE_ERROR(TSDB_CODE_PAR_CANNOT_DROP_PRIMARY_KEY, "Primary timestamp column cannot be dropped")
TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_MODIFY_COL, "Only varbinary/binary/nchar/geometry column length could be modified, and the length can only be increased, not decreased")

68
tests/army/cluster/strongPassword.py Normal file
View File

@ -0,0 +1,68 @@
import taos
import sys
import os
import subprocess
import glob
import shutil
import time
from frame.log import *
from frame.cases import *
from frame.sql import *
from frame.srvCtl import *
from frame.caseBase import *
from frame import *
from frame.autogen import *
from frame import epath
# from frame.server.dnodes import *
# from frame.server.cluster import *
class TDTestCase(TBase):
def init(self, conn, logSql, replicaVar=1):
super(TDTestCase, self).init(conn, logSql, replicaVar=1, checkColName="c1")
tdSql.init(conn.cursor(), logSql)
def run(self):
# strong
tdSql.error("create user test pass '12345678' sysinfo 0;", expectErrInfo="Invalid password")
tdSql.execute("create user test pass '12345678@Abc' sysinfo 0;")
tdSql.error("alter user test pass '23456789'", expectErrInfo="Invalid password")
tdSql.execute("alter user test pass '23456789@Abc';")
# change setting
tdSql.execute("ALTER ALL DNODES 'enableStrongPassword' '0'")
time.sleep(3)
# weak
tdSql.execute("create user test1 pass '12345678' sysinfo 0;")
tdSql.execute("alter user test1 pass '12345678';")
# pass length
tdSql.error("alter user test1 pass '1234567';", expectErrInfo="Password too short or empty")
tdSql.error("create user test2 pass '1234567' sysinfo 0;", expectErrInfo="Password too short or empty")
tdSql.error("create user test2 pass '1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456' sysinfo 0;", expectErrInfo="Name or password too long")
tdSql.execute("create user test2 pass '123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345' sysinfo 0;")
cmd = "taos -u test2 -p123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345 -s 'show databases;'"
if os.system(cmd) != 0:
raise Exception("failed to execute system command. cmd: %s" % cmd)
tdSql.error("alter user test2 pass '1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456';", expectErrInfo="Name or password too long")
def stop(self):
tdSql.close()
tdLog.success(f"{__file__} successfully executed")
tdCases.addLinux(__file__, TDTestCase())
tdCases.addWindows(__file__, TDTestCase())

114
tests/army/insert/auto_create_bench.py Normal file
View File

@ -0,0 +1,114 @@
import time
import taos
conn = taos.connect()
total_batches = 100
tables_per_batch = 100
def prepare_database():
cursor = conn.cursor()
cursor.execute("DROP DATABASE IF EXISTS test")
cursor.execute("CREATE DATABASE IF NOT EXISTS test")
cursor.execute("USE test")
cursor.execute("CREATE STABLE IF NOT EXISTS stb (ts TIMESTAMP, a INT, b FLOAT, c BINARY(10)) TAGS (e_id INT)")
cursor.close()
def test_auto_create_tables():
"""测试场景1：自动建表插入"""
cursor = conn.cursor()
cursor.execute("USE test")
print("开始测试自动建表插入...")
start_time = time.time()
for _ in range(100):
for batch in range(total_batches):
# 生成当前批次的子表ID范围
start_id = batch * tables_per_batch
end_id = start_id + tables_per_batch
# 构建批量插入SQL
sql_parts = []
for i in range(start_id, end_id):
sql_part = f"t_{i} USING stb TAGS ({i}) VALUES ('2024-01-01 00:00:00', 1, 2.0, 'test')"
sql_parts.append(sql_part)
# 执行批量插入
full_sql = "INSERT INTO " + " ".join(sql_parts)
cursor.execute(full_sql)
elapsed = time.time() - start_time
print(f"自动建表插入耗时: {elapsed:.2f} 秒")
cursor.close()
return elapsed
def precreate_tables():
"""预处理：创建所有子表结构"""
cursor = conn.cursor()
cursor.execute("USE test")
print("\n开始预创建子表...")
start_time = time.time()
for batch in range(total_batches):
start_id = batch * tables_per_batch
end_id = start_id + tables_per_batch
for i in range(start_id, end_id):
sql_part = f"CREATE TABLE t_{i} USING stb TAGS ({i})"
cursor.execute(sql_part)
elapsed = time.time() - start_time
print(f"子表预创建耗时: {elapsed:.2f} 秒")
cursor.close()
def test_direct_insert():
"""测试场景2：直接插入已存在的子表"""
cursor = conn.cursor()
cursor.execute("USE test")
print("\n开始测试直接插入...")
start_time = time.time()
for _ in range(100):
for batch in range(total_batches):
start_id = batch * tables_per_batch
end_id = start_id + tables_per_batch
# 构建批量插入SQL
sql_parts = []
for i in range(start_id, end_id):
sql_part = f"t_{i} VALUES ('2024-01-01 00:00:01', 1, 2.0, 'test')"
sql_parts.append(sql_part)
# 执行批量插入
full_sql = "INSERT INTO " + " ".join(sql_parts)
cursor.execute(full_sql)
elapsed = time.time() - start_time
print(f"直接插入耗时: {elapsed:.2f} 秒")
cursor.close()
return elapsed
if __name__ == "__main__":
# 初始化数据库环境
prepare_database()
# 预创建所有子表
precreate_tables()
# 测试场景1：自动建表插入
auto_create_time = test_auto_create_tables()
# # 清理环境并重新初始化
# prepare_database()
# # 预创建所有子表
# precreate_tables()
# # 测试场景2：直接插入
# direct_insert_time = test_direct_insert()
# 打印最终结果
print("\n测试结果对比:")
print(f"自动建表插入耗时: {auto_create_time:.2f} 秒")
# print(f"直接插入耗时: {direct_insert_time:.2f} 秒")
# print(f"性能差异: {auto_create_time/direct_insert_time:.1f} 倍")

195
tests/army/insert/auto_create_insert.py Normal file
View File

@ -0,0 +1,195 @@
###################################################################
# Copyright (c) 2016 by TAOS Technologies, Inc.
# All rights reserved.
#
# This file is proprietary and confidential to TAOS Technologies.
# No part of this file may be reproduced, stored, transmitted,
# disclosed or used in any form or by any means other than as
# expressly provided by the written permission from Jianhui Tao
#
###################################################################
# -*- coding: utf-8 -*-
import sys
import time
import random
import taos
import frame
import frame.etool
from frame.log import *
from frame.cases import *
from frame.sql import *
from frame.caseBase import *
from frame import *
class TDTestCase(TBase):
def prepare_database(self):
tdLog.info(f"prepare database")
tdSql.execute("DROP DATABASE IF EXISTS test")
tdSql.execute("CREATE DATABASE IF NOT EXISTS test")
tdSql.execute("USE test")
tdSql.execute("CREATE STABLE IF NOT EXISTS stb (ts TIMESTAMP, a INT, b FLOAT, c BINARY(10)) TAGS (e_id INT)")
def insert_table_auto_create(self):
tdLog.info(f"insert table auto create")
tdSql.execute("USE test")
tdLog.info("start to test auto create insert...")
tdSql.execute("INSERT INTO t_0 USING stb TAGS (0) VALUES ('2024-01-01 00:00:00', 1, 2.0, 'test')")
tdSql.execute("INSERT INTO t_0 USING stb TAGS (0) VALUES ('2024-01-01 00:00:01', 1, 2.0, 'test')")
tdSql.query("select * from t_0")
tdSql.checkRows(2)
def insert_table_pre_create(self):
tdLog.info(f"insert table pre create")
tdSql.execute("USE test")
tdLog.info("start to pre create table...")
tdSql.execute("CREATE TABLE t_1 USING stb TAGS (1)")
tdLog.info("start to test pre create insert...")
tdSql.execute("INSERT INTO t_1 USING stb TAGS (1) VALUES ('2024-01-01 00:00:00', 1, 2.0, 'test')")
tdSql.execute("INSERT INTO t_1 VALUES ('2024-01-01 00:00:01', 1, 2.0, 'test')")
tdSql.query("select * from t_1")
tdSql.checkRows(2)
def insert_table_auto_insert_with_cache(self):
tdLog.info(f"insert table auto insert with cache")
tdSql.execute("USE test")
tdLog.info("start to test auto insert with cache...")
tdSql.execute("CREATE TABLE t_2 USING stb TAGS (2)")
tdLog.info("start to insert to init cache...")
tdSql.execute("INSERT INTO t_2 VALUES ('2024-01-01 00:00:00', 1, 2.0, 'test')")
tdSql.execute("INSERT INTO t_2 USING stb TAGS (2) VALUES ('2024-01-01 00:00:01', 1, 2.0, 'test')")
tdSql.query("select * from t_2")
tdSql.checkRows(2)
def insert_table_auto_insert_with_multi_rows(self):
tdLog.info(f"insert table auto insert with multi rows")
tdSql.execute("USE test")
tdLog.info("start to test auto insert with multi rows...")
tdSql.execute("CREATE TABLE t_3 USING stb TAGS (3)")
tdLog.info("start to insert multi rows...")
tdSql.execute("INSERT INTO t_3 VALUES ('2024-01-01 00:00:00', 1, 2.0, 'test'), ('2024-01-01 00:00:01', 1, 2.0, 'test')")
tdSql.query("select * from t_3")
tdSql.checkRows(2)
tdLog.info("start to insert multi rows with direct insert and auto create...")
tdSql.execute("INSERT INTO t_4 USING stb TAGS (4) VALUES ('2024-01-01 00:00:00', 1, 2.0, 'test'), t_3 VALUES ('2024-01-01 00:00:02', 1, 2.0, 'test')")
tdSql.query("select * from t_4")
tdSql.checkRows(1)
tdSql.query("select * from t_3")
tdSql.checkRows(3)
tdLog.info("start to insert multi rows with auto create and direct insert...")
tdSql.execute("INSERT INTO t_3 VALUES ('2024-01-01 00:00:03', 1, 2.0, 'test'),t_4 USING stb TAGS (4) VALUES ('2024-01-01 00:00:01', 1, 2.0, 'test'),")
tdSql.query("select * from t_4")
tdSql.checkRows(2)
tdSql.query("select * from t_3")
tdSql.checkRows(4)
tdLog.info("start to insert multi rows with auto create into same table...")
tdSql.execute("INSERT INTO t_10 USING stb TAGS (10) VALUES ('2024-01-01 00:00:04', 1, 2.0, 'test'),t_10 USING stb TAGS (10) VALUES ('2024-01-01 00:00:05', 1, 2.0, 'test'),")
tdSql.query("select * from t_10")
tdSql.checkRows(2)
def check_some_err_case(self):
tdLog.info(f"check some err case")
tdSql.execute("USE test")
tdLog.info("start to test err stb name...")
tdSql.error("INSERT INTO t_5 USING errrrxx TAGS (5) VALUES ('2024-01-01 00:00:00', 1, 2.0, 'test')", expectErrInfo="Table does not exist")
tdLog.info("start to test err syntax name...")
tdSql.error("INSERT INTO t_5 USING stb TAG (5) VALUES ('2024-01-01 00:00:00', 1, 2.0, 'test')", expectErrInfo="syntax error")
tdLog.info("start to test err syntax values...")
tdSql.error("INSERT INTO t_5 USING stb TAG (5) VALUS ('2024-01-01 00:00:00', 1, 2.0, 'test')", expectErrInfo="syntax error")
tdLog.info("start to test err tag counts...")
tdSql.error("INSERT INTO t_5 USING stb TAG (5,1) VALUS ('2024-01-01 00:00:00', 1, 2.0, 'test')", expectErrInfo="syntax error")
tdLog.info("start to test err tag counts...")
tdSql.error("INSERT INTO t_5 USING stb TAG ('dasds') VALUS ('2024-01-01 00:00:00', 1, 2.0, 'test')", expectErrInfo="syntax error")
tdLog.info("start to test err values counts...")
tdSql.error("INSERT INTO t_5 USING stb TAGS (5) VALUES ('2024-01-01 00:00:00', 1, 1 ,2.0, 'test')", expectErrInfo="Illegal number of columns")
tdLog.info("start to test err values...")
tdSql.error("INSERT INTO t_5 USING stb TAGS (5) VALUES ('2024-01-01 00:00:00', 'dasdsa', 1 ,2.0, 'test')", expectErrInfo="syntax error")
def check_same_table_same_ts(self):
tdLog.info(f"check same table same ts")
tdSql.execute("USE test")
tdSql.execute("INSERT INTO t_6 USING stb TAGS (6) VALUES ('2024-01-01 00:00:00', 1, 2.0, 'test') t_6 USING stb TAGS (6) VALUES ('2024-01-01 00:00:00', 1, 2.0, 'test')")
tdSql.query("select * from t_6")
tdSql.checkRows(1)
def check_tag_parse_error_with_cache(self):
tdLog.info(f"check tag parse error with cache")
tdSql.execute("USE test")
tdSql.execute("INSERT INTO t_7 USING stb TAGS (7) VALUES ('2024-01-01 00:00:00', 1, 2.0, 'test')")
tdSql.error("INSERT INTO t_7 USING stb TAGS ('ddd') VALUES ('2024-01-01 00:00:00', 1, 2.0, 'test')", expectErrInfo="syntax error")
tdSql.query("select * from t_7")
tdSql.checkRows(1)
def check_duplicate_table_with_err_tag(self):
tdLog.info(f"check tag parse error with cache")
tdSql.execute("USE test")
tdSql.execute("INSERT INTO t_8 USING stb TAGS (8) VALUES ('2024-01-01 00:00:00', 1, 2.0, 'test') t_8 USING stb TAGS (ddd) VALUES ('2024-01-01 00:00:00', 1, 2.0, 'test')")
tdSql.query("select * from t_8")
tdSql.checkRows(1)
def check_table_with_another_stb_name(self):
tdLog.info(f"check table with another stb name")
tdSql.execute("USE test")
tdSql.execute("CREATE STABLE IF NOT EXISTS stb2 (ts TIMESTAMP, a INT, b FLOAT, c BINARY(10)) TAGS (e_id INT)")
tdSql.execute("INSERT INTO t_20 USING stb2 TAGS (20) VALUES ('2024-01-01 00:00:00', 1, 2.0, 'test')")
tdSql.query("select * from t_20")
tdSql.checkRows(1)
tdSql.error("INSERT INTO t_20 USING stb TAGS (20) VALUES ('2024-01-01 00:00:00', 1, 2.0, 'test')", expectErrInfo="Table already exists in other stables")
tdSql.error("INSERT INTO t_20 USING stb TAGS (20) VALUES ('2024-01-01 00:00:00', 1, 2.0, 'test')", expectErrInfo="Table already exists in other stables")
# run
def run(self):
tdLog.debug(f"start to excute {__file__}")
# prepare database
self.prepare_database()
# insert table auto create
self.insert_table_auto_create()
# insert table pre create
self.insert_table_pre_create()
# insert table auto insert with cache
self.insert_table_auto_insert_with_cache()
# insert table auto insert with multi rows
self.insert_table_auto_insert_with_multi_rows()
# check some err case
self.check_some_err_case()
# check same table same ts
self.check_same_table_same_ts()
# check tag parse error with cache
self.check_tag_parse_error_with_cache()
# check duplicate table with err tag
self.check_duplicate_table_with_err_tag()
# check table with another stb name
self.check_table_with_another_stb_name()
tdLog.success(f"{__file__} successfully executed")
tdCases.addLinux(__file__, TDTestCase())
tdCases.addWindows(__file__, TDTestCase())

237
tests/army/tools/benchmark/basic/csv-export.py Normal file
View File

@ -0,0 +1,237 @@
###################################################################
# Copyright (c) 2016 by TAOS Technologies, Inc.
# All rights reserved.
#
# This file is proprietary and confidential to TAOS Technologies.
# No part of this file may be reproduced, stored, transmitted,
# disclosed or used in any form or by any means other than as
# expressly provided by the written permission from Jianhui Tao
#
###################################################################
# -*- coding: utf-8 -*-
import os
import json
import csv
import datetime
import frame
import frame.eos
import frame.etool
from frame.log import *
from frame.cases import *
from frame.sql import *
from frame.caseBase import *
from frame import *
class TDTestCase(TBase):
def caseDescription(self):
"""
[TS-5089] taosBenchmark support exporting csv
"""
def clear_directory(self, target_dir: str = 'csv'):
try:
if not os.path.exists(target_dir):
return
for entry in os.listdir(target_dir):
entry_path = os.path.join(target_dir, entry)
if os.path.isfile(entry_path) or os.path.islink(entry_path):
os.unlink(entry_path)
else:
shutil.rmtree(entry_path)
tdLog.debug("clear succ, dir: %s " % (target_dir))
except OSError as e:
tdLog.exit("clear fail, dir: %s " % (target_dir))
def convert_timestamp(self, ts, ts_format):
dt_object = datetime.datetime.fromtimestamp(ts / 1000)
formatted_time = dt_object.strftime(ts_format)
return formatted_time
def calc_time_slice_partitions(self, total_start_ts, total_end_ts, ts_step, ts_format, ts_interval):
interval_days = int(ts_interval[:-1])
n_days_millis = interval_days * 24 * 60 * 60 * 1000
dt_start = datetime.datetime.fromtimestamp(total_start_ts / 1000.0)
formatted_str = dt_start.strftime(ts_format)
s0_dt = datetime.datetime.strptime(formatted_str, ts_format)
s0 = int(s0_dt.timestamp() * 1000)
partitions = []
current_s = s0
while current_s <= total_end_ts:
current_end = current_s + n_days_millis
start_actual = max(current_s, total_start_ts)
end_actual = min(current_end, total_end_ts)
if start_actual >= end_actual:
count = 0
else:
delta = end_actual - start_actual
delta
delta_start = start_actual - total_start_ts
delta_end = end_actual - total_start_ts
if delta % ts_step:
count = delta // ts_step + 1
else:
count = delta // ts_step
partitions.append({
"start_ts": current_s,
"end_ts": current_end,
"start_time": self.convert_timestamp(current_s, ts_format),
"end_time": self.convert_timestamp(current_end, ts_format),
"count": count
})
current_s += n_days_millis
# partitions = [p for p in partitions if p['count'] > 0]
return partitions
def check_stb_csv_correct(self, csv_file_name, all_rows, interlace_rows):
# open as csv
tbname_idx = 14
count = 0
batch = 0
name = ""
header = True
with open(csv_file_name) as file:
rows = csv.reader(file)
for row in rows:
if header:
header = False
continue
# interlace_rows
if name == "":
name = row[tbname_idx]
batch = 1
else:
if name == row[tbname_idx]:
batch += 1
else:
# switch to another child table
if batch != interlace_rows:
tdLog.exit(f"interlace_rows invalid. tbName={name} actual={batch} expected={interlace_rows} i={count} csv_file_name={csv_file_name}")
batch = 1
name = row[tbname_idx]
# count ++
count += 1
# batch
if batch != interlace_rows:
tdLog.exit(f"interlace_rows invalid. tbName={name} actual={batch} expected={interlace_rows} i={count} csv_file_name={csv_file_name}")
# check all rows
if count != all_rows:
tdLog.exit(f"all_rows invalid. actual={count} expected={all_rows} csv_file_name={csv_file_name}")
tdLog.info(f"Check generate csv file successfully. csv_file_name={csv_file_name} count={count} interlace_rows={batch}")
# check correct
def check_stb_correct(self, data, db, stb):
filepath = data["output_dir"]
stbName = stb["name"]
child_count = stb["childtable_to"] - stb["childtable_from"]
insert_rows = stb["insert_rows"]
interlace_rows = stb["interlace_rows"]
csv_file_prefix = stb["csv_file_prefix"]
csv_ts_format = stb.get("csv_ts_format", None)
csv_ts_interval = stb.get("csv_ts_interval", None)
ts_step = stb["timestamp_step"]
total_start_ts = stb["start_timestamp"]
total_end_ts = total_start_ts + ts_step * insert_rows
all_rows = child_count * insert_rows
if interlace_rows > 0:
# interlace
if not csv_ts_format:
# normal
csv_file_name = f"{filepath}{csv_file_prefix}.csv"
self.check_stb_csv_correct(csv_file_name, all_rows, interlace_rows)
else:
# time slice
partitions = self.calc_time_slice_partitions(total_start_ts, total_end_ts, ts_step, csv_ts_format, csv_ts_interval)
for part in partitions:
csv_file_name = f"{filepath}{csv_file_prefix}_{part['start_time']}_{part['end_time']}.csv"
self.check_stb_csv_correct(csv_file_name, part['count'] * child_count, interlace_rows)
else:
# batch
thread_count = stb["thread_count"]
interlace_rows = insert_rows
if not csv_ts_format:
# normal
for i in range(thread_count):
csv_file_name = f"{filepath}{csv_file_prefix}_{i + 1}.csv"
if i < child_count % thread_count:
self.check_stb_csv_correct(csv_file_name, insert_rows * (child_count // thread_count + 1), interlace_rows)
else:
self.check_stb_csv_correct(csv_file_name, insert_rows * (child_count // thread_count), interlace_rows)
else:
# time slice
for i in range(thread_count):
partitions = self.calc_time_slice_partitions(total_start_ts, total_end_ts, ts_step, csv_ts_format, csv_ts_interval)
for part in partitions:
csv_file_name = f"{filepath}{csv_file_prefix}_{i + 1}_{part['start_time']}_{part['end_time']}.csv"
if i < child_count % thread_count:
slice_rows = part['count'] * (child_count // thread_count + 1)
else:
slice_rows = part['count'] * (child_count // thread_count)
self.check_stb_csv_correct(csv_file_name, slice_rows, part['count'])
# check result
def check_result(self, jsonFile):
# csv
with open(jsonFile) as file:
data = json.load(file)
# read json
database = data["databases"][0]
stables = database["super_tables"]
for stable in stables:
# check csv context correct
self.check_stb_correct(data, database, stable)
def check_export_csv(self, benchmark, jsonFile, options=""):
# clear
self.clear_directory()
# exec
cmd = f"{benchmark} {options} -f {jsonFile}"
eos.exe(cmd)
# check result
self.check_result(jsonFile)
def run(self):
# path
benchmark = etool.benchMarkFile()
# do check interlace normal
json = "tools/benchmark/basic/json/csv-export.json"
self.check_export_csv(benchmark, json)
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())

110
tests/army/tools/benchmark/basic/exportCsv.py
View File

@ -1,110 +0,0 @@
###################################################################
# Copyright (c) 2016 by TAOS Technologies, Inc.
# All rights reserved.
#
# This file is proprietary and confidential to TAOS Technologies.
# No part of this file may be reproduced, stored, transmitted,
# disclosed or used in any form or by any means other than as
# expressly provided by the written permission from Jianhui Tao
#
###################################################################
# -*- coding: utf-8 -*-
import os
import json
import csv
import frame
import frame.etool
from frame.log import *
from frame.cases import *
from frame.sql import *
from frame.caseBase import *
from frame import *
class TDTestCase(TBase):
def caseDescription(self):
"""
[TD-11510] taosBenchmark test cases
"""
# check correct
def checkCorrect(self, csvFile, allRows, interlaceRows):
# open as csv
count = 0
batch = 0
name = ""
with open(csvFile) as file:
rows = csv.reader(file)
for row in rows:
# interlaceRows
if name == "":
name = row[0]
batch = 1
else:
if name == row[0]:
batch += 1
else:
# switch to another child table
if batch != interlaceRows:
tdLog.exit(f"interlaceRows invalid. tbName={name} real={batch} expect={interlaceRows} i={count} csvFile={csvFile}")
batch = 1
name = row[0]
# count ++
count += 1
# batch
if batch != interlaceRows:
tdLog.exit(f"interlaceRows invalid. tbName={name} real={batch} expect={interlaceRows} i={count} csvFile={csvFile}")
# check all rows
if count != allRows:
tdLog.exit(f"allRows invalid. real={count} expect={allRows} csvFile={csvFile}")
tdLog.info(f"Check generate csv file successfully. csvFile={csvFile} count={count} interlaceRows={batch}")
# check result
def checResult(self, jsonFile):
# csv
with open(jsonFile) as file:
data = json.load(file)
# read json
database = data["databases"][0]
out = data["csvPath"]
dbName = database["dbinfo"]["name"]
stables = database["super_tables"]
for stable in stables:
stbName = stable["name"]
childs = stable["childtable_count"]
insertRows = stable["insert_rows"]
interlaceRows = stable["interlace_rows"]
csvFile = f"{out}{dbName}-{stbName}.csv"
rows = childs * insertRows
if interlaceRows == 0:
interlaceRows = insertRows
# check csv context correct
self.checkCorrect(csvFile, rows, interlaceRows)
def checkExportCsv(self, benchmark, jsonFile, options=""):
# exec
cmd = f"{benchmark} {options} -f {jsonFile}"
os.system(cmd)
# check result
self.checResult(jsonFile)
def run(self):
# path
benchmark = etool.benchMarkFile()
# do check
json = "tools/benchmark/basic/json/exportCsv.json"
self.checkExportCsv(benchmark, json)
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())

172
tests/army/tools/benchmark/basic/json/csv-export.json Normal file
View File

@ -0,0 +1,172 @@
{
"filetype": "csvfile",
"output_dir": "./csv/",
"databases": [
{
"dbinfo": {
"name": "csvdb",
"precision": "ms"
},
"super_tables": [
{
"name": "interlace-normal",
"childtable_count": 1010,
"insert_rows": 1000,
"interlace_rows": 1,
"childtable_prefix": "d",
"timestamp_step": 1000000,
"start_timestamp":1700000000000,
"childtable_from": 1000,
"childtable_to": 1010,
"csv_file_prefix": "data",
"csv_output_header": "yes",
"csv_tbname_alias": "device_id",
"csv_compress_level": "none",
"columns": [
{ "type": "bool", "name": "bc"},
{ "type": "float", "name": "fc", "min": 1},
{ "type": "double", "name": "dc", "min":10, "max":10},
{ "type": "tinyint", "name": "ti"},
{ "type": "smallint", "name": "si"},
{ "type": "int", "name": "ic", "fillNull":"false"},
{ "type": "bigint", "name": "bi"},
{ "type": "utinyint", "name": "uti"},
{ "type": "usmallint", "name": "usi", "min":100, "max":120},
{ "type": "uint", "name": "ui"},
{ "type": "ubigint", "name": "ubi"},
{ "type": "binary", "name": "bin", "len": 16},
{ "type": "nchar", "name": "nch", "len": 16}
],
"tags": [
{"type": "tinyint", "name": "groupid","max": 10,"min": 1},
{"type": "binary", "name": "location", "len": 16,
"values": ["San Francisco", "Los Angles", "San Diego",
"San Jose", "Palo Alto", "Campbell", "Mountain View",
"Sunnyvale", "Santa Clara", "Cupertino"]
}
]
},
{
"name": "interlace-timeslice",
"childtable_count": 1010,
"insert_rows": 1000,
"interlace_rows": 1,
"childtable_prefix": "d",
"timestamp_step": 1000000,
"start_timestamp":1700000000000,
"childtable_from": 1000,
"childtable_to": 1010,
"csv_file_prefix": "data",
"csv_ts_format": "%Y%m%d",
"csv_ts_interval": "1d",
"csv_output_header": "yes",
"csv_tbname_alias": "device_id",
"csv_compress_level": "none",
"columns": [
{ "type": "bool", "name": "bc"},
{ "type": "float", "name": "fc", "min": 1},
{ "type": "double", "name": "dc", "min":10, "max":10},
{ "type": "tinyint", "name": "ti"},
{ "type": "smallint", "name": "si"},
{ "type": "int", "name": "ic", "fillNull":"false"},
{ "type": "bigint", "name": "bi"},
{ "type": "utinyint", "name": "uti"},
{ "type": "usmallint", "name": "usi", "min":100, "max":120},
{ "type": "uint", "name": "ui"},
{ "type": "ubigint", "name": "ubi"},
{ "type": "binary", "name": "bin", "len": 16},
{ "type": "nchar", "name": "nch", "len": 16}
],
"tags": [
{"type": "tinyint", "name": "groupid","max": 10,"min": 1},
{"type": "binary", "name": "location", "len": 16,
"values": ["San Francisco", "Los Angles", "San Diego",
"San Jose", "Palo Alto", "Campbell", "Mountain View",
"Sunnyvale", "Santa Clara", "Cupertino"]
}
]
},
{
"name": "batch-normal",
"childtable_count": 1010,
"insert_rows": 1000,
"interlace_rows": 0,
"thread_count": 8,
"childtable_prefix": "d",
"timestamp_step": 1000000,
"start_timestamp":1700000000000,
"childtable_from": 1000,
"childtable_to": 1010,
"csv_file_prefix": "data",
"csv_output_header": "yes",
"csv_tbname_alias": "device_id",
"csv_compress_level": "none",
"columns": [
{ "type": "bool", "name": "bc"},
{ "type": "float", "name": "fc", "min": 1},
{ "type": "double", "name": "dc", "min":10, "max":10},
{ "type": "tinyint", "name": "ti"},
{ "type": "smallint", "name": "si"},
{ "type": "int", "name": "ic", "fillNull":"false"},
{ "type": "bigint", "name": "bi"},
{ "type": "utinyint", "name": "uti"},
{ "type": "usmallint", "name": "usi", "min":100, "max":120},
{ "type": "uint", "name": "ui"},
{ "type": "ubigint", "name": "ubi"},
{ "type": "binary", "name": "bin", "len": 16},
{ "type": "nchar", "name": "nch", "len": 16}
],
"tags": [
{"type": "tinyint", "name": "groupid","max": 10,"min": 1},
{"type": "binary", "name": "location", "len": 16,
"values": ["San Francisco", "Los Angles", "San Diego",
"San Jose", "Palo Alto", "Campbell", "Mountain View",
"Sunnyvale", "Santa Clara", "Cupertino"]
}
]
},
{
"name": "batch-timeslice",
"childtable_count": 1010,
"insert_rows": 1000,
"interlace_rows": 0,
"thread_count": 8,
"childtable_prefix": "d",
"timestamp_step": 1000000,
"start_timestamp":1700000000000,
"childtable_from": 1000,
"childtable_to": 1010,
"csv_file_prefix": "data",
"csv_ts_format": "%Y%m%d",
"csv_ts_interval": "1d",
"csv_output_header": "yes",
"csv_tbname_alias": "device_id",
"csv_compress_level": "none",
"columns": [
{ "type": "bool", "name": "bc"},
{ "type": "float", "name": "fc", "min": 1},
{ "type": "double", "name": "dc", "min":10, "max":10},
{ "type": "tinyint", "name": "ti"},
{ "type": "smallint", "name": "si"},
{ "type": "int", "name": "ic", "fillNull":"false"},
{ "type": "bigint", "name": "bi"},
{ "type": "utinyint", "name": "uti"},
{ "type": "usmallint", "name": "usi", "min":100, "max":120},
{ "type": "uint", "name": "ui"},
{ "type": "ubigint", "name": "ubi"},
{ "type": "binary", "name": "bin", "len": 16},
{ "type": "nchar", "name": "nch", "len": 16}
],
"tags": [
{"type": "tinyint", "name": "groupid","max": 10,"min": 1},
{"type": "binary", "name": "location", "len": 16,
"values": ["San Francisco", "Los Angles", "San Diego",
"San Jose", "Palo Alto", "Campbell", "Mountain View",
"Sunnyvale", "Santa Clara", "Cupertino"]
}
]
}
]
}
]
}

78
tests/army/tools/benchmark/basic/json/exportCsv.json
View File

@ -1,78 +0,0 @@
{
"filetype": "csvfile",
"csvPath": "./csv/",
"num_of_records_per_req": 10000,
"databases": [
{
"dbinfo": {
"name": "csvdb"
},
"super_tables": [
{
"name": "batchTable",
"childtable_count": 5,
"insert_rows": 100,
"interlace_rows": 0,
"childtable_prefix": "d",
"timestamp_step": 10,
"start_timestamp":1600000000000,
"columns": [
{ "type": "bool", "name": "bc"},
{ "type": "float", "name": "fc", "min": 1},
{ "type": "double", "name": "dc", "min":10, "max":10},
{ "type": "tinyint", "name": "ti"},
{ "type": "smallint", "name": "si"},
{ "type": "int", "name": "ic", "fillNull":"false"},
{ "type": "bigint", "name": "bi"},
{ "type": "utinyint", "name": "uti"},
{ "type": "usmallint", "name": "usi", "min":100, "max":120},
{ "type": "uint", "name": "ui"},
{ "type": "ubigint", "name": "ubi"},
{ "type": "binary", "name": "bin", "len": 16},
{ "type": "nchar", "name": "nch", "len": 16}
],
"tags": [
{"type": "tinyint", "name": "groupid","max": 10,"min": 1},
{"type": "binary", "name": "location", "len": 16,
"values": ["San Francisco", "Los Angles", "San Diego",
"San Jose", "Palo Alto", "Campbell", "Mountain View",
"Sunnyvale", "Santa Clara", "Cupertino"]
}
]
},
{
"name": "interlaceTable",
"childtable_count": 5,
"insert_rows": 100,
"interlace_rows": 10,
"childtable_prefix": "d",
"timestamp_step": 1000,
"start_timestamp":1700000000000,
"columns": [
{ "type": "bool", "name": "bc"},
{ "type": "float", "name": "fc", "min":16},
{ "type": "double", "name": "dc", "min":16},
{ "type": "tinyint", "name": "ti"},
{ "type": "smallint", "name": "si"},
{ "type": "int", "name": "ic", "fillNull":"false"},
{ "type": "bigint", "name": "bi"},
{ "type": "utinyint", "name": "uti"},
{ "type": "usmallint", "name": "usi"},
{ "type": "uint", "name": "ui"},
{ "type": "ubigint", "name": "ubi"},
{ "type": "binary", "name": "bin", "len": 32},
{ "type": "nchar", "name": "nch", "len": 64}
],
"tags": [
{"type": "tinyint", "name": "groupid","max": 10,"min": 1},
{"type": "binary", "name": "location", "len": 16,
"values": ["San Francisco", "Los Angles", "San Diego",
"San Jose", "Palo Alto", "Campbell", "Mountain View",
"Sunnyvale", "Santa Clara", "Cupertino"]
}
]
}
]
}
]
}

2
tests/ci/container_build_newmachine.sh
View File

@ -61,7 +61,7 @@ docker run \
-v /root/go/pkg/mod:/root/go/pkg/mod \
-v /root/.cache/go-build:/root/.cache/go-build \
-v /root/.cos-local.1:/root/.cos-local.2 \
--rm --ulimit core=-1 taos_test:v1.0 sh -c "pip uninstall taospy -y;pip3 install taospy==2.7.2;cd $REP_DIR;rm -rf debug;mkdir -p debug;cd debug;cmake .. -DBUILD_HTTP=false -DBUILD_TOOLS=true -DBUILD_TEST=true -DWEBSOCKET=true -DBUILD_SANITIZER=1 -DTOOLS_SANITIZE=true $CMAKE_BUILD_TYPE -DTOOLS_BUILD_TYPE=Debug -DBUILD_TAOSX=false -DJEMALLOC_ENABLED=0;make -j 10|| exit 1 "
--rm --ulimit core=-1 taos_test:v1.0 sh -c "pip uninstall taospy -y; pip3 install taospy==2.7.2; cd $REP_DIR; rm -rf debug ;mkdir -p debug; cd debug; cmake .. -DBUILD_HTTP=false -DBUILD_TOOLS=true -DBUILD_TEST=ON -DWEBSOCKET=true -DBUILD_SANITIZER=1 -DTOOLS_SANITIZE=true $CMAKE_BUILD_TYPE -DTOOLS_BUILD_TYPE=Debug -DBUILD_TAOSX=false -DJEMALLOC_ENABLED=OFF; make -j 10|| exit 1 "
# -v ${REP_REAL_PATH}/community/contrib/jemalloc/:${REP_DIR}/community/contrib/jemalloc \
if [[ -d ${WORKDIR}/debugNoSan ]] ;then

5
tests/parallel_test/cases.task
View File

@ -46,11 +46,13 @@
,,y,army,./pytest.sh python3 ./test.py -f cluster/incSnapshot.py -N 3
,,y,army,./pytest.sh python3 ./test.py -f cluster/clusterBasic.py -N 5
,,y,army,./pytest.sh python3 ./test.py -f cluster/tsdbSnapshot.py -N 3 -M 3
,,y,army,./pytest.sh python3 ./test.py -f cluster/strongPassword.py
,,y,army,./pytest.sh python3 ./test.py -f query/query_basic.py -N 3
,,y,army,./pytest.sh python3 ./test.py -f query/accuracy/test_query_accuracy.py
,,y,army,./pytest.sh python3 ./test.py -f query/accuracy/test_ts5400.py
,,y,army,./pytest.sh python3 ./test.py -f query/accuracy/test_having.py
,,y,army,./pytest.sh python3 ./test.py -f insert/insert_basic.py -N 3
,,y,army,./pytest.sh python3 ./test.py -f insert/auto_create_insert.py
,,y,army,./pytest.sh python3 ./test.py -f cluster/splitVgroupByLearner.py -N 3
,,y,army,./pytest.sh python3 ./test.py -f authorith/authBasic.py -N 3
,,n,army,python3 ./test.py -f cmdline/fullopt.py
@ -91,7 +93,8 @@
,,y,army,./pytest.sh python3 ./test.py -f tools/benchmark/basic/default_json.py
,,y,army,./pytest.sh python3 ./test.py -f tools/benchmark/basic/demo.py
,,y,army,./pytest.sh python3 ./test.py -f tools/benchmark/basic/exportCsv.py
,,y,army,./pytest.sh python3 ./test.py -f tools/benchmark/basic/csv-export.py
# ,,y,army,./pytest.sh python3 ./test.py -f tools/benchmark/basic/csv-import.py
,,y,army,./pytest.sh python3 ./test.py -f tools/benchmark/basic/from-to.py
,,y,army,./pytest.sh python3 ./test.py -f tools/benchmark/basic/from-to-continue.py

6
tests/parallel_test/container_build.sh
View File

@ -81,7 +81,7 @@ docker run \
-v ${REP_REAL_PATH}/community/contrib/pcre2/:${REP_DIR}/community/contrib/pcre2 \
-v ${REP_REAL_PATH}/community/contrib/zlib/:${REP_DIR}/community/contrib/zlib \
-v ${REP_REAL_PATH}/community/contrib/zstd/:${REP_DIR}/community/contrib/zstd \
--rm --ulimit core=-1 taos_test:v1.0 sh -c "cd $REP_DIR;rm -rf debug;mkdir -p debug;cd debug;cmake .. -DBUILD_HTTP=false -DBUILD_TOOLS=true -DBUILD_TEST=true -DWEBSOCKET=true -DBUILD_TAOSX=false -DJEMALLOC_ENABLED=0 -DCMAKE_EXPORT_COMPILE_COMMANDS=1 ;make -j|| exit 1"
--rm --ulimit core=-1 taos_test:v1.0 sh -c "cd $REP_DIR; rm -rf debug; mkdir -p debug; cd debug; cmake .. -DBUILD_HTTP=false -DBUILD_TOOLS=true -DBUILD_TEST=ON -DWEBSOCKET=true -DBUILD_TAOSX=false -DJEMALLOC_ENABLED=OFF -DCMAKE_EXPORT_COMPILE_COMMANDS=1 ;make -j|| exit 1"
# -v ${REP_REAL_PATH}/community/contrib/jemalloc/:${REP_DIR}/community/contrib/jemalloc \
if [[ -d ${WORKDIR}/debugNoSan ]] ;then
@ -133,7 +133,7 @@ docker run \
-v ${REP_REAL_PATH}/community/contrib/pcre2/:${REP_DIR}/community/contrib/pcre2 \
-v ${REP_REAL_PATH}/community/contrib/zlib/:${REP_DIR}/community/contrib/zlib \
-v ${REP_REAL_PATH}/community/contrib/zstd/:${REP_DIR}/community/contrib/zstd \
--rm --ulimit core=-1 taos_test:v1.0 sh -c "cd $REP_DIR;rm -rf debug;mkdir -p debug;cd debug;cmake .. -DBUILD_HTTP=false -DBUILD_TOOLS=true -DBUILD_TEST=false -DWEBSOCKET=true -DBUILD_SANITIZER=1 -DTOOLS_SANITIZE=true -DCMAKE_BUILD_TYPE=Debug -DTOOLS_BUILD_TYPE=Debug -DBUILD_TAOSX=false -DJEMALLOC_ENABLED=0;make -j|| exit 1 "
--rm --ulimit core=-1 taos_test:v1.0 sh -c "cd $REP_DIR; rm -rf debug; mkdir -p debug; cd debug; cmake .. -DBUILD_HTTP=false -DBUILD_TOOLS=true -DBUILD_TEST=ON -DWEBSOCKET=true -DBUILD_SANITIZER=1 -DTOOLS_SANITIZE=true -DCMAKE_BUILD_TYPE=Debug -DTOOLS_BUILD_TYPE=Debug -DBUILD_TAOSX=false -DJEMALLOC_ENABLED=OFF; make -j|| exit 1 "
mv ${REP_REAL_PATH}/debug ${WORKDIR}/debugSan
@ -168,7 +168,7 @@ date
# -v ${REP_REAL_PATH}/community/contrib/xml2/:${REP_DIR}/community/contrib/xml2 \
# -v ${REP_REAL_PATH}/community/contrib/zlib/:${REP_DIR}/community/contrib/zlib \
# -v ${REP_REAL_PATH}/community/contrib/zstd/:${REP_DIR}/community/contrib/zstd \
# --rm --ulimit core=-1 taos_test:v1.0 sh -c "pip uninstall taospy -y;pip3 install taospy==2.7.2;cd $REP_DIR;rm -rf debug;mkdir -p debug;cd debug;cmake .. -DBUILD_HTTP=false -DBUILD_TOOLS=true -DBUILD_TEST=false -DWEBSOCKET=true -DCMAKE_BUILD_TYPE=Release -DBUILD_TAOSX=false -DJEMALLOC_ENABLED=0;make -j || exit 1 "
# --rm --ulimit core=-1 taos_test:v1.0 sh -c "pip uninstall taospy -y; pip3 install taospy==2.7.2; cd $REP_DIR; rm -rf debug; mkdir -p debug; cd debug; cmake .. -DBUILD_HTTP=false -DBUILD_TOOLS=true -DBUILD_TEST=ON -DWEBSOCKET=true -DCMAKE_BUILD_TYPE=Release -DBUILD_TAOSX=false -DJEMALLOC_ENABLED=OFF; make -j || exit 1 "
# mv ${REP_REAL_PATH}/debug ${WORKDIR}/debugRelease

11
tests/pytest/util/sql.py
View File

@ -705,6 +705,17 @@ class TDSql:
return True
self.print_error_frame_info(elm, expect_elm)
def checkGreater(self, elm, expect_elm):
if elm > expect_elm:
tdLog.info("sql:%s, elm:%s > expect_elm:%s" % (self.sql, elm, expect_elm))
return True
else:
caller = inspect.getframeinfo(inspect.stack()[1][0])
args = (caller.filename, caller.lineno, self.sql, elm, expect_elm)
tdLog.info("%s(%d) failed: sql:%s, elm:%s <= expect_elm:%s" % args)
self.print_error_frame_info(elm, expect_elm)
return False
def checkNotEqual(self, elm, expect_elm):
if elm != expect_elm:
tdLog.info("sql:%s, elm:%s != expect_elm:%s" % (self.sql, elm, expect_elm))

2
tests/script/tsim/show/basic.sim
View File

@ -230,7 +230,7 @@ endi
sql_error show create stable t0;
sql show variables;
if $rows != 87 then
if $rows != 88 then
return -1
endi

13
tests/script/tsim/sync/3Replica1VgElect.sim
View File

@ -91,19 +91,19 @@ endi
if $data[0][4] == leader then
if $data[0][7] == follower then
if $data[0][10] == follower then
print ---- vgroup $data[0][0] leader locate on dnode $data[0][3]
print ---- vgroup $data[0][0] leader locate on dnode $data[0][4]
endi
endi
elif $data[0][6] == leader then
if $data[0][7] == follower then
elif $data[0][7] == leader then
if $data[0][10] == follower then
print ---- vgroup $data[0][0] leader locate on dnode $data[0][5]
if $data[0][4] == follower then
print ---- vgroup $data[0][0] leader locate on dnode $data[0][7]
endi
endi
elif $data[0][10] == leader then
if $data[0][4] == follower then
if $data[0][7] == follower then
print ---- vgroup $data[0][0] leader locate on dnode $data[0][7]
print ---- vgroup $data[0][0] leader locate on dnode $data[0][10]
endi
endi
else
@ -462,6 +462,3 @@ system sh/exec.sh -n dnode1 -s stop -x SIGINT
system sh/exec.sh -n dnode2 -s stop -x SIGINT
system sh/exec.sh -n dnode3 -s stop -x SIGINT
system sh/exec.sh -n dnode4 -s stop -x SIGINT

6
tests/script/tsim/user/password.sim
View File

@ -86,7 +86,7 @@ sql create user user_p6 pass 'abcd!@123456'
sql create user user_p7 pass 'abcd!@1234567'
sql create user user_p8 pass 'abcd!@123456789'
sql create user user_p9 pass 'abcd!@1234567890'
sql_error create user user_p10 pass 'abcd!@1234567890T'
sql_error create user user_p10 pass 'abcd!@123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345T'
sql drop user user_p2
sql drop user user_p3
sql drop user user_p4
@ -273,9 +273,9 @@ sql create user u27 pass 'taosdata1.'
sql CREATE USER `_xTest1` PASS '2729c41a99b2c5222aa7dd9fc1ce3de7' SYSINFO 1 CREATEDB 0 IS_IMPORT 1 HOST '127.0.0.1';
sql_error CREATE USER `_xTest2` PASS '2729c41a99b2c5222aa7dd9fc1ce3de7' SYSINFO 1 CREATEDB 0 IS_IMPORT 0 HOST '127.0.0.1';
sql CREATE USER `_xTest3` PASS '2729c41' SYSINFO 1 CREATEDB 0 IS_IMPORT 1 HOST '127.0.0.1';
sql_error CREATE USER `_xTest3` PASS '2729c41' SYSINFO 1 CREATEDB 0 IS_IMPORT 1 HOST '127.0.0.1';
sql_error CREATE USER `_xTest4` PASS '2729c417' SYSINFO 1 CREATEDB 0 IS_IMPORT 0 HOST '127.0.0.1';
sql CREATE USER `_xTest5` PASS '2xF' SYSINFO 1 CREATEDB 0 IS_IMPORT 1 HOST '127.0.0.1';
sql_error CREATE USER `_xTest5` PASS '2xF' SYSINFO 1 CREATEDB 0 IS_IMPORT 1' HOST '127.0.0.1';
sql_error CREATE USER `_xTest6` PASS '2xF' SYSINFO 1 CREATEDB 0 IS_IMPORT 0 HOST '127.0.0.1';

2
tests/script/tsim/valgrind/checkError1.sim
View File

@ -120,7 +120,7 @@ if $rows != 3 then
endi
sql show variables;
if $rows != 87 then
if $rows != 88 then
return -1
endi

22
tests/system-test/0-others/udfTest.py
View File

@ -622,17 +622,17 @@ class TDTestCase:
tdLog.info("taosd found in %s" % buildPath)
cfgPath = buildPath + "/../sim/dnode1/cfg"
udfdPath = buildPath +'/build/bin/udfd'
udfdPath = buildPath +'/build/bin/taosudf'
for i in range(3):
tdLog.info(" loop restart udfd %d_th" % i)
tdLog.info(" loop restart taosudf %d_th" % i)
tdSql.query("select udf2(sub1.c1 ,sub1.c2), udf2(sub2.c2 ,sub2.c1) from sub1, sub2 where sub1.ts=sub2.ts and sub1.c1 is not null")
tdSql.checkData(0,0,169.661427555)
tdSql.checkData(0,1,169.661427555)
# stop udfd cmds
get_processID = "ps -ef | grep -w udfd | grep -v grep| grep -v defunct | awk '{print $2}'"
# stop taosudf cmds
get_processID = "ps -ef | grep -w taosudf | grep -v grep| grep -v defunct | awk '{print $2}'"
processID = subprocess.check_output(get_processID, shell=True).decode("utf-8")
stop_udfd = " kill -9 %s" % processID
os.system(stop_udfd)
@ -643,9 +643,9 @@ class TDTestCase:
tdSql.checkData(0,0,169.661427555)
tdSql.checkData(0,1,169.661427555)
# # start udfd cmds
# # start taosudf cmds
# start_udfd = "nohup " + udfdPath +'-c' +cfgPath +" > /dev/null 2>&1 &"
# tdLog.info("start udfd : %s " % start_udfd)
# tdLog.info("start taosudf : %s " % start_udfd)
def test_function_name(self):
tdLog.info(" create function name is not build_in functions ")
@ -680,15 +680,15 @@ class TDTestCase:
time.sleep(2)
def test_udfd_cmd(self):
tdLog.info(" test udfd -V ")
os.system("udfd -V")
tdLog.info(" test udfd -c ")
os.system("udfd -c")
tdLog.info(" test taosudf -V ")
os.system("taosudf -V")
tdLog.info(" test taosudf -c ")
os.system("taosudf -c")
letters = string.ascii_letters + string.digits + '\\'
path = ''.join(random.choice(letters) for i in range(5000))
os.system(f"udfd -c {path}")
os.system(f"taosudf -c {path}")
def test_change_udf_normal(self, func_name):
# create function with normal file

12
tests/system-test/0-others/udf_cfg2.py
View File

@ -592,17 +592,17 @@ class TDTestCase:
tdLog.info("taosd found in %s" % buildPath)
cfgPath = buildPath + "/../sim/dnode1/cfg"
udfdPath = buildPath +'/build/bin/udfd'
udfdPath = buildPath +'/build/bin/taosudf'
for i in range(3):
tdLog.info(" loop restart udfd %d_th" % i)
tdLog.info(" loop restart taosudf %d_th" % i)
tdSql.query("select udf2(sub1.c1 ,sub1.c2), udf2(sub2.c2 ,sub2.c1) from sub1, sub2 where sub1.ts=sub2.ts and sub1.c1 is not null")
tdSql.checkData(0,0,169.661427555)
tdSql.checkData(0,1,169.661427555)
# stop udfd cmds
get_processID = "ps -ef | grep -w udfd | grep -v grep| grep -v defunct | awk '{print $2}'"
# stop taosudf cmds
get_processID = "ps -ef | grep -w taosudf | grep -v grep| grep -v defunct | awk '{print $2}'"
processID = subprocess.check_output(get_processID, shell=True).decode("utf-8")
stop_udfd = " kill -9 %s" % processID
os.system(stop_udfd)
@ -613,9 +613,9 @@ class TDTestCase:
tdSql.checkData(0,0,169.661427555)
tdSql.checkData(0,1,169.661427555)
# # start udfd cmds
# # start taosudf cmds
# start_udfd = "nohup " + udfdPath +'-c' +cfgPath +" > /dev/null 2>&1 &"
# tdLog.info("start udfd : %s " % start_udfd)
# tdLog.info("start taosudf : %s " % start_udfd)
def test_function_name(self):
tdLog.info(" create function name is not build_in functions ")

10
tests/system-test/0-others/udf_cluster.py
View File

@ -301,14 +301,14 @@ class TDTestCase:
cfgPath = dnode.cfgDir
udfdPath = buildPath +'/build/bin/udfd'
udfdPath = buildPath +'/build/bin/taosudf'
for i in range(5):
tdLog.info(" loop restart udfd %d_th at dnode_index : %s" % (i ,dnode.index))
tdLog.info(" loop restart taosudf %d_th at dnode_index : %s" % (i ,dnode.index))
self.basic_udf_query(dnode)
# stop udfd cmds
get_processID = "ps -ef | grep -w udfd | grep %s | grep 'root' | grep -v grep| grep -v defunct | awk '{print $2}'"%cfgPath
# stop taosudf cmds
get_processID = "ps -ef | grep -w taosudf | grep %s | grep 'root' | grep -v grep| grep -v defunct | awk '{print $2}'"%cfgPath
processID = subprocess.check_output(get_processID, shell=True).decode("utf-8")
stop_udfd = " kill -9 %s" % processID
os.system(stop_udfd)
@ -317,7 +317,7 @@ class TDTestCase:
def test_restart_udfd_All_dnodes(self):
for dnode in self.TDDnodes.dnodes:
tdLog.info(" start restart udfd for dnode_index :%s" %dnode.index )
tdLog.info(" start restart taosudf for dnode_index :%s" %dnode.index )
self.restart_udfd(dnode)

14
tests/system-test/0-others/udf_create.py
View File

@ -570,7 +570,7 @@ class TDTestCase:
time.sleep(1)
hwnd = win32gui.FindWindow(None, "Microsoft Visual C++ Runtime Library")
if hwnd:
os.system("TASKKILL /F /IM udfd.exe")
os.system("TASKKILL /F /IM taosudf.exe")
def unexpected_create(self):
if (platform.system().lower() == 'windows' and tdDnodes.dnodes[0].remoteIP == ""):
@ -627,17 +627,17 @@ class TDTestCase:
tdLog.info("taosd found in %s" % buildPath)
cfgPath = buildPath + "/../sim/dnode1/cfg"
udfdPath = buildPath +'/build/bin/udfd'
udfdPath = buildPath +'/build/bin/taosudf'
for i in range(3):
tdLog.info(" loop restart udfd %d_th" % i)
tdLog.info(" loop restart taosudf %d_th" % i)
tdSql.query("select udf2(sub1.c1 ,sub1.c2), udf2(sub2.c2 ,sub2.c1) from sub1, sub2 where sub1.ts=sub2.ts and sub1.c1 is not null")
tdSql.checkData(0,0,169.661427555)
tdSql.checkData(0,1,169.661427555)
# stop udfd cmds
get_processID = "ps -ef | grep -w udfd | grep -v grep| grep -v defunct | awk '{print $2}'"
# stop taosudf cmds
get_processID = "ps -ef | grep -w taosudf | grep -v grep| grep -v defunct | awk '{print $2}'"
processID = subprocess.check_output(get_processID, shell=True).decode("utf-8")
stop_udfd = " kill -9 %s" % processID
os.system(stop_udfd)
@ -648,9 +648,9 @@ class TDTestCase:
tdSql.checkData(0,0,169.661427555)
tdSql.checkData(0,1,169.661427555)
# # start udfd cmds
# # start taosudf cmds
# start_udfd = "nohup " + udfdPath +'-c' +cfgPath +" > /dev/null 2>&1 &"
# tdLog.info("start udfd : %s " % start_udfd)
# tdLog.info("start taosudf : %s " % start_udfd)
def test_function_name(self):
tdLog.info(" create function name is not build_in functions ")

12
tests/system-test/0-others/udf_restart_taosd.py
View File

@ -589,17 +589,17 @@ class TDTestCase:
tdLog.info("taosd found in %s" % buildPath)
cfgPath = buildPath + "/../sim/dnode1/cfg"
udfdPath = buildPath +'/build/bin/udfd'
udfdPath = buildPath +'/build/bin/taosudf'
for i in range(3):
tdLog.info(" loop restart udfd %d_th" % i)
tdLog.info(" loop restart taosudf %d_th" % i)
tdSql.query("select udf2(sub1.c1 ,sub1.c2), udf2(sub2.c2 ,sub2.c1) from sub1, sub2 where sub1.ts=sub2.ts and sub1.c1 is not null")
tdSql.checkData(0,0,169.661427555)
tdSql.checkData(0,1,169.661427555)
# stop udfd cmds
get_processID = "ps -ef | grep -w udfd | grep -v grep| grep -v defunct | awk '{print $2}'"
# stop taosudf cmds
get_processID = "ps -ef | grep -w taosudf | grep -v grep| grep -v defunct | awk '{print $2}'"
processID = subprocess.check_output(get_processID, shell=True).decode("utf-8")
stop_udfd = " kill -9 %s" % processID
os.system(stop_udfd)
@ -610,9 +610,9 @@ class TDTestCase:
tdSql.checkData(0,0,169.661427555)
tdSql.checkData(0,1,169.661427555)
# # start udfd cmds
# # start taosudf cmds
# start_udfd = "nohup " + udfdPath +'-c' +cfgPath +" > /dev/null 2>&1 &"
# tdLog.info("start udfd : %s " % start_udfd)
# tdLog.info("start taosudf : %s " % start_udfd)
def test_function_name(self):
tdLog.info(" create function name is not build_in functions ")

2
tests/system-test/1-insert/boundary.py
View File

@ -33,7 +33,7 @@ class TDTestCase:
self.colname_length_boundary = self.boundary.COL_KEY_MAX_LENGTH
self.tagname_length_boundary = self.boundary.TAG_KEY_MAX_LENGTH
self.username_length_boundary = 23
self.password_length_boundary = 14
self.password_length_boundary = 253
def dbname_length_check(self):
dbname_length = randint(1,self.dbname_length_boundary-1)
for dbname in [tdCom.get_long_name(self.dbname_length_boundary),tdCom.get_long_name(dbname_length)]:

63
tests/system-test/2-query/db.py
View File

@ -46,8 +46,9 @@ class TDTestCase:
tdSql.checkRows(2)
def case2(self):
tdSql.query("show variables")
tdSql.checkRows(87)
tdSql.checkGreater(tdSql.getRows(), 80)
for i in range(self.replicaVar):
tdSql.query("show dnode %d variables like 'debugFlag'" % (i + 1))
@ -86,6 +87,58 @@ class TDTestCase:
tdSql.checkData(0, 1, 's3UploadDelaySec')
tdSql.checkData(0, 2, 60)
def show_local_variables_like(self):
tdSql.query("show local variables")
tdSql.checkGreater(tdSql.getRows(), 80)
tdSql.query("show local variables like 'debugFlag'")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 'debugFlag')
tdSql.checkData(0, 1, 0)
tdSql.query("show local variables like '%debugFlag'")
tdSql.checkRows(9)
tdSql.query("show local variables like '____debugFlag'")
tdSql.checkRows(0)
tdSql.query("show local variables like 's3MigrateEnab%'")
tdSql.checkRows(0)
tdSql.query("show local variables like 'mini%'")
tdSql.checkRows(3)
tdSql.checkData(0, 0, 'minimalTmpDirGB')
tdSql.query("show local variables like '%info'")
tdSql.checkRows(2)
def show_cluster_variables_like(self):
zones = ["", "cluster"]
for zone in zones:
tdLog.info(f"show {zone} variables")
tdSql.query(f"show {zone} variables")
tdSql.checkGreater(tdSql.getRows(), 80)
tdLog.info(f"show {zone} variables like 'debugFlag'")
#tdSql.query(f"show {zone} variables like 'debugFlag'")
#tdSql.checkRows(0)
tdSql.query(f"show {zone} variables like 's3%'")
tdSql.checkRows(6)
tdSql.query(f"show {zone} variables like 'Max%'")
tdSql.checkRows(3)
tdSql.query(f"show {zone} variables like 'ttl%'")
tdSql.checkRows(5)
tdSql.query(f"show {zone} variables like 'ttl34343434%'")
tdSql.checkRows(0)
tdSql.query(f"show {zone} variables like 'jdlkfdjdfkdfnldlfdnfkdkfdmfdlfmnnnnnjkjk'")
tdSql.checkRows(0)
def threadTest(self, threadID):
print(f"Thread {threadID} starting...")
tdsqln = tdCom.newTdSql()
@ -128,6 +181,14 @@ class TDTestCase:
self.case3()
tdLog.printNoPrefix("==========end case3 run ...............")
tdLog.printNoPrefix("==========start show_local_variables_like run ...............")
self.show_local_variables_like()
tdLog.printNoPrefix("==========end show_local_variables_like run ...............")
tdLog.printNoPrefix("==========start show_cluster_variables_like run ...............")
self.show_cluster_variables_like()
tdLog.printNoPrefix("==========end show_cluster_variables_like run ...............")
def stop(self):
tdSql.close()
tdLog.success(f"{__file__} successfully executed")

2
tests/taosc_test/CMakeLists.txt
View File

@ -16,7 +16,7 @@ aux_source_directory(src OS_SRC)
# taoscTest
add_executable(taoscTest "taoscTest.cpp")
target_link_libraries(taoscTest ${TAOS_LIB} os gtest_main)
target_link_libraries(taoscTest PUBLIC ${TAOS_LIB} os util common gtest_main)
target_include_directories(
taoscTest
PUBLIC "${TD_SOURCE_DIR}/include/os"

Some files were not shown because too many files have changed in this diff Show More