xxq250
/
homework-jianmu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Merge pull request #29469 from taosdata/docs/readme 

update README structure for support linux/win/mac platforms by charles
This commit is contained in:
WANG Xu 2025-01-17 14:11:46 +08:00 committed by GitHub
parent afb3a5b90c 37948a93b0
commit c37a96a916
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
10 changed files with 562 additions and 233 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

394
README.md
View File

@ -26,24 +26,33 @@ English | [简体中文](README-CN.md) | [TDengine Cloud](https://cloud.tdengine
# Table of Contents # Table of Contents
1. [What is TDengine?](#1-what-is-tdengine) 1. [Introduction](#1-introduction)
2. [Documentation](#2-documentation) 1. [Documentation](#2-documentation)
3. [Building](#3-building) 1. [Prerequisites](#3-prerequisites)
1. [Install build tools](#31-install-build-tools) - [3.1 Prerequisites On Linux](#31-on-linux)
1. [Get the source codes](#32-get-the-source-codes) - [3.2 Prerequisites On macOS](#32-on-macos)
1. [Special Note](#33-special-note) - [3.3 Prerequisites On Windows](#33-on-windows)
1. [Build TDengine](#34-build-tdengine) - [3.4 Clone the repo](#34-clone-the-repo)
4. [Installing](#4-installing) 1. [Building](#4-building)
1. [On Linux platform](#41-on-linux-platform) - [4.1 Build on Linux](#41-build-on-linux)
1. [On Windows platform](#42-on-windows-platform) - [4.2 Build on macOS](#42-build-on-macos)
1. [On macOS platform](#43-on-macos-platform) - [4.3 Build On Windows](#43-build-on-windows)
1. [Quick Run](#44-quick-run) 1. [Packaging](#5-packaging)
5. [Try TDengine](#5-try-tdengine) 1. [Installation](#6-installation)
6. [Developing with TDengine](#6-developing-with-tdengine) - [6.1 Install on Linux](#61-install-on-linux)
7. [Contribute to TDengine](#7-contribute-to-tdengine) - [6.2 Install on macOS](#62-install-on-macos)
8. [Join the TDengine Community](#8-join-the-tdengine-community) - [6.3 Install on Windows](#63-install-on-windows)
1. [Running](#7-running)
- [7.1 Run TDengine on Linux](#71-run-tdengine-on-linux)
- [7.2 Run TDengine on macOS](#72-run-tdengine-on-macos)
- [7.3 Run TDengine on Windows](#73-run-tdengine-on-windows)
1. [Testing](#8-testing)
1. [Releasing](#9-releasing)
1. [Workflow](#10-workflow)
1. [Coverage](#11-coverage)
1. [Contributing](#12-contributing)
# 1. What is TDengine？ # 1. Introduction
TDengine is an open source, high-performance, cloud native [time-series database](https://tdengine.com/tsdb/) optimized for Internet of Things (IoT), Connected Cars, and Industrial IoT. It enables efficient, real-time data ingestion, processing, and monitoring of TB and even PB scale data per day, generated by billions of sensors and data collectors. TDengine differentiates itself from other time-series databases with the following advantages: TDengine is an open source, high-performance, cloud native [time-series database](https://tdengine.com/tsdb/) optimized for Internet of Things (IoT), Connected Cars, and Industrial IoT. It enables efficient, real-time data ingestion, processing, and monitoring of TB and even PB scale data per day, generated by billions of sensors and data collectors. TDengine differentiates itself from other time-series databases with the following advantages:
@ -65,132 +74,91 @@ For a full list of TDengine competitive advantages, please [check here](https://
For user manual, system design and architecture, please refer to [TDengine Documentation](https://docs.tdengine.com) ([TDengine 文档](https://docs.taosdata.com)) For user manual, system design and architecture, please refer to [TDengine Documentation](https://docs.tdengine.com) ([TDengine 文档](https://docs.taosdata.com))
# 3. Building # 3. Prerequisites
At the moment, TDengine server supports running on Linux/Windows/macOS systems. Any application can also choose the RESTful interface provided by taosAdapter to connect the taosd service . TDengine supports X64/ARM64 CPU, and it will support MIPS64, Alpha64, ARM32, RISC-V and other CPU architectures in the future. Right now we don't support build with cross-compiling environment. ## 3.1 On Linux
You can choose to install through source code, [container](https://docs.tdengine.com/get-started/docker/), [installation package](https://docs.tdengine.com/get-started/package/) or [Kubernetes](https://docs.tdengine.com/deployment/k8s/). This quick guide only applies to installing from source. <details>
TDengine provide a few useful tools such as taosBenchmark (was named taosdemo) and taosdump. They were part of TDengine. By default, TDengine compiling does not include taosTools. You can use `cmake .. -DBUILD_TOOLS=true` to make them be compiled with TDengine. <summary>Install required tools on Linux</summary>
To build TDengine, use [CMake](https://cmake.org/) 3.13.0 or higher versions in the project directory. ### For Ubuntu 18.04、20.04、22.04
## 3.1 Install build tools
### Ubuntu 18.04 and above or Debian
```bash ```bash
sudo apt-get install -y gcc cmake build-essential git libssl-dev libgflags2.2 libgflags-dev sudo apt-get udpate
sudo apt-get install -y gcc cmake build-essential git libjansson-dev \
libsnappy-dev liblzma-dev zlib1g-dev pkg-config
``` ```
#### Install build dependencies for taosTools ### For CentOS 8
To build the [taosTools](https://github.com/taosdata/taos-tools) on Ubuntu/Debian, the following packages need to be installed.
```bash ```bash
sudo apt install build-essential libjansson-dev libsnappy-dev liblzma-dev libz-dev zlib1g pkg-config
```
### CentOS 7.9
```bash
sudo yum install epel-release
sudo yum update sudo yum update
sudo yum install -y gcc gcc-c++ make cmake3 gflags git openssl-devel yum install -y epel-release gcc gcc-c++ make cmake git perl dnf-plugins-core
sudo ln -sf /usr/bin/cmake3 /usr/bin/cmake yum config-manager --set-enabled powertools
yum install -y zlib-static xz-devel snappy-devel jansson-devel pkgconfig libatomic-static libstdc++-static
``` ```
### CentOS 8/Fedora/Rocky Linux </details>
## 3.2 On macOS
<details>
<summary>Install required tools on macOS</summary>
Please intall the dependencies with [brew](https://brew.sh/).
```bash ```bash
sudo dnf install -y gcc gcc-c++ make cmake epel-release gflags git openssl-devel
```
#### Install build dependencies for taosTools on CentOS
#### CentOS 7.9
```
sudo yum install -y zlib-devel zlib-static xz-devel snappy-devel jansson jansson-devel pkgconfig libatomic libatomic-static libstdc++-static openssl-devel
```
#### CentOS 8/Fedora/Rocky Linux
```
sudo yum install -y epel-release
sudo yum install -y dnf-plugins-core
sudo yum config-manager --set-enabled powertools
sudo yum install -y zlib-devel zlib-static xz-devel snappy-devel jansson jansson-devel pkgconfig libatomic libatomic-static libstdc++-static openssl-devel
```
Note: Since snappy lacks pkg-config support (refer to [link](https://github.com/google/snappy/pull/86)), it leads a cmake prompt libsnappy not found. But snappy still works well.
If the PowerTools installation fails, you can try to use:
```
sudo yum config-manager --set-enabled powertools
```
#### For CentOS + devtoolset
Besides above dependencies, please run following commands:
```
sudo yum install centos-release-scl
sudo yum install devtoolset-9 devtoolset-9-libatomic-devel
scl enable devtoolset-9 -- bash
```
### macOS
```
brew install argp-standalone gflags pkgconfig brew install argp-standalone gflags pkgconfig
``` ```
### Setup golang environment </details>
TDengine includes a few components like taosAdapter developed by Go language. Please refer to golang.org official documentation for golang environment setup. ## 3.3 On Windows
Please use version 1.20+. For the user in China, we recommend using a proxy to accelerate package downloading. <details>
``` <summary>Install required tools on Windows</summary>
go env -w GO111MODULE=on
go env -w GOPROXY=https://goproxy.cn,direct
```
The default will not build taosAdapter, but you can use the following command to build taosAdapter as the service for RESTful interface. Work in Progress.
``` </details>
cmake .. -DBUILD_HTTP=false
```
### Setup rust environment ## 3.4 Clone the repo
TDengine includes a few components developed by Rust language. Please refer to rust-lang.org official documentation for rust environment setup. <details>
## 3.2 Get the source codes <summary>Clone the repo</summary>
First of all, you may clone the source codes from github: Clone the repository to the target machine:
```bash ```bash
git clone https://github.com/taosdata/TDengine.git git clone https://github.com/taosdata/TDengine.git
cd TDengine cd TDengine
``` ```
You can modify the file ~/.gitconfig to use ssh protocol instead of https for better download speed. You will need to upload ssh public key to GitHub first. Please refer to GitHub official documentation for detail.
``` > **NOTE:**
[url "git@github.com:"] > TDengine Connectors can be found in following repositories: [JDBC Connector](https://github.com/taosdata/taos-connector-jdbc), [Go Connector](https://github.com/taosdata/driver-go), [Python Connector](https://github.com/taosdata/taos-connector-python), [Node.js Connector](https://github.com/taosdata/taos-connector-node), [C# Connector](https://github.com/taosdata/taos-connector-dotnet), [Rust Connector](https://github.com/taosdata/taos-connector-rust).
insteadOf = https://github.com/
```
## 3.3 Special Note </details>
[JDBC Connector](https://github.com/taosdata/taos-connector-jdbc)， [Go Connector](https://github.com/taosdata/driver-go)，[Python Connector](https://github.com/taosdata/taos-connector-python)，[Node.js Connector](https://github.com/taosdata/taos-connector-node)，[C# Connector](https://github.com/taosdata/taos-connector-dotnet) ，[Rust Connector](https://github.com/taosdata/taos-connector-rust) and [Grafana plugin](https://github.com/taosdata/grafanaplugin) has been moved to standalone repository. # 4. Building
## 3.4 Build TDengine At the moment, TDengine server supports running on Linux/Windows/MacOS systems. Any application can also choose the RESTful interface provided by taosAdapter to connect the taosd service. TDengine supports X64/ARM64 CPU, and it will support MIPS64, Alpha64, ARM32, RISC-V and other CPU architectures in the future. Right now we don't support build with cross-compiling environment.
### On Linux platform You can choose to install through source code, [container](https://docs.tdengine.com/get-started/deploy-in-docker/), [installation package](https://docs.tdengine.com/get-started/deploy-from-package/) or [Kubernetes](https://docs.tdengine.com/operations-and-maintenance/deploy-your-cluster/#kubernetes-deployment). This quick guide only applies to install from source.
TDengine provide a few useful tools such as taosBenchmark (was named taosdemo) and taosdump. They were part of TDengine. By default, TDengine compiling does not include taosTools. You can use `cmake .. -DBUILD_TOOLS=true` to make them be compiled with TDengine.
To build TDengine, use [CMake](https://cmake.org/) 3.13.0 or higher versions in the project directory.
## 4.1 Build on Linux
<details>
<summary>Detailed steps to build on Linux</summary>
You can run the bash script `build.sh` to build both TDengine and taosTools including taosBenchmark and taosdump as below: You can run the bash script `build.sh` to build both TDengine and taosTools including taosBenchmark and taosdump as below:
@ -201,29 +169,46 @@ You can run the bash script `build.sh` to build both TDengine and taosTools incl
It equals to execute following commands: It equals to execute following commands:
```bash ```bash
mkdir debug mkdir debug && cd debug
cd debug
cmake .. -DBUILD_TOOLS=true -DBUILD_CONTRIB=true cmake .. -DBUILD_TOOLS=true -DBUILD_CONTRIB=true
make make
``` ```
You can use Jemalloc as memory allocator instead of glibc: You can use Jemalloc as memory allocator instead of glibc:
``` ```bash
apt install autoconf
cmake .. -DJEMALLOC_ENABLED=true cmake .. -DJEMALLOC_ENABLED=true
``` ```
TDengine build script can detect the host machine's architecture on X86-64, X86, arm64 platform. TDengine build script can auto-detect the host machine's architecture on x86, x86-64, arm64 platform.
You can also specify CPUTYPE option like aarch64 too if the detection result is not correct: You can also specify architecture manually by CPUTYPE option:
aarch64:
```bash ```bash
cmake .. -DCPUTYPE=aarch64 && cmake --build . cmake .. -DCPUTYPE=aarch64 && cmake --build .
``` ```
### On Windows platform </details>
## 4.2 Build on macOS
<details>
<summary>Detailed steps to build on macOS</summary>
Please install XCode command line tools and cmake. Verified with XCode 11.4+ on Catalina and Big Sur.
```shell
mkdir debug && cd debug
cmake .. && cmake --build .
```
</details>
## 4.3 Build on Windows
<details>
<summary>Detailed steps to build on Windows</summary>
If you use the Visual Studio 2013, please open a command window by executing "cmd.exe". If you use the Visual Studio 2013, please open a command window by executing "cmd.exe".
Please specify "amd64" for 64 bits Windows or specify "x86" for 32 bits Windows when you execute vcvarsall.bat. Please specify "amd64" for 64 bits Windows or specify "x86" for 32 bits Windows when you execute vcvarsall.bat.
@ -254,31 +239,67 @@ mkdir debug && cd debug
cmake .. -G "NMake Makefiles" cmake .. -G "NMake Makefiles"
nmake nmake
``` ```
</details>
### On macOS platform # 5. Packaging
Please install XCode command line tools and cmake. Verified with XCode 11.4+ on Catalina and Big Sur. The TDengine community installer can NOT be created by this repository only, due to some component dependencies. We are still working on this improvement.
```shell # 6. Installation
mkdir debug && cd debug
cmake .. && cmake --build .
```
# 4. Installing ## 6.1 Install on Linux
## 4.1 On Linux platform <details>
After building successfully, TDengine can be installed by <summary>Detailed steps to install on Linux</summary>
After building successfully, TDengine can be installed by:
```bash ```bash
sudo make install sudo make install
``` ```
Users can find more information about directories installed on the system in the [directory and files](https://docs.tdengine.com/reference/directory/) section. Installing from source code will also configure service management for TDengine. Users can also choose to [install from packages](https://docs.tdengine.com/get-started/deploy-from-package/) for it.
Installing from source code will also configure service management for TDengine.Users can also choose to [install from packages](https://docs.tdengine.com/get-started/package/) for it. </details>
To start the service after installation, in a terminal, use: ## 6.2 Install on macOS
<details>
<summary>Detailed steps to install on macOS</summary>
After building successfully, TDengine can be installed by:
```bash
sudo make install
```
</details>
## 6.3 Install on Windows
<details>
<summary>Detailed steps to install on windows</summary>
After building successfully, TDengine can be installed by:
```cmd
nmake install
```
</details>
# 7. Running
## 7.1 Run TDengine on Linux
<details>
<summary>Detailed steps to run on Linux</summary>
To start the service after installation on linux, in a terminal, use:
```bash ```bash
sudo systemctl start taosd sudo systemctl start taosd
@ -292,27 +313,29 @@ taos
If TDengine CLI connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown. If TDengine CLI connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown.
## 4.2 On Windows platform If you don't want to run TDengine as a service, you can run it in current shell. For example, to quickly start a TDengine server after building, run the command below in terminal: (We take Linux as an example, command on Windows will be `taosd.exe`)
After building successfully, TDengine can be installed by:
```cmd
nmake install
```
## 4.3 On macOS platform
After building successfully, TDengine can be installed by:
```bash ```bash
sudo make install ./build/bin/taosd -c test/cfg
``` ```
Users can find more information about directories installed on the system in the [directory and files](https://docs.tdengine.com/reference/directory/) section. In another terminal, use the TDengine CLI to connect the server:
Installing from source code will also configure service management for TDengine.Users can also choose to [install from packages](https://docs.tdengine.com/get-started/package/) for it. ```bash
./build/bin/taos -c test/cfg
```
To start the service after installation, double-click the /applications/TDengine to start the program, or in a terminal, use: Option `-c test/cfg` specifies the system configuration file directory.
</details>
## 7.2 Run TDengine on macOS
<details>
<summary>Detailed steps to run on macOS</summary>
To start the service after installation on macOS, double-click the /applications/TDengine to start the program, or in a terminal, use:
```bash ```bash
sudo launchctl start com.tdengine.taosd sudo launchctl start com.tdengine.taosd
@ -326,64 +349,63 @@ taos
If TDengine CLI connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown. If TDengine CLI connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown.
## 4.4 Quick Run </details>
If you don't want to run TDengine as a service, you can run it in current shell. For example, to quickly start a TDengine server after building, run the command below in terminal: (We take Linux as an example, command on Windows will be `taosd.exe`)
```bash ## 7.3 Run TDengine on Windows
./build/bin/taosd -c test/cfg
<details>
<summary>Detailed steps to run on windows</summary>
You can start TDengine server on Windows platform with below commands:
```cmd
.\build\bin\taosd.exe -c test\cfg
``` ```
In another terminal, use the TDengine CLI to connect the server: In another terminal, use the TDengine CLI to connect the server:
```bash ```cmd
./build/bin/taos -c test/cfg .\build\bin\taos.exe -c test\cfg
``` ```
option "-c test/cfg" specifies the system configuration file directory. option "-c test/cfg" specifies the system configuration file directory.
# 5. Try TDengine </details>
It is easy to run SQL commands from TDengine CLI which is the same as other SQL databases. # 8. Testing
```sql For how to run different types of tests on TDengine, please see [Testing TDengine](./tests/README.md).
CREATE DATABASE demo;
USE demo; # 9. Releasing
CREATE TABLE t (ts TIMESTAMP, speed INT);
INSERT INTO t VALUES('2019-07-15 00:00:00', 10); For the complete list of TDengine Releases, please see [Releases](https://github.com/taosdata/TDengine/releases).
INSERT INTO t VALUES('2019-07-15 01:00:00', 20);
SELECT * FROM t; # 10. Workflow
ts | speed |
=================================== TDengine build check workflow can be found in this [Github Action](https://github.com/taosdata/TDengine/actions/workflows/taosd-ci-build.yml). More workflows will be available soon.
19-07-15 00:00:00.000| 10|
19-07-15 01:00:00.000| 20| # 11. Coverage
Query OK, 2 row(s) in set (0.001700s)
Latest TDengine test coverage report can be found on [coveralls.io](https://coveralls.io/github/taosdata/TDengine)
<details>
<summary>How to run the coverage report locally?</summary>
To create the test coverage report (in HTML format) locally, please run following commands:
```bash
cd tests
bash setup-lcov.sh -v 1.16 && ./run_local_coverage.sh -b main -c task
# on main branch and run cases in longtimeruning_cases.task
# for more infomation about options please refer to ./run_local_coverage.sh -h
``` ```
> **NOTE:**
> Please note that the -b and -i options will recompile TDengine with the -DCOVER=true option, which may take a amount of time.
# 6. Developing with TDengine </details>
## Official Connectors # 12. Contributing
TDengine provides abundant developing tools for users to develop on TDengine. Follow the links below to find your desired connectors and relevant documentation. Please follow the [contribution guidelines](CONTRIBUTING.md) to contribute to TDengine.
- [Java](https://docs.tdengine.com/reference/connectors/java/)
- [C/C++](https://docs.tdengine.com/reference/connectors/cpp/)
- [Python](https://docs.tdengine.com/reference/connectors/python/)
- [Go](https://docs.tdengine.com/reference/connectors/go/)
- [Node.js](https://docs.tdengine.com/reference/connectors/node/)
- [Rust](https://docs.tdengine.com/reference/connectors/rust/)
- [C#](https://docs.tdengine.com/reference/connectors/csharp/)
- [RESTful API](https://docs.tdengine.com/reference/connectors/rest-api/)
# 7. Contribute to TDengine
Please follow the [contribution guidelines](CONTRIBUTING.md) to contribute to the project.
# 8. Join the TDengine Community
For more information about TDengine, you can follow us on social media and join our Discord server:
- [Discord](https://discord.com/invite/VZdSuUg4pS)
- [Twitter](https://twitter.com/TDengineDB)
- [LinkedIn](https://www.linkedin.com/company/tdengine/)
- [YouTube](https://www.youtube.com/@tdengine)

14
packaging/smokeTest/test_smoking_selfhost.sh
View File

@ -6,12 +6,6 @@ SUCCESS_FILE="success.txt"
FAILED_FILE="failed.txt" FAILED_FILE="failed.txt"
REPORT_FILE="report.txt" REPORT_FILE="report.txt"
# Initialize/clear result files
> "$SUCCESS_FILE"
> "$FAILED_FILE"
> "$LOG_FILE"
> "$REPORT_FILE"
# Switch to the target directory # Switch to the target directory
TARGET_DIR="../../tests/system-test/" TARGET_DIR="../../tests/system-test/"
@ -24,6 +18,12 @@ else
exit 1 exit 1
fi fi
# Initialize/clear result files
> "$SUCCESS_FILE"
> "$FAILED_FILE"
> "$LOG_FILE"
> "$REPORT_FILE"
# Define the Python commands to execute # Define the Python commands to execute
commands=( commands=(
"python3 ./test.py -f 2-query/join.py" "python3 ./test.py -f 2-query/join.py"
@ -102,4 +102,4 @@ fi
echo "Detailed logs can be found in: $(realpath "$LOG_FILE")" echo "Detailed logs can be found in: $(realpath "$LOG_FILE")"
echo "Successful commands can be found in: $(realpath "$SUCCESS_FILE")" echo "Successful commands can be found in: $(realpath "$SUCCESS_FILE")"
echo "Failed commands can be found in: $(realpath "$FAILED_FILE")" echo "Failed commands can be found in: $(realpath "$FAILED_FILE")"
echo "Test report can be found in: $(realpath "$REPORT_FILE")" echo "Test report can be found in: $(realpath "$REPORT_FILE")"

233
tests/README.md Normal file
View File

@ -0,0 +1,233 @@
# Table of Contents
1. [Introduction](#1-introduction)
1. [Prerequisites](#2-prerequisites)
1. [Testing Guide](#3-testing-guide)
- [3.1 Unit Test](#31-unit-test)
- [3.2 System Test](#32-system-test)
- [3.3 Legacy Test](#33-legacy-test)
- [3.4 Smoke Test](#34-smoke-test)
- [3.5 Chaos Test](#35-chaos-test)
- [3.6 CI Test](#36-ci-test)
# 1. Introduction
This manual is intended to give developers a comprehensive guidance to test TDengine efficiently. It is divided into three main sections: introduction, prerequisites and testing guide.
> [!NOTE]
> - The commands and scripts below are verified on Linux (Ubuntu 18.04/20.04/22.04).
> - The commands and steps described below are to run the tests on a single host.
# 2. Prerequisites
- Install Python3
```bash
apt install python3
apt install python3-pip
```
- Install Python dependencies
```bash
pip3 install pandas psutil fabric2 requests faker simplejson \
toml pexpect tzlocal distro decorator loguru hyperloglog
```
- Install Python connector for TDengine
```bash
pip3 install taospy taos-ws-py
```
- Building
Before testing, please make sure the building operation with option `-DBUILD_TOOLS=true -DBUILD_TEST=true -DBUILD_CONTRIB=true` has been done, otherwise execute commands below:
```bash
cd debug
cmake .. -DBUILD_TOOLS=true -DBUILD_TEST=true -DBUILD_CONTRIB=true
make && make install
```
# 3. Testing Guide
In `tests` directory, there are different types of tests for TDengine. Below is a brief introduction about how to run them and how to add new cases.
### 3.1 Unit Test
Unit tests are the smallest testable units, which are used to test functions, methods or classes in TDengine code.
### 3.1.1 How to run single test case?
```bash
cd debug/build/bin
./osTimeTests
```
### 3.1.2 How to run all unit test cases?
```bash
cd tests/unit-test/
bash test.sh -e 0
```
### 3.1.3 How to add new cases?
<details>
<summary>Detailed steps to add new unit test case</summary>
The Google test framwork is used for unit testing to specific function module, please refer to steps below to add a new test case:
##### a. Create test case file and develop the test scripts
In the test directory corresponding to the target function module, create test files in CPP format and write corresponding test cases.
##### b. Update build configuration
Modify the CMakeLists.txt file in this directory to ensure that the new test files are properly included in the compilation process. See the `source/os/test/CMakeLists.txt` file for configuration examples.
##### c. Compile test code
In the root directory of the project, create a compilation directory (e.g., debug), switch to the directory and run CMake commands (e.g., `cmake .. -DBUILD_TEST=1`) to generate a compilation file,
and then run a compilation command (e.g. make) to complete the compilation of the test code.
##### d. Execute the test program
Find the executable file in the compiled directory(e.g. `TDengine/debug/build/bin/`) and run it.
##### e. Integrate into CI tests
Use the add_test command to add new compiled test cases into CI test collection, ensure that the new added test cases can be run for every build.
</details>
## 3.2 System Test
System tests are end-to-end test cases written in Python from a system point of view. Some of them are designed to test features only in enterprise ediiton, so when running on community edition, they may fail. We'll fix this issue by separating the cases into different gruops in the future.
### 3.2.1 How to run a single test case?
Take test file `system-test/2-query/avg.py` for example:
```bash
cd tests/system-test
python3 ./test.py -f 2-query/avg.py
```
### 3.2.2 How to run all system test cases?
```bash
cd tests
./run_all_ci_cases.sh -t python # all python cases
```
### 3.2.3 How to add new case?
<details>
<summary>Detailed steps to add new system test case</summary>
The Python test framework is developed by TDengine team, and test.py is the test case execution and monitoring of the entry program, Use `python3 ./test.py -h` to view more features.
Please refer to steps below for how to add a new test case:
##### a. Create a test case file and develop the test cases
Create a file in `tests/system-test` containing each functional directory and refer to the use case template `tests/system-test/0-others/test_case_template.py` to add a new test case.
##### b. Execute the test case
Ensure the test case execution is successful.
``` bash
cd tests/system-test && python3 ./test.py -f 0-others/test_case_template.py
```
##### c. Integrate into CI tests
Edit `tests/parallel_test/cases.task` and add the testcase path and executions in the specified format. The third column indicates whether to use Address Sanitizer mode for testing.
```bash
#caseID,rerunTimes,Run with Sanitizer,casePath,caseCommand
,,n,system-test, python3 ./test.py -f 0-others/test_case_template.py
```
</details>
## 3.3 Legacy Test
In the early stage of TDengine development, test cases are run by an internal test framework called TSIM, which is developed in C++.
### 3.3.1 How to run single test case?
To run the legacy test cases, please execute the following commands:
```bash
cd tests/script
./test.sh -f tsim/db/basic1.sim
```
### 3.3.2 How to run all legacy test cases?
```bash
cd tests
./run_all_ci_cases.sh -t legacy # all legacy cases
```
### 3.3.3 How to add new cases?
> [!NOTE]
> TSIM test framwork is deprecated by system test now, it is encouraged to add new test cases in system test, please refer to [System Test](#32-system-test) for details.
## 3.4 Smoke Test
Smoke test is a group of test cases selected from system test, which is also known as sanity test to ensure the critical functionalities of TDengine.
### 3.4.1 How to run test?
```bash
cd /root/TDengine/packaging/smokeTest
./test_smoking_selfhost.sh
```
### 3.4.2 How to add new cases?
New cases can be added by updating the value of `commands` variable in `test_smoking_selfhost.sh`.
## 3.5 Chaos Test
A simple tool to execute various functions of the system in a randomized way, hoping to expose potential problems without a pre-defined test scenario.
### 3.5.1 How to run test?
```bash
cd tests/pytest
python3 auto_crash_gen.py
```
### 3.5.2 How to add new cases?
1. Add a function, such as `TaskCreateNewFunction` in `pytest/crash_gen/crash_gen_main.py`.
2. Integrate `TaskCreateNewFunction` into the `balance_pickTaskType` function in `crash_gen_main.py`.
## 3.6 CI Test
CI testing (Continuous Integration testing), is an important practice in software development that aims to automate frequent integration of code into a shared codebase, build and test it to ensure code quality and stability.
TDengine CI testing will run all the test cases from the following three types of tests: unit test, system test and legacy test.
### 3.6.1 How to run all CI test cases?
If this is the first time to run all the CI test cases, it is recommended to add the test branch, please run it with following commands:
```bash
cd tests
./run_all_ci_cases.sh -b main # on main branch
```
### 3.6.2 How to add new cases?
Please refer to the [Unit Test](#31-unit-test)、[System Test](#32-system-test) and [Legacy Test](#33-legacy-test) sections for detailed steps to add new test cases, when new cases are added in aboved tests, they will be run automatically by CI test.

15
tests/pytest/auto_crash_gen.py
View File

@ -244,7 +244,7 @@ def start_taosd():
else: else:
pass pass
start_cmd = 'cd %s && python3 test.py >>/dev/null '%(start_path) start_cmd = 'cd %s && python3 test.py -G >>/dev/null '%(start_path)
os.system(start_cmd) os.system(start_cmd)
def get_cmds(args_list): def get_cmds(args_list):
@ -371,7 +371,7 @@ Result: {msg_dict[status]}
Details Details
Owner: Jayden Jia Owner: Jayden Jia
Start time: {starttime} Start time: {starttime}
End time: {endtime} End time: {endtime}
Hostname: {hostname} Hostname: {hostname}
Commit: {git_commit} Commit: {git_commit}
Cmd: {cmd} Cmd: {cmd}
@ -380,14 +380,13 @@ Core dir: {core_dir}
''' '''
text_result=text.split("Result: ")[1].split("Details")[0].strip() text_result=text.split("Result: ")[1].split("Details")[0].strip()
print(text_result) print(text_result)
if text_result == "success": if text_result == "success":
send_msg(notification_robot_url, get_msg(text)) send_msg(notification_robot_url, get_msg(text))
else: else:
send_msg(alert_robot_url, get_msg(text)) send_msg(alert_robot_url, get_msg(text))
send_msg(notification_robot_url, get_msg(text)) send_msg(notification_robot_url, get_msg(text))
#send_msg(get_msg(text)) #send_msg(get_msg(text))
except Exception as e: except Exception as e:
print("exception:", e) print("exception:", e)
exit(status) exit(status)

14
tests/pytest/auto_crash_gen_valgrind.py
View File

@ -245,7 +245,7 @@ def start_taosd():
else: else:
pass pass
start_cmd = 'cd %s && python3 test.py '%(start_path) start_cmd = 'cd %s && python3 test.py -G'%(start_path)
os.system(start_cmd +">>/dev/null") os.system(start_cmd +">>/dev/null")
def get_cmds(args_list): def get_cmds(args_list):
@ -404,24 +404,24 @@ Result: {msg_dict[status]}
Details Details
Owner: Jayden Jia Owner: Jayden Jia
Start time: {starttime} Start time: {starttime}
End time: {endtime} End time: {endtime}
Hostname: {hostname} Hostname: {hostname}
Commit: {git_commit} Commit: {git_commit}
Cmd: {cmd} Cmd: {cmd}
Log dir: {log_dir} Log dir: {log_dir}
Core dir: {core_dir} Core dir: {core_dir}
''' '''
text_result=text.split("Result: ")[1].split("Details")[0].strip() text_result=text.split("Result: ")[1].split("Details")[0].strip()
print(text_result) print(text_result)
if text_result == "success": if text_result == "success":
send_msg(notification_robot_url, get_msg(text)) send_msg(notification_robot_url, get_msg(text))
else: else:
send_msg(alert_robot_url, get_msg(text)) send_msg(alert_robot_url, get_msg(text))
send_msg(notification_robot_url, get_msg(text)) send_msg(notification_robot_url, get_msg(text))
#send_msg(get_msg(text)) #send_msg(get_msg(text))
except Exception as e: except Exception as e:
print("exception:", e) print("exception:", e)
exit(status) exit(status)

18
tests/pytest/auto_crash_gen_valgrind_cluster.py
View File

@ -236,7 +236,7 @@ def start_taosd():
else: else:
pass pass
start_cmd = 'cd %s && python3 test.py -N 4 -M 1 '%(start_path) start_cmd = 'cd %s && python3 test.py -N 4 -M 1 -G '%(start_path)
os.system(start_cmd +">>/dev/null") os.system(start_cmd +">>/dev/null")
def get_cmds(args_list): def get_cmds(args_list):
@ -388,28 +388,28 @@ def main():
text = f''' text = f'''
Result: {msg_dict[status]} Result: {msg_dict[status]}
Details Details
Owner: Jayden Jia Owner: Jayden Jia
Start time: {starttime} Start time: {starttime}
End time: {endtime} End time: {endtime}
Hostname: {hostname} Hostname: {hostname}
Commit: {git_commit} Commit: {git_commit}
Cmd: {cmd} Cmd: {cmd}
Log dir: {log_dir} Log dir: {log_dir}
Core dir: {core_dir} Core dir: {core_dir}
''' '''
text_result=text.split("Result: ")[1].split("Details")[0].strip() text_result=text.split("Result: ")[1].split("Details")[0].strip()
print(text_result) print(text_result)
if text_result == "success": if text_result == "success":
send_msg(notification_robot_url, get_msg(text)) send_msg(notification_robot_url, get_msg(text))
else: else:
send_msg(alert_robot_url, get_msg(text)) send_msg(alert_robot_url, get_msg(text))
send_msg(notification_robot_url, get_msg(text)) send_msg(notification_robot_url, get_msg(text))
#send_msg(get_msg(text)) #send_msg(get_msg(text))
except Exception as e: except Exception as e:
print("exception:", e) print("exception:", e)
exit(status) exit(status)

23
tests/run_all_ci_cases.sh
View File

@ -23,20 +23,24 @@ function printHelp() {
echo " -b [Build test branch] Build test branch (default: null)" echo " -b [Build test branch] Build test branch (default: null)"
echo " Options: " echo " Options: "
echo " e.g., -b main (pull main branch, build and install)" echo " e.g., -b main (pull main branch, build and install)"
echo " -t [Run test cases] Run test cases type(default: all)"
echo " Options: "
echo " e.g., -t all/python/legacy"
echo " -s [Save cases log] Save cases log(default: notsave)" echo " -s [Save cases log] Save cases log(default: notsave)"
echo " Options:" echo " Options:"
echo " e.g., -c notsave : do not save the log " echo " e.g., -s notsave : do not save the log "
echo " -c save : default save ci case log in Project dir/tests/ci_bak" echo " -s save : default save ci case log in Project dir/tests/ci_bak"
exit 0 exit 0
} }
# Initialization parameter # Initialization parameter
PROJECT_DIR="" PROJECT_DIR=""
BRANCH="" BRANCH=""
TEST_TYPE=""
SAVE_LOG="notsave" SAVE_LOG="notsave"
# Parse command line parameters # Parse command line parameters
while getopts "hb:d:s:" arg; do while getopts "hb:d:t:s:" arg; do
case $arg in case $arg in
d) d)
PROJECT_DIR=$OPTARG PROJECT_DIR=$OPTARG
@ -44,6 +48,9 @@ while getopts "hb:d:s:" arg; do
b) b)
BRANCH=$OPTARG BRANCH=$OPTARG
;; ;;
t)
TEST_TYPE=$OPTARG
;;
s) s)
SAVE_LOG=$OPTARG SAVE_LOG=$OPTARG
;; ;;
@ -315,9 +322,9 @@ function runTest() {
[ -d sim ] && rm -rf sim [ -d sim ] && rm -rf sim
[ -f $TDENGINE_ALLCI_REPORT ] && rm $TDENGINE_ALLCI_REPORT [ -f $TDENGINE_ALLCI_REPORT ] && rm $TDENGINE_ALLCI_REPORT
runUnitTest
runSimCases runSimCases
runPythonCases runPythonCases
runUnitTest
stopTaosd stopTaosd
cd $TDENGINE_DIR/tests/script cd $TDENGINE_DIR/tests/script
@ -361,7 +368,13 @@ print_color "$GREEN" "Run all ci test cases" | tee -a $WORK_DIR/date.log
stopTaosd stopTaosd
runTest if [ -z "$TEST_TYPE" -o "$TEST_TYPE" = "all" -o "$TEST_TYPE" = "ALL" ]; then
runTest
elif [ "$TEST_TYPE" = "python" -o "$TEST_TYPE" = "PYTHON" ]; then
runPythonCases
elif [ "$TEST_TYPE" = "legacy" -o "$TEST_TYPE" = "LEGACY" ]; then
runSimCases
fi
date >> $WORK_DIR/date.log date >> $WORK_DIR/date.log
print_color "$GREEN" "End of ci test cases" | tee -a $WORK_DIR/date.log print_color "$GREEN" "End of ci test cases" | tee -a $WORK_DIR/date.log

55
tests/system-test/0-others/test_case_template.py Normal file
View File

@ -0,0 +1,55 @@
from util.log import tdLog
from util.cases import tdCases
from util.sql import tdSql
from util.dnodes import tdDnodes
from util.dnodes import *
from util.common import *
class TDTestCase:
"""
Here is the class description for the whole file cases
"""
# add the configuration of the client and server here
clientCfgDict = {'debugFlag': 131}
updatecfgDict = {
"debugFlag" : "131",
"queryBufferSize" : 10240,
'clientCfg' : clientCfgDict
}
def init(self, conn, logSql, replicaVar=1):
tdLog.debug(f"start to excute {__file__}")
tdSql.init(conn.cursor())
self.replicaVar = int(replicaVar)
def test_function(self): # case function should be named start with test_
"""
Here is the function description for single test:
Test case for custom function
"""
tdLog.info(f"Test case test custom function")
# excute the sql
tdSql.execute(f"create database db_test_function")
tdSql.execute(f"create table db_test_function.stb (ts timestamp, c1 int, c2 float, c3 double) tags (t1 int unsigned);")
# qury the result and return the result
tdSql.query(f"show databases")
# print result and check the result
database_info = tdLog.info(f"{tdSql.queryResult}")
tdSql.checkRows(3)
tdSql.checkData(2,0,"db_test_function")
def run(self):
self.test_function()
def stop(self):
tdSql.close()
tdLog.success(f"{__file__} successfully executed")
tdCases.addLinux(__file__, TDTestCase())
tdCases.addWindows(__file__, TDTestCase())

25
tests/system-test/test.py
View File

@ -58,12 +58,12 @@ def checkRunTimeError():
if hwnd: if hwnd:
os.system("TASKKILL /F /IM taosd.exe") os.system("TASKKILL /F /IM taosd.exe")
# #
# run case on previous cluster # run case on previous cluster
# #
def runOnPreviousCluster(host, config, fileName): def runOnPreviousCluster(host, config, fileName):
print("enter run on previeous") print("enter run on previeous")
# load case module # load case module
sep = "/" sep = "/"
if platform.system().lower() == 'windows': if platform.system().lower() == 'windows':
@ -113,8 +113,9 @@ if __name__ == "__main__":
asan = False asan = False
independentMnode = False independentMnode = False
previousCluster = False previousCluster = False
opts, args = getopt.gnu_getopt(sys.argv[1:], 'f:p:m:l:scghrd:k:e:N:M:Q:C:RWD:n:i:aP', [ crashGen = False
'file=', 'path=', 'master', 'logSql', 'stop', 'cluster', 'valgrind', 'help', 'restart', 'updateCfgDict', 'killv', 'execCmd','dnodeNums','mnodeNums','queryPolicy','createDnodeNums','restful','websocket','adaptercfgupdate','replicaVar','independentMnode','previous']) opts, args = getopt.gnu_getopt(sys.argv[1:], 'f:p:m:l:scghrd:k:e:N:M:Q:C:RWD:n:i:aP:G', [
'file=', 'path=', 'master', 'logSql', 'stop', 'cluster', 'valgrind', 'help', 'restart', 'updateCfgDict', 'killv', 'execCmd','dnodeNums','mnodeNums','queryPolicy','createDnodeNums','restful','websocket','adaptercfgupdate','replicaVar','independentMnode','previous',"crashGen"])
for key, value in opts: for key, value in opts:
if key in ['-h', '--help']: if key in ['-h', '--help']:
tdLog.printNoPrefix( tdLog.printNoPrefix(
@ -141,6 +142,7 @@ if __name__ == "__main__":
tdLog.printNoPrefix('-i independentMnode Mnode') tdLog.printNoPrefix('-i independentMnode Mnode')
tdLog.printNoPrefix('-a address sanitizer mode') tdLog.printNoPrefix('-a address sanitizer mode')
tdLog.printNoPrefix('-P run case with [P]revious cluster, do not create new cluster to run case.') tdLog.printNoPrefix('-P run case with [P]revious cluster, do not create new cluster to run case.')
tdLog.printNoPrefix('-G crashGen mode')
sys.exit(0) sys.exit(0)
@ -208,7 +210,7 @@ if __name__ == "__main__":
if key in ['-R', '--restful']: if key in ['-R', '--restful']:
restful = True restful = True
if key in ['-W', '--websocket']: if key in ['-W', '--websocket']:
websocket = True websocket = True
@ -228,6 +230,10 @@ if __name__ == "__main__":
if key in ['-P', '--previous']: if key in ['-P', '--previous']:
previousCluster = True previousCluster = True
if key in ['-G', '--crashGen']:
crashGen = True
# #
# do exeCmd command # do exeCmd command
# #
@ -405,7 +411,7 @@ if __name__ == "__main__":
for dnode in tdDnodes.dnodes: for dnode in tdDnodes.dnodes:
tdDnodes.starttaosd(dnode.index) tdDnodes.starttaosd(dnode.index)
tdCases.logSql(logSql) tdCases.logSql(logSql)
if restful or websocket: if restful or websocket:
tAdapter.deploy(adapter_cfg_dict) tAdapter.deploy(adapter_cfg_dict)
tAdapter.start() tAdapter.start()
@ -450,7 +456,7 @@ if __name__ == "__main__":
else: else:
tdLog.debug(res) tdLog.debug(res)
tdLog.exit(f"alter queryPolicy to {queryPolicy} failed") tdLog.exit(f"alter queryPolicy to {queryPolicy} failed")
if ucase is not None and hasattr(ucase, 'noConn') and ucase.noConn == True: if ucase is not None and hasattr(ucase, 'noConn') and ucase.noConn == True:
conn = None conn = None
else: else:
@ -640,7 +646,7 @@ if __name__ == "__main__":
else: else:
tdLog.debug(res) tdLog.debug(res)
tdLog.exit(f"alter queryPolicy to {queryPolicy} failed") tdLog.exit(f"alter queryPolicy to {queryPolicy} failed")
# run case # run case
if testCluster: if testCluster:
@ -692,6 +698,7 @@ if __name__ == "__main__":
# tdDnodes.StopAllSigint() # tdDnodes.StopAllSigint()
tdLog.info("Address sanitizer mode finished") tdLog.info("Address sanitizer mode finished")
else: else:
tdDnodes.stopAll() if not crashGen:
tdDnodes.stopAll()
tdLog.info("stop all td process finished") tdLog.info("stop all td process finished")
sys.exit(0) sys.exit(0)

4
tests/unit-test/test.sh
View File

@ -7,10 +7,10 @@ function usage() {
} }
ent=1 ent=1
while getopts "eh" opt; do while getopts "e:h" opt; do
case $opt in case $opt in
e) e)
ent=1 ent="$OPTARG"
;; ;;
h) h)
usage usage