udpate README to remove the second layer section for Building and Testing by charles

This commit is contained in:
Feng Chao 2025-01-07 16:44:39 +08:00
parent 1e2e526f96
commit 16d1542712
1 changed files with 259 additions and 269 deletions

528
README.md
View File

@ -30,20 +30,11 @@ English | [简体中文](README-CN.md) | [TDengine Cloud](https://cloud.tdengine
- [1. Introduction](#1-introduction)
- [2. Documentation](#2-documentation)
- [3. Prerequisites](#3-prerequisites)
- [3.1 Install dependencies tools](#31-install-dependencies-tools)
- [3.2 Get the source codes](#32-get-the-source-codes)
- [3.3 Special Note](#33-special-note)
- [4. Building](#4-building)
- [5. Packaging](#5-packaging)
- [6. Installation](#6-installation)
- [7. Running](#7-running)
- [8. Testing](#8-testing)
- [8.1 Run the TSIM test script](#81-run-the-tsim-test-script)
- [8.2 Run the Python test script](#82-run-the-python-test-script)
- [8.3 Run unittest](#83-run-unittest)
- [8.4 Smoke Testing](#84-smoke-testing)
- [8.5 TSBS Test --replace](#85-tsbs-test---replace)
- [8.6 Crash\_gen Test](#86-crash_gen-test)
- [9. Releasing](#9-releasing)
- [10. CI/CD](#10-cicd)
- [11. Coverage --replace](#11-coverage---replace)
@ -81,131 +72,130 @@ List the software and tools required to work on the project.
Step-by-step instructions to set up the prerequisites software.
## 3.1 Install dependencies tools
- Install dependencies tools
Install the required package for linux
- The required package for linux
```bash
apt-get install -y llvm gcc make cmake libssl-dev pkg-config perl g++ lzma curl locales psmisc sudo tree libgeos-dev libgflags2.2 libgflags-dev libgoogle-glog-dev libjansson-dev libsnappy-dev liblzma-dev libz-dev zlib1g build-essential valgrind rsync vim libjemalloc-dev openssh-server screen sshpass net-tools dirmngr gnupg apt-transport-https ca-certificates software-properties-common r-base iputils-ping
```
```bash
apt-get install -y llvm gcc make cmake libssl-dev pkg-config perl g++ lzma curl locales psmisc sudo tree libgeos-dev libgflags2.2 libgflags-dev libgoogle-glog-dev libjansson-dev libsnappy-dev liblzma-dev libz-dev zlib1g build-essential valgrind rsync vim libjemalloc-dev openssh-server screen sshpass net-tools dirmngr gnupg apt-transport-https ca-certificates software-properties-common r-base iputils-ping
```
Install Go for linux
Install Go for linux
Update the installation package to version 1.23.3.
Update the installation package to version 1.23.3.
```bash
cd /usr/local/
wget https://golang.google.cn/dl/go1.23.3.linux-amd64.tar.gz
rm -rf /usr/local/go && tar -C /usr/local -xzf go1.23.3.linux-amd64.tar.gz
```
```bash
cd /usr/local/
wget https://golang.google.cn/dl/go1.23.3.linux-amd64.tar.gz
rm -rf /usr/local/go && tar -C /usr/local -xzf go1.23.3.linux-amd64.tar.gz
```
Set up environment variables, first add the following content to the end of the `~/.bashrc` file.
Set up environment variables, first add the following content to the end of the `~/.bashrc` file.
```bash
export GO_HOME=/usr/local/go
export PATH=$GO_HOME/bin:$PATH
export CGO_ENABLED=1
```
```bash
export GO_HOME=/usr/local/go
export PATH=$GO_HOME/bin:$PATH
export CGO_ENABLED=1
```
Then make the environment variables take effect.
Then make the environment variables take effect.
```bash
source ~/.bashrc
```
```bash
source ~/.bashrc
```
Configure proxy to accelerate the download of Go dependencies.
Configure proxy to accelerate the download of Go dependencies.
```bash
go env -w GOPROXY=https://goproxy.cn,direct
go env -w GO111MODULE=on
```
```bash
go env -w GOPROXY=https://goproxy.cn,direct
go env -w GO111MODULE=on
```
Check if the environment variables have taken effect and if the version is the installed version.
Check if the environment variables have taken effect and if the version is the installed version.
```bash
go env
go version
```
```bash
go env
go version
```
Install node for linux
Install node for linux
Recommend install node using nvm.
Recommend install node using nvm.
```bash
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.8/install.sh | bash
```
```bash
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.8/install.sh | bash
```
Set up environment variables, add the following content to the end of the `~/.bashrc` file.
Set up environment variables, add the following content to the end of the `~/.bashrc` file.
```bash
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
```
```bash
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
```
Then make the environment variables take effect.
Then make the environment variables take effect.
```bash
source ~/.bashrc
```
```bash
source ~/.bashrc
```
Finally, Install node and yarn.
Finally, Install node and yarn.
```bash
nvm install 16.20.2
npm config set registry=https://registry.npmmirror.com
npm install -g yarn
```
```bash
nvm install 16.20.2
npm config set registry=https://registry.npmmirror.com
npm install -g yarn
```
Install Python-connector for linux
Install Python-connector for linux
Install Python3.
Install Python3.
```bash
apt install python3
apt install python3-pip
```
```bash
apt install python3
apt install python3-pip
```
Install the dependent Python components.
Install the dependent Python components.
```bash
pip3 install pandas psutil fabric2 requests faker simplejson toml pexpect tzlocal distro decorator loguru hyperloglog
```
```bash
pip3 install pandas psutil fabric2 requests faker simplejson toml pexpect tzlocal distro decorator loguru hyperloglog
```
Install the Python connector for TDengine.
Install the Python connector for TDengine.
```bash
pip3 install taospy taos-ws-py
```
```bash
pip3 install taospy taos-ws-py
```
Install the required package for windows
- The required package for windows
to be updated...
to be updated...
Install the required package for macOS
- The required package for macOS
```
brew install argp-standalone gflags pkgconfig
```
```
brew install argp-standalone gflags pkgconfig
```
## 3.2 Get the source codes
- Get the source codes
First of all, you may clone the source codes from github:
First of all, you may clone the source codes from github:
```bash
git clone https://github.com/taosdata/TDengine.git
cd TDengine
```
```bash
git clone https://github.com/taosdata/TDengine.git
cd TDengine
```
You can modify the file ~/.gitconfig to use ssh protocol instead of https for better download speed. You will need to upload ssh public key to GitHub first. Please refer to GitHub official documentation for detail.
You can modify the file ~/.gitconfig to use ssh protocol instead of https for better download speed. You will need to upload ssh public key to GitHub first. Please refer to GitHub official documentation for detail.
```
[url "git@github.com:"]
insteadOf = https://github.com/
```
```
[url "git@github.com:"]
insteadOf = https://github.com/
```
## 3.3 Special Note
[JDBC Connector](https://github.com/taosdata/taos-connector-jdbc) [Go Connector](https://github.com/taosdata/driver-go)[Python Connector](https://github.com/taosdata/taos-connector-python)[Node.js Connector](https://github.com/taosdata/taos-connector-node)[C# Connector](https://github.com/taosdata/taos-connector-dotnet) [Rust Connector](https://github.com/taosdata/taos-connector-rust) and [Grafana plugin](https://github.com/taosdata/grafanaplugin) has been moved to standalone repository.
- Special Note
[JDBC Connector](https://github.com/taosdata/taos-connector-jdbc) [Go Connector](https://github.com/taosdata/driver-go)[Python Connector](https://github.com/taosdata/taos-connector-python)[Node.js Connector](https://github.com/taosdata/taos-connector-node)[C# Connector](https://github.com/taosdata/taos-connector-dotnet) [Rust Connector](https://github.com/taosdata/taos-connector-rust) and [Grafana plugin](https://github.com/taosdata/grafanaplugin) has been moved to standalone repository.
# 4. Building
@ -217,258 +207,258 @@ TDengine provide a few useful tools such as taosBenchmark (was named taosdemo) a
To build TDengine, use [CMake](https://cmake.org/) 3.13.0 or higher versions in the project directory.
Linux platform building
- Linux platform building
You can run the bash script `build.sh` to build both TDengine and taosTools including taosBenchmark and taosdump as below:
You can run the bash script `build.sh` to build both TDengine and taosTools including taosBenchmark and taosdump as below:
```bash
./build.sh
```
```bash
./build.sh
```
It equals to execute following commands:
It equals to execute following commands:
```bash
mkdir debug
cd debug
cmake .. -DBUILD_TOOLS=true -DBUILD_CONTRIB=true
make
```
```bash
mkdir debug
cd debug
cmake .. -DBUILD_TOOLS=true -DBUILD_CONTRIB=true
make
```
You can use Jemalloc as memory allocator instead of glibc:
You can use Jemalloc as memory allocator instead of glibc:
```
apt install autoconf
cmake .. -DJEMALLOC_ENABLED=true
```
```
apt install autoconf
cmake .. -DJEMALLOC_ENABLED=true
```
TDengine build script can detect the host machine's architecture on X86-64, X86, arm64 platform.
You can also specify CPUTYPE option like aarch64 too if the detection result is not correct:
TDengine build script can detect the host machine's architecture on X86-64, X86, arm64 platform.
You can also specify CPUTYPE option like aarch64 too if the detection result is not correct:
aarch64:
aarch64:
```bash
cmake .. -DCPUTYPE=aarch64 && cmake --build .
```
```bash
cmake .. -DCPUTYPE=aarch64 && cmake --build .
```
4.2 Windows platform building
- Windows platform building
If you use the Visual Studio 2013, please open a command window by executing "cmd.exe".
Please specify "amd64" for 64 bits Windows or specify "x86" for 32 bits Windows when you execute vcvarsall.bat.
If you use the Visual Studio 2013, please open a command window by executing "cmd.exe".
Please specify "amd64" for 64 bits Windows or specify "x86" for 32 bits Windows when you execute vcvarsall.bat.
```cmd
mkdir debug && cd debug
"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\vcvarsall.bat" < amd64 | x86 >
cmake .. -G "NMake Makefiles"
nmake
```
```cmd
mkdir debug && cd debug
"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\vcvarsall.bat" < amd64 | x86 >
cmake .. -G "NMake Makefiles"
nmake
```
If you use the Visual Studio 2019 or 2017:
If you use the Visual Studio 2019 or 2017:
please open a command window by executing "cmd.exe".
Please specify "x64" for 64 bits Windows or specify "x86" for 32 bits Windows when you execute vcvarsall.bat.
please open a command window by executing "cmd.exe".
Please specify "x64" for 64 bits Windows or specify "x86" for 32 bits Windows when you execute vcvarsall.bat.
```cmd
mkdir debug && cd debug
"c:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" < x64 | x86 >
cmake .. -G "NMake Makefiles"
nmake
```
```cmd
mkdir debug && cd debug
"c:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" < x64 | x86 >
cmake .. -G "NMake Makefiles"
nmake
```
Or, you can simply open a command window by clicking Windows Start -> "Visual Studio < 2019 | 2017 >" folder -> "x64 Native Tools Command Prompt for VS < 2019 | 2017 >" or "x86 Native Tools Command Prompt for VS < 2019 | 2017 >" depends what architecture your Windows is, then execute commands as follows:
Or, you can simply open a command window by clicking Windows Start -> "Visual Studio < 2019 | 2017 >" folder -> "x64 Native Tools Command Prompt for VS < 2019 | 2017 >" or "x86 Native Tools Command Prompt for VS < 2019 | 2017 >" depends what architecture your Windows is, then execute commands as follows:
```cmd
mkdir debug && cd debug
cmake .. -G "NMake Makefiles"
nmake
```
```cmd
mkdir debug && cd debug
cmake .. -G "NMake Makefiles"
nmake
```
MacOS platform building
- MacOS platform building
Please install XCode command line tools and cmake. Verified with XCode 11.4+ on Catalina and Big Sur.
Please install XCode command line tools and cmake. Verified with XCode 11.4+ on Catalina and Big Sur.
```shell
mkdir debug && cd debug
cmake .. && cmake --build .
```
```shell
mkdir debug && cd debug
cmake .. && cmake --build .
```
# 5. Packaging
Linux platform packaging
to be updated...
Windows platform packaging
to be updated...
MacOS platform packaging
to be updated...
To be updated...
# 6. Installation
Linux platform installation
- Linux platform installation
After building successfully, TDengine can be installed by
After building successfully, TDengine can be installed by
```bash
sudo make install
```
```bash
sudo make install
```
Users can find more information about directories installed on the system in the [directory and files](https://docs.tdengine.com/reference/directory/) section.
Users can find more information about directories installed on the system in the [directory and files](https://docs.tdengine.com/reference/directory/) section.
Installing from source code will also configure service management for TDengine.Users can also choose to [install from packages](https://docs.tdengine.com/get-started/deploy-from-package/) for it.
Installing from source code will also configure service management for TDengine.Users can also choose to [install from packages](https://docs.tdengine.com/get-started/deploy-from-package/) for it.
Windows platform installation
- Windows platform installation
After building successfully, TDengine can be installed by:
After building successfully, TDengine can be installed by:
```cmd
nmake install
```
```cmd
nmake install
```
MacOS platform installation
- MacOS platform installation
After building successfully, TDengine can be installed by:
After building successfully, TDengine can be installed by:
```bash
sudo make install
```
```bash
sudo make install
```
# 7. Running
To start the service after installation on linux, in a terminal, use:
- To start the service after installation on linux, in a terminal, use:
```bash
sudo systemctl start taosd
```
```bash
sudo systemctl start taosd
```
Then users can use the TDengine CLI to connect the TDengine server. In a terminal, use:
Then users can use the TDengine CLI to connect the TDengine server. In a terminal, use:
```bash
taos
```
```bash
taos
```
If TDengine CLI connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown.
If TDengine CLI connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown.
If you don't want to run TDengine as a service, you can run it in current shell. For example, to quickly start a TDengine server after building, run the command below in terminal: (We take Linux as an example, command on Windows will be `taosd.exe`)
If you don't want to run TDengine as a service, you can run it in current shell. For example, to quickly start a TDengine server after building, run the command below in terminal: (We take Linux as an example, command on Windows will be `taosd.exe`)
```bash
./build/bin/taosd -c test/cfg
```
```bash
./build/bin/taosd -c test/cfg
```
In another terminal, use the TDengine CLI to connect the server:
In another terminal, use the TDengine CLI to connect the server:
```bash
./build/bin/taos -c test/cfg
```
```bash
./build/bin/taos -c test/cfg
```
option "-c test/cfg" specifies the system configuration file directory.
option "-c test/cfg" specifies the system configuration file directory.
To start the service after installation on windows, in a terminal, use:
- To start the service after installation on windows, in a terminal, use:
```bash
to be updated
```
Then users can use the TDengine CLI to connect the TDengine server. In a terminal, use:
```bash
to be updated
```
Then users can use the TDengine CLI to connect the TDengine server. In a terminal, use:
```bash
to be updated
```
```bash
to be updated
```
To start the service after installation on macOS, double-click the /applications/TDengine to start the program, or in a terminal, use:
- To start the service after installation on macOS, double-click the /applications/TDengine to start the program, or in a terminal, use:
```bash
sudo launchctl start com.tdengine.taosd
```
```bash
sudo launchctl start com.tdengine.taosd
```
Then users can use the TDengine CLI to connect the TDengine server. In a terminal, use:
Then users can use the TDengine CLI to connect the TDengine server. In a terminal, use:
```bash
taos
```
```bash
taos
```
If TDengine CLI connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown.
# 8. Testing
## 8.1 Run the TSIM test script
- Run the TSIM test script
TSIM test framework is developed by c++ in the start-up period of TDengine the test scripts are basic cases, you can run
the script with below command:
TSIM test framework is developed by c++ in the start-up period of TDengine the test scripts are basic cases, you can run
the script with below command:
```bash
cd /root/TDengine/tests/script
./test.sh -f tsim/db/basic1.sim
```
```bash
cd /root/TDengine/tests/script
./test.sh -f tsim/db/basic1.sim
```
## 8.2 Run the Python test script
Python test script includes almost all of the functions of TDengine, so some test case maybe fail cause the function only
work for TDengine enterprise version, you can run the script with below command:
- Run the Python test script
Python test script includes almost all of the functions of TDengine, so some test case maybe fail cause the function only
work for TDengine enterprise version, you can run the script with below command:
```bash
cd /root/TDengine/tests/system-test
python3 ./test.py -f 2-query/floor.py
```
```bash
cd /root/TDengine/tests/system-test
python3 ./test.py -f 2-query/floor.py
```
## 8.3 Run unittest
Unit test script is the smallest testable part and developed for some function, method or class of TDengine, you can run
the script with below command:
- Run unittest
Unit test script is the smallest testable part and developed for some function, method or class of TDengine, you can run
the script with below command:
```bash
cd /root/TDengine/tests/unit-test/
bash test.sh
```
```bash
cd /root/TDengine/tests/unit-test/
bash test.sh
```
## 8.4 Smoke Testing
Smoke test script is known as sanity testing to ensure that the critical functionalities of TDengine, you can run the
script with below command:
- Smoke Testing
Smoke test script is known as sanity testing to ensure that the critical functionalities of TDengine, you can run the
script with below command:
```bash
cd /root/TDengine/packaging/smokeTest
./test_smoking_selfhost.sh
```
```bash
cd /root/TDengine/packaging/smokeTest
./test_smoking_selfhost.sh
```
## 8.5 TSBS Test --replace
The Time Series Benchmark Suite(TSBS) test script provides a standardized approach to evaluate the performance of various
time series databases, you can run the script with below command:
- TSBS Test --replace
1. Clone the code
```bash
cd /root && git clone https://github.com/taosdata/tsbs.git && cd tsbs/scripts/tsdbComp
```
1. Modify IP and host of client and server in `test.ini`
```ini
clientIP="192.168.0.203" # client ip
clientHost="trd03" # client hostname
serverIP="192.168.0.204" # server ip
serverHost="trd04" # server hostname
```
1. Set up passwordless login between the client and server; otherwise, you'll need to configure the server password:
```ini
serverPass="taosdata123" # server root password
```
1. Run the following command to start the test:
```bash
nohup bash tsdbComparison.sh > test.log &
```
1. When the test is done, the result can be found in `/data2/` directory, which can also be configured in `test.ini`.
The Time Series Benchmark Suite(TSBS) test script provides a standardized approach to evaluate the performance of various
time series databases, you can run the script with below command:
## 8.6 Crash_gen Test
Crash_gen is a chaotic testing tool developed by TDengine, it can evaluate the resilience of TDengine through simulating
normal operation、intentional fault injection and their combination, you can run the script with below command:
- Clone the code
```bash
cd /root/TDengine/tests/pytest/ && ./crash_gen.sh
```
```bash
cd /root && git clone https://github.com/taosdata/tsbs.git && cd tsbs/scripts/tsdbComp
```
- Modify IP and host of client and server in `test.ini`
```ini
clientIP="192.168.0.203" # client ip
clientHost="trd03" # client hostname
serverIP="192.168.0.204" # server ip
serverHost="trd04" # server hostname
```
- Set up passwordless login between the client and server; otherwise, you'll need to configure the server password:
```ini
serverPass="taosdata123" # server root password
```
- Run the following command to start the test:
```bash
nohup bash tsdbComparison.sh > test.log &
```
When the test is done, the result can be found in `/data2/` directory, which can also be configured in `test.ini`.
- Crash_gen Test
Crash_gen is a chaotic testing tool developed by TDengine, it can evaluate the resilience of TDengine through simulating
normal operation、intentional fault injection and their combination, you can run the script with below command:
```bash
cd /root/TDengine/tests/pytest/ && ./crash_gen.sh
```
Windows platform testing
to be updated...
to be updated...
MacOS platform testing
to be updated...
to be updated...
# 9. Releasing