From c4b522ff09e80356c3a522788c5fc83ec81607c5 Mon Sep 17 00:00:00 2001 From: Feng Chao Date: Fri, 3 Jan 2025 14:15:29 +0800 Subject: [PATCH] update README structure for support linux/win/mac platforms by charles --- README.md | 337 ++++++++++++++++++++++++++++++++---------------------- 1 file changed, 198 insertions(+), 139 deletions(-) diff --git a/README.md b/README.md index e4814cee67..dfd1be23c7 100644 --- a/README.md +++ b/README.md @@ -26,24 +26,51 @@ English | [简体中文](README-CN.md) | [TDengine Cloud](https://cloud.tdengine # Table of Contents -1. [What is TDengine?](#1-what-is-tdengine) -2. [Documentation](#2-documentation) -3. [Building](#3-building) - 1. [Install build tools](#31-install-build-tools) - 1. [Get the source codes](#32-get-the-source-codes) - 1. [Special Note](#33-special-note) - 1. [Build TDengine](#34-build-tdengine) -4. [Installing](#4-installing) - 1. [On Linux platform](#41-on-linux-platform) - 1. [On Windows platform](#42-on-windows-platform) - 1. [On macOS platform](#43-on-macos-platform) - 1. [Quick Run](#44-quick-run) -5. [Try TDengine](#5-try-tdengine) -6. [Developing with TDengine](#6-developing-with-tdengine) -7. [Contribute to TDengine](#7-contribute-to-tdengine) -8. [Join the TDengine Community](#8-join-the-tdengine-community) +- [Table of Contents](#table-of-contents) +- [1. Introduction](#1-introduction) +- [2. Documentation](#2-documentation) +- [3. Prerequisites](#3-prerequisites) + - [3.1 Install dependencies tools](#31-install-dependencies-tools) + - [3.1.1 Linux platform](#311-linux-platform) + - [Install the required package for linux](#install-the-required-package-for-linux) + - [Install Go for linux](#install-go-for-linux) + - [Install node for linux](#install-node-for-linux) + - [Install Python-connector for linux](#install-python-connector-for-linux) + - [3.1.2 Windows platform](#312-windows-platform) + - [Install the required package for windows](#install-the-required-package-for-windows) + - [3.1.3 MacOS platform](#313-macos-platform) + - [Install the required package for macOS](#install-the-required-package-for-macos) + - [3.2 Get the source codes](#32-get-the-source-codes) + - [3.3 Special Note](#33-special-note) +- [4. Building](#4-building) + - [4.1 Linux platform building](#41-linux-platform-building) + - [4.2 Windows platform building](#42-windows-platform-building) + - [4.3 MacOS platform building](#43-macos-platform-building) +- [5. Packaging](#5-packaging) + - [5.1 Linux platform packaging](#51-linux-platform-packaging) + - [5.2 Windows platform packaging](#52-windows-platform-packaging) + - [5.3 MacOS platform packaging](#53-macos-platform-packaging) +- [6. Installation](#6-installation) + - [6.1 Linux platform installation](#61-linux-platform-installation) + - [6.2 Windows platform installation](#62-windows-platform-installation) + - [6.3 MacOS platform installation](#63-macos-platform-installation) +- [7. Running](#7-running) + - [7.1 Linux platform](#71-linux-platform) + - [7.2 Windows platform](#72-windows-platform) + - [7.3 MacOS platform](#73-macos-platform) +- [8. Testing](#8-testing) + - [8.1 Linux platform testing](#81-linux-platform-testing) + - [8.2 Windows platform testing](#82-windows-platform-testing) + - [8.3 MacOS platform testing](#83-macos-platform-testing) +- [9. Releasing](#9-releasing) + - [9.1 Linux platform releasing](#91-linux-platform-releasing) + - [9.2 Windows platform realeasing](#92-windows-platform-realeasing) + - [9.3 MacOS platform releasing](#93-macos-platform-releasing) +- [10. CI/CD](#10-cicd) +- [11. Coverage](#11-coverage) +- [12. Contributing](#12-contributing) -# 1. What is TDengine？ +# 1. Introduction TDengine is an open source, high-performance, cloud native [time-series database](https://tdengine.com/tsdb/) optimized for Internet of Things (IoT), Connected Cars, and Industrial IoT. It enables efficient, real-time data ingestion, processing, and monitoring of TB and even PB scale data per day, generated by billions of sensors and data collectors. TDengine differentiates itself from other time-series databases with the following advantages: @@ -65,109 +92,127 @@ For a full list of TDengine competitive advantages, please [check here](https:// For user manual, system design and architecture, please refer to [TDengine Documentation](https://docs.tdengine.com) ([TDengine 文档](https://docs.taosdata.com)) -# 3. Building +# 3. Prerequisites -At the moment, TDengine server supports running on Linux/Windows/macOS systems. Any application can also choose the RESTful interface provided by taosAdapter to connect the taosd service . TDengine supports X64/ARM64 CPU, and it will support MIPS64, Alpha64, ARM32, RISC-V and other CPU architectures in the future. Right now we don't support build with cross-compiling environment. +List the software and tools required to work on the project. -You can choose to install through source code, [container](https://docs.tdengine.com/get-started/docker/), [installation package](https://docs.tdengine.com/get-started/package/) or [Kubernetes](https://docs.tdengine.com/deployment/k8s/). This quick guide only applies to installing from source. +- go 1.20+ (for taosadapter) +- node 16.20.2 (for taos-explorer) +- python 3.10.12+ (for test) -TDengine provide a few useful tools such as taosBenchmark (was named taosdemo) and taosdump. They were part of TDengine. By default, TDengine compiling does not include taosTools. You can use `cmake .. -DBUILD_TOOLS=true` to make them be compiled with TDengine. +Step-by-step instructions to set up the prerequisites software. -To build TDengine, use [CMake](https://cmake.org/) 3.13.0 or higher versions in the project directory. +## 3.1 Install dependencies tools -## 3.1 Install build tools +### 3.1.1 Linux platform -### Ubuntu 18.04 and above or Debian +#### Install the required package for linux ```bash -sudo apt-get install -y gcc cmake build-essential git libssl-dev libgflags2.2 libgflags-dev +apt-get install -y llvm gcc make cmake libssl-dev pkg-config perl g++ lzma curl locales psmisc sudo tree libgeos-dev libgflags2.2 libgflags-dev libgoogle-glog-dev libjansson-dev libsnappy-dev liblzma-dev libz-dev zlib1g build-essential valgrind rsync vim libjemalloc-dev openssh-server screen sshpass net-tools dirmngr gnupg apt-transport-https ca-certificates software-properties-common r-base iputils-ping ``` -#### Install build dependencies for taosTools +#### Install Go for linux -To build the [taosTools](https://github.com/taosdata/taos-tools) on Ubuntu/Debian, the following packages need to be installed. +Update the installation package to version 1.23.3. ```bash -sudo apt install build-essential libjansson-dev libsnappy-dev liblzma-dev libz-dev zlib1g pkg-config +cd /usr/local/ +wget https://golang.google.cn/dl/go1.23.3.linux-amd64.tar.gz +rm -rf /usr/local/go && tar -C /usr/local -xzf go1.23.3.linux-amd64.tar.gz ``` -### CentOS 7.9 +Set up environment variables, first add the following content to the end of the `~/.bashrc` file. ```bash -sudo yum install epel-release -sudo yum update -sudo yum install -y gcc gcc-c++ make cmake3 gflags git openssl-devel -sudo ln -sf /usr/bin/cmake3 /usr/bin/cmake +export GO_HOME=/usr/local/go +export PATH=$GO_HOME/bin:$PATH +export CGO_ENABLED=1 ``` -### CentOS 8/Fedora/Rocky Linux +Then make the environment variables take effect. ```bash -sudo dnf install -y gcc gcc-c++ make cmake epel-release gflags git openssl-devel +source ~/.bashrc ``` -#### Install build dependencies for taosTools on CentOS +Configure proxy to accelerate the download of Go dependencies. -#### CentOS 7.9 - -``` -sudo yum install -y zlib-devel zlib-static xz-devel snappy-devel jansson jansson-devel pkgconfig libatomic libatomic-static libstdc++-static openssl-devel +```bash +go env -w GOPROXY=https://goproxy.cn,direct +go env -w GO111MODULE=on ``` -#### CentOS 8/Fedora/Rocky Linux +Check if the environment variables have taken effect and if the version is the installed version. -``` -sudo yum install -y epel-release -sudo yum install -y dnf-plugins-core -sudo yum config-manager --set-enabled powertools -sudo yum install -y zlib-devel zlib-static xz-devel snappy-devel jansson jansson-devel pkgconfig libatomic libatomic-static libstdc++-static openssl-devel +```bash +go env +go version ``` -Note: Since snappy lacks pkg-config support (refer to [link](https://github.com/google/snappy/pull/86)), it leads a cmake prompt libsnappy not found. But snappy still works well. +#### Install node for linux -If the PowerTools installation fails, you can try to use: +Recommend install node using nvm. -``` -sudo yum config-manager --set-enabled powertools +```bash +curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.8/install.sh | bash ``` -#### For CentOS + devtoolset +Set up environment variables, add the following content to the end of the `~/.bashrc` file. -Besides above dependencies, please run following commands: - -``` -sudo yum install centos-release-scl -sudo yum install devtoolset-9 devtoolset-9-libatomic-devel -scl enable devtoolset-9 -- bash +```bash +export NVM_DIR="$HOME/.nvm" +[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" ``` -### macOS +Then make the environment variables take effect. + +```bash +source ~/.bashrc +``` + +Finally, Install node and yarn. + +```bash +nvm install 16.20.2 +npm config set registry=https://registry.npmmirror.com +npm install -g yarn +``` + +#### Install Python-connector for linux + +Install Python3. + +```bash +apt install python3 +apt install python3-pip +``` + +Install the dependent Python components. + +```bash +pip3 install pandas psutil fabric2 requests faker simplejson toml pexpect tzlocal distro decorator loguru hyperloglog +``` + +Install the Python connector for TDengine. + +```bash +pip3 install taospy taos-ws-py +``` + +### 3.1.2 Windows platform + +#### Install the required package for windows + + +### 3.1.3 MacOS platform + +#### Install the required package for macOS ``` brew install argp-standalone gflags pkgconfig ``` -### Setup golang environment - -TDengine includes a few components like taosAdapter developed by Go language. Please refer to golang.org official documentation for golang environment setup. - -Please use version 1.20+. For the user in China, we recommend using a proxy to accelerate package downloading. - -``` -go env -w GO111MODULE=on -go env -w GOPROXY=https://goproxy.cn,direct -``` - -The default will not build taosAdapter, but you can use the following command to build taosAdapter as the service for RESTful interface. - -``` -cmake .. -DBUILD_HTTP=false -``` - -### Setup rust environment - -TDengine includes a few components developed by Rust language. Please refer to rust-lang.org official documentation for rust environment setup. - ## 3.2 Get the source codes First of all, you may clone the source codes from github: @@ -188,9 +233,17 @@ You can modify the file ~/.gitconfig to use ssh protocol instead of https for be [JDBC Connector](https://github.com/taosdata/taos-connector-jdbc)， [Go Connector](https://github.com/taosdata/driver-go)，[Python Connector](https://github.com/taosdata/taos-connector-python)，[Node.js Connector](https://github.com/taosdata/taos-connector-node)，[C# Connector](https://github.com/taosdata/taos-connector-dotnet) ，[Rust Connector](https://github.com/taosdata/taos-connector-rust) and [Grafana plugin](https://github.com/taosdata/grafanaplugin) has been moved to standalone repository. -## 3.4 Build TDengine +# 4. Building -### On Linux platform +At the moment, TDengine server supports running on Linux/Windows/MacOS systems. Any application can also choose the RESTful interface provided by taosAdapter to connect the taosd service. TDengine supports X64/ARM64 CPU, and it will support MIPS64, Alpha64, ARM32, RISC-V and other CPU architectures in the future. Right now we don't support build with cross-compiling environment. + +You can choose to install through source code, [container](https://docs.tdengine.com/get-started/docker/), [installation package](https://docs.tdengine.com/get-started/package/) or [Kubernetes](https://docs.tdengine.com/deployment/k8s/). This quick guide only applies to installing from source. + +TDengine provide a few useful tools such as taosBenchmark (was named taosdemo) and taosdump. They were part of TDengine. By default, TDengine compiling does not include taosTools. You can use `cmake .. -DBUILD_TOOLS=true` to make them be compiled with TDengine. + +To build TDengine, use [CMake](https://cmake.org/) 3.13.0 or higher versions in the project directory. + +## 4.1 Linux platform building You can run the bash script `build.sh` to build both TDengine and taosTools including taosBenchmark and taosdump as below: @@ -223,7 +276,7 @@ aarch64: cmake .. -DCPUTYPE=aarch64 && cmake --build . ``` -### On Windows platform +## 4.2 Windows platform building If you use the Visual Studio 2013, please open a command window by executing "cmd.exe". Please specify "amd64" for 64 bits Windows or specify "x86" for 32 bits Windows when you execute vcvarsall.bat. @@ -255,7 +308,7 @@ cmake .. -G "NMake Makefiles" nmake ``` -### On macOS platform +## 4.3 MacOS platform building Please install XCode command line tools and cmake. Verified with XCode 11.4+ on Catalina and Big Sur. @@ -264,9 +317,17 @@ mkdir debug && cd debug cmake .. && cmake --build . ``` -# 4. Installing +# 5. Packaging -## 4.1 On Linux platform +## 5.1 Linux platform packaging + +## 5.2 Windows platform packaging + +## 5.3 MacOS platform packaging + +# 6. Installation + +## 6.1 Linux platform installation After building successfully, TDengine can be installed by @@ -278,21 +339,7 @@ Users can find more information about directories installed on the system in the Installing from source code will also configure service management for TDengine.Users can also choose to [install from packages](https://docs.tdengine.com/get-started/package/) for it. -To start the service after installation, in a terminal, use: - -```bash -sudo systemctl start taosd -``` - -Then users can use the TDengine CLI to connect the TDengine server. In a terminal, use: - -```bash -taos -``` - -If TDengine CLI connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown. - -## 4.2 On Windows platform +## 6.2 Windows platform installation After building successfully, TDengine can be installed by: @@ -300,7 +347,7 @@ After building successfully, TDengine can be installed by: nmake install ``` -## 4.3 On macOS platform +## 6.3 MacOS platform installation After building successfully, TDengine can be installed by: @@ -326,7 +373,23 @@ taos If TDengine CLI connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown. -## 4.4 Quick Run +# 7. Running + +## 7.1 Linux platform + +To start the service after installation, in a terminal, use: + +```bash +sudo systemctl start taosd +``` + +Then users can use the TDengine CLI to connect the TDengine server. In a terminal, use: + +```bash +taos +``` + +If TDengine CLI connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown. If you don't want to run TDengine as a service, you can run it in current shell. For example, to quickly start a TDengine server after building, run the command below in terminal: (We take Linux as an example, command on Windows will be `taosd.exe`) @@ -342,48 +405,44 @@ In another terminal, use the TDengine CLI to connect the server: option "-c test/cfg" specifies the system configuration file directory. -# 5. Try TDengine +## 7.2 Windows platform -It is easy to run SQL commands from TDengine CLI which is the same as other SQL databases. +## 7.3 MacOS platform -```sql -CREATE DATABASE demo; -USE demo; -CREATE TABLE t (ts TIMESTAMP, speed INT); -INSERT INTO t VALUES('2019-07-15 00:00:00', 10); -INSERT INTO t VALUES('2019-07-15 01:00:00', 20); -SELECT * FROM t; - ts | speed | -=================================== - 19-07-15 00:00:00.000| 10| - 19-07-15 01:00:00.000| 20| -Query OK, 2 row(s) in set (0.001700s) +# 8. Testing + +## 8.1 Linux platform testing + +## 8.2 Windows platform testing + +## 8.3 MacOS platform testing + +# 9. Releasing + +## 9.1 Linux platform releasing + +## 9.2 Windows platform realeasing + +## 9.3 MacOS platform releasing + +# 10. CI/CD +We use jenkins for CI/CD workflow configuration. See http://ci.bl.taosdata.com:8080/job/NewTest/view/change-requests/ +We can also run ci script locally. + +```bash +cd /root/TDengine/tests +./run_all_ci_cases.sh ``` -# 6. Developing with TDengine +# 11. Coverage +We can see coverage result in https://coveralls.io/github/taosdata/TDengine +We can also run coverage script locally. -## Official Connectors +```bash +cd /root/TDengine/tests +./run_local_coverage.sh +``` -TDengine provides abundant developing tools for users to develop on TDengine. Follow the links below to find your desired connectors and relevant documentation. - -- [Java](https://docs.tdengine.com/reference/connectors/java/) -- [C/C++](https://docs.tdengine.com/reference/connectors/cpp/) -- [Python](https://docs.tdengine.com/reference/connectors/python/) -- [Go](https://docs.tdengine.com/reference/connectors/go/) -- [Node.js](https://docs.tdengine.com/reference/connectors/node/) -- [Rust](https://docs.tdengine.com/reference/connectors/rust/) -- [C#](https://docs.tdengine.com/reference/connectors/csharp/) -- [RESTful API](https://docs.tdengine.com/reference/connectors/rest-api/) - -# 7. Contribute to TDengine +# 12. Contributing Please follow the [contribution guidelines](CONTRIBUTING.md) to contribute to the project. - -# 8. Join the TDengine Community - -For more information about TDengine, you can follow us on social media and join our Discord server: - -- [Discord](https://discord.com/invite/VZdSuUg4pS) -- [Twitter](https://twitter.com/TDengineDB) -- [LinkedIn](https://www.linkedin.com/company/tdengine/) -- [YouTube](https://www.youtube.com/@tdengine)