xxq250
/
homework-jianmu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Merge branch 'main' into merge/mainto3.0

This commit is contained in:
Simon Guan 2025-03-22 23:41:35 +08:00
parent 62da23b848 b8e14e148b
commit c00f876638
7 changed files with 151 additions and 83 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

114
.github/workflows/tdengine-release-build.yml vendored Normal file
View File

@ -0,0 +1,114 @@
name: TDengine Release Build
on:
push:
branches:
- 'main'
- '3.*'
paths-ignore:
- 'docs/**'
- 'packaging/**'
- 'tests/**'
- '**/*.md'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
name: Run on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os:
- ubuntu-22.04
- macos-14
steps:
- name: Checkout the repository
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: 1.18
- name: Install dependencies on Linux
if: runner.os == 'Linux'
run: |
sudo apt update -y
sudo apt install -y \
build-essential \
cmake \
gawk \
libgeos-dev \
libjansson-dev \
liblzma-dev \
libsnappy-dev \
libssl-dev \
libz-dev \
pkg-config \
zlib1g
- name: Install dependencies on macOS
if: runner.os == 'macOS'
run: |
brew update
brew install \
argp-standalone \
gawk \
gflags \
geos \
jansson \
openssl \
pkg-config \
snappy \
zlib
- name: Build and install TDengine
run: |
mkdir debug && cd debug
cmake .. -DBUILD_TOOLS=true \
-DBUILD_KEEPER=true \
-DBUILD_HTTP=false \
-DBUILD_TEST=true \
-DWEBSOCKET=true \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_DEPENDENCY_TESTS=false
make -j 4
sudo make install
which taosd
which taosadapter
which taoskeeper
- name: Statistics ldd
run: |
find ${{ github.workspace }}/debug/build/lib -type f -name "*.so" -print0 | xargs -0 ldd || true
find ${{ github.workspace }}/debug/build/bin -type f -print0 | xargs -0 ldd || true
- name: Statistics size
run: |
find ${{ github.workspace }}/debug/build/lib -type f -print0 | xargs -0 ls -lhrS
find ${{ github.workspace }}/debug/build/bin -type f -print0 | xargs -0 ls -lhrS
- name: Start taosd
run: |
cp /etc/taos/taos.cfg ./
sudo echo "supportVnodes 256" >> taos.cfg
nohup sudo taosd -c taos.cfg &
- name: Start taosadapter
run: nohup sudo taosadapter &
- name: Run tests with taosBenchmark
run: |
taosBenchmark -t 10 -n 10 -y
taos -s "select count(*) from test.meters"
- name: Clean up
if: always()
run: |
if pgrep taosd; then sudo pkill taosd; fi
if pgrep taosadapter; then sudo pkill taosadapter; fi

81
README.md
View File

@ -8,7 +8,7 @@
</a>
</p>
[![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/taosdata/tdengine/tdengine-test.yml)](https://github.com/taosdata/TDengine/actions/workflows/tdengine-test.yml)
[![TDengine Release Build](https://github.com/taosdata/TDengine/actions/workflows/tdengine-release-build.yml/badge.svg)](https://github.com/taosdata/TDengine/actions/workflows/tdengine-release-build.yml)
[![Coverage Status](https://coveralls.io/repos/github/taosdata/TDengine/badge.svg?branch=3.0)](https://coveralls.io/github/taosdata/TDengine?branch=3.0)
[![GitHub commit activity](https://img.shields.io/github/commit-activity/m/taosdata/tdengine)](https://github.com/feici02/TDengine/commits/main/)
<br />
@ -82,7 +82,9 @@ For contributing/building/testing TDengine Connectors, please check the followin
# 3. Prerequisites
At the moment, TDengine server supports running on Linux/Windows/MacOS systems. Any application can also choose the RESTful interface provided by taosAdapter to connect the taosd service. TDengine supports X64/ARM64 CPU, and it will support MIPS64, Alpha64, ARM32, RISC-V and other CPU architectures in the future. Right now we don't support build with cross-compiling environment.
At the moment, TDengine server supports running on Linux/MacOS systems. Any application can also choose the RESTful interface provided by taosAdapter to connect the taosd service. TDengine supports X64/ARM64 CPU, and it will support MIPS64, Alpha64, ARM32, RISC-V and other CPU architectures in the future. Right now we don't support build with cross-compiling environment.
Starting from version 3.1.0.0, TDengine supports the Windows system exclusively in its Enterprise edition.
If you want to compile taosAdapter or taosKeeper, you need to install Go 1.18 or above.
@ -127,13 +129,7 @@ brew install argp-standalone gflags pkgconfig
## 3.3 Prerequisites on Windows
<details>
<summary>Install required tools on Windows</summary>
Work in Progress.
</details>
Not available for community edition.
## 3.4 Clone the repo
@ -212,40 +208,7 @@ If you want to compile taosKeeper, you need to add the `-DBUILD_KEEPER=true` opt
## 4.3 Build on Windows
<details>
<summary>Detailed steps to build on Windows</summary>
If you use the Visual Studio 2013, please open a command window by executing "cmd.exe".
Please specify "amd64" for 64 bits Windows or specify "x86" for 32 bits Windows when you execute vcvarsall.bat.
```cmd
mkdir debug && cd debug
"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\vcvarsall.bat" < amd64 | x86 >
cmake .. -G "NMake Makefiles"
nmake
```
If you use the Visual Studio 2019 or 2017:
please open a command window by executing "cmd.exe".
Please specify "x64" for 64 bits Windows or specify "x86" for 32 bits Windows when you execute vcvarsall.bat.
```cmd
mkdir debug && cd debug
"c:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" < x64 | x86 >
cmake .. -G "NMake Makefiles"
nmake
```
Or, you can simply open a command window by clicking Windows Start -> "Visual Studio < 2019 | 2017 >" folder -> "x64 Native Tools Command Prompt for VS < 2019 | 2017 >" or "x86 Native Tools Command Prompt for VS < 2019 | 2017 >" depends what architecture your Windows is, then execute commands as follows:
```cmd
mkdir debug && cd debug
cmake .. -G "NMake Makefiles"
nmake
```
</details>
Not available for community edition.
# 5. Packaging
@ -285,17 +248,7 @@ sudo make install
## 6.3 Install on Windows
<details>
<summary>Detailed steps to install on windows</summary>
After building successfully, TDengine can be installed by:
```cmd
nmake install
```
</details>
Not available for community edition.
# 7. Running
@ -360,25 +313,7 @@ If TDengine CLI connects the server successfully, welcome messages and version i
## 7.3 Run TDengine on Windows
<details>
<summary>Detailed steps to run on windows</summary>
You can start TDengine server on Windows platform with below commands:
```cmd
.\build\bin\taosd.exe -c test\cfg
```
In another terminal, use the TDengine CLI to connect the server:
```cmd
.\build\bin\taos.exe -c test\cfg
```
option "-c test/cfg" specifies the system configuration file directory.
</details>
Not available for community edition.
# 8. Testing

14
docs/en/10-third-party/03-visual/01-grafana.md vendored
View File

@ -151,15 +151,22 @@ Using docker-compose, configure Grafana Provisioning for automated setup, and ex
services:
tdengine:
image: tdengine/tdengine:3.3.0.0
image: tdengine/tdengine:latest
container_name: tdengine
hostname: tdengine
environment:
TAOS_FQDN: tdengine
MONITOR_FQDN: tdengine
EXPLORER_CLUSTER: http://tdengine:6041
TAOS_KEEPER_TDENGINE_HOST: tdengine
volumes:
- tdengine-data:/var/lib/taos/
ports:
- 6060:6060
grafana:
image: grafana/grafana:9.3.6
image: grafana/grafana:latest
volumes:
- ./tdengine.yml/:/etc/grafana/provisioning/tdengine.yml
- ./tdengine.yml:/etc/grafana/provisioning/tdengine.yml
- grafana-data:/var/lib/grafana
environment:
# install tdengine plugin at start
@ -169,6 +176,7 @@ Using docker-compose, configure Grafana Provisioning for automated setup, and ex
TDENGINE_BASIC_AUTH: "cm9vdDp0YmFzZTEyNQ=="
ports:
- 3000:3000
volumes:
grafana-data:
tdengine-data:

14
docs/zh/10-third-party/03-visual/01-grafana.mdx vendored
View File

@ -138,15 +138,22 @@ docker run -d \
services:
tdengine:
image: tdengine/tdengine:3.3.0.0
image: tdengine/tdengine:latest
container_name: tdengine
hostname: tdengine
environment:
TAOS_FQDN: tdengine
MONITOR_FQDN: tdengine
EXPLORER_CLUSTER: http://tdengine:6041
TAOS_KEEPER_TDENGINE_HOST: tdengine
volumes:
- tdengine-data:/var/lib/taos/
ports:
- 6060:6060
grafana:
image: grafana/grafana:9.3.6
image: grafana/grafana:latest
volumes:
- ./tdengine.yml/:/etc/grafana/provisioning/tdengine.yml
- ./tdengine.yml:/etc/grafana/provisioning/tdengine.yml
- grafana-data:/var/lib/grafana
environment:
# install tdengine plugin at start
@ -156,6 +163,7 @@ docker run -d \
TDENGINE_BASIC_AUTH: "cm9vdDp0YmFzZTEyNQ=="
ports:
- 3000:3000
volumes:
grafana-data:
tdengine-data:

4
source/dnode/vnode/src/tsdb/tsdbRetention.c
View File

@ -692,13 +692,13 @@ static int32_t tsdbDoS3Migrate(SRTNer *rtner) {
int32_t r = taosStatFile(fobj->fname, &size, &mtime, NULL);
if (size > chunksize && mtime < rtner->now - tsS3UploadDelaySec) {
if (pCfg->s3Compact && lcn < 0) {
extern int32_t tsdbAsyncCompact(STsdb * tsdb, const STimeWindow *tw, bool sync);
extern int32_t tsdbAsyncCompact(STsdb * tsdb, const STimeWindow *tw, bool sync,bool s3Migrate);
STimeWindow win = {0};
tsdbFidKeyRange(fset->fid, rtner->tsdb->keepCfg.days, rtner->tsdb->keepCfg.precision, &win.skey, &win.ekey);
tsdbInfo("vgId:%d, async compact begin lcn: %d.", TD_VID(rtner->tsdb->pVnode), lcn);
code = tsdbAsyncCompact(rtner->tsdb, &win, pCfg->sttTrigger == 1);
code = tsdbAsyncCompact(rtner->tsdb, &win, pCfg->sttTrigger == 1, true);
tsdbInfo("vgId:%d, async compact end lcn: %d.", TD_VID(rtner->tsdb->pVnode), lcn);
goto _exit;
return code;

4
source/libs/stream/src/streamBackendRocksdb.c
View File

@ -3134,7 +3134,7 @@ rocksdb_iterator_t* streamStateIterCreate(SStreamState* pState, const char* cfKe
taosMemoryFree(err); \
code = TSDB_CODE_THIRDPARTY_ERROR; \
} else { \
stInfo("[InternalERR] write streamState str:%s succ to write to %s, rowValLen:%d, ttlValLen:%d, %p", toString, \
stDebug("[InternalERR] write streamState str:%s succ to write to %s, rowValLen:%d, ttlValLen:%d, %p", toString, \
funcname, vLen, ttlVLen, wrapper); \
} \
taosMemoryFree(ttlV); \
@ -4593,7 +4593,7 @@ int32_t streamStatePutBatchOptimize(SStreamState* pState, int32_t cfIdx, rocksdb
int32_t klen = ginitDict[cfIdx].enFunc((void*)key, buf);
ginitDict[cfIdx].toStrFunc((void*)key, toString);
qInfo("[InternalERR] write cfIdx:%d key:%s vlen:%d", cfIdx, toString, vlen);
stDebug("[InternalERR] write cfIdx:%d key:%s vlen:%d", cfIdx, toString, vlen);
char* ttlV = tmpBuf;
int32_t ttlVLen = ginitDict[cfIdx].enValueFunc(dst, size, ttl, &ttlV);

3
tests/README.md
View File

@ -19,6 +19,9 @@ This manual is intended to give developers a comprehensive guidance to test TDen
> - The commands and scripts below are verified on Linux (Ubuntu 18.04/20.04/22.04).
> - [taos-connector-python](https://github.com/taosdata/taos-connector-python) is used by tests written in Python, which requires Python 3.7+.
> - The commands and steps described below are to run the tests on a single host.
> - The testing framework is currently compatible with Python versions 3.8 through 3.10.
> - Vitural Environment is advised when setting up the environment, pease refer to [venv](https://docs.python.org/3/library/venv.html) for details.
# 2. Prerequisites