From 24db39d4ce4a938baf8f5dce3b9ef427aa502fde Mon Sep 17 00:00:00 2001 From: Feng Chao Date: Thu, 16 Jan 2025 18:08:07 +0800 Subject: [PATCH 1/2] update testing guide structure by charles --- tests/README.md | 61 +++++++++++++++++++++++-------------------------- 1 file changed, 29 insertions(+), 32 deletions(-) diff --git a/tests/README.md b/tests/README.md index f684c8977c..4349e4bc65 100644 --- a/tests/README.md +++ b/tests/README.md @@ -3,13 +3,12 @@ 1. [Introduction](#1-introduction) 1. [Prerequisites](#2-prerequisites) 1. [Testing Guide](#3-testing-guide) - - [3.1 CI Test](#31-ci-test) - - [3.1.1 Unit Test](#311-unit-test) - - [3.1.2 System Test](#312-system-test) - - [3.1.3 Legacy Test](#313-legacy-test) - - [3.2 Smoke Test](#32-smoke-test) - - [3.3 Chaos Test](#33-chaos-test) - + - [3.1 Unit Test](#31-unit-test) + - [3.2 System Test](#32-system-test) + - [3.3 Legacy Test](#33-legacy-test) + - [3.4 Smoke Test](#34-smoke-test) + - [3.5 Chaos Test](#35-chaos-test) + - [3.6 CI Test](#36-ci-test) # 1. Introduction @@ -54,25 +53,7 @@ make && make install In `tests` directory, there are different types of tests for TDengine. Below is a brief introduction about how to run them and how to add new cases. - -## 3.1 CI Test - -CI testing (Continuous Integration testing), is an important practice in software development that aims to automate frequent integration of code into a shared codebase, build and test it to ensure code quality and stability. TDengine CI testing includes three part of test cases: unit test、system test and legacy test - -### How to run all CI test cases? - -If this is the first time to run all the CI test cases, it is recommended to add the test branch, please run it with following commands: - -```bash -cd tests -./run_all_ci_cases.sh -b main # on main branch -``` - -### How to add new cases? - -You can refer the below child sections to add new test cases for CI test. - -### 3.1.1 Unit Test +### 3.1 Unit Test Unit test script is the smallest testable part and developed for some function, method or class of TDengine. @@ -90,7 +71,7 @@ cd tests/unit-test/ bash test.sh -e 0 ``` -#### How to add new cases? +#### How to add new cases? {#test-id}
@@ -120,7 +101,7 @@ Use the add_test command to add new compiled test cases into CI test collection,
-## 3.1.2 System Test +## 3.2 System Test Python test script includes all of the functions of TDengine OSS, so some test case maybe fail cause the function only work for TDengine Enterprise Edition. @@ -169,8 +150,7 @@ Use the add_test command to add new compiled test cases into CI test collection, - -## 3.1.3 Legacy Test +## 3.3 Legacy Test In the early stage of TDengine development, test cases are run by an internal test framework called TSIM, which is developed in C++. @@ -195,7 +175,7 @@ cd tests > [!NOTE] > TSIM test framwork is replaced by system test currently, suggest to add new test scripts to system test, you can refer [System Test](#312-system-test) for detail steps. -## 3.2 Smoke Test +## 3.4 Smoke Test Smoke test script is from system test and known as sanity testing to ensure that the critical functionalities of TDengine. @@ -210,7 +190,7 @@ cd /root/TDengine/packaging/smokeTest You can update python commands part of test_smoking_selfhost.sh file to add any system test case into smoke test. -## 3.3 Chaos Test +## 3.5 Chaos Test A simple tool to exercise various functions of the system in a randomized fashion, hoping to expose maximum number of problems without a pre-determined scenario. @@ -225,3 +205,20 @@ python3 auto_crash_gen.py Add a function, such as TaskCreateNewFunction, to pytest/crash_gen/crash_gen_main.py. Integrate TaskCreateNewFunction into the balance_pickTaskType function in crash_gen_main.py. + +## 3.6 CI Test + +CI testing (Continuous Integration testing), is an important practice in software development that aims to automate frequent integration of code into a shared codebase, build and test it to ensure code quality and stability. TDengine CI testing includes three part of test cases: unit test、system test and legacy test + +### How to run all CI test cases? + +If this is the first time to run all the CI test cases, it is recommended to add the test branch, please run it with following commands: + +```bash +cd tests +./run_all_ci_cases.sh -b main # on main branch +``` + +### How to add new cases? + +You can refer the [Unit Test](#31-unit-test)、[System Test](#32-system-test) and [Legacy Test](#33-legacy-test) sections for detail steps to add new test cases for CI test. From 20f911d009ab106487d8b848d9f1fd2edcffae7e Mon Sep 17 00:00:00 2001 From: happyguoxy Date: Thu, 16 Jan 2025 18:52:43 +0800 Subject: [PATCH 2/2] test:add note --- README.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 91232136c5..8ceab44b3f 100644 --- a/README.md +++ b/README.md @@ -387,7 +387,10 @@ TDengine build check workflow can be found in this [Github Action](https://githu # 11. Coverage -Latest TDengine test coverage report can be found on [coveralls.io](https://coveralls.io/github/taosdata/TDengine). To create the test coverage report (in HTML format) locally, please run following commands: +
+ +Latest TDengine test coverage report can be found on [coveralls.io](https://coveralls.io/github/taosdata/TDengine). +To create the test coverage report (in HTML format) locally, please run following commands: ```bash cd tests @@ -395,9 +398,12 @@ bash setup-lcov.sh -v 1.16 && ./run_local_coverage.sh -b main -c task # on main branch and run cases in longtimeruning_cases.task # for more infomation about options please refer to ./run_local_coverage.sh -h ``` -> [!NOTE] +> [**NOTE**] > Please note that the -b and -i options will recompile TDengine with the -DCOVER=true option, which may take a amount of time. +
+ + # 12. Contributing Please follow the [contribution guidelines](CONTRIBUTING.md) to contribute to TDengine.