Merge pull request #1921 from taosdata/feature/add-how-to-add-test-into-readme
Feature/add how to add test into readme
This commit is contained in:
Shengliang Guan
GitHub
commit
d299208f82
249
README.md
249
README.md
|
|
@ -115,251 +115,6 @@ TDengine provides abundant developing tools for users to develop on TDengine. Fo
|
||||||
- [RESTful API](https://www.taosdata.com/en/documentation/connector/#RESTful-Connector)
|
- [RESTful API](https://www.taosdata.com/en/documentation/connector/#RESTful-Connector)
|
||||||
- [Node.js](https://www.taosdata.com/en/documentation/connector/#Node.js-Connector)
|
- [Node.js](https://www.taosdata.com/en/documentation/connector/#Node.js-Connector)
|
||||||
|
|
||||||
# How to run the test cases and how to add a new test case?
|
|
||||||
|
|
||||||
### Prepare development environment
|
|
||||||
|
|
||||||
1. sudo apt install
|
|
||||||
build-essential cmake net-tools python-pip python-setuptools python3-pip
|
|
||||||
python3-setuptools valgrind psmisc curl
|
|
||||||
|
|
||||||
2. git clone <https://github.com/taosdata/TDengine>; cd TDengine
|
|
||||||
|
|
||||||
3. mkdir debug; cd debug; cmake ..; make ; sudo make install
|
|
||||||
|
|
||||||
4. pip install src/connector/python/linux/python2 ; pip3 install
|
|
||||||
src/connector/python/linux/python3
|
|
||||||
|
|
||||||
### How to run TSIM test suite
|
|
||||||
|
|
||||||
1. cd \<TDengine\>/tests/script
|
|
||||||
|
|
||||||
2. sudo ./test.sh
|
|
||||||
|
|
||||||
### How to run Python test suite
|
|
||||||
|
|
||||||
1. cd \<TDengine\>/tests/pytest
|
|
||||||
|
|
||||||
2. ./smoketest.sh \# for smoke test
|
|
||||||
|
|
||||||
3. ./smoketest.sh -g \# for memory leak detection test with valgrind
|
|
||||||
|
|
||||||
4. ./fulltest.sh \# for full test
|
|
||||||
|
|
||||||
> Note1: TDengine daemon's configuration and data files are stored in
|
|
||||||
> \<TDengine\>/sim directory. As a historical design, it's same place with
|
|
||||||
> TSIM script. So after the TSIM script ran with sudo privilege, the directory
|
|
||||||
> has been used by TSIM then the python script cannot write it by a normal
|
|
||||||
> user. You need to remove the directory completely first before running the
|
|
||||||
> Python test case. We should consider using two different locations to store
|
|
||||||
> for TSIM and Python script.
|
|
||||||
|
|
||||||
> Note2: if you need to debug crash problem with a core dump, you need
|
|
||||||
> manually edit smoketest.sh or fulltest.sh to add "ulimit -c unlimited"
|
|
||||||
> before the script line. Then you can look for the core file in
|
|
||||||
> \<TDengine\>/tests/pytest after the program crash.
|
|
||||||
|
|
||||||
### How to add a new test case
|
|
||||||
|
|
||||||
**1. add a new TSIM test cases:**
|
|
||||||
|
|
||||||
TSIM test cases are now included in the new development branch and can be
|
|
||||||
added to the TDengine/tests/script/test.sh script based on the manual test
|
|
||||||
methods necessary to add test cases as described above.
|
|
||||||
|
|
||||||
**2. add a new Python test cases:**
|
|
||||||
|
|
||||||
**2.1 Please refer to \<TDengine\>/tests/pytest/insert/basic.py to add a new
|
|
||||||
test case.** The new test case must implement 3 functions, where self.init()
|
|
||||||
and self.stop() simply copy the contents of insert/basic.py and the test
|
|
||||||
logic is implemented in self.run(). You can refer to the code in the util
|
|
||||||
directory for more information.
|
|
||||||
|
|
||||||
**2.2 Edit smoketest.sh to add the path and filename of the new test case**
|
|
||||||
|
|
||||||
Note: The Python test framework may continue to be improved in the future,
|
|
||||||
hopefully, to provide more functionality and ease of writing test cases. The
|
|
||||||
method of writing the test case above does not exclude that it will also be
|
|
||||||
affected.
|
|
||||||
|
|
||||||
**2.3 What test.py does in detail:**
|
|
||||||
|
|
||||||
test.py is the entry program for test case execution and monitoring.
|
|
||||||
|
|
||||||
test.py has the following functions.
|
|
||||||
|
|
||||||
\-f --file, Specifies the test case file name to be executed
|
|
||||||
-p --path, Specifies deployment path
|
|
||||||
|
|
||||||
\-m --master, Specifies the master server IP for cluster deployment
|
|
||||||
-c--cluster, test cluster function
|
|
||||||
-s--stop, terminates all running nodes
|
|
||||||
|
|
||||||
\-g--valgrind, load valgrind for memory leak detection test
|
|
||||||
|
|
||||||
\-h--help, display help
|
|
||||||
|
|
||||||
**2.4 What util/log.py does in detail:**
|
|
||||||
|
|
||||||
log.py is quite simple, the main thing is that you can print the output in
|
|
||||||
different colors as needed. The success() should be called for successful
|
|
||||||
test case execution and the success() will print green text. The exit() will
|
|
||||||
print red text and exit the program, exit() should be called for test
|
|
||||||
failure.
|
|
||||||
|
|
||||||
**util/log.py**
|
|
||||||
|
|
||||||
...
|
|
||||||
|
|
||||||
def info(self, info):
|
|
||||||
|
|
||||||
printf("%s %s" % (datetime.datetime.now(), info))
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
def sleep(self, sec):
|
|
||||||
|
|
||||||
printf("%s sleep %d seconds" % (datetime.datetime.now(), sec))
|
|
||||||
|
|
||||||
time.sleep(sec)
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
def debug(self, err):
|
|
||||||
|
|
||||||
printf("\\033[1;36m%s %s\\033[0m" % (datetime.datetime.now(), err))
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
def success(self, info):
|
|
||||||
|
|
||||||
printf("\\033[1;32m%s %s\\033[0m" % (datetime.datetime.now(), info))
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
def notice(self, err):
|
|
||||||
|
|
||||||
printf("\\033[1;33m%s %s\\033[0m" % (datetime.datetime.now(), err))
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
def exit(self, err):
|
|
||||||
|
|
||||||
printf("\\033[1;31m%s %s\\033[0m" % (datetime.datetime.now(), err))
|
|
||||||
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
def printNoPrefix(self, info):
|
|
||||||
|
|
||||||
printf("\\033[1;36m%s\\033[0m" % (info)
|
|
||||||
|
|
||||||
...
|
|
||||||
|
|
||||||
**2.5 What util/sql.py does in detail:**
|
|
||||||
|
|
||||||
SQL.py is mainly used to execute SQL statements to manipulate the database,
|
|
||||||
and the code is extracted and commented as follows:
|
|
||||||
|
|
||||||
**util/sql.py**
|
|
||||||
|
|
||||||
\# prepare() is mainly used to set up the environment for testing table and
|
|
||||||
data, and to set up the database db for testing. do not call prepare() if you
|
|
||||||
need to test the database operation command.
|
|
||||||
|
|
||||||
def prepare(self):
|
|
||||||
|
|
||||||
tdLog.info("prepare database:db")
|
|
||||||
|
|
||||||
self.cursor.execute('reset query cache')
|
|
||||||
|
|
||||||
self.cursor.execute('drop database if exists db')
|
|
||||||
|
|
||||||
self.cursor.execute('create database db')
|
|
||||||
|
|
||||||
self.cursor.execute('use db')
|
|
||||||
|
|
||||||
...
|
|
||||||
|
|
||||||
\# query() is mainly used to execute select statements for normal syntax input
|
|
||||||
|
|
||||||
def query(self, sql):
|
|
||||||
|
|
||||||
...
|
|
||||||
|
|
||||||
\# error() is mainly used to execute the select statement with the wrong syntax
|
|
||||||
input, the error will be caught as a reasonable behavior, if not caught it will
|
|
||||||
prove that the test failed
|
|
||||||
|
|
||||||
def error()
|
|
||||||
|
|
||||||
...
|
|
||||||
|
|
||||||
\# checkRows() is used to check the number of returned lines after calling
|
|
||||||
query(select ...) after calling the query(select ...) to check the number of
|
|
||||||
rows of returned results.
|
|
||||||
|
|
||||||
def checkRows(self, expectRows):
|
|
||||||
|
|
||||||
...
|
|
||||||
|
|
||||||
\# checkData() is used to check the returned result data after calling
|
|
||||||
query(select ...) after the query(select ...) is called, failure to meet
|
|
||||||
expectation is
|
|
||||||
|
|
||||||
def checkData(self, row, col, data):
|
|
||||||
|
|
||||||
...
|
|
||||||
|
|
||||||
\# getData() returns the result data after calling query(select ...) to return
|
|
||||||
the resulting data after calling query(select ...)
|
|
||||||
|
|
||||||
def getData(self, row, col):
|
|
||||||
|
|
||||||
...
|
|
||||||
|
|
||||||
\# execute() used to execute sql and return the number of affected rows
|
|
||||||
|
|
||||||
def execute(self, sql):
|
|
||||||
|
|
||||||
...
|
|
||||||
|
|
||||||
\# executeTimes() Multiple executions of the same sql statement
|
|
||||||
|
|
||||||
def executeTimes(self, sql, times):
|
|
||||||
|
|
||||||
...
|
|
||||||
|
|
||||||
\# CheckAffectedRows() Check if the number of affected rows is as expected
|
|
||||||
|
|
||||||
def checkAffectedRows(self, expectAffectedRows):
|
|
||||||
|
|
||||||
...
|
|
||||||
|
|
||||||
> Note: Both Python2 and Python3 are currently supported by the Python test
|
|
||||||
> case. Since Python2 is no longer officially supported by January 1, 2020, it
|
|
||||||
> is recommended that subsequent test case development be guaranteed to run
|
|
||||||
> correctly on Python3. For Python2, please consider being compatible if
|
|
||||||
> appropriate without additional
|
|
||||||
> burden. <https://nakedsecurity.sophos.com/2020/01/03/python-is-dead-long-live-python/>
|
|
||||||
|
|
||||||
### CI Covenant submission adoption principle.
|
|
||||||
|
|
||||||
- Every commit / PR compilation must pass. Currently, the warning is treated
|
|
||||||
as an error, so the warning must also be resolved.
|
|
||||||
|
|
||||||
- Test cases that already exist must pass.
|
|
||||||
|
|
||||||
- Because CI is very important to support build and automatically test
|
|
||||||
procedure, it is necessary to manually test the test case before adding it
|
|
||||||
and do as many iterations as possible to ensure that the test case provides
|
|
||||||
stable and reliable test results when added.
|
|
||||||
|
|
||||||
> Note: In the future, according to the requirements and test development
|
|
||||||
> progress will add stress testing, performance testing, code style,
|
|
||||||
> and other features based on functional testing.
|
|
||||||
|
|
||||||
### Third Party Connectors
|
### Third Party Connectors
|
||||||
|
|
||||||
The TDengine community has also kindly built some of their own connectors! Follow the links below to find the source code for them.
|
The TDengine community has also kindly built some of their own connectors! Follow the links below to find the source code for them.
|
||||||
|
|
@ -367,6 +122,10 @@ The TDengine community has also kindly built some of their own connectors! Follo
|
||||||
- [Rust Connector](https://github.com/taosdata/TDengine/tree/master/tests/examples/rust)
|
- [Rust Connector](https://github.com/taosdata/TDengine/tree/master/tests/examples/rust)
|
||||||
- [.Net Core Connector](https://github.com/maikebing/Maikebing.EntityFrameworkCore.Taos)
|
- [.Net Core Connector](https://github.com/maikebing/Maikebing.EntityFrameworkCore.Taos)
|
||||||
|
|
||||||
|
# How to run the test cases and how to add a new test case?
|
||||||
|
TDengine's test framework and all test cases are fully open source.
|
||||||
|
Please refer to [this document](tests/How-To-Run-Test-And-How-To-Add-New-Test-Case.md) for how to run test and develop new test case.
|
||||||
|
|
||||||
# TDengine Roadmap
|
# TDengine Roadmap
|
||||||
- Support event-driven stream computing
|
- Support event-driven stream computing
|
||||||
- Support user defined functions
|
- Support user defined functions
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,235 @@
|
||||||
|
### Prepare development environment
|
||||||
|
|
||||||
|
1. sudo apt install
|
||||||
|
build-essential cmake net-tools python-pip python-setuptools python3-pip
|
||||||
|
python3-setuptools valgrind psmisc curl
|
||||||
|
|
||||||
|
2. git clone <https://github.com/taosdata/TDengine>; cd TDengine
|
||||||
|
|
||||||
|
3. mkdir debug; cd debug; cmake ..; make ; sudo make install
|
||||||
|
|
||||||
|
4. pip install src/connector/python/linux/python2 ; pip3 install
|
||||||
|
src/connector/python/linux/python3
|
||||||
|
|
||||||
|
### How to run Python test suite
|
||||||
|
|
||||||
|
1. cd \<TDengine\>/tests/pytest
|
||||||
|
|
||||||
|
2. ./smoketest.sh \# for smoke test
|
||||||
|
|
||||||
|
3. ./smoketest.sh -g \# for memory leak detection test with valgrind
|
||||||
|
|
||||||
|
4. ./fulltest.sh \# for full test
|
||||||
|
|
||||||
|
> Note1: TDengine daemon's configuration and data files are stored in
|
||||||
|
> \<TDengine\>/sim directory. As a historical design, it's same place with
|
||||||
|
> TSIM script. So after the TSIM script ran with sudo privilege, the directory
|
||||||
|
> has been used by TSIM then the python script cannot write it by a normal
|
||||||
|
> user. You need to remove the directory completely first before running the
|
||||||
|
> Python test case. We should consider using two different locations to store
|
||||||
|
> for TSIM and Python script.
|
||||||
|
|
||||||
|
> Note2: if you need to debug crash problem with a core dump, you need
|
||||||
|
> manually edit smoketest.sh or fulltest.sh to add "ulimit -c unlimited"
|
||||||
|
> before the script line. Then you can look for the core file in
|
||||||
|
> \<TDengine\>/tests/pytest after the program crash.
|
||||||
|
|
||||||
|
|
||||||
|
### How to add a new test case
|
||||||
|
|
||||||
|
**1. TSIM test cases:**
|
||||||
|
|
||||||
|
TSIM was the testing framework has been used internally. Now it still be used to run the test cases we develop in the past as a legacy system. We are turning to use Python to develop new test case and are abandoning TSIM gradually.
|
||||||
|
|
||||||
|
**2. Python test cases:**
|
||||||
|
|
||||||
|
**2.1 Please refer to \<TDengine\>/tests/pytest/insert/basic.py to add a new
|
||||||
|
test case.** The new test case must implement 3 functions, where self.init()
|
||||||
|
and self.stop() simply copy the contents of insert/basic.py and the test
|
||||||
|
logic is implemented in self.run(). You can refer to the code in the util
|
||||||
|
directory for more information.
|
||||||
|
|
||||||
|
**2.2 Edit smoketest.sh to add the path and filename of the new test case**
|
||||||
|
|
||||||
|
Note: The Python test framework may continue to be improved in the future,
|
||||||
|
hopefully, to provide more functionality and ease of writing test cases. The
|
||||||
|
method of writing the test case above does not exclude that it will also be
|
||||||
|
affected.
|
||||||
|
|
||||||
|
**2.3 What test.py does in detail:**
|
||||||
|
|
||||||
|
test.py is the entry program for test case execution and monitoring.
|
||||||
|
|
||||||
|
test.py has the following functions.
|
||||||
|
|
||||||
|
\-f --file, Specifies the test case file name to be executed
|
||||||
|
-p --path, Specifies deployment path
|
||||||
|
|
||||||
|
\-m --master, Specifies the master server IP for cluster deployment
|
||||||
|
-c--cluster, test cluster function
|
||||||
|
-s--stop, terminates all running nodes
|
||||||
|
|
||||||
|
\-g--valgrind, load valgrind for memory leak detection test
|
||||||
|
|
||||||
|
\-h--help, display help
|
||||||
|
|
||||||
|
**2.4 What util/log.py does in detail:**
|
||||||
|
|
||||||
|
log.py is quite simple, the main thing is that you can print the output in
|
||||||
|
different colors as needed. The success() should be called for successful
|
||||||
|
test case execution and the success() will print green text. The exit() will
|
||||||
|
print red text and exit the program, exit() should be called for test
|
||||||
|
failure.
|
||||||
|
|
||||||
|
**util/log.py**
|
||||||
|
|
||||||
|
...
|
||||||
|
|
||||||
|
def info(self, info):
|
||||||
|
|
||||||
|
printf("%s %s" % (datetime.datetime.now(), info))
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
def sleep(self, sec):
|
||||||
|
|
||||||
|
printf("%s sleep %d seconds" % (datetime.datetime.now(), sec))
|
||||||
|
|
||||||
|
time.sleep(sec)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
def debug(self, err):
|
||||||
|
|
||||||
|
printf("\\033[1;36m%s %s\\033[0m" % (datetime.datetime.now(), err))
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
def success(self, info):
|
||||||
|
|
||||||
|
printf("\\033[1;32m%s %s\\033[0m" % (datetime.datetime.now(), info))
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
def notice(self, err):
|
||||||
|
|
||||||
|
printf("\\033[1;33m%s %s\\033[0m" % (datetime.datetime.now(), err))
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
def exit(self, err):
|
||||||
|
|
||||||
|
printf("\\033[1;31m%s %s\\033[0m" % (datetime.datetime.now(), err))
|
||||||
|
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
def printNoPrefix(self, info):
|
||||||
|
|
||||||
|
printf("\\033[1;36m%s\\033[0m" % (info)
|
||||||
|
|
||||||
|
...
|
||||||
|
|
||||||
|
**2.5 What util/sql.py does in detail:**
|
||||||
|
|
||||||
|
SQL.py is mainly used to execute SQL statements to manipulate the database,
|
||||||
|
and the code is extracted and commented as follows:
|
||||||
|
|
||||||
|
**util/sql.py**
|
||||||
|
|
||||||
|
\# prepare() is mainly used to set up the environment for testing table and
|
||||||
|
data, and to set up the database db for testing. do not call prepare() if you
|
||||||
|
need to test the database operation command.
|
||||||
|
|
||||||
|
def prepare(self):
|
||||||
|
|
||||||
|
tdLog.info("prepare database:db")
|
||||||
|
|
||||||
|
self.cursor.execute('reset query cache')
|
||||||
|
|
||||||
|
self.cursor.execute('drop database if exists db')
|
||||||
|
|
||||||
|
self.cursor.execute('create database db')
|
||||||
|
|
||||||
|
self.cursor.execute('use db')
|
||||||
|
|
||||||
|
...
|
||||||
|
|
||||||
|
\# query() is mainly used to execute select statements for normal syntax input
|
||||||
|
|
||||||
|
def query(self, sql):
|
||||||
|
|
||||||
|
...
|
||||||
|
|
||||||
|
\# error() is mainly used to execute the select statement with the wrong syntax
|
||||||
|
input, the error will be caught as a reasonable behavior, if not caught it will
|
||||||
|
prove that the test failed
|
||||||
|
|
||||||
|
def error()
|
||||||
|
|
||||||
|
...
|
||||||
|
|
||||||
|
\# checkRows() is used to check the number of returned lines after calling
|
||||||
|
query(select ...) after calling the query(select ...) to check the number of
|
||||||
|
rows of returned results.
|
||||||
|
|
||||||
|
def checkRows(self, expectRows):
|
||||||
|
|
||||||
|
...
|
||||||
|
|
||||||
|
\# checkData() is used to check the returned result data after calling
|
||||||
|
query(select ...) after the query(select ...) is called, failure to meet
|
||||||
|
expectation is
|
||||||
|
|
||||||
|
def checkData(self, row, col, data):
|
||||||
|
|
||||||
|
...
|
||||||
|
|
||||||
|
\# getData() returns the result data after calling query(select ...) to return
|
||||||
|
the resulting data after calling query(select ...)
|
||||||
|
|
||||||
|
def getData(self, row, col):
|
||||||
|
|
||||||
|
...
|
||||||
|
|
||||||
|
\# execute() used to execute sql and return the number of affected rows
|
||||||
|
|
||||||
|
def execute(self, sql):
|
||||||
|
|
||||||
|
...
|
||||||
|
|
||||||
|
\# executeTimes() Multiple executions of the same sql statement
|
||||||
|
|
||||||
|
def executeTimes(self, sql, times):
|
||||||
|
|
||||||
|
...
|
||||||
|
|
||||||
|
\# CheckAffectedRows() Check if the number of affected rows is as expected
|
||||||
|
|
||||||
|
def checkAffectedRows(self, expectAffectedRows):
|
||||||
|
|
||||||
|
...
|
||||||
|
|
||||||
|
> Note: Both Python2 and Python3 are currently supported by the Python test
|
||||||
|
> case. Since Python2 is no longer officially supported by January 1, 2020, it
|
||||||
|
> is recommended that subsequent test case development be guaranteed to run
|
||||||
|
> correctly on Python3. For Python2, please consider being compatible if
|
||||||
|
> appropriate without additional
|
||||||
|
> burden. <https://nakedsecurity.sophos.com/2020/01/03/python-is-dead-long-live-python/>
|
||||||
|
|
||||||
|
### CI submission adoption principle.
|
||||||
|
|
||||||
|
- Every commit / PR compilation must pass. Currently, the warning is treated
|
||||||
|
as an error, so the warning must also be resolved.
|
||||||
|
|
||||||
|
- Test cases that already exist must pass.
|
||||||
|
|
||||||
|
- Because CI is very important to support build and automatically test
|
||||||
|
procedure, it is necessary to manually test the test case before adding it
|
||||||
|
and do as many iterations as possible to ensure that the test case provides
|
||||||
|
stable and reliable test results when added.
|
||||||
|
|
||||||
|
> Note: In the future, according to the requirements and test development
|
||||||
|
> progress will add stress testing, performance testing, code style,
|
||||||
|
> and other features based on functional testing.
|
||||||
Loading…
Reference in New Issue