xxq250
/
homework-jianmu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

[td-225]merge developpp

This commit is contained in:
Haojun Liao 2021-06-21 18:55:48 +08:00
parent 1dd0c56d87 b7fded8b4f
commit 2eb86a90ac
34 changed files with 1921 additions and 789 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
documentation20/cn/08.connector/01.java/docs.md
View File

@ -49,6 +49,7 @@ TDengine 的 JDBC 驱动实现尽可能与关系型数据库驱动保持一致
</tr> </tr>
</table> </table>
注意：与 JNI 方式不同，RESTful 接口是无状态的，因此 `USE db_name` 指令没有效果，RESTful 下所有对表名、超级表名的引用都需要指定数据库名前缀。
## 如何获取 taos-jdbcdriver ## 如何获取 taos-jdbcdriver

4
documentation20/cn/08.connector/docs.md
View File

@ -585,7 +585,9 @@ conn.close()
## <a class="anchor" id="restful"></a>RESTful Connector ## <a class="anchor" id="restful"></a>RESTful Connector
为支持各种不同类型平台的开发，TDengine提供符合REST设计标准的API，即RESTful API。为最大程度降低学习成本，不同于其他数据库RESTful API的设计方法，TDengine直接通过HTTP POST 请求BODY中包含的SQL语句来操作数据库，仅需要一个URL。RESTful连接器的使用参见[视频教程](https://www.taosdata.com/blog/2020/11/11/1965.html)。 为支持各种不同类型平台的开发，TDengine 提供符合 REST 设计标准的 API，即 RESTful API。为最大程度降低学习成本，不同于其他数据库 RESTful API 的设计方法，TDengine 直接通过 HTTP POST 请求 BODY 中包含的 SQL 语句来操作数据库，仅需要一个 URL。RESTful 连接器的使用参见[视频教程](https://www.taosdata.com/blog/2020/11/11/1965.html)。
注意：与标准连接器的一个区别是，RESTful 接口是无状态的，因此 `USE db_name` 指令没有效果，所有对表名、超级表名的引用都需要指定数据库名前缀。
### HTTP请求格式 ### HTTP请求格式

29
documentation20/cn/09.connections/docs.md
View File

@ -77,48 +77,43 @@ sudo cp -rf /usr/local/taos/connector/grafanaplugin /var/lib/grafana/plugins/tde
## <a class="anchor" id="matlab"></a>MATLAB ## <a class="anchor" id="matlab"></a>MATLAB
MATLAB可以通过安装包内提供的JDBC Driver直接连接到TDengine获取数据到本地工作空间。 MATLAB 可以通过安装包内提供的 JDBC Driver 直接连接到 TDengine 获取数据到本地工作空间。
### MATLAB的JDBC接口适配 ### MATLAB 的 JDBC 接口适配
MATLAB的适配有下面几个步骤，下面以Windows10上适配MATLAB2017a为例： MATLAB 的适配有下面几个步骤，下面以 Windows 10 上适配 MATLAB2021a 为例：
- 将TDengine安装包内的驱动程序JDBCDriver-1.0.0-dist.jar拷贝到${matlab_root}\MATLAB\R2017a\java\jar\toolbox - 将 TDengine 客户端安装路径下的 `\TDengine\connector\jdbc的驱动程序taos-jdbcdriver-2.0.25-dist.jar` 拷贝到 `${matlab_root}\MATLAB\R2021a\java\jar\toolbox`。
- 将TDengine安装包内的taos.lib文件拷贝至${matlab_ root _dir}\MATLAB\R2017a\lib\win64 - 将 TDengine 安装包内的 `taos.lib` 文件拷贝至 `${matlab_root_dir}\MATLAB\R2021\lib\win64`。
- 将新添加的驱动jar包加入MATLAB的classpath。在${matlab_ root _dir}\MATLAB\R2017a\toolbox\local\classpath.txt文件中添加下面一行 - 将新添加的驱动 jar 包加入 MATLAB 的 classpath。在 `${matlab_root_dir}\MATLAB\R2021a\toolbox\local\classpath.txt` 文件中添加下面一行：
​
``` ```
$matlabroot/java/jar/toolbox/JDBCDriver-1.0.0-dist.jar $matlabroot/java/jar/toolbox/taos-jdbcdriver-2.0.25-dist.jar
``` ```
- 在${user_home}\AppData\Roaming\MathWorks\MATLAB\R2017a\下添加一个文件javalibrarypath.txt, 并在该文件中添加taos.dll的路径，比如您的taos.dll是在安装时拷贝到了C:\Windows\System32下，那么就应该在javalibrarypath.txt中添加如下一行： - 在 `${user_home}\AppData\Roaming\MathWorks\MATLAB\R2021a\` 下添加一个文件 `javalibrarypath.txt`，并在该文件中添加 taos.dll 的路径，比如您的 taos.dll 是在安装时拷贝到了 `C:\Windows\System32` 下，那么就应该在 `javalibrarypath.txt` 中添加如下一行：
​
``` ```
C:\Windows\System32 C:\Windows\System32
``` ```
### 在MATLAB中连接TDengine获取数据 ### 在 MATLAB 中连接 TDengine 获取数据
在成功进行了上述配置后，打开MATLAB。 在成功进行了上述配置后，打开 MATLAB。
- 创建一个连接： - 创建一个连接：
```matlab ```matlab
conn = database(‘db’, ‘root’, ‘taosdata’, ‘com.taosdata.jdbc.TSDBDriver’, ‘jdbc:TSDB://127.0.0.1:0/’) conn = database(‘test’, ‘root’, ‘taosdata’, ‘com.taosdata.jdbc.TSDBDriver’, ‘jdbc:TSDB://192.168.1.94:6030/’)
``` ```
- 执行一次查询： - 执行一次查询：
```matlab ```matlab
sql0 = [‘select * from tb’] sql0 = [‘select * from tb’]
data = select(conn, sql0); data = select(conn, sql0);
``` ```
- 插入一条记录: - 插入一条记录:
```matlab ```matlab
sql1 = [‘insert into tb values (now, 1)’] sql1 = [‘insert into tb values (now, 1)’]
exec(conn, sql1) exec(conn, sql1)
``` ```
更多例子细节请参考安装包内examples\Matlab\TDengineDemo.m文件。 更多例子细节请参考安装包内 `examples\Matlab\TDengineDemo.m` 文件。
## <a class="anchor" id="r"></a>R ## <a class="anchor" id="r"></a>R

24
documentation20/cn/12.taos-sql/docs.md
View File

@ -144,7 +144,7 @@ TDengine 缺省的时间戳是毫秒精度，但通过在 CREATE DATABASE 时传
``` ```
UPDATE 参数控制是否允许更新数据。缺省值为 0，取值范围为 [0, 1]。0 表示会直接丢弃后写入的相同时间戳的数据；1 表示会使用后写入的数据覆盖已有的相同时间戳的数据。 UPDATE 参数控制是否允许更新数据。缺省值为 0，取值范围为 [0, 1]。0 表示会直接丢弃后写入的相同时间戳的数据；1 表示会使用后写入的数据覆盖已有的相同时间戳的数据。
**Tips**: 以上所有参数修改后都可以用show databases来确认是否修改成功。另外，从 2.1.1.0 版本开始，修改这些参数后无需重启服务器即可生效。 **Tips**: 以上所有参数修改后都可以用show databases来确认是否修改成功。另外，从 2.1.3.0 版本开始，修改这些参数后无需重启服务器即可生效。
- **显示系统所有数据库** - **显示系统所有数据库**
@ -263,6 +263,14 @@ TDengine 缺省的时间戳是毫秒精度，但通过在 CREATE DATABASE 时传
``` ```
如果表是通过超级表创建，更改表结构的操作只能对超级表进行。同时针对超级表的结构更改对所有通过该结构创建的表生效。对于不是通过超级表创建的表，可以直接修改表结构。 如果表是通过超级表创建，更改表结构的操作只能对超级表进行。同时针对超级表的结构更改对所有通过该结构创建的表生效。对于不是通过超级表创建的表，可以直接修改表结构。
- **表修改列宽**
```mysql
ALTER TABLE tb_name MODIFY COLUMN field_name data_type(length);
```
如果数据列的类型是可变长格式（BINARY 或 NCHAR），那么可以使用此指令修改其宽度（只能改大，不能改小）。（2.1.3.0 版本新增）
如果表是通过超级表创建，更改表结构的操作只能对超级表进行。同时针对超级表的结构更改对所有通过该结构创建的表生效。对于不是通过超级表创建的表，可以直接修改表结构。
## <a class="anchor" id="super-table"></a>超级表STable管理 ## <a class="anchor" id="super-table"></a>超级表STable管理
注意：在 2.0.15.0 及以后的版本中，开始支持 STABLE 保留字。也即，在本节后文的指令说明中，CREATE、DROP、ALTER 三个指令在老版本中保留字需写作 TABLE 而不是 STABLE。 注意：在 2.0.15.0 及以后的版本中，开始支持 STABLE 保留字。也即，在本节后文的指令说明中，CREATE、DROP、ALTER 三个指令在老版本中保留字需写作 TABLE 而不是 STABLE。
@ -323,6 +331,13 @@ TDengine 缺省的时间戳是毫秒精度，但通过在 CREATE DATABASE 时传
ALTER STABLE stb_name DROP COLUMN field_name; ALTER STABLE stb_name DROP COLUMN field_name;
``` ```
- **超级表修改列宽**
```mysql
ALTER STABLE stb_name MODIFY COLUMN field_name data_type(length);
```
如果数据列的类型是可变长格式（BINARY 或 NCHAR），那么可以使用此指令修改其宽度（只能改大，不能改小）。（2.1.3.0 版本新增）
## <a class="anchor" id="tags"></a>超级表 STable 中 TAG 管理 ## <a class="anchor" id="tags"></a>超级表 STable 中 TAG 管理
- **添加标签** - **添加标签**
@ -346,6 +361,13 @@ TDengine 缺省的时间戳是毫秒精度，但通过在 CREATE DATABASE 时传
``` ```
修改超级表的标签名，从超级表修改某个标签名后，该超级表下的所有子表也会自动更新该标签名。 修改超级表的标签名，从超级表修改某个标签名后，该超级表下的所有子表也会自动更新该标签名。
- **修改标签列宽度**
```mysql
ALTER STABLE stb_name MODIFY TAG tag_name data_type(length);
```
如果标签的类型是可变长格式（BINARY 或 NCHAR），那么可以使用此指令修改其宽度（只能改大，不能改小）。（2.1.3.0 版本新增）
- **修改子表标签值** - **修改子表标签值**
```mysql ```mysql

245
packaging/check_package.sh Executable file
View File

@ -0,0 +1,245 @@
#!/bin/bash
#
# This file is used to install database on linux systems. The operating system
# is required to use systemd to manage services at boot
set -e
#set -x
verMode=edge
pagMode=full
iplist=""
serverFqdn=""
# -----------------------Variables definition---------------------
script_dir="../release"
# Dynamic directory
data_dir="/var/lib/taos"
log_dir="/var/log/taos"
data_link_dir="/usr/local/taos/data"
log_link_dir="/usr/local/taos/log"
cfg_install_dir="/etc/taos"
bin_link_dir="/usr/bin"
lib_link_dir="/usr/lib"
lib64_link_dir="/usr/lib64"
inc_link_dir="/usr/include"
#install main path
install_main_dir="/usr/local/taos"
# old bin dir
sbin_dir="/usr/local/taos/bin"
temp_version=""
fin_result=""
service_config_dir="/etc/systemd/system"
nginx_port=6060
nginx_dir="/usr/local/nginxd"
# Color setting
RED='\033[0;31m'
GREEN='\033[1;32m'
GREEN_DARK='\033[0;32m'
GREEN_UNDERLINE='\033[4;32m'
NC='\033[0m'
csudo=""
if command -v sudo > /dev/null; then
csudo="sudo"
fi
# ============================= get input parameters =================================================
# install.sh -v [server | client] -e [yes | no] -i [systemd | service | ...]
# set parameters by default value
interactiveFqdn=yes # [yes | no]
verType=server # [server | client]
initType=systemd # [systemd | service | ...]
while getopts "hv:d:" arg
do
case $arg in
d)
#echo "interactiveFqdn=$OPTARG"
script_dir=$( echo $OPTARG )
;;
h)
echo "Usage: `basename $0` -d scripy_path"
exit 0
;;
?) #unknow option
echo "unkonw argument"
exit 1
;;
esac
done
#echo "verType=${verType} interactiveFqdn=${interactiveFqdn}"
function kill_process() {
pid=$(ps -ef | grep "$1" | grep -v "grep" | awk '{print $2}')
if [ -n "$pid" ]; then
${csudo} kill -9 $pid || :
fi
}
function check_file() {
#check file whether exists
if [ ! -e $1/$2 ];then
echo -e "$1/$2 \033[31mnot exists\033[0m!quit"
fin_result=$fin_result"\033[31m$temp_version\033[0m test failed!\n"
echo -e $fin_result
exit 8
fi
}
function get_package_name() {
var=$1
if [[ $1 =~ 'aarch' ]];then
echo ${var::-21}
else
echo ${var::-17}
fi
}
function check_link() {
#check Link whether exists or broken
if [ -L $1 ] ; then
if [ ! -e $1 ] ; then
echo -e "$1 \033[31Broken link\033[0m"
fin_result=$fin_result"\033[31m$temp_version\033[0m test failed!\n"
echo -e $fin_result
exit 8
fi
else
echo -e "$1 \033[31mnot exists\033[0m!quit"
fin_result=$fin_result"\033[31m$temp_version\033[0m test failed!\n"
echo -e $fin_result
exit 8
fi
}
function check_main_path() {
#check install main dir and all sub dir
main_dir=("" "cfg" "bin" "connector" "driver" "examples" "include" "init.d")
for i in ${main_dir[@]};do
check_file ${install_main_dir} $i
done
if [ "$verMode" == "cluster" ]; then
nginx_main_dir=("admin" "conf" "html" "sbin" "logs")
for i in ${nginx_main_dir[@]};do
check_file ${nginx_dir} $i
done
fi
echo -e "Check main path:\033[32mOK\033[0m!"
}
function check_bin_path() {
# check install bin dir and all sub dir
bin_dir=("taos" "taosd" "taosdemo" "taosdump" "remove.sh" "tarbitrator" "set_core.sh")
for i in ${bin_dir[@]};do
check_file ${sbin_dir} $i
done
lbin_dir=("taos" "taosd" "taosdemo" "taosdump" "rmtaos" "tarbitrator" "set_core")
for i in ${lbin_dir[@]};do
check_link ${bin_link_dir}/$i
done
if [ "$verMode" == "cluster" ]; then
check_file ${nginx_dir}/sbin nginx
fi
echo -e "Check bin path:\033[32mOK\033[0m!"
}
function check_lib_path() {
# check all links
check_link ${lib_link_dir}/libtaos.so
check_link ${lib_link_dir}/libtaos.so.1
if [[ -d ${lib64_link_dir} ]]; then
check_link ${lib64_link_dir}/libtaos.so
check_link ${lib64_link_dir}/libtaos.so.1
fi
echo -e "Check lib path:\033[32mOK\033[0m!"
}
function check_header_path() {
# check all header
header_dir=("taos.h" "taoserror.h")
for i in ${header_dir[@]};do
check_link ${inc_link_dir}/$i
done
echo -e "Check bin path:\033[32mOK\033[0m!"
}
function check_config_dir() {
# check all config
check_file ${cfg_install_dir} taos.cfg
check_file ${install_main_dir}/cfg taos.cfg.org
echo -e "Check conf path:\033[32mOK\033[0m!"
}
function check_log_path() {
# check log path
check_file ${log_dir}
echo -e "Check log path:\033[32mOK\033[0m!"
}
function check_data_path() {
# check data path
check_file ${data_dir}
echo -e "Check data path:\033[32mOK\033[0m!"
}
function install_TDengine() {
cd ${script_dir}
tar zxf $1
temp_version=$(get_package_name $1)
cd $(get_package_name $1)
echo -e "\033[32muninstall TDengine && install TDengine...\033[0m"
rmtaos >/dev/null 2>&1 || echo 'taosd not installed' && echo -e '\n\n' |./install.sh >/dev/null 2>&1
echo -e "\033[32mTDengine has been installed!\033[0m"
echo -e "\033[32mTDengine is starting...\033[0m"
kill_process taos && systemctl start taosd && sleep 10
}
function test_TDengine() {
check_main_path
check_bin_path
check_lib_path
check_header_path
check_config_dir
check_log_path
check_data_path
result=`taos -s 'create database test ;create table test.tt(ts timestamp ,i int);insert into test.tt values(now,11);select * from test.tt' 2>&1 ||:`
if [[ $result =~ "Unable to establish" ]];then
echo -e "\033[31mTDengine connect failed\033[0m"
fin_result=$fin_result"\033[31m$temp_version\033[0m test failed!\n"
echo -e $fin_result
exit 8
fi
echo -e "Check TDengine connect:\033[32mOK\033[0m!"
fin_result=$fin_result"\033[32m$temp_version\033[0m test OK!\n"
}
# ## ==============================Main program starts from here============================
TD_package_name=`ls ${script_dir}/*server*gz |awk -F '/' '{print $NF}' `
temp=`pwd`
for i in $TD_package_name;do
if [[ $i =~ 'enterprise' ]];then
verMode="cluster"
else
verMode=""
fi
cd $temp
install_TDengine $i
test_TDengine
done
echo "============================================================"
echo -e $fin_result

143
src/client/src/tscSQLParser.c
View File

@ -63,7 +63,8 @@ static SExprInfo* doAddProjectCol(SQueryInfo* pQueryInfo, int32_t colIndex, int3
static int32_t setShowInfo(SSqlObj* pSql, SSqlInfo* pInfo); static int32_t setShowInfo(SSqlObj* pSql, SSqlInfo* pInfo);
static char* getAccountId(SSqlObj* pSql); static char* getAccountId(SSqlObj* pSql);
static bool serializeExprListToVariant(SArray* pList, tVariant **dest, int16_t colType); static int convertTimestampStrToInt64(tVariant *pVar, int32_t precision);
static bool serializeExprListToVariant(SArray* pList, tVariant **dest, int16_t colType, uint8_t precision);
static int32_t validateParamOfRelationIn(tVariant *pVar, int32_t colType); static int32_t validateParamOfRelationIn(tVariant *pVar, int32_t colType);
static bool has(SArray* pFieldList, int32_t startIdx, const char* name); static bool has(SArray* pFieldList, int32_t startIdx, const char* name);
@ -149,7 +150,7 @@ int16_t getNewResColId(SSqlCmd* pCmd) {
// serialize expr in exprlist to binary // serialize expr in exprlist to binary
// formate "type | size | value" // formate "type | size | value"
bool serializeExprListToVariant(SArray* pList, tVariant **dst, int16_t colType) { bool serializeExprListToVariant(SArray* pList, tVariant **dst, int16_t colType, uint8_t precision) {
bool ret = false; bool ret = false;
if (!pList || pList->size <= 0 || colType < 0) { if (!pList || pList->size <= 0 || colType < 0) {
return ret; return ret;
@ -159,14 +160,17 @@ bool serializeExprListToVariant(SArray* pList, tVariant **dst, int16_t colType)
int32_t firstTokenType = item->pNode->token.type; int32_t firstTokenType = item->pNode->token.type;
int32_t type = firstTokenType; int32_t type = firstTokenType;
//nchar to binary and //nchar to binary and other xxint to bigint
toTSDBType(type); toTSDBType(type);
if (colType != TSDB_DATA_TYPE_TIMESTAMP && !IS_UNSIGNED_NUMERIC_TYPE(colType)) {
if (type != colType && (type != TSDB_DATA_TYPE_BINARY || colType != TSDB_DATA_TYPE_NCHAR)) { if (type != colType && (type != TSDB_DATA_TYPE_BINARY || colType != TSDB_DATA_TYPE_NCHAR)) {
return false; return false;
} }
}
type = colType; type = colType;
SBufferWriter bw = tbufInitWriter( NULL, false ); SBufferWriter bw = tbufInitWriter( NULL, false);
tbufEnsureCapacity(&bw, 512); tbufEnsureCapacity(&bw, 512);
int32_t size = (int32_t)(pList->size); int32_t size = (int32_t)(pList->size);
@ -180,15 +184,22 @@ bool serializeExprListToVariant(SArray* pList, tVariant **dst, int16_t colType)
if (firstTokenType != pSub->token.type) { if (firstTokenType != pSub->token.type) {
break; break;
} }
toTSDBType(pSub->token.type); toTSDBType(pSub->token.type);
tVariant var; tVariant var;
tVariantCreate(&var, &pSub->token); tVariantCreate(&var, &pSub->token);
if (type == TSDB_DATA_TYPE_BOOL || type == TSDB_DATA_TYPE_TINYINT || type == TSDB_DATA_TYPE_SMALLINT if (type == TSDB_DATA_TYPE_BOOL || IS_SIGNED_NUMERIC_TYPE(type)) {
|| type == TSDB_DATA_TYPE_BIGINT || type == TSDB_DATA_TYPE_INT) {
tbufWriteInt64(&bw, var.i64); tbufWriteInt64(&bw, var.i64);
} else if (type == TSDB_DATA_TYPE_DOUBLE || type == TSDB_DATA_TYPE_FLOAT) { } else if (IS_UNSIGNED_NUMERIC_TYPE(type)) {
// ugly code, refactor later
if (IS_UNSIGNED_NUMERIC_TYPE(pSub->token.type) || IS_SIGNED_NUMERIC_TYPE(pSub->token.type)) {
tbufWriteUint64(&bw, var.i64);
} else {
tVariantDestroy(&var);
break;
}
}
else if (type == TSDB_DATA_TYPE_DOUBLE || type == TSDB_DATA_TYPE_FLOAT) {
tbufWriteDouble(&bw, var.dKey); tbufWriteDouble(&bw, var.dKey);
} else if (type == TSDB_DATA_TYPE_BINARY){ } else if (type == TSDB_DATA_TYPE_BINARY){
tbufWriteBinary(&bw, var.pz, var.nLen); tbufWriteBinary(&bw, var.pz, var.nLen);
@ -201,6 +212,16 @@ bool serializeExprListToVariant(SArray* pList, tVariant **dst, int16_t colType)
} }
tbufWriteBinary(&bw, buf, twcslen((wchar_t *)buf) * TSDB_NCHAR_SIZE); tbufWriteBinary(&bw, buf, twcslen((wchar_t *)buf) * TSDB_NCHAR_SIZE);
free(buf); free(buf);
} else if (type == TSDB_DATA_TYPE_TIMESTAMP) {
if (var.nType == TSDB_DATA_TYPE_BINARY) {
if (convertTimestampStrToInt64(&var, precision) < 0) {
tVariantDestroy(&var);
break;
}
tbufWriteInt64(&bw, var.i64);
} else if (var.nType == TSDB_DATA_TYPE_BIGINT) {
tbufWriteInt64(&bw, var.i64);
}
} }
tVariantDestroy(&var); tVariantDestroy(&var);
@ -286,29 +307,33 @@ static int32_t invalidOperationMsg(char* dstBuffer, const char* errMsg) {
return tscInvalidOperationMsg(dstBuffer, errMsg, NULL); return tscInvalidOperationMsg(dstBuffer, errMsg, NULL);
} }
static int setColumnFilterInfoForTimestamp(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tVariant* pVar) { static int convertTimestampStrToInt64(tVariant *pVar, int32_t precision) {
int64_t time = 0; int64_t time = 0;
const char* msg = "invalid timestamp";
strdequote(pVar->pz); strdequote(pVar->pz);
char* seg = strnchr(pVar->pz, '-', pVar->nLen, false); char* seg = strnchr(pVar->pz, '-', pVar->nLen, false);
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta);
if (seg != NULL) { if (seg != NULL) {
if (taosParseTime(pVar->pz, &time, pVar->nLen, tinfo.precision, tsDaylight) != TSDB_CODE_SUCCESS) { if (taosParseTime(pVar->pz, &time, pVar->nLen, precision, tsDaylight) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); return -1;
} }
} else { } else {
if (tVariantDump(pVar, (char*)&time, TSDB_DATA_TYPE_BIGINT, true)) { if (tVariantDump(pVar, (char*)&time, TSDB_DATA_TYPE_BIGINT, true)) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); return -1;
} }
} }
tVariantDestroy(pVar); tVariantDestroy(pVar);
tVariantCreateFromBinary(pVar, (char*)&time, 0, TSDB_DATA_TYPE_BIGINT); tVariantCreateFromBinary(pVar, (char*)&time, 0, TSDB_DATA_TYPE_BIGINT);
return 0;
}
static int setColumnFilterInfoForTimestamp(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tVariant* pVar) {
const char* msg = "invalid timestamp";
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta);
if (convertTimestampStrToInt64(pVar, tinfo.precision) < -1) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg);
}
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
@ -949,8 +974,10 @@ int32_t validateIntervalNode(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNode* pS
static int32_t validateStateWindowNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSqlNode, bool isStable) { static int32_t validateStateWindowNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSqlNode, bool isStable) {
const char* msg1 = "invalid column name"; const char* msg1 = "invalid column name";
const char* msg2 = "invalid column type";
const char* msg3 = "not support state_window with group by "; const char* msg3 = "not support state_window with group by ";
const char* msg4 = "function not support for super table query"; const char* msg4 = "function not support for super table query";
const char* msg5 = "not support state_window on tag column";
SStrToken *col = &(pSqlNode->windowstateVal.col) ; SStrToken *col = &(pSqlNode->windowstateVal.col) ;
if (col->z == NULL || col->n <= 0) { if (col->z == NULL || col->n <= 0) {
@ -978,8 +1005,10 @@ static int32_t validateStateWindowNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SS
STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex);
STableMeta* pTableMeta = pTableMetaInfo->pTableMeta; STableMeta* pTableMeta = pTableMetaInfo->pTableMeta;
int32_t numOfCols = tscGetNumOfColumns(pTableMeta); int32_t numOfCols = tscGetNumOfColumns(pTableMeta);
if (index.columnIndex == TSDB_TBNAME_COLUMN_INDEX || index.columnIndex >= numOfCols) { if (index.columnIndex == TSDB_TBNAME_COLUMN_INDEX) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3);
} else if (index.columnIndex >= numOfCols) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5);
} }
SGroupbyExpr* pGroupExpr = &pQueryInfo->groupbyExpr; SGroupbyExpr* pGroupExpr = &pQueryInfo->groupbyExpr;
@ -988,8 +1017,10 @@ static int32_t validateStateWindowNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SS
} }
SSchema* pSchema = tscGetTableColumnSchema(pTableMeta, index.columnIndex); SSchema* pSchema = tscGetTableColumnSchema(pTableMeta, index.columnIndex);
if (pSchema->type == TSDB_DATA_TYPE_TIMESTAMP || pSchema->type == TSDB_DATA_TYPE_FLOAT || pSchema->type == TSDB_DATA_TYPE_DOUBLE) { if (pSchema->type == TSDB_DATA_TYPE_TIMESTAMP || pSchema->type == TSDB_DATA_TYPE_FLOAT
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); || pSchema->type == TSDB_DATA_TYPE_DOUBLE || pSchema->type == TSDB_DATA_TYPE_NCHAR
|| pSchema->type == TSDB_DATA_TYPE_BINARY) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2);
} }
tscColumnListInsert(pQueryInfo->colList, index.columnIndex, pTableMeta->id.uid, pSchema); tscColumnListInsert(pQueryInfo->colList, index.columnIndex, pTableMeta->id.uid, pSchema);
@ -1276,7 +1307,7 @@ static bool validateTagParams(SArray* pTagsList, SArray* pFieldList, SSqlCmd* pC
const char* msg1 = "invalid number of tag columns"; const char* msg1 = "invalid number of tag columns";
const char* msg2 = "tag length too long"; const char* msg2 = "tag length too long";
const char* msg3 = "duplicated column names"; const char* msg3 = "duplicated column names";
const char* msg4 = "timestamp not allowed in tags"; //const char* msg4 = "timestamp not allowed in tags";
const char* msg5 = "invalid data type in tags"; const char* msg5 = "invalid data type in tags";
const char* msg6 = "invalid tag name"; const char* msg6 = "invalid tag name";
const char* msg7 = "invalid binary/nchar tag length"; const char* msg7 = "invalid binary/nchar tag length";
@ -1292,10 +1323,10 @@ static bool validateTagParams(SArray* pTagsList, SArray* pFieldList, SSqlCmd* pC
for (int32_t i = 0; i < numOfTags; ++i) { for (int32_t i = 0; i < numOfTags; ++i) {
TAOS_FIELD* p = taosArrayGet(pTagsList, i); TAOS_FIELD* p = taosArrayGet(pTagsList, i);
if (p->type == TSDB_DATA_TYPE_TIMESTAMP) { //if (p->type == TSDB_DATA_TYPE_TIMESTAMP) {
invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); // invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4);
return false; // return false;
} //}
if (!isValidDataType(p->type)) { if (!isValidDataType(p->type)) {
invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5);
@ -1353,7 +1384,7 @@ static bool validateTagParams(SArray* pTagsList, SArray* pFieldList, SSqlCmd* pC
* tags name /column name is truncated in sql.y * tags name /column name is truncated in sql.y
*/ */
bool validateOneTags(SSqlCmd* pCmd, TAOS_FIELD* pTagField) { bool validateOneTags(SSqlCmd* pCmd, TAOS_FIELD* pTagField) {
const char* msg1 = "timestamp not allowed in tags"; //const char* msg1 = "timestamp not allowed in tags";
const char* msg2 = "duplicated column names"; const char* msg2 = "duplicated column names";
const char* msg3 = "tag length too long"; const char* msg3 = "tag length too long";
const char* msg4 = "invalid tag name"; const char* msg4 = "invalid tag name";
@ -1376,10 +1407,10 @@ bool validateOneTags(SSqlCmd* pCmd, TAOS_FIELD* pTagField) {
} }
// no timestamp allowable // no timestamp allowable
if (pTagField->type == TSDB_DATA_TYPE_TIMESTAMP) { //if (pTagField->type == TSDB_DATA_TYPE_TIMESTAMP) {
invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); // invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
return false; // return false;
} //}
if ((pTagField->type < TSDB_DATA_TYPE_BOOL) || (pTagField->type > TSDB_DATA_TYPE_UBIGINT)) { if ((pTagField->type < TSDB_DATA_TYPE_BOOL) || (pTagField->type > TSDB_DATA_TYPE_UBIGINT)) {
invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg6); invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg6);
@ -3335,7 +3366,7 @@ static int32_t doExtractColumnFilterInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo,
// TK_GT,TK_GE,TK_EQ,TK_NE are based on the pColumn->lowerBndd // TK_GT,TK_GE,TK_EQ,TK_NE are based on the pColumn->lowerBndd
} else if (pExpr->tokenId == TK_IN) { } else if (pExpr->tokenId == TK_IN) {
tVariant *pVal; tVariant *pVal;
if (pRight->tokenId != TK_SET || !serializeExprListToVariant(pRight->pParam, &pVal, colType) || colType == TSDB_DATA_TYPE_TIMESTAMP) { if (pRight->tokenId != TK_SET || !serializeExprListToVariant(pRight->pParam, &pVal, colType, timePrecision)) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg);
} }
if (validateParamOfRelationIn(pVal, colType) != TSDB_CODE_SUCCESS) { if (validateParamOfRelationIn(pVal, colType) != TSDB_CODE_SUCCESS) {
@ -3347,7 +3378,6 @@ static int32_t doExtractColumnFilterInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo,
pColumnFilter->len = pVal->nLen; pColumnFilter->len = pVal->nLen;
pColumnFilter->filterstr = 1; pColumnFilter->filterstr = 1;
memcpy((char *)(pColumnFilter->pz), (char *)(pVal->pz), pVal->nLen); memcpy((char *)(pColumnFilter->pz), (char *)(pVal->pz), pVal->nLen);
//retVal = tVariantDump(pVal, (char *)(pColumnFilter->pz), TSDB_DATA_TYPE_BINARY, false);
tVariantDestroy(pVal); tVariantDestroy(pVal);
free(pVal); free(pVal);
@ -3480,6 +3510,7 @@ static int32_t extractColumnFilterInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SC
const char* msg1 = "non binary column not support like operator"; const char* msg1 = "non binary column not support like operator";
const char* msg2 = "binary column not support this operator"; const char* msg2 = "binary column not support this operator";
const char* msg3 = "bool column not support this operator"; const char* msg3 = "bool column not support this operator";
const char* msg4 = "primary key not support this operator";
SColumn* pColumn = tscColumnListInsert(pQueryInfo->colList, pIndex->columnIndex, pTableMeta->id.uid, pSchema); SColumn* pColumn = tscColumnListInsert(pQueryInfo->colList, pIndex->columnIndex, pTableMeta->id.uid, pSchema);
SColumnFilterInfo* pColFilter = NULL; SColumnFilterInfo* pColFilter = NULL;
@ -3536,6 +3567,9 @@ static int32_t extractColumnFilterInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SC
pColumn->columnIndex = pIndex->columnIndex; pColumn->columnIndex = pIndex->columnIndex;
pColumn->tableUid = pTableMeta->id.uid; pColumn->tableUid = pTableMeta->id.uid;
if (pColumn->columnIndex == PRIMARYKEY_TIMESTAMP_COL_INDEX && pExpr->tokenId == TK_IN) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4);
}
STableComInfo tinfo = tscGetTableInfo(pTableMeta); STableComInfo tinfo = tscGetTableInfo(pTableMeta);
return doExtractColumnFilterInfo(pCmd, pQueryInfo, tinfo.precision, pColFilter, pSchema->type, pExpr); return doExtractColumnFilterInfo(pCmd, pQueryInfo, tinfo.precision, pColFilter, pSchema->type, pExpr);
@ -6834,6 +6868,7 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) {
// too long tag values will return invalid sql, not be truncated automatically // too long tag values will return invalid sql, not be truncated automatically
SSchema *pTagSchema = tscGetTableTagSchema(pStableMetaInfo->pTableMeta); SSchema *pTagSchema = tscGetTableTagSchema(pStableMetaInfo->pTableMeta);
STableComInfo tinfo = tscGetTableInfo(pStableMetaInfo->pTableMeta);
STagData *pTag = &pCreateTableInfo->tagdata; STagData *pTag = &pCreateTableInfo->tagdata;
SKVRowBuilder kvRowBuilder = {0}; SKVRowBuilder kvRowBuilder = {0};
@ -6883,6 +6918,15 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) {
tdDestroyKVRowBuilder(&kvRowBuilder); tdDestroyKVRowBuilder(&kvRowBuilder);
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3);
} }
} else if (pSchema->type == TSDB_DATA_TYPE_TIMESTAMP) {
if (pItem->pVar.nType == TSDB_DATA_TYPE_BINARY) {
ret = convertTimestampStrToInt64(&(pItem->pVar), tinfo.precision);
if (ret != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4);
}
} else if (pItem->pVar.nType == TSDB_DATA_TYPE_TIMESTAMP) {
pItem->pVar.i64 = convertTimePrecision(pItem->pVar.i64, TSDB_TIME_PRECISION_NANO, tinfo.precision);
}
} }
ret = tVariantDump(&(pItem->pVar), tagVal, pSchema->type, true); ret = tVariantDump(&(pItem->pVar), tagVal, pSchema->type, true);
@ -6929,7 +6973,17 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) {
tdDestroyKVRowBuilder(&kvRowBuilder); tdDestroyKVRowBuilder(&kvRowBuilder);
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3);
} }
} else if (pSchema->type == TSDB_DATA_TYPE_TIMESTAMP) {
if (pItem->pVar.nType == TSDB_DATA_TYPE_BINARY) {
ret = convertTimestampStrToInt64(&(pItem->pVar), tinfo.precision);
if (ret != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4);
} }
} else if (pItem->pVar.nType == TSDB_DATA_TYPE_TIMESTAMP) {
pItem->pVar.i64 = convertTimePrecision(pItem->pVar.i64, TSDB_TIME_PRECISION_NANO, tinfo.precision);
}
}
ret = tVariantDump(&(pItem->pVar), tagVal, pSchema->type, true); ret = tVariantDump(&(pItem->pVar), tagVal, pSchema->type, true);
@ -8031,12 +8085,22 @@ int32_t exprTreeFromSqlExpr(SSqlCmd* pCmd, tExprNode **pExpr, const tSqlExpr* pS
assert(pSqlExpr->pRight == NULL); assert(pSqlExpr->pRight == NULL);
if (pSqlExpr->type == SQL_NODE_VALUE) { if (pSqlExpr->type == SQL_NODE_VALUE) {
int32_t ret = TSDB_CODE_SUCCESS;
*pExpr = calloc(1, sizeof(tExprNode)); *pExpr = calloc(1, sizeof(tExprNode));
(*pExpr)->nodeType = TSQL_NODE_VALUE; (*pExpr)->nodeType = TSQL_NODE_VALUE;
(*pExpr)->pVal = calloc(1, sizeof(tVariant)); (*pExpr)->pVal = calloc(1, sizeof(tVariant));
tVariantAssign((*pExpr)->pVal, &pSqlExpr->value); tVariantAssign((*pExpr)->pVal, &pSqlExpr->value);
return TSDB_CODE_SUCCESS;
STableMeta* pTableMeta = tscGetMetaInfo(pQueryInfo, 0)->pTableMeta;
if (pCols != NULL && taosArrayGetSize(pCols) > 0) {
SColIndex* idx = taosArrayGet(pCols, 0);
SSchema* pSchema = tscGetTableColumnSchema(pTableMeta, idx->colIndex);
// convert time by precision
if (pSchema != NULL && TSDB_DATA_TYPE_TIMESTAMP == pSchema->type && TSDB_DATA_TYPE_BINARY == (*pExpr)->pVal->nType) {
ret = setColumnFilterInfoForTimestamp(pCmd, pQueryInfo, (*pExpr)->pVal);
}
}
return ret;
} else if (pSqlExpr->type == SQL_NODE_SQLFUNCTION) { } else if (pSqlExpr->type == SQL_NODE_SQLFUNCTION) {
// arithmetic expression on the results of aggregation functions // arithmetic expression on the results of aggregation functions
*pExpr = calloc(1, sizeof(tExprNode)); *pExpr = calloc(1, sizeof(tExprNode));
@ -8093,21 +8157,22 @@ int32_t exprTreeFromSqlExpr(SSqlCmd* pCmd, tExprNode **pExpr, const tSqlExpr* pS
} else if (pSqlExpr->tokenId == TK_SET) { } else if (pSqlExpr->tokenId == TK_SET) {
int32_t colType = -1; int32_t colType = -1;
STableMeta* pTableMeta = tscGetMetaInfo(pQueryInfo, 0)->pTableMeta; STableMeta* pTableMeta = tscGetMetaInfo(pQueryInfo, 0)->pTableMeta;
if (pCols != NULL) { if (pCols != NULL && taosArrayGetSize(pCols) > 0) {
SColIndex* idx = taosArrayGet(pCols, 0); SColIndex* idx = taosArrayGet(pCols, 0);
SSchema* pSchema = tscGetTableColumnSchema(pTableMeta, idx->colIndex); SSchema* pSchema = tscGetTableColumnSchema(pTableMeta, idx->colIndex);
if (pSchema != NULL) { if (pSchema != NULL) {
colType = pSchema->type; colType = pSchema->type;
} }
} }
tVariant *pVal; tVariant *pVal;
if (colType >= TSDB_DATA_TYPE_TINYINT && colType <= TSDB_DATA_TYPE_BIGINT) { if (colType >= TSDB_DATA_TYPE_TINYINT && colType <= TSDB_DATA_TYPE_BIGINT) {
colType = TSDB_DATA_TYPE_BIGINT; colType = TSDB_DATA_TYPE_BIGINT;
} else if (colType == TSDB_DATA_TYPE_FLOAT || colType == TSDB_DATA_TYPE_DOUBLE) { } else if (colType == TSDB_DATA_TYPE_FLOAT || colType == TSDB_DATA_TYPE_DOUBLE) {
colType = TSDB_DATA_TYPE_DOUBLE; colType = TSDB_DATA_TYPE_DOUBLE;
} }
if (serializeExprListToVariant(pSqlExpr->pParam, &pVal, colType) == false) { STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta);
if (serializeExprListToVariant(pSqlExpr->pParam, &pVal, colType, tinfo.precision) == false) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), "not support filter expression"); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), "not support filter expression");
} }
*pExpr = calloc(1, sizeof(tExprNode)); *pExpr = calloc(1, sizeof(tExprNode));

9
src/common/src/texpr.c
View File

@ -476,7 +476,14 @@ void buildFilterSetFromBinary(void **q, const char *buf, int32_t len) {
int dummy = -1; int dummy = -1;
int32_t sz = tbufReadInt32(&br); int32_t sz = tbufReadInt32(&br);
for (int32_t i = 0; i < sz; i++) { for (int32_t i = 0; i < sz; i++) {
if (type == TSDB_DATA_TYPE_BOOL || type == TSDB_DATA_TYPE_TINYINT || type == TSDB_DATA_TYPE_SMALLINT || type == TSDB_DATA_TYPE_BIGINT || type == TSDB_DATA_TYPE_INT) { if (type == TSDB_DATA_TYPE_BOOL || IS_SIGNED_NUMERIC_TYPE(type)) {
int64_t val = tbufReadInt64(&br);
taosHashPut(pObj, (char *)&val, sizeof(val), &dummy, sizeof(dummy));
} else if (IS_UNSIGNED_NUMERIC_TYPE(type)) {
uint64_t val = tbufReadUint64(&br);
taosHashPut(pObj, (char *)&val, sizeof(val), &dummy, sizeof(dummy));
}
else if (type == TSDB_DATA_TYPE_TIMESTAMP) {
int64_t val = tbufReadInt64(&br); int64_t val = tbufReadInt64(&br);
taosHashPut(pObj, (char *)&val, sizeof(val), &dummy, sizeof(dummy)); taosHashPut(pObj, (char *)&val, sizeof(val), &dummy, sizeof(dummy));
} else if (type == TSDB_DATA_TYPE_DOUBLE || type == TSDB_DATA_TYPE_FLOAT) { } else if (type == TSDB_DATA_TYPE_DOUBLE || type == TSDB_DATA_TYPE_FLOAT) {

10
src/common/src/ttypes.c
View File

@ -632,7 +632,15 @@ int32_t tStrToInteger(const char* z, int16_t type, int32_t n, int64_t* value, bo
} }
// the string may be overflow according to errno // the string may be overflow according to errno
*value = issigned? strtoll(z, &endPtr, radix):strtoull(z, &endPtr, radix); if (!issigned) {
const char *p = z;
while(*p != 0 && *p == ' ') p++;
if (*p != 0 && *p == '-') { return -1;}
*value = strtoull(z, &endPtr, radix);
} else {
*value = strtoll(z, &endPtr, radix);
}
// not a valid integer number, return error // not a valid integer number, return error
if (endPtr - z != n || errno == ERANGE) { if (endPtr - z != n || errno == ERANGE) {

4
src/common/src/tvariant.c
View File

@ -77,6 +77,10 @@ void tVariantCreate(tVariant *pVar, SStrToken *token) {
pVar->nLen = strRmquote(pVar->pz, token->n); pVar->nLen = strRmquote(pVar->pz, token->n);
break; break;
} }
case TSDB_DATA_TYPE_TIMESTAMP: {
pVar->i64 = taosGetTimestamp(TSDB_TIME_PRECISION_NANO);
break;
}
default: { // nType == 0 means the null value default: { // nType == 0 means the null value
type = TSDB_DATA_TYPE_NULL; type = TSDB_DATA_TYPE_NULL;

5
src/connector/python/README.md
View File

@ -5,12 +5,13 @@
## Install ## Install
```sh ```sh
pip install git+https://github.com/taosdata/TDengine-connector-python git clone --depth 1 https://github.com/taosdata/TDengine.git
pip install ./TDengine/src/connector/python
``` ```
## Source Code ## Source Code
[TDengine] connector for Python source code is hosted on [GitHub](https://github.com/taosdata/TDengine-connector-python). [TDengine] connector for Python source code is hosted on [GitHub](https://github.com/taosdata/TDengine/tree/develop/src/connector/python).
## License - AGPL ## License - AGPL

2
src/connector/python/taos/cinterface.py
View File

@ -15,7 +15,7 @@ def _convert_microsecond_to_datetime(micro):
def _convert_nanosecond_to_datetime(nanosec): def _convert_nanosecond_to_datetime(nanosec):
return datetime.datetime.fromtimestamp(nanosec / 1000000000.0) return nanosec
def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, precision=FieldType.C_TIMESTAMP_UNKNOWN): def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, precision=FieldType.C_TIMESTAMP_UNKNOWN):

46
src/inc/ttokendef.h
View File

@ -128,29 +128,29 @@
#define TK_USING 110 #define TK_USING 110
#define TK_AS 111 #define TK_AS 111
#define TK_NULL 112 #define TK_NULL 112
#define TK_SELECT 113 #define TK_NOW 113
#define TK_UNION 114 #define TK_SELECT 114
#define TK_ALL 115 #define TK_UNION 115
#define TK_DISTINCT 116 #define TK_ALL 116
#define TK_FROM 117 #define TK_DISTINCT 117
#define TK_VARIABLE 118 #define TK_FROM 118
#define TK_INTERVAL 119 #define TK_VARIABLE 119
#define TK_SESSION 120 #define TK_INTERVAL 120
#define TK_STATE_WINDOW 121 #define TK_SESSION 121
#define TK_FILL 122 #define TK_STATE_WINDOW 122
#define TK_SLIDING 123 #define TK_FILL 123
#define TK_ORDER 124 #define TK_SLIDING 124
#define TK_BY 125 #define TK_ORDER 125
#define TK_ASC 126 #define TK_BY 126
#define TK_DESC 127 #define TK_ASC 127
#define TK_GROUP 128 #define TK_DESC 128
#define TK_HAVING 129 #define TK_GROUP 129
#define TK_LIMIT 130 #define TK_HAVING 130
#define TK_OFFSET 131 #define TK_LIMIT 131
#define TK_SLIMIT 132 #define TK_OFFSET 132
#define TK_SOFFSET 133 #define TK_SLIMIT 133
#define TK_WHERE 134 #define TK_SOFFSET 134
#define TK_NOW 135 #define TK_WHERE 135
#define TK_RESET 136 #define TK_RESET 136
#define TK_QUERY 137 #define TK_QUERY 137
#define TK_SYNCDB 138 #define TK_SYNCDB 138

16
src/os/inc/osFile.h
View File

@ -53,6 +53,22 @@ int64_t taosFSendFile(FILE *outfile, FILE *infile, int64_t *offset, int64_t size
void taosGetTmpfilePath(const char *fileNamePrefix, char *dstPath); void taosGetTmpfilePath(const char *fileNamePrefix, char *dstPath);
void taosClose(FileFd fd); void taosClose(FileFd fd);
#ifdef TAOS_RANDOM_FILE_FAIL
void taosSetRandomFileFailFactor(int32_t factor);
void taosSetRandomFileFailOutput(const char *path);
#ifdef TAOS_RANDOM_FILE_FAIL_TEST
int64_t taosReadFileRandomFail(int32_t fd, void *buf, int32_t count, const char *file, uint32_t line);
int64_t taosWriteFileRandomFail(int32_t fd, void *buf, int32_t count, const char *file, uint32_t line);
int64_t taosLSeekRandomFail(int32_t fd, int64_t offset, int32_t whence, const char *file, uint32_t line);
#undef taosRead
#undef taosWrite
#undef taosLSeek
#define taosRead(fd, buf, count) taosReadFileRandomFail(fd, buf, count, __FILE__, __LINE__)
#define taosWrite(fd, buf, count) taosWriteFileRandomFail(fd, buf, count, __FILE__, __LINE__)
#define taosLSeek(fd, offset, whence) taosLSeekRandomFail(fd, offset, whence, __FILE__, __LINE__)
#endif
#endif
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif

6
src/os/src/detail/osFail.c
View File

@ -113,7 +113,7 @@ int64_t taosReadFileRandomFail(int32_t fd, void *buf, int32_t count, const char
} }
} }
return taosReadImp(fd, buf, count); return taosRead(fd, buf, count);
} }
int64_t taosWriteFileRandomFail(int32_t fd, void *buf, int32_t count, const char *file, uint32_t line) { int64_t taosWriteFileRandomFail(int32_t fd, void *buf, int32_t count, const char *file, uint32_t line) {
@ -124,7 +124,7 @@ int64_t taosWriteFileRandomFail(int32_t fd, void *buf, int32_t count, const char
} }
} }
return taosWriteImp(fd, buf, count); return taosWrite(fd, buf, count);
} }
int64_t taosLSeekRandomFail(int32_t fd, int64_t offset, int32_t whence, const char *file, uint32_t line) { int64_t taosLSeekRandomFail(int32_t fd, int64_t offset, int32_t whence, const char *file, uint32_t line) {
@ -135,7 +135,7 @@ int64_t taosLSeekRandomFail(int32_t fd, int64_t offset, int32_t whence, const ch
} }
} }
return taosLSeekImp(fd, offset, whence); return taosLSeek(fd, offset, whence);
} }
#endif //TAOS_RANDOM_FILE_FAIL #endif //TAOS_RANDOM_FILE_FAIL

1
src/query/inc/sql.y
View File

@ -437,6 +437,7 @@ tagitem(A) ::= FLOAT(X). { toTSDBType(X.type); tVariantCreate(&A, &X); }
tagitem(A) ::= STRING(X). { toTSDBType(X.type); tVariantCreate(&A, &X); } tagitem(A) ::= STRING(X). { toTSDBType(X.type); tVariantCreate(&A, &X); }
tagitem(A) ::= BOOL(X). { toTSDBType(X.type); tVariantCreate(&A, &X); } tagitem(A) ::= BOOL(X). { toTSDBType(X.type); tVariantCreate(&A, &X); }
tagitem(A) ::= NULL(X). { X.type = 0; tVariantCreate(&A, &X); } tagitem(A) ::= NULL(X). { X.type = 0; tVariantCreate(&A, &X); }
tagitem(A) ::= NOW(X). { X.type = TSDB_DATA_TYPE_TIMESTAMP; tVariantCreate(&A, &X);}
tagitem(A) ::= MINUS(X) INTEGER(Y).{ tagitem(A) ::= MINUS(X) INTEGER(Y).{
X.n += Y.n; X.n += Y.n;

4
src/query/src/qExecutor.c
View File

@ -2914,7 +2914,9 @@ void setTagValue(SOperatorInfo* pOperatorInfo, void *pTable, SQLFunctionCtx* pCt
doSetTagValueInParam(pTable, pLocalExprInfo->base.colInfo.colId, &pCtx[idx].tag, pLocalExprInfo->base.resType, doSetTagValueInParam(pTable, pLocalExprInfo->base.colInfo.colId, &pCtx[idx].tag, pLocalExprInfo->base.resType,
pLocalExprInfo->base.resBytes); pLocalExprInfo->base.resBytes);
if (IS_NUMERIC_TYPE(pLocalExprInfo->base.resType) || pLocalExprInfo->base.resType == TSDB_DATA_TYPE_BOOL) { if (IS_NUMERIC_TYPE(pLocalExprInfo->base.resType)
|| pLocalExprInfo->base.resType == TSDB_DATA_TYPE_BOOL
|| pLocalExprInfo->base.resType == TSDB_DATA_TYPE_TIMESTAMP) {
memcpy(pRuntimeEnv->tagVal + offset, &pCtx[idx].tag.i64, pLocalExprInfo->base.resBytes); memcpy(pRuntimeEnv->tagVal + offset, &pCtx[idx].tag.i64, pLocalExprInfo->base.resBytes);
} else { } else {
memcpy(pRuntimeEnv->tagVal + offset, pCtx[idx].tag.pz, pCtx[idx].tag.nLen); memcpy(pRuntimeEnv->tagVal + offset, pCtx[idx].tag.pz, pCtx[idx].tag.nLen);

14
src/query/src/qFilterfunc.c
View File

@ -254,15 +254,23 @@ bool notNullOperator(SColumnFilterElem *pFilter, const char* minval, const char*
return true; return true;
} }
bool inOperator(SColumnFilterElem *pFilter, const char* minval, const char* maxval, int16_t type) { bool inOperator(SColumnFilterElem *pFilter, const char* minval, const char* maxval, int16_t type) {
if (type == TSDB_DATA_TYPE_BOOL || type == TSDB_DATA_TYPE_TINYINT || type == TSDB_DATA_TYPE_SMALLINT || type == TSDB_DATA_TYPE_BIGINT || type == TSDB_DATA_TYPE_INT) { if (type == TSDB_DATA_TYPE_BOOL || IS_SIGNED_NUMERIC_TYPE(type) || type == TSDB_DATA_TYPE_TIMESTAMP) {
int64_t minv = -1, maxv = -1; int64_t minv = -1, maxv = -1;
GET_TYPED_DATA(minv, int64_t, type, minval); GET_TYPED_DATA(minv, int64_t, type, minval);
GET_TYPED_DATA(maxv, int64_t, type, maxval); GET_TYPED_DATA(maxv, int64_t, type, maxval);
if (minv == maxv) { if (minv == maxv) {
return NULL != taosHashGet((SHashObj *)pFilter->q, (char *)&minv, sizeof(minv)); return NULL != taosHashGet((SHashObj *)pFilter->q, (char *)&minv, sizeof(minv));
} }
return true; return false;
} else if (type == TSDB_DATA_TYPE_DOUBLE || type == TSDB_DATA_TYPE_FLOAT) { } else if (IS_UNSIGNED_NUMERIC_TYPE(type)) {
uint64_t minv = 0, maxv = 0;
GET_TYPED_DATA(minv, uint64_t, type, minval);
GET_TYPED_DATA(maxv, uint64_t, type, maxval);
if (minv == maxv) {
return NULL != taosHashGet((SHashObj *)pFilter->q, (char *)&minv, sizeof(minv));
}
return false;
}else if (type == TSDB_DATA_TYPE_DOUBLE || type == TSDB_DATA_TYPE_FLOAT) {
double v; double v;
GET_TYPED_DATA(v, double, type, minval); GET_TYPED_DATA(v, double, type, minval);
return NULL != taosHashGet((SHashObj *)pFilter->q, (char *)&v, sizeof(v)); return NULL != taosHashGet((SHashObj *)pFilter->q, (char *)&v, sizeof(v));

1284
src/query/src/sql.c
View File

File diff suppressed because it is too large Load Diff

9
src/tsdb/src/tsdbRead.c
View File

@ -3355,11 +3355,16 @@ static bool tableFilterFp(const void* pNode, void* param) {
} }
} else if (pInfo->optr == TSDB_RELATION_IN) { } else if (pInfo->optr == TSDB_RELATION_IN) {
int type = pInfo->sch.type; int type = pInfo->sch.type;
if (type == TSDB_DATA_TYPE_BOOL || type == TSDB_DATA_TYPE_TINYINT || type == TSDB_DATA_TYPE_SMALLINT || type == TSDB_DATA_TYPE_BIGINT || type == TSDB_DATA_TYPE_INT) { if (type == TSDB_DATA_TYPE_BOOL || IS_SIGNED_NUMERIC_TYPE(type) || type == TSDB_DATA_TYPE_TIMESTAMP) {
int64_t v; int64_t v;
GET_TYPED_DATA(v, int64_t, pInfo->sch.type, val); GET_TYPED_DATA(v, int64_t, pInfo->sch.type, val);
return NULL != taosHashGet((SHashObj *)pInfo->q, (char *)&v, sizeof(v)); return NULL != taosHashGet((SHashObj *)pInfo->q, (char *)&v, sizeof(v));
} else if (type == TSDB_DATA_TYPE_DOUBLE || type == TSDB_DATA_TYPE_DOUBLE) { } else if (IS_UNSIGNED_NUMERIC_TYPE(type)) {
uint64_t v;
GET_TYPED_DATA(v, uint64_t, pInfo->sch.type, val);
return NULL != taosHashGet((SHashObj *)pInfo->q, (char *)&v, sizeof(v));
}
else if (type == TSDB_DATA_TYPE_DOUBLE || type == TSDB_DATA_TYPE_DOUBLE) {
double v; double v;
GET_TYPED_DATA(v, double, pInfo->sch.type, val); GET_TYPED_DATA(v, double, pInfo->sch.type, val);
return NULL != taosHashGet((SHashObj *)pInfo->q, (char *)&v, sizeof(v)); return NULL != taosHashGet((SHashObj *)pInfo->q, (char *)&v, sizeof(v));

54
tests/examples/JDBC/JDBCDemo/pom.xml
View File

@ -9,11 +9,15 @@
<version>SNAPSHOT</version> <version>SNAPSHOT</version>
<packaging>jar</packaging> <packaging>jar</packaging>
<properties>
<project.assembly.dir>src/main/resources/assembly</project.assembly.dir>
</properties>
<dependencies> <dependencies>
<dependency> <dependency>
<groupId>com.taosdata.jdbc</groupId> <groupId>com.taosdata.jdbc</groupId>
<artifactId>taos-jdbcdriver</artifactId> <artifactId>taos-jdbcdriver</artifactId>
<version>2.0.22</version> <version>2.0.30</version>
</dependency> </dependency>
</dependencies> </dependencies>
@ -22,20 +26,60 @@
<plugin> <plugin>
<groupId>org.apache.maven.plugins</groupId> <groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId> <artifactId>maven-assembly-plugin</artifactId>
<version>3.1.0</version> <version>3.3.0</version>
<executions>
<!-- jdbcDemo -->
<execution>
<id>JdbcDemo</id>
<configuration> <configuration>
<finalName>JdbcDemo</finalName>
<archive> <archive>
<manifest> <manifest>
<mainClass>com.taosdata.example.JDBCDemo</mainClass> <mainClass>com.taosdata.example.JdbcDemo</mainClass>
</manifest> </manifest>
</archive> </archive>
<descriptorRefs> <descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef> <descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs> </descriptorRefs>
</configuration> </configuration>
<executions> <phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
<execution> <execution>
<id>make-assembly</id> <id>JdbcRestfulDemo</id>
<configuration>
<finalName>JdbcRestfulDemo</finalName>
<archive>
<manifest>
<mainClass>com.taosdata.example.JdbcRestfulDemo</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
<execution>
<id>SubscribeDemo</id>
<configuration>
<finalName>SubscribeDemo</finalName>
<archive>
<manifest>
<mainClass>com.taosdata.example.SubscribeDemo</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<phase>package</phase> <phase>package</phase>
<goals> <goals>
<goal>single</goal> <goal>single</goal>

4
tests/examples/JDBC/JDBCDemo/readme.md
View File

@ -11,12 +11,12 @@ Download the tdengine package on our website: ``https://www.taosdata.com/cn/all-
## Run jdbcDemo using mvn plugin ## Run jdbcDemo using mvn plugin
run command: run command:
``` ```
mvn clean compile exec:java -Dexec.mainClass="com.taosdata.example.JDBCDemo" mvn clean compile exec:java -Dexec.mainClass="com.taosdata.example.JdbcDemo"
``` ```
and run with your customed args and run with your customed args
``` ```
mvn clean compile exec:java -Dexec.mainClass="com.taosdata.example.JDBCDemo" -Dexec.args="-host [HOSTNAME]" mvn clean compile exec:java -Dexec.mainClass="com.taosdata.example.JdbcDemo" -Dexec.args="-host [HOSTNAME]"
``` ```
## Compile the Demo Code and Run It ## Compile the Demo Code and Run It

4
tests/examples/JDBC/JDBCDemo/src/main/java/com/taosdata/example/JDBCDemo.java → tests/examples/JDBC/JDBCDemo/src/main/java/com/taosdata/example/JdbcDemo.java
View File

@ -3,7 +3,7 @@ package com.taosdata.example;
import java.sql.*; import java.sql.*;
import java.util.Properties; import java.util.Properties;
public class JDBCDemo { public class JdbcDemo {
private static String host; private static String host;
private static final String dbName = "test"; private static final String dbName = "test";
private static final String tbName = "weather"; private static final String tbName = "weather";
@ -17,7 +17,7 @@ public class JDBCDemo {
if (host == null) { if (host == null) {
printHelp(); printHelp();
} }
JDBCDemo demo = new JDBCDemo(); JdbcDemo demo = new JdbcDemo();
demo.init(); demo.init();
demo.createDatabase(); demo.createDatabase();
demo.useDatabase(); demo.useDatabase();

2
tests/examples/JDBC/JDBCDemo/src/main/java/com/taosdata/example/JdbcRestfulDemo.java
View File

@ -4,7 +4,7 @@ import java.sql.*;
import java.util.Properties; import java.util.Properties;
public class JdbcRestfulDemo { public class JdbcRestfulDemo {
private static final String host = "master"; private static final String host = "127.0.0.1";
public static void main(String[] args) { public static void main(String[] args) {
try { try {

205
tests/pytest/crash_gen/valgrind_taos.supp
View File

@ -17518,3 +17518,208 @@
fun:taosCheckGlobalCfg fun:taosCheckGlobalCfg
fun:taos_init_imp fun:taos_init_imp
} }
{
<insert_a_suppression_name_here>
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
obj:/usr/bin/python3.8
fun:PyTuple_Pack
fun:__pyx_pymod_exec_mtrand
fun:PyModule_ExecDef
obj:/usr/bin/python3.8
obj:/usr/bin/python3.8
fun:PyVectorcall_Call
fun:_PyEval_EvalFrameDefault
fun:_PyEval_EvalCodeWithName
fun:_PyFunction_Vectorcall
fun:_PyEval_EvalFrameDefault
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
fun:PyCode_NewWithPosOnlyArgs
fun:PyCode_New
fun:__Pyx_InitCachedConstants
fun:__pyx_pymod_exec__generator
fun:PyModule_ExecDef
obj:/usr/bin/python3.8
obj:/usr/bin/python3.8
fun:PyVectorcall_Call
fun:_PyEval_EvalFrameDefault
fun:_PyEval_EvalCodeWithName
fun:_PyFunction_Vectorcall
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
obj:/usr/bin/python3.8
fun:PyTuple_Pack
fun:__pyx_pymod_exec_bit_generator
fun:PyModule_ExecDef
obj:/usr/bin/python3.8
obj:/usr/bin/python3.8
fun:PyVectorcall_Call
fun:_PyEval_EvalFrameDefault
fun:_PyEval_EvalCodeWithName
fun:_PyFunction_Vectorcall
fun:_PyEval_EvalFrameDefault
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
obj:/usr/bin/python3.8
fun:PyTuple_Pack
fun:__pyx_pymod_exec__common
fun:PyModule_ExecDef
obj:/usr/bin/python3.8
obj:/usr/bin/python3.8
fun:PyVectorcall_Call
fun:_PyEval_EvalFrameDefault
fun:_PyEval_EvalCodeWithName
fun:_PyFunction_Vectorcall
fun:_PyEval_EvalFrameDefault
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
obj:/usr/bin/python3.8
fun:PyTuple_Pack
fun:__pyx_pymod_exec__bounded_integers
fun:PyModule_ExecDef
obj:/usr/bin/python3.8
obj:/usr/bin/python3.8
fun:PyVectorcall_Call
fun:_PyEval_EvalFrameDefault
fun:_PyEval_EvalCodeWithName
fun:_PyFunction_Vectorcall
fun:_PyEval_EvalFrameDefault
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
obj:/usr/bin/python3.8
fun:PyTuple_Pack
fun:__pyx_pymod_exec__mt19937
fun:PyModule_ExecDef
obj:/usr/bin/python3.8
obj:/usr/bin/python3.8
fun:PyVectorcall_Call
fun:_PyEval_EvalFrameDefault
fun:_PyEval_EvalCodeWithName
fun:_PyFunction_Vectorcall
fun:_PyEval_EvalFrameDefault
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
obj:/usr/bin/python3.8
fun:PyTuple_Pack
fun:__pyx_pymod_exec__philox
fun:PyModule_ExecDef
obj:/usr/bin/python3.8
obj:/usr/bin/python3.8
fun:PyVectorcall_Call
fun:_PyEval_EvalFrameDefault
fun:_PyEval_EvalCodeWithName
fun:_PyFunction_Vectorcall
fun:_PyEval_EvalFrameDefault
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
obj:/usr/bin/python3.8
fun:PyTuple_Pack
fun:__pyx_pymod_exec__pcg64
fun:PyModule_ExecDef
obj:/usr/bin/python3.8
obj:/usr/bin/python3.8
fun:PyVectorcall_Call
fun:_PyEval_EvalFrameDefault
fun:_PyEval_EvalCodeWithName
fun:_PyFunction_Vectorcall
fun:_PyEval_EvalFrameDefault
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
obj:/usr/bin/python3.8
fun:PyTuple_Pack
fun:__pyx_pymod_exec__sfc64
fun:PyModule_ExecDef
obj:/usr/bin/python3.8
obj:/usr/bin/python3.8
fun:PyVectorcall_Call
fun:_PyEval_EvalFrameDefault
fun:_PyEval_EvalCodeWithName
fun:_PyFunction_Vectorcall
fun:_PyEval_EvalFrameDefault
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
obj:/usr/bin/python3.8
fun:PyTuple_Pack
fun:__Pyx_InitCachedConstants
fun:__pyx_pymod_exec__generator
fun:PyModule_ExecDef
obj:/usr/bin/python3.8
obj:/usr/bin/python3.8
fun:PyVectorcall_Call
fun:_PyEval_EvalFrameDefault
fun:_PyEval_EvalCodeWithName
fun:_PyFunction_Vectorcall
fun:_PyEval_EvalFrameDefault
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
fun:PyCode_NewWithPosOnlyArgs
fun:PyCode_New
fun:__pyx_pymod_exec_mtrand
fun:PyModule_ExecDef
obj:/usr/bin/python3.8
obj:/usr/bin/python3.8
fun:PyVectorcall_Call
fun:_PyEval_EvalFrameDefault
fun:_PyEval_EvalCodeWithName
fun:_PyFunction_Vectorcall
fun:_PyEval_EvalFrameDefault
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
fun:PyCode_NewWithPosOnlyArgs
fun:PyCode_New
fun:__pyx_pymod_exec_bit_generator
fun:PyModule_ExecDef
obj:/usr/bin/python3.8
obj:/usr/bin/python3.8
fun:PyVectorcall_Call
fun:_PyEval_EvalFrameDefault
fun:_PyEval_EvalCodeWithName
fun:_PyFunction_Vectorcall
fun:_PyEval_EvalFrameDefault
}

210
tests/pytest/dbmgmt/nanoSecondCheck.py Normal file
View File

@ -0,0 +1,210 @@
# #################################################################
# Copyright (c) 2016 by TAOS Technologies, Inc.
# All rights reserved.
# This file is proprietary and confidential to TAOS Technologies.
# No part of this file may be reproduced, stored, transmitted,
# disclosed or used in any form or by any means other than as
# expressly provided by the written permission from Jianhui Tao
# #################################################################
# -*- coding: utf-8 -*-
# TODO: after TD-4518 and TD-4510 is resolved, add the exception test case for these situations
import sys
from util.log import *
from util.cases import *
from util.sql import *
import time
from datetime import datetime
import os
class TDTestCase:
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor(), logSql)
def run(self):
tdSql.prepare()
tdSql.execute('reset query cache')
tdSql.execute('drop database if exists db')
tdSql.execute('create database db precision "ns";')
tdSql.query('show databases;')
tdSql.checkData(0,16,'ns')
tdSql.execute('use db')
tdLog.debug('testing nanosecond support in 1st timestamp')
tdSql.execute('create table tb (ts timestamp, speed int)')
tdSql.execute('insert into tb values(\'2021-06-10 0:00:00.100000001\', 1);')
tdSql.execute('insert into tb values(1623254400150000000, 2);')
tdSql.execute('import into tb values(1623254400300000000, 3);')
tdSql.execute('import into tb values(1623254400299999999, 4);')
tdSql.execute('insert into tb values(1623254400300000001, 5);')
tdSql.execute('insert into tb values(1623254400999999999, 7);')
tdSql.query('select * from tb;')
tdSql.checkData(0,0,'2021-06-10 0:00:00.100000001')
tdSql.checkData(1,0,'2021-06-10 0:00:00.150000000')
tdSql.checkData(2,0,'2021-06-10 0:00:00.299999999')
tdSql.checkData(3,1,3)
tdSql.checkData(4,1,5)
tdSql.checkData(5,1,7)
tdSql.checkRows(6)
tdSql.query('select count(*) from tb where ts > 1623254400100000000 and ts < 1623254400100000002;')
tdSql.checkData(0,0,1)
tdSql.query('select count(*) from tb where ts > \'2021-06-10 0:00:00.100000001\' and ts < \'2021-06-10 0:00:00.160000000\';')
tdSql.checkData(0,0,1)
tdSql.query('select count(*) from tb where ts > 1623254400100000000 and ts < 1623254400150000000;')
tdSql.checkData(0,0,1)
tdSql.query('select count(*) from tb where ts > \'2021-06-10 0:00:00.100000000\' and ts < \'2021-06-10 0:00:00.150000000\';')
tdSql.checkData(0,0,1)
tdSql.query('select count(*) from tb where ts > 1623254400400000000;')
tdSql.checkData(0,0,1)
tdSql.query('select count(*) from tb where ts < \'2021-06-10 00:00:00.400000000\';')
tdSql.checkData(0,0,5)
tdSql.query('select count(*) from tb where ts > now + 400000000b;')
tdSql.checkRows(0)
tdSql.query('select count(*) from tb where ts >= \'2021-06-10 0:00:00.100000001\';')
tdSql.checkData(0,0,6)
tdSql.query('select count(*) from tb where ts <= 1623254400300000000;')
tdSql.checkData(0,0,4)
tdSql.query('select count(*) from tb where ts = \'2021-06-10 0:00:00.000000000\';')
tdSql.checkRows(0)
tdSql.query('select count(*) from tb where ts = 1623254400150000000;')
tdSql.checkData(0,0,1)
tdSql.query('select count(*) from tb where ts = \'2021-06-10 0:00:00.100000001\';')
tdSql.checkData(0,0,1)
tdSql.query('select count(*) from tb where ts between 1623254400000000000 and 1623254400400000000;')
tdSql.checkData(0,0,5)
tdSql.query('select count(*) from tb where ts between \'2021-06-10 0:00:00.299999999\' and \'2021-06-10 0:00:00.300000001\';')
tdSql.checkData(0,0,3)
tdSql.query('select avg(speed) from tb interval(5000000000b);')
tdSql.checkRows(1)
tdSql.query('select avg(speed) from tb interval(100000000b)')
tdSql.checkRows(4)
tdSql.query('select avg(speed) from tb interval(100000000b) sliding (100000000b);')
tdSql.checkRows(4)
tdSql.query('select last(*) from tb')
tdSql.checkData(0,0, '2021-06-10 0:00:00.999999999')
tdSql.checkData(0,0, 1623254400999999999)
tdSql.query('select first(*) from tb')
tdSql.checkData(0,0, 1623254400100000001)
tdSql.checkData(0,0, '2021-06-10 0:00:00.100000001')
tdSql.execute('insert into tb values(now + 500000000b, 6);')
tdSql.query('select * from tb;')
tdSql.checkRows(7)
tdLog.debug('testing nanosecond support in other timestamps')
tdSql.execute('create table tb2 (ts timestamp, speed int, ts2 timestamp);')
tdSql.execute('insert into tb2 values(\'2021-06-10 0:00:00.100000001\', 1, \'2021-06-11 0:00:00.100000001\');')
tdSql.execute('insert into tb2 values(1623254400150000000, 2, 1623340800150000000);')
tdSql.execute('import into tb2 values(1623254400300000000, 3, 1623340800300000000);')
tdSql.execute('import into tb2 values(1623254400299999999, 4, 1623340800299999999);')
tdSql.execute('insert into tb2 values(1623254400300000001, 5, 1623340800300000001);')
tdSql.execute('insert into tb2 values(1623254400999999999, 7, 1623513600999999999);')
tdSql.query('select * from tb2;')
tdSql.checkData(0,0,'2021-06-10 0:00:00.100000001')
tdSql.checkData(1,0,'2021-06-10 0:00:00.150000000')
tdSql.checkData(2,1,4)
tdSql.checkData(3,1,3)
tdSql.checkData(4,2,'2021-06-11 00:00:00.300000001')
tdSql.checkData(5,2,'2021-06-13 00:00:00.999999999')
tdSql.checkRows(6)
tdSql.query('select count(*) from tb2 where ts2 > 1623340800000000000 and ts2 < 1623340800150000000;')
tdSql.checkData(0,0,1)
tdSql.query('select count(*) from tb2 where ts2 > \'2021-06-11 0:00:00.100000000\' and ts2 < \'2021-06-11 0:00:00.100000002\';')
tdSql.checkData(0,0,1)
tdSql.query('select count(*) from tb2 where ts2 > 1623340800500000000;')
tdSql.checkData(0,0,1)
tdSql.query('select count(*) from tb2 where ts2 < \'2021-06-11 0:00:00.400000000\';')
tdSql.checkData(0,0,5)
tdSql.query('select count(*) from tb2 where ts2 > now + 400000000b;')
tdSql.checkRows(0)
tdSql.query('select count(*) from tb2 where ts2 >= \'2021-06-11 0:00:00.100000001\';')
tdSql.checkData(0,0,6)
tdSql.query('select count(*) from tb2 where ts2 <= 1623340800400000000;')
tdSql.checkData(0,0,5)
tdSql.query('select count(*) from tb2 where ts2 = \'2021-06-11 0:00:00.000000000\';')
tdSql.checkRows(0)
tdSql.query('select count(*) from tb2 where ts2 = \'2021-06-11 0:00:00.300000001\';')
tdSql.checkData(0,0,1)
tdSql.query('select count(*) from tb2 where ts2 = 1623340800300000001;')
tdSql.checkData(0,0,1)
tdSql.query('select count(*) from tb2 where ts2 between 1623340800000000000 and 1623340800450000000;')
tdSql.checkData(0,0,5)
tdSql.query('select count(*) from tb2 where ts2 between \'2021-06-11 0:00:00.299999999\' and \'2021-06-11 0:00:00.300000001\';')
tdSql.checkData(0,0,3)
tdSql.query('select count(*) from tb2 where ts2 <> 1623513600999999999;')
tdSql.checkData(0,0,5)
tdSql.query('select count(*) from tb2 where ts2 <> \'2021-06-11 0:00:00.100000001\';')
tdSql.checkData(0,0,5)
tdSql.query('select count(*) from tb2 where ts2 <> \'2021-06-11 0:00:00.100000000\';')
tdSql.checkData(0,0,6)
tdSql.query('select count(*) from tb2 where ts2 != 1623513600999999999;')
tdSql.checkData(0,0,5)
tdSql.query('select count(*) from tb2 where ts2 != \'2021-06-11 0:00:00.100000001\';')
tdSql.checkData(0,0,5)
tdSql.query('select count(*) from tb2 where ts2 != \'2021-06-11 0:00:00.100000000\';')
tdSql.checkData(0,0,6)
tdSql.execute('insert into tb2 values(now + 500000000b, 6, now +2d);')
tdSql.query('select * from tb2;')
tdSql.checkRows(7)
tdLog.debug('testing ill nanosecond format handling')
tdSql.execute('create table tb3 (ts timestamp, speed int);')
tdSql.error('insert into tb3 values(16232544001500000, 2);')
tdSql.execute('insert into tb3 values(\'2021-06-10 0:00:00.123456\', 2);')
tdSql.query('select * from tb3 where ts = \'2021-06-10 0:00:00.123456000\';')
tdSql.checkRows(1)
tdSql.execute('insert into tb3 values(\'2021-06-10 0:00:00.123456789000\', 2);')
tdSql.query('select * from tb3 where ts = \'2021-06-10 0:00:00.123456789\';')
tdSql.checkRows(1)
os.system('sudo timedatectl set-ntp on')
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())

2
tests/pytest/fulltest.sh
View File

@ -73,8 +73,10 @@ python3 ./test.py -f tag_lite/int.py
python3 ./test.py -f tag_lite/set.py python3 ./test.py -f tag_lite/set.py
python3 ./test.py -f tag_lite/smallint.py python3 ./test.py -f tag_lite/smallint.py
python3 ./test.py -f tag_lite/tinyint.py python3 ./test.py -f tag_lite/tinyint.py
python3 ./test.py -f tag_lite/timestamp.py
#python3 ./test.py -f dbmgmt/database-name-boundary.py #python3 ./test.py -f dbmgmt/database-name-boundary.py
python3 test.py -f dbmgmt/nanoSecondCheck.py
python3 ./test.py -f import_merge/importBlock1HO.py python3 ./test.py -f import_merge/importBlock1HO.py
python3 ./test.py -f import_merge/importBlock1HPO.py python3 ./test.py -f import_merge/importBlock1HPO.py

7
tests/pytest/functions/function_derivative.py
View File

@ -130,6 +130,13 @@ class TDTestCase:
tdSql.prepare() tdSql.prepare()
self.insertAndCheckData() self.insertAndCheckData()
tdSql.execute("create table st(ts timestamp, c1 int, c2 int) tags(id int)")
tdSql.execute("insert into dev1(ts, c1) using st tags(1) values(now, 1)")
tdSql.error("select derivative(c1, 10s, 0) from (select c1 from st)")
tdSql.query("select diff(c1) from (select derivative(c1, 1s, 0) c1 from dev1)")
tdSql.checkRows(0)
def stop(self): def stop(self):
tdSql.close() tdSql.close()
tdLog.success("%s successfully executed" % __file__) tdLog.success("%s successfully executed" % __file__)

2
tests/pytest/functions/function_operations.py
View File

@ -16,7 +16,7 @@ import taos
from util.log import * from util.log import *
from util.cases import * from util.cases import *
from util.sql import * from util.sql import *
import numpy as np #import numpy as np
class TDTestCase: class TDTestCase:

102
tests/pytest/tag_lite/timestamp.py Normal file
View File

@ -0,0 +1,102 @@
# -*- coding: utf-8 -*-
import sys
from util.log import *
from util.cases import *
from util.sql import *
class TDTestCase:
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor(), logSql)
def run(self):
tdSql.prepare()
tdLog.info('======================== dnode1 start')
tbPrefix = "ta_fl_tb"
mtPrefix = "ta_fl_mt"
tbNum = 10
rowNum = 20
totalNum = 200
tdLog.info('=============== step1')
i = 0
mt = "%s%d" % (mtPrefix, i)
tdSql.execute(
'create table %s (ts timestamp, tbcol int) TAGS(tgcol float, tgTs timestamp, tgcol2 int)' %(mt))
i = 0
ts = 1605045600000
tsStr = "2020-11-11 06:00:00"
while (i < 5):
tb = "%s%d" % (tbPrefix, i)
tdLog.info('create table %s using %s tags(%d, %d, %d)' % (tb, mt, i, ts + i, i))
tdSql.execute('create table %s using %s tags(%d, %d, %d)' % (tb, mt, i, ts + i, i))
x = 0
while (x < rowNum):
ms = x * 60000
#tdLog.info(
# "insert into %s values (%d, %d)" %
# (tb, 1605045600000 + ms, x))
tdSql.execute(
"insert into %s values (%d, %d)" %
(tb, 1605045600000 + ms, x))
x = x + 1
i = i + 1
tdLog.info('=============== step2')
tdSql.query('select * from %s' % (mt))
tdSql.checkRows(5 * rowNum)
tdSql.query('select * from %s where tgTs = %ld and tgcol2 = 0' % (mt, ts))
tdSql.checkRows(rowNum)
tdSql.query('select * from %s where tgTs = \"%s\" and tgcol2 = 0' % (mt, tsStr))
tdSql.checkRows(rowNum)
tdLog.info('=============== step3')
i = 0
while (i < 5):
tb = "%s%d" % (tbPrefix, i + 100)
tdLog.info('create table %s using %s tags(%d, \"%s\", %d)' % (tb, mt, i + 100, tsStr, i + 100))
tdSql.execute('create table %s using %s tags(%d, \"%s\", %d)' % (tb, mt, i + 100, tsStr, i + 100))
x = 0
while (x < rowNum):
ms = x * 60000
#tdLog.info(
# "insert into %s values (%d, %d)" %
# (tb, 1605045600000 + ms, x))
tdSql.execute(
"insert into %s values (%d, %d)" %
(tb, 1605045600000 + ms, x))
x = x + 1
i = i + 1
tdSql.query('select * from %s where tgTs = %ld and tgcol2 = 100' % (mt, ts))
tdSql.checkRows(rowNum)
tdSql.query('select * from %s where tgTs = \"%s\" and tgcol2 = 100' % (mt, tsStr))
tdSql.checkRows(rowNum)
tdLog.info('=============== step4')
i = 0
tb = "%s%d"%(tbPrefix, i + 1000)
tdSql.execute('insert into %s using %s tags(%d, \"%s\", %d) values(now, 10)' % (tb, mt, i + 100, tsStr, i + 1000))
tdSql.execute('insert into %s using %s tags(%d, \"%s\", %d) values(now+2s, 10)' % (tb, mt, i + 100, tsStr, i + 1000))
tdSql.execute('insert into %s using %s tags(%d, \"%s\", %d) values(now+3s, 10)' % (tb, mt, i + 100, tsStr, i + 1000))
tdSql.query('select * from %s where tgTs = \"%s\" and tgcol2 = 1000' % (mt, tsStr))
tdSql.checkRows(3)
i = 0
tb = "%s%d"%(tbPrefix, i + 10000)
tdSql.execute('create table %s using %s tags(%d, now, %d)' % (tb, mt, i + 10000,i + 10000))
tdSql.checkRows(3)
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())

10
tests/pytest/util/sql.py
View File

@ -18,6 +18,7 @@ import datetime
import inspect import inspect
import psutil import psutil
import shutil import shutil
import pandas as pd
from util.log import * from util.log import *
@ -136,7 +137,14 @@ class TDSql:
def checkData(self, row, col, data): def checkData(self, row, col, data):
self.checkRowCol(row, col) self.checkRowCol(row, col)
if self.queryResult[row][col] != data: if self.queryResult[row][col] != data:
if self.cursor.istype(col, "TIMESTAMP") and self.queryResult[row][col] == datetime.datetime.fromisoformat(data): if self.cursor.istype(col, "TIMESTAMP"):
# suppose user want to check nanosecond timestamp if a longer data passed
if (len(data) >= 28):
if pd.to_datetime(self.queryResult[row][col]) == pd.to_datetime(data):
tdLog.info("sql:%s, row:%d col:%d data:%d == expect:%s" %
(self.sql, row, col, self.queryResult[row][col], data))
else:
if self.queryResult[row][col] == datetime.datetime.fromisoformat(data):
tdLog.info("sql:%s, row:%d col:%d data:%s == expect:%s" % tdLog.info("sql:%s, row:%d col:%d data:%s == expect:%s" %
(self.sql, row, col, self.queryResult[row][col], data)) (self.sql, row, col, self.queryResult[row][col], data))
return return

166
tests/script/general/field/unsigined_bigint.sim Normal file
View File

@ -0,0 +1,166 @@
system sh/stop_dnodes.sh
system sh/deploy.sh -n dnode1 -i 1
system sh/cfg.sh -n dnode1 -c walLevel -v 1
system sh/exec.sh -n dnode1 -s start
sleep 2000
sql connect
print ======================== dnode1 start
$dbPrefix = db
$tbPrefix = tb
$mtPrefix = st
$tbNum = 10
$rowNum = 20
$totalNum = 200
print =============== step1
$i = 0
$db = $dbPrefix . $i
$mt = $mtPrefix . $i
sql create database $db
sql use $db
sql create table $mt (ts timestamp, tbcol bigint unsigned) TAGS(tgcol bigint unsigned)
$i = 0
while $i < 5
$tb = $tbPrefix . $i
sql create table $tb using $mt tags( 0 )
sql create table $tb using $mt tags( -111 )
$x = 0
while $x < $rowNum
$ms = $x . m
sql insert into $tb values (now + $ms , 0 )
$x = $x + 1
sql_error insert into $tb values (now + $ms , -10)
sql_error insert into $tb values (now + $ms , -1000)
sql_error insert into $tb values (now + $ms , -10000000)
endw
$i = $i + 1
endw
while $i < 10
$tb = $tbPrefix . $i
sql create table $tb using $mt tags( 1 )
$x = 0
while $x < $rowNum
$ms = $x . m
sql insert into $tb values (now + $ms , 1 )
$x = $x + 1
endw
$i = $i + 1
endw
print =============== step2
sql select * from $mt where tbcol = 0
if $rows != 100 then
return -1
endi
sql select * from $mt where tbcol <> 0
if $rows != 100 then
return -1
endi
sql select * from $mt where tbcol = 1
if $rows != 100 then
return -1
endi
sql select * from $mt where tbcol <> 1
if $rows != 100 then
return -1
endi
sql select * from $mt where tbcol = 1
if $rows != 100 then
return -1
endi
sql select * from $mt where tbcol <> 1
if $rows != 100 then
return -1
endi
sql select * from $mt where tbcol = 0
if $rows != 100 then
return -1
endi
sql select * from $mt where tbcol <> 0
if $rows != 100 then
return -1
endi
print =============== step3
sql select * from $mt where ts > now + 4m and tbcol = 1
if $rows != 75 then
return -1
endi
sql select * from $mt where ts > now + 4m and tbcol <> 1
if $rows != 75 then
return -1
endi
sql select * from $mt where ts < now + 4m and tbcol = 0
if $rows != 25 then
return -1
endi
sql select * from $mt where ts < now + 4m and tbcol <> 0
if $rows != 25 then
return -1
endi
sql select * from $mt where ts <= now + 4m and tbcol = 0
if $rows != 25 then
return -1
endi
sql select * from $mt where ts <= now + 4m and tbcol <> 0
if $rows != 25 then
return -1
endi
sql select * from $mt where ts > now + 4m and ts < now + 5m and tbcol <> 0
if $rows != 5 then
return -1
endi
sql select * from $mt where ts > now + 4m and tbcol <> 0 and ts < now + 5m
if $rows != 5 then
return -1
endi
print =============== step4
sql select count(tbcol), avg(tbcol), sum(tbcol), min(tbcol), max(tbcol), first(tbcol), last(tbcol) from $mt
print $data00 $data01 $data02 $data03 $data04 $data05 $data06
if $data00 != 200 then
return -1
endi
print =============== step5
sql select count(tbcol), avg(tbcol), sum(tbcol), min(tbcol), max(tbcol), first(tbcol), last(tbcol) from $mt where tbcol = 1
print $data00 $data01 $data02 $data03 $data04 $data05 $data06
if $data00 != 100 then
return -1
endi
print =============== step6
sql select count(tbcol), avg(tbcol), sum(tbcol), min(tbcol), max(tbcol), first(tbcol), last(tbcol) from $mt where tbcol = 1 group by tgcol
print $data00 $data01 $data02 $data03 $data04 $data05 $data06
print $data10 $data11 $data12 $data13 $data14 $data15 $data16
if $data00 != 100 then
print expect 100, actual $data00
return -1
endi
print =============== step7
sql select count(tbcol), avg(tbcol), sum(tbcol), min(tbcol), max(tbcol), first(tbcol), last(tbcol) from $mt where ts < now + 4m and tbcol = 1 group by tgcol
print $data00 $data01 $data02 $data03 $data04 $data05 $data06
if $data00 != 25 then
return -1
endi
print =============== step8
sql select count(tbcol), avg(tbcol), sum(tbcol), min(tbcol), max(tbcol), first(tbcol), last(tbcol) from $mt where tbcol = 1 interval(1d) group by tgcol order by tgcol desc
print $data00 $data01 $data02 $data03 $data04 $data05 $data06
if $data01 != 100 then
return -1
endi
print =============== clear
sql drop database $db
sql show databases
if $rows != 0 then
return -1
endi
system sh/exec.sh -n dnode1 -s stop -x SIGINT

2
tests/script/general/parser/create_mt.sim
View File

@ -99,7 +99,7 @@ $i_binary2 = varchar(20) # illegal string
$i_bool = boolean $i_bool = boolean
$nchar = nchar # nchar with unspecified length $nchar = nchar # nchar with unspecified length
print ========== create_mt.sim case4: illegal data types in tags test print ========== create_mt.sim case4: illegal data types in tags test
sql_error create table $mt (ts timestamp, col int) tags (tag1 timestamp ) ##sql_error create table $mt (ts timestamp, col int) tags (tag1 timestamp )
sql_error create table $mt (ts timestamp, col int) tags (tag1 $i_ts ) sql_error create table $mt (ts timestamp, col int) tags (tag1 $i_ts )
sql_error create table $mt (ts timestamp, col int) tags (tag1 $i_binary ) sql_error create table $mt (ts timestamp, col int) tags (tag1 $i_binary )
sql_error create table $mt (ts timestamp, col int) tags (tag1 $i_bigint ) sql_error create table $mt (ts timestamp, col int) tags (tag1 $i_bigint )

4
tests/script/general/parser/where.sim
View File

@ -155,8 +155,8 @@ sql_error select last(*) from wh_mt1_tb1 where c6 in ('1')
#sql_error select last(*) from wh_mt1_tb1 where c7 in ('binary') #sql_error select last(*) from wh_mt1_tb1 where c7 in ('binary')
#sql_error select last(*) from wh_mt1 where c8 in ('nchar') #sql_error select last(*) from wh_mt1 where c8 in ('nchar')
#sql_error select last(*) from wh_mt1_tb1 where c9 in (true, false) #sql_error select last(*) from wh_mt1_tb1 where c9 in (true, false)
sql_error select last(*) from wh_mt1 where c10 in ('2019-01-01 00:00:00.000') #sql_error select last(*) from wh_mt1 where c10 in ('2019-01-01 00:00:00.000')
sql_error select last(*) from wh_mt1_tb1 where c10 in ('2019-01-01 00:00:00.000') #sql_error select last(*) from wh_mt1_tb1 where c10 in ('2019-01-01 00:00:00.000')
sql select last(*) from wh_mt1 where c1 = 1 sql select last(*) from wh_mt1 where c1 = 1
if $rows != 1 then if $rows != 1 then
return -1 return -1