xxq250
/
homework-jianmu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Merge branch 'develop' into feature/crash_gen

This commit is contained in:
Shengliang Guan 2020-06-20 09:06:06 +00:00
parent 815cb83904 31c20a354f
commit c44bb0c647
298 changed files with 16567 additions and 9550 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

99
.travis.yml
View File

@ -4,6 +4,13 @@
#
# Build Matrix
#
branches:
only:
- master
- develop
- coverity_scan
- /^.*ci-.*$/
matrix:
- os: linux
dist: bionic
@ -45,7 +52,9 @@ matrix:
cd ${TRAVIS_BUILD_DIR}/debug
make install > /dev/null || travis_terminate $?
pip install numpy
pip install --user ${TRAVIS_BUILD_DIR}/src/connector/python/linux/python2/
pip3 install numpy
pip3 install --user ${TRAVIS_BUILD_DIR}/src/connector/python/linux/python3/
cd ${TRAVIS_BUILD_DIR}/tests
@ -164,7 +173,9 @@ matrix:
cd ${TRAVIS_BUILD_DIR}/debug
make install > /dev/null || travis_terminate $?
pip install numpy
pip install --user ${TRAVIS_BUILD_DIR}/src/connector/python/linux/python2/
pip3 install numpy
pip3 install --user ${TRAVIS_BUILD_DIR}/src/connector/python/linux/python3/
cd ${TRAVIS_BUILD_DIR}/tests
@ -256,19 +267,75 @@ matrix:
- cmake .. > /dev/null
- make > /dev/null
# - os: osx
# language: c
# compiler: clang
# env: DESC="mac/clang build"
# git:
# - depth: 1
# addons:
# homebrew:
# - cmake
#
# script:
# - cd ${TRAVIS_BUILD_DIR}
# - mkdir debug
# - cd debug
# - cmake .. > /dev/null
# - make > /dev/null
- os: linux
arch: arm64
dist: bionic
language: c
compiler: clang
env: DESC="linux/clang build"
git:
- depth: 1
addons:
apt:
packages:
- build-essential
- cmake
before_script:
- cd ${TRAVIS_BUILD_DIR}
- mkdir debug
- cd debug
script:
- if [ "${TRAVIS_CPU_ARCH}" == "arm64" ]; then
cmake .. -DCPUTYPE=aarch64 > /dev/null;
else
cmake .. > /dev/null;
fi
- make > /dev/null
- os: linux
arch: arm64
dist: trusty
language: c
git:
- depth: 1
addons:
apt:
packages:
- build-essential
- cmake
env:
- DESC="trusty/gcc-4.8 build"
before_script:
- cd ${TRAVIS_BUILD_DIR}
- mkdir debug
- cd debug
script:
- if [ "${TRAVIS_CPU_ARCH}" == "arm64" ]; then
cmake .. -DCPUTYPE=aarch64 > /dev/null;
else
cmake .. > /dev/null;
fi
- make > /dev/null
# - os: osx
# language: c
# compiler: clang
# env: DESC="mac/clang build"
# git:
# - depth: 1
# addons:
# homebrew:
# - cmake
#
# script:
# - cd ${TRAVIS_BUILD_DIR}
# - mkdir debug
# - cd debug
# - cmake .. > /dev/null
# - make > /dev/null

2
documentation/webdocs/markdowndocs/Super Table-ch.md
View File

@ -53,7 +53,7 @@ STable从属于库，一个STable只属于一个库，但一个库可以有一
说明：
1. TAGS列总长度不能超过64k bytes；
1. TAGS列总长度不能超过16k bytes；
2. TAGS列的数据类型不能是timestamp；
3. TAGS列名不能与其他列名相同;
4. TAGS列名不能为预留关键字.

2
documentation/webdocs/markdowndocs/Super Table.md
View File

@ -22,7 +22,7 @@ New keyword "tags" is introduced, where tag_name is the tag name, and tag_type i
Note：
1. The bytes of all tags together shall be less than 64k
1. The bytes of all tags together shall be less than 16k
2. Tag's data type can not be time stamp
3. Tag name shall be different from the field name
4. Tag name shall not be the same as system keywords

4
documentation/webdocs/markdowndocs/TAOS SQL-ch.md
View File

@ -63,7 +63,7 @@ TDengine缺省的时间戳是毫秒精度，但通过修改配置参数enableMic
| 3 | BIGINT | 8 | 长整型，范围 [-2^63+1, 2^63-1], -2^63用于NULL |
| 4 | FLOAT | 4 | 浮点型，有效位数6-7，范围 [-3.4E38, 3.4E38] |
| 5 | DOUBLE | 8 | 双精度浮点型，有效位数15-16，范围 [-1.7E308, 1.7E308] |
| 6 | BINARY | 自定义 | 用于记录字符串，理论上，最长可以有65526字节，但由于每行数据最多64K字节，实际上限一般小于理论值。 binary仅支持字符串输入，字符串两端使用单引号引用，否则英文全部自动转化为小写。使用时须指定大小，如binary(20)定义了最长为20个字符的字符串，每个字符占1byte的存储空间。如果用户字符串超出20字节将会报错。对于字符串内的单引号，可以用转义字符反斜线加单引号来表示， 即 **\’**。 |
| 6 | BINARY | 自定义 | 用于记录字符串，理论上，最长可以有16374字节，但由于每行数据最多16K字节，实际上限一般小于理论值。 binary仅支持字符串输入，字符串两端使用单引号引用，否则英文全部自动转化为小写。使用时须指定大小，如binary(20)定义了最长为20个字符的字符串，每个字符占1byte的存储空间。如果用户字符串超出20字节将会报错。对于字符串内的单引号，可以用转义字符反斜线加单引号来表示， 即 **\’**。 |
| 7 | SMALLINT | 2 | 短整型， 范围 [-32767, 32767], -32768用于NULL |
| 8 | TINYINT | 1 | 单字节整型，范围 [-127, 127], -128用于NULL |
| 9 | BOOL | 1 | 布尔型，{true, false} |
@ -106,7 +106,7 @@ TDengine缺省的时间戳是毫秒精度，但通过修改配置参数enableMic
```mysql
CREATE TABLE [IF NOT EXISTS] tb_name (timestamp_field_name TIMESTAMP, field1_name data_type1 [, field2_name data_type2 ...])
```
说明：1）表的第一个字段必须是TIMESTAMP，并且系统自动将其设为主键；2）表的每行长度不能超过64K字节；3）使用数据类型binary或nchar，需指定其最长的字节数，如binary(20)，表示20字节。
说明：1）表的第一个字段必须是TIMESTAMP，并且系统自动将其设为主键；2）表的每行长度不能超过16K字节；3）使用数据类型binary或nchar，需指定其最长的字节数，如binary(20)，表示20字节。
- **删除数据表**

4
documentation/webdocs/markdowndocs/TAOS SQL.md
View File

@ -39,7 +39,7 @@ The full list of data types is listed below. For string types of data, we will
| 6 | DOUBLE | 8 | A standard nullable double float type with 15-16 significant digits and a range of [-1.7E308, 1.7E308]​ |
| 7 | BOOL | 1 | A nullable boolean type, [**`true`**, **`false`**] |
| 8 | TIMESTAMP | 8 | A nullable timestamp type with the same usage as the primary column timestamp |
| 9 | BINARY(*M*) | *M* | A nullable string type whose length is *M*, error should be threw with exceeded chars, the maximum length of *M* is 65526, but as maximum row size is 64K bytes, the actual upper limit will generally less than 65526. This type of string only supports ASCii encoded chars. |
| 9 | BINARY(*M*) | *M* | A nullable string type whose length is *M*, error should be threw with exceeded chars, the maximum length of *M* is 16374, but as maximum row size is 16K bytes, the actual upper limit will generally less than 16374. This type of string only supports ASCii encoded chars. |
| 10 | NCHAR(*M*) | 4 * *M* | A nullable string type whose length is *M*, error should be threw with exceeded chars. The **`NCHAR`** type supports Unicode encoded chars. |
All the keywords in a SQL statement are case-insensitive, but strings values are case-sensitive and must be quoted by a pair of `'` or `"`. To quote a `'` or a `"` , you can use the escape character `\`.
@ -86,7 +86,7 @@ All the keywords in a SQL statement are case-insensitive, but strings values are
1) The first column must be a `timestamp`, and the system will set it as the primary key.
2) The record size is limited to 64k bytes
2) The record size is limited to 16k bytes
3) For `binary` or `nchar` data types, the length must be specified. For example, binary(20) means a binary data type with 20 bytes.

219
packaging/cfg/taos.cfg
View File

@ -6,10 +6,22 @@
########################################################
# first full-qualified domain name (FQDN) for TDengine system
# first 127.0.0.1
# first hostname1:6030
# second full-qualified domain name (FQDN) for TDengine system, for cluster edition only
# second 127.0.0.1
# second cluster_hostname2:6030
# the arbitrator's full-qualified domain name (FQDN) for TDengine system, for cluster edition only
# arbitrator arbitrator_hostname:6030
# the full-qualified domain name (FQDN) of dnode
# fqdn hostname:6030
# port for MNode connect to Client, default udp[6030-6055] tcp[6030]
# serverPort 6030
# http service port, default tcp[6020]
# httpPort 6020
# set socket type ("udp" and "tcp")
# the server and client should have the same socket type. Otherwise, connect will fail
@ -30,51 +42,29 @@
# log file's directory
# logDir /var/log/taos
# http service port, default tcp[6020]
# httpPort 6020
# number of management nodes in the system
# numOfMnodes 3
# port for MNode connect to Client, default udp[6030-6055] tcp[6030]
# serverPort 6030
# optional roles for dnode. 0 - any, 1 - mnode, 2 - dnode
# alternativeRole 0
# number of threads per CPU core
# numOfThreadsPerCore 1
# numOfThreadsPerCore 1.0
# number of vnodes per core in DNode
# numOfVnodesPerCore 8
# the ratio of threads responsible for querying in the total thread
# ratioOfQueryThreads 0.5
# number of total vnodes in DNode
# numOfTotalVnodes 0
# max number of tables per vnode
# maxtablesPerVnode 1000
# cache block size
# cache 16384
# row of records in file block
# minRows 100
# maxRows 4096
# number of cache blocks per vnode
# blocks 2
# interval of system monitor
# monitorInterval 60
# the compressed rpc message, option:
# -1 (no compression)
# 0 (all message compressed),
# > 0 (rpc message body which larger than this value will be compressed)
# compressMsgSize -1
# RPC re-try timer, millisecond
# rpcTimer 300
# RPC maximum time for ack, seconds
# rpcMaxTime 600
# commit interval，unit is second
# ctime 3600
# interval of check load balance when the management node is in normal operation
# balanceInterval 300
# interval of DNode report status to MNode, unit is Second, for cluster version only
# statusInterval 1
@ -82,29 +72,29 @@
# interval of Shell send HB to MNode, unit is Second
# shellActivityTimer 3
# interval of DNode send HB to DNode, unit is Second, for cluster version only
# vnodePeerHBTimer 1
# interval of system monitor
# monitorInterval 30
# interval of MNode send HB to MNode, unit is Second, for cluster version only
# mgmtPeerHBTimer 1
# RPC re-try timer, millisecond
# rpcTimer 300
# RPC maximum time for ack, seconds
# rpcMaxTime 600
# duration of to keep tableMeta kept in Cache, seconds
# tableMetaKeepTimer 7200
# max number of users
# maxUsers 1000
# Minimum sliding window time
# minSlidingTime 10
# max number of databases
# maxDbs 1000
# Time window minimum
# minIntervalTime 10
# max number of tables
# maxTables 650000
# max length of an SQL
# maxSQLLength 65380
# max number of Dnodes, for cluster version only
# maxDnodes 1000
# Max number of VGroups, for cluster version only
# maxVGroups 1000
# Support the maximum number of records allowed for super table time sorting
# maxNumOfOrderedRes 100000
# system locale
# locale en_US.UTF-8
@ -121,6 +111,24 @@
# enable/disable async log
# asyncLog 1
# the compressed rpc message, option:
# -1 (no compression)
# 0 (all message compressed),
# > 0 (rpc message body which larger than this value will be compressed)
# compressMsgSize -1
# cache block size (Mbyte)
# cache 16
# number of cache blocks per vnode
# blocks 4
# min row of records in file block
# minRows 100
# max row of records in file block
# maxRows 4096
# enable/disable compression
# comp 1
@ -131,31 +139,25 @@
# keep 3650
# number of replications, for cluster version only
# replications 1
# replica 1
# client default database(database should be created)
# defaultDB
# max number of connections from client for dnode
# maxShellConns 5000
# client default username
# defaultUser root
# max numerber of connections to one database
# maxConnections 5000
# client default password
# defaultPass taosdata
# Stop writing logs when the disk size of the log folder is less than this value
# minimalLogDirGB 0.1
# max number of connections from client for mgmt node
# maxShellConns 2000
# Stop writing temporary files when the disk size of the log folder is less than this value
# minimalTmpDirGB 0.1
# max numerber of meter Connections
# maxMeterConnections 10000
# max connection to management node
# maxMgmtConnections 2000
# max connection to Vnode
# maxVnodeConnections 10000
# Stop writing data when the disk size of the log folder is less than this value
# minimalDataDirGB 0.1
# mnode take into account while balance, for cluster version only
# mnodeEqualVnodeNum 4
# mnodeEqualVnodeNum 4
# number of seconds allowed for a dnode to be offline, for cluster version only
# offlineThreshold 864000
@ -170,10 +172,10 @@
# mqtt 0
# mqtt uri
# mqttBrokerAddress mqtt://username:password@hostname:1883/taos/
# mqttBrokerAddress mqtt://username:password@hostname:1883/taos/
# mqtt client name
# mqttBrokerClientId taos_mqtt
# mqttBrokerClientId taos_mqtt
# maximum number of rows returned by the restful interface
# restfulRowLimit 10240
@ -181,41 +183,17 @@
# number of threads used to process http requests
# httpMaxThreads 2
# pre-allocated number of http sessions
# httpCacheSessions 100
# The minimum time to wait before the first stream execution
# maxFirstStreamCompDelay 10000
# whether to enable HTTP compression transmission
# httpEnableCompress 0
# Retry wait time benchmark
# retryStreamCompDelay 10
# the delayed time for launching each continuous query. 10% of the whole computing time window by default.
# streamCompDelayRatio 0.1
# the max allowed delayed time for launching continuous query. 20ms by default
# tsMaxStreamComputDelay 20000
# whether the telegraf table name contains the number of tags and the number of fields
# telegrafUseFieldNum 0
# enable table names generated by telegraf can be configured
# use /etc/taos/taos.telegraf.cfg as the configuration file, such as
# {
# "metrics": [
# {
# "name" : "system",
# "tbname" : "system_uptime",
# "fields": [
# "uptime"
# ]
# },
# {
# "name": "system",
# "tbname" : "system_uptime_format",
# "fields": [
# "uptime_format"
# ]
# }
# ]
# }
# maxStreamCompDelay 20000
# The following parameter is used to limit the maximum number of lines in log files.
# max number of rows per log filters
@ -226,35 +204,62 @@
# 131: output warning and error，135: output info, warning and error to log.
# 199: output debug, info, warning and error to both screen and file
# debug flag for basic utils
# debugFlag 131
# debug flag for meta management messages
# mDebugFlag 135
# debug flag for dnode messages
# dDebugFlag 131
# debug flag for TDengine SDB
# sDebugFlag 135
# debug flag for TDengine SDB
# sdbDebugFlag 135
# debug flag for RPC
# rpcDebugFlag 131
# debug flag for basic utils
# debugFlag 131
# debug flag for TAOS TIMER
# tmrDebugFlag 131
# debug flag for TDengine client
# cDebugFlag 131
# debug flag for JNI
# jniDebugflag 131
# debug flag for ODBC
# odbcDebugflag 131
# debug flag for storage
# uDebugflag 131
# debug flag for http server
# httpDebugFlag 131
# debug flag for mqtt
# mqttDebugFlag 131
# debug flag for monitor
# monitorDebugFlag 131
# debug flag for query
# qDebugflag 131
# debug flag for http server
# httpDebugFlag 131
# tsdbDebugFlag 131
# debug flag for system monitor
# monitorDebugFlag 131
# Record the SQL through restful interface
# httpEnableRecordSql 0
#debug flag for mqtt client
# mqttDebugFlag 131
# Record the SQL in taos client
# tscEnableRecordSql 0
# debug flag for TAOS TIMER
# tmrDebugFlag 131
# if generate core file when service crash
# enableCoreFile 1
# The maximum display width of binary and nchar fields in the shell. The parts exceeding this limit will be hidden
# maxBinaryDisplayWidth 30

25
src/client/inc/tscSecondaryMerge.h → src/client/inc/tscLocalMerge.h
View File

@ -13,8 +13,8 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef TDENGINE_TSCSECONARYMERGE_H
#define TDENGINE_TSCSECONARYMERGE_H
#ifndef TDENGINE_TSCLOCALMERGE_H
#define TDENGINE_TSCLOCALMERGE_H
#ifdef __cplusplus
extern "C" {
@ -27,14 +27,7 @@ extern "C" {
#include "tsclient.h"
#define MAX_NUM_OF_SUBQUERY_RETRY 3
/*
* @version 0.1
* @date 2018/01/05
* @author liaohj
* management of client-side reducer for metric query
*/
struct SQLFunctionCtx;
typedef struct SLocalDataSource {
@ -60,7 +53,6 @@ typedef struct SLocalReducer {
char * prevRowOfInput;
tFilePage * pResultBuf;
int32_t nResultBufSize;
// char * pBufForInterpo; // intermediate buffer for interpolation
tFilePage * pTempBuffer;
struct SQLFunctionCtx *pCtx;
int32_t rowSize; // size of each intermediate result.
@ -81,13 +73,8 @@ typedef struct SLocalReducer {
} SLocalReducer;
typedef struct SSubqueryState {
/*
* the number of completed retrieval subquery, once this value equals to numOfVnodes,
* all retrieval are completed.Local merge is launched.
*/
int32_t numOfCompleted;
int32_t numOfTotal; // number of total sub-queries
int32_t code; // code from subqueries
int32_t numOfRemain; // the number of remain unfinished subquery
int32_t numOfTotal; // the number of total sub-queries
uint64_t numOfRetrievedRows; // total number of points in this query
} SSubqueryState;
@ -128,4 +115,4 @@ int32_t tscDoLocalMerge(SSqlObj *pSql);
}
#endif
#endif // TDENGINE_TSCSECONARYMERGE_H
#endif // TDENGINE_TSCLOCALMERGE_H

2
src/client/inc/tscSubquery.h
View File

@ -26,11 +26,9 @@ extern "C" {
void tscFetchDatablockFromSubquery(SSqlObj* pSql);
void tscSetupOutputColumnIndex(SSqlObj* pSql);
int32_t tscLaunchSecondPhaseSubqueries(SSqlObj* pSql);
void tscJoinQueryCallback(void* param, TAOS_RES* tres, int code);
SJoinSupporter* tscCreateJoinSupporter(SSqlObj* pSql, SSubqueryState* pState, int32_t index);
void tscDestroyJoinSupporter(SJoinSupporter* pSupporter);
int32_t tscHandleMasterJoinQuery(SSqlObj* pSql);

17
src/client/inc/tscUtil.h
View File

@ -28,7 +28,7 @@ extern "C" {
#include "exception.h"
#include "qextbuffer.h"
#include "taosdef.h"
#include "tscSecondaryMerge.h"
#include "tscLocalMerge.h"
#include "tsclient.h"
#define UTIL_TABLE_IS_SUPER_TABLE(metaInfo) \
@ -64,7 +64,8 @@ typedef struct SJoinSupporter {
SSubqueryState* pState;
SSqlObj* pObj; // parent SqlObj
int32_t subqueryIndex; // index of sub query
int64_t interval; // interval time
int64_t intervalTime; // interval time
int64_t slidingTime; // sliding time
SLimitVal limit; // limit info
uint64_t uid; // query meter uid
SArray* colList; // previous query information, no need to use this attribute, and the corresponding attribution
@ -122,15 +123,13 @@ bool tscNonOrderedProjectionQueryOnSTable(SQueryInfo *pQueryInfo, int32_t tableI
bool tscOrderedProjectionQueryOnSTable(SQueryInfo* pQueryInfo, int32_t tableIndex);
bool tscIsProjectionQueryOnSTable(SQueryInfo* pQueryInfo, int32_t tableIndex);
bool tscProjectionQueryOnTable(SQueryInfo* pQueryInfo);
bool tscIsProjectionQuery(SQueryInfo* pQueryInfo);
bool tscIsTwoStageSTableQuery(SQueryInfo* pQueryInfo, int32_t tableIndex);
bool tscQueryOnSTable(SSqlCmd* pCmd);
bool tscQueryTags(SQueryInfo* pQueryInfo);
bool tscIsSelectivityWithTagQuery(SSqlCmd* pCmd);
void tscAddSpecialColumnForSelect(SQueryInfo* pQueryInfo, int32_t outputColIndex, int16_t functionId, SColumnIndex* pIndex,
SSchema* pColSchema, int16_t isTag);
SSchema* pColSchema, int16_t colType);
int32_t tscSetTableFullName(STableMetaInfo* pTableMetaInfo, SSQLToken* pzTableName, SSqlObj* pSql);
void tscClearInterpInfo(SQueryInfo* pQueryInfo);
@ -139,7 +138,7 @@ bool tscIsInsertData(char* sqlstr);
/* use for keep current db info temporarily, for handle table with db prefix */
// todo remove it
void tscGetDBInfoFromMeterId(char* tableId, char* db);
void tscGetDBInfoFromTableFullName(char* tableId, char* db);
int tscAllocPayload(SSqlCmd* pCmd, int size);
@ -248,12 +247,14 @@ void tscDoQuery(SSqlObj* pSql);
* @param pPrevSql
* @return
*/
SSqlObj* createSimpleSubObj(SSqlObj* pSql, void (*fp)(), void* param, int32_t cmd);
SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, void (*fp)(), void* param, int32_t cmd, SSqlObj* pPrevSql);
void addGroupInfoForSubquery(SSqlObj* pParentObj, SSqlObj* pSql, int32_t subClauseIndex, int32_t tableIndex);
void doAddGroupColumnForSubquery(SQueryInfo* pQueryInfo, int32_t tagIndex);
int16_t tscGetJoinTagColIndexByUid(STagCond* pTagCond, uint64_t uid);
int16_t tscGetJoinTagColIdByUid(STagCond* pTagCond, uint64_t uid);
void tscPrintSelectClause(SSqlObj* pSql, int32_t subClauseIndex);

14
src/client/inc/tschemautil.h
View File

@ -64,12 +64,20 @@ SSchema* tscGetTableSchema(const STableMeta* pTableMeta);
SSchema *tscGetTableTagSchema(const STableMeta *pMeta);
/**
*
* get the column schema according to the column index
* @param pMeta
* @param startCol
* @param colIndex
* @return
*/
SSchema *tscGetTableColumnSchema(const STableMeta *pMeta, int32_t startCol);
SSchema *tscGetTableColumnSchema(const STableMeta *pMeta, int32_t colIndex);
/**
* get the column schema according to the column id
* @param pTableMeta
* @param colId
* @return
*/
SSchema* tscGetTableColumnSchemaById(STableMeta* pTableMeta, int16_t colId);
/**
* check if the schema is valid or not, including following aspects:

66
src/client/inc/tsclient.h
View File

@ -85,7 +85,7 @@ typedef struct SSqlExpr {
int16_t functionId; // function id in aAgg array
int16_t resType; // return value type
int16_t resBytes; // length of return value
int32_t interBytes; // inter result buffer size
int32_t interBytes; // inter result buffer size
int16_t numOfParams; // argument value of each function
tVariant param[3]; // parameters are not more than 3
int32_t offset; // sub result column value of arithmetic expression.
@ -123,7 +123,7 @@ typedef struct SCond {
typedef struct SJoinNode {
char tableId[TSDB_TABLE_ID_LEN];
uint64_t uid;
int16_t tagCol;
int16_t tagColId;
} SJoinNode;
typedef struct SJoinInfo {
@ -155,20 +155,19 @@ typedef struct SParamInfo {
} SParamInfo;
typedef struct STableDataBlocks {
char tableId[TSDB_TABLE_ID_LEN];
int8_t tsSource; // where does the UNIX timestamp come from, server or client
bool ordered; // if current rows are ordered or not
int64_t vgId; // virtual group id
int64_t prevTS; // previous timestamp, recorded to decide if the records array is ts ascending
int32_t numOfTables; // number of tables in current submit block
int32_t rowSize; // row size for current table
char tableId[TSDB_TABLE_ID_LEN];
int8_t tsSource; // where does the UNIX timestamp come from, server or client
bool ordered; // if current rows are ordered or not
int64_t vgId; // virtual group id
int64_t prevTS; // previous timestamp, recorded to decide if the records array is ts ascending
int32_t numOfTables; // number of tables in current submit block
int32_t rowSize; // row size for current table
uint32_t nAllocSize;
uint32_t headerSize; // header for metadata (submit metadata)
uint32_t headerSize; // header for table info (uid, tid, submit metadata)
uint32_t size;
/*
* the metermeta for current table, the metermeta will be used during submit stage, keep a ref
* the table meta of table, the table meta will be used during submit, keep a ref
* to avoid it to be removed from cache
*/
STableMeta *pTableMeta;
@ -191,32 +190,28 @@ typedef struct SDataBlockList { // todo remove
} SDataBlockList;
typedef struct SQueryInfo {
int16_t command; // the command may be different for each subclause, so keep it seperately.
uint32_t type; // query/insert/import type
int16_t command; // the command may be different for each subclause, so keep it seperately.
uint32_t type; // query/insert/import type
char slidingTimeUnit;
STimeWindow window;
int64_t intervalTime; // aggregation time interval
int64_t slidingTime; // sliding window in mseconds
SSqlGroupbyExpr groupbyExpr; // group by tags info
SArray * colList; // SArray<SColumn*>
SArray * colList; // SArray<SColumn*>
SFieldInfo fieldsInfo;
SArray * exprList; // SArray<SSqlExpr*>
SArray * exprList; // SArray<SSqlExpr*>
SLimitVal limit;
SLimitVal slimit;
STagCond tagCond;
SOrderVal order;
int16_t fillType; // final result fill type
int16_t fillType; // final result fill type
int16_t numOfTables;
STableMetaInfo **pTableMetaInfo;
struct STSBuf * tsBuf;
int64_t * fillVal; // default value for fill
char * msg; // pointer to the pCmd->payload to keep error message temporarily
int64_t clauseLimit; // limit for current sub clause
// offset value in the original sql expression, NOT sent to virtual node, only applied at client side
int64_t prjOffset;
int64_t * fillVal; // default value for fill
char * msg; // pointer to the pCmd->payload to keep error message temporarily
int64_t clauseLimit; // limit for current sub clause
int64_t prjOffset; // offset value in the original sql expression, only applied at client side
} SQueryInfo;
typedef struct {
@ -286,7 +281,7 @@ typedef struct STscObj {
char user[TSDB_USER_LEN];
char pass[TSDB_KEY_LEN];
char acctId[TSDB_ACCT_LEN];
char db[TSDB_DB_NAME_LEN];
char db[TSDB_ACCT_LEN + TSDB_DB_NAME_LEN];
char sversion[TSDB_VERSION_LEN];
char writeAuth : 1;
char superAuth : 1;
@ -299,11 +294,12 @@ typedef struct STscObj {
} STscObj;
typedef struct SSqlObj {
void * signature;
STscObj *pTscObj;
void (*fp)();
void (*fetchFp)();
void * param;
void *signature;
STscObj *pTscObj;
void *pRpcCtx;
void (*fp)();
void (*fetchFp)();
void *param;
int64_t stime;
uint32_t queryId;
void * pStream;
@ -312,8 +308,7 @@ typedef struct SSqlObj {
char retry;
char maxRetry;
SRpcIpSet ipList;
char freed : 4;
char listed : 4;
char listed;
tsem_t rspSem;
SSqlCmd cmd;
SSqlRes res;
@ -353,7 +348,7 @@ typedef struct SSqlStream {
int32_t tscInitRpc(const char *user, const char *secret, void** pDnodeConn);
void tscInitMsgsFp();
int tsParseSql(SSqlObj *pSql, bool multiVnodeInsertion);
int tsParseSql(SSqlObj *pSql, bool initial);
void tscProcessMsgFromServer(SRpcMsg *rpcMsg, SRpcIpSet *pIpSet);
int tscProcessSql(SSqlObj *pSql);
@ -409,6 +404,7 @@ void tscKillSTableQuery(SSqlObj *pSql);
void tscInitResObjForLocalQuery(SSqlObj *pObj, int32_t numOfRes, int32_t rowLen);
bool tscIsUpdateQuery(SSqlObj* pSql);
bool tscHasReachLimitation(SQueryInfo *pQueryInfo, SSqlRes *pRes);
bool tscResultsetFetchCompleted(TAOS_RES *result);
char *tscGetErrorMsgPayload(SSqlCmd *pCmd);
@ -431,7 +427,7 @@ extern int (*tscBuildMsg[TSDB_SQL_MAX])(SSqlObj *pSql, SSqlInfo *pInfo);
typedef void (*__async_cb_func_t)(void *param, TAOS_RES *tres, int numOfRows);
int32_t tscCompareTidTags(const void* p1, const void* p2);
void tscBuildVgroupTableInfo(STableMetaInfo* pTableMetaInfo, SArray* tables);
void tscBuildVgroupTableInfo(SSqlObj* pSql, STableMetaInfo* pTableMetaInfo, SArray* tables);
#ifdef __cplusplus
}

34
src/client/src/TSDBJNIConnector.c
View File

@ -151,8 +151,8 @@ JNIEXPORT void JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_dumpMemoryLeakImp
JNIEXPORT void JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_initImp(JNIEnv *env, jobject jobj, jstring jconfigDir) {
if (jconfigDir != NULL) {
const char *confDir = (*env)->GetStringUTFChars(env, jconfigDir, NULL);
if (confDir && strlen(configDir) != 0) {
strcpy(configDir, confDir);
if (confDir && strlen(confDir) != 0) {
tstrncpy(configDir, confDir, TSDB_FILENAME_LEN);
}
(*env)->ReleaseStringUTFChars(env, jconfigDir, confDir);
}
@ -192,7 +192,7 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setOptions(JNIEnv
const char *tz1 = (*env)->GetStringUTFChars(env, optionValue, NULL);
if (tz1 && strlen(tz1) != 0) {
res = taos_options(TSDB_OPTION_TIMEZONE, tz1);
jniTrace("set timezone to %s, result:%d", timezone, res);
jniTrace("set timezone to %s, result:%d", tz1, res);
} else {
jniTrace("input timezone is empty");
}
@ -227,12 +227,10 @@ JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_connectImp(JNIEn
}
if (user == NULL) {
jniTrace("jobj:%p, user is null, use tsDefaultUser", jobj);
user = tsDefaultUser;
jniTrace("jobj:%p, user is null, use default user %s", jobj, TSDB_DEFAULT_USER);
}
if (pass == NULL) {
jniTrace("jobj:%p, pass is null, use tsDefaultPass", jobj);
pass = tsDefaultPass;
jniTrace("jobj:%p, pass is null, use default password", jobj);
}
/*
@ -252,8 +250,8 @@ JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_connectImp(JNIEn
if (host != NULL) (*env)->ReleaseStringUTFChars(env, jhost, host);
if (dbname != NULL) (*env)->ReleaseStringUTFChars(env, jdbName, dbname);
if (user != NULL && user != tsDefaultUser) (*env)->ReleaseStringUTFChars(env, juser, user);
if (pass != NULL && pass != tsDefaultPass) (*env)->ReleaseStringUTFChars(env, jpass, pass);
if (user != NULL) (*env)->ReleaseStringUTFChars(env, juser, user);
if (pass != NULL) (*env)->ReleaseStringUTFChars(env, jpass, pass);
return ret;
}
@ -344,7 +342,7 @@ JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getResultSetImp(
STscObj *pObj = pSql->pTscObj;
if (tscIsUpdateQuery(pSql)) {
taos_free_result(pSql); // free result here
// taos_free_result(pSql); // free result here
jniTrace("jobj:%p, conn:%p, no resultset, %p", jobj, pObj, (void *)tres);
return 0;
} else {
@ -385,7 +383,7 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getAffectedRowsIm
}
jint ret = taos_affected_rows((SSqlObj *)res);
jniTrace("jobj:%p, conn:%p, sql:%p, affect rows:%d", jobj, tscon, (void *)con, res, ret);
jniTrace("jobj:%p, conn:%p, sql:%p, res: %p, affect rows:%d", jobj, tscon, (void *)con, (void *)res, ret);
return ret;
}
@ -411,10 +409,10 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getSchemaMetaData
// jobject arrayListObj = (*env)->NewObject(env, g_arrayListClass, g_arrayListConstructFp, "");
if (num_fields == 0) {
jniError("jobj:%p, conn:%p, resultset:%p, fields size is %d", jobj, tscon, res, num_fields);
jniError("jobj:%p, conn:%p, resultset:%p, fields size is %d", jobj, tscon, (void *)res, num_fields);
return JNI_NUM_OF_FIELDS_0;
} else {
jniTrace("jobj:%p, conn:%p, resultset:%p, fields size is %d", jobj, tscon, res, num_fields);
jniTrace("jobj:%p, conn:%p, resultset:%p, fields size is %d", jobj, tscon, (void *)res, num_fields);
for (int i = 0; i < num_fields; ++i) {
jobject metadataObj = (*env)->NewObject(env, g_metadataClass, g_metadataConstructFp);
(*env)->SetIntField(env, metadataObj, g_metadataColtypeField, fields[i].type);
@ -465,7 +463,7 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_fetchRowImp(JNIEn
int num_fields = taos_num_fields(result);
if (num_fields == 0) {
jniError("jobj:%p, conn:%p, resultset:%p, fields size is %d", jobj, tscon, res, num_fields);
jniError("jobj:%p, conn:%p, resultset:%p, fields size is %d", jobj, tscon, (void*)res, num_fields);
return JNI_NUM_OF_FIELDS_0;
}
@ -473,7 +471,7 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_fetchRowImp(JNIEn
if (row == NULL) {
int tserrno = taos_errno(result);
if (tserrno == 0) {
jniTrace("jobj:%p, conn:%p, resultset:%p, fields size is %d, fetch row to the end", jobj, tscon, res, num_fields);
jniTrace("jobj:%p, conn:%p, resultset:%p, fields size is %d, fetch row to the end", jobj, tscon, (void*)res, num_fields);
return JNI_FETCH_END;
} else {
jniTrace("jobj:%p, conn:%p, interruptted query", jobj, tscon);
@ -571,9 +569,9 @@ JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_subscribeImp(JNI
sub = (jlong)tsub;
if (sub == 0) {
jniTrace("jobj:%p, failed to subscribe: topic:%s", jobj, jtopic);
jniTrace("jobj:%p, failed to subscribe: topic:%s", jobj, topic);
} else {
jniTrace("jobj:%p, successfully subscribe: topic: %s", jobj, jtopic);
jniTrace("jobj:%p, successfully subscribe: topic: %s", jobj, topic);
}
if (topic != NULL) (*env)->ReleaseStringUTFChars(env, jtopic, topic);
@ -583,7 +581,7 @@ JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_subscribeImp(JNI
}
JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_consumeImp(JNIEnv *env, jobject jobj, jlong sub) {
jniTrace("jobj:%p, in TSDBJNIConnector_consumeImp, sub:%ld", jobj, sub);
jniTrace("jobj:%p, in TSDBJNIConnector_consumeImp, sub:%lld", jobj, sub);
jniGetGlobalMethod(env);
TAOS_SUB *tsub = (TAOS_SUB *)sub;

44
src/client/src/tscAsync.c
View File

@ -19,9 +19,8 @@
#include "tnote.h"
#include "trpc.h"
#include "tscLog.h"
#include "tscProfile.h"
#include "tscSubquery.h"
#include "tscSecondaryMerge.h"
#include "tscLocalMerge.h"
#include "tscUtil.h"
#include "tsched.h"
#include "tschemautil.h"
@ -40,38 +39,26 @@ static void tscProcessAsyncRetrieveImpl(void *param, TAOS_RES *tres, int numOfRo
static void tscAsyncFetchRowsProxy(void *param, TAOS_RES *tres, int numOfRows);
static void tscAsyncFetchSingleRowProxy(void *param, TAOS_RES *tres, int numOfRows);
int doAsyncParseSql(SSqlObj* pSql) {
SSqlCmd* pCmd = &pSql->cmd;
SSqlRes* pRes = &pSql->res;
int32_t code = tscAllocPayload(pCmd, TSDB_DEFAULT_PAYLOAD_SIZE);
if (code != TSDB_CODE_SUCCESS) {
tscError("failed to malloc payload");
tscQueueAsyncError(pSql->fp, pSql->param, TSDB_CODE_TSC_OUT_OF_MEMORY);
return code;
}
pRes->qhandle = 0;
pRes->numOfRows = 1;
tscDump("%p SQL: %s", pSql, pSql->sqlstr);
return tsParseSql(pSql, true);
}
void doAsyncQuery(STscObj* pObj, SSqlObj* pSql, void (*fp)(), void* param, const char* sqlstr, size_t sqlLen) {
pSql->signature = pSql;
pSql->param = param;
pSql->pTscObj = pObj;
pSql->maxRetry = TSDB_MAX_REPLICA_NUM;
pSql->fp = fp;
pSql->sqlstr = calloc(1, sqlLen + 1);
if (pSql->sqlstr == NULL) {
tscError("%p failed to malloc sql string buffer", pSql);
tscQueueAsyncError(pSql->fp, pSql->param, TSDB_CODE_TSC_OUT_OF_MEMORY);
return;
}
strtolower(pSql->sqlstr, sqlstr);
int32_t code = doAsyncParseSql(pSql);
tscDump("%p SQL: %s", pSql, pSql->sqlstr);
pSql->cmd.curSql = pSql->sqlstr;
int32_t code = tsParseSql(pSql, true);
if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) return;
if (code != TSDB_CODE_SUCCESS) {
@ -211,7 +198,8 @@ void taos_fetch_rows_a(TAOS_RES *taosa, void (*fp)(void *, TAOS_RES *, int), voi
if (pRes->qhandle == 0) {
tscError("qhandle is NULL");
tscQueueAsyncError(fp, param, TSDB_CODE_TSC_INVALID_QHANDLE);
pRes->code = TSDB_CODE_TSC_INVALID_QHANDLE;
tscQueueAsyncRes(pSql);
return;
}
@ -285,7 +273,7 @@ void taos_fetch_row_a(TAOS_RES *taosa, void (*fp)(void *, TAOS_RES *, TAOS_ROW),
tscProcessSql(pSql);
} else {
SSchedMsg schedMsg;
SSchedMsg schedMsg = { 0 };
schedMsg.fp = tscProcessFetchRow;
schedMsg.ahandle = pSql;
schedMsg.thandle = pRes->tsrow;
@ -368,7 +356,9 @@ void tscProcessAsyncRes(SSchedMsg *pMsg) {
pSql->fp = pSql->fetchFp;
}
(*pSql->fp)(pSql->param, taosres, code);
if (pSql->fp) {
(*pSql->fp)(pSql->param, taosres, code);
}
if (shouldFree) {
tscTrace("%p sqlObj is automatically freed in async res", pSql);
@ -385,7 +375,7 @@ void tscQueueAsyncError(void(*fp), void *param, int32_t code) {
int32_t* c = malloc(sizeof(int32_t));
*c = code;
SSchedMsg schedMsg;
SSchedMsg schedMsg = { 0 };
schedMsg.fp = tscProcessAsyncError;
schedMsg.ahandle = fp;
schedMsg.thandle = param;
@ -401,7 +391,7 @@ void tscQueueAsyncRes(SSqlObj *pSql) {
tscError("%p add into queued async res, code:%s", pSql, tstrerror(pSql->res.code));
}
SSchedMsg schedMsg;
SSchedMsg schedMsg = { 0 };
schedMsg.fp = tscProcessAsyncRes;
schedMsg.ahandle = pSql;
schedMsg.thandle = (void *)1;
@ -418,7 +408,7 @@ void tscProcessAsyncFree(SSchedMsg *pMsg) {
void tscQueueAsyncFreeResult(SSqlObj *pSql) {
tscTrace("%p sqlObj put in queue to async free", pSql);
SSchedMsg schedMsg;
SSchedMsg schedMsg = { 0 };
schedMsg.fp = tscProcessAsyncFree;
schedMsg.ahandle = pSql;
schedMsg.thandle = (void *)1;

48
src/client/src/tscFunctionImpl.c
View File

@ -153,7 +153,7 @@ typedef struct SRateInfo {
int32_t getResultDataInfo(int32_t dataType, int32_t dataBytes, int32_t functionId, int32_t param, int16_t *type,
int16_t *bytes, int32_t *interBytes, int16_t extLength, bool isSuperTable) {
if (!isValidDataType(dataType, dataBytes)) {
if (!isValidDataType(dataType)) {
tscError("Illegal data type %d or data type length %d", dataType, dataBytes);
return TSDB_CODE_TSC_INVALID_SQL;
}
@ -699,7 +699,7 @@ static int32_t first_data_req_info(SQLFunctionCtx *pCtx, TSKEY start, TSKEY end,
}
static int32_t last_data_req_info(SQLFunctionCtx *pCtx, TSKEY start, TSKEY end, int32_t colId) {
if (pCtx->order == TSDB_ORDER_ASC) {
if (pCtx->order != pCtx->param[0].i64Key) {
return BLK_DATA_NO_NEEDED;
}
@ -727,7 +727,7 @@ static int32_t first_dist_data_req_info(SQLFunctionCtx *pCtx, TSKEY start, TSKEY
}
static int32_t last_dist_data_req_info(SQLFunctionCtx *pCtx, TSKEY start, TSKEY end, int32_t colId) {
if (pCtx->order == TSDB_ORDER_ASC) {
if (pCtx->order != pCtx->param[0].i64Key) {
return BLK_DATA_NO_NEEDED;
}
@ -1593,7 +1593,7 @@ static void first_dist_function_f(SQLFunctionCtx *pCtx, int32_t index) {
if (pCtx->hasNull && isNull(pData, pCtx->inputType)) {
return;
}
if (pCtx->order == TSDB_ORDER_DESC) {
return;
}
@ -1652,7 +1652,7 @@ static void first_dist_func_second_merge(SQLFunctionCtx *pCtx) {
* least one data in this block that is not null.(TODO opt for this case)
*/
static void last_function(SQLFunctionCtx *pCtx) {
if (pCtx->order == TSDB_ORDER_ASC) {
if (pCtx->order != pCtx->param[0].i64Key) {
return;
}
@ -1681,7 +1681,6 @@ static void last_function(SQLFunctionCtx *pCtx) {
}
static void last_function_f(SQLFunctionCtx *pCtx, int32_t index) {
assert(pCtx->order != TSDB_ORDER_ASC);
void *pData = GET_INPUT_CHAR_INDEX(pCtx, index);
if (pCtx->hasNull && isNull(pData, pCtx->inputType)) {
return;
@ -1725,7 +1724,7 @@ static void last_dist_function(SQLFunctionCtx *pCtx) {
* 1. for scan data in asc order, no need to check data
* 2. for data blocks that are not loaded, no need to check data
*/
if (pCtx->order == TSDB_ORDER_ASC) {
if (pCtx->order != pCtx->param[0].i64Key) {
return;
}
@ -1763,7 +1762,7 @@ static void last_dist_function_f(SQLFunctionCtx *pCtx, int32_t index) {
* 1. for scan data in asc order, no need to check data
* 2. for data blocks that are not loaded, no need to check data
*/
if (pCtx->order == TSDB_ORDER_ASC) {
if (pCtx->order != pCtx->param[0].i64Key) {
return;
}
@ -1854,26 +1853,14 @@ static void last_row_function(SQLFunctionCtx *pCtx) {
static void last_row_finalizer(SQLFunctionCtx *pCtx) {
// do nothing at the first stage
SResultInfo *pResInfo = GET_RES_INFO(pCtx);
if (pCtx->currentStage == SECONDARY_STAGE_MERGE) {
if (pResInfo->hasResult != DATA_SET_FLAG) {
if (pCtx->outputType == TSDB_DATA_TYPE_BINARY || pCtx->outputType == TSDB_DATA_TYPE_NCHAR) {
setVardataNull(pCtx->aOutputBuf, pCtx->outputType);
} else {
setNull(pCtx->aOutputBuf, pCtx->outputType, pCtx->outputBytes);
}
return;
}
} else {
if (pResInfo->hasResult != DATA_SET_FLAG) {
if (pCtx->outputType == TSDB_DATA_TYPE_BINARY || pCtx->outputType == TSDB_DATA_TYPE_NCHAR) {
setVardataNull(pCtx->aOutputBuf, pCtx->outputType);
} else {
setNull(pCtx->aOutputBuf, pCtx->outputType, pCtx->outputBytes);
}
return;
if (pResInfo->hasResult != DATA_SET_FLAG) {
if (pCtx->outputType == TSDB_DATA_TYPE_BINARY || pCtx->outputType == TSDB_DATA_TYPE_NCHAR) {
setVardataNull(pCtx->aOutputBuf, pCtx->outputType);
} else {
setNull(pCtx->aOutputBuf, pCtx->outputType, pCtx->outputBytes);
}
return;
}
GET_RES_INFO(pCtx)->numOfRes = 1;
@ -2990,12 +2977,12 @@ static void tag_project_function_f(SQLFunctionCtx *pCtx, int32_t index) {
*/
static void tag_function(SQLFunctionCtx *pCtx) {
SET_VAL(pCtx, 1, 1);
tVariantDump(&pCtx->tag, pCtx->aOutputBuf, pCtx->tag.nType, true);
tVariantDump(&pCtx->tag, pCtx->aOutputBuf, pCtx->outputType, true);
}
static void tag_function_f(SQLFunctionCtx *pCtx, int32_t index) {
SET_VAL(pCtx, 1, 1);
tVariantDump(&pCtx->tag, pCtx->aOutputBuf, pCtx->tag.nType, true);
tVariantDump(&pCtx->tag, pCtx->aOutputBuf, pCtx->outputType, true);
}
static void copy_function(SQLFunctionCtx *pCtx) {
@ -3904,7 +3891,7 @@ static bool ts_comp_function_setup(SQLFunctionCtx *pCtx) {
SResultInfo *pResInfo = GET_RES_INFO(pCtx);
STSCompInfo *pInfo = pResInfo->interResultBuf;
pInfo->pTSBuf = tsBufCreate(false);
pInfo->pTSBuf = tsBufCreate(false, pCtx->order);
pInfo->pTSBuf->tsOrder = pCtx->order;
return true;
}
@ -3926,7 +3913,6 @@ static void ts_comp_function(SQLFunctionCtx *pCtx) {
}
SET_VAL(pCtx, pCtx->size, 1);
pResInfo->hasResult = DATA_SET_FLAG;
}

31
src/client/src/tscLocal.c
View File

@ -22,7 +22,6 @@
#include "taosdef.h"
#include "tscLog.h"
#include "qextbuffer.h"
#include "tscSecondaryMerge.h"
#include "tschemautil.h"
#include "tname.h"
@ -132,14 +131,14 @@ static int32_t tscSetValueToResObj(SSqlObj *pSql, int32_t rowLen) {
for (int32_t i = 0; i < numOfRows; ++i) {
TAOS_FIELD *pField = tscFieldInfoGetField(&pQueryInfo->fieldsInfo, 0);
char* dst = pRes->data + tscFieldInfoGetOffset(pQueryInfo, 0) * totalNumOfRows + pField->bytes * i;
STR_WITH_MAXSIZE_TO_VARSTR(dst, pSchema[i].name, TSDB_COL_NAME_LEN - 1);
STR_WITH_MAXSIZE_TO_VARSTR(dst, pSchema[i].name, pField->bytes);
char *type = tDataTypeDesc[pSchema[i].type].aName;
pField = tscFieldInfoGetField(&pQueryInfo->fieldsInfo, 1);
dst = pRes->data + tscFieldInfoGetOffset(pQueryInfo, 1) * totalNumOfRows + pField->bytes * i;
STR_TO_VARSTR(dst, type);
STR_WITH_MAXSIZE_TO_VARSTR(dst, type, pField->bytes);
int32_t bytes = pSchema[i].bytes;
if (pSchema[i].type == TSDB_DATA_TYPE_BINARY || pSchema[i].type == TSDB_DATA_TYPE_NCHAR) {
@ -157,7 +156,7 @@ static int32_t tscSetValueToResObj(SSqlObj *pSql, int32_t rowLen) {
if (i >= tscGetNumOfColumns(pMeta) && tscGetNumOfTags(pMeta) != 0) {
char* output = pRes->data + tscFieldInfoGetOffset(pQueryInfo, 3) * totalNumOfRows + pField->bytes * i;
const char *src = "TAG";
STR_WITH_SIZE_TO_VARSTR(output, src, strlen(src));
STR_WITH_MAXSIZE_TO_VARSTR(output, src, pField->bytes);
}
}
@ -171,7 +170,7 @@ static int32_t tscSetValueToResObj(SSqlObj *pSql, int32_t rowLen) {
// field name
TAOS_FIELD *pField = tscFieldInfoGetField(&pQueryInfo->fieldsInfo, 0);
char* output = pRes->data + tscFieldInfoGetOffset(pQueryInfo, 0) * totalNumOfRows + pField->bytes * i;
STR_WITH_MAXSIZE_TO_VARSTR(output, pSchema[i].name, TSDB_COL_NAME_LEN - 1);
STR_WITH_MAXSIZE_TO_VARSTR(output, pSchema[i].name, pField->bytes);
// type name
pField = tscFieldInfoGetField(&pQueryInfo->fieldsInfo, 1);
@ -183,8 +182,12 @@ static int32_t tscSetValueToResObj(SSqlObj *pSql, int32_t rowLen) {
// type length
int32_t bytes = pSchema[i].bytes;
pField = tscFieldInfoGetField(&pQueryInfo->fieldsInfo, 2);
if (pSchema[i].type == TSDB_DATA_TYPE_NCHAR) {
bytes = bytes / TSDB_NCHAR_SIZE;
if (pSchema[i].type == TSDB_DATA_TYPE_BINARY || pSchema[i].type == TSDB_DATA_TYPE_NCHAR) {
bytes -= VARSTR_HEADER_SIZE;
if (pSchema[i].type == TSDB_DATA_TYPE_NCHAR) {
bytes = bytes / TSDB_NCHAR_SIZE;
}
}
*(int32_t *)(pRes->data + tscFieldInfoGetOffset(pQueryInfo, 2) * totalNumOfRows + pField->bytes * i) = bytes;
@ -193,7 +196,7 @@ static int32_t tscSetValueToResObj(SSqlObj *pSql, int32_t rowLen) {
pField = tscFieldInfoGetField(&pQueryInfo->fieldsInfo, 3);
char *target = pRes->data + tscFieldInfoGetOffset(pQueryInfo, 3) * totalNumOfRows + pField->bytes * i;
const char *src = "TAG";
STR_WITH_SIZE_TO_VARSTR(target, src, strlen(src));
STR_WITH_MAXSIZE_TO_VARSTR(target, src, pField->bytes);
pTagValue += pSchema[i].bytes;
}
@ -220,15 +223,15 @@ static int32_t tscBuildTableSchemaResultFields(SSqlObj *pSql, int32_t numOfCols,
rowLen += ((TSDB_COL_NAME_LEN - 1) + VARSTR_HEADER_SIZE);
f.bytes = typeColLength;
f.bytes = typeColLength + VARSTR_HEADER_SIZE;
f.type = TSDB_DATA_TYPE_BINARY;
tstrncpy(f.name, "Type", sizeof(f.name));
pInfo = tscFieldInfoAppend(&pQueryInfo->fieldsInfo, &f);
pInfo->pSqlExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_BINARY, typeColLength,
pInfo->pSqlExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_BINARY, typeColLength + VARSTR_HEADER_SIZE,
typeColLength, false);
rowLen += typeColLength;
rowLen += typeColLength + VARSTR_HEADER_SIZE;
f.bytes = sizeof(int32_t);
f.type = TSDB_DATA_TYPE_INT;
@ -240,15 +243,15 @@ static int32_t tscBuildTableSchemaResultFields(SSqlObj *pSql, int32_t numOfCols,
rowLen += sizeof(int32_t);
f.bytes = noteColLength;
f.bytes = noteColLength + VARSTR_HEADER_SIZE;
f.type = TSDB_DATA_TYPE_BINARY;
tstrncpy(f.name, "Note", sizeof(f.name));
pInfo = tscFieldInfoAppend(&pQueryInfo->fieldsInfo, &f);
pInfo->pSqlExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_BINARY, noteColLength,
pInfo->pSqlExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_BINARY, noteColLength + VARSTR_HEADER_SIZE,
noteColLength, false);
rowLen += noteColLength;
rowLen += noteColLength + VARSTR_HEADER_SIZE;
return rowLen;
}

44
src/client/src/tscSecondaryMerge.c → src/client/src/tscLocalMerge.c
View File

@ -13,7 +13,6 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include "tscSecondaryMerge.h"
#include "os.h"
#include "tlosertree.h"
#include "tscUtil.h"
@ -21,6 +20,7 @@
#include "tsclient.h"
#include "tutil.h"
#include "tscLog.h"
#include "tscLocalMerge.h"
typedef struct SCompareParam {
SLocalDataSource **pLocalData;
@ -230,6 +230,7 @@ void tscCreateLocalReducer(tExtMemBuffer **pMemBuffer, int32_t numOfBuffer, tOrd
if (ds == NULL) {
tscError("%p failed to create merge structure", pSql);
pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY;
tfree(pReducer);
return;
}
@ -266,6 +267,7 @@ void tscCreateLocalReducer(tExtMemBuffer **pMemBuffer, int32_t numOfBuffer, tOrd
// no data actually, no need to merge result.
if (idx == 0) {
tfree(pReducer);
return;
}
@ -282,6 +284,7 @@ void tscCreateLocalReducer(tExtMemBuffer **pMemBuffer, int32_t numOfBuffer, tOrd
pRes->code = tLoserTreeCreate(&pReducer->pLoserTree, pReducer->numOfBuffer, param, treeComparator);
if (pReducer->pLoserTree == NULL || pRes->code != 0) {
tfree(pReducer);
return;
}
@ -325,7 +328,7 @@ void tscCreateLocalReducer(tExtMemBuffer **pMemBuffer, int32_t numOfBuffer, tOrd
tfree(pReducer->pResultBuf);
tfree(pReducer->pFinalRes);
tfree(pReducer->prevRowOfInput);
tfree(pReducer);
pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY;
return;
}
@ -353,7 +356,7 @@ void tscCreateLocalReducer(tExtMemBuffer **pMemBuffer, int32_t numOfBuffer, tOrd
pRes->numOfGroups = 0;
STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0);
STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta);;
STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta);
TSKEY stime = MIN(pQueryInfo->window.skey, pQueryInfo->window.ekey);
int64_t revisedSTime =
@ -410,13 +413,13 @@ static int32_t tscFlushTmpBufferImpl(tExtMemBuffer *pMemoryBuf, tOrderDescriptor
}
int32_t tscFlushTmpBuffer(tExtMemBuffer *pMemoryBuf, tOrderDescriptor *pDesc, tFilePage *pPage, int32_t orderType) {
int32_t ret = tscFlushTmpBufferImpl(pMemoryBuf, pDesc, pPage, orderType);
if (ret != 0) {
return -1;
int32_t ret = 0;
if ((ret = tscFlushTmpBufferImpl(pMemoryBuf, pDesc, pPage, orderType)) != 0) {
return ret;
}
if (!tExtMemBufferFlush(pMemoryBuf)) {
return -1;
if ((ret = tExtMemBufferFlush(pMemoryBuf)) != 0) {
return ret;
}
return 0;
@ -437,9 +440,9 @@ int32_t saveToBuffer(tExtMemBuffer *pMemoryBuf, tOrderDescriptor *pDesc, tFilePa
// current buffer is full, need to flushed to disk
assert(pPage->num == pModel->capacity);
int32_t ret = tscFlushTmpBuffer(pMemoryBuf, pDesc, pPage, orderType);
if (ret != 0) {
return -1;
int32_t code = tscFlushTmpBuffer(pMemoryBuf, pDesc, pPage, orderType);
if (code != 0) {
return code;
}
int32_t remain = numOfRows - numOfRemainEntries;
@ -455,8 +458,8 @@ int32_t saveToBuffer(tExtMemBuffer *pMemoryBuf, tOrderDescriptor *pDesc, tFilePa
tColModelAppend(pModel, pPage, data, numOfRows - remain, numOfWriteElems, numOfRows);
if (pPage->num == pModel->capacity) {
if (tscFlushTmpBuffer(pMemoryBuf, pDesc, pPage, orderType) != TSDB_CODE_SUCCESS) {
return -1;
if ((code = tscFlushTmpBuffer(pMemoryBuf, pDesc, pPage, orderType)) != TSDB_CODE_SUCCESS) {
return code;
}
} else {
pPage->num = numOfWriteElems;
@ -685,6 +688,7 @@ int32_t tscLocalReducerEnvCreate(SSqlObj *pSql, tExtMemBuffer ***pMemBuffer, tOr
if (createOrderDescriptor(pOrderDesc, pCmd, pModel) != TSDB_CODE_SUCCESS) {
pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY;
tfree(pSchema);
return pRes->code;
}
@ -1092,14 +1096,6 @@ static int64_t getNumOfResultLocal(SQueryInfo *pQueryInfo, SQLFunctionCtx *pCtx)
size_t size = tscSqlExprNumOfExprs(pQueryInfo);
for (int32_t j = 0; j < size; ++j) {
// SSqlExpr* pExpr = pQueryInfo->fieldsInfo.pSqlExpr[j];
// if (pExpr == NULL) {
// assert(pQueryInfo->fieldsInfo.pExpr[j] != NULL);
//
// maxOutput = 1;
// continue;
// }
/*
* ts, tag, tagprj function can not decide the output number of current query
* the number of output result is decided by main output
@ -1109,8 +1105,9 @@ static int64_t getNumOfResultLocal(SQueryInfo *pQueryInfo, SQLFunctionCtx *pCtx)
continue;
}
if (maxOutput < GET_RES_INFO(&pCtx[j])->numOfRes) {
maxOutput = GET_RES_INFO(&pCtx[j])->numOfRes;
SResultInfo* pResInfo = GET_RES_INFO(&pCtx[j]);
if (maxOutput < pResInfo->numOfRes) {
maxOutput = pResInfo->numOfRes;
}
}
@ -1260,7 +1257,6 @@ bool doGenerateFinalResults(SSqlObj *pSql, SLocalReducer *pLocalReducer, bool no
#ifdef _DEBUG_VIEW
printf("final result before interpo:\n");
assert(0);
// tColModelDisplay(pLocalReducer->resColModel, pLocalReducer->pBufForInterpo, pResBuf->num, pResBuf->num);
#endif

120
src/client/src/tscParseInsert.c
View File

@ -97,7 +97,7 @@ int tsParseTime(SSQLToken *pToken, int64_t *time, char **next, char *error, int1
useconds = str2int64(pToken->z);
} else {
// strptime("2001-11-12 18:31:01", "%Y-%m-%d %H:%M:%S", &tm);
if (taosParseTime(pToken->z, time, pToken->n, timePrec) != TSDB_CODE_SUCCESS) {
if (taosParseTime(pToken->z, time, pToken->n, timePrec, tsDaylight) != TSDB_CODE_SUCCESS) {
return tscInvalidSQLErrMsg(error, "invalid timestamp format", pToken->z);
}
@ -252,7 +252,7 @@ int32_t tsParseOneColumnData(SSchema *pSchema, SSQLToken *pToken, char *payload,
numType = tscToInteger(pToken, &iv, &endptr);
if (TK_ILLEGAL == numType) {
return tscInvalidSQLErrMsg(msg, "invalid bigint data", pToken->z);
} else if (errno == ERANGE || iv > INT64_MAX || iv <= INT64_MIN) {
} else if (errno == ERANGE || iv == INT64_MIN) {
return tscInvalidSQLErrMsg(msg, "bigint data overflow", pToken->z);
}
@ -594,7 +594,6 @@ int32_t tscAllocateMemIfNeed(STableDataBlocks *pDataBlock, int32_t rowSize, int3
size_t remain = pDataBlock->nAllocSize - pDataBlock->size;
const int factor = 5;
uint32_t nAllocSizeOld = pDataBlock->nAllocSize;
assert(pDataBlock->headerSize >= 0);
// expand the allocated size
if (remain < rowSize * factor) {
@ -1015,46 +1014,41 @@ static int32_t validateDataSource(SSqlCmd *pCmd, int8_t type, const char *sql) {
* @param pSql
* @return
*/
int doParseInsertSql(SSqlObj *pSql, char *str) {
int tsParseInsertSql(SSqlObj *pSql) {
SSqlCmd *pCmd = &pSql->cmd;
char* str = pCmd->curSql;
int32_t totalNum = 0;
int32_t code = TSDB_CODE_SUCCESS;
STableMetaInfo *pTableMetaInfo = NULL;
SQueryInfo *pQueryInfo = tscGetQueryInfoDetail(pCmd, 0);
assert(pQueryInfo != NULL);
STableMetaInfo *pTableMetaInfo = NULL;
if (pQueryInfo->numOfTables == 0) {
pTableMetaInfo = tscAddEmptyMetaInfo(pQueryInfo);
} else {
pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
}
// TODO: 2048 is added because TSDB_MAX_TAGS_LEN now is 65536
// but TSDB_PAYLOAD_SIZE is 65380
// TODO: 2048 is added because TSDB_MAX_TAGS_LEN now is 65536, but TSDB_PAYLOAD_SIZE is 65380
if ((code = tscAllocPayload(pCmd, TSDB_PAYLOAD_SIZE + 2048)) != TSDB_CODE_SUCCESS) {
return code;
}
assert(((NULL == pCmd->curSql) && (NULL == pCmd->pTableList))
|| ((NULL != pCmd->curSql) && (NULL != pCmd->pTableList)));
if ((NULL == pCmd->curSql) && (NULL == pCmd->pTableList)) {
if (NULL == pCmd->pTableList) {
pCmd->pTableList = taosHashInit(128, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), false);
pSql->cmd.pDataBlocks = tscCreateBlockArrayList();
if (NULL == pCmd->pTableList || NULL == pSql->cmd.pDataBlocks) {
code = TSDB_CODE_TSC_OUT_OF_MEMORY;
goto _error_clean;
goto _error;
}
} else {
assert((NULL != pCmd->curSql) && (NULL != pCmd->pTableList));
str = pCmd->curSql;
}
tscTrace("%p create data block list for submit data:%p, curSql:%p, pTableList:%p", pSql, pSql->cmd.pDataBlocks, pCmd->curSql, pCmd->pTableList);
tscTrace("%p create data block list for submit data:%p, pTableList:%p", pSql, pCmd->pDataBlocks, pCmd->pTableList);
while (1) {
int32_t index = 0;
@ -1076,7 +1070,7 @@ int doParseInsertSql(SSqlObj *pSql, char *str) {
*/
if (totalNum == 0) {
code = TSDB_CODE_TSC_INVALID_SQL;
goto _error_clean;
goto _error;
} else {
break;
}
@ -1087,37 +1081,30 @@ int doParseInsertSql(SSqlObj *pSql, char *str) {
// Check if the table name available or not
if (validateTableName(sToken.z, sToken.n) != TSDB_CODE_SUCCESS) {
code = tscInvalidSQLErrMsg(pCmd->payload, "table name invalid", sToken.z);
goto _error_clean;
goto _error;
}
if ((code = tscSetTableFullName(pTableMetaInfo, &sToken, pSql)) != TSDB_CODE_SUCCESS) {
goto _error_clean;
goto _error;
}
ptrdiff_t pos = pCmd->curSql - pSql->sqlstr;
if ((code = tscCheckIfCreateTable(&str, pSql)) != TSDB_CODE_SUCCESS) {
/*
* For async insert, after get the table meta from server, the sql string will not be
* parsed using the new table meta to avoid the overhead cause by get table meta data information.
* And during the getMeterMetaCallback function, the sql string will be parsed from the
* interrupted position.
* After retrieving the table meta from server, the sql string will be parsed from the paused position.
* And during the getTableMetaCallback function, the sql string will be parsed from the paused position.
*/
if (TSDB_CODE_TSC_ACTION_IN_PROGRESS == code) {
tscTrace("%p waiting for get table meta during insert, then resume from offset: %" PRId64 " , %s", pSql,
pos, pCmd->curSql);
return code;
}
// todo add to return
tscError("%p async insert parse error, code:%d, %s", pSql, code, tstrerror(code));
tscError("%p async insert parse error, code:%s", pSql, tstrerror(code));
pCmd->curSql = NULL;
goto _error_clean; // TODO: should _clean or _error_clean to async flow ????
goto _error;
}
if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) {
code = tscInvalidSQLErrMsg(pCmd->payload, "insert data into super table is not supported", NULL);
goto _error_clean;
goto _error;
}
index = 0;
@ -1126,7 +1113,7 @@ int doParseInsertSql(SSqlObj *pSql, char *str) {
if (sToken.n == 0) {
code = tscInvalidSQLErrMsg(pCmd->payload, "keyword VALUES or FILE required", sToken.z);
goto _error_clean;
goto _error;
}
STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta);
@ -1138,7 +1125,7 @@ int doParseInsertSql(SSqlObj *pSql, char *str) {
tscSetAssignedColumnInfo(&spd, pSchema, tinfo.numOfColumns);
if (validateDataSource(pCmd, DATA_FROM_SQL_STRING, sToken.z) != TSDB_CODE_SUCCESS) {
goto _error_clean;
goto _error;
}
/*
@ -1147,11 +1134,11 @@ int doParseInsertSql(SSqlObj *pSql, char *str) {
*/
code = doParseInsertStatement(pSql, pCmd->pTableList, &str, &spd, &totalNum);
if (code != TSDB_CODE_SUCCESS) {
goto _error_clean;
goto _error;
}
} else if (sToken.type == TK_FILE) {
if (validateDataSource(pCmd, DATA_FROM_DATA_FILE, sToken.z) != TSDB_CODE_SUCCESS) {
goto _error_clean;
goto _error;
}
index = 0;
@ -1159,7 +1146,7 @@ int doParseInsertSql(SSqlObj *pSql, char *str) {
str += index;
if (sToken.n == 0) {
code = tscInvalidSQLErrMsg(pCmd->payload, "file path is required following keyword FILE", sToken.z);
goto _error_clean;
goto _error;
}
char fname[PATH_MAX] = {0};
@ -1169,7 +1156,7 @@ int doParseInsertSql(SSqlObj *pSql, char *str) {
wordexp_t full_path;
if (wordexp(fname, &full_path, 0) != 0) {
code = tscInvalidSQLErrMsg(pCmd->payload, "invalid filename", sToken.z);
goto _error_clean;
goto _error;
}
strcpy(fname, full_path.we_wordv[0]);
wordfree(&full_path);
@ -1180,7 +1167,7 @@ int doParseInsertSql(SSqlObj *pSql, char *str) {
int32_t ret = tscCreateDataBlock(PATH_MAX, tinfo.rowSize, sizeof(SSubmitBlk), pTableMetaInfo->name,
pTableMeta, &pDataBlock);
if (ret != TSDB_CODE_SUCCESS) {
goto _error_clean;
goto _error;
}
tscAppendDataBlock(pCmd->pDataBlocks, pDataBlock);
@ -1191,7 +1178,7 @@ int doParseInsertSql(SSqlObj *pSql, char *str) {
SSchema * pSchema = tscGetTableSchema(pTableMeta);
if (validateDataSource(pCmd, DATA_FROM_SQL_STRING, sToken.z) != TSDB_CODE_SUCCESS) {
goto _error_clean;
goto _error;
}
SParsedDataColInfo spd = {0};
@ -1227,7 +1214,7 @@ int doParseInsertSql(SSqlObj *pSql, char *str) {
if (spd.hasVal[t] == true) {
code = tscInvalidSQLErrMsg(pCmd->payload, "duplicated column name", sToken.z);
goto _error_clean;
goto _error;
}
spd.hasVal[t] = true;
@ -1238,13 +1225,13 @@ int doParseInsertSql(SSqlObj *pSql, char *str) {
if (!findColumnIndex) {
code = tscInvalidSQLErrMsg(pCmd->payload, "invalid column name", sToken.z);
goto _error_clean;
goto _error;
}
}
if (spd.numOfAssignedCols == 0 || spd.numOfAssignedCols > tinfo.numOfColumns) {
code = tscInvalidSQLErrMsg(pCmd->payload, "column name expected", sToken.z);
goto _error_clean;
goto _error;
}
index = 0;
@ -1253,16 +1240,16 @@ int doParseInsertSql(SSqlObj *pSql, char *str) {
if (sToken.type != TK_VALUES) {
code = tscInvalidSQLErrMsg(pCmd->payload, "keyword VALUES is expected", sToken.z);
goto _error_clean;
goto _error;
}
code = doParseInsertStatement(pSql, pCmd->pTableList, &str, &spd, &totalNum);
if (code != TSDB_CODE_SUCCESS) {
goto _error_clean;
goto _error;
}
} else {
code = tscInvalidSQLErrMsg(pCmd->payload, "keyword VALUES or FILE are required", sToken.z);
goto _error_clean;
goto _error;
}
}
@ -1273,7 +1260,7 @@ int doParseInsertSql(SSqlObj *pSql, char *str) {
if (pCmd->pDataBlocks->nSize > 0) { // merge according to vgId
if ((code = tscMergeTableDataBlocks(pSql, pCmd->pDataBlocks)) != TSDB_CODE_SUCCESS) {
goto _error_clean;
goto _error;
}
} else {
pCmd->pDataBlocks = tscDestroyBlockArrayList(pCmd->pDataBlocks);
@ -1282,7 +1269,7 @@ int doParseInsertSql(SSqlObj *pSql, char *str) {
code = TSDB_CODE_SUCCESS;
goto _clean;
_error_clean:
_error:
pCmd->pDataBlocks = tscDestroyBlockArrayList(pCmd->pDataBlocks);
_clean:
@ -1295,7 +1282,7 @@ _clean:
return code;
}
int tsParseInsertSql(SSqlObj *pSql) {
int tsInsertInitialCheck(SSqlObj *pSql) {
if (!pSql->pTscObj->writeAuth) {
return TSDB_CODE_TSC_NO_WRITE_AUTH;
}
@ -1313,28 +1300,23 @@ int tsParseInsertSql(SSqlObj *pSql) {
SQueryInfo *pQueryInfo = NULL;
tscGetQueryInfoDetailSafely(pCmd, pCmd->clauseIndex, &pQueryInfo);
TSDB_QUERY_SET_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_INSERT);
TSDB_QUERY_SET_TYPE(pQueryInfo->type, pCmd->insertType);
TSDB_QUERY_SET_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_INSERT | pCmd->insertType);
sToken = tStrGetToken(pSql->sqlstr, &index, false, 0, NULL);
if (sToken.type != TK_INTO) {
return tscInvalidSQLErrMsg(pCmd->payload, "keyword INTO is expected", sToken.z);
}
return doParseInsertSql(pSql, pSql->sqlstr + index);
pCmd->curSql = sToken.z + sToken.n;
return TSDB_CODE_SUCCESS;
}
int tsParseSql(SSqlObj *pSql, bool initialParse) {
int tsParseSql(SSqlObj *pSql, bool initial) {
int32_t ret = TSDB_CODE_SUCCESS;
if (initialParse) {
char* p = pSql->sqlstr;
pSql->sqlstr = NULL;
tscPartiallyFreeSqlObj(pSql);
pSql->sqlstr = p;
} else {
tscTrace("continue parse sql: %s", pSql->cmd.curSql);
SSqlCmd* pCmd = &pSql->cmd;
if ((!pCmd->parseFinished) && (!initial)) {
tscTrace("%p resume to parse sql: %s", pSql, pCmd->curSql);
}
if (tscIsInsertData(pSql->sqlstr)) {
@ -1342,18 +1324,22 @@ int tsParseSql(SSqlObj *pSql, bool initialParse) {
* Set the fp before parse the sql string, in case of getTableMeta failed, in which
* the error handle callback function can rightfully restore the user-defined callback function (fp).
*/
if (initialParse && (pSql->cmd.insertType != TSDB_QUERY_TYPE_STMT_INSERT)) {
if (initial && (pSql->cmd.insertType != TSDB_QUERY_TYPE_STMT_INSERT)) {
pSql->fetchFp = pSql->fp;
pSql->fp = (void(*)())tscHandleMultivnodeInsert;
}
if (initial && ((ret = tsInsertInitialCheck(pSql)) != TSDB_CODE_SUCCESS)) {
return ret;
}
ret = tsParseInsertSql(pSql);
} else {
ret = tscAllocPayload(&pSql->cmd, TSDB_DEFAULT_PAYLOAD_SIZE);
if (TSDB_CODE_SUCCESS != ret) {
return ret;
}
SSqlInfo SQLInfo = qSQLParse(pSql->sqlstr);
ret = tscToSQLCmd(pSql, &SQLInfo);
SQLInfoDestroy(&SQLInfo);
@ -1361,11 +1347,9 @@ int tsParseSql(SSqlObj *pSql, bool initialParse) {
/*
* the pRes->code may be modified or released by another thread in tscTableMetaCallBack function,
* so do NOT use pRes->code to determine if the getTableMeta/getMetricMeta function
* invokes new threads to get data from mnode or simply retrieves data from cache.
*
* do NOT assign return code to pRes->code for the same reason since it may be released by another thread
* pRes->code = ret;
* so do NOT use pRes->code to determine if the getTableMeta function
* invokes new threads to get data from mgmt node or simply retrieves data from cache.
* do NOT assign return code to pRes->code for the same reason since it may be released by another thread already.
*/
return ret;
}

290
src/client/src/tscSQLParser.c
View File

@ -98,8 +98,6 @@ static int32_t setKillInfo(SSqlObj* pSql, struct SSqlInfo* pInfo, int32_t killTy
static bool validateOneTags(SSqlCmd* pCmd, TAOS_FIELD* pTagField);
static bool hasTimestampForPointInterpQuery(SQueryInfo* pQueryInfo);
static void updateTagColumnIndex(SQueryInfo* pQueryInfo, int32_t tableIndex);
static int32_t parseLimitClause(SQueryInfo* pQueryInfo, int32_t index, SQuerySQL* pQuerySql, SSqlObj* pSql);
static int32_t parseCreateDBOptions(SSqlCmd* pCmd, SCreateDBInfo* pCreateDbSql);
static int32_t getColumnIndexByName(const SSQLToken* pToken, SQueryInfo* pQueryInfo, SColumnIndex* pIndex);
@ -138,7 +136,7 @@ static int setColumnFilterInfoForTimestamp(SQueryInfo* pQueryInfo, tVariant* pVa
STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta);
if (seg != NULL) {
if (taosParseTime(pVar->pz, &time, pVar->nLen, tinfo.precision) != TSDB_CODE_SUCCESS) {
if (taosParseTime(pVar->pz, &time, pVar->nLen, tinfo.precision, tsDaylight) != TSDB_CODE_SUCCESS) {
return invalidSqlErrMsg(pQueryInfo->msg, msg);
}
} else {
@ -640,17 +638,11 @@ int32_t parseIntervalClause(SQueryInfo* pQueryInfo, SQuerySQL* pQuerySql) {
return TSDB_CODE_TSC_INVALID_SQL;
}
SSchema s = {.bytes = TSDB_KEYSIZE, .type = TSDB_DATA_TYPE_TIMESTAMP, .colId = PRIMARYKEY_TIMESTAMP_COL_INDEX};
tstrncpy(s.name, aAggs[TSDB_FUNC_TS].aName, sizeof(s.name));
SColumnIndex index = {tableIndex, PRIMARYKEY_TIMESTAMP_COL_INDEX};
SSqlExpr* pExpr = tscSqlExprInsert(pQueryInfo, 0, TSDB_FUNC_TS, &index, TSDB_DATA_TYPE_TIMESTAMP, TSDB_KEYSIZE,
TSDB_KEYSIZE, false);
SColumnList ids = getColumnList(1, 0, PRIMARYKEY_TIMESTAMP_COL_INDEX);
int32_t ret =
insertResultField(pQueryInfo, 0, &ids, TSDB_KEYSIZE, TSDB_DATA_TYPE_TIMESTAMP, aAggs[TSDB_FUNC_TS].aName, pExpr);
if (ret != TSDB_CODE_SUCCESS) {
return ret;
}
tscAddSpecialColumnForSelect(pQueryInfo, 0, TSDB_FUNC_TS, &index, &s, TSDB_COL_NORMAL);
if (parseSlidingClause(pQueryInfo, pQuerySql) != TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_SQL;
@ -760,6 +752,10 @@ static bool validateTableColumnInfo(tFieldList* pFieldList, SSqlCmd* pCmd) {
int32_t nLen = 0;
for (int32_t i = 0; i < pFieldList->nField; ++i) {
if (pFieldList->p[i].bytes == 0) {
invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5);
return false;
}
nLen += pFieldList->p[i].bytes;
}
@ -816,6 +812,10 @@ static bool validateTagParams(tFieldList* pTagsList, tFieldList* pFieldList, SSq
int32_t nLen = 0;
for (int32_t i = 0; i < pTagsList->nField; ++i) {
if (pTagsList->p[i].bytes == 0) {
invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg7);
return false;
}
nLen += pTagsList->p[i].bytes;
}
@ -1042,7 +1042,7 @@ int32_t setObjFullName(char* fullName, const char* account, SSQLToken* pDB, SSQL
/* db name is not specified, the tableName dose not include db name */
if (pDB != NULL) {
if (pDB->n >= TSDB_DB_NAME_LEN) {
if (pDB->n >= TSDB_ACCT_LEN + TSDB_DB_NAME_LEN) {
return TSDB_CODE_TSC_INVALID_SQL;
}
@ -1241,11 +1241,11 @@ int32_t parseSelectClause(SSqlCmd* pCmd, int32_t clauseIndex, tSQLExprList* pSel
return invalidSqlErrMsg(pQueryInfo->msg, msg2);
}
/*
* transfer sql functions that need secondary merge into another format
* in dealing with metric queries such as: count/first/last
*/
if (isSTable) {
/*
* transfer sql functions that need secondary merge into another format
* in dealing with metric queries such as: count/first/last
*/
tscTansformSQLFuncForSTableQuery(pQueryInfo);
if (hasUnsupportFunctionsForSTableQuery(pQueryInfo)) {
@ -1272,7 +1272,7 @@ int32_t insertResultField(SQueryInfo* pQueryInfo, int32_t outputIndex, SColumnLi
}
TAOS_FIELD f = tscCreateField(type, fieldName, bytes);
SFieldSupInfo* pInfo =tscFieldInfoInsert(&pQueryInfo->fieldsInfo, outputIndex, &f);
SFieldSupInfo* pInfo = tscFieldInfoInsert(&pQueryInfo->fieldsInfo, outputIndex, &f);
pInfo->pSqlExpr = pSqlExpr;
return TSDB_CODE_SUCCESS;
@ -1324,8 +1324,9 @@ static void addProjectQueryCol(SQueryInfo* pQueryInfo, int32_t startPos, SColumn
void tscAddSpecialColumnForSelect(SQueryInfo* pQueryInfo, int32_t outputColIndex, int16_t functionId,
SColumnIndex* pIndex, SSchema* pColSchema, int16_t flag) {
SSqlExpr* pExpr = tscSqlExprAppend(pQueryInfo, functionId, pIndex, pColSchema->type,
SSqlExpr* pExpr = tscSqlExprInsert(pQueryInfo, outputColIndex, functionId, pIndex, pColSchema->type,
pColSchema->bytes, pColSchema->bytes, flag);
tstrncpy(pExpr->aliasName, pColSchema->name, sizeof(pExpr->aliasName));
SColumnList ids = getColumnList(1, pIndex->tableIndex, pIndex->columnIndex);
if (TSDB_COL_IS_TAG(flag)) {
@ -1403,7 +1404,7 @@ int32_t addProjectionExprAndResultField(SQueryInfo* pQueryInfo, tSQLExprItem* pI
if (index.columnIndex == TSDB_TBNAME_COLUMN_INDEX) {
SSchema colSchema = tGetTableNameColumnSchema();
tscAddSpecialColumnForSelect(pQueryInfo, startPos, TSDB_FUNC_TAGPRJ, &index, &colSchema, true);
tscAddSpecialColumnForSelect(pQueryInfo, startPos, TSDB_FUNC_TAGPRJ, &index, &colSchema, TSDB_COL_TAG);
} else {
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex);
STableMeta* pTableMeta = pTableMetaInfo->pTableMeta;
@ -1445,13 +1446,20 @@ static int32_t setExprInfoForFunctions(SQueryInfo* pQueryInfo, SSchema* pSchema,
}
if (aliasName != NULL) {
strcpy(columnName, aliasName);
tstrncpy(columnName, aliasName, sizeof(columnName));
} else {
getRevisedName(columnName, functionID, sizeof(columnName) - 1, pSchema[pColIndex->columnIndex].name);
}
SSqlExpr* pExpr = tscSqlExprAppend(pQueryInfo, functionID, pColIndex, type, bytes, bytes, false);
tstrncpy(pExpr->aliasName, columnName, sizeof(pExpr->aliasName));
// set reverse order scan data blocks for last query
if (functionID == TSDB_FUNC_LAST) {
pExpr->numOfParams = 1;
pExpr->param[0].i64Key = TSDB_ORDER_DESC;
pExpr->param[0].nType = TSDB_DATA_TYPE_INT;
}
// for all queries, the timestamp column needs to be loaded
SColumnIndex index = {.tableIndex = pColIndex->tableIndex, .columnIndex = PRIMARYKEY_TIMESTAMP_COL_INDEX};
@ -1474,7 +1482,8 @@ int32_t addExprAndResultField(SQueryInfo* pQueryInfo, int32_t colIndex, tSQLExpr
const char* msg5 = "parameter is out of range [0, 100]";
const char* msg6 = "function applied to tags not allowed";
const char* msg7 = "normal table can not apply this function";
const char* msg8 = "multi-columns selection does not support alias column name";
switch (optr) {
case TK_COUNT: {
if (pItem->pNode->pParam != NULL && pItem->pNode->pParam->nExpr != 1) {
@ -1681,6 +1690,10 @@ int32_t addExprAndResultField(SQueryInfo* pQueryInfo, int32_t colIndex, tSQLExpr
return invalidSqlErrMsg(pQueryInfo->msg, msg3);
}
if (pItem->pNode->pParam->nExpr > 1 && (pItem->aliasName != NULL && strlen(pItem->aliasName) > 0)) {
return invalidSqlErrMsg(pQueryInfo->msg, msg8);
}
/* in first/last function, multiple columns can be add to resultset */
for (int32_t i = 0; i < pItem->pNode->pParam->nExpr; ++i) {
tSQLExprItem* pParamElem = &(pItem->pNode->pParam->a[i]);
@ -1724,6 +1737,22 @@ int32_t addExprAndResultField(SQueryInfo* pQueryInfo, int32_t colIndex, tSQLExpr
if (setExprInfoForFunctions(pQueryInfo, pSchema, functionID, pItem->aliasName, colIndex + i, &index) != 0) {
return TSDB_CODE_TSC_INVALID_SQL;
}
if (optr == TK_LAST) { // todo refactor
SSqlGroupbyExpr* pGroupBy = &pQueryInfo->groupbyExpr;
if (pGroupBy->numOfGroupCols > 0) {
for(int32_t k = 0; k < pGroupBy->numOfGroupCols; ++k) {
SColIndex* pIndex = taosArrayGet(pGroupBy->columnInfo, k);
if (!TSDB_COL_IS_TAG(pIndex->flag) && pIndex->colIndex < tscGetNumOfColumns(pTableMetaInfo->pTableMeta)) { // group by normal columns
SSqlExpr* pExpr = taosArrayGetP(pQueryInfo->exprList, colIndex + i);
pExpr->numOfParams = 1;
pExpr->param->i64Key = TSDB_ORDER_ASC;
break;
}
}
}
}
}
}
@ -1731,6 +1760,11 @@ int32_t addExprAndResultField(SQueryInfo* pQueryInfo, int32_t colIndex, tSQLExpr
} else { // select * from xxx
int32_t numOfFields = 0;
// multicolumn selection does not support alias name
if (pItem->aliasName != NULL && strlen(pItem->aliasName) > 0) {
return invalidSqlErrMsg(pQueryInfo->msg, msg8);
}
for (int32_t j = 0; j < pQueryInfo->numOfTables; ++j) {
pTableMetaInfo = tscGetMetaInfo(pQueryInfo, j);
SSchema* pSchema = tscGetTableSchema(pTableMetaInfo->pTableMeta);
@ -1823,7 +1857,7 @@ int32_t addExprAndResultField(SQueryInfo* pQueryInfo, int32_t colIndex, tSQLExpr
} else {
tVariantDump(pVariant, val, TSDB_DATA_TYPE_BIGINT, true);
int64_t nTop = *((int32_t*)val);
int64_t nTop = GET_INT32_VAL(val);
if (nTop <= 0 || nTop > 100) { // todo use macro
return invalidSqlErrMsg(pQueryInfo->msg, msg5);
}
@ -1833,12 +1867,14 @@ int32_t addExprAndResultField(SQueryInfo* pQueryInfo, int32_t colIndex, tSQLExpr
return TSDB_CODE_TSC_INVALID_SQL;
}
// todo REFACTOR
// set the first column ts for top/bottom query
SColumnIndex index1 = {0, PRIMARYKEY_TIMESTAMP_COL_INDEX};
pExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TS, &index1, TSDB_DATA_TYPE_TIMESTAMP, TSDB_KEYSIZE,
TSDB_KEYSIZE, false);
tstrncpy(pExpr->aliasName, aAggs[TSDB_FUNC_TS].aName, sizeof(pExpr->aliasName));
const int32_t TS_COLUMN_INDEX = 0;
const int32_t TS_COLUMN_INDEX = PRIMARYKEY_TIMESTAMP_COL_INDEX;
SColumnList ids = getColumnList(1, 0, TS_COLUMN_INDEX);
insertResultField(pQueryInfo, TS_COLUMN_INDEX, &ids, TSDB_KEYSIZE, TSDB_DATA_TYPE_TIMESTAMP,
aAggs[TSDB_FUNC_TS].aName, pExpr);
@ -1990,6 +2026,7 @@ static int16_t doGetColumnIndex(SQueryInfo* pQueryInfo, int32_t index, SSQLToken
if (strncasecmp(pSchema[i].name, pToken->z, pToken->n) == 0) {
columnIndex = i;
break;
}
}
@ -2197,7 +2234,6 @@ int32_t setShowInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
// db prefix in tagCond, show table conds in payload
SSQLToken* pDbPrefixToken = &pShowInfo->prefix;
if (pDbPrefixToken->type != 0) {
assert(pDbPrefixToken->n >= 0);
if (pDbPrefixToken->n >= TSDB_DB_NAME_LEN) { // db name is too long
return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3);
@ -2446,62 +2482,10 @@ static bool functionCompatibleCheck(SQueryInfo* pQueryInfo) {
return true;
}
void updateTagColumnIndex(SQueryInfo* pQueryInfo, int32_t tableIndex) {
// STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, tableIndex);
//
// // update tags column index for expression
// size_t size = tscSqlExprNumOfExprs(pQueryInfo);
// for (int32_t i = 0; i < size; ++i) {
// SSqlExpr* pExpr = tscSqlExprGet(pQueryInfo, i);
//
// if (!TSDB_COL_IS_TAG(pExpr->colInfo.flag)) { // not tags, continue
// continue;
// }
//
// // not belongs to this table
// if (pExpr->uid != pTableMetaInfo->pTableMeta->uid) {
// continue;
// }
// for (int32_t j = 0; j < pTableMetaInfo->numOfTags; ++j) {
// if (pExpr->colInfo.colIndex == pTableMetaInfo->tagColumnIndex[j]) {
// pExpr->colInfo.colIndex = j;
// break;
// }
// }
// }
// update join condition tag column index
// SJoinInfo* pJoinInfo = &pQueryInfo->tagCond.joinInfo;
// if (!pJoinInfo->hasJoin) { // not join query
// return;
// }
//
// assert(pJoinInfo->left.uid != pJoinInfo->right.uid);
//
// // the join condition expression node belongs to this table(super table)
// assert(0);
// if (pTableMetaInfo->pTableMeta->uid == pJoinInfo->left.uid) {
// for (int32_t i = 0; i < pTableMetaInfo->numOfTags; ++i) {
// if (pJoinInfo->left.tagCol == pTableMetaInfo->tagColumnIndex[i]) {
// pJoinInfo->left.tagCol = i;
// }
// }
// }
//
// if (pTableMetaInfo->pTableMeta->uid == pJoinInfo->right.uid) {
// for (int32_t i = 0; i < pTableMetaInfo->numOfTags; ++i) {
// if (pJoinInfo->right.tagCol == pTableMetaInfo->tagColumnIndex[i]) {
// pJoinInfo->right.tagCol = i;
// }
// }
// }
}
int32_t parseGroupbyClause(SQueryInfo* pQueryInfo, tVariantList* pList, SSqlCmd* pCmd) {
const char* msg1 = "too many columns in group by clause";
const char* msg2 = "invalid column name in group by clause";
const char* msg3 = "group by columns must belong to one table";
// const char* msg3 = "group by columns must belong to one table";
const char* msg7 = "not support group by expression";
const char* msg8 = "not allowed column type for group by";
const char* msg9 = "tags not allowed for table query";
@ -2537,10 +2521,6 @@ int32_t parseGroupbyClause(SQueryInfo* pQueryInfo, tVariantList* pList, SSqlCmd*
return invalidSqlErrMsg(pQueryInfo->msg, msg2);
}
if (tableIndex != index.tableIndex && tableIndex >= 0) {
return invalidSqlErrMsg(pQueryInfo->msg, msg3);
}
tableIndex = index.tableIndex;
pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex);
@ -2586,9 +2566,7 @@ int32_t parseGroupbyClause(SQueryInfo* pQueryInfo, tVariantList* pList, SSqlCmd*
tscColumnListInsert(pQueryInfo->colList, &index);
SColIndex colIndex = {
.colIndex = index.columnIndex, .flag = TSDB_COL_NORMAL, .colId = pSchema->colId,
};
SColIndex colIndex = { .colIndex = index.columnIndex, .flag = TSDB_COL_NORMAL, .colId = pSchema->colId };
taosArrayPush(pGroupExpr->columnInfo, &colIndex);
pQueryInfo->groupbyExpr.orderType = TSDB_ORDER_ASC;
@ -2599,7 +2577,6 @@ int32_t parseGroupbyClause(SQueryInfo* pQueryInfo, tVariantList* pList, SSqlCmd*
}
pQueryInfo->groupbyExpr.tableIndex = tableIndex;
return TSDB_CODE_SUCCESS;
}
@ -2655,7 +2632,7 @@ static int32_t doExtractColumnFilterInfo(SQueryInfo* pQueryInfo, SColumnFilterIn
tVariantDump(&pRight->val, (char*)&pColumnFilter->upperBndd, colType, false);
} else { // TK_GT,TK_GE,TK_EQ,TK_NE are based on the pColumn->lowerBndd
if (colType == TSDB_DATA_TYPE_BINARY) {
pColumnFilter->pz = (int64_t)calloc(1, pRight->val.nLen + 1);
pColumnFilter->pz = (int64_t)calloc(1, pRight->val.nLen + TSDB_NCHAR_SIZE);
pColumnFilter->len = pRight->val.nLen;
tVariantDump(&pRight->val, (char*)pColumnFilter->pz, colType, false);
@ -2665,7 +2642,7 @@ static int32_t doExtractColumnFilterInfo(SQueryInfo* pQueryInfo, SColumnFilterIn
tVariantDump(&pRight->val, (char*)pColumnFilter->pz, colType, false);
size_t len = wcslen((wchar_t*)pColumnFilter->pz);
size_t len = twcslen((wchar_t*)pColumnFilter->pz);
pColumnFilter->len = len * TSDB_NCHAR_SIZE;
} else {
tVariantDump(&pRight->val, (char*)&pColumnFilter->lowerBndd, colType, false);
@ -2886,7 +2863,8 @@ static int32_t extractColumnFilterInfo(SQueryInfo* pQueryInfo, SColumnIndex* pIn
SSchema* pSchema = tscGetTableColumnSchema(pTableMeta, pIndex->columnIndex);
const char* msg1 = "non binary column not support like operator";
const char* msg2 = "binary column not support this operator";
const char* msg2 = "binary column not support this operator";
const char* msg3 = "bool column not support this operator";
SColumn* pColumn = tscColumnListInsert(pQueryInfo->colList, pIndex);
SColumnFilterInfo* pColFilter = NULL;
@ -2920,6 +2898,12 @@ static int32_t extractColumnFilterInfo(SQueryInfo* pQueryInfo, SColumnIndex* pIn
if (pExpr->nSQLOptr == TK_LIKE) {
return invalidSqlErrMsg(pQueryInfo->msg, msg1);
}
if (pSchema->type == TSDB_DATA_TYPE_BOOL) {
if (pExpr->nSQLOptr != TK_EQ && pExpr->nSQLOptr != TK_NE) {
return invalidSqlErrMsg(pQueryInfo->msg, msg3);
}
}
}
pColumn->colIndex = *pIndex;
@ -3022,14 +3006,17 @@ static int32_t getColumnQueryCondInfo(SQueryInfo* pQueryInfo, tSQLExpr* pExpr, i
}
static int32_t getJoinCondInfo(SQueryInfo* pQueryInfo, tSQLExpr* pExpr) {
const char* msg = "invalid join query condition";
const char* msg1 = "invalid join query condition";
const char* msg2 = "join on binary/nchar not supported";
const char* msg3 = "type of join columns must be identical";
const char* msg4 = "invalid column name in join condition";
if (pExpr == NULL) {
return TSDB_CODE_SUCCESS;
}
if (!isExprDirectParentOfLeaftNode(pExpr)) {
return invalidSqlErrMsg(pQueryInfo->msg, msg);
return invalidSqlErrMsg(pQueryInfo->msg, msg1);
}
STagCond* pTagCond = &pQueryInfo->tagCond;
@ -3038,28 +3025,36 @@ static int32_t getJoinCondInfo(SQueryInfo* pQueryInfo, tSQLExpr* pExpr) {
SColumnIndex index = COLUMN_INDEX_INITIALIZER;
if (getColumnIndexByName(&pExpr->pLeft->colInfo, pQueryInfo, &index) != TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_SQL;
return invalidSqlErrMsg(pQueryInfo->msg, msg4);
}
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex);
int16_t tagColIndex = index.columnIndex - tscGetNumOfColumns(pTableMetaInfo->pTableMeta);
SSchema* pTagSchema1 = tscGetTableColumnSchema(pTableMetaInfo->pTableMeta, index.columnIndex);
pLeft->uid = pTableMetaInfo->pTableMeta->uid;
pLeft->tagCol = tagColIndex;
pLeft->tagColId = pTagSchema1->colId;
strcpy(pLeft->tableId, pTableMetaInfo->name);
index = (SColumnIndex)COLUMN_INDEX_INITIALIZER;
if (getColumnIndexByName(&pExpr->pRight->colInfo, pQueryInfo, &index) != TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_SQL;
return invalidSqlErrMsg(pQueryInfo->msg, msg4);
}
pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex);
tagColIndex = index.columnIndex - tscGetNumOfColumns(pTableMetaInfo->pTableMeta);
SSchema* pTagSchema2 = tscGetTableColumnSchema(pTableMetaInfo->pTableMeta, index.columnIndex);
pRight->uid = pTableMetaInfo->pTableMeta->uid;
pRight->tagCol = tagColIndex;
pRight->tagColId = pTagSchema2->colId;
strcpy(pRight->tableId, pTableMetaInfo->name);
if (pTagSchema1->type != pTagSchema2->type) {
return invalidSqlErrMsg(pQueryInfo->msg, msg3);
}
if (pTagSchema1->type == TSDB_DATA_TYPE_BINARY || pTagSchema1->type == TSDB_DATA_TYPE_NCHAR) {
return invalidSqlErrMsg(pQueryInfo->msg, msg2);
}
pTagCond->joinInfo.hasJoin = true;
return TSDB_CODE_SUCCESS;
}
@ -3787,6 +3782,10 @@ static int32_t getTagQueryCondExpr(SQueryInfo* pQueryInfo, SCondExpr* pCondExpr,
for (int32_t i = 0; i < pQueryInfo->numOfTables; ++i) {
tSQLExpr* p1 = extractExprForSTable(pExpr, pQueryInfo, i);
if (p1 == NULL) { // no query condition on this table
continue;
}
tExprNode* p = NULL;
SArray* colList = taosArrayInit(10, sizeof(SColIndex));
@ -3921,7 +3920,7 @@ int32_t getTimeRange(STimeWindow* win, tSQLExpr* pRight, int32_t optr, int16_t t
char* seg = strnchr(pRight->val.pz, '-', pRight->val.nLen, false);
if (seg != NULL) {
if (taosParseTime(pRight->val.pz, &val, pRight->val.nLen, TSDB_TIME_PRECISION_MICRO) == TSDB_CODE_SUCCESS) {
if (taosParseTime(pRight->val.pz, &val, pRight->val.nLen, TSDB_TIME_PRECISION_MICRO, tsDaylight) == TSDB_CODE_SUCCESS) {
parsed = true;
} else {
return TSDB_CODE_TSC_INVALID_SQL;
@ -4479,7 +4478,7 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
if (pTagsSchema->type != TSDB_DATA_TYPE_BINARY && pTagsSchema->type != TSDB_DATA_TYPE_NCHAR) {
len = tDataTypeDesc[pTagsSchema->type].nSize;
} else {
len = varDataLen(pUpdateMsg->data);
len = varDataTLen(pUpdateMsg->data);
}
pUpdateMsg->tagValLen = htonl(len); // length may be changed after dump data
@ -4736,7 +4735,7 @@ int32_t parseLimitClause(SQueryInfo* pQueryInfo, int32_t clauseIndex, SQuerySQL*
pQueryInfo->clauseLimit = pQueryInfo->limit.limit;
pQueryInfo->slimit = pQuerySql->slimit;
tscTrace("%p limit:%d, offset:%" PRId64 " slimit:%d, soffset:%" PRId64, pSql, pQueryInfo->limit.limit,
tscTrace("%p limit:%" PRId64 ", offset:%" PRId64 " slimit:%" PRId64 ", soffset:%" PRId64, pSql, pQueryInfo->limit.limit,
pQueryInfo->limit.offset, pQueryInfo->slimit.limit, pQueryInfo->slimit.offset);
if (pQueryInfo->slimit.offset < 0 || pQueryInfo->limit.offset < 0) {
@ -4920,25 +4919,25 @@ int32_t parseCreateDBOptions(SSqlCmd* pCmd, SCreateDBInfo* pCreateDbSql) {
return TSDB_CODE_SUCCESS;
}
void tscAddTimestampColumn(SQueryInfo* pQueryInfo, int16_t functionId, int16_t tableIndex) {
// the first column not timestamp column, add it
SSqlExpr* pExpr = NULL;
if (tscSqlExprNumOfExprs(pQueryInfo) > 0) {
pExpr = tscSqlExprGet(pQueryInfo, 0);
}
if (pExpr == NULL || pExpr->colInfo.colId != PRIMARYKEY_TIMESTAMP_COL_INDEX || pExpr->functionId != functionId) {
SColumnIndex index = {tableIndex, PRIMARYKEY_TIMESTAMP_COL_INDEX};
pExpr = tscSqlExprInsert(pQueryInfo, 0, functionId, &index, TSDB_DATA_TYPE_TIMESTAMP, TSDB_KEYSIZE, TSDB_KEYSIZE, false);
pExpr->colInfo.flag = TSDB_COL_NORMAL;
// NOTE: tag column does not add to source column list
SColumnList ids = getColumnList(1, tableIndex, PRIMARYKEY_TIMESTAMP_COL_INDEX);
insertResultField(pQueryInfo, 0, &ids, TSDB_KEYSIZE, TSDB_DATA_TYPE_TIMESTAMP, "ts", pExpr);
}
}
//void tscAddTimestampColumn(SQueryInfo* pQueryInfo, int16_t functionId, int16_t tableIndex) {
// // the first column not timestamp column, add it
// SSqlExpr* pExpr = NULL;
// if (tscSqlExprNumOfExprs(pQueryInfo) > 0) {
// pExpr = tscSqlExprGet(pQueryInfo, 0);
// }
//
// if (pExpr == NULL || pExpr->colInfo.colId != PRIMARYKEY_TIMESTAMP_COL_INDEX || pExpr->functionId != functionId) {
// SColumnIndex index = {tableIndex, PRIMARYKEY_TIMESTAMP_COL_INDEX};
//
// pExpr = tscSqlExprInsert(pQueryInfo, 0, functionId, &index, TSDB_DATA_TYPE_TIMESTAMP, TSDB_KEYSIZE, TSDB_KEYSIZE, false);
// pExpr->colInfo.flag = TSDB_COL_NORMAL;
//
// // NOTE: tag column does not add to source column list
// SColumnList ids = getColumnList(1, tableIndex, PRIMARYKEY_TIMESTAMP_COL_INDEX);
//
// insertResultField(pQueryInfo, 0, &ids, TSDB_KEYSIZE, TSDB_DATA_TYPE_TIMESTAMP, "ts", pExpr);
// }
//}
void addGroupInfoForSubquery(SSqlObj* pParentObj, SSqlObj* pSql, int32_t subClauseIndex, int32_t tableIndex) {
SQueryInfo* pParentQueryInfo = tscGetQueryInfoDetail(&pParentObj->cmd, subClauseIndex);
@ -4951,7 +4950,7 @@ void addGroupInfoForSubquery(SSqlObj* pParentObj, SSqlObj* pSql, int32_t subClau
if (pExpr->functionId != TSDB_FUNC_TAG) {
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, tableIndex);
int16_t columnInfo = tscGetJoinTagColIndexByUid(&pQueryInfo->tagCond, pTableMetaInfo->pTableMeta->uid);
int16_t columnInfo = tscGetJoinTagColIdByUid(&pQueryInfo->tagCond, pTableMetaInfo->pTableMeta->uid);
SColumnIndex index = {.tableIndex = 0, .columnIndex = columnInfo};
SSchema* pSchema = tscGetTableTagSchema(pTableMetaInfo->pTableMeta);
@ -4987,27 +4986,17 @@ static void doLimitOutputNormalColOfGroupby(SSqlExpr* pExpr) {
void doAddGroupColumnForSubquery(SQueryInfo* pQueryInfo, int32_t tagIndex) {
SColIndex* pColIndex = taosArrayGet(pQueryInfo->groupbyExpr.columnInfo, tagIndex);
int32_t index = pColIndex->colIndex;
size_t size = tscSqlExprNumOfExprs(pQueryInfo);
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
SSchema* pSchema = tscGetTableColumnSchema(pTableMetaInfo->pTableMeta, index);
SColumnIndex colIndex = {.tableIndex = 0, .columnIndex = index};
size_t size = tscSqlExprNumOfExprs(pQueryInfo);
SSqlExpr* pExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_PRJ, &colIndex, pSchema->type, pSchema->bytes,
pSchema->bytes, false);
SSchema* pSchema = tscGetTableColumnSchema(pTableMetaInfo->pTableMeta, pColIndex->colIndex);
SColumnIndex colIndex = {.tableIndex = 0, .columnIndex = pColIndex->colIndex};
pExpr->colInfo.flag = TSDB_COL_NORMAL;
doLimitOutputNormalColOfGroupby(pExpr);
tscAddSpecialColumnForSelect(pQueryInfo, size, TSDB_FUNC_PRJ, &colIndex, pSchema, TSDB_COL_NORMAL);
// NOTE: tag column does not add to source column list
SColumnList list = {0};
list.num = 1;
list.ids[0] = colIndex;
insertResultField(pQueryInfo, size, &list, pSchema->bytes, pSchema->type, pSchema->name, pExpr);
SFieldSupInfo* pInfo = tscFieldInfoGetSupp(&pQueryInfo->fieldsInfo, size - 1);
SFieldSupInfo* pInfo = tscFieldInfoGetSupp(&pQueryInfo->fieldsInfo, size);
doLimitOutputNormalColOfGroupby(pInfo->pSqlExpr);
pInfo->visible = false;
}
@ -5219,6 +5208,7 @@ static int32_t doAddGroupbyColumnsOnDemand(SQueryInfo* pQueryInfo) {
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
SSchema s = tGetTableNameColumnSchema();
SSchema* pSchema = tscGetTableSchema(pTableMetaInfo->pTableMeta);
int16_t bytes = 0;
int16_t type = 0;
@ -5226,10 +5216,8 @@ static int32_t doAddGroupbyColumnsOnDemand(SQueryInfo* pQueryInfo) {
for (int32_t i = 0; i < pQueryInfo->groupbyExpr.numOfGroupCols; ++i) {
SColIndex* pColIndex = taosArrayGet(pQueryInfo->groupbyExpr.columnInfo, i);
int16_t colIndex = pColIndex->colIndex;
if (colIndex == TSDB_TBNAME_COLUMN_INDEX) {
SSchema s = tGetTableNameColumnSchema();
type = s.type;
bytes = s.bytes;
name = s.name;
@ -5767,6 +5755,7 @@ int32_t doCheckForQuery(SSqlObj* pSql, SQuerySQL* pQuerySql, int32_t index) {
const char* msg7 = "illegal number of tables in from clause";
const char* msg8 = "too many columns in selection clause";
const char* msg9 = "TWA query requires both the start and end time";
const char* msg10= "too many tables in from clause";
int32_t code = TSDB_CODE_SUCCESS;
@ -5802,6 +5791,10 @@ int32_t doCheckForQuery(SSqlObj* pSql, SQuerySQL* pQuerySql, int32_t index) {
pQueryInfo->command = TSDB_SQL_SELECT;
if (pQuerySql->from->nExpr > 2) {
return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg10);
}
// set all query tables, which are maybe more than one.
for (int32_t i = 0; i < pQuerySql->from->nExpr; ++i) {
tVariant* pTableItem = &pQuerySql->from->a[i].pVar;
@ -5926,10 +5919,6 @@ int32_t doCheckForQuery(SSqlObj* pSql, SQuerySQL* pQuerySql, int32_t index) {
setColumnOffsetValueInResultset(pQueryInfo);
for (int32_t i = 0; i < pQueryInfo->numOfTables; ++i) {
updateTagColumnIndex(pQueryInfo, i);
}
/*
* fill options are set at the end position, when all columns are set properly
* the columns may be increased due to group by operation
@ -6052,6 +6041,19 @@ int32_t exprTreeFromSqlExpr(tExprNode **pExpr, const tSQLExpr* pSqlExpr, SArray*
}
}
}
if ((*pExpr)->_node.optr != TSDB_RELATION_EQUAL && (*pExpr)->_node.optr != TSDB_RELATION_NOT_EQUAL) {
if (pRight->nodeType == TSQL_NODE_VALUE) {
if (pRight->pVal->nType == TSDB_DATA_TYPE_BOOL) {
return TSDB_CODE_TSC_INVALID_SQL;
}
if ((pRight->pVal->nType == TSDB_DATA_TYPE_BINARY || pRight->pVal->nType == TSDB_DATA_TYPE_NCHAR)
&& (*pExpr)->_node.optr != TSDB_RELATION_LIKE) {
return TSDB_CODE_TSC_INVALID_SQL;
}
}
}
}
return TSDB_CODE_SUCCESS;

26
src/client/src/tscSchemaUtil.c
View File

@ -32,7 +32,6 @@ int32_t tscGetNumOfTags(const STableMeta* pTableMeta) {
}
if (pTableMeta->tableType == TSDB_SUPER_TABLE || pTableMeta->tableType == TSDB_CHILD_TABLE) {
assert(tinfo.numOfTags >= 0);
return tinfo.numOfTags;
}
@ -64,14 +63,6 @@ SSchema* tscGetTableTagSchema(const STableMeta* pTableMeta) {
STableComInfo tscGetTableInfo(const STableMeta* pTableMeta) {
assert(pTableMeta != NULL);
#if 0
if (pTableMeta->tableType == TSDB_CHILD_TABLE) {
assert (pTableMeta->pSTable != NULL);
return pTableMeta->pSTable->tableInfo;
}
#endif
return pTableMeta->tableInfo;
}
@ -119,11 +110,24 @@ bool isValidSchema(struct SSchema* pSchema, int32_t numOfCols) {
return (rowLen <= TSDB_MAX_BYTES_PER_ROW);
}
SSchema* tscGetTableColumnSchema(const STableMeta* pTableMeta, int32_t startCol) {
SSchema* tscGetTableColumnSchema(const STableMeta* pTableMeta, int32_t colIndex) {
assert(pTableMeta != NULL);
SSchema* pSchema = (SSchema*) pTableMeta->schema;
return &pSchema[startCol];
return &pSchema[colIndex];
}
// TODO for large number of columns, employ the binary search method
SSchema* tscGetTableColumnSchemaById(STableMeta* pTableMeta, int16_t colId) {
STableComInfo tinfo = tscGetTableInfo(pTableMeta);
for(int32_t i = 0; i < tinfo.numOfColumns + tinfo.numOfTags; ++i) {
if (pTableMeta->schema[i].colId == colId) {
return &pTableMeta->schema[i];
}
}
return NULL;
}
struct SSchema tscGetTbnameColumnSchema() {

218
src/client/src/tscServer.c
View File

@ -14,20 +14,18 @@
*/
#include "os.h"
#include "qsqltype.h"
#include "tcache.h"
#include "trpc.h"
#include "tscLocalMerge.h"
#include "tscLog.h"
#include "tscProfile.h"
#include "tscSecondaryMerge.h"
#include "tscSubquery.h"
#include "tscUtil.h"
#include "tschemautil.h"
#include "tsclient.h"
#include "tsocket.h"
#include "ttime.h"
#include "ttimer.h"
#include "tutil.h"
#include "tscLog.h"
#include "qsqltype.h"
#define TSC_MGMT_VNODE 999
@ -198,30 +196,35 @@ int tscSendMsgToServer(SSqlObj *pSql) {
.handle = pSql,
.code = 0
};
rpcSendRequest(pObj->pDnodeConn, &pSql->ipList, &rpcMsg);
pSql->pRpcCtx = rpcSendRequest(pObj->pDnodeConn, &pSql->ipList, &rpcMsg);
return TSDB_CODE_SUCCESS;
}
void tscProcessMsgFromServer(SRpcMsg *rpcMsg, SRpcIpSet *pIpSet) {
SSqlObj *pSql = (SSqlObj *)rpcMsg->handle;
if (pSql == NULL) {
if (pSql == NULL || pSql->signature != pSql) {
tscError("%p sql is already released", pSql->signature);
return;
}
if (pSql->signature != pSql) {
tscError("%p sql is already released, signature:%p", pSql, pSql->signature);
STscObj *pObj = pSql->pTscObj;
SSqlRes *pRes = &pSql->res;
SSqlCmd *pCmd = &pSql->cmd;
if (pObj->signature != pObj) {
tscTrace("%p DB connection is closed, cmd:%d pObj:%p signature:%p", pSql, pCmd->command, pObj, pObj->signature);
tscFreeSqlObj(pSql);
rpcFreeCont(rpcMsg->pCont);
return;
}
SSqlRes *pRes = &pSql->res;
SSqlCmd *pCmd = &pSql->cmd;
STscObj *pObj = pSql->pTscObj;
if (pObj->signature != pObj || pSql->freed == 1) {
tscTrace("%p sqlObj needs to be released or DB connection is closed, freed:%d pObj:%p signature:%p", pSql, pSql->freed,
SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, 0);
if (pQueryInfo != NULL && pQueryInfo->type == TSDB_QUERY_TYPE_FREE_RESOURCE) {
tscTrace("%p sqlObj needs to be released or DB connection is closed, cmd:%d pObj:%p signature:%p", pSql, pCmd->command,
pObj, pObj->signature);
tscFreeSqlObj(pSql);
rpcFreeCont(rpcMsg->pCont);
return;
@ -274,7 +277,7 @@ void tscProcessMsgFromServer(SRpcMsg *rpcMsg, SRpcIpSet *pIpSet) {
if (pRes->code != TSDB_CODE_TSC_QUERY_CANCELLED) {
pRes->code = (rpcMsg->code != TSDB_CODE_SUCCESS) ? rpcMsg->code : TSDB_CODE_RPC_NETWORK_UNAVAIL;
} else {
tscTrace("%p query is cancelled, code:%d", pSql, tstrerror(pRes->code));
tscTrace("%p query is cancelled, code:%s", pSql, tstrerror(pRes->code));
}
if (pRes->code == TSDB_CODE_SUCCESS) {
@ -414,7 +417,6 @@ void tscKillSTableQuery(SSqlObj *pSql) {
for (int i = 0; i < pSql->numOfSubs; ++i) {
SSqlObj *pSub = pSql->pSubs[i];
if (pSub == NULL) {
continue;
}
@ -424,13 +426,13 @@ void tscKillSTableQuery(SSqlObj *pSql) {
* sub-queries not correctly released and master sql object of super table query reaches an abnormal state.
*/
pSql->pSubs[i]->res.code = TSDB_CODE_TSC_QUERY_CANCELLED;
// taosStopRpcConn(pSql->pSubs[i]->);
rpcCancelRequest(pSql->pSubs[i]->pRpcCtx);
}
/*
* 1. if the subqueries are not launched or partially launched, we need to waiting the launched
* query return to successfully free allocated resources.
* 2. if no any subqueries are launched yet, which means the metric query only in parse sql stage,
* 2. if no any subqueries are launched yet, which means the super table query only in parse sql stage,
* set the res.code, and return.
*/
const int64_t MAX_WAITING_TIME = 10000; // 10 Sec.
@ -644,14 +646,10 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pQueryMsg->numOfGroupCols = htons(pQueryInfo->groupbyExpr.numOfGroupCols);
pQueryMsg->numOfTags = htonl(numOfTags);
pQueryMsg->tagNameRelType = htons(pQueryInfo->tagCond.relType);
pQueryMsg->queryType = htons(pQueryInfo->type);
pQueryMsg->queryType = htonl(pQueryInfo->type);
size_t numOfOutput = tscSqlExprNumOfExprs(pQueryInfo);
pQueryMsg->numOfOutput = htons(numOfOutput);
if (numOfOutput < 0) {
tscError("%p illegal value of number of output columns in query msg: %d", pSql, numOfOutput);
return -1;
}
// set column list ids
size_t numOfCols = taosArrayGetSize(pQueryInfo->colList);
@ -665,7 +663,7 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
if (pCol->colIndex.columnIndex >= tscGetNumOfColumns(pTableMeta) || pColSchema->type < TSDB_DATA_TYPE_BOOL ||
pColSchema->type > TSDB_DATA_TYPE_NCHAR) {
tscError("%p sid:%d uid:%" PRIu64" id:%s, column index out of range, numOfColumns:%d, index:%d, column name:%s",
pSql, pTableMeta->sid, pTableMeta->uid, pTableMetaInfo->name, tscGetNumOfColumns(pTableMeta), pCol->colIndex,
pSql, pTableMeta->sid, pTableMeta->uid, pTableMetaInfo->name, tscGetNumOfColumns(pTableMeta), pCol->colIndex.columnIndex,
pColSchema->name);
return TSDB_CODE_TSC_INVALID_SQL;
@ -723,6 +721,7 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pMsg += sizeof(SSqlFuncMsg);
for (int32_t j = 0; j < pExpr->numOfParams; ++j) {
// todo add log
pSqlFuncExpr->arg[j].argType = htons((uint16_t)pExpr->param[j].nType);
pSqlFuncExpr->arg[j].argBytes = htons(pExpr->param[j].nLen);
@ -784,7 +783,7 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
(pColSchema->type < TSDB_DATA_TYPE_BOOL || pColSchema->type > TSDB_DATA_TYPE_NCHAR)) {
tscError("%p sid:%d uid:%" PRIu64 " id:%s, tag index out of range, totalCols:%d, numOfTags:%d, index:%d, column name:%s",
pSql, pTableMeta->sid, pTableMeta->uid, pTableMetaInfo->name, total, numOfTagColumns,
pCol->colIndex, pColSchema->name);
pCol->colIndex.columnIndex, pColSchema->name);
return TSDB_CODE_TSC_INVALID_SQL;
}
@ -800,6 +799,27 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
}
}
// serialize tag column query condition
if (pQueryInfo->tagCond.pCond != NULL && taosArrayGetSize(pQueryInfo->tagCond.pCond) > 0) {
STagCond* pTagCond = &pQueryInfo->tagCond;
SCond *pCond = tsGetSTableQueryCond(pTagCond, pTableMeta->uid);
if (pCond != NULL && pCond->cond != NULL) {
pQueryMsg->tagCondLen = htons(pCond->len);
memcpy(pMsg, pCond->cond, pCond->len);
pMsg += pCond->len;
}
}
if (pQueryInfo->tagCond.tbnameCond.cond == NULL) {
*pMsg = 0;
pMsg++;
} else {
strcpy(pMsg, pQueryInfo->tagCond.tbnameCond.cond);
pMsg += strlen(pQueryInfo->tagCond.tbnameCond.cond) + 1;
}
// compressed ts block
pQueryMsg->tsOffset = htonl(pMsg - pStart);
int32_t tsLen = 0;
@ -824,27 +844,6 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pQueryMsg->tsOrder = htonl(pQueryInfo->tsBuf->tsOrder);
}
// serialize tag column query condition
if (pQueryInfo->tagCond.pCond != NULL && taosArrayGetSize(pQueryInfo->tagCond.pCond) > 0) {
STagCond* pTagCond = &pQueryInfo->tagCond;
SCond *pCond = tsGetSTableQueryCond(pTagCond, pTableMeta->uid);
if (pCond != NULL && pCond->cond != NULL) {
pQueryMsg->tagCondLen = htons(pCond->len);
memcpy(pMsg, pCond->cond, pCond->len);
pMsg += pCond->len;
}
}
if (pQueryInfo->tagCond.tbnameCond.cond == NULL) {
*pMsg = 0;
pMsg++;
} else {
strcpy(pMsg, pQueryInfo->tagCond.tbnameCond.cond);
pMsg += strlen(pQueryInfo->tagCond.tbnameCond.cond) + 1;
}
int32_t msgLen = pMsg - pStart;
tscTrace("%p msg built success,len:%d bytes", pSql, msgLen);
@ -983,7 +982,7 @@ int32_t tscBuildDropDbMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
SCMDropDbMsg *pDropDbMsg = (SCMDropDbMsg*)pCmd->payload;
STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0);
strncpy(pDropDbMsg->db, pTableMetaInfo->name, tListLen(pDropDbMsg->db));
tstrncpy(pDropDbMsg->db, pTableMetaInfo->name, sizeof(pDropDbMsg->db));
pDropDbMsg->ignoreNotExists = pInfo->pDCLInfo->existsCheck ? 1 : 0;
pCmd->msgType = TSDB_MSG_TYPE_CM_DROP_DB;
@ -1053,7 +1052,7 @@ int32_t tscBuildDropAcctMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
SCMDropUserMsg *pDropMsg = (SCMDropUserMsg*)pCmd->payload;
STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0);
strcpy(pDropMsg->user, pTableMetaInfo->name);
tstrncpy(pDropMsg->user, pTableMetaInfo->name, sizeof(pDropMsg->user));
return TSDB_CODE_SUCCESS;
}
@ -1175,7 +1174,7 @@ int tscBuildCreateTableMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
strcpy(pCreateTableMsg->tableId, pTableMetaInfo->name);
// use dbinfo from table id without modifying current db info
tscGetDBInfoFromMeterId(pTableMetaInfo->name, pCreateTableMsg->db);
tscGetDBInfoFromTableFullName(pTableMetaInfo->name, pCreateTableMsg->db);
SCreateTableSQL *pCreateTable = pInfo->pCreateTableInfo;
@ -1252,7 +1251,7 @@ int tscBuildAlterTableMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
}
SCMAlterTableMsg *pAlterTableMsg = (SCMAlterTableMsg *)pCmd->payload;
tscGetDBInfoFromMeterId(pTableMetaInfo->name, pAlterTableMsg->db);
tscGetDBInfoFromTableFullName(pTableMetaInfo->name, pAlterTableMsg->db);
strcpy(pAlterTableMsg->tableId, pTableMetaInfo->name);
pAlterTableMsg->type = htons(pAlterInfo->type);
@ -1473,7 +1472,7 @@ int tscBuildTableMetaMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pMsg += len;
}
pCmd->payloadLen = pMsg - (char*)pInfoMsg;;
pCmd->payloadLen = pMsg - (char*)pInfoMsg;
pCmd->msgType = TSDB_MSG_TYPE_CM_TABLE_META;
tfree(tmpData);
@ -1577,7 +1576,7 @@ int tscBuildSTableVgroupMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pMsg = pStart;
SMgmtHead *pMgmt = (SMgmtHead *)pMsg;
tscGetDBInfoFromMeterId(pTableMetaInfo->name, pMgmt->db);
tscGetDBInfoFromTableFullName(pTableMetaInfo->name, pMgmt->db);
pMsg += sizeof(SMgmtHead);
@ -1813,6 +1812,7 @@ int tscProcessTableMetaRsp(SSqlObj *pSql) {
// todo handle out of memory case
if (pTableMetaInfo->pTableMeta == NULL) {
free(pTableMeta);
return TSDB_CODE_TSC_OUT_OF_MEMORY;
}
@ -1932,113 +1932,6 @@ int tscProcessMultiMeterMetaRsp(SSqlObj *pSql) {
}
int tscProcessSTableVgroupRsp(SSqlObj *pSql) {
#if 0
void ** metricMetaList = NULL;
int32_t * sizes = NULL;
int32_t num = htons(*(int16_t *)rsp);
rsp += sizeof(int16_t);
metricMetaList = calloc(1, POINTER_BYTES * num);
sizes = calloc(1, sizeof(int32_t) * num);
// return with error code
if (metricMetaList == NULL || sizes == NULL) {
tfree(metricMetaList);
tfree(sizes);
pSql->res.code = TSDB_CODE_TSC_OUT_OF_MEMORY;
return pSql->res.code;
}
for (int32_t k = 0; k < num; ++k) {
pMeta = (SSuperTableMeta *)rsp;
size_t size = (size_t)pSql->res.rspLen - 1;
rsp = rsp + sizeof(SSuperTableMeta);
pMeta->numOfTables = htonl(pMeta->numOfTables);
pMeta->numOfVnodes = htonl(pMeta->numOfVnodes);
pMeta->tagLen = htons(pMeta->tagLen);
size += pMeta->numOfVnodes * sizeof(SVnodeSidList *) + pMeta->numOfTables * sizeof(STableIdInfo *);
char *pBuf = calloc(1, size);
if (pBuf == NULL) {
pSql->res.code = TSDB_CODE_TSC_OUT_OF_MEMORY;
goto _error_clean;
}
SSuperTableMeta *pNewMetricMeta = (SSuperTableMeta *)pBuf;
metricMetaList[k] = pNewMetricMeta;
pNewMetricMeta->numOfTables = pMeta->numOfTables;
pNewMetricMeta->numOfVnodes = pMeta->numOfVnodes;
pNewMetricMeta->tagLen = pMeta->tagLen;
pBuf = pBuf + sizeof(SSuperTableMeta) + pNewMetricMeta->numOfVnodes * sizeof(SVnodeSidList *);
for (int32_t i = 0; i < pMeta->numOfVnodes; ++i) {
SVnodeSidList *pSidLists = (SVnodeSidList *)rsp;
memcpy(pBuf, pSidLists, sizeof(SVnodeSidList));
pNewMetricMeta->list[i] = pBuf - (char *)pNewMetricMeta; // offset value
SVnodeSidList *pLists = (SVnodeSidList *)pBuf;
tscTrace("%p metricmeta:vid:%d,numOfTables:%d", pSql, i, pLists->numOfSids);
pBuf += sizeof(SVnodeSidList) + sizeof(STableIdInfo *) * pSidLists->numOfSids;
rsp += sizeof(SVnodeSidList);
size_t elemSize = sizeof(STableIdInfo) + pNewMetricMeta->tagLen;
for (int32_t j = 0; j < pSidLists->numOfSids; ++j) {
pLists->pSidExtInfoList[j] = pBuf - (char *)pLists;
memcpy(pBuf, rsp, elemSize);
((STableIdInfo *)pBuf)->uid = htobe64(((STableIdInfo *)pBuf)->uid);
((STableIdInfo *)pBuf)->sid = htonl(((STableIdInfo *)pBuf)->sid);
rsp += elemSize;
pBuf += elemSize;
}
}
sizes[k] = pBuf - (char *)pNewMetricMeta;
}
SQueryInfo *pQueryInfo = tscGetQueryInfoDetail(&pSql->cmd, 0);
for (int32_t i = 0; i < num; ++i) {
char name[TSDB_MAX_TAGS_LEN + 1] = {0};
STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, i);
tscGetMetricMetaCacheKey(pQueryInfo, name, pTableMetaInfo->pTableMeta->uid);
#ifdef _DEBUG_VIEW
printf("generate the metric key:%s, index:%d\n", name, i);
#endif
// release the used metricmeta
taosCacheRelease(tscCacheHandle, (void **)&(pTableMetaInfo->pMetricMeta), false);
pTableMetaInfo->pMetricMeta = (SSuperTableMeta *)taosCachePut(tscCacheHandle, name, (char *)metricMetaList[i],
sizes[i], tsMetricMetaKeepTimer);
tfree(metricMetaList[i]);
// failed to put into cache
if (pTableMetaInfo->pMetricMeta == NULL) {
pSql->res.code = TSDB_CODE_TSC_OUT_OF_MEMORY;
goto _error_clean;
}
}
_error_clean:
// free allocated resource
for (int32_t i = 0; i < num; ++i) {
tfree(metricMetaList[i]);
}
free(sizes);
free(metricMetaList);
#endif
SSqlRes* pRes = &pSql->res;
// NOTE: the order of several table must be preserved.
@ -2201,7 +2094,7 @@ int tscProcessDropTableRsp(SSqlObj *pSql) {
* The cached information is expired, however, we may have lost the ref of original meter. So, clear whole cache
* instead.
*/
tscTrace("%p force release metermeta after drop table:%s", pSql, pTableMetaInfo->name);
tscTrace("%p force release table meta after drop table:%s", pSql, pTableMetaInfo->name);
taosCacheRelease(tscCacheHandle, (void **)&pTableMeta, true);
if (pTableMetaInfo->pTableMeta) {
@ -2291,7 +2184,7 @@ int tscProcessRetrieveRspFromNode(SSqlObj *pSql) {
}
pRes->row = 0;
tscTrace("%p numOfRows:%d, offset:%d, complete:%d", pSql, pRes->numOfRows, pRes->offset, pRes->completed);
tscTrace("%p numOfRows:%" PRId64 ", offset:%" PRId64 ", complete:%d", pSql, pRes->numOfRows, pRes->offset, pRes->completed);
return 0;
}
@ -2432,6 +2325,7 @@ int tscGetSTableVgroupInfo(SSqlObj *pSql, int32_t clauseIndex) {
SQueryInfo *pNewQueryInfo = NULL;
if ((code = tscGetQueryInfoDetailSafely(&pNew->cmd, 0, &pNewQueryInfo)) != TSDB_CODE_SUCCESS) {
tscFreeSqlObj(pNew);
return code;
}

32
src/client/src/tscSql.c
View File

@ -20,14 +20,9 @@
#include "tnote.h"
#include "trpc.h"
#include "tscLog.h"
#include "tscProfile.h"
#include "tscSecondaryMerge.h"
#include "tscSubquery.h"
#include "tscUtil.h"
#include "tsclient.h"
#include "tscompression.h"
#include "tsocket.h"
#include "ttimer.h"
#include "ttokendef.h"
#include "tutil.h"
@ -152,6 +147,8 @@ static void syncConnCallback(void *param, TAOS_RES *tres, int code) {
TAOS *taos_connect(const char *ip, const char *user, const char *pass, const char *db, uint16_t port) {
tscTrace("try to create a connection to %s:%u, user:%s db:%s", ip, port, user, db);
if (user == NULL) user = TSDB_DEFAULT_USER;
if (pass == NULL) pass = TSDB_DEFAULT_PASS;
STscObj* pObj = NULL;
SSqlObj *pSql = taosConnectImpl(ip, user, pass, db, port, syncConnCallback, NULL, (void**) &pObj);
@ -491,8 +488,7 @@ static bool tscFreeQhandleInVnode(SSqlObj* pSql) {
(pCmd->command == TSDB_SQL_SELECT && pSql->pStream == NULL && pTableMetaInfo->pTableMeta != NULL)) {
pCmd->command = (pCmd->command > TSDB_SQL_MGMT) ? TSDB_SQL_RETRIEVE : TSDB_SQL_FETCH;
tscTrace("%p start to send msg to free qhandle in dnode, command:%s", pSql, sqlCmd[pCmd->command]);
pSql->freed = 1;
tscTrace("%p send msg to dnode to free qhandle ASAP, command:%s", pSql, sqlCmd[pCmd->command]);
tscProcessSql(pSql);
// in case of sync model query, waits for response and then goes on
@ -513,18 +509,17 @@ static bool tscFreeQhandleInVnode(SSqlObj* pSql) {
void taos_free_result(TAOS_RES *res) {
SSqlObj *pSql = (SSqlObj *)res;
tscTrace("%p start to free result", res);
if (pSql == NULL || pSql->signature != pSql) {
tscTrace("%p result has been freed", pSql);
tscTrace("%p sqlObj has been freed", pSql);
return;
}
// The semaphore can not be changed while freeing async sub query objects.
SSqlRes *pRes = &pSql->res;
if (pRes == NULL || pRes->qhandle == 0) {
tscTrace("%p SqlObj is freed by app, qhandle is null", pSql);
tscFreeSqlObj(pSql);
tscTrace("%p SqlObj is freed by app, qhandle is null", pSql);
return;
}
@ -532,6 +527,7 @@ void taos_free_result(TAOS_RES *res) {
SQueryInfo *pQueryInfo = tscGetQueryInfoDetail(&pSql->cmd, 0);
if (pQueryInfo == NULL) {
tscFreeSqlObj(pSql);
tscTrace("%p SqlObj is freed by app", pSql);
return;
}
@ -589,7 +585,7 @@ char *taos_errstr(TAOS_RES *tres) {
void taos_config(int debug, char *log_path) {
uDebugFlag = debug;
strcpy(tsLogDir, log_path);
tstrncpy(tsLogDir, log_path, TSDB_FILENAME_LEN);
}
char *taos_get_server_info(TAOS *taos) {
@ -612,7 +608,9 @@ int* taos_fetch_lengths(TAOS_RES *res) {
char *taos_get_client_info() { return version; }
void taos_stop_query(TAOS_RES *res) {
if (res == NULL) return;
if (res == NULL) {
return;
}
SSqlObj *pSql = (SSqlObj *)res;
SSqlCmd *pCmd = &pSql->cmd;
@ -632,7 +630,7 @@ void taos_stop_query(TAOS_RES *res) {
return;
}
//taosStopRpcConn(pSql->thandle);
rpcCancelRequest(pSql->pRpcCtx);
tscTrace("%p query is cancelled", res);
}
@ -714,7 +712,6 @@ int taos_validate_sql(TAOS *taos, const char *sql) {
SSqlRes *pRes = &pSql->res;
SSqlCmd *pCmd = &pSql->cmd;
pRes->numOfRows = 1;
pRes->numOfTotal = 0;
pRes->numOfClauseTotal = 0;
@ -724,6 +721,7 @@ int taos_validate_sql(TAOS *taos, const char *sql) {
if (sqlLen > tsMaxSQLStringLen) {
tscError("%p sql too long", pSql);
pRes->code = TSDB_CODE_TSC_INVALID_SQL;
tfree(pSql);
return pRes->code;
}
@ -732,6 +730,7 @@ int taos_validate_sql(TAOS *taos, const char *sql) {
pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY;
tscError("%p failed to malloc sql string buffer", pSql);
tscTrace("%p Valid SQL result:%d, %s pObj:%p", pSql, pRes->code, taos_errstr(taos), pObj);
tfree(pSql);
return pRes->code;
}
@ -856,6 +855,7 @@ int taos_load_table_info(TAOS *taos, const char *tableNameList) {
if (tblListLen > MAX_TABLE_NAME_LENGTH) {
tscError("%p tableNameList too long, length:%d, maximum allowed:%d", pSql, tblListLen, MAX_TABLE_NAME_LENGTH);
pRes->code = TSDB_CODE_TSC_INVALID_SQL;
tfree(pSql);
return pRes->code;
}
@ -863,6 +863,7 @@ int taos_load_table_info(TAOS *taos, const char *tableNameList) {
if (str == NULL) {
pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY;
tscError("%p failed to malloc sql string buffer", pSql);
tfree(pSql);
return pRes->code;
}
@ -878,6 +879,7 @@ int taos_load_table_info(TAOS *taos, const char *tableNameList) {
free(str);
if (pRes->code != TSDB_CODE_SUCCESS) {
tscFreeSqlObj(pSql);
return pRes->code;
}

12
src/client/src/tscStream.c
View File

@ -125,7 +125,7 @@ static void tscProcessStreamTimer(void *handle, void *tmrId) {
}
// launch stream computing in a new thread
SSchedMsg schedMsg;
SSchedMsg schedMsg = { 0 };
schedMsg.fp = tscProcessStreamLaunchQuery;
schedMsg.ahandle = pStream;
schedMsg.thandle = (void *)1;
@ -239,7 +239,7 @@ static void tscProcessStreamRetrieveResult(void *param, TAOS_RES *res, int numOf
/* no resuls in the query range, retry */
// todo set retry dynamic time
int32_t retry = tsProjectExecInterval;
tscError("%p stream:%p, retrieve no data, code:%d, retry in %" PRId64 "ms", pSql, pStream, numOfRows, retry);
tscError("%p stream:%p, retrieve no data, code:%d, retry in %" PRId32 "ms", pSql, pStream, numOfRows, retry);
tscSetRetryTimer(pStream, pStream->pSql, retry);
return;
@ -250,7 +250,7 @@ static void tscProcessStreamRetrieveResult(void *param, TAOS_RES *res, int numOf
}
}
tscTrace("%p stream:%p, query on:%s, fetch result completed, fetched rows:%d", pSql, pStream, pTableMetaInfo->name,
tscTrace("%p stream:%p, query on:%s, fetch result completed, fetched rows:%" PRId64, pSql, pStream, pTableMetaInfo->name,
pStream->numOfRes);
// release the metric/meter meta information reference, so data in cache can be updated
@ -499,12 +499,14 @@ TAOS_STREAM *taos_open_stream(TAOS *taos, const char *sqlstr, void (*fp)(void *p
if (pSql->sqlstr == NULL) {
tscError("%p failed to malloc sql string buffer", pSql);
tscFreeSqlObj(pSql);
return NULL;;
return NULL;
}
strtolower(pSql->sqlstr, sqlstr);
tscDump("%p SQL: %s", pSql, pSql->sqlstr);
tsem_init(&pSql->rspSem, 0, 0);
int32_t code = doAsyncParseSql(pSql);
int32_t code = tsParseSql(pSql, true);
if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) {
sem_wait(&pSql->rspSem);
}

30
src/client/src/tscSub.c
View File

@ -182,21 +182,25 @@ static SArray* getTableList( SSqlObj* pSql ) {
char* sql = alloca(strlen(p) + 32);
sprintf(sql, "select tbid(tbname)%s", p);
SSqlObj* pSql1 = taos_query(pSql->pTscObj, sql);
if (terrno != TSDB_CODE_SUCCESS) {
tscError("failed to retrieve table id: %s", tstrerror(terrno));
SSqlObj* pNew = taos_query(pSql->pTscObj, sql);
if (pNew == NULL) {
tscError("failed to retrieve table id: cannot create new sql object.");
return NULL;
} else if (taos_errno(pNew) != TSDB_CODE_SUCCESS) {
tscError("failed to retrieve table id: %s", tstrerror(taos_errno(pNew)));
return NULL;
}
TAOS_ROW row;
SArray* result = taosArrayInit( 128, sizeof(STidTags) );
while ((row = taos_fetch_row(pSql1))) {
while ((row = taos_fetch_row(pNew))) {
STidTags tags;
memcpy(&tags, row[0], sizeof(tags));
taosArrayPush(result, &tags);
}
taos_free_result(pSql1);
taos_free_result(pNew);
return result;
}
@ -222,6 +226,9 @@ static int tscUpdateSubscription(STscObj* pObj, SSub* pSub) {
}
SArray* tables = getTableList(pSql);
if (tables == NULL) {
return 0;
}
size_t numOfTables = taosArrayGetSize(tables);
SArray* progress = taosArrayInit(numOfTables, sizeof(SSubscriptionProgress));
@ -238,10 +245,11 @@ static int tscUpdateSubscription(STscObj* pObj, SSub* pSub) {
if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) {
taosArraySort( tables, tscCompareTidTags );
tscBuildVgroupTableInfo( pTableMetaInfo, tables );
tscBuildVgroupTableInfo(pSql, pTableMetaInfo, tables);
}
taosArrayDestroy(tables);
TSDB_QUERY_SET_TYPE(tscGetQueryInfoDetail(pCmd, 0)->type, TSDB_QUERY_TYPE_MULTITABLE_QUERY);
return 1;
}
@ -291,7 +299,7 @@ static int tscLoadSubscriptionProgress(SSub* pSub) {
fclose(fp);
taosArraySort(progress, tscCompareSubscriptionProgress);
tscTrace("subscription progress loaded, %d tables: %s", taosArrayGetSize(progress), pSub->topic);
tscTrace("subscription progress loaded, %zu tables: %s", taosArrayGetSize(progress), pSub->topic);
return 1;
}
@ -350,7 +358,7 @@ TAOS_SUB *taos_subscribe(TAOS *taos, int restart, const char* topic, const char
pSub->interval = interval;
if (fp != NULL) {
tscTrace("asynchronize subscription, create new timer", topic);
tscTrace("asynchronize subscription, create new timer: %s", topic);
pSub->fp = fp;
pSub->param = param;
taosTmrReset(tscProcessSubscriptionTimer, interval, pSub, tscTmr, &pSub->pTimer);
@ -413,7 +421,7 @@ TAOS_RES *taos_consume(TAOS_SUB *tsub) {
}
if (pRes->code != TSDB_CODE_SUCCESS) {
tscError("failed to query data, error code=%d", pRes->code);
tscError("failed to query data: %s", tstrerror(pRes->code));
tscRemoveFromSqlList(pSql);
return NULL;
}
@ -435,7 +443,9 @@ void taos_unsubscribe(TAOS_SUB *tsub, int keepProgress) {
} else {
char path[256];
sprintf(path, "%s/subscribe/%s", tsDataDir, pSub->topic);
remove(path);
if (remove(path) != 0) {
tscError("failed to remove progress file, topic = %s, error = %s", pSub->topic, strerror(errno));
}
}
tscFreeSqlObj(pSub->pSql);

1162
src/client/src/tscSubquery.c
View File

File diff suppressed because it is too large Load Diff

4
src/client/src/tscSystem.c
View File

@ -220,7 +220,7 @@ static int taos_options_imp(TSDB_OPTION option, const char *pStr) {
if (strlen(tsLocale) == 0) { // locale does not set yet
char* defaultLocale = setlocale(LC_CTYPE, "");
strcpy(tsLocale, defaultLocale);
tstrncpy(tsLocale, defaultLocale, sizeof(tsLocale));
}
// set the user specified locale
@ -304,7 +304,7 @@ static int taos_options_imp(TSDB_OPTION option, const char *pStr) {
assert(cfg != NULL);
if (cfg->cfgStatus <= TAOS_CFG_CSTATUS_OPTION) {
strcpy(tsTimezone, pStr);
tstrncpy(tsTimezone, pStr, sizeof(tsTimezone));
tsSetTimeZone();
cfg->cfgStatus = TAOS_CFG_CSTATUS_OPTION;
tscTrace("timezone set:%s, input:%s by taos_options", tsTimezone, pStr);

194
src/client/src/tscUtil.c
View File

@ -22,7 +22,7 @@
#include "tkey.h"
#include "tmd5.h"
#include "tscProfile.h"
#include "tscSecondaryMerge.h"
#include "tscLocalMerge.h"
#include "tscSubquery.h"
#include "tschemautil.h"
#include "tsclient.h"
@ -32,7 +32,7 @@
static void freeQueryInfoImpl(SQueryInfo* pQueryInfo);
static void clearAllTableMetaInfo(SQueryInfo* pQueryInfo, const char* address, bool removeFromCache);
SCond* tsGetSTableQueryCond(STagCond* pTagCond, uint64_t uid) {
if (pTagCond->pCond == NULL) {
return NULL;
@ -70,13 +70,6 @@ void tsSetSTableQueryCond(STagCond* pTagCond, uint64_t uid, SBufferWriter* bw) {
taosArrayPush(pTagCond->pCond, &cond);
}
bool tscQueryOnSTable(SSqlCmd* pCmd) {
SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, 0);
return ((pQueryInfo->type & TSDB_QUERY_TYPE_STABLE_QUERY) == TSDB_QUERY_TYPE_STABLE_QUERY) &&
(pCmd->msgType == TSDB_MSG_TYPE_QUERY);
}
bool tscQueryTags(SQueryInfo* pQueryInfo) {
for (int32_t i = 0; i < pQueryInfo->fieldsInfo.numOfOutput; ++i) {
SSqlExpr* pExpr = tscSqlExprGet(pQueryInfo, i);
@ -95,32 +88,8 @@ bool tscQueryTags(SQueryInfo* pQueryInfo) {
return true;
}
bool tscIsSelectivityWithTagQuery(SSqlCmd* pCmd) {
bool hasTags = false;
int32_t numOfSelectivity = 0;
SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, 0);
for (int32_t i = 0; i < pQueryInfo->fieldsInfo.numOfOutput; ++i) {
int32_t functId = tscSqlExprGet(pQueryInfo, i)->functionId;
if (functId == TSDB_FUNC_TAG_DUMMY) {
hasTags = true;
continue;
}
if ((aAggs[functId].nStatus & TSDB_FUNCSTATE_SELECTIVITY) != 0) {
numOfSelectivity++;
}
}
if (numOfSelectivity > 0 && hasTags) {
return true;
}
return false;
}
void tscGetDBInfoFromMeterId(char* tableId, char* db) {
// todo refactor, extract methods and move the common module
void tscGetDBInfoFromTableFullName(char* tableId, char* db) {
char* st = strstr(tableId, TS_PATH_DELIMITER);
if (st != NULL) {
char* end = strstr(st + 1, TS_PATH_DELIMITER);
@ -181,8 +150,14 @@ bool tscIsProjectionQueryOnSTable(SQueryInfo* pQueryInfo, int32_t tableIndex) {
for (int32_t i = 0; i < numOfExprs; ++i) {
int32_t functionId = tscSqlExprGet(pQueryInfo, i)->functionId;
if (functionId != TSDB_FUNC_PRJ && functionId != TSDB_FUNC_TAGPRJ && functionId != TSDB_FUNC_TAG &&
functionId != TSDB_FUNC_TS && functionId != TSDB_FUNC_ARITHM) {
if (functionId != TSDB_FUNC_PRJ &&
functionId != TSDB_FUNC_TAGPRJ &&
functionId != TSDB_FUNC_TAG &&
functionId != TSDB_FUNC_TS &&
functionId != TSDB_FUNC_ARITHM &&
functionId != TSDB_FUNC_TS_COMP &&
functionId != TSDB_FUNC_TID_TAG) {
return false;
}
}
@ -209,10 +184,14 @@ bool tscOrderedProjectionQueryOnSTable(SQueryInfo* pQueryInfo, int32_t tableInde
return pQueryInfo->order.orderColId >= 0;
}
bool tscProjectionQueryOnTable(SQueryInfo* pQueryInfo) {
for (int32_t i = 0; i < pQueryInfo->fieldsInfo.numOfOutput; ++i) {
bool tscIsProjectionQuery(SQueryInfo* pQueryInfo) {
size_t size = tscSqlExprNumOfExprs(pQueryInfo);
for (int32_t i = 0; i < size; ++i) {
int32_t functionId = tscSqlExprGet(pQueryInfo, i)->functionId;
if (functionId != TSDB_FUNC_PRJ && functionId != TSDB_FUNC_TS) {
if (functionId != TSDB_FUNC_PRJ && functionId != TSDB_FUNC_TAGPRJ && functionId != TSDB_FUNC_TAG &&
functionId != TSDB_FUNC_TS && functionId != TSDB_FUNC_ARITHM) {
return false;
}
}
@ -225,9 +204,10 @@ bool tscIsPointInterpQuery(SQueryInfo* pQueryInfo) {
for (int32_t i = 0; i < size; ++i) {
SSqlExpr* pExpr = tscSqlExprGet(pQueryInfo, i);
if (pExpr == NULL) {
return false;
}
assert(pExpr != NULL);
// if (pExpr == NULL) {
// return false;
// }
int32_t functionId = pExpr->functionId;
if (functionId == TSDB_FUNC_TAG) {
@ -238,6 +218,7 @@ bool tscIsPointInterpQuery(SQueryInfo* pQueryInfo) {
return false;
}
}
return true;
}
@ -383,7 +364,6 @@ void tscPartiallyFreeSqlObj(SSqlObj* pSql) {
tscFreeSqlResult(pSql);
tfree(pSql->pSubs);
pSql->freed = 0;
pSql->numOfSubs = 0;
tscResetSqlCmdObj(pCmd);
@ -1451,8 +1431,6 @@ bool tscShouldBeFreed(SSqlObj* pSql) {
return false;
}
assert(pSql->fp != NULL);
STscObj* pTscObj = pSql->pTscObj;
if (pSql->pStream != NULL || pTscObj->pHb == pSql || pSql->pSubscription != NULL) {
return false;
@ -1602,24 +1580,6 @@ void tscClearSubqueryInfo(SSqlCmd* pCmd) {
}
}
void doRemoveTableMetaInfo(SQueryInfo* pQueryInfo, int32_t index, bool removeFromCache) {
if (index < 0 || index >= pQueryInfo->numOfTables) {
return;
}
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index);
tscClearTableMetaInfo(pTableMetaInfo, removeFromCache);
free(pTableMetaInfo);
int32_t after = pQueryInfo->numOfTables - index - 1;
if (after > 0) {
memmove(&pQueryInfo->pTableMetaInfo[index], &pQueryInfo->pTableMetaInfo[index + 1], after * POINTER_BYTES);
}
pQueryInfo->numOfTables -= 1;
}
void clearAllTableMetaInfo(SQueryInfo* pQueryInfo, const char* address, bool removeFromCache) {
tscTrace("%p deref the table meta in cache, numOfTables:%d", address, pQueryInfo->numOfTables);
@ -1692,6 +1652,46 @@ void tscResetForNextRetrieve(SSqlRes* pRes) {
pRes->numOfRows = 0;
}
SSqlObj* createSimpleSubObj(SSqlObj* pSql, void (*fp)(), void* param, int32_t cmd) {
SSqlObj* pNew = (SSqlObj*)calloc(1, sizeof(SSqlObj));
if (pNew == NULL) {
tscError("%p new subquery failed, tableIndex:%d", pSql, 0);
return NULL;
}
pNew->pTscObj = pSql->pTscObj;
pNew->signature = pNew;
SSqlCmd* pCmd = &pNew->cmd;
pCmd->command = cmd;
if (tscAddSubqueryInfo(pCmd) != TSDB_CODE_SUCCESS) {
tscFreeSqlObj(pNew);
return NULL;
}
pNew->fp = fp;
pNew->param = param;
pNew->maxRetry = TSDB_MAX_REPLICA_NUM;
pNew->sqlstr = strdup(pSql->sqlstr);
if (pNew->sqlstr == NULL) {
tscError("%p new subquery failed", pSql);
free(pNew);
return NULL;
}
SQueryInfo* pQueryInfo = NULL;
tscGetQueryInfoDetailSafely(pCmd, 0, &pQueryInfo);
assert(pSql->cmd.clauseIndex == 0);
STableMetaInfo* pMasterTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, pSql->cmd.clauseIndex, 0);
tscAddTableMetaInfo(pQueryInfo, pMasterTableMetaInfo->name, NULL, NULL, NULL);
return pNew;
}
SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, void (*fp)(), void* param, int32_t cmd, SSqlObj* pPrevSql) {
SSqlCmd* pCmd = &pSql->cmd;
SSqlObj* pNew = (SSqlObj*)calloc(1, sizeof(SSqlObj));
@ -1774,7 +1774,7 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, void (*fp)(), void
SQueryInfo* pPrevQueryInfo = tscGetQueryInfoDetail(&pPrevSql->cmd, pPrevSql->cmd.clauseIndex);
pNewQueryInfo->type = pPrevQueryInfo->type;
} else {
pNewQueryInfo->type |= TSDB_QUERY_TYPE_SUBQUERY; // it must be the subquery
TSDB_QUERY_SET_TYPE(pNewQueryInfo->type, TSDB_QUERY_TYPE_SUBQUERY);// it must be the subquery
}
uint64_t uid = pTableMetaInfo->pTableMeta->uid;
@ -1799,19 +1799,26 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, void (*fp)(), void
}
// make sure the the sqlExpr for each fields is correct
// todo handle the agg arithmetic expression
// todo handle the agg arithmetic expression
numOfExprs = tscSqlExprNumOfExprs(pNewQueryInfo);
for(int32_t f = 0; f < pNewQueryInfo->fieldsInfo.numOfOutput; ++f) {
TAOS_FIELD* field = tscFieldInfoGetField(&pNewQueryInfo->fieldsInfo, f);
numOfExprs = tscSqlExprNumOfExprs(pNewQueryInfo);
bool matched = false;
for(int32_t k1 = 0; k1 < numOfExprs; ++k1) {
SSqlExpr* pExpr1 = tscSqlExprGet(pNewQueryInfo, k1);
if (strcmp(field->name, pExpr1->aliasName) == 0) { // eatablish link according to the result field name
if (strcmp(field->name, pExpr1->aliasName) == 0) { // establish link according to the result field name
SFieldSupInfo* pInfo = tscFieldInfoGetSupp(&pNewQueryInfo->fieldsInfo, f);
pInfo->pSqlExpr = pExpr1;
matched = true;
break;
}
}
assert(matched);
}
tscFieldInfoUpdateOffset(pNewQueryInfo);
@ -1827,6 +1834,7 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, void (*fp)(), void
if (pPrevSql == NULL) {
STableMeta* pTableMeta = taosCacheAcquireByData(tscCacheHandle, pTableMetaInfo->pTableMeta); // get by name may failed due to the cache cleanup
assert(pTableMeta != NULL);
pFinalInfo = tscAddTableMetaInfo(pNewQueryInfo, name, pTableMeta, pTableMetaInfo->vgroupList, pTableMetaInfo->tagColList);
} else { // transfer the ownership of pTableMeta to the newly create sql object.
STableMetaInfo* pPrevInfo = tscGetTableMetaInfoFromCmd(&pPrevSql->cmd, pPrevSql->cmd.clauseIndex, 0);
@ -1849,12 +1857,12 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, void (*fp)(), void
if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) {
assert(pFinalInfo->vgroupList != NULL);
}
if (cmd == TSDB_SQL_SELECT) {
size_t size = taosArrayGetSize(pNewQueryInfo->colList);
tscTrace(
"%p new subquery:%p, tableIndex:%d, vgroupIndex:%d, type:%d, exprInfo:%d, colList:%d,"
"%p new subquery:%p, tableIndex:%d, vgroupIndex:%d, type:%d, exprInfo:%zu, colList:%zu,"
"fieldInfo:%d, name:%s, qrang:%" PRId64 " - %" PRId64 " order:%d, limit:%" PRId64,
pSql, pNew, tableIndex, pTableMetaInfo->vgroupIndex, pNewQueryInfo->type, tscSqlExprNumOfExprs(pNewQueryInfo),
size, pNewQueryInfo->fieldsInfo.numOfOutput, pFinalInfo->name, pNewQueryInfo->window.skey,
@ -1900,16 +1908,21 @@ void tscDoQuery(SSqlObj* pSql) {
}
if (QUERY_IS_JOIN_QUERY(type)) {
if ((pQueryInfo->type & TSDB_QUERY_TYPE_SUBQUERY) == 0) {
if (!TSDB_QUERY_HAS_TYPE(type, TSDB_QUERY_TYPE_SUBQUERY)) {
tscHandleMasterJoinQuery(pSql);
return;
} else {
// for first stage sub query, iterate all vnodes to get all timestamp
if ((pQueryInfo->type & TSDB_QUERY_TYPE_JOIN_SEC_STAGE) != TSDB_QUERY_TYPE_JOIN_SEC_STAGE) {
// doProcessSql(pSql);
assert(0);
} else { // for first stage sub query, iterate all vnodes to get all timestamp
if (!TSDB_QUERY_HAS_TYPE(type, TSDB_QUERY_TYPE_JOIN_SEC_STAGE)) {
tscProcessSql(pSql);
} else { // secondary stage join query.
if (tscIsTwoStageSTableQuery(pQueryInfo, 0)) { // super table query
tscHandleMasterSTableQuery(pSql);
} else {
tscProcessSql(pSql);
}
}
}
return;
} else if (tscIsTwoStageSTableQuery(pQueryInfo, 0)) { // super table query
tscHandleMasterSTableQuery(pSql);
return;
@ -1919,11 +1932,13 @@ void tscDoQuery(SSqlObj* pSql) {
}
}
int16_t tscGetJoinTagColIndexByUid(STagCond* pTagCond, uint64_t uid) {
int16_t tscGetJoinTagColIdByUid(STagCond* pTagCond, uint64_t uid) {
if (pTagCond->joinInfo.left.uid == uid) {
return pTagCond->joinInfo.left.tagCol;
return pTagCond->joinInfo.left.tagColId;
} else if (pTagCond->joinInfo.right.uid == uid) {
return pTagCond->joinInfo.right.tagColId;
} else {
return pTagCond->joinInfo.right.tagCol;
assert(0);
}
}
@ -1967,6 +1982,11 @@ bool tscHasReachLimitation(SQueryInfo* pQueryInfo, SSqlRes* pRes) {
return (pQueryInfo->clauseLimit > 0 && pRes->numOfClauseTotal >= pQueryInfo->clauseLimit);
}
bool tscResultsetFetchCompleted(TAOS_RES *result) {
SSqlRes* pRes = result;
return pRes->completed;
}
char* tscGetErrorMsgPayload(SSqlCmd* pCmd) { return pCmd->payload; }
/**
@ -1980,11 +2000,10 @@ bool hasMoreVnodesToTry(SSqlObj* pSql) {
return false;
}
SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, pCmd->clauseIndex);
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
assert(pRes->completed);
SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, pCmd->clauseIndex);
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
// for normal table, no need to try any more if results are all retrieved from one vnode
if (!UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo) || (pTableMetaInfo->vgroupList == NULL)) {
return false;
@ -2006,12 +2025,11 @@ void tscTryQueryNextVnode(SSqlObj* pSql, __async_cb_func_t fp) {
* if case of: multi-vnode super table projection query
*/
assert(pRes->numOfRows == 0 && tscNonOrderedProjectionQueryOnSTable(pQueryInfo, 0) && !tscHasReachLimitation(pQueryInfo, pRes));
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
int32_t totalVgroups = pTableMetaInfo->vgroupList->numOfVgroups;
while (++pTableMetaInfo->vgroupIndex < totalVgroups) {
tscTrace("%p results from vgroup index:%d completed, try next:%d. total vgroups:%d. current numOfRes:%d", pSql,
tscTrace("%p results from vgroup index:%d completed, try next:%d. total vgroups:%d. current numOfRes:%" PRId64, pSql,
pTableMetaInfo->vgroupIndex - 1, pTableMetaInfo->vgroupIndex, totalVgroups, pRes->numOfClauseTotal);
/*

74
src/client/tests/timeParseTest.cpp
View File

@ -24,19 +24,19 @@ TEST(testCase, parse_time) {
int64_t time = 0, time1 = 0;
taosParseTime(t1, &time, strlen(t1), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t1, &time, strlen(t1), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, 1514739661952);
taosParseTime(t13, &time, strlen(t13), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t13, &time, strlen(t13), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, timezone * MILLISECOND_PER_SECOND);
char t2[] = "2018-1-1T1:1:1.952Z";
taosParseTime(t2, &time, strlen(t2), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t2, &time, strlen(t2), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, 1514739661952 + 28800000);
char t3[] = "2018-1-1 1:01:01.952";
taosParseTime(t3, &time, strlen(t3), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t3, &time, strlen(t3), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, 1514739661952);
char t4[] = "2018-1-1 1:01:01.9";
@ -45,122 +45,122 @@ TEST(testCase, parse_time) {
char t7[] = "2018-01-01 01:01:01.9";
char t8[] = "2018-01-01 01:01:01.9007865";
taosParseTime(t4, &time, strlen(t4), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t5, &time1, strlen(t5), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t4, &time, strlen(t4), TSDB_TIME_PRECISION_MILLI, 0);
taosParseTime(t5, &time1, strlen(t5), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, time1);
taosParseTime(t4, &time, strlen(t4), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t6, &time1, strlen(t6), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t4, &time, strlen(t4), TSDB_TIME_PRECISION_MILLI, 0);
taosParseTime(t6, &time1, strlen(t6), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, time1);
taosParseTime(t4, &time, strlen(t4), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t7, &time1, strlen(t7), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t4, &time, strlen(t4), TSDB_TIME_PRECISION_MILLI, 0);
taosParseTime(t7, &time1, strlen(t7), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, time1);
taosParseTime(t5, &time, strlen(t5), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t8, &time1, strlen(t8), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t5, &time, strlen(t5), TSDB_TIME_PRECISION_MILLI, 0);
taosParseTime(t8, &time1, strlen(t8), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, time1);
char t9[] = "2017-4-3 1:1:2.980";
char t10[] = "2017-4-3T2:1:2.98+9:00";
taosParseTime(t9, &time, strlen(t9), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t10, &time1, strlen(t10), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t9, &time, strlen(t9), TSDB_TIME_PRECISION_MILLI, 0);
taosParseTime(t10, &time1, strlen(t10), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, time1);
char t11[] = "2017-4-3T2:1:2.98+09:00";
taosParseTime(t11, &time, strlen(t11), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t10, &time1, strlen(t10), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t11, &time, strlen(t11), TSDB_TIME_PRECISION_MILLI, 0);
taosParseTime(t10, &time1, strlen(t10), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, time1);
char t12[] = "2017-4-3T2:1:2.98+0900";
taosParseTime(t11, &time, strlen(t11), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t12, &time1, strlen(t12), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t11, &time, strlen(t11), TSDB_TIME_PRECISION_MILLI, 0);
taosParseTime(t12, &time1, strlen(t12), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, time1);
taos_options(TSDB_OPTION_TIMEZONE, "UTC");
deltaToUtcInitOnce();
taosParseTime(t13, &time, strlen(t13), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t13, &time, strlen(t13), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, 0);
taos_options(TSDB_OPTION_TIMEZONE, "Asia/Shanghai");
deltaToUtcInitOnce();
char t14[] = "1970-1-1T0:0:0Z";
taosParseTime(t14, &time, strlen(t14), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t14, &time, strlen(t14), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, 0);
char t40[] = "1970-1-1 0:0:0.999999999";
taosParseTime(t40, &time, strlen(t40), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t40, &time, strlen(t40), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, 999 + timezone * MILLISECOND_PER_SECOND);
char t41[] = "1997-1-1 0:0:0.999999999";
taosParseTime(t41, &time, strlen(t41), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t41, &time, strlen(t41), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, 852048000999);
int64_t k = timezone;
char t42[] = "1997-1-1T0:0:0.999999999Z";
taosParseTime(t42, &time, strlen(t42), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t42, &time, strlen(t42), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, 852048000999 - timezone * MILLISECOND_PER_SECOND);
////////////////////////////////////////////////////////////////////
// illegal timestamp format
char t15[] = "2017-12-33 0:0:0";
EXPECT_EQ(taosParseTime(t15, &time, strlen(t15), TSDB_TIME_PRECISION_MILLI), -1);
EXPECT_EQ(taosParseTime(t15, &time, strlen(t15), TSDB_TIME_PRECISION_MILLI, 0), -1);
char t16[] = "2017-12-31 99:0:0";
EXPECT_EQ(taosParseTime(t16, &time, strlen(t16), TSDB_TIME_PRECISION_MILLI), -1);
EXPECT_EQ(taosParseTime(t16, &time, strlen(t16), TSDB_TIME_PRECISION_MILLI, 0), -1);
char t17[] = "2017-12-31T9:0:0";
EXPECT_EQ(taosParseTime(t17, &time, strlen(t17), TSDB_TIME_PRECISION_MILLI), -1);
EXPECT_EQ(taosParseTime(t17, &time, strlen(t17), TSDB_TIME_PRECISION_MILLI, 0), -1);
char t18[] = "2017-12-31T9:0:0.Z";
EXPECT_EQ(taosParseTime(t18, &time, strlen(t18), TSDB_TIME_PRECISION_MILLI), -1);
EXPECT_EQ(taosParseTime(t18, &time, strlen(t18), TSDB_TIME_PRECISION_MILLI, 0), -1);
char t19[] = "2017-12-31 9:0:0.-1";
EXPECT_EQ(taosParseTime(t19, &time, strlen(t19), TSDB_TIME_PRECISION_MILLI), -1);
EXPECT_EQ(taosParseTime(t19, &time, strlen(t19), TSDB_TIME_PRECISION_MILLI, 0), -1);
char t20[] = "2017-12-31 9:0:0.1+12:99";
EXPECT_EQ(taosParseTime(t20, &time, strlen(t20), TSDB_TIME_PRECISION_MILLI), 0);
EXPECT_EQ(taosParseTime(t20, &time, strlen(t20), TSDB_TIME_PRECISION_MILLI, 0), 0);
EXPECT_EQ(time, 1514682000100);
char t21[] = "2017-12-31T9:0:0.1+12:99";
EXPECT_EQ(taosParseTime(t21, &time, strlen(t21), TSDB_TIME_PRECISION_MILLI), -1);
EXPECT_EQ(taosParseTime(t21, &time, strlen(t21), TSDB_TIME_PRECISION_MILLI, 0), -1);
char t22[] = "2017-12-31 9:0:0.1+13:1";
EXPECT_EQ(taosParseTime(t22, &time, strlen(t22), TSDB_TIME_PRECISION_MILLI), 0);
EXPECT_EQ(taosParseTime(t22, &time, strlen(t22), TSDB_TIME_PRECISION_MILLI, 0), 0);
char t23[] = "2017-12-31T9:0:0.1+13:1";
EXPECT_EQ(taosParseTime(t23, &time, strlen(t23), TSDB_TIME_PRECISION_MILLI), 0);
EXPECT_EQ(taosParseTime(t23, &time, strlen(t23), TSDB_TIME_PRECISION_MILLI, 0), 0);
//======================== add some case ============================//
char b1[] = "9999-12-31 23:59:59.999";
taosParseTime(b1, &time, strlen(b1), TSDB_TIME_PRECISION_MILLI);
taosParseTime(b1, &time, strlen(b1), TSDB_TIME_PRECISION_MILLI,0);
EXPECT_EQ(time, 253402271999999);
char b2[] = "2020-01-01 01:01:01.321";
taosParseTime(b2, &time, strlen(b2), TSDB_TIME_PRECISION_MILLI);
taosParseTime(b2, &time, strlen(b2), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, 1577811661321);
taos_options(TSDB_OPTION_TIMEZONE, "America/New_York");
deltaToUtcInitOnce();
taosParseTime(t13, &time, strlen(t13), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t13, &time, strlen(t13), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, 18000 * MILLISECOND_PER_SECOND);
taos_options(TSDB_OPTION_TIMEZONE, "Asia/Tokyo");
deltaToUtcInitOnce();
taosParseTime(t13, &time, strlen(t13), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t13, &time, strlen(t13), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, -32400 * MILLISECOND_PER_SECOND);
taos_options(TSDB_OPTION_TIMEZONE, "Asia/Shanghai");
deltaToUtcInitOnce();
taosParseTime(t13, &time, strlen(t13), TSDB_TIME_PRECISION_MILLI);
taosParseTime(t13, &time, strlen(t13), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, -28800 * MILLISECOND_PER_SECOND);
}

16
src/common/inc/tdataformat.h
View File

@ -36,7 +36,7 @@ extern "C" {
#define STR_WITH_MAXSIZE_TO_VARSTR(x, str, _maxs) \
do { \
char *_e = stpncpy(varDataVal(x), (str), (_maxs)); \
char *_e = stpncpy(varDataVal(x), (str), (_maxs)-VARSTR_HEADER_SIZE); \
varDataSetLen(x, (_e - (x)-VARSTR_HEADER_SIZE)); \
} while (0)
@ -69,8 +69,8 @@ typedef struct {
int version; // version
int numOfCols; // Number of columns appended
int tlen; // maximum length of a SDataRow without the header part
uint16_t flen; // First part length in a SDataRow after the header part
uint16_t vlen; // pure value part length, excluded the overhead
uint16_t flen; // First part length in a SDataRow after the header part
uint16_t vlen; // pure value part length, excluded the overhead
STColumn columns[];
} STSchema;
@ -83,8 +83,8 @@ typedef struct {
#define tdFreeSchema(s) tfree((s))
STSchema *tdDupSchema(STSchema *pSchema);
void * tdEncodeSchema(void *dst, STSchema *pSchema);
STSchema *tdDecodeSchema(void **psrc);
int tdEncodeSchema(void **buf, STSchema *pSchema);
void * tdDecodeSchema(void *buf, STSchema **pRSchema);
static FORCE_INLINE int comparColId(const void *key1, const void *key2) {
if (*(int16_t *)key1 > ((STColumn *)key2)->colId) {
@ -107,8 +107,8 @@ typedef struct {
int tCols;
int nCols;
int tlen;
uint16_t flen;
uint16_t vlen;
uint16_t flen;
uint16_t vlen;
int version;
STColumn *columns;
} STSchemaBuilder;
@ -288,7 +288,7 @@ typedef struct {
SKVRow tdKVRowDup(SKVRow row);
int tdSetKVRowDataOfCol(SKVRow *orow, int16_t colId, int8_t type, void *value);
void * tdEncodeKVRow(void *buf, SKVRow row);
int tdEncodeKVRow(void **buf, SKVRow row);
void * tdDecodeKVRow(void *buf, SKVRow *row);
static FORCE_INLINE int comparTagId(const void *key1, const void *key2) {

5
src/common/inc/tglobal.h
View File

@ -92,10 +92,6 @@ extern int32_t tsNumOfMnodes;
extern int32_t tsMaxShellConns;
extern int32_t tsMaxTables;
extern char tsDefaultDB[];
extern char tsDefaultUser[];
extern char tsDefaultPass[];
extern char tsMqttBrokerAddress[];
extern char tsMqttBrokerClientId[];
@ -170,6 +166,7 @@ extern char gitinfo[];
extern char gitinfoOfInternal[];
extern char buildinfo[];
extern int8_t tsDaylight;
extern char tsTimezone[64];
extern char tsLocale[64];
extern char tsCharset[64]; // default encode string

54
src/common/src/tdataformat.c
View File

@ -14,6 +14,7 @@
*/
#include "tdataformat.h"
#include "talgo.h"
#include "tcoding.h"
#include "wchar.h"
/**
@ -33,50 +34,50 @@ STSchema *tdDupSchema(STSchema *pSchema) {
/**
* Encode a schema to dst, and return the next pointer
*/
void *tdEncodeSchema(void *dst, STSchema *pSchema) {
int tdEncodeSchema(void **buf, STSchema *pSchema) {
int tlen = 0;
tlen += taosEncodeFixedI32(buf, schemaVersion(pSchema));
tlen += taosEncodeFixedI32(buf, schemaNCols(pSchema));
T_APPEND_MEMBER(dst, pSchema, STSchema, version);
T_APPEND_MEMBER(dst, pSchema, STSchema, numOfCols);
for (int i = 0; i < schemaNCols(pSchema); i++) {
STColumn *pCol = schemaColAt(pSchema, i);
T_APPEND_MEMBER(dst, pCol, STColumn, type);
T_APPEND_MEMBER(dst, pCol, STColumn, colId);
T_APPEND_MEMBER(dst, pCol, STColumn, bytes);
tlen += taosEncodeFixedI8(buf, colType(pCol));
tlen += taosEncodeFixedI16(buf, colColId(pCol));
tlen += taosEncodeFixedI32(buf, colBytes(pCol));
}
return dst;
return tlen;
}
/**
* Decode a schema from a binary.
*/
STSchema *tdDecodeSchema(void **psrc) {
int totalCols = 0;
void *tdDecodeSchema(void *buf, STSchema **pRSchema) {
int version = 0;
STSchemaBuilder schemaBuilder = {0};
int numOfCols = 0;
STSchemaBuilder schemaBuilder;
T_READ_MEMBER(*psrc, int, version);
T_READ_MEMBER(*psrc, int, totalCols);
buf = taosDecodeFixedI32(buf, &version);
buf = taosDecodeFixedI32(buf, &numOfCols);
if (tdInitTSchemaBuilder(&schemaBuilder, version) < 0) return NULL;
for (int i = 0; i < totalCols; i++) {
for (int i = 0; i < numOfCols; i++) {
int8_t type = 0;
int16_t colId = 0;
int32_t bytes = 0;
T_READ_MEMBER(*psrc, int8_t, type);
T_READ_MEMBER(*psrc, int16_t, colId);
T_READ_MEMBER(*psrc, int32_t, bytes);
buf = taosDecodeFixedI8(buf, &type);
buf = taosDecodeFixedI16(buf, &colId);
buf = taosDecodeFixedI32(buf, &bytes);
if (tdAddColToSchema(&schemaBuilder, type, colId, bytes) < 0) {
tdDestroyTSchemaBuilder(&schemaBuilder);
return NULL;
}
}
STSchema *pSchema = tdGetSchemaFromBuilder(&schemaBuilder);
*pRSchema = tdGetSchemaFromBuilder(&schemaBuilder);
tdDestroyTSchemaBuilder(&schemaBuilder);
return pSchema;
return buf;
}
int tdInitTSchemaBuilder(STSchemaBuilder *pBuilder, int32_t version) {
@ -105,7 +106,7 @@ void tdResetTSchemaBuilder(STSchemaBuilder *pBuilder, int32_t version) {
}
int tdAddColToSchema(STSchemaBuilder *pBuilder, int8_t type, int16_t colId, int32_t bytes) {
if (!isValidDataType(type, 0)) return -1;
if (!isValidDataType(type)) return -1;
if (pBuilder->nCols >= pBuilder->tCols) {
pBuilder->tCols *= 2;
@ -588,7 +589,7 @@ int tdSetKVRowDataOfCol(SKVRow *orow, int16_t colId, int8_t type, void *value) {
if (kvRowNCols(nrow) - colIdx - 1 > 0) {
for (int i = colIdx + 1; i < kvRowNCols(nrow); i++) {
kvRowColIdxAt(nrow, i)->colId = kvRowColIdxAt(row, i)->colId;
kvRowColIdxAt(nrow, i)->offset += diff;
kvRowColIdxAt(nrow, i)->offset = kvRowColIdxAt(row, i)->offset + diff;
}
memcpy(kvRowColVal(nrow, kvRowColIdxAt(nrow, colIdx + 1)), kvRowColVal(row, kvRowColIdxAt(row, colIdx + 1)),
POINTER_DISTANCE(kvRowEnd(row), kvRowColVal(row, kvRowColIdxAt(row, colIdx + 1))));
@ -605,14 +606,19 @@ int tdSetKVRowDataOfCol(SKVRow *orow, int16_t colId, int8_t type, void *value) {
return 0;
}
void *tdEncodeKVRow(void *buf, SKVRow row) {
int tdEncodeKVRow(void **buf, SKVRow row) {
// May change the encode purpose
kvRowCpy(buf, row);
return POINTER_SHIFT(buf, kvRowLen(row));
if (buf != NULL) {
kvRowCpy(*buf, row);
*buf = POINTER_SHIFT(*buf, kvRowLen(row));
}
return kvRowLen(row);
}
void *tdDecodeKVRow(void *buf, SKVRow *row) {
*row = tdKVRowDup(buf);
if (*row == NULL) return NULL;
return POINTER_SHIFT(buf, kvRowLen(*row));
}

36
src/common/src/tglobal.c
View File

@ -109,10 +109,6 @@ int32_t tsReplications = TSDB_DEFAULT_REPLICA_NUM;
int16_t tsAffectedRowsMod = 0;
int32_t tsNumOfMnodes = 3;
int32_t tsMaxShellConns = 5000;
char tsDefaultDB[TSDB_DB_NAME_LEN] = {0};
char tsDefaultUser[64] = "root";
char tsDefaultPass[64] = "taosdata";
int32_t tsMaxConnections = 5000;
int32_t tsBalanceInterval = 300; // seconds
@ -198,6 +194,7 @@ char tsMonitorDbName[TSDB_DB_NAME_LEN] = "log";
char tsInternalPass[] = "secretkey";
int32_t tsMonitorInterval = 30; // seconds
int8_t tsDaylight = 0;
char tsTimezone[64] = {0};
char tsLocale[TSDB_LOCALE_LEN] = {0};
char tsCharset[TSDB_LOCALE_LEN] = {0}; // default encode string
@ -712,37 +709,6 @@ static void doInitGlobalConfig() {
cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg);
// login configs
cfg.option = "defaultDB";
cfg.ptr = tsDefaultDB;
cfg.valType = TAOS_CFG_VTYPE_STRING;
cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_CLIENT;
cfg.minValue = 0;
cfg.maxValue = 0;
cfg.ptrLength = TSDB_DB_NAME_LEN - 1;
cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg);
cfg.option = "defaultUser";
cfg.ptr = tsDefaultUser;
cfg.valType = TAOS_CFG_VTYPE_STRING;
cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_CLIENT;
cfg.minValue = 0;
cfg.maxValue = 0;
cfg.ptrLength = TSDB_USER_LEN - 1;
cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg);
cfg.option = "defaultPass";
cfg.ptr = tsDefaultPass;
cfg.valType = TAOS_CFG_VTYPE_STRING;
cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_CLIENT | TSDB_CFG_CTYPE_B_NOT_PRINT;
cfg.minValue = 0;
cfg.maxValue = 0;
cfg.ptrLength = TSDB_PASSWORD_LEN - 1;
cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg);
cfg.option = "mqttBrokerAddress";
cfg.ptr = tsMqttBrokerAddress;
cfg.valType = TAOS_CFG_VTYPE_STRING;

1
src/common/src/ttimezone.c
View File

@ -58,6 +58,7 @@ void tsSetTimeZone() {
* (BST, +0100)
*/
sprintf(tsTimezone, "(%s, %s%02d00)", tzname[daylight], tz >= 0 ? "+" : "-", abs(tz));
tsDaylight = daylight;
uPrint("timezone format changed to %s", tsTimezone);
}

12
src/common/src/ttypes.c
View File

@ -363,16 +363,8 @@ char tTokenTypeSwitcher[13] = {
TSDB_DATA_TYPE_NCHAR, // TK_NCHAR
};
bool isValidDataType(int32_t type, int32_t length) {
if (type < TSDB_DATA_TYPE_NULL || type > TSDB_DATA_TYPE_NCHAR) {
return false;
}
if (type == TSDB_DATA_TYPE_BINARY || type == TSDB_DATA_TYPE_NCHAR) {
// return length >= 0 && length <= TSDB_MAX_BINARY_LEN;
}
return true;
bool isValidDataType(int32_t type) {
return type >= TSDB_DATA_TYPE_NULL && type <= TSDB_DATA_TYPE_NCHAR;
}
bool isNull(const char *val, int32_t type) {

49
src/connector/jdbc/.classpath Normal file
View File

@ -0,0 +1,49 @@
<?xml version="1.0" encoding="UTF-8"?>
<classpath>
<classpathentry kind="src" output="target/classes" path="src/main/java">
<attributes>
<attribute name="optional" value="true"/>
<attribute name="maven.pomderived" value="true"/>
</attributes>
</classpathentry>
<classpathentry excluding="**" kind="src" output="target/classes" path="src/main/resources">
<attributes>
<attribute name="maven.pomderived" value="true"/>
</attributes>
</classpathentry>
<classpathentry kind="src" output="target/test-classes" path="src/test/java">
<attributes>
<attribute name="optional" value="true"/>
<attribute name="maven.pomderived" value="true"/>
<attribute name="test" value="true"/>
</attributes>
</classpathentry>
<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-1.8">
<attributes>
<attribute name="maven.pomderived" value="true"/>
</attributes>
</classpathentry>
<classpathentry kind="con" path="org.eclipse.m2e.MAVEN2_CLASSPATH_CONTAINER">
<attributes>
<attribute name="maven.pomderived" value="true"/>
</attributes>
</classpathentry>
<classpathentry kind="src" path="target/generated-sources/annotations">
<attributes>
<attribute name="optional" value="true"/>
<attribute name="maven.pomderived" value="true"/>
<attribute name="ignore_optional_problems" value="true"/>
<attribute name="m2e-apt" value="true"/>
</attributes>
</classpathentry>
<classpathentry kind="src" output="target/test-classes" path="target/generated-test-sources/test-annotations">
<attributes>
<attribute name="optional" value="true"/>
<attribute name="maven.pomderived" value="true"/>
<attribute name="ignore_optional_problems" value="true"/>
<attribute name="m2e-apt" value="true"/>
<attribute name="test" value="true"/>
</attributes>
</classpathentry>
<classpathentry kind="output" path="target/classes"/>
</classpath>

23
src/connector/jdbc/.project Normal file
View File

@ -0,0 +1,23 @@
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>taos-jdbcdriver</name>
<comment></comment>
<projects>
</projects>
<buildSpec>
<buildCommand>
<name>org.eclipse.jdt.core.javabuilder</name>
<arguments>
</arguments>
</buildCommand>
<buildCommand>
<name>org.eclipse.m2e.core.maven2Builder</name>
<arguments>
</arguments>
</buildCommand>
</buildSpec>
<natures>
<nature>org.eclipse.jdt.core.javanature</nature>
<nature>org.eclipse.m2e.core.maven2Nature</nature>
</natures>
</projectDescription>

5
src/connector/jdbc/.settings/org.eclipse.core.resources.prefs
View File

@ -1,5 +0,0 @@
eclipse.preferences.version=1
encoding//src/main/java=GBK
encoding//src/main/java/com/taosdata/jdbc/TSDBJNIConnector.java=UTF-8
encoding//src/main/resources=UTF-8
encoding/<project>=UTF-8

13
src/connector/jdbc/.settings/org.eclipse.jdt.core.prefs
View File

@ -1,13 +0,0 @@
eclipse.preferences.version=1
org.eclipse.jdt.core.compiler.codegen.inlineJsrBytecode=enabled
org.eclipse.jdt.core.compiler.codegen.methodParameters=do not generate
org.eclipse.jdt.core.compiler.codegen.targetPlatform=1.6
org.eclipse.jdt.core.compiler.codegen.unusedLocal=preserve
org.eclipse.jdt.core.compiler.compliance=1.6
org.eclipse.jdt.core.compiler.debug.lineNumber=generate
org.eclipse.jdt.core.compiler.debug.localVariable=generate
org.eclipse.jdt.core.compiler.debug.sourceFile=generate
org.eclipse.jdt.core.compiler.problem.assertIdentifier=error
org.eclipse.jdt.core.compiler.problem.enumIdentifier=error
org.eclipse.jdt.core.compiler.problem.forbiddenReference=warning
org.eclipse.jdt.core.compiler.source=1.6

4
src/connector/jdbc/.settings/org.eclipse.m2e.core.prefs
View File

@ -1,4 +0,0 @@
activeProfiles=
eclipse.preferences.version=1
resolveWorkspaceProjects=true
version=1

237
src/connector/jdbc/pom.xml
View File

@ -1,105 +1,136 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.taosdata.jdbc</groupId>
<artifactId>taos-jdbcdriver</artifactId>
<version>1.0.3</version>
<packaging>jar</packaging>
<name>JDBCDriver</name>
<url>https://github.com/taosdata/TDengine/tree/master/src/connector/jdbc</url>
<description>TDengine JDBC Driver</description>
<licenses>
<license>
<name>GNU AFFERO GENERAL PUBLIC LICENSE Version 3</name>
<url>https://github.com/taosdata/TDengine/blob/master/LICENSE</url>
<distribution>repo</distribution>
</license>
</licenses>
<scm>
<connection>scm:git:git://github.com/taosdata/TDengine.git</connection>
<developerConnection>scm:git:git@github.com:taosdata/TDengine.git</developerConnection>
<url>https://github.com/taosdata/TDengine/tree/master/src/connector/jdbc</url>
<tag>HEAD</tag>
</scm>
<developers>
<developer>
<name>taosdata</name>
<email>support@taosdata.com</email>
<organization>https://www.taosdata.com/</organization>
<organizationUrl>https://www.taosdata.com/</organizationUrl>
</developer>
</developers>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<java.version>1.8</java.version>
<maven-compiler-plugin.version>3.6.0</maven-compiler-plugin.version>
<commons-logging.version>1.1.2</commons-logging.version>
<commons-lang3.version>3.5</commons-lang3.version>
</properties>
<dependencies>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>${commons-logging.version}</version>
<exclusions>
<exclusion>
<groupId>*</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>${commons-lang3.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.0.0</version>
<configuration>
<descriptors>
<!--<descriptor>src/main/assembly/assembly.xml</descriptor>-->
<descriptor>src/main/assembly/assembly-jar.xml</descriptor>
</descriptors>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven-compiler-plugin.version}</version>
<configuration>
<encoding>UTF-8</encoding>
<source>${java.version}</source>
<target>${java.version}</target>
<debug>true</debug>
<showDeprecation>true</showDeprecation>
</configuration>
</plugin>
</plugins>
</build>
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.taosdata.jdbc</groupId>
<artifactId>taos-jdbcdriver</artifactId>
<version>1.0.3</version>
<packaging>jar</packaging>
<name>JDBCDriver</name>
<url>https://github.com/taosdata/TDengine/tree/master/src/connector/jdbc</url>
<description>TDengine JDBC Driver</description>
<licenses>
<license>
<name>GNU AFFERO GENERAL PUBLIC LICENSE Version 3</name>
<url>https://github.com/taosdata/TDengine/blob/master/LICENSE</url>
<distribution>repo</distribution>
</license>
</licenses>
<scm>
<connection>scm:git:git://github.com/taosdata/TDengine.git</connection>
<developerConnection>scm:git:git@github.com:taosdata/TDengine.git</developerConnection>
<url>https://github.com/taosdata/TDengine/tree/master/src/connector/jdbc</url>
<tag>HEAD</tag>
</scm>
<developers>
<developer>
<name>taosdata</name>
<email>support@taosdata.com</email>
<organization>https://www.taosdata.com/</organization>
<organizationUrl>https://www.taosdata.com/</organizationUrl>
</developer>
</developers>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<java.version>1.8</java.version>
<maven-compiler-plugin.version>3.6.0</maven-compiler-plugin.version>
<commons-logging.version>1.1.2</commons-logging.version>
<commons-lang3.version>3.5</commons-lang3.version>
</properties>
<dependencies>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>${commons-logging.version}</version>
<exclusions>
<exclusion>
<groupId>*</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>${commons-lang3.version}</version>
</dependency>
<dependency>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.3</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.0.0</version>
<configuration>
<descriptors>
<descriptor>src/main/assembly/assembly-jar.xml</descriptor>
</descriptors>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven-compiler-plugin.version}</version>
<configuration>
<encoding>UTF-8</encoding>
<source>${java.version}</source>
<target>${java.version}</target>
<debug>true</debug>
<showDeprecation>true</showDeprecation>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.4</version>
<configuration>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.3</version>
<configuration>
<includes>
<include>com/**/*</include>
</includes>
</configuration>
<executions>
<execution>
<id>pre-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>post-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>

5
src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBDatabaseMetaData.java
View File

@ -527,8 +527,9 @@ public class TSDBDatabaseMetaData implements java.sql.DatabaseMetaData {
public ResultSet getTables(String catalog, String schemaPattern, String tableNamePattern, String[] types)
throws SQLException {
if (conn != null && !conn.isClosed()) {
Statement stmt = conn.createStatement();
Statement stmt = null;
if (null != conn && !conn.isClosed()) {
stmt = conn.createStatement();
if (catalog == null || catalog.length() < 1) {
catalog = conn.getCatalog();
}

223
src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBError.java
View File

@ -1,223 +0,0 @@
/***************************************************************************
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*****************************************************************************/
package com.taosdata.jdbc;
import java.util.HashMap;
import java.util.Map;
/**
*
* TDengine error code and error message enumeration.
*
*/
public enum TSDBError {
TSDB_CODE_SUCCESS(0, "success"),
TSDB_CODE_ACTION_IN_PROGRESS(1, "in progress"),
TSDB_CODE_LAST_SESSION_NOT_FINISHED(5, "last session not finished"),
TSDB_CODE_INVALID_SESSION_ID(6, "invalid session ID"),
TSDB_CODE_INVALID_TRAN_ID(7, "invalid tran ID"),
TSDB_CODE_INVALID_MSG_TYPE(8, "invalid msg type"),
TSDB_CODE_ALREADY_PROCESSED(9, "alredy processed"),
TSDB_CODE_AUTH_FAILURE(10, "authentication failure"),
TSDB_CODE_WRONG_MSG_SIZE(11, "wrong msg size"),
TSDB_CODE_UNEXPECTED_RESPONSE(12, "unexpected response"),
TSDB_CODE_INVALID_RESPONSE_TYPE(13, "invalid response type"),
TSDB_CODE_NO_RESOURCE(14, "no resource"),
TSDB_CODE_INVALID_TIME_STAMP(15, "invalid time stamp"),
TSDB_CODE_MISMATCHED_METER_ID(16, "mismatched meter ID"),
TSDB_CODE_ACTION_TRANS_NOT_FINISHED(17, "transcation not finished"),
TSDB_CODE_ACTION_NOT_ONLINE(18, "not online"),
TSDB_CODE_ACTION_SEND_FAILD(19, "send failed"),
TSDB_CODE_NOT_ACTIVE_SESSION(20, "not active session"),
TSDB_CODE_INSERT_FAILED(21, "insert failed"),
TSDB_CODE_APP_ERROR(22, "App error"),
TSDB_CODE_INVALID_IE(23, "invalid IE"),
TSDB_CODE_INVALID_VALUE(24, "invalid value"),
TSDB_CODE_REDIRECT(25, "service not available"),
TSDB_CODE_ALREADY_THERE(26, "already there"),
TSDB_CODE_INVALID_TABLE_ID(27, "invalid meter ID"),
TSDB_CODE_INVALID_SQL(28, "invalid SQL"), // this message often comes with additional info which will vary based on the specific error situation
TSDB_CODE_NETWORK_UNAVAIL(29, "failed to connect to server"),
TSDB_CODE_INVALID_MSG_LEN(30, "invalid msg len"),
TSDB_CODE_INVALID_DB(31, "invalid DB"),
TSDB_CODE_INVALID_TABLE(32, "invalid table"),
TSDB_CODE_DB_ALREADY_EXIST(33, "DB already there"),
TSDB_CODE_TABLE_ALREADY_EXIST(34, "table already there"),
TSDB_CODE_INVALID_USER(35, "invalid user name"),
TSDB_CODE_INVALID_ACCT(36, "invalid acct name"),
TSDB_CODE_INVALID_PASS(37, "invalid password"),
TSDB_CODE_DB_NOT_SELECTED(38, "DB not selected"),
TSDB_CODE_MEMORY_CORRUPTED(39, "memory corrupted"),
TSDB_CODE_USER_ALREADY_EXIST(40, "user name exists"),
TSDB_CODE_NO_RIGHTS(41, "not authorized"),
TSDB_CODE_DISCONNECTED(42, "login disconnected), login again"),
TSDB_CODE_NO_MASTER(43, "mgmt master node not available"),
TSDB_CODE_NOT_CONFIGURED(44, "not configured"),
TSDB_CODE_INVALID_OPTION(45, "invalid option"),
TSDB_CODE_NODE_OFFLINE(46, "node offline"),
TSDB_CODE_SYNC_REQUIRED(47, "sync required"),
TSDB_CODE_NO_ENOUGH_DNODES(48, "more dnodes are needed"),
TSDB_CODE_UNSYNCED(49, "node in unsynced state"),
TSDB_CODE_TOO_SLOW(50, "too slow"),
TSDB_CODE_OTHERS(51, "others"),
TSDB_CODE_NO_REMOVE_MASTER(52, "can't remove dnode which is master"),
TSDB_CODE_WRONG_SCHEMA(53, "wrong schema"),
TSDB_CODE_NO_RESULT(54, "no results"),
TSDB_CODE_TOO_MANY_USERS(55, "num of users execeed maxUsers"),
TSDB_CODE_TOO_MANY_DATABSES(56, "num of databases execeed maxDbs"),
TSDB_CODE_TOO_MANY_TABLES(57, "num of tables execeed maxTables"),
TSDB_CODE_TOO_MANY_DNODES(58, "num of dnodes execeed maxDnodes"),
TSDB_CODE_TOO_MANY_ACCTS(59, "num of accounts execeed maxAccts"),
TSDB_CODE_ACCT_ALREADY_EXIST(60, "accout name exists"),
TSDB_CODE_DNODE_ALREADY_EXIST(61, "dnode ip exists"),
TSDB_CODE_SDB_ERROR(62, "sdb error"),
TSDB_CODE_METRICMETA_EXPIRED(63, "metric meta expired"), // local cached metric-meta expired causes error in metric query
TSDB_CODE_NOT_READY(64, "not ready"), // peer is not ready to process data
TSDB_CODE_MAX_SESSIONS(65, "too many sessions on server"), // too many sessions
TSDB_CODE_MAX_CONNECTIONS(66, "too many sessions from app"), // too many connections
TSDB_CODE_SESSION_ALREADY_EXIST(67, "session to dest is already there"),
TSDB_CODE_NO_QSUMMARY(68, "query list not there), please show again"),
TSDB_CODE_SERV_OUT_OF_MEMORY(69, "server out of memory"),
TSDB_CODE_INVALID_QHANDLE(70, "invalid query handle"),
TSDB_CODE_RELATED_TABLES_EXIST(71, "tables related to metric exist"),
TSDB_CODE_MONITOR_DB_FORBEIDDEN(72, "can't drop monitor database or tables"),
TSDB_CODE_VG_COMMITLOG_INIT_FAILED(73, "commit log init failed"),
TSDB_CODE_VG_INIT_FAILED(74, "vgroup init failed"),
TSDB_CODE_DATA_ALREADY_IMPORTED(75, "data is already imported"),
TSDB_CODE_OPS_NOT_SUPPORT(76, "not supported operation"),
TSDB_CODE_INVALID_QUERY_ID(77, "invalid query id string"),
TSDB_CODE_INVALID_STREAM_ID(78, "invalid stream id string"),
TSDB_CODE_INVALID_CONNECTION(79, "invalid connection string"),
TSDB_CODE_ACTION_NOT_BALANCED(80, "dnode not balanced"),
TSDB_CODE_CLI_OUT_OF_MEMORY(81, "client out of memory"),
TSDB_CODE_DATA_OVERFLOW(82, "data value overflow"),
TSDB_CODE_QUERY_CANCELLED(83, "query cancelled"),
TSDB_CODE_GRANT_POINT_LIMITED(84, "grant points limited"),
TSDB_CODE_GRANT_EXPIRED(85, "grant expired"),
TSDB_CODE_CLI_NO_DISKSPACE(86, "client no disk space"),
TSDB_CODE_FILE_CORRUPTED(87, "DB file corrupted"),
TSDB_CODE_INVALID_CLIENT_VERSION(88, "version of client and server not match");
private long errCode;
private String errMessage;
private static Map<Integer, String> errorCodeMap = new HashMap<>(86);
static {
errorCodeMap.put(0, "success");
errorCodeMap.put(1, "in progress");
errorCodeMap.put(5, "last session not finished");
errorCodeMap.put(6, "invalid session ID");
errorCodeMap.put(7, "invalid tran ID");
errorCodeMap.put(8, "invalid msg type");
errorCodeMap.put(9, "alredy processed");
errorCodeMap.put(10, "authentication failure");
errorCodeMap.put(11, "wrong msg size");
errorCodeMap.put(12, "unexpected response");
errorCodeMap.put(13, "invalid response type");
errorCodeMap.put(14, "no resource");
errorCodeMap.put(15, "invalid time stamp");
errorCodeMap.put(16, "mismatched meter ID");
errorCodeMap.put(17, "transcation not finished");
errorCodeMap.put(18, "not online");
errorCodeMap.put(19, "send failed");
errorCodeMap.put(20, "not active session");
errorCodeMap.put(21, "insert failed");
errorCodeMap.put(22, "App error");
errorCodeMap.put(23, "invalid IE");
errorCodeMap.put(24, "invalid value");
errorCodeMap.put(25, "service not available");
errorCodeMap.put(26, "already there");
errorCodeMap.put(27, "invalid meter ID");
errorCodeMap.put(28, "invalid SQL"); // this message often comes with additional info which will vary based on the specific error situation
errorCodeMap.put(29, "failed to connect to server");
errorCodeMap.put(30, "invalid msg len");
errorCodeMap.put(31, "invalid DB");
errorCodeMap.put(32, "invalid table");
errorCodeMap.put(33, "DB already there");
errorCodeMap.put(34, "table already there");
errorCodeMap.put(35, "invalid user name");
errorCodeMap.put(36, "invalid acct name");
errorCodeMap.put(37, "invalid password");
errorCodeMap.put(38, "DB not selected");
errorCodeMap.put(39, "memory corrupted");
errorCodeMap.put(40, "user name exists");
errorCodeMap.put(41, "not authorized");
errorCodeMap.put(42, "login disconnected); login again");
errorCodeMap.put(43, "mgmt master node not available");
errorCodeMap.put(44, "not configured");
errorCodeMap.put(45, "invalid option");
errorCodeMap.put(46, "node offline");
errorCodeMap.put(47, "sync required");
errorCodeMap.put(48, "more dnodes are needed");
errorCodeMap.put(49, "node in unsynced state");
errorCodeMap.put(50, "too slow");
errorCodeMap.put(51, "others");
errorCodeMap.put(52, "can't remove dnode which is master");
errorCodeMap.put(53, "wrong schema");
errorCodeMap.put(54, "no results");
errorCodeMap.put(55, "num of users execeed maxUsers");
errorCodeMap.put(56, "num of databases execeed maxDbs");
errorCodeMap.put(57, "num of tables execeed maxTables");
errorCodeMap.put(58, "num of dnodes execeed maxDnodes");
errorCodeMap.put(59, "num of accounts execeed maxAccts");
errorCodeMap.put(60, "accout name exists");
errorCodeMap.put(61, "dnode ip exists");
errorCodeMap.put(62, "sdb error");
errorCodeMap.put(63, "metric meta expired"); // local cached metric-meta expired causes error in metric query
errorCodeMap.put(64, "not ready"); // peer is not ready to process data
errorCodeMap.put(65, "too many sessions on server"); // too many sessions
errorCodeMap.put(66, "too many sessions from app"); // too many connections
errorCodeMap.put(67, "session to dest is already there");
errorCodeMap.put(68, "query list not there); please show again");
errorCodeMap.put(69, "server out of memory");
errorCodeMap.put(70, "invalid query handle");
errorCodeMap.put(71, "tables related to metric exist");
errorCodeMap.put(72, "can't drop monitor database or tables");
errorCodeMap.put(73, "commit log init failed");
errorCodeMap.put(74, "vgroup init failed");
errorCodeMap.put(75, "data is already imported");
errorCodeMap.put(76, "not supported operation");
errorCodeMap.put(77, "invalid query id string");
errorCodeMap.put(78, "invalid stream id string");
errorCodeMap.put(79, "invalid connection string");
errorCodeMap.put(80, "dnode not balanced");
errorCodeMap.put(81, "client out of memory");
errorCodeMap.put(82, "data value overflow");
errorCodeMap.put(83, "query cancelled");
errorCodeMap.put(84, "grant points limited");
errorCodeMap.put(85, "grant expired");
errorCodeMap.put(86, "client no disk space");
errorCodeMap.put(87, "DB file corrupted");
errorCodeMap.put(88, "version of client and server not match");
}
TSDBError(long code, String message) {
this.errCode = code;
this.errMessage = message;
}
public long getErrCode() {
return this.errCode;
}
public String getErrMessage() {
return this.errMessage;
}
public static String getErrMessageByCode(long errCode) {
return errorCodeMap.get(errCode);
}
}

15
src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBJNIConnector.java
View File

@ -19,7 +19,7 @@ import java.sql.SQLWarning;
import java.util.List;
public class TSDBJNIConnector {
static volatile Boolean isInitialized = false;
private static volatile Boolean isInitialized = false;
static {
System.loadLibrary("taos");
@ -51,8 +51,6 @@ public class TSDBJNIConnector {
/**
* Returns the status of last result set in current connection
*
* @return
*/
public boolean isResultsetClosed() {
return this.isResultsetClosed;
@ -112,7 +110,7 @@ public class TSDBJNIConnector {
*
* @throws SQLException
*/
public int executeQuery(String sql) throws SQLException {
public long executeQuery(String sql) throws SQLException {
if (!this.isResultsetClosed) {
freeResultSet(taosResultSetPointer);
}
@ -127,7 +125,6 @@ public class TSDBJNIConnector {
}
int code = this.getErrCode(pSql);
affectedRows = code;
if (code < 0) {
affectedRows = -1;
if (code == TSDBConstants.JNI_TDENGINE_ERROR) {
@ -146,7 +143,7 @@ public class TSDBJNIConnector {
if (taosResultSetPointer != TSDBConstants.JNI_NULL_POINTER) {
isResultsetClosed = false;
}
return code;
return pSql;
}
private native long executeQueryImp(byte[] sqlBytes, long connection);
@ -199,8 +196,6 @@ public class TSDBJNIConnector {
/**
* Close the open result set which is associated to the current connection. If the result set is already
* closed, return 0 for success.
*
* @return
*/
public int freeResultSet() {
int resCode = TSDBConstants.JNI_SUCCESS;
@ -217,7 +212,7 @@ public class TSDBJNIConnector {
/**
* Get affected rows count
*/
public int getAffectedRows(Long pSql) {
public int getAffectedRows(long pSql) {
int affectedRows = this.affectedRows;
if (affectedRows < 0) {
affectedRows = this.getAffectedRowsImp(this.taos, pSql);
@ -225,7 +220,7 @@ public class TSDBJNIConnector {
return affectedRows;
}
private native int getAffectedRowsImp(long connection, Long pSql);
private native int getAffectedRowsImp(long connection, long pSql);
/**
* Get schema metadata

19
src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBStatement.java
View File

@ -51,13 +51,16 @@ public class TSDBStatement implements Statement {
if (isClosed) {
throw new SQLException("Invalid method call on a closed statement.");
}
this.connecter.executeQuery(sql);
pSql = this.connecter.executeQuery(sql);
long resultSetPointer = this.connecter.getResultSet();
if (resultSetPointer == TSDBConstants.JNI_CONNECTION_NULL) {
this.connecter.freeResultSet(pSql);
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_CONNECTION_NULL));
} else if (resultSetPointer == TSDBConstants.JNI_NULL_POINTER) {
// create/insert/update/del/alter
this.connecter.freeResultSet(pSql);
return null;
} else {
return new TSDBResultSet(this.connecter, resultSetPointer);
@ -68,17 +71,18 @@ public class TSDBStatement implements Statement {
if (isClosed) {
throw new SQLException("Invalid method call on a closed statement.");
}
long res = this.connecter.executeQuery(sql);
pSql = this.connecter.executeQuery(sql);
long resultSetPointer = this.connecter.getResultSet();
if (resultSetPointer == TSDBConstants.JNI_CONNECTION_NULL) {
this.connecter.freeResultSet(res);
this.connecter.freeResultSet(pSql);
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_CONNECTION_NULL));
} else if (resultSetPointer != TSDBConstants.JNI_NULL_POINTER) {
this.connecter.freeResultSet();
throw new SQLException("The executed SQL is not a DML or a DDL");
} else {
int num = this.connecter.getAffectedRows(res);
int num = this.connecter.getAffectedRows(pSql);
this.connecter.freeResultSet(pSql);
return num;
}
}
@ -146,16 +150,19 @@ public class TSDBStatement implements Statement {
throw new SQLException("Invalid method call on a closed statement.");
}
boolean res = true;
this.connecter.executeQuery(sql);
pSql = this.connecter.executeQuery(sql);
long resultSetPointer = this.connecter.getResultSet();
if (resultSetPointer == TSDBConstants.JNI_CONNECTION_NULL) {
this.connecter.freeResultSet(pSql);
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_CONNECTION_NULL));
} else if (resultSetPointer == TSDBConstants.JNI_NULL_POINTER) {
// no result set is retrieved
res = false;
}
return res;
this.connecter.freeResultSet(pSql);
return res;
}
public ResultSet getResultSet() throws SQLException {

3
src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBSubscribe.java
View File

@ -171,8 +171,7 @@ public class TSDBSubscribe {
state = 1;
try {
TSDBResultSet resultSet = consume(subscription);
callBack.invoke(resultSet);
callBack.invoke(consume(subscription));
} catch (Exception e) {
this.cancel();
throw new RuntimeException(e);

15
src/connector/jdbc/src/test/java/TestAsyncTSDBSubscribe.java
View File

@ -3,10 +3,11 @@ import org.apache.commons.lang3.StringUtils;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.util.Properties;
public class TestAsyncTSDBSubscribe {
public static void main(String[] args) {
public static void main(String[] args) throws SQLException {
String usage = "java -cp taos-jdbcdriver-1.0.3_dev-dist.jar com.taosdata.jdbc.TSDBSubscribe -db dbName -topic topicName " +
"-tname tableName -h host";
if (args.length < 2) {
@ -38,15 +39,18 @@ public class TestAsyncTSDBSubscribe {
}
Connection connection = null;
TSDBSubscribe subscribe = null;
long subscribId = 0;
try {
Class.forName("com.taosdata.jdbc.TSDBDriver");
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host);
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host);
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
connection = DriverManager.getConnection("jdbc:TAOS://" + host + ":0/" + dbName + "?user=root&password=taosdata", properties);
String rawSql = "select * from " + tName + ";";
subscribe = ((TSDBConnection) connection).createSubscribe();
TSDBSubscribe subscribe = ((TSDBConnection) connection).createSubscribe();
subscribId = subscribe.subscribe(topic, rawSql, false, 1000, new CallBack("first"));
long subscribId2 = subscribe.subscribe("test", rawSql, false, 1000, new CallBack("second"));
int a = 0;
@ -55,6 +59,9 @@ public class TestAsyncTSDBSubscribe {
System.err.println("cancel subscribe");
} catch (Exception e) {
e.printStackTrace();
if (null != connection && !connection.isClosed()) {
connection.close();
}
}
}

13
src/connector/jdbc/src/test/java/TestPreparedStatement.java
View File

@ -6,24 +6,31 @@ import java.util.Properties;
public class TestPreparedStatement {
public static void main(String[] args) {
public static void main(String[] args) throws SQLException {
Connection connection = null;
try {
Class.forName("com.taosdata.jdbc.TSDBDriver");
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, "localhost");
Connection connection = DriverManager.getConnection("jdbc:TAOS://localhost:0/?user=root&password=taosdata", properties);
connection = DriverManager.getConnection("jdbc:TAOS://localhost:0/?user=root&password=taosdata", properties);
String rawSql = "select * from test.log0601";
// String[] params = new String[]{"ts", "c1"};
PreparedStatement pstmt = (TSDBPreparedStatement) connection.prepareStatement(rawSql);
ResultSet resSet = pstmt.executeQuery();
while(resSet.next()) {
for (int i = 1; i <= resSet.getMetaData().getColumnCount(); i++) {
System.out.printf("%d: %s\n", i, resSet.getString(i));
System.out.printf("%d: %s \n", i, resSet.getString(i));
}
}
resSet.close();
pstmt.close();
connection.close();
} catch (Exception e) {
e.printStackTrace();
if (null != connection) {
connection.close();
}
}
}
}

22
src/connector/jdbc/src/test/java/TestTSDBDatabaseMetaData.java
View File

@ -1,29 +1,33 @@
import com.taosdata.jdbc.TSDBDriver;
import java.sql.Connection;
import java.sql.DatabaseMetaData;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.*;
import java.util.Properties;
public class TestTSDBDatabaseMetaData {
public static void main(String[] args) {
public static void main(String[] args) throws SQLException {
Connection connection = null;
DatabaseMetaData dbMetaData = null;
ResultSet resSet = null;
try {
Class.forName("com.taosdata.jdbc.TSDBDriver");
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, "192.168.1.114");
Connection connection = DriverManager.getConnection("jdbc:TAOS://192.168.1.114:0/?user=root&password=taosdata", properties);
DatabaseMetaData dbMetaData = connection.getMetaData();
ResultSet resSet = dbMetaData.getCatalogs();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, "localhost");
connection = DriverManager.getConnection("jdbc:TAOS://localhost:0/?user=root&password=taosdata", properties);
dbMetaData = connection.getMetaData();
resSet = dbMetaData.getCatalogs();
while(resSet.next()) {
for (int i = 1; i <= resSet.getMetaData().getColumnCount(); i++) {
System.out.printf("dbMetaData.getCatalogs(%d) = %s\n", i, resSet.getString(i));
}
}
resSet.close();
} catch (Exception e) {
e.printStackTrace();
if (null != connection) {
connection.close();
}
}
}
}

8
src/connector/jdbc/src/test/java/TestTSDBSubscribe.java
View File

@ -46,16 +46,20 @@ public class TestTSDBSubscribe {
try {
Class.forName("com.taosdata.jdbc.TSDBDriver");
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host);
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host);
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
connection = DriverManager.getConnection("jdbc:TAOS://" + host + ":0/" + dbName + "?user=root&password=taosdata"
, properties);
String rawSql = "select * from " + tName + ";";
subscribe = ((TSDBConnection) connection).createSubscribe();
subscribId = subscribe.subscribe(topic, rawSql, false, 1000);
int a = 0;
TSDBResultSet resSet = null;
while (true) {
Thread.sleep(900);
TSDBResultSet resSet = subscribe.consume(subscribId);
resSet = subscribe.consume(subscribId);
while (resSet.next()) {
for (int i = 1; i <= resSet.getMetaData().getColumnCount(); i++) {

99
src/connector/jdbc/src/test/java/com/taosdata/jdbc/AsyncSubscribeTest.java Normal file
View File

@ -0,0 +1,99 @@
package com.taosdata.jdbc;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Properties;
import static org.junit.Assert.assertTrue;
public class AsyncSubscribeTest {
Connection connection = null;
Statement statement = null;
String dbName = "test";
String tName = "t0";
String host = "localhost";
String topic = "test";
long subscribId = 0;
@Before
public void createDatabase() throws SQLException {
try {
Class.forName("com.taosdata.jdbc.TSDBDriver");
} catch (ClassNotFoundException e) {
return;
}
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host);
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
connection = DriverManager.getConnection("jdbc:TAOS://" + host + ":0/" + "?user=root&password=taosdata"
, properties);
statement = connection.createStatement();
statement.executeUpdate("create database if not exists " + dbName);
statement.executeUpdate("create table if not exists " + dbName + "." + tName + " (ts timestamp, k int, v int)");
long ts = System.currentTimeMillis();
for (int i = 0; i < 2; i++) {
ts += i;
statement.executeUpdate("insert into \" + dbName + \".\" + tName + \" values (" + ts + ", " + (100 + i) + ", " + i + ")");
}
}
@Test
public void subscribe() throws Exception {
TSDBSubscribe subscribe = null;
try {
String rawSql = "select * from " + dbName + "." + tName + ";";
System.out.println(rawSql);
subscribe = ((TSDBConnection) connection).createSubscribe();
subscribId = subscribe.subscribe(topic, rawSql, false, 1000, new CallBack("first"));
assertTrue(subscribId > 0);
} catch (Exception e) {
e.printStackTrace();
}
Thread.sleep(2000);
subscribe.unsubscribe(subscribId, true);
}
private static class CallBack implements TSDBSubscribeCallBack {
private String name = "";
public CallBack(String name) {
this.name = name;
}
@Override
public void invoke(TSDBResultSet resultSet) {
try {
while (null != resultSet && resultSet.next()) {
System.out.print("callback_" + name + ": ");
for (int i = 1; i <= resultSet.getMetaData().getColumnCount(); i++) {
System.out.printf(i + ": " + resultSet.getString(i) + "\t");
}
System.out.println();
}
resultSet.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
@After
public void close() throws Exception {
statement.executeQuery("drop database test");
statement.close();
connection.close();
Thread.sleep(10);
}
}

62
src/connector/jdbc/src/test/java/com/taosdata/jdbc/ConnectionTest.java Normal file
View File

@ -0,0 +1,62 @@
package com.taosdata.jdbc;
import org.junit.Test;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Properties;
import static org.junit.Assert.assertTrue;
public class ConnectionTest {
static Connection connection = null;
static Statement statement = null;
static String dbName = "test";
static String stbName = "st";
static String host = "localhost";
@Test
public void testConnection() throws SQLException {
try {
Class.forName("com.taosdata.jdbc.TSDBDriver");
} catch (ClassNotFoundException e) {
return;
}
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host);
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
connection = DriverManager.getConnection("jdbc:TAOS://" + host + ":0/" + "?user=root&password=taosdata"
, properties);
assertTrue(null != connection);
statement = connection.createStatement();
assertTrue(null != statement);
// try reconnect
connection = DriverManager.getConnection("jdbc:TAOS://" + host + ":0/" + "?user=root&password=taosdata"
, properties);
try {
statement.execute("create database if not exists " + dbName);
} catch (SQLException e) {
assert false : "create database error: " + e.getMessage();
}
try {
if (!connection.isClosed()) {
if (!statement.isClosed()) {
statement.executeUpdate("drop database " + dbName);
statement.close();
}
connection.close();
Thread.sleep(10);
}
} catch (Exception e) {
assert false : "close connection error: " + e.getMessage();
}
}
}

245
src/connector/jdbc/src/test/java/com/taosdata/jdbc/DatabaseMetaDataTest.java Normal file
View File

@ -0,0 +1,245 @@
package com.taosdata.jdbc;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import java.sql.*;
import java.util.Properties;
public class DatabaseMetaDataTest {
static Connection connection = null;
static PreparedStatement statement = null;
static String dbName = "test";
static String tName = "t0";
static String host = "localhost";
@BeforeClass
public static void createConnection() throws SQLException {
try {
Class.forName("com.taosdata.jdbc.TSDBDriver");
} catch (ClassNotFoundException e) {
return;
}
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host);
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
connection = DriverManager.getConnection("jdbc:TAOS://" + host + ":0/" + "?user=root&password=taosdata",
properties);
String sql = "drop database if exists " + dbName;
statement = (TSDBPreparedStatement) connection.prepareStatement(sql);
statement.executeUpdate("create database if not exists " + dbName);
statement.executeUpdate("create table if not exists " + dbName + "." + tName + " (ts timestamp, k int, v int)");
}
@Test
public void testMetaDataTest() throws SQLException {
DatabaseMetaData databaseMetaData = connection.getMetaData();
ResultSet resultSet = databaseMetaData.getTables(dbName, "t*", "t*", new String[]{"t"});
while (resultSet.next()) {
for (int i = 1; i <= resultSet.getMetaData().getColumnCount(); i++) {
System.out.printf("%d: %s\n", i, resultSet.getString(i));
}
}
resultSet.close();
databaseMetaData.isWrapperFor(null);
databaseMetaData.allProceduresAreCallable();
databaseMetaData.allTablesAreSelectable();
databaseMetaData.getURL();
databaseMetaData.getUserName();
databaseMetaData.isReadOnly();
databaseMetaData.nullsAreSortedHigh();
databaseMetaData.nullsAreSortedLow();
databaseMetaData.nullsAreSortedAtStart();
databaseMetaData.nullsAreSortedAtEnd();
databaseMetaData.getDatabaseProductName();
databaseMetaData.getDatabaseProductVersion();
databaseMetaData.getDriverName();
databaseMetaData.getDriverVersion();
databaseMetaData.getDriverMajorVersion();
databaseMetaData.getDriverMinorVersion();
databaseMetaData.usesLocalFiles();
databaseMetaData.usesLocalFilePerTable();
databaseMetaData.supportsMixedCaseIdentifiers();
databaseMetaData.storesUpperCaseIdentifiers();
databaseMetaData.storesLowerCaseIdentifiers();
databaseMetaData.storesMixedCaseIdentifiers();
databaseMetaData.supportsMixedCaseQuotedIdentifiers();
databaseMetaData.storesUpperCaseQuotedIdentifiers();
databaseMetaData.storesLowerCaseQuotedIdentifiers();
databaseMetaData.storesMixedCaseQuotedIdentifiers();
databaseMetaData.getIdentifierQuoteString();
databaseMetaData.getSQLKeywords();
databaseMetaData.getNumericFunctions();
databaseMetaData.getStringFunctions();
databaseMetaData.getSystemFunctions();
databaseMetaData.getTimeDateFunctions();
databaseMetaData.getSearchStringEscape();
databaseMetaData.getExtraNameCharacters();
databaseMetaData.supportsAlterTableWithAddColumn();
databaseMetaData.supportsAlterTableWithDropColumn();
databaseMetaData.supportsColumnAliasing();
databaseMetaData.nullPlusNonNullIsNull();
databaseMetaData.supportsConvert();
databaseMetaData.supportsConvert(0, 0);
databaseMetaData.supportsTableCorrelationNames();
databaseMetaData.supportsDifferentTableCorrelationNames();
databaseMetaData.supportsExpressionsInOrderBy();
databaseMetaData.supportsOrderByUnrelated();
databaseMetaData.supportsGroupBy();
databaseMetaData.supportsGroupByUnrelated();
databaseMetaData.supportsGroupByBeyondSelect();
databaseMetaData.supportsLikeEscapeClause();
databaseMetaData.supportsMultipleResultSets();
databaseMetaData.supportsMultipleTransactions();
databaseMetaData.supportsNonNullableColumns();
databaseMetaData.supportsMinimumSQLGrammar();
databaseMetaData.supportsCoreSQLGrammar();
databaseMetaData.supportsExtendedSQLGrammar();
databaseMetaData.supportsANSI92EntryLevelSQL();
databaseMetaData.supportsANSI92IntermediateSQL();
databaseMetaData.supportsANSI92FullSQL();
databaseMetaData.supportsIntegrityEnhancementFacility();
databaseMetaData.supportsOuterJoins();
databaseMetaData.supportsFullOuterJoins();
databaseMetaData.supportsLimitedOuterJoins();
databaseMetaData.getSchemaTerm();
databaseMetaData.getProcedureTerm();
databaseMetaData.getCatalogTerm();
databaseMetaData.isCatalogAtStart();
databaseMetaData.getCatalogSeparator();
databaseMetaData.supportsSchemasInDataManipulation();
databaseMetaData.supportsSchemasInProcedureCalls();
databaseMetaData.supportsSchemasInTableDefinitions();
databaseMetaData.supportsSchemasInIndexDefinitions();
databaseMetaData.supportsSchemasInPrivilegeDefinitions();
databaseMetaData.supportsCatalogsInDataManipulation();
databaseMetaData.supportsCatalogsInProcedureCalls();
databaseMetaData.supportsCatalogsInTableDefinitions();
databaseMetaData.supportsCatalogsInIndexDefinitions();
databaseMetaData.supportsCatalogsInPrivilegeDefinitions();
databaseMetaData.supportsPositionedDelete();
databaseMetaData.supportsPositionedUpdate();
databaseMetaData.supportsSelectForUpdate();
databaseMetaData.supportsStoredProcedures();
databaseMetaData.supportsSubqueriesInComparisons();
databaseMetaData.supportsSubqueriesInExists();
databaseMetaData.supportsSubqueriesInIns();
databaseMetaData.supportsSubqueriesInQuantifieds();
databaseMetaData.supportsCorrelatedSubqueries();
databaseMetaData.supportsUnion();
databaseMetaData.supportsUnionAll();
databaseMetaData.supportsOpenCursorsAcrossCommit();
databaseMetaData.supportsOpenCursorsAcrossRollback();
databaseMetaData.supportsOpenStatementsAcrossCommit();
databaseMetaData.supportsOpenStatementsAcrossRollback();
databaseMetaData.getMaxBinaryLiteralLength();
databaseMetaData.getMaxCharLiteralLength();
databaseMetaData.getMaxColumnNameLength();
databaseMetaData.getMaxColumnsInGroupBy();
databaseMetaData.getMaxColumnsInIndex();
databaseMetaData.getMaxColumnsInOrderBy();
databaseMetaData.getMaxColumnsInSelect();
databaseMetaData.getMaxColumnsInTable();
databaseMetaData.getMaxConnections();
databaseMetaData.getMaxCursorNameLength();
databaseMetaData.getMaxIndexLength();
databaseMetaData.getMaxSchemaNameLength();
databaseMetaData.getMaxProcedureNameLength();
databaseMetaData.getMaxCatalogNameLength();
databaseMetaData.getMaxRowSize();
databaseMetaData.doesMaxRowSizeIncludeBlobs();
databaseMetaData.getMaxStatementLength();
databaseMetaData.getMaxStatements();
databaseMetaData.getMaxTableNameLength();
databaseMetaData.getMaxTablesInSelect();
databaseMetaData.getMaxUserNameLength();
databaseMetaData.getDefaultTransactionIsolation();
databaseMetaData.supportsTransactions();
databaseMetaData.supportsTransactionIsolationLevel(0);
databaseMetaData.supportsDataDefinitionAndDataManipulationTransactions();
databaseMetaData.supportsDataManipulationTransactionsOnly();
databaseMetaData.dataDefinitionCausesTransactionCommit();
databaseMetaData.dataDefinitionIgnoredInTransactions();
try {
databaseMetaData.getProcedures("", "", "");
} catch (Exception e) {
}
try {
databaseMetaData.getProcedureColumns("", "", "", "");
} catch (Exception e) {
}
try {
databaseMetaData.getTables("", "", "", new String[]{""});
} catch (Exception e) {
}
databaseMetaData.getSchemas();
databaseMetaData.getCatalogs();
// databaseMetaData.getTableTypes();
databaseMetaData.getColumns("", "", "", "");
databaseMetaData.getColumnPrivileges("", "", "", "");
databaseMetaData.getTablePrivileges("", "", "");
databaseMetaData.getBestRowIdentifier("", "", "", 0, false);
databaseMetaData.getVersionColumns("", "", "");
databaseMetaData.getPrimaryKeys("", "", "");
databaseMetaData.getImportedKeys("", "", "");
databaseMetaData.getExportedKeys("", "", "");
databaseMetaData.getCrossReference("", "", "", "", "", "");
databaseMetaData.getTypeInfo();
databaseMetaData.getIndexInfo("", "", "", false, false);
databaseMetaData.supportsResultSetType(0);
databaseMetaData.supportsResultSetConcurrency(0, 0);
databaseMetaData.ownUpdatesAreVisible(0);
databaseMetaData.ownDeletesAreVisible(0);
databaseMetaData.ownInsertsAreVisible(0);
databaseMetaData.othersUpdatesAreVisible(0);
databaseMetaData.othersDeletesAreVisible(0);
databaseMetaData.othersInsertsAreVisible(0);
databaseMetaData.updatesAreDetected(0);
databaseMetaData.deletesAreDetected(0);
databaseMetaData.insertsAreDetected(0);
databaseMetaData.supportsBatchUpdates();
databaseMetaData.getUDTs("", "", "", new int[]{0});
databaseMetaData.getConnection();
databaseMetaData.supportsSavepoints();
databaseMetaData.supportsNamedParameters();
databaseMetaData.supportsMultipleOpenResults();
databaseMetaData.supportsGetGeneratedKeys();
databaseMetaData.getSuperTypes("", "", "");
databaseMetaData.getSuperTables("", "", "");
databaseMetaData.getAttributes("", "", "", "");
databaseMetaData.supportsResultSetHoldability(0);
databaseMetaData.getResultSetHoldability();
databaseMetaData.getDatabaseMajorVersion();
databaseMetaData.getDatabaseMinorVersion();
databaseMetaData.getJDBCMajorVersion();
databaseMetaData.getJDBCMinorVersion();
databaseMetaData.getSQLStateType();
databaseMetaData.locatorsUpdateCopy();
databaseMetaData.supportsStatementPooling();
databaseMetaData.getRowIdLifetime();
databaseMetaData.getSchemas("", "");
databaseMetaData.supportsStoredFunctionsUsingCallSyntax();
databaseMetaData.autoCommitFailureClosesAllResultSets();
databaseMetaData.getClientInfoProperties();
databaseMetaData.getFunctions("", "", "");
databaseMetaData.getFunctionColumns("", "", "", "");
databaseMetaData.getPseudoColumns("", "", "", "");
databaseMetaData.generatedKeyAlwaysReturned();
}
@AfterClass
public static void close() throws Exception {
statement.executeUpdate("drop database " + dbName);
statement.close();
connection.close();
Thread.sleep(10);
}
}

93
src/connector/jdbc/src/test/java/com/taosdata/jdbc/ImportTest.java Normal file
View File

@ -0,0 +1,93 @@
package com.taosdata.jdbc;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import java.sql.*;
import java.util.Properties;
import static org.junit.Assert.assertEquals;
public class ImportTest {
Connection connection = null;
Statement statement = null;
String dbName = "test";
String tName = "t0";
String host = "localhost";
@Before
public void createDatabase() throws SQLException {
try {
Class.forName("com.taosdata.jdbc.TSDBDriver");
} catch (ClassNotFoundException e) {
return;
}
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host);
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
connection = DriverManager.getConnection("jdbc:TAOS://" + host + ":0/" + "?user=root&password=taosdata"
, properties);
statement = connection.createStatement();
statement.executeUpdate("drop database if exists " + dbName);
statement.executeUpdate("create database if not exists " + dbName);
statement.executeUpdate("create table if not exists " + dbName + "." + tName + " (ts timestamp, k int, v int)");
}
@Test
public void insertData() throws Exception {
long ts = 1496732686000l;
for (int i = 0; i < 50; i++) {
ts++;
int row = statement.executeUpdate("insert into " + dbName + "." + tName + " values (" + ts + ", " + (100 + i) + ", " + i + ")");
System.out.println("insert into " + dbName + "." + tName + " values (" + ts + ", " + (100 + i) + ", " + i + ")\t" + row);
assertEquals(1, row);
}
}
@Test
public void selectData() throws Exception {
insertData();
String sql = "select * from test.t0";
ResultSet resSet = statement.executeQuery(sql);
while (resSet.next()) {
for (int i = 1; i <= resSet.getMetaData().getColumnCount(); i++) {
System.out.printf(i + ": " + resSet.getString(i) + "\t");
}
}
resSet.close();
}
@Test
public void importData() throws Exception {
// 避免时间重复
long ts = 1496732686000l;
StringBuilder sqlBuilder = new StringBuilder("insert into ").append(dbName).append(".").append(tName).append(" values ");
for (int i = 0; i < 50; i++) {
int a = i / 5;
long t = ts + a;
sqlBuilder.append("(").append(t).append(",").append((100 + i)).append(",").append(i).append(") ");
}
System.out.println(sqlBuilder.toString());
int rows = statement.executeUpdate(sqlBuilder.toString());
System.out.println(rows);
assertEquals(10, rows);
}
@After
public void close() throws Exception {
statement.executeUpdate("drop database " + dbName);
statement.close();
connection.close();
Thread.sleep(10);
}
}

201
src/connector/jdbc/src/test/java/com/taosdata/jdbc/PreparedStatementTest.java Normal file
View File

@ -0,0 +1,201 @@
package com.taosdata.jdbc;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.FixMethodOrder;
import org.junit.Test;
import org.junit.runners.MethodSorters;
import java.sql.*;
import java.util.Properties;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
@FixMethodOrder(MethodSorters.DEFAULT)
public class PreparedStatementTest {
static Connection connection = null;
static PreparedStatement statement = null;
static String dbName = "test";
static String tName = "t0";
static String host = "localhost";
static ResultSet resSet = null;
@BeforeClass
public static void createConnection() throws SQLException {
try {
Class.forName("com.taosdata.jdbc.TSDBDriver");
} catch (ClassNotFoundException e) {
return;
}
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host);
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
connection = DriverManager.getConnection("jdbc:TAOS://" + host + ":0/" + "?user=root&password=taosdata",
properties);
String sql = "drop database if exists " + dbName;
statement = (TSDBPreparedStatement) connection.prepareStatement(sql);
}
@Test
public void createTableAndQuery() throws SQLException {
long ts = System.currentTimeMillis();
statement.executeUpdate("create database if not exists " + dbName);
statement.executeUpdate("create table if not exists " + dbName + "." + tName + "(ts timestamp, k1 int)");
statement.executeUpdate("insert into " + dbName + "." + tName + " values (" + ts + ", 1)");
PreparedStatement selectStatement = connection.prepareStatement("select * from " + dbName + "." + tName);
ResultSet resultSet = selectStatement.executeQuery();
assertTrue(null != resultSet);
boolean isClosed = statement.isClosed();
assertEquals(false, isClosed);
}
@Test
public void testPreparedStatement() throws SQLException {
long ts = System.currentTimeMillis() + 20000;
PreparedStatement saveStatement = connection
.prepareStatement("insert into " + dbName + "." + tName + " values (" + ts + ", 1)");
int affectedRows = saveStatement.executeUpdate();
assertTrue(1 == affectedRows);
}
@Test
public void testSavedPreparedStatement() throws SQLException {
long ts = System.currentTimeMillis();
TSDBPreparedStatement saveStatement = (TSDBPreparedStatement) connection
.prepareStatement("insert into " + dbName + "." + tName + " values (?, ?)");
saveStatement.setObject(1, ts + 10000);
saveStatement.setObject(2, 3);
int rows = saveStatement.executeUpdate();
assertEquals(1, rows);
}
@Test
public void testUnsupport() {
// if(null == resSet) {
// return;
// }
TSDBPreparedStatement tsdbStatement = (TSDBPreparedStatement) statement;
try {
tsdbStatement.unwrap(null);
} catch (SQLException e) {
}
try {
tsdbStatement.isWrapperFor(null);
} catch (SQLException e) {
}
try {
tsdbStatement.getMaxFieldSize();
} catch (SQLException e) {
}
try {
tsdbStatement.setMaxFieldSize(0);
} catch (SQLException e) {
}
try {
tsdbStatement.setEscapeProcessing(true);
} catch (SQLException e) {
}
try {
tsdbStatement.cancel();
} catch (SQLException e) {
}
try {
tsdbStatement.getWarnings();
} catch (SQLException e) {
}
try {
tsdbStatement.clearWarnings();
} catch (SQLException e) {
}
try {
tsdbStatement.setCursorName(null);
} catch (SQLException e) {
}
try {
tsdbStatement.getMoreResults();
} catch (SQLException e) {
}
try {
tsdbStatement.setFetchDirection(0);
} catch (SQLException e) {
}
try {
tsdbStatement.getFetchDirection();
} catch (SQLException e) {
}
try {
tsdbStatement.getResultSetConcurrency();
} catch (SQLException e) {
}
try {
tsdbStatement.getResultSetType();
} catch (SQLException e) {
}
try {
tsdbStatement.getConnection();
} catch (SQLException e) {
}
try {
tsdbStatement.getMoreResults();
} catch (SQLException e) {
}
try {
tsdbStatement.getGeneratedKeys();
} catch (SQLException e) {
}
try {
tsdbStatement.executeUpdate(null, 0);
} catch (SQLException e) {
}
try {
tsdbStatement.executeUpdate(null, new int[]{0});
} catch (SQLException e) {
}
try {
tsdbStatement.executeUpdate(null, new String[]{"str1", "str2"});
} catch (SQLException e) {
}
try {
tsdbStatement.getResultSetHoldability();
} catch (SQLException e) {
}
try {
tsdbStatement.setPoolable(true);
} catch (SQLException e) {
}
try {
tsdbStatement.isPoolable();
} catch (SQLException e) {
}
try {
tsdbStatement.closeOnCompletion();
} catch (SQLException e) {
}
try {
tsdbStatement.isCloseOnCompletion();
} catch (SQLException e) {
}
}
@AfterClass
public static void close() throws Exception {
statement.executeUpdate("drop database " + dbName);
statement.close();
connection.close();
Thread.sleep(10);
}
}

829
src/connector/jdbc/src/test/java/com/taosdata/jdbc/ResultSetTest.java Normal file
View File

@ -0,0 +1,829 @@
package com.taosdata.jdbc;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import javax.sql.rowset.serial.SerialBlob;
import javax.sql.rowset.serial.SerialClob;
import java.sql.*;
import java.util.HashMap;
import java.util.Properties;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
public class ResultSetTest {
static Connection connection = null;
static Statement statement = null;
static String dbName = "test";
static String tName = "t0";
static String host = "localhost";
static ResultSet resSet = null;
@BeforeClass
public static void createDatabaseAndTable() throws SQLException {
try {
Class.forName("com.taosdata.jdbc.TSDBDriver");
} catch (ClassNotFoundException e) {
return;
}
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host);
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
connection = DriverManager.getConnection("jdbc:TAOS://" + host + ":0/" + "?user=root&password=taosdata"
, properties);
statement = connection.createStatement();
statement.executeUpdate("drop database if exists " + dbName);
statement.executeUpdate("create database if not exists " + dbName);
statement.executeUpdate("create table if not exists " + dbName + "." + tName +
" (ts timestamp, k1 int, k2 bigint, k3 float, k4 double, k5 binary(30), k6 smallint, k7 bool, k8 nchar(20))");
statement.executeQuery("use " + dbName);
}
@Test
public void testResultSet() {
String sql = null;
long ts = 1496732686000l;
int v1 = 2147483600;
long v2 = ts + 1000;
float v3 = 3.1415926f;
double v4 = 3.1415926535897;
String v5 = "涛思数据，强~！";
short v6 = 12;
boolean v7 = false;
String v8 = "TDengine is powerful";
sql = "insert into " + dbName + "." + tName + " values (" + ts + "," + v1 + "," + v2 + "," + v3 + "," + v4
+ ",\"" + v5 + "\"," + v6 + "," + v7 + ",\"" + v8 + "\")";
try {
statement.executeUpdate(sql);
assertEquals(1, statement.getUpdateCount());
} catch (SQLException e) {
assert false : "insert error " + e.getMessage();
}
try {
statement.executeQuery("select * from " + dbName + "." + tName);
resSet = statement.getResultSet();
System.out.println(((TSDBResultSet) resSet).getRowData());
while (resSet.next()) {
assertEquals(ts, resSet.getLong(1));
assertEquals(ts, resSet.getLong("ts"));
System.out.println(resSet.getTimestamp(1));
assertEquals(v1, resSet.getInt(2));
assertEquals(v1, resSet.getInt("k1"));
assertEquals(v2, resSet.getLong(3));
assertEquals(v2, resSet.getLong("k2"));
assertEquals(v3, resSet.getFloat(4), 7);
assertEquals(v3, resSet.getFloat("k3"), 7);
assertEquals(v4, resSet.getDouble(5), 13);
assertEquals(v4, resSet.getDouble("k4"), 13);
assertEquals(v5, resSet.getString(6));
assertEquals(v5, resSet.getString("k5"));
assertEquals(v6, resSet.getShort(7));
assertEquals(v6, resSet.getShort("k6"));
assertEquals(v7, resSet.getBoolean(8));
assertEquals(v7, resSet.getBoolean("k7"));
assertEquals(v8, resSet.getString(9));
assertEquals(v8, resSet.getString("k8"));
resSet.getBytes(9);
resSet.getObject(6);
resSet.getObject("k8");
}
if (!resSet.isClosed()) {
resSet.close();
}
} catch (SQLException e) {
assert false : "insert error " + e.getMessage();
}
}
@Test
public void testUnsupport() throws SQLException {
statement.executeQuery("show databases");
resSet = statement.getResultSet();
try {
resSet.unwrap(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.isWrapperFor(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getAsciiStream(0);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getUnicodeStream(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getBinaryStream(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getAsciiStream("");
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getUnicodeStream(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getBinaryStream(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getWarnings();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.clearWarnings();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getCursorName();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getCharacterStream(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getCharacterStream(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.isBeforeFirst();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.isAfterLast();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.isFirst();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.isLast();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.beforeFirst();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.afterLast();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.first();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.last();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getRow();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.absolute(1);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.relative(1);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.previous();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.setFetchDirection(0);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getFetchDirection();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.setFetchSize(0);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getFetchSize();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getConcurrency();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.rowUpdated();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.rowInserted();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.rowDeleted();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateNull(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateBoolean(0, true);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateByte(0, (byte) 2);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateShort(0, (short) 1);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateInt(0, 0);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateLong(0, 0l);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateFloat(0, 3.14f);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateDouble(0, 3.1415);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateBigDecimal(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateString(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateBytes(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateDate(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateTime(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateTimestamp(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateAsciiStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateBinaryStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateCharacterStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateObject(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateObject(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateNull(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateBoolean("", false);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateByte("", (byte) 1);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateShort("", (short) 1);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateInt("", 0);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateLong("", 0l);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateFloat("", 3.14f);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateDouble("", 3.1415);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateBigDecimal(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateString(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateBytes(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateDate(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateTime(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateTimestamp(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateAsciiStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateBinaryStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateCharacterStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateObject(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateObject(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.insertRow();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateRow();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.deleteRow();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.refreshRow();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.cancelRowUpdates();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.moveToInsertRow();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.moveToCurrentRow();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getStatement();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getObject(0, new HashMap<>());
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getRef(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getBlob(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getClob(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getArray(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getObject("", new HashMap<>());
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getRef(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getBlob(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getClob(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getArray(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getDate(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getDate(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getTime(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getTime(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getTimestamp(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getTimestamp(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getURL(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getURL(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateRef(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateRef(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateBlob(0, new SerialBlob("".getBytes("UTF8")));
} catch (Exception e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateBlob("", new SerialBlob("".getBytes("UTF8")));
} catch (Exception e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateClob("", new SerialClob("".toCharArray()));
} catch (Exception e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateClob(0, new SerialClob("".toCharArray()));
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateArray(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateArray(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getRowId(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getRowId(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateRowId(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateRowId(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getHoldability();
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateNString(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateNString(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getNClob(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getNClob(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getSQLXML(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getSQLXML(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateSQLXML(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateSQLXML(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getNCharacterStream(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.getNCharacterStream(null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateNCharacterStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateNCharacterStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateAsciiStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateBinaryStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateCharacterStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateAsciiStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateBinaryStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateCharacterStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateNCharacterStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateNCharacterStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateAsciiStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateBinaryStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateCharacterStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateAsciiStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateBinaryStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
try {
resSet.updateCharacterStream(null, null);
} catch (SQLException e) {
assertTrue(e.getMessage().contains("this operation is NOT supported currently!"));
}
}
@Test
public void testBatch() throws SQLException {
String[] sqls = new String[]{"insert into test.t0 values (1496732686001,2147483600,1496732687000,3.1415925,3.1415926\n" +
"535897,\"涛思数据，强~！\",12,12,\"TDengine is powerful\")", "insert into test.t0 values (1496732686002,2147483600,1496732687000,3.1415925,3.1415926\n" +
"535897,\"涛思数据，强~！\",12,12,\"TDengine is powerful\")"};
for (String sql : sqls) {
statement.addBatch(sql);
}
int[] res = statement.executeBatch();
assertEquals(res.length, 2);
statement.clearBatch();
}
@AfterClass
public static void close() throws Exception {
statement.executeUpdate("drop database " + dbName);
statement.close();
connection.close();
Thread.sleep(10);
}
}

72
src/connector/jdbc/src/test/java/com/taosdata/jdbc/SelectTest.java Normal file
View File

@ -0,0 +1,72 @@
package com.taosdata.jdbc;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import java.sql.*;
import java.util.Properties;
import static org.junit.Assert.assertEquals;
public class SelectTest {
Connection connection = null;
Statement statement = null;
String dbName = "test";
String tName = "t0";
String host = "localhost";
@Before
public void createDatabaseAndTable() throws SQLException {
try {
Class.forName("com.taosdata.jdbc.TSDBDriver");
} catch (ClassNotFoundException e) {
return;
}
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host);
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
connection = DriverManager.getConnection("jdbc:TAOS://" + host + ":0/" + "?user=root&password=taosdata"
, properties);
statement = connection.createStatement();
statement.executeUpdate("drop database if exists " + dbName);
statement.executeUpdate("create database if not exists " + dbName);
statement.executeUpdate("create table if not exists " + dbName + "." + tName + " (ts timestamp, k int, v int)");
}
@Test
public void selectData() throws SQLException {
long ts = 1496732686000l;
for (int i = 0; i < 50; i++) {
ts++;
int row = statement.executeUpdate("insert into " + dbName + "." + tName + " values (" + ts + ", " + (100 + i) + ", " + i + ")");
System.out.println("insert into " + dbName + "." + tName + " values (" + ts + ", " + (100 + i) + ", " + i + ")\t" + row);
assertEquals(1, row);
}
String sql = "select * from " + dbName + "." + tName;
ResultSet resSet = statement.executeQuery(sql);
int num = 0;
while (resSet.next()) {
num++;
}
resSet.close();
assertEquals(num, 50);
}
@After
public void close() throws Exception {
statement.executeUpdate("drop database " + dbName);
statement.close();
connection.close();
Thread.sleep(10);
}
}

123
src/connector/jdbc/src/test/java/com/taosdata/jdbc/StableTest.java Normal file
View File

@ -0,0 +1,123 @@
package com.taosdata.jdbc;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.FixMethodOrder;
import org.junit.Test;
import org.junit.runners.MethodSorters;
import java.sql.*;
import java.util.Properties;
import static org.junit.Assert.assertEquals;
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class StableTest {
static Connection connection = null;
static Statement statement = null;
static String dbName = "test";
static String stbName = "st";
static String host = "localhost";
@BeforeClass
public static void createDatabase() throws SQLException {
try {
Class.forName("com.taosdata.jdbc.TSDBDriver");
} catch (ClassNotFoundException e) {
return;
}
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host);
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
connection = DriverManager.getConnection("jdbc:TAOS://" + host + ":0/" + "?user=root&password=taosdata"
, properties);
statement = connection.createStatement();
statement.executeUpdate("create database if not exists " + dbName);
statement.executeQuery("use " + dbName);
}
// @Test
public void createStable() {
String sql = "create table " + stbName + " (ts timestamp, v1 int, v2 int) tags (tg nchar(20)) ";
try {
statement.executeUpdate(sql);
} catch (SQLException e) {
assert false : "error create stable" + e.getMessage();
}
}
// @Test
public void createTable() {
String sql = "create table t1 using " + stbName + " tags (\"beijing\")";
try {
statement.executeUpdate(sql);
} catch (SQLException e) {
assert false : "error create table" + e.getMessage();
}
}
@Test
public void describeSTable() {
createStable();
String sql = "describe " + stbName;
int num = 0;
System.out.println("describe stable");
try {
ResultSet res = statement.executeQuery(sql);
while (res.next()) {
for (int i = 1; i <= res.getMetaData().getColumnCount(); i++) {
System.out.printf("%d: %s\n", i, res.getString(i));
}
num++;
}
res.close();
assertEquals(4, num);
} catch (SQLException e) {
assert false : "error describe stable" + e.getMessage();
}
}
@Test
public void describeTable() {
createTable();
String sql = "describe t1";
int num = 0;
System.out.println("describe table");
try {
ResultSet res = statement.executeQuery(sql);
while (res.next()) {
for (int i = 1; i <= res.getMetaData().getColumnCount(); i++) {
System.out.printf("%d: %s\n", i, res.getString(i));
}
num++;
}
res.close();
assertEquals(4, num);
} catch (SQLException e) {
assert false : "error describe stable" + e.getMessage();
}
}
// @Test
public void validCreateSql() {
String sql = "create table t2 using " + stbName + " tags (\"beijing\")";
boolean valid = ((TSDBConnection) connection).getConnection().validateCreateTableSql(sql);
assertEquals(true, valid);
}
@AfterClass
public static void close() throws Exception {
if (!statement.isClosed()) {
statement.executeUpdate("drop database " + dbName);
statement.close();
connection.close();
Thread.sleep(10);
}
}
}

174
src/connector/jdbc/src/test/java/com/taosdata/jdbc/StatementTest.java Normal file
View File

@ -0,0 +1,174 @@
package com.taosdata.jdbc;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import java.sql.*;
import java.util.Properties;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
public class StatementTest {
static Connection connection = null;
static Statement statement = null;
static String dbName = "test";
static String tName = "t0";
static String host = "localhost";
static ResultSet resSet = null;
@BeforeClass
public static void createConnection() throws SQLException {
try {
Class.forName("com.taosdata.jdbc.TSDBDriver");
} catch (ClassNotFoundException e) {
return;
}
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host);
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
connection = DriverManager.getConnection("jdbc:TAOS://" + host + ":0/" + "?user=root&password=taosdata"
, properties);
statement = connection.createStatement();
statement.executeUpdate("drop database if exists " + dbName);
}
@Test
public void createTableAndQuery() throws SQLException {
long ts = System.currentTimeMillis();
statement.executeUpdate("create database if not exists " + dbName);
statement.executeUpdate("create table if not exists " + dbName + "." + tName + "(ts timestamp, k1 int)");
statement.executeUpdate("insert into " + dbName + "." + tName + " values (" + ts + ", 1)");
statement.executeQuery("select * from " + dbName + "." + tName);
ResultSet resultSet = statement.getResultSet();
assertTrue(null != resultSet);
boolean isClosed = statement.isClosed();
assertEquals(false, isClosed);
}
@Test
public void testUnsupport() {
// if(null == resSet) {
// return;
// }
TSDBStatement tsdbStatement = (TSDBStatement) statement;
try {
tsdbStatement.unwrap(null);
} catch (SQLException e) {
}
try {
tsdbStatement.isWrapperFor(null);
} catch (SQLException e) {
}
try {
tsdbStatement.getMaxFieldSize();
} catch (SQLException e) {
}
try {
tsdbStatement.setMaxFieldSize(0);
} catch (SQLException e) {
}
try {
tsdbStatement.setEscapeProcessing(true);
} catch (SQLException e) {
}
try {
tsdbStatement.cancel();
} catch (SQLException e) {
}
try {
tsdbStatement.getWarnings();
} catch (SQLException e) {
}
try {
tsdbStatement.clearWarnings();
} catch (SQLException e) {
}
try {
tsdbStatement.setCursorName(null);
} catch (SQLException e) {
}
try {
tsdbStatement.getMoreResults();
} catch (SQLException e) {
}
try {
tsdbStatement.setFetchDirection(0);
} catch (SQLException e) {
}
try {
tsdbStatement.getFetchDirection();
} catch (SQLException e) {
}
try {
tsdbStatement.getResultSetConcurrency();
} catch (SQLException e) {
}
try {
tsdbStatement.getResultSetType();
} catch (SQLException e) {
}
try {
tsdbStatement.getConnection();
} catch (SQLException e) {
}
try {
tsdbStatement.getMoreResults();
} catch (SQLException e) {
}
try {
tsdbStatement.getGeneratedKeys();
} catch (SQLException e) {
}
try {
tsdbStatement.executeUpdate(null, 0);
} catch (SQLException e) {
}
try {
tsdbStatement.executeUpdate(null, new int[]{0});
} catch (SQLException e) {
}
try {
tsdbStatement.executeUpdate(null, new String[]{"str1", "str2"});
} catch (SQLException e) {
}
try {
tsdbStatement.getResultSetHoldability();
} catch (SQLException e) {
}
try {
tsdbStatement.setPoolable(true);
} catch (SQLException e) {
}
try {
tsdbStatement.isPoolable();
} catch (SQLException e) {
}
try {
tsdbStatement.closeOnCompletion();
} catch (SQLException e) {
}
try {
tsdbStatement.isCloseOnCompletion();
} catch (SQLException e) {
}
}
@AfterClass
public static void close() throws Exception {
if (!statement.isClosed()) {
statement.executeUpdate("drop database " + dbName);
statement.close();
connection.close();
Thread.sleep(10);
}
}
}

94
src/connector/jdbc/src/test/java/com/taosdata/jdbc/SubscribeTest.java Normal file
View File

@ -0,0 +1,94 @@
package com.taosdata.jdbc;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Properties;
import static org.junit.Assert.assertTrue;
public class SubscribeTest {
Connection connection = null;
Statement statement = null;
String dbName = "test";
String tName = "t0";
String host = "localhost";
String topic = "test";
@Before
public void createDatabase() throws SQLException {
try {
Class.forName("com.taosdata.jdbc.TSDBDriver");
} catch (ClassNotFoundException e) {
return;
}
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host);
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
connection = DriverManager.getConnection("jdbc:TAOS://" + host + ":0/" + "?user=root&password=taosdata"
, properties);
statement = connection.createStatement();
statement.executeUpdate("create database if not exists " + dbName);
statement.executeUpdate("create table if not exists " + dbName + "." + tName + " (ts timestamp, k int, v int)");
long ts = System.currentTimeMillis();
for (int i = 0; i < 2; i++) {
ts += i;
statement.executeUpdate("insert into \" + dbName + \".\" + tName + \" values (" + ts + ", " + (100 + i) + ", " + i + ")");
}
}
@Test
public void subscribe() throws Exception {
TSDBSubscribe subscribe = null;
long subscribId = 0;
try {
String rawSql = "select * from " + dbName + "." + tName + ";";
System.out.println(rawSql);
subscribe = ((TSDBConnection) connection).createSubscribe();
subscribId = subscribe.subscribe(topic, rawSql, false, 1000);
assertTrue(subscribId > 0);
int a = 0;
while (true) {
Thread.sleep(900);
TSDBResultSet resSet = subscribe.consume(subscribId);
while (resSet.next()) {
for (int i = 1; i <= resSet.getMetaData().getColumnCount(); i++) {
System.out.printf(i + ": " + resSet.getString(i) + "\t");
}
System.out.println("\n======" + a + "==========");
}
resSet.close();
a++;
if (a >= 2) {
break;
}
}
} catch (Exception e) {
e.printStackTrace();
} finally {
if (null != subscribe && 0 != subscribId) {
subscribe.unsubscribe(subscribId, true);
}
}
}
@After
public void close() throws Exception {
statement.executeQuery("drop database " + dbName);
statement.close();
connection.close();
Thread.sleep(10);
}
}

21
src/cq/src/cqMain.c
View File

@ -256,11 +256,30 @@ static void cqProcessStreamRes(void *param, TAOS_RES *tres, TAOS_ROW row) {
SDataRow trow = (SDataRow)pBlk->data;
tdInitDataRow(trow, pSchema);
union {
char buf[sizeof(int64_t)];
tstr str;
} nullVal;
for (int32_t i = 0; i < pSchema->numOfCols; i++) {
STColumn *c = pSchema->columns + i;
char* val = (char*)row[i];
if (IS_VAR_DATA_TYPE(c->type)) {
val -= sizeof(VarDataLenT);
if (val == NULL) {
val = nullVal.buf;
if (c->type == TSDB_DATA_TYPE_BINARY) {
setNull(nullVal.str.data, TSDB_DATA_TYPE_BINARY, 1);
nullVal.str.len = 1;
} else {
setNull(nullVal.str.data, TSDB_DATA_TYPE_NCHAR, 4);
nullVal.str.len = 4;
}
} else {
val -= sizeof(VarDataLenT);
}
} else if (val == NULL) {
val = nullVal.buf;
setNull(val, c->type, c->bytes);
}
tdAppendColVal(trow, val, c->type, c->bytes, c->offset);
}

4
src/cq/test/cqtest.c
View File

@ -48,8 +48,8 @@ int main(int argc, char *argv[]) {
taosInitLog("cq.log", 100000, 10);
SCqCfg cqCfg;
strcpy(cqCfg.user, "root");
strcpy(cqCfg.pass, "taosdata");
strcpy(cqCfg.user, TSDB_DEFAULT_USER);
strcpy(cqCfg.pass, TSDB_DEFAULT_PASS);
cqCfg.vgId = 2;
cqCfg.cqWrite = writeToQueue;

23
src/dnode/src/dnodeMWrite.c
View File

@ -118,6 +118,8 @@ void dnodeDispatchToMnodeWriteQueue(SRpcMsg *pMsg) {
SMnodeMsg *pWrite = (SMnodeMsg *)taosAllocateQitem(sizeof(SMnodeMsg));
mnodeCreateMsg(pWrite, pMsg);
dTrace("app:%p:%p, msg:%s is put into mwrite queue", pWrite->rpcMsg.ahandle, pWrite, taosMsg[pWrite->rpcMsg.msgType]);
taosWriteQitem(tsMWriteQueue, TAOS_QTYPE_RPC, pWrite);
}
@ -128,6 +130,7 @@ static void dnodeFreeMnodeWriteMsg(SMnodeMsg *pWrite) {
void dnodeSendRpcMnodeWriteRsp(void *pRaw, int32_t code) {
SMnodeMsg *pWrite = pRaw;
if (pWrite == NULL) return;
if (code == TSDB_CODE_MND_ACTION_IN_PROGRESS) return;
if (code == TSDB_CODE_MND_ACTION_NEED_REPROCESSED) {
dnodeReprocessMnodeWriteMsg(pWrite);
@ -146,19 +149,21 @@ void dnodeSendRpcMnodeWriteRsp(void *pRaw, int32_t code) {
}
static void *dnodeProcessMnodeWriteQueue(void *param) {
SMnodeMsg *pWriteMsg;
SMnodeMsg *pWrite;
int32_t type;
void * unUsed;
while (1) {
if (taosReadQitemFromQset(tsMWriteQset, &type, (void **)&pWriteMsg, &unUsed) == 0) {
if (taosReadQitemFromQset(tsMWriteQset, &type, (void **)&pWrite, &unUsed) == 0) {
dTrace("dnodeProcessMnodeWriteQueue: got no message from qset, exiting...");
break;
}
dTrace("%p, msg:%s will be processed in mwrite queue", pWriteMsg->rpcMsg.ahandle, taosMsg[pWriteMsg->rpcMsg.msgType]);
int32_t code = mnodeProcessWrite(pWriteMsg);
dnodeSendRpcMnodeWriteRsp(pWriteMsg, code);
dTrace("app:%p:%p, msg:%s will be processed in mwrite queue", pWrite->rpcMsg.ahandle, pWrite,
taosMsg[pWrite->rpcMsg.msgType]);
int32_t code = mnodeProcessWrite(pWrite);
dnodeSendRpcMnodeWriteRsp(pWrite, code);
}
return NULL;
@ -168,9 +173,15 @@ void dnodeReprocessMnodeWriteMsg(void *pMsg) {
SMnodeMsg *pWrite = pMsg;
if (!mnodeIsRunning() || tsMWriteQueue == NULL) {
dTrace("app:%p:%p, msg:%s is redirected for mnode not running, retry times:%d", pWrite->rpcMsg.ahandle, pWrite,
taosMsg[pWrite->rpcMsg.msgType], pWrite->retry);
dnodeSendRedirectMsg(pMsg, true);
dnodeFreeMnodeWriteMsg(pWrite);
} else {
} else {
dTrace("app:%p:%p, msg:%s is reput into mwrite queue, retry times:%d", pWrite->rpcMsg.ahandle, pWrite,
taosMsg[pWrite->rpcMsg.msgType], pWrite->retry);
taosWriteQitem(tsMWriteQueue, TAOS_QTYPE_RPC, pWrite);
}
}

16
src/dnode/src/dnodeMgmt.c
View File

@ -407,11 +407,7 @@ static void dnodeProcessStatusRsp(SRpcMsg *pMsg) {
pMnodeInfo->nodeId = htonl(pMnodeInfo->nodeId);
}
SDMVgroupAccess *pVgAcccess = pStatusRsp->vgAccess;
for (int32_t i = 0; i < pCfg->numOfVnodes; ++i) {
pVgAcccess[i].vgId = htonl(pVgAcccess[i].vgId);
}
vnodeSetAccess(pStatusRsp->vgAccess, pCfg->numOfVnodes);
dnodeProcessModuleStatus(pCfg->moduleStatus);
dnodeUpdateDnodeCfg(pCfg);
@ -616,6 +612,16 @@ static void dnodeSendStatusMsg(void *handle, void *tmrId) {
pStatus->numOfCores = htons((uint16_t) tsNumOfCores);
pStatus->diskAvailable = tsAvailDataDirGB;
pStatus->alternativeRole = (uint8_t) tsAlternativeRole;
// fill cluster cfg parameters
pStatus->clusterCfg.numOfMnodes = tsNumOfMnodes;
pStatus->clusterCfg.mnodeEqualVnodeNum = tsMnodeEqualVnodeNum;
pStatus->clusterCfg.offlineThreshold = tsOfflineThreshold;
pStatus->clusterCfg.statusInterval = tsStatusInterval;
strcpy(pStatus->clusterCfg.arbitrator, tsArbitrator);
strcpy(pStatus->clusterCfg.timezone, tsTimezone);
strcpy(pStatus->clusterCfg.locale, tsLocale);
strcpy(pStatus->clusterCfg.charset, tsCharset);
vnodeBuildStatusMsg(pStatus);
contLen = sizeof(SDMStatusMsg) + pStatus->openVnodes * sizeof(SVnodeLoad);

20
src/inc/query.h
View File

@ -32,11 +32,11 @@ int32_t qCreateQueryInfo(void* tsdb, int32_t vgId, SQueryTableMsg* pQueryTableMs
/**
* Destroy QInfo object
*
* @param qinfo
* @return
* @param qinfo qhandle
* @param fp destroy callback function, while the qhandle is destoried, invoke the fp
* @param param free callback params
*/
void qDestroyQueryInfo(qinfo_t qinfo);
void qDestroyQueryInfo(qinfo_t qinfo, void (*fp)(void*), void* param);
/**
* the main query execution function, including query on both table and multitables,
@ -45,7 +45,7 @@ void qDestroyQueryInfo(qinfo_t qinfo);
* @param qinfo
* @return
*/
void qTableQuery(qinfo_t qinfo);
void qTableQuery(qinfo_t qinfo, void (*fp)(void*), void* param);
/**
* Retrieve the produced results information, if current query is not paused or completed,
@ -70,7 +70,8 @@ int32_t qRetrieveQueryResultInfo(qinfo_t qinfo);
int32_t qDumpRetrieveResult(qinfo_t qinfo, SRetrieveTableRsp** pRsp, int32_t* contLen);
/**
* Decide if more results will be produced or not
* Decide if more results will be produced or not, NOTE: this function will increase the ref count of QInfo,
* so it can be only called once for each retrieve
*
* @param qinfo
* @return
@ -79,9 +80,12 @@ bool qHasMoreResultsToRetrieve(qinfo_t qinfo);
/**
* kill current ongoing query and free query handle automatically
* @param qinfo
* @param qinfo qhandle
* @param fp destroy callback function, while the qhandle is destoried, invoke the fp
* @param param free callback params
* @return
*/
int32_t qKillQuery(qinfo_t qinfo);
int32_t qKillQuery(qinfo_t qinfo, void (*fp)(void*), void* param);
#ifdef __cplusplus
}

2
src/inc/taos.h
View File

@ -55,7 +55,7 @@ typedef enum {
typedef struct taosField {
char name[65];
uint8_t type;
short bytes;
int16_t bytes;
} TAOS_FIELD;
#ifdef _TD_GO_DLL_

9
src/inc/taosdef.h
View File

@ -83,6 +83,9 @@ extern const int32_t TYPE_BYTES[11];
#define TSDB_DATA_NULL_STR "NULL"
#define TSDB_DATA_NULL_STR_L "null"
#define TSDB_DEFAULT_USER "root"
#define TSDB_DEFAULT_PASS "taosdata"
#define TSDB_TRUE 1
#define TSDB_FALSE 0
#define TSDB_OK 0
@ -156,7 +159,7 @@ typedef struct tDataTypeDescriptor {
extern tDataTypeDescriptor tDataTypeDesc[11];
#define POINTER_BYTES sizeof(void *) // 8 by default assert(sizeof(ptrdiff_t) == sizseof(void*)
bool isValidDataType(int32_t type, int32_t length);
bool isValidDataType(int32_t type);
bool isNull(const char *val, int32_t type);
void setVardataNull(char* val, int32_t type);
@ -209,8 +212,8 @@ void tsDataSwap(void *pLeft, void *pRight, int32_t type, int32_t size);
#define TSDB_MAX_SQL_SHOW_LEN 256
#define TSDB_MAX_ALLOWED_SQL_LEN (8*1024*1024U) // sql length should be less than 8mb
#define TSDB_MAX_BYTES_PER_ROW 65535
#define TSDB_MAX_TAGS_LEN 65535
#define TSDB_MAX_BYTES_PER_ROW 16384
#define TSDB_MAX_TAGS_LEN 16384
#define TSDB_MAX_TAGS 128
#define TSDB_AUTH_LEN 16

20
src/inc/taoserror.h
View File

@ -113,14 +113,19 @@ TAOS_DEFINE_ERROR(TSDB_CODE_MND_INVALID_QUERY_ID, 0, 0x030C, "mnode inva
TAOS_DEFINE_ERROR(TSDB_CODE_MND_INVALID_STREAM_ID, 0, 0x030D, "mnode invalid stream id")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_INVALID_CONN_ID, 0, 0x030E, "mnode invalid connection")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_SDB_OBJ_ALREADY_THERE, 0, 0x0320, "mnode object already there")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_SDB_ERROR, 0, 0x0321, "mnode sdb error")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_SDB_OBJ_ALREADY_THERE, 0, 0x0320, "sdb object already there")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_SDB_ERROR, 0, 0x0321, "sdb app error")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_SDB_INVALID_TABLE_TYPE, 0, 0x0322, "sdb invalid table type")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_SDB_OBJ_NOT_THERE, 0, 0x0323, "sdb object not there")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_SDB_INVAID_META_ROW, 0, 0x0324, "sdb invalid meta row")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_SDB_INVAID_KEY_TYPE, 0, 0x0325, "sdb invalid key type")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_DNODE_ALREADY_EXIST, 0, 0x0330, "mnode dnode already exist")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_DNODE_NOT_EXIST, 0, 0x0331, "mnode dnode not exist")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_VGROUP_NOT_EXIST, 0, 0x0332, "mnode vgroup not exist")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_NO_REMOVE_MASTER, 0, 0x0333, "mnode cant not remove master")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_NO_ENOUGH_DNODES, 0, 0x0334, "mnode no enough dnodes")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_CLUSTER_CFG_INCONSISTENT, 0, 0x0335, "mnode cluster cfg inconsistent")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_ACCT_ALREADY_EXIST, 0, 0x0340, "mnode accounts already exist")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_INVALID_ACCT, 0, 0x0341, "mnode invalid account")
@ -174,11 +179,12 @@ TAOS_DEFINE_ERROR(TSDB_CODE_VND_NO_DISK_PERMISSIONS, 0, 0x0506, "vnode no d
TAOS_DEFINE_ERROR(TSDB_CODE_VND_NO_SUCH_FILE_OR_DIR, 0, 0x0507, "vnode no such file or directory")
TAOS_DEFINE_ERROR(TSDB_CODE_VND_OUT_OF_MEMORY, 0, 0x0508, "vnode out of memory")
TAOS_DEFINE_ERROR(TSDB_CODE_VND_APP_ERROR, 0, 0x0509, "vnode app error")
TAOS_DEFINE_ERROR(TSDB_CODE_VND_NO_WRITE_AUTH, 0, 0x0214, "vnode no write auth")
// tsdb
TAOS_DEFINE_ERROR(TSDB_CODE_TDB_INVALID_TABLE_ID, 0, 0x0600, "tsdb invalid table id")
TAOS_DEFINE_ERROR(TSDB_CODE_TDB_INVALID_TABLE_TYPE, 0, 0x0601, "tsdb invalid table schema version")
TAOS_DEFINE_ERROR(TSDB_CODE_TDB_TABLE_SCHEMA_VERSION, 0, 0x0602, "tsdb invalid table schema version")
TAOS_DEFINE_ERROR(TSDB_CODE_TDB_INVALID_TABLE_TYPE, 0, 0x0601, "tsdb invalid table type")
TAOS_DEFINE_ERROR(TSDB_CODE_TDB_IVD_TB_SCHEMA_VERSION, 0, 0x0602, "tsdb invalid table schema version")
TAOS_DEFINE_ERROR(TSDB_CODE_TDB_TABLE_ALREADY_EXIST, 0, 0x0603, "tsdb table already exist")
TAOS_DEFINE_ERROR(TSDB_CODE_TDB_INVALID_CONFIG, 0, 0x0604, "tsdb invalid configuration")
TAOS_DEFINE_ERROR(TSDB_CODE_TDB_INIT_FAILED, 0, 0x0605, "tsdb init failed")
@ -188,6 +194,11 @@ TAOS_DEFINE_ERROR(TSDB_CODE_TDB_FILE_CORRUPTED, 0, 0x0608, "tsdb file
TAOS_DEFINE_ERROR(TSDB_CODE_TDB_OUT_OF_MEMORY, 0, 0x0609, "tsdb out of memory")
TAOS_DEFINE_ERROR(TSDB_CODE_TDB_TAG_VER_OUT_OF_DATE, 0, 0x060A, "tsdb tag version is out of date")
TAOS_DEFINE_ERROR(TSDB_CODE_TDB_TIMESTAMP_OUT_OF_RANGE, 0, 0x060B, "tsdb timestamp is out of range")
TAOS_DEFINE_ERROR(TSDB_CODE_TDB_SUBMIT_MSG_MSSED_UP, 0, 0x060C, "tsdb submit message is messed up")
TAOS_DEFINE_ERROR(TSDB_CODE_TDB_INVALID_ACTION, 0, 0x060D, "tsdb invalid action")
TAOS_DEFINE_ERROR(TSDB_CODE_TDB_INVALID_CREATE_TB_MSG, 0, 0x060E, "tsdb invalid create table message")
TAOS_DEFINE_ERROR(TSDB_CODE_TDB_NO_TABLE_DATA_IN_MEM, 0, 0x060F, "tsdb no table data in memory skiplist")
TAOS_DEFINE_ERROR(TSDB_CODE_TDB_FILE_ALREADY_EXISTS, 0, 0x0610, "tsdb file already exists")
// query
TAOS_DEFINE_ERROR(TSDB_CODE_QRY_INVALID_QHANDLE, 0, 0x0700, "query invalid handle")
@ -195,6 +206,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_QRY_INVALID_MSG, 0, 0x0701, "query inva
TAOS_DEFINE_ERROR(TSDB_CODE_QRY_NO_DISKSPACE, 0, 0x0702, "query no diskspace")
TAOS_DEFINE_ERROR(TSDB_CODE_QRY_OUT_OF_MEMORY, 0, 0x0703, "query out of memory")
TAOS_DEFINE_ERROR(TSDB_CODE_QRY_APP_ERROR, 0, 0x0704, "query app error")
TAOS_DEFINE_ERROR(TSDB_CODE_QRY_DUP_JOIN_KEY, 0, 0x0705, "query duplicated join key")
// grant
TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_EXPIRED, 0, 0x0800, "grant expired")

45
src/inc/taosmsg.h
View File

@ -250,7 +250,7 @@ typedef struct {
typedef struct {
char tableId[TSDB_TABLE_ID_LEN];
char db[TSDB_DB_NAME_LEN];
char db[TSDB_ACCT_LEN + TSDB_DB_NAME_LEN];
int8_t igExists;
int8_t getMeta;
int16_t numOfTags;
@ -268,7 +268,7 @@ typedef struct {
typedef struct {
char tableId[TSDB_TABLE_ID_LEN];
char db[TSDB_DB_NAME_LEN];
char db[TSDB_ACCT_LEN + TSDB_DB_NAME_LEN];
int16_t type; /* operation type */
int16_t numOfCols; /* number of schema */
int32_t tagValLen;
@ -455,7 +455,7 @@ typedef struct {
int16_t orderType; // used in group by xx order by xxx
int64_t limit;
int64_t offset;
uint16_t queryType; // denote another query process
uint32_t queryType; // denote another query process
int16_t numOfOutput; // final output columns numbers
int16_t tagNameRelType; // relation of tag criteria and tbname criteria
int16_t fillType; // interpolate type
@ -543,6 +543,7 @@ typedef struct {
int32_t dnodeId;
uint32_t moduleStatus;
uint32_t numOfVnodes;
uint32_t reserved;
} SDMDnodeCfg;
typedef struct {
@ -557,18 +558,30 @@ typedef struct {
} SDMMnodeInfos;
typedef struct {
uint32_t version;
int32_t dnodeId;
char dnodeEp[TSDB_EP_LEN];
uint32_t moduleStatus;
uint32_t lastReboot; // time stamp for last reboot
uint16_t numOfTotalVnodes; // from config file
uint16_t openVnodes;
uint16_t numOfCores;
float diskAvailable; // GB
uint8_t alternativeRole;
uint8_t reserve[15];
SVnodeLoad load[];
int32_t numOfMnodes; // tsNumOfMnodes
int32_t mnodeEqualVnodeNum; // tsMnodeEqualVnodeNum
int32_t offlineThreshold; // tsOfflineThreshold
int32_t statusInterval; // tsStatusInterval
char arbitrator[TSDB_EP_LEN]; // tsArbitrator
char timezone[64]; // tsTimezone
char locale[TSDB_LOCALE_LEN]; // tsLocale
char charset[TSDB_LOCALE_LEN]; // tsCharset
} SClusterCfg;
typedef struct {
uint32_t version;
int32_t dnodeId;
char dnodeEp[TSDB_EP_LEN];
uint32_t moduleStatus;
uint32_t lastReboot; // time stamp for last reboot
uint16_t numOfTotalVnodes; // from config file
uint16_t openVnodes;
uint16_t numOfCores;
float diskAvailable; // GB
uint8_t alternativeRole;
uint8_t reserve[15];
SClusterCfg clusterCfg;
SVnodeLoad load[];
} SDMStatusMsg;
typedef struct {
@ -670,7 +683,7 @@ typedef struct {
*/
typedef struct {
int8_t type;
char db[TSDB_DB_NAME_LEN];
char db[TSDB_ACCT_LEN + TSDB_DB_NAME_LEN];
uint16_t payloadLen;
char payload[];
} SCMShowMsg;

3
src/inc/trpc.h
View File

@ -78,12 +78,13 @@ void rpcClose(void *);
void *rpcMallocCont(int contLen);
void rpcFreeCont(void *pCont);
void *rpcReallocCont(void *ptr, int contLen);
void rpcSendRequest(void *thandle, const SRpcIpSet *pIpSet, const SRpcMsg *pMsg);
void *rpcSendRequest(void *thandle, const SRpcIpSet *pIpSet, const SRpcMsg *pMsg);
void rpcSendResponse(const SRpcMsg *pMsg);
void rpcSendRedirectRsp(void *pConn, const SRpcIpSet *pIpSet);
int rpcGetConnInfo(void *thandle, SRpcConnInfo *pInfo);
void rpcSendRecv(void *shandle, SRpcIpSet *pIpSet, const SRpcMsg *pReq, SRpcMsg *pRsp);
int rpcReportProgress(void *pConn, char *pCont, int contLen);
void rpcCancelRequest(void *pContext);
#ifdef __cplusplus
}

84
src/inc/tsdb.h
View File

@ -19,11 +19,11 @@
#include <stdbool.h>
#include <stdint.h>
#include "tdataformat.h"
#include "tname.h"
#include "taosdef.h"
#include "taosmsg.h"
#include "tarray.h"
#include "tdataformat.h"
#include "tname.h"
#ifdef __cplusplus
extern "C" {
@ -35,7 +35,7 @@ extern "C" {
#define TSDB_INVALID_SUPER_TABLE_ID -1
#define TSDB_STATUS_COMMIT_START 1
#define TSDB_STATUS_COMMIT_OVER 2
#define TSDB_STATUS_COMMIT_OVER 2
// --------- TSDB APPLICATION HANDLE DEFINITION
typedef struct {
@ -53,9 +53,9 @@ typedef struct {
int32_t tsdbId;
int32_t cacheBlockSize;
int32_t totalBlocks;
int32_t maxTables; // maximum number of tables this repository can have
int32_t daysPerFile; // day per file sharding policy
int32_t keep; // day of data to keep
int32_t maxTables; // maximum number of tables this repository can have
int32_t daysPerFile; // day per file sharding policy
int32_t keep; // day of data to keep
int32_t keep1;
int32_t keep2;
int32_t minRowsPerFileBlock; // minimum rows per file block
@ -72,19 +72,16 @@ typedef struct {
int64_t pointsWritten; // total data points written
} STsdbStat;
typedef void TsdbRepoT; // use void to hide implementation details from outside
typedef void TSDB_REPO_T; // use void to hide implementation details from outside
void tsdbSetDefaultCfg(STsdbCfg *pCfg);
STsdbCfg *tsdbCreateDefaultCfg();
void tsdbFreeCfg(STsdbCfg *pCfg);
STsdbCfg *tsdbGetCfg(const TsdbRepoT *repo);
STsdbCfg *tsdbGetCfg(const TSDB_REPO_T *repo);
// --------- TSDB REPOSITORY DEFINITION
int tsdbCreateRepo(char *rootDir, STsdbCfg *pCfg, void *limiter);
int32_t tsdbDropRepo(TsdbRepoT *repo);
TsdbRepoT *tsdbOpenRepo(char *rootDir, STsdbAppH *pAppH);
int32_t tsdbCloseRepo(TsdbRepoT *repo, int toCommit);
int32_t tsdbConfigRepo(TsdbRepoT *repo, STsdbCfg *pCfg);
int tsdbCreateRepo(char *rootDir, STsdbCfg *pCfg);
int32_t tsdbDropRepo(char *rootDir);
TSDB_REPO_T *tsdbOpenRepo(char *rootDir, STsdbAppH *pAppH);
void tsdbCloseRepo(TSDB_REPO_T *repo, int toCommit);
int32_t tsdbConfigRepo(TSDB_REPO_T *repo, STsdbCfg *pCfg);
// --------- TSDB TABLE DEFINITION
typedef struct {
@ -106,28 +103,19 @@ typedef struct {
char * sql;
} STableCfg;
int tsdbInitTableCfg(STableCfg *config, ETableType type, uint64_t uid, int32_t tid);
int tsdbTableSetSuperUid(STableCfg *config, uint64_t uid);
int tsdbTableSetSchema(STableCfg *config, STSchema *pSchema, bool dup);
int tsdbTableSetTagSchema(STableCfg *config, STSchema *pSchema, bool dup);
int tsdbTableSetTagValue(STableCfg *config, SKVRow row, bool dup);
int tsdbTableSetName(STableCfg *config, char *name, bool dup);
int tsdbTableSetSName(STableCfg *config, char *sname, bool dup);
int tsdbTableSetStreamSql(STableCfg *config, char *sql, bool dup);
void tsdbClearTableCfg(STableCfg *config);
void* tsdbGetTableTagVal(TsdbRepoT* repo, const STableId* id, int32_t colId, int16_t type, int16_t bytes);
char* tsdbGetTableName(TsdbRepoT *repo, const STableId *id);
void * tsdbGetTableTagVal(TSDB_REPO_T *repo, const STableId *id, int32_t colId, int16_t type, int16_t bytes);
char * tsdbGetTableName(TSDB_REPO_T *repo, const STableId *id);
STableCfg *tsdbCreateTableCfgFromMsg(SMDCreateTableMsg *pMsg);
int tsdbCreateTable(TsdbRepoT *repo, STableCfg *pCfg);
int tsdbDropTable(TsdbRepoT *pRepo, STableId tableId);
int tsdbAlterTable(TsdbRepoT *repo, STableCfg *pCfg);
int tsdbUpdateTagValue(TsdbRepoT *repo, SUpdateTableTagValMsg *pMsg);
TSKEY tsdbGetTableLastKey(TsdbRepoT *repo, uint64_t uid);
void tsdbStartStream(TsdbRepoT *repo);
int tsdbCreateTable(TSDB_REPO_T *repo, STableCfg *pCfg);
int tsdbDropTable(TSDB_REPO_T *pRepo, STableId tableId);
int tsdbUpdateTagValue(TSDB_REPO_T *repo, SUpdateTableTagValMsg *pMsg);
TSKEY tsdbGetTableLastKey(TSDB_REPO_T *repo, uint64_t uid);
void tsdbStartStream(TSDB_REPO_T *repo);
uint32_t tsdbGetFileInfo(TsdbRepoT *repo, char *name, uint32_t *index, uint32_t eindex, int32_t *size);
uint32_t tsdbGetFileInfo(TSDB_REPO_T *repo, char *name, uint32_t *index, uint32_t eindex, int32_t *size);
// the TSDB repository info
typedef struct STsdbRepoInfo {
@ -137,7 +125,7 @@ typedef struct STsdbRepoInfo {
int64_t tsdbTotalDiskSize; // the total disk size taken by this TSDB repository
// TODO: Other informations to add
} STsdbRepoInfo;
STsdbRepoInfo *tsdbGetStatus(TsdbRepoT *pRepo);
STsdbRepoInfo *tsdbGetStatus(TSDB_REPO_T *pRepo);
// the meter information report structure
typedef struct {
@ -146,7 +134,7 @@ typedef struct {
int64_t tableTotalDataSize; // In bytes
int64_t tableTotalDiskSize; // In bytes
} STableInfo;
STableInfo *tsdbGetTableInfo(TsdbRepoT *pRepo, STableId tid);
STableInfo *tsdbGetTableInfo(TSDB_REPO_T *pRepo, STableId tid);
// -- FOR INSERT DATA
/**
@ -156,7 +144,7 @@ STableInfo *tsdbGetTableInfo(TsdbRepoT *pRepo, STableId tid);
*
* @return the number of points inserted, -1 for failure and the error number is set
*/
int32_t tsdbInsertData(TsdbRepoT *repo, SSubmitMsg *pMsg, SShellSubmitRspMsg * pRsp) ;
int32_t tsdbInsertData(TSDB_REPO_T *repo, SSubmitMsg *pMsg, SShellSubmitRspMsg *pRsp);
// -- FOR QUERY TIME SERIES DATA
@ -164,10 +152,10 @@ typedef void *TsdbQueryHandleT; // Use void to hide implementation details
// query condition to build vnode iterator
typedef struct STsdbQueryCond {
STimeWindow twindow;
int32_t order; // desc|asc order to iterate the data block
int32_t numOfCols;
SColumnInfo *colList;
STimeWindow twindow;
int32_t order; // desc|asc order to iterate the data block
int32_t numOfCols;
SColumnInfo *colList;
} STsdbQueryCond;
typedef struct SDataBlockInfo {
@ -199,7 +187,7 @@ typedef void *TsdbPosT;
* @param qinfo query info handle from query processor
* @return
*/
TsdbQueryHandleT *tsdbQueryTables(TsdbRepoT *tsdb, STsdbQueryCond *pCond, STableGroupInfo *groupInfo, void* qinfo);
TsdbQueryHandleT *tsdbQueryTables(TSDB_REPO_T *tsdb, STsdbQueryCond *pCond, STableGroupInfo *groupInfo, void *qinfo);
/**
* Get the last row of the given query time window for all the tables in STableGroupInfo object.
@ -207,15 +195,17 @@ TsdbQueryHandleT *tsdbQueryTables(TsdbRepoT *tsdb, STsdbQueryCond *pCond, STable
* all tables in this group.
*
* @param tsdb tsdb handle
* @param pCond query condition, including time window, result set order, and basic required columns for each block
* @param pCond query condition, including time window, result set order, and basic required columns for each
* block
* @param groupInfo tableId list.
* @return
*/
TsdbQueryHandleT tsdbQueryLastRow(TsdbRepoT *tsdb, STsdbQueryCond *pCond, STableGroupInfo *groupInfo, void* qinfo);
TsdbQueryHandleT tsdbQueryLastRow(TSDB_REPO_T *tsdb, STsdbQueryCond *pCond, STableGroupInfo *groupInfo, void *qinfo);
SArray* tsdbGetQueriedTableIdList(TsdbQueryHandleT *pHandle);
SArray *tsdbGetQueriedTableIdList(TsdbQueryHandleT *pHandle);
TsdbQueryHandleT tsdbQueryRowsInExternalWindow(TsdbRepoT *tsdb, STsdbQueryCond* pCond, STableGroupInfo *groupList, void* qinfo);
TsdbQueryHandleT tsdbQueryRowsInExternalWindow(TSDB_REPO_T *tsdb, STsdbQueryCond *pCond, STableGroupInfo *groupList,
void *qinfo);
/**
* move to next block if exists
@ -293,7 +283,7 @@ SArray *tsdbGetTableList(TsdbQueryHandleT *pQueryHandle);
* @param stableid. super table sid
* @param pTagCond. tag query condition
*/
int32_t tsdbQuerySTableByTagCond(TsdbRepoT *tsdb, uint64_t uid, const char *pTagCond, size_t len,
int32_t tsdbQuerySTableByTagCond(TSDB_REPO_T *tsdb, uint64_t uid, const char *pTagCond, size_t len,
int16_t tagNameRelType, const char *tbnameCond, STableGroupInfo *pGroupList,
SColIndex *pColIndex, int32_t numOfCols);
@ -305,7 +295,7 @@ int32_t tsdbQuerySTableByTagCond(TsdbRepoT *tsdb, uint64_t uid, const char *pTag
* @param pGroupInfo the generated result
* @return
*/
int32_t tsdbGetOneTableGroup(TsdbRepoT *tsdb, uint64_t uid, STableGroupInfo *pGroupInfo);
int32_t tsdbGetOneTableGroup(TSDB_REPO_T *tsdb, uint64_t uid, STableGroupInfo *pGroupInfo);
/**
* clean up the query handle

1
src/inc/vnode.h
View File

@ -58,6 +58,7 @@ void* vnodeGetWal(void *pVnode);
int32_t vnodeProcessWrite(void *pVnode, int qtype, void *pHead, void *item);
void vnodeBuildStatusMsg(void * param);
void vnodeSetAccess(SDMVgroupAccess *pAccess, int32_t numOfVnodes);
int32_t vnodeProcessRead(void *pVnode, SReadMsg *pReadMsg);

48
src/kit/shell/src/shellEngine.c
View File

@ -56,11 +56,11 @@ TAOS *shellInit(SShellArguments *args) {
if (args->is_use_passwd) {
if (args->password == NULL) args->password = getpass("Enter password: ");
} else {
args->password = tsDefaultPass;
args->password = TSDB_DEFAULT_PASS;
}
if (args->user == NULL) {
args->user = tsDefaultUser;
args->user = TSDB_DEFAULT_USER;
}
taos_init();
@ -276,6 +276,8 @@ void shellRunCommandOnServer(TAOS *con, char command[]) {
st = taosGetTimestampUs();
TAOS_RES* pSql = taos_query(con, command);
result = pSql; // set it into the global variable
if (taos_errno(pSql)) {
taos_error(pSql);
return;
@ -284,7 +286,8 @@ void shellRunCommandOnServer(TAOS *con, char command[]) {
if (regex_match(command, "^\\s*use\\s+[a-zA-Z0-9_]+\\s*;\\s*$", REG_EXTENDED | REG_ICASE)) {
fprintf(stdout, "Database changed.\n\n");
fflush(stdout);
result = NULL;
taos_free_result(pSql);
return;
}
@ -294,6 +297,7 @@ void shellRunCommandOnServer(TAOS *con, char command[]) {
int error_no = 0;
int numOfRows = shellDumpResult(pSql, fname, &error_no, printMode);
if (numOfRows < 0) {
result = NULL;
taos_free_result(pSql);
return;
}
@ -315,7 +319,8 @@ void shellRunCommandOnServer(TAOS *con, char command[]) {
if (fname != NULL) {
wordfree(&full_path);
}
result = NULL;
taos_free_result(pSql);
}
@ -419,8 +424,8 @@ static void dumpFieldToFile(FILE* fp, const char* val, TAOS_FIELD* field, int32_
}
}
static int dumpResultToFile(const char* fname, TAOS_RES* result) {
TAOS_ROW row = taos_fetch_row(result);
static int dumpResultToFile(const char* fname, TAOS_RES* tres) {
TAOS_ROW row = taos_fetch_row(tres);
if (row == NULL) {
return 0;
}
@ -441,9 +446,9 @@ static int dumpResultToFile(const char* fname, TAOS_RES* result) {
wordfree(&full_path);
int num_fields = taos_num_fields(result);
TAOS_FIELD *fields = taos_fetch_fields(result);
int precision = taos_result_precision(result);
int num_fields = taos_num_fields(tres);
TAOS_FIELD *fields = taos_fetch_fields(tres);
int precision = taos_result_precision(tres);
for (int col = 0; col < num_fields; col++) {
if (col > 0) {
@ -455,7 +460,7 @@ static int dumpResultToFile(const char* fname, TAOS_RES* result) {
int numOfRows = 0;
do {
int32_t* length = taos_fetch_lengths(result);
int32_t* length = taos_fetch_lengths(tres);
for (int i = 0; i < num_fields; i++) {
if (i > 0) {
fputc(',', fp);
@ -465,11 +470,13 @@ static int dumpResultToFile(const char* fname, TAOS_RES* result) {
fputc('\n', fp);
numOfRows++;
row = taos_fetch_row(result);
row = taos_fetch_row(tres);
} while( row != NULL);
result = NULL;
taos_free_result(tres);
fclose(fp);
taos_free_result(result);
return numOfRows;
}
@ -770,8 +777,7 @@ void write_history() {
void taos_error(TAOS_RES *tres) {
fprintf(stderr, "\nDB error: %s\n", taos_errstr(tres));
/* free local resouce: allocated memory/metric-meta refcnt */
result = NULL;
taos_free_result(tres);
}
@ -846,9 +852,9 @@ void shellGetGrantInfo(void *con) {
char sql[] = "show grants";
result = taos_query(con, sql);
TAOS_RES* tres = taos_query(con, sql);
int code = taos_errno(result);
int code = taos_errno(tres);
if (code != TSDB_CODE_SUCCESS) {
if (code == TSDB_CODE_COM_OPS_NOT_SUPPORT) {
fprintf(stdout, "Server is Community Edition, version is %s\n\n", taos_get_server_info(con));
@ -858,18 +864,18 @@ void shellGetGrantInfo(void *con) {
return;
}
int num_fields = taos_field_count(result);
int num_fields = taos_field_count(tres);
if (num_fields == 0) {
fprintf(stderr, "\nInvalid grant information.\n");
exit(0);
} else {
if (result == NULL) {
if (tres == NULL) {
fprintf(stderr, "\nGrant information is null.\n");
exit(0);
}
TAOS_FIELD *fields = taos_fetch_fields(result);
TAOS_ROW row = taos_fetch_row(result);
TAOS_FIELD *fields = taos_fetch_fields(tres);
TAOS_ROW row = taos_fetch_row(tres);
if (row == NULL) {
fprintf(stderr, "\nFailed to get grant information from server. Abort.\n");
exit(0);
@ -889,8 +895,8 @@ void shellGetGrantInfo(void *con) {
fprintf(stdout, "Server is Enterprise %s Edition, version is %s and will expire at %s.\n", serverVersion, taos_get_server_info(con), expiretime);
}
taos_free_result(result);
result = NULL;
taos_free_result(tres);
}
fprintf(stdout, "\n");

9
src/kit/shell/src/shellMain.c
View File

@ -15,7 +15,6 @@
#include "os.h"
#include "shell.h"
#include "tsclient.h"
pthread_t pid;
@ -23,14 +22,6 @@ pthread_t pid;
void interruptHandler(int signum) {
#ifdef LINUX
taos_stop_query(result);
if (result != NULL) {
/*
* we need to free result in async model, in order to avoid free
* results while the master thread is waiting for server response.
*/
tscQueueAsyncFreeResult(result);
}
result = NULL;
#else
printf("\nReceive ctrl+c or other signal, quit shell.\n");

32
src/kit/taosdemo/taosdemo.c
View File

@ -297,7 +297,7 @@ void *deleteTable();
void *asyncWrite(void *sarg);
void generateData(char *res, char **data_type, int num_of_cols, int64_t timestamp, int len_of_binary);
int generateData(char *res, char **data_type, int num_of_cols, int64_t timestamp, int len_of_binary);
void rand_string(char *str, int size);
@ -306,7 +306,7 @@ double getCurrentTime();
void callBack(void *param, TAOS_RES *res, int code);
int main(int argc, char *argv[]) {
SDemoArguments arguments = {NULL, // host
SDemoArguments arguments = { NULL, // host
0, // port
"root", // user
"taosdata", // password
@ -817,7 +817,7 @@ void queryDB(TAOS *taos, char *command) {
i--;
}
if (i == 0) {
if (code != 0) {
fprintf(stderr, "Failed to run %s, reason: %s\n", command, taos_errstr(pSql));
taos_free_result(pSql);
@ -846,14 +846,19 @@ void *syncWrite(void *sarg) {
int k;
for (k = 0; k < winfo->nrecords_per_request;) {
int rand_num = rand() % 100;
if (winfo->data_of_order ==1 && rand_num < winfo->data_of_rate)
{
int len = -1;
if (winfo->data_of_order ==1 && rand_num < winfo->data_of_rate) {
long d = tmp_time - rand() % 1000000 + rand_num;
generateData(data, data_type, ncols_per_record, d, len_of_binary);
} else
{
generateData(data, data_type, ncols_per_record, tmp_time += 1000, len_of_binary);
len = generateData(data, data_type, ncols_per_record, d, len_of_binary);
} else {
len = generateData(data, data_type, ncols_per_record, tmp_time += 1000, len_of_binary);
}
//assert(len + pstr - buffer < BUFFER_SIZE);
if (len + pstr - buffer >= BUFFER_SIZE) { // too long
break;
}
pstr += sprintf(pstr, " %s", data);
inserted++;
k++;
@ -968,7 +973,7 @@ double getCurrentTime() {
return tv.tv_sec + tv.tv_usec / 1E6;
}
void generateData(char *res, char **data_type, int num_of_cols, int64_t timestamp, int len_of_binary) {
int32_t generateData(char *res, char **data_type, int num_of_cols, int64_t timestamp, int len_of_binary) {
memset(res, 0, MAX_DATA_SIZE);
char *pstr = res;
pstr += sprintf(pstr, "(%" PRId64, timestamp);
@ -1002,9 +1007,16 @@ void generateData(char *res, char **data_type, int num_of_cols, int64_t timestam
rand_string(s, len_of_binary);
pstr += sprintf(pstr, ", \"%s\"", s);
}
if (pstr - res > MAX_DATA_SIZE) {
perror("column length too long, abort");
exit(-1);
}
}
pstr += sprintf(pstr, ")");
return pstr - res;
}
static const char charset[] = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJK1234567890";

36
src/kit/taosdump/taosdump.c
View File

@ -34,6 +34,8 @@
#include "taosdef.h"
#include "tutil.h"
#include "tglobal.h"
#define COMMAND_SIZE 65536
#define DEFAULT_DUMP_FILE "taosdump.sql"
@ -293,7 +295,6 @@ static error_t parse_opt(int key, char *arg, struct argp_state *state) {
static struct argp argp = {options, parse_opt, args_doc, doc};
TAOS *taos = NULL;
TAOS_RES *result = NULL;
char *command = NULL;
char *lcommand = NULL;
char *buffer = NULL;
@ -324,7 +325,7 @@ void taosFreeDbInfos();
int main(int argc, char *argv[]) {
SDumpArguments arguments = {
// connection option
NULL, "root", "taosdata", 0,
NULL, TSDB_DEFAULT_USER, TSDB_DEFAULT_PASS, 0,
// output file
DEFAULT_DUMP_FILE, DEFAULT_DUMP_FILE, NULL,
// dump unit option
@ -424,6 +425,7 @@ int taosGetTableRecordInfo(char *table, STableRecordInfo *pTableRecordInfo) {
int taosDumpOut(SDumpArguments *arguments) {
TAOS_ROW row;
TAOS_RES* result = NULL;
char *temp = NULL;
FILE *fp = NULL;
int count = 0;
@ -466,7 +468,7 @@ int taosDumpOut(SDumpArguments *arguments) {
result = taos_query(taos, command);
int32_t code = taos_errno(result);
if (code != 0) {
fprintf(stderr, "failed to run command: %s, reason: %s\n", command, taos_errstr(taos));
fprintf(stderr, "failed to run command: %s, reason: %s\n", command, taos_errstr(result));
taos_free_result(result);
goto _exit_failure;
}
@ -502,7 +504,7 @@ int taosDumpOut(SDumpArguments *arguments) {
}
strncpy(dbInfos[count]->name, (char *)row[TSDB_SHOW_DB_NAME_INDEX], fields[TSDB_SHOW_DB_NAME_INDEX].bytes);
if (strcmp(arguments->user, "root") == 0) {
if (strcmp(arguments->user, TSDB_DEFAULT_USER) == 0) {
dbInfos[count]->replica = (int)(*((int16_t *)row[TSDB_SHOW_DB_REPLICA_INDEX]));
dbInfos[count]->days = (int)(*((int16_t *)row[TSDB_SHOW_DB_DAYS_INDEX]));
dbInfos[count]->keep = *((int *)row[TSDB_SHOW_DB_KEEP_INDEX]);
@ -570,7 +572,7 @@ int taosDumpOut(SDumpArguments *arguments) {
fclose(fp);
taos_close(taos);
taos_free_result(result);
free(temp);
tfree(temp);
taosFreeDbInfos();
return 0;
@ -578,7 +580,7 @@ _exit_failure:
fclose(fp);
taos_close(taos);
taos_free_result(result);
free(temp);
tfree(temp);
taosFreeDbInfos();
return -1;
}
@ -613,7 +615,7 @@ int taosDumpDb(SDbInfo *dbInfo, SDumpArguments *arguments, FILE *fp) {
fprintf(fp, "USE %s\n\n", dbInfo->name);
sprintf(command, "show tables");
result = taos_query(taos,command);
TAOS_RES* result = taos_query(taos,command);
int32_t code = taos_errno(result);
if (code != 0) {
fprintf(stderr, "failed to run command %s, error: %s\n", command, taos_errstr(result));
@ -717,7 +719,7 @@ void taosDumpCreateMTableClause(STableDef *tableDes, char *metric, int numOfCols
sprintf(command, "select %s from %s limit 1", tableDes->cols[counter].field, tableDes->name);
result = taos_query(taos, command);
TAOS_RES* result = taos_query(taos, command);
int32_t code = taos_errno(result);
if (code != 0) {
fprintf(stderr, "failed to run command %s, error: %s\n", command, taos_errstr(result));
@ -795,7 +797,7 @@ int taosGetTableDes(char *table, STableDef *tableDes) {
sprintf(command, "describe %s", table);
result = taos_query(taos, command);
TAOS_RES* result = taos_query(taos, command);
int32_t code = taos_errno(result);
if (code != 0) {
fprintf(stderr, "failed to run command %s, error: %s\n", command, taos_errstr(result));
@ -875,7 +877,7 @@ int32_t taosDumpMetric(char *metric, SDumpArguments *arguments, FILE *fp) {
tstrncpy(tableRecord.metric, metric, TSDB_TABLE_NAME_LEN);
sprintf(command, "select tbname from %s", metric);
result = taos_query(taos, command);
TAOS_RES* result = taos_query(taos, command);
int32_t code = taos_errno(result);
if (code != 0) {
fprintf(stderr, "failed to run command %s, error: %s\n", command, taos_errstr(result));
@ -928,7 +930,7 @@ int taosDumpTableData(FILE *fp, char *tbname, SDumpArguments *arguments) {
sprintf(command, "select * from %s where _c0 >= %" PRId64 " and _c0 <= %" PRId64 " order by _c0 asc", tbname, arguments->start_time,
arguments->end_time);
result = taos_query(taos, command);
TAOS_RES* result = taos_query(taos, command);
int32_t code = taos_errno(result);
if (code != 0) {
fprintf(stderr, "failed to run command %s, reason: %s\n", command, taos_errstr(result));
@ -1177,9 +1179,13 @@ int taosDumpIn(SDumpArguments *arguments) {
tcommand = command;
}
taosReplaceCtrlChar(tcommand);
if (taos_query(taos, tcommand) == NULL)
TAOS_RES* result = taos_query(taos, tcommand);
if (taos_errno(result) != 0){
fprintf(stderr, "linenu: %" PRId64 " failed to run command %s reason:%s \ncontinue...\n", linenu, command,
taos_errstr(taos));
taos_errstr(result));
taos_free_result(result);
}
pstr = command;
pstr[0] = '\0';
@ -1225,12 +1231,12 @@ int taosDumpIn(SDumpArguments *arguments) {
tcommand = command;
}
taosReplaceCtrlChar(tcommand);
result = taos_query(taos, tcommand);
TAOS_RES* result = taos_query(taos, tcommand);
int32_t code = taos_errno(result);
if (code != 0)
{
fprintf(stderr, "linenu:%" PRId64 " failed to run command %s reason: %s \ncontinue...\n", linenu, command,
taos_errstr(taos));
taos_errstr(result));
}
taos_free_result(result);
}

48
src/kit/taosnetwork/client.c
View File

@ -13,6 +13,7 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include <argp.h>
#include <arpa/inet.h>
#include <errno.h>
#include <fcntl.h>
@ -34,6 +35,36 @@ typedef struct {
char *host[15];
} info;
typedef struct Arguments {
char * host;
uint16_t port;
uint16_t max_port;
} SArguments;
static struct argp_option options[] = {
{0, 'h', "host", 0, "The host to connect to TDEngine. Default is localhost.", 0},
{0, 'p', "port", 0, "The TCP or UDP port number to use for the connection. Default is 6020.", 1},
{0, 'm', "max port", 0, "The max TCP or UDP port number to use for the connection. Default is 6050.", 2}};
static error_t parse_opt(int key, char *arg, struct argp_state *state) {
SArguments *arguments = state->input;
switch (key) {
case 'h':
arguments->host = arg;
break;
case 'p':
arguments->port = atoi(arg);
break;
case 'm':
arguments->max_port = atoi(arg);
break;
}
return 0;
}
static struct argp argp = {options, parse_opt, 0, 0};
void *checkPort(void *sarg) {
info *pinfo = (info *)sarg;
int port = pinfo->port;
@ -97,7 +128,7 @@ void *checkUPort(void *sarg) {
sprintf(sendbuf, "send msg port_%d by udp", port);
socklen_t sin_size = sizeof(*(struct sockaddr*)&serverAddr);
socklen_t sin_size = sizeof(*(struct sockaddr *)&serverAddr);
sendto(clientSocket, sendbuf, strlen(sendbuf), 0, (struct sockaddr *)&serverAddr, (int)sin_size);
@ -113,14 +144,19 @@ void *checkUPort(void *sarg) {
return NULL;
}
int main() {
int port = 6020;
char *host = "127.0.0.1";
int main(int argc, char *argv[]) {
SArguments arguments = {"127.0.0.1", 6020, 6050};
argp_parse(&argp, argc, argv, 0, 0, &arguments);
printf("host: %s\tport: %d\tmax_port: %d\n", arguments.host, arguments.port, arguments.max_port);
int port = arguments.port;
char *host = arguments.host;
info *tinfo = malloc(sizeof(info));
info *uinfo = malloc(sizeof(info));
for (size_t i = 0; i < 30; i++) {
port++;
for (; port < arguments.max_port; port++) {
printf("For test: %s:%d\n", host, port);
*tinfo->host = host;

58
src/kit/taosnetwork/server.c
View File

@ -13,6 +13,7 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include <argp.h>
#include <arpa/inet.h>
#include <errno.h>
#include <fcntl.h>
@ -34,6 +35,36 @@ typedef struct {
int type; // 0: tcp, 1: udo, default: 0
} info;
typedef struct Arguments {
char * host;
uint16_t port;
uint16_t max_port;
} SArguments;
static struct argp_option options[] = {
{0, 'h', "host", 0, "The host to connect to TDEngine. Default is localhost.", 0},
{0, 'p', "port", 0, "The TCP or UDP port number to use for the connection. Default is 6020.", 1},
{0, 'm', "max port", 0, "The max TCP or UDP port number to use for the connection. Default is 6050.", 2}};
static error_t parse_opt(int key, char *arg, struct argp_state *state) {
SArguments *arguments = state->input;
switch (key) {
case 'h':
arguments->host = arg;
break;
case 'p':
arguments->port = atoi(arg);
break;
case 'm':
arguments->max_port = atoi(arg);
break;
}
return 0;
}
static struct argp argp = {options, parse_opt, 0, 0};
static void *bindPort(void *sarg) {
info *pinfo = (info *)sarg;
int port = pinfo->port;
@ -170,15 +201,21 @@ static void *bindUPort(void *sarg) {
}
int main() {
int port = 6020;
pthread_t *pids = malloc(60 * sizeof(pthread_t));
info * infos = malloc(30 * sizeof(info));
info * uinfos = malloc(30 * sizeof(info));
int main(int argc, char *argv[]) {
SArguments arguments = {"127.0.0.1", 6020, 6050};
argp_parse(&argp, argc, argv, 0, 0, &arguments);
int port = arguments.port;
for (size_t i = 0; i < 30; i++) {
port++;
int num = arguments.max_port - arguments.port;
if (num < 0) {
num = 1;
}
pthread_t *pids = malloc(2 * num * sizeof(pthread_t));
info * infos = malloc(num * sizeof(info));
info * uinfos = malloc(num * sizeof(info));
for (size_t i = 0; i < num; i++) {
info *pinfo = infos++;
pinfo->port = port;
@ -191,14 +228,15 @@ int main() {
info *uinfo = uinfos++;
uinfo->port = port;
uinfo->type = 1;
if (pthread_create(pids + 30 + i, NULL, bindUPort, uinfo) != 0) //创建线程
port++;
if (pthread_create(pids + num + i, NULL, bindUPort, uinfo) != 0) //创建线程
{ //创建线程失败
printf("创建线程失败: %d.\n", port);
exit(0);
}
}
for (int i = 0; i < 30; i++) {
for (int i = 0; i < num; i++) {
pthread_join(pids[i], NULL);
pthread_join(pids[(10 + i)], NULL);
pthread_join(pids[(num + i)], NULL);
}
}

66
src/kit/taosnetwork/taosnetwork_client.c
View File

@ -1,66 +0,0 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include <netinet/in.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <unistd.h>
#define SERVER_PORT 8000
#define SIZE 200
int main() {
struct sockaddr_in servaddr, cliaddr;
socklen_t cliaddr_len;
int client_sockfd;
char buf[SIZE];
char recvbuf[SIZE];
int i, n, flag = 0;
int len, iDataNum;
client_sockfd = socket(AF_INET, SOCK_STREAM, 0);
bzero(&servaddr, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
servaddr.sin_port = htons(SERVER_PORT);
if (connect(client_sockfd, (struct sockaddr *)&servaddr, sizeof(servaddr)) < 0) {
printf("Connected error..\n");
return 0;
}
printf("Connected to server..\n");
/*循环的发送接收信息并打印接收信息（可以按需发送）--recv返回接收到的字节数，send返回发送的字节数*/
while (1) {
printf("Enter string to send:");
scanf("%s", buf);
if (!strcmp(buf, "quit")) {
break;
}
len = (sizeof buf);
recvbuf[0] = '\0';
iDataNum = recv(client_sockfd, recvbuf, SIZE, 0);
recvbuf[iDataNum] = '\0';
printf("%s\n", recvbuf);
}
return 0;
}

57
src/kit/taosnetwork/taosnetwork_server.c
View File

@ -1,57 +0,0 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include <netinet/in.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <unistd.h>
#define SERVER_PORT 8000
#define SIZE 200
int main() {
struct sockaddr_in servaddr, cliaddr;
socklen_t cliaddr_len;
int listenfd, connfd;
char buf[BUFSIZ];
int i, n, flag = 0;
listenfd = socket(AF_INET, SOCK_STREAM, 0);
bzero(&servaddr, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
servaddr.sin_port = htons(SERVER_PORT);
bind(listenfd, (struct sockaddr *)&servaddr, sizeof(servaddr));
listen(listenfd, 20);
printf("Accepting connections..\n");
while (1) {
cliaddr_len = sizeof(cliaddr);
connfd = accept(listenfd, (struct sockaddr *)&cliaddr,
&cliaddr_len); //如果得不到客户端发来的消息，将会被阻塞，一直等到消息到来
n = read(connfd, buf, SIZE); //如果n<=0，表示客户端已断开
while (1) {
if (n != 0) {
for (i = 0; i < n; i++) printf("%c", buf[i]); //输出客户端发来的信息
} else {
printf("Client say close the connection..\n");
break;
}
n = read(connfd, buf, SIZE);
}
close(connfd);
}
}

50
src/kit/taosnetwork/test_client.c
View File

@ -1,50 +0,0 @@
#include <netinet/in.h>
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/types.h>
#define SERVER_PORT 8888
#define BUFF_LEN 512
#define SERVER_IP "172.0.5.182"
void udp_msg_sender(int fd, struct sockaddr* dst) {}
/*
client:
socket-->sendto-->revcfrom-->close
*/
int main(int argc, char* argv[]) {
int client_fd;
struct sockaddr_in ser_addr;
client_fd = socket(AF_INET, SOCK_DGRAM, 0);
if (client_fd < 0) {
printf("create socket fail!\n");
return -1;
}
memset(&ser_addr, 0, sizeof(ser_addr));
ser_addr.sin_family = AF_INET;
// ser_addr.sin_addr.s_addr = inet_addr(SERVER_IP);
ser_addr.sin_addr.s_addr = htonl(INADDR_ANY); //注意网络序转换
ser_addr.sin_port = htons(SERVER_PORT); //注意网络序转换
socklen_t len;
struct sockaddr_in src;
while (1) {
char buf[BUFF_LEN] = "TEST UDP MSG!\n";
len = sizeof(*(struct sockaddr*)&ser_addr);
printf("client:%s\n", buf); //打印自己发送的信息
sendto(client_fd, buf, BUFF_LEN, 0, (struct sockaddr*)&ser_addr, len);
memset(buf, 0, BUFF_LEN);
recvfrom(client_fd, buf, BUFF_LEN, 0, (struct sockaddr*)&src, &len); //接收来自server的信息
printf("server:%s\n", buf);
sleep(1); //一秒发送一次消息
}
close(client_fd);
return 0;
}

63
src/kit/taosnetwork/test_server.c
View File

@ -1,63 +0,0 @@
#include <netinet/in.h>
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/types.h>
#define SERVER_PORT 8888
#define BUFF_LEN 1024
void handle_udp_msg(int fd) {
char buf[BUFF_LEN]; //接收缓冲区，1024字节
socklen_t len;
int count;
struct sockaddr_in clent_addr; // clent_addr用于记录发送方的地址信息
while (1) {
memset(buf, 0, BUFF_LEN);
len = sizeof(clent_addr);
count =
recvfrom(fd, buf, BUFF_LEN, 0, (struct sockaddr*)&clent_addr, &len); // recvfrom是拥塞函数，没有数据就一直拥塞
if (count == -1) {
printf("recieve data fail!\n");
return;
}
printf("client:%s\n", buf); //打印client发过来的信息
memset(buf, 0, BUFF_LEN);
sprintf(buf, "I have recieved %d bytes data!\n", count); //回复client
printf("server:%s\n", buf); //打印自己发送的信息给
sendto(fd, buf, BUFF_LEN, 0, (struct sockaddr*)&clent_addr,
len); //发送信息给client，注意使用了clent_addr结构体指针
}
}
/*
server:
socket-->bind-->recvfrom-->sendto-->close
*/
int main(int argc, char* argv[]) {
int server_fd, ret;
struct sockaddr_in ser_addr;
server_fd = socket(AF_INET, SOCK_DGRAM, 0); // AF_INET:IPV4;SOCK_DGRAM:UDP
if (server_fd < 0) {
printf("create socket fail!\n");
return -1;
}
memset(&ser_addr, 0, sizeof(ser_addr));
ser_addr.sin_family = AF_INET;
ser_addr.sin_addr.s_addr = htonl(INADDR_ANY); // IP地址，需要进行网络序转换，INADDR_ANY：本地地址
ser_addr.sin_port = htons(SERVER_PORT); //端口号，需要网络序转换
ret = bind(server_fd, (struct sockaddr*)&ser_addr, sizeof(ser_addr));
if (ret < 0) {
printf("socket bind fail!\n");
return -1;
}
handle_udp_msg(server_fd); //处理接收到的数据
close(server_fd);
return 0;
}

97
src/mnode/inc/mnodeDef.h
View File

@ -29,21 +29,28 @@ struct SAcctObj;
struct SUserObj;
struct SMnodeObj;
/*
struct define notes:
1. The first field must be the xxxxId field or name field , e.g. 'int32_t dnodeId', 'int32_t mnodeId', 'char name[]', 'char user[]', ...
2. From the dnodeId field to the updataEnd field, these information will be falled disc;
3. The fields behind the updataEnd field can be changed;
*/
typedef struct SDnodeObj {
int32_t dnodeId;
int32_t openVnodes;
int64_t createdTime;
int32_t totalVnodes; // from dnode status msg, config information
int32_t customScore; // config by user
uint32_t lastAccess;
uint16_t numOfCores; // from dnode status msg
uint16_t dnodePort;
char dnodeFqdn[TSDB_FQDN_LEN];
char dnodeEp[TSDB_EP_LEN];
int64_t createdTime;
uint32_t lastAccess;
int32_t openVnodes;
int32_t totalVnodes; // from dnode status msg, config information
int32_t customScore; // config by user
uint16_t numOfCores; // from dnode status msg
int8_t alternativeRole; // from dnode status msg, 0-any, 1-mgmt, 2-dnode
int8_t status; // set in balance function
int8_t isMgmt;
int8_t reserved[15];
int8_t reserved0[14];
int8_t updateEnd[1];
int32_t refCount;
uint32_t moduleStatus;
@ -54,15 +61,18 @@ typedef struct SDnodeObj {
int16_t cpuAvgUsage; // calc from sys.cpu
int16_t memoryAvgUsage; // calc from sys.mem
int16_t bandwidthUsage; // calc from sys.band
int8_t reserved1[2];
} SDnodeObj;
typedef struct SMnodeObj {
int32_t mnodeId;
int8_t reserved0[4];
int64_t createdTime;
int8_t reserved[14];
int8_t reserved1[7];
int8_t updateEnd[1];
int32_t refCount;
int8_t role;
int8_t reserved2[3];
} SMnodeObj;
typedef struct STableObj {
@ -71,35 +81,38 @@ typedef struct STableObj {
} STableObj;
typedef struct SSuperTableObj {
STableObj info;
STableObj info;
int8_t reserved0[1]; // for fill struct STableObj to 4byte align
int16_t nextColId;
int32_t sversion;
uint64_t uid;
int64_t createdTime;
int32_t sversion;
int32_t tversion;
int32_t numOfColumns;
int32_t numOfTags;
int8_t reserved[15];
int8_t reserved1[3];
int8_t updateEnd[1];
int32_t refCount;
int32_t numOfTables;
int16_t nextColId;
SSchema * schema;
void * vgHash;
int8_t reserved2[6];
} SSuperTableObj;
typedef struct {
STableObj info;
STableObj info;
int8_t reserved0[1]; // for fill struct STableObj to 4byte align
int16_t nextColId; //used by normal table
int32_t sversion; //used by normal table
uint64_t uid;
uint64_t suid;
int64_t createdTime;
int32_t sversion; //used by normal table
int32_t numOfColumns; //used by normal table
int32_t sid;
int32_t vgId;
uint64_t suid;
int32_t sqlLen;
int8_t reserved[1];
int8_t updateEnd[1];
int16_t nextColId; //used by normal table
int8_t reserved1[1];
int32_t refCount;
char* sql; //used by normal table
SSchema* schema; //used by normal table
@ -115,22 +128,24 @@ typedef struct {
typedef struct SVgObj {
uint32_t vgId;
char dbName[TSDB_DB_NAME_LEN];
int64_t createdTime;
SVnodeGid vnodeGid[TSDB_MAX_REPLICA];
int32_t numOfVnodes;
int64_t createdTime;
int32_t lbDnodeId;
int32_t lbTime;
char dbName[TSDB_DB_NAME_LEN];
int8_t inUse;
int8_t reserved[13];
int8_t accessState;
int8_t reserved0[5];
SVnodeGid vnodeGid[TSDB_MAX_REPLICA];
int8_t reserved1[7];
int8_t updateEnd[1];
int32_t refCount;
struct SVgObj *prev, *next;
struct SDbObj *pDb;
int32_t numOfTables;
int64_t totalStorage;
int64_t compStorage;
int64_t pointsWritten;
struct SVgObj *prev, *next;
struct SDbObj *pDb;
void * idPool;
SChildTableObj **tableList;
} SVgObj;
@ -150,17 +165,18 @@ typedef struct {
int8_t compression;
int8_t walLevel;
int8_t replications;
int8_t reserved[16];
int8_t reserved[12];
} SDbCfg;
typedef struct SDbObj {
char name[TSDB_ACCT_LEN + TSDB_DB_NAME_LEN];
int8_t reserved0[4];
char acct[TSDB_USER_LEN];
int64_t createdTime;
int32_t cfgVersion;
SDbCfg cfg;
int8_t status;
int8_t reserved[14];
int8_t reserved1[14];
int8_t updateEnd[1];
int32_t refCount;
int32_t numOfVgroups;
@ -185,13 +201,6 @@ typedef struct SUserObj {
} SUserObj;
typedef struct {
int32_t numOfUsers;
int32_t numOfDbs;
int32_t numOfTimeSeries;
int32_t numOfPointsPerSecond;
int32_t numOfConns;
int32_t numOfQueries;
int32_t numOfStreams;
int64_t totalStorage; // Total storage wrtten from this account
int64_t compStorage; // Compressed storage on disk
int64_t queryTime;
@ -199,34 +208,44 @@ typedef struct {
int64_t inblound;
int64_t outbound;
int64_t sKey;
int32_t numOfUsers;
int32_t numOfDbs;
int32_t numOfTimeSeries;
int32_t numOfPointsPerSecond;
int32_t numOfConns;
int32_t numOfQueries;
int32_t numOfStreams;
int8_t accessState; // Checked by mgmt heartbeat message
int8_t reserved[3];
} SAcctInfo;
typedef struct SAcctObj {
char user[TSDB_USER_LEN];
char pass[TSDB_KEY_LEN];
SAcctCfg cfg;
int32_t acctId;
int64_t createdTime;
int32_t acctId;
int8_t status;
int8_t reserved[14];
int8_t reserved0[10];
int8_t updateEnd[1];
int32_t refCount;
SAcctInfo acctInfo;
int32_t refCount;
int8_t reserved1[4];
pthread_mutex_t mutex;
} SAcctObj;
typedef struct {
int8_t type;
int32_t index;
char db[TSDB_DB_NAME_LEN];
void * pIter;
int8_t type;
int16_t numOfColumns;
int32_t index;
int32_t rowSize;
int32_t numOfRows;
int32_t numOfReads;
void * pIter;
int16_t offset[TSDB_MAX_COLUMNS];
int16_t bytes[TSDB_MAX_COLUMNS];
int32_t numOfReads;
int8_t reserved0[2];
uint16_t payloadLen;
char payload[];
} SShowObj;

2
src/mnode/inc/mnodeDnode.h
View File

@ -47,7 +47,7 @@ void mnodeDecDnodeRef(SDnodeObj *pDnode);
void * mnodeGetDnode(int32_t dnodeId);
void * mnodeGetDnodeByEp(char *ep);
void mnodeUpdateDnode(SDnodeObj *pDnode);
int32_t mnodeDropDnode(SDnodeObj *pDnode);
int32_t mnodeDropDnode(SDnodeObj *pDnode, void *pMsg);
extern int32_t tsAccessSquence;

8
src/mnode/inc/mnodeInt.h
View File

@ -36,10 +36,10 @@ extern int32_t sdbDebugFlag;
#define mLWarn(...) { monitorSaveLog(1, __VA_ARGS__); mWarn(__VA_ARGS__) }
#define mLPrint(...) { monitorSaveLog(0, __VA_ARGS__); mPrint(__VA_ARGS__) }
#define sdbError(...) { if (sdbDebugFlag & DEBUG_ERROR) { taosPrintLog("ERROR MND-SDB ", 255, __VA_ARGS__); }}
#define sdbWarn(...) { if (sdbDebugFlag & DEBUG_WARN) { taosPrintLog("WARN MND-SDB ", sdbDebugFlag, __VA_ARGS__); }}
#define sdbTrace(...) { if (sdbDebugFlag & DEBUG_TRACE) { taosPrintLog("MND-SDB ", sdbDebugFlag, __VA_ARGS__);}}
#define sdbPrint(...) { taosPrintLog("MND-SDB ", 255, __VA_ARGS__); }
#define sdbError(...) { if (sdbDebugFlag & DEBUG_ERROR) { taosPrintLog("ERROR SDB ", 255, __VA_ARGS__); }}
#define sdbWarn(...) { if (sdbDebugFlag & DEBUG_WARN) { taosPrintLog("WARN SDB ", sdbDebugFlag, __VA_ARGS__); }}
#define sdbTrace(...) { if (sdbDebugFlag & DEBUG_TRACE) { taosPrintLog("SDB ", sdbDebugFlag, __VA_ARGS__);}}
#define sdbPrint(...) { taosPrintLog("SDB ", 255, __VA_ARGS__); }
#define sdbLError(...) { monitorSaveLog(2, __VA_ARGS__); sdbError(__VA_ARGS__) }
#define sdbLWarn(...) { monitorSaveLog(1, __VA_ARGS__); sdbWarn(__VA_ARGS__) }

7
src/mnode/inc/mnodeSdb.h
View File

@ -20,6 +20,8 @@
extern "C" {
#endif
struct SMnodeMsg;
typedef enum {
SDB_TABLE_DNODE = 0,
SDB_TABLE_MNODE = 1,
@ -48,8 +50,11 @@ typedef struct {
ESdbOper type;
void * table;
void * pObj;
int32_t rowSize;
void * rowData;
int32_t rowSize;
int32_t retCode; // for callback in sdb queue
int32_t (*cb)(struct SMnodeMsg *pMsg, int32_t code);
struct SMnodeMsg *pMsg;
} SSdbOper;
typedef struct {

3
src/mnode/inc/mnodeUser.h
View File

@ -28,7 +28,8 @@ void * mnodeGetNextUser(void *pIter, SUserObj **pUser);
void mnodeIncUserRef(SUserObj *pUser);
void mnodeDecUserRef(SUserObj *pUser);
SUserObj *mnodeGetUserFromConn(void *pConn);
int32_t mnodeCreateUser(SAcctObj *pAcct, char *name, char *pass);
char * mnodeGetUserFromMsg(void *pMnodeMsg);
int32_t mnodeCreateUser(SAcctObj *pAcct, char *name, char *pass, void *pMsg);
void mnodeDropAllUsers(SAcctObj *pAcct);
#ifdef __cplusplus

3
src/mnode/inc/mnodeVgroup.h
View File

@ -34,7 +34,8 @@ void mnodeUpdateAllDbVgroups(SDbObj *pAlterDb);
void * mnodeGetNextVgroup(void *pIter, SVgObj **pVgroup);
void mnodeUpdateVgroup(SVgObj *pVgroup);
void mnodeUpdateVgroupStatus(SVgObj *pVgroup, SDnodeObj *dnodeId, SVnodeLoad *pVload);
void mnodeUpdateVgroupStatus(SVgObj *pVgroup, SDnodeObj *pDnode, SVnodeLoad *pVload);
void mnodeCheckUnCreatedVgroup(SDnodeObj *pDnode, SVnodeLoad *pVloads, int32_t openVnodes);
int32_t mnodeCreateVgroup(struct SMnodeMsg *pMsg, SDbObj *pDb);
void mnodeDropVgroup(SVgObj *pVgroup, void *ahandle);

13
src/mnode/src/mnodeAcct.c
View File

@ -25,6 +25,8 @@
#include "mnodeSdb.h"
#include "mnodeUser.h"
#include "tglobal.h"
void * tsAcctSdb = NULL;
static int32_t tsAcctUpdateSize;
static int32_t mnodeCreateRootAcct();
@ -39,6 +41,7 @@ static int32_t mnodeAcctActionDestroy(SSdbOper *pOper) {
static int32_t mnodeAcctActionInsert(SSdbOper *pOper) {
SAcctObj *pAcct = pOper->pObj;
memset(&pAcct->acctInfo, 0, sizeof(SAcctInfo));
pAcct->acctInfo.accessState = TSDB_VN_ALL_ACCCESS;
pthread_mutex_init(&pAcct->mutex, NULL);
return TSDB_CODE_SUCCESS;
}
@ -78,7 +81,9 @@ static int32_t mnodeAcctActionDecode(SSdbOper *pOper) {
}
static int32_t mnodeAcctActionRestored() {
if (dnodeIsFirstDeploy()) {
int32_t numOfRows = sdbGetNumOfRows(tsAcctSdb);
if (numOfRows <= 0 && dnodeIsFirstDeploy()) {
mPrint("dnode first deploy, create root acct");
int32_t code = mnodeCreateRootAcct();
if (code != TSDB_CODE_SUCCESS) {
mError("failed to create root account, reason:%s", tstrerror(code));
@ -171,8 +176,8 @@ static int32_t mnodeCreateRootAcct() {
SAcctObj *pAcct = malloc(sizeof(SAcctObj));
memset(pAcct, 0, sizeof(SAcctObj));
strcpy(pAcct->user, "root");
taosEncryptPass((uint8_t*)"taosdata", strlen("taosdata"), pAcct->pass);
strcpy(pAcct->user, TSDB_DEFAULT_USER);
taosEncryptPass((uint8_t *)TSDB_DEFAULT_PASS, strlen(TSDB_DEFAULT_PASS), pAcct->pass);
pAcct->cfg = (SAcctCfg){
.maxUsers = 10,
.maxDbs = 64,
@ -204,4 +209,4 @@ int32_t acctInit() { return TSDB_CODE_SUCCESS; }
void acctCleanUp() {}
int32_t acctCheck(void *pAcct, EAcctGrantType type) { return TSDB_CODE_SUCCESS; }
#endif
#endif

117
src/mnode/src/mnodeDb.c
View File

@ -41,7 +41,7 @@
static void * tsDbSdb = NULL;
static int32_t tsDbUpdateSize;
static int32_t mnodeCreateDb(SAcctObj *pAcct, SCMCreateDbMsg *pCreate);
static int32_t mnodeCreateDb(SAcctObj *pAcct, SCMCreateDbMsg *pCreate, void *pMsg);
static int32_t mnodeDropDb(SMnodeMsg *newMsg);
static int32_t mnodeSetDbDropping(SDbObj *pDb);
static int32_t mnodeGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn);
@ -84,9 +84,12 @@ static int32_t mnodeDbActionDelete(SSdbOper *pOper) {
mnodeDropAllChildTables(pDb);
mnodeDropAllSuperTables(pDb);
mnodeDropAllDbVgroups(pDb);
mnodeDropDbFromAcct(pAcct, pDb);
mnodeDecAcctRef(pAcct);
if (pAcct) {
mnodeDropDbFromAcct(pAcct, pDb);
mnodeDecAcctRef(pAcct);
}
return TSDB_CODE_SUCCESS;
}
@ -308,7 +311,7 @@ static void mnodeSetDefaultDbCfg(SDbCfg *pCfg) {
if (pCfg->replications < 0) pCfg->replications = tsReplications;
}
static int32_t mnodeCreateDb(SAcctObj *pAcct, SCMCreateDbMsg *pCreate) {
static int32_t mnodeCreateDb(SAcctObj *pAcct, SCMCreateDbMsg *pCreate, void *pMsg) {
int32_t code = acctCheck(pAcct, ACCT_GRANT_DB);
if (code != 0) return code;
@ -361,12 +364,15 @@ static int32_t mnodeCreateDb(SAcctObj *pAcct, SCMCreateDbMsg *pCreate) {
.table = tsDbSdb,
.pObj = pDb,
.rowSize = sizeof(SDbObj),
.pMsg = pMsg
};
code = sdbInsertRow(&oper);
if (code != TSDB_CODE_SUCCESS) {
tfree(pDb);
code = TSDB_CODE_MND_SDB_ERROR;
} else {
mLPrint("db:%s, is created by %s", pDb->name, mnodeGetUserFromMsg(pMsg));
if (pMsg != NULL) code = TSDB_CODE_MND_ACTION_IN_PROGRESS;
}
return code;
@ -472,7 +478,7 @@ static int32_t mnodeGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn
cols++;
#ifndef __CLOUD_VERSION__
if (strcmp(pUser->user, "root") == 0) {
if (strcmp(pUser->user, TSDB_DEFAULT_USER) == 0) {
#endif
pShow->bytes[cols] = 4;
pSchema[cols].type = TSDB_DATA_TYPE_INT;
@ -484,7 +490,7 @@ static int32_t mnodeGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn
#endif
#ifndef __CLOUD_VERSION__
if (strcmp(pUser->user, "root") == 0) {
if (strcmp(pUser->user, TSDB_DEFAULT_USER) == 0) {
#endif
pShow->bytes[cols] = 2;
pSchema[cols].type = TSDB_DATA_TYPE_SMALLINT;
@ -508,7 +514,7 @@ static int32_t mnodeGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn
cols++;
#ifndef __CLOUD_VERSION__
if (strcmp(pUser->user, "root") == 0) {
if (strcmp(pUser->user, TSDB_DEFAULT_USER) == 0) {
#endif
pShow->bytes[cols] = 4;
pSchema[cols].type = TSDB_DATA_TYPE_INT;
@ -610,7 +616,7 @@ static int32_t mnodeRetrieveDbs(SShowObj *pShow, char *data, int32_t rows, void
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
char* name = mnodeGetDbStr(pDb->name);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, name, TSDB_DB_NAME_LEN - 1);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, name, pShow->bytes[cols]);
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
@ -622,7 +628,7 @@ static int32_t mnodeRetrieveDbs(SShowObj *pShow, char *data, int32_t rows, void
cols++;
#ifndef __CLOUD_VERSION__
if (strcmp(pUser->user, "root") == 0) {
if (strcmp(pUser->user, TSDB_DEFAULT_USER) == 0) {
#endif
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
*(int32_t *)pWrite = pDb->numOfVgroups;
@ -632,7 +638,7 @@ static int32_t mnodeRetrieveDbs(SShowObj *pShow, char *data, int32_t rows, void
#endif
#ifndef __CLOUD_VERSION__
if (strcmp(pUser->user, "root") == 0) {
if (strcmp(pUser->user, TSDB_DEFAULT_USER) == 0) {
#endif
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
*(int16_t *)pWrite = pDb->cfg.replications;
@ -653,7 +659,7 @@ static int32_t mnodeRetrieveDbs(SShowObj *pShow, char *data, int32_t rows, void
cols++;
#ifndef __CLOUD_VERSION__
if (strcmp(pUser->user, "root") == 0) {
if (strcmp(pUser->user, TSDB_DEFAULT_USER) == 0) {
#endif
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
*(int32_t *)pWrite = pDb->cfg.maxTables; // table num can be created should minus 1
@ -768,12 +774,7 @@ static int32_t mnodeProcessCreateDbMsg(SMnodeMsg *pMsg) {
} else if (!pMsg->pUser->writeAuth) {
code = TSDB_CODE_MND_NO_RIGHTS;
} else {
code = mnodeCreateDb(pMsg->pUser->pAcct, pCreate);
if (code == TSDB_CODE_SUCCESS) {
mLPrint("db:%s, is created by %s", pCreate->db, pMsg->pUser->user);
} else {
mError("db:%s, failed to create, reason:%s", pCreate->db, tstrerror(code));
}
code = mnodeCreateDb(pMsg->pUser->pAcct, pCreate, pMsg);
}
return code;
@ -890,7 +891,31 @@ static SDbCfg mnodeGetAlterDbOption(SDbObj *pDb, SCMAlterDbMsg *pAlter) {
return newCfg;
}
static int32_t mnodeAlterDb(SDbObj *pDb, SCMAlterDbMsg *pAlter) {
static int32_t mnodeAlterDbCb(SMnodeMsg *pMsg, int32_t code) {
if (code != TSDB_CODE_SUCCESS) return code;
SDbObj *pDb = pMsg->pDb;
void *pIter = NULL;
while (1) {
SVgObj *pVgroup = NULL;
pIter = mnodeGetNextVgroup(pIter, &pVgroup);
if (pVgroup == NULL) break;
if (pVgroup->pDb == pDb) {
mnodeSendCreateVgroupMsg(pVgroup, NULL);
}
mnodeDecVgroupRef(pVgroup);
}
sdbFreeIter(pIter);
mTrace("db:%s, all vgroups is altered", pDb->name);
mLPrint("db:%s, is alterd by %s", pDb->name, mnodeGetUserFromMsg(pMsg));
balanceNotify();
return TSDB_CODE_SUCCESS;
}
static int32_t mnodeAlterDb(SDbObj *pDb, SCMAlterDbMsg *pAlter, void *pMsg) {
SDbCfg newCfg = mnodeGetAlterDbOption(pDb, pAlter);
if (terrno != TSDB_CODE_SUCCESS) {
return terrno;
@ -901,38 +926,24 @@ static int32_t mnodeAlterDb(SDbObj *pDb, SCMAlterDbMsg *pAlter) {
return code;
}
int32_t oldReplica = pDb->cfg.replications;
if (memcmp(&newCfg, &pDb->cfg, sizeof(SDbCfg)) != 0) {
pDb->cfg = newCfg;
pDb->cfgVersion++;
SSdbOper oper = {
.type = SDB_OPER_GLOBAL,
.type = SDB_OPER_GLOBAL,
.table = tsDbSdb,
.pObj = pDb
.pObj = pDb,
.pMsg = pMsg,
.cb = mnodeAlterDbCb
};
int32_t code = sdbUpdateRow(&oper);
if (code != TSDB_CODE_SUCCESS) {
return TSDB_CODE_MND_SDB_ERROR;
code = sdbUpdateRow(&oper);
if (code == TSDB_CODE_SUCCESS) {
if (pMsg != NULL) code = TSDB_CODE_MND_ACTION_IN_PROGRESS;
}
}
void *pIter = NULL;
while (1) {
SVgObj *pVgroup = NULL;
pIter = mnodeGetNextVgroup(pIter, &pVgroup);
if (pVgroup == NULL) break;
mnodeSendCreateVgroupMsg(pVgroup, NULL);
mnodeDecVgroupRef(pVgroup);
}
sdbFreeIter(pIter);
if (oldReplica != pDb->cfg.replications) {
balanceNotify();
}
return TSDB_CODE_SUCCESS;
return code;
}
static int32_t mnodeProcessAlterDbMsg(SMnodeMsg *pMsg) {
@ -945,28 +956,26 @@ static int32_t mnodeProcessAlterDbMsg(SMnodeMsg *pMsg) {
return TSDB_CODE_MND_INVALID_DB;
}
int32_t code = mnodeAlterDb(pMsg->pDb, pAlter);
if (code != TSDB_CODE_SUCCESS) {
mError("db:%s, failed to alter, invalid db option", pAlter->db);
return code;
}
mTrace("db:%s, all vgroups is altered", pMsg->pDb->name);
return TSDB_CODE_SUCCESS;
return mnodeAlterDb(pMsg->pDb, pAlter, pMsg);
}
static int32_t mnodeDropDb(SMnodeMsg *pMsg) {
if (pMsg == NULL) return TSDB_CODE_MND_APP_ERROR;
SDbObj *pDb = pMsg->pDb;
mPrint("db:%s, drop db from sdb", pDb->name);
SSdbOper oper = {
.type = SDB_OPER_GLOBAL,
.type = SDB_OPER_GLOBAL,
.table = tsDbSdb,
.pObj = pDb
.pObj = pDb,
.pMsg = pMsg
};
int32_t code = sdbDeleteRow(&oper);
if (code != 0) {
code = TSDB_CODE_MND_SDB_ERROR;
if (code == TSDB_CODE_SUCCESS) {
mLPrint("db:%s, is dropped by %s", pDb->name, mnodeGetUserFromMsg(pMsg));
code = TSDB_CODE_MND_ACTION_IN_PROGRESS;
}
return code;

141
src/mnode/src/mnodeDnode.c
View File

@ -44,7 +44,7 @@ static int32_t tsDnodeUpdateSize = 0;
extern void * tsMnodeSdb;
extern void * tsVgroupSdb;
static int32_t mnodeCreateDnode(char *ep);
static int32_t mnodeCreateDnode(char *ep, SMnodeMsg *pMsg);
static int32_t mnodeProcessCreateDnodeMsg(SMnodeMsg *pMsg);
static int32_t mnodeProcessDropDnodeMsg(SMnodeMsg *pMsg);
static int32_t mnodeProcessCfgDnodeMsg(SMnodeMsg *pMsg);
@ -90,11 +90,12 @@ static int32_t mnodeDnodeActionDelete(SSdbOper *pOper) {
static int32_t mnodeDnodeActionUpdate(SSdbOper *pOper) {
SDnodeObj *pDnode = pOper->pObj;
SDnodeObj *pSaved = mnodeGetDnode(pDnode->dnodeId);
if (pDnode != pSaved && pDnode != NULL && pSaved != NULL) {
if (pSaved != NULL && pDnode != pSaved) {
memcpy(pSaved, pDnode, pOper->rowSize);
free(pDnode);
mnodeDecDnodeRef(pSaved);
}
mnodeDecDnodeRef(pSaved);
return TSDB_CODE_SUCCESS;
}
@ -117,10 +118,13 @@ static int32_t mnodeDnodeActionDecode(SSdbOper *pOper) {
static int32_t mnodeDnodeActionRestored() {
int32_t numOfRows = sdbGetNumOfRows(tsDnodeSdb);
if (numOfRows <= 0 && dnodeIsFirstDeploy()) {
mnodeCreateDnode(tsLocalEp);
mPrint("dnode first deploy, create dnode:%s", tsLocalEp);
mnodeCreateDnode(tsLocalEp, NULL);
SDnodeObj *pDnode = mnodeGetDnodeByEp(tsLocalEp);
mnodeAddMnode(pDnode->dnodeId);
mnodeDecDnodeRef(pDnode);
if (pDnode != NULL) {
mnodeAddMnode(pDnode->dnodeId);
mnodeDecDnodeRef(pDnode);
}
}
return TSDB_CODE_SUCCESS;
@ -250,7 +254,7 @@ static int32_t mnodeProcessCfgDnodeMsg(SMnodeMsg *pMsg) {
// TODO temporary disabled for compiling: strcpy(pCmCfgDnode->ep, pCmCfgDnode->ep);
}
if (strcmp(pMsg->pUser->user, "root") != 0) {
if (strcmp(pMsg->pUser->user, TSDB_DEFAULT_USER) != 0) {
return TSDB_CODE_MND_NO_RIGHTS;
}
@ -277,6 +281,20 @@ static void mnodeProcessCfgDnodeMsgRsp(SRpcMsg *rpcMsg) {
mPrint("cfg dnode rsp is received");
}
static bool mnodeCheckClusterCfgPara(const SClusterCfg *clusterCfg) {
if (clusterCfg->numOfMnodes != tsNumOfMnodes) return false;
if (clusterCfg->mnodeEqualVnodeNum != tsMnodeEqualVnodeNum) return false;
if (clusterCfg->offlineThreshold != tsOfflineThreshold) return false;
if (clusterCfg->statusInterval != tsStatusInterval) return false;
if (0 != strncasecmp(clusterCfg->arbitrator, tsArbitrator, strlen(tsArbitrator))) return false;
if (0 != strncasecmp(clusterCfg->timezone, tsTimezone, strlen(tsTimezone))) return false;
if (0 != strncasecmp(clusterCfg->locale, tsLocale, strlen(tsLocale))) return false;
if (0 != strncasecmp(clusterCfg->charset, tsCharset, strlen(tsCharset))) return false;
return true;
}
static int32_t mnodeProcessDnodeStatusMsg(SMnodeMsg *pMsg) {
SDMStatusMsg *pStatus = pMsg->rpcMsg.pCont;
pStatus->dnodeId = htonl(pStatus->dnodeId);
@ -312,7 +330,6 @@ static int32_t mnodeProcessDnodeStatusMsg(SMnodeMsg *pMsg) {
pDnode->alternativeRole = pStatus->alternativeRole;
pDnode->totalVnodes = pStatus->numOfTotalVnodes;
pDnode->moduleStatus = pStatus->moduleStatus;
pDnode->lastAccess = tsAccessSquence;
if (pStatus->dnodeId == 0) {
mTrace("dnode:%d %s, first access", pDnode->dnodeId, pDnode->dnodeEp);
@ -321,6 +338,19 @@ static int32_t mnodeProcessDnodeStatusMsg(SMnodeMsg *pMsg) {
}
int32_t openVnodes = htons(pStatus->openVnodes);
int32_t contLen = sizeof(SDMStatusRsp) + openVnodes * sizeof(SDMVgroupAccess);
SDMStatusRsp *pRsp = rpcMallocCont(contLen);
if (pRsp == NULL) {
mnodeDecDnodeRef(pDnode);
return TSDB_CODE_MND_OUT_OF_MEMORY;
}
pRsp->dnodeCfg.dnodeId = htonl(pDnode->dnodeId);
pRsp->dnodeCfg.moduleStatus = htonl((int32_t)pDnode->isMgmt);
pRsp->dnodeCfg.numOfVnodes = htonl(openVnodes);
mnodeGetMnodeInfos(&pRsp->mnodes);
SDMVgroupAccess *pAccess = (SDMVgroupAccess *)((char *)pRsp + sizeof(SDMStatusRsp));
for (int32_t j = 0; j < openVnodes; ++j) {
SVnodeLoad *pVload = &pStatus->load[j];
pVload->vgId = htonl(pVload->vgId);
@ -333,42 +363,44 @@ static int32_t mnodeProcessDnodeStatusMsg(SMnodeMsg *pMsg) {
mnodeSendDropVnodeMsg(pVload->vgId, &ipSet, NULL);
} else {
mnodeUpdateVgroupStatus(pVgroup, pDnode, pVload);
pAccess->vgId = htonl(pVload->vgId);
pAccess->accessState = pVgroup->accessState;
pAccess++;
mnodeDecVgroupRef(pVgroup);
}
}
if (pDnode->status == TAOS_DN_STATUS_OFFLINE) {
// Verify whether the cluster parameters are consistent when status change from offline to ready
bool ret = mnodeCheckClusterCfgPara(&(pStatus->clusterCfg));
if (false == ret) {
mnodeDecDnodeRef(pDnode);
rpcFreeCont(pRsp);
mError("dnode %s cluster cfg parameters inconsistent", pStatus->dnodeEp);
return TSDB_CODE_MND_CLUSTER_CFG_INCONSISTENT;
}
mTrace("dnode:%d, from offline to online", pDnode->dnodeId);
pDnode->status = TAOS_DN_STATUS_READY;
balanceUpdateMnode();
balanceNotify();
}
mnodeDecDnodeRef(pDnode);
int32_t contLen = sizeof(SDMStatusRsp) + TSDB_MAX_VNODES * sizeof(SDMVgroupAccess);
SDMStatusRsp *pRsp = rpcMallocCont(contLen);
if (pRsp == NULL) {
return TSDB_CODE_MND_OUT_OF_MEMORY;
if (openVnodes != pDnode->openVnodes) {
mnodeCheckUnCreatedVgroup(pDnode, pStatus->load, openVnodes);
}
mnodeGetMnodeInfos(&pRsp->mnodes);
pDnode->lastAccess = tsAccessSquence;
mnodeDecDnodeRef(pDnode);
pRsp->dnodeCfg.dnodeId = htonl(pDnode->dnodeId);
pRsp->dnodeCfg.moduleStatus = htonl((int32_t)pDnode->isMgmt);
pRsp->dnodeCfg.numOfVnodes = 0;
contLen = sizeof(SDMStatusRsp);
//TODO: set vnode access
pMsg->rpcRsp.len = contLen;
pMsg->rpcRsp.rsp = pRsp;
return TSDB_CODE_SUCCESS;
}
static int32_t mnodeCreateDnode(char *ep) {
static int32_t mnodeCreateDnode(char *ep, SMnodeMsg *pMsg) {
int32_t grantCode = grantCheck(TSDB_GRANT_DNODE);
if (grantCode != TSDB_CODE_SUCCESS) {
return grantCode;
@ -392,7 +424,8 @@ static int32_t mnodeCreateDnode(char *ep) {
.type = SDB_OPER_GLOBAL,
.table = tsDnodeSdb,
.pObj = pDnode,
.rowSize = sizeof(SDnodeObj)
.rowSize = sizeof(SDnodeObj),
.pMsg = pMsg
};
int32_t code = sdbInsertRow(&oper);
@ -400,30 +433,32 @@ static int32_t mnodeCreateDnode(char *ep) {
int dnodeId = pDnode->dnodeId;
tfree(pDnode);
mError("failed to create dnode:%d, result:%s", dnodeId, tstrerror(code));
return TSDB_CODE_MND_SDB_ERROR;
} else {
mPrint("dnode:%d is created, result:%s", pDnode->dnodeId, tstrerror(code));
if (pMsg != NULL) code = TSDB_CODE_MND_ACTION_IN_PROGRESS;
}
mPrint("dnode:%d is created, result:%s", pDnode->dnodeId, tstrerror(code));
return code;
}
int32_t mnodeDropDnode(SDnodeObj *pDnode) {
int32_t mnodeDropDnode(SDnodeObj *pDnode, void *pMsg) {
SSdbOper oper = {
.type = SDB_OPER_GLOBAL,
.table = tsDnodeSdb,
.pObj = pDnode
.pObj = pDnode,
.pMsg = pMsg
};
int32_t code = sdbDeleteRow(&oper);
if (code != TSDB_CODE_SUCCESS) {
code = TSDB_CODE_MND_SDB_ERROR;
int32_t code = sdbDeleteRow(&oper);
if (code == TSDB_CODE_SUCCESS) {
mLPrint("dnode:%d, is dropped from cluster, result:%s", pDnode->dnodeId, tstrerror(code));
if (pMsg != NULL) code = TSDB_CODE_MND_ACTION_IN_PROGRESS;
}
mLPrint("dnode:%d, is dropped from cluster, result:%s", pDnode->dnodeId, tstrerror(code));
return code;
}
static int32_t mnodeDropDnodeByEp(char *ep) {
static int32_t mnodeDropDnodeByEp(char *ep, SMnodeMsg *pMsg) {
SDnodeObj *pDnode = mnodeGetDnodeByEp(ep);
if (pDnode == NULL) {
mError("dnode:%s, is not exist", ep);
@ -438,7 +473,7 @@ static int32_t mnodeDropDnodeByEp(char *ep) {
mPrint("dnode:%d, start to drop it", pDnode->dnodeId);
#ifndef _SYNC
return mnodeDropDnode(pDnode);
return mnodeDropDnode(pDnode, pMsg);
#else
return balanceDropDnode(pDnode);
#endif
@ -447,38 +482,20 @@ static int32_t mnodeDropDnodeByEp(char *ep) {
static int32_t mnodeProcessCreateDnodeMsg(SMnodeMsg *pMsg) {
SCMCreateDnodeMsg *pCreate = pMsg->rpcMsg.pCont;
if (strcmp(pMsg->pUser->user, "root") != 0) {
if (strcmp(pMsg->pUser->user, TSDB_DEFAULT_USER) != 0) {
return TSDB_CODE_MND_NO_RIGHTS;
} else {
int32_t code = mnodeCreateDnode(pCreate->ep);
if (code == TSDB_CODE_SUCCESS) {
SDnodeObj *pDnode = mnodeGetDnodeByEp(pCreate->ep);
mLPrint("dnode:%d, %s is created by %s", pDnode->dnodeId, pCreate->ep, pMsg->pUser->user);
mnodeDecDnodeRef(pDnode);
} else {
mError("failed to create dnode:%s, reason:%s", pCreate->ep, tstrerror(code));
}
return code;
return mnodeCreateDnode(pCreate->ep, pMsg);
}
}
static int32_t mnodeProcessDropDnodeMsg(SMnodeMsg *pMsg) {
SCMDropDnodeMsg *pDrop = pMsg->rpcMsg.pCont;
if (strcmp(pMsg->pUser->user, "root") != 0) {
if (strcmp(pMsg->pUser->user, TSDB_DEFAULT_USER) != 0) {
return TSDB_CODE_MND_NO_RIGHTS;
} else {
int32_t code = mnodeDropDnodeByEp(pDrop->ep);
if (code == TSDB_CODE_SUCCESS) {
mLPrint("dnode:%s is dropped by %s", pDrop->ep, pMsg->pUser->user);
} else {
mError("failed to drop dnode:%s, reason:%s", pDrop->ep, tstrerror(code));
}
return code;
return mnodeDropDnodeByEp(pDrop->ep, pMsg);
}
}
@ -486,7 +503,7 @@ static int32_t mnodeGetDnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pC
SUserObj *pUser = mnodeGetUserFromConn(pConn);
if (pUser == NULL) return 0;
if (strcmp(pUser->pAcct->user, "root") != 0) {
if (strcmp(pUser->pAcct->user, TSDB_DEFAULT_USER) != 0) {
mnodeDecUserRef(pUser);
return TSDB_CODE_MND_NO_RIGHTS;
}
@ -570,7 +587,7 @@ static int32_t mnodeRetrieveDnodes(SShowObj *pShow, char *data, int32_t rows, vo
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pDnode->dnodeEp, pShow->bytes[cols] - VARSTR_HEADER_SIZE);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pDnode->dnodeEp, pShow->bytes[cols]);
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
@ -615,7 +632,7 @@ static int32_t mnodeGetModuleMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *p
SUserObj *pUser = mnodeGetUserFromConn(pConn);
if (pUser == NULL) return 0;
if (strcmp(pUser->user, "root") != 0) {
if (strcmp(pUser->user, TSDB_DEFAULT_USER) != 0) {
mnodeDecUserRef(pUser);
return TSDB_CODE_MND_NO_RIGHTS;
}
@ -725,7 +742,7 @@ static int32_t mnodeGetConfigMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *p
SUserObj *pUser = mnodeGetUserFromConn(pConn);
if (pUser == NULL) return 0;
if (strcmp(pUser->user, "root") != 0) {
if (strcmp(pUser->user, TSDB_DEFAULT_USER) != 0) {
mnodeDecUserRef(pUser);
return TSDB_CODE_MND_NO_RIGHTS;
}
@ -812,7 +829,7 @@ static int32_t mnodeGetVnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pC
SUserObj *pUser = mnodeGetUserFromConn(pConn);
if (pUser == NULL) return 0;
if (strcmp(pUser->user, "root") != 0) {
if (strcmp(pUser->user, TSDB_DEFAULT_USER) != 0) {
mnodeDecUserRef(pUser);
return TSDB_CODE_MND_NO_RIGHTS;
}

10
src/mnode/src/mnodeMnode.c
View File

@ -31,6 +31,8 @@
#include "mnodeShow.h"
#include "mnodeUser.h"
#include "tglobal.h"
static void * tsMnodeSdb = NULL;
static int32_t tsMnodeUpdateSize = 0;
static SRpcIpSet tsMnodeIpSetForShell;
@ -333,7 +335,7 @@ static int32_t mnodeGetMnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pC
SUserObj *pUser = mnodeGetUserFromConn(pConn);
if (pUser == NULL) return 0;
if (strcmp(pUser->pAcct->user, "root") != 0) {
if (strcmp(pUser->pAcct->user, TSDB_DEFAULT_USER) != 0) {
mnodeDecUserRef(pUser);
return TSDB_CODE_MND_NO_RIGHTS;
}
@ -401,9 +403,9 @@ static int32_t mnodeRetrieveMnodes(SShowObj *pShow, char *data, int32_t rows, vo
SDnodeObj *pDnode = mnodeGetDnode(pMnode->mnodeId);
if (pDnode != NULL) {
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pDnode->dnodeEp, pShow->bytes[cols] - VARSTR_HEADER_SIZE);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pDnode->dnodeEp, pShow->bytes[cols]);
} else {
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, "invalid ep", pShow->bytes[cols] - VARSTR_HEADER_SIZE);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, "invalid ep", pShow->bytes[cols]);
}
mnodeDecDnodeRef(pDnode);
@ -411,7 +413,7 @@ static int32_t mnodeRetrieveMnodes(SShowObj *pShow, char *data, int32_t rows, vo
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
char* roles = mnodeGetMnodeRoleStr(pMnode->role);
STR_TO_VARSTR(pWrite, roles);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, roles, pShow->bytes[cols]);
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;

70
src/mnode/src/mnodeProfile.c
View File

@ -39,6 +39,7 @@
#define CONN_KEEP_TIME (tsShellActivityTimer * 3)
#define CONN_CHECK_TIME (tsShellActivityTimer * 2)
#define QUERY_ID_SIZE 20
#define QUERY_STREAM_SAVE_SIZE 20
extern void *tsMnodeTmr;
static SCacheObj *tsMnodeConnCache = NULL;
@ -108,8 +109,8 @@ SConnObj *mnodeCreateConn(char *user, uint32_t ip, uint16_t port) {
}
void mnodeReleaseConn(SConnObj *pConn) {
if(pConn == NULL) return;
taosCacheRelease(tsMnodeConnCache, (void**)&pConn, false);
if (pConn == NULL) return;
taosCacheRelease(tsMnodeConnCache, (void **)&pConn, false);
}
SConnObj *mnodeAccquireConn(uint32_t connId, char *user, uint32_t ip, uint16_t port) {
@ -138,7 +139,7 @@ SConnObj *mnodeAccquireConn(uint32_t connId, char *user, uint32_t ip, uint16_t p
static void mnodeFreeConn(void *data) {
SConnObj *pConn = data;
tfree(pConn->pQueries);
tfree(pConn->pQueries);
tfree(pConn->pStreams);
mTrace("connId:%d, is destroyed", pConn->connId);
}
@ -168,7 +169,7 @@ static void *mnodeGetNextConn(SHashMutableIterator *pIter, SConnObj **pConn) {
static int32_t mnodeGetConnsMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
SUserObj *pUser = mnodeGetUserFromConn(pConn);
if (pUser == NULL) return 0;
if (strcmp(pUser->user, "root") != 0) return TSDB_CODE_MND_NO_RIGHTS;
if (strcmp(pUser->user, TSDB_DEFAULT_USER) != 0) return TSDB_CODE_MND_NO_RIGHTS;
int32_t cols = 0;
SSchema *pSchema = pMeta->schema;
@ -235,14 +236,12 @@ static int32_t mnodeRetrieveConns(SShowObj *pShow, char *data, int32_t rows, voi
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
size_t size = sizeof(pConnObj->user);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pConnObj->user, size);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pConnObj->user, pShow->bytes[cols]);
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
snprintf(ipStr, sizeof(ipStr), "%s:%u", taosIpStr(pConnObj->ip), pConnObj->port);
size = sizeof(ipStr);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, ipStr, size);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, ipStr, pShow->bytes[cols]);
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
@ -266,16 +265,27 @@ static int32_t mnodeRetrieveConns(SShowObj *pShow, char *data, int32_t rows, voi
// not thread safe, need optimized
int32_t mnodeSaveQueryStreamList(SConnObj *pConn, SCMHeartBeatMsg *pHBMsg) {
pConn->numOfQueries = htonl(pHBMsg->numOfQueries);
if (pConn->numOfQueries > 0 && pConn->numOfQueries < 20) {
pConn->pQueries = calloc(sizeof(SQueryDesc), pConn->numOfQueries);
memcpy(pConn->pQueries, pHBMsg->pData, pConn->numOfQueries * sizeof(SQueryDesc));
if (pConn->numOfQueries > 0) {
if (pConn->pQueries == NULL) {
pConn->pQueries = calloc(sizeof(SQueryDesc), QUERY_STREAM_SAVE_SIZE);
}
int32_t saveSize = MIN(QUERY_STREAM_SAVE_SIZE, pConn->numOfQueries) * sizeof(SQueryDesc);
if (saveSize > 0 && pConn->pQueries != NULL) {
memcpy(pConn->pQueries, pHBMsg->pData, saveSize);
}
}
pConn->numOfStreams = htonl(pHBMsg->numOfStreams);
if (pConn->numOfStreams > 0 && pConn->numOfStreams < 20) {
pConn->pStreams = calloc(sizeof(SStreamDesc), pConn->numOfStreams);
memcpy(pConn->pStreams, pHBMsg->pData + pConn->numOfQueries * sizeof(SQueryDesc),
pConn->numOfStreams * sizeof(SStreamDesc));
if (pConn->numOfStreams > 0) {
if (pConn->pStreams == NULL) {
pConn->pStreams = calloc(sizeof(SStreamDesc), QUERY_STREAM_SAVE_SIZE);
}
int32_t saveSize = MIN(QUERY_STREAM_SAVE_SIZE, pConn->numOfStreams) * sizeof(SStreamDesc);
if (saveSize > 0 && pConn->pStreams != NULL) {
memcpy(pConn->pStreams, pHBMsg->pData + pConn->numOfQueries * sizeof(SQueryDesc), saveSize);
}
}
return TSDB_CODE_SUCCESS;
@ -284,7 +294,7 @@ int32_t mnodeSaveQueryStreamList(SConnObj *pConn, SCMHeartBeatMsg *pHBMsg) {
static int32_t mnodeGetQueryMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
SUserObj *pUser = mnodeGetUserFromConn(pConn);
if (pUser == NULL) return 0;
if (strcmp(pUser->user, "root") != 0) return TSDB_CODE_MND_NO_RIGHTS;
if (strcmp(pUser->user, TSDB_DEFAULT_USER) != 0) return TSDB_CODE_MND_NO_RIGHTS;
int32_t cols = 0;
SSchema *pSchema = pMeta->schema;
@ -356,18 +366,16 @@ static int32_t mnodeRetrieveQueries(SShowObj *pShow, char *data, int32_t rows, v
snprintf(ipStr, QUERY_ID_SIZE + 1, "%u:%u", pConnObj->connId, htonl(pDesc->queryId));
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, ipStr, QUERY_ID_SIZE);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, ipStr, pShow->bytes[cols]);
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
size_t size = sizeof(pConnObj->user);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pConnObj->user, size);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pConnObj->user, pShow->bytes[cols]);
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
snprintf(ipStr, sizeof(ipStr), "%s:%u", taosIpStr(pConnObj->ip), pConnObj->port);
size = sizeof(ipStr);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, ipStr, size);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, ipStr, pShow->bytes[cols]);
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
@ -379,7 +387,7 @@ static int32_t mnodeRetrieveQueries(SShowObj *pShow, char *data, int32_t rows, v
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pDesc->sql, TSDB_SHOW_SQL_LEN);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pDesc->sql, pShow->bytes[cols]);
cols++;
numOfRows++;
@ -395,7 +403,7 @@ static int32_t mnodeRetrieveQueries(SShowObj *pShow, char *data, int32_t rows, v
static int32_t mnodeGetStreamMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
SUserObj *pUser = mnodeGetUserFromConn(pConn);
if (pUser == NULL) return 0;
if (strcmp(pUser->user, "root") != 0) return TSDB_CODE_MND_NO_RIGHTS;
if (strcmp(pUser->user, TSDB_DEFAULT_USER) != 0) return TSDB_CODE_MND_NO_RIGHTS;
int32_t cols = 0;
SSchema *pSchema = pMeta->schema;
@ -479,18 +487,16 @@ static int32_t mnodeRetrieveStreams(SShowObj *pShow, char *data, int32_t rows, v
snprintf(ipStr, QUERY_ID_SIZE + 1, "%u:%u", pConnObj->connId, htonl(pDesc->streamId));
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, ipStr, QUERY_ID_SIZE);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, ipStr, pShow->bytes[cols]);
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
size_t size = sizeof(pConnObj->user);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pConnObj->user, size);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pConnObj->user, pShow->bytes[cols]);
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
snprintf(ipStr, sizeof(ipStr), "%s:%u", taosIpStr(pConnObj->ip), pConnObj->port);
size = sizeof(ipStr);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, ipStr, size);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, ipStr, pShow->bytes[cols]);
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
@ -506,7 +512,7 @@ static int32_t mnodeRetrieveStreams(SShowObj *pShow, char *data, int32_t rows, v
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pDesc->sql, TSDB_SHOW_SQL_LEN);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pDesc->sql, pShow->bytes[cols]);
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
@ -525,7 +531,7 @@ static int32_t mnodeRetrieveStreams(SShowObj *pShow, char *data, int32_t rows, v
static int32_t mnodeProcessKillQueryMsg(SMnodeMsg *pMsg) {
SUserObj *pUser = pMsg->pUser;
if (strcmp(pUser->user, "root") != 0) return TSDB_CODE_MND_NO_RIGHTS;
if (strcmp(pUser->user, TSDB_DEFAULT_USER) != 0) return TSDB_CODE_MND_NO_RIGHTS;
SCMKillQueryMsg *pKill = pMsg->rpcMsg.pCont;
mPrint("kill query msg is received, queryId:%s", pKill->queryId);
@ -555,7 +561,7 @@ static int32_t mnodeProcessKillQueryMsg(SMnodeMsg *pMsg) {
static int32_t mnodeProcessKillStreamMsg(SMnodeMsg *pMsg) {
SUserObj *pUser = pMsg->pUser;
if (strcmp(pUser->user, "root") != 0) return TSDB_CODE_MND_NO_RIGHTS;
if (strcmp(pUser->user, TSDB_DEFAULT_USER) != 0) return TSDB_CODE_MND_NO_RIGHTS;
SCMKillQueryMsg *pKill = pMsg->rpcMsg.pCont;
mPrint("kill stream msg is received, streamId:%s", pKill->queryId);
@ -585,7 +591,7 @@ static int32_t mnodeProcessKillStreamMsg(SMnodeMsg *pMsg) {
static int32_t mnodeProcessKillConnectionMsg(SMnodeMsg *pMsg) {
SUserObj *pUser = pMsg->pUser;
if (strcmp(pUser->user, "root") != 0) return TSDB_CODE_MND_NO_RIGHTS;
if (strcmp(pUser->user, TSDB_DEFAULT_USER) != 0) return TSDB_CODE_MND_NO_RIGHTS;
SCMKillConnMsg *pKill = pMsg->rpcMsg.pCont;
SConnObj * pConn = taosCacheAcquireByName(tsMnodeConnCache, pKill->queryId);

377
src/mnode/src/mnodeSdb.c
View File

@ -24,6 +24,7 @@
#include "tsync.h"
#include "tglobal.h"
#include "dnode.h"
#include "mnode.h"
#include "mnodeDef.h"
#include "mnodeInt.h"
#include "mnodeMnode.h"
@ -31,6 +32,7 @@
#include "mnodeSdb.h"
#define SDB_TABLE_LEN 12
#define SDB_SYNC_HACK 16
typedef enum {
SDB_ACTION_INSERT,
@ -83,8 +85,29 @@ typedef struct {
void * row;
} SSdbRow;
typedef struct {
pthread_t thread;
int32_t workerId;
} SSdbWriteWorker;
typedef struct {
int32_t num;
SSdbWriteWorker *writeWorker;
} SSdbWriteWorkerPool;
static SSdbObject tsSdbObj = {0};
static int sdbWrite(void *param, void *data, int type);
static taos_qset tsSdbWriteQset;
static taos_qall tsSdbWriteQall;
static taos_queue tsSdbWriteQueue;
static SSdbWriteWorkerPool tsSdbPool;
static int sdbWrite(void *param, void *data, int type);
static int sdbWriteToQueue(void *param, void *data, int type);
static void * sdbWorkerFp(void *param);
static int32_t sdbInitWriteWorker();
static void sdbCleanupWriteWorker();
static int32_t sdbAllocWriteQueue();
static void sdbFreeWritequeue();
int32_t sdbGetId(void *handle) {
return ((SSdbTable *)handle)->autoIndex;
@ -302,7 +325,7 @@ void sdbUpdateSync() {
syncInfo.ahandle = NULL;
syncInfo.getWalInfo = sdbGetWalInfo;
syncInfo.getFileInfo = sdbGetFileInfo;
syncInfo.writeToCache = sdbWrite;
syncInfo.writeToCache = sdbWriteToQueue;
syncInfo.confirmForward = sdbConfirmForward;
syncInfo.notifyRole = sdbNotifyRole;
tsSdbObj.cfg = syncCfg;
@ -319,10 +342,14 @@ int32_t sdbInit() {
pthread_mutex_init(&tsSdbObj.mutex, NULL);
sem_init(&tsSdbObj.sem, 0, 0);
if (sdbInitWriteWorker() != 0) {
return -1;
}
if (sdbInitWal() != 0) {
return -1;
}
sdbRestoreTables();
if (mnodeGetMnodesNum() == 1) {
@ -340,6 +367,8 @@ void sdbCleanUp() {
tsSdbObj.status = SDB_STATUS_CLOSING;
sdbCleanupWriteWorker();
if (tsSdbObj.sync) {
syncStop(tsSdbObj.sync);
tsSdbObj.sync = NULL;
@ -475,7 +504,7 @@ static int32_t sdbDeleteHash(SSdbTable *pTable, SSdbOper *pOper) {
pTable->numOfRows--;
pthread_mutex_unlock(&pTable->mutex);
sdbTrace("table:%s, delete record:%s from hash, numOfRows:%" PRId64 "version:%" PRIu64, pTable->tableName,
sdbTrace("table:%s, delete record:%s from hash, numOfRows:%" PRId64 " version:%" PRIu64, pTable->tableName,
sdbGetKeyStrFromObj(pTable, pOper->pObj), pTable->numOfRows, sdbGetVersion());
int8_t *updateEnd = pOper->pObj + pTable->refCountPos - 1;
@ -494,9 +523,10 @@ static int32_t sdbUpdateHash(SSdbTable *pTable, SSdbOper *pOper) {
}
static int sdbWrite(void *param, void *data, int type) {
SSdbOper *pOper = param;
SWalHead *pHead = data;
int32_t tableId = pHead->msgType / 10;
int32_t action = pHead->msgType % 10;
int32_t tableId = pHead->msgType / 10;
int32_t action = pHead->msgType % 10;
SSdbTable *pTable = sdbGetTableFromId(tableId);
assert(pTable != NULL);
@ -531,21 +561,22 @@ static int sdbWrite(void *param, void *data, int type) {
pthread_mutex_unlock(&tsSdbObj.mutex);
return code;
}
walFsync(tsSdbObj.wal);
code = sdbForwardToPeer(pHead);
pthread_mutex_unlock(&tsSdbObj.mutex);
// from app, oper is created
if (param != NULL) {
//sdbTrace("request from app is disposed, version:%" PRIu64 " code:%s", pHead->version, tstrerror(code));
if (pOper != NULL) {
sdbTrace("record from app is disposed, version:%" PRIu64 " result:%s", pHead->version, tstrerror(code));
return code;
}
// from wal or forward msg, oper not created, should add into hash
if (tsSdbObj.sync != NULL) {
sdbTrace("forward request is received, version:%" PRIu64 " result:%s, confirm it", pHead->version, tstrerror(code));
sdbTrace("record from wal forward is disposed, version:%" PRIu64 " confirm it", pHead->version);
syncConfirmForward(tsSdbObj.sync, pHead->version, code);
} else {
sdbTrace("record from wal restore is disposed, version:%" PRIu64 , pHead->version);
}
if (action == SDB_ACTION_INSERT) {
@ -568,7 +599,7 @@ static int sdbWrite(void *param, void *data, int type) {
int32_t sdbInsertRow(SSdbOper *pOper) {
SSdbTable *pTable = (SSdbTable *)pOper->table;
if (pTable == NULL) return -1;
if (pTable == NULL) return TSDB_CODE_MND_SDB_INVALID_TABLE_TYPE;
if (sdbGetRowFromObj(pTable, pOper->pObj)) {
sdbError("table:%s, failed to insert record:%s, already exist", pTable->tableName, sdbGetKeyStrFromObj(pTable, pOper->pObj));
@ -587,98 +618,133 @@ int32_t sdbInsertRow(SSdbOper *pOper) {
pthread_mutex_unlock(&pTable->mutex);
}
if (pOper->type == SDB_OPER_GLOBAL) {
int32_t size = sizeof(SWalHead) + pTable->maxRowSize;
SWalHead *pHead = taosAllocateQitem(size);
pHead->version = 0;
pHead->len = pOper->rowSize;
pHead->msgType = pTable->tableId * 10 + SDB_ACTION_INSERT;
pOper->rowData = pHead->cont;
(*pTable->encodeFp)(pOper);
pHead->len = pOper->rowSize;
int32_t code = sdbWrite(pOper, pHead, pHead->msgType);
taosFreeQitem(pHead);
if (code < 0) return code;
int32_t code = sdbInsertHash(pTable, pOper);
if (code != TSDB_CODE_SUCCESS) {
sdbError("table:%s, failed to insert into hash", pTable->tableName);
return code;
}
return sdbInsertHash(pTable, pOper);
// just insert data into memory
if (pOper->type != SDB_OPER_GLOBAL) {
return TSDB_CODE_SUCCESS;
}
int32_t size = sizeof(SSdbOper) + sizeof(SWalHead) + pTable->maxRowSize + SDB_SYNC_HACK;
SSdbOper *pNewOper = taosAllocateQitem(size);
SWalHead *pHead = (void *)pNewOper + sizeof(SSdbOper) + SDB_SYNC_HACK;
pHead->version = 0;
pHead->len = pOper->rowSize;
pHead->msgType = pTable->tableId * 10 + SDB_ACTION_INSERT;
pOper->rowData = pHead->cont;
(*pTable->encodeFp)(pOper);
pHead->len = pOper->rowSize;
memcpy(pNewOper, pOper, sizeof(SSdbOper));
if (pNewOper->pMsg != NULL) {
sdbTrace("app:%p:%p, insert action is add to sdb queue", pNewOper->pMsg->rpcMsg.ahandle, pNewOper->pMsg);
}
taosWriteQitem(tsSdbWriteQueue, TAOS_QTYPE_RPC, pNewOper);
return TSDB_CODE_SUCCESS;
}
int32_t sdbDeleteRow(SSdbOper *pOper) {
SSdbTable *pTable = (SSdbTable *)pOper->table;
if (pTable == NULL) return -1;
if (pTable == NULL) return TSDB_CODE_MND_SDB_INVALID_TABLE_TYPE;
SSdbRow *pMeta = sdbGetRowMetaFromObj(pTable, pOper->pObj);
if (pMeta == NULL) {
sdbTrace("table:%s, record is not there, delete failed", pTable->tableName);
return -1;
return TSDB_CODE_MND_SDB_OBJ_NOT_THERE;
}
void * pMetaRow = pMeta->row;
assert(pMetaRow != NULL);
if (pOper->type == SDB_OPER_GLOBAL) {
void * key = sdbGetObjKey(pTable, pOper->pObj);
int32_t keySize = 0;
switch (pTable->keyType) {
case SDB_KEY_STRING:
case SDB_KEY_VAR_STRING:
keySize = strlen((char *)key) + 1;
break;
case SDB_KEY_INT:
case SDB_KEY_AUTO:
keySize = sizeof(uint32_t);
break;
default:
return -1;
}
int32_t size = sizeof(SWalHead) + keySize;
SWalHead *pHead = taosAllocateQitem(size);
pHead->version = 0;
pHead->len = keySize;
pHead->msgType = pTable->tableId * 10 + SDB_ACTION_DELETE;
memcpy(pHead->cont, key, keySize);
int32_t code = sdbWrite(pOper, pHead, pHead->msgType);
taosFreeQitem(pHead);
if (code < 0) return code;
void *pMetaRow = pMeta->row;
if (pMetaRow == NULL) {
sdbError("table:%s, record meta is null", pTable->tableName);
return TSDB_CODE_MND_SDB_INVAID_META_ROW;
}
return sdbDeleteHash(pTable, pOper);
int32_t code = sdbDeleteHash(pTable, pOper);
if (code != TSDB_CODE_SUCCESS) {
sdbError("table:%s, failed to delete from hash", pTable->tableName);
return code;
}
// just delete data from memory
if (pOper->type != SDB_OPER_GLOBAL) {
return TSDB_CODE_SUCCESS;
}
int32_t size = sizeof(SSdbOper) + sizeof(SWalHead) + pTable->maxRowSize + SDB_SYNC_HACK;
SSdbOper *pNewOper = taosAllocateQitem(size);
SWalHead *pHead = (void *)pNewOper + sizeof(SSdbOper) + SDB_SYNC_HACK;
pHead->version = 0;
pHead->msgType = pTable->tableId * 10 + SDB_ACTION_DELETE;
pOper->rowData = pHead->cont;
(*pTable->encodeFp)(pOper);
pHead->len = pOper->rowSize;
memcpy(pNewOper, pOper, sizeof(SSdbOper));
if (pNewOper->pMsg != NULL) {
sdbTrace("app:%p:%p, delete action is add to sdb queue", pNewOper->pMsg->rpcMsg.ahandle, pNewOper->pMsg);
}
taosWriteQitem(tsSdbWriteQueue, TAOS_QTYPE_RPC, pNewOper);
return TSDB_CODE_SUCCESS;
}
int32_t sdbUpdateRow(SSdbOper *pOper) {
SSdbTable *pTable = (SSdbTable *)pOper->table;
if (pTable == NULL) return -1;
if (pTable == NULL) return TSDB_CODE_MND_SDB_INVALID_TABLE_TYPE;
SSdbRow *pMeta = sdbGetRowMetaFromObj(pTable, pOper->pObj);
if (pMeta == NULL) {
sdbTrace("table:%s, record is not there, delete failed", pTable->tableName);
return -1;
sdbTrace("table:%s, record is not there, update failed", pTable->tableName);
return TSDB_CODE_MND_SDB_OBJ_NOT_THERE;
}
void * pMetaRow = pMeta->row;
assert(pMetaRow != NULL);
void *pMetaRow = pMeta->row;
if (pMetaRow == NULL) {
sdbError("table:%s, record meta is null", pTable->tableName);
return TSDB_CODE_MND_SDB_INVAID_META_ROW;
}
if (pOper->type == SDB_OPER_GLOBAL) {
int32_t size = sizeof(SWalHead) + pTable->maxRowSize;
SWalHead *pHead = taosAllocateQitem(size);
pHead->version = 0;
pHead->msgType = pTable->tableId * 10 + SDB_ACTION_UPDATE;
int32_t code = sdbUpdateHash(pTable, pOper);
if (code != TSDB_CODE_SUCCESS) {
sdbError("table:%s, failed to update hash", pTable->tableName);
return code;
}
pOper->rowData = pHead->cont;
(*pTable->encodeFp)(pOper);
pHead->len = pOper->rowSize;
// just update data in memory
if (pOper->type != SDB_OPER_GLOBAL) {
return TSDB_CODE_SUCCESS;
}
int32_t code = sdbWrite(pOper, pHead, pHead->msgType);
taosFreeQitem(pHead);
if (code < 0) return code;
}
return sdbUpdateHash(pTable, pOper);
int32_t size = sizeof(SSdbOper) + sizeof(SWalHead) + pTable->maxRowSize + SDB_SYNC_HACK;
SSdbOper *pNewOper = taosAllocateQitem(size);
SWalHead *pHead = (void *)pNewOper + sizeof(SSdbOper) + SDB_SYNC_HACK;
pHead->version = 0;
pHead->msgType = pTable->tableId * 10 + SDB_ACTION_UPDATE;
pOper->rowData = pHead->cont;
(*pTable->encodeFp)(pOper);
pHead->len = pOper->rowSize;
memcpy(pNewOper, pOper, sizeof(SSdbOper));
if (pNewOper->pMsg != NULL) {
sdbTrace("app:%p:%p, update action is add to sdb queue", pNewOper->pMsg->rpcMsg.ahandle, pNewOper->pMsg);
}
taosWriteQitem(tsSdbWriteQueue, TAOS_QTYPE_RPC, pNewOper);
return TSDB_CODE_SUCCESS;
}
void *sdbFetchRow(void *handle, void *pNode, void **ppRow) {
@ -775,3 +841,158 @@ void sdbCloseTable(void *handle) {
free(pTable);
}
int32_t sdbInitWriteWorker() {
tsSdbPool.num = 1;
tsSdbPool.writeWorker = (SSdbWriteWorker *)calloc(sizeof(SSdbWriteWorker), tsSdbPool.num);
if (tsSdbPool.writeWorker == NULL) return -1;
for (int32_t i = 0; i < tsSdbPool.num; ++i) {
SSdbWriteWorker *pWorker = tsSdbPool.writeWorker + i;
pWorker->workerId = i;
}
sdbAllocWriteQueue();
mPrint("sdb write is opened");
return 0;
}
void sdbCleanupWriteWorker() {
for (int32_t i = 0; i < tsSdbPool.num; ++i) {
SSdbWriteWorker *pWorker = tsSdbPool.writeWorker + i;
if (pWorker->thread) {
taosQsetThreadResume(tsSdbWriteQset);
}
}
for (int32_t i = 0; i < tsSdbPool.num; ++i) {
SSdbWriteWorker *pWorker = tsSdbPool.writeWorker + i;
if (pWorker->thread) {
pthread_join(pWorker->thread, NULL);
}
}
sdbFreeWritequeue();
mPrint("sdb write is closed");
}
int32_t sdbAllocWriteQueue() {
tsSdbWriteQueue = taosOpenQueue();
if (tsSdbWriteQueue == NULL) return TSDB_CODE_MND_OUT_OF_MEMORY;
tsSdbWriteQset = taosOpenQset();
if (tsSdbWriteQset == NULL) {
taosCloseQueue(tsSdbWriteQueue);
return TSDB_CODE_MND_OUT_OF_MEMORY;
}
taosAddIntoQset(tsSdbWriteQset, tsSdbWriteQueue, NULL);
tsSdbWriteQall = taosAllocateQall();
if (tsSdbWriteQall == NULL) {
taosCloseQset(tsSdbWriteQset);
taosCloseQueue(tsSdbWriteQueue);
return TSDB_CODE_MND_OUT_OF_MEMORY;
}
for (int32_t i = 0; i < tsSdbPool.num; ++i) {
SSdbWriteWorker *pWorker = tsSdbPool.writeWorker + i;
pWorker->workerId = i;
pthread_attr_t thAttr;
pthread_attr_init(&thAttr);
pthread_attr_setdetachstate(&thAttr, PTHREAD_CREATE_JOINABLE);
if (pthread_create(&pWorker->thread, &thAttr, sdbWorkerFp, pWorker) != 0) {
mError("failed to create thread to process sdb write queue, reason:%s", strerror(errno));
taosFreeQall(tsSdbWriteQall);
taosCloseQset(tsSdbWriteQset);
taosCloseQueue(tsSdbWriteQueue);
return TSDB_CODE_MND_OUT_OF_MEMORY;
}
pthread_attr_destroy(&thAttr);
mTrace("sdb write worker:%d is launched, total:%d", pWorker->workerId, tsSdbPool.num);
}
mTrace("sdb write queue:%p is allocated", tsSdbWriteQueue);
return TSDB_CODE_SUCCESS;
}
void sdbFreeWritequeue() {
taosCloseQset(tsSdbWriteQueue);
taosFreeQall(tsSdbWriteQall);
taosCloseQset(tsSdbWriteQset);
tsSdbWriteQall = NULL;
tsSdbWriteQset = NULL;
tsSdbWriteQueue = NULL;
}
int sdbWriteToQueue(void *param, void *data, int type) {
SWalHead *pHead = data;
int size = sizeof(SWalHead) + pHead->len;
SWalHead *pWal = (SWalHead *)taosAllocateQitem(size);
memcpy(pWal, pHead, size);
taosWriteQitem(tsSdbWriteQueue, type, pWal);
return 0;
}
static void *sdbWorkerFp(void *param) {
SWalHead *pHead;
SSdbOper *pOper;
int32_t type;
int32_t numOfMsgs;
void * item;
void * unUsed;
while (1) {
numOfMsgs = taosReadAllQitemsFromQset(tsSdbWriteQset, tsSdbWriteQall, &unUsed);
if (numOfMsgs == 0) {
sdbTrace("sdbWorkerFp: got no message from qset, exiting...");
break;
}
for (int32_t i = 0; i < numOfMsgs; ++i) {
taosGetQitem(tsSdbWriteQall, &type, &item);
if (type == TAOS_QTYPE_RPC) {
pOper = (SSdbOper *)item;
pHead = (void *)pOper + sizeof(SSdbOper) + SDB_SYNC_HACK;
} else {
pHead = (SWalHead *)item;
pOper = NULL;
}
if (pOper != NULL && pOper->pMsg != NULL) {
sdbTrace("app:%p:%p, will be processed in sdb queue", pOper->pMsg->rpcMsg.ahandle, pOper->pMsg);
}
int32_t code = sdbWrite(pOper, pHead, type);
if (pOper) pOper->retCode = code;
}
walFsync(tsSdbObj.wal);
// browse all items, and process them one by one
taosResetQitems(tsSdbWriteQall);
for (int32_t i = 0; i < numOfMsgs; ++i) {
taosGetQitem(tsSdbWriteQall, &type, &item);
if (type == TAOS_QTYPE_RPC) {
pOper = (SSdbOper *)item;
if (pOper != NULL && pOper->cb != NULL) {
pOper->retCode = (*pOper->cb)(pOper->pMsg, pOper->retCode);
}
if (pOper != NULL && pOper->pMsg != NULL) {
sdbTrace("app:%p:%p, msg is processed, result:%s", pOper->pMsg->rpcMsg.ahandle, pOper->pMsg,
tstrerror(pOper->retCode));
}
dnodeSendRpcMnodeWriteRsp(pOper->pMsg, pOper->retCode);
}
taosFreeQitem(item);
}
}
return NULL;
}

14
src/mnode/src/mnodeShow.c
View File

@ -65,7 +65,7 @@ int32_t mnodeInitShow() {
mnodeAddReadMsgHandle(TSDB_MSG_TYPE_CM_CONNECT, mnodeProcessConnectMsg);
mnodeAddReadMsgHandle(TSDB_MSG_TYPE_CM_USE_DB, mnodeProcessUseMsg);
tsMnodeShowCache = taosCacheInitWithCb(10, mnodeFreeShowObj);
tsMnodeShowCache = taosCacheInitWithCb(5, mnodeFreeShowObj);
return 0;
}
@ -122,7 +122,7 @@ static int32_t mnodeProcessShowMsg(SMnodeMsg *pMsg) {
SShowObj *pShow = (SShowObj *) calloc(1, showObjSize);
pShow->type = pShowMsg->type;
pShow->payloadLen = htons(pShowMsg->payloadLen);
strcpy(pShow->db, pShowMsg->db);
tstrncpy(pShow->db, pShowMsg->db, TSDB_DB_NAME_LEN);
memcpy(pShow->payload, pShowMsg->payload, pShow->payloadLen);
pShow = mnodePutShowObj(pShow, showObjSize);
@ -139,7 +139,7 @@ static int32_t mnodeProcessShowMsg(SMnodeMsg *pMsg) {
pShowRsp->qhandle = htobe64((uint64_t) pShow);
int32_t code = (*tsMnodeShowMetaFp[pShowMsg->type])(&pShowRsp->tableMeta, pShow, pMsg->rpcMsg.handle);
mTrace("%p, show type:%s index:%d, get meta finished, rows:%d cols:%d result:%s", pShow,
mTrace("%p, show type:%s index:%d, get meta finished, numOfRows:%d cols:%d result:%s", pShow,
mnodeGetShowType(pShowMsg->type), pShow->index, pShow->numOfRows, pShow->numOfColumns, tstrerror(code));
if (code == TSDB_CODE_SUCCESS) {
@ -179,7 +179,7 @@ static int32_t mnodeProcessRetrieveMsg(SMnodeMsg *pMsg) {
mTrace("%p, show is already read finished, numOfReads:%d numOfRows:%d", pShow, pShow->numOfReads, pShow->numOfRows);
pShow->numOfReads = pShow->numOfRows;
}
if ((pRetrieve->free & TSDB_QUERY_TYPE_FREE_RESOURCE) != TSDB_QUERY_TYPE_FREE_RESOURCE) {
rowsToRead = pShow->numOfRows - pShow->numOfReads;
}
@ -219,8 +219,10 @@ static int32_t mnodeProcessRetrieveMsg(SMnodeMsg *pMsg) {
if (rowsToRead == 0 || (rowsRead == rowsToRead && pShow->numOfRows == pShow->numOfReads)) {
pRsp->completed = 1;
mTrace("%p, retrieve completed", pShow);
mnodeReleaseShowObj(pShow, true);
} else {
mTrace("%p, retrieve not completed yet", pShow);
mnodeReleaseShowObj(pShow, false);
}
@ -379,10 +381,10 @@ static void *mnodePutShowObj(SShowObj *pShow, int32_t size) {
pShow->index = atomic_add_fetch_32(&tsShowObjIndex, 1);
sprintf(key, "%d", pShow->index);
SShowObj *newQhandle = taosCachePut(tsMnodeShowCache, key, pShow, size, 60);
SShowObj *newQhandle = taosCachePut(tsMnodeShowCache, key, pShow, size, 6);
free(pShow);
mTrace("%p, show is put into cache", newQhandle);
mTrace("%p, show is put into cache, index:%s", newQhandle, key);
return newQhandle;
}

686
src/mnode/src/mnodeTable.c
View File

File diff suppressed because it is too large Load Diff

104
src/mnode/src/mnodeUser.c
View File

@ -102,11 +102,13 @@ static int32_t mnodeUserActionDecode(SSdbOper *pOper) {
}
static int32_t mnodeUserActionRestored() {
if (dnodeIsFirstDeploy()) {
SAcctObj *pAcct = mnodeGetAcct("root");
mnodeCreateUser(pAcct, "root", "taosdata");
mnodeCreateUser(pAcct, "monitor", tsInternalPass);
mnodeCreateUser(pAcct, "_root", tsInternalPass);
int32_t numOfRows = sdbGetNumOfRows(tsUserSdb);
if (numOfRows <= 0 && dnodeIsFirstDeploy()) {
mPrint("dnode first deploy, create root user");
SAcctObj *pAcct = mnodeGetAcct(TSDB_DEFAULT_USER);
mnodeCreateUser(pAcct, TSDB_DEFAULT_USER, TSDB_DEFAULT_PASS, NULL);
mnodeCreateUser(pAcct, "monitor", tsInternalPass, NULL);
mnodeCreateUser(pAcct, "_"TSDB_DEFAULT_USER, tsInternalPass, NULL);
mnodeDecAcctRef(pAcct);
}
@ -170,22 +172,24 @@ void mnodeDecUserRef(SUserObj *pUser) {
return sdbDecRef(tsUserSdb, pUser);
}
static int32_t mnodeUpdateUser(SUserObj *pUser) {
static int32_t mnodeUpdateUser(SUserObj *pUser, void *pMsg) {
SSdbOper oper = {
.type = SDB_OPER_GLOBAL,
.type = SDB_OPER_GLOBAL,
.table = tsUserSdb,
.pObj = pUser
.pObj = pUser,
.pMsg = pMsg
};
int32_t code = sdbUpdateRow(&oper);
if (code != TSDB_CODE_SUCCESS) {
code = TSDB_CODE_MND_SDB_ERROR;
if (code == TSDB_CODE_SUCCESS) {
mLPrint("user:%s, is altered by %s", pUser->user, mnodeGetUserFromMsg(pMsg));
if (pMsg != NULL) code = TSDB_CODE_MND_ACTION_IN_PROGRESS;
}
return code;
}
int32_t mnodeCreateUser(SAcctObj *pAcct, char *name, char *pass) {
int32_t mnodeCreateUser(SAcctObj *pAcct, char *name, char *pass, void *pMsg) {
int32_t code = acctCheck(pAcct, ACCT_GRANT_USER);
if (code != TSDB_CODE_SUCCESS) {
return code;
@ -212,42 +216,47 @@ int32_t mnodeCreateUser(SAcctObj *pAcct, char *name, char *pass) {
}
pUser = calloc(1, sizeof(SUserObj));
strcpy(pUser->user, name);
tstrncpy(pUser->user, name, TSDB_USER_LEN);
taosEncryptPass((uint8_t*) pass, strlen(pass), pUser->pass);
strcpy(pUser->acct, pAcct->user);
pUser->createdTime = taosGetTimestampMs();
pUser->superAuth = 0;
pUser->writeAuth = 1;
if (strcmp(pUser->user, "root") == 0 || strcmp(pUser->user, pUser->acct) == 0) {
if (strcmp(pUser->user, TSDB_DEFAULT_USER) == 0 || strcmp(pUser->user, pUser->acct) == 0) {
pUser->superAuth = 1;
}
SSdbOper oper = {
.type = SDB_OPER_GLOBAL,
.table = tsUserSdb,
.pObj = pUser,
.rowSize = sizeof(SUserObj)
.type = SDB_OPER_GLOBAL,
.table = tsUserSdb,
.pObj = pUser,
.rowSize = sizeof(SUserObj),
.pMsg = pMsg
};
code = sdbInsertRow(&oper);
if (code != TSDB_CODE_SUCCESS) {
tfree(pUser);
code = TSDB_CODE_MND_SDB_ERROR;
} else {
mLPrint("user:%s, is created by %s", pUser->user, mnodeGetUserFromMsg(pMsg));
if (pMsg != NULL) code = TSDB_CODE_MND_ACTION_IN_PROGRESS;
}
return code;
}
static int32_t mnodeDropUser(SUserObj *pUser) {
static int32_t mnodeDropUser(SUserObj *pUser, void *pMsg) {
SSdbOper oper = {
.type = SDB_OPER_GLOBAL,
.type = SDB_OPER_GLOBAL,
.table = tsUserSdb,
.pObj = pUser
.pObj = pUser,
.pMsg = pMsg
};
int32_t code = sdbDeleteRow(&oper);
if (code != TSDB_CODE_SUCCESS) {
code = TSDB_CODE_MND_SDB_ERROR;
if (code == TSDB_CODE_SUCCESS) {
mLPrint("user:%s, is dropped by %s", pUser->user, mnodeGetUserFromMsg(pMsg));
if (pMsg != NULL) code = TSDB_CODE_MND_ACTION_IN_PROGRESS;
}
return code;
@ -315,8 +324,7 @@ static int32_t mnodeRetrieveUsers(SShowObj *pShow, char *data, int32_t rows, voi
cols = 0;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
size_t size = sizeof(pUser->user);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pUser->user, size);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pUser->user, pShow->bytes[cols]);
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
@ -337,7 +345,7 @@ static int32_t mnodeRetrieveUsers(SShowObj *pShow, char *data, int32_t rows, voi
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pUser->acct, sizeof(pUser->user));
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pUser->acct, pShow->bytes[cols]);
cols++;
numOfRows++;
@ -358,22 +366,25 @@ SUserObj *mnodeGetUserFromConn(void *pConn) {
}
}
char *mnodeGetUserFromMsg(void *pMsg) {
SMnodeMsg *pMnodeMsg = pMsg;
if (pMnodeMsg != NULL && pMnodeMsg->pUser != NULL) {
return pMnodeMsg->pUser->user;
} else {
return "system";
}
}
static int32_t mnodeProcessCreateUserMsg(SMnodeMsg *pMsg) {
int32_t code;
SUserObj *pOperUser = pMsg->pUser;
if (pOperUser->superAuth) {
SCMCreateUserMsg *pCreate = pMsg->rpcMsg.pCont;
code = mnodeCreateUser(pOperUser->pAcct, pCreate->user, pCreate->pass);
if (code == TSDB_CODE_SUCCESS) {
mLPrint("user:%s, is created by %s", pCreate->user, pOperUser->user);
}
return mnodeCreateUser(pOperUser->pAcct, pCreate->user, pCreate->pass, pMsg);
} else {
mError("user:%s, no rights to create user", pOperUser->user);
code = TSDB_CODE_MND_NO_RIGHTS;
return TSDB_CODE_MND_NO_RIGHTS;
}
return code;
}
static int32_t mnodeProcessAlterUserMsg(SMnodeMsg *pMsg) {
@ -393,12 +404,12 @@ static int32_t mnodeProcessAlterUserMsg(SMnodeMsg *pMsg) {
if ((pAlter->flag & TSDB_ALTER_USER_PASSWD) != 0) {
bool hasRight = false;
if (strcmp(pOperUser->user, "root") == 0) {
if (strcmp(pOperUser->user, TSDB_DEFAULT_USER) == 0) {
hasRight = true;
} else if (strcmp(pUser->user, pOperUser->user) == 0) {
hasRight = true;
} else if (pOperUser->superAuth) {
if (strcmp(pUser->user, "root") == 0) {
if (strcmp(pUser->user, TSDB_DEFAULT_USER) == 0) {
hasRight = false;
} else if (strcmp(pOperUser->acct, pUser->acct) != 0) {
hasRight = false;
@ -410,8 +421,7 @@ static int32_t mnodeProcessAlterUserMsg(SMnodeMsg *pMsg) {
if (hasRight) {
memset(pUser->pass, 0, sizeof(pUser->pass));
taosEncryptPass((uint8_t*)pAlter->pass, strlen(pAlter->pass), pUser->pass);
code = mnodeUpdateUser(pUser);
mLPrint("user:%s, password is altered by %s, result:%s", pUser->user, pOperUser->user, tstrerror(code));
code = mnodeUpdateUser(pUser, pMsg);
} else {
mError("user:%s, no rights to alter user", pOperUser->user);
code = TSDB_CODE_MND_NO_RIGHTS;
@ -419,16 +429,16 @@ static int32_t mnodeProcessAlterUserMsg(SMnodeMsg *pMsg) {
} else if ((pAlter->flag & TSDB_ALTER_USER_PRIVILEGES) != 0) {
bool hasRight = false;
if (strcmp(pUser->user, "root") == 0) {
if (strcmp(pUser->user, TSDB_DEFAULT_USER) == 0) {
hasRight = false;
} else if (strcmp(pUser->user, pUser->acct) == 0) {
hasRight = false;
} else if (strcmp(pOperUser->user, "root") == 0) {
} else if (strcmp(pOperUser->user, TSDB_DEFAULT_USER) == 0) {
hasRight = true;
} else if (strcmp(pUser->user, pOperUser->user) == 0) {
hasRight = false;
} else if (pOperUser->superAuth) {
if (strcmp(pUser->user, "root") == 0) {
if (strcmp(pUser->user, TSDB_DEFAULT_USER) == 0) {
hasRight = false;
} else if (strcmp(pOperUser->acct, pUser->acct) != 0) {
hasRight = false;
@ -451,8 +461,7 @@ static int32_t mnodeProcessAlterUserMsg(SMnodeMsg *pMsg) {
pUser->writeAuth = 1;
}
code = mnodeUpdateUser(pUser);
mLPrint("user:%s, privilege is altered by %s, result:%s", pUser->user, pOperUser->user, tstrerror(code));
code = mnodeUpdateUser(pUser, pMsg);
} else {
mError("user:%s, no rights to alter user", pOperUser->user);
code = TSDB_CODE_MND_NO_RIGHTS;
@ -483,9 +492,9 @@ static int32_t mnodeProcessDropUserMsg(SMnodeMsg *pMsg) {
}
bool hasRight = false;
if (strcmp(pUser->user, "root") == 0) {
if (strcmp(pUser->user, TSDB_DEFAULT_USER) == 0) {
hasRight = false;
} else if (strcmp(pOperUser->user, "root") == 0) {
} else if (strcmp(pOperUser->user, TSDB_DEFAULT_USER) == 0) {
hasRight = true;
} else if (strcmp(pUser->user, pOperUser->user) == 0) {
hasRight = false;
@ -498,10 +507,7 @@ static int32_t mnodeProcessDropUserMsg(SMnodeMsg *pMsg) {
}
if (hasRight) {
code = mnodeDropUser(pUser);
if (code == TSDB_CODE_SUCCESS) {
mLPrint("user:%s, is dropped by %s, result:%s", pUser->user, pOperUser->user, tstrerror(code));
}
code = mnodeDropUser(pUser, pMsg);
} else {
code = TSDB_CODE_MND_NO_RIGHTS;
}

105
src/mnode/src/mnodeVgroup.c
View File

@ -74,6 +74,7 @@ static int32_t mnodeVgroupActionInsert(SSdbOper *pOper) {
pVgroup->pDb = pDb;
pVgroup->prev = NULL;
pVgroup->next = NULL;
pVgroup->accessState = TSDB_VN_ALL_ACCCESS;
int32_t size = sizeof(SChildTableObj *) * pDb->cfg.maxTables;
pVgroup->tableList = calloc(pDb->cfg.maxTables, sizeof(SChildTableObj *));
@ -132,7 +133,7 @@ static void mnodeVgroupUpdateIdPool(SVgObj *pVgroup) {
taosUpdateIdPool(pVgroup->idPool, pDb->cfg.maxTables);
int32_t size = sizeof(SChildTableObj *) * pDb->cfg.maxTables;
pVgroup->tableList = (SChildTableObj **)realloc(pVgroup->tableList, size);
memset(pVgroup->tableList + oldTables, 0, (pDb->cfg.maxTables - oldTables) * sizeof(SChildTableObj **));
memset(pVgroup->tableList + oldTables, 0, (pDb->cfg.maxTables - oldTables) * sizeof(SChildTableObj *));
}
}
}
@ -149,8 +150,7 @@ static int32_t mnodeVgroupActionUpdate(SSdbOper *pOper) {
}
}
memcpy(pVgroup, pNew, pOper->rowSize);
free(pNew);
memcpy(pVgroup, pNew, tsVgUpdateSize);
for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) {
SDnodeObj *pDnode = mnodeGetDnode(pVgroup->vnodeGid[i].dnodeId);
@ -252,10 +252,44 @@ void mnodeUpdateVgroup(SVgObj *pVgroup) {
.pObj = pVgroup
};
sdbUpdateRow(&oper);
if (sdbUpdateRow(&oper) != TSDB_CODE_SUCCESS) {
mError("vgId:%d, failed to update vgroup", pVgroup->vgId);
}
mnodeSendCreateVgroupMsg(pVgroup, NULL);
}
/*
Traverse all vgroups on mnode, if there no such vgId on a dnode, so send msg to this dnode for re-creating this vgId/vnode
*/
void mnodeCheckUnCreatedVgroup(SDnodeObj *pDnode, SVnodeLoad *pVloads, int32_t openVnodes) {
SVnodeLoad *pNextV = NULL;
void *pIter = NULL;
while (1) {
SVgObj *pVgroup;
pIter = mnodeGetNextVgroup(pIter, &pVgroup);
if (pVgroup == NULL) break;
pNextV = pVloads;
int32_t i;
for (i = 0; i < openVnodes; ++i) {
if ((pVgroup->vnodeGid[i].pDnode == pDnode) && (pVgroup->vgId == pNextV->vgId)) {
break;
}
pNextV++;
}
if (i == openVnodes) {
mnodeSendCreateVgroupMsg(pVgroup, NULL);
}
mnodeDecVgroupRef(pVgroup);
}
sdbFreeIter(pIter);
return;
}
void mnodeUpdateVgroupStatus(SVgObj *pVgroup, SDnodeObj *pDnode, SVnodeLoad *pVload) {
bool dnodeExist = false;
for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) {
@ -299,11 +333,35 @@ void *mnodeGetNextVgroup(void *pIter, SVgObj **pVgroup) {
return sdbFetchRow(tsVgroupSdb, pIter, (void **)pVgroup);
}
static int32_t mnodeCreateVgroupCb(SMnodeMsg *pMsg, int32_t code) {
if (code != TSDB_CODE_SUCCESS) {
pMsg->pVgroup = NULL;
return code;
}
SVgObj *pVgroup = pMsg->pVgroup;
SDbObj *pDb = pMsg->pDb;
mPrint("vgId:%d, is created in mnode, db:%s replica:%d", pVgroup->vgId, pDb->name, pVgroup->numOfVnodes);
for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) {
mPrint("vgId:%d, index:%d, dnode:%d", pVgroup->vgId, i, pVgroup->vnodeGid[i].dnodeId);
}
mnodeIncVgroupRef(pVgroup);
pMsg->expected = pVgroup->numOfVnodes;
mnodeSendCreateVgroupMsg(pVgroup, pMsg);
return TSDB_CODE_MND_ACTION_IN_PROGRESS;
}
int32_t mnodeCreateVgroup(SMnodeMsg *pMsg, SDbObj *pDb) {
if (pMsg == NULL) return TSDB_CODE_MND_APP_ERROR;
SVgObj *pVgroup = (SVgObj *)calloc(1, sizeof(SVgObj));
strcpy(pVgroup->dbName, pDb->name);
tstrncpy(pVgroup->dbName, pDb->name, TSDB_DB_NAME_LEN);
pVgroup->numOfVnodes = pDb->cfg.replications;
pVgroup->createdTime = taosGetTimestampMs();
pVgroup->accessState = TSDB_VN_ALL_ACCCESS;
if (balanceAllocVnodes(pVgroup) != 0) {
mError("db:%s, no enough dnode to alloc %d vnodes to vgroup", pDb->name, pVgroup->numOfVnodes);
free(pVgroup);
@ -314,26 +372,22 @@ int32_t mnodeCreateVgroup(SMnodeMsg *pMsg, SDbObj *pDb) {
.type = SDB_OPER_GLOBAL,
.table = tsVgroupSdb,
.pObj = pVgroup,
.rowSize = sizeof(SVgObj)
.rowSize = sizeof(SVgObj),
.pMsg = pMsg,
.cb = mnodeCreateVgroupCb
};
pMsg->pVgroup = pVgroup;
int32_t code = sdbInsertRow(&oper);
if (code != TSDB_CODE_SUCCESS) {
pMsg->pVgroup = NULL;
tfree(pVgroup);
return TSDB_CODE_MND_SDB_ERROR;
} else {
code = TSDB_CODE_MND_ACTION_IN_PROGRESS;
}
mPrint("vgId:%d, is created in mnode, db:%s replica:%d", pVgroup->vgId, pDb->name, pVgroup->numOfVnodes);
for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) {
mPrint("vgId:%d, index:%d, dnode:%d", pVgroup->vgId, i, pVgroup->vnodeGid[i].dnodeId);
}
mnodeIncVgroupRef(pVgroup);
pMsg->pVgroup = pVgroup;
pMsg->expected = pVgroup->numOfVnodes;
mnodeSendCreateVgroupMsg(pVgroup, pMsg);
return TSDB_CODE_MND_ACTION_IN_PROGRESS;
return code;
}
void mnodeDropVgroup(SVgObj *pVgroup, void *ahandle) {
@ -479,12 +533,12 @@ int32_t mnodeRetrieveVgroups(SShowObj *pShow, char *data, int32_t rows, void *pC
if (pDnode != NULL) {
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pDnode->dnodeEp, pShow->bytes[cols] - VARSTR_HEADER_SIZE);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pDnode->dnodeEp, pShow->bytes[cols]);
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
char *role = mnodeGetMnodeRoleStr(pVgroup->vnodeGid[i].role);
STR_TO_VARSTR(pWrite, role);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, role, pShow->bytes[cols]);
cols++;
} else {
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
@ -596,7 +650,6 @@ SRpcIpSet mnodeGetIpSetFromIp(char *ep) {
}
void mnodeSendCreateVnodeMsg(SVgObj *pVgroup, SRpcIpSet *ipSet, void *ahandle) {
mTrace("vgId:%d, send create vnode:%d msg, ahandle:%p db:%s", pVgroup->vgId, pVgroup->vgId, ahandle, pVgroup->dbName);
SMDCreateVnodeMsg *pCreate = mnodeBuildCreateVnodeMsg(pVgroup);
SRpcMsg rpcMsg = {
.handle = ahandle,
@ -609,9 +662,12 @@ void mnodeSendCreateVnodeMsg(SVgObj *pVgroup, SRpcIpSet *ipSet, void *ahandle) {
}
void mnodeSendCreateVgroupMsg(SVgObj *pVgroup, void *ahandle) {
mTrace("vgId:%d, send create all vnodes msg, ahandle:%p", pVgroup->vgId, ahandle);
mTrace("vgId:%d, send create all vnodes msg, numOfVnodes:%d db:%s", pVgroup->vgId, pVgroup->numOfVnodes,
pVgroup->dbName);
for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) {
SRpcIpSet ipSet = mnodeGetIpSetFromIp(pVgroup->vnodeGid[i].pDnode->dnodeEp);
mTrace("vgId:%d, index:%d, send create vnode msg to dnode %s, ahandle:%p", pVgroup->vgId,
i, pVgroup->vnodeGid[i].pDnode->dnodeEp, ahandle);
mnodeSendCreateVnodeMsg(pVgroup, &ipSet, ahandle);
}
}
@ -717,7 +773,7 @@ static int32_t mnodeProcessVnodeCfgMsg(SMnodeMsg *pMsg) {
SDnodeObj *pDnode = mnodeGetDnode(pCfg->dnodeId);
if (pDnode == NULL) {
mTrace("dnode:%s, invalid dnode", taosIpStr(pCfg->dnodeId), pCfg->vgId);
mTrace("dnode:%s, vgId:%d, invalid dnode", taosIpStr(pCfg->dnodeId), pCfg->vgId);
return TSDB_CODE_MND_VGROUP_NOT_EXIST;
}
mnodeDecDnodeRef(pDnode);
@ -729,6 +785,7 @@ static int32_t mnodeProcessVnodeCfgMsg(SMnodeMsg *pMsg) {
}
mnodeDecVgroupRef(pVgroup);
mTrace("vgId:%d, send create vnode msg to dnode %s for vnode cfg msg", pVgroup->vgId, pDnode->dnodeEp);
SRpcIpSet ipSet = mnodeGetIpSetFromIp(pDnode->dnodeEp);
mnodeSendCreateVnodeMsg(pVgroup, &ipSet, NULL);
@ -834,4 +891,4 @@ void mnodeSendDropAllDbVgroupsMsg(SDbObj *pDropDb) {
sdbFreeIter(pIter);
mPrint("db:%s, all vgroups:%d drop msg is sent to dnode", pDropDb->name, numOfVgroups);
}
}

16
src/mnode/src/mnodeWrite.c
View File

@ -43,7 +43,7 @@ void mnodeAddWriteMsgHandle(uint8_t msgType, int32_t (*fp)(SMnodeMsg *mnodeMsg))
int32_t mnodeProcessWrite(SMnodeMsg *pMsg) {
if (pMsg->rpcMsg.pCont == NULL) {
mError("%p, msg:%s in mwrite queue, content is null", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType]);
mError("app:%p:%p, msg:%s content is null", pMsg->rpcMsg.ahandle, pMsg, taosMsg[pMsg->rpcMsg.msgType]);
return TSDB_CODE_MND_INVALID_MSG_LEN;
}
@ -54,27 +54,31 @@ int32_t mnodeProcessWrite(SMnodeMsg *pMsg) {
rpcRsp->rsp = ipSet;
rpcRsp->len = sizeof(SRpcIpSet);
mTrace("%p, msg:%s in mwrite queue, will be redireced inUse:%d", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType], ipSet->inUse);
mTrace("app:%p:%p, msg:%s will be redireced inUse:%d", pMsg->rpcMsg.ahandle, pMsg, taosMsg[pMsg->rpcMsg.msgType],
ipSet->inUse);
for (int32_t i = 0; i < ipSet->numOfIps; ++i) {
mTrace("mnode index:%d ip:%s:%d", i, ipSet->fqdn[i], htons(ipSet->port[i]));
mTrace("app:%p:%p, mnode index:%d ip:%s:%d", pMsg->rpcMsg.ahandle, pMsg, i, ipSet->fqdn[i],
htons(ipSet->port[i]));
}
return TSDB_CODE_RPC_REDIRECT;
}
if (tsMnodeProcessWriteMsgFp[pMsg->rpcMsg.msgType] == NULL) {
mError("%p, msg:%s in mwrite queue, not processed", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType]);
mError("app:%p:%p, msg:%s not processed", pMsg->rpcMsg.ahandle, pMsg, taosMsg[pMsg->rpcMsg.msgType]);
return TSDB_CODE_MND_MSG_NOT_PROCESSED;
}
int32_t code = mnodeInitMsg(pMsg);
if (code != TSDB_CODE_SUCCESS) {
mError("%p, msg:%s in mwrite queue, not processed reason:%s", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType], tstrerror(code));
mError("app:%p:%p, msg:%s not processed, reason:%s", pMsg->rpcMsg.ahandle, pMsg, taosMsg[pMsg->rpcMsg.msgType],
tstrerror(code));
return code;
}
if (!pMsg->pUser->writeAuth) {
mError("%p, msg:%s in mwrite queue, not processed, no write auth", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType]);
mError("app:%p:%p, msg:%s not processed, no write auth", pMsg->rpcMsg.ahandle, pMsg,
taosMsg[pMsg->rpcMsg.msgType]);
return TSDB_CODE_MND_NO_RIGHTS;
}

10
src/plugins/http/inc/gcHandle.h
View File

@ -16,15 +16,11 @@
#ifndef TDENGINE_GC_HANDLE_H
#define TDENGINE_GC_HANDLE_H
#include <stdbool.h>
#include <stdint.h>
#include <string.h>
#include <unistd.h>
#include "http.h"
#include "httpCode.h"
#include "httpHandle.h"
#include "httpInt.h"
#include "httpUtil.h"
#include "httpResp.h"
#include "httpSql.h"
#define GC_ROOT_URL_POS 0
#define GC_ACTION_URL_POS 1

23
src/plugins/http/inc/httpAuth.h Normal file
View File

@ -0,0 +1,23 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef TDENGINE_HTTP_TOKEN_H
#define TDENGINE_HTTP_TOKEN_H
bool httpParseBasicAuthToken(HttpContext *pContext, char *token, int len);
bool httpParseTaosdAuthToken(HttpContext *pContext, char *token, int len);
bool httpGenTaosdAuthToken(HttpContext *pContext, char *token, int maxLen);
#endif

34
src/plugins/http/inc/httpContext.h Normal file
View File

@ -0,0 +1,34 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef TDENGINE_HTTP_CONTEXT_H
#define TDENGINE_HTTP_CONTEXT_H
#include "httpInt.h"
bool httpInitContexts();
void httpCleanupContexts();
const char *httpContextStateStr(HttpContextState state);
HttpContext *httpCreateContext(int32_t fd);
bool httpInitContext(HttpContext *pContext);
HttpContext *httpGetContext(void * pContext);
void httpReleaseContext(HttpContext *pContext);
void httpCloseContextByServer(HttpContext *pContext);
void httpCloseContextByApp(HttpContext *pContext);
void httpNotifyContextClose(HttpContext *pContext);
bool httpAlterContextState(HttpContext *pContext, HttpContextState srcState, HttpContextState destState);
#endif

Some files were not shown because too many files have changed in this diff Show More