xxq250
/
homework-jianmu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

other: merge 3.0

This commit is contained in:
Haojun Liao 2024-09-26 09:59:29 +08:00
parent 3c7f718210 6d8ab955cf
commit 09600132ec
109 changed files with 1992 additions and 886 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
docs/examples/rust/restexample/examples/tmq.rs
View File

@ -83,7 +83,6 @@ async fn main() -> anyhow::Result<()> {
eprintln!("Failed to execute insert: {:?}", e);
}
tokio::time::sleep(Duration::from_millis(10)).await;
println!("Succed to execute insert 1 row");
}
});
});

2
docs/zh/05-basic/03-query.md
View File

@ -181,7 +181,7 @@ INTERVAL(interval_val [, interval_offset])
- FILL：用于指定窗口区间数据缺失的情况下，数据的填充模式。
对于时间窗口，interval_val 和 sliding_val 都表示时间段， 语法上支持三种方式。例如：
1. INTERVAL(1s, 500a) SLIDING(1s)，带时间单位的形式，其中的时间单位是单字符表示， 分别为： a （毫秒）， b （纳秒）， d （天）， h （小时）， m （分钟）， n （月）， s （秒）， u （微妙）， w （周）， y （年）；
1. INTERVAL(1s, 500a) SLIDING(1s)，带时间单位的形式，其中的时间单位是单字符表示， 分别为： a （毫秒）， b （纳秒）， d （天）， h （小时）， m （分钟）， n （月）， s （秒）， u （微秒）， w （周）， y （年）；
2. INTERVAL(1000, 500) SLIDING(1000)，不带时间单位的形式，将使用查询库的时间精度作为默认时间单位，当存在多个库时默认采用精度更高的库；
3. INTERVAL('1s', '500a') SLIDING('1s')，带时间单位的字符串形式，字符串内部不能有任何空格等其它字符。

3
docs/zh/07-develop/index.md
View File

@ -14,7 +14,8 @@ description: 让开发者能够快速上手的指南
7. 在很多场景下（如车辆管理），应用需要获取每个数据采集点的最新状态，那么建议你采用 TDengine 的 Cache 功能，而不用单独部署 Redis 等缓存软件。
8. 如果你发现 TDengine 的函数无法满足你的要求，那么你可以使用用户自定义函数（UDF）来解决问题。
本部分内容就是按照上述顺序组织的。为便于理解，TDengine 为每个功能和每个支持的编程语言都提供了示例代码。如果你希望深入了解 SQL 的使用，需要查看[SQL 手册](../reference/taos-sql/)。如果想更深入地了解各连接器的使用，请阅读[连接器参考指南](../reference/connector/)。如果还希望想将 TDengine 与第三方系统集成起来，比如 Grafana, 请参考[第三方工具](../third-party/)。
本部分内容就是按照上述顺序组织的。为便于理解，TDengine 为每个功能和每个支持的编程语言都提供了示例代码，位于 [示例代码](https://github.com/taosdata/TDengine/tree/main/docs/examples)。所有示例代码都会有 CI 保证正确性，脚本位于 [示例代码 CI](https://github.com/taosdata/TDengine/tree/main/tests/docs-examples-test)。
如果你希望深入了解 SQL 的使用，需要查看[SQL 手册](../reference/taos-sql/)。如果想更深入地了解各连接器的使用，请阅读[连接器参考指南](../reference/connector/)。如果还希望想将 TDengine 与第三方系统集成起来，比如 Grafana, 请参考[第三方工具](../third-party/)。
如果在开发过程中遇到任何问题，请点击每个页面下方的["反馈问题"](https://github.com/taosdata/TDengine/issues/new/choose), 在 GitHub 上直接递交 Issue。

91
docs/zh/14-reference/01-components/04-taosx.md
View File

@ -421,7 +421,6 @@ taosX 会将监控指标上报给 taosKeeper，这些监控指标会被 taosKeep
| write_raw_fails | 本次运行写入 raw meta 失败的次数 |
| success_blocks | 本次写入成功的数据块数 |
### taosX 其他数据源 任务
这些数据源包括： InfluxDB，OpenTSDB，OPC UA，OPC DA，PI，CSV，MQTT，AVEVA Historian 和 Kafka。
@ -452,3 +451,93 @@ taosX 会将监控指标上报给 taosKeeper，这些监控指标会被 taosKeep
| written_blocks | 本次运行此任务写人成功的 raw block 数 |
| failed_blocks | 本次运行此任务写入失败的 raw block 数 |
## taosX 数据解析插件
接入 kafka / mqtt 消息中间件时，需要对原始数据进行解析，如果使用 json/regex 等模式解析器无法满足解析需求，同时 UDT(自定义解析脚本) 也无法满足性能要求时，可以自定义数据解析插件。
### 插件概述
taosX Parser 插件是一个要求用 C/Rust 语言开发的 C ABI 兼容动态库，该动态库要实现约定的 API 并编译为在 taosX 所在运行环境中能够正确运行的动态库，然后复制到约定位置由 taosX 在运行时加载，并在处理数据的 Parsing 阶段调用。
### 插件部署
完成插件开发后，编译环境需要和目标运行环境兼容，将编译好的插件动态库复制到插件目录下，taosX 启动后，系统首次使用插件时初始化加载插件。可以在 explorer 的 kafka 或者 mqtt 数据接入配置页面中，检查是否加载成功。如下图，如果加载成功，则在解析器选择列表中展示出来。
![插件示例](./plugin-01.png)
插件目录在 `taosx.toml` 配置文件中复用 plugins 配置，追加`/parsers`作为插件安装路径，默认值在 UNIX 环境下为 `/usr/local/taos/plugins/parsers`，在 Windows 下为 `C:\TDengine\plugins\parsers`。
### 插件 api 说明
#### 1. 获取插件名称
获取插件名，用于前端显示。
**函数签名**：const char* parser_name()
**返回值**：字符串。
#### 2. 获取插件版本
插件版本，方便问题定位。
**函数签名**：const char* parser_version()
**返回值**：字符串。
#### 3. 配置解析器
将一个字符串参数解析为一个配置对象，仅插件内部使用。
**函数签名**：parser_resp_t parser_new(char* ctx, uint32_t len);
char* ctx: 用户自定义配置字符串。
uint32_t len: 该字符串的二进制长度（不含 `\0`）。
**返回值**：
``` c
struct parser_resp_t {
int e; // 0 if success.
void* p; // Success if contains.
}
```
当创建对象失败时，e 不为 0。
当创建成功时，e = 0，p 为解析器对象。
#### 4. 解析数据
**函数签名**：
对输入 payload 进行解析，返回结果为 JSON 格式 [u8] 。返回的 JSON 将使用默认的 JSON 解析器进行完全解码（展开根数组和所有的对象）。
``` c
const char* parser_mutate(
void* parser,
const uint8_t* in_ptr, uint32_t in_len,
const void* uint8_t* out_ptr, uint32_t* out_len
);
```
`void* parser`: parser_new 生成的对象指针;
`const uint8_t* in_ptr`：输入 Payload 的指针;
`uint32_t in_len`: 输入 Payload 的 bytes 长度（不含 `\0`）;
`const void* uint8_t* out_ptr`:输出 JSON 字符串的指针（不含 \0）。当 out_ptr 指向为空时，表示输出为空。
`uint32_t * out_len`：输出 JSON 字符串长度。
**返回值**： 当调用成功时，返回值为 NULL。
#### 5. 释放解析器
释放解析器对象内存。
**函数签名**： void parser_free(void* parser);
void* parser: parser_new 生成的对象指针。

BIN
docs/zh/14-reference/01-components/plugin-01.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 27 KiB

1
docs/zh/14-reference/03-taos-sql/02-database.md
View File

@ -80,7 +80,6 @@ database_option: {
```sql
create database if not exists db vgroups 10 buffer 10
```
以上示例创建了一个有 10 个 vgroup 名为 db 的数据库， 其中每个 vnode 分配 10MB 的写入缓存

2
docs/zh/14-reference/03-taos-sql/10-function.md
View File

@ -1214,7 +1214,7 @@ TO_TIMESTAMP(ts_str_literal, format_str_literal)
- 如果没有指定完整的时间，那么默认时间值为指定或默认时区的 `1970-01-01 00:00:00`, 未指定部分使用该默认值中的对应部分. 暂不支持只指定年日而不指定月日的格式, 如'yyyy-mm-DDD', 支持'yyyy-mm-DD'.
- 如果格式串中有`AM`, `PM`等, 那么小时必须是12小时制, 范围必须是01-12.
- `to_timestamp`转换具有一定的容错机制, 在格式串和时间戳串不完全对应时, 有时也可转换, 如: `to_timestamp('200101/2', 'yyyyMM1/dd')`, 格式串中多出来的1会被丢弃. 格式串与时间戳串中多余的空格字符(空格, tab等)也会被 自动忽略. 如`to_timestamp(' 23 年 - 1 月 - 01 日 ', 'yy 年-MM月-dd日')` 可以被成功转换. 虽然`MM`等字段需要两个数字对应(只有一位时前面补0), 在`to_timestamp`时, 一个数字也可以成功转换.
- 输出时间戳的精度与查询表的精度相同, 若查询未指定表, 则输出精度为毫秒. 如`select to_timestamp('2023-08-1 10:10:10.123456789', 'yyyy-mm-dd hh:mi:ss.ns')`的输出将会把微妙和纳秒进行截断. 如果指定一张纳秒表, 那么就不会发生截断, 如`select to_timestamp('2023-08-1 10:10:10.123456789', 'yyyy-mm-dd hh:mi:ss.ns') from db_ns.table_ns limit 1`.
- 输出时间戳的精度与查询表的精度相同, 若查询未指定表, 则输出精度为毫秒. 如`select to_timestamp('2023-08-1 10:10:10.123456789', 'yyyy-mm-dd hh:mi:ss.ns')`的输出将会把微秒和纳秒进行截断. 如果指定一张纳秒表, 那么就不会发生截断, 如`select to_timestamp('2023-08-1 10:10:10.123456789', 'yyyy-mm-dd hh:mi:ss.ns') from db_ns.table_ns limit 1`.
### 时间和日期函数

8
docs/zh/14-reference/03-taos-sql/16-operators.md
View File

@ -41,9 +41,11 @@ TDengine 支持 `UNION ALL` 和 `UNION` 操作符。UNION ALL 将查询返回的
| 5 | IS [NOT] NULL | 所有类型 | 是否为空值 |
| 6 | [NOT] BETWEEN AND | 除 BOOL、BLOB、MEDIUMBLOB、JSON 和 GEOMETRY 外的所有类型 | 闭区间比较 |
| 7 | IN | 除 BLOB、MEDIUMBLOB 和 JSON 外的所有类型，且不可以为表的时间戳主键列 | 与列表内的任意值相等 |
| 8 | LIKE | BINARY、NCHAR 和 VARCHAR | 通配符匹配 |
| 9 | MATCH, NMATCH | BINARY、NCHAR 和 VARCHAR | 正则表达式匹配 |
| 10 | CONTAINS | JSON | JSON 中是否存在某键 |
| 8 | NOT IN | 除 BLOB、MEDIUMBLOB 和 JSON 外的所有类型，且不可以为表的时间戳主键列 | 与列表内的任意值都不相等 |
| 9 | LIKE | BINARY、NCHAR 和 VARCHAR | 通配符匹配所指定的模式串 |
| 10 | NOT LIKE | BINARY、NCHAR 和 VARCHAR | 通配符不匹配所指定的模式串 |
| 11 | MATCH, NMATCH | BINARY、NCHAR 和 VARCHAR | 正则表达式匹配 |
| 12 | CONTAINS | JSON | JSON 中是否存在某键 |
LIKE 条件使用通配符字符串进行匹配检查，规则如下：

2
docs/zh/14-reference/03-taos-sql/28-index.md
View File

@ -28,7 +28,7 @@ TSMA只能基于超级表和普通表创建, 不能基于子表创建.
由于TSMA输出为一张超级表, 因此输出表的行长度受最大行长度限制, 不同函数的`中间结果`大小各异, 一般都大于原始数据大小, 若输出表的行长度大于最大行长度限制, 将会报`Row length exceeds max length`错误. 此时需要减少函数个数或者将常用的函数进行分组拆分到多个TSMA中.
窗口大小的限制为[1m ~ 1y/12n]. INTERVAL 的单位与查询中INTERVAL子句相同, 如 a (毫秒), b (纳秒), h (小时), m (分钟), s (秒), u (微妙), d (天), w(周), n(月), y(年).
窗口大小的限制为[1m ~ 1y/12n]. INTERVAL 的单位与查询中INTERVAL子句相同, 如 a (毫秒), b (纳秒), h (小时), m (分钟), s (秒), u (微秒), d (天), w(周), n(月), y(年).
TSMA为库内对象, 但名字全局唯一. 集群内一共可创建TSMA个数受参数`maxTsmaNum`限制, 参数默认值为3, 范围: [0-3]. 注意, 由于TSMA后台计算使用流计算, 因此每创建一条TSMA, 将会创建一条流, 因此能够创建的TSMA条数也受当前已经存在的流条数和最大可创建流条数限制.

1
include/common/cos.h
View File

@ -32,7 +32,6 @@ extern int32_t tsS3PageCacheSize;
extern int32_t tsS3UploadDelaySec;
int32_t s3Init();
void s3CleanUp();
int32_t s3CheckCfg();
int32_t s3PutObjectFromFile(const char *file, const char *object);
int32_t s3PutObjectFromFile2(const char *file, const char *object, int8_t withcp);

2
include/common/tdataformat.h
View File

@ -115,7 +115,7 @@ typedef struct {
} SValueColumnCompressInfo;
int32_t tValueColumnInit(SValueColumn *valCol);
int32_t tValueColumnDestroy(SValueColumn *valCol);
void tValueColumnDestroy(SValueColumn *valCol);
void tValueColumnClear(SValueColumn *valCol);
int32_t tValueColumnAppend(SValueColumn *valCol, const SValue *value);
int32_t tValueColumnUpdate(SValueColumn *valCol, int32_t idx, const SValue *value);

4
include/util/tcompression.h
View File

@ -72,8 +72,8 @@ extern "C" {
#ifdef TD_TSZ
extern bool lossyFloat;
extern bool lossyDouble;
int32_t tsCompressInit(char *lossyColumns, float fPrecision, double dPrecision, uint32_t maxIntervals,
uint32_t intervals, int32_t ifAdtFse, const char *compressor);
void tsCompressInit(char *lossyColumns, float fPrecision, double dPrecision, uint32_t maxIntervals, uint32_t intervals,
int32_t ifAdtFse, const char *compressor);
void tsCompressExit();

19
source/client/inc/clientInt.h
View File

@ -26,10 +26,10 @@ extern "C" {
#include "query.h"
#include "taos.h"
#include "tcommon.h"
#include "tmisce.h"
#include "tdef.h"
#include "thash.h"
#include "tlist.h"
#include "tmisce.h"
#include "tmsg.h"
#include "tmsgtype.h"
#include "trpc.h"
@ -319,11 +319,13 @@ void syncCatalogFn(SMetaData* pResult, void* param, int32_t code);
TAOS_RES* taosQueryImpl(TAOS* taos, const char* sql, bool validateOnly, int8_t source);
TAOS_RES* taosQueryImplWithReqid(TAOS* taos, const char* sql, bool validateOnly, int64_t reqid);
void taosAsyncQueryImpl(uint64_t connId, const char* sql, __taos_async_fn_t fp, void* param, bool validateOnly, int8_t source);
void taosAsyncQueryImpl(uint64_t connId, const char* sql, __taos_async_fn_t fp, void* param, bool validateOnly,
int8_t source);
void taosAsyncQueryImplWithReqid(uint64_t connId, const char* sql, __taos_async_fn_t fp, void* param, bool validateOnly,
int64_t reqid);
void taosAsyncFetchImpl(SRequestObj* pRequest, __taos_async_fn_t fp, void* param);
int32_t clientParseSql(void* param, const char* dbName, const char* sql, bool parseOnly, const char* effectiveUser, SParseSqlRes* pRes);
int32_t clientParseSql(void* param, const char* dbName, const char* sql, bool parseOnly, const char* effectiveUser,
SParseSqlRes* pRes);
void syncQueryFn(void* param, void* res, int32_t code);
int32_t getVersion1BlockMetaSize(const char* p, int32_t numOfCols);
@ -422,7 +424,7 @@ typedef struct SSqlCallbackWrapper {
} SSqlCallbackWrapper;
void setQueryRequest(int64_t rId);
SRequestObj* launchQueryImpl(SRequestObj* pRequest, SQuery* pQuery, bool keepQuery, void** res);
void launchQueryImpl(SRequestObj* pRequest, SQuery* pQuery, bool keepQuery, void** res);
int32_t scheduleQuery(SRequestObj* pRequest, SQueryPlan* pDag, SArray* pNodeList);
void launchAsyncQuery(SRequestObj* pRequest, SQuery* pQuery, SMetaData* pResultMeta, SSqlCallbackWrapper* pWrapper);
int32_t refreshMeta(STscObj* pTscObj, SRequestObj* pRequest);
@ -444,7 +446,8 @@ void doRequestCallback(SRequestObj* pRequest, int32_t code);
void freeQueryParam(SSyncQueryParam* param);
#ifdef TD_ENTERPRISE
int32_t clientParseSqlImpl(void* param, const char* dbName, const char* sql, bool parseOnly, const char* effeciveUser, SParseSqlRes* pRes);
int32_t clientParseSqlImpl(void* param, const char* dbName, const char* sql, bool parseOnly, const char* effeciveUser,
SParseSqlRes* pRes);
#endif
#define TSC_ERR_RET(c) \
@ -474,11 +477,7 @@ int32_t clientParseSqlImpl(void* param, const char* dbName, const char* sql, boo
void slowQueryLog(int64_t rid, bool killed, int32_t code, int32_t cost);
enum {
MONITORSQLTYPESELECT = 0,
MONITORSQLTYPEINSERT = 1,
MONITORSQLTYPEDELETE = 2
};
enum { MONITORSQLTYPESELECT = 0, MONITORSQLTYPEINSERT = 1, MONITORSQLTYPEDELETE = 2 };
void sqlReqLog(int64_t rid, bool killed, int32_t code, int8_t type);

54
source/client/src/clientImpl.c
View File

@ -679,9 +679,15 @@ _return:
return TSDB_CODE_SUCCESS;
}
void freeVgList(void* list) {
SArray* pList = *(SArray**)list;
taosArrayDestroy(pList);
}
int32_t buildAsyncExecNodeList(SRequestObj* pRequest, SArray** pNodeList, SArray* pMnodeList, SMetaData* pResultMeta) {
SArray* pDbVgList = NULL;
SArray* pQnodeList = NULL;
FDelete fp = NULL;
int32_t code = 0;
switch (tsQueryPolicy) {
@ -705,6 +711,43 @@ int32_t buildAsyncExecNodeList(SRequestObj* pRequest, SArray** pNodeList, SArray
goto _return;
}
}
} else {
fp = freeVgList;
int32_t dbNum = taosArrayGetSize(pRequest->dbList);
if (dbNum > 0) {
SCatalog* pCtg = NULL;
SAppInstInfo* pInst = pRequest->pTscObj->pAppInfo;
code = catalogGetHandle(pInst->clusterId, &pCtg);
if (code != TSDB_CODE_SUCCESS) {
goto _return;
}
pDbVgList = taosArrayInit(dbNum, POINTER_BYTES);
if (NULL == pDbVgList) {
code = terrno;
goto _return;
}
SArray* pVgList = NULL;
for (int32_t i = 0; i < dbNum; ++i) {
char* dbFName = taosArrayGet(pRequest->dbList, i);
SRequestConnInfo conn = {.pTrans = pInst->pTransporter,
.requestId = pRequest->requestId,
.requestObjRefId = pRequest->self,
.mgmtEps = getEpSet_s(&pInst->mgmtEp)};
// catalogGetDBVgList will handle dbFName == null.
code = catalogGetDBVgList(pCtg, &conn, dbFName, &pVgList);
if (code) {
goto _return;
}
if (NULL == taosArrayPush(pDbVgList, &pVgList)) {
code = terrno;
goto _return;
}
}
}
}
code = buildVnodePolicyNodeList(pRequest, pNodeList, pMnodeList, pDbVgList);
@ -745,17 +788,12 @@ int32_t buildAsyncExecNodeList(SRequestObj* pRequest, SArray** pNodeList, SArray
}
_return:
taosArrayDestroy(pDbVgList);
taosArrayDestroyEx(pDbVgList, fp);
taosArrayDestroy(pQnodeList);
return code;
}
void freeVgList(void* list) {
SArray* pList = *(SArray**)list;
taosArrayDestroy(pList);
}
int32_t buildSyncExecNodeList(SRequestObj* pRequest, SArray** pNodeList, SArray* pMnodeList) {
SArray* pDbVgList = NULL;
SArray* pQnodeList = NULL;
@ -1211,7 +1249,7 @@ void schedulerExecCb(SExecResult* pResult, void* param, int32_t code) {
}
}
SRequestObj* launchQueryImpl(SRequestObj* pRequest, SQuery* pQuery, bool keepQuery, void** res) {
void launchQueryImpl(SRequestObj* pRequest, SQuery* pQuery, bool keepQuery, void** res) {
int32_t code = 0;
if (pQuery->pRoot) {
@ -1297,8 +1335,6 @@ SRequestObj* launchQueryImpl(SRequestObj* pRequest, SQuery* pQuery, bool keepQue
*res = pRequest->body.resInfo.execRes.res;
pRequest->body.resInfo.execRes.res = NULL;
}
return pRequest;
}
static int32_t asyncExecSchQuery(SRequestObj* pRequest, SQuery* pQuery, SMetaData* pResultMeta,

43
source/client/src/clientRawBlockWrite.c
View File

@ -361,7 +361,7 @@ static void processCreateStb(SMqMetaRsp* metaRsp, cJSON** pJson) {
buildCreateTableJson(&req.schemaRow, &req.schemaTag, req.name, req.suid, TSDB_SUPER_TABLE, &req.colCmpr, pJson);
end:
uDebug("create stable return, sql json:%s", cJSON_PrintUnformatted(*pJson));
uDebug("create stable return");
tDecoderClear(&coder);
}
@ -381,7 +381,7 @@ static void processAlterStb(SMqMetaRsp* metaRsp, cJSON** pJson) {
buildAlterSTableJson(req.alterOriData, req.alterOriDataLen, pJson);
end:
uDebug("alter stable return, sql json:%s", cJSON_PrintUnformatted(*pJson));
uDebug("alter stable return");
tDecoderClear(&coder);
}
@ -393,7 +393,7 @@ static void buildChildElement(cJSON* json, SVCreateTbReq* pCreateReq) {
int64_t id = pCreateReq->uid;
uint8_t tagNum = pCreateReq->ctb.tagNum;
int32_t code = 0;
cJSON* tags = NULL;
cJSON* tableName = cJSON_CreateString(name);
RAW_NULL_CHECK(tableName);
RAW_FALSE_CHECK(cJSON_AddItemToObject(json, "tableName", tableName));
@ -404,7 +404,7 @@ static void buildChildElement(cJSON* json, SVCreateTbReq* pCreateReq) {
RAW_NULL_CHECK(tagNumJson);
RAW_FALSE_CHECK(cJSON_AddItemToObject(json, "tagNum", tagNumJson));
cJSON* tags = cJSON_CreateArray();
tags = cJSON_CreateArray();
RAW_NULL_CHECK(tags);
SArray* pTagVals = NULL;
RAW_RETURN_CHECK(tTagToValArray(pTag, &pTagVals));
@ -543,7 +543,7 @@ static void processCreateTable(SMqMetaRsp* metaRsp, cJSON** pJson) {
}
end:
uDebug("create table return, sql json:%s", cJSON_PrintUnformatted(*pJson));
uDebug("create table return");
tDeleteSVCreateTbBatchReq(&req);
tDecoderClear(&decoder);
}
@ -772,7 +772,7 @@ static void processAlterTable(SMqMetaRsp* metaRsp, cJSON** pJson) {
}
end:
uDebug("alter table return, sql json:%s", cJSON_PrintUnformatted(json));
uDebug("alter table return");
tDecoderClear(&decoder);
*pJson = json;
}
@ -807,7 +807,7 @@ static void processDropSTable(SMqMetaRsp* metaRsp, cJSON** pJson) {
RAW_FALSE_CHECK(cJSON_AddItemToObject(json, "tableName", tableName));
end:
uDebug("processDropSTable return, sql json:%s", cJSON_PrintUnformatted(json));
uDebug("processDropSTable return");
tDecoderClear(&decoder);
*pJson = json;
}
@ -843,7 +843,7 @@ static void processDeleteTable(SMqMetaRsp* metaRsp, cJSON** pJson) {
RAW_FALSE_CHECK(cJSON_AddItemToObject(json, "sql", sqlJson));
end:
uDebug("processDeleteTable return, sql json:%s", cJSON_PrintUnformatted(json));
uDebug("processDeleteTable return");
tDecoderClear(&coder);
*pJson = json;
}
@ -880,7 +880,7 @@ static void processDropTable(SMqMetaRsp* metaRsp, cJSON** pJson) {
RAW_FALSE_CHECK(cJSON_AddItemToObject(json, "tableNameList", tableNameList));
end:
uDebug("processDropTable return, json sql:%s", cJSON_PrintUnformatted(json));
uDebug("processDropTable return");
tDecoderClear(&decoder);
*pJson = json;
}
@ -978,7 +978,7 @@ static int32_t taosCreateStb(TAOS* taos, void* meta, int32_t metaLen) {
pQuery.msgType = pQuery.pCmdMsg->msgType;
pQuery.stableQuery = true;
(void)launchQueryImpl(pRequest, &pQuery, true, NULL); // ignore, because return value is pRequest
launchQueryImpl(pRequest, &pQuery, true, NULL); // ignore, because return value is pRequest
taosMemoryFree(pCmdMsg.pMsg);
@ -1082,7 +1082,7 @@ static int32_t taosDropStb(TAOS* taos, void* meta, int32_t metaLen) {
pQuery.msgType = pQuery.pCmdMsg->msgType;
pQuery.stableQuery = true;
(void)launchQueryImpl(pRequest, &pQuery, true, NULL); // ignore, because return value is pRequest
launchQueryImpl(pRequest, &pQuery, true, NULL); // ignore, because return value is pRequest
taosMemoryFree(pCmdMsg.pMsg);
if (pRequest->code == TSDB_CODE_SUCCESS) {
// ignore the error code
@ -1236,7 +1236,7 @@ static int32_t taosCreateTable(TAOS* taos, void* meta, int32_t metaLen) {
RAW_RETURN_CHECK(rewriteToVnodeModifyOpStmt(pQuery, pBufArray));
(void)launchQueryImpl(pRequest, pQuery, true, NULL);
launchQueryImpl(pRequest, pQuery, true, NULL);
if (pRequest->code == TSDB_CODE_SUCCESS) {
RAW_RETURN_CHECK(removeMeta(pTscObj, pRequest->tableList, false));
}
@ -1365,7 +1365,7 @@ static int32_t taosDropTable(TAOS* taos, void* meta, int32_t metaLen) {
if (TSDB_CODE_SUCCESS != code) goto end;
RAW_RETURN_CHECK(rewriteToVnodeModifyOpStmt(pQuery, pBufArray));
(void)launchQueryImpl(pRequest, pQuery, true, NULL);
launchQueryImpl(pRequest, pQuery, true, NULL);
if (pRequest->code == TSDB_CODE_SUCCESS) {
RAW_RETURN_CHECK(removeMeta(pTscObj, pRequest->tableList, false));
}
@ -1510,7 +1510,7 @@ static int32_t taosAlterTable(TAOS* taos, void* meta, int32_t metaLen) {
if (TSDB_CODE_SUCCESS != code) goto end;
RAW_RETURN_CHECK(rewriteToVnodeModifyOpStmt(pQuery, pArray));
(void)launchQueryImpl(pRequest, pQuery, true, NULL);
launchQueryImpl(pRequest, pQuery, true, NULL);
pVgData = NULL;
pArray = NULL;
@ -1587,7 +1587,7 @@ int taos_write_raw_block_with_fields_with_reqid(TAOS* taos, int rows, char* pDat
RAW_RETURN_CHECK(rawBlockBindData(pQuery, pTableMeta, pData, NULL, fields, numFields, false, NULL, 0));
RAW_RETURN_CHECK(smlBuildOutput(pQuery, pVgHash));
(void)launchQueryImpl(pRequest, pQuery, true, NULL);
launchQueryImpl(pRequest, pQuery, true, NULL);
code = pRequest->code;
end:
@ -1647,7 +1647,7 @@ int taos_write_raw_block_with_reqid(TAOS* taos, int rows, char* pData, const cha
RAW_RETURN_CHECK(rawBlockBindData(pQuery, pTableMeta, pData, NULL, NULL, 0, false, NULL, 0));
RAW_RETURN_CHECK(smlBuildOutput(pQuery, pVgHash));
(void)launchQueryImpl(pRequest, pQuery, true, NULL);
launchQueryImpl(pRequest, pQuery, true, NULL);
code = pRequest->code;
end:
@ -1766,7 +1766,7 @@ static int32_t tmqWriteRawDataImpl(TAOS* taos, void* data, int32_t dataLen) {
RAW_RETURN_CHECK(smlBuildOutput(pQuery, pVgHash));
(void)launchQueryImpl(pRequest, pQuery, true, NULL);
launchQueryImpl(pRequest, pQuery, true, NULL);
code = pRequest->code;
end:
@ -1923,6 +1923,10 @@ static int32_t tmqWriteRawMetaDataImpl(TAOS* taos, void* data, int32_t dataLen)
code = rawBlockBindData(pQuery, pTableMeta, rawData, &pCreateReqDst, fields, pSW->nCols, true, err, ERR_MSG_LEN);
taosMemoryFree(fields);
taosMemoryFreeClear(pTableMeta);
if (pCreateReqDst) {
tdDestroySVCreateTbReq(pCreateReqDst);
taosMemoryFreeClear(pCreateReqDst);
}
if (code != TSDB_CODE_SUCCESS) {
SET_ERROR_MSG("table:%s, err:%s", tbName, err);
goto end;
@ -1931,7 +1935,7 @@ static int32_t tmqWriteRawMetaDataImpl(TAOS* taos, void* data, int32_t dataLen)
RAW_RETURN_CHECK(smlBuildOutput(pQuery, pVgHash));
(void)launchQueryImpl(pRequest, pQuery, true, NULL);
launchQueryImpl(pRequest, pQuery, true, NULL);
code = pRequest->code;
end:
@ -1944,7 +1948,7 @@ end:
taosMemoryFreeClear(pTableMeta);
if (pCreateReqDst) {
tdDestroySVCreateTbReq(pCreateReqDst);
taosMemoryFree(pCreateReqDst);
taosMemoryFreeClear(pCreateReqDst);
}
return code;
}
@ -2038,6 +2042,7 @@ char* tmq_get_json_meta(TAOS_RES* res) {
processSimpleMeta(&pMetaRspObj->metaRsp, &pJson);
char* string = cJSON_PrintUnformatted(pJson);
cJSON_Delete(pJson);
uDebug("tmq_get_json_meta string:%s", string);
return string;
}

36
source/client/src/clientSml.c
View File

@ -538,7 +538,8 @@ static int32_t smlParseTableName(SArray *tags, char *childTableName, char *tbnam
// handle child table name
if (childTableNameLen == tag->keyLen && strncmp(tag->key, tbnameKey, tag->keyLen) == 0) {
(void)memset(childTableName, 0, TSDB_TABLE_NAME_LEN);
(void)strncpy(childTableName, tag->value, (tag->length < TSDB_TABLE_NAME_LEN ? tag->length : TSDB_TABLE_NAME_LEN));
(void)strncpy(childTableName, tag->value,
(tag->length < TSDB_TABLE_NAME_LEN ? tag->length : TSDB_TABLE_NAME_LEN));
if (tsSmlDot2Underline) {
smlStrReplace(childTableName, strlen(childTableName));
}
@ -906,8 +907,8 @@ static int32_t smlFindNearestPowerOf2(int32_t length, uint8_t type) {
return result;
}
static int32_t smlProcessSchemaAction(SSmlHandle *info, SSchema *schemaField, SHashObj *schemaHash, SArray *cols, SArray *checkDumplicateCols,
ESchemaAction *action, bool isTag) {
static int32_t smlProcessSchemaAction(SSmlHandle *info, SSchema *schemaField, SHashObj *schemaHash, SArray *cols,
SArray *checkDumplicateCols, ESchemaAction *action, bool isTag) {
int32_t code = TSDB_CODE_SUCCESS;
for (int j = 0; j < taosArrayGetSize(cols); ++j) {
if (j == 0 && !isTag) continue;
@ -1144,7 +1145,7 @@ static int32_t smlSendMetaMsg(SSmlHandle *info, SName *pName, SArray *pColumns,
pQuery.msgType = pQuery.pCmdMsg->msgType;
pQuery.stableQuery = true;
(void)launchQueryImpl(pRequest, &pQuery, true, NULL); // no need to check return value
launchQueryImpl(pRequest, &pQuery, true, NULL); // no need to check return value
if (pRequest->code == TSDB_CODE_SUCCESS) {
code = catalogRemoveTableMeta(info->pCatalog, pName);
@ -1246,7 +1247,6 @@ static int32_t smlModifyDBSchemas(SSmlHandle *info) {
goto end;
}
} else if (code == TSDB_CODE_SUCCESS) {
if (smlIsPKTable(pTableMeta)) {
code = TSDB_CODE_SML_NOT_SUPPORT_PK;
goto end;
@ -1267,7 +1267,8 @@ static int32_t smlModifyDBSchemas(SSmlHandle *info) {
}
ESchemaAction action = SCHEMA_ACTION_NULL;
code = smlProcessSchemaAction(info, pTableMeta->schema, hashTmp, sTableData->tags, sTableData->cols, &action, true);
code =
smlProcessSchemaAction(info, pTableMeta->schema, hashTmp, sTableData->tags, sTableData->cols, &action, true);
if (code != TSDB_CODE_SUCCESS) {
goto end;
}
@ -1363,7 +1364,8 @@ static int32_t smlModifyDBSchemas(SSmlHandle *info) {
}
}
action = SCHEMA_ACTION_NULL;
code = smlProcessSchemaAction(info, pTableMeta->schema, hashTmp, sTableData->cols, sTableData->tags, &action, false);
code =
smlProcessSchemaAction(info, pTableMeta->schema, hashTmp, sTableData->cols, sTableData->tags, &action, false);
if (code != TSDB_CODE_SUCCESS) {
goto end;
}
@ -1512,7 +1514,8 @@ static int32_t smlInsertMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols
return TSDB_CODE_SUCCESS;
}
static int32_t smlUpdateMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols, bool isTag, SSmlMsgBuf *msg, SHashObj* checkDuplicate) {
static int32_t smlUpdateMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols, bool isTag, SSmlMsgBuf *msg,
SHashObj *checkDuplicate) {
for (int i = 0; i < taosArrayGetSize(cols); ++i) {
SSmlKv *kv = (SSmlKv *)taosArrayGet(cols, i);
if (kv == NULL) {
@ -1759,9 +1762,11 @@ static int32_t smlParseLineBottom(SSmlHandle *info) {
if (tableMeta) { // update meta
uDebug("SML:0x%" PRIx64 " smlParseLineBottom update meta, format:%d, linenum:%d", info->id, info->dataFormat,
info->lineNum);
ret = smlUpdateMeta((*tableMeta)->colHash, (*tableMeta)->cols, elements->colArray, false, &info->msgBuf, (*tableMeta)->tagHash);
ret = smlUpdateMeta((*tableMeta)->colHash, (*tableMeta)->cols, elements->colArray, false, &info->msgBuf,
(*tableMeta)->tagHash);
if (ret == TSDB_CODE_SUCCESS) {
ret = smlUpdateMeta((*tableMeta)->tagHash, (*tableMeta)->tags, tinfo->tags, true, &info->msgBuf, (*tableMeta)->colHash);
ret = smlUpdateMeta((*tableMeta)->tagHash, (*tableMeta)->tags, tinfo->tags, true, &info->msgBuf,
(*tableMeta)->colHash);
}
if (ret != TSDB_CODE_SUCCESS) {
uError("SML:0x%" PRIx64 " smlUpdateMeta failed, ret:%d", info->id, ret);
@ -1906,7 +1911,7 @@ static int32_t smlInsertData(SSmlHandle *info) {
SAppClusterSummary *pActivity = &info->taos->pAppInfo->summary;
(void)atomic_add_fetch_64((int64_t *)&pActivity->numOfInsertsReq, 1); // no need to check return code
(void)launchQueryImpl(info->pRequest, info->pQuery, true, NULL); // no need to check return code
launchQueryImpl(info->pRequest, info->pQuery, true, NULL); // no need to check return code
uDebug("SML:0x%" PRIx64 " smlInsertData end, format:%d, code:%d,%s", info->id, info->dataFormat, info->pRequest->code,
tstrerror(info->pRequest->code));
@ -2306,14 +2311,17 @@ static void getRawLineLen(char *lines, int len, int32_t *totalRows, int protocol
}
TAOS_RES *taos_schemaless_insert_raw_ttl_with_reqid_tbname_key(TAOS *taos, char *lines, int len, int32_t *totalRows,
int protocol, int precision, int32_t ttl, int64_t reqid, char *tbnameKey){
int protocol, int precision, int32_t ttl, int64_t reqid,
char *tbnameKey) {
getRawLineLen(lines, len, totalRows, protocol);
return taos_schemaless_insert_inner(taos, NULL, lines, lines + len, *totalRows, protocol, precision, ttl, reqid, tbnameKey);
return taos_schemaless_insert_inner(taos, NULL, lines, lines + len, *totalRows, protocol, precision, ttl, reqid,
tbnameKey);
}
TAOS_RES *taos_schemaless_insert_raw_ttl_with_reqid(TAOS *taos, char *lines, int len, int32_t *totalRows, int protocol,
int precision, int32_t ttl, int64_t reqid) {
return taos_schemaless_insert_raw_ttl_with_reqid_tbname_key(taos, lines, len, totalRows, protocol, precision, ttl, reqid, NULL);
return taos_schemaless_insert_raw_ttl_with_reqid_tbname_key(taos, lines, len, totalRows, protocol, precision, ttl,
reqid, NULL);
}
TAOS_RES *taos_schemaless_insert_raw_with_reqid(TAOS *taos, char *lines, int len, int32_t *totalRows, int protocol,

5
source/client/src/clientStmt.c
View File

@ -1216,7 +1216,6 @@ int stmtBindBatch(TAOS_STMT* stmt, TAOS_MULTI_BIND* bind, int32_t colIdx) {
STMT_ERR_RET(stmtSwitchStatus(pStmt, STMT_BIND));
if (pStmt->bInfo.needParse && pStmt->sql.runTimes && pStmt->sql.type > 0 &&
STMT_TYPE_MULTI_INSERT != pStmt->sql.type) {
pStmt->bInfo.needParse = false;
@ -1549,7 +1548,7 @@ int stmtExec(TAOS_STMT* stmt) {
STMT_ERR_RET(stmtSwitchStatus(pStmt, STMT_EXECUTE));
if (STMT_TYPE_QUERY == pStmt->sql.type) {
(void)launchQueryImpl(pStmt->exec.pRequest, pStmt->sql.pQuery, true, NULL);
launchQueryImpl(pStmt->exec.pRequest, pStmt->sql.pQuery, true, NULL);
} else {
if (pStmt->sql.stbInterlaceMode) {
int64_t startTs = taosGetTimestampUs();
@ -1571,7 +1570,7 @@ int stmtExec(TAOS_STMT* stmt) {
STMT_ERR_RET(qBuildStmtOutput(pStmt->sql.pQuery, pStmt->sql.pVgHash, pStmt->exec.pBlockHash));
}
(void)launchQueryImpl(pStmt->exec.pRequest, pStmt->sql.pQuery, true, NULL);
launchQueryImpl(pStmt->exec.pRequest, pStmt->sql.pQuery, true, NULL);
}
if (pStmt->exec.pRequest->code && NEED_CLIENT_HANDLE_ERROR(pStmt->exec.pRequest->code)) {

18
source/client/src/clientStmt2.c
View File

@ -1645,7 +1645,7 @@ int stmtExec2(TAOS_STMT2* stmt, int* affected_rows) {
__taos_async_fn_t fp = pStmt->options.asyncExecFn;
if (!fp) {
(void)launchQueryImpl(pStmt->exec.pRequest, pStmt->sql.pQuery, true, NULL);
launchQueryImpl(pStmt->exec.pRequest, pStmt->sql.pQuery, true, NULL);
if (pStmt->exec.pRequest->code && NEED_CLIENT_HANDLE_ERROR(pStmt->exec.pRequest->code)) {
code = refreshMeta(pStmt->exec.pRequest->pTscObj, pStmt->exec.pRequest);
@ -1873,6 +1873,8 @@ int stmtGetParamNum2(TAOS_STMT2* stmt, int* nums) {
int stmtGetParamTbName(TAOS_STMT2* stmt, int* nums) {
STscStmt2* pStmt = (STscStmt2*)stmt;
int32_t code = 0;
int32_t preCode = pStmt->errCode;
STMT_DLOG_E("start to get param num");
@ -1895,17 +1897,19 @@ int stmtGetParamTbName(TAOS_STMT2* stmt, int* nums) {
STMT_ERR_RET(stmtCreateRequest(pStmt));
if (pStmt->bInfo.needParse) {
STMT_ERR_RET(stmtParseSql(pStmt));
STMT_ERRI_JRET(stmtParseSql(pStmt));
}
if (TSDB_CODE_TSC_STMT_TBNAME_ERROR == pStmt->errCode) {
*nums = 1;
pStmt->errCode = TSDB_CODE_SUCCESS;
} else {
*nums = STMT_TYPE_MULTI_INSERT == pStmt->sql.type ? 1 : 0;
_return:
if (TSDB_CODE_TSC_STMT_TBNAME_ERROR == code) {
*nums = 1;
code = TSDB_CODE_SUCCESS;
}
return TSDB_CODE_SUCCESS;
pStmt->errCode = preCode;
return code;
}
/*
int stmtGetParam(TAOS_STMT* stmt, int idx, int* type, int* bytes) {

27
source/common/src/cos.c
View File

@ -63,13 +63,10 @@ int32_t s3Begin() {
TAOS_RETURN(TSDB_CODE_SUCCESS);
}
void s3End() { (void)S3_deinitialize(); }
void s3End() { S3_deinitialize(); }
int32_t s3Init() { TAOS_RETURN(TSDB_CODE_SUCCESS); /*s3Begin();*/ }
void s3CleanUp() { /*s3End();*/
}
static int32_t s3ListBucket(char const *bucketname);
static void s3DumpCfgByEp(int8_t epIndex) {
@ -506,7 +503,9 @@ S3Status initial_multipart_callback(const char *upload_id, void *callbackData) {
}
S3Status MultipartResponseProperiesCallback(const S3ResponseProperties *properties, void *callbackData) {
(void)responsePropertiesCallbackNull(properties, callbackData);
if (S3StatusOK != responsePropertiesCallbackNull(properties, callbackData)) {
uError("%s failed at line %d to process null callback.", __func__, __LINE__);
}
MultipartPartData *data = (MultipartPartData *)callbackData;
int seq = data->seq;
@ -517,7 +516,9 @@ S3Status MultipartResponseProperiesCallback(const S3ResponseProperties *properti
}
S3Status MultipartResponseProperiesCallbackWithCp(const S3ResponseProperties *properties, void *callbackData) {
(void)responsePropertiesCallbackNull(properties, callbackData);
if (S3StatusOK != responsePropertiesCallbackNull(properties, callbackData)) {
uError("%s failed at line %d to process null callback.", __func__, __LINE__);
}
MultipartPartData *data = (MultipartPartData *)callbackData;
int seq = data->seq;
@ -897,8 +898,6 @@ upload:
if (partData.put_object_data.status != S3StatusOK) {
s3PrintError(__FILE__, __LINE__, __func__, partData.put_object_data.status, partData.put_object_data.err_msg);
TAOS_CHECK_GOTO(TAOS_SYSTEM_ERROR(EIO), &lino, _exit);
//(void)cos_cp_dump(&cp);
}
if (!manager.etags[seq - 1]) {
@ -952,7 +951,9 @@ _exit:
}
if (cp.thefile) {
(void)cos_cp_close(cp.thefile);
if (cos_cp_close(cp.thefile)) {
uError("%s failed at line %d to close cp file.", __func__, lino);
}
}
if (cp.parts) {
taosMemoryFree(cp.parts);
@ -1292,7 +1293,10 @@ int32_t s3DeleteObjects(const char *object_name[], int nobject) {
void s3DeleteObjectsByPrefix(const char *prefix) {
SArray *objectArray = getListByPrefix(prefix);
if (objectArray == NULL) return;
(void)s3DeleteObjects(TARRAY_DATA(objectArray), TARRAY_SIZE(objectArray));
int32_t code = s3DeleteObjects(TARRAY_DATA(objectArray), TARRAY_SIZE(objectArray));
if (!code) {
uError("%s failed at line %d since %s.", __func__, __LINE__, tstrerror(code));
}
taosArrayDestroyEx(objectArray, s3FreeObjectKey);
}
@ -1539,7 +1543,7 @@ int32_t s3Init() {
TAOS_RETURN(TSDB_CODE_SUCCESS);
}
void s3CleanUp() { cos_http_io_deinitialize(); }
// void s3CleanUp() { cos_http_io_deinitialize(); }
static void log_status(cos_status_t *s) {
cos_warn_log("status->code: %d", s->code);
@ -1963,7 +1967,6 @@ long s3Size(const char *object_name) {
#else
int32_t s3Init() { return 0; }
void s3CleanUp() {}
int32_t s3PutObjectFromFile(const char *file, const char *object) { return 0; }
int32_t s3PutObjectFromFile2(const char *file, const char *object, int8_t withcp) { return 0; }
int32_t s3PutObjectFromFileOffset(const char *file, const char *object_name, int64_t offset, int64_t size) { return 0; }

2
source/common/src/cos_cp.c
View File

@ -309,7 +309,7 @@ int32_t cos_cp_dump(SCheckpoint* cp) {
if (!item) {
TAOS_CHECK_GOTO(TSDB_CODE_OUT_OF_MEMORY, &lino, _exit);
}
(void)cJSON_AddItemToArray(ajson, item);
if (!cJSON_AddItemToArray(ajson, item)) goto _exit;
if (NULL == cJSON_AddNumberToObject(item, "index", cp->parts[i].index)) {
TAOS_CHECK_GOTO(TSDB_CODE_OUT_OF_MEMORY, &lino, _exit);

4
source/common/src/tdataformat.c
View File

@ -4171,12 +4171,12 @@ int32_t tValueColumnInit(SValueColumn *valCol) {
return 0;
}
int32_t tValueColumnDestroy(SValueColumn *valCol) {
void tValueColumnDestroy(SValueColumn *valCol) {
valCol->type = TSDB_DATA_TYPE_NULL;
valCol->numOfValues = 0;
tBufferDestroy(&valCol->data);
tBufferDestroy(&valCol->offsets);
return 0;
return;
}
void tValueColumnClear(SValueColumn *valCol) {

77
source/dnode/mgmt/exe/dmMain.c
View File

@ -81,11 +81,21 @@ static void dmSetAssert(int32_t signum, void *sigInfo, void *context) { tsAssert
static void dmStopDnode(int signum, void *sigInfo, void *context) {
// taosIgnSignal(SIGUSR1);
// taosIgnSignal(SIGUSR2);
(void)taosIgnSignal(SIGTERM);
(void)taosIgnSignal(SIGHUP);
(void)taosIgnSignal(SIGINT);
(void)taosIgnSignal(SIGABRT);
(void)taosIgnSignal(SIGBREAK);
if (taosIgnSignal(SIGTERM) != 0) {
dWarn("failed to ignore signal SIGTERM");
}
if (taosIgnSignal(SIGHUP) != 0) {
dWarn("failed to ignore signal SIGHUP");
}
if (taosIgnSignal(SIGINT) != 0) {
dWarn("failed to ignore signal SIGINT");
}
if (taosIgnSignal(SIGABRT) != 0) {
dWarn("failed to ignore signal SIGABRT");
}
if (taosIgnSignal(SIGBREAK) != 0) {
dWarn("failed to ignore signal SIGBREAK");
}
dInfo("shut down signal is %d", signum);
#ifndef WINDOWS
@ -103,11 +113,19 @@ void dmLogCrash(int signum, void *sigInfo, void *context) {
// taosIgnSignal(SIGBREAK);
#ifndef WINDOWS
(void)taosIgnSignal(SIGBUS);
if (taosIgnSignal(SIGBUS) != 0) {
dWarn("failed to ignore signal SIGBUS");
}
#endif
(void)taosIgnSignal(SIGABRT);
(void)taosIgnSignal(SIGFPE);
(void)taosIgnSignal(SIGSEGV);
if (taosIgnSignal(SIGABRT) != 0) {
dWarn("failed to ignore signal SIGABRT");
}
if (taosIgnSignal(SIGFPE) != 0) {
dWarn("failed to ignore signal SIGABRT");
}
if (taosIgnSignal(SIGSEGV) != 0) {
dWarn("failed to ignore signal SIGABRT");
}
char *pMsg = NULL;
const char *flags = "UTL FATAL ";
@ -136,24 +154,31 @@ _return:
}
static void dmSetSignalHandle() {
(void)taosSetSignal(SIGUSR1, dmSetDebugFlag);
(void)taosSetSignal(SIGUSR2, dmSetAssert);
(void)taosSetSignal(SIGTERM, dmStopDnode);
(void)taosSetSignal(SIGHUP, dmStopDnode);
(void)taosSetSignal(SIGINT, dmStopDnode);
(void)taosSetSignal(SIGBREAK, dmStopDnode);
if (taosSetSignal(SIGUSR1, dmSetDebugFlag) != 0) {
dWarn("failed to set signal SIGUSR1");
}
if (taosSetSignal(SIGUSR2, dmSetAssert) != 0) {
dWarn("failed to set signal SIGUSR1");
}
if (taosSetSignal(SIGTERM, dmStopDnode) != 0) {
dWarn("failed to set signal SIGUSR1");
}
if (taosSetSignal(SIGHUP, dmStopDnode) != 0) {
dWarn("failed to set signal SIGUSR1");
}
if (taosSetSignal(SIGINT, dmStopDnode) != 0) {
dWarn("failed to set signal SIGUSR1");
}
if (taosSetSignal(SIGBREAK, dmStopDnode) != 0) {
dWarn("failed to set signal SIGUSR1");
}
#ifndef WINDOWS
(void)taosSetSignal(SIGTSTP, dmStopDnode);
(void)taosSetSignal(SIGQUIT, dmStopDnode);
#endif
#if 0
#ifndef WINDOWS
(void)taosSetSignal(SIGBUS, dmLogCrash);
#endif
(void)taosSetSignal(SIGABRT, dmLogCrash);
(void)taosSetSignal(SIGFPE, dmLogCrash);
(void)taosSetSignal(SIGSEGV, dmLogCrash);
if (taosSetSignal(SIGTSTP, dmStopDnode) != 0) {
dWarn("failed to set signal SIGUSR1");
}
if (taosSetSignal(SIGQUIT, dmStopDnode) != 0) {
dWarn("failed to set signal SIGUSR1");
}
#endif
}

16
source/dnode/mgmt/mgmt_dnode/src/dmHandle.c
View File

@ -45,7 +45,9 @@ static void dmMayShouldUpdateIpWhiteList(SDnodeMgmt *pMgmt, int64_t ver) {
if (pMgmt->pData->ipWhiteVer == ver) {
if (ver == 0) {
dDebug("disable ip-white-list on dnode ver: %" PRId64 ", status ver: %" PRId64 "", pMgmt->pData->ipWhiteVer, ver);
(void)rpcSetIpWhite(pMgmt->msgCb.serverRpc, NULL);
if (rpcSetIpWhite(pMgmt->msgCb.serverRpc, NULL) != 0) {
dError("failed to disable ip white list on dnode");
}
}
return;
}
@ -91,7 +93,9 @@ static void dmProcessStatusRsp(SDnodeMgmt *pMgmt, SRpcMsg *pRsp) {
dGInfo("dnode:%d, set to dropped since not exist in mnode, statusSeq:%d", pMgmt->pData->dnodeId,
pMgmt->statusSeq);
pMgmt->pData->dropped = 1;
(void)dmWriteEps(pMgmt->pData);
if (dmWriteEps(pMgmt->pData) != 0) {
dError("failed to write dnode file");
}
dInfo("dnode will exit since it is in the dropped state");
(void)raise(SIGINT);
}
@ -147,7 +151,9 @@ void dmSendStatusReq(SDnodeMgmt *pMgmt) {
req.clusterCfg.monitorParas.tsSlowLogThresholdTest = tsSlowLogThresholdTest;
tstrncpy(req.clusterCfg.monitorParas.tsSlowLogExceptDb, tsSlowLogExceptDb, TSDB_DB_NAME_LEN);
char timestr[32] = "1970-01-01 00:00:00.00";
(void)taosParseTime(timestr, &req.clusterCfg.checkTime, (int32_t)strlen(timestr), TSDB_TIME_PRECISION_MILLI, 0);
if (taosParseTime(timestr, &req.clusterCfg.checkTime, (int32_t)strlen(timestr), TSDB_TIME_PRECISION_MILLI, 0) != 0) {
dError("failed to parse time since %s", tstrerror(code));
}
memcpy(req.clusterCfg.timezone, tsTimezoneStr, TD_TIMEZONE_LEN);
memcpy(req.clusterCfg.locale, tsLocale, TD_LOCALE_LEN);
memcpy(req.clusterCfg.charset, tsCharset, TD_LOCALE_LEN);
@ -243,7 +249,9 @@ void dmSendNotifyReq(SDnodeMgmt *pMgmt, SNotifyReq *pReq) {
SEpSet epSet = {0};
dmGetMnodeEpSet(pMgmt->pData, &epSet);
(void)rpcSendRequest(pMgmt->msgCb.clientRpc, &epSet, &rpcMsg, NULL);
if (rpcSendRequest(pMgmt->msgCb.clientRpc, &epSet, &rpcMsg, NULL) != 0) {
dError("failed to send notify req");
}
}
int32_t dmProcessAuthRsp(SDnodeMgmt *pMgmt, SRpcMsg *pMsg) {

9
source/dnode/mgmt/mgmt_dnode/src/dmWorker.c
View File

@ -305,11 +305,16 @@ int32_t dmStartNotifyThread(SDnodeMgmt *pMgmt) {
void dmStopNotifyThread(SDnodeMgmt *pMgmt) {
if (taosCheckPthreadValid(pMgmt->notifyThread)) {
(void)tsem_post(&dmNotifyHdl.sem);
if (tsem_post(&dmNotifyHdl.sem) != 0) {
dError("failed to post notify sem");
}
(void)taosThreadJoin(pMgmt->notifyThread, NULL);
taosThreadClear(&pMgmt->notifyThread);
}
(void)tsem_destroy(&dmNotifyHdl.sem);
if (tsem_destroy(&dmNotifyHdl.sem) != 0) {
dError("failed to destroy notify sem");
}
}
int32_t dmStartMonitorThread(SDnodeMgmt *pMgmt) {

4
source/dnode/mgmt/mgmt_mnode/src/mmHandle.c
View File

@ -17,7 +17,9 @@
#include "mmInt.h"
void mmGetMonitorInfo(SMnodeMgmt *pMgmt, SMonMmInfo *pInfo) {
(void)mndGetMonitorInfo(pMgmt->pMnode, &pInfo->cluster, &pInfo->vgroup, &pInfo->stb, &pInfo->grant);
if (mndGetMonitorInfo(pMgmt->pMnode, &pInfo->cluster, &pInfo->vgroup, &pInfo->stb, &pInfo->grant) != 0) {
dError("failed to get monitor info");
}
}
void mmGetMnodeLoads(SMnodeMgmt *pMgmt, SMonMloadInfo *pInfo) {

2
source/dnode/mgmt/mgmt_qnode/src/qmWorker.c
View File

@ -23,7 +23,7 @@ static inline void qmSendRsp(SRpcMsg *pMsg, int32_t code) {
.contLen = pMsg->info.rspLen,
.info = pMsg->info,
};
(void)tmsgSendRsp(&rsp);
tmsgSendRsp(&rsp);
}
static void qmProcessQueue(SQueueInfo *pInfo, SRpcMsg *pMsg) {

4
source/dnode/mgmt/mgmt_snode/src/smWorker.c
View File

@ -23,7 +23,7 @@ static inline void smSendRsp(SRpcMsg *pMsg, int32_t code) {
.contLen = pMsg->info.rspLen,
.info = pMsg->info,
};
(void)tmsgSendRsp(&rsp);
tmsgSendRsp(&rsp);
}
static void smProcessWriteQueue(SQueueInfo *pInfo, STaosQall *qall, int32_t numOfMsgs) {
@ -31,7 +31,7 @@ static void smProcessWriteQueue(SQueueInfo *pInfo, STaosQall *qall, int32_t numO
for (int32_t i = 0; i < numOfMsgs; i++) {
SRpcMsg *pMsg = NULL;
(void)taosGetQitem(qall, (void **)&pMsg);
int32_t num = taosGetQitem(qall, (void **)&pMsg);
const STraceId *trace = &pMsg->info.traceId;
dTrace("msg:%p, get from snode-write queue", pMsg);

16
source/dnode/mgmt/mgmt_vnode/src/vmHandle.c
View File

@ -35,10 +35,14 @@ void vmGetVnodeLoads(SVnodeMgmt *pMgmt, SMonVloadInfo *pInfo, bool isReset) {
SVnodeObj *pVnode = *ppVnode;
SVnodeLoad vload = {.vgId = pVnode->vgId};
if (!pVnode->failed) {
(void)vnodeGetLoad(pVnode->pImpl, &vload);
if (vnodeGetLoad(pVnode->pImpl, &vload) != 0) {
dError("failed to get vnode load");
}
if (isReset) vnodeResetLoad(pVnode->pImpl, &vload);
}
(void)taosArrayPush(pInfo->pVloads, &vload);
if (taosArrayPush(pInfo->pVloads, &vload) == NULL) {
dError("failed to push vnode load");
}
pIter = taosHashIterate(pMgmt->hash, pIter);
}
@ -116,7 +120,9 @@ void vmGetMonitorInfo(SVnodeMgmt *pMgmt, SMonVmInfo *pInfo) {
pMgmt->state.numOfBatchInsertReqs = numOfBatchInsertReqs;
pMgmt->state.numOfBatchInsertSuccessReqs = numOfBatchInsertSuccessReqs;
(void)tfsGetMonitorInfo(pMgmt->pTfs, &pInfo->tfs);
if (tfsGetMonitorInfo(pMgmt->pTfs, &pInfo->tfs) != 0) {
dError("failed to get tfs monitor info");
}
taosArrayDestroy(pVloads);
}
@ -845,7 +851,9 @@ int32_t vmProcessDropVnodeReq(SVnodeMgmt *pMgmt, SRpcMsg *pMsg) {
}
vmCloseVnode(pMgmt, pVnode, false);
(void)vmWriteVnodeListToFile(pMgmt);
if (vmWriteVnodeListToFile(pMgmt) != 0) {
dError("vgId:%d, failed to write vnode list since %s", vgId, terrstr());
}
dInfo("vgId:%d, is dropped", vgId);
return 0;

16
source/dnode/mgmt/mgmt_vnode/src/vmInt.c
View File

@ -233,8 +233,12 @@ void vmCloseVnode(SVnodeMgmt *pMgmt, SVnodeObj *pVnode, bool commitAndRemoveWal)
if (commitAndRemoveWal) {
dInfo("vgId:%d, commit data for vnode split", pVnode->vgId);
(void)vnodeSyncCommit(pVnode->pImpl);
(void)vnodeBegin(pVnode->pImpl);
if (vnodeSyncCommit(pVnode->pImpl) != 0) {
dError("vgId:%d, failed to commit data", pVnode->vgId);
}
if (vnodeBegin(pVnode->pImpl) != 0) {
dError("vgId:%d, failed to begin", pVnode->vgId);
}
dInfo("vgId:%d, commit data finished", pVnode->vgId);
}
@ -248,8 +252,12 @@ _closed:
if (commitAndRemoveWal) {
snprintf(path, TSDB_FILENAME_LEN, "vnode%svnode%d%swal", TD_DIRSEP, pVnode->vgId, TD_DIRSEP);
dInfo("vgId:%d, remove all wals, path:%s", pVnode->vgId, path);
(void)tfsRmdir(pMgmt->pTfs, path);
(void)tfsMkdir(pMgmt->pTfs, path);
if (tfsRmdir(pMgmt->pTfs, path) != 0) {
dTrace("vgId:%d, failed to remove wals, path:%s", pVnode->vgId, path);
}
if (tfsMkdir(pMgmt->pTfs, path) != 0) {
dTrace("vgId:%d, failed to create wals, path:%s", pVnode->vgId, path);
}
}
if (pVnode->dropped) {

30
source/dnode/mgmt/mgmt_vnode/src/vmWorker.c
View File

@ -187,7 +187,9 @@ static void vmProcessSyncQueue(SQueueInfo *pInfo, STaosQall *qall, int32_t numOf
static void vmSendResponse(SRpcMsg *pMsg) {
if (pMsg->info.handle) {
SRpcMsg rsp = {.info = pMsg->info, .code = terrno};
(void)rpcSendResponse(&rsp);
if (rpcSendResponse(&rsp) != 0) {
dError("failed to send response since %s", terrstr());
}
}
}
@ -389,10 +391,28 @@ int32_t vmAllocQueue(SVnodeMgmt *pMgmt, SVnodeObj *pVnode) {
SMultiWorkerCfg scfg = {.max = 1, .name = "vnode-sync", .fp = (FItems)vmProcessSyncQueue, .param = pVnode};
SMultiWorkerCfg sccfg = {.max = 1, .name = "vnode-sync-rd", .fp = (FItems)vmProcessSyncQueue, .param = pVnode};
SMultiWorkerCfg acfg = {.max = 1, .name = "vnode-apply", .fp = (FItems)vnodeApplyWriteMsg, .param = pVnode->pImpl};
(void)tMultiWorkerInit(&pVnode->pWriteW, &wcfg);
(void)tMultiWorkerInit(&pVnode->pSyncW, &scfg);
(void)tMultiWorkerInit(&pVnode->pSyncRdW, &sccfg);
(void)tMultiWorkerInit(&pVnode->pApplyW, &acfg);
code = tMultiWorkerInit(&pVnode->pWriteW, &wcfg);
if (code) {
return code;
}
code = tMultiWorkerInit(&pVnode->pSyncW, &scfg);
if (code) {
tMultiWorkerCleanup(&pVnode->pWriteW);
return code;
}
code = tMultiWorkerInit(&pVnode->pSyncRdW, &sccfg);
if (code) {
tMultiWorkerCleanup(&pVnode->pWriteW);
tMultiWorkerCleanup(&pVnode->pSyncW);
return code;
}
code = tMultiWorkerInit(&pVnode->pApplyW, &acfg);
if (code) {
tMultiWorkerCleanup(&pVnode->pWriteW);
tMultiWorkerCleanup(&pVnode->pSyncW);
tMultiWorkerCleanup(&pVnode->pSyncRdW);
return code;
}
pVnode->pQueryQ = tQueryAutoQWorkerAllocQueue(&pMgmt->queryPool, pVnode, (FItem)vmProcessQueryQueue);
pVnode->pStreamQ = tAutoQWorkerAllocQueue(&pMgmt->streamPool, pVnode, (FItem)vmProcessStreamQueue);

14
source/dnode/mgmt/node_mgmt/src/dmEnv.c
View File

@ -47,8 +47,14 @@ static int32_t dmCheckRepeatInit(SDnode *pDnode) {
}
static int32_t dmInitSystem() {
(void)taosIgnSIGPIPE();
(void)taosBlockSIGPIPE();
if (taosIgnSIGPIPE() != 0) {
dError("failed to ignore SIGPIPE");
}
if (taosBlockSIGPIPE() != 0) {
dError("failed to block SIGPIPE");
}
taosResolveCRC();
return 0;
}
@ -204,7 +210,9 @@ void dmCleanup() {
auditCleanup();
syncCleanUp();
walCleanUp();
(void)udfcClose();
if (udfcClose() != 0) {
dError("failed to close udfc");
}
udfStopUdfd();
taosStopCacheRefreshWorker();
(void)dmDiskClose();

21
source/dnode/mgmt/node_mgmt/src/dmMgmt.c
View File

@ -47,8 +47,7 @@ int32_t dmInitDnode(SDnode *pDnode) {
}
// compress module init
(void)tsCompressInit(tsLossyColumns, tsFPrecision, tsDPrecision, tsMaxRange, tsCurRange, (int)tsIfAdtFse,
tsCompressor);
tsCompressInit(tsLossyColumns, tsFPrecision, tsDPrecision, tsMaxRange, tsCurRange, (int)tsIfAdtFse, tsCompressor);
pDnode->wrappers[DNODE].func = dmGetMgmtFunc();
pDnode->wrappers[MNODE].func = mmGetMgmtFunc();
@ -226,7 +225,10 @@ void dmClearVars(SDnode *pDnode) {
(void)taosThreadRwlockDestroy(&pWrapper->lock);
}
if (pDnode->lockfile != NULL) {
(void)taosUnLockFile(pDnode->lockfile);
if (taosUnLockFile(pDnode->lockfile) != 0) {
dError("failed to unlock file");
}
(void)taosCloseFile(&pDnode->lockfile);
pDnode->lockfile = NULL;
}
@ -343,7 +345,9 @@ void dmProcessNetTestReq(SDnode *pDnode, SRpcMsg *pMsg) {
rsp.contLen = pMsg->contLen;
}
(void)rpcSendResponse(&rsp);
if (rpcSendResponse(&rsp) != 0) {
dError("failed to send response, msg:%p", &rsp);
}
rpcFreeCont(pMsg->pCont);
}
@ -360,11 +364,16 @@ void dmProcessServerStartupStatus(SDnode *pDnode, SRpcMsg *pMsg) {
} else {
rsp.pCont = rpcMallocCont(contLen);
if (rsp.pCont != NULL) {
(void)tSerializeSServerStatusRsp(rsp.pCont, contLen, &statusRsp);
if (tSerializeSServerStatusRsp(rsp.pCont, contLen, &statusRsp) < 0) {
rsp.code = TSDB_CODE_APP_ERROR;
} else {
rsp.contLen = contLen;
}
}
}
(void)rpcSendResponse(&rsp);
if (rpcSendResponse(&rsp) != 0) {
dError("failed to send response, msg:%p", &rsp);
}
rpcFreeCont(pMsg->pCont);
}

39
source/dnode/mgmt/node_mgmt/src/dmTransport.c
View File

@ -18,7 +18,11 @@
#include "qworker.h"
#include "tversion.h"
static inline void dmSendRsp(SRpcMsg *pMsg) { (void)rpcSendResponse(pMsg); }
static inline void dmSendRsp(SRpcMsg *pMsg) {
if (rpcSendResponse(pMsg) != 0) {
dError("failed to send response, msg:%p", pMsg);
}
}
static inline void dmBuildMnodeRedirectRsp(SDnode *pDnode, SRpcMsg *pMsg) {
SEpSet epSet = {0};
@ -113,7 +117,11 @@ static void dmProcessRpcMsg(SDnode *pDnode, SRpcMsg *pRpc, SEpSet *pEpSet) {
pRpc->info.handle, pRpc->contLen, pRpc->code, pRpc->info.ahandle, pRpc->info.refId);
int32_t svrVer = 0;
(void)taosVersionStrToInt(version, &svrVer);
code = taosVersionStrToInt(version, &svrVer);
if (code != 0) {
dError("failed to convert version string:%s to int, code:%d", version, code);
goto _OVER;
}
if ((code = taosCheckVersionCompatible(pRpc->info.cliVer, svrVer, 3)) != 0) {
dError("Version not compatible, cli ver: %d, svr ver: %d, ip:0x%x", pRpc->info.cliVer, svrVer,
pRpc->info.conn.clientIp);
@ -253,7 +261,9 @@ _OVER:
if (pWrapper != NULL) {
dmSendRsp(&rsp);
} else {
(void)rpcSendResponse(&rsp);
if (rpcSendResponse(&rsp) != 0) {
dError("failed to send response, msg:%p", &rsp);
}
}
}
@ -310,7 +320,9 @@ static inline int32_t dmSendReq(const SEpSet *pEpSet, SRpcMsg *pMsg) {
return code;
} else {
pMsg->info.handle = 0;
(void)rpcSendRequest(pDnode->trans.clientRpc, pEpSet, pMsg, NULL);
if (rpcSendRequest(pDnode->trans.clientRpc, pEpSet, pMsg, NULL) != 0) {
dError("failed to send rpc msg");
}
return 0;
}
}
@ -396,7 +408,9 @@ int32_t dmInitClient(SDnode *pDnode) {
rpcInit.timeToGetConn = tsTimeToGetAvailableConn;
rpcInit.notWaitAvaliableConn = 0;
(void)taosVersionStrToInt(version, &(rpcInit.compatibilityVer));
if (taosVersionStrToInt(version, &(rpcInit.compatibilityVer)) != 0) {
dError("failed to convert version string:%s to int", version);
}
pTrans->clientRpc = rpcOpen(&rpcInit);
if (pTrans->clientRpc == NULL) {
@ -440,7 +454,10 @@ int32_t dmInitStatusClient(SDnode *pDnode) {
rpcInit.supportBatch = 1;
rpcInit.batchSize = 8 * 1024;
rpcInit.timeToGetConn = tsTimeToGetAvailableConn;
(void)taosVersionStrToInt(version, &(rpcInit.compatibilityVer));
if (taosVersionStrToInt(version, &(rpcInit.compatibilityVer)) != 0) {
dError("failed to convert version string:%s to int", version);
}
pTrans->statusRpc = rpcOpen(&rpcInit);
if (pTrans->statusRpc == NULL) {
@ -485,7 +502,9 @@ int32_t dmInitSyncClient(SDnode *pDnode) {
rpcInit.supportBatch = 1;
rpcInit.batchSize = 8 * 1024;
rpcInit.timeToGetConn = tsTimeToGetAvailableConn;
(void)taosVersionStrToInt(version, &(rpcInit.compatibilityVer));
if (taosVersionStrToInt(version, &(rpcInit.compatibilityVer)) != 0) {
dError("failed to convert version string:%s to int", version);
}
pTrans->syncRpc = rpcOpen(&rpcInit);
if (pTrans->syncRpc == NULL) {
@ -536,7 +555,11 @@ int32_t dmInitServer(SDnode *pDnode) {
rpcInit.idleTime = tsShellActivityTimer * 1000;
rpcInit.parent = pDnode;
rpcInit.compressSize = tsCompressMsgSize;
(void)taosVersionStrToInt(version, &(rpcInit.compatibilityVer));
if (taosVersionStrToInt(version, &(rpcInit.compatibilityVer)) != 0) {
dError("failed to convert version string:%s to int", version);
}
pTrans->serverRpc = rpcOpen(&rpcInit);
if (pTrans->serverRpc == NULL) {
dError("failed to init dnode rpc server since:%s", tstrerror(terrno));

22
source/dnode/mgmt/node_util/src/dmEps.c
View File

@ -259,7 +259,9 @@ _OVER:
if (taosArrayGetSize(pData->dnodeEps) == 0) {
SDnodeEp dnodeEp = {0};
dnodeEp.isMnode = 1;
(void)taosGetFqdnPortFromEp(tsFirst, &dnodeEp.ep);
if (taosGetFqdnPortFromEp(tsFirst, &dnodeEp.ep) != 0) {
dError("failed to get fqdn and port from ep:%s", tsFirst);
}
if (taosArrayPush(pData->dnodeEps, &dnodeEp) == NULL) {
return terrno;
}
@ -370,11 +372,19 @@ int32_t dmGetDnodeSize(SDnodeData *pData) {
}
void dmUpdateEps(SDnodeData *pData, SArray *eps) {
(void)taosThreadRwlockWrlock(&pData->lock);
if (taosThreadRwlockWrlock(&pData->lock) != 0) {
dError("failed to lock dnode lock");
}
dDebug("new dnode list get from mnode, dnodeVer:%" PRId64, pData->dnodeVer);
dmResetEps(pData, eps);
(void)dmWriteEps(pData);
(void)taosThreadRwlockUnlock(&pData->lock);
if (dmWriteEps(pData) != 0) {
dError("failed to write dnode file");
}
if (taosThreadRwlockUnlock(&pData->lock) != 0) {
dError("failed to unlock dnode lock");
}
}
static void dmResetEps(SDnodeData *pData, SArray *dnodeEps) {
@ -590,7 +600,9 @@ void dmRemoveDnodePairs(SDnodeData *pData) {
snprintf(file, sizeof(file), "%s%sdnode%sep.json", tsDataDir, TD_DIRSEP, TD_DIRSEP);
snprintf(bak, sizeof(bak), "%s%sdnode%sep.json.bak", tsDataDir, TD_DIRSEP, TD_DIRSEP);
dInfo("dnode file:%s is rename to bak file", file);
(void)taosRenameFile(file, bak);
if (taosRenameFile(file, bak) != 0) {
dError("failed to rename dnode file:%s to bak file:%s since %s", file, bak, tstrerror(terrno));
}
}
static int32_t dmReadDnodePairs(SDnodeData *pData) {

9
source/dnode/vnode/src/inc/tsdb.h
View File

@ -949,7 +949,7 @@ int32_t tsdbBICacheRelease(SLRUCache *pCache, LRUHandle *h);
int32_t tsdbCacheGetBlockS3(SLRUCache *pCache, STsdbFD *pFD, LRUHandle **handle);
int32_t tsdbCacheGetPageS3(SLRUCache *pCache, STsdbFD *pFD, int64_t pgno, LRUHandle **handle);
int32_t tsdbCacheSetPageS3(SLRUCache *pCache, STsdbFD *pFD, int64_t pgno, uint8_t *pPage);
void tsdbCacheSetPageS3(SLRUCache *pCache, STsdbFD *pFD, int64_t pgno, uint8_t *pPage);
int32_t tsdbCacheDeleteLastrow(SLRUCache *pCache, tb_uid_t uid, TSKEY eKey);
int32_t tsdbCacheDeleteLast(SLRUCache *pCache, tb_uid_t uid, TSKEY eKey);
@ -1069,6 +1069,13 @@ int32_t tsdbSnapPrepDescription(SVnode *pVnode, SSnapshot *pSnap);
void tsdbRemoveFile(const char *path);
#define taosCloseFileWithLog(fd) \
do { \
if (taosCloseFile(fd) < 0) { \
tsdbTrace("failed to close file"); \
} \
} while (0)
#ifdef __cplusplus
}
#endif

13
source/dnode/vnode/src/meta/metaCache.c
View File

@ -216,7 +216,7 @@ void metaCacheClose(SMeta* pMeta) {
}
}
static int32_t metaRehashCache(SMetaCache* pCache, int8_t expand) {
static void metaRehashCache(SMetaCache* pCache, int8_t expand) {
int32_t code = 0;
int32_t nBucket;
@ -228,8 +228,7 @@ static int32_t metaRehashCache(SMetaCache* pCache, int8_t expand) {
SMetaCacheEntry** aBucket = (SMetaCacheEntry**)taosMemoryCalloc(nBucket, sizeof(SMetaCacheEntry*));
if (aBucket == NULL) {
code = terrno;
goto _exit;
return;
}
// rehash
@ -250,9 +249,7 @@ static int32_t metaRehashCache(SMetaCache* pCache, int8_t expand) {
taosMemoryFree(pCache->sEntryCache.aBucket);
pCache->sEntryCache.nBucket = nBucket;
pCache->sEntryCache.aBucket = aBucket;
_exit:
return code;
return;
}
int32_t metaCacheUpsert(SMeta* pMeta, SMetaInfo* pInfo) {
@ -279,7 +276,7 @@ int32_t metaCacheUpsert(SMeta* pMeta, SMetaInfo* pInfo) {
}
} else { // insert
if (pCache->sEntryCache.nEntry >= pCache->sEntryCache.nBucket) {
TAOS_UNUSED(metaRehashCache(pCache, 1));
metaRehashCache(pCache, 1);
iBucket = TABS(pInfo->uid) % pCache->sEntryCache.nBucket;
}
@ -317,7 +314,7 @@ int32_t metaCacheDrop(SMeta* pMeta, int64_t uid) {
pCache->sEntryCache.nEntry--;
if (pCache->sEntryCache.nEntry < pCache->sEntryCache.nBucket / 4 &&
pCache->sEntryCache.nBucket > META_CACHE_BASE_BUCKET) {
TAOS_UNUSED(metaRehashCache(pCache, 0));
metaRehashCache(pCache, 0);
}
} else {
code = TSDB_CODE_NOT_FOUND;

4
source/dnode/vnode/src/meta/metaOpen.c
View File

@ -60,7 +60,7 @@ int32_t metaOpen(SVnode *pVnode, SMeta **ppMeta, int8_t rollback) {
pMeta->path = (char *)&pMeta[1];
strcpy(pMeta->path, path);
(void)taosRealPath(pMeta->path, NULL, strlen(path) + 1);
int32_t ret = taosRealPath(pMeta->path, NULL, strlen(path) + 1);
pMeta->pVnode = pVnode;
@ -98,7 +98,7 @@ int32_t metaOpen(SVnode *pVnode, SMeta **ppMeta, int8_t rollback) {
TSDB_CHECK_CODE(code, lino, _exit);
sprintf(indexFullPath, "%s/%s", pMeta->path, "invert");
TAOS_UNUSED(taosMkDir(indexFullPath));
ret = taosMkDir(indexFullPath);
SIndexOpts opts = {.cacheSize = 8 * 1024 * 1024};
code = indexOpen(&opts, indexFullPath, (SIndex **)&pMeta->pTagIvtIdx);

12
source/dnode/vnode/src/meta/metaQuery.c
View File

@ -62,7 +62,10 @@ int metaGetTableEntryByVersion(SMetaReader *pReader, int64_t version, tb_uid_t u
tDecoderInit(&pReader->coder, pReader->pBuf, pReader->szBuf);
code = metaDecodeEntry(&pReader->coder, &pReader->me);
if (code) return code;
if (code) {
tDecoderClear(&pReader->coder);
return code;
}
// taosMemoryFreeClear(pReader->me.colCmpr.pColCmpr);
return 0;
@ -393,6 +396,7 @@ _query:
tDecoderInit(&dc, pData, nData);
int32_t code = metaDecodeEntry(&dc, &me);
if (code) {
tDecoderClear(&dc);
goto _err;
}
if (me.type == TSDB_SUPER_TABLE) {
@ -1277,7 +1281,11 @@ int32_t metaFilterTableIds(void *pVnode, SMetaFltParam *arg, SArray *pUids) {
tDecoderInit(&dc, pData, nData);
TAOS_CHECK_GOTO(metaDecodeEntry(&dc, &oStbEntry), NULL, END);
code = metaDecodeEntry(&dc, &oStbEntry);
if (code) {
tDecoderClear(&dc);
goto END;
}
if (oStbEntry.stbEntry.schemaTag.pSchema == NULL || oStbEntry.stbEntry.schemaTag.pSchema == NULL) {
TAOS_CHECK_GOTO(TSDB_CODE_INVALID_PARA, NULL, END);

1
source/dnode/vnode/src/meta/metaSnapshot.c
View File

@ -605,6 +605,7 @@ int32_t getTableInfoFromSnapshot(SSnapContext* ctx, void** pBuf, int32_t* contLe
tDecoderInit(&dc, pVal, vLen);
ret = metaDecodeEntry(&dc, &me);
if (ret < 0) {
tDecoderClear(&dc);
ret = TAOS_GET_TERRNO(ret);
goto END;
}

8
source/dnode/vnode/src/meta/metaTable.c
View File

@ -285,7 +285,9 @@ static inline void metaTimeSeriesNotifyCheck(SMeta *pMeta) {
int64_t deltaTS = nTimeSeries - pMeta->pVnode->config.vndStats.numOfReportedTimeSeries;
if (deltaTS > tsTimeSeriesThreshold) {
if (0 == atomic_val_compare_exchange_8(&dmNotifyHdl.state, 1, 2)) {
(void)tsem_post(&dmNotifyHdl.sem);
if (tsem_post(&dmNotifyHdl.sem) != 0) {
metaError("vgId:%d, failed to post semaphore, errno:%d", TD_VID(pMeta->pVnode), errno);
}
}
}
#endif
@ -563,6 +565,7 @@ int metaAlterSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
if (ret < 0) {
metaError("vgId:%d, failed to decode stb:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name,
pReq->suid, tstrerror(ret));
tDecoderClear(&dc);
tdbTbcClose(pTbDbc);
tdbTbcClose(pUidIdxc);
return terrno;
@ -1502,6 +1505,7 @@ static int metaDropTableByUid(SMeta *pMeta, tb_uid_t uid, int *type, tb_uid_t *p
tDecoderInit(&tdc, tData, tLen);
int32_t ret = metaDecodeEntry(&tdc, &stbEntry);
if (ret < 0) {
tDecoderClear(&tdc);
metaError("vgId:%d, failed to decode child table:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), e.name,
e.ctbEntry.suid, tstrerror(ret));
return ret;
@ -2397,6 +2401,7 @@ static int metaAddTagIndex(SMeta *pMeta, int64_t version, SVAlterTbReq *pAlterTb
tDecoderInit(&dc, pVal, nVal);
ret = metaDecodeEntry(&dc, &stbEntry);
if (ret < 0) {
tDecoderClear(&dc);
goto _err;
}
@ -2538,6 +2543,7 @@ static int metaDropTagIndex(SMeta *pMeta, int64_t version, SVAlterTbReq *pAlterT
tDecoderInit(&dc, pVal, nVal);
ret = metaDecodeEntry(&dc, &stbEntry);
if (ret < 0) {
tDecoderClear(&dc);
goto _err;
}

13
source/dnode/vnode/src/sma/smaRollup.c
View File

@ -820,7 +820,10 @@ static int32_t tdExecuteRSmaImplAsync(SSma *pSma, int64_t version, const void *p
int64_t nItems = atomic_fetch_add_64(&pRSmaStat->nBufItems, 1);
if (atomic_load_8(&pInfo->assigned) == 0) {
(void)tsem_post(&(pRSmaStat->notEmpty));
if (tsem_post(&(pRSmaStat->notEmpty)) != 0) {
smaError("vgId:%d, failed to post notEmpty semaphore for rsma %" PRIi64 " since %s", SMA_VID(pSma), suid,
tstrerror(terrno));
}
}
// smoothing consume
@ -1385,7 +1388,8 @@ static void tdRSmaFetchTrigger(void *param, void *tmrId) {
if (rsmaTriggerStat == TASK_TRIGGER_STAT_PAUSED) {
bool ret = taosTmrReset(tdRSmaFetchTrigger, RSMA_FETCH_INTERVAL, pItem, smaMgmt.tmrHandle, &pItem->tmrId);
if (!ret) {
smaWarn("vgId:%d, rsma fetch task not reset for level %" PRIi8 " since tmr reset failed, rsetId:%d refId:%" PRIi64,
smaWarn("vgId:%d, rsma fetch task not reset for level %" PRIi8
" since tmr reset failed, rsetId:%d refId:%" PRIi64,
SMA_VID(pSma), pItem->level, smaMgmt.rsetId, pRSmaRef->refId);
}
}
@ -1407,7 +1411,10 @@ static void tdRSmaFetchTrigger(void *param, void *tmrId) {
atomic_store_8(&pItem->fetchLevel, 1);
if (atomic_load_8(&pRSmaInfo->assigned) == 0) {
(void)tsem_post(&(pStat->notEmpty));
if (tsem_post(&(pStat->notEmpty)) != 0) {
smaError("vgId:%d, rsma fetch task not start for level:%" PRIi8 " suid:%" PRIi64 " since sem post failed",
SMA_VID(pSma), pItem->level, pRSmaInfo->suid);
}
}
} break;
case TASK_TRIGGER_STAT_INACTIVE: {

7
source/dnode/vnode/src/tsdb/tsdbCache.c
View File

@ -3682,8 +3682,7 @@ int32_t tsdbCacheGetPageS3(SLRUCache *pCache, STsdbFD *pFD, int64_t pgno, LRUHan
return code;
}
int32_t tsdbCacheSetPageS3(SLRUCache *pCache, STsdbFD *pFD, int64_t pgno, uint8_t *pPage) {
int32_t code = 0;
void tsdbCacheSetPageS3(SLRUCache *pCache, STsdbFD *pFD, int64_t pgno, uint8_t *pPage) {
char key[128] = {0};
int keyLen = 0;
LRUHandle *handle = NULL;
@ -3696,7 +3695,7 @@ int32_t tsdbCacheSetPageS3(SLRUCache *pCache, STsdbFD *pFD, int64_t pgno, uint8_
_taos_lru_deleter_t deleter = deleteBCache;
uint8_t *pPg = taosMemoryMalloc(charge);
if (!pPg) {
TAOS_RETURN(terrno);
return; // ignore error with s3 cache and leave error untouched
}
memcpy(pPg, pPage, charge);
@ -3710,6 +3709,4 @@ int32_t tsdbCacheSetPageS3(SLRUCache *pCache, STsdbFD *pFD, int64_t pgno, uint8_
(void)taosThreadMutexUnlock(&pFD->pTsdb->pgMutex);
tsdbCacheRelease(pFD->pTsdb->pgCache, handle);
TAOS_RETURN(code);
}

36
source/dnode/vnode/src/tsdb/tsdbCommit2.c
View File

@ -199,9 +199,9 @@ _exit:
return code;
}
static int32_t tsdbCommitCloseReader(SCommitter2 *committer) {
static void tsdbCommitCloseReader(SCommitter2 *committer) {
TARRAY2_CLEAR(committer->sttReaderArray, tsdbSttFileReaderClose);
return 0;
return;
}
static int32_t tsdbCommitOpenReader(SCommitter2 *committer) {
@ -243,19 +243,19 @@ static int32_t tsdbCommitOpenReader(SCommitter2 *committer) {
_exit:
if (code) {
TAOS_UNUSED(tsdbCommitCloseReader(committer));
tsdbCommitCloseReader(committer);
tsdbError("vgId:%d %s failed at %s:%d since %s", TD_VID(committer->tsdb->pVnode), __func__, __FILE__, lino,
tstrerror(code));
}
return code;
}
static int32_t tsdbCommitCloseIter(SCommitter2 *committer) {
static void tsdbCommitCloseIter(SCommitter2 *committer) {
tsdbIterMergerClose(&committer->tombIterMerger);
tsdbIterMergerClose(&committer->dataIterMerger);
TARRAY2_CLEAR(committer->tombIterArray, tsdbIterClose);
TARRAY2_CLEAR(committer->dataIterArray, tsdbIterClose);
return 0;
return;
}
static int32_t tsdbCommitOpenIter(SCommitter2 *committer) {
@ -309,7 +309,7 @@ static int32_t tsdbCommitOpenIter(SCommitter2 *committer) {
_exit:
if (code) {
TAOS_UNUSED(tsdbCommitCloseIter(committer));
tsdbCommitCloseIter(committer);
tsdbError("vgId:%d %s failed at %s:%d since %s", TD_VID(committer->tsdb->pVnode), __func__, __FILE__, lino,
tstrerror(code));
}
@ -322,7 +322,7 @@ static int32_t tsdbCommitFileSetBegin(SCommitter2 *committer) {
STsdb *tsdb = committer->tsdb;
// check if can commit
TAOS_UNUSED(tsdbFSCheckCommit(tsdb, committer->ctx->info->fid));
tsdbFSCheckCommit(tsdb, committer->ctx->info->fid);
committer->ctx->expLevel = tsdbFidLevel(committer->ctx->info->fid, &tsdb->keepCfg, committer->now);
tsdbFidKeyRange(committer->ctx->info->fid, committer->minutes, committer->precision, &committer->ctx->minKey,
@ -355,8 +355,8 @@ static int32_t tsdbCommitFileSetEnd(SCommitter2 *committer) {
int32_t lino = 0;
TAOS_CHECK_GOTO(tsdbCommitCloseWriter(committer), &lino, _exit);
TAOS_CHECK_GOTO(tsdbCommitCloseIter(committer), &lino, _exit);
TAOS_CHECK_GOTO(tsdbCommitCloseReader(committer), &lino, _exit);
tsdbCommitCloseIter(committer);
tsdbCommitCloseReader(committer);
_exit:
if (code) {
@ -409,11 +409,11 @@ static uint32_t tFileSetCommitInfoHash(const void *arg) {
return MurmurHash3_32((const char *)&info->fid, sizeof(info->fid));
}
static int32_t tsdbCommitInfoDestroy(STsdb *pTsdb) {
static void tsdbCommitInfoDestroy(STsdb *pTsdb) {
if (pTsdb->commitInfo) {
for (int32_t i = 0; i < taosArrayGetSize(pTsdb->commitInfo->arr); i++) {
SFileSetCommitInfo *info = *(SFileSetCommitInfo **)taosArrayGet(pTsdb->commitInfo->arr, i);
TAOS_UNUSED(vHashDrop(pTsdb->commitInfo->ht, info));
int32_t ret = vHashDrop(pTsdb->commitInfo->ht, info);
tsdbTFileSetClear(&info->fset);
taosMemoryFree(info);
}
@ -423,7 +423,7 @@ static int32_t tsdbCommitInfoDestroy(STsdb *pTsdb) {
pTsdb->commitInfo->arr = NULL;
taosMemoryFreeClear(pTsdb->commitInfo);
}
return 0;
return;
}
static int32_t tsdbCommitInfoInit(STsdb *pTsdb) {
@ -444,7 +444,7 @@ static int32_t tsdbCommitInfoInit(STsdb *pTsdb) {
_exit:
if (code) {
TAOS_UNUSED(tsdbCommitInfoDestroy(pTsdb));
tsdbCommitInfoDestroy(pTsdb);
tsdbError("vgId:%d %s failed at %s:%d since %s", TD_VID(pTsdb->pVnode), __func__, __FILE__, lino, tstrerror(code));
}
return code;
@ -514,7 +514,7 @@ static int32_t tsdbCommitInfoBuild(STsdb *tsdb) {
SFileSetCommitInfo tinfo = {
.fid = fid,
};
TAOS_UNUSED(vHashGet(tsdb->commitInfo->ht, &tinfo, (void **)&info));
int32_t ret = vHashGet(tsdb->commitInfo->ht, &tinfo, (void **)&info);
if (info == NULL) {
TAOS_CHECK_GOTO(tsdbCommitInfoAdd(tsdb, fid), &lino, _exit);
}
@ -538,7 +538,7 @@ static int32_t tsdbCommitInfoBuild(STsdb *tsdb) {
};
// check if the file set already on the commit list
TAOS_UNUSED(vHashGet(tsdb->commitInfo->ht, &tinfo, (void **)&info));
int32_t ret = vHashGet(tsdb->commitInfo->ht, &tinfo, (void **)&info);
if (info != NULL) {
continue;
}
@ -586,7 +586,7 @@ static int32_t tsdbCommitInfoBuild(STsdb *tsdb) {
_exit:
if (code) {
TAOS_UNUSED(tsdbCommitInfoDestroy(tsdb));
tsdbCommitInfoDestroy(tsdb);
tsdbError("vgId:%d %s failed at %s:%d since %s", TD_VID(tsdb->pVnode), __func__, __FILE__, lino, tstrerror(code));
}
return code;
@ -716,7 +716,7 @@ int32_t tsdbCommitCommit(STsdb *tsdb) {
(void)taosThreadMutexUnlock(&tsdb->mutex);
TAOS_UNUSED(tsdbCommitInfoDestroy(tsdb));
tsdbCommitInfoDestroy(tsdb);
tsdbUnrefMemTable(pMemTable, NULL, true);
}
@ -745,7 +745,7 @@ int32_t tsdbCommitAbort(STsdb *pTsdb) {
}
}
(void)taosThreadMutexUnlock(&pTsdb->mutex);
TAOS_UNUSED(tsdbCommitInfoDestroy(pTsdb));
tsdbCommitInfoDestroy(pTsdb);
_exit:
if (code) {

12
source/dnode/vnode/src/tsdb/tsdbFS.c
View File

@ -130,7 +130,7 @@ _exit:
tsdbError("%s failed at line %d since %s, fname:%s", __func__, lino, tstrerror(code), fname);
}
taosMemoryFree(pData);
(void)taosCloseFile(&pFD);
taosCloseFileWithLog(&pFD);
return code;
}
@ -300,26 +300,26 @@ static int32_t load_fs(const char *fname, STsdbFS *pFS) {
int64_t size;
code = taosFStatFile(pFD, &size, NULL);
if (code != 0) {
(void)taosCloseFile(&pFD);
taosCloseFileWithLog(&pFD);
TSDB_CHECK_CODE(code, lino, _exit);
}
pData = taosMemoryMalloc(size);
if (pData == NULL) {
code = terrno;
(void)taosCloseFile(&pFD);
taosCloseFileWithLog(&pFD);
TSDB_CHECK_CODE(code, lino, _exit);
}
if (taosReadFile(pFD, pData, size) < 0) {
code = terrno;
(void)taosCloseFile(&pFD);
taosCloseFileWithLog(&pFD);
TSDB_CHECK_CODE(code, lino, _exit);
}
if (!taosCheckChecksumWhole(pData, size)) {
code = TSDB_CODE_FILE_CORRUPTED;
(void)taosCloseFile(&pFD);
taosCloseFileWithLog(&pFD);
TSDB_CHECK_CODE(code, lino, _exit);
}
@ -331,7 +331,7 @@ _exit:
tsdbError("%s failed at line %d since %s, fname:%s", __func__, lino, tstrerror(code), fname);
}
taosMemoryFree(pData);
(void)taosCloseFile(&pFD);
taosCloseFileWithLog(&pFD);
return code;
}

37
source/dnode/vnode/src/tsdb/tsdbFS2.c
View File

@ -44,7 +44,12 @@ static int32_t create_fs(STsdb *pTsdb, STFileSystem **fs) {
}
fs[0]->tsdb = pTsdb;
(void)tsem_init(&fs[0]->canEdit, 0, 1);
int32_t code = tsem_init(&fs[0]->canEdit, 0, 1);
if (code) {
taosMemoryFree(fs[0]);
return code;
}
fs[0]->fsstate = TSDB_FS_STATE_NORMAL;
fs[0]->neid = 0;
TARRAY2_INIT(fs[0]->fSetArr);
@ -58,7 +63,9 @@ static void destroy_fs(STFileSystem **fs) {
TARRAY2_DESTROY(fs[0]->fSetArr, NULL);
TARRAY2_DESTROY(fs[0]->fSetArrTmp, NULL);
(void)tsem_destroy(&fs[0]->canEdit);
if (tsem_destroy(&fs[0]->canEdit) != 0) {
tsdbError("failed to destroy semaphore");
}
taosMemoryFree(fs[0]);
fs[0] = NULL;
}
@ -100,7 +107,7 @@ _exit:
tsdbError("%s failed at %s:%d since %s", __func__, fname, __LINE__, tstrerror(code));
}
taosMemoryFree(data);
(void)taosCloseFile(&fp);
taosCloseFileWithLog(&fp);
return code;
}
@ -140,7 +147,7 @@ _exit:
tsdbError("%s failed at %s:%d since %s", __func__, fname, __LINE__, tstrerror(code));
json[0] = NULL;
}
(void)taosCloseFile(&fp);
taosCloseFileWithLog(&fp);
taosMemoryFree(data);
return code;
}
@ -803,7 +810,11 @@ void tsdbEnableBgTask(STsdb *pTsdb) {
void tsdbCloseFS(STFileSystem **fs) {
if (fs[0] == NULL) return;
TAOS_UNUSED(tsdbDisableAndCancelAllBgTask((*fs)->tsdb));
int32_t code = tsdbDisableAndCancelAllBgTask((*fs)->tsdb);
if (code) {
tsdbError("vgId:%d %s failed at line %d since %s", TD_VID((*fs)->tsdb->pVnode), __func__, __LINE__,
tstrerror(code));
}
close_file_system(fs[0]);
destroy_fs(fs);
return;
@ -833,7 +844,9 @@ int32_t tsdbFSEditBegin(STFileSystem *fs, const TFileOpArray *opArray, EFEditT e
current_fname(fs->tsdb, current_t, TSDB_FCURRENT_M);
}
(void)tsem_wait(&fs->canEdit);
if (tsem_wait(&fs->canEdit) != 0) {
tsdbError("vgId:%d failed to wait semaphore", TD_VID(fs->tsdb->pVnode));
}
fs->etype = etype;
// edit
@ -865,7 +878,7 @@ static void tsdbFSSetBlockCommit(STFileSet *fset, bool block) {
}
}
int32_t tsdbFSCheckCommit(STsdb *tsdb, int32_t fid) {
void tsdbFSCheckCommit(STsdb *tsdb, int32_t fid) {
(void)taosThreadMutexLock(&tsdb->mutex);
STFileSet *fset;
tsdbFSGetFSet(tsdb->pFS, fid, &fset);
@ -877,7 +890,7 @@ int32_t tsdbFSCheckCommit(STsdb *tsdb, int32_t fid) {
}
}
(void)taosThreadMutexUnlock(&tsdb->mutex);
return 0;
return;
}
// IMPORTANT: the caller must hold fs->tsdb->mutex
@ -939,13 +952,17 @@ _exit:
} else {
tsdbInfo("vgId:%d %s done, etype:%d", TD_VID(fs->tsdb->pVnode), __func__, fs->etype);
}
(void)tsem_post(&fs->canEdit);
if (tsem_post(&fs->canEdit) != 0) {
tsdbError("vgId:%d failed to post semaphore", TD_VID(fs->tsdb->pVnode));
}
return code;
}
int32_t tsdbFSEditAbort(STFileSystem *fs) {
int32_t code = abort_edit(fs);
(void)tsem_post(&fs->canEdit);
if (tsem_post(&fs->canEdit) != 0) {
tsdbError("vgId:%d failed to post semaphore", TD_VID(fs->tsdb->pVnode));
}
return code;
}

2
source/dnode/vnode/src/tsdb/tsdbFS2.h
View File

@ -60,7 +60,7 @@ int32_t tsdbFSEditCommit(STFileSystem *fs);
int32_t tsdbFSEditAbort(STFileSystem *fs);
// other
void tsdbFSGetFSet(STFileSystem *fs, int32_t fid, STFileSet **fset);
int32_t tsdbFSCheckCommit(STsdb *tsdb, int32_t fid);
void tsdbFSCheckCommit(STsdb *tsdb, int32_t fid);
void tsdbBeginTaskOnFileSet(STsdb *tsdb, int32_t fid, STFileSet **fset);
void tsdbFinishTaskOnFileSet(STsdb *tsdb, int32_t fid);
// utils

6
source/dnode/vnode/src/tsdb/tsdbReaderWriter.c
View File

@ -436,7 +436,7 @@ static int32_t tsdbReadFileS3(STsdbFD *pFD, int64_t offset, uint8_t *pBuf, int64
code = tsdbCacheGetPageS3(pFD->pTsdb->pgCache, pFD, pgno, &handle);
if (code != TSDB_CODE_SUCCESS) {
if (handle) {
(void)tsdbCacheRelease(pFD->pTsdb->pgCache, handle);
tsdbCacheRelease(pFD->pTsdb->pgCache, handle);
}
TSDB_CHECK_CODE(code, lino, _exit);
}
@ -447,7 +447,7 @@ static int32_t tsdbReadFileS3(STsdbFD *pFD, int64_t offset, uint8_t *pBuf, int64
uint8_t *pPage = (uint8_t *)taosLRUCacheValue(pFD->pTsdb->pgCache, handle);
memcpy(pFD->pBuf, pPage, pFD->szPage);
(void)tsdbCacheRelease(pFD->pTsdb->pgCache, handle);
tsdbCacheRelease(pFD->pTsdb->pgCache, handle);
// check
if (pgno > 1 && !taosCheckChecksumWhole(pFD->pBuf, pFD->szPage)) {
@ -483,7 +483,7 @@ static int32_t tsdbReadFileS3(STsdbFD *pFD, int64_t offset, uint8_t *pBuf, int64
int nPage = pgnoEnd - pgno + 1;
for (int i = 0; i < nPage; ++i) {
if (pFD->szFile != pgno) { // DONOT cache last volatile page
(void)tsdbCacheSetPageS3(pFD->pTsdb->pgCache, pFD, pgno, pBlock + i * pFD->szPage);
tsdbCacheSetPageS3(pFD->pTsdb->pgCache, pFD, pgno, pBlock + i * pFD->szPage);
}
if (szHint > 0 && n >= size) {

41
source/dnode/vnode/src/tsdb/tsdbRetention.c
View File

@ -99,8 +99,12 @@ _exit:
tsdbError("vgId:%d, %s failed at %s:%d since %s", TD_VID(rtner->tsdb->pVnode), __func__, __FILE__, lino,
tstrerror(code));
}
(void)taosCloseFile(&fdFrom);
(void)taosCloseFile(&fdTo);
if (taosCloseFile(&fdFrom) != 0) {
tsdbError("vgId:%d, failed to close file %s", TD_VID(rtner->tsdb->pVnode), fname_from);
}
if (taosCloseFile(&fdTo) != 0) {
tsdbError("vgId:%d, failed to close file %s", TD_VID(rtner->tsdb->pVnode), fname_to);
}
return code;
}
@ -136,8 +140,12 @@ _exit:
tsdbError("vgId:%d, %s failed at %s:%d since %s", TD_VID(rtner->tsdb->pVnode), __func__, __FILE__, lino,
tstrerror(code));
}
(void)taosCloseFile(&fdFrom);
(void)taosCloseFile(&fdTo);
if (taosCloseFile(&fdFrom) != 0) {
tsdbTrace("vgId:%d, failed to close file", TD_VID(rtner->tsdb->pVnode));
}
if (taosCloseFile(&fdTo) != 0) {
tsdbTrace("vgId:%d, failed to close file", TD_VID(rtner->tsdb->pVnode));
}
return code;
}
@ -441,7 +449,9 @@ _exit:
tsdbError("vgId:%d %s failed at line %s:%d since %s", TD_VID(rtner->tsdb->pVnode), __func__, __FILE__, lino,
tstrerror(code));
}
(void)taosCloseFile(&fdFrom);
if (taosCloseFile(&fdFrom) != 0) {
tsdbTrace("vgId:%d, failed to close file", TD_VID(rtner->tsdb->pVnode));
}
return code;
}
@ -541,8 +551,13 @@ _exit:
tsdbError("vgId:%d %s failed at line %s:%d since %s", TD_VID(rtner->tsdb->pVnode), __func__, __FILE__, lino,
tstrerror(code));
}
(void)taosCloseFile(&fdFrom);
(void)taosCloseFile(&fdTo);
if (taosCloseFile(&fdFrom) != 0) {
tsdbTrace("vgId:%d, failed to close file", TD_VID(rtner->tsdb->pVnode));
}
if (taosCloseFile(&fdTo) != 0) {
tsdbTrace("vgId:%d, failed to close file", TD_VID(rtner->tsdb->pVnode));
}
return code;
}
@ -639,8 +654,12 @@ _exit:
tsdbError("vgId:%d %s failed at line %s:%d since %s", TD_VID(rtner->tsdb->pVnode), __func__, __FILE__, lino,
tstrerror(code));
}
(void)taosCloseFile(&fdFrom);
(void)taosCloseFile(&fdTo);
if (taosCloseFile(&fdFrom) != 0) {
tsdbTrace("vgId:%d, failed to close file", TD_VID(rtner->tsdb->pVnode));
}
if (taosCloseFile(&fdTo) != 0) {
tsdbTrace("vgId:%d, failed to close file", TD_VID(rtner->tsdb->pVnode));
}
return code;
}
@ -699,7 +718,9 @@ static int32_t tsdbDoS3Migrate(SRTNer *rtner) {
if (taosCheckExistFile(fname1)) {
int32_t mtime = 0;
int64_t size = 0;
(void)taosStatFile(fname1, &size, &mtime, NULL);
if (taosStatFile(fname1, &size, &mtime, NULL) != 0) {
tsdbError("vgId:%d, %s failed at %s:%d ", TD_VID(rtner->tsdb->pVnode), __func__, __FILE__, __LINE__);
}
if (size > chunksize && mtime < rtner->now - tsS3UploadDelaySec) {
TAOS_CHECK_GOTO(tsdbMigrateDataFileLCS3(rtner, fobj, size, chunksize), &lino, _exit);
}

20
source/dnode/vnode/src/tsdb/tsdbSnapshot.c
View File

@ -57,10 +57,10 @@ struct STsdbSnapReader {
STombBlock tombBlock[1];
};
static int32_t tsdbSnapReadFileSetCloseReader(STsdbSnapReader* reader) {
static void tsdbSnapReadFileSetCloseReader(STsdbSnapReader* reader) {
TARRAY2_CLEAR(reader->sttReaderArr, tsdbSttFileReaderClose);
tsdbDataFileReaderClose(&reader->dataReader);
return 0;
return;
}
static int32_t tsdbSnapReadFileSetOpenReader(STsdbSnapReader* reader) {
@ -112,7 +112,7 @@ static int32_t tsdbSnapReadFileSetOpenReader(STsdbSnapReader* reader) {
_exit:
if (code) {
TAOS_UNUSED(tsdbSnapReadFileSetCloseReader(reader));
tsdbSnapReadFileSetCloseReader(reader);
TSDB_ERROR_LOG(TD_VID(reader->tsdb->pVnode), code, lino);
}
return code;
@ -190,12 +190,12 @@ _exit:
return code;
}
static int32_t tsdbSnapReadFileSetCloseIter(STsdbSnapReader* reader) {
static void tsdbSnapReadFileSetCloseIter(STsdbSnapReader* reader) {
tsdbIterMergerClose(&reader->dataIterMerger);
tsdbIterMergerClose(&reader->tombIterMerger);
TARRAY2_CLEAR(reader->dataIterArr, tsdbIterClose);
TARRAY2_CLEAR(reader->tombIterArr, tsdbIterClose);
return 0;
return;
}
static int32_t tsdbSnapReadRangeBegin(STsdbSnapReader* reader) {
@ -222,8 +222,8 @@ _exit:
}
static int32_t tsdbSnapReadRangeEnd(STsdbSnapReader* reader) {
TAOS_UNUSED(tsdbSnapReadFileSetCloseIter(reader));
TAOS_UNUSED(tsdbSnapReadFileSetCloseReader(reader));
tsdbSnapReadFileSetCloseIter(reader);
tsdbSnapReadFileSetCloseReader(reader);
reader->ctx->fsr = NULL;
return 0;
}
@ -373,7 +373,7 @@ static int32_t tsdbSnapReadTombData(STsdbSnapReader* reader, uint8_t** data) {
int32_t lino = 0;
SMetaInfo info;
TAOS_UNUSED(tTombBlockClear(reader->tombBlock));
tTombBlockClear(reader->tombBlock);
TABLEID tbid[1] = {0};
for (STombRecord* record; (record = tsdbIterMergerGetTombRecord(reader->tombIterMerger)) != NULL;) {
@ -463,7 +463,7 @@ void tsdbSnapReaderClose(STsdbSnapReader** reader) {
tDestroyTSchema(reader[0]->skmTb->pTSchema);
for (int32_t i = 0; i < ARRAY_SIZE(reader[0]->buffers); ++i) {
TAOS_UNUSED(tBufferDestroy(reader[0]->buffers + i));
tBufferDestroy(reader[0]->buffers + i);
}
taosMemoryFree(reader[0]);
@ -1000,7 +1000,7 @@ static int32_t tsdbSnapWriteDecmprTombBlock(SSnapDataHdr* hdr, STombBlock* tombB
int32_t code = 0;
int32_t lino = 0;
TAOS_UNUSED(tTombBlockClear(tombBlock));
tTombBlockClear(tombBlock);
int64_t size = hdr->size;
size = size / TOMB_RECORD_ELEM_NUM;

4
source/dnode/vnode/src/tsdb/tsdbSttFileRW.c
View File

@ -415,7 +415,7 @@ int32_t tsdbSttFileReadStatisBlock(SSttFileReader *reader, const SStatisBlk *sta
&lino, _exit);
// decode data
TAOS_UNUSED(tStatisBlockClear(statisBlock));
tStatisBlockClear(statisBlock);
statisBlock->numOfPKs = statisBlk->numOfPKs;
statisBlock->numOfRecords = statisBlk->numRec;
SBufferReader br = BUFFER_READER_INITIALIZER(0, buffer0);
@ -654,7 +654,7 @@ static int32_t tsdbSttFileDoWriteStatisBlock(SSttFileWriter *writer) {
TAOS_CHECK_GOTO(TARRAY2_APPEND_PTR(writer->statisBlkArray, &statisBlk), &lino, _exit);
TAOS_UNUSED(tStatisBlockClear(writer->staticBlock));
tStatisBlockClear(writer->staticBlock);
_exit:
if (code) {

5
source/dnode/vnode/src/tsdb/tsdbUtil.c
View File

@ -609,7 +609,10 @@ void tsdbRowGetColVal(TSDBROW *pRow, STSchema *pTSchema, int32_t iCol, SColVal *
SValue value;
if (pRow->type == TSDBROW_ROW_FMT) {
(void)tRowGet(pRow->pTSRow, pTSchema, iCol, pColVal);
int32_t ret = tRowGet(pRow->pTSRow, pTSchema, iCol, pColVal);
if (ret != 0) {
tsdbError("failed to get column value, code:%d", ret);
}
} else if (pRow->type == TSDBROW_COL_FMT) {
if (iCol == 0) {
*pColVal =

20
source/dnode/vnode/src/tsdb/tsdbUtil2.c
View File

@ -93,25 +93,25 @@ void tStatisBlockDestroy(STbStatisBlock *statisBlock) {
statisBlock->numOfPKs = 0;
statisBlock->numOfRecords = 0;
for (int32_t i = 0; i < ARRAY_SIZE(statisBlock->buffers); ++i) {
TAOS_UNUSED(tBufferDestroy(&statisBlock->buffers[i]));
tBufferDestroy(&statisBlock->buffers[i]);
}
for (int32_t i = 0; i < TD_MAX_PK_COLS; ++i) {
TAOS_UNUSED(tValueColumnDestroy(&statisBlock->firstKeyPKs[i]));
TAOS_UNUSED(tValueColumnDestroy(&statisBlock->lastKeyPKs[i]));
tValueColumnDestroy(&statisBlock->firstKeyPKs[i]);
tValueColumnDestroy(&statisBlock->lastKeyPKs[i]);
}
}
int32_t tStatisBlockClear(STbStatisBlock *statisBlock) {
void tStatisBlockClear(STbStatisBlock *statisBlock) {
statisBlock->numOfPKs = 0;
statisBlock->numOfRecords = 0;
for (int32_t i = 0; i < ARRAY_SIZE(statisBlock->buffers); ++i) {
TAOS_UNUSED(tBufferClear(&statisBlock->buffers[i]));
tBufferClear(&statisBlock->buffers[i]);
}
for (int32_t i = 0; i < TD_MAX_PK_COLS; ++i) {
tValueColumnClear(&statisBlock->firstKeyPKs[i]);
tValueColumnClear(&statisBlock->lastKeyPKs[i]);
}
return 0;
return;
}
static int32_t tStatisBlockAppend(STbStatisBlock *block, SRowInfo *row) {
@ -252,11 +252,11 @@ void tBrinBlockDestroy(SBrinBlock *brinBlock) {
brinBlock->numOfPKs = 0;
brinBlock->numOfRecords = 0;
for (int32_t i = 0; i < ARRAY_SIZE(brinBlock->buffers); ++i) {
TAOS_UNUSED(tBufferDestroy(&brinBlock->buffers[i]));
tBufferDestroy(&brinBlock->buffers[i]);
}
for (int32_t i = 0; i < TD_MAX_PK_COLS; ++i) {
TAOS_UNUSED(tValueColumnDestroy(&brinBlock->firstKeyPKs[i]));
TAOS_UNUSED(tValueColumnDestroy(&brinBlock->lastKeyPKs[i]));
tValueColumnDestroy(&brinBlock->firstKeyPKs[i]);
tValueColumnDestroy(&brinBlock->lastKeyPKs[i]);
}
}
@ -264,7 +264,7 @@ void tBrinBlockClear(SBrinBlock *brinBlock) {
brinBlock->numOfPKs = 0;
brinBlock->numOfRecords = 0;
for (int32_t i = 0; i < ARRAY_SIZE(brinBlock->buffers); ++i) {
TAOS_UNUSED(tBufferClear(&brinBlock->buffers[i]));
tBufferClear(&brinBlock->buffers[i]);
}
for (int32_t i = 0; i < TD_MAX_PK_COLS; ++i) {
tValueColumnClear(&brinBlock->firstKeyPKs[i]);

2
source/dnode/vnode/src/tsdb/tsdbUtil2.h
View File

@ -113,7 +113,7 @@ typedef struct {
int32_t tStatisBlockInit(STbStatisBlock *statisBlock);
void tStatisBlockDestroy(STbStatisBlock *statisBlock);
int32_t tStatisBlockClear(STbStatisBlock *statisBlock);
void tStatisBlockClear(STbStatisBlock *statisBlock);
int32_t tStatisBlockPut(STbStatisBlock *statisBlock, SRowInfo *row, int32_t maxRecords);
int32_t tStatisBlockGet(STbStatisBlock *statisBlock, int32_t idx, STbStatisRecord *record);

20
source/dnode/vnode/src/vnd/vnodeAsync.c
View File

@ -187,10 +187,12 @@ static void vnodeAsyncCancelAllTasks(SVAsync *async, SArray *cancelArray) {
task->prev->next = task->next;
task->next->prev = task->prev;
if (task->cancel) {
TAOS_UNUSED(taosArrayPush(cancelArray, &(SVATaskCancelInfo){
if (taosArrayPush(cancelArray, &(SVATaskCancelInfo){
.cancel = task->cancel,
.arg = task->arg,
}));
}) == NULL) {
vError("failed to push cancel task into array");
};
}
vnodeAsyncTaskDone(async, task);
}
@ -430,7 +432,7 @@ static void vnodeAsyncLaunchWorker(SVAsync *async) {
if (async->workers[i].state == EVA_WORKER_STATE_ACTIVE) {
continue;
} else if (async->workers[i].state == EVA_WORKER_STATE_STOP) {
TAOS_UNUSED(taosThreadJoin(async->workers[i].thread, NULL));
int32_t ret = taosThreadJoin(async->workers[i].thread, NULL);
async->workers[i].state = EVA_WORKER_STATE_UINIT;
}
@ -748,10 +750,12 @@ int32_t vnodeAChannelDestroy(SVAChannelID *channelID, bool waitRunning) {
task->prev->next = task->next;
task->next->prev = task->prev;
if (task->cancel) {
TAOS_UNUSED(taosArrayPush(cancelArray, &(SVATaskCancelInfo){
if (taosArrayPush(cancelArray, &(SVATaskCancelInfo){
.cancel = task->cancel,
.arg = task->arg,
}));
}) == NULL) {
vError("failed to push cancel info");
};
}
vnodeAsyncTaskDone(async, task);
}
@ -763,10 +767,12 @@ int32_t vnodeAChannelDestroy(SVAChannelID *channelID, bool waitRunning) {
channel->scheduled->prev->next = channel->scheduled->next;
channel->scheduled->next->prev = channel->scheduled->prev;
if (channel->scheduled->cancel) {
TAOS_UNUSED(taosArrayPush(cancelArray, &(SVATaskCancelInfo){
if (taosArrayPush(cancelArray, &(SVATaskCancelInfo){
.cancel = channel->scheduled->cancel,
.arg = channel->scheduled->arg,
}));
}) == NULL) {
vError("failed to push cancel info");
}
}
vnodeAsyncTaskDone(async, channel->scheduled);
}

12
source/dnode/vnode/src/vnd/vnodeCommit.c
View File

@ -201,7 +201,9 @@ _exit:
vInfo("vgId:%d, vnode info is saved, fname:%s replica:%d selfIndex:%d changeVersion:%d", pInfo->config.vgId, fname,
pInfo->config.syncCfg.replicaNum, pInfo->config.syncCfg.myIndex, pInfo->config.syncCfg.changeVersion);
}
(void)taosCloseFile(&pFile);
if (taosCloseFile(&pFile) != 0) {
vError("vgId:%d, failed to close file", pInfo->config.vgId);
}
taosMemoryFree(data);
return code;
}
@ -263,7 +265,9 @@ _exit:
}
}
taosMemoryFree(pData);
(void)taosCloseFile(&pFile);
if (taosCloseFile(&pFile) != 0) {
vError("vgId:%d, failed to close file", pInfo->config.vgId);
}
return code;
}
@ -496,7 +500,9 @@ void vnodeRollback(SVnode *pVnode) {
offset = strlen(tFName);
snprintf(tFName + offset, TSDB_FILENAME_LEN - offset - 1, "%s%s", TD_DIRSEP, VND_INFO_FNAME_TMP);
TAOS_UNUSED(taosRemoveFile(tFName));
if (taosRemoveFile(tFName) != 0) {
vError("vgId:%d, failed to remove file %s since %s", TD_VID(pVnode), tFName, tstrerror(terrno));
}
}
static int vnodeEncodeState(const void *pObj, SJson *pJson) {

5
source/dnode/vnode/src/vnd/vnodeOpen.c
View File

@ -414,7 +414,10 @@ SVnode *vnodeOpen(const char *path, int32_t diskPrimary, STfs *pTfs, SMsgCb msgC
pVnode->blocked = false;
pVnode->disableWrite = false;
(void)tsem_init(&pVnode->syncSem, 0, 0);
if (tsem_init(&pVnode->syncSem, 0, 0) != 0) {
vError("vgId:%d, failed to init semaphore", TD_VID(pVnode));
goto _err;
}
(void)taosThreadMutexInit(&pVnode->mutex, NULL);
(void)taosThreadCondInit(&pVnode->poolNotEmpty, NULL);

16
source/dnode/vnode/src/vnd/vnodeSnapshot.c
View File

@ -274,13 +274,17 @@ int32_t vnodeSnapRead(SVSnapReader *pReader, uint8_t **ppData, uint32_t *nData)
int64_t size;
code = taosFStatFile(pFile, &size, NULL);
if (code != 0) {
(void)taosCloseFile(&pFile);
if (taosCloseFile(&pFile) != 0) {
vError("vgId:%d, failed to close file", vgId);
}
TSDB_CHECK_CODE(code, lino, _exit);
}
*ppData = taosMemoryMalloc(sizeof(SSnapDataHdr) + size + 1);
if (*ppData == NULL) {
(void)taosCloseFile(&pFile);
if (taosCloseFile(&pFile) != 0) {
vError("vgId:%d, failed to close file", vgId);
}
TSDB_CHECK_CODE(code = terrno, lino, _exit);
}
((SSnapDataHdr *)(*ppData))->type = SNAP_DATA_CFG;
@ -289,11 +293,15 @@ int32_t vnodeSnapRead(SVSnapReader *pReader, uint8_t **ppData, uint32_t *nData)
if (taosReadFile(pFile, ((SSnapDataHdr *)(*ppData))->data, size) < 0) {
taosMemoryFree(*ppData);
(void)taosCloseFile(&pFile);
if (taosCloseFile(&pFile) != 0) {
vError("vgId:%d, failed to close file", vgId);
}
TSDB_CHECK_CODE(code = terrno, lino, _exit);
}
(void)taosCloseFile(&pFile);
if (taosCloseFile(&pFile) != 0) {
vError("vgId:%d, failed to close file", vgId);
}
pReader->cfgDone = 1;
goto _exit;

4
source/dnode/vnode/src/vnd/vnodeSvr.c
View File

@ -1833,7 +1833,9 @@ static int32_t vnodeProcessSubmitReq(SVnode *pVnode, int64_t ver, void *pReq, in
}
if (info.suid) {
(void)metaGetInfo(pVnode->pMeta, info.suid, &info, NULL);
if (metaGetInfo(pVnode->pMeta, info.suid, &info, NULL) != 0) {
vWarn("vgId:%d, table uid:%" PRId64 " not exists", TD_VID(pVnode), info.suid);
}
}
if (pSubmitTbData->sver != info.skmVer) {

24
source/dnode/vnode/src/vnd/vnodeSync.c
View File

@ -28,7 +28,9 @@ static inline void vnodeWaitBlockMsg(SVnode *pVnode, const SRpcMsg *pMsg) {
const STraceId *trace = &pMsg->info.traceId;
vGTrace("vgId:%d, msg:%p wait block, type:%s sec:%d seq:%" PRId64, pVnode->config.vgId, pMsg,
TMSG_INFO(pMsg->msgType), pVnode->blockSec, pVnode->blockSeq);
(void)tsem_wait(&pVnode->syncSem);
if (tsem_wait(&pVnode->syncSem) != 0) {
vError("vgId:%d, failed to wait sem", pVnode->config.vgId);
}
}
static inline void vnodePostBlockMsg(SVnode *pVnode, const SRpcMsg *pMsg) {
@ -41,7 +43,9 @@ static inline void vnodePostBlockMsg(SVnode *pVnode, const SRpcMsg *pMsg) {
pVnode->blocked = false;
pVnode->blockSec = 0;
pVnode->blockSeq = 0;
(void)tsem_post(&pVnode->syncSem);
if (tsem_post(&pVnode->syncSem) != 0) {
vError("vgId:%d, failed to post sem", pVnode->config.vgId);
}
}
(void)taosThreadMutexUnlock(&pVnode->lock);
}
@ -613,7 +617,9 @@ static void vnodeBecomeFollower(const SSyncFSM *pFsm) {
if (pVnode->blocked) {
pVnode->blocked = false;
vDebug("vgId:%d, become follower and post block", pVnode->config.vgId);
(void)tsem_post(&pVnode->syncSem);
if (tsem_post(&pVnode->syncSem) != 0) {
vError("vgId:%d, failed to post sync semaphore", pVnode->config.vgId);
}
}
(void)taosThreadMutexUnlock(&pVnode->lock);
@ -633,7 +639,9 @@ static void vnodeBecomeLearner(const SSyncFSM *pFsm) {
if (pVnode->blocked) {
pVnode->blocked = false;
vDebug("vgId:%d, become learner and post block", pVnode->config.vgId);
(void)tsem_post(&pVnode->syncSem);
if (tsem_post(&pVnode->syncSem) != 0) {
vError("vgId:%d, failed to post sync semaphore", pVnode->config.vgId);
}
}
(void)taosThreadMutexUnlock(&pVnode->lock);
}
@ -766,7 +774,9 @@ void vnodeSyncPreClose(SVnode *pVnode) {
if (pVnode->blocked) {
vInfo("vgId:%d, post block after close sync", pVnode->config.vgId);
pVnode->blocked = false;
(void)tsem_post(&pVnode->syncSem);
if (tsem_post(&pVnode->syncSem) != 0) {
vError("vgId:%d, failed to post block", pVnode->config.vgId);
}
}
(void)taosThreadMutexUnlock(&pVnode->lock);
}
@ -801,7 +811,9 @@ void vnodeSyncCheckTimeout(SVnode *pVnode) {
pVnode->blocked = false;
pVnode->blockSec = 0;
pVnode->blockSeq = 0;
(void)tsem_post(&pVnode->syncSem);
if (tsem_post(&pVnode->syncSem) != 0) {
vError("vgId:%d, failed to post block", pVnode->config.vgId);
}
}
}
(void)taosThreadMutexUnlock(&pVnode->lock);

6
source/libs/executor/src/aggregateoperator.c
View File

@ -152,15 +152,17 @@ _error:
}
void destroyAggOperatorInfo(void* param) {
if (param == NULL) {
return;
}
SAggOperatorInfo* pInfo = (SAggOperatorInfo*)param;
cleanupBasicInfo(&pInfo->binfo);
if (pInfo->pOperator != NULL) {
if (pInfo->pOperator) {
cleanupResultInfo(pInfo->pOperator->pTaskInfo, &pInfo->pOperator->exprSupp, pInfo->aggSup.pResultBuf,
&pInfo->groupResInfo, pInfo->aggSup.pResultRowHashTable);
pInfo->pOperator = NULL;
}
cleanupAggSup(&pInfo->aggSup);
cleanupExprSupp(&pInfo->scalarExprSup);
cleanupGroupResInfo(&pInfo->groupResInfo);

12
source/libs/executor/src/cachescanoperator.c
View File

@ -347,11 +347,7 @@ static int32_t doScanCacheNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
SExprSupp* pSup = &pInfo->pseudoExprSup;
code = addTagPseudoColumnData(&pInfo->readHandle, pSup->pExprInfo, pSup->numOfExprs, pRes, pRes->info.rows,
pTaskInfo, NULL);
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
(*ppRes) = NULL;
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
pRes->info.id.groupId = tableListGetTableGroupId(pTableList, pRes->info.id.uid);
pInfo->indexOfBufferedRes += 1;
@ -414,11 +410,7 @@ static int32_t doScanCacheNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
pInfo->pRes->info.id.uid = *(tb_uid_t*)pUid;
code = addTagPseudoColumnData(&pInfo->readHandle, pSup->pExprInfo, pSup->numOfExprs, pInfo->pRes,
pInfo->pRes->info.rows, pTaskInfo, NULL);
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
(*ppRes) = NULL;
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
}
}

2
source/libs/executor/src/dynqueryctrloperator.c
View File

@ -938,7 +938,9 @@ _return:
}
if (code) {
qError("%s failed since %s", __func__, tstrerror(code));
pOperator->pTaskInfo->code = code;
T_LONG_JMP(pOperator->pTaskInfo->env, code);
} else {
seqStableJoinComposeRes(pStbJoin, *pRes);
}

2
source/libs/executor/src/eventwindowoperator.c
View File

@ -155,7 +155,7 @@ _error:
}
void cleanupResultInfoInEventWindow(SOperatorInfo* pOperator, SEventWindowOperatorInfo* pInfo) {
if (pInfo == NULL || pInfo->pRow == NULL) {
if (pInfo == NULL || pInfo->pRow == NULL || pOperator == NULL) {
return;
}
SExprSupp* pSup = &pOperator->exprSupp;

18
source/libs/executor/src/exchangeoperator.c
View File

@ -1094,6 +1094,7 @@ int32_t addDynamicExchangeSource(SOperatorInfo* pOperator) {
int32_t prepareLoadRemoteData(SOperatorInfo* pOperator) {
SExchangeInfo* pExchangeInfo = pOperator->info;
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
if ((OPTR_IS_OPENED(pOperator) && !pExchangeInfo->dynamicOp) ||
(pExchangeInfo->dynamicOp && NULL == pOperator->pOperatorGetParam)) {
return TSDB_CODE_SUCCESS;
@ -1101,23 +1102,26 @@ int32_t prepareLoadRemoteData(SOperatorInfo* pOperator) {
if (pExchangeInfo->dynamicOp) {
code = addDynamicExchangeSource(pOperator);
if (code) {
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
}
int64_t st = taosGetTimestampUs();
if (!pExchangeInfo->seqLoadData) {
int32_t code = prepareConcurrentlyLoad(pOperator);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
code = prepareConcurrentlyLoad(pOperator);
QUERY_CHECK_CODE(code, lino, _end);
pExchangeInfo->openedTs = taosGetTimestampUs();
}
OPTR_SET_OPENED(pOperator);
pOperator->cost.openCost = (taosGetTimestampUs() - st) / 1000.0;
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pOperator->pTaskInfo->code = code;
T_LONG_JMP(pOperator->pTaskInfo->env, code);
}
return TSDB_CODE_SUCCESS;
}

10
source/libs/executor/src/groupoperator.c
View File

@ -76,10 +76,10 @@ static void freeGroupKey(void* param) {
}
static void destroyGroupOperatorInfo(void* param) {
SGroupbyOperatorInfo* pInfo = (SGroupbyOperatorInfo*)param;
if (pInfo == NULL) {
if (param == NULL) {
return;
}
SGroupbyOperatorInfo* pInfo = (SGroupbyOperatorInfo*)param;
cleanupBasicInfo(&pInfo->binfo);
taosMemoryFreeClear(pInfo->keyBuf);
@ -454,7 +454,7 @@ static int32_t hashGroupbyAggregateNext(SOperatorInfo* pOperator, SSDataBlock**
QRY_PARAM_CHECK(ppRes);
if (pOperator->status == OP_EXEC_DONE) {
return TSDB_CODE_SUCCESS;
return code;
}
if (pOperator->status == OP_RES_TO_RETURN) {
@ -506,6 +506,7 @@ _end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
} else {
(*ppRes) = buildGroupResultDataBlockByHash(pOperator);
}
@ -1537,8 +1538,9 @@ static int32_t doStreamHashPartitionNext(SOperatorInfo* pOperator, SSDataBlock**
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
(*ppRes) = NULL;
return code;

38
source/libs/executor/src/hashjoinoperator.c
View File

@ -993,6 +993,7 @@ static int32_t hJoinMainProcess(struct SOperatorInfo* pOperator, SSDataBlock** p
SHJoinOperatorInfo* pJoin = pOperator->info;
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
SSDataBlock* pRes = pJoin->finBlk;
int64_t st = 0;
@ -1003,7 +1004,7 @@ static int32_t hJoinMainProcess(struct SOperatorInfo* pOperator, SSDataBlock** p
if (pOperator->status == OP_EXEC_DONE) {
pRes->info.rows = 0;
goto _return;
goto _end;
}
if (!pJoin->keyHashBuilt) {
@ -1011,13 +1012,10 @@ static int32_t hJoinMainProcess(struct SOperatorInfo* pOperator, SSDataBlock** p
bool queryDone = false;
code = hJoinBuildHash(pOperator, &queryDone);
if (code) {
pTaskInfo->code = code;
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
if (queryDone) {
goto _return;
goto _end;
}
}
@ -1025,17 +1023,11 @@ static int32_t hJoinMainProcess(struct SOperatorInfo* pOperator, SSDataBlock** p
if (pJoin->ctx.rowRemains) {
code = (*pJoin->joinFp)(pOperator);
if (code) {
pTaskInfo->code = code;
return pTaskInfo->code;
}
QUERY_CHECK_CODE(code, lino, _end);
if (pRes->info.rows > 0 && pJoin->pFinFilter != NULL) {
code = doFilter(pRes, pJoin->pFinFilter, NULL);
if (code) {
pTaskInfo->code = code;
return pTaskInfo->code;
}
QUERY_CHECK_CODE(code, lino, _end);
}
if (pRes->info.rows > 0) {
@ -1055,10 +1047,7 @@ static int32_t hJoinMainProcess(struct SOperatorInfo* pOperator, SSDataBlock** p
pJoin->execInfo.probeBlkRows += pBlock->info.rows;
code = hJoinPrepareStart(pOperator, pBlock);
if (code) {
pTaskInfo->code = code;
return pTaskInfo->code;
}
QUERY_CHECK_CODE(code, lino, _end);
if (!hJoinBlkReachThreshold(pJoin, pRes->info.rows)) {
continue;
@ -1066,10 +1055,7 @@ static int32_t hJoinMainProcess(struct SOperatorInfo* pOperator, SSDataBlock** p
if (pRes->info.rows > 0 && pJoin->pFinFilter != NULL) {
code = doFilter(pRes, pJoin->pFinFilter, NULL);
if (code) {
pTaskInfo->code = code;
return pTaskInfo->code;
}
QUERY_CHECK_CODE(code, lino, _end);
}
if (pRes->info.rows > 0) {
@ -1077,11 +1063,15 @@ static int32_t hJoinMainProcess(struct SOperatorInfo* pOperator, SSDataBlock** p
}
}
_return:
_end:
if (pOperator->cost.openCost == 0) {
pOperator->cost.openCost = (taosGetTimestampUs() - st) / 1000.0;
}
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
if (pRes->info.rows > 0) {
*pResBlock = pRes;
}

1
source/libs/executor/src/mergejoinoperator.c
View File

@ -1731,6 +1731,7 @@ int32_t mJoinMainProcess(struct SOperatorInfo* pOperator, SSDataBlock** pResBloc
if (pJoin->pFinFilter != NULL) {
code = doFilter(pBlock, pJoin->pFinFilter, NULL);
if (code) {
qError("%s failed at line %d since %s", __func__, __LINE__, tstrerror(code));
pJoin->errCode = code;
T_LONG_JMP(pOperator->pTaskInfo->env, pJoin->errCode);
}

23
source/libs/executor/src/mergeoperator.c
View File

@ -495,6 +495,8 @@ int32_t openMultiwayMergeOperator(SOperatorInfo* pOperator) {
pOperator->status = OP_RES_TO_RETURN;
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, __LINE__, tstrerror(code));
pOperator->pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, terrno);
}
@ -504,6 +506,8 @@ int32_t openMultiwayMergeOperator(SOperatorInfo* pOperator) {
int32_t doMultiwayMerge(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
QRY_PARAM_CHECK(pResBlock);
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
if (pOperator->status == OP_EXEC_DONE) {
return 0;
@ -512,19 +516,12 @@ int32_t doMultiwayMerge(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
SMultiwayMergeOperatorInfo* pInfo = pOperator->info;
int32_t code = pOperator->fpSet._openFn(pOperator);
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
return code;
}
code = pOperator->fpSet._openFn(pOperator);
QUERY_CHECK_CODE(code, lino, _end);
if (NULL != gMultiwayMergeFps[pInfo->type].getNextFn) {
code = (*gMultiwayMergeFps[pInfo->type].getNextFn)(pOperator, pResBlock);
if (code) {
qError("failed to get next data block from upstream, code:%s", tstrerror(code));
pTaskInfo->code = code;
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
}
if ((*pResBlock) != NULL) {
@ -534,6 +531,12 @@ int32_t doMultiwayMerge(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
setOperatorCompleted(pOperator);
}
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
}

14
source/libs/executor/src/operator.c
View File

@ -881,15 +881,17 @@ SSDataBlock* getNextBlockFromDownstreamRemain(struct SOperatorInfo* pOperator, i
int32_t optrDefaultGetNextExtFn(struct SOperatorInfo* pOperator, SOperatorParam* pParam, SSDataBlock** pRes) {
QRY_PARAM_CHECK(pRes);
int32_t lino = 0;
int32_t code = setOperatorParams(pOperator, pParam, OP_GET_PARAM);
if (TSDB_CODE_SUCCESS != code) {
pOperator->pTaskInfo->code = code;
} else {
QUERY_CHECK_CODE(code, lino, _end);
code = pOperator->fpSet.getNextFn(pOperator, pRes);
if (code) {
qError("failed to get next data block from upstream, %s code:%s", __func__, tstrerror(code));
QUERY_CHECK_CODE(code, lino, _end);
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, __LINE__, tstrerror(code));
pOperator->pTaskInfo->code = code;
}
T_LONG_JMP(pOperator->pTaskInfo->env, code);
}
return code;

58
source/libs/executor/src/projectoperator.c
View File

@ -270,6 +270,7 @@ int32_t doProjectOperation(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
SSDataBlock* pRes = pInfo->pRes;
SSDataBlock* pFinalRes = pProjectInfo->pFinalRes;
int32_t code = 0;
int32_t lino = 0;
int64_t st = 0;
int32_t order = pInfo->inputTsOrder;
int32_t scanFlag = 0;
@ -290,9 +291,7 @@ int32_t doProjectOperation(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
if (downstream == NULL) {
code = doGenerateSourceData(pOperator);
if (code != TSDB_CODE_SUCCESS) {
T_LONG_JMP(pTaskInfo->env, code);
}
QUERY_CHECK_CODE(code, lino, _end);
if (pProjectInfo->outputIgnoreGroup) {
pRes->info.id.groupId = 0;
@ -348,20 +347,14 @@ int32_t doProjectOperation(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
}
code = setInputDataBlock(pSup, pBlock, order, scanFlag, false);
if (code) {
T_LONG_JMP(pTaskInfo->env, code);
}
QUERY_CHECK_CODE(code, lino, _end);
code = blockDataEnsureCapacity(pInfo->pRes, pInfo->pRes->info.rows + pBlock->info.rows);
if (code != TSDB_CODE_SUCCESS) {
T_LONG_JMP(pTaskInfo->env, code);
}
QUERY_CHECK_CODE(code, lino, _end);
code = projectApplyFunctions(pSup->pExprInfo, pInfo->pRes, pBlock, pSup->pCtx, pSup->numOfExprs,
pProjectInfo->pPseudoColInfo);
if (code != TSDB_CODE_SUCCESS) {
T_LONG_JMP(pTaskInfo->env, code);
}
QUERY_CHECK_CODE(code, lino, _end);
status = doIngroupLimitOffset(pLimitInfo, pBlock->info.id.groupId, pInfo->pRes, pOperator);
if (status == PROJECT_RETRIEVE_CONTINUE) {
@ -377,11 +370,8 @@ int32_t doProjectOperation(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
pFinalRes->info.version = pRes->info.version;
// continue merge data, ignore the group id
int32_t ret = blockDataMerge(pFinalRes, pRes);
if (ret < 0) {
pTaskInfo->code = code;
return code;
}
code = blockDataMerge(pFinalRes, pRes);
QUERY_CHECK_CODE(code, lino, _end);
if (pFinalRes->info.rows + pRes->info.rows <= pOperator->resultInfo.threshold && (pOperator->status != OP_EXEC_DONE)) {
continue;
@ -390,10 +380,7 @@ int32_t doProjectOperation(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
// do apply filter
code = doFilter(pFinalRes, pOperator->exprSupp.pFilterInfo, NULL);
if (code) {
pTaskInfo->code = code;
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
// when apply the limit/offset for each group, pRes->info.rows may be 0, due to limit constraint.
if (pFinalRes->info.rows > 0 || (pOperator->status == OP_EXEC_DONE)) {
@ -404,10 +391,7 @@ int32_t doProjectOperation(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
// do apply filter
if (pRes->info.rows > 0) {
code = doFilter(pRes, pOperator->exprSupp.pFilterInfo, NULL);
if (code) {
pTaskInfo->code = code;
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
if (pRes->info.rows == 0) {
continue;
@ -436,6 +420,13 @@ int32_t doProjectOperation(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
}
*pResBlock = (p->info.rows > 0)? p:NULL;
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
}
@ -578,14 +569,15 @@ int32_t doApplyIndefinitFunction(SOperatorInfo* pOperator, SSDataBlock** pResBlo
SOptrBasicInfo* pInfo = &pIndefInfo->binfo;
SExprSupp* pSup = &pOperator->exprSupp;
int64_t st = 0;
int32_t code = 0;
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
SSDataBlock* pRes = pInfo->pRes;
blockDataCleanup(pRes);
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
if (pOperator->status == OP_EXEC_DONE) {
return 0;
return code;
}
if (pOperator->cost.openCost == 0) {
@ -637,10 +629,7 @@ int32_t doApplyIndefinitFunction(SOperatorInfo* pOperator, SSDataBlock** pResBlo
}
code = doFilter(pInfo->pRes, pOperator->exprSupp.pFilterInfo, NULL);
if (code) {
pTaskInfo->code = code;
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
size_t rows = pInfo->pRes->info.rows;
if (rows > 0 || pOperator->status == OP_EXEC_DONE) {
@ -658,6 +647,13 @@ int32_t doApplyIndefinitFunction(SOperatorInfo* pOperator, SSDataBlock** pResBlo
}
*pResBlock = (rows > 0) ? pInfo->pRes : NULL;
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
}

55
source/libs/executor/src/scanoperator.c
View File

@ -951,7 +951,8 @@ static int32_t doTableScanImplNext(SOperatorInfo* pOperator, SSDataBlock** ppRes
if (isTaskKilled(pTaskInfo)) {
pAPI->tsdReader.tsdReaderReleaseDataBlock(pTableScanInfo->base.dataReader);
return pTaskInfo->code;
code = pTaskInfo->code;
goto _end;
}
if (pOperator->status == OP_EXEC_DONE) {
@ -996,6 +997,7 @@ _end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
}
@ -1416,6 +1418,7 @@ _end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s %s failed at line %d since %s", GET_TASKID(pTaskInfo), __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
@ -2944,8 +2947,9 @@ static int32_t doQueueScanNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
if (pResult && pResult->info.rows > 0) {
bool hasPrimaryKey = pAPI->tqReaderFn.tqGetTablePrimaryKey(pInfo->tqReader);
code = processPrimaryKey(pResult, hasPrimaryKey, &pTaskInfo->streamInfo.currentOffset);
QUERY_CHECK_CODE(code, lino, _end);
qDebug("tmqsnap doQueueScan get data utid:%" PRId64 "", pResult->info.id.uid);
if (pResult->info.rows > 0 || code != TSDB_CODE_SUCCESS) {
if (pResult->info.rows > 0) {
(*ppRes) = pResult;
return code;
}
@ -3009,8 +3013,9 @@ static int32_t doQueueScanNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
(*ppRes) = NULL;
return code;
@ -3340,9 +3345,7 @@ FETCH_NEXT_BLOCK:
if (pBlock->info.parTbName[0]) {
code =
pAPI->stateStore.streamStatePutParName(pStreamInfo->pState, pBlock->info.id.groupId, pBlock->info.parTbName);
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
}
QUERY_CHECK_CODE(code, lino, _end);
}
// TODO move into scan
@ -3482,7 +3485,7 @@ FETCH_NEXT_BLOCK:
return code;
}
qError("%s===stream=== %s failed at line %d since pInfo->pUpdateRes is empty", GET_TASKID(pTaskInfo), __func__,
lino);
__LINE__);
blockDataCleanup(pInfo->pUpdateDataRes);
pInfo->scanMode = STREAM_SCAN_FROM_READERHANDLE;
} break;
@ -3496,7 +3499,7 @@ FETCH_NEXT_BLOCK:
return code;
}
qError("%s===stream=== %s failed at line %d since pInfo->pUpdateRes is empty", GET_TASKID(pTaskInfo), __func__,
lino);
__LINE__);
blockDataCleanup(pInfo->pUpdateDataRes);
pInfo->scanMode = STREAM_SCAN_FROM_READERHANDLE;
} break;
@ -3658,8 +3661,9 @@ FETCH_NEXT_BLOCK:
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
(*ppRes) = NULL;
return code;
@ -3730,6 +3734,7 @@ static int32_t doRawScanNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
if (pBlock && pBlock->info.rows > 0) {
bool hasPrimaryKey = pAPI->snapshotFn.taosXGetTablePrimaryKey(pInfo->sContext);
code = processPrimaryKey(pBlock, hasPrimaryKey, &pTaskInfo->streamInfo.currentOffset);
QUERY_CHECK_CODE(code, lino, _end);
qDebug("tmqsnap doRawScan get data uid:%" PRId64 "", pBlock->info.id.uid);
(*ppRes) = pBlock;
return code;
@ -3741,7 +3746,7 @@ static int32_t doRawScanNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
QUERY_CHECK_CODE(code, lino, _end);
if (code != 0) {
tDeleteSchemaWrapper(mtInfo.schema);
goto _end;
QUERY_CHECK_CODE(code, lino, _end);
}
STqOffsetVal offset = {0};
if (mtInfo.uid == 0 || pInfo->sContext->withMeta == ONLY_META) { // read snapshot done, change to get data from wal
@ -3831,6 +3836,7 @@ _end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
(*ppRes) = NULL;
@ -4677,6 +4683,7 @@ _end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
@ -4684,6 +4691,7 @@ _end:
static int32_t doTagScanFromMetaEntryNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
if (pOperator->status == OP_EXEC_DONE) {
(*ppRes) = NULL;
return code;
@ -4699,10 +4707,7 @@ static int32_t doTagScanFromMetaEntryNext(SOperatorInfo* pOperator, SSDataBlock*
int32_t size = 0;
code = tableListGetSize(pInfo->pTableListInfo, &size);
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, __LINE__, tstrerror(code));
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
if (size == 0) {
setTaskStatus(pTaskInfo, TASK_COMPLETED);
@ -4716,11 +4721,11 @@ static int32_t doTagScanFromMetaEntryNext(SOperatorInfo* pOperator, SSDataBlock*
while (pInfo->curPos < size && count < pOperator->resultInfo.capacity) {
code = doTagScanOneTable(pOperator, pRes, count, &mr, &pTaskInfo->storageAPI);
if (code == TSDB_CODE_OUT_OF_MEMORY) {
break;
} else {
if (code != TSDB_CODE_OUT_OF_MEMORY) {
// ignore other error
code = TSDB_CODE_SUCCESS;
}
QUERY_CHECK_CODE(code, lino, _end);
++count;
if (++pInfo->curPos >= size) {
@ -4744,6 +4749,13 @@ static int32_t doTagScanFromMetaEntryNext(SOperatorInfo* pOperator, SSDataBlock*
pOperator->resultInfo.totalRows += pRes->info.rows;
(*ppRes) = (pRes->info.rows == 0) ? NULL : pInfo->pRes;
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
}
@ -5429,6 +5441,7 @@ _end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
} else {
(*ppRes) = pBlock;
}
@ -5946,6 +5959,7 @@ _end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
} else {
(*ppRes) = pBlock;
}
@ -6461,7 +6475,12 @@ static int32_t doTableCountScanNext(SOperatorInfo* pOperator, SSDataBlock** ppRe
}
code = buildVnodeDbTableCount(pOperator, pInfo, pSupp, pRes);
if ((pRes->info.rows > 0) && (code == 0)) {
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed since %s", __func__, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
if (pRes->info.rows > 0) {
*ppRes = pRes;
}

97
source/libs/executor/src/sortoperator.c
View File

@ -356,82 +356,84 @@ void applyScalarFunction(SSDataBlock* pBlock, void* param) {
int32_t doOpenSortOperator(SOperatorInfo* pOperator) {
SSortOperatorInfo* pInfo = pOperator->info;
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
SSortSource* pSource =NULL;
if (OPTR_IS_OPENED(pOperator)) {
return TSDB_CODE_SUCCESS;
return code;
}
pInfo->startTs = taosGetTimestampUs();
// pInfo->binfo.pRes is not equalled to the input datablock.
pInfo->pSortHandle = NULL;
int32_t code =
code =
tsortCreateSortHandle(pInfo->pSortInfo, SORT_SINGLESOURCE_SORT, -1, -1, NULL, pTaskInfo->id.str, pInfo->maxRows,
pInfo->maxTupleLength, tsPQSortMemThreshold * 1024 * 1024, &pInfo->pSortHandle);
if (code) {
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
tsortSetFetchRawDataFp(pInfo->pSortHandle, loadNextDataBlock, applyScalarFunction, pOperator);
SSortSource* pSource = taosMemoryCalloc(1, sizeof(SSortSource));
if (pSource == NULL) {
return terrno;
}
pSource = taosMemoryCalloc(1, sizeof(SSortSource));
QUERY_CHECK_NULL(pSource, code, lino, _end, terrno);
pSource->param = pOperator->pDownstream[0];
pSource->onlyRef = true;
code = tsortAddSource(pInfo->pSortHandle, pSource);
if (code) {
taosMemoryFree(pSource);
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
pSource = NULL;
code = tsortOpen(pInfo->pSortHandle);
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
} else {
QUERY_CHECK_CODE(code, lino, _end);
pOperator->cost.openCost = (taosGetTimestampUs() - pInfo->startTs) / 1000.0;
pOperator->status = OP_RES_TO_RETURN;
OPTR_SET_OPENED(pOperator);
}
_end:
if (pSource) {
taosMemoryFree(pSource);
}
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
}
int32_t doSort(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
QRY_PARAM_CHECK(pResBlock);
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
if (pOperator->status == OP_EXEC_DONE) {
return 0;
return code;
}
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
SSortOperatorInfo* pInfo = pOperator->info;
int32_t code = pOperator->fpSet._openFn(pOperator);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
code = pOperator->fpSet._openFn(pOperator);
QUERY_CHECK_CODE(code, lino, _end);
// multi-group case not handle here
SSDataBlock* pBlock = NULL;
while (1) {
if (tsortIsClosed(pInfo->pSortHandle)) {
code = TSDB_CODE_TSC_QUERY_CANCELLED;
T_LONG_JMP(pTaskInfo->env, code);
QUERY_CHECK_CODE(code, lino, _end);
}
code = getSortedBlockData(pInfo->pSortHandle, pInfo->binfo.pRes, pOperator->resultInfo.capacity,
pInfo->matchInfo.pList, pInfo, &pBlock);
if (pBlock == NULL || code != 0) {
QUERY_CHECK_CODE(code, lino, _end);
if (pBlock == NULL) {
setOperatorCompleted(pOperator);
return code;
}
code = doFilter(pBlock, pOperator->exprSupp.pFilterInfo, &pInfo->matchInfo);
if (code) {
break;
}
QUERY_CHECK_CODE(code, lino, _end);
if (blockDataGetNumOfRows(pBlock) == 0) {
continue;
@ -450,6 +452,12 @@ int32_t doSort(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
}
*pResBlock = blockDataGetNumOfRows(pBlock) > 0 ? pBlock : NULL;
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
}
@ -699,16 +707,16 @@ int32_t doGroupSort(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
QRY_PARAM_CHECK(pResBlock);
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
SGroupSortOperatorInfo* pInfo = pOperator->info;
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
if (pOperator->status == OP_EXEC_DONE) {
return 0;
}
int32_t code = pOperator->fpSet._openFn(pOperator);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
code = pOperator->fpSet._openFn(pOperator);
QUERY_CHECK_CODE(code, lino, _end);
if (!pInfo->hasGroupId) {
pInfo->hasGroupId = true;
@ -721,30 +729,25 @@ int32_t doGroupSort(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
pInfo->currGroupId = pInfo->prefetchedSortInput->info.id.groupId;
pInfo->childOpStatus = CHILD_OP_NEW_GROUP;
code = beginSortGroup(pOperator);
if (code) {
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
}
SSDataBlock* pBlock = NULL;
while (pInfo->pCurrSortHandle != NULL) {
if (tsortIsClosed(pInfo->pCurrSortHandle)) {
code = TSDB_CODE_TSC_QUERY_CANCELLED;
T_LONG_JMP(pTaskInfo->env, code);
QUERY_CHECK_CODE(code, lino, _end);
}
// beginSortGroup would fetch all child blocks of pInfo->currGroupId;
if (pInfo->childOpStatus == CHILD_OP_SAME_GROUP) {
pTaskInfo->code = code = TSDB_CODE_QRY_EXECUTOR_INTERNAL_ERROR;
qError("%s failed at line %d since %s", __func__, __LINE__, tstrerror(code));
return code;
code = TSDB_CODE_QRY_EXECUTOR_INTERNAL_ERROR;
QUERY_CHECK_CODE(code, lino, _end);
}
code = getGroupSortedBlockData(pInfo->pCurrSortHandle, pInfo->binfo.pRes, pOperator->resultInfo.capacity,
pInfo->matchInfo.pList, pInfo, &pBlock);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
if (pBlock != NULL) {
pBlock->info.id.groupId = pInfo->currGroupId;
pOperator->resultInfo.totalRows += pBlock->info.rows;
@ -755,9 +758,7 @@ int32_t doGroupSort(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
(void) finishSortGroup(pOperator);
pInfo->currGroupId = pInfo->prefetchedSortInput->info.id.groupId;
code = beginSortGroup(pOperator);
if (code) {
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
} else if (pInfo->childOpStatus == CHILD_OP_FINISHED) {
(void) finishSortGroup(pOperator);
setOperatorCompleted(pOperator);
@ -766,6 +767,12 @@ int32_t doGroupSort(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
}
}
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
}

8
source/libs/executor/src/streamcountwindowoperator.c
View File

@ -46,11 +46,16 @@ typedef struct SBuffInfo {
} SBuffInfo;
void destroyStreamCountAggOperatorInfo(void* param) {
if (param == NULL) {
return;
}
SStreamCountAggOperatorInfo* pInfo = (SStreamCountAggOperatorInfo*)param;
cleanupBasicInfo(&pInfo->binfo);
if (pInfo->pOperator) {
cleanupResultInfoInStream(pInfo->pOperator->pTaskInfo, pInfo->streamAggSup.pState, &pInfo->pOperator->exprSupp,
&pInfo->groupResInfo);
pInfo->pOperator = NULL;
}
destroyStreamAggSupporter(&pInfo->streamAggSup);
cleanupExprSupp(&pInfo->scalarSupp);
clearGroupResInfo(&pInfo->groupResInfo);
@ -744,8 +749,9 @@ static int32_t doStreamCountAggNext(SOperatorInfo* pOperator, SSDataBlock** ppRe
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s. task:%s", __func__, lino, tstrerror(code), GET_TASKID(pTaskInfo));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
setStreamOperatorCompleted(pOperator);
(*ppRes) = NULL;

9
source/libs/executor/src/streameventwindowoperator.c
View File

@ -48,9 +48,11 @@ void destroyStreamEventOperatorInfo(void* param) {
}
SStreamEventAggOperatorInfo* pInfo = (SStreamEventAggOperatorInfo*)param;
cleanupBasicInfo(&pInfo->binfo);
if (pInfo->pOperator) {
cleanupResultInfoInStream(pInfo->pOperator->pTaskInfo, pInfo->streamAggSup.pState, &pInfo->pOperator->exprSupp,
&pInfo->groupResInfo);
pInfo->pOperator = NULL;
}
destroyStreamAggSupporter(&pInfo->streamAggSup);
clearGroupResInfo(&pInfo->groupResInfo);
taosArrayDestroyP(pInfo->pUpdated, destroyFlusedPos);
@ -179,7 +181,7 @@ _end:
pAggSup->stateStore.streamStateSessionDel(pAggSup->pState, &pCurWin->winInfo.sessionWin);
}
pAggSup->stateStore.streamStateFreeCur(pCur);
qDebug("===stream===set event next win buff. skey:%" PRId64 ", endkey:%" PRId64, pCurWin->winInfo.sessionWin.win.skey,
qDebug("===stream===set event cur win buff. skey:%" PRId64 ", endkey:%" PRId64, pCurWin->winInfo.sessionWin.win.skey,
pCurWin->winInfo.sessionWin.win.ekey);
_error:
@ -233,7 +235,7 @@ int32_t updateEventWindowInfo(SStreamAggSupporter* pAggSup, SEventWindowInfo* pW
pWinInfo->pWinFlag->endFlag = ends[i];
} else if (pWin->ekey == pTsData[i]) {
pWinInfo->pWinFlag->endFlag |= ends[i];
} else {
} else if (ends[i] && !pWinInfo->pWinFlag->endFlag) {
*pRebuild = true;
pWinInfo->pWinFlag->endFlag |= ends[i];
(*pWinRow) = i + 1 - start;
@ -734,8 +736,9 @@ static int32_t doStreamEventAggNext(SOperatorInfo* pOperator, SSDataBlock** ppRe
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s. task:%s", __func__, lino, tstrerror(code), GET_TASKID(pTaskInfo));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
setStreamOperatorCompleted(pOperator);
(*ppRes) = NULL;

3
source/libs/executor/src/streamfilloperator.c
View File

@ -1155,8 +1155,9 @@ static int32_t doStreamFillNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s. task:%s", __func__, lino, tstrerror(code), GET_TASKID(pTaskInfo));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
setOperatorCompleted(pOperator);
resetStreamFillInfo(pInfo);

27
source/libs/executor/src/streamtimewindowoperator.c
View File

@ -473,13 +473,11 @@ void destroyStreamFinalIntervalOperatorInfo(void* param) {
}
SStreamIntervalOperatorInfo* pInfo = (SStreamIntervalOperatorInfo*)param;
cleanupBasicInfo(&pInfo->binfo);
if (pInfo->pOperator != NULL) {
if (pInfo->pOperator) {
cleanupResultInfo(pInfo->pOperator->pTaskInfo, &pInfo->pOperator->exprSupp, pInfo->aggSup.pResultBuf,
&pInfo->groupResInfo, pInfo->aggSup.pResultRowHashTable);
pInfo->pOperator = NULL;
}
cleanupAggSup(&pInfo->aggSup);
clearGroupResInfo(&pInfo->groupResInfo);
taosArrayDestroyP(pInfo->pUpdated, destroyFlusedPos);
@ -1806,8 +1804,9 @@ static int32_t doStreamFinalIntervalAggNext(SOperatorInfo* pOperator, SSDataBloc
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s. task:%s", __func__, lino, tstrerror(code), GET_TASKID(pTaskInfo));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
setStreamOperatorCompleted(pOperator);
(*ppRes) = NULL;
@ -2096,9 +2095,11 @@ void destroyStreamSessionAggOperatorInfo(void* param) {
}
SStreamSessionAggOperatorInfo* pInfo = (SStreamSessionAggOperatorInfo*)param;
cleanupBasicInfo(&pInfo->binfo);
if (pInfo->pOperator) {
cleanupResultInfoInStream(pInfo->pOperator->pTaskInfo, pInfo->streamAggSup.pState, &pInfo->pOperator->exprSupp,
&pInfo->groupResInfo);
pInfo->pOperator = NULL;
}
destroyStreamAggSupporter(&pInfo->streamAggSup);
cleanupExprSupp(&pInfo->scalarSupp);
clearGroupResInfo(&pInfo->groupResInfo);
@ -3575,8 +3576,9 @@ static int32_t doStreamSessionAggNext(SOperatorInfo* pOperator, SSDataBlock** pp
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s. task:%s", __func__, lino, tstrerror(code), GET_TASKID(pTaskInfo));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
setStreamOperatorCompleted(pOperator);
(*ppRes) = NULL;
@ -4080,8 +4082,9 @@ static int32_t doStreamSessionSemiAggNext(SOperatorInfo* pOperator, SSDataBlock*
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s. task:%s", __func__, lino, tstrerror(code), GET_TASKID(pTaskInfo));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
clearFunctionContext(&pOperator->exprSupp);
@ -4186,9 +4189,11 @@ void destroyStreamStateOperatorInfo(void* param) {
}
SStreamStateAggOperatorInfo* pInfo = (SStreamStateAggOperatorInfo*)param;
cleanupBasicInfo(&pInfo->binfo);
if (pInfo->pOperator) {
cleanupResultInfoInStream(pInfo->pOperator->pTaskInfo, pInfo->streamAggSup.pState, &pInfo->pOperator->exprSupp,
&pInfo->groupResInfo);
pInfo->pOperator = NULL;
}
destroyStreamAggSupporter(&pInfo->streamAggSup);
clearGroupResInfo(&pInfo->groupResInfo);
taosArrayDestroyP(pInfo->pUpdated, destroyFlusedPos);
@ -4820,8 +4825,9 @@ static int32_t doStreamStateAggNext(SOperatorInfo* pOperator, SSDataBlock** ppRe
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s. task:%s", __func__, lino, tstrerror(code), GET_TASKID(pTaskInfo));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
setStreamOperatorCompleted(pOperator);
(*ppRes) = NULL;
@ -5220,6 +5226,7 @@ static int32_t doStreamIntervalAggNext(SOperatorInfo* pOperator, SSDataBlock** p
code = TSDB_CODE_SUCCESS;
break;
}
QUERY_CHECK_CODE(code, lino, _end);
pInfo->twAggSup.maxTs = TMAX(pInfo->twAggSup.maxTs, pBlock->info.window.ekey);
pInfo->twAggSup.minTs = TMIN(pInfo->twAggSup.minTs, pBlock->info.window.skey);
}
@ -5262,8 +5269,9 @@ static int32_t doStreamIntervalAggNext(SOperatorInfo* pOperator, SSDataBlock** p
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s. task:%s", __func__, lino, tstrerror(code), GET_TASKID(pTaskInfo));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
setStreamOperatorCompleted(pOperator);
(*ppRes) = NULL;
@ -5799,8 +5807,9 @@ static int32_t doStreamMidIntervalAggNext(SOperatorInfo* pOperator, SSDataBlock*
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s. task:%s", __func__, lino, tstrerror(code), GET_TASKID(pTaskInfo));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
(*ppRes) = NULL;
return code;

14
source/libs/executor/src/sysscanoperator.c
View File

@ -2049,7 +2049,7 @@ static int32_t doSysTableScanNext(SOperatorInfo* pOperator, SSDataBlock** ppRes)
if (isTaskKilled(pOperator->pTaskInfo)) {
setOperatorCompleted(pOperator);
(*ppRes) = NULL;
return pTaskInfo->code;
break;
}
blockDataCleanup(pInfo->pRes);
@ -2092,13 +2092,19 @@ static int32_t doSysTableScanNext(SOperatorInfo* pOperator, SSDataBlock** ppRes)
continue;
}
(*ppRes) = pBlock;
return pTaskInfo->code;
} else {
(*ppRes) = NULL;
}
break;
}
_end:
if (pTaskInfo->code) {
qError("%s failed since %s", __func__, tstrerror(pTaskInfo->code));
T_LONG_JMP(pTaskInfo->env, pTaskInfo->code);
}
return pTaskInfo->code;
}
}
}
static void sysTableScanFillTbName(SOperatorInfo* pOperator, const SSysTableScanInfo* pInfo, const char* name,
SSDataBlock* pBlock) {

3
source/libs/executor/src/timewindowoperator.c
View File

@ -1223,6 +1223,9 @@ _end:
}
static void destroyStateWindowOperatorInfo(void* param) {
if (param == NULL) {
return;
}
SStateWindowOperatorInfo* pInfo = (SStateWindowOperatorInfo*)param;
cleanupBasicInfo(&pInfo->binfo);
taosMemoryFreeClear(pInfo->stateKey.pData);

55
source/libs/executor/src/tsort.c
View File

@ -437,41 +437,36 @@ int32_t tsortAddSource(SSortHandle* pSortHandle, void* pSource) {
static int32_t doAddNewExternalMemSource(SDiskbasedBuf* pBuf, SArray* pAllSources, SSDataBlock* pBlock,
int32_t* sourceId, SArray* pPageIdList) {
int32_t code = 0;
int32_t lino = 0;
SSortSource* pSource = taosMemoryCalloc(1, sizeof(SSortSource));
if (pSource == NULL) {
taosArrayDestroy(pPageIdList);
return terrno;
}
QUERY_CHECK_NULL(pSource, code, lino, _err, terrno);
pSource->src.pBlock = pBlock;
pSource->pageIdList = pPageIdList;
void* p = taosArrayPush(pAllSources, &pSource);
if (p == NULL) {
taosArrayDestroy(pPageIdList);
return terrno;
}
SSortSource** p = taosArrayPush(pAllSources, &pSource);
QUERY_CHECK_NULL(p, code, lino, _err, terrno);
pSource = NULL;
(*sourceId) += 1;
int32_t rowSize = blockDataGetSerialRowSize(pSource->src.pBlock);
int32_t rowSize = blockDataGetSerialRowSize((*p)->src.pBlock);
// The value of numOfRows must be greater than 0, which is guaranteed by the previous memory allocation
int32_t numOfRows =
(getBufPageSize(pBuf) - blockDataGetSerialMetaSize(taosArrayGetSize(pBlock->pDataBlock))) / rowSize;
if (numOfRows <= 0) {
qError("sort failed at: %s:%d", __func__, __LINE__);
taosArrayDestroy(pPageIdList);
return TSDB_CODE_QRY_EXECUTOR_INTERNAL_ERROR;
}
QUERY_CHECK_CONDITION((numOfRows > 0), code, lino, _err, TSDB_CODE_QRY_EXECUTOR_INTERNAL_ERROR);
int32_t code = blockDataEnsureCapacity(pSource->src.pBlock, numOfRows);
if (code != 0) {
qError("sort failed at: %s:%d", __func__, __LINE__);
taosArrayDestroy(pPageIdList);
}
code = blockDataEnsureCapacity((*p)->src.pBlock, numOfRows);
QUERY_CHECK_CODE(code, lino, _err);
return code;
_err:
if (pSource) taosMemoryFree(pSource);
qError("sort failed at %s:%d since %s", __func__, lino, tstrerror(code));
return code;
}
static int32_t doAddToBuf(SSDataBlock* pDataBlock, SSortHandle* pHandle) {
@ -554,7 +549,12 @@ static int32_t doAddToBuf(SSDataBlock* pDataBlock, SSortHandle* pHandle) {
return code;
}
return doAddNewExternalMemSource(pHandle->pBuf, pHandle->pOrderedSource, pBlock, &pHandle->sourceId, pPageIdList);
code = doAddNewExternalMemSource(pHandle->pBuf, pHandle->pOrderedSource, pBlock, &pHandle->sourceId, pPageIdList);
if (code) {
blockDataDestroy(pBlock);
taosArrayDestroy(pPageIdList);
}
return code;
}
static void setCurrentSourceDone(SSortSource* pSource, SSortHandle* pHandle) {
@ -1023,6 +1023,9 @@ static int32_t doSortForEachGroup(SSortHandle* pHandle, int32_t sortTimes, int32
QUERY_CHECK_CODE(code, lino, _err);
code = doAddNewExternalMemSource(pHandle->pBuf, pResList, pBlock, &pHandle->sourceId, pPageIdList);
if (code != TSDB_CODE_SUCCESS) {
blockDataDestroy(pBlock);
}
QUERY_CHECK_CODE(code, lino, _err);
}
@ -2144,6 +2147,10 @@ static int32_t sortBlocksToExtSource(SSortHandle* pHandle, SArray* aBlk, SArray*
if (code) goto _error;
code = doAddNewExternalMemSource(pHandle->pBuf, aExtSrc, pMemSrcBlk, &pHandle->sourceId, aPgId);
if (code != TSDB_CODE_SUCCESS) {
blockDataDestroy(pMemSrcBlk);
goto _error;
}
cleanupMergeSup(&sup);
tMergeTreeDestroy(&pTree);
@ -2306,9 +2313,15 @@ static int32_t createBlocksMergeSortInitialSources(SSortHandle* pHandle) {
}
code = tSimpleHashPut(mUidBlk, &pBlk->info.id.uid, sizeof(pBlk->info.id.uid), &tBlk, POINTER_BYTES);
if (code != TSDB_CODE_SUCCESS) {
blockDataDestroy(tBlk);
}
QUERY_CHECK_CODE(code, lino, _err);
void* px = taosArrayPush(aBlkSort, &tBlk);
if (px == NULL) {
blockDataDestroy(tBlk);
}
QUERY_CHECK_NULL(px, code, lino, _err, terrno);
}
}

36
source/libs/index/src/index.c
View File

@ -137,8 +137,12 @@ int32_t indexOpen(SIndexOpts* opts, const char* path, SIndex** index) {
TAOS_CHECK_GOTO(terrno, NULL, END);
}
TAOS_UNUSED(taosThreadMutexInit(&idx->mtx, NULL));
TAOS_UNUSED(tsem_init(&idx->sem, 0, 0));
if (taosThreadMutexInit(&idx->mtx, NULL) != 0) {
TAOS_CHECK_GOTO(terrno, NULL, END);
}
if (tsem_init(&idx->sem, 0, 0) != 0) {
TAOS_CHECK_GOTO(terrno, NULL, END);
}
idx->refId = idxAddRef(idx);
idx->opts = *opts;
@ -213,7 +217,10 @@ void idxReleaseRef(int64_t ref) {
int32_t indexPut(SIndex* index, SIndexMultiTerm* fVals, uint64_t uid) {
// TODO(yihao): reduce the lock range
int32_t code = 0;
TAOS_UNUSED(taosThreadMutexLock(&index->mtx));
if (taosThreadMutexLock(&index->mtx) != 0) {
indexError("failed to lock index mutex");
}
for (int i = 0; i < taosArrayGetSize(fVals); i++) {
SIndexTerm* p = taosArrayGetP(fVals, i);
@ -231,7 +238,9 @@ int32_t indexPut(SIndex* index, SIndexMultiTerm* fVals, uint64_t uid) {
}
}
}
TAOS_UNUSED(taosThreadMutexUnlock(&index->mtx));
if (taosThreadMutexUnlock(&index->mtx) != 0) {
indexError("failed to unlock index mutex");
}
if (code != 0) {
return code;
@ -463,7 +472,10 @@ static int32_t idxTermSearch(SIndex* sIdx, SIndexTermQuery* query, SArray** resu
int32_t sz = idxSerialCacheKey(&key, buf);
TAOS_UNUSED(taosThreadMutexLock(&sIdx->mtx));
if (taosThreadMutexLock(&sIdx->mtx) != 0) {
indexError("failed to lock index mutex");
}
IndexCache** pCache = taosHashGet(sIdx->colObj, buf, sz);
cache = (pCache == NULL) ? NULL : *pCache;
TAOS_UNUSED(taosThreadMutexUnlock(&sIdx->mtx));
@ -757,7 +769,9 @@ static int64_t idxGetAvailableVer(SIndex* sIdx, IndexCache* cache) {
IndexTFile* tf = (IndexTFile*)(sIdx->tindex);
TAOS_UNUSED(taosThreadMutexLock(&tf->mtx));
if (taosThreadMutexLock(&tf->mtx) != 0) {
indexError("failed to lock tfile mutex");
}
TFileReader* rd = tfileCacheGet(tf->cache, &key);
TAOS_UNUSED(taosThreadMutexUnlock(&tf->mtx));
@ -801,9 +815,15 @@ static int32_t idxGenTFile(SIndex* sIdx, IndexCache* cache, SArray* batch) {
TFileHeader* header = &reader->header;
ICacheKey key = {.suid = cache->suid, .colName = header->colName, .nColName = strlen(header->colName)};
TAOS_UNUSED(taosThreadMutexLock(&tf->mtx));
if (taosThreadMutexLock(&tf->mtx) != 0) {
indexError("failed to lock tfile mutex");
}
code = tfileCachePut(tf->cache, &key, reader);
TAOS_UNUSED(taosThreadMutexUnlock(&tf->mtx));
if (taosThreadMutexUnlock(&tf->mtx) != 0) {
indexError("failed to unlock tfile mutex");
}
return code;

61
source/libs/index/src/indexCache.c
View File

@ -398,8 +398,17 @@ IndexCache* idxCacheCreate(SIndex* idx, uint64_t suid, const char* colName, int8
cache->suid = suid;
cache->occupiedMem = 0;
TAOS_UNUSED(taosThreadMutexInit(&cache->mtx, NULL));
TAOS_UNUSED(taosThreadCondInit(&cache->finished, NULL));
if (taosThreadMutexInit(&cache->mtx, NULL) != 0) {
indexError("failed to create mutex for index cache");
taosMemoryFree(cache);
return NULL;
}
if (taosThreadCondInit(&cache->finished, NULL) != 0) {
indexError("failed to create cond for index cache");
taosMemoryFree(cache);
return NULL;
}
idxCacheRef(cache);
if (idx != NULL) {
@ -410,10 +419,16 @@ IndexCache* idxCacheCreate(SIndex* idx, uint64_t suid, const char* colName, int8
void idxCacheDebug(IndexCache* cache) {
MemTable* tbl = NULL;
TAOS_UNUSED(taosThreadMutexLock(&cache->mtx));
if ((taosThreadMutexLock(&cache->mtx)) != 0) {
indexError("failed to lock cache mutex");
}
tbl = cache->mem;
idxMemRef(tbl);
TAOS_UNUSED(taosThreadMutexUnlock(&cache->mtx));
if (taosThreadMutexUnlock(&cache->mtx) != 0) {
indexError("failed to unlock cache mutex");
}
{
SSkipList* slt = tbl->mem;
@ -432,7 +447,9 @@ void idxCacheDebug(IndexCache* cache) {
}
{
TAOS_UNUSED(taosThreadMutexLock(&cache->mtx));
if (taosThreadMutexLock(&cache->mtx) != 0) {
indexError("failed to lock cache mutex");
}
tbl = cache->imm;
idxMemRef(tbl);
TAOS_UNUSED(taosThreadMutexUnlock(&cache->mtx));
@ -480,7 +497,9 @@ void idxCacheDestroyImm(IndexCache* cache) {
return;
}
MemTable* tbl = NULL;
TAOS_UNUSED(taosThreadMutexLock(&cache->mtx));
if (taosThreadMutexLock(&cache->mtx) != 0) {
indexError("failed to lock cache mutex");
}
tbl = cache->imm;
cache->imm = NULL; // or throw int bg thread
@ -517,7 +536,11 @@ Iterate* idxCacheIteratorCreate(IndexCache* cache) {
if (iter == NULL) {
return NULL;
}
TAOS_UNUSED(taosThreadMutexLock(&cache->mtx));
if (taosThreadMutexLock(&cache->mtx) != 0) {
indexError("failed to lock cache mutex");
taosMemoryFree(iter);
return NULL;
}
idxMemRef(cache->imm);
@ -615,7 +638,9 @@ int idxCachePut(void* cache, SIndexTerm* term, uint64_t uid) {
// ugly code, refactor later
int64_t estimate = sizeof(ct) + strlen(ct->colVal);
TAOS_UNUSED(taosThreadMutexLock(&pCache->mtx));
if (taosThreadMutexLock(&pCache->mtx) != 0) {
indexError("failed to lock cache mutex");
}
pCache->occupiedMem += estimate;
idxCacheMakeRoomForWrite(pCache);
MemTable* tbl = pCache->mem;
@ -623,7 +648,9 @@ int idxCachePut(void* cache, SIndexTerm* term, uint64_t uid) {
TAOS_UNUSED(tSkipListPut(tbl->mem, (char*)ct));
idxMemUnRef(tbl);
TAOS_UNUSED(taosThreadMutexUnlock(&pCache->mtx));
if (taosThreadMutexUnlock(&pCache->mtx) != 0) {
indexError("failed to unlock cache mutex");
}
idxCacheUnRef(pCache);
return 0;
}
@ -631,13 +658,17 @@ void idxCacheForceToMerge(void* cache) {
IndexCache* pCache = cache;
idxCacheRef(pCache);
TAOS_UNUSED(taosThreadMutexLock(&pCache->mtx));
if (taosThreadMutexLock(&pCache->mtx) != 0) {
indexError("failed to lock cache mutex");
}
indexInfo("%p is forced to merge into tfile", pCache);
pCache->occupiedMem += MEM_SIGNAL_QUIT;
idxCacheMakeRoomForWrite(pCache);
TAOS_UNUSED(taosThreadMutexUnlock(&pCache->mtx));
if (taosThreadMutexUnlock(&pCache->mtx) != 0) {
indexError("failed to unlock cache mutex");
}
idxCacheUnRef(pCache);
return;
}
@ -668,12 +699,16 @@ int idxCacheSearch(void* cache, SIndexTermQuery* query, SIdxTRslt* result, STerm
IndexCache* pCache = cache;
MemTable *mem = NULL, *imm = NULL;
TAOS_UNUSED(taosThreadMutexLock(&pCache->mtx));
if (taosThreadMutexLock(&pCache->mtx) != 0) {
indexError("failed to lock cache mutex");
}
mem = pCache->mem;
imm = pCache->imm;
idxMemRef(mem);
idxMemRef(imm);
TAOS_UNUSED(taosThreadMutexUnlock(&pCache->mtx));
if (taosThreadMutexUnlock(&pCache->mtx) != 0) {
indexError("failed to unlock cache mutex");
}
int64_t st = taosGetTimestampUs();

4
source/libs/index/src/indexFst.c
View File

@ -994,7 +994,9 @@ Fst* fstCreate(FstSlice* slice) {
*s = fstSliceCopy(slice, 0, FST_SLICE_LEN(slice) - 1);
fst->data = s;
TAOS_UNUSED(taosThreadMutexInit(&fst->mtx, NULL));
if (taosThreadMutexInit(&fst->mtx, NULL) != 0) {
goto FST_CREAT_FAILED;
}
return fst;
FST_CREAT_FAILED:

25
source/libs/index/src/indexTfile.c
View File

@ -739,7 +739,11 @@ IndexTFile* idxTFileCreate(SIndex* idx, const char* path) {
tfileCacheDestroy(cache);
return NULL;
}
TAOS_UNUSED(taosThreadMutexInit(&tfile->mtx, NULL));
if (taosThreadMutexInit(&tfile->mtx, NULL) != 0) {
taosMemoryFree(tfile);
tfileCacheDestroy(cache);
return NULL;
}
tfile->cache = cache;
return tfile;
}
@ -764,9 +768,16 @@ int idxTFileSearch(void* tfile, SIndexTermQuery* query, SIdxTRslt* result) {
SIndexTerm* term = query->term;
ICacheKey key = {.suid = term->suid, .colType = term->colType, .colName = term->colName, .nColName = term->nColName};
TAOS_UNUSED(taosThreadMutexLock(&pTfile->mtx));
if (taosThreadMutexLock(&pTfile->mtx) != 0) {
indexError("failed to lock tfile mutex");
}
TFileReader* reader = tfileCacheGet(pTfile->cache, &key);
TAOS_UNUSED(taosThreadMutexUnlock(&pTfile->mtx));
if (taosThreadMutexUnlock(&pTfile->mtx) != 0) {
indexError("failed to unlock tfile mutex");
}
if (reader == NULL) {
return 0;
}
@ -883,9 +894,13 @@ TFileReader* tfileGetReaderByCol(IndexTFile* tf, uint64_t suid, char* colName) {
TFileReader* rd = NULL;
ICacheKey key = {.suid = suid, .colType = TSDB_DATA_TYPE_BINARY, .colName = colName, .nColName = strlen(colName)};
TAOS_UNUSED(taosThreadMutexLock(&tf->mtx));
if (taosThreadMutexLock(&tf->mtx) != 0) {
indexError("failed to lock tfile mutex");
}
rd = tfileCacheGet(tf->cache, &key);
TAOS_UNUSED(taosThreadMutexUnlock(&tf->mtx));
if (taosThreadMutexUnlock(&tf->mtx) != 0) {
indexError("failed to unlock tfile mutex");
}
return rd;
}

60
source/libs/monitor/src/monFramework.c
View File

@ -100,7 +100,9 @@ extern char* tsMonFwUri;
#define VNODE_ROLE "taosd_vnodes_info:role"
void monInitMonitorFW(){
(void)taos_collector_registry_default_init();
if (taos_collector_registry_default_init() != 0) {
uError("failed to init default collector registry");
}
tsMonitor.metrics = taosHashInit(16, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_ENTRY_LOCK);
taos_gauge_t *gauge = NULL;
@ -115,7 +117,9 @@ void monInitMonitorFW(){
for(int32_t i = 0; i < 25; i++){
gauge= taos_gauge_new(dnodes_gauges[i], "", dnodes_label_count, dnodes_sample_labels);
if(taos_collector_registry_register_metric(gauge) == 1){
(void)taos_counter_destroy(gauge);
if (taos_counter_destroy(gauge) != 0) {
uError("failed to delete metric %s", dnodes_gauges[i]);
}
}
if (taosHashPut(tsMonitor.metrics, dnodes_gauges[i], strlen(dnodes_gauges[i]), &gauge, sizeof(taos_gauge_t *)) !=
0) {
@ -129,7 +133,9 @@ void monInitMonitorFW(){
for(int32_t i = 0; i < 3; i++){
gauge= taos_gauge_new(dnodes_data_gauges[i], "", dnodes_data_label_count, dnodes_data_sample_labels);
if(taos_collector_registry_register_metric(gauge) == 1){
(void)taos_counter_destroy(gauge);
if (taos_counter_destroy(gauge) != 0) {
uError("failed to delete metric %s", dnodes_data_gauges[i]);
}
}
if (taosHashPut(tsMonitor.metrics, dnodes_data_gauges[i], strlen(dnodes_data_gauges[i]), &gauge,
sizeof(taos_gauge_t *)) != 0) {
@ -143,7 +149,9 @@ void monInitMonitorFW(){
for(int32_t i = 0; i < 3; i++){
gauge= taos_gauge_new(dnodes_log_gauges[i], "", dnodes_log_label_count, dnodes_log_sample_labels);
if(taos_collector_registry_register_metric(gauge) == 1){
(void)taos_counter_destroy(gauge);
if (taos_counter_destroy(gauge) != 0) {
uError("failed to delete metric %s", dnodes_log_gauges[i]);
}
}
if (taosHashPut(tsMonitor.metrics, dnodes_log_gauges[i], strlen(dnodes_log_gauges[i]), &gauge,
sizeof(taos_gauge_t *)) != 0) {
@ -154,7 +162,9 @@ void monInitMonitorFW(){
void monCleanupMonitorFW(){
taosHashCleanup(tsMonitor.metrics);
(void)taos_collector_registry_destroy(TAOS_COLLECTOR_REGISTRY_DEFAULT);
if (taos_collector_registry_destroy(TAOS_COLLECTOR_REGISTRY_DEFAULT) != 0) {
uError("failed to destroy default collector registry");
}
TAOS_COLLECTOR_REGISTRY_DEFAULT = NULL;
}
@ -174,7 +184,9 @@ void monGenClusterInfoTable(SMonInfo *pMonitor){
uError("failed to delete metric %s", metric_names[i]);
}
(void)taosHashRemove(tsMonitor.metrics, metric_names[i], strlen(metric_names[i]));
if (taosHashRemove(tsMonitor.metrics, metric_names[i], strlen(metric_names[i])) != 0) {
uError("failed to remove metric %s", metric_names[i]);
}
}
if(pBasicInfo->cluster_id == 0) {
@ -191,7 +203,9 @@ void monGenClusterInfoTable(SMonInfo *pMonitor){
for(int32_t i = 0; i < 18; i++){
gauge= taos_gauge_new(metric_names[i], "", label_count, sample_labels1);
if(taos_collector_registry_register_metric(gauge) == 1){
(void)taos_counter_destroy(gauge);
if (taos_counter_destroy(gauge) != 0) {
uError("failed to delete metric %s", metric_names[i]);
}
}
if (taosHashPut(tsMonitor.metrics, metric_names[i], strlen(metric_names[i]), &gauge, sizeof(taos_gauge_t *)) != 0) {
uError("failed to add cluster gauge at%d:%s", i, metric_names[i]);
@ -317,11 +331,15 @@ void monGenVgroupInfoTable(SMonInfo *pMonitor){
const char *vgroup_sample_labels[] = {"cluster_id", "vgroup_id", "database_name"};
taos_gauge_t *tableNumGauge = taos_gauge_new(TABLES_NUM, "", vgroup_label_count, vgroup_sample_labels);
if(taos_collector_registry_register_metric(tableNumGauge) == 1){
(void)taos_counter_destroy(tableNumGauge);
if (taos_counter_destroy(tableNumGauge) != 0) {
uError("failed to delete metric " TABLES_NUM);
}
}
taos_gauge_t *statusGauge = taos_gauge_new(STATUS, "", vgroup_label_count, vgroup_sample_labels);
if(taos_collector_registry_register_metric(statusGauge) == 1){
(void)taos_counter_destroy(statusGauge);
if (taos_counter_destroy(statusGauge) != 0) {
uError("failed to delete metric " STATUS);
}
}
char cluster_id[TSDB_CLUSTER_ID_LEN] = {0};
@ -530,7 +548,9 @@ void monGenDnodeStatusInfoTable(SMonInfo *pMonitor){
gauge= taos_gauge_new(DNODE_STATUS, "", dnodes_label_count, dnodes_sample_labels);
if(taos_collector_registry_register_metric(gauge) == 1){
(void)taos_counter_destroy(gauge);
if (taos_counter_destroy(gauge) != 0) {
uError("failed to delete metric " DNODE_STATUS);
}
}
char cluster_id[TSDB_CLUSTER_ID_LEN];
@ -633,7 +653,9 @@ void monGenMnodeRoleTable(SMonInfo *pMonitor){
uError("failed to delete metric %s", mnodes_role_gauges[i]);
}
(void)taosHashRemove(tsMonitor.metrics, mnodes_role_gauges[i], strlen(mnodes_role_gauges[i]));
if (taosHashRemove(tsMonitor.metrics, mnodes_role_gauges[i], strlen(mnodes_role_gauges[i])) != 0) {
uError("failed to remove metric %s", mnodes_role_gauges[i]);
}
}
SMonClusterInfo *pInfo = &pMonitor->mmInfo.cluster;
@ -647,7 +669,9 @@ void monGenMnodeRoleTable(SMonInfo *pMonitor){
for(int32_t i = 0; i < 1; i++){
gauge= taos_gauge_new(mnodes_role_gauges[i], "", mnodes_role_label_count, mnodes_role_sample_labels);
if(taos_collector_registry_register_metric(gauge) == 1){
(void)taos_counter_destroy(gauge);
if (taos_counter_destroy(gauge) != 0) {
uError("failed to destroy gauge");
}
}
if (taosHashPut(tsMonitor.metrics, mnodes_role_gauges[i], strlen(mnodes_role_gauges[i]), &gauge,
sizeof(taos_gauge_t *)) != 0) {
@ -702,7 +726,9 @@ void monGenVnodeRoleTable(SMonInfo *pMonitor){
uError("failed to delete metric %s", vnodes_role_gauges[i]);
}
(void)taosHashRemove(tsMonitor.metrics, vnodes_role_gauges[i], strlen(vnodes_role_gauges[i]));
if (taosHashRemove(tsMonitor.metrics, vnodes_role_gauges[i], strlen(vnodes_role_gauges[i])) != 0) {
uError("failed to remove metric %s", vnodes_role_gauges[i]);
}
}
SMonVgroupInfo *pInfo = &pMonitor->mmInfo.vgroup;
@ -716,7 +742,9 @@ void monGenVnodeRoleTable(SMonInfo *pMonitor){
for(int32_t i = 0; i < 1; i++){
gauge= taos_gauge_new(vnodes_role_gauges[i], "", vnodes_role_label_count, vnodes_role_sample_labels);
if(taos_collector_registry_register_metric(gauge) == 1){
(void)taos_counter_destroy(gauge);
if (taos_counter_destroy(gauge) != 0) {
uError("failed to destroy gauge");
}
}
if (taosHashPut(tsMonitor.metrics, vnodes_role_gauges[i], strlen(vnodes_role_gauges[i]), &gauge,
sizeof(taos_gauge_t *)) != 0) {
@ -774,7 +802,9 @@ void monSendPromReport() {
tmp) != 0) {
uError("failed to send monitor msg");
} else {
(void)taos_collector_registry_clear_batch(TAOS_COLLECTOR_REGISTRY_DEFAULT);
if (taos_collector_registry_clear_batch(TAOS_COLLECTOR_REGISTRY_DEFAULT) != 0) {
uError("failed to clear batch");
}
}
taosMemoryFreeClear(pCont);
}

209
source/libs/monitor/src/monMain.c
View File

@ -145,7 +145,9 @@ void monInitVnode() {
counter = taos_counter_new(VNODE_METRIC_SQL_COUNT, "counter for insert sql", label_count, sample_labels);
uDebug("new metric:%p", counter);
if (taos_collector_registry_register_metric(counter) == 1) {
(void)taos_counter_destroy(counter);
if (taos_counter_destroy(counter) != 0) {
uError("failed to destroy metric:%p", counter);
}
uError("failed to register metric:%p", counter);
} else {
tsInsertCounter = counter;
@ -226,14 +228,17 @@ static void monGenBasicJson(SMonInfo *pMonitor) {
SJson *pJson = pMonitor->pJson;
char buf[40] = {0};
(void)taosFormatUtcTime(buf, sizeof(buf), pMonitor->curTime, TSDB_TIME_PRECISION_MILLI);
if (taosFormatUtcTime(buf, sizeof(buf), pMonitor->curTime, TSDB_TIME_PRECISION_MILLI) != 0) {
uError("failed to format time");
return;
}
(void)tjsonAddStringToObject(pJson, "ts", buf);
(void)tjsonAddDoubleToObject(pJson, "dnode_id", pInfo->dnode_id);
(void)tjsonAddStringToObject(pJson, "dnode_ep", pInfo->dnode_ep);
if (tjsonAddStringToObject(pJson, "ts", buf) != 0) uError("failed to add ts");
if (tjsonAddDoubleToObject(pJson, "dnode_id", pInfo->dnode_id) != 0) uError("failed to add dnode_id");
if (tjsonAddStringToObject(pJson, "dnode_ep", pInfo->dnode_ep) != 0) uError("failed to add dnode_ep");
snprintf(buf, sizeof(buf), "%" PRId64, pInfo->cluster_id);
(void)tjsonAddStringToObject(pJson, "cluster_id", buf);
(void)tjsonAddDoubleToObject(pJson, "protocol", pInfo->protocol);
if (tjsonAddStringToObject(pJson, "cluster_id", buf) != 0) uError("failed to add cluster_id");
if (tjsonAddDoubleToObject(pJson, "protocol", pInfo->protocol) != 0) uError("failed to add protocol");
}
static void monGenBasicJsonBasic(SMonInfo *pMonitor) {
@ -244,12 +249,12 @@ static void monGenBasicJsonBasic(SMonInfo *pMonitor) {
char buf[40] = {0};
sprintf(buf, "%" PRId64, taosGetTimestamp(TSDB_TIME_PRECISION_MILLI));
(void)tjsonAddStringToObject(pJson, "ts", buf);
(void)tjsonAddDoubleToObject(pJson, "dnode_id", pInfo->dnode_id);
(void)tjsonAddStringToObject(pJson, "dnode_ep", pInfo->dnode_ep);
if (tjsonAddStringToObject(pJson, "ts", buf) != 0) uError("failed to add ts");
if (tjsonAddDoubleToObject(pJson, "dnode_id", pInfo->dnode_id) != 0) uError("failed to add dnode_id");
if (tjsonAddStringToObject(pJson, "dnode_ep", pInfo->dnode_ep) != 0) uError("failed to add dnode_ep");
snprintf(buf, sizeof(buf), "%" PRId64, pInfo->cluster_id);
(void)tjsonAddStringToObject(pJson, "cluster_id", buf);
(void)tjsonAddDoubleToObject(pJson, "protocol", pInfo->protocol);
if (tjsonAddStringToObject(pJson, "cluster_id", buf) != 0) uError("failed to add cluster_id");
if (tjsonAddDoubleToObject(pJson, "protocol", pInfo->protocol) != 0) uError("failed to add protocol");
}
static void monGenClusterJson(SMonInfo *pMonitor) {
@ -263,21 +268,24 @@ static void monGenClusterJson(SMonInfo *pMonitor) {
return;
}
(void)tjsonAddStringToObject(pJson, "first_ep", pInfo->first_ep);
(void)tjsonAddDoubleToObject(pJson, "first_ep_dnode_id", pInfo->first_ep_dnode_id);
(void)tjsonAddStringToObject(pJson, "version", pInfo->version);
(void)tjsonAddDoubleToObject(pJson, "master_uptime", pInfo->master_uptime);
(void)tjsonAddDoubleToObject(pJson, "monitor_interval", pInfo->monitor_interval);
(void)tjsonAddDoubleToObject(pJson, "dbs_total", pInfo->dbs_total);
(void)tjsonAddDoubleToObject(pJson, "tbs_total", pInfo->tbs_total);
(void)tjsonAddDoubleToObject(pJson, "stbs_total", pInfo->stbs_total);
(void)tjsonAddDoubleToObject(pJson, "vgroups_total", pInfo->vgroups_total);
(void)tjsonAddDoubleToObject(pJson, "vgroups_alive", pInfo->vgroups_alive);
(void)tjsonAddDoubleToObject(pJson, "vnodes_total", pInfo->vnodes_total);
(void)tjsonAddDoubleToObject(pJson, "vnodes_alive", pInfo->vnodes_alive);
(void)tjsonAddDoubleToObject(pJson, "connections_total", pInfo->connections_total);
(void)tjsonAddDoubleToObject(pJson, "topics_total", pInfo->topics_toal);
(void)tjsonAddDoubleToObject(pJson, "streams_total", pInfo->streams_total);
if (tjsonAddStringToObject(pJson, "first_ep", pInfo->first_ep) != 0) uError("failed to add first_ep");
if (tjsonAddDoubleToObject(pJson, "first_ep_dnode_id", pInfo->first_ep_dnode_id) != 0)
uError("failed to add first_ep_dnode_id");
if (tjsonAddStringToObject(pJson, "version", pInfo->version) != 0) uError("failed to add version");
if (tjsonAddDoubleToObject(pJson, "master_uptime", pInfo->master_uptime) != 0) uError("failed to add master_uptime");
if (tjsonAddDoubleToObject(pJson, "monitor_interval", pInfo->monitor_interval) != 0)
uError("failed to add monitor_interval");
if (tjsonAddDoubleToObject(pJson, "dbs_total", pInfo->dbs_total) != 0) uError("failed to add dbs_total");
if (tjsonAddDoubleToObject(pJson, "tbs_total", pInfo->tbs_total) != 0) uError("failed to add tbs_total");
if (tjsonAddDoubleToObject(pJson, "stbs_total", pInfo->stbs_total) != 0) uError("failed to add stbs_total");
if (tjsonAddDoubleToObject(pJson, "vgroups_total", pInfo->vgroups_total) != 0) uError("failed to add vgroups_total");
if (tjsonAddDoubleToObject(pJson, "vgroups_alive", pInfo->vgroups_alive) != 0) uError("failed to add vgroups_alive");
if (tjsonAddDoubleToObject(pJson, "vnodes_total", pInfo->vnodes_total) != 0) uError("failed to add vnodes_total");
if (tjsonAddDoubleToObject(pJson, "vnodes_alive", pInfo->vnodes_alive) != 0) uError("failed to add vnodes_alive");
if (tjsonAddDoubleToObject(pJson, "connections_total", pInfo->connections_total) != 0)
uError("failed to add connections_total");
if (tjsonAddDoubleToObject(pJson, "topics_total", pInfo->topics_toal) != 0) uError("failed to add topics_total");
if (tjsonAddDoubleToObject(pJson, "streams_total", pInfo->streams_total) != 0) uError("failed to add streams_total");
SJson *pDnodesJson = tjsonAddArrayToObject(pJson, "dnodes");
if (pDnodesJson == NULL) return;
@ -287,9 +295,9 @@ static void monGenClusterJson(SMonInfo *pMonitor) {
if (pDnodeJson == NULL) continue;
SMonDnodeDesc *pDnodeDesc = taosArrayGet(pInfo->dnodes, i);
(void)tjsonAddDoubleToObject(pDnodeJson, "dnode_id", pDnodeDesc->dnode_id);
(void)tjsonAddStringToObject(pDnodeJson, "dnode_ep", pDnodeDesc->dnode_ep);
(void)tjsonAddStringToObject(pDnodeJson, "status", pDnodeDesc->status);
if (tjsonAddDoubleToObject(pDnodeJson, "dnode_id", pDnodeDesc->dnode_id) != 0) uError("failed to add dnode_id");
if (tjsonAddStringToObject(pDnodeJson, "dnode_ep", pDnodeDesc->dnode_ep) != 0) uError("failed to add dnode_ep");
if (tjsonAddStringToObject(pDnodeJson, "status", pDnodeDesc->status) != 0) uError("failed to add status");
if (tjsonAddItemToArray(pDnodesJson, pDnodeJson) != 0) tjsonDelete(pDnodeJson);
}
@ -302,9 +310,9 @@ static void monGenClusterJson(SMonInfo *pMonitor) {
if (pMnodeJson == NULL) continue;
SMonMnodeDesc *pMnodeDesc = taosArrayGet(pInfo->mnodes, i);
(void)tjsonAddDoubleToObject(pMnodeJson, "mnode_id", pMnodeDesc->mnode_id);
(void)tjsonAddStringToObject(pMnodeJson, "mnode_ep", pMnodeDesc->mnode_ep);
(void)tjsonAddStringToObject(pMnodeJson, "role", pMnodeDesc->role);
if (tjsonAddDoubleToObject(pMnodeJson, "mnode_id", pMnodeDesc->mnode_id) != 0) uError("failed to add mnode_id");
if (tjsonAddStringToObject(pMnodeJson, "mnode_ep", pMnodeDesc->mnode_ep) != 0) uError("failed to add mnode_ep");
if (tjsonAddStringToObject(pMnodeJson, "role", pMnodeDesc->role) != 0) uError("failed to add role");
if (tjsonAddItemToArray(pMnodesJson, pMnodeJson) != 0) tjsonDelete(pMnodeJson);
}
@ -314,11 +322,11 @@ static void monGenClusterJsonBasic(SMonInfo *pMonitor) {
SMonClusterInfo *pInfo = &pMonitor->mmInfo.cluster;
if (pMonitor->mmInfo.cluster.first_ep_dnode_id == 0) return;
// (void)tjsonAddStringToObject(pMonitor->pJson, "first_ep", pInfo->first_ep);
(void)tjsonAddStringToObject(pMonitor->pJson, "first_ep", tsFirst);
(void)tjsonAddDoubleToObject(pMonitor->pJson, "first_ep_dnode_id", pInfo->first_ep_dnode_id);
(void)tjsonAddStringToObject(pMonitor->pJson, "cluster_version", pInfo->version);
// (void)tjsonAddDoubleToObject(pMonitor->pJson, "monitor_interval", pInfo->monitor_interval);
if (tjsonAddStringToObject(pMonitor->pJson, "first_ep", tsFirst) != 0) uError("failed to add first_ep");
if (tjsonAddDoubleToObject(pMonitor->pJson, "first_ep_dnode_id", pInfo->first_ep_dnode_id) != 0)
uError("failed to add first_ep_dnode_id");
if (tjsonAddStringToObject(pMonitor->pJson, "cluster_version", pInfo->version) != 0)
uError("failed to add cluster_version");
}
static void monGenVgroupJson(SMonInfo *pMonitor) {
@ -337,10 +345,13 @@ static void monGenVgroupJson(SMonInfo *pMonitor) {
}
SMonVgroupDesc *pVgroupDesc = taosArrayGet(pInfo->vgroups, i);
(void)tjsonAddDoubleToObject(pVgroupJson, "vgroup_id", pVgroupDesc->vgroup_id);
(void)tjsonAddStringToObject(pVgroupJson, "database_name", pVgroupDesc->database_name);
(void)tjsonAddDoubleToObject(pVgroupJson, "tables_num", pVgroupDesc->tables_num);
(void)tjsonAddStringToObject(pVgroupJson, "status", pVgroupDesc->status);
if (tjsonAddDoubleToObject(pVgroupJson, "vgroup_id", pVgroupDesc->vgroup_id) != 0)
uError("failed to add vgroup_id");
if (tjsonAddStringToObject(pVgroupJson, "database_name", pVgroupDesc->database_name) != 0)
uError("failed to add database_name");
if (tjsonAddDoubleToObject(pVgroupJson, "tables_num", pVgroupDesc->tables_num) != 0)
uError("failed to add tables_num");
if (tjsonAddStringToObject(pVgroupJson, "status", pVgroupDesc->status) != 0) uError("failed to add status");
SJson *pVnodesJson = tjsonAddArrayToObject(pVgroupJson, "vnodes");
if (pVnodesJson == NULL) continue;
@ -352,8 +363,9 @@ static void monGenVgroupJson(SMonInfo *pMonitor) {
SJson *pVnodeJson = tjsonCreateObject();
if (pVnodeJson == NULL) continue;
(void)tjsonAddDoubleToObject(pVnodeJson, "dnode_id", pVnodeDesc->dnode_id);
(void)tjsonAddStringToObject(pVnodeJson, "vnode_role", pVnodeDesc->vnode_role);
if (tjsonAddDoubleToObject(pVnodeJson, "dnode_id", pVnodeDesc->dnode_id) != 0) uError("failed to add dnode_id");
if (tjsonAddStringToObject(pVnodeJson, "vnode_role", pVnodeDesc->vnode_role) != 0)
uError("failed to add vnode_role");
if (tjsonAddItemToArray(pVnodesJson, pVnodeJson) != 0) tjsonDelete(pVnodeJson);
}
@ -376,8 +388,9 @@ static void monGenStbJson(SMonInfo *pMonitor) {
}
SMonStbDesc *pStbDesc = taosArrayGet(pInfo->stbs, i);
(void)tjsonAddStringToObject(pStbJson, "stb_name", pStbDesc->stb_name);
(void)tjsonAddStringToObject(pStbJson, "database_name", pStbDesc->database_name);
if (tjsonAddStringToObject(pStbJson, "stb_name", pStbDesc->stb_name) != 0) uError("failed to add stb_name");
if (tjsonAddStringToObject(pStbJson, "database_name", pStbDesc->database_name) != 0)
uError("failed to add database_name");
}
}
@ -392,9 +405,11 @@ static void monGenGrantJson(SMonInfo *pMonitor) {
return;
}
(void)tjsonAddDoubleToObject(pJson, "expire_time", pInfo->expire_time);
(void)tjsonAddDoubleToObject(pJson, "timeseries_used", pInfo->timeseries_used);
(void)tjsonAddDoubleToObject(pJson, "timeseries_total", pInfo->timeseries_total);
if (tjsonAddDoubleToObject(pJson, "expire_time", pInfo->expire_time) != 0) uError("failed to add expire_time");
if (tjsonAddDoubleToObject(pJson, "timeseries_used", pInfo->timeseries_used) != 0)
uError("failed to add timeseries_used");
if (tjsonAddDoubleToObject(pJson, "timeseries_total", pInfo->timeseries_total) != 0)
uError("failed to add timeseries_total");
}
static void monGenDnodeJson(SMonInfo *pMonitor) {
@ -451,36 +466,40 @@ static void monGenDnodeJson(SMonInfo *pMonitor) {
double io_read_disk_rate = io_read_disk / interval;
double io_write_disk_rate = io_write_disk / interval;
(void)tjsonAddDoubleToObject(pJson, "uptime", pInfo->uptime);
(void)tjsonAddDoubleToObject(pJson, "cpu_engine", cpu_engine);
(void)tjsonAddDoubleToObject(pJson, "cpu_system", pSys->cpu_system);
(void)tjsonAddDoubleToObject(pJson, "cpu_cores", pSys->cpu_cores);
(void)tjsonAddDoubleToObject(pJson, "mem_engine", mem_engine);
(void)tjsonAddDoubleToObject(pJson, "mem_system", pSys->mem_system);
(void)tjsonAddDoubleToObject(pJson, "mem_total", pSys->mem_total);
(void)tjsonAddDoubleToObject(pJson, "disk_engine", pSys->disk_engine);
(void)tjsonAddDoubleToObject(pJson, "disk_used", pSys->disk_used);
(void)tjsonAddDoubleToObject(pJson, "disk_total", pSys->disk_total);
(void)tjsonAddDoubleToObject(pJson, "net_in", net_in_rate);
(void)tjsonAddDoubleToObject(pJson, "net_out", net_out_rate);
(void)tjsonAddDoubleToObject(pJson, "io_read", io_read_rate);
(void)tjsonAddDoubleToObject(pJson, "io_write", io_write_rate);
(void)tjsonAddDoubleToObject(pJson, "io_read_disk", io_read_disk_rate);
(void)tjsonAddDoubleToObject(pJson, "io_write_disk", io_write_disk_rate);
(void)tjsonAddDoubleToObject(pJson, "req_select", pStat->numOfSelectReqs);
(void)tjsonAddDoubleToObject(pJson, "req_select_rate", req_select_rate);
(void)tjsonAddDoubleToObject(pJson, "req_insert", pStat->numOfInsertReqs);
(void)tjsonAddDoubleToObject(pJson, "req_insert_success", pStat->numOfInsertSuccessReqs);
(void)tjsonAddDoubleToObject(pJson, "req_insert_rate", req_insert_rate);
(void)tjsonAddDoubleToObject(pJson, "req_insert_batch", pStat->numOfBatchInsertReqs);
(void)tjsonAddDoubleToObject(pJson, "req_insert_batch_success", pStat->numOfBatchInsertSuccessReqs);
(void)tjsonAddDoubleToObject(pJson, "req_insert_batch_rate", req_insert_batch_rate);
(void)tjsonAddDoubleToObject(pJson, "errors", pStat->errors);
(void)tjsonAddDoubleToObject(pJson, "vnodes_num", pStat->totalVnodes);
(void)tjsonAddDoubleToObject(pJson, "masters", pStat->masterNum);
(void)tjsonAddDoubleToObject(pJson, "has_mnode", pInfo->has_mnode);
(void)tjsonAddDoubleToObject(pJson, "has_qnode", pInfo->has_qnode);
(void)tjsonAddDoubleToObject(pJson, "has_snode", pInfo->has_snode);
if (tjsonAddDoubleToObject(pJson, "uptime", pInfo->uptime) != 0) uError("failed to add uptime");
if (tjsonAddDoubleToObject(pJson, "cpu_engine", cpu_engine) != 0) uError("failed to add cpu_engine");
if (tjsonAddDoubleToObject(pJson, "cpu_system", pSys->cpu_system) != 0) uError("failed to add cpu_system");
if (tjsonAddDoubleToObject(pJson, "cpu_cores", pSys->cpu_cores) != 0) uError("failed to add cpu_cores");
if (tjsonAddDoubleToObject(pJson, "mem_engine", mem_engine) != 0) uError("failed to add mem_engine");
if (tjsonAddDoubleToObject(pJson, "mem_system", pSys->mem_system) != 0) uError("failed to add mem_system");
if (tjsonAddDoubleToObject(pJson, "mem_total", pSys->mem_total) != 0) uError("failed to add mem_total");
if (tjsonAddDoubleToObject(pJson, "disk_engine", pSys->disk_engine) != 0) uError("failed to add disk_engine");
if (tjsonAddDoubleToObject(pJson, "disk_used", pSys->disk_used) != 0) uError("failed to add disk_used");
if (tjsonAddDoubleToObject(pJson, "disk_total", pSys->disk_total) != 0) uError("failed to add disk_total");
if (tjsonAddDoubleToObject(pJson, "net_in", net_in_rate) != 0) uError("failed to add net_in");
if (tjsonAddDoubleToObject(pJson, "net_out", net_out_rate) != 0) uError("failed to add net_out");
if (tjsonAddDoubleToObject(pJson, "io_read", io_read_rate) != 0) uError("failed to add io_read");
if (tjsonAddDoubleToObject(pJson, "io_write", io_write_rate) != 0) uError("failed to add io_write");
if (tjsonAddDoubleToObject(pJson, "io_read_disk", io_read_disk_rate) != 0) uError("failed to add io_read_disk");
if (tjsonAddDoubleToObject(pJson, "io_write_disk", io_write_disk_rate) != 0) uError("failed to add io_write_disk");
if (tjsonAddDoubleToObject(pJson, "req_select", pStat->numOfSelectReqs) != 0) uError("failed to add req_select");
if (tjsonAddDoubleToObject(pJson, "req_select_rate", req_select_rate) != 0) uError("failed to add req_select_rate");
if (tjsonAddDoubleToObject(pJson, "req_insert", pStat->numOfInsertReqs) != 0) uError("failed to add req_insert");
if (tjsonAddDoubleToObject(pJson, "req_insert_success", pStat->numOfInsertSuccessReqs) != 0)
uError("failed to add req_insert_success");
if (tjsonAddDoubleToObject(pJson, "req_insert_rate", req_insert_rate) != 0) uError("failed to add req_insert_rate");
if (tjsonAddDoubleToObject(pJson, "req_insert_batch", pStat->numOfBatchInsertReqs) != 0)
uError("failed to add req_insert_batch");
if (tjsonAddDoubleToObject(pJson, "req_insert_batch_success", pStat->numOfBatchInsertSuccessReqs) != 0)
uError("failed to add req_insert_batch_success");
if (tjsonAddDoubleToObject(pJson, "req_insert_batch_rate", req_insert_batch_rate) != 0)
uError("failed to add req_insert_batch_rate");
if (tjsonAddDoubleToObject(pJson, "errors", pStat->errors) != 0) uError("failed to add errors");
if (tjsonAddDoubleToObject(pJson, "vnodes_num", pStat->totalVnodes) != 0) uError("failed to add vnodes_num");
if (tjsonAddDoubleToObject(pJson, "masters", pStat->masterNum) != 0) uError("failed to add masters");
if (tjsonAddDoubleToObject(pJson, "has_mnode", pInfo->has_mnode) != 0) uError("failed to add has_mnode");
if (tjsonAddDoubleToObject(pJson, "has_qnode", pInfo->has_qnode) != 0) uError("failed to add has_qnode");
if (tjsonAddDoubleToObject(pJson, "has_snode", pInfo->has_snode) != 0) uError("failed to add has_snode");
}
static void monGenDiskJson(SMonInfo *pMonitor) {
@ -515,18 +534,18 @@ static void monGenDiskJson(SMonInfo *pMonitor) {
SJson *pLogdirJson = tjsonCreateObject();
if (pLogdirJson == NULL) return;
if (tjsonAddItemToObject(pJson, "logdir", pLogdirJson) != 0) return;
(void)tjsonAddStringToObject(pLogdirJson, "name", pLogDesc->name);
(void)tjsonAddDoubleToObject(pLogdirJson, "avail", pLogDesc->size.avail);
(void)tjsonAddDoubleToObject(pLogdirJson, "used", pLogDesc->size.used);
(void)tjsonAddDoubleToObject(pLogdirJson, "total", pLogDesc->size.total);
if (tjsonAddStringToObject(pLogdirJson, "name", pLogDesc->name) != 0) uError("failed to add string to json");
if (tjsonAddDoubleToObject(pLogdirJson, "avail", pLogDesc->size.avail) != 0) uError("failed to add double to json");
if (tjsonAddDoubleToObject(pLogdirJson, "used", pLogDesc->size.used) != 0) uError("failed to add double to json");
if (tjsonAddDoubleToObject(pLogdirJson, "total", pLogDesc->size.total) != 0) uError("failed to add double to json");
SJson *pTempdirJson = tjsonCreateObject();
if (pTempdirJson == NULL) return;
if (tjsonAddItemToObject(pJson, "tempdir", pTempdirJson) != 0) return;
(void)tjsonAddStringToObject(pTempdirJson, "name", pTempDesc->name);
(void)tjsonAddDoubleToObject(pTempdirJson, "avail", pTempDesc->size.avail);
(void)tjsonAddDoubleToObject(pTempdirJson, "used", pTempDesc->size.used);
(void)tjsonAddDoubleToObject(pTempdirJson, "total", pTempDesc->size.total);
if (tjsonAddStringToObject(pTempdirJson, "name", pTempDesc->name) != 0) uError("failed to add string to json");
if (tjsonAddDoubleToObject(pTempdirJson, "avail", pTempDesc->size.avail) != 0) uError("failed to add double to json");
if (tjsonAddDoubleToObject(pTempdirJson, "used", pTempDesc->size.used) != 0) uError("failed to add double to json");
if (tjsonAddDoubleToObject(pTempdirJson, "total", pTempDesc->size.total) != 0) uError("failed to add double to json");
}
static const char *monLogLevelStr(ELogLevel level) {
@ -571,26 +590,26 @@ static void monGenLogJson(SMonInfo *pMonitor) {
SJson *pLogError = tjsonCreateObject();
if (pLogError == NULL) return;
(void)tjsonAddStringToObject(pLogError, "level", "error");
(void)tjsonAddDoubleToObject(pLogError, "total", numOfErrorLogs);
if (tjsonAddStringToObject(pLogError, "level", "error") != 0) uError("failed to add string to json");
if (tjsonAddDoubleToObject(pLogError, "total", numOfErrorLogs) != 0) uError("failed to add double to json");
if (tjsonAddItemToArray(pSummaryJson, pLogError) != 0) tjsonDelete(pLogError);
SJson *pLogInfo = tjsonCreateObject();
if (pLogInfo == NULL) return;
(void)tjsonAddStringToObject(pLogInfo, "level", "info");
(void)tjsonAddDoubleToObject(pLogInfo, "total", numOfInfoLogs);
if (tjsonAddStringToObject(pLogInfo, "level", "info") != 0) uError("failed to add string to json");
if (tjsonAddDoubleToObject(pLogInfo, "total", numOfInfoLogs) != 0) uError("failed to add double to json");
if (tjsonAddItemToArray(pSummaryJson, pLogInfo) != 0) tjsonDelete(pLogInfo);
SJson *pLogDebug = tjsonCreateObject();
if (pLogDebug == NULL) return;
(void)tjsonAddStringToObject(pLogDebug, "level", "debug");
(void)tjsonAddDoubleToObject(pLogDebug, "total", numOfDebugLogs);
if (tjsonAddStringToObject(pLogDebug, "level", "debug") != 0) uError("failed to add string to json");
if (tjsonAddDoubleToObject(pLogDebug, "total", numOfDebugLogs) != 0) uError("failed to add double to json");
if (tjsonAddItemToArray(pSummaryJson, pLogDebug) != 0) tjsonDelete(pLogDebug);
SJson *pLogTrace = tjsonCreateObject();
if (pLogTrace == NULL) return;
(void)tjsonAddStringToObject(pLogTrace, "level", "trace");
(void)tjsonAddDoubleToObject(pLogTrace, "total", numOfTraceLogs);
if (tjsonAddStringToObject(pLogTrace, "level", "trace") != 0) uError("failed to add string to json");
if (tjsonAddDoubleToObject(pLogTrace, "total", numOfTraceLogs) != 0) uError("failed to add double to json");
if (tjsonAddItemToArray(pSummaryJson, pLogTrace) != 0) tjsonDelete(pLogTrace);
}

9
source/libs/parser/src/parTranslater.c
View File

@ -14705,7 +14705,7 @@ static int32_t rewriteDropSuperTablewithOpt(STranslateContext* pCxt, SQuery* pQu
break;
}
if (!isdigit(pStmt->tableName[i])) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_TABLE_NOT_EXIST, "Table does not exist: `%s`.`%s`",
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_TABLE_NOT_EXIST, "STable not exist: `%s`.`%s`",
pStmt->dbName, pStmt->tableName);
}
}
@ -14715,8 +14715,11 @@ static int32_t rewriteDropSuperTablewithOpt(STranslateContext* pCxt, SQuery* pQu
toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, &name);
code = getTargetName(pCxt, &name, pTableName);
if (TSDB_CODE_SUCCESS != code) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, code, "%s: db:`%s`, tbuid:`%s`", tstrerror(code), pStmt->dbName,
pStmt->tableName);
return generateSyntaxErrMsgExt(&pCxt->msgBuf, code, "%s: db:`%s`, tbuid:`%s`",
(code == TSDB_CODE_PAR_TABLE_NOT_EXIST || code == TSDB_CODE_TDB_TABLE_NOT_EXIST)
? "STable not exist"
: tstrerror(code),
pStmt->dbName, pStmt->tableName);
}
tstrncpy(pStmt->tableName, pTableName, TSDB_TABLE_NAME_LEN); // rewrite table uid to table name

1
source/libs/stream/test/checkpointTest.cpp
View File

@ -43,7 +43,6 @@
// }
// strcpy(tsSnodeAddress, "127.0.0.1");
// int ret = RUN_ALL_TESTS();
// s3CleanUp();
// return ret;
// }

14
source/libs/sync/src/syncPipeline.c
View File

@ -1139,10 +1139,15 @@ int32_t syncLogReplProcessReply(SSyncLogReplMgr* pMgr, SSyncNode* pNode, SyncApp
pMgr->peerStartTime = pMsg->startTime;
}
int32_t code = 0;
if (pMgr->restored) {
TAOS_CHECK_RETURN(syncLogReplContinue(pMgr, pNode, pMsg));
if ((code = syncLogReplContinue(pMgr, pNode, pMsg)) != 0) {
sError("vgId:%d, failed to continue sync log repl since %s", pNode->vgId, tstrerror(code));
}
} else {
TAOS_CHECK_RETURN(syncLogReplRecover(pMgr, pNode, pMsg));
if ((code = syncLogReplRecover(pMgr, pNode, pMsg)) != 0) {
sError("vgId:%d, failed to recover sync log repl since %s", pNode->vgId, tstrerror(code));
}
}
(void)taosThreadMutexUnlock(&pBuf->mutex);
return 0;
@ -1439,7 +1444,10 @@ int32_t syncLogBufferReset(SSyncLogBuffer* pBuf, SSyncNode* pNode) {
if (lastVer != pBuf->matchIndex) return TSDB_CODE_SYN_INTERNAL_ERROR;
SyncIndex index = pBuf->endIndex - 1;
TAOS_CHECK_RETURN(syncLogBufferRollback(pBuf, pNode, pBuf->matchIndex + 1));
int32_t code = 0;
if ((code = syncLogBufferRollback(pBuf, pNode, pBuf->matchIndex + 1)) != 0) {
sError("vgId:%d, failed to rollback sync log buffer since %s", pNode->vgId, tstrerror(code));
}
sInfo("vgId:%d, reset sync log buffer. buffer: [%" PRId64 " %" PRId64 " %" PRId64 ", %" PRId64 ")", pNode->vgId,
pBuf->startIndex, pBuf->commitIndex, pBuf->matchIndex, pBuf->endIndex);

19
source/libs/transport/src/transCli.c
View File

@ -1634,7 +1634,7 @@ static void cliHandleFreeById(SCliMsg* pMsg, SCliThrd* pThrd) {
int64_t refId = (int64_t)(pMsg->msg.info.handle);
SExHandle* exh = transAcquireExHandle(transGetRefMgt(), refId);
if (exh == NULL) {
tDebug("id %" PRId64 " already released", refId);
tDebug("refId %" PRId64 " already released", refId);
destroyCmsg(pMsg);
return;
}
@ -1646,7 +1646,7 @@ static void cliHandleFreeById(SCliMsg* pMsg, SCliThrd* pThrd) {
if (conn == NULL || conn->refId != refId) {
TAOS_CHECK_GOTO(TSDB_CODE_REF_INVALID_ID, NULL, _exception);
}
tDebug("do free conn %p by id %" PRId64 "", conn, refId);
tDebug("do free conn %p by refId %" PRId64 "", conn, refId);
int32_t size = transQueueSize(&conn->cliMsgs);
if (size == 0) {
@ -3316,12 +3316,13 @@ int32_t transAllocHandle(int64_t* refId) {
QUEUE_INIT(&exh->q);
taosInitRWLatch(&exh->latch);
tDebug("pre alloc refId %" PRId64 "", exh->refId);
tDebug("pre alloc refId %" PRId64 ", alloc exhandle %p", exh->refId, exh);
*refId = exh->refId;
return 0;
}
int32_t transFreeConnById(void* shandle, int64_t transpointId) {
int32_t code = 0;
SCliMsg* pCli = NULL;
STrans* pTransInst = (STrans*)transAcquireExHandle(transGetInstMgt(), (int64_t)shandle);
if (pTransInst == NULL) {
return TSDB_CODE_RPC_MODULE_QUIT;
@ -3336,7 +3337,7 @@ int32_t transFreeConnById(void* shandle, int64_t transpointId) {
TAOS_CHECK_GOTO(TSDB_CODE_REF_INVALID_ID, NULL, _exception);
}
SCliMsg* pCli = taosMemoryCalloc(1, sizeof(SCliMsg));
pCli = taosMemoryCalloc(1, sizeof(SCliMsg));
if (pCli == NULL) {
TAOS_CHECK_GOTO(terrno, NULL, _exception);
}
@ -3349,11 +3350,19 @@ int32_t transFreeConnById(void* shandle, int64_t transpointId) {
code = transAsyncSend(pThrd->asyncPool, &pCli->q);
if (code != 0) {
taosMemoryFree(pCli);
taosMemoryFreeClear(pCli);
TAOS_CHECK_GOTO(code, NULL, _exception);
}
_exception:
transReleaseExHandle(transGetInstMgt(), (int64_t)shandle);
if (code != 0) {
if (transpointId != 0) {
(void)transReleaseExHandle(transGetRefMgt(), transpointId);
(void)transRemoveExHandle(transGetRefMgt(), transpointId);
}
taosMemoryFree(pCli);
}
return code;
}

1
source/libs/transport/src/transComm.c
View File

@ -768,6 +768,7 @@ void transDestroyExHandle(void* handle) {
if (!QUEUE_IS_EMPTY(&eh->q)) {
tDebug("handle %p mem leak", handle);
}
tDebug("free exhandle %p", handle);
taosMemoryFree(handle);
}

1
source/libs/wal/src/walMgmt.c
View File

@ -189,6 +189,7 @@ SWal *walOpen(const char *path, SWalCfg *pCfg) {
_err:
taosArrayDestroy(pWal->fileInfoSet);
taosArrayDestroy(pWal->toDeleteFiles);
taosHashCleanup(pWal->pRefHash);
TAOS_UNUSED(taosThreadRwlockDestroy(&pWal->mutex));
taosMemoryFreeClear(pWal);

2
source/os/src/osSemaphore.c
View File

@ -232,7 +232,7 @@ int32_t tsem_init(tsem_t *psem, int flags, unsigned int count) {
if(sem_init(psem, flags, count) == 0) {
return 0;
} else {
return TAOS_SYSTEM_ERROR(errno);
return terrno = TAOS_SYSTEM_ERROR(errno);
}
}

6
source/util/src/tcompression.c
View File

@ -324,8 +324,8 @@ bool lossyFloat = false;
bool lossyDouble = false;
// init call
int32_t tsCompressInit(char *lossyColumns, float fPrecision, double dPrecision, uint32_t maxIntervals,
uint32_t intervals, int32_t ifAdtFse, const char *compressor) {
void tsCompressInit(char *lossyColumns, float fPrecision, double dPrecision, uint32_t maxIntervals, uint32_t intervals,
int32_t ifAdtFse, const char *compressor) {
// config
lossyFloat = strstr(lossyColumns, "float") != NULL;
lossyDouble = strstr(lossyColumns, "double") != NULL;
@ -333,7 +333,7 @@ int32_t tsCompressInit(char *lossyColumns, float fPrecision, double dPrecision,
tdszInit(fPrecision, dPrecision, maxIntervals, intervals, ifAdtFse, compressor);
if (lossyFloat) uTrace("lossy compression float is opened. ");
if (lossyDouble) uTrace("lossy compression double is opened. ");
return 0;
return;
}
// exit call
void tsCompressExit() { tdszExit(); }

138
source/util/src/tlog.c
View File

@ -153,14 +153,16 @@ static void getDay(char* buf){
time_t t = taosTime(NULL);
struct tm tmInfo;
if (taosLocalTime(&t, &tmInfo, buf) != NULL) {
(void)strftime(buf, LOG_FILE_DAY_LEN, "%Y-%m-%d", &tmInfo);
TAOS_UNUSED(strftime(buf, LOG_FILE_DAY_LEN, "%Y-%m-%d", &tmInfo));
}
}
static int64_t getTimestampToday() {
time_t t = taosTime(NULL);
struct tm tm;
(void) taosLocalTime(&t, &tm, NULL);
if (taosLocalTime(&t, &tm, NULL) == NULL) {
return 0;
}
tm.tm_hour = 0;
tm.tm_min = 0;
tm.tm_sec = 0;
@ -203,7 +205,7 @@ int32_t taosInitSlowLog() {
tsLogObj.slowHandle = taosLogBuffNew(LOG_SLOW_BUF_SIZE);
if (tsLogObj.slowHandle == NULL) return terrno;
(void)taosUmaskFile(0);
TAOS_UNUSED(taosUmaskFile(0));
tsLogObj.slowHandle->pFile = taosOpenFile(name, TD_FILE_CREATE | TD_FILE_WRITE | TD_FILE_APPEND);
if (tsLogObj.slowHandle->pFile == NULL) {
(void)printf("\nfailed to open slow log file:%s, reason:%s\n", name, strerror(errno));
@ -281,7 +283,10 @@ static void taosUnLockLogFile(TdFilePtr pFile) {
if (pFile == NULL) return;
if (tsLogObj.fileNum > 1) {
(void)taosUnLockFile(pFile);
int32_t code = taosUnLockFile(pFile);
if (code != 0) {
TAOS_UNUSED(printf("failed to unlock log file:%p, reason:%s\n", pFile, tstrerror(code)));
}
}
}
@ -310,7 +315,10 @@ static void taosKeepOldLog(char *oldName) {
char compressFileName[PATH_MAX + 20];
snprintf(compressFileName, PATH_MAX + 20, "%s.gz", oldName);
if (taosCompressFile(oldName, compressFileName) == 0) {
(void)taosRemoveFile(oldName);
int32_t code = taosRemoveFile(oldName);
if (code != 0) {
TAOS_UNUSED(printf("failed to remove file:%s, reason:%s\n", oldName, tstrerror(code)));
}
}
}
@ -331,7 +339,7 @@ static OldFileKeeper *taosOpenNewFile() {
char name[PATH_MAX + 20];
sprintf(name, "%s.%d", tsLogObj.logName, tsLogObj.flag);
(void)taosUmaskFile(0);
TAOS_UNUSED(taosUmaskFile(0));
TdFilePtr pFile = taosOpenFile(name, TD_FILE_CREATE | TD_FILE_WRITE | TD_FILE_TRUNC);
if (pFile == NULL) {
@ -341,8 +349,10 @@ static OldFileKeeper *taosOpenNewFile() {
return NULL;
}
(void)taosLockLogFile(pFile);
(void)taosLSeekFile(pFile, 0, SEEK_SET);
TAOS_UNUSED(taosLockLogFile(pFile));
if (taosLSeekFile(pFile, 0, SEEK_SET) < 0) {
uWarn("failed to seek file:%s, reason:%s", name, tstrerror(terrno));
}
TdFilePtr pOldFile = tsLogObj.logHandle->pFile;
tsLogObj.logHandle->pFile = pFile;
@ -384,7 +394,14 @@ static int32_t taosOpenNewLogFile() {
(void)taosThreadAttrSetDetachState(&attr, PTHREAD_CREATE_DETACHED);
OldFileKeeper *oldFileKeeper = taosOpenNewFile();
(void)taosThreadCreate(&thread, &attr, taosThreadToCloseOldFile, oldFileKeeper);
if (!oldFileKeeper) {
TAOS_UNUSED(taosThreadMutexUnlock(&tsLogObj.logMutex));
return terrno;
}
if (taosThreadCreate(&thread, &attr, taosThreadToCloseOldFile, oldFileKeeper) != 0) {
uError("failed to create thread to close old log file");
taosMemoryFreeClear(oldFileKeeper);
}
(void)taosThreadAttrDestroy(&attr);
}
@ -404,7 +421,7 @@ static void taosOpenNewSlowLogFile() {
for (int32_t i = 1; atomic_val_compare_exchange_32(&tsLogObj.slowHandle->lock, 0, 1) == 1; ++i) {
if (i % 1000 == 0) {
(void)sched_yield();
TAOS_UNUSED(sched_yield());
}
}
tsLogObj.slowHandle->lastDuration = LOG_MAX_WAIT_MSEC; // force write
@ -435,7 +452,10 @@ void taosResetLog() {
tsLogObj.lines = tsNumOfLogLines + 10;
if (tsLogObj.logHandle) {
(void)taosOpenNewLogFile();
int32_t code = taosOpenNewLogFile();
if(code != 0){
uError("failed to open new log file, reason:%s", tstrerror(code));
}
uInfo("==================================");
uInfo(" reset log file ");
}
@ -521,6 +541,7 @@ static void processLogFileName(const char* logName , int32_t maxFileNum){
}
static int32_t taosInitNormalLog(const char *logName, int32_t maxFileNum) {
int32_t code = 0;
#ifdef WINDOWS_STASH
/*
* always set maxFileNum to 1
@ -535,7 +556,7 @@ static int32_t taosInitNormalLog(const char *logName, int32_t maxFileNum) {
(void)sprintf(name, "%s.%d", tsLogObj.logName, tsLogObj.flag);
(void)taosThreadMutexInit(&tsLogObj.logMutex, NULL);
(void)taosUmaskFile(0);
TAOS_UNUSED(taosUmaskFile(0));
tsLogObj.logHandle = taosLogBuffNew(LOG_DEFAULT_BUF_SIZE);
if (tsLogObj.logHandle == NULL) return terrno;
@ -544,24 +565,41 @@ static int32_t taosInitNormalLog(const char *logName, int32_t maxFileNum) {
(void)printf("\nfailed to open log file:%s, reason:%s\n", name, strerror(errno));
return terrno;
}
(void)taosLockLogFile(tsLogObj.logHandle->pFile);
TAOS_UNUSED(taosLockLogFile(tsLogObj.logHandle->pFile));
// only an estimate for number of lines
int64_t filesize = 0;
if (taosFStatFile(tsLogObj.logHandle->pFile, &filesize, NULL) != 0) {
(void)printf("\nfailed to fstat log file:%s, reason:%s\n", name, strerror(errno));
taosUnLockLogFile(tsLogObj.logHandle->pFile);
return terrno;
}
tsLogObj.lines = (int32_t)(filesize / 60);
(void)taosLSeekFile(tsLogObj.logHandle->pFile, 0, SEEK_END);
if ((code = taosLSeekFile(tsLogObj.logHandle->pFile, 0, SEEK_END)) < 0) {
TAOS_UNUSED(printf("failed to seek to the end of log file:%s, reason:%s\n", name, tstrerror(code)));
taosUnLockLogFile(tsLogObj.logHandle->pFile);
return code;
}
(void)sprintf(name, "==================================================\n");
(void)taosWriteFile(tsLogObj.logHandle->pFile, name, (uint32_t)strlen(name));
if (taosWriteFile(tsLogObj.logHandle->pFile, name, (uint32_t)strlen(name)) <= 0) {
TAOS_UNUSED(printf("failed to write to log file:%s, reason:%s\n", name, tstrerror(terrno)));
taosUnLockLogFile(tsLogObj.logHandle->pFile);
return terrno;
}
(void)sprintf(name, " new log file \n");
(void)taosWriteFile(tsLogObj.logHandle->pFile, name, (uint32_t)strlen(name));
if (taosWriteFile(tsLogObj.logHandle->pFile, name, (uint32_t)strlen(name)) <= 0) {
TAOS_UNUSED(printf("failed to write to log file:%s, reason:%s\n", name, tstrerror(terrno)));
taosUnLockLogFile(tsLogObj.logHandle->pFile);
return terrno;
}
(void)sprintf(name, "==================================================\n");
(void)taosWriteFile(tsLogObj.logHandle->pFile, name, (uint32_t)strlen(name));
if (taosWriteFile(tsLogObj.logHandle->pFile, name, (uint32_t)strlen(name)) <= 0) {
TAOS_UNUSED(printf("failed to write to log file:%s, reason:%s\n", name, tstrerror(terrno)));
taosUnLockLogFile(tsLogObj.logHandle->pFile);
return terrno;
}
return 0;
}
@ -569,17 +607,17 @@ static int32_t taosInitNormalLog(const char *logName, int32_t maxFileNum) {
static void taosUpdateLogNums(ELogLevel level) {
switch (level) {
case DEBUG_ERROR:
(void)atomic_add_fetch_64(&tsNumOfErrorLogs, 1);
TAOS_UNUSED(atomic_add_fetch_64(&tsNumOfErrorLogs, 1));
break;
case DEBUG_INFO:
(void)atomic_add_fetch_64(&tsNumOfInfoLogs, 1);
TAOS_UNUSED(atomic_add_fetch_64(&tsNumOfInfoLogs, 1));
break;
case DEBUG_DEBUG:
(void)atomic_add_fetch_64(&tsNumOfDebugLogs, 1);
TAOS_UNUSED(atomic_add_fetch_64(&tsNumOfDebugLogs, 1));
break;
case DEBUG_DUMP:
case DEBUG_TRACE:
(void)atomic_add_fetch_64(&tsNumOfTraceLogs, 1);
TAOS_UNUSED(atomic_add_fetch_64(&tsNumOfTraceLogs, 1));
break;
default:
break;
@ -590,7 +628,7 @@ static inline int32_t taosBuildLogHead(char *buffer, const char *flags) {
struct tm Tm, *ptm;
struct timeval timeSecs;
(void)taosGetTimeOfDay(&timeSecs);
TAOS_UNUSED(taosGetTimeOfDay(&timeSecs));
time_t curTime = timeSecs.tv_sec;
ptm = taosLocalTime(&curTime, &Tm, NULL);
@ -603,15 +641,18 @@ static inline void taosPrintLogImp(ELogLevel level, int32_t dflag, const char *b
if ((dflag & DEBUG_FILE) && tsLogObj.logHandle && tsLogObj.logHandle->pFile != NULL && osLogSpaceSufficient()) {
taosUpdateLogNums(level);
if (tsAsyncLog) {
(void)taosPushLogBuffer(tsLogObj.logHandle, buffer, len);
TAOS_UNUSED(taosPushLogBuffer(tsLogObj.logHandle, buffer, len));
} else {
(void)taosWriteFile(tsLogObj.logHandle->pFile, buffer, len);
TAOS_UNUSED(taosWriteFile(tsLogObj.logHandle->pFile, buffer, len));
}
if (tsNumOfLogLines > 0) {
(void)atomic_add_fetch_32(&tsLogObj.lines, 1);
TAOS_UNUSED(atomic_add_fetch_32(&tsLogObj.lines, 1));
if ((tsLogObj.lines > tsNumOfLogLines) && (tsLogObj.openInProgress == 0)) {
(void)taosOpenNewLogFile();
int32_t code = taosOpenNewLogFile();
if (code != 0) {
uError("failed to open new log file, reason:%s", tstrerror(code));
}
}
}
}
@ -619,7 +660,9 @@ static inline void taosPrintLogImp(ELogLevel level, int32_t dflag, const char *b
if (dflag & DEBUG_SCREEN) {
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wunused-result"
(void)write(1, buffer, (uint32_t)len);
if (write(1, buffer, (uint32_t)len) < 0) {
TAOS_UNUSED(printf("failed to write log to screen, reason:%s\n", strerror(errno)));
}
#pragma GCC diagnostic pop
}
}
@ -690,12 +733,12 @@ void taosPrintSlowLog(const char *format, ...) {
buffer[len++] = '\n';
buffer[len] = 0;
(void)atomic_add_fetch_64(&tsNumOfSlowLogs, 1);
TAOS_UNUSED(atomic_add_fetch_64(&tsNumOfSlowLogs, 1));
if (tsAsyncLog) {
(void)taosPushLogBuffer(tsLogObj.slowHandle, buffer, len);
TAOS_UNUSED(taosPushLogBuffer(tsLogObj.slowHandle, buffer, len));
} else {
(void)taosWriteFile(tsLogObj.slowHandle->pFile, buffer, len);
TAOS_UNUSED(taosWriteFile(tsLogObj.slowHandle->pFile, buffer, len));
}
taosMemoryFree(buffer);
@ -714,7 +757,7 @@ void taosDumpData(unsigned char *msg, int32_t len) {
pos += 3;
if (c >= 16) {
temp[pos++] = '\n';
(void)taosWriteFile(tsLogObj.logHandle->pFile, temp, (uint32_t)pos);
TAOS_UNUSED((taosWriteFile(tsLogObj.logHandle->pFile, temp, (uint32_t)pos) <= 0));
c = 0;
pos = 0;
}
@ -722,7 +765,7 @@ void taosDumpData(unsigned char *msg, int32_t len) {
temp[pos++] = '\n';
(void)taosWriteFile(tsLogObj.logHandle->pFile, temp, (uint32_t)pos);
TAOS_UNUSED(taosWriteFile(tsLogObj.logHandle->pFile, temp, (uint32_t)pos));
}
static void taosCloseLogByFd(TdFilePtr pFile) {
@ -855,12 +898,12 @@ static void taosWriteLog(SLogBuff *pLogBuf) {
pLogBuf->lastDuration = 0;
if (start < end) {
(void)taosWriteFile(pLogBuf->pFile, LOG_BUF_BUFFER(pLogBuf) + start, pollSize);
TAOS_UNUSED(taosWriteFile(pLogBuf->pFile, LOG_BUF_BUFFER(pLogBuf) + start, pollSize));
} else {
int32_t tsize = LOG_BUF_SIZE(pLogBuf) - start;
(void)taosWriteFile(pLogBuf->pFile, LOG_BUF_BUFFER(pLogBuf) + start, tsize);
TAOS_UNUSED(taosWriteFile(pLogBuf->pFile, LOG_BUF_BUFFER(pLogBuf) + start, tsize));
(void)taosWriteFile(pLogBuf->pFile, LOG_BUF_BUFFER(pLogBuf), end);
TAOS_UNUSED(taosWriteFile(pLogBuf->pFile, LOG_BUF_BUFFER(pLogBuf), end));
}
dbgWN++;
@ -981,11 +1024,14 @@ void taosLogCrashInfo(char *nodeType, char *pMsg, int64_t msgLen, int signum, vo
goto _return;
}
(void)taosLockFile(pFile);
if (taosLockFile(pFile) < 0) {
taosPrintLog(flags, level, dflag, "failed to lock file:%s since %s", filepath, terrstr());
goto _return;
}
int64_t writeSize = taosWriteFile(pFile, &msgLen, sizeof(msgLen));
if (sizeof(msgLen) != writeSize) {
(void)taosUnLockFile(pFile);
TAOS_UNUSED(taosUnLockFile(pFile));
taosPrintLog(flags, level, dflag, "failed to write len to file:%s,%p wlen:%" PRId64 " tlen:%lu since %s",
filepath, pFile, writeSize, sizeof(msgLen), terrstr());
goto _return;
@ -993,13 +1039,13 @@ void taosLogCrashInfo(char *nodeType, char *pMsg, int64_t msgLen, int signum, vo
writeSize = taosWriteFile(pFile, pMsg, msgLen);
if (msgLen != writeSize) {
(void)taosUnLockFile(pFile);
TAOS_UNUSED(taosUnLockFile(pFile));
taosPrintLog(flags, level, dflag, "failed to write file:%s,%p wlen:%" PRId64 " tlen:%" PRId64 " since %s",
filepath, pFile, writeSize, msgLen, terrstr());
goto _return;
}
(void)taosUnLockFile(pFile);
TAOS_UNUSED(taosUnLockFile(pFile));
}
_return:
@ -1054,7 +1100,7 @@ void taosReadCrashInfo(char *filepath, char **pMsg, int64_t *pMsgLen, TdFilePtr
return;
}
(void)taosLockFile(pFile);
TAOS_UNUSED(taosLockFile(pFile));
} else {
pFile = *pFd;
}
@ -1093,10 +1139,10 @@ void taosReadCrashInfo(char *filepath, char **pMsg, int64_t *pMsgLen, TdFilePtr
_return:
if (truncateFile) {
(void)taosFtruncateFile(pFile, 0);
TAOS_UNUSED(taosFtruncateFile(pFile, 0));
}
(void)taosUnLockFile(pFile);
(void)taosCloseFile(&pFile);
TAOS_UNUSED(taosUnLockFile(pFile));
TAOS_UNUSED(taosCloseFile(&pFile));
taosMemoryFree(buf);
*pMsg = NULL;
@ -1106,11 +1152,11 @@ _return:
void taosReleaseCrashLogFile(TdFilePtr pFile, bool truncateFile) {
if (truncateFile) {
(void)taosFtruncateFile(pFile, 0);
TAOS_UNUSED(taosFtruncateFile(pFile, 0));
}
(void)taosUnLockFile(pFile);
(void)taosCloseFile(&pFile);
TAOS_UNUSED(taosUnLockFile(pFile));
TAOS_UNUSED(taosCloseFile(&pFile));
}
#ifdef NDEBUG

29
source/util/src/tqueue.c
View File

@ -224,7 +224,9 @@ int32_t taosWriteQitem(STaosQueue *queue, void *pItem) {
(void)taosThreadMutexUnlock(&queue->mutex);
if (queue->qset) {
(void)tsem_post(&queue->qset->sem);
if (tsem_post(&queue->qset->sem) != 0) {
uError("failed to post semaphore for queue set:%p", queue->qset);
}
}
return code;
}
@ -333,7 +335,10 @@ int32_t taosOpenQset(STaosQset **qset) {
}
(void)taosThreadMutexInit(&(*qset)->mutex, NULL);
(void)tsem_init(&(*qset)->sem, 0, 0);
if (tsem_init(&(*qset)->sem, 0, 0) != 0) {
taosMemoryFree(*qset);
return terrno;
}
uDebug("qset:%p is opened", qset);
return 0;
@ -354,7 +359,9 @@ void taosCloseQset(STaosQset *qset) {
(void)taosThreadMutexUnlock(&qset->mutex);
(void)taosThreadMutexDestroy(&qset->mutex);
(void)tsem_destroy(&qset->sem);
if (tsem_destroy(&qset->sem) != 0) {
uError("failed to destroy semaphore for qset:%p", qset);
}
taosMemoryFree(qset);
uDebug("qset:%p is closed", qset);
}
@ -364,7 +371,9 @@ void taosCloseQset(STaosQset *qset) {
// thread to exit.
void taosQsetThreadResume(STaosQset *qset) {
uDebug("qset:%p, it will exit", qset);
(void)tsem_post(&qset->sem);
if (tsem_post(&qset->sem) != 0) {
uError("failed to post semaphore for qset:%p", qset);
}
}
int32_t taosAddIntoQset(STaosQset *qset, STaosQueue *queue, void *ahandle) {
@ -432,7 +441,9 @@ int32_t taosReadQitemFromQset(STaosQset *qset, void **ppItem, SQueueInfo *qinfo)
STaosQnode *pNode = NULL;
int32_t code = 0;
(void)tsem_wait(&qset->sem);
if (tsem_wait(&qset->sem) != 0) {
uError("failed to wait semaphore for qset:%p", qset);
}
(void)taosThreadMutexLock(&qset->mutex);
@ -476,7 +487,9 @@ int32_t taosReadAllQitemsFromQset(STaosQset *qset, STaosQall *qall, SQueueInfo *
STaosQueue *queue;
int32_t code = 0;
(void)tsem_wait(&qset->sem);
if (tsem_wait(&qset->sem) != 0) {
uError("failed to wait semaphore for qset:%p", qset);
}
(void)taosThreadMutexLock(&qset->mutex);
for (int32_t i = 0; i < qset->numOfQueues; ++i) {
@ -510,7 +523,9 @@ int32_t taosReadAllQitemsFromQset(STaosQset *qset, STaosQall *qall, SQueueInfo *
(void)atomic_sub_fetch_32(&qset->numOfItems, qall->numOfItems);
for (int32_t j = 1; j < qall->numOfItems; ++j) {
(void)tsem_wait(&qset->sem);
if (tsem_wait(&qset->sem) != 0) {
uError("failed to wait semaphore for qset:%p", qset);
}
}
}

3
source/util/src/tworker.c
View File

@ -440,7 +440,8 @@ STaosQueue *tWWorkerAllocQueue(SWWorkerPool *pool, void *ahandle, FItems fp) {
TdThreadAttr thAttr;
(void)taosThreadAttrInit(&thAttr);
(void)taosThreadAttrSetDetachState(&thAttr, PTHREAD_CREATE_JOINABLE);
if (taosThreadCreate(&worker->thread, &thAttr, (ThreadFp)tWWorkerThreadFp, worker) != 0) goto _OVER;
code = taosThreadCreate(&worker->thread, &thAttr, (ThreadFp)tWWorkerThreadFp, worker);
if ((code)) goto _OVER;
uInfo("worker:%s:%d is launched, max:%d", pool->name, worker->id, pool->max);
pool->nextId = (pool->nextId + 1) % pool->max;

87
tests/docs-examples-test/rust.sh Normal file
View File

@ -0,0 +1,87 @@
#!/bin/bash
set -e
check_transactions() {
for i in {1..30}
do
output=$(taos -s "show transactions;")
if [[ $output == *"Query OK, 0 row(s)"* ]]; then
echo "Success: No transactions are in progress."
return 0
fi
sleep 1
done
echo "Error: Transactions are still in progress after 30 attempts."
return 1
}
reset_cache() {
response=$(curl --location -uroot:taosdata 'http://127.0.0.1:6041/rest/sql' --data 'reset query cache')
if [[ $response == \{\"code\":0* ]]; then
echo "Success: Query cache reset successfully."
else
echo "Error: Failed to reset query cache. Response: $response"
return 1
fi
}
taosd >>/dev/null 2>&1 &
taosadapter >>/dev/null 2>&1 &
sleep 5
cd ../../docs/examples/rust/nativeexample
cargo run --example connect
cargo run --example createdb
cargo run --example insert
cargo run --example query
taos -s "drop database if exists power"
check_transactions || exit 1
reset_cache || exit 1
cargo run --example schemaless
taos -s "drop database if exists power"
check_transactions || exit 1
reset_cache || exit 1
cargo run --example stmt
cargo run --example tmq
cd ../restexample
cargo run --example connect
cargo run --example createdb
cargo run --example insert
cargo run --example query
taos -s "drop database if exists power"
check_transactions || exit 1
reset_cache || exit 1
taos -s "create database if not exists power"
cargo run --example schemaless
taos -s "drop database if exists power"
check_transactions || exit 1
reset_cache || exit 1
cargo run --example stmt
cargo run --example tmq

2
tests/parallel_test/cases.task
View File

@ -280,6 +280,7 @@
,,y,system-test,./pytest.sh python3 ./test.py -f 7-tmq/dataFromTsdbNWal.py
,,y,system-test,./pytest.sh python3 ./test.py -f 7-tmq/dataFromTsdbNWal-multiCtb.py
,,y,system-test,./pytest.sh python3 ./test.py -f 7-tmq/tmq_taosx.py
,,y,system-test,./pytest.sh python3 ./test.py -f 7-tmq/tmq_ts-5473.py
,,y,system-test,./pytest.sh python3 ./test.py -f 7-tmq/tmq_ts4563.py
,,y,system-test,./pytest.sh python3 ./test.py -f 7-tmq/tmq_replay.py
,,y,system-test,./pytest.sh python3 ./test.py -f 7-tmq/tmqSeekAndCommit.py
@ -1556,5 +1557,6 @@
#,,n,docs-examples-test,bash node.sh
,,n,docs-examples-test,bash csharp.sh
,,n,docs-examples-test,bash jdbc.sh
,,n,docs-examples-test,bash rust.sh
,,n,docs-examples-test,bash go.sh
,,n,docs-examples-test,bash test_R.sh

7
tests/pytest/util/csv.py
View File

@ -1,5 +1,6 @@
import csv
import os
import platform
class TDCsv:
def __init__(self):
@ -25,7 +26,11 @@ class TDCsv:
@property
def file(self):
if self.file_name and self.file_path:
return os.path.join(self.file_path, self.file_name)
print(f"self.file_path {self.file_path}, self.file_name {self.file_name}")
csv_file = os.path.join(self.file_path, self.file_name)
if platform.system().lower() == 'windows':
csv_file = csv_file.replace("\\", "/")
return csv_file
return None

42
tests/pytest/util/sql.py
View File

@ -24,26 +24,48 @@ from util.log import *
from util.constant import *
import ctypes
import random
# from datetime import timezone
import datetime
import time
from tzlocal import get_localzone
def _parse_ns_timestamp(timestr):
dt_obj = datetime.datetime.strptime(timestr[:len(timestr)-3], "%Y-%m-%d %H:%M:%S.%f")
tz = int(int((dt_obj-datetime.datetime.fromtimestamp(0,dt_obj.tzinfo)).total_seconds())*1e9) + int(dt_obj.microsecond * 1000) + int(timestr[-3:])
return tz
def _parse_datetime(timestr):
# defined timestr formats
formats = [
'%Y-%m-%d %H:%M:%S.%f%z', # 包含微秒和时区偏移
'%Y-%m-%d %H:%M:%S%z', # 不包含微秒但包含时区偏移
'%Y-%m-%d %H:%M:%S.%f', # 包含微秒
'%Y-%m-%d %H:%M:%S' # 不包含微秒
]
for fmt in formats:
try:
return datetime.datetime.strptime(timestr, '%Y-%m-%d %H:%M:%S.%f')
# try to parse the string with the current format
dt = datetime.datetime.strptime(timestr, fmt)
# 如果字符串包含时区信息，则返回 aware 对象
# if sting contains timezone info, return aware object
if dt.tzinfo is not None:
return dt
else:
# if sting does not contain timezone info, assume it is in local timezone
# get local timezone
local_timezone = get_localzone()
# print("Timezone:", local_timezone)
return dt.replace(tzinfo=local_timezone)
except ValueError:
pass
try:
return datetime.datetime.strptime(timestr, '%Y-%m-%d %H:%M:%S')
except ValueError:
pass
continue # if the current format does not match, try the next format
# 如果所有格式都不匹配，返回 None
# if none of the formats match, return
raise ValueError(f"input format does not match. correct formats include: '{', '.join(formats)}'")
class TDSql:
def __init__(self):
self.queryRows = 0
self.queryCols = 0
@ -408,6 +430,7 @@ class TDSql:
if self.queryResult[row][col] != data:
if self.cursor.istype(col, "TIMESTAMP"):
# tdLog.debug(f"self.queryResult[row][col]:{self.queryResult[row][col]}, data:{data},len(data):{len(data)}, isinstance(data,str) :{isinstance(data,str)}")
# suppose user want to check nanosecond timestamp if a longer data passed``
if isinstance(data,str) :
if (len(data) >= 28):
@ -419,8 +442,9 @@ class TDSql:
args = (caller.filename, caller.lineno, self.sql, row, col, self.queryResult[row][col], data)
tdLog.exit("%s(%d) failed: sql:%s row:%d col:%d data:%s != expect:%s" % args)
else:
# tdLog.info(f"datetime.timezone.utc:{datetime.timezone.utc},data:{data},_parse_datetime(data).astimezone(datetime.timezone.utc):{_parse_datetime(data).astimezone(datetime.timezone.utc)}")
if self.queryResult[row][col].astimezone(datetime.timezone.utc) == _parse_datetime(data).astimezone(datetime.timezone.utc):
# tdLog.info(f"sql:{self.sql}, row:{row} col:{col} data:{self.queryResult[row][col]} == expect:{data}")
# tdLog.info(f"sql:{self.sql}, row:{row} col:{col} data:{self.queryResult[row][col].astimezone(datetime.timezone.utc)} == expect:{_parse_datetime(data).astimezone(datetime.timezone.utc)}")
if(show):
tdLog.info("check successfully")
else:

Some files were not shown because too many files have changed in this diff Show More