xxq250
/
homework-jianmu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/3.0' into enh/opt-transport

This commit is contained in:
yihaoDeng 2024-09-26 21:40:48 +08:00
parent 15b7b116b7 54bc8c3e22
commit ce052c9c8a
159 changed files with 2940 additions and 1942 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
docs/examples/rust/restexample/examples/tmq.rs
View File

@ -83,7 +83,6 @@ async fn main() -> anyhow::Result<()> {
eprintln!("Failed to execute insert: {:?}", e);
}
tokio::time::sleep(Duration::from_millis(10)).await;
println!("Succed to execute insert 1 row");
}
});
});

2
docs/zh/05-basic/03-query.md
View File

@ -181,7 +181,7 @@ INTERVAL(interval_val [, interval_offset])
- FILL：用于指定窗口区间数据缺失的情况下，数据的填充模式。
对于时间窗口，interval_val 和 sliding_val 都表示时间段， 语法上支持三种方式。例如：
1. INTERVAL(1s, 500a) SLIDING(1s)，带时间单位的形式，其中的时间单位是单字符表示， 分别为： a （毫秒）， b （纳秒）， d （天）， h （小时）， m （分钟）， n （月）， s （秒）， u （微妙）， w （周）， y （年）；
1. INTERVAL(1s, 500a) SLIDING(1s)，带时间单位的形式，其中的时间单位是单字符表示， 分别为： a （毫秒）， b （纳秒）， d （天）， h （小时）， m （分钟）， n （月）， s （秒）， u （微秒）， w （周）， y （年）；
2. INTERVAL(1000, 500) SLIDING(1000)，不带时间单位的形式，将使用查询库的时间精度作为默认时间单位，当存在多个库时默认采用精度更高的库；
3. INTERVAL('1s', '500a') SLIDING('1s')，带时间单位的字符串形式，字符串内部不能有任何空格等其它字符。

44
docs/zh/06-advanced/05-data-in/08-kafka.md
View File

@ -44,8 +44,50 @@ TDengine 可以高效地从 Kafka 读取数据并将其写入 TDengine，以实
如果服务端开启了 SASL 认证机制，此处需要启用 SASL 并配置相关内容，目前支持 PLAIN/SCRAM-SHA-256/GSSAPI 三种认证机制，请按实际情况进行选择。
#### 4.1. PLAIN 认证
选择 `PLAIN` 认证机制，输入用户名和密码：
![kafka-04-sasl-plain.png](./kafka-04-sasl-plain.png)
#### 4.1. SCRAM(SCRAM-SHA-256) 认证
选择 `SCRAM-SHA-256` 认证机制，输入用户名和密码：
![kafka-04.png](./kafka-04.png)
#### 4.3. GSSAPI 认证
选择 `GSSAPI` ，将通过 [RDkafka 客户端](https://github.com/confluentinc/librdkafka) 调用 GSSAPI 应用 Kerberos 认证机制：
![kafka-04-sasl-gssapi.png](./kafka-04-sasl-gssapi.png)
需要输入的信息有：
- Kerberos 服务名，一般是 `kafka`；
- Kerberos 认证主体，即认证用户名，例如 `kafkaclient`；
- Kerberos 初始化命令（可选，一般不用填写）；
- Kerberos 密钥表，需提供文件并上传；
以上信息均需由 Kafka 服务管理者提供。
除此之外，在服务器上需要配置 [Kerberos](https://web.mit.edu/kerberos/) 认证服务。在 Ubuntu 下使用 `apt install krb5-user` ；在 CentOS 下，使用 `yum install krb5-workstation`；即可。
配置完成后，可以使用 [kcat](https://github.com/edenhill/kcat) 工具进行 Kafka 主题消费验证：
```bash
kcat <topic> \
-b <kafka-server:port> \
-G kcat \
-X security.protocol=SASL_PLAINTEXT \
-X sasl.mechanism=GSSAPI \
-X sasl.kerberos.keytab=</path/to/kafkaclient.keytab> \
-X sasl.kerberos.principal=<kafkaclient> \
-X sasl.kerberos.service.name=kafka
```
如果出现错误：“Server xxxx not found in kerberos database”，则需要配置 Kafka 节点对应的域名并在 Kerberos 客户端配置文件 `/etc/krb5.conf` 中配置反向域名解析 `rdns = true`。
### 5. 配置 SSL 证书
如果服务端开启了 SSL 加密认证,此处需要启用 SSL 并配置相关内容。
@ -160,4 +202,4 @@ json 数据支持 JSONObject 或者 JSONArray，使用 json 解析器可以解
### 9. 创建完成
点击 **提交** 按钮，完成创建 Kafka 到 TDengine 的数据同步任务，回到**数据源列表**页面可查看任务执行情况。
点击 **提交** 按钮，完成创建 Kafka 到 TDengine 的数据同步任务，回到**数据源列表**页面可查看任务执行情况。

BIN
docs/zh/06-advanced/05-data-in/kafka-04-sasl-gssapi.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 43 KiB

BIN
docs/zh/06-advanced/05-data-in/kafka-04-sasl-plain.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 20 KiB

3
docs/zh/07-develop/index.md
View File

@ -14,7 +14,8 @@ description: 让开发者能够快速上手的指南
7. 在很多场景下（如车辆管理），应用需要获取每个数据采集点的最新状态，那么建议你采用 TDengine 的 Cache 功能，而不用单独部署 Redis 等缓存软件。
8. 如果你发现 TDengine 的函数无法满足你的要求，那么你可以使用用户自定义函数（UDF）来解决问题。
本部分内容就是按照上述顺序组织的。为便于理解，TDengine 为每个功能和每个支持的编程语言都提供了示例代码。如果你希望深入了解 SQL 的使用，需要查看[SQL 手册](../reference/taos-sql/)。如果想更深入地了解各连接器的使用，请阅读[连接器参考指南](../reference/connector/)。如果还希望想将 TDengine 与第三方系统集成起来，比如 Grafana, 请参考[第三方工具](../third-party/)。
本部分内容就是按照上述顺序组织的。为便于理解，TDengine 为每个功能和每个支持的编程语言都提供了示例代码，位于 [示例代码](https://github.com/taosdata/TDengine/tree/main/docs/examples)。所有示例代码都会有 CI 保证正确性，脚本位于 [示例代码 CI](https://github.com/taosdata/TDengine/tree/main/tests/docs-examples-test)。
如果你希望深入了解 SQL 的使用，需要查看[SQL 手册](../reference/taos-sql/)。如果想更深入地了解各连接器的使用，请阅读[连接器参考指南](../reference/connector/)。如果还希望想将 TDengine 与第三方系统集成起来，比如 Grafana, 请参考[第三方工具](../third-party/)。
如果在开发过程中遇到任何问题，请点击每个页面下方的["反馈问题"](https://github.com/taosdata/TDengine/issues/new/choose), 在 GitHub 上直接递交 Issue。

91
docs/zh/14-reference/01-components/04-taosx.md
View File

@ -421,7 +421,6 @@ taosX 会将监控指标上报给 taosKeeper，这些监控指标会被 taosKeep
| write_raw_fails | 本次运行写入 raw meta 失败的次数 |
| success_blocks | 本次写入成功的数据块数 |
### taosX 其他数据源 任务
这些数据源包括： InfluxDB，OpenTSDB，OPC UA，OPC DA，PI，CSV，MQTT，AVEVA Historian 和 Kafka。
@ -452,3 +451,93 @@ taosX 会将监控指标上报给 taosKeeper，这些监控指标会被 taosKeep
| written_blocks | 本次运行此任务写人成功的 raw block 数 |
| failed_blocks | 本次运行此任务写入失败的 raw block 数 |
## taosX 数据解析插件
接入 kafka / mqtt 消息中间件时，需要对原始数据进行解析，如果使用 json/regex 等模式解析器无法满足解析需求，同时 UDT(自定义解析脚本) 也无法满足性能要求时，可以自定义数据解析插件。
### 插件概述
taosX Parser 插件是一个要求用 C/Rust 语言开发的 C ABI 兼容动态库，该动态库要实现约定的 API 并编译为在 taosX 所在运行环境中能够正确运行的动态库，然后复制到约定位置由 taosX 在运行时加载，并在处理数据的 Parsing 阶段调用。
### 插件部署
完成插件开发后，编译环境需要和目标运行环境兼容，将编译好的插件动态库复制到插件目录下，taosX 启动后，系统首次使用插件时初始化加载插件。可以在 explorer 的 kafka 或者 mqtt 数据接入配置页面中，检查是否加载成功。如下图，如果加载成功，则在解析器选择列表中展示出来。
![插件示例](./plugin-01.png)
插件目录在 `taosx.toml` 配置文件中复用 plugins 配置，追加`/parsers`作为插件安装路径，默认值在 UNIX 环境下为 `/usr/local/taos/plugins/parsers`，在 Windows 下为 `C:\TDengine\plugins\parsers`。
### 插件 api 说明
#### 1. 获取插件名称
获取插件名，用于前端显示。
**函数签名**：const char* parser_name()
**返回值**：字符串。
#### 2. 获取插件版本
插件版本，方便问题定位。
**函数签名**：const char* parser_version()
**返回值**：字符串。
#### 3. 配置解析器
将一个字符串参数解析为一个配置对象，仅插件内部使用。
**函数签名**：parser_resp_t parser_new(char* ctx, uint32_t len);
char* ctx: 用户自定义配置字符串。
uint32_t len: 该字符串的二进制长度（不含 `\0`）。
**返回值**：
``` c
struct parser_resp_t {
int e; // 0 if success.
void* p; // Success if contains.
}
```
当创建对象失败时，e 不为 0。
当创建成功时，e = 0，p 为解析器对象。
#### 4. 解析数据
**函数签名**：
对输入 payload 进行解析，返回结果为 JSON 格式 [u8] 。返回的 JSON 将使用默认的 JSON 解析器进行完全解码（展开根数组和所有的对象）。
``` c
const char* parser_mutate(
void* parser,
const uint8_t* in_ptr, uint32_t in_len,
const void* uint8_t* out_ptr, uint32_t* out_len
);
```
`void* parser`: parser_new 生成的对象指针;
`const uint8_t* in_ptr`：输入 Payload 的指针;
`uint32_t in_len`: 输入 Payload 的 bytes 长度（不含 `\0`）;
`const void* uint8_t* out_ptr`:输出 JSON 字符串的指针（不含 \0）。当 out_ptr 指向为空时，表示输出为空。
`uint32_t * out_len`：输出 JSON 字符串长度。
**返回值**： 当调用成功时，返回值为 NULL。
#### 5. 释放解析器
释放解析器对象内存。
**函数签名**： void parser_free(void* parser);
void* parser: parser_new 生成的对象指针。

BIN
docs/zh/14-reference/01-components/plugin-01.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 27 KiB

1
docs/zh/14-reference/03-taos-sql/02-database.md
View File

@ -80,7 +80,6 @@ database_option: {
```sql
create database if not exists db vgroups 10 buffer 10
```
以上示例创建了一个有 10 个 vgroup 名为 db 的数据库， 其中每个 vnode 分配 10MB 的写入缓存

2
docs/zh/14-reference/03-taos-sql/10-function.md
View File

@ -1214,7 +1214,7 @@ TO_TIMESTAMP(ts_str_literal, format_str_literal)
- 如果没有指定完整的时间，那么默认时间值为指定或默认时区的 `1970-01-01 00:00:00`, 未指定部分使用该默认值中的对应部分. 暂不支持只指定年日而不指定月日的格式, 如'yyyy-mm-DDD', 支持'yyyy-mm-DD'.
- 如果格式串中有`AM`, `PM`等, 那么小时必须是12小时制, 范围必须是01-12.
- `to_timestamp`转换具有一定的容错机制, 在格式串和时间戳串不完全对应时, 有时也可转换, 如: `to_timestamp('200101/2', 'yyyyMM1/dd')`, 格式串中多出来的1会被丢弃. 格式串与时间戳串中多余的空格字符(空格, tab等)也会被 自动忽略. 如`to_timestamp(' 23 年 - 1 月 - 01 日 ', 'yy 年-MM月-dd日')` 可以被成功转换. 虽然`MM`等字段需要两个数字对应(只有一位时前面补0), 在`to_timestamp`时, 一个数字也可以成功转换.
- 输出时间戳的精度与查询表的精度相同, 若查询未指定表, 则输出精度为毫秒. 如`select to_timestamp('2023-08-1 10:10:10.123456789', 'yyyy-mm-dd hh:mi:ss.ns')`的输出将会把微妙和纳秒进行截断. 如果指定一张纳秒表, 那么就不会发生截断, 如`select to_timestamp('2023-08-1 10:10:10.123456789', 'yyyy-mm-dd hh:mi:ss.ns') from db_ns.table_ns limit 1`.
- 输出时间戳的精度与查询表的精度相同, 若查询未指定表, 则输出精度为毫秒. 如`select to_timestamp('2023-08-1 10:10:10.123456789', 'yyyy-mm-dd hh:mi:ss.ns')`的输出将会把微秒和纳秒进行截断. 如果指定一张纳秒表, 那么就不会发生截断, 如`select to_timestamp('2023-08-1 10:10:10.123456789', 'yyyy-mm-dd hh:mi:ss.ns') from db_ns.table_ns limit 1`.
### 时间和日期函数

8
docs/zh/14-reference/03-taos-sql/16-operators.md
View File

@ -41,9 +41,11 @@ TDengine 支持 `UNION ALL` 和 `UNION` 操作符。UNION ALL 将查询返回的
| 5 | IS [NOT] NULL | 所有类型 | 是否为空值 |
| 6 | [NOT] BETWEEN AND | 除 BOOL、BLOB、MEDIUMBLOB、JSON 和 GEOMETRY 外的所有类型 | 闭区间比较 |
| 7 | IN | 除 BLOB、MEDIUMBLOB 和 JSON 外的所有类型，且不可以为表的时间戳主键列 | 与列表内的任意值相等 |
| 8 | LIKE | BINARY、NCHAR 和 VARCHAR | 通配符匹配 |
| 9 | MATCH, NMATCH | BINARY、NCHAR 和 VARCHAR | 正则表达式匹配 |
| 10 | CONTAINS | JSON | JSON 中是否存在某键 |
| 8 | NOT IN | 除 BLOB、MEDIUMBLOB 和 JSON 外的所有类型，且不可以为表的时间戳主键列 | 与列表内的任意值都不相等 |
| 9 | LIKE | BINARY、NCHAR 和 VARCHAR | 通配符匹配所指定的模式串 |
| 10 | NOT LIKE | BINARY、NCHAR 和 VARCHAR | 通配符不匹配所指定的模式串 |
| 11 | MATCH, NMATCH | BINARY、NCHAR 和 VARCHAR | 正则表达式匹配 |
| 12 | CONTAINS | JSON | JSON 中是否存在某键 |
LIKE 条件使用通配符字符串进行匹配检查，规则如下：

2
docs/zh/14-reference/03-taos-sql/28-index.md
View File

@ -28,7 +28,7 @@ TSMA只能基于超级表和普通表创建, 不能基于子表创建.
由于TSMA输出为一张超级表, 因此输出表的行长度受最大行长度限制, 不同函数的`中间结果`大小各异, 一般都大于原始数据大小, 若输出表的行长度大于最大行长度限制, 将会报`Row length exceeds max length`错误. 此时需要减少函数个数或者将常用的函数进行分组拆分到多个TSMA中.
窗口大小的限制为[1m ~ 1y/12n]. INTERVAL 的单位与查询中INTERVAL子句相同, 如 a (毫秒), b (纳秒), h (小时), m (分钟), s (秒), u (微妙), d (天), w(周), n(月), y(年).
窗口大小的限制为[1m ~ 1y/12n]. INTERVAL 的单位与查询中INTERVAL子句相同, 如 a (毫秒), b (纳秒), h (小时), m (分钟), s (秒), u (微秒), d (天), w(周), n(月), y(年).
TSMA为库内对象, 但名字全局唯一. 集群内一共可创建TSMA个数受参数`maxTsmaNum`限制, 参数默认值为3, 范围: [0-3]. 注意, 由于TSMA后台计算使用流计算, 因此每创建一条TSMA, 将会创建一条流, 因此能够创建的TSMA条数也受当前已经存在的流条数和最大可创建流条数限制.

1
include/common/cos.h
View File

@ -32,7 +32,6 @@ extern int32_t tsS3PageCacheSize;
extern int32_t tsS3UploadDelaySec;
int32_t s3Init();
void s3CleanUp();
int32_t s3CheckCfg();
int32_t s3PutObjectFromFile(const char *file, const char *object);
int32_t s3PutObjectFromFile2(const char *file, const char *object, int8_t withcp);

2
include/common/tdataformat.h
View File

@ -115,7 +115,7 @@ typedef struct {
} SValueColumnCompressInfo;
int32_t tValueColumnInit(SValueColumn *valCol);
int32_t tValueColumnDestroy(SValueColumn *valCol);
void tValueColumnDestroy(SValueColumn *valCol);
void tValueColumnClear(SValueColumn *valCol);
int32_t tValueColumnAppend(SValueColumn *valCol, const SValue *value);
int32_t tValueColumnUpdate(SValueColumn *valCol, int32_t idx, const SValue *value);

13
include/util/tcompression.h
View File

@ -72,8 +72,8 @@ extern "C" {
#ifdef TD_TSZ
extern bool lossyFloat;
extern bool lossyDouble;
int32_t tsCompressInit(char *lossyColumns, float fPrecision, double dPrecision, uint32_t maxIntervals,
uint32_t intervals, int32_t ifAdtFse, const char *compressor);
void tsCompressInit(char *lossyColumns, float fPrecision, double dPrecision, uint32_t maxIntervals, uint32_t intervals,
int32_t ifAdtFse, const char *compressor);
void tsCompressExit();
@ -153,11 +153,10 @@ int32_t tsDecompressBigint(void *pIn, int32_t nIn, int32_t nEle, void *pOut, int
int32_t getWordLength(char type);
int32_t tsDecompressIntImpl_Hw(const char *const input, const int32_t nelements, char *const output, const char type);
int32_t tsDecompressFloatImplAvx512(const char *const input, const int32_t nelements, char *const output);
int32_t tsDecompressFloatImplAvx2(const char *const input, const int32_t nelements, char *const output);
int32_t tsDecompressTimestampAvx512(const char *const input, const int32_t nelements, char *const output,
bool bigEndian);
int32_t tsDecompressTimestampAvx2(const char *const input, const int32_t nelements, char *const output, bool bigEndian);
void tsDecompressFloatImplAvx512(const char *const input, const int32_t nelements, char *const output);
void tsDecompressFloatImplAvx2(const char *const input, const int32_t nelements, char *const output);
void tsDecompressTimestampAvx512(const char *const input, const int32_t nelements, char *const output, bool bigEndian);
void tsDecompressTimestampAvx2(const char *const input, const int32_t nelements, char *const output, bool bigEndian);
/*************************************************************************
* REGULAR COMPRESSION 2

81
source/client/inc/clientInt.h
View File

@ -26,10 +26,10 @@ extern "C" {
#include "query.h"
#include "taos.h"
#include "tcommon.h"
#include "tmisce.h"
#include "tdef.h"
#include "thash.h"
#include "tlist.h"
#include "tmisce.h"
#include "tmsg.h"
#include "tmsgtype.h"
#include "trpc.h"
@ -86,7 +86,7 @@ typedef struct {
int8_t threadStop;
int8_t quitByKill;
TdThread thread;
TdThreadMutex lock; // used when app init and cleanup
TdThreadMutex lock; // used when app init and cleanup
SHashObj* appSummary;
SHashObj* appHbHash; // key: clusterId
SArray* appHbMgrs; // SArray<SAppHbMgr*> one for each cluster
@ -95,11 +95,11 @@ typedef struct {
} SClientHbMgr;
typedef struct SQueryExecMetric {
int64_t start; // start timestamp, us
int64_t ctgStart; // start to parse, us
int64_t execStart; // start to parse, us
int64_t start; // start timestamp, us
int64_t ctgStart; // start to parse, us
int64_t execStart; // start to parse, us
int64_t parseCostUs;
int64_t parseCostUs;
int64_t ctgCostUs;
int64_t analyseCostUs;
int64_t planCostUs;
@ -193,7 +193,7 @@ typedef struct SReqResultInfo {
char** convertBuf;
TAOS_ROW row;
SResultColumn* pCol;
uint64_t numOfRows; // from int32_t change to int64_t
uint64_t numOfRows; // from int32_t change to int64_t
uint64_t totalRows;
uint64_t current;
bool localResultFetched;
@ -319,12 +319,14 @@ void syncCatalogFn(SMetaData* pResult, void* param, int32_t code);
TAOS_RES* taosQueryImpl(TAOS* taos, const char* sql, bool validateOnly, int8_t source);
TAOS_RES* taosQueryImplWithReqid(TAOS* taos, const char* sql, bool validateOnly, int64_t reqid);
void taosAsyncQueryImpl(uint64_t connId, const char* sql, __taos_async_fn_t fp, void* param, bool validateOnly, int8_t source);
void taosAsyncQueryImpl(uint64_t connId, const char* sql, __taos_async_fn_t fp, void* param, bool validateOnly,
int8_t source);
void taosAsyncQueryImplWithReqid(uint64_t connId, const char* sql, __taos_async_fn_t fp, void* param, bool validateOnly,
int64_t reqid);
void taosAsyncFetchImpl(SRequestObj *pRequest, __taos_async_fn_t fp, void *param);
int32_t clientParseSql(void* param, const char* dbName, const char* sql, bool parseOnly, const char* effectiveUser, SParseSqlRes* pRes);
void syncQueryFn(void* param, void* res, int32_t code);
void taosAsyncFetchImpl(SRequestObj* pRequest, __taos_async_fn_t fp, void* param);
int32_t clientParseSql(void* param, const char* dbName, const char* sql, bool parseOnly, const char* effectiveUser,
SParseSqlRes* pRes);
void syncQueryFn(void* param, void* res, int32_t code);
int32_t getVersion1BlockMetaSize(const char* p, int32_t numOfCols);
@ -333,7 +335,7 @@ static FORCE_INLINE SReqResultInfo* tmqGetCurResInfo(TAOS_RES* res) {
return (SReqResultInfo*)&msg->common.resInfo;
}
int32_t tmqGetNextResInfo(TAOS_RES* res, bool convertUcs4, SReqResultInfo** pResInfo);
int32_t tmqGetNextResInfo(TAOS_RES* res, bool convertUcs4, SReqResultInfo** pResInfo);
static FORCE_INLINE SReqResultInfo* tscGetCurResInfo(TAOS_RES* res) {
if (TD_RES_QUERY(res)) return &(((SRequestObj*)res)->body.resInfo);
return tmqGetCurResInfo(res);
@ -349,8 +351,8 @@ __async_send_cb_fn_t getMsgRspHandle(int32_t msgType);
SMsgSendInfo* buildMsgInfoImpl(SRequestObj* pReqObj);
int32_t createTscObj(const char *user, const char *auth, const char *db, int32_t connType, SAppInstInfo *pAppInfo,
STscObj **p);
int32_t createTscObj(const char* user, const char* auth, const char* db, int32_t connType, SAppInstInfo* pAppInfo,
STscObj** p);
void destroyTscObj(void* pObj);
STscObj* acquireTscObj(int64_t rid);
void releaseTscObj(int64_t rid);
@ -358,7 +360,7 @@ void destroyAppInst(void* pAppInfo);
uint64_t generateRequestId();
int32_t createRequest(uint64_t connId, int32_t type, int64_t reqid, SRequestObj **pRequest);
int32_t createRequest(uint64_t connId, int32_t type, int64_t reqid, SRequestObj** pRequest);
void destroyRequest(SRequestObj* pRequest);
SRequestObj* acquireRequest(int64_t rid);
int32_t releaseRequest(int64_t rid);
@ -372,9 +374,9 @@ void resetConnectDB(STscObj* pTscObj);
int taos_options_imp(TSDB_OPTION option, const char* str);
int32_t openTransporter(const char* user, const char* auth, int32_t numOfThreads, void **pDnodeConn);
void tscStopCrashReport();
void cleanupAppInfo();
int32_t openTransporter(const char* user, const char* auth, int32_t numOfThreads, void** pDnodeConn);
void tscStopCrashReport();
void cleanupAppInfo();
typedef struct AsyncArg {
SRpcMsg msg;
@ -402,17 +404,17 @@ int32_t hbMgrInit();
void hbMgrCleanUp();
// cluster level
int32_t appHbMgrInit(SAppInstInfo *pAppInstInfo, char *key, SAppHbMgr **pAppHbMgr);
void appHbMgrCleanup(void);
void hbRemoveAppHbMrg(SAppHbMgr** pAppHbMgr);
void destroyAllRequests(SHashObj* pRequests);
void stopAllRequests(SHashObj* pRequests);
int32_t appHbMgrInit(SAppInstInfo* pAppInstInfo, char* key, SAppHbMgr** pAppHbMgr);
void appHbMgrCleanup(void);
void hbRemoveAppHbMrg(SAppHbMgr** pAppHbMgr);
void destroyAllRequests(SHashObj* pRequests);
void stopAllRequests(SHashObj* pRequests);
//SAppInstInfo* getAppInstInfo(const char* clusterKey);
// SAppInstInfo* getAppInstInfo(const char* clusterKey);
// conn level
int32_t hbRegisterConn(SAppHbMgr* pAppHbMgr, int64_t tscRefId, int64_t clusterId, int8_t connType);
void hbDeregisterConn(STscObj* pTscObj, SClientHbKey connKey);
void hbDeregisterConn(STscObj* pTscObj, SClientHbKey connKey);
typedef struct SSqlCallbackWrapper {
SParseContext* pParseCtx;
@ -421,9 +423,9 @@ typedef struct SSqlCallbackWrapper {
void* pPlanInfo;
} SSqlCallbackWrapper;
void setQueryRequest(int64_t rId);
SRequestObj* launchQueryImpl(SRequestObj* pRequest, SQuery* pQuery, bool keepQuery, void** res);
int32_t scheduleQuery(SRequestObj* pRequest, SQueryPlan* pDag, SArray* pNodeList);
void setQueryRequest(int64_t rId);
void launchQueryImpl(SRequestObj* pRequest, SQuery* pQuery, bool keepQuery, void** res);
int32_t scheduleQuery(SRequestObj* pRequest, SQueryPlan* pDag, SArray* pNodeList);
void launchAsyncQuery(SRequestObj* pRequest, SQuery* pQuery, SMetaData* pResultMeta, SSqlCallbackWrapper* pWrapper);
int32_t refreshMeta(STscObj* pTscObj, SRequestObj* pRequest);
int32_t updateQnodeList(SAppInstInfo* pInfo, SArray* pNodeList);
@ -431,20 +433,21 @@ void doAsyncQuery(SRequestObj* pRequest, bool forceUpdateMeta);
int32_t removeMeta(STscObj* pTscObj, SArray* tbList, bool isView);
int32_t handleAlterTbExecRes(void* res, struct SCatalog* pCatalog);
int32_t handleCreateTbExecRes(void* res, SCatalog* pCatalog);
int32_t qnodeRequired(SRequestObj* pRequest, bool *required);
int32_t qnodeRequired(SRequestObj* pRequest, bool* required);
void continueInsertFromCsv(SSqlCallbackWrapper* pWrapper, SRequestObj* pRequest);
void destorySqlCallbackWrapper(SSqlCallbackWrapper* pWrapper);
void handleQueryAnslyseRes(SSqlCallbackWrapper *pWrapper, SMetaData *pResultMeta, int32_t code);
void restartAsyncQuery(SRequestObj *pRequest, int32_t code);
int32_t buildPreviousRequest(SRequestObj *pRequest, const char* sql, SRequestObj** pNewRequest);
int32_t prepareAndParseSqlSyntax(SSqlCallbackWrapper **ppWrapper, SRequestObj *pRequest, bool updateMetaForce);
void handleQueryAnslyseRes(SSqlCallbackWrapper* pWrapper, SMetaData* pResultMeta, int32_t code);
void restartAsyncQuery(SRequestObj* pRequest, int32_t code);
int32_t buildPreviousRequest(SRequestObj* pRequest, const char* sql, SRequestObj** pNewRequest);
int32_t prepareAndParseSqlSyntax(SSqlCallbackWrapper** ppWrapper, SRequestObj* pRequest, bool updateMetaForce);
void returnToUser(SRequestObj* pRequest);
void stopAllQueries(SRequestObj *pRequest);
void stopAllQueries(SRequestObj* pRequest);
void doRequestCallback(SRequestObj* pRequest, int32_t code);
void freeQueryParam(SSyncQueryParam* param);
#ifdef TD_ENTERPRISE
int32_t clientParseSqlImpl(void* param, const char* dbName, const char* sql, bool parseOnly, const char* effeciveUser, SParseSqlRes* pRes);
int32_t clientParseSqlImpl(void* param, const char* dbName, const char* sql, bool parseOnly, const char* effeciveUser,
SParseSqlRes* pRes);
#endif
#define TSC_ERR_RET(c) \
@ -474,13 +477,9 @@ int32_t clientParseSqlImpl(void* param, const char* dbName, const char* sql, boo
void slowQueryLog(int64_t rid, bool killed, int32_t code, int32_t cost);
enum {
MONITORSQLTYPESELECT = 0,
MONITORSQLTYPEINSERT = 1,
MONITORSQLTYPEDELETE = 2
};
enum { MONITORSQLTYPESELECT = 0, MONITORSQLTYPEINSERT = 1, MONITORSQLTYPEDELETE = 2 };
void sqlReqLog(int64_t rid, bool killed, int32_t code, int8_t type);
void sqlReqLog(int64_t rid, bool killed, int32_t code, int8_t type);
void tmqMgmtClose(void);

7
source/client/src/clientEnv.c
View File

@ -197,7 +197,12 @@ static int32_t generateWriteSlowLog(STscObj *pTscObj, SRequestObj *pRequest, int
ENV_JSON_FALSE_CHECK(cJSON_AddItemToObject(json, "db", cJSON_CreateString("")));
}
char *value = cJSON_PrintUnformatted(json);
char *value = cJSON_PrintUnformatted(json);
if (value == NULL) {
tscError("failed to print json");
code = TSDB_CODE_FAILED;
goto _end;
}
MonitorSlowLogData data = {0};
data.clusterId = pTscObj->pAppInfo->clusterId;
data.type = SLOW_LOG_WRITE;

58
source/client/src/clientImpl.c
View File

@ -678,9 +678,15 @@ _return:
return TSDB_CODE_SUCCESS;
}
void freeVgList(void* list) {
SArray* pList = *(SArray**)list;
taosArrayDestroy(pList);
}
int32_t buildAsyncExecNodeList(SRequestObj* pRequest, SArray** pNodeList, SArray* pMnodeList, SMetaData* pResultMeta) {
SArray* pDbVgList = NULL;
SArray* pQnodeList = NULL;
FDelete fp = NULL;
int32_t code = 0;
switch (tsQueryPolicy) {
@ -704,6 +710,43 @@ int32_t buildAsyncExecNodeList(SRequestObj* pRequest, SArray** pNodeList, SArray
goto _return;
}
}
} else {
fp = freeVgList;
int32_t dbNum = taosArrayGetSize(pRequest->dbList);
if (dbNum > 0) {
SCatalog* pCtg = NULL;
SAppInstInfo* pInst = pRequest->pTscObj->pAppInfo;
code = catalogGetHandle(pInst->clusterId, &pCtg);
if (code != TSDB_CODE_SUCCESS) {
goto _return;
}
pDbVgList = taosArrayInit(dbNum, POINTER_BYTES);
if (NULL == pDbVgList) {
code = terrno;
goto _return;
}
SArray* pVgList = NULL;
for (int32_t i = 0; i < dbNum; ++i) {
char* dbFName = taosArrayGet(pRequest->dbList, i);
SRequestConnInfo conn = {.pTrans = pInst->pTransporter,
.requestId = pRequest->requestId,
.requestObjRefId = pRequest->self,
.mgmtEps = getEpSet_s(&pInst->mgmtEp)};
// catalogGetDBVgList will handle dbFName == null.
code = catalogGetDBVgList(pCtg, &conn, dbFName, &pVgList);
if (code) {
goto _return;
}
if (NULL == taosArrayPush(pDbVgList, &pVgList)) {
code = terrno;
goto _return;
}
}
}
}
code = buildVnodePolicyNodeList(pRequest, pNodeList, pMnodeList, pDbVgList);
@ -744,17 +787,12 @@ int32_t buildAsyncExecNodeList(SRequestObj* pRequest, SArray** pNodeList, SArray
}
_return:
taosArrayDestroy(pDbVgList);
taosArrayDestroyEx(pDbVgList, fp);
taosArrayDestroy(pQnodeList);
return code;
}
void freeVgList(void* list) {
SArray* pList = *(SArray**)list;
taosArrayDestroy(pList);
}
int32_t buildSyncExecNodeList(SRequestObj* pRequest, SArray** pNodeList, SArray* pMnodeList) {
SArray* pDbVgList = NULL;
SArray* pQnodeList = NULL;
@ -1210,7 +1248,7 @@ void schedulerExecCb(SExecResult* pResult, void* param, int32_t code) {
}
}
SRequestObj* launchQueryImpl(SRequestObj* pRequest, SQuery* pQuery, bool keepQuery, void** res) {
void launchQueryImpl(SRequestObj* pRequest, SQuery* pQuery, bool keepQuery, void** res) {
int32_t code = 0;
if (pQuery->pRoot) {
@ -1296,8 +1334,6 @@ SRequestObj* launchQueryImpl(SRequestObj* pRequest, SQuery* pQuery, bool keepQue
*res = pRequest->body.resInfo.execRes.res;
pRequest->body.resInfo.execRes.res = NULL;
}
return pRequest;
}
static int32_t asyncExecSchQuery(SRequestObj* pRequest, SQuery* pQuery, SMetaData* pResultMeta,
@ -2896,8 +2932,8 @@ TAOS_RES* taosQueryImpl(TAOS* taos, const char* sql, bool validateOnly, int8_t s
return NULL;
}
code = tsem_destroy(&param->sem);
if(TSDB_CODE_SUCCESS != code) {
tscError("failed to destroy semaphore since %s", tstrerror(code));
if (TSDB_CODE_SUCCESS != code) {
tscError("failed to destroy semaphore since %s", tstrerror(code));
}
SRequestObj* pRequest = NULL;

10
source/client/src/clientMain.c
View File

@ -1316,9 +1316,9 @@ void doAsyncQuery(SRequestObj *pRequest, bool updateMetaForce) {
tscDebug("0x%" PRIx64 " client retry to handle the error, code:%d - %s, tryCount:%d,QID:0x%" PRIx64,
pRequest->self, code, tstrerror(code), pRequest->retry, pRequest->requestId);
code = refreshMeta(pRequest->pTscObj, pRequest);
if (code != 0){
tscWarn("0x%" PRIx64 " refresh meta failed, code:%d - %s,QID:0x%" PRIx64, pRequest->self, code,
tstrerror(code), pRequest->requestId);
if (code != 0) {
tscWarn("0x%" PRIx64 " refresh meta failed, code:%d - %s,QID:0x%" PRIx64, pRequest->self, code, tstrerror(code),
pRequest->requestId);
}
pRequest->prevCode = code;
doAsyncQuery(pRequest, true);
@ -1985,7 +1985,9 @@ int taos_stmt2_bind_param(TAOS_STMT2 *stmt, TAOS_STMT2_BINDV *bindv, int32_t col
STscStmt2 *pStmt = (STscStmt2 *)stmt;
if (pStmt->options.asyncExecFn && !pStmt->semWaited) {
(void)tsem_wait(&pStmt->asyncQuerySem);
if (tsem_wait(&pStmt->asyncQuerySem) != 0) {
tscError("wait async query sem failed");
}
pStmt->semWaited = true;
}

5
source/client/src/clientMonitor.c
View File

@ -447,20 +447,19 @@ static char* readFile(TdFilePtr pFile, int64_t* offset, int64_t size) {
char* pCont = NULL;
int64_t totalSize = 0;
if (size - *offset >= SLOW_LOG_SEND_SIZE_MAX) {
pCont = taosMemoryCalloc(1, 4 + SLOW_LOG_SEND_SIZE_MAX); // 4 reserved for []
totalSize = 4 + SLOW_LOG_SEND_SIZE_MAX;
} else {
pCont = taosMemoryCalloc(1, 4 + (size - *offset));
totalSize = 4 + (size - *offset);
}
pCont = taosMemoryCalloc(1, totalSize); // 4 reserved for []
if (pCont == NULL) {
tscError("failed to allocate memory for slow log, size:%" PRId64, totalSize);
return NULL;
}
char* buf = pCont;
(void)strcat(buf++, "[");
int64_t readSize = taosReadFile(pFile, buf, SLOW_LOG_SEND_SIZE_MAX);
int64_t readSize = taosReadFile(pFile, buf, totalSize - 4); // 4 reserved for []
if (readSize <= 0) {
if (readSize < 0) {
tscError("failed to read len from file:%p since %s", pFile, terrstr());

49
source/client/src/clientRawBlockWrite.c
View File

@ -361,7 +361,7 @@ static void processCreateStb(SMqMetaRsp* metaRsp, cJSON** pJson) {
buildCreateTableJson(&req.schemaRow, &req.schemaTag, req.name, req.suid, TSDB_SUPER_TABLE, &req.colCmpr, pJson);
end:
uDebug("create stable return, sql json:%s", cJSON_PrintUnformatted(*pJson));
uDebug("create stable return");
tDecoderClear(&coder);
}
@ -381,7 +381,7 @@ static void processAlterStb(SMqMetaRsp* metaRsp, cJSON** pJson) {
buildAlterSTableJson(req.alterOriData, req.alterOriDataLen, pJson);
end:
uDebug("alter stable return, sql json:%s", cJSON_PrintUnformatted(*pJson));
uDebug("alter stable return");
tDecoderClear(&coder);
}
@ -393,8 +393,8 @@ static void buildChildElement(cJSON* json, SVCreateTbReq* pCreateReq) {
int64_t id = pCreateReq->uid;
uint8_t tagNum = pCreateReq->ctb.tagNum;
int32_t code = 0;
cJSON* tableName = cJSON_CreateString(name);
cJSON* tags = NULL;
cJSON* tableName = cJSON_CreateString(name);
RAW_NULL_CHECK(tableName);
RAW_FALSE_CHECK(cJSON_AddItemToObject(json, "tableName", tableName));
cJSON* using = cJSON_CreateString(sname);
@ -404,7 +404,7 @@ static void buildChildElement(cJSON* json, SVCreateTbReq* pCreateReq) {
RAW_NULL_CHECK(tagNumJson);
RAW_FALSE_CHECK(cJSON_AddItemToObject(json, "tagNum", tagNumJson));
cJSON* tags = cJSON_CreateArray();
tags = cJSON_CreateArray();
RAW_NULL_CHECK(tags);
SArray* pTagVals = NULL;
RAW_RETURN_CHECK(tTagToValArray(pTag, &pTagVals));
@ -417,7 +417,7 @@ static void buildChildElement(cJSON* json, SVCreateTbReq* pCreateReq) {
}
char* pJson = NULL;
parseTagDatatoJson(pTag, &pJson);
if(pJson == NULL) {
if (pJson == NULL) {
uError("parseTagDatatoJson failed, pJson == NULL");
goto end;
}
@ -543,7 +543,7 @@ static void processCreateTable(SMqMetaRsp* metaRsp, cJSON** pJson) {
}
end:
uDebug("create table return, sql json:%s", cJSON_PrintUnformatted(*pJson));
uDebug("create table return");
tDeleteSVCreateTbBatchReq(&req);
tDecoderClear(&decoder);
}
@ -731,7 +731,7 @@ static void processAlterTable(SMqMetaRsp* metaRsp, cJSON** pJson) {
goto end;
}
parseTagDatatoJson(vAlterTbReq.pTagVal, &buf);
if(buf == NULL) {
if (buf == NULL) {
uError("parseTagDatatoJson failed, buf == NULL");
goto end;
}
@ -772,7 +772,7 @@ static void processAlterTable(SMqMetaRsp* metaRsp, cJSON** pJson) {
}
end:
uDebug("alter table return, sql json:%s", cJSON_PrintUnformatted(json));
uDebug("alter table return");
tDecoderClear(&decoder);
*pJson = json;
}
@ -807,7 +807,7 @@ static void processDropSTable(SMqMetaRsp* metaRsp, cJSON** pJson) {
RAW_FALSE_CHECK(cJSON_AddItemToObject(json, "tableName", tableName));
end:
uDebug("processDropSTable return, sql json:%s", cJSON_PrintUnformatted(json));
uDebug("processDropSTable return");
tDecoderClear(&decoder);
*pJson = json;
}
@ -843,7 +843,7 @@ static void processDeleteTable(SMqMetaRsp* metaRsp, cJSON** pJson) {
RAW_FALSE_CHECK(cJSON_AddItemToObject(json, "sql", sqlJson));
end:
uDebug("processDeleteTable return, sql json:%s", cJSON_PrintUnformatted(json));
uDebug("processDeleteTable return");
tDecoderClear(&coder);
*pJson = json;
}
@ -880,7 +880,7 @@ static void processDropTable(SMqMetaRsp* metaRsp, cJSON** pJson) {
RAW_FALSE_CHECK(cJSON_AddItemToObject(json, "tableNameList", tableNameList));
end:
uDebug("processDropTable return, json sql:%s", cJSON_PrintUnformatted(json));
uDebug("processDropTable return");
tDecoderClear(&decoder);
*pJson = json;
}
@ -978,7 +978,7 @@ static int32_t taosCreateStb(TAOS* taos, void* meta, int32_t metaLen) {
pQuery.msgType = pQuery.pCmdMsg->msgType;
pQuery.stableQuery = true;
(void)launchQueryImpl(pRequest, &pQuery, true, NULL); // ignore, because return value is pRequest
launchQueryImpl(pRequest, &pQuery, true, NULL); // ignore, because return value is pRequest
taosMemoryFree(pCmdMsg.pMsg);
@ -1082,7 +1082,7 @@ static int32_t taosDropStb(TAOS* taos, void* meta, int32_t metaLen) {
pQuery.msgType = pQuery.pCmdMsg->msgType;
pQuery.stableQuery = true;
(void)launchQueryImpl(pRequest, &pQuery, true, NULL); // ignore, because return value is pRequest
launchQueryImpl(pRequest, &pQuery, true, NULL); // ignore, because return value is pRequest
taosMemoryFree(pCmdMsg.pMsg);
if (pRequest->code == TSDB_CODE_SUCCESS) {
// ignore the error code
@ -1236,7 +1236,7 @@ static int32_t taosCreateTable(TAOS* taos, void* meta, int32_t metaLen) {
RAW_RETURN_CHECK(rewriteToVnodeModifyOpStmt(pQuery, pBufArray));
(void)launchQueryImpl(pRequest, pQuery, true, NULL);
launchQueryImpl(pRequest, pQuery, true, NULL);
if (pRequest->code == TSDB_CODE_SUCCESS) {
RAW_RETURN_CHECK(removeMeta(pTscObj, pRequest->tableList, false));
}
@ -1365,7 +1365,7 @@ static int32_t taosDropTable(TAOS* taos, void* meta, int32_t metaLen) {
if (TSDB_CODE_SUCCESS != code) goto end;
RAW_RETURN_CHECK(rewriteToVnodeModifyOpStmt(pQuery, pBufArray));
(void)launchQueryImpl(pRequest, pQuery, true, NULL);
launchQueryImpl(pRequest, pQuery, true, NULL);
if (pRequest->code == TSDB_CODE_SUCCESS) {
RAW_RETURN_CHECK(removeMeta(pTscObj, pRequest->tableList, false));
}
@ -1510,7 +1510,7 @@ static int32_t taosAlterTable(TAOS* taos, void* meta, int32_t metaLen) {
if (TSDB_CODE_SUCCESS != code) goto end;
RAW_RETURN_CHECK(rewriteToVnodeModifyOpStmt(pQuery, pArray));
(void)launchQueryImpl(pRequest, pQuery, true, NULL);
launchQueryImpl(pRequest, pQuery, true, NULL);
pVgData = NULL;
pArray = NULL;
@ -1587,7 +1587,7 @@ int taos_write_raw_block_with_fields_with_reqid(TAOS* taos, int rows, char* pDat
RAW_RETURN_CHECK(rawBlockBindData(pQuery, pTableMeta, pData, NULL, fields, numFields, false, NULL, 0));
RAW_RETURN_CHECK(smlBuildOutput(pQuery, pVgHash));
(void)launchQueryImpl(pRequest, pQuery, true, NULL);
launchQueryImpl(pRequest, pQuery, true, NULL);
code = pRequest->code;
end:
@ -1647,7 +1647,7 @@ int taos_write_raw_block_with_reqid(TAOS* taos, int rows, char* pData, const cha
RAW_RETURN_CHECK(rawBlockBindData(pQuery, pTableMeta, pData, NULL, NULL, 0, false, NULL, 0));
RAW_RETURN_CHECK(smlBuildOutput(pQuery, pVgHash));
(void)launchQueryImpl(pRequest, pQuery, true, NULL);
launchQueryImpl(pRequest, pQuery, true, NULL);
code = pRequest->code;
end:
@ -1766,7 +1766,7 @@ static int32_t tmqWriteRawDataImpl(TAOS* taos, void* data, int32_t dataLen) {
RAW_RETURN_CHECK(smlBuildOutput(pQuery, pVgHash));
(void)launchQueryImpl(pRequest, pQuery, true, NULL);
launchQueryImpl(pRequest, pQuery, true, NULL);
code = pRequest->code;
end:
@ -1923,6 +1923,10 @@ static int32_t tmqWriteRawMetaDataImpl(TAOS* taos, void* data, int32_t dataLen)
code = rawBlockBindData(pQuery, pTableMeta, rawData, &pCreateReqDst, fields, pSW->nCols, true, err, ERR_MSG_LEN);
taosMemoryFree(fields);
taosMemoryFreeClear(pTableMeta);
if (pCreateReqDst) {
tdDestroySVCreateTbReq(pCreateReqDst);
taosMemoryFreeClear(pCreateReqDst);
}
if (code != TSDB_CODE_SUCCESS) {
SET_ERROR_MSG("table:%s, err:%s", tbName, err);
goto end;
@ -1931,7 +1935,7 @@ static int32_t tmqWriteRawMetaDataImpl(TAOS* taos, void* data, int32_t dataLen)
RAW_RETURN_CHECK(smlBuildOutput(pQuery, pVgHash));
(void)launchQueryImpl(pRequest, pQuery, true, NULL);
launchQueryImpl(pRequest, pQuery, true, NULL);
code = pRequest->code;
end:
@ -1944,7 +1948,7 @@ end:
taosMemoryFreeClear(pTableMeta);
if (pCreateReqDst) {
tdDestroySVCreateTbReq(pCreateReqDst);
taosMemoryFree(pCreateReqDst);
taosMemoryFreeClear(pCreateReqDst);
}
return code;
}
@ -2038,6 +2042,7 @@ char* tmq_get_json_meta(TAOS_RES* res) {
processSimpleMeta(&pMetaRspObj->metaRsp, &pJson);
char* string = cJSON_PrintUnformatted(pJson);
cJSON_Delete(pJson);
uDebug("tmq_get_json_meta string:%s", string);
return string;
}

214
source/client/src/clientSml.c
View File

@ -112,7 +112,7 @@ static int32_t smlCheckAuth(SSmlHandle *info, SRequestConnInfo *conn, const char
SUserAuthInfo pAuth = {0};
(void)snprintf(pAuth.user, sizeof(pAuth.user), "%s", info->taos->user);
if (NULL == pTabName) {
if (tNameSetDbName(&pAuth.tbName, info->taos->acctId, info->pRequest->pDb, strlen(info->pRequest->pDb)) != 0){
if (tNameSetDbName(&pAuth.tbName, info->taos->acctId, info->pRequest->pDb, strlen(info->pRequest->pDb)) != 0) {
return TSDB_CODE_SML_INVALID_DATA;
}
} else {
@ -165,7 +165,7 @@ int64_t smlGetTimeValue(const char *value, int32_t len, uint8_t fromPrecision, u
return convertTimePrecision(tsInt64, fromPrecision, toPrecision);
}
int32_t smlBuildTableInfo(int numRows, const char *measure, int32_t measureLen, SSmlTableInfo** tInfo) {
int32_t smlBuildTableInfo(int numRows, const char *measure, int32_t measureLen, SSmlTableInfo **tInfo) {
SSmlTableInfo *tag = (SSmlTableInfo *)taosMemoryCalloc(sizeof(SSmlTableInfo), 1);
if (!tag) {
return terrno;
@ -203,13 +203,13 @@ static void smlDestroySTableMeta(void *para) {
taosMemoryFree(meta);
}
int32_t smlBuildSuperTableInfo(SSmlHandle *info, SSmlLineInfo *currElement, SSmlSTableMeta** sMeta) {
int32_t code = TSDB_CODE_SUCCESS;
char *measure = currElement->measure;
int measureLen = currElement->measureLen;
int32_t smlBuildSuperTableInfo(SSmlHandle *info, SSmlLineInfo *currElement, SSmlSTableMeta **sMeta) {
int32_t code = TSDB_CODE_SUCCESS;
char *measure = currElement->measure;
int measureLen = currElement->measureLen;
if (currElement->measureEscaped) {
measure = (char *)taosMemoryMalloc(measureLen);
if (measure == NULL){
if (measure == NULL) {
return terrno;
}
(void)memcpy(measure, currElement->measure, measureLen);
@ -233,7 +233,7 @@ int32_t smlBuildSuperTableInfo(SSmlHandle *info, SSmlLineInfo *currElement, SSml
}
(*sMeta)->tableMeta = pTableMeta;
code = taosHashPut(info->superTables, currElement->measure, currElement->measureLen, sMeta, POINTER_BYTES);
if (code != TSDB_CODE_SUCCESS){
if (code != TSDB_CODE_SUCCESS) {
smlDestroySTableMeta(*sMeta);
return code;
}
@ -250,11 +250,11 @@ int32_t smlBuildSuperTableInfo(SSmlHandle *info, SSmlLineInfo *currElement, SSml
}
if (i < pTableMeta->tableInfo.numOfColumns) {
if(taosArrayPush((*sMeta)->cols, &kv) == NULL){
if (taosArrayPush((*sMeta)->cols, &kv) == NULL) {
return terrno;
}
} else {
if(taosArrayPush((*sMeta)->tags, &kv) == NULL){
if (taosArrayPush((*sMeta)->tags, &kv) == NULL) {
return terrno;
}
}
@ -277,7 +277,7 @@ bool isSmlColAligned(SSmlHandle *info, int cnt, SSmlKv *kv) {
goto END;
}
SSmlKv *maxKV = (SSmlKv *)taosArrayGet(info->maxColKVs, cnt);
if (maxKV == NULL){
if (maxKV == NULL) {
goto END;
}
if (unlikely(!IS_SAME_KEY)) {
@ -336,9 +336,9 @@ int32_t smlJoinMeasureTag(SSmlLineInfo *elements) {
return TSDB_CODE_SUCCESS;
}
static bool smlIsPKTable(STableMeta *pTableMeta){
for(int i = 0; i < pTableMeta->tableInfo.numOfColumns; i++){
if(pTableMeta->schema[i].flags & COL_IS_KEY){
static bool smlIsPKTable(STableMeta *pTableMeta) {
for (int i = 0; i < pTableMeta->tableInfo.numOfColumns; i++) {
if (pTableMeta->schema[i].flags & COL_IS_KEY) {
return true;
}
}
@ -368,14 +368,14 @@ int32_t smlProcessSuperTable(SSmlHandle *info, SSmlLineInfo *elements) {
info->maxTagKVs = sMeta->tags;
info->maxColKVs = sMeta->cols;
if(smlIsPKTable(sMeta->tableMeta)){
if (smlIsPKTable(sMeta->tableMeta)) {
return TSDB_CODE_SML_NOT_SUPPORT_PK;
}
return 0;
}
int32_t smlProcessChildTable(SSmlHandle *info, SSmlLineInfo *elements) {
int32_t code = TSDB_CODE_SUCCESS;
int32_t code = TSDB_CODE_SUCCESS;
SSmlTableInfo **oneTable =
(SSmlTableInfo **)taosHashGet(info->childTables, elements->measureTag, elements->measureTagsLen);
SSmlTableInfo *tinfo = NULL;
@ -385,19 +385,19 @@ int32_t smlProcessChildTable(SSmlHandle *info, SSmlLineInfo *elements) {
return code;
}
code = taosHashPut(info->childTables, elements->measureTag, elements->measureTagsLen, &tinfo, POINTER_BYTES);
if(code != 0){
if (code != 0) {
smlDestroyTableInfo(&tinfo);
return code;
}
tinfo->tags = taosArrayDup(info->preLineTagKV, NULL);
if(tinfo->tags == NULL){
if (tinfo->tags == NULL) {
smlDestroyTableInfo(&tinfo);
return TSDB_CODE_OUT_OF_MEMORY;
}
for (size_t i = 0; i < taosArrayGetSize(info->preLineTagKV); i++) {
SSmlKv *kv = (SSmlKv *)taosArrayGet(info->preLineTagKV, i);
if(kv == NULL){
if (kv == NULL) {
smlDestroyTableInfo(&tinfo);
return TSDB_CODE_SML_INVALID_DATA;
}
@ -406,12 +406,12 @@ int32_t smlProcessChildTable(SSmlHandle *info, SSmlLineInfo *elements) {
}
code = smlSetCTableName(tinfo, info->tbnameKey);
if (code != TSDB_CODE_SUCCESS){
if (code != TSDB_CODE_SUCCESS) {
smlDestroyTableInfo(&tinfo);
return code;
}
code = getTableUid(info, elements, tinfo);
if (code != TSDB_CODE_SUCCESS){
if (code != TSDB_CODE_SUCCESS) {
smlDestroyTableInfo(&tinfo);
return code;
}
@ -458,10 +458,10 @@ int32_t smlParseEndTelnetJson(SSmlHandle *info, SSmlLineInfo *elements, SSmlKv *
return terrno;
}
}
if (taosArrayPush(elements->colArray, kvTs) == NULL){
if (taosArrayPush(elements->colArray, kvTs) == NULL) {
return terrno;
}
if (taosArrayPush(elements->colArray, kv) == NULL){
if (taosArrayPush(elements->colArray, kv) == NULL) {
return terrno;
}
}
@ -495,11 +495,11 @@ int32_t smlParseEndLine(SSmlHandle *info, SSmlLineInfo *elements, SSmlKv *kvTs)
static int32_t smlParseTableName(SArray *tags, char *childTableName, char *tbnameKey) {
bool autoChildName = false;
size_t delimiter = strlen(tsSmlAutoChildTableNameDelimiter);
if(delimiter > 0 && tbnameKey == NULL){
if (delimiter > 0 && tbnameKey == NULL) {
size_t totalNameLen = delimiter * (taosArrayGetSize(tags) - 1);
for (int i = 0; i < taosArrayGetSize(tags); i++) {
SSmlKv *tag = (SSmlKv *)taosArrayGet(tags, i);
if(tag == NULL){
if (tag == NULL) {
return TSDB_CODE_SML_INVALID_DATA;
}
totalNameLen += tag->length;
@ -512,7 +512,7 @@ static int32_t smlParseTableName(SArray *tags, char *childTableName, char *tbnam
(void)memset(childTableName, 0, TSDB_TABLE_NAME_LEN);
for (int i = 0; i < taosArrayGetSize(tags); i++) {
SSmlKv *tag = (SSmlKv *)taosArrayGet(tags, i);
if(tag == NULL){
if (tag == NULL) {
return TSDB_CODE_SML_INVALID_DATA;
}
(void)strncat(childTableName, tag->value, tag->length);
@ -523,8 +523,8 @@ static int32_t smlParseTableName(SArray *tags, char *childTableName, char *tbnam
if (tsSmlDot2Underline) {
smlStrReplace(childTableName, strlen(childTableName));
}
}else{
if (tbnameKey == NULL){
} else {
if (tbnameKey == NULL) {
tbnameKey = tsSmlChildTableName;
}
size_t childTableNameLen = strlen(tbnameKey);
@ -532,13 +532,14 @@ static int32_t smlParseTableName(SArray *tags, char *childTableName, char *tbnam
for (int i = 0; i < taosArrayGetSize(tags); i++) {
SSmlKv *tag = (SSmlKv *)taosArrayGet(tags, i);
if(tag == NULL){
if (tag == NULL) {
return TSDB_CODE_SML_INVALID_DATA;
}
// handle child table name
if (childTableNameLen == tag->keyLen && strncmp(tag->key, tbnameKey, tag->keyLen) == 0) {
(void)memset(childTableName, 0, TSDB_TABLE_NAME_LEN);
(void)strncpy(childTableName, tag->value, (tag->length < TSDB_TABLE_NAME_LEN ? tag->length : TSDB_TABLE_NAME_LEN));
(void)strncpy(childTableName, tag->value,
(tag->length < TSDB_TABLE_NAME_LEN ? tag->length : TSDB_TABLE_NAME_LEN));
if (tsSmlDot2Underline) {
smlStrReplace(childTableName, strlen(childTableName));
}
@ -553,7 +554,7 @@ static int32_t smlParseTableName(SArray *tags, char *childTableName, char *tbnam
int32_t smlSetCTableName(SSmlTableInfo *oneTable, char *tbnameKey) {
int32_t code = smlParseTableName(oneTable->tags, oneTable->childTableName, tbnameKey);
if(code != TSDB_CODE_SUCCESS){
if (code != TSDB_CODE_SUCCESS) {
return code;
}
@ -562,7 +563,7 @@ int32_t smlSetCTableName(SSmlTableInfo *oneTable, char *tbnameKey) {
if (dst == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
}
if(oneTable->sTableNameLen >= TSDB_TABLE_NAME_LEN){
if (oneTable->sTableNameLen >= TSDB_TABLE_NAME_LEN) {
uError("SML:smlSetCTableName super table name is too long");
taosArrayDestroy(dst);
return TSDB_CODE_SML_INTERNAL_ERROR;
@ -578,7 +579,7 @@ int32_t smlSetCTableName(SSmlTableInfo *oneTable, char *tbnameKey) {
}
code = buildChildTableName(&rName);
if (code != TSDB_CODE_SUCCESS){
if (code != TSDB_CODE_SUCCESS) {
return code;
}
taosArrayDestroy(dst);
@ -906,13 +907,13 @@ static int32_t smlFindNearestPowerOf2(int32_t length, uint8_t type) {
return result;
}
static int32_t smlProcessSchemaAction(SSmlHandle *info, SSchema *schemaField, SHashObj *schemaHash, SArray *cols, SArray *checkDumplicateCols,
ESchemaAction *action, bool isTag) {
static int32_t smlProcessSchemaAction(SSmlHandle *info, SSchema *schemaField, SHashObj *schemaHash, SArray *cols,
SArray *checkDumplicateCols, ESchemaAction *action, bool isTag) {
int32_t code = TSDB_CODE_SUCCESS;
for (int j = 0; j < taosArrayGetSize(cols); ++j) {
if (j == 0 && !isTag) continue;
SSmlKv *kv = (SSmlKv *)taosArrayGet(cols, j);
if (kv == NULL){
if (kv == NULL) {
return TSDB_CODE_SML_INVALID_DATA;
}
code = smlGenerateSchemaAction(schemaField, schemaHash, kv, isTag, action, info);
@ -923,10 +924,10 @@ static int32_t smlProcessSchemaAction(SSmlHandle *info, SSchema *schemaField, SH
for (int j = 0; j < taosArrayGetSize(checkDumplicateCols); ++j) {
SSmlKv *kv = (SSmlKv *)taosArrayGet(checkDumplicateCols, j);
if (kv == NULL){
if (kv == NULL) {
return TSDB_CODE_SML_INVALID_DATA;
}
if(taosHashGet(schemaHash, kv->key, kv->keyLen) != NULL){
if (taosHashGet(schemaHash, kv->key, kv->keyLen) != NULL) {
return TSDB_CODE_PAR_DUPLICATED_COLUMN;
}
}
@ -934,16 +935,16 @@ static int32_t smlProcessSchemaAction(SSmlHandle *info, SSchema *schemaField, SH
}
static int32_t smlCheckMeta(SSchema *schema, int32_t length, SArray *cols, bool isTag) {
int32_t code = TSDB_CODE_SUCCESS;
int32_t code = TSDB_CODE_SUCCESS;
SHashObj *hashTmp = taosHashInit(length, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_NO_LOCK);
if (hashTmp == NULL) {
code = terrno;
goto END;
}
int32_t i = 0;
int32_t i = 0;
for (; i < length; i++) {
code = taosHashPut(hashTmp, schema[i].name, strlen(schema[i].name), &i, SHORT_BYTES);
if (code != 0){
if (code != 0) {
goto END;
}
}
@ -955,7 +956,7 @@ static int32_t smlCheckMeta(SSchema *schema, int32_t length, SArray *cols, bool
}
for (; i < taosArrayGetSize(cols); i++) {
SSmlKv *kv = (SSmlKv *)taosArrayGet(cols, i);
if (kv == NULL){
if (kv == NULL) {
code = TSDB_CODE_SML_INVALID_DATA;
goto END;
}
@ -982,8 +983,8 @@ static int32_t getBytes(uint8_t type, int32_t length) {
static int32_t smlBuildFieldsList(SSmlHandle *info, SSchema *schemaField, SHashObj *schemaHash, SArray *cols,
SArray *results, int32_t numOfCols, bool isTag) {
for (int j = 0; j < taosArrayGetSize(cols); ++j) {
SSmlKv *kv = (SSmlKv *)taosArrayGet(cols, j);
if (kv == NULL){
SSmlKv *kv = (SSmlKv *)taosArrayGet(cols, j);
if (kv == NULL) {
return TSDB_CODE_SML_INVALID_DATA;
}
ESchemaAction action = SCHEMA_ACTION_NULL;
@ -996,7 +997,7 @@ static int32_t smlBuildFieldsList(SSmlHandle *info, SSchema *schemaField, SHashO
field.type = kv->type;
field.bytes = getBytes(kv->type, kv->length);
(void)memcpy(field.name, kv->key, kv->keyLen);
if (taosArrayPush(results, &field) == NULL){
if (taosArrayPush(results, &field) == NULL) {
return terrno;
}
} else if (action == SCHEMA_ACTION_CHANGE_COLUMN_SIZE || action == SCHEMA_ACTION_CHANGE_TAG_SIZE) {
@ -1008,7 +1009,7 @@ static int32_t smlBuildFieldsList(SSmlHandle *info, SSchema *schemaField, SHashO
uint16_t newIndex = *index;
if (isTag) newIndex -= numOfCols;
SField *field = (SField *)taosArrayGet(results, newIndex);
if (field == NULL){
if (field == NULL) {
return TSDB_CODE_SML_INVALID_DATA;
}
field->bytes = getBytes(kv->type, kv->length);
@ -1019,7 +1020,7 @@ static int32_t smlBuildFieldsList(SSmlHandle *info, SSchema *schemaField, SHashO
int32_t len = 0;
for (int j = 0; j < taosArrayGetSize(results); ++j) {
SField *field = taosArrayGet(results, j);
if (field == NULL){
if (field == NULL) {
return TSDB_CODE_SML_INVALID_DATA;
}
len += field->bytes;
@ -1051,14 +1052,14 @@ static int32_t smlSendMetaMsg(SSmlHandle *info, SName *pName, SArray *pColumns,
}
for (int32_t i = 0; i < pReq.numOfColumns; ++i) {
SField *pField = taosArrayGet(pColumns, i);
if (pField == NULL){
if (pField == NULL) {
code = TSDB_CODE_SML_INVALID_DATA;
goto end;
}
SFieldWithOptions fieldWithOption = {0};
setFieldWithOptions(&fieldWithOption, pField);
setDefaultOptionsForField(&fieldWithOption);
if (taosArrayPush(pReq.pColumns, &fieldWithOption) == NULL){
if (taosArrayPush(pReq.pColumns, &fieldWithOption) == NULL) {
code = terrno;
goto end;
}
@ -1105,7 +1106,7 @@ static int32_t smlSendMetaMsg(SSmlHandle *info, SName *pName, SArray *pColumns,
field.type = TSDB_DATA_TYPE_NCHAR;
field.bytes = TSDB_NCHAR_SIZE + VARSTR_HEADER_SIZE;
tstrncpy(field.name, tsSmlTagName, sizeof(field.name));
if (taosArrayPush(pReq.pTags, &field) == NULL){
if (taosArrayPush(pReq.pTags, &field) == NULL) {
code = terrno;
goto end;
}
@ -1121,7 +1122,7 @@ static int32_t smlSendMetaMsg(SSmlHandle *info, SName *pName, SArray *pColumns,
pCmdMsg.epSet = getEpSet_s(&info->taos->pAppInfo->mgmtEp);
pCmdMsg.msgType = TDMT_MND_CREATE_STB;
pCmdMsg.msgLen = tSerializeSMCreateStbReq(NULL, 0, &pReq);
if (pCmdMsg.msgLen < 0){
if (pCmdMsg.msgLen < 0) {
code = TSDB_CODE_OUT_OF_MEMORY;
goto end;
}
@ -1131,7 +1132,7 @@ static int32_t smlSendMetaMsg(SSmlHandle *info, SName *pName, SArray *pColumns,
goto end;
}
if (tSerializeSMCreateStbReq(pCmdMsg.pMsg, pCmdMsg.msgLen, &pReq) < 0){
if (tSerializeSMCreateStbReq(pCmdMsg.pMsg, pCmdMsg.msgLen, &pReq) < 0) {
code = TSDB_CODE_OUT_OF_MEMORY;
taosMemoryFree(pCmdMsg.pMsg);
goto end;
@ -1144,11 +1145,11 @@ static int32_t smlSendMetaMsg(SSmlHandle *info, SName *pName, SArray *pColumns,
pQuery.msgType = pQuery.pCmdMsg->msgType;
pQuery.stableQuery = true;
(void)launchQueryImpl(pRequest, &pQuery, true, NULL); // no need to check return value
launchQueryImpl(pRequest, &pQuery, true, NULL); // no need to check return value
if (pRequest->code == TSDB_CODE_SUCCESS) {
code = catalogRemoveTableMeta(info->pCatalog, pName);
if (code != TSDB_CODE_SUCCESS){
if (code != TSDB_CODE_SUCCESS) {
goto end;
}
}
@ -1187,7 +1188,7 @@ static int32_t smlModifyDBSchemas(SSmlHandle *info) {
size_t superTableLen = 0;
void *superTable = taosHashGetKey(tmp, &superTableLen);
char *measure = taosMemoryMalloc(superTableLen);
if (measure == NULL){
if (measure == NULL) {
code = terrno;
goto end;
}
@ -1246,28 +1247,28 @@ static int32_t smlModifyDBSchemas(SSmlHandle *info) {
goto end;
}
} else if (code == TSDB_CODE_SUCCESS) {
if(smlIsPKTable(pTableMeta)){
if (smlIsPKTable(pTableMeta)) {
code = TSDB_CODE_SML_NOT_SUPPORT_PK;
goto end;
}
hashTmp = taosHashInit(pTableMeta->tableInfo.numOfTags, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true,
HASH_NO_LOCK);
if (hashTmp == NULL){
if (hashTmp == NULL) {
code = terrno;
goto end;
}
for (uint16_t i = pTableMeta->tableInfo.numOfColumns;
i < pTableMeta->tableInfo.numOfColumns + pTableMeta->tableInfo.numOfTags; i++) {
code = taosHashPut(hashTmp, pTableMeta->schema[i].name, strlen(pTableMeta->schema[i].name), &i, SHORT_BYTES);
if (code != 0){
if (code != 0) {
goto end;
}
}
ESchemaAction action = SCHEMA_ACTION_NULL;
code = smlProcessSchemaAction(info, pTableMeta->schema, hashTmp, sTableData->tags, sTableData->cols, &action, true);
code =
smlProcessSchemaAction(info, pTableMeta->schema, hashTmp, sTableData->tags, sTableData->cols, &action, true);
if (code != TSDB_CODE_SUCCESS) {
goto end;
}
@ -1280,13 +1281,13 @@ static int32_t smlModifyDBSchemas(SSmlHandle *info) {
action);
SArray *pColumns =
taosArrayInit(taosArrayGetSize(sTableData->cols) + pTableMeta->tableInfo.numOfColumns, sizeof(SField));
if (pColumns == NULL){
if (pColumns == NULL) {
code = terrno;
goto end;
}
SArray *pTags =
taosArrayInit(taosArrayGetSize(sTableData->tags) + pTableMeta->tableInfo.numOfTags, sizeof(SField));
if (pTags == NULL){
if (pTags == NULL) {
taosArrayDestroy(pColumns);
code = terrno;
goto end;
@ -1297,14 +1298,14 @@ static int32_t smlModifyDBSchemas(SSmlHandle *info) {
field.bytes = pTableMeta->schema[i].bytes;
tstrncpy(field.name, pTableMeta->schema[i].name, sizeof(field.name));
if (i < pTableMeta->tableInfo.numOfColumns) {
if (taosArrayPush(pColumns, &field) == NULL){
if (taosArrayPush(pColumns, &field) == NULL) {
taosArrayDestroy(pColumns);
taosArrayDestroy(pTags);
code = terrno;
goto end;
}
} else {
if (taosArrayPush(pTags, &field) == NULL){
if (taosArrayPush(pTags, &field) == NULL) {
taosArrayDestroy(pColumns);
taosArrayDestroy(pTags);
code = terrno;
@ -1363,7 +1364,8 @@ static int32_t smlModifyDBSchemas(SSmlHandle *info) {
}
}
action = SCHEMA_ACTION_NULL;
code = smlProcessSchemaAction(info, pTableMeta->schema, hashTmp, sTableData->cols, sTableData->tags, &action, false);
code =
smlProcessSchemaAction(info, pTableMeta->schema, hashTmp, sTableData->cols, sTableData->tags, &action, false);
if (code != TSDB_CODE_SUCCESS) {
goto end;
}
@ -1376,13 +1378,13 @@ static int32_t smlModifyDBSchemas(SSmlHandle *info) {
action);
SArray *pColumns =
taosArrayInit(taosArrayGetSize(sTableData->cols) + pTableMeta->tableInfo.numOfColumns, sizeof(SField));
if (pColumns == NULL){
if (pColumns == NULL) {
code = terrno;
goto end;
}
SArray *pTags =
taosArrayInit(taosArrayGetSize(sTableData->tags) + pTableMeta->tableInfo.numOfTags, sizeof(SField));
if (pTags == NULL){
if (pTags == NULL) {
taosArrayDestroy(pColumns);
code = terrno;
goto end;
@ -1393,14 +1395,14 @@ static int32_t smlModifyDBSchemas(SSmlHandle *info) {
field.bytes = pTableMeta->schema[i].bytes;
tstrncpy(field.name, pTableMeta->schema[i].name, sizeof(field.name));
if (i < pTableMeta->tableInfo.numOfColumns) {
if (taosArrayPush(pColumns, &field) == NULL){
if (taosArrayPush(pColumns, &field) == NULL) {
taosArrayDestroy(pColumns);
taosArrayDestroy(pTags);
code = terrno;
goto end;
}
} else {
if (taosArrayPush(pTags, &field) == NULL){
if (taosArrayPush(pTags, &field) == NULL) {
taosArrayDestroy(pColumns);
taosArrayDestroy(pTags);
code = terrno;
@ -1483,7 +1485,7 @@ end:
taosHashCancelIterate(info->superTables, tmp);
taosHashCleanup(hashTmp);
taosMemoryFreeClear(pTableMeta);
(void)catalogRefreshTableMeta(info->pCatalog, &conn, &pName, 1); // ignore refresh meta code if there is an error
(void)catalogRefreshTableMeta(info->pCatalog, &conn, &pName, 1); // ignore refresh meta code if there is an error
uError("SML:0x%" PRIx64 " smlModifyDBSchemas end failed:%d:%s, format:%d, needModifySchema:%d", info->id, code,
tstrerror(code), info->dataFormat, info->needModifySchema);
@ -1494,34 +1496,35 @@ static int32_t smlInsertMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols
terrno = 0;
for (int16_t i = 0; i < taosArrayGetSize(cols); ++i) {
SSmlKv *kv = (SSmlKv *)taosArrayGet(cols, i);
if (kv == NULL){
if (kv == NULL) {
return TSDB_CODE_SML_INVALID_DATA;
}
int ret = taosHashPut(metaHash, kv->key, kv->keyLen, &i, SHORT_BYTES);
int ret = taosHashPut(metaHash, kv->key, kv->keyLen, &i, SHORT_BYTES);
if (ret == 0) {
if (taosArrayPush(metaArray, kv) == NULL){
if (taosArrayPush(metaArray, kv) == NULL) {
return terrno;
}
if(taosHashGet(checkDuplicate, kv->key, kv->keyLen) != NULL) {
if (taosHashGet(checkDuplicate, kv->key, kv->keyLen) != NULL) {
return TSDB_CODE_PAR_DUPLICATED_COLUMN;
}
}else if(terrno == TSDB_CODE_DUP_KEY){
} else if (terrno == TSDB_CODE_DUP_KEY) {
return TSDB_CODE_PAR_DUPLICATED_COLUMN;
}
}
return TSDB_CODE_SUCCESS;
}
static int32_t smlUpdateMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols, bool isTag, SSmlMsgBuf *msg, SHashObj* checkDuplicate) {
static int32_t smlUpdateMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols, bool isTag, SSmlMsgBuf *msg,
SHashObj *checkDuplicate) {
for (int i = 0; i < taosArrayGetSize(cols); ++i) {
SSmlKv *kv = (SSmlKv *)taosArrayGet(cols, i);
if (kv == NULL){
if (kv == NULL) {
return TSDB_CODE_SML_INVALID_DATA;
}
int16_t *index = (int16_t *)taosHashGet(metaHash, kv->key, kv->keyLen);
if (index) {
SSmlKv *value = (SSmlKv *)taosArrayGet(metaArray, *index);
if (value == NULL){
if (value == NULL) {
return TSDB_CODE_SML_INVALID_DATA;
}
@ -1549,13 +1552,13 @@ static int32_t smlUpdateMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols
int16_t size = tmp;
int ret = taosHashPut(metaHash, kv->key, kv->keyLen, &size, SHORT_BYTES);
if (ret == 0) {
if(taosArrayPush(metaArray, kv) == NULL){
if (taosArrayPush(metaArray, kv) == NULL) {
return terrno;
}
if(taosHashGet(checkDuplicate, kv->key, kv->keyLen) != NULL) {
if (taosHashGet(checkDuplicate, kv->key, kv->keyLen) != NULL) {
return TSDB_CODE_PAR_DUPLICATED_COLUMN;
}
}else{
} else {
return ret;
}
}
@ -1586,7 +1589,7 @@ void freeSSmlKv(void *data) {
void smlDestroyInfo(SSmlHandle *info) {
if (!info) return;
// qDestroyQuery(info->pQuery);
// qDestroyQuery(info->pQuery);
taosHashCleanup(info->pVgHash);
taosHashCleanup(info->childTables);
@ -1657,7 +1660,7 @@ int32_t smlBuildSmlInfo(TAOS *taos, SSmlHandle **handle) {
info->id = smlGenId();
code = smlInitHandle(&info->pQuery);
if (code != TSDB_CODE_SUCCESS){
if (code != TSDB_CODE_SUCCESS) {
goto FAILED;
}
info->dataFormat = true;
@ -1688,7 +1691,7 @@ static int32_t smlPushCols(SArray *colsArray, SArray *cols) {
}
for (size_t i = 0; i < taosArrayGetSize(cols); i++) {
SSmlKv *kv = (SSmlKv *)taosArrayGet(cols, i);
if (kv == NULL){
if (kv == NULL) {
taosHashCleanup(kvHash);
return TSDB_CODE_SML_INVALID_DATA;
}
@ -1698,7 +1701,7 @@ static int32_t smlPushCols(SArray *colsArray, SArray *cols) {
taosHashCleanup(kvHash);
return TSDB_CODE_PAR_DUPLICATED_COLUMN;
}
if (code != TSDB_CODE_SUCCESS){
if (code != TSDB_CODE_SUCCESS) {
taosHashCleanup(kvHash);
return code;
}
@ -1759,9 +1762,11 @@ static int32_t smlParseLineBottom(SSmlHandle *info) {
if (tableMeta) { // update meta
uDebug("SML:0x%" PRIx64 " smlParseLineBottom update meta, format:%d, linenum:%d", info->id, info->dataFormat,
info->lineNum);
ret = smlUpdateMeta((*tableMeta)->colHash, (*tableMeta)->cols, elements->colArray, false, &info->msgBuf, (*tableMeta)->tagHash);
ret = smlUpdateMeta((*tableMeta)->colHash, (*tableMeta)->cols, elements->colArray, false, &info->msgBuf,
(*tableMeta)->tagHash);
if (ret == TSDB_CODE_SUCCESS) {
ret = smlUpdateMeta((*tableMeta)->tagHash, (*tableMeta)->tags, tinfo->tags, true, &info->msgBuf, (*tableMeta)->colHash);
ret = smlUpdateMeta((*tableMeta)->tagHash, (*tableMeta)->tags, tinfo->tags, true, &info->msgBuf,
(*tableMeta)->colHash);
}
if (ret != TSDB_CODE_SUCCESS) {
uError("SML:0x%" PRIx64 " smlUpdateMeta failed, ret:%d", info->id, ret);
@ -1801,17 +1806,17 @@ static int32_t smlInsertData(SSmlHandle *info) {
if (info->pRequest->dbList == NULL) {
info->pRequest->dbList = taosArrayInit(1, TSDB_DB_FNAME_LEN);
if (info->pRequest->dbList == NULL){
if (info->pRequest->dbList == NULL) {
return terrno;
}
}
char *data = (char *)taosArrayReserve(info->pRequest->dbList, 1);
if (data == NULL){
if (data == NULL) {
return terrno;
}
SName pName = {TSDB_TABLE_NAME_T, info->taos->acctId, {0}, {0}};
tstrncpy(pName.dbname, info->pRequest->pDb, sizeof(pName.dbname));
(void)tNameGetFullDbName(&pName, data); //ignore
(void)tNameGetFullDbName(&pName, data); // ignore
SSmlTableInfo **oneTable = (SSmlTableInfo **)taosHashIterate(info->childTables, NULL);
while (oneTable) {
@ -1819,7 +1824,7 @@ static int32_t smlInsertData(SSmlHandle *info) {
int measureLen = tableData->sTableNameLen;
char *measure = (char *)taosMemoryMalloc(tableData->sTableNameLen);
if (measure == NULL){
if (measure == NULL) {
return terrno;
}
(void)memcpy(measure, tableData->sTableName, tableData->sTableNameLen);
@ -1830,11 +1835,11 @@ static int32_t smlInsertData(SSmlHandle *info) {
if (info->pRequest->tableList == NULL) {
info->pRequest->tableList = taosArrayInit(1, sizeof(SName));
if (info->pRequest->tableList == NULL){
if (info->pRequest->tableList == NULL) {
return terrno;
}
}
if (taosArrayPush(info->pRequest->tableList, &pName) == NULL){
if (taosArrayPush(info->pRequest->tableList, &pName) == NULL) {
return terrno;
}
@ -1862,7 +1867,7 @@ static int32_t smlInsertData(SSmlHandle *info) {
return code;
}
code = taosHashPut(info->pVgHash, (const char *)&vg.vgId, sizeof(vg.vgId), (char *)&vg, sizeof(vg));
if (code != TSDB_CODE_SUCCESS){
if (code != TSDB_CODE_SUCCESS) {
uError("SML:0x%" PRIx64 " taosHashPut failed. table name: %s", info->id, tableData->childTableName);
taosMemoryFree(measure);
taosHashCancelIterate(info->childTables, oneTable);
@ -1904,9 +1909,9 @@ static int32_t smlInsertData(SSmlHandle *info) {
info->cost.insertRpcTime = taosGetTimestampUs();
SAppClusterSummary *pActivity = &info->taos->pAppInfo->summary;
(void)atomic_add_fetch_64((int64_t *)&pActivity->numOfInsertsReq, 1); // no need to check return code
(void)atomic_add_fetch_64((int64_t *)&pActivity->numOfInsertsReq, 1); // no need to check return code
(void)launchQueryImpl(info->pRequest, info->pQuery, true, NULL); // no need to check return code
launchQueryImpl(info->pRequest, info->pQuery, true, NULL); // no need to check return code
uDebug("SML:0x%" PRIx64 " smlInsertData end, format:%d, code:%d,%s", info->id, info->dataFormat, info->pRequest->code,
tstrerror(info->pRequest->code));
@ -1975,12 +1980,12 @@ static bool getLine(SSmlHandle *info, char *lines[], char **rawLine, char *rawLi
if (*rawLine != NULL && (uDebugFlag & DEBUG_DEBUG)) {
char *print = taosMemoryCalloc(*len + 1, 1);
if (print != NULL){
if (print != NULL) {
(void)memcpy(print, *tmp, *len);
uDebug("SML:0x%" PRIx64 " smlParseLine is raw, numLines:%d, protocol:%d, len:%d, data:%s", info->id, numLines,
info->protocol, *len, print);
taosMemoryFree(print);
} else{
} else {
uError("SML:0x%" PRIx64 " smlParseLine taosMemoryCalloc failed", info->id);
}
} else {
@ -2228,7 +2233,7 @@ TAOS_RES *taos_schemaless_insert_inner(TAOS *taos, char *lines[], char *rawLine,
uInfo("SML:%" PRIx64 " retry:%d/10,ver is old retry or object is creating code:%d, msg:%s", info->id, cnt, code,
tstrerror(code));
code = refreshMeta(request->pTscObj, request);
if (code != 0){
if (code != 0) {
uInfo("SML:%" PRIx64 " refresh meta error code:%d, msg:%s", info->id, code, tstrerror(code));
}
smlDestroyInfo(info);
@ -2266,7 +2271,7 @@ end:
*/
TAOS_RES *taos_schemaless_insert_ttl_with_reqid_tbname_key(TAOS *taos, char *lines[], int numLines, int protocol,
int precision, int32_t ttl, int64_t reqid, char *tbnameKey){
int precision, int32_t ttl, int64_t reqid, char *tbnameKey) {
return taos_schemaless_insert_inner(taos, lines, NULL, NULL, numLines, protocol, precision, ttl, reqid, tbnameKey);
}
@ -2306,14 +2311,17 @@ static void getRawLineLen(char *lines, int len, int32_t *totalRows, int protocol
}
TAOS_RES *taos_schemaless_insert_raw_ttl_with_reqid_tbname_key(TAOS *taos, char *lines, int len, int32_t *totalRows,
int protocol, int precision, int32_t ttl, int64_t reqid, char *tbnameKey){
int protocol, int precision, int32_t ttl, int64_t reqid,
char *tbnameKey) {
getRawLineLen(lines, len, totalRows, protocol);
return taos_schemaless_insert_inner(taos, NULL, lines, lines + len, *totalRows, protocol, precision, ttl, reqid, tbnameKey);
return taos_schemaless_insert_inner(taos, NULL, lines, lines + len, *totalRows, protocol, precision, ttl, reqid,
tbnameKey);
}
TAOS_RES *taos_schemaless_insert_raw_ttl_with_reqid(TAOS *taos, char *lines, int len, int32_t *totalRows, int protocol,
int precision, int32_t ttl, int64_t reqid) {
return taos_schemaless_insert_raw_ttl_with_reqid_tbname_key(taos, lines, len, totalRows, protocol, precision, ttl, reqid, NULL);
return taos_schemaless_insert_raw_ttl_with_reqid_tbname_key(taos, lines, len, totalRows, protocol, precision, ttl,
reqid, NULL);
}
TAOS_RES *taos_schemaless_insert_raw_with_reqid(TAOS *taos, char *lines, int len, int32_t *totalRows, int protocol,

23
source/client/src/clientStmt.c
View File

@ -25,7 +25,7 @@ static FORCE_INLINE int32_t stmtAllocQNodeFromBuf(STableBufInfo* pTblBuf, void**
return terrno;
}
if(taosArrayPush(pTblBuf->pBufList, &buff) == NULL){
if (taosArrayPush(pTblBuf->pBufList, &buff) == NULL) {
return terrno;
}
@ -224,8 +224,8 @@ int32_t stmtUpdateBindInfo(TAOS_STMT* stmt, STableMeta* pTableMeta, void* tags,
bool autoCreateTbl) {
STscStmt* pStmt = (STscStmt*)stmt;
char tbFName[TSDB_TABLE_FNAME_LEN];
int32_t code = tNameExtractFullName(tbName, tbFName);
if (code != 0){
int32_t code = tNameExtractFullName(tbName, tbFName);
if (code != 0) {
return code;
}
@ -772,7 +772,7 @@ void* stmtBindThreadFunc(void* param) {
}
int ret = stmtAsyncOutput(pStmt, asyncParam);
if (ret != 0){
if (ret != 0) {
qError("stmtAsyncOutput failed, reason:%s", tstrerror(ret));
}
}
@ -821,7 +821,7 @@ int32_t stmtInitTableBuf(STableBufInfo* pTblBuf) {
return terrno;
}
if (taosArrayPush(pTblBuf->pBufList, &buff) == NULL){
if (taosArrayPush(pTblBuf->pBufList, &buff) == NULL) {
return terrno;
}
@ -967,7 +967,7 @@ int32_t stmtInitStbInterlaceTableInfo(STscStmt* pStmt) {
}
int stmtSetDbName(TAOS_STMT* stmt, const char* dbName) {
STscStmt *pStmt = (STscStmt *) stmt;
STscStmt* pStmt = (STscStmt*)stmt;
STMT_DLOG("start to set dbName: %s", dbName);
@ -1045,7 +1045,7 @@ int stmtSetTbTags(TAOS_STMT* stmt, TAOS_MULTI_BIND* tags) {
STMT_ERR_RET(stmtSwitchStatus(pStmt, STMT_SETTAGS));
SBoundColInfo *tags_info = (SBoundColInfo*)pStmt->bInfo.boundTags;
SBoundColInfo* tags_info = (SBoundColInfo*)pStmt->bInfo.boundTags;
if (tags_info->numOfBound <= 0 || tags_info->numOfCols <= 0) {
tscWarn("no tags bound in sql, will not bound tags");
return TSDB_CODE_SUCCESS;
@ -1192,7 +1192,7 @@ static FORCE_INLINE int32_t stmtGetTableColsFromCache(STscStmt* pStmt, SArray**
return terrno;
}
if (taosArrayPush(pStmt->sql.siInfo.pTableCols, &pTblCols) == NULL){
if (taosArrayPush(pStmt->sql.siInfo.pTableCols, &pTblCols) == NULL) {
return terrno;
}
}
@ -1216,7 +1216,6 @@ int stmtBindBatch(TAOS_STMT* stmt, TAOS_MULTI_BIND* bind, int32_t colIdx) {
STMT_ERR_RET(stmtSwitchStatus(pStmt, STMT_BIND));
if (pStmt->bInfo.needParse && pStmt->sql.runTimes && pStmt->sql.type > 0 &&
STMT_TYPE_MULTI_INSERT != pStmt->sql.type) {
pStmt->bInfo.needParse = false;
@ -1256,7 +1255,7 @@ int stmtBindBatch(TAOS_STMT* stmt, TAOS_MULTI_BIND* bind, int32_t colIdx) {
if (pStmt->sql.pQuery->haveResultSet) {
STMT_ERR_RET(setResSchemaInfo(&pStmt->exec.pRequest->body.resInfo, pStmt->sql.pQuery->pResSchema,
pStmt->sql.pQuery->numOfResCols));
pStmt->sql.pQuery->numOfResCols));
taosMemoryFreeClear(pStmt->sql.pQuery->pResSchema);
setResPrecision(&pStmt->exec.pRequest->body.resInfo, pStmt->sql.pQuery->precision);
}
@ -1549,7 +1548,7 @@ int stmtExec(TAOS_STMT* stmt) {
STMT_ERR_RET(stmtSwitchStatus(pStmt, STMT_EXECUTE));
if (STMT_TYPE_QUERY == pStmt->sql.type) {
(void)launchQueryImpl(pStmt->exec.pRequest, pStmt->sql.pQuery, true, NULL);
launchQueryImpl(pStmt->exec.pRequest, pStmt->sql.pQuery, true, NULL);
} else {
if (pStmt->sql.stbInterlaceMode) {
int64_t startTs = taosGetTimestampUs();
@ -1571,7 +1570,7 @@ int stmtExec(TAOS_STMT* stmt) {
STMT_ERR_RET(qBuildStmtOutput(pStmt->sql.pQuery, pStmt->sql.pVgHash, pStmt->exec.pBlockHash));
}
(void)launchQueryImpl(pStmt->exec.pRequest, pStmt->sql.pQuery, true, NULL);
launchQueryImpl(pStmt->exec.pRequest, pStmt->sql.pQuery, true, NULL);
}
if (pStmt->exec.pRequest->code && NEED_CLIENT_HANDLE_ERROR(pStmt->exec.pRequest->code)) {

40
source/client/src/clientStmt2.c
View File

@ -698,7 +698,9 @@ static void* stmtBindThreadFunc(void* param) {
continue;
}
(void)stmtAsyncOutput(pStmt, asyncParam);
if (stmtAsyncOutput(pStmt, asyncParam) != 0) {
qError("stmt async output failed");
}
}
qInfo("stmt bind thread stopped");
@ -822,7 +824,11 @@ TAOS_STMT2* stmtInit2(STscObj* taos, TAOS_STMT2_OPTION* pOptions) {
pStmt->sql.siInfo.tableColsReady = true;
if (pStmt->options.asyncExecFn) {
(void)tsem_init(&pStmt->asyncQuerySem, 0, 1);
if (tsem_init(&pStmt->asyncQuerySem, 0, 1) != 0) {
terrno = TAOS_SYSTEM_ERROR(errno);
(void)stmtClose(pStmt);
return NULL;
}
}
pStmt->semWaited = false;
@ -1603,7 +1609,9 @@ static void asyncQueryCb(void* userdata, TAOS_RES* res, int code) {
(void)stmtCleanExecInfo(pStmt, (code ? false : true), false);
++pStmt->sql.runTimes;
(void)tsem_post(&pStmt->asyncQuerySem);
if (tsem_post(&pStmt->asyncQuerySem) != 0) {
tscError("failed to post asyncQuerySem");
}
}
int stmtExec2(TAOS_STMT2* stmt, int* affected_rows) {
@ -1645,7 +1653,7 @@ int stmtExec2(TAOS_STMT2* stmt, int* affected_rows) {
__taos_async_fn_t fp = pStmt->options.asyncExecFn;
if (!fp) {
(void)launchQueryImpl(pStmt->exec.pRequest, pStmt->sql.pQuery, true, NULL);
launchQueryImpl(pStmt->exec.pRequest, pStmt->sql.pQuery, true, NULL);
if (pStmt->exec.pRequest->code && NEED_CLIENT_HANDLE_ERROR(pStmt->exec.pRequest->code)) {
code = refreshMeta(pStmt->exec.pRequest->pTscObj, pStmt->exec.pRequest);
@ -1710,7 +1718,9 @@ int stmtClose2(TAOS_STMT2* stmt) {
}
if (pStmt->options.asyncExecFn && !pStmt->semWaited) {
(void)tsem_wait(&pStmt->asyncQuerySem);
if (tsem_wait(&pStmt->asyncQuerySem) != 0) {
tscError("failed to wait asyncQuerySem");
}
}
STMT_DLOG("stmt %p closed, stbInterlaceMode: %d, statInfo: ctgGetTbMetaNum=>%" PRId64 ", getCacheTbInfo=>%" PRId64
@ -1727,7 +1737,9 @@ int stmtClose2(TAOS_STMT2* stmt) {
STMT_ERR_RET(stmtCleanSQLInfo(pStmt));
if (pStmt->options.asyncExecFn) {
(void)tsem_destroy(&pStmt->asyncQuerySem);
if (tsem_destroy(&pStmt->asyncQuerySem) != 0) {
tscError("failed to destroy asyncQuerySem");
}
}
taosMemoryFree(stmt);
@ -1873,6 +1885,8 @@ int stmtGetParamNum2(TAOS_STMT2* stmt, int* nums) {
int stmtGetParamTbName(TAOS_STMT2* stmt, int* nums) {
STscStmt2* pStmt = (STscStmt2*)stmt;
int32_t code = 0;
int32_t preCode = pStmt->errCode;
STMT_DLOG_E("start to get param num");
@ -1895,17 +1909,19 @@ int stmtGetParamTbName(TAOS_STMT2* stmt, int* nums) {
STMT_ERR_RET(stmtCreateRequest(pStmt));
if (pStmt->bInfo.needParse) {
STMT_ERR_RET(stmtParseSql(pStmt));
STMT_ERRI_JRET(stmtParseSql(pStmt));
}
if (TSDB_CODE_TSC_STMT_TBNAME_ERROR == pStmt->errCode) {
*nums = STMT_TYPE_MULTI_INSERT == pStmt->sql.type ? 1 : 0;
_return:
if (TSDB_CODE_TSC_STMT_TBNAME_ERROR == code) {
*nums = 1;
pStmt->errCode = TSDB_CODE_SUCCESS;
} else {
*nums = STMT_TYPE_MULTI_INSERT == pStmt->sql.type ? 1 : 0;
code = TSDB_CODE_SUCCESS;
}
return TSDB_CODE_SUCCESS;
pStmt->errCode = preCode;
return code;
}
/*
int stmtGetParam(TAOS_STMT* stmt, int idx, int* type, int* bytes) {

51
source/common/src/cos.c
View File

@ -63,13 +63,10 @@ int32_t s3Begin() {
TAOS_RETURN(TSDB_CODE_SUCCESS);
}
void s3End() { (void)S3_deinitialize(); }
void s3End() { S3_deinitialize(); }
int32_t s3Init() { TAOS_RETURN(TSDB_CODE_SUCCESS); /*s3Begin();*/ }
void s3CleanUp() { /*s3End();*/
}
static int32_t s3ListBucket(char const *bucketname);
static void s3DumpCfgByEp(int8_t epIndex) {
@ -506,7 +503,9 @@ S3Status initial_multipart_callback(const char *upload_id, void *callbackData) {
}
S3Status MultipartResponseProperiesCallback(const S3ResponseProperties *properties, void *callbackData) {
(void)responsePropertiesCallbackNull(properties, callbackData);
if (S3StatusOK != responsePropertiesCallbackNull(properties, callbackData)) {
uError("%s failed at line %d to process null callback.", __func__, __LINE__);
}
MultipartPartData *data = (MultipartPartData *)callbackData;
int seq = data->seq;
@ -517,7 +516,9 @@ S3Status MultipartResponseProperiesCallback(const S3ResponseProperties *properti
}
S3Status MultipartResponseProperiesCallbackWithCp(const S3ResponseProperties *properties, void *callbackData) {
(void)responsePropertiesCallbackNull(properties, callbackData);
if (S3StatusOK != responsePropertiesCallbackNull(properties, callbackData)) {
uError("%s failed at line %d to process null callback.", __func__, __LINE__);
}
MultipartPartData *data = (MultipartPartData *)callbackData;
int seq = data->seq;
@ -897,8 +898,6 @@ upload:
if (partData.put_object_data.status != S3StatusOK) {
s3PrintError(__FILE__, __LINE__, __func__, partData.put_object_data.status, partData.put_object_data.err_msg);
TAOS_CHECK_GOTO(TAOS_SYSTEM_ERROR(EIO), &lino, _exit);
//(void)cos_cp_dump(&cp);
}
if (!manager.etags[seq - 1]) {
@ -952,7 +951,9 @@ _exit:
}
if (cp.thefile) {
(void)cos_cp_close(cp.thefile);
if (cos_cp_close(cp.thefile)) {
uError("%s failed at line %d to close cp file.", __func__, lino);
}
}
if (cp.parts) {
taosMemoryFree(cp.parts);
@ -1292,7 +1293,10 @@ int32_t s3DeleteObjects(const char *object_name[], int nobject) {
void s3DeleteObjectsByPrefix(const char *prefix) {
SArray *objectArray = getListByPrefix(prefix);
if (objectArray == NULL) return;
(void)s3DeleteObjects(TARRAY_DATA(objectArray), TARRAY_SIZE(objectArray));
int32_t code = s3DeleteObjects(TARRAY_DATA(objectArray), TARRAY_SIZE(objectArray));
if (!code) {
uError("%s failed at line %d since %s.", __func__, __LINE__, tstrerror(code));
}
taosArrayDestroyEx(objectArray, s3FreeObjectKey);
}
@ -1539,7 +1543,7 @@ int32_t s3Init() {
TAOS_RETURN(TSDB_CODE_SUCCESS);
}
void s3CleanUp() { cos_http_io_deinitialize(); }
// void s3CleanUp() { cos_http_io_deinitialize(); }
static void log_status(cos_status_t *s) {
cos_warn_log("status->code: %d", s->code);
@ -1745,20 +1749,20 @@ bool s3Get(const char *object_name, const char *path) {
cos_table_t *headers = NULL;
int traffic_limit = 0;
//创建内存池
// 创建内存池
cos_pool_create(&p, NULL);
//初始化请求选项
// 初始化请求选项
options = cos_request_options_create(p);
s3InitRequestOptions(options, is_cname);
cos_str_set(&bucket, tsS3BucketName);
if (traffic_limit) {
//限速值设置范围为819200 - 838860800，即100KB/s - 100MB/s，如果超出该范围将返回400错误
// 限速值设置范围为819200 - 838860800，即100KB/s - 100MB/s，如果超出该范围将返回400错误
headers = cos_table_make(p, 1);
cos_table_add_int(headers, "x-cos-traffic-limit", 819200);
}
//下载对象
// 下载对象
cos_str_set(&file, path);
cos_str_set(&object, object_name);
s = cos_get_object_to_file(options, &bucket, &object, headers, NULL, &file, &resp_headers);
@ -1769,7 +1773,7 @@ bool s3Get(const char *object_name, const char *path) {
cos_warn_log("get object failed\n");
}
//销毁内存池
// 销毁内存池
cos_pool_destroy(p);
return ret;
@ -1791,10 +1795,10 @@ int32_t s3GetObjectBlock(const char *object_name, int64_t offset, int64_t block_
// int traffic_limit = 0;
char range_buf[64];
//创建内存池
// 创建内存池
cos_pool_create(&p, NULL);
//初始化请求选项
// 初始化请求选项
options = cos_request_options_create(p);
// init_test_request_options(options, is_cname);
s3InitRequestOptions(options, is_cname);
@ -1843,7 +1847,7 @@ int32_t s3GetObjectBlock(const char *object_name, int64_t offset, int64_t block_
// cos_warn_log("Download data=%s", buf);
_exit:
//销毁内存池
// 销毁内存池
cos_pool_destroy(p);
*ppBlock = buf;
@ -1932,15 +1936,15 @@ long s3Size(const char *object_name) {
cos_string_t object;
cos_table_t *resp_headers = NULL;
//创建内存池
// 创建内存池
cos_pool_create(&p, NULL);
//初始化请求选项
// 初始化请求选项
options = cos_request_options_create(p);
s3InitRequestOptions(options, is_cname);
cos_str_set(&bucket, tsS3BucketName);
//获取对象元数据
// 获取对象元数据
cos_str_set(&object, object_name);
s = cos_head_object(options, &bucket, &object, NULL, &resp_headers);
// print_headers(resp_headers);
@ -1954,7 +1958,7 @@ long s3Size(const char *object_name) {
cos_warn_log("head object failed\n");
}
//销毁内存池
// 销毁内存池
cos_pool_destroy(p);
return size;
@ -1963,7 +1967,6 @@ long s3Size(const char *object_name) {
#else
int32_t s3Init() { return 0; }
void s3CleanUp() {}
int32_t s3PutObjectFromFile(const char *file, const char *object) { return 0; }
int32_t s3PutObjectFromFile2(const char *file, const char *object, int8_t withcp) { return 0; }
int32_t s3PutObjectFromFileOffset(const char *file, const char *object_name, int64_t offset, int64_t size) { return 0; }

2
source/common/src/cos_cp.c
View File

@ -309,7 +309,7 @@ int32_t cos_cp_dump(SCheckpoint* cp) {
if (!item) {
TAOS_CHECK_GOTO(TSDB_CODE_OUT_OF_MEMORY, &lino, _exit);
}
(void)cJSON_AddItemToArray(ajson, item);
if (!cJSON_AddItemToArray(ajson, item)) goto _exit;
if (NULL == cJSON_AddNumberToObject(item, "index", cp->parts[i].index)) {
TAOS_CHECK_GOTO(TSDB_CODE_OUT_OF_MEMORY, &lino, _exit);

4
source/common/src/tdataformat.c
View File

@ -4171,12 +4171,12 @@ int32_t tValueColumnInit(SValueColumn *valCol) {
return 0;
}
int32_t tValueColumnDestroy(SValueColumn *valCol) {
void tValueColumnDestroy(SValueColumn *valCol) {
valCol->type = TSDB_DATA_TYPE_NULL;
valCol->numOfValues = 0;
tBufferDestroy(&valCol->data);
tBufferDestroy(&valCol->offsets);
return 0;
return;
}
void tValueColumnClear(SValueColumn *valCol) {

51
source/common/src/tglobal.c
View File

@ -2300,11 +2300,14 @@ static int taosLogVarComp(void const *lp, void const *rp) {
return strcasecmp(lpVar->name, rpVar->name);
}
static int32_t taosCheckAndSetDebugFlag(int32_t *pFlagPtr, char *name, int32_t flag, SArray *noNeedToSetVars) {
static void taosCheckAndSetDebugFlag(int32_t *pFlagPtr, char *name, int32_t flag, SArray *noNeedToSetVars) {
if (noNeedToSetVars != NULL && taosArraySearch(noNeedToSetVars, name, taosLogVarComp, TD_EQ) != NULL) {
TAOS_RETURN(TSDB_CODE_SUCCESS);
return;
}
return taosSetDebugFlag(pFlagPtr, name, flag);
if (taosSetDebugFlag(pFlagPtr, name, flag) != 0) {
uError("failed to set flag %s to %d", name, flag);
}
return;
}
int32_t taosSetGlobalDebugFlag(int32_t flag) { return taosSetAllDebugFlag(tsCfg, flag); }
@ -2321,29 +2324,29 @@ static int32_t taosSetAllDebugFlag(SConfig *pCfg, int32_t flag) {
pItem->i32 = flag;
noNeedToSetVars = pItem->array;
(void)taosCheckAndSetDebugFlag(&simDebugFlag, "simDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&uDebugFlag, "uDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&rpcDebugFlag, "rpcDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&qDebugFlag, "qDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&simDebugFlag, "simDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&uDebugFlag, "uDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&rpcDebugFlag, "rpcDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&qDebugFlag, "qDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&jniDebugFlag, "jniDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&cDebugFlag, "cDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&jniDebugFlag, "jniDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&cDebugFlag, "cDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&dDebugFlag, "dDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&vDebugFlag, "vDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&mDebugFlag, "mDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&wDebugFlag, "wDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&sDebugFlag, "sDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&tsdbDebugFlag, "tsdbDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&tqDebugFlag, "tqDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&fsDebugFlag, "fsDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&udfDebugFlag, "udfDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&smaDebugFlag, "smaDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&idxDebugFlag, "idxDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&tdbDebugFlag, "tdbDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&metaDebugFlag, "metaDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&stDebugFlag, "stDebugFlag", flag, noNeedToSetVars);
(void)taosCheckAndSetDebugFlag(&sndDebugFlag, "sndDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&dDebugFlag, "dDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&vDebugFlag, "vDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&mDebugFlag, "mDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&wDebugFlag, "wDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&sDebugFlag, "sDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&tsdbDebugFlag, "tsdbDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&tqDebugFlag, "tqDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&fsDebugFlag, "fsDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&udfDebugFlag, "udfDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&smaDebugFlag, "smaDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&idxDebugFlag, "idxDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&tdbDebugFlag, "tdbDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&metaDebugFlag, "metaDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&stDebugFlag, "stDebugFlag", flag, noNeedToSetVars);
taosCheckAndSetDebugFlag(&sndDebugFlag, "sndDebugFlag", flag, noNeedToSetVars);
taosArrayClear(noNeedToSetVars); // reset array

6
source/common/src/trow.c
View File

@ -354,8 +354,10 @@ bool tdSKvRowGetVal(STSRow *pRow, col_id_t colId, col_id_t colIdx, SCellVal *pVa
}
void *pBitmap = tdGetBitmapAddrKv(pRow, tdRowGetNCols(pRow));
(void)tdGetKvRowValOfCol(pVal, pRow, pBitmap, pColIdx->offset,
POINTER_DISTANCE(pColIdx, TD_ROW_COL_IDX(pRow)) / sizeof(SKvRowIdx));
if (tdGetKvRowValOfCol(pVal, pRow, pBitmap, pColIdx->offset,
POINTER_DISTANCE(pColIdx, TD_ROW_COL_IDX(pRow)) / sizeof(SKvRowIdx)) != TSDB_CODE_SUCCESS) {
return false;
}
return true;
}

28
source/common/src/ttime.c
View File

@ -30,7 +30,9 @@ static int64_t m_deltaUtc = 0;
void deltaToUtcInitOnce() {
struct tm tm = {0};
(void)taosStrpTime("1970-01-01 00:00:00", (const char*)("%Y-%m-%d %H:%M:%S"), &tm);
if (taosStrpTime("1970-01-01 00:00:00", (const char*)("%Y-%m-%d %H:%M:%S"), &tm) != 0) {
uError("failed to parse time string");
}
m_deltaUtc = (int64_t)taosMktime(&tm);
// printf("====delta:%lld\n\n", seconds);
}
@ -689,10 +691,10 @@ int64_t taosTimeAdd(int64_t t, int64_t duration, char unit, int32_t precision) {
int64_t numOfMonth = (unit == 'y') ? duration * 12 : duration;
int64_t fraction = t % TSDB_TICK_PER_SECOND(precision);
struct tm tm;
time_t tt = (time_t)(t / TSDB_TICK_PER_SECOND(precision));
(void)taosLocalTime(&tt, &tm, NULL);
int32_t mon = tm.tm_year * 12 + tm.tm_mon + (int32_t)numOfMonth;
struct tm tm;
time_t tt = (time_t)(t / TSDB_TICK_PER_SECOND(precision));
struct tm* ptm = taosLocalTime(&tt, &tm, NULL);
int32_t mon = tm.tm_year * 12 + tm.tm_mon + (int32_t)numOfMonth;
tm.tm_year = mon / 12;
tm.tm_mon = mon % 12;
int daysOfMonth[] = {31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31};
@ -750,13 +752,13 @@ int32_t taosTimeCountIntervalForFill(int64_t skey, int64_t ekey, int64_t interva
skey /= (int64_t)(TSDB_TICK_PER_SECOND(precision));
ekey /= (int64_t)(TSDB_TICK_PER_SECOND(precision));
struct tm tm;
time_t t = (time_t)skey;
(void)taosLocalTime(&t, &tm, NULL);
int32_t smon = tm.tm_year * 12 + tm.tm_mon;
struct tm tm;
time_t t = (time_t)skey;
struct tm* ptm = taosLocalTime(&t, &tm, NULL);
int32_t smon = tm.tm_year * 12 + tm.tm_mon;
t = (time_t)ekey;
(void)taosLocalTime(&t, &tm, NULL);
ptm = taosLocalTime(&t, &tm, NULL);
int32_t emon = tm.tm_year * 12 + tm.tm_mon;
if (unit == 'y') {
@ -778,9 +780,9 @@ int64_t taosTimeTruncate(int64_t ts, const SInterval* pInterval) {
if (IS_CALENDAR_TIME_DURATION(pInterval->slidingUnit)) {
start /= (int64_t)(TSDB_TICK_PER_SECOND(precision));
struct tm tm;
time_t tt = (time_t)start;
(void)taosLocalTime(&tt, &tm, NULL);
struct tm tm;
time_t tt = (time_t)start;
struct tm* ptm = taosLocalTime(&tt, &tm, NULL);
tm.tm_sec = 0;
tm.tm_min = 0;
tm.tm_hour = 0;

77
source/dnode/mgmt/exe/dmMain.c
View File

@ -81,11 +81,21 @@ static void dmSetAssert(int32_t signum, void *sigInfo, void *context) { tsAssert
static void dmStopDnode(int signum, void *sigInfo, void *context) {
// taosIgnSignal(SIGUSR1);
// taosIgnSignal(SIGUSR2);
(void)taosIgnSignal(SIGTERM);
(void)taosIgnSignal(SIGHUP);
(void)taosIgnSignal(SIGINT);
(void)taosIgnSignal(SIGABRT);
(void)taosIgnSignal(SIGBREAK);
if (taosIgnSignal(SIGTERM) != 0) {
dWarn("failed to ignore signal SIGTERM");
}
if (taosIgnSignal(SIGHUP) != 0) {
dWarn("failed to ignore signal SIGHUP");
}
if (taosIgnSignal(SIGINT) != 0) {
dWarn("failed to ignore signal SIGINT");
}
if (taosIgnSignal(SIGABRT) != 0) {
dWarn("failed to ignore signal SIGABRT");
}
if (taosIgnSignal(SIGBREAK) != 0) {
dWarn("failed to ignore signal SIGBREAK");
}
dInfo("shut down signal is %d", signum);
#ifndef WINDOWS
@ -103,11 +113,19 @@ void dmLogCrash(int signum, void *sigInfo, void *context) {
// taosIgnSignal(SIGBREAK);
#ifndef WINDOWS
(void)taosIgnSignal(SIGBUS);
if (taosIgnSignal(SIGBUS) != 0) {
dWarn("failed to ignore signal SIGBUS");
}
#endif
(void)taosIgnSignal(SIGABRT);
(void)taosIgnSignal(SIGFPE);
(void)taosIgnSignal(SIGSEGV);
if (taosIgnSignal(SIGABRT) != 0) {
dWarn("failed to ignore signal SIGABRT");
}
if (taosIgnSignal(SIGFPE) != 0) {
dWarn("failed to ignore signal SIGABRT");
}
if (taosIgnSignal(SIGSEGV) != 0) {
dWarn("failed to ignore signal SIGABRT");
}
char *pMsg = NULL;
const char *flags = "UTL FATAL ";
@ -136,24 +154,31 @@ _return:
}
static void dmSetSignalHandle() {
(void)taosSetSignal(SIGUSR1, dmSetDebugFlag);
(void)taosSetSignal(SIGUSR2, dmSetAssert);
(void)taosSetSignal(SIGTERM, dmStopDnode);
(void)taosSetSignal(SIGHUP, dmStopDnode);
(void)taosSetSignal(SIGINT, dmStopDnode);
(void)taosSetSignal(SIGBREAK, dmStopDnode);
if (taosSetSignal(SIGUSR1, dmSetDebugFlag) != 0) {
dWarn("failed to set signal SIGUSR1");
}
if (taosSetSignal(SIGUSR2, dmSetAssert) != 0) {
dWarn("failed to set signal SIGUSR1");
}
if (taosSetSignal(SIGTERM, dmStopDnode) != 0) {
dWarn("failed to set signal SIGUSR1");
}
if (taosSetSignal(SIGHUP, dmStopDnode) != 0) {
dWarn("failed to set signal SIGUSR1");
}
if (taosSetSignal(SIGINT, dmStopDnode) != 0) {
dWarn("failed to set signal SIGUSR1");
}
if (taosSetSignal(SIGBREAK, dmStopDnode) != 0) {
dWarn("failed to set signal SIGUSR1");
}
#ifndef WINDOWS
(void)taosSetSignal(SIGTSTP, dmStopDnode);
(void)taosSetSignal(SIGQUIT, dmStopDnode);
#endif
#if 0
#ifndef WINDOWS
(void)taosSetSignal(SIGBUS, dmLogCrash);
#endif
(void)taosSetSignal(SIGABRT, dmLogCrash);
(void)taosSetSignal(SIGFPE, dmLogCrash);
(void)taosSetSignal(SIGSEGV, dmLogCrash);
if (taosSetSignal(SIGTSTP, dmStopDnode) != 0) {
dWarn("failed to set signal SIGUSR1");
}
if (taosSetSignal(SIGQUIT, dmStopDnode) != 0) {
dWarn("failed to set signal SIGUSR1");
}
#endif
}

16
source/dnode/mgmt/mgmt_dnode/src/dmHandle.c
View File

@ -45,7 +45,9 @@ static void dmMayShouldUpdateIpWhiteList(SDnodeMgmt *pMgmt, int64_t ver) {
if (pMgmt->pData->ipWhiteVer == ver) {
if (ver == 0) {
dDebug("disable ip-white-list on dnode ver: %" PRId64 ", status ver: %" PRId64 "", pMgmt->pData->ipWhiteVer, ver);
(void)rpcSetIpWhite(pMgmt->msgCb.serverRpc, NULL);
if (rpcSetIpWhite(pMgmt->msgCb.serverRpc, NULL) != 0) {
dError("failed to disable ip white list on dnode");
}
}
return;
}
@ -91,7 +93,9 @@ static void dmProcessStatusRsp(SDnodeMgmt *pMgmt, SRpcMsg *pRsp) {
dGInfo("dnode:%d, set to dropped since not exist in mnode, statusSeq:%d", pMgmt->pData->dnodeId,
pMgmt->statusSeq);
pMgmt->pData->dropped = 1;
(void)dmWriteEps(pMgmt->pData);
if (dmWriteEps(pMgmt->pData) != 0) {
dError("failed to write dnode file");
}
dInfo("dnode will exit since it is in the dropped state");
(void)raise(SIGINT);
}
@ -147,7 +151,9 @@ void dmSendStatusReq(SDnodeMgmt *pMgmt) {
req.clusterCfg.monitorParas.tsSlowLogThresholdTest = tsSlowLogThresholdTest;
tstrncpy(req.clusterCfg.monitorParas.tsSlowLogExceptDb, tsSlowLogExceptDb, TSDB_DB_NAME_LEN);
char timestr[32] = "1970-01-01 00:00:00.00";
(void)taosParseTime(timestr, &req.clusterCfg.checkTime, (int32_t)strlen(timestr), TSDB_TIME_PRECISION_MILLI, 0);
if (taosParseTime(timestr, &req.clusterCfg.checkTime, (int32_t)strlen(timestr), TSDB_TIME_PRECISION_MILLI, 0) != 0) {
dError("failed to parse time since %s", tstrerror(code));
}
memcpy(req.clusterCfg.timezone, tsTimezoneStr, TD_TIMEZONE_LEN);
memcpy(req.clusterCfg.locale, tsLocale, TD_LOCALE_LEN);
memcpy(req.clusterCfg.charset, tsCharset, TD_LOCALE_LEN);
@ -243,7 +249,9 @@ void dmSendNotifyReq(SDnodeMgmt *pMgmt, SNotifyReq *pReq) {
SEpSet epSet = {0};
dmGetMnodeEpSet(pMgmt->pData, &epSet);
(void)rpcSendRequest(pMgmt->msgCb.clientRpc, &epSet, &rpcMsg, NULL);
if (rpcSendRequest(pMgmt->msgCb.clientRpc, &epSet, &rpcMsg, NULL) != 0) {
dError("failed to send notify req");
}
}
int32_t dmProcessAuthRsp(SDnodeMgmt *pMgmt, SRpcMsg *pMsg) {

9
source/dnode/mgmt/mgmt_dnode/src/dmWorker.c
View File

@ -305,11 +305,16 @@ int32_t dmStartNotifyThread(SDnodeMgmt *pMgmt) {
void dmStopNotifyThread(SDnodeMgmt *pMgmt) {
if (taosCheckPthreadValid(pMgmt->notifyThread)) {
(void)tsem_post(&dmNotifyHdl.sem);
if (tsem_post(&dmNotifyHdl.sem) != 0) {
dError("failed to post notify sem");
}
(void)taosThreadJoin(pMgmt->notifyThread, NULL);
taosThreadClear(&pMgmt->notifyThread);
}
(void)tsem_destroy(&dmNotifyHdl.sem);
if (tsem_destroy(&dmNotifyHdl.sem) != 0) {
dError("failed to destroy notify sem");
}
}
int32_t dmStartMonitorThread(SDnodeMgmt *pMgmt) {

4
source/dnode/mgmt/mgmt_mnode/src/mmHandle.c
View File

@ -17,7 +17,9 @@
#include "mmInt.h"
void mmGetMonitorInfo(SMnodeMgmt *pMgmt, SMonMmInfo *pInfo) {
(void)mndGetMonitorInfo(pMgmt->pMnode, &pInfo->cluster, &pInfo->vgroup, &pInfo->stb, &pInfo->grant);
if (mndGetMonitorInfo(pMgmt->pMnode, &pInfo->cluster, &pInfo->vgroup, &pInfo->stb, &pInfo->grant) != 0) {
dError("failed to get monitor info");
}
}
void mmGetMnodeLoads(SMnodeMgmt *pMgmt, SMonMloadInfo *pInfo) {

2
source/dnode/mgmt/mgmt_qnode/src/qmWorker.c
View File

@ -23,7 +23,7 @@ static inline void qmSendRsp(SRpcMsg *pMsg, int32_t code) {
.contLen = pMsg->info.rspLen,
.info = pMsg->info,
};
(void)tmsgSendRsp(&rsp);
tmsgSendRsp(&rsp);
}
static void qmProcessQueue(SQueueInfo *pInfo, SRpcMsg *pMsg) {

6
source/dnode/mgmt/mgmt_snode/src/smWorker.c
View File

@ -23,15 +23,15 @@ static inline void smSendRsp(SRpcMsg *pMsg, int32_t code) {
.contLen = pMsg->info.rspLen,
.info = pMsg->info,
};
(void)tmsgSendRsp(&rsp);
tmsgSendRsp(&rsp);
}
static void smProcessWriteQueue(SQueueInfo *pInfo, STaosQall *qall, int32_t numOfMsgs) {
SSnodeMgmt *pMgmt = pInfo->ahandle;
for (int32_t i = 0; i < numOfMsgs; i++) {
SRpcMsg *pMsg = NULL;
(void)taosGetQitem(qall, (void **)&pMsg);
SRpcMsg *pMsg = NULL;
int32_t num = taosGetQitem(qall, (void **)&pMsg);
const STraceId *trace = &pMsg->info.traceId;
dTrace("msg:%p, get from snode-write queue", pMsg);

16
source/dnode/mgmt/mgmt_vnode/src/vmHandle.c
View File

@ -35,10 +35,14 @@ void vmGetVnodeLoads(SVnodeMgmt *pMgmt, SMonVloadInfo *pInfo, bool isReset) {
SVnodeObj *pVnode = *ppVnode;
SVnodeLoad vload = {.vgId = pVnode->vgId};
if (!pVnode->failed) {
(void)vnodeGetLoad(pVnode->pImpl, &vload);
if (vnodeGetLoad(pVnode->pImpl, &vload) != 0) {
dError("failed to get vnode load");
}
if (isReset) vnodeResetLoad(pVnode->pImpl, &vload);
}
(void)taosArrayPush(pInfo->pVloads, &vload);
if (taosArrayPush(pInfo->pVloads, &vload) == NULL) {
dError("failed to push vnode load");
}
pIter = taosHashIterate(pMgmt->hash, pIter);
}
@ -116,7 +120,9 @@ void vmGetMonitorInfo(SVnodeMgmt *pMgmt, SMonVmInfo *pInfo) {
pMgmt->state.numOfBatchInsertReqs = numOfBatchInsertReqs;
pMgmt->state.numOfBatchInsertSuccessReqs = numOfBatchInsertSuccessReqs;
(void)tfsGetMonitorInfo(pMgmt->pTfs, &pInfo->tfs);
if (tfsGetMonitorInfo(pMgmt->pTfs, &pInfo->tfs) != 0) {
dError("failed to get tfs monitor info");
}
taosArrayDestroy(pVloads);
}
@ -845,7 +851,9 @@ int32_t vmProcessDropVnodeReq(SVnodeMgmt *pMgmt, SRpcMsg *pMsg) {
}
vmCloseVnode(pMgmt, pVnode, false);
(void)vmWriteVnodeListToFile(pMgmt);
if (vmWriteVnodeListToFile(pMgmt) != 0) {
dError("vgId:%d, failed to write vnode list since %s", vgId, terrstr());
}
dInfo("vgId:%d, is dropped", vgId);
return 0;

24
source/dnode/mgmt/mgmt_vnode/src/vmInt.c
View File

@ -24,7 +24,7 @@ int32_t vmGetPrimaryDisk(SVnodeMgmt *pMgmt, int32_t vgId) {
SVnodeObj *pVnode = NULL;
(void)taosThreadRwlockRdlock(&pMgmt->lock);
(void)taosHashGetDup(pMgmt->hash, &vgId, sizeof(int32_t), (void *)&pVnode);
int32_t r = taosHashGetDup(pMgmt->hash, &vgId, sizeof(int32_t), (void *)&pVnode);
if (pVnode != NULL) {
diskId = pVnode->diskPrimary;
}
@ -97,7 +97,7 @@ SVnodeObj *vmAcquireVnodeImpl(SVnodeMgmt *pMgmt, int32_t vgId, bool strict) {
SVnodeObj *pVnode = NULL;
(void)taosThreadRwlockRdlock(&pMgmt->lock);
(void)taosHashGetDup(pMgmt->hash, &vgId, sizeof(int32_t), (void *)&pVnode);
int32_t r = taosHashGetDup(pMgmt->hash, &vgId, sizeof(int32_t), (void *)&pVnode);
if (pVnode == NULL || strict && (pVnode->dropped || pVnode->failed)) {
terrno = TSDB_CODE_VND_INVALID_VGROUP_ID;
pVnode = NULL;
@ -165,7 +165,7 @@ int32_t vmOpenVnode(SVnodeMgmt *pMgmt, SWrapperCfg *pCfg, SVnode *pImpl) {
(void)taosThreadRwlockWrlock(&pMgmt->lock);
SVnodeObj *pOld = NULL;
(void)taosHashGetDup(pMgmt->hash, &pVnode->vgId, sizeof(int32_t), (void *)&pOld);
int32_t r = taosHashGetDup(pMgmt->hash, &pVnode->vgId, sizeof(int32_t), (void *)&pOld);
if (pOld) {
vmFreeVnodeObj(&pOld);
}
@ -184,7 +184,7 @@ void vmCloseVnode(SVnodeMgmt *pMgmt, SVnodeObj *pVnode, bool commitAndRemoveWal)
}
(void)taosThreadRwlockWrlock(&pMgmt->lock);
(void)taosHashRemove(pMgmt->hash, &pVnode->vgId, sizeof(int32_t));
int32_t r = taosHashRemove(pMgmt->hash, &pVnode->vgId, sizeof(int32_t));
(void)taosThreadRwlockUnlock(&pMgmt->lock);
vmReleaseVnode(pMgmt, pVnode);
@ -233,8 +233,12 @@ void vmCloseVnode(SVnodeMgmt *pMgmt, SVnodeObj *pVnode, bool commitAndRemoveWal)
if (commitAndRemoveWal) {
dInfo("vgId:%d, commit data for vnode split", pVnode->vgId);
(void)vnodeSyncCommit(pVnode->pImpl);
(void)vnodeBegin(pVnode->pImpl);
if (vnodeSyncCommit(pVnode->pImpl) != 0) {
dError("vgId:%d, failed to commit data", pVnode->vgId);
}
if (vnodeBegin(pVnode->pImpl) != 0) {
dError("vgId:%d, failed to begin", pVnode->vgId);
}
dInfo("vgId:%d, commit data finished", pVnode->vgId);
}
@ -248,8 +252,12 @@ _closed:
if (commitAndRemoveWal) {
snprintf(path, TSDB_FILENAME_LEN, "vnode%svnode%d%swal", TD_DIRSEP, pVnode->vgId, TD_DIRSEP);
dInfo("vgId:%d, remove all wals, path:%s", pVnode->vgId, path);
(void)tfsRmdir(pMgmt->pTfs, path);
(void)tfsMkdir(pMgmt->pTfs, path);
if (tfsRmdir(pMgmt->pTfs, path) != 0) {
dTrace("vgId:%d, failed to remove wals, path:%s", pVnode->vgId, path);
}
if (tfsMkdir(pMgmt->pTfs, path) != 0) {
dTrace("vgId:%d, failed to create wals, path:%s", pVnode->vgId, path);
}
}
if (pVnode->dropped) {

30
source/dnode/mgmt/mgmt_vnode/src/vmWorker.c
View File

@ -187,7 +187,9 @@ static void vmProcessSyncQueue(SQueueInfo *pInfo, STaosQall *qall, int32_t numOf
static void vmSendResponse(SRpcMsg *pMsg) {
if (pMsg->info.handle) {
SRpcMsg rsp = {.info = pMsg->info, .code = terrno};
(void)rpcSendResponse(&rsp);
if (rpcSendResponse(&rsp) != 0) {
dError("failed to send response since %s", terrstr());
}
}
}
@ -389,10 +391,28 @@ int32_t vmAllocQueue(SVnodeMgmt *pMgmt, SVnodeObj *pVnode) {
SMultiWorkerCfg scfg = {.max = 1, .name = "vnode-sync", .fp = (FItems)vmProcessSyncQueue, .param = pVnode};
SMultiWorkerCfg sccfg = {.max = 1, .name = "vnode-sync-rd", .fp = (FItems)vmProcessSyncQueue, .param = pVnode};
SMultiWorkerCfg acfg = {.max = 1, .name = "vnode-apply", .fp = (FItems)vnodeApplyWriteMsg, .param = pVnode->pImpl};
(void)tMultiWorkerInit(&pVnode->pWriteW, &wcfg);
(void)tMultiWorkerInit(&pVnode->pSyncW, &scfg);
(void)tMultiWorkerInit(&pVnode->pSyncRdW, &sccfg);
(void)tMultiWorkerInit(&pVnode->pApplyW, &acfg);
code = tMultiWorkerInit(&pVnode->pWriteW, &wcfg);
if (code) {
return code;
}
code = tMultiWorkerInit(&pVnode->pSyncW, &scfg);
if (code) {
tMultiWorkerCleanup(&pVnode->pWriteW);
return code;
}
code = tMultiWorkerInit(&pVnode->pSyncRdW, &sccfg);
if (code) {
tMultiWorkerCleanup(&pVnode->pWriteW);
tMultiWorkerCleanup(&pVnode->pSyncW);
return code;
}
code = tMultiWorkerInit(&pVnode->pApplyW, &acfg);
if (code) {
tMultiWorkerCleanup(&pVnode->pWriteW);
tMultiWorkerCleanup(&pVnode->pSyncW);
tMultiWorkerCleanup(&pVnode->pSyncRdW);
return code;
}
pVnode->pQueryQ = tQueryAutoQWorkerAllocQueue(&pMgmt->queryPool, pVnode, (FItem)vmProcessQueryQueue);
pVnode->pStreamQ = tAutoQWorkerAllocQueue(&pMgmt->streamPool, pVnode, (FItem)vmProcessStreamQueue);

14
source/dnode/mgmt/node_mgmt/src/dmEnv.c
View File

@ -47,8 +47,14 @@ static int32_t dmCheckRepeatInit(SDnode *pDnode) {
}
static int32_t dmInitSystem() {
(void)taosIgnSIGPIPE();
(void)taosBlockSIGPIPE();
if (taosIgnSIGPIPE() != 0) {
dError("failed to ignore SIGPIPE");
}
if (taosBlockSIGPIPE() != 0) {
dError("failed to block SIGPIPE");
}
taosResolveCRC();
return 0;
}
@ -204,7 +210,9 @@ void dmCleanup() {
auditCleanup();
syncCleanUp();
walCleanUp();
(void)udfcClose();
if (udfcClose() != 0) {
dError("failed to close udfc");
}
udfStopUdfd();
taosStopCacheRefreshWorker();
(void)dmDiskClose();

23
source/dnode/mgmt/node_mgmt/src/dmMgmt.c
View File

@ -47,8 +47,7 @@ int32_t dmInitDnode(SDnode *pDnode) {
}
// compress module init
(void)tsCompressInit(tsLossyColumns, tsFPrecision, tsDPrecision, tsMaxRange, tsCurRange, (int)tsIfAdtFse,
tsCompressor);
tsCompressInit(tsLossyColumns, tsFPrecision, tsDPrecision, tsMaxRange, tsCurRange, (int)tsIfAdtFse, tsCompressor);
pDnode->wrappers[DNODE].func = dmGetMgmtFunc();
pDnode->wrappers[MNODE].func = mmGetMgmtFunc();
@ -226,7 +225,10 @@ void dmClearVars(SDnode *pDnode) {
(void)taosThreadRwlockDestroy(&pWrapper->lock);
}
if (pDnode->lockfile != NULL) {
(void)taosUnLockFile(pDnode->lockfile);
if (taosUnLockFile(pDnode->lockfile) != 0) {
dError("failed to unlock file");
}
(void)taosCloseFile(&pDnode->lockfile);
pDnode->lockfile = NULL;
}
@ -343,7 +345,9 @@ void dmProcessNetTestReq(SDnode *pDnode, SRpcMsg *pMsg) {
rsp.contLen = pMsg->contLen;
}
(void)rpcSendResponse(&rsp);
if (rpcSendResponse(&rsp) != 0) {
dError("failed to send response, msg:%p", &rsp);
}
rpcFreeCont(pMsg->pCont);
}
@ -360,11 +364,16 @@ void dmProcessServerStartupStatus(SDnode *pDnode, SRpcMsg *pMsg) {
} else {
rsp.pCont = rpcMallocCont(contLen);
if (rsp.pCont != NULL) {
(void)tSerializeSServerStatusRsp(rsp.pCont, contLen, &statusRsp);
rsp.contLen = contLen;
if (tSerializeSServerStatusRsp(rsp.pCont, contLen, &statusRsp) < 0) {
rsp.code = TSDB_CODE_APP_ERROR;
} else {
rsp.contLen = contLen;
}
}
}
(void)rpcSendResponse(&rsp);
if (rpcSendResponse(&rsp) != 0) {
dError("failed to send response, msg:%p", &rsp);
}
rpcFreeCont(pMsg->pCont);
}

44
source/dnode/mgmt/node_mgmt/src/dmTransport.c
View File

@ -18,7 +18,11 @@
#include "qworker.h"
#include "tversion.h"
static inline void dmSendRsp(SRpcMsg *pMsg) { (void)rpcSendResponse(pMsg); }
static inline void dmSendRsp(SRpcMsg *pMsg) {
if (rpcSendResponse(pMsg) != 0) {
dError("failed to send response, msg:%p", pMsg);
}
}
static inline void dmBuildMnodeRedirectRsp(SDnode *pDnode, SRpcMsg *pMsg) {
SEpSet epSet = {0};
@ -113,7 +117,11 @@ static void dmProcessRpcMsg(SDnode *pDnode, SRpcMsg *pRpc, SEpSet *pEpSet) {
pRpc->info.handle, pRpc->contLen, pRpc->code, pRpc->info.ahandle, pRpc->info.refId);
int32_t svrVer = 0;
(void)taosVersionStrToInt(version, &svrVer);
code = taosVersionStrToInt(version, &svrVer);
if (code != 0) {
dError("failed to convert version string:%s to int, code:%d", version, code);
goto _OVER;
}
if ((code = taosCheckVersionCompatible(pRpc->info.cliVer, svrVer, 3)) != 0) {
dError("Version not compatible, cli ver: %d, svr ver: %d, ip:0x%x", pRpc->info.cliVer, svrVer,
pRpc->info.conn.clientIp);
@ -253,7 +261,9 @@ _OVER:
if (pWrapper != NULL) {
dmSendRsp(&rsp);
} else {
(void)rpcSendResponse(&rsp);
if (rpcSendResponse(&rsp) != 0) {
dError("failed to send response, msg:%p", &rsp);
}
}
}
@ -310,7 +320,9 @@ static inline int32_t dmSendReq(const SEpSet *pEpSet, SRpcMsg *pMsg) {
return code;
} else {
pMsg->info.handle = 0;
(void)rpcSendRequest(pDnode->trans.clientRpc, pEpSet, pMsg, NULL);
if (rpcSendRequest(pDnode->trans.clientRpc, pEpSet, pMsg, NULL) != 0) {
dError("failed to send rpc msg");
}
return 0;
}
}
@ -396,9 +408,12 @@ int32_t dmInitClient(SDnode *pDnode) {
rpcInit.shareConn = 1;
rpcInit.timeToGetConn = tsTimeToGetAvailableConn;
rpcInit.notWaitAvaliableConn = 0;
(void)taosVersionStrToInt(version, &(rpcInit.compatibilityVer));
rpcInit.startReadTimer = 1;
if (taosVersionStrToInt(version, &(rpcInit.compatibilityVer)) != 0) {
dError("failed to convert version string:%s to int", version);
}
pTrans->clientRpc = rpcOpen(&rpcInit);
if (pTrans->clientRpc == NULL) {
dError("failed to init dnode rpc client since:%s", tstrerror(terrno));
@ -442,7 +457,10 @@ int32_t dmInitStatusClient(SDnode *pDnode) {
rpcInit.batchSize = 8 * 1024;
rpcInit.timeToGetConn = tsTimeToGetAvailableConn;
rpcInit.startReadTimer = 1;
(void)taosVersionStrToInt(version, &(rpcInit.compatibilityVer));
if (taosVersionStrToInt(version, &(rpcInit.compatibilityVer)) != 0) {
dError("failed to convert version string:%s to int", version);
}
pTrans->statusRpc = rpcOpen(&rpcInit);
if (pTrans->statusRpc == NULL) {
@ -487,8 +505,12 @@ int32_t dmInitSyncClient(SDnode *pDnode) {
rpcInit.supportBatch = 1;
rpcInit.batchSize = 8 * 1024;
rpcInit.timeToGetConn = tsTimeToGetAvailableConn;
(void)taosVersionStrToInt(version, &(rpcInit.compatibilityVer));
rpcInit.startReadTimer = 1;
if (taosVersionStrToInt(version, &(rpcInit.compatibilityVer)) != 0) {
dError("failed to convert version string:%s to int", version);
}
pTrans->syncRpc = rpcOpen(&rpcInit);
if (pTrans->syncRpc == NULL) {
dError("failed to init dnode rpc sync client since %s", tstrerror(terrno));
@ -538,7 +560,11 @@ int32_t dmInitServer(SDnode *pDnode) {
rpcInit.idleTime = tsShellActivityTimer * 1000;
rpcInit.parent = pDnode;
rpcInit.compressSize = tsCompressMsgSize;
(void)taosVersionStrToInt(version, &(rpcInit.compatibilityVer));
if (taosVersionStrToInt(version, &(rpcInit.compatibilityVer)) != 0) {
dError("failed to convert version string:%s to int", version);
}
pTrans->serverRpc = rpcOpen(&rpcInit);
if (pTrans->serverRpc == NULL) {
dError("failed to init dnode rpc server since:%s", tstrerror(terrno));

22
source/dnode/mgmt/node_util/src/dmEps.c
View File

@ -259,7 +259,9 @@ _OVER:
if (taosArrayGetSize(pData->dnodeEps) == 0) {
SDnodeEp dnodeEp = {0};
dnodeEp.isMnode = 1;
(void)taosGetFqdnPortFromEp(tsFirst, &dnodeEp.ep);
if (taosGetFqdnPortFromEp(tsFirst, &dnodeEp.ep) != 0) {
dError("failed to get fqdn and port from ep:%s", tsFirst);
}
if (taosArrayPush(pData->dnodeEps, &dnodeEp) == NULL) {
return terrno;
}
@ -370,11 +372,19 @@ int32_t dmGetDnodeSize(SDnodeData *pData) {
}
void dmUpdateEps(SDnodeData *pData, SArray *eps) {
(void)taosThreadRwlockWrlock(&pData->lock);
if (taosThreadRwlockWrlock(&pData->lock) != 0) {
dError("failed to lock dnode lock");
}
dDebug("new dnode list get from mnode, dnodeVer:%" PRId64, pData->dnodeVer);
dmResetEps(pData, eps);
(void)dmWriteEps(pData);
(void)taosThreadRwlockUnlock(&pData->lock);
if (dmWriteEps(pData) != 0) {
dError("failed to write dnode file");
}
if (taosThreadRwlockUnlock(&pData->lock) != 0) {
dError("failed to unlock dnode lock");
}
}
static void dmResetEps(SDnodeData *pData, SArray *dnodeEps) {
@ -590,7 +600,9 @@ void dmRemoveDnodePairs(SDnodeData *pData) {
snprintf(file, sizeof(file), "%s%sdnode%sep.json", tsDataDir, TD_DIRSEP, TD_DIRSEP);
snprintf(bak, sizeof(bak), "%s%sdnode%sep.json.bak", tsDataDir, TD_DIRSEP, TD_DIRSEP);
dInfo("dnode file:%s is rename to bak file", file);
(void)taosRenameFile(file, bak);
if (taosRenameFile(file, bak) != 0) {
dError("failed to rename dnode file:%s to bak file:%s since %s", file, bak, tstrerror(terrno));
}
}
static int32_t dmReadDnodePairs(SDnodeData *pData) {

1
source/dnode/mnode/impl/inc/mndDef.h
View File

@ -106,6 +106,7 @@ typedef enum {
// TRN_CONFLICT_TOPIC = 4,
// TRN_CONFLICT_TOPIC_INSIDE = 5,
TRN_CONFLICT_ARBGROUP = 6,
TRN_CONFLICT_TSMA = 7,
} ETrnConflct;
typedef enum {

4
source/dnode/mnode/impl/src/mndDb.c
View File

@ -2281,6 +2281,10 @@ static void mndDumpDbInfoData(SMnode *pMnode, SSDataBlock *pBlock, SDbObj *pDb,
int32_t cols = 0;
int32_t bytes = pShow->pMeta->pSchemas[cols].bytes;
char *buf = taosMemoryMalloc(bytes);
if (buf == NULL) {
mError("db:%s, failed to malloc buffer", pDb->name);
return;
}
int32_t code = 0;
int32_t lino = 0;

20
source/dnode/mnode/impl/src/mndFunc.c
View File

@ -184,6 +184,7 @@ static int32_t mndFuncActionDelete(SSdb *pSdb, SFuncObj *pFunc) {
}
static int32_t mndFuncActionUpdate(SSdb *pSdb, SFuncObj *pOld, SFuncObj *pNew) {
int32_t code = 0;
mTrace("func:%s, perform update action, old row:%p new row:%p", pOld->name, pOld, pNew);
taosWLockLatch(&pOld->lock);
@ -205,6 +206,11 @@ static int32_t mndFuncActionUpdate(SSdb *pSdb, SFuncObj *pOld, SFuncObj *pNew) {
if (pNew->commentSize > 0 && pNew->pComment != NULL) {
pOld->commentSize = pNew->commentSize;
pOld->pComment = taosMemoryMalloc(pOld->commentSize);
if (pOld->pComment == NULL) {
code = terrno;
taosWUnLockLatch(&pOld->lock);
return code;
}
(void)memcpy(pOld->pComment, pNew->pComment, pOld->commentSize);
}
@ -215,6 +221,11 @@ static int32_t mndFuncActionUpdate(SSdb *pSdb, SFuncObj *pOld, SFuncObj *pNew) {
if (pNew->codeSize > 0 && pNew->pCode != NULL) {
pOld->codeSize = pNew->codeSize;
pOld->pCode = taosMemoryMalloc(pOld->codeSize);
if (pOld->pCode == NULL) {
code = terrno;
taosWUnLockLatch(&pOld->lock);
return code;
}
(void)memcpy(pOld->pCode, pNew->pCode, pOld->codeSize);
}
@ -261,6 +272,10 @@ static int32_t mndCreateFunc(SMnode *pMnode, SRpcMsg *pReq, SCreateFuncReq *pCre
if (NULL != pCreate->pComment) {
func.commentSize = strlen(pCreate->pComment) + 1;
func.pComment = taosMemoryMalloc(func.commentSize);
if (func.pComment == NULL) {
code = terrno;
goto _OVER;
}
}
func.codeSize = pCreate->codeLen;
func.pCode = taosMemoryMalloc(func.codeSize);
@ -716,6 +731,11 @@ static int32_t mndRetrieveFuncs(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBl
? TSDB_MAX_BINARY_LEN
: pFunc->codeSize + VARSTR_HEADER_SIZE;
char *b4 = taosMemoryMalloc(varCodeLen);
if (b4 == NULL) {
code = terrno;
sdbRelease(pSdb, pFunc);
TAOS_RETURN(code);
}
(void)memcpy(varDataVal(b4), pFunc->pCode, varCodeLen - VARSTR_HEADER_SIZE);
varDataSetLen(b4, varCodeLen - VARSTR_HEADER_SIZE);
code = colDataSetVal(pColInfo, numOfRows, (const char *)b4, false);

16
source/dnode/mnode/impl/src/mndMnode.c
View File

@ -343,6 +343,10 @@ static int32_t mndBuildCreateMnodeRedoAction(STrans *pTrans, SDCreateMnodeReq *p
int32_t code = 0;
int32_t contLen = tSerializeSDCreateMnodeReq(NULL, 0, pCreateReq);
void *pReq = taosMemoryMalloc(contLen);
if (pReq == NULL) {
code = terrno;
return code;
}
code = tSerializeSDCreateMnodeReq(pReq, contLen, pCreateReq);
if (code < 0) {
taosMemoryFree(pReq);
@ -369,6 +373,10 @@ static int32_t mndBuildAlterMnodeTypeRedoAction(STrans *pTrans, SDAlterMnodeType
int32_t code = 0;
int32_t contLen = tSerializeSDCreateMnodeReq(NULL, 0, pAlterMnodeTypeReq);
void *pReq = taosMemoryMalloc(contLen);
if (pReq == NULL) {
code = terrno;
return code;
}
code = tSerializeSDCreateMnodeReq(pReq, contLen, pAlterMnodeTypeReq);
if (code < 0) {
taosMemoryFree(pReq);
@ -395,6 +403,10 @@ static int32_t mndBuildAlterMnodeRedoAction(STrans *pTrans, SDCreateMnodeReq *pA
int32_t code = 0;
int32_t contLen = tSerializeSDCreateMnodeReq(NULL, 0, pAlterReq);
void *pReq = taosMemoryMalloc(contLen);
if (pReq == NULL) {
code = terrno;
return code;
}
code = tSerializeSDCreateMnodeReq(pReq, contLen, pAlterReq);
if (code < 0) {
taosMemoryFree(pReq);
@ -420,6 +432,10 @@ static int32_t mndBuildDropMnodeRedoAction(STrans *pTrans, SDDropMnodeReq *pDrop
int32_t code = 0;
int32_t contLen = tSerializeSCreateDropMQSNodeReq(NULL, 0, pDropReq);
void *pReq = taosMemoryMalloc(contLen);
if (pReq == NULL) {
code = terrno;
return code;
}
code = tSerializeSCreateDropMQSNodeReq(pReq, contLen, pDropReq);
if (code < 0) {
taosMemoryFree(pReq);

4
source/dnode/mnode/impl/src/mndSma.c
View File

@ -1692,7 +1692,7 @@ static int32_t mndCreateTSMATxnPrepare(SCreateTSMACxt* pCxt) {
STransAction dropStbUndoAction = {0};
SMDropStbReq dropStbReq = {0};
STrans *pTrans =
mndTransCreate(pCxt->pMnode, TRN_POLICY_ROLLBACK, TRN_CONFLICT_NOTHING, pCxt->pRpcReq, "create-tsma");
mndTransCreate(pCxt->pMnode, TRN_POLICY_ROLLBACK, TRN_CONFLICT_TSMA, pCxt->pRpcReq, "create-tsma");
if (!pTrans) {
code = terrno;
goto _OVER;
@ -1974,7 +1974,7 @@ _OVER:
static int32_t mndDropTSMA(SCreateTSMACxt* pCxt) {
int32_t code = -1;
STransAction dropStreamRedoAction = {0};
STrans *pTrans = mndTransCreate(pCxt->pMnode, TRN_POLICY_RETRY, TRN_CONFLICT_NOTHING, pCxt->pRpcReq, "drop-tsma");
STrans *pTrans = mndTransCreate(pCxt->pMnode, TRN_POLICY_RETRY, TRN_CONFLICT_TSMA, pCxt->pRpcReq, "drop-tsma");
if (!pTrans) {
code = terrno;
goto _OVER;

10
source/dnode/mnode/impl/src/mndStb.c
View File

@ -2352,6 +2352,11 @@ static int32_t mndBuildSMAlterStbRsp(SDbObj *pDb, SStbObj *pObj, void **pCont, i
}
void *cont = taosMemoryMalloc(contLen);
if (NULL == cont) {
code = terrno;
tFreeSMAlterStbRsp(&alterRsp);
TAOS_RETURN(code);
}
tEncoderInit(&ec, cont, contLen);
code = tEncodeSMAlterStbRsp(&ec, &alterRsp);
tEncoderClear(&ec);
@ -2407,6 +2412,11 @@ int32_t mndBuildSMCreateStbRsp(SMnode *pMnode, char *dbFName, char *stbFName, vo
}
void *cont = taosMemoryMalloc(contLen);
if (NULL == cont) {
code = terrno;
tFreeSMCreateStbRsp(&stbRsp);
goto _OVER;
}
tEncoderInit(&ec, cont, contLen);
TAOS_CHECK_GOTO(tEncodeSMCreateStbRsp(&ec, &stbRsp), NULL, _OVER);
tEncoderClear(&ec);

9
source/dnode/mnode/impl/src/mndTrans.c
View File

@ -367,6 +367,7 @@ SSdbRow *mndTransDecode(SSdbRaw *pRaw) {
SDB_GET_INT32(pRaw, dataPos, &pTrans->paramLen, _OVER)
if (pTrans->paramLen != 0) {
pTrans->param = taosMemoryMalloc(pTrans->paramLen);
if (pTrans->param == NULL) goto _OVER;
SDB_GET_BINARY(pRaw, dataPos, pTrans->param, pTrans->paramLen, _OVER);
}
@ -902,6 +903,14 @@ static bool mndCheckTransConflict(SMnode *pMnode, STrans *pNew) {
}
}
if (pNew->conflict == TRN_CONFLICT_TSMA) {
if (pTrans->conflict == TRN_CONFLICT_GLOBAL || pTrans->conflict == TRN_CONFLICT_TSMA) {
mndTransLogConflict(pNew, pTrans, true, &conflict);
} else {
mndTransLogConflict(pNew, pTrans, false, &conflict);
}
}
sdbRelease(pMnode->pSdb, pTrans);
}

6
source/dnode/mnode/impl/src/mndVgroup.c
View File

@ -14,6 +14,7 @@
*/
#define _DEFAULT_SOURCE
#include "mndVgroup.h"
#include "audit.h"
#include "mndArbGroup.h"
#include "mndDb.h"
@ -26,7 +27,6 @@
#include "mndTopic.h"
#include "mndTrans.h"
#include "mndUser.h"
#include "mndVgroup.h"
#include "tmisce.h"
#define VGROUP_VER_NUMBER 1
@ -1670,7 +1670,9 @@ int32_t mndAddNewVgPrepareAction(SMnode *pMnode, STrans *pTrans, SVgObj *pVg) {
}
TAOS_CHECK_GOTO(mndTransAppendPrepareLog(pTrans, pRaw), NULL, _err);
(void)sdbSetRawStatus(pRaw, SDB_STATUS_CREATING);
if (sdbSetRawStatus(pRaw, SDB_STATUS_CREATING) != 0) {
mError("vgId:%d, failed to set raw status at line:%d", pVg->vgId, __LINE__);
}
if (code != 0) {
mError("vgId:%d, failed to set raw status since %s at line:%d", pVg->vgId, tstrerror(code), __LINE__);
TAOS_RETURN(code);

10
source/dnode/snode/src/snode.c
View File

@ -38,7 +38,7 @@ int32_t sndBuildStreamTask(SSnode *pSnode, SStreamTask *pTask, int64_t nextProce
streamTaskOpenAllUpstreamInput(pTask);
streamTaskResetUpstreamStageInfo(pTask);
(void)streamSetupScheduleTrigger(pTask);
streamSetupScheduleTrigger(pTask);
SCheckpointInfo *pChkInfo = &pTask->chkInfo;
tqSetRestoreVersionInfo(pTask);
@ -93,14 +93,18 @@ FAIL:
}
int32_t sndInit(SSnode *pSnode) {
(void)streamTaskSchedTask(&pSnode->msgCb, pSnode->pMeta->vgId, 0, 0, STREAM_EXEC_T_START_ALL_TASKS);
if (streamTaskSchedTask(&pSnode->msgCb, pSnode->pMeta->vgId, 0, 0, STREAM_EXEC_T_START_ALL_TASKS) != 0) {
sndError("failed to start all tasks");
}
return 0;
}
void sndClose(SSnode *pSnode) {
stopRsync();
streamMetaNotifyClose(pSnode->pMeta);
(void)streamMetaCommit(pSnode->pMeta);
if (streamMetaCommit(pSnode->pMeta) != 0) {
sndError("failed to commit stream meta");
}
streamMetaClose(pSnode->pMeta);
taosMemoryFree(pSnode);
}

13
source/dnode/vnode/src/inc/tsdb.h
View File

@ -292,7 +292,7 @@ int32_t tsdbReadDelData(SDelFReader *pReader, SDelIdx *pDelIdx, SArray *aDelData
int32_t tsdbReadDelIdx(SDelFReader *pReader, SArray *aDelIdx);
// tsdbRead.c ==============================================================================================
int32_t tsdbTakeReadSnap2(STsdbReader *pReader, _query_reseek_func_t reseek, STsdbReadSnap **ppSnap, const char* id);
int32_t tsdbTakeReadSnap2(STsdbReader *pReader, _query_reseek_func_t reseek, STsdbReadSnap **ppSnap, const char *id);
void tsdbUntakeReadSnap2(STsdbReader *pReader, STsdbReadSnap *pSnap, bool proactive);
int32_t tsdbGetTableSchema(SMeta *pMeta, int64_t uid, STSchema **pSchema, int64_t *suid);
@ -375,7 +375,7 @@ struct STsdb {
struct {
SVHashTable *ht;
SArray *arr;
} *commitInfo;
} * commitInfo;
};
struct TSDBKEY {
@ -949,7 +949,7 @@ int32_t tsdbBICacheRelease(SLRUCache *pCache, LRUHandle *h);
int32_t tsdbCacheGetBlockS3(SLRUCache *pCache, STsdbFD *pFD, LRUHandle **handle);
int32_t tsdbCacheGetPageS3(SLRUCache *pCache, STsdbFD *pFD, int64_t pgno, LRUHandle **handle);
int32_t tsdbCacheSetPageS3(SLRUCache *pCache, STsdbFD *pFD, int64_t pgno, uint8_t *pPage);
void tsdbCacheSetPageS3(SLRUCache *pCache, STsdbFD *pFD, int64_t pgno, uint8_t *pPage);
int32_t tsdbCacheDeleteLastrow(SLRUCache *pCache, tb_uid_t uid, TSKEY eKey);
int32_t tsdbCacheDeleteLast(SLRUCache *pCache, tb_uid_t uid, TSKEY eKey);
@ -1069,6 +1069,13 @@ int32_t tsdbSnapPrepDescription(SVnode *pVnode, SSnapshot *pSnap);
void tsdbRemoveFile(const char *path);
#define taosCloseFileWithLog(fd) \
do { \
if (taosCloseFile(fd) < 0) { \
tsdbTrace("failed to close file"); \
} \
} while (0)
#ifdef __cplusplus
}
#endif

201
source/dnode/vnode/src/meta/metaCache.c
View File

@ -40,7 +40,7 @@ typedef struct SMetaStbStatsEntry {
} SMetaStbStatsEntry;
typedef struct STagFilterResEntry {
SList list; // the linked list of md5 digest, extracted from the serialized tag query condition
SHashObj *set; // the set of md5 digest, extracted from the serialized tag query condition
uint32_t hitTimes; // queried times for current super table
} STagFilterResEntry;
@ -112,7 +112,7 @@ static void statsCacheClose(SMeta* pMeta) {
static void freeCacheEntryFp(void* param) {
STagFilterResEntry** p = param;
tdListEmpty(&(*p)->list);
taosHashCleanup((*p)->set);
taosMemoryFreeClear(*p);
}
@ -200,10 +200,12 @@ void metaCacheClose(SMeta* pMeta) {
entryCacheClose(pMeta);
statsCacheClose(pMeta);
taosHashClear(pMeta->pCache->sTagFilterResCache.pTableEntry);
taosLRUCacheCleanup(pMeta->pCache->sTagFilterResCache.pUidResCache);
(void)taosThreadMutexDestroy(&pMeta->pCache->sTagFilterResCache.lock);
taosHashCleanup(pMeta->pCache->sTagFilterResCache.pTableEntry);
taosHashClear(pMeta->pCache->STbGroupResCache.pTableEntry);
taosLRUCacheCleanup(pMeta->pCache->STbGroupResCache.pResCache);
(void)taosThreadMutexDestroy(&pMeta->pCache->STbGroupResCache.lock);
taosHashCleanup(pMeta->pCache->STbGroupResCache.pTableEntry);
@ -216,7 +218,7 @@ void metaCacheClose(SMeta* pMeta) {
}
}
static int32_t metaRehashCache(SMetaCache* pCache, int8_t expand) {
static void metaRehashCache(SMetaCache* pCache, int8_t expand) {
int32_t code = 0;
int32_t nBucket;
@ -228,8 +230,7 @@ static int32_t metaRehashCache(SMetaCache* pCache, int8_t expand) {
SMetaCacheEntry** aBucket = (SMetaCacheEntry**)taosMemoryCalloc(nBucket, sizeof(SMetaCacheEntry*));
if (aBucket == NULL) {
code = terrno;
goto _exit;
return;
}
// rehash
@ -250,9 +251,7 @@ static int32_t metaRehashCache(SMetaCache* pCache, int8_t expand) {
taosMemoryFree(pCache->sEntryCache.aBucket);
pCache->sEntryCache.nBucket = nBucket;
pCache->sEntryCache.aBucket = aBucket;
_exit:
return code;
return;
}
int32_t metaCacheUpsert(SMeta* pMeta, SMetaInfo* pInfo) {
@ -279,7 +278,7 @@ int32_t metaCacheUpsert(SMeta* pMeta, SMetaInfo* pInfo) {
}
} else { // insert
if (pCache->sEntryCache.nEntry >= pCache->sEntryCache.nBucket) {
TAOS_UNUSED(metaRehashCache(pCache, 1));
metaRehashCache(pCache, 1);
iBucket = TABS(pInfo->uid) % pCache->sEntryCache.nBucket;
}
@ -317,7 +316,7 @@ int32_t metaCacheDrop(SMeta* pMeta, int64_t uid) {
pCache->sEntryCache.nEntry--;
if (pCache->sEntryCache.nEntry < pCache->sEntryCache.nBucket / 4 &&
pCache->sEntryCache.nBucket > META_CACHE_BASE_BUCKET) {
TAOS_UNUSED(metaRehashCache(pCache, 0));
metaRehashCache(pCache, 0);
}
} else {
code = TSDB_CODE_NOT_FOUND;
@ -474,34 +473,6 @@ int32_t metaStatsCacheGet(SMeta* pMeta, int64_t uid, SMetaStbStats* pInfo) {
return code;
}
static int checkAllEntriesInCache(const STagFilterResEntry* pEntry, SArray* pInvalidRes, int32_t keyLen,
SLRUCache* pCache, uint64_t suid) {
SListIter iter = {0};
tdListInitIter((SList*)&(pEntry->list), &iter, TD_LIST_FORWARD);
SListNode* pNode = NULL;
uint64_t buf[3];
buf[0] = suid;
int32_t len = sizeof(uint64_t) * tListLen(buf);
while ((pNode = tdListNext(&iter)) != NULL) {
memcpy(&buf[1], pNode->data, keyLen);
// check whether it is existed in LRU cache, and remove it from linked list if not.
LRUHandle* pRes = taosLRUCacheLookup(pCache, buf, len);
if (pRes == NULL) { // remove the item in the linked list
if (taosArrayPush(pInvalidRes, &pNode) == NULL) {
return terrno;
}
} else {
bool ret = taosLRUCacheRelease(pCache, pRes, false);
}
}
return 0;
}
static FORCE_INLINE void setMD5DigestInKey(uint64_t* pBuf, const char* key, int32_t keyLen) {
memcpy(&pBuf[2], key, keyLen);
}
@ -587,22 +558,11 @@ static void freeUidCachePayload(const void* key, size_t keyLen, void* value, voi
if (pEntry != NULL && (*pEntry) != NULL) {
int64_t st = taosGetTimestampUs();
SListIter iter = {0};
tdListInitIter((SList*)&((*pEntry)->list), &iter, TD_LIST_FORWARD);
SListNode* pNode = NULL;
while ((pNode = tdListNext(&iter)) != NULL) {
uint64_t* digest = (uint64_t*)pNode->data;
if (digest[0] == p[2] && digest[1] == p[3]) {
void* tmp = tdListPopNode(&((*pEntry)->list), pNode);
taosMemoryFree(tmp);
double el = (taosGetTimestampUs() - st) / 1000.0;
metaInfo("clear items in meta-cache, remain cached item:%d, elapsed time:%.2fms", listNEles(&((*pEntry)->list)),
el);
break;
}
int32_t code = taosHashRemove((*pEntry)->set, &p[2], sizeof(uint64_t) * 2);
if (code == TSDB_CODE_SUCCESS) {
double el = (taosGetTimestampUs() - st) / 1000.0;
metaInfo("clear items in meta-cache, remain cached item:%d, elapsed time:%.2fms", taosHashGetSize((*pEntry)->set),
el);
}
}
@ -610,16 +570,30 @@ static void freeUidCachePayload(const void* key, size_t keyLen, void* value, voi
}
static int32_t addNewEntry(SHashObj* pTableEntry, const void* pKey, int32_t keyLen, uint64_t suid) {
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
STagFilterResEntry* p = taosMemoryMalloc(sizeof(STagFilterResEntry));
if (p == NULL) {
return terrno;
}
TSDB_CHECK_NULL(p, code, lino, _end, terrno);
p->hitTimes = 0;
tdListInit(&p->list, keyLen);
TAOS_CHECK_RETURN(taosHashPut(pTableEntry, &suid, sizeof(uint64_t), &p, POINTER_BYTES));
TAOS_CHECK_RETURN(tdListAppend(&p->list, pKey));
return 0;
p->set = taosHashInit(1024, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), false, HASH_NO_LOCK);
TSDB_CHECK_NULL(p->set, code, lino, _end, terrno);
code = taosHashPut(p->set, pKey, keyLen, NULL, 0);
TSDB_CHECK_CODE(code, lino, _end);
code = taosHashPut(pTableEntry, &suid, sizeof(uint64_t), &p, POINTER_BYTES);
TSDB_CHECK_CODE(code, lino, _end);
_end:
if (code != TSDB_CODE_SUCCESS) {
metaError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
if (p != NULL) {
if (p->set != NULL) {
taosHashCleanup(p->set);
}
taosMemoryFree(p);
}
}
return code;
}
// check both the payload size and selectivity ratio
@ -660,25 +634,14 @@ int32_t metaUidFilterCachePut(void* pVnode, uint64_t suid, const void* pKey, int
goto _end;
}
} else { // check if it exists or not
size_t size = listNEles(&(*pEntry)->list);
if (size == 0) {
code = tdListAppend(&(*pEntry)->list, pKey);
if (code) {
goto _end;
}
} else {
SListNode* pNode = listHead(&(*pEntry)->list);
uint64_t* p = (uint64_t*)pNode->data;
if (p[1] == ((uint64_t*)pKey)[1] && p[0] == ((uint64_t*)pKey)[0]) {
// we have already found the existed items, no need to added to cache anymore.
(void)taosThreadMutexUnlock(pLock);
return TSDB_CODE_SUCCESS;
} else { // not equal, append it
code = tdListAppend(&(*pEntry)->list, pKey);
if (code) {
goto _end;
}
}
code = taosHashPut((*pEntry)->set, pKey, keyLen, NULL, 0);
if (code == TSDB_CODE_DUP_KEY) {
// we have already found the existed items, no need to added to cache anymore.
(void)taosThreadMutexUnlock(pLock);
return TSDB_CODE_SUCCESS;
}
if (code != TSDB_CODE_SUCCESS) {
goto _end;
}
}
@ -706,23 +669,20 @@ int32_t metaUidCacheClear(SMeta* pMeta, uint64_t suid) {
(void)taosThreadMutexLock(pLock);
STagFilterResEntry** pEntry = taosHashGet(pEntryHashMap, &suid, sizeof(uint64_t));
if (pEntry == NULL || listNEles(&(*pEntry)->list) == 0) {
if (pEntry == NULL || taosHashGetSize((*pEntry)->set) == 0) {
(void)taosThreadMutexUnlock(pLock);
return TSDB_CODE_SUCCESS;
}
(*pEntry)->hitTimes = 0;
SListIter iter = {0};
tdListInitIter(&(*pEntry)->list, &iter, TD_LIST_FORWARD);
SListNode* pNode = NULL;
while ((pNode = tdListNext(&iter)) != NULL) {
setMD5DigestInKey(p, pNode->data, 2 * sizeof(uint64_t));
char *iter = taosHashIterate((*pEntry)->set, NULL);
while (iter != NULL) {
setMD5DigestInKey(p, iter, 2 * sizeof(uint64_t));
taosLRUCacheErase(pMeta->pCache->sTagFilterResCache.pUidResCache, p, TAG_FILTER_RES_KEY_LEN);
iter = taosHashIterate((*pEntry)->set, iter);
}
tdListEmpty(&(*pEntry)->list);
taosHashClear((*pEntry)->set);
(void)taosThreadMutexUnlock(pLock);
metaDebug("vgId:%d suid:%" PRId64 " cached related tag filter uid list cleared", vgId, suid);
@ -792,22 +752,11 @@ static void freeTbGroupCachePayload(const void* key, size_t keyLen, void* value,
if (pEntry != NULL && (*pEntry) != NULL) {
int64_t st = taosGetTimestampUs();
SListIter iter = {0};
tdListInitIter((SList*)&((*pEntry)->list), &iter, TD_LIST_FORWARD);
SListNode* pNode = NULL;
while ((pNode = tdListNext(&iter)) != NULL) {
uint64_t* digest = (uint64_t*)pNode->data;
if (digest[0] == p[2] && digest[1] == p[3]) {
void* tmp = tdListPopNode(&((*pEntry)->list), pNode);
taosMemoryFree(tmp);
double el = (taosGetTimestampUs() - st) / 1000.0;
metaDebug("clear one item in tb group cache, remain cached item:%d, elapsed time:%.2fms",
listNEles(&((*pEntry)->list)), el);
break;
}
int32_t code = taosHashRemove((*pEntry)->set, &p[2], sizeof(uint64_t) * 2);
if (code == TSDB_CODE_SUCCESS) {
double el = (taosGetTimestampUs() - st) / 1000.0;
metaDebug("clear one item in tb group cache, remain cached item:%d, elapsed time:%.2fms",
taosHashGetSize((*pEntry)->set), el);
}
}
@ -843,25 +792,14 @@ int32_t metaPutTbGroupToCache(void* pVnode, uint64_t suid, const void* pKey, int
goto _end;
}
} else { // check if it exists or not
size_t size = listNEles(&(*pEntry)->list);
if (size == 0) {
code = tdListAppend(&(*pEntry)->list, pKey);
if (code) {
goto _end;
}
} else {
SListNode* pNode = listHead(&(*pEntry)->list);
uint64_t* p = (uint64_t*)pNode->data;
if (p[1] == ((uint64_t*)pKey)[1] && p[0] == ((uint64_t*)pKey)[0]) {
// we have already found the existed items, no need to added to cache anymore.
(void)taosThreadMutexUnlock(pLock);
return TSDB_CODE_SUCCESS;
} else { // not equal, append it
code = tdListAppend(&(*pEntry)->list, pKey);
if (code) {
goto _end;
}
}
code = taosHashPut((*pEntry)->set, pKey, keyLen, NULL, 0);
if (code == TSDB_CODE_DUP_KEY) {
// we have already found the existed items, no need to added to cache anymore.
(void)taosThreadMutexUnlock(pLock);
return TSDB_CODE_SUCCESS;
}
if (code != TSDB_CODE_SUCCESS) {
goto _end;
}
}
@ -889,23 +827,20 @@ int32_t metaTbGroupCacheClear(SMeta* pMeta, uint64_t suid) {
(void)taosThreadMutexLock(pLock);
STagFilterResEntry** pEntry = taosHashGet(pEntryHashMap, &suid, sizeof(uint64_t));
if (pEntry == NULL || listNEles(&(*pEntry)->list) == 0) {
if (pEntry == NULL || taosHashGetSize((*pEntry)->set) == 0) {
(void)taosThreadMutexUnlock(pLock);
return TSDB_CODE_SUCCESS;
}
(*pEntry)->hitTimes = 0;
SListIter iter = {0};
tdListInitIter(&(*pEntry)->list, &iter, TD_LIST_FORWARD);
SListNode* pNode = NULL;
while ((pNode = tdListNext(&iter)) != NULL) {
setMD5DigestInKey(p, pNode->data, 2 * sizeof(uint64_t));
char *iter = taosHashIterate((*pEntry)->set, NULL);
while (iter != NULL) {
setMD5DigestInKey(p, iter, 2 * sizeof(uint64_t));
taosLRUCacheErase(pMeta->pCache->STbGroupResCache.pResCache, p, TAG_FILTER_RES_KEY_LEN);
iter = taosHashIterate((*pEntry)->set, iter);
}
tdListEmpty(&(*pEntry)->list);
taosHashClear((*pEntry)->set);
(void)taosThreadMutexUnlock(pLock);
metaDebug("vgId:%d suid:%" PRId64 " cached related tb group cleared", vgId, suid);

4
source/dnode/vnode/src/meta/metaOpen.c
View File

@ -60,7 +60,7 @@ int32_t metaOpen(SVnode *pVnode, SMeta **ppMeta, int8_t rollback) {
pMeta->path = (char *)&pMeta[1];
strcpy(pMeta->path, path);
(void)taosRealPath(pMeta->path, NULL, strlen(path) + 1);
int32_t ret = taosRealPath(pMeta->path, NULL, strlen(path) + 1);
pMeta->pVnode = pVnode;
@ -98,7 +98,7 @@ int32_t metaOpen(SVnode *pVnode, SMeta **ppMeta, int8_t rollback) {
TSDB_CHECK_CODE(code, lino, _exit);
sprintf(indexFullPath, "%s/%s", pMeta->path, "invert");
TAOS_UNUSED(taosMkDir(indexFullPath));
ret = taosMkDir(indexFullPath);
SIndexOpts opts = {.cacheSize = 8 * 1024 * 1024};
code = indexOpen(&opts, indexFullPath, (SIndex **)&pMeta->pTagIvtIdx);

12
source/dnode/vnode/src/meta/metaQuery.c
View File

@ -62,7 +62,10 @@ int metaGetTableEntryByVersion(SMetaReader *pReader, int64_t version, tb_uid_t u
tDecoderInit(&pReader->coder, pReader->pBuf, pReader->szBuf);
code = metaDecodeEntry(&pReader->coder, &pReader->me);
if (code) return code;
if (code) {
tDecoderClear(&pReader->coder);
return code;
}
// taosMemoryFreeClear(pReader->me.colCmpr.pColCmpr);
return 0;
@ -393,6 +396,7 @@ _query:
tDecoderInit(&dc, pData, nData);
int32_t code = metaDecodeEntry(&dc, &me);
if (code) {
tDecoderClear(&dc);
goto _err;
}
if (me.type == TSDB_SUPER_TABLE) {
@ -1277,7 +1281,11 @@ int32_t metaFilterTableIds(void *pVnode, SMetaFltParam *arg, SArray *pUids) {
tDecoderInit(&dc, pData, nData);
TAOS_CHECK_GOTO(metaDecodeEntry(&dc, &oStbEntry), NULL, END);
code = metaDecodeEntry(&dc, &oStbEntry);
if (code) {
tDecoderClear(&dc);
goto END;
}
if (oStbEntry.stbEntry.schemaTag.pSchema == NULL || oStbEntry.stbEntry.schemaTag.pSchema == NULL) {
TAOS_CHECK_GOTO(TSDB_CODE_INVALID_PARA, NULL, END);

1
source/dnode/vnode/src/meta/metaSnapshot.c
View File

@ -605,6 +605,7 @@ int32_t getTableInfoFromSnapshot(SSnapContext* ctx, void** pBuf, int32_t* contLe
tDecoderInit(&dc, pVal, vLen);
ret = metaDecodeEntry(&dc, &me);
if (ret < 0) {
tDecoderClear(&dc);
ret = TAOS_GET_TERRNO(ret);
goto END;
}

8
source/dnode/vnode/src/meta/metaTable.c
View File

@ -285,7 +285,9 @@ static inline void metaTimeSeriesNotifyCheck(SMeta *pMeta) {
int64_t deltaTS = nTimeSeries - pMeta->pVnode->config.vndStats.numOfReportedTimeSeries;
if (deltaTS > tsTimeSeriesThreshold) {
if (0 == atomic_val_compare_exchange_8(&dmNotifyHdl.state, 1, 2)) {
(void)tsem_post(&dmNotifyHdl.sem);
if (tsem_post(&dmNotifyHdl.sem) != 0) {
metaError("vgId:%d, failed to post semaphore, errno:%d", TD_VID(pMeta->pVnode), errno);
}
}
}
#endif
@ -563,6 +565,7 @@ int metaAlterSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
if (ret < 0) {
metaError("vgId:%d, failed to decode stb:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name,
pReq->suid, tstrerror(ret));
tDecoderClear(&dc);
tdbTbcClose(pTbDbc);
tdbTbcClose(pUidIdxc);
return terrno;
@ -1502,6 +1505,7 @@ static int metaDropTableByUid(SMeta *pMeta, tb_uid_t uid, int *type, tb_uid_t *p
tDecoderInit(&tdc, tData, tLen);
int32_t ret = metaDecodeEntry(&tdc, &stbEntry);
if (ret < 0) {
tDecoderClear(&tdc);
metaError("vgId:%d, failed to decode child table:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), e.name,
e.ctbEntry.suid, tstrerror(ret));
return ret;
@ -2397,6 +2401,7 @@ static int metaAddTagIndex(SMeta *pMeta, int64_t version, SVAlterTbReq *pAlterTb
tDecoderInit(&dc, pVal, nVal);
ret = metaDecodeEntry(&dc, &stbEntry);
if (ret < 0) {
tDecoderClear(&dc);
goto _err;
}
@ -2538,6 +2543,7 @@ static int metaDropTagIndex(SMeta *pMeta, int64_t version, SVAlterTbReq *pAlterT
tDecoderInit(&dc, pVal, nVal);
ret = metaDecodeEntry(&dc, &stbEntry);
if (ret < 0) {
tDecoderClear(&dc);
goto _err;
}

16
source/dnode/vnode/src/sma/smaEnv.c
View File

@ -211,7 +211,10 @@ static int32_t tdInitSmaStat(SSmaStat **pSmaStat, int8_t smaType, const SSma *pS
SRSmaStat *pRSmaStat = (SRSmaStat *)(*pSmaStat);
pRSmaStat->pSma = (SSma *)pSma;
atomic_store_8(RSMA_TRIGGER_STAT(pRSmaStat), TASK_TRIGGER_STAT_INIT);
(void)tsem_init(&pRSmaStat->notEmpty, 0, 0);
if (tsem_init(&pRSmaStat->notEmpty, 0, 0) != 0) {
code = terrno;
TAOS_CHECK_GOTO(code, &lino, _exit);
}
if (!(pRSmaStat->blocks = taosArrayInit(1, sizeof(SSDataBlock)))) {
code = terrno;
TAOS_CHECK_GOTO(code, &lino, _exit);
@ -295,7 +298,10 @@ static void tdDestroyRSmaStat(void *pRSmaStat) {
taosHashCleanup(RSMA_INFO_HASH(pStat));
// step 5: free pStat
(void)tsem_destroy(&(pStat->notEmpty));
if (tsem_destroy(&(pStat->notEmpty)) != 0) {
smaError("vgId:%d, failed to destroy notEmpty semaphore for rsma stat:%p since %s", SMA_VID(pSma), pRSmaStat,
tstrerror(terrno));
}
taosArrayDestroy(pStat->blocks);
taosMemoryFreeClear(pStat);
}
@ -399,7 +405,7 @@ int32_t tdCheckAndInitSmaEnv(SSma *pSma, int8_t smaType) {
void *tdRSmaExecutorFunc(void *param) {
setThreadName("vnode-rsma");
if(tdRSmaProcessExecImpl((SSma *)param, RSMA_EXEC_OVERFLOW) < 0){
if (tdRSmaProcessExecImpl((SSma *)param, RSMA_EXEC_OVERFLOW) < 0) {
smaError("vgId:%d, failed to process rsma exec", SMA_VID((SSma *)param));
}
return NULL;
@ -444,7 +450,9 @@ static int32_t tdRsmaStopExecutor(const SSma *pSma) {
pthread = (TdThread *)&pStat->data;
for (int32_t i = 0; i < tsNumOfVnodeRsmaThreads; ++i) {
(void)tsem_post(&(pRSmaStat->notEmpty));
if (tsem_post(&(pRSmaStat->notEmpty)) != 0) {
smaError("vgId:%d, failed to post notEmpty semaphore for rsma since %s", SMA_VID(pSma), tstrerror(terrno));
}
}
for (int32_t i = 0; i < tsNumOfVnodeRsmaThreads; ++i) {

23
source/dnode/vnode/src/sma/smaRollup.c
View File

@ -89,7 +89,7 @@ void *tdFreeRSmaInfo(SSma *pSma, SRSmaInfo *pInfo) {
if (pItem->tmrId) {
smaDebug("vgId:%d, stop fetch timer %p for table %" PRIi64 " level %d", SMA_VID(pSma), pItem->tmrId,
pInfo->suid, i + 1);
if(!taosTmrStopA(&pItem->tmrId)){
if (!taosTmrStopA(&pItem->tmrId)) {
smaError("vgId:%d, failed to stop fetch timer for table %" PRIi64 " level %d", SMA_VID(pSma), pInfo->suid,
i + 1);
}
@ -404,7 +404,7 @@ int32_t tdRSmaProcessCreateImpl(SSma *pSma, SRSmaParam *param, int64_t suid, con
}
STSchema *pTSchema;
code = metaGetTbTSchemaNotNull(SMA_META(pSma), suid, -1, 1, &pTSchema);
code = metaGetTbTSchemaNotNull(SMA_META(pSma), suid, -1, 1, &pTSchema);
TAOS_CHECK_EXIT(code);
pRSmaInfo->pSma = pSma;
pRSmaInfo->pTSchema = pTSchema;
@ -820,7 +820,10 @@ static int32_t tdExecuteRSmaImplAsync(SSma *pSma, int64_t version, const void *p
int64_t nItems = atomic_fetch_add_64(&pRSmaStat->nBufItems, 1);
if (atomic_load_8(&pInfo->assigned) == 0) {
(void)tsem_post(&(pRSmaStat->notEmpty));
if (tsem_post(&(pRSmaStat->notEmpty)) != 0) {
smaError("vgId:%d, failed to post notEmpty semaphore for rsma %" PRIi64 " since %s", SMA_VID(pSma), suid,
tstrerror(terrno));
}
}
// smoothing consume
@ -1385,7 +1388,8 @@ static void tdRSmaFetchTrigger(void *param, void *tmrId) {
if (rsmaTriggerStat == TASK_TRIGGER_STAT_PAUSED) {
bool ret = taosTmrReset(tdRSmaFetchTrigger, RSMA_FETCH_INTERVAL, pItem, smaMgmt.tmrHandle, &pItem->tmrId);
if (!ret) {
smaWarn("vgId:%d, rsma fetch task not reset for level %" PRIi8 " since tmr reset failed, rsetId:%d refId:%" PRIi64,
smaWarn("vgId:%d, rsma fetch task not reset for level %" PRIi8
" since tmr reset failed, rsetId:%d refId:%" PRIi64,
SMA_VID(pSma), pItem->level, smaMgmt.rsetId, pRSmaRef->refId);
}
}
@ -1407,7 +1411,10 @@ static void tdRSmaFetchTrigger(void *param, void *tmrId) {
atomic_store_8(&pItem->fetchLevel, 1);
if (atomic_load_8(&pRSmaInfo->assigned) == 0) {
(void)tsem_post(&(pStat->notEmpty));
if (tsem_post(&(pStat->notEmpty)) != 0) {
smaError("vgId:%d, rsma fetch task not start for level:%" PRIi8 " suid:%" PRIi64 " since sem post failed",
SMA_VID(pSma), pItem->level, pRSmaInfo->suid);
}
}
} break;
case TASK_TRIGGER_STAT_INACTIVE: {
@ -1641,7 +1648,7 @@ int32_t tdRSmaProcessExecImpl(SSma *pSma, ERsmaExecType type) {
batchMax = 100 / batchMax;
batchMax = TMAX(batchMax, 4);
}
while (occupied || (++batchCnt < batchMax)) { // greedy mode
while (occupied || (++batchCnt < batchMax)) { // greedy mode
TAOS_UNUSED(taosReadAllQitems(pInfo->queue, pInfo->qall)); // queue has mutex lock
int32_t qallItemSize = taosQallItemSize(pInfo->qall);
if (qallItemSize > 0) {
@ -1700,7 +1707,9 @@ int32_t tdRSmaProcessExecImpl(SSma *pSma, ERsmaExecType type) {
break;
}
(void)tsem_wait(&pRSmaStat->notEmpty);
if (tsem_wait(&pRSmaStat->notEmpty) != 0) {
smaError("vgId:%d, failed to wait for not empty since %s", TD_VID(pVnode), tstrerror(terrno));
}
if ((pEnv->flag & SMA_ENV_FLG_CLOSE) && (atomic_load_64(&pRSmaStat->nBufItems) <= 0)) {
smaDebug("vgId:%d, exec task end, flag:%" PRIi8 ", nBufItems:%" PRIi64, SMA_VID(pSma), pEnv->flag,

10
source/dnode/vnode/src/tsdb/tsdbCache.c
View File

@ -1474,6 +1474,9 @@ int32_t tsdbCacheColFormatUpdate(STsdb *pTsdb, tb_uid_t suid, tb_uid_t uid, SBlo
TAOS_CHECK_RETURN(metaGetTbTSchemaEx(pTsdb->pVnode->pMeta, suid, uid, sver, &pTSchema));
ctxArray = taosArrayInit(pBlockData->nColData, sizeof(SLastUpdateCtx));
if (ctxArray == NULL) {
TAOS_CHECK_GOTO(terrno, &lino, _exit);
}
// 1. prepare last
STsdbRowKey tsdbRowKey = {0};
@ -3682,8 +3685,7 @@ int32_t tsdbCacheGetPageS3(SLRUCache *pCache, STsdbFD *pFD, int64_t pgno, LRUHan
return code;
}
int32_t tsdbCacheSetPageS3(SLRUCache *pCache, STsdbFD *pFD, int64_t pgno, uint8_t *pPage) {
int32_t code = 0;
void tsdbCacheSetPageS3(SLRUCache *pCache, STsdbFD *pFD, int64_t pgno, uint8_t *pPage) {
char key[128] = {0};
int keyLen = 0;
LRUHandle *handle = NULL;
@ -3696,7 +3698,7 @@ int32_t tsdbCacheSetPageS3(SLRUCache *pCache, STsdbFD *pFD, int64_t pgno, uint8_
_taos_lru_deleter_t deleter = deleteBCache;
uint8_t *pPg = taosMemoryMalloc(charge);
if (!pPg) {
TAOS_RETURN(terrno);
return; // ignore error with s3 cache and leave error untouched
}
memcpy(pPg, pPage, charge);
@ -3710,6 +3712,4 @@ int32_t tsdbCacheSetPageS3(SLRUCache *pCache, STsdbFD *pFD, int64_t pgno, uint8_
(void)taosThreadMutexUnlock(&pFD->pTsdb->pgMutex);
tsdbCacheRelease(pFD->pTsdb->pgCache, handle);
TAOS_RETURN(code);
}

40
source/dnode/vnode/src/tsdb/tsdbCommit2.c
View File

@ -199,9 +199,9 @@ _exit:
return code;
}
static int32_t tsdbCommitCloseReader(SCommitter2 *committer) {
static void tsdbCommitCloseReader(SCommitter2 *committer) {
TARRAY2_CLEAR(committer->sttReaderArray, tsdbSttFileReaderClose);
return 0;
return;
}
static int32_t tsdbCommitOpenReader(SCommitter2 *committer) {
@ -243,19 +243,19 @@ static int32_t tsdbCommitOpenReader(SCommitter2 *committer) {
_exit:
if (code) {
TAOS_UNUSED(tsdbCommitCloseReader(committer));
tsdbCommitCloseReader(committer);
tsdbError("vgId:%d %s failed at %s:%d since %s", TD_VID(committer->tsdb->pVnode), __func__, __FILE__, lino,
tstrerror(code));
}
return code;
}
static int32_t tsdbCommitCloseIter(SCommitter2 *committer) {
static void tsdbCommitCloseIter(SCommitter2 *committer) {
tsdbIterMergerClose(&committer->tombIterMerger);
tsdbIterMergerClose(&committer->dataIterMerger);
TARRAY2_CLEAR(committer->tombIterArray, tsdbIterClose);
TARRAY2_CLEAR(committer->dataIterArray, tsdbIterClose);
return 0;
return;
}
static int32_t tsdbCommitOpenIter(SCommitter2 *committer) {
@ -309,7 +309,7 @@ static int32_t tsdbCommitOpenIter(SCommitter2 *committer) {
_exit:
if (code) {
TAOS_UNUSED(tsdbCommitCloseIter(committer));
tsdbCommitCloseIter(committer);
tsdbError("vgId:%d %s failed at %s:%d since %s", TD_VID(committer->tsdb->pVnode), __func__, __FILE__, lino,
tstrerror(code));
}
@ -322,7 +322,7 @@ static int32_t tsdbCommitFileSetBegin(SCommitter2 *committer) {
STsdb *tsdb = committer->tsdb;
// check if can commit
TAOS_UNUSED(tsdbFSCheckCommit(tsdb, committer->ctx->info->fid));
tsdbFSCheckCommit(tsdb, committer->ctx->info->fid);
committer->ctx->expLevel = tsdbFidLevel(committer->ctx->info->fid, &tsdb->keepCfg, committer->now);
tsdbFidKeyRange(committer->ctx->info->fid, committer->minutes, committer->precision, &committer->ctx->minKey,
@ -331,7 +331,9 @@ static int32_t tsdbCommitFileSetBegin(SCommitter2 *committer) {
TAOS_CHECK_GOTO(tfsAllocDisk(committer->tsdb->pVnode->pTfs, committer->ctx->expLevel, &committer->ctx->did), &lino,
_exit);
TAOS_UNUSED(tfsMkdirRecurAt(committer->tsdb->pVnode->pTfs, committer->tsdb->path, committer->ctx->did));
if (tfsMkdirRecurAt(committer->tsdb->pVnode->pTfs, committer->tsdb->path, committer->ctx->did) != 0) {
tsdbError("vgId:%d failed to create directory %s", TD_VID(committer->tsdb->pVnode), committer->tsdb->path);
}
committer->ctx->tbid->suid = 0;
committer->ctx->tbid->uid = 0;
@ -355,8 +357,8 @@ static int32_t tsdbCommitFileSetEnd(SCommitter2 *committer) {
int32_t lino = 0;
TAOS_CHECK_GOTO(tsdbCommitCloseWriter(committer), &lino, _exit);
TAOS_CHECK_GOTO(tsdbCommitCloseIter(committer), &lino, _exit);
TAOS_CHECK_GOTO(tsdbCommitCloseReader(committer), &lino, _exit);
tsdbCommitCloseIter(committer);
tsdbCommitCloseReader(committer);
_exit:
if (code) {
@ -409,11 +411,11 @@ static uint32_t tFileSetCommitInfoHash(const void *arg) {
return MurmurHash3_32((const char *)&info->fid, sizeof(info->fid));
}
static int32_t tsdbCommitInfoDestroy(STsdb *pTsdb) {
static void tsdbCommitInfoDestroy(STsdb *pTsdb) {
if (pTsdb->commitInfo) {
for (int32_t i = 0; i < taosArrayGetSize(pTsdb->commitInfo->arr); i++) {
SFileSetCommitInfo *info = *(SFileSetCommitInfo **)taosArrayGet(pTsdb->commitInfo->arr, i);
TAOS_UNUSED(vHashDrop(pTsdb->commitInfo->ht, info));
int32_t ret = vHashDrop(pTsdb->commitInfo->ht, info);
tsdbTFileSetClear(&info->fset);
taosMemoryFree(info);
}
@ -423,7 +425,7 @@ static int32_t tsdbCommitInfoDestroy(STsdb *pTsdb) {
pTsdb->commitInfo->arr = NULL;
taosMemoryFreeClear(pTsdb->commitInfo);
}
return 0;
return;
}
static int32_t tsdbCommitInfoInit(STsdb *pTsdb) {
@ -444,7 +446,7 @@ static int32_t tsdbCommitInfoInit(STsdb *pTsdb) {
_exit:
if (code) {
TAOS_UNUSED(tsdbCommitInfoDestroy(pTsdb));
tsdbCommitInfoDestroy(pTsdb);
tsdbError("vgId:%d %s failed at %s:%d since %s", TD_VID(pTsdb->pVnode), __func__, __FILE__, lino, tstrerror(code));
}
return code;
@ -514,7 +516,7 @@ static int32_t tsdbCommitInfoBuild(STsdb *tsdb) {
SFileSetCommitInfo tinfo = {
.fid = fid,
};
TAOS_UNUSED(vHashGet(tsdb->commitInfo->ht, &tinfo, (void **)&info));
int32_t ret = vHashGet(tsdb->commitInfo->ht, &tinfo, (void **)&info);
if (info == NULL) {
TAOS_CHECK_GOTO(tsdbCommitInfoAdd(tsdb, fid), &lino, _exit);
}
@ -538,7 +540,7 @@ static int32_t tsdbCommitInfoBuild(STsdb *tsdb) {
};
// check if the file set already on the commit list
TAOS_UNUSED(vHashGet(tsdb->commitInfo->ht, &tinfo, (void **)&info));
int32_t ret = vHashGet(tsdb->commitInfo->ht, &tinfo, (void **)&info);
if (info != NULL) {
continue;
}
@ -586,7 +588,7 @@ static int32_t tsdbCommitInfoBuild(STsdb *tsdb) {
_exit:
if (code) {
TAOS_UNUSED(tsdbCommitInfoDestroy(tsdb));
tsdbCommitInfoDestroy(tsdb);
tsdbError("vgId:%d %s failed at %s:%d since %s", TD_VID(tsdb->pVnode), __func__, __FILE__, lino, tstrerror(code));
}
return code;
@ -716,7 +718,7 @@ int32_t tsdbCommitCommit(STsdb *tsdb) {
(void)taosThreadMutexUnlock(&tsdb->mutex);
TAOS_UNUSED(tsdbCommitInfoDestroy(tsdb));
tsdbCommitInfoDestroy(tsdb);
tsdbUnrefMemTable(pMemTable, NULL, true);
}
@ -745,7 +747,7 @@ int32_t tsdbCommitAbort(STsdb *pTsdb) {
}
}
(void)taosThreadMutexUnlock(&pTsdb->mutex);
TAOS_UNUSED(tsdbCommitInfoDestroy(pTsdb));
tsdbCommitInfoDestroy(pTsdb);
_exit:
if (code) {

4
source/dnode/vnode/src/tsdb/tsdbDataFileRW.c
View File

@ -1362,7 +1362,7 @@ int32_t tsdbFileWriteTombBlock(STsdbFD *fd, STombBlock *tombBlock, int8_t cmprAl
};
for (int i = 0; i < TOMB_BLOCK_SIZE(tombBlock); i++) {
STombRecord record;
TAOS_UNUSED(tTombBlockGet(tombBlock, i, &record));
TAOS_CHECK_RETURN(tTombBlockGet(tombBlock, i, &record));
if (i == 0) {
tombBlk.minTbid.suid = record.suid;
@ -1519,7 +1519,7 @@ static int32_t tsdbDataFileDoWriteTombRecord(SDataFileWriter *writer, const STom
while (writer->ctx->hasOldTomb) {
for (; writer->ctx->tombBlockIdx < TOMB_BLOCK_SIZE(writer->ctx->tombBlock); writer->ctx->tombBlockIdx++) {
STombRecord record1[1];
TAOS_UNUSED(tTombBlockGet(writer->ctx->tombBlock, writer->ctx->tombBlockIdx, record1));
TAOS_CHECK_GOTO(tTombBlockGet(writer->ctx->tombBlock, writer->ctx->tombBlockIdx, record1), &lino, _exit);
int32_t c = tTombRecordCompare(record, record1);
if (c < 0) {

12
source/dnode/vnode/src/tsdb/tsdbFS.c
View File

@ -130,7 +130,7 @@ _exit:
tsdbError("%s failed at line %d since %s, fname:%s", __func__, lino, tstrerror(code), fname);
}
taosMemoryFree(pData);
(void)taosCloseFile(&pFD);
taosCloseFileWithLog(&pFD);
return code;
}
@ -300,26 +300,26 @@ static int32_t load_fs(const char *fname, STsdbFS *pFS) {
int64_t size;
code = taosFStatFile(pFD, &size, NULL);
if (code != 0) {
(void)taosCloseFile(&pFD);
taosCloseFileWithLog(&pFD);
TSDB_CHECK_CODE(code, lino, _exit);
}
pData = taosMemoryMalloc(size);
if (pData == NULL) {
code = terrno;
(void)taosCloseFile(&pFD);
taosCloseFileWithLog(&pFD);
TSDB_CHECK_CODE(code, lino, _exit);
}
if (taosReadFile(pFD, pData, size) < 0) {
code = terrno;
(void)taosCloseFile(&pFD);
taosCloseFileWithLog(&pFD);
TSDB_CHECK_CODE(code, lino, _exit);
}
if (!taosCheckChecksumWhole(pData, size)) {
code = TSDB_CODE_FILE_CORRUPTED;
(void)taosCloseFile(&pFD);
taosCloseFileWithLog(&pFD);
TSDB_CHECK_CODE(code, lino, _exit);
}
@ -331,7 +331,7 @@ _exit:
tsdbError("%s failed at line %d since %s, fname:%s", __func__, lino, tstrerror(code), fname);
}
taosMemoryFree(pData);
(void)taosCloseFile(&pFD);
taosCloseFileWithLog(&pFD);
return code;
}

37
source/dnode/vnode/src/tsdb/tsdbFS2.c
View File

@ -44,7 +44,12 @@ static int32_t create_fs(STsdb *pTsdb, STFileSystem **fs) {
}
fs[0]->tsdb = pTsdb;
(void)tsem_init(&fs[0]->canEdit, 0, 1);
int32_t code = tsem_init(&fs[0]->canEdit, 0, 1);
if (code) {
taosMemoryFree(fs[0]);
return code;
}
fs[0]->fsstate = TSDB_FS_STATE_NORMAL;
fs[0]->neid = 0;
TARRAY2_INIT(fs[0]->fSetArr);
@ -58,7 +63,9 @@ static void destroy_fs(STFileSystem **fs) {
TARRAY2_DESTROY(fs[0]->fSetArr, NULL);
TARRAY2_DESTROY(fs[0]->fSetArrTmp, NULL);
(void)tsem_destroy(&fs[0]->canEdit);
if (tsem_destroy(&fs[0]->canEdit) != 0) {
tsdbError("failed to destroy semaphore");
}
taosMemoryFree(fs[0]);
fs[0] = NULL;
}
@ -100,7 +107,7 @@ _exit:
tsdbError("%s failed at %s:%d since %s", __func__, fname, __LINE__, tstrerror(code));
}
taosMemoryFree(data);
(void)taosCloseFile(&fp);
taosCloseFileWithLog(&fp);
return code;
}
@ -140,7 +147,7 @@ _exit:
tsdbError("%s failed at %s:%d since %s", __func__, fname, __LINE__, tstrerror(code));
json[0] = NULL;
}
(void)taosCloseFile(&fp);
taosCloseFileWithLog(&fp);
taosMemoryFree(data);
return code;
}
@ -803,7 +810,11 @@ void tsdbEnableBgTask(STsdb *pTsdb) {
void tsdbCloseFS(STFileSystem **fs) {
if (fs[0] == NULL) return;
TAOS_UNUSED(tsdbDisableAndCancelAllBgTask((*fs)->tsdb));
int32_t code = tsdbDisableAndCancelAllBgTask((*fs)->tsdb);
if (code) {
tsdbError("vgId:%d %s failed at line %d since %s", TD_VID((*fs)->tsdb->pVnode), __func__, __LINE__,
tstrerror(code));
}
close_file_system(fs[0]);
destroy_fs(fs);
return;
@ -833,7 +844,9 @@ int32_t tsdbFSEditBegin(STFileSystem *fs, const TFileOpArray *opArray, EFEditT e
current_fname(fs->tsdb, current_t, TSDB_FCURRENT_M);
}
(void)tsem_wait(&fs->canEdit);
if (tsem_wait(&fs->canEdit) != 0) {
tsdbError("vgId:%d failed to wait semaphore", TD_VID(fs->tsdb->pVnode));
}
fs->etype = etype;
// edit
@ -865,7 +878,7 @@ static void tsdbFSSetBlockCommit(STFileSet *fset, bool block) {
}
}
int32_t tsdbFSCheckCommit(STsdb *tsdb, int32_t fid) {
void tsdbFSCheckCommit(STsdb *tsdb, int32_t fid) {
(void)taosThreadMutexLock(&tsdb->mutex);
STFileSet *fset;
tsdbFSGetFSet(tsdb->pFS, fid, &fset);
@ -877,7 +890,7 @@ int32_t tsdbFSCheckCommit(STsdb *tsdb, int32_t fid) {
}
}
(void)taosThreadMutexUnlock(&tsdb->mutex);
return 0;
return;
}
// IMPORTANT: the caller must hold fs->tsdb->mutex
@ -939,13 +952,17 @@ _exit:
} else {
tsdbInfo("vgId:%d %s done, etype:%d", TD_VID(fs->tsdb->pVnode), __func__, fs->etype);
}
(void)tsem_post(&fs->canEdit);
if (tsem_post(&fs->canEdit) != 0) {
tsdbError("vgId:%d failed to post semaphore", TD_VID(fs->tsdb->pVnode));
}
return code;
}
int32_t tsdbFSEditAbort(STFileSystem *fs) {
int32_t code = abort_edit(fs);
(void)tsem_post(&fs->canEdit);
if (tsem_post(&fs->canEdit) != 0) {
tsdbError("vgId:%d failed to post semaphore", TD_VID(fs->tsdb->pVnode));
}
return code;
}

8
source/dnode/vnode/src/tsdb/tsdbFS2.h
View File

@ -59,10 +59,10 @@ int32_t tsdbFSEditBegin(STFileSystem *fs, const TFileOpArray *opArray, EFEditT e
int32_t tsdbFSEditCommit(STFileSystem *fs);
int32_t tsdbFSEditAbort(STFileSystem *fs);
// other
void tsdbFSGetFSet(STFileSystem *fs, int32_t fid, STFileSet **fset);
int32_t tsdbFSCheckCommit(STsdb *tsdb, int32_t fid);
void tsdbBeginTaskOnFileSet(STsdb *tsdb, int32_t fid, STFileSet **fset);
void tsdbFinishTaskOnFileSet(STsdb *tsdb, int32_t fid);
void tsdbFSGetFSet(STFileSystem *fs, int32_t fid, STFileSet **fset);
void tsdbFSCheckCommit(STsdb *tsdb, int32_t fid);
void tsdbBeginTaskOnFileSet(STsdb *tsdb, int32_t fid, STFileSet **fset);
void tsdbFinishTaskOnFileSet(STsdb *tsdb, int32_t fid);
// utils
int32_t save_fs(const TFileSetArray *arr, const char *fname);
void current_fname(STsdb *pTsdb, char *fname, EFCurrentT ftype);

2
source/dnode/vnode/src/tsdb/tsdbIter.c
View File

@ -225,7 +225,7 @@ static int32_t tsdbMemTableIterNext(STsdbIter *iter, const TABLEID *tbid) {
iter->row->row = row[0];
TAOS_UNUSED(tsdbTbDataIterNext(iter->memtData->tbIter));
bool r = tsdbTbDataIterNext(iter->memtData->tbIter);
goto _exit;
}

11
source/dnode/vnode/src/tsdb/tsdbMemTable.c
View File

@ -193,7 +193,11 @@ int32_t tsdbDeleteTableData(STsdb *pTsdb, int64_t version, tb_uid_t suid, tb_uid
pMemTable->minVer = TMIN(pMemTable->minVer, version);
pMemTable->maxVer = TMAX(pMemTable->maxVer, version);
TAOS_UNUSED(tsdbCacheDel(pTsdb, suid, uid, sKey, eKey));
if (tsdbCacheDel(pTsdb, suid, uid, sKey, eKey) != 0) {
tsdbError("vgId:%d, failed to delete cache data from table suid:%" PRId64 " uid:%" PRId64 " skey:%" PRId64
" eKey:%" PRId64 " at version %" PRId64,
TD_VID(pTsdb->pVnode), suid, uid, sKey, eKey, version);
}
tsdbTrace("vgId:%d, delete data from table suid:%" PRId64 " uid:%" PRId64 " skey:%" PRId64 " eKey:%" PRId64
" at version %" PRId64,
@ -652,7 +656,10 @@ static int32_t tsdbInsertColDataToTable(SMemTable *pMemTable, STbData *pTbData,
}
if (!TSDB_CACHE_NO(pMemTable->pTsdb->pVnode->config)) {
TAOS_UNUSED(tsdbCacheColFormatUpdate(pMemTable->pTsdb, pTbData->suid, pTbData->uid, pBlockData));
if (tsdbCacheColFormatUpdate(pMemTable->pTsdb, pTbData->suid, pTbData->uid, pBlockData) != 0) {
tsdbError("vgId:%d, failed to update cache data from table suid:%" PRId64 " uid:%" PRId64 " at version %" PRId64,
TD_VID(pMemTable->pTsdb->pVnode), pTbData->suid, pTbData->uid, version);
}
}
// SMemTable

3
source/dnode/vnode/src/tsdb/tsdbMergeTree.c
View File

@ -403,7 +403,8 @@ static int32_t loadSttStatisticsBlockData(SSttFileReader *pSttFileReader, SSttBl
pBlockLoadInfo->cost.loadStatisBlocks += num;
STbStatisBlock block;
TAOS_UNUSED(tStatisBlockInit(&block));
code = tStatisBlockInit(&block);
QUERY_CHECK_CODE(code, lino, _end);
int64_t st = taosGetTimestampUs();

6
source/dnode/vnode/src/tsdb/tsdbReaderWriter.c
View File

@ -436,7 +436,7 @@ static int32_t tsdbReadFileS3(STsdbFD *pFD, int64_t offset, uint8_t *pBuf, int64
code = tsdbCacheGetPageS3(pFD->pTsdb->pgCache, pFD, pgno, &handle);
if (code != TSDB_CODE_SUCCESS) {
if (handle) {
(void)tsdbCacheRelease(pFD->pTsdb->pgCache, handle);
tsdbCacheRelease(pFD->pTsdb->pgCache, handle);
}
TSDB_CHECK_CODE(code, lino, _exit);
}
@ -447,7 +447,7 @@ static int32_t tsdbReadFileS3(STsdbFD *pFD, int64_t offset, uint8_t *pBuf, int64
uint8_t *pPage = (uint8_t *)taosLRUCacheValue(pFD->pTsdb->pgCache, handle);
memcpy(pFD->pBuf, pPage, pFD->szPage);
(void)tsdbCacheRelease(pFD->pTsdb->pgCache, handle);
tsdbCacheRelease(pFD->pTsdb->pgCache, handle);
// check
if (pgno > 1 && !taosCheckChecksumWhole(pFD->pBuf, pFD->szPage)) {
@ -483,7 +483,7 @@ static int32_t tsdbReadFileS3(STsdbFD *pFD, int64_t offset, uint8_t *pBuf, int64
int nPage = pgnoEnd - pgno + 1;
for (int i = 0; i < nPage; ++i) {
if (pFD->szFile != pgno) { // DONOT cache last volatile page
(void)tsdbCacheSetPageS3(pFD->pTsdb->pgCache, pFD, pgno, pBlock + i * pFD->szPage);
tsdbCacheSetPageS3(pFD->pTsdb->pgCache, pFD, pgno, pBlock + i * pFD->szPage);
}
if (szHint > 0 && n >= size) {

43
source/dnode/vnode/src/tsdb/tsdbRetention.c
View File

@ -99,8 +99,12 @@ _exit:
tsdbError("vgId:%d, %s failed at %s:%d since %s", TD_VID(rtner->tsdb->pVnode), __func__, __FILE__, lino,
tstrerror(code));
}
(void)taosCloseFile(&fdFrom);
(void)taosCloseFile(&fdTo);
if (taosCloseFile(&fdFrom) != 0) {
tsdbError("vgId:%d, failed to close file %s", TD_VID(rtner->tsdb->pVnode), fname_from);
}
if (taosCloseFile(&fdTo) != 0) {
tsdbError("vgId:%d, failed to close file %s", TD_VID(rtner->tsdb->pVnode), fname_to);
}
return code;
}
@ -136,8 +140,12 @@ _exit:
tsdbError("vgId:%d, %s failed at %s:%d since %s", TD_VID(rtner->tsdb->pVnode), __func__, __FILE__, lino,
tstrerror(code));
}
(void)taosCloseFile(&fdFrom);
(void)taosCloseFile(&fdTo);
if (taosCloseFile(&fdFrom) != 0) {
tsdbTrace("vgId:%d, failed to close file", TD_VID(rtner->tsdb->pVnode));
}
if (taosCloseFile(&fdTo) != 0) {
tsdbTrace("vgId:%d, failed to close file", TD_VID(rtner->tsdb->pVnode));
}
return code;
}
@ -441,7 +449,9 @@ _exit:
tsdbError("vgId:%d %s failed at line %s:%d since %s", TD_VID(rtner->tsdb->pVnode), __func__, __FILE__, lino,
tstrerror(code));
}
(void)taosCloseFile(&fdFrom);
if (taosCloseFile(&fdFrom) != 0) {
tsdbTrace("vgId:%d, failed to close file", TD_VID(rtner->tsdb->pVnode));
}
return code;
}
@ -541,8 +551,13 @@ _exit:
tsdbError("vgId:%d %s failed at line %s:%d since %s", TD_VID(rtner->tsdb->pVnode), __func__, __FILE__, lino,
tstrerror(code));
}
(void)taosCloseFile(&fdFrom);
(void)taosCloseFile(&fdTo);
if (taosCloseFile(&fdFrom) != 0) {
tsdbTrace("vgId:%d, failed to close file", TD_VID(rtner->tsdb->pVnode));
}
if (taosCloseFile(&fdTo) != 0) {
tsdbTrace("vgId:%d, failed to close file", TD_VID(rtner->tsdb->pVnode));
}
return code;
}
@ -639,8 +654,12 @@ _exit:
tsdbError("vgId:%d %s failed at line %s:%d since %s", TD_VID(rtner->tsdb->pVnode), __func__, __FILE__, lino,
tstrerror(code));
}
(void)taosCloseFile(&fdFrom);
(void)taosCloseFile(&fdTo);
if (taosCloseFile(&fdFrom) != 0) {
tsdbTrace("vgId:%d, failed to close file", TD_VID(rtner->tsdb->pVnode));
}
if (taosCloseFile(&fdTo) != 0) {
tsdbTrace("vgId:%d, failed to close file", TD_VID(rtner->tsdb->pVnode));
}
return code;
}
@ -672,7 +691,7 @@ static int32_t tsdbDoS3Migrate(SRTNer *rtner) {
if (/*lcn < 1 && */ taosCheckExistFile(fobj->fname)) {
int32_t mtime = 0;
int64_t size = 0;
(void)taosStatFile(fobj->fname, &size, &mtime, NULL);
int32_t r = taosStatFile(fobj->fname, &size, &mtime, NULL);
if (size > chunksize && mtime < rtner->now - tsS3UploadDelaySec) {
if (pCfg->s3Compact && lcn < 0) {
extern int32_t tsdbAsyncCompact(STsdb * tsdb, const STimeWindow *tw, bool sync);
@ -699,7 +718,9 @@ static int32_t tsdbDoS3Migrate(SRTNer *rtner) {
if (taosCheckExistFile(fname1)) {
int32_t mtime = 0;
int64_t size = 0;
(void)taosStatFile(fname1, &size, &mtime, NULL);
if (taosStatFile(fname1, &size, &mtime, NULL) != 0) {
tsdbError("vgId:%d, %s failed at %s:%d ", TD_VID(rtner->tsdb->pVnode), __func__, __FILE__, __LINE__);
}
if (size > chunksize && mtime < rtner->now - tsS3UploadDelaySec) {
TAOS_CHECK_GOTO(tsdbMigrateDataFileLCS3(rtner, fobj, size, chunksize), &lino, _exit);
}

24
source/dnode/vnode/src/tsdb/tsdbSnapshot.c
View File

@ -57,10 +57,10 @@ struct STsdbSnapReader {
STombBlock tombBlock[1];
};
static int32_t tsdbSnapReadFileSetCloseReader(STsdbSnapReader* reader) {
static void tsdbSnapReadFileSetCloseReader(STsdbSnapReader* reader) {
TARRAY2_CLEAR(reader->sttReaderArr, tsdbSttFileReaderClose);
tsdbDataFileReaderClose(&reader->dataReader);
return 0;
return;
}
static int32_t tsdbSnapReadFileSetOpenReader(STsdbSnapReader* reader) {
@ -112,7 +112,7 @@ static int32_t tsdbSnapReadFileSetOpenReader(STsdbSnapReader* reader) {
_exit:
if (code) {
TAOS_UNUSED(tsdbSnapReadFileSetCloseReader(reader));
tsdbSnapReadFileSetCloseReader(reader);
TSDB_ERROR_LOG(TD_VID(reader->tsdb->pVnode), code, lino);
}
return code;
@ -190,12 +190,12 @@ _exit:
return code;
}
static int32_t tsdbSnapReadFileSetCloseIter(STsdbSnapReader* reader) {
static void tsdbSnapReadFileSetCloseIter(STsdbSnapReader* reader) {
tsdbIterMergerClose(&reader->dataIterMerger);
tsdbIterMergerClose(&reader->tombIterMerger);
TARRAY2_CLEAR(reader->dataIterArr, tsdbIterClose);
TARRAY2_CLEAR(reader->tombIterArr, tsdbIterClose);
return 0;
return;
}
static int32_t tsdbSnapReadRangeBegin(STsdbSnapReader* reader) {
@ -222,8 +222,8 @@ _exit:
}
static int32_t tsdbSnapReadRangeEnd(STsdbSnapReader* reader) {
TAOS_UNUSED(tsdbSnapReadFileSetCloseIter(reader));
TAOS_UNUSED(tsdbSnapReadFileSetCloseReader(reader));
tsdbSnapReadFileSetCloseIter(reader);
tsdbSnapReadFileSetCloseReader(reader);
reader->ctx->fsr = NULL;
return 0;
}
@ -373,7 +373,7 @@ static int32_t tsdbSnapReadTombData(STsdbSnapReader* reader, uint8_t** data) {
int32_t lino = 0;
SMetaInfo info;
TAOS_UNUSED(tTombBlockClear(reader->tombBlock));
tTombBlockClear(reader->tombBlock);
TABLEID tbid[1] = {0};
for (STombRecord* record; (record = tsdbIterMergerGetTombRecord(reader->tombIterMerger)) != NULL;) {
@ -463,7 +463,7 @@ void tsdbSnapReaderClose(STsdbSnapReader** reader) {
tDestroyTSchema(reader[0]->skmTb->pTSchema);
for (int32_t i = 0; i < ARRAY_SIZE(reader[0]->buffers); ++i) {
TAOS_UNUSED(tBufferDestroy(reader[0]->buffers + i));
tBufferDestroy(reader[0]->buffers + i);
}
taosMemoryFree(reader[0]);
@ -821,7 +821,9 @@ static int32_t tsdbSnapWriteFileSetBegin(STsdbSnapWriter* writer, int32_t fid) {
code = TSDB_CODE_NO_AVAIL_DISK;
TSDB_CHECK_CODE(code, lino, _exit);
}
TAOS_UNUSED(tfsMkdirRecurAt(writer->tsdb->pVnode->pTfs, writer->tsdb->path, writer->ctx->did));
if (tfsMkdirRecurAt(writer->tsdb->pVnode->pTfs, writer->tsdb->path, writer->ctx->did) != 0) {
tsdbError("vgId:%d failed to create directory %s", TD_VID(writer->tsdb->pVnode), writer->tsdb->path);
}
writer->ctx->hasData = true;
writer->ctx->hasTomb = true;
@ -1000,7 +1002,7 @@ static int32_t tsdbSnapWriteDecmprTombBlock(SSnapDataHdr* hdr, STombBlock* tombB
int32_t code = 0;
int32_t lino = 0;
TAOS_UNUSED(tTombBlockClear(tombBlock));
tTombBlockClear(tombBlock);
int64_t size = hdr->size;
size = size / TOMB_RECORD_ELEM_NUM;

4
source/dnode/vnode/src/tsdb/tsdbSttFileRW.c
View File

@ -415,7 +415,7 @@ int32_t tsdbSttFileReadStatisBlock(SSttFileReader *reader, const SStatisBlk *sta
&lino, _exit);
// decode data
TAOS_UNUSED(tStatisBlockClear(statisBlock));
tStatisBlockClear(statisBlock);
statisBlock->numOfPKs = statisBlk->numOfPKs;
statisBlock->numOfRecords = statisBlk->numRec;
SBufferReader br = BUFFER_READER_INITIALIZER(0, buffer0);
@ -654,7 +654,7 @@ static int32_t tsdbSttFileDoWriteStatisBlock(SSttFileWriter *writer) {
TAOS_CHECK_GOTO(TARRAY2_APPEND_PTR(writer->statisBlkArray, &statisBlk), &lino, _exit);
TAOS_UNUSED(tStatisBlockClear(writer->staticBlock));
tStatisBlockClear(writer->staticBlock);
_exit:
if (code) {

7
source/dnode/vnode/src/tsdb/tsdbUtil.c
View File

@ -106,7 +106,7 @@ _exit:
#endif
void tMapDataGetItemByIdx(SMapData *pMapData, int32_t idx, void *pItem, int32_t (*tGetItemFn)(uint8_t *, void *)) {
TAOS_UNUSED(tGetItemFn(pMapData->pData + pMapData->aOffset[idx], pItem));
int32_t r = tGetItemFn(pMapData->pData + pMapData->aOffset[idx], pItem);
}
#ifdef BUILD_NO_CALL
@ -609,7 +609,10 @@ void tsdbRowGetColVal(TSDBROW *pRow, STSchema *pTSchema, int32_t iCol, SColVal *
SValue value;
if (pRow->type == TSDBROW_ROW_FMT) {
(void)tRowGet(pRow->pTSRow, pTSchema, iCol, pColVal);
int32_t ret = tRowGet(pRow->pTSRow, pTSchema, iCol, pColVal);
if (ret != 0) {
tsdbError("failed to get column value, code:%d", ret);
}
} else if (pRow->type == TSDBROW_COL_FMT) {
if (iCol == 0) {
*pColVal =

20
source/dnode/vnode/src/tsdb/tsdbUtil2.c
View File

@ -93,25 +93,25 @@ void tStatisBlockDestroy(STbStatisBlock *statisBlock) {
statisBlock->numOfPKs = 0;
statisBlock->numOfRecords = 0;
for (int32_t i = 0; i < ARRAY_SIZE(statisBlock->buffers); ++i) {
TAOS_UNUSED(tBufferDestroy(&statisBlock->buffers[i]));
tBufferDestroy(&statisBlock->buffers[i]);
}
for (int32_t i = 0; i < TD_MAX_PK_COLS; ++i) {
TAOS_UNUSED(tValueColumnDestroy(&statisBlock->firstKeyPKs[i]));
TAOS_UNUSED(tValueColumnDestroy(&statisBlock->lastKeyPKs[i]));
tValueColumnDestroy(&statisBlock->firstKeyPKs[i]);
tValueColumnDestroy(&statisBlock->lastKeyPKs[i]);
}
}
int32_t tStatisBlockClear(STbStatisBlock *statisBlock) {
void tStatisBlockClear(STbStatisBlock *statisBlock) {
statisBlock->numOfPKs = 0;
statisBlock->numOfRecords = 0;
for (int32_t i = 0; i < ARRAY_SIZE(statisBlock->buffers); ++i) {
TAOS_UNUSED(tBufferClear(&statisBlock->buffers[i]));
tBufferClear(&statisBlock->buffers[i]);
}
for (int32_t i = 0; i < TD_MAX_PK_COLS; ++i) {
tValueColumnClear(&statisBlock->firstKeyPKs[i]);
tValueColumnClear(&statisBlock->lastKeyPKs[i]);
}
return 0;
return;
}
static int32_t tStatisBlockAppend(STbStatisBlock *block, SRowInfo *row) {
@ -252,11 +252,11 @@ void tBrinBlockDestroy(SBrinBlock *brinBlock) {
brinBlock->numOfPKs = 0;
brinBlock->numOfRecords = 0;
for (int32_t i = 0; i < ARRAY_SIZE(brinBlock->buffers); ++i) {
TAOS_UNUSED(tBufferDestroy(&brinBlock->buffers[i]));
tBufferDestroy(&brinBlock->buffers[i]);
}
for (int32_t i = 0; i < TD_MAX_PK_COLS; ++i) {
TAOS_UNUSED(tValueColumnDestroy(&brinBlock->firstKeyPKs[i]));
TAOS_UNUSED(tValueColumnDestroy(&brinBlock->lastKeyPKs[i]));
tValueColumnDestroy(&brinBlock->firstKeyPKs[i]);
tValueColumnDestroy(&brinBlock->lastKeyPKs[i]);
}
}
@ -264,7 +264,7 @@ void tBrinBlockClear(SBrinBlock *brinBlock) {
brinBlock->numOfPKs = 0;
brinBlock->numOfRecords = 0;
for (int32_t i = 0; i < ARRAY_SIZE(brinBlock->buffers); ++i) {
TAOS_UNUSED(tBufferClear(&brinBlock->buffers[i]));
tBufferClear(&brinBlock->buffers[i]);
}
for (int32_t i = 0; i < TD_MAX_PK_COLS; ++i) {
tValueColumnClear(&brinBlock->firstKeyPKs[i]);

2
source/dnode/vnode/src/tsdb/tsdbUtil2.h
View File

@ -113,7 +113,7 @@ typedef struct {
int32_t tStatisBlockInit(STbStatisBlock *statisBlock);
void tStatisBlockDestroy(STbStatisBlock *statisBlock);
int32_t tStatisBlockClear(STbStatisBlock *statisBlock);
void tStatisBlockClear(STbStatisBlock *statisBlock);
int32_t tStatisBlockPut(STbStatisBlock *statisBlock, SRowInfo *row, int32_t maxRecords);
int32_t tStatisBlockGet(STbStatisBlock *statisBlock, int32_t idx, STbStatisRecord *record);

32
source/dnode/vnode/src/vnd/vnodeAsync.c
View File

@ -187,10 +187,12 @@ static void vnodeAsyncCancelAllTasks(SVAsync *async, SArray *cancelArray) {
task->prev->next = task->next;
task->next->prev = task->prev;
if (task->cancel) {
TAOS_UNUSED(taosArrayPush(cancelArray, &(SVATaskCancelInfo){
.cancel = task->cancel,
.arg = task->arg,
}));
if (taosArrayPush(cancelArray, &(SVATaskCancelInfo){
.cancel = task->cancel,
.arg = task->arg,
}) == NULL) {
vError("failed to push cancel task into array");
};
}
vnodeAsyncTaskDone(async, task);
}
@ -430,7 +432,7 @@ static void vnodeAsyncLaunchWorker(SVAsync *async) {
if (async->workers[i].state == EVA_WORKER_STATE_ACTIVE) {
continue;
} else if (async->workers[i].state == EVA_WORKER_STATE_STOP) {
TAOS_UNUSED(taosThreadJoin(async->workers[i].thread, NULL));
int32_t ret = taosThreadJoin(async->workers[i].thread, NULL);
async->workers[i].state = EVA_WORKER_STATE_UINIT;
}
@ -748,10 +750,12 @@ int32_t vnodeAChannelDestroy(SVAChannelID *channelID, bool waitRunning) {
task->prev->next = task->next;
task->next->prev = task->prev;
if (task->cancel) {
TAOS_UNUSED(taosArrayPush(cancelArray, &(SVATaskCancelInfo){
.cancel = task->cancel,
.arg = task->arg,
}));
if (taosArrayPush(cancelArray, &(SVATaskCancelInfo){
.cancel = task->cancel,
.arg = task->arg,
}) == NULL) {
vError("failed to push cancel info");
};
}
vnodeAsyncTaskDone(async, task);
}
@ -763,10 +767,12 @@ int32_t vnodeAChannelDestroy(SVAChannelID *channelID, bool waitRunning) {
channel->scheduled->prev->next = channel->scheduled->next;
channel->scheduled->next->prev = channel->scheduled->prev;
if (channel->scheduled->cancel) {
TAOS_UNUSED(taosArrayPush(cancelArray, &(SVATaskCancelInfo){
.cancel = channel->scheduled->cancel,
.arg = channel->scheduled->arg,
}));
if (taosArrayPush(cancelArray, &(SVATaskCancelInfo){
.cancel = channel->scheduled->cancel,
.arg = channel->scheduled->arg,
}) == NULL) {
vError("failed to push cancel info");
}
}
vnodeAsyncTaskDone(async, channel->scheduled);
}

12
source/dnode/vnode/src/vnd/vnodeCommit.c
View File

@ -201,7 +201,9 @@ _exit:
vInfo("vgId:%d, vnode info is saved, fname:%s replica:%d selfIndex:%d changeVersion:%d", pInfo->config.vgId, fname,
pInfo->config.syncCfg.replicaNum, pInfo->config.syncCfg.myIndex, pInfo->config.syncCfg.changeVersion);
}
(void)taosCloseFile(&pFile);
if (taosCloseFile(&pFile) != 0) {
vError("vgId:%d, failed to close file", pInfo->config.vgId);
}
taosMemoryFree(data);
return code;
}
@ -263,7 +265,9 @@ _exit:
}
}
taosMemoryFree(pData);
(void)taosCloseFile(&pFile);
if (taosCloseFile(&pFile) != 0) {
vError("vgId:%d, failed to close file", pInfo->config.vgId);
}
return code;
}
@ -496,7 +500,9 @@ void vnodeRollback(SVnode *pVnode) {
offset = strlen(tFName);
snprintf(tFName + offset, TSDB_FILENAME_LEN - offset - 1, "%s%s", TD_DIRSEP, VND_INFO_FNAME_TMP);
TAOS_UNUSED(taosRemoveFile(tFName));
if (taosRemoveFile(tFName) != 0) {
vError("vgId:%d, failed to remove file %s since %s", TD_VID(pVnode), tFName, tstrerror(terrno));
}
}
static int vnodeEncodeState(const void *pObj, SJson *pJson) {

15
source/dnode/vnode/src/vnd/vnodeOpen.c
View File

@ -414,7 +414,10 @@ SVnode *vnodeOpen(const char *path, int32_t diskPrimary, STfs *pTfs, SMsgCb msgC
pVnode->blocked = false;
pVnode->disableWrite = false;
(void)tsem_init(&pVnode->syncSem, 0, 0);
if (tsem_init(&pVnode->syncSem, 0, 0) != 0) {
vError("vgId:%d, failed to init semaphore", TD_VID(pVnode));
goto _err;
}
(void)taosThreadMutexInit(&pVnode->mutex, NULL);
(void)taosThreadCondInit(&pVnode->poolNotEmpty, NULL);
@ -449,7 +452,8 @@ SVnode *vnodeOpen(const char *path, int32_t diskPrimary, STfs *pTfs, SMsgCb msgC
// open wal
sprintf(tdir, "%s%s%s", dir, TD_DIRSEP, VNODE_WAL_DIR);
(void)taosRealPath(tdir, NULL, sizeof(tdir));
ret = taosRealPath(tdir, NULL, sizeof(tdir));
TAOS_UNUSED(ret);
pVnode->pWal = walOpen(tdir, &(pVnode->config.walCfg));
if (pVnode->pWal == NULL) {
@ -459,7 +463,8 @@ SVnode *vnodeOpen(const char *path, int32_t diskPrimary, STfs *pTfs, SMsgCb msgC
// open tq
sprintf(tdir, "%s%s%s", dir, TD_DIRSEP, VNODE_TQ_DIR);
(void)taosRealPath(tdir, NULL, sizeof(tdir));
ret = taosRealPath(tdir, NULL, sizeof(tdir));
TAOS_UNUSED(ret);
// open query
if (vnodeQueryOpen(pVnode)) {
@ -541,7 +546,9 @@ void vnodeClose(SVnode *pVnode) {
vnodeCloseBufPool(pVnode);
// destroy handle
(void)tsem_destroy(&pVnode->syncSem);
if (tsem_destroy(&pVnode->syncSem) != 0) {
vError("vgId:%d, failed to destroy semaphore", TD_VID(pVnode));
}
(void)taosThreadCondDestroy(&pVnode->poolNotEmpty);
(void)taosThreadMutexDestroy(&pVnode->mutex);
(void)taosThreadMutexDestroy(&pVnode->lock);

16
source/dnode/vnode/src/vnd/vnodeSnapshot.c
View File

@ -274,13 +274,17 @@ int32_t vnodeSnapRead(SVSnapReader *pReader, uint8_t **ppData, uint32_t *nData)
int64_t size;
code = taosFStatFile(pFile, &size, NULL);
if (code != 0) {
(void)taosCloseFile(&pFile);
if (taosCloseFile(&pFile) != 0) {
vError("vgId:%d, failed to close file", vgId);
}
TSDB_CHECK_CODE(code, lino, _exit);
}
*ppData = taosMemoryMalloc(sizeof(SSnapDataHdr) + size + 1);
if (*ppData == NULL) {
(void)taosCloseFile(&pFile);
if (taosCloseFile(&pFile) != 0) {
vError("vgId:%d, failed to close file", vgId);
}
TSDB_CHECK_CODE(code = terrno, lino, _exit);
}
((SSnapDataHdr *)(*ppData))->type = SNAP_DATA_CFG;
@ -289,11 +293,15 @@ int32_t vnodeSnapRead(SVSnapReader *pReader, uint8_t **ppData, uint32_t *nData)
if (taosReadFile(pFile, ((SSnapDataHdr *)(*ppData))->data, size) < 0) {
taosMemoryFree(*ppData);
(void)taosCloseFile(&pFile);
if (taosCloseFile(&pFile) != 0) {
vError("vgId:%d, failed to close file", vgId);
}
TSDB_CHECK_CODE(code = terrno, lino, _exit);
}
(void)taosCloseFile(&pFile);
if (taosCloseFile(&pFile) != 0) {
vError("vgId:%d, failed to close file", vgId);
}
pReader->cfgDone = 1;
goto _exit;

33
source/dnode/vnode/src/vnd/vnodeSvr.c
View File

@ -210,7 +210,9 @@ static int32_t vnodePreProcessDropTtlMsg(SVnode *pVnode, SRpcMsg *pMsg) {
TSDB_CHECK_CODE(code, lino, _exit);
}
(void)tSerializeSVDropTtlTableReq((char *)pContNew + sizeof(SMsgHead), reqLenNew, &ttlReq);
if (tSerializeSVDropTtlTableReq((char *)pContNew + sizeof(SMsgHead), reqLenNew, &ttlReq) != 0) {
vError("vgId:%d %s:%d failed to serialize drop ttl request", TD_VID(pVnode), __func__, lino);
}
pContNew->contLen = htonl(reqLenNew);
pContNew->vgId = pContOld->vgId;
@ -420,7 +422,9 @@ static int32_t vnodePreProcessDeleteMsg(SVnode *pVnode, SRpcMsg *pMsg) {
((SMsgHead *)pCont)->vgId = TD_VID(pVnode);
tEncoderInit(pCoder, pCont + sizeof(SMsgHead), size);
(void)tEncodeDeleteRes(pCoder, &res);
if (tEncodeDeleteRes(pCoder, &res) != 0) {
vError("vgId:%d %s failed to encode delete response", TD_VID(pVnode), __func__);
}
tEncoderClear(pCoder);
rpcFreeCont(pMsg->pCont);
@ -647,7 +651,9 @@ int32_t vnodeProcessWriteMsg(SVnode *pVnode, SRpcMsg *pMsg, int64_t ver, SRpcMsg
} break;
case TDMT_STREAM_CONSEN_CHKPT: {
if (pVnode->restored) {
(void)tqProcessTaskConsenChkptIdReq(pVnode->pTq, pMsg);
if (tqProcessTaskConsenChkptIdReq(pVnode->pTq, pMsg) < 0) {
goto _err;
}
}
} break;
case TDMT_STREAM_TASK_PAUSE: {
@ -664,7 +670,9 @@ int32_t vnodeProcessWriteMsg(SVnode *pVnode, SRpcMsg *pMsg, int64_t ver, SRpcMsg
} break;
case TDMT_VND_STREAM_TASK_RESET: {
if (pVnode->restored && vnodeIsLeader(pVnode)) {
(void)tqProcessTaskResetReq(pVnode->pTq, pMsg);
if (tqProcessTaskResetReq(pVnode->pTq, pMsg) < 0) {
goto _err;
}
}
} break;
case TDMT_VND_ALTER_CONFIRM:
@ -1215,7 +1223,9 @@ static int32_t vnodeProcessCreateTbReq(SVnode *pVnode, int64_t ver, void *pReq,
int64_t clusterId = pVnode->config.syncCfg.nodeInfo[0].clusterId;
SName name = {0};
(void)tNameFromString(&name, pVnode->config.dbname, T_NAME_ACCT | T_NAME_DB);
if (tNameFromString(&name, pVnode->config.dbname, T_NAME_ACCT | T_NAME_DB) < 0) {
vError("vgId:%d, failed to get name from string", TD_VID(pVnode));
}
SStringBuilder sb = {0};
for (int32_t i = 0; i < tbNames->size; i++) {
@ -1833,7 +1843,9 @@ static int32_t vnodeProcessSubmitReq(SVnode *pVnode, int64_t ver, void *pReq, in
}
if (info.suid) {
(void)metaGetInfo(pVnode->pMeta, info.suid, &info, NULL);
if (metaGetInfo(pVnode->pMeta, info.suid, &info, NULL) != 0) {
vWarn("vgId:%d, table uid:%" PRId64 " not exists", TD_VID(pVnode), info.suid);
}
}
if (pSubmitTbData->sver != info.skmVer) {
@ -1940,7 +1952,9 @@ _exit:
tEncodeSize(tEncodeSSubmitRsp2, pSubmitRsp, pRsp->contLen, ret);
pRsp->pCont = rpcMallocCont(pRsp->contLen);
tEncoderInit(&ec, pRsp->pCont, pRsp->contLen);
(void)tEncodeSSubmitRsp2(&ec, pSubmitRsp);
if (tEncodeSSubmitRsp2(&ec, pSubmitRsp) < 0) {
vError("vgId:%d, failed to encode submit response", TD_VID(pVnode));
}
tEncoderClear(&ec);
// update statistics
@ -2221,7 +2235,10 @@ static int32_t vnodeProcessBatchDeleteReq(SVnode *pVnode, int64_t ver, void *pRe
SBatchDeleteReq deleteReq;
SDecoder decoder;
tDecoderInit(&decoder, pReq, len);
(void)tDecodeSBatchDeleteReq(&decoder, &deleteReq);
if (tDecodeSBatchDeleteReq(&decoder, &deleteReq) < 0) {
tDecoderClear(&decoder);
return terrno = TSDB_CODE_INVALID_MSG;
}
SMetaReader mr = {0};
metaReaderDoInit(&mr, pVnode->pMeta, META_READER_NOLOCK);

24
source/dnode/vnode/src/vnd/vnodeSync.c
View File

@ -28,7 +28,9 @@ static inline void vnodeWaitBlockMsg(SVnode *pVnode, const SRpcMsg *pMsg) {
const STraceId *trace = &pMsg->info.traceId;
vGTrace("vgId:%d, msg:%p wait block, type:%s sec:%d seq:%" PRId64, pVnode->config.vgId, pMsg,
TMSG_INFO(pMsg->msgType), pVnode->blockSec, pVnode->blockSeq);
(void)tsem_wait(&pVnode->syncSem);
if (tsem_wait(&pVnode->syncSem) != 0) {
vError("vgId:%d, failed to wait sem", pVnode->config.vgId);
}
}
static inline void vnodePostBlockMsg(SVnode *pVnode, const SRpcMsg *pMsg) {
@ -41,7 +43,9 @@ static inline void vnodePostBlockMsg(SVnode *pVnode, const SRpcMsg *pMsg) {
pVnode->blocked = false;
pVnode->blockSec = 0;
pVnode->blockSeq = 0;
(void)tsem_post(&pVnode->syncSem);
if (tsem_post(&pVnode->syncSem) != 0) {
vError("vgId:%d, failed to post sem", pVnode->config.vgId);
}
}
(void)taosThreadMutexUnlock(&pVnode->lock);
}
@ -613,7 +617,9 @@ static void vnodeBecomeFollower(const SSyncFSM *pFsm) {
if (pVnode->blocked) {
pVnode->blocked = false;
vDebug("vgId:%d, become follower and post block", pVnode->config.vgId);
(void)tsem_post(&pVnode->syncSem);
if (tsem_post(&pVnode->syncSem) != 0) {
vError("vgId:%d, failed to post sync semaphore", pVnode->config.vgId);
}
}
(void)taosThreadMutexUnlock(&pVnode->lock);
@ -633,7 +639,9 @@ static void vnodeBecomeLearner(const SSyncFSM *pFsm) {
if (pVnode->blocked) {
pVnode->blocked = false;
vDebug("vgId:%d, become learner and post block", pVnode->config.vgId);
(void)tsem_post(&pVnode->syncSem);
if (tsem_post(&pVnode->syncSem) != 0) {
vError("vgId:%d, failed to post sync semaphore", pVnode->config.vgId);
}
}
(void)taosThreadMutexUnlock(&pVnode->lock);
}
@ -766,7 +774,9 @@ void vnodeSyncPreClose(SVnode *pVnode) {
if (pVnode->blocked) {
vInfo("vgId:%d, post block after close sync", pVnode->config.vgId);
pVnode->blocked = false;
(void)tsem_post(&pVnode->syncSem);
if (tsem_post(&pVnode->syncSem) != 0) {
vError("vgId:%d, failed to post block", pVnode->config.vgId);
}
}
(void)taosThreadMutexUnlock(&pVnode->lock);
}
@ -801,7 +811,9 @@ void vnodeSyncCheckTimeout(SVnode *pVnode) {
pVnode->blocked = false;
pVnode->blockSec = 0;
pVnode->blockSeq = 0;
(void)tsem_post(&pVnode->syncSem);
if (tsem_post(&pVnode->syncSem) != 0) {
vError("vgId:%d, failed to post block", pVnode->config.vgId);
}
}
}
(void)taosThreadMutexUnlock(&pVnode->lock);

1
source/libs/catalog/inc/catalogInt.h
View File

@ -334,6 +334,7 @@ typedef struct SCtgViewCache {
typedef struct SCtgTSMACache {
SRWLatch tsmaLock;
SArray* pTsmas; // SArray<STSMACache*>
bool retryFetch;
} SCtgTSMACache;
typedef struct SCtgDBCache {

18
source/libs/catalog/src/ctgCache.c
View File

@ -2997,6 +2997,7 @@ int32_t ctgOpDropTbTSMA(SCtgCacheOperation *operation) {
taosArrayDestroyP(pCtgCache->pTsmas, tFreeAndClearTableTSMAInfo);
pCtgCache->pTsmas = NULL;
pCtgCache->retryFetch = true;
ctgDebug("all tsmas for table dropped: %s.%s", msg->dbFName, msg->tbName);
code = taosHashRemove(dbCache->tsmaCache, msg->tbName, TSDB_TABLE_NAME_LEN);
@ -3975,17 +3976,25 @@ int32_t ctgGetTbTSMAFromCache(SCatalog* pCtg, SCtgTbTSMACtx* pCtx, int32_t dbIdx
// get tsma cache
pCache = taosHashAcquire(dbCache->tsmaCache, tsmaSourceTbName.tname, strlen(tsmaSourceTbName.tname));
if (!pCache || !pCache->pTsmas || pCache->pTsmas->size == 0) {
if (!pCache) {
if (NULL == taosArrayPush(pCtx->pResList, &(SMetaRes){0})) {
ctgReleaseTSMAToCache(pCtg, dbCache, pCache);
CTG_ERR_RET(terrno);
}
continue;
}
CTG_LOCK(CTG_READ, &pCache->tsmaLock);
if ((!pCache->pTsmas || pCache->pTsmas->size == 0) && !pCache->retryFetch) {
if (NULL == taosArrayPush(pCtx->pResList, &(SMetaRes){0})) {
ctgReleaseTSMAToCache(pCtg, dbCache, pCache);
CTG_ERR_RET(terrno);
}
CTG_UNLOCK(CTG_READ, &pCache->tsmaLock);
taosHashRelease(dbCache->tsmaCache, pCache);
continue;
}
CTG_LOCK(CTG_READ, &pCache->tsmaLock);
if (hasOutOfDateTSMACache(pCache->pTsmas)) {
if (pCache->retryFetch || hasOutOfDateTSMACache(pCache->pTsmas)) {
CTG_UNLOCK(CTG_READ, &pCache->tsmaLock);
taosHashRelease(dbCache->tsmaCache, pCache);
@ -3997,6 +4006,7 @@ int32_t ctgGetTbTSMAFromCache(SCatalog* pCtg, SCtgTbTSMACtx* pCtx, int32_t dbIdx
}
CTG_CACHE_NHIT_INC(CTG_CI_TBL_TSMA, 1);
pCache->retryFetch = false;
continue;
}

11
source/libs/executor/src/aggregateoperator.c
View File

@ -152,12 +152,17 @@ _error:
}
void destroyAggOperatorInfo(void* param) {
if (param == NULL) {
return;
}
SAggOperatorInfo* pInfo = (SAggOperatorInfo*)param;
cleanupBasicInfo(&pInfo->binfo);
cleanupResultInfo(pInfo->pOperator->pTaskInfo, &pInfo->pOperator->exprSupp, pInfo->aggSup.pResultBuf,
&pInfo->groupResInfo, pInfo->aggSup.pResultRowHashTable);
pInfo->pOperator = NULL;
if (pInfo->pOperator) {
cleanupResultInfo(pInfo->pOperator->pTaskInfo, &pInfo->pOperator->exprSupp, pInfo->aggSup.pResultBuf,
&pInfo->groupResInfo, pInfo->aggSup.pResultRowHashTable);
pInfo->pOperator = NULL;
}
cleanupAggSup(&pInfo->aggSup);
cleanupExprSupp(&pInfo->scalarExprSup);
cleanupGroupResInfo(&pInfo->groupResInfo);

12
source/libs/executor/src/cachescanoperator.c
View File

@ -347,11 +347,7 @@ static int32_t doScanCacheNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
SExprSupp* pSup = &pInfo->pseudoExprSup;
code = addTagPseudoColumnData(&pInfo->readHandle, pSup->pExprInfo, pSup->numOfExprs, pRes, pRes->info.rows,
pTaskInfo, NULL);
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
(*ppRes) = NULL;
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
pRes->info.id.groupId = tableListGetTableGroupId(pTableList, pRes->info.id.uid);
pInfo->indexOfBufferedRes += 1;
@ -414,11 +410,7 @@ static int32_t doScanCacheNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
pInfo->pRes->info.id.uid = *(tb_uid_t*)pUid;
code = addTagPseudoColumnData(&pInfo->readHandle, pSup->pExprInfo, pSup->numOfExprs, pInfo->pRes,
pInfo->pRes->info.rows, pTaskInfo, NULL);
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
(*ppRes) = NULL;
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
}
}

2
source/libs/executor/src/dynqueryctrloperator.c
View File

@ -938,7 +938,9 @@ _return:
}
if (code) {
qError("%s failed since %s", __func__, tstrerror(code));
pOperator->pTaskInfo->code = code;
T_LONG_JMP(pOperator->pTaskInfo->env, code);
} else {
seqStableJoinComposeRes(pStbJoin, *pRes);
}

2
source/libs/executor/src/eventwindowoperator.c
View File

@ -155,7 +155,7 @@ _error:
}
void cleanupResultInfoInEventWindow(SOperatorInfo* pOperator, SEventWindowOperatorInfo* pInfo) {
if (pInfo == NULL || pInfo->pRow == NULL) {
if (pInfo == NULL || pInfo->pRow == NULL || pOperator == NULL) {
return;
}
SExprSupp* pSup = &pOperator->exprSupp;

18
source/libs/executor/src/exchangeoperator.c
View File

@ -1094,6 +1094,7 @@ int32_t addDynamicExchangeSource(SOperatorInfo* pOperator) {
int32_t prepareLoadRemoteData(SOperatorInfo* pOperator) {
SExchangeInfo* pExchangeInfo = pOperator->info;
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
if ((OPTR_IS_OPENED(pOperator) && !pExchangeInfo->dynamicOp) ||
(pExchangeInfo->dynamicOp && NULL == pOperator->pOperatorGetParam)) {
return TSDB_CODE_SUCCESS;
@ -1101,23 +1102,26 @@ int32_t prepareLoadRemoteData(SOperatorInfo* pOperator) {
if (pExchangeInfo->dynamicOp) {
code = addDynamicExchangeSource(pOperator);
if (code) {
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
}
int64_t st = taosGetTimestampUs();
if (!pExchangeInfo->seqLoadData) {
int32_t code = prepareConcurrentlyLoad(pOperator);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
code = prepareConcurrentlyLoad(pOperator);
QUERY_CHECK_CODE(code, lino, _end);
pExchangeInfo->openedTs = taosGetTimestampUs();
}
OPTR_SET_OPENED(pOperator);
pOperator->cost.openCost = (taosGetTimestampUs() - st) / 1000.0;
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pOperator->pTaskInfo->code = code;
T_LONG_JMP(pOperator->pTaskInfo->env, code);
}
return TSDB_CODE_SUCCESS;
}

18
source/libs/executor/src/groupoperator.c
View File

@ -76,21 +76,23 @@ static void freeGroupKey(void* param) {
}
static void destroyGroupOperatorInfo(void* param) {
SGroupbyOperatorInfo* pInfo = (SGroupbyOperatorInfo*)param;
if (pInfo == NULL) {
if (param == NULL) {
return;
}
SGroupbyOperatorInfo* pInfo = (SGroupbyOperatorInfo*)param;
cleanupBasicInfo(&pInfo->binfo);
taosMemoryFreeClear(pInfo->keyBuf);
taosArrayDestroy(pInfo->pGroupCols);
taosArrayDestroyEx(pInfo->pGroupColVals, freeGroupKey);
cleanupExprSupp(&pInfo->scalarSup);
cleanupResultInfo(pInfo->pOperator->pTaskInfo, &pInfo->pOperator->exprSupp, pInfo->aggSup.pResultBuf,
&pInfo->groupResInfo, pInfo->aggSup.pResultRowHashTable);
if (pInfo->pOperator) {
cleanupResultInfo(pInfo->pOperator->pTaskInfo, &pInfo->pOperator->exprSupp, pInfo->aggSup.pResultBuf,
&pInfo->groupResInfo, pInfo->aggSup.pResultRowHashTable);
pInfo->pOperator = NULL;
}
cleanupGroupResInfo(&pInfo->groupResInfo);
cleanupAggSup(&pInfo->aggSup);
pInfo->pOperator = NULL;
taosMemoryFreeClear(param);
}
@ -450,7 +452,7 @@ static int32_t hashGroupbyAggregateNext(SOperatorInfo* pOperator, SSDataBlock**
QRY_PARAM_CHECK(ppRes);
if (pOperator->status == OP_EXEC_DONE) {
return TSDB_CODE_SUCCESS;
return code;
}
if (pOperator->status == OP_RES_TO_RETURN) {
@ -502,6 +504,7 @@ _end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
} else {
(*ppRes) = buildGroupResultDataBlockByHash(pOperator);
}
@ -1533,8 +1536,9 @@ static int32_t doStreamHashPartitionNext(SOperatorInfo* pOperator, SSDataBlock**
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
(*ppRes) = NULL;
return code;

38
source/libs/executor/src/hashjoinoperator.c
View File

@ -993,6 +993,7 @@ static int32_t hJoinMainProcess(struct SOperatorInfo* pOperator, SSDataBlock** p
SHJoinOperatorInfo* pJoin = pOperator->info;
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
SSDataBlock* pRes = pJoin->finBlk;
int64_t st = 0;
@ -1003,7 +1004,7 @@ static int32_t hJoinMainProcess(struct SOperatorInfo* pOperator, SSDataBlock** p
if (pOperator->status == OP_EXEC_DONE) {
pRes->info.rows = 0;
goto _return;
goto _end;
}
if (!pJoin->keyHashBuilt) {
@ -1011,13 +1012,10 @@ static int32_t hJoinMainProcess(struct SOperatorInfo* pOperator, SSDataBlock** p
bool queryDone = false;
code = hJoinBuildHash(pOperator, &queryDone);
if (code) {
pTaskInfo->code = code;
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
if (queryDone) {
goto _return;
goto _end;
}
}
@ -1025,17 +1023,11 @@ static int32_t hJoinMainProcess(struct SOperatorInfo* pOperator, SSDataBlock** p
if (pJoin->ctx.rowRemains) {
code = (*pJoin->joinFp)(pOperator);
if (code) {
pTaskInfo->code = code;
return pTaskInfo->code;
}
QUERY_CHECK_CODE(code, lino, _end);
if (pRes->info.rows > 0 && pJoin->pFinFilter != NULL) {
code = doFilter(pRes, pJoin->pFinFilter, NULL);
if (code) {
pTaskInfo->code = code;
return pTaskInfo->code;
}
QUERY_CHECK_CODE(code, lino, _end);
}
if (pRes->info.rows > 0) {
@ -1055,10 +1047,7 @@ static int32_t hJoinMainProcess(struct SOperatorInfo* pOperator, SSDataBlock** p
pJoin->execInfo.probeBlkRows += pBlock->info.rows;
code = hJoinPrepareStart(pOperator, pBlock);
if (code) {
pTaskInfo->code = code;
return pTaskInfo->code;
}
QUERY_CHECK_CODE(code, lino, _end);
if (!hJoinBlkReachThreshold(pJoin, pRes->info.rows)) {
continue;
@ -1066,10 +1055,7 @@ static int32_t hJoinMainProcess(struct SOperatorInfo* pOperator, SSDataBlock** p
if (pRes->info.rows > 0 && pJoin->pFinFilter != NULL) {
code = doFilter(pRes, pJoin->pFinFilter, NULL);
if (code) {
pTaskInfo->code = code;
return pTaskInfo->code;
}
QUERY_CHECK_CODE(code, lino, _end);
}
if (pRes->info.rows > 0) {
@ -1077,11 +1063,15 @@ static int32_t hJoinMainProcess(struct SOperatorInfo* pOperator, SSDataBlock** p
}
}
_return:
_end:
if (pOperator->cost.openCost == 0) {
pOperator->cost.openCost = (taosGetTimestampUs() - st) / 1000.0;
}
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
if (pRes->info.rows > 0) {
*pResBlock = pRes;
}

1
source/libs/executor/src/mergejoinoperator.c
View File

@ -1731,6 +1731,7 @@ int32_t mJoinMainProcess(struct SOperatorInfo* pOperator, SSDataBlock** pResBloc
if (pJoin->pFinFilter != NULL) {
code = doFilter(pBlock, pJoin->pFinFilter, NULL);
if (code) {
qError("%s failed at line %d since %s", __func__, __LINE__, tstrerror(code));
pJoin->errCode = code;
T_LONG_JMP(pOperator->pTaskInfo->env, pJoin->errCode);
}

22
source/libs/executor/src/mergeoperator.c
View File

@ -492,6 +492,8 @@ int32_t openMultiwayMergeOperator(SOperatorInfo* pOperator) {
pOperator->status = OP_RES_TO_RETURN;
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, __LINE__, tstrerror(code));
pOperator->pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, terrno);
}
@ -501,6 +503,8 @@ int32_t openMultiwayMergeOperator(SOperatorInfo* pOperator) {
int32_t doMultiwayMerge(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
QRY_PARAM_CHECK(pResBlock);
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
if (pOperator->status == OP_EXEC_DONE) {
return 0;
@ -509,18 +513,12 @@ int32_t doMultiwayMerge(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
SMultiwayMergeOperatorInfo* pInfo = pOperator->info;
int32_t code = pOperator->fpSet._openFn(pOperator);
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
return code;
}
code = pOperator->fpSet._openFn(pOperator);
QUERY_CHECK_CODE(code, lino, _end);
if (NULL != gMultiwayMergeFps[pInfo->type].getNextFn) {
code = (*gMultiwayMergeFps[pInfo->type].getNextFn)(pOperator, pResBlock);
if (code) {
pTaskInfo->code = code;
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
}
if ((*pResBlock) != NULL) {
@ -530,6 +528,12 @@ int32_t doMultiwayMerge(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
setOperatorCompleted(pOperator);
}
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
}

15
source/libs/executor/src/operator.c
View File

@ -881,14 +881,17 @@ SSDataBlock* getNextBlockFromDownstreamRemain(struct SOperatorInfo* pOperator, i
int32_t optrDefaultGetNextExtFn(struct SOperatorInfo* pOperator, SOperatorParam* pParam, SSDataBlock** pRes) {
QRY_PARAM_CHECK(pRes);
int32_t lino = 0;
int32_t code = setOperatorParams(pOperator, pParam, OP_GET_PARAM);
if (TSDB_CODE_SUCCESS != code) {
QUERY_CHECK_CODE(code, lino, _end);
code = pOperator->fpSet.getNextFn(pOperator, pRes);
QUERY_CHECK_CODE(code, lino, _end);
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, __LINE__, tstrerror(code));
pOperator->pTaskInfo->code = code;
} else {
code = pOperator->fpSet.getNextFn(pOperator, pRes);
if (code) {
pOperator->pTaskInfo->code = code;
}
T_LONG_JMP(pOperator->pTaskInfo->env, code);
}
return code;

58
source/libs/executor/src/projectoperator.c
View File

@ -270,6 +270,7 @@ int32_t doProjectOperation(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
SSDataBlock* pRes = pInfo->pRes;
SSDataBlock* pFinalRes = pProjectInfo->pFinalRes;
int32_t code = 0;
int32_t lino = 0;
int64_t st = 0;
int32_t order = pInfo->inputTsOrder;
int32_t scanFlag = 0;
@ -290,9 +291,7 @@ int32_t doProjectOperation(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
if (downstream == NULL) {
code = doGenerateSourceData(pOperator);
if (code != TSDB_CODE_SUCCESS) {
T_LONG_JMP(pTaskInfo->env, code);
}
QUERY_CHECK_CODE(code, lino, _end);
if (pProjectInfo->outputIgnoreGroup) {
pRes->info.id.groupId = 0;
@ -348,20 +347,14 @@ int32_t doProjectOperation(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
}
code = setInputDataBlock(pSup, pBlock, order, scanFlag, false);
if (code) {
T_LONG_JMP(pTaskInfo->env, code);
}
QUERY_CHECK_CODE(code, lino, _end);
code = blockDataEnsureCapacity(pInfo->pRes, pInfo->pRes->info.rows + pBlock->info.rows);
if (code != TSDB_CODE_SUCCESS) {
T_LONG_JMP(pTaskInfo->env, code);
}
QUERY_CHECK_CODE(code, lino, _end);
code = projectApplyFunctions(pSup->pExprInfo, pInfo->pRes, pBlock, pSup->pCtx, pSup->numOfExprs,
pProjectInfo->pPseudoColInfo);
if (code != TSDB_CODE_SUCCESS) {
T_LONG_JMP(pTaskInfo->env, code);
}
QUERY_CHECK_CODE(code, lino, _end);
status = doIngroupLimitOffset(pLimitInfo, pBlock->info.id.groupId, pInfo->pRes, pOperator);
if (status == PROJECT_RETRIEVE_CONTINUE) {
@ -377,11 +370,8 @@ int32_t doProjectOperation(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
pFinalRes->info.version = pRes->info.version;
// continue merge data, ignore the group id
int32_t ret = blockDataMerge(pFinalRes, pRes);
if (ret < 0) {
pTaskInfo->code = code;
return code;
}
code = blockDataMerge(pFinalRes, pRes);
QUERY_CHECK_CODE(code, lino, _end);
if (pFinalRes->info.rows + pRes->info.rows <= pOperator->resultInfo.threshold && (pOperator->status != OP_EXEC_DONE)) {
continue;
@ -390,10 +380,7 @@ int32_t doProjectOperation(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
// do apply filter
code = doFilter(pFinalRes, pOperator->exprSupp.pFilterInfo, NULL);
if (code) {
pTaskInfo->code = code;
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
// when apply the limit/offset for each group, pRes->info.rows may be 0, due to limit constraint.
if (pFinalRes->info.rows > 0 || (pOperator->status == OP_EXEC_DONE)) {
@ -404,10 +391,7 @@ int32_t doProjectOperation(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
// do apply filter
if (pRes->info.rows > 0) {
code = doFilter(pRes, pOperator->exprSupp.pFilterInfo, NULL);
if (code) {
pTaskInfo->code = code;
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
if (pRes->info.rows == 0) {
continue;
@ -436,6 +420,13 @@ int32_t doProjectOperation(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
}
*pResBlock = (p->info.rows > 0)? p:NULL;
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
}
@ -578,14 +569,15 @@ int32_t doApplyIndefinitFunction(SOperatorInfo* pOperator, SSDataBlock** pResBlo
SOptrBasicInfo* pInfo = &pIndefInfo->binfo;
SExprSupp* pSup = &pOperator->exprSupp;
int64_t st = 0;
int32_t code = 0;
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
SSDataBlock* pRes = pInfo->pRes;
blockDataCleanup(pRes);
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
if (pOperator->status == OP_EXEC_DONE) {
return 0;
return code;
}
if (pOperator->cost.openCost == 0) {
@ -637,10 +629,7 @@ int32_t doApplyIndefinitFunction(SOperatorInfo* pOperator, SSDataBlock** pResBlo
}
code = doFilter(pInfo->pRes, pOperator->exprSupp.pFilterInfo, NULL);
if (code) {
pTaskInfo->code = code;
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
size_t rows = pInfo->pRes->info.rows;
if (rows > 0 || pOperator->status == OP_EXEC_DONE) {
@ -658,6 +647,13 @@ int32_t doApplyIndefinitFunction(SOperatorInfo* pOperator, SSDataBlock** pResBlo
}
*pResBlock = (rows > 0) ? pInfo->pRes : NULL;
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
}

55
source/libs/executor/src/scanoperator.c
View File

@ -951,7 +951,8 @@ static int32_t doTableScanImplNext(SOperatorInfo* pOperator, SSDataBlock** ppRes
if (isTaskKilled(pTaskInfo)) {
pAPI->tsdReader.tsdReaderReleaseDataBlock(pTableScanInfo->base.dataReader);
return pTaskInfo->code;
code = pTaskInfo->code;
goto _end;
}
if (pOperator->status == OP_EXEC_DONE) {
@ -996,6 +997,7 @@ _end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
}
@ -1416,6 +1418,7 @@ _end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
@ -2944,8 +2947,9 @@ static int32_t doQueueScanNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
if (pResult && pResult->info.rows > 0) {
bool hasPrimaryKey = pAPI->tqReaderFn.tqGetTablePrimaryKey(pInfo->tqReader);
code = processPrimaryKey(pResult, hasPrimaryKey, &pTaskInfo->streamInfo.currentOffset);
QUERY_CHECK_CODE(code, lino, _end);
qDebug("tmqsnap doQueueScan get data utid:%" PRId64 "", pResult->info.id.uid);
if (pResult->info.rows > 0 || code != TSDB_CODE_SUCCESS) {
if (pResult->info.rows > 0) {
(*ppRes) = pResult;
return code;
}
@ -3009,8 +3013,9 @@ static int32_t doQueueScanNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
(*ppRes) = NULL;
return code;
@ -3340,9 +3345,7 @@ FETCH_NEXT_BLOCK:
if (pBlock->info.parTbName[0]) {
code =
pAPI->stateStore.streamStatePutParName(pStreamInfo->pState, pBlock->info.id.groupId, pBlock->info.parTbName);
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
}
QUERY_CHECK_CODE(code, lino, _end);
}
// TODO move into scan
@ -3482,7 +3485,7 @@ FETCH_NEXT_BLOCK:
return code;
}
qError("%s===stream=== %s failed at line %d since pInfo->pUpdateRes is empty", GET_TASKID(pTaskInfo), __func__,
lino);
__LINE__);
blockDataCleanup(pInfo->pUpdateDataRes);
pInfo->scanMode = STREAM_SCAN_FROM_READERHANDLE;
} break;
@ -3496,7 +3499,7 @@ FETCH_NEXT_BLOCK:
return code;
}
qError("%s===stream=== %s failed at line %d since pInfo->pUpdateRes is empty", GET_TASKID(pTaskInfo), __func__,
lino);
__LINE__);
blockDataCleanup(pInfo->pUpdateDataRes);
pInfo->scanMode = STREAM_SCAN_FROM_READERHANDLE;
} break;
@ -3658,8 +3661,9 @@ FETCH_NEXT_BLOCK:
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
(*ppRes) = NULL;
return code;
@ -3730,6 +3734,7 @@ static int32_t doRawScanNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
if (pBlock && pBlock->info.rows > 0) {
bool hasPrimaryKey = pAPI->snapshotFn.taosXGetTablePrimaryKey(pInfo->sContext);
code = processPrimaryKey(pBlock, hasPrimaryKey, &pTaskInfo->streamInfo.currentOffset);
QUERY_CHECK_CODE(code, lino, _end);
qDebug("tmqsnap doRawScan get data uid:%" PRId64 "", pBlock->info.id.uid);
(*ppRes) = pBlock;
return code;
@ -3741,7 +3746,7 @@ static int32_t doRawScanNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
QUERY_CHECK_CODE(code, lino, _end);
if (code != 0) {
tDeleteSchemaWrapper(mtInfo.schema);
goto _end;
QUERY_CHECK_CODE(code, lino, _end);
}
STqOffsetVal offset = {0};
if (mtInfo.uid == 0 || pInfo->sContext->withMeta == ONLY_META) { // read snapshot done, change to get data from wal
@ -3831,6 +3836,7 @@ _end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
(*ppRes) = NULL;
@ -4677,6 +4683,7 @@ _end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
@ -4684,6 +4691,7 @@ _end:
static int32_t doTagScanFromMetaEntryNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
if (pOperator->status == OP_EXEC_DONE) {
(*ppRes) = NULL;
return code;
@ -4699,10 +4707,7 @@ static int32_t doTagScanFromMetaEntryNext(SOperatorInfo* pOperator, SSDataBlock*
int32_t size = 0;
code = tableListGetSize(pInfo->pTableListInfo, &size);
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, __LINE__, tstrerror(code));
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
if (size == 0) {
setTaskStatus(pTaskInfo, TASK_COMPLETED);
@ -4716,11 +4721,11 @@ static int32_t doTagScanFromMetaEntryNext(SOperatorInfo* pOperator, SSDataBlock*
while (pInfo->curPos < size && count < pOperator->resultInfo.capacity) {
code = doTagScanOneTable(pOperator, pRes, count, &mr, &pTaskInfo->storageAPI);
if (code == TSDB_CODE_OUT_OF_MEMORY) {
break;
} else {
if (code != TSDB_CODE_OUT_OF_MEMORY) {
// ignore other error
code = TSDB_CODE_SUCCESS;
}
QUERY_CHECK_CODE(code, lino, _end);
++count;
if (++pInfo->curPos >= size) {
@ -4744,6 +4749,13 @@ static int32_t doTagScanFromMetaEntryNext(SOperatorInfo* pOperator, SSDataBlock*
pOperator->resultInfo.totalRows += pRes->info.rows;
(*ppRes) = (pRes->info.rows == 0) ? NULL : pInfo->pRes;
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
}
@ -5429,6 +5441,7 @@ _end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
} else {
(*ppRes) = pBlock;
}
@ -5945,6 +5958,7 @@ _end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
} else {
(*ppRes) = pBlock;
}
@ -6460,7 +6474,12 @@ static int32_t doTableCountScanNext(SOperatorInfo* pOperator, SSDataBlock** ppRe
}
code = buildVnodeDbTableCount(pOperator, pInfo, pSupp, pRes);
if ((pRes->info.rows > 0) && (code == 0)) {
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed since %s", __func__, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
if (pRes->info.rows > 0) {
*ppRes = pRes;
}

103
source/libs/executor/src/sortoperator.c
View File

@ -349,82 +349,84 @@ void applyScalarFunction(SSDataBlock* pBlock, void* param) {
int32_t doOpenSortOperator(SOperatorInfo* pOperator) {
SSortOperatorInfo* pInfo = pOperator->info;
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
SSortSource* pSource =NULL;
if (OPTR_IS_OPENED(pOperator)) {
return TSDB_CODE_SUCCESS;
return code;
}
pInfo->startTs = taosGetTimestampUs();
// pInfo->binfo.pRes is not equalled to the input datablock.
pInfo->pSortHandle = NULL;
int32_t code =
code =
tsortCreateSortHandle(pInfo->pSortInfo, SORT_SINGLESOURCE_SORT, -1, -1, NULL, pTaskInfo->id.str, pInfo->maxRows,
pInfo->maxTupleLength, tsPQSortMemThreshold * 1024 * 1024, &pInfo->pSortHandle);
if (code) {
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
tsortSetFetchRawDataFp(pInfo->pSortHandle, loadNextDataBlock, applyScalarFunction, pOperator);
SSortSource* pSource = taosMemoryCalloc(1, sizeof(SSortSource));
if (pSource == NULL) {
return terrno;
}
pSource = taosMemoryCalloc(1, sizeof(SSortSource));
QUERY_CHECK_NULL(pSource, code, lino, _end, terrno);
pSource->param = pOperator->pDownstream[0];
pSource->onlyRef = true;
code = tsortAddSource(pInfo->pSortHandle, pSource);
if (code) {
taosMemoryFree(pSource);
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
pSource = NULL;
code = tsortOpen(pInfo->pSortHandle);
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
} else {
pOperator->cost.openCost = (taosGetTimestampUs() - pInfo->startTs) / 1000.0;
pOperator->status = OP_RES_TO_RETURN;
OPTR_SET_OPENED(pOperator);
}
QUERY_CHECK_CODE(code, lino, _end);
pOperator->cost.openCost = (taosGetTimestampUs() - pInfo->startTs) / 1000.0;
pOperator->status = OP_RES_TO_RETURN;
OPTR_SET_OPENED(pOperator);
_end:
if (pSource) {
taosMemoryFree(pSource);
}
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
}
int32_t doSort(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
QRY_PARAM_CHECK(pResBlock);
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
if (pOperator->status == OP_EXEC_DONE) {
return 0;
return code;
}
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
SSortOperatorInfo* pInfo = pOperator->info;
int32_t code = pOperator->fpSet._openFn(pOperator);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
code = pOperator->fpSet._openFn(pOperator);
QUERY_CHECK_CODE(code, lino, _end);
// multi-group case not handle here
SSDataBlock* pBlock = NULL;
while (1) {
if (tsortIsClosed(pInfo->pSortHandle)) {
code = TSDB_CODE_TSC_QUERY_CANCELLED;
T_LONG_JMP(pTaskInfo->env, code);
QUERY_CHECK_CODE(code, lino, _end);
}
code = getSortedBlockData(pInfo->pSortHandle, pInfo->binfo.pRes, pOperator->resultInfo.capacity,
pInfo->matchInfo.pList, pInfo, &pBlock);
if (pBlock == NULL || code != 0) {
QUERY_CHECK_CODE(code, lino, _end);
if (pBlock == NULL) {
setOperatorCompleted(pOperator);
return code;
}
code = doFilter(pBlock, pOperator->exprSupp.pFilterInfo, &pInfo->matchInfo);
if (code) {
break;
}
QUERY_CHECK_CODE(code, lino, _end);
if (blockDataGetNumOfRows(pBlock) == 0) {
continue;
@ -443,6 +445,12 @@ int32_t doSort(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
}
*pResBlock = blockDataGetNumOfRows(pBlock) > 0 ? pBlock : NULL;
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
}
@ -692,16 +700,16 @@ int32_t doGroupSort(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
QRY_PARAM_CHECK(pResBlock);
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
SGroupSortOperatorInfo* pInfo = pOperator->info;
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
if (pOperator->status == OP_EXEC_DONE) {
return 0;
}
int32_t code = pOperator->fpSet._openFn(pOperator);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
code = pOperator->fpSet._openFn(pOperator);
QUERY_CHECK_CODE(code, lino, _end);
if (!pInfo->hasGroupId) {
pInfo->hasGroupId = true;
@ -714,30 +722,25 @@ int32_t doGroupSort(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
pInfo->currGroupId = pInfo->prefetchedSortInput->info.id.groupId;
pInfo->childOpStatus = CHILD_OP_NEW_GROUP;
code = beginSortGroup(pOperator);
if (code) {
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
}
SSDataBlock* pBlock = NULL;
while (pInfo->pCurrSortHandle != NULL) {
if (tsortIsClosed(pInfo->pCurrSortHandle)) {
code = TSDB_CODE_TSC_QUERY_CANCELLED;
T_LONG_JMP(pTaskInfo->env, code);
QUERY_CHECK_CODE(code, lino, _end);
}
// beginSortGroup would fetch all child blocks of pInfo->currGroupId;
if (pInfo->childOpStatus == CHILD_OP_SAME_GROUP) {
pTaskInfo->code = code = TSDB_CODE_QRY_EXECUTOR_INTERNAL_ERROR;
qError("%s failed at line %d since %s", __func__, __LINE__, tstrerror(code));
return code;
code = TSDB_CODE_QRY_EXECUTOR_INTERNAL_ERROR;
QUERY_CHECK_CODE(code, lino, _end);
}
code = getGroupSortedBlockData(pInfo->pCurrSortHandle, pInfo->binfo.pRes, pOperator->resultInfo.capacity,
pInfo->matchInfo.pList, pInfo, &pBlock);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
if (pBlock != NULL) {
pBlock->info.id.groupId = pInfo->currGroupId;
pOperator->resultInfo.totalRows += pBlock->info.rows;
@ -748,9 +751,7 @@ int32_t doGroupSort(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
(void) finishSortGroup(pOperator);
pInfo->currGroupId = pInfo->prefetchedSortInput->info.id.groupId;
code = beginSortGroup(pOperator);
if (code) {
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
} else if (pInfo->childOpStatus == CHILD_OP_FINISHED) {
(void) finishSortGroup(pOperator);
setOperatorCompleted(pOperator);
@ -759,6 +760,12 @@ int32_t doGroupSort(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
}
}
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
return code;
}

14
source/libs/executor/src/streamcountwindowoperator.c
View File

@ -46,11 +46,16 @@ typedef struct SBuffInfo {
} SBuffInfo;
void destroyStreamCountAggOperatorInfo(void* param) {
if (param == NULL) {
return;
}
SStreamCountAggOperatorInfo* pInfo = (SStreamCountAggOperatorInfo*)param;
cleanupBasicInfo(&pInfo->binfo);
cleanupResultInfoInStream(pInfo->pOperator->pTaskInfo, pInfo->streamAggSup.pState, &pInfo->pOperator->exprSupp,
&pInfo->groupResInfo);
pInfo->pOperator = NULL;
if (pInfo->pOperator) {
cleanupResultInfoInStream(pInfo->pOperator->pTaskInfo, pInfo->streamAggSup.pState, &pInfo->pOperator->exprSupp,
&pInfo->groupResInfo);
pInfo->pOperator = NULL;
}
destroyStreamAggSupporter(&pInfo->streamAggSup);
cleanupExprSupp(&pInfo->scalarSupp);
clearGroupResInfo(&pInfo->groupResInfo);
@ -744,8 +749,9 @@ static int32_t doStreamCountAggNext(SOperatorInfo* pOperator, SSDataBlock** ppRe
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s. task:%s", __func__, lino, tstrerror(code), GET_TASKID(pTaskInfo));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
setStreamOperatorCompleted(pOperator);
(*ppRes) = NULL;

15
source/libs/executor/src/streameventwindowoperator.c
View File

@ -48,9 +48,11 @@ void destroyStreamEventOperatorInfo(void* param) {
}
SStreamEventAggOperatorInfo* pInfo = (SStreamEventAggOperatorInfo*)param;
cleanupBasicInfo(&pInfo->binfo);
cleanupResultInfoInStream(pInfo->pOperator->pTaskInfo, pInfo->streamAggSup.pState, &pInfo->pOperator->exprSupp,
&pInfo->groupResInfo);
pInfo->pOperator = NULL;
if (pInfo->pOperator) {
cleanupResultInfoInStream(pInfo->pOperator->pTaskInfo, pInfo->streamAggSup.pState, &pInfo->pOperator->exprSupp,
&pInfo->groupResInfo);
pInfo->pOperator = NULL;
}
destroyStreamAggSupporter(&pInfo->streamAggSup);
clearGroupResInfo(&pInfo->groupResInfo);
taosArrayDestroyP(pInfo->pUpdated, destroyFlusedPos);
@ -179,7 +181,7 @@ _end:
pAggSup->stateStore.streamStateSessionDel(pAggSup->pState, &pCurWin->winInfo.sessionWin);
}
pAggSup->stateStore.streamStateFreeCur(pCur);
qDebug("===stream===set event next win buff. skey:%" PRId64 ", endkey:%" PRId64, pCurWin->winInfo.sessionWin.win.skey,
qDebug("===stream===set event cur win buff. skey:%" PRId64 ", endkey:%" PRId64, pCurWin->winInfo.sessionWin.win.skey,
pCurWin->winInfo.sessionWin.win.ekey);
_error:
@ -233,7 +235,7 @@ int32_t updateEventWindowInfo(SStreamAggSupporter* pAggSup, SEventWindowInfo* pW
pWinInfo->pWinFlag->endFlag = ends[i];
} else if (pWin->ekey == pTsData[i]) {
pWinInfo->pWinFlag->endFlag |= ends[i];
} else {
} else if (ends[i] && !pWinInfo->pWinFlag->endFlag) {
*pRebuild = true;
pWinInfo->pWinFlag->endFlag |= ends[i];
(*pWinRow) = i + 1 - start;
@ -734,8 +736,9 @@ static int32_t doStreamEventAggNext(SOperatorInfo* pOperator, SSDataBlock** ppRe
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s. task:%s", __func__, lino, tstrerror(code), GET_TASKID(pTaskInfo));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
setStreamOperatorCompleted(pOperator);
(*ppRes) = NULL;

3
source/libs/executor/src/streamfilloperator.c
View File

@ -1155,8 +1155,9 @@ static int32_t doStreamFillNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
_end:
if (code != TSDB_CODE_SUCCESS) {
pTaskInfo->code = code;
qError("%s failed at line %d since %s. task:%s", __func__, lino, tstrerror(code), GET_TASKID(pTaskInfo));
pTaskInfo->code = code;
T_LONG_JMP(pTaskInfo->env, code);
}
setOperatorCompleted(pOperator);
resetStreamFillInfo(pInfo);

Some files were not shown because too many files have changed in this diff Show More