xxq250
/
homework-jianmu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Merge branch '3.0' into test/3.0/TS-4893

This commit is contained in:
qevolg 2024-09-14 23:14:17 +08:00
parent 4b5148314d fcf39a444a
commit 46e04d3278
94 changed files with 1952 additions and 650 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/zh/06-advanced/05-data-in/05-opcua.md
View File

@ -169,8 +169,8 @@ CSV 文件中的每个 Row 配置一个 OPC 数据点位。Row 的规则如下
- **采集模式**：可使用 `subscribe` 或 `observe` 模式。
- `subscribe`：订阅模式，变更时上报数据并写入 TDengine。
- `observe`：按照`采集间隔`时长，轮询读取点位最新值并写入 TDengine。
- 采集间隔：默认为 10 秒，数据点位采集间隔，从上次采集数据结束后开始计时，轮询读取点位最新值并写入 TDengine。
- **采集超时**：向 OPC 服务器读取点位数据时如果超过设定时间未返回数据，则读取失败，默认为 10 秒。
- **采集间隔**：默认为 10 秒，数据点位采集间隔，从上次采集数据结束后开始计时，轮询读取点位最新值并写入 TDengine。仅在 **采集模式** 为 `observe` 时可配置。
- **采集超时**：向 OPC 服务器读取点位数据时如果超过设定时间未返回数据，则读取失败，默认为 10 秒。仅在 **采集模式** 为 `observe` 时可配置。
当 **点位集** 中使用 **选择数据点位** 方式时，采集配置中可以配置 **点位更新模式** 和 **点位更新间隔** 来启用动态点位更新。**动态点位更新** 是指，在任务运行期间，OPC Server增加或删除了点位后，符合条件的点位会自动添加到当前任务中，不需要重启 OPC 任务。

6
docs/zh/06-advanced/05-data-in/08-kafka.md
View File

@ -32,7 +32,11 @@ TDengine 可以高效地从 Kafka 读取数据并将其写入 TDengine，以实
### 3. 配置连接信息
在 **连接配置** 区域填写 **bootstrap-servers**，例如：`192.168.1.92:9092`。
**bootstrap-server**，例如：`192.168.1.92`。
**服务端口**，例如：`9092`。
多个 broker 地址时，在连接配置右下增加 **新增 Broker** 按钮，成对增加 bootstrap-server 和服务端口。
![kafka-03.png](./kafka-03.png)

BIN
docs/zh/06-advanced/05-data-in/kafka-03.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 6.5 KiB

After

Width:  |  Height:  |  Size: 45 KiB

36
docs/zh/08-develop/04-schemaless.md
View File

@ -13,7 +13,7 @@ import TabItem from "@theme/TabItem";
值得注意的是，通过无模式写入方式创建的超级表及其对应的子表与通过 SQL 直接创建的超级表和子表在功能上没有区别，用户仍然可以使用 SQL 直接向其中写入数据。然而，由于无模式写入方式生成的表名是基于标签值按照固定的映射规则生成的，因此这些表名可能缺乏可读性，不易于理解。
**采用无模式写入方式时会自动创建表，无须手动创建表。**
**采用无模式写入方式时会自动创建表，无须手动创建表。手动建表的话可能会出现未知的错误。**
## 无模式写入行协议
@ -30,6 +30,7 @@ measurement,tag_set field_set timestamp
- tag_set 格式形如 `<tag_key>=<tag_value>, <tag_key>=<tag_value>`，表示标签列数据，使用英文逗号分隔，与 field_set 之间使用一个半角空格分隔。
- field_set 格式形如 `<field_key>=<field_value>, <field_key>=<field_value>`，表示普通列，同样使用英文逗号来分隔，与 timestamp 之间使用一个半角空格分隔。
- timestamp 为本行数据对应的主键时间戳。
- 无模式写入不支持含第二主键列的表的数据写入。
tag_set 中的所有的数据自动转化为 nchar 数据类型，并不需要使用双引号。
在无模式写入数据行协议中，field_set 中的每个数据项都需要对自身的数据类型进行描述，具体要求如下。
@ -81,7 +82,38 @@ st,t1=3,t2=4,t3=t3 c1=3i64,c3="passit",c2=false,c4=4f64 1626006833639000000
需要注意的是，如果描述数据类型后缀时出现大小写错误，或者为数据指定的数据类型有误，均可能引发报错提示而导致数据写入失败。
TDengine提供数据写入的幂等性保证，即用户可以反复调用API进行出错数据的写入操作。无模式写入TDengine的主要处理逻辑请参考TDengine的官方网站，此处不赘述。
TDengine 提供数据写入的幂等性保证，即您可以反复调用 API 进行出错数据的写入操作。但是不提供多行数据写入的原子性保证。即在多行数据一批次写入过程中，会出现部分数据写入成功，部分数据写入失败的情况。
## 无模式写入处理规则
无模式写入按照如下原则来处理行数据：
1. 将使用如下规则来生成子表名：首先将 measurement 的名称和标签的 key 和 value 组合成为如下的字符串
```json
"measurement,tag_key1=tag_value1,tag_key2=tag_value2"
```
- 需要注意的是，这里的 tag_key1, tag_key2 并不是用户输入的标签的原始顺序，而是使用了标签名称按照字符串升序排列后的结果。所以，tag_key1 并不是在行协议中输入的第一个标签。
排列完成以后计算该字符串的 MD5 散列值 "md5_val"。然后将计算的结果与字符串组合生成表名：“t_md5_val”。其中的 “t_” 是固定的前缀，每个通过该映射关系自动生成的表都具有该前缀。
- 如果不想用自动生成的表名，有两种指定子表名的方式(第一种优先级更高)。
1. 通过在taos.cfg里配置 smlAutoChildTableNameDelimiter 参数来指定（`@ # 空格 回车 换行 制表符`除外)。
1. 举例如下：配置 smlAutoChildTableNameDelimiter=- 插入数据为 st,t0=cpu1,t1=4 c1=3 1626006833639000000 则创建的表名为 cpu1-4。
2. 通过在taos.cfg里配置 smlChildTableName 参数来指定。
1. 举例如下：配置 smlChildTableName=tname 插入数据为 st,tname=cpu1,t1=4 c1=3 1626006833639000000 则创建的表名为 cpu1，注意如果多行数据 tname 相同，但是后面的 tag_set 不同，则使用第一行自动建表时指定的 tag_set，其他的行会忽略。
2. 如果解析行协议获得的超级表不存在，则会创建这个超级表（不建议手动创建超级表，不然插入数据可能异常）。
3. 如果解析行协议获得子表不存在，则 Schemaless 会按照步骤 1 或 2 确定的子表名来创建子表。
4. 如果数据行中指定的标签列或普通列不存在，则在超级表中增加对应的标签列或普通列（只增不减）。
5. 如果超级表中存在一些标签列或普通列未在一个数据行中被指定取值，那么这些列的值在这一行中会被置为 NULL。
6. 对 BINARY 或 NCHAR 列，如果数据行中所提供值的长度超出了列类型的限制，自动增加该列允许存储的字符长度上限（只增不减），以保证数据的完整保存。
7. 整个处理过程中遇到的错误会中断写入过程，并返回错误代码。
8. 为了提高写入的效率，默认假设同一个超级表中 field_set 的顺序是一样的（第一条数据包含所有的 field，后面的数据按照这个顺序），如果顺序不一样，需要配置参数 smlDataFormat 为 false，否则，数据写入按照相同顺序写入，库中数据会异常，从3.0.3.0开始，自动检测顺序是否一致，该配置废弃。
9. 由于sql建表表名不支持点号（.），所以schemaless也对点号（.）做了处理，如果schemaless自动建表的表名如果有点号（.），会自动替换为下划线（\_）。如果手动指定子表名的话，子表名里有点号（.），同样转化为下划线（\_）。
10. taos.cfg 增加 smlTsDefaultName 配置（值为字符串），只在client端起作用，配置后，schemaless自动建表的时间列名字可以通过该配置设置。不配置的话，默认为 _ts。
11. 无模式写入的数据超级表或子表名区分大小写。
12. 无模式写入仍然遵循 TDengine 对数据结构的底层限制，例如每行数据的总长度不能超过 48KB（从 3.0.5.0 版本开始为 64KB），标签值的总长度不超过16KB。
## 时间分辨率识别

6
docs/zh/14-reference/01-components/01-taosd.md
View File

@ -154,6 +154,12 @@ charset 的有效值是 UTF-8。
| :-----------: | :-------------------------------------------------------------------------: |
| supportVnodes | dnode 支持的最大 vnode 数目，取值范围：0-4096，缺省值： CPU 核数的 2 倍 + 5 |
### 内存相关
| 参数名称 | 参数说明 |
| :----------------: | :---------------------------------------------: |
| rpcQueueMemoryAllowed | 一个 dnode 允许的 rpc 消息占用的内存最大值，单位 bytes，取值范围：10485760-INT64_MAX，缺省值：服务器内存的 1/10 |
| syncLogBufferMemoryAllowed | 一个 dnode 允许的 sync 日志缓存消息占用的内存最大值，单位 bytes，取值范围：10485760-INT64_MAX，缺省值：服务器内存的 1/10，3.1.3.2/3.3.2.13 版本开始生效 |
### 性能调优
| 参数名称 | 参数说明 |

2
docs/zh/14-reference/01-components/02-taosc.md
View File

@ -33,7 +33,7 @@ TDengine 客户端驱动提供了应用编程所需要的全部 API，并且在
|logKeepDays | 日志文件的最长保存时间; 缺省值: 0，表示无限保存; 大于 0 时，日志文件会被重命名为 taosdlog.xxx，其中 xxx 为日志文件最后修改的时间戳|
|smlChildTableName | schemaless 自定义的子表名的 key, 无缺省值 |
|smlAutoChildTableNameDelimiter | schemaless tag之间的连接符，连起来作为子表名，无缺省值 |
|smlTagName | schemaless tag 为空时默认的 tag 名字, 无缺省值 |
|smlTagName | schemaless tag 为空时默认的 tag 名字, 缺省值 "_tag_null" |
|smlTsDefaultName | schemaless自动建表的时间列名字通过该配置设置, 缺省值 "_ts" |
|enableCoreFile | crash 时是否生成 core 文件，0: 不生成， 1： 生成；缺省值：1 |
|enableScience | 是否开启科学计数法显示浮点数; 0: 不开始, 1: 开启；缺省值：1 |

898
docs/zh/14-reference/03-taos-sql/10-function.md
View File

@ -169,14 +169,43 @@ POW(expr1, expr2)
#### ROUND
```sql
ROUND(expr)
ROUND(expr[, digits])
```
**功能说明**：获得指定字段的四舍五入的结果。
其他使用说明参见 CEIL 函数描述。
**返回结果类型**：与指定字段的原始数据类型一致。
**适用数据类型**：
- `expr`：数值类型。
- `digits`：数值类型。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 若 `expr` 或 `digits` 为 NULL，返回 NULL。
- 若指定了`digits`，则会保留 `digits` 位小数，默认为 0。
- 若输入值是 INTEGER 类型, 无论 `digits` 值为多少，都只会返回 INTEGER 类型，不会保留小数。
- `digits` 大于零表示对小数位进行操作，四舍五入到 `digits` 位小数。若小数位数小于 `digits` 位，不进行四舍五入操作，直接返回。
- `digits` 小于零表示丢掉小数位，并将数字四舍五入到小数点左侧 `digits` 位。若小数点左侧的位数小于 `digits`位，返回 0。
- 由于暂未支持 DECIMAL 类型，所以该函数会用 DOUBLE 和 FLOAT 来表示包含小数的结果，但是 DOUBLE 和 FLOAT 是有精度上限的，当位数太多时使用该函数可能没有意义。
- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
**举例**：
```sql
taos> select round(8888.88);
round(8888.88) |
============================
8889.000000000000000 |
taos> select round(8888.88,-1);
round(8888.88,-1) |
============================
8890.000000000000000 |
```
#### SIN
@ -232,6 +261,297 @@ TAN(expr)
**使用说明**：只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
#### PI
```sql
PI()
```
**功能说明**：返回圆周率 π 的值。
**返回结果类型**：DOUBLE。
**适用数据类型**：无。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- π ≈ 3.141592653589793。
- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
**举例**：
```sql
taos> select pi();
pi() |
============================
3.141592653589793 |
```
##### TRUNCATE
```sql
TRUNCATE(expr, digits)
```
**功能说明**：获得指定字段按照指定位数截断的值。
**返回结果类型**：与 `expr` 字段的原始数据类型一致。
**适用数据类型**：
- `expr`：数值类型。
- `digits`：数值类型。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 若 `expr` 或 `digits` 为 NULL，返回 NULL。
- 截取指按保留位数直接进行截取，没有四舍五入。
- `digits` 大于零表示对小数位进行操作，截取到 `digits` 位小数，若小数位数小于 `digits` 位，不进行截取操作，直接返回。
- `digits` 等于零表示丢掉小数位。
- `digits` 小于零表示丢掉小数位，并将小数点左侧 `digits` 位置 `0`。若小数点左侧的位数小于 `digits`位，返回 0。
- 由于暂未支持 DECIMAL 类型，所以该函数会用 DOUBLE 和 FLOAT 来表示包含小数的结果，但是 DOUBLE 和 FLOAT 是有精度上限的，当位数太多时使用该函数可能没有意义。
- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
**举例**：
```sql
taos> select truncate(8888.88, 0);
truncate(8888.88, 0) |
============================
8888.000000000000000 |
taos> select truncate(8888.88, -1);
truncate(8888.88, -1) |
============================
8880.000000000000000 |
```
#### EXP
```sql
EXP(expr)
```
**功能说明**：返回 e（自然对数的底）的指定乘方后的值。
**返回结果类型**：DOUBLE。
**适用数据类型**：数值类型。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 如果 `expr` 为 NULL，返回 NULL。
- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
**举例**：
```sql
taos> select exp(2);
exp(2) |
============================
7.389056098930650 |
```
#### LN
```sql
LN(expr)
```
**功能说明**：返回指定参数的自然对数。
**返回结果类型**：DOUBLE。
**适用数据类型**：数值类型。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 如果 `expr` 为 NULL，返回 NULL。
- 如果 `epxr` 小于等于 0，返回 NULL。
- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
**举例**：
```sql
taos> select ln(10);
ln(10) |
============================
2.302585092994046 |
```
#### MOD
```sql
MOD(expr1, expr2)
```
**功能说明**：计算 expr1 % expr2 的结果。
**返回结果类型**：DOUBLE。
**适用数据类型**：数值类型。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 如果 `expr2` 为 0 则返回 NULL。
- 如果 `expr1` 或 `expr2` 为 NULL，返回 NULL。
- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
**举例**：
``` sql
taos> select mod(10,3);
mod(10,3) |
============================
1.000000000000000 |
taos> select mod(1,0);
mod(1,0) |
============================
NULL |
```
#### RAND
```sql
RAND([seed])
```
**功能说明**：返回一个从0到1均匀分布的随机数。
**返回结果类型**：DOUBLE。
**适用数据类型**：
- `seed`：INTEGER。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 如果指定了 `seed` 值，那么将会用指定的 `seed` 作为随机种子，确保生成的随机数序列具有确定性。
- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
**举例**：
``` sql
taos> select rand();
rand() |
============================
0.202092426923147 |
taos> select rand();
rand() |
============================
0.131537788143166 |
taos> select rand(1);
rand(1) |
============================
0.000007826369259 |
taos> select rand(1);
rand(1) |
============================
0.000007826369259 |
```
#### SIGN
```sql
SIGN(expr)
```
**功能说明**：返回指定参数的符号。
**返回结果类型**：与指定字段的原始数据类型一致。
**适用数据类型**：数值类型。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 如果 `expr` 为负，返回 -1 ，
- 如果 `expr` 为正，返回 1 ，
- 如果 `expr` 为 0 ，返回 0 。
- 如果 `expr` 为 NULL ，返回 NULL 。
- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
**举例**：
```sql
taos> select sign(-1);
sign(-1) |
========================
-1 |
taos> select sign(1);
sign(1) |
========================
1 |
taos> select sign(0);
sign(0) |
========================
0 |
```
#### DEGREES
```sql
DEGREES(expr)
```
**功能说明**：计算指定参数由弧度值转为角度后的值。
**返回结果类型**：DOUBLE。
**适用数据类型**：数值类型。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 如果 `expr` 为 NULL，则返回 NULL。
- degree = radian * 180 / π。
- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
**举例**：
```sql
taos> select degrees(PI());
degrees(pi()) |
============================
180.000000000000000 |
```
#### RADIANS
```sql
RADIANS(expr)
```
**功能说明**：计算指定参数由角度值转为弧度后的值。
**返回结果类型**：DOUBLE。
**适用数据类型**：数值类型。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 如果 `expr` 为 NULL，则返回 NULL。
- radian = degree * π / 180。
- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
**举例**：
```sql
taos> select radians(180);
radians(180) |
============================
3.141592653589793 |
```
### 字符串函数
字符串函数的输入参数为字符串类型，返回结果为数值类型或字符串类型。
@ -252,6 +572,27 @@ CHAR_LENGTH(expr)
**适用于**: 表和超级表。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 与 `LENGTH()` 函数不同在于，对于多字节字符，比如中文字符， `CHAR_LENGTH()` 函数会将其算做一个字符，长度为 1，而 `LENGTH()` 会计算其字节数，长度为 3。比如 `CHAR_LENGTH('你好') = 2`， `LENGTH('你好') = 6`。
- 如果 `expr` 为 NULL，返回 NULL。
**举例**：
```sql
taos> select char_length('Hello world');
char_length('Hello world') |
=============================
11 |
taos> select char_length('你好 世界');
char_length('你好 世界') |
===============================
5 |
```
#### CONCAT
```sql
@ -353,23 +694,150 @@ RTRIM(expr)
**适用于**: 表和超级表。
#### SUBSTR
#### TRIM
```sql
SUBSTR(expr, pos [,len])
TRIM([{LEADING | TRAILING | BOTH} [remstr] FROM] expr)
TRIM([remstr FROM] expr)
```
**功能说明**：从源字符串 str 中的指定位置 pos 开始取一个长度为 len 的子串并返回。如果输入参数 len 被忽略，返回的子串包含从 pos 开始的整个字串。
**功能说明**：返回去掉了所有 remstr 前缀或后缀的字符串 epxr 。
**返回结果类型**：与输入字段的原始类型相同。
**返回结果类型**：与输入字段 epxr 的原始类型相同。
**适用数据类型**：VARCHAR, NCHAR。输入参数 pos 可以为正数，也可以为负数。如果 pos 是正数，表示开始位置从字符串开头正数计算。如果 pos 为负数，表示开始位置从字符串结尾倒数计算。
**适用数据类型**：
- remstr：VARCHAR,NCHAR。
- epxr：VARCHAR,NCHAR。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 第一个可选变量[LEADING | BOTH | TRAILING]指定要剪裁字符串的哪一侧：
- LEADING 将移除字符串开头的指定字符。
- TRAILING 将移除字符串末尾的指定字符。
- BOTH（默认值）将移除字符串开头和末尾的指定字符。
- 第二个可选变量[remstr]指定要裁剪掉的字符串：
- 如果不指定 remstr ，默认裁剪空格。
- remstr 可以指定多个字符，如trim('ab' from 'abacd') ，此时会将 'ab' 看做一个整体来裁剪，得到裁剪结果 'acd'。
- 若 expr 为 NULL, 返回 NULL。
- 该函数是多字节安全的。
**举例**：
```sql
taos> select trim(' a ');
trim(' a ') |
=============================
a |
taos> select trim(leading from ' a ');
trim(leading from ' a ') |
==========================================
a |
taos> select trim(leading 'b' from 'bbbbbbbba ');
trim(leading 'b' from 'bbbbbbbba ') |
==============================================
a |
taos> select trim(both 'b' from 'bbbbbabbbbbb');
trim(both 'b' from 'bbbbbabbbbbb') |
=====================================
a |
```
#### SUBSTRING/SUBSTR
```sql
SUBSTRING/SUBSTR(expr, pos [, len])
SUBSTRING/SUBSTR(expr FROM pos [FOR len])
```
**功能说明**：返回字符串 `expr` 在 `pos` 位置开始的子串，若指定了 `len` ，则返回在 `pos` 位置开始，长度为 `len` 的子串。
**返回结果类型**：与输入字段 `expr` 的原始类型相同。
**适用数据类型**：
- `expr`：VARCHAR,NCHAR。
- `pos`：整数类型。
- `len`：整数类型。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 若 `pos` 为正数，则返回的结果为 `expr` 从左到右开始数 `pos` 位置开始的右侧的子串。
- 若 `pos` 为负数，则返回的结果为 `expr` 从右到左开始数 `pos` 位置开始的右侧的子串。
- 任意参数为 NULL，返回 NULL。
- 该函数是多字节安全的。
- 若 `len` 小于 1，返回空串。
- `pos` 是 1-base 的，若 `pos` 为 0，返回空串。
- 若 `pos` + `len` 大于 `len(expr)`，返回从 `pos` 开始到字符串结尾的子串，等同于执行 `substring(expr, pos)`。
**举例**：
```sql
taos> select substring('tdengine', 0);
substring('tdengine', 0) |
===========================
|
taos> select substring('tdengine', 3);
substring('tdengine', 3) |
===========================
engine |
taos> select substring('tdengine', 3,3);
substring('tdengine', 3,3) |
=============================
eng |
taos> select substring('tdengine', -3,3);
substring('tdengine', -3,3) |
==============================
ine |
taos> select substring('tdengine', -3,-3);
substring('tdengine', -3,-3) |
===============================
|
```
#### SUBSTRING_INDEX
```sql
SUBSTRING_INDEX(expr, delim, count)
```
**功能说明**：返回字符串 `expr` 在出现指定次数分隔符的位置截取的子串。
**返回结果类型**：与输入字段 `expr` 的原始类型相同。
**适用数据类型**：
- `expr`：VARCHAR,NCHAR。
- `delim`：VARCHAR, NCHAR。
- `count`：INTEGER。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 若 `count` 为正数，则返回的结果为 `expr` 从左到右开始数第 `count` 次 出现 `delim` 的位置左侧的字符串。
- 若 `count` 为负数，则返回的结果为 `expr` 从右到左开始数第 `count` 的绝对值次 出现 `delim` 的位置右侧的字符串。
- 任意参数为 NULL，返回 NULL。
- 该函数是多字节安全的。
**举例**：
```sql
taos> select substring_index('www.taosdata.com','.',2);
substring_index('www.taosdata.com','.',2) |
============================================
www.taosdata |
taos> select substring_index('www.taosdata.com','.',-2);
substring_index('www.taosdata.com','.',-2) |
=============================================
taosdata.com |
```
#### UPPER
@ -387,7 +855,182 @@ UPPER(expr)
**适用于**: 表和超级表。
#### CHAR
```sql
CHAR(expr1 [, expr2] [, epxr3] ...)
```
**功能说明**：将输入参数当作整数，并返回这些整数在 ASCII 编码中对应的字符。
**返回结果类型**：VARCHAR。
**适用数据类型**：整数类型,VARCHAR,NCHAR。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 输入的值超过 255 会被转化成多字节的结果，如 `CHAR(256)` 等同于 `CHAR(1,0)`, `CHAR(256 * 256)` 等同于 `CHAR(1,0,0)`。
- 输入参数的 NULL 值会被跳过。
- 输入参数若为字符串类型，会将其转换为数值类型处理。
- 若输入的参数对应的字符为不可打印字符，返回值中仍有该参数对应的字符，但是可能无法显示出来。
**举例**：
```sql
taos> select char(77);
char(77) |
===========
M |
taos> select char(77,77);
char(77,77) |
==============
MM |
taos> select char(77 * 256 + 77);
char(77 * 256 + 77) |
======================
MM |
taos> select char(77,NULL,77);
char(77,null,77) |
===================
MM |
```
#### ASCII
```sql
ASCII(expr)
```
**功能说明**：返回字符串第一个字符的 ASCII 码。
**返回结果数据类型**：BIGINT。
**适用数据类型**：VARCHAR, NCHAR。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 如果 `expr` 为 NULL，返回 NULL。
- 如果 `expr` 的第一个字符为多字节字符，只会返回该字符第一个字节的值对应的 ASCII 码。
**举例**：
```sql
taos> select ascii('testascii');
ascii('testascii') |
=====================
116 |
```
#### POSITION
```sql
POSITION(expr1 IN expr2)
```
**功能说明**：计算字符串 `expr1` 在字符串 `expr2` 中的位置。
**返回结果类型**：BIGINT。
**适用数据类型**：
- `expr1`：VARCHAR, NCHAR。
- `expr2`：VARCHAR, NCHAR。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 若 `expr1` 或 `expr2` 为 NULL，返回 NULL。
- 若 `expr1` 在 `expr2` 中不存在，返回 0。
- 若 `expr1` 为空串，认为 `expr1` 在 `expr2` 中总能匹配成功，返回 1。
- 返回的位置是 1-base 的。
- 该函数是多字节安全的。
**举例**：
```sql
taos> select position('a' in 'cba');
position('a' in 'cba') |
=========================
3 |
taos> select position('' in 'cba');
position('' in 'cba') |
========================
1 |
taos> select position('d' in 'cba');
position('d' in 'cba') |
=========================
0 |
```
#### REPLACE
```sql
REPLACE(expr, from_str, to_str)
```
**功能说明**：将字符串中的 `from_str` 全部替换为 `to_str`。
**返回结果类型**：与输入字段 `expr` 的原始类型相同。
**适用数据类型**：
- `expr`：VARCHAR, NCHAR。
- `from_str`：VARCHAR, NCHAR。
- `to_str`：VARCHAR, NCHAR。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 该函数是大小写敏感的。
- 任意参数为 NULL，返回 NULL。
- 该函数是多字节安全的。
**举例**：
```sql
taos> select replace('aabbccAABBCC', 'AA', 'DD');
replace('aabbccAABBCC', 'AA', 'DD') |
======================================
aabbccDDBBCC |
```
#### REPEAT
```sql
REPEAT(expr, count)
```
**功能说明**：返回将字符串重复指定次数得到的字符串。
**返回结果类型**：与输入字段 `expr` 的原始类型相同。
**适用数据类型**：
- `expr`： VARCHAR,NCHAR。
- `count`：INTEGER。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 若 `count < 1`，返回空串。
- 若 `expr` 或 `count` 为 NULL，返回 NULL。
**举例**：
```sql
taos> select repeat('abc',5);
repeat('abc',5) |
============================
abcabcabcabcabc |
taos> select repeat('abc',-1);
repeat('abc',-1) |
===================
|
```
### 转换函数
转换函数将值从一种数据类型转换为另一种数据类型。
@ -609,22 +1252,39 @@ NOW()
TIMEDIFF(expr1, expr2 [, time_unit])
```
**功能说明**：计算两个时间戳之间的差值，并近似到时间单位 time_unit 指定的精度。
**功能说明**：返回时间戳 `expr1` - `expr2` 的结果，结果可能为负，并近似到时间单位 `time_unit` 指定的精度。
**返回结果数据类型**：BIGINT。
**返回结果类型**：BIGINT。
**应用字段**：表示 UNIX 时间戳的 BIGINT, TIMESTAMP 类型，或符合日期时间格式的 VARCHAR, NCHAR 类型。
**适用于**：表和超级表。
**适用数据类型**：
- `expr1`：表示 UNIX 时间戳的 BIGINT, TIMESTAMP 类型，或符合日期时间格式的 VARCHAR, NCHAR 类型。
- `expr2`：表示 UNIX 时间戳的 BIGINT, TIMESTAMP 类型，或符合日期时间格式的 VARCHAR, NCHAR 类型。
- `time_unit`：见使用说明。
**嵌套子查询支持**：适用于内层查询和外层查询。
**使用说明**：
- 支持的时间单位 time_unit 如下：
1b(纳秒), 1u(微秒)，1a(毫秒)，1s(秒)，1m(分)，1h(小时)，1d(天), 1w(周)。
- 如果时间单位 time_unit 未指定， 返回的时间差值精度与当前 DATABASE 设置的时间精度一致。
- 输入包含不符合时间日期格式的字符串则返回 NULL。
**适用于**: 表和超级表。
**使用说明**：
- 支持的时间单位 `time_unit` 如下： 1b(纳秒), 1u(微秒)，1a(毫秒)，1s(秒)，1m(分)，1h(小时)，1d(天), 1w(周)。
- 如果时间单位 `time_unit` 未指定， 返回的时间差值精度与当前 DATABASE 设置的时间精度一致。
- 输入包含不符合时间日期格式的字符串则返回 NULL。
- `expr1` 或 `expr2` 为 NULL，返回 NULL。
- `time_unit` 为 NULL，等同于未指定时间单位。
- 输入时间戳的精度由所查询表的精度确定, 若未指定表, 则精度为毫秒.
**举例**：
```sql
taos> select timediff('2022-01-01 08:00:00', '2022-01-01 08:00:01',1s);
timediff('2022-01-01 08:00:00', '2022-01-01 08:00:01',1s) |
============================================================
-1 |
taos> select timediff('2022-01-01 08:00:01', '2022-01-01 08:00:00',1s);
timediff('2022-01-01 08:00:01', '2022-01-01 08:00:00',1s) |
============================================================
1 |
```
#### TIMETRUNCATE
@ -695,6 +1355,148 @@ TODAY()
b(纳秒)，u(微秒)，a(毫秒)，s(秒)，m(分)，h(小时)，d(天)，w(周)。
- 返回的时间戳精度与当前 DATABASE 设置的时间精度一致。
#### WEEK
```sql
WEEK(expr [, mode])
```
**功能说明**：返回输入日期的周数。
**返回结果类型**：BIGINT。
**适用数据类型**：
- `expr`：表示 UNIX 时间戳的 BIGINT, TIMESTAMP 类型，或符合日期时间格式的 VARCHAR, NCHAR 类型。
- `mode`：0 - 7 之间的整数。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 若 `expr` 为 NULL，返回 NULL。
- 输入时间戳的精度由所查询表的精度确定, 若未指定表, 则精度为毫秒.
- `mode` 用来指定一周是从周日开始还是周一开始，以及指定返回值范围是 1 - 53 还是 0 - 53。下表详细描述不同的 mode 对应的计算方法：
| Mode | 每周的第一天 | 返回值范围 | 第 1 周的计算方法 |
| ---- | ------ | ------ | ------------------ |
| 0 | 周日 | 0 - 53 | 第一个包含周日的周为第 1 周 |
| 1 | 周一 | 0 - 53 | 第一个包含四天及以上的周为第 1 周 |
| 2 | 周日 | 1 - 53 | 第一个包含周日的周为第 1 周 |
| 3 | 周一 | 1 - 53 | 第一个包含四天及以上的周为第 1 周 |
| 4 | 周日 | 0 - 53 | 第一个包含四天及以上的周为第 1 周 |
| 5 | 周一 | 0 - 53 | 第一个包含周一的周为第 1 周 |
| 6 | 周日 | 1 - 53 | 第一个包含四天及以上的周为第 1 周 |
| 7 | 周一 | 1 - 53 | 第一个包含周一的周为第 1 周 |
- 当返回值范围为0 - 53时，第 1 周之前的日期为第 0 周。
- 当返回值范围为1 - 53时，第 1 周之前的日期为上一年的最后一周。
- 以`2000-01-01`为例，
- 在 `mode=0`时返回值为`0`，因为该年第一个周日为`2000-01-02`，从`2000-01-02`起才是第 1 周，所以 `2000-01-01`为第 0 周，返回 0。
- 在 `mode=1`时返回值为`0`，因为`2000-01-01`所在的周只有两天，分别是 `2000-01-01(周六)`和`2000-01-02(周日)`，所以`2000-01-03`起才是第一周，`2000-01-01`为第 0 周，返回 0。
- 在 `mode=2`时返回值为`52`，因为从`2000-01-02`起才是第 1 周，并且返回值范围为 1-53，所以`2000-01-01`算做上一年的最后一周，即 1999 年的第 52 周，返回 52。
**举例**：
```sql
taos> select week('2000-01-01',0);
week('2000-01-01',0) |
========================
0 |
taos> select week('2000-01-01',1);
week('2000-01-01',1) |
========================
0 |
taos> select week('2000-01-01',2);
week('2000-01-01',2) |
========================
52 |
taos> select week('2000-01-01',3);
week('2000-01-01',3) |
========================
52 |
```
#### WEEKOFYEAR
```sql
WEEKOFYEAR(expr)
```
**功能说明**：返回输入日期的周数。
**返回结果类型**：BIGINT。
**适用数据类型**：表示 UNIX 时间戳的 BIGINT, TIMESTAMP 类型，或符合日期时间格式的 VARCHAR, NCHAR 类型。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 等同于`WEEK(expr, 3)`，即在每周第一天是周一，返回值范围为 1 - 53，第一个包含四天及以上的周为第 1 周的条件下判断输入日期的周数。
- 若 `expr` 为 NULL，返回 NULL。
- 输入时间戳的精度由所查询表的精度确定, 若未指定表, 则精度为毫秒.
**举例**：
```sql
taos> select weekofyear('2000-01-01');
weekofyear('2000-01-01') |
===========================
52 |
```
#### WEEKDAY
```sql
WEEKDAY(expr)
```
**功能说明**：返回输入日期是周几。
**返回结果类型**：BIGINT。
**适用数据类型**：表示 UNIX 时间戳的 BIGINT, TIMESTAMP 类型，或符合日期时间格式的 VARCHAR, NCHAR 类型。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 返回值 0 代表周日，1 代表周一 ... 6 代表周六。
- 若 `expr` 为 NULL，返回 NULL。
- 输入时间戳的精度由所查询表的精度确定, 若未指定表, 则精度为毫秒.
**举例**：
```sql
taos> select weekday('2000-01-01');
weekday('2000-01-01') |
========================
5 |
```
#### DAYOFWEEK
```sql
DAYOFWEEK(expr)
```
**功能说明**：返回输入日期是周几。
**返回结果类型**：BIGINT。
**适用数据类型**：表示 UNIX 时间戳的 BIGINT, TIMESTAMP 类型，或符合日期时间格式的 VARCHAR, NCHAR 类型。
**嵌套子查询支持**：适用于内层查询和外层查询。
**适用于**: 表和超级表。
**使用说明**：
- 返回值 1 代表周日，2 代表周一 ... 7 代表周六
- 若 `expr` 为 NULL，返回 NULL。
- 输入时间戳的精度由所查询表的精度确定, 若未指定表, 则精度为毫秒.
**举例**：
```sql
taos> select dayofweek('2000-01-01');
dayofweek('2000-01-01') |
==========================
7 |
```
## 聚合函数
@ -817,13 +1619,13 @@ SPREAD(expr)
**适用于**：表和超级表。
### STDDEV
### STDDEV/STDDEV_POP
```sql
STDDEV(expr)
STDDEV/STDDEV_POP(expr)
```
**功能说明**：统计表中某列的均方差。
**功能说明**：统计表中某列的总体标准差。
**返回数据类型**：DOUBLE。
@ -831,7 +1633,51 @@ STDDEV(expr)
**适用于**：表和超级表。
**举例**：
```sql
taos> select id from test_stddev;
id |
==============
1 |
2 |
3 |
4 |
5 |
taos> select stddev_pop(id) from test_stddev;
stddev_pop(id) |
============================
1.414213562373095 |
```
### VAR_POP
```sql
VAR_POP(expr)
```
**功能说明**：统计表中某列的总体方差。
**返回数据类型**：DOUBLE。
**适用数据类型**：数值类型。
**适用于**：表和超级表。
**举例**:
```sql
taos> select id from test_var;
id |
==============
3 |
1 |
2 |
4 |
5 |
taos> select var_pop(id) from test_var;
var_pop(id) |
============================
2.000000000000000 |
```
### SUM
```sql
@ -1045,10 +1891,11 @@ MAX(expr)
**返回数据类型**：同应用的字段。
**适用数据类型**：数值类型。
**适用数据类型**：数值类型, VARCHAR，NCHAR。
**适用于**：表和超级表。
**使用说明**：max 函数可以接受字符串作为输入参数，当输入参数为字符串类型时，返回最大的字符串值。
### MIN
@ -1060,10 +1907,11 @@ MIN(expr)
**返回数据类型**：同应用的字段。
**适用数据类型**：数值类型。
**适用数据类型**：数值类型, VARCHAR，NCHAR。
**适用于**：表和超级表。
**使用说明**：min 函数可以接受字符串作为输入参数，当输入参数为字符串类型时，返回最大的字符串值。
### MODE

4
docs/zh/14-reference/03-taos-sql/22-meta.md
View File

@ -93,8 +93,8 @@ TDengine 内置了一个名为 `INFORMATION_SCHEMA` 的数据库，提供对数
| 4 | vgroups | INT | 数据库中有多少个 vgroup。需要注意，`vgroups` 为 TDengine 关键字，作为列名使用时需要使用 ` 进行转义。 |
| 6 | replica | INT | 副本数。需要注意，`replica` 为 TDengine 关键字，作为列名使用时需要使用 ` 进行转义。 |
| 7 | strict | VARCHAR(4) | 废弃参数 |
| 8 | duration | VARCHAR(10) | 单文件存储数据的时间跨度。需要注意，`duration` 为 TDengine 关键字，作为列名使用时需要使用 ` 进行转义。 |
| 9 | keep | VARCHAR(32) | 数据保留时长。需要注意，`keep` 为 TDengine 关键字，作为列名使用时需要使用 ` 进行转义。 |
| 8 | duration | VARCHAR(10) | 单文件存储数据的时间跨度。需要注意，`duration` 为 TDengine 关键字，作为列名使用时需要使用 ` 进行转义。内部存储单位为分钟，查询时有可能转换为天或小时展示 |
| 9 | keep | VARCHAR(32) | 数据保留时长。需要注意，`keep` 为 TDengine 关键字，作为列名使用时需要使用 ` 进行转义。 内部存储单位为分钟，查询时有可能转换为天或小时展示 |
| 10 | buffer | INT | 每个 vnode 写缓存的内存块大小，单位 MB。需要注意，`buffer` 为 TDengine 关键字，作为列名使用时需要使用 ` 进行转义。 |
| 11 | pagesize | INT | 每个 VNODE 中元数据存储引擎的页大小，单位为 KB。需要注意，`pagesize` 为 TDengine 关键字，作为列名使用时需要使用 ` 进行转义。 |
| 12 | pages | INT | 每个 vnode 元数据存储引擎的缓存页个数。需要注意，`pages` 为 TDengine 关键字，作为列名使用时需要使用 ` 进行转义。 |

1
include/common/tglobal.h
View File

@ -94,6 +94,7 @@ extern int32_t tsElectInterval;
extern int32_t tsHeartbeatInterval;
extern int32_t tsHeartbeatTimeout;
extern int32_t tsSnapReplMaxWaitN;
extern int64_t tsLogBufferMemoryAllowed; // maximum allowed log buffer size in bytes for each dnode
// arbitrator
extern int32_t tsArbHeartBeatIntervalSec;

6
include/libs/executor/storageapi.h
View File

@ -408,9 +408,9 @@ typedef struct SStateStore {
SStreamStateCur* (*streamStateSessionSeekKeyCurrentPrev)(SStreamState* pState, const SSessionKey* key);
SStreamStateCur* (*streamStateSessionSeekKeyCurrentNext)(SStreamState* pState, const SSessionKey* key);
struct SStreamFileState* (*streamFileStateInit)(int64_t memSize, uint32_t keySize, uint32_t rowSize,
uint32_t selectRowSize, GetTsFun fp, void* pFile, TSKEY delMark,
const char* id, int64_t ckId, int8_t type);
int32_t (*streamFileStateInit)(int64_t memSize, uint32_t keySize, uint32_t rowSize, uint32_t selectRowSize,
GetTsFun fp, void* pFile, TSKEY delMark, const char* id, int64_t ckId, int8_t type,
struct SStreamFileState** ppFileState);
void (*streamFileStateDestroy)(struct SStreamFileState* pFileState);
void (*streamFileStateClear)(struct SStreamFileState* pFileState);

10
include/libs/stream/tstreamFileState.h
View File

@ -45,9 +45,9 @@ typedef int32_t (*_state_fun_get_fn)(SStreamFileState* pFileState, void* pKey, i
typedef int32_t (*range_cmpr_fn)(const SSessionKey* pWin1, const SSessionKey* pWin2);
SStreamFileState* streamFileStateInit(int64_t memSize, uint32_t keySize, uint32_t rowSize, uint32_t selectRowSize,
GetTsFun fp, void* pFile, TSKEY delMark, const char* taskId, int64_t checkpointId,
int8_t type);
int32_t streamFileStateInit(int64_t memSize, uint32_t keySize, uint32_t rowSize, uint32_t selectRowSize, GetTsFun fp,
void* pFile, TSKEY delMark, const char* taskId, int64_t checkpointId, int8_t type,
struct SStreamFileState** ppFileState);
void streamFileStateDestroy(SStreamFileState* pFileState);
void streamFileStateClear(SStreamFileState* pFileState);
bool needClearDiskBuff(SStreamFileState* pFileState);
@ -63,7 +63,7 @@ int32_t putFreeBuff(SStreamFileState* pFileState, SRowBuffPos* pPos);
SStreamSnapshot* getSnapshot(SStreamFileState* pFileState);
void flushSnapshot(SStreamFileState* pFileState, SStreamSnapshot* pSnapshot, bool flushState);
void recoverSnapshot(SStreamFileState* pFileState, int64_t ckId);
int32_t recoverSnapshot(SStreamFileState* pFileState, int64_t ckId);
int32_t getSnapshotIdList(SStreamFileState* pFileState, SArray* list);
int32_t deleteExpiredCheckPoint(SStreamFileState* pFileState, TSKEY mark);
@ -89,7 +89,7 @@ int32_t allocSessioncWinBuffByNextPosition(SStreamFileState* pFileState, SStream
const SSessionKey* pWinKey, void** ppVal, int32_t* pVLen);
SRowBuffPos* createSessionWinBuff(SStreamFileState* pFileState, SSessionKey* pKey, void* p, int32_t* pVLen);
void recoverSesssion(SStreamFileState* pFileState, int64_t ckId);
int32_t recoverSesssion(SStreamFileState* pFileState, int64_t ckId);
void sessionWinStateClear(SStreamFileState* pFileState);
void sessionWinStateCleanup(void* pBuff);

2
include/os/osTimezone.h
View File

@ -54,7 +54,7 @@ enum TdTimezone {
TdEastZone12
};
void taosGetSystemTimezone(char *outTimezone, enum TdTimezone *tsTimezone);
int32_t taosGetSystemTimezone(char *outTimezone, enum TdTimezone *tsTimezone);
int32_t taosSetSystemTimezone(const char *inTimezone, char *outTimezone, int8_t *outDaylight, enum TdTimezone *tsTimezone);
#ifdef __cplusplus

2
include/util/taoserror.h
View File

@ -157,6 +157,7 @@ int32_t taosGetErrSize();
#define TSDB_CODE_INTERNAL_ERROR TAOS_DEF_ERROR_CODE(0, 0x0138)
#define TSDB_CODE_SOCKET_ERROR TAOS_DEF_ERROR_CODE(0, 0x0139)
#define TSDB_CODE_UNSUPPORT_OS TAOS_DEF_ERROR_CODE(0, 0x013A)
#define TSDB_CODE_TIME_ERROR TAOS_DEF_ERROR_CODE(0, 0x013B)
//client
#define TSDB_CODE_TSC_INVALID_OPERATION TAOS_DEF_ERROR_CODE(0, 0x0200)
@ -966,6 +967,7 @@ int32_t taosGetErrSize();
#define TSDB_CODE_TMQ_REPLAY_NEED_ONE_VGROUP TAOS_DEF_ERROR_CODE(0, 0x4013)
#define TSDB_CODE_TMQ_REPLAY_NOT_SUPPORT TAOS_DEF_ERROR_CODE(0, 0x4014)
#define TSDB_CODE_TMQ_NO_TABLE_QUALIFIED TAOS_DEF_ERROR_CODE(0, 0x4015)
#define TSDB_CODE_TMQ_NO_NEED_REBALANCE TAOS_DEF_ERROR_CODE(0, 0x4016)
// stream
#define TSDB_CODE_STREAM_TASK_NOT_EXIST TAOS_DEF_ERROR_CODE(0, 0x4100)

1
include/util/tdef.h
View File

@ -332,6 +332,7 @@ typedef enum ELogicConditionType {
#define TSDB_MAX_LEARNER_REPLICA 10
#define TSDB_SYNC_LOG_BUFFER_SIZE 4096
#define TSDB_SYNC_LOG_BUFFER_RETENTION 256
#define TSDB_SYNC_LOG_BUFFER_THRESHOLD (1024 * 1024 * 5)
#define TSDB_SYNC_APPLYQ_SIZE_LIMIT 512
#define TSDB_SYNC_NEGOTIATION_WIN 512

1
include/util/tref.h
View File

@ -46,6 +46,7 @@ void *taosAcquireRef(int32_t rsetId, int64_t rid);
// release ref, rid is the reference ID returned by taosAddRef
// return 0 if success. On error, -1 is returned, and terrno is set appropriately
int32_t taosReleaseRef(int32_t rsetId, int64_t rid);
int32_t taosReleaseRefEx(int32_t rsetId, int64_t rid, int32_t* isReleased);
// return the first reference if rid is 0, otherwise return the next after current reference.
// if return value is NULL, it means list is over(if terrno is set, it means error happens)

5
source/client/src/clientHb.c
View File

@ -1636,8 +1636,9 @@ void hbDeregisterConn(STscObj *pTscObj, SClientHbKey connKey) {
SClientHbReq *pReq = taosHashAcquire(pAppHbMgr->activeInfo, &connKey, sizeof(SClientHbKey));
if (pReq) {
tFreeClientHbReq(pReq);
if (TSDB_CODE_SUCCESS != taosHashRemove(pAppHbMgr->activeInfo, &connKey, sizeof(SClientHbKey))) {
tscError("failed to remove connKey from activeInfo");
code = taosHashRemove(pAppHbMgr->activeInfo, &connKey, sizeof(SClientHbKey));
if (TSDB_CODE_SUCCESS != code) {
tscError("hbDeregisterConn taosHashRemove error, code:%s", tstrerror(TAOS_SYSTEM_ERROR(code)));
}
taosHashRelease(pAppHbMgr->activeInfo, pReq);
(void)atomic_sub_fetch_32(&pAppHbMgr->connKeyCnt, 1);

12
source/client/src/clientImpl.c
View File

@ -77,7 +77,10 @@ bool chkRequestKilled(void* param) {
return killed;
}
void cleanupAppInfo() { taosHashCleanup(appInfo.pInstMap); }
void cleanupAppInfo() {
taosHashCleanup(appInfo.pInstMap);
taosHashCleanup(appInfo.pInstMapByClusterId);
}
static int32_t taosConnectImpl(const char* user, const char* auth, const char* db, __taos_async_fn_t fp, void* param,
SAppInstInfo* pAppInfo, int connType, STscObj** pTscObj);
@ -1162,6 +1165,7 @@ void schedulerExecCb(SExecResult* pResult, void* param, int32_t code) {
(void)atomic_add_fetch_64((int64_t*)&pActivity->numOfInsertRows, pResult->numOfRows);
}
}
schedulerFreeJob(&pRequest->body.queryJob, 0);
}
taosMemoryFree(pResult);
@ -1276,9 +1280,9 @@ SRequestObj* launchQueryImpl(SRequestObj* pRequest, SQuery* pQuery, bool keepQue
}
if (NEED_CLIENT_RM_TBLMETA_REQ(pRequest->type) && NULL == pRequest->body.resInfo.execRes.res) {
code = removeMeta(pRequest->pTscObj, pRequest->targetTableList, IS_VIEW_REQUEST(pRequest->type));
if (TSDB_CODE_SUCCESS != code) {
tscError("0x%" PRIx64 " remove meta failed,QID:0x%" PRIx64, pRequest->self, pRequest->requestId);
int ret = removeMeta(pRequest->pTscObj, pRequest->targetTableList, IS_VIEW_REQUEST(pRequest->type));
if (TSDB_CODE_SUCCESS != ret) {
tscError("0x%" PRIx64 " remove meta failed,code:%d,QID:0x%" PRIx64, pRequest->self, ret, pRequest->requestId);
}
}

14
source/client/src/clientMain.c
View File

@ -73,6 +73,8 @@ void taos_cleanup(void) {
tscWarn("failed to cleanup task queue");
}
tmqMgmtClose();
int32_t id = clientReqRefPool;
clientReqRefPool = -1;
taosCloseRef(id);
@ -87,9 +89,6 @@ void taos_cleanup(void) {
tscDebug("rpc cleanup");
taosConvDestroy();
tmqMgmtClose();
DestroyRegexCache();
tscInfo("all local resources released");
@ -1316,9 +1315,10 @@ void doAsyncQuery(SRequestObj *pRequest, bool updateMetaForce) {
if (NEED_CLIENT_HANDLE_ERROR(code)) {
tscDebug("0x%" PRIx64 " client retry to handle the error, code:%d - %s, tryCount:%d,QID:0x%" PRIx64,
pRequest->self, code, tstrerror(code), pRequest->retry, pRequest->requestId);
if (TSDB_CODE_SUCCESS != refreshMeta(pRequest->pTscObj, pRequest)) {
tscWarn("0x%" PRIx64 " refresh meta failed, code:%d - %s,QID:0x%" PRIx64, pRequest->self, code, tstrerror(code),
pRequest->requestId);
code = refreshMeta(pRequest->pTscObj, pRequest);
if (code != 0){
tscWarn("0x%" PRIx64 " refresh meta failed, code:%d - %s,QID:0x%" PRIx64, pRequest->self, code,
tstrerror(code), pRequest->requestId);
}
pRequest->prevCode = code;
doAsyncQuery(pRequest, true);
@ -1525,7 +1525,7 @@ int taos_get_table_vgId(TAOS *taos, const char *db, const char *table, int *vgId
conn.mgmtEps = getEpSet_s(&pTscObj->pAppInfo->mgmtEp);
SName tableName;
SName tableName = {0};
toName(pTscObj->acctId, db, table, &tableName);
SVgroupInfo vgInfo;

69
source/client/src/clientMonitor.c
View File

@ -68,9 +68,14 @@ static void destroyMonitorClient(void* data) {
if (pMonitor == NULL) {
return;
}
(void)taosTmrStopA(&pMonitor->timer);
if (!taosTmrStopA(&pMonitor->timer)) {
tscError("failed to stop timer, pMonitor:%p", pMonitor);
}
taosHashCleanup(pMonitor->counters);
(void)taos_collector_registry_destroy(pMonitor->registry);
int ret = taos_collector_registry_destroy(pMonitor->registry);
if (ret){
tscError("failed to destroy registry, pMonitor:%p ret:%d", pMonitor, ret);
}
taosMemoryFree(pMonitor);
}
@ -186,7 +191,10 @@ static void generateClusterReport(taos_collector_registry_t* registry, void* pTr
}
if (strlen(pCont) != 0 && sendReport(pTransporter, epSet, pCont, MONITOR_TYPE_COUNTER, NULL) == 0) {
(void)taos_collector_registry_clear_batch(registry);
int ret = taos_collector_registry_clear_batch(registry);
if (ret){
tscError("failed to clear registry, ret:%d", ret);
}
}
taosMemoryFreeClear(pCont);
}
@ -207,7 +215,10 @@ static void reportSendProcess(void* param, void* tmrId) {
SEpSet ep = getEpSet_s(&pInst->mgmtEp);
generateClusterReport(pMonitor->registry, pInst->pTransporter, &ep);
(void)taosTmrReset(reportSendProcess, pInst->monitorParas.tsMonitorInterval * 1000, param, monitorTimer, &tmrId);
bool reset = taosTmrReset(reportSendProcess, pInst->monitorParas.tsMonitorInterval * 1000, param, monitorTimer, &tmrId);
if (!reset){
tscError("failed to reset timer, pMonitor:%p", pMonitor);
}
taosRUnLockLatch(&monitorLock);
}
@ -255,7 +266,11 @@ void monitorCreateClient(int64_t clusterId) {
goto fail;
}
(void)taos_collector_registry_register_collector(pMonitor->registry, pMonitor->colector);
int r = taos_collector_registry_register_collector(pMonitor->registry, pMonitor->colector);
if (r){
tscError("failed to register collector, ret:%d", r);
goto fail;
}
pMonitor->counters =
(SHashObj*)taosHashInit(64, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_ENTRY_LOCK);
if (pMonitor->counters == NULL) {
@ -304,12 +319,18 @@ void monitorCreateClientCounter(int64_t clusterId, const char* name, const char*
MonitorClient* pMonitor = *ppMonitor;
if (taos_collector_add_metric(pMonitor->colector, newCounter) != 0) {
tscError("failed to add metric to collector");
(void)taos_counter_destroy(newCounter);
int r = taos_counter_destroy(newCounter);
if (r){
tscError("failed to destroy counter, code: %d", r);
}
goto end;
}
if (taosHashPut(pMonitor->counters, name, strlen(name), &newCounter, POINTER_BYTES) != 0) {
tscError("failed to put counter to monitor");
(void)taos_counter_destroy(newCounter);
int r = taos_counter_destroy(newCounter);
if (r){
tscError("failed to destroy counter, code: %d", r);
}
goto end;
}
tscInfo("[monitor] monitorCreateClientCounter %" PRIx64 "(%p):%s : %p.", pMonitor->clusterId, pMonitor, name,
@ -374,7 +395,10 @@ static void monitorWriteSlowLog2File(MonitorSlowLogData* slowLogData, char* tmpP
SlowLogClient* pClient = taosMemoryCalloc(1, sizeof(SlowLogClient));
if (pClient == NULL) {
tscError("failed to allocate memory for slow log client");
(void)taosCloseFile(&pFile);
int32_t ret = taosCloseFile(&pFile);
if (ret != 0){
tscError("failed to close file:%p ret:%d", pFile, ret);
}
return;
}
pClient->lastCheckTime = taosGetMonoTimestampMs();
@ -383,7 +407,10 @@ static void monitorWriteSlowLog2File(MonitorSlowLogData* slowLogData, char* tmpP
pClient->pFile = pFile;
if (taosHashPut(monitorSlowLogHash, &slowLogData->clusterId, LONG_BYTES, &pClient, POINTER_BYTES) != 0) {
tscError("failed to put clusterId:%" PRId64 " to hash table", slowLogData->clusterId);
(void)taosCloseFile(&pFile);
int32_t ret = taosCloseFile(&pFile);
if (ret != 0){
tscError("failed to close file:%p ret:%d", pFile, ret);
}
taosMemoryFree(pClient);
return;
}
@ -609,7 +636,11 @@ static void processFileRemoved(SlowLogClient* pClient) {
tscError("failed to unlock file:%s since %d", pClient->path, terrno);
return;
}
(void)taosCloseFile(&(pClient->pFile));
int32_t ret = taosCloseFile(&(pClient->pFile));
if (ret != 0){
tscError("failed to close file:%p ret:%d", pClient->pFile, ret);
return;
}
TdFilePtr pFile =
taosOpenFile(pClient->path, TD_FILE_CREATE | TD_FILE_WRITE | TD_FILE_APPEND | TD_FILE_READ | TD_FILE_TRUNC);
@ -698,7 +729,10 @@ static void monitorSendAllSlowLogFromTempDir(int64_t clusterId) {
}
if (taosLockFile(pFile) < 0) {
tscInfo("failed to lock file:%s since %s, maybe used by other process", filename, terrstr());
(void)taosCloseFile(&pFile);
int32_t ret = taosCloseFile(&pFile);
if (ret != 0){
tscError("failed to close file:%p ret:%d", pFile, ret);
}
continue;
}
char* tmp = taosStrdup(filename);
@ -706,7 +740,10 @@ static void monitorSendAllSlowLogFromTempDir(int64_t clusterId) {
taosMemoryFree(tmp);
}
(void)taosCloseDir(&pDir);
int32_t ret = taosCloseDir(&pDir);
if (ret != 0){
tscError("failed to close dir, ret:%d", ret);
}
}
static void* monitorThreadFunc(void* param) {
@ -852,7 +889,9 @@ void monitorClose() {
taosHashCleanup(monitorSlowLogHash);
taosTmrCleanUp(monitorTimer);
taosCloseQueue(monitorQueue);
(void)tsem2_destroy(&monitorSem);
if(tsem2_destroy(&monitorSem) != 0) {
tscError("failed to destroy semaphore");
}
taosWUnLockLatch(&monitorLock);
}
@ -874,7 +913,9 @@ int32_t monitorPutData2MonitorQueue(MonitorSlowLogData data) {
tscDebug("[monitor] write slow log to queue, clusterId:%" PRIx64 " type:%s, data:%s", slowLogData->clusterId,
queueTypeStr[slowLogData->type], slowLogData->data);
if (taosWriteQitem(monitorQueue, slowLogData) == 0) {
(void)tsem2_post(&monitorSem);
if(tsem2_post(&monitorSem) != 0) {
tscError("failed to post semaphore");
}
} else {
if (taosCloseFile(&(slowLogData->pFile)) != 0) {
tscError("failed to close file:%p", slowLogData->pFile);

75
source/client/src/clientMsgHandler.c
View File

@ -50,7 +50,9 @@ int32_t genericRspCallback(void* param, SDataBuf* pMsg, int32_t code) {
if (pRequest->body.queryFp != NULL) {
doRequestCallback(pRequest, code);
} else {
(void)tsem_post(&pRequest->body.rspSem);
if (tsem_post(&pRequest->body.rspSem) != 0){
tscError("failed to post semaphore");
}
}
return code;
}
@ -58,12 +60,10 @@ int32_t genericRspCallback(void* param, SDataBuf* pMsg, int32_t code) {
int32_t processConnectRsp(void* param, SDataBuf* pMsg, int32_t code) {
SRequestObj* pRequest = acquireRequest(*(int64_t*)param);
if (NULL == pRequest) {
goto End;
goto EXIT;
}
if (code != TSDB_CODE_SUCCESS) {
setErrno(pRequest, code);
(void)tsem_post(&pRequest->body.rspSem);
goto End;
}
@ -71,23 +71,17 @@ int32_t processConnectRsp(void* param, SDataBuf* pMsg, int32_t code) {
if (NULL == pTscObj->pAppInfo) {
code = TSDB_CODE_TSC_DISCONNECTED;
setErrno(pRequest, code);
(void)tsem_post(&pRequest->body.rspSem);
goto End;
}
SConnectRsp connectRsp = {0};
if (tDeserializeSConnectRsp(pMsg->pData, pMsg->len, &connectRsp) != 0) {
code = TSDB_CODE_TSC_INVALID_VERSION;
setErrno(pRequest, code);
(void)tsem_post(&pRequest->body.rspSem);
goto End;
}
if ((code = taosCheckVersionCompatibleFromStr(version, connectRsp.sVer, 3)) != 0) {
tscError("version not compatible. client version: %s, server version: %s", version, connectRsp.sVer);
setErrno(pRequest, code);
(void)tsem_post(&pRequest->body.rspSem);
goto End;
}
@ -96,15 +90,11 @@ int32_t processConnectRsp(void* param, SDataBuf* pMsg, int32_t code) {
if (delta > timestampDeltaLimit) {
code = TSDB_CODE_TIME_UNSYNCED;
tscError("time diff:%ds is too big", delta);
setErrno(pRequest, code);
(void)tsem_post(&pRequest->body.rspSem);
goto End;
}
if (connectRsp.epSet.numOfEps == 0) {
code = TSDB_CODE_APP_ERROR;
setErrno(pRequest, code);
(void)tsem_post(&pRequest->body.rspSem);
goto End;
}
@ -177,8 +167,6 @@ int32_t processConnectRsp(void* param, SDataBuf* pMsg, int32_t code) {
} else {
(void)taosThreadMutexUnlock(&clientHbMgr.lock);
code = TSDB_CODE_TSC_DISCONNECTED;
setErrno(pRequest, code);
(void)tsem_post(&pRequest->body.rspSem);
goto End;
}
(void)taosThreadMutexUnlock(&clientHbMgr.lock);
@ -186,13 +174,19 @@ int32_t processConnectRsp(void* param, SDataBuf* pMsg, int32_t code) {
tscDebug("0x%" PRIx64 " clusterId:%" PRId64 ", totalConn:%" PRId64, pRequest->requestId, connectRsp.clusterId,
pTscObj->pAppInfo->numOfConns);
(void)tsem_post(&pRequest->body.rspSem);
End:
if (code != 0){
setErrno(pRequest, code);
}
if (tsem_post(&pRequest->body.rspSem) != 0){
tscError("failed to post semaphore");
}
if (pRequest) {
(void)releaseRequest(pRequest->self);
}
EXIT:
taosMemoryFree(param);
taosMemoryFree(pMsg->pEpSet);
taosMemoryFree(pMsg->pData);
@ -245,7 +239,9 @@ int32_t processCreateDbRsp(void* param, SDataBuf* pMsg, int32_t code) {
if (pRequest->body.queryFp) {
doRequestCallback(pRequest, code);
} else {
(void)tsem_post(&pRequest->body.rspSem);
if (tsem_post(&pRequest->body.rspSem) != 0){
tscError("failed to post semaphore");
}
}
return code;
}
@ -285,7 +281,9 @@ int32_t processUseDbRsp(void* param, SDataBuf* pMsg, int32_t code) {
doRequestCallback(pRequest, pRequest->code);
} else {
(void)tsem_post(&pRequest->body.rspSem);
if (tsem_post(&pRequest->body.rspSem) != 0){
tscError("failed to post semaphore");
}
}
return code;
@ -363,7 +361,9 @@ int32_t processUseDbRsp(void* param, SDataBuf* pMsg, int32_t code) {
if (pRequest->body.queryFp != NULL) {
doRequestCallback(pRequest, pRequest->code);
} else {
(void)tsem_post(&pRequest->body.rspSem);
if (tsem_post(&pRequest->body.rspSem) != 0){
tscError("failed to post semaphore");
}
}
return 0;
}
@ -386,7 +386,12 @@ int32_t processCreateSTableRsp(void* param, SDataBuf* pMsg, int32_t code) {
SMCreateStbRsp createRsp = {0};
SDecoder coder = {0};
tDecoderInit(&coder, pMsg->pData, pMsg->len);
(void)tDecodeSMCreateStbRsp(&coder, &createRsp); // pMsg->len == 0
if (pMsg->len > 0){
code = tDecodeSMCreateStbRsp(&coder, &createRsp); // pMsg->len == 0
if (code != TSDB_CODE_SUCCESS) {
setErrno(pRequest, code);
}
}
tDecoderClear(&coder);
pRequest->body.resInfo.execRes.msgType = TDMT_MND_CREATE_STB;
@ -413,7 +418,9 @@ int32_t processCreateSTableRsp(void* param, SDataBuf* pMsg, int32_t code) {
doRequestCallback(pRequest, code);
} else {
(void)tsem_post(&pRequest->body.rspSem);
if (tsem_post(&pRequest->body.rspSem) != 0){
tscError("failed to post semaphore");
}
}
return code;
}
@ -451,14 +458,15 @@ int32_t processDropDbRsp(void* param, SDataBuf* pMsg, int32_t code) {
}
}
END:
taosMemoryFree(pMsg->pData);
taosMemoryFree(pMsg->pEpSet);
if (pRequest->body.queryFp != NULL) {
doRequestCallback(pRequest, code);
} else {
(void)tsem_post(&pRequest->body.rspSem);
if (tsem_post(&pRequest->body.rspSem) != 0){
tscError("failed to post semaphore");
}
}
return code;
}
@ -471,7 +479,12 @@ int32_t processAlterStbRsp(void* param, SDataBuf* pMsg, int32_t code) {
SMAlterStbRsp alterRsp = {0};
SDecoder coder = {0};
tDecoderInit(&coder, pMsg->pData, pMsg->len);
(void)tDecodeSMAlterStbRsp(&coder, &alterRsp); // pMsg->len = 0
if (pMsg->len > 0){
code = tDecodeSMAlterStbRsp(&coder, &alterRsp); // pMsg->len == 0
if (code != TSDB_CODE_SUCCESS) {
setErrno(pRequest, code);
}
}
tDecoderClear(&coder);
pRequest->body.resInfo.execRes.msgType = TDMT_MND_ALTER_STB;
@ -498,7 +511,9 @@ int32_t processAlterStbRsp(void* param, SDataBuf* pMsg, int32_t code) {
doRequestCallback(pRequest, code);
} else {
(void)tsem_post(&pRequest->body.rspSem);
if (tsem_post(&pRequest->body.rspSem) != 0){
tscError("failed to post semaphore");
}
}
return code;
}
@ -649,7 +664,9 @@ int32_t processShowVariablesRsp(void* param, SDataBuf* pMsg, int32_t code) {
if (pRequest->body.queryFp != NULL) {
doRequestCallback(pRequest, code);
} else {
(void)tsem_post(&pRequest->body.rspSem);
if (tsem_post(&pRequest->body.rspSem) != 0){
tscError("failed to post semaphore");
}
}
return code;
}
@ -801,7 +818,9 @@ int32_t processCompactDbRsp(void* param, SDataBuf* pMsg, int32_t code) {
if (pRequest->body.queryFp != NULL) {
pRequest->body.queryFp(((SSyncQueryParam *)pRequest->body.interParam)->userParam, pRequest, code);
} else {
(void)tsem_post(&pRequest->body.rspSem);
if (tsem_post(&pRequest->body.rspSem) != 0){
tscError("failed to post semaphore");
}
}
return code;
}

24
source/client/src/clientRawBlockWrite.c
View File

@ -960,6 +960,7 @@ static int32_t taosCreateStb(TAOS* taos, void* meta, int32_t metaLen) {
RAW_NULL_CHECK(pCmdMsg.pMsg);
if (tSerializeSMCreateStbReq(pCmdMsg.pMsg, pCmdMsg.msgLen, &pReq) <= 0) {
code = TSDB_CODE_INVALID_PARA;
taosMemoryFree(pCmdMsg.pMsg);
goto end;
}
@ -971,15 +972,15 @@ static int32_t taosCreateStb(TAOS* taos, void* meta, int32_t metaLen) {
(void)launchQueryImpl(pRequest, &pQuery, true, NULL); // ignore, because return value is pRequest
taosMemoryFree(pCmdMsg.pMsg);
if (pRequest->code == TSDB_CODE_SUCCESS) {
SCatalog* pCatalog = NULL;
// ignore the return value
(void)catalogGetHandle(pTscObj->pAppInfo->clusterId, &pCatalog);
(void)catalogRemoveTableMeta(pCatalog, &tableName);
RAW_RETURN_CHECK(catalogGetHandle(pTscObj->pAppInfo->clusterId, &pCatalog));
RAW_RETURN_CHECK(catalogRemoveTableMeta(pCatalog, &tableName));
}
code = pRequest->code;
taosMemoryFree(pCmdMsg.pMsg);
end:
uDebug(LOG_ID_TAG " create stable return, msg:%s", LOG_ID_VALUE, tstrerror(code));
@ -1022,8 +1023,7 @@ static int32_t taosDropStb(TAOS* taos, void* meta, int32_t metaLen) {
.requestObjRefId = pRequest->self,
.mgmtEps = getEpSet_s(&pRequest->pTscObj->pAppInfo->mgmtEp)};
SName pName = {0};
toName(pRequest->pTscObj->acctId, pRequest->pDb, req.name,
&pName); // ignore the return value, always return pName
toName(pRequest->pTscObj->acctId, pRequest->pDb, req.name, &pName);
STableMeta* pTableMeta = NULL;
code = catalogGetTableMeta(pCatalog, &conn, &pName, &pTableMeta);
if (code == TSDB_CODE_PAR_TABLE_NOT_EXIST) {
@ -1064,6 +1064,7 @@ static int32_t taosDropStb(TAOS* taos, void* meta, int32_t metaLen) {
RAW_NULL_CHECK(pCmdMsg.pMsg);
if (tSerializeSMDropStbReq(pCmdMsg.pMsg, pCmdMsg.msgLen, &pReq) <= 0) {
code = TSDB_CODE_INVALID_PARA;
taosMemoryFree(pCmdMsg.pMsg);
goto end;
}
@ -1074,15 +1075,14 @@ static int32_t taosDropStb(TAOS* taos, void* meta, int32_t metaLen) {
pQuery.stableQuery = true;
(void)launchQueryImpl(pRequest, &pQuery, true, NULL); // ignore, because return value is pRequest
taosMemoryFree(pCmdMsg.pMsg);
if (pRequest->code == TSDB_CODE_SUCCESS) {
// ignore the error code
(void)catalogGetHandle(pTscObj->pAppInfo->clusterId, &pCatalog);
(void)catalogRemoveTableMeta(pCatalog, &tableName);
RAW_RETURN_CHECK(catalogGetHandle(pTscObj->pAppInfo->clusterId, &pCatalog));
RAW_RETURN_CHECK(catalogRemoveTableMeta(pCatalog, &tableName));
}
code = pRequest->code;
taosMemoryFree(pCmdMsg.pMsg);
end:
uDebug(LOG_ID_TAG " drop stable return, msg:%s", LOG_ID_VALUE, tstrerror(code));
@ -1230,7 +1230,7 @@ static int32_t taosCreateTable(TAOS* taos, void* meta, int32_t metaLen) {
(void)launchQueryImpl(pRequest, pQuery, true, NULL);
if (pRequest->code == TSDB_CODE_SUCCESS) {
(void)removeMeta(pTscObj, pRequest->tableList, false);
RAW_RETURN_CHECK(removeMeta(pTscObj, pRequest->tableList, false));
}
code = pRequest->code;
@ -1359,7 +1359,7 @@ static int32_t taosDropTable(TAOS* taos, void* meta, int32_t metaLen) {
(void)launchQueryImpl(pRequest, pQuery, true, NULL);
if (pRequest->code == TSDB_CODE_SUCCESS) {
(void)removeMeta(pTscObj, pRequest->tableList, false);
RAW_RETURN_CHECK(removeMeta(pTscObj, pRequest->tableList, false));
}
code = pRequest->code;

7
source/client/src/clientSml.c
View File

@ -116,7 +116,7 @@ static int32_t smlCheckAuth(SSmlHandle *info, SRequestConnInfo *conn, const char
return TSDB_CODE_SML_INVALID_DATA;
}
} else {
toName(info->taos->acctId, info->pRequest->pDb, pTabName, &pAuth.tbName); //ignore
toName(info->taos->acctId, info->pRequest->pDb, pTabName, &pAuth.tbName);
}
pAuth.type = type;
@ -2217,9 +2217,12 @@ TAOS_RES *taos_schemaless_insert_inner(TAOS *taos, char *lines[], char *rawLine,
break;
}
taosMsleep(100);
(void)refreshMeta(request->pTscObj, request); //ignore return code,try again
uInfo("SML:%" PRIx64 " retry:%d/10,ver is old retry or object is creating code:%d, msg:%s", info->id, cnt, code,
tstrerror(code));
code = refreshMeta(request->pTscObj, request);
if (code != 0){
uInfo("SML:%" PRIx64 " refresh meta error code:%d, msg:%s", info->id, code, tstrerror(code));
}
smlDestroyInfo(info);
info = NULL;
taos_free_result(request);

5
source/client/src/clientStmt.c
View File

@ -771,7 +771,10 @@ void* stmtBindThreadFunc(void* param) {
continue;
}
(void)stmtAsyncOutput(pStmt, asyncParam);
int ret = stmtAsyncOutput(pStmt, asyncParam);
if (ret != 0){
qError("stmtAsyncOutput failed, reason:%s", tstrerror(ret));
}
}
qInfo("stmt bind thread stopped");

7
source/client/src/clientStmt2.c
View File

@ -1889,7 +1889,12 @@ int stmtGetParamTbName(TAOS_STMT2* stmt, int* nums) {
STMT_ERR_RET(stmtParseSql(pStmt));
}
*nums = STMT_TYPE_MULTI_INSERT == pStmt->sql.type ? 1 : 0;
if (TSDB_CODE_TSC_STMT_TBNAME_ERROR == pStmt->errCode) {
*nums = 1;
pStmt->errCode = TSDB_CODE_SUCCESS;
} else {
*nums = STMT_TYPE_MULTI_INSERT == pStmt->sql.type ? 1 : 0;
}
return TSDB_CODE_SUCCESS;
}

276
source/client/src/clientTmq.c
View File

@ -807,11 +807,18 @@ static void generateTimedTask(int64_t refId, int32_t type) {
if (code == TSDB_CODE_SUCCESS) {
*pTaskType = type;
if (taosWriteQitem(tmq->delayedTask, pTaskType) == 0) {
(void)tsem2_post(&tmq->rspSem);
if (tsem2_post(&tmq->rspSem) != 0){
tscError("consumer:0x%" PRIx64 " failed to post sem, type:%d", tmq->consumerId, type);
}
}else{
taosFreeQitem(pTaskType);
}
}
(void)taosReleaseRef(tmqMgmt.rsetId, refId);
code = taosReleaseRef(tmqMgmt.rsetId, refId);
if (code != 0){
tscError("failed to release ref:%"PRId64 ", type:%d, code:%d", refId, type, code);
}
}
void tmqAssignAskEpTask(void* param, void* tmrId) {
@ -824,8 +831,13 @@ void tmqReplayTask(void* param, void* tmrId) {
tmq_t* tmq = taosAcquireRef(tmqMgmt.rsetId, refId);
if (tmq == NULL) return;
(void)tsem2_post(&tmq->rspSem);
(void)taosReleaseRef(tmqMgmt.rsetId, refId);
if (tsem2_post(&tmq->rspSem) != 0){
tscError("consumer:0x%" PRIx64 " failed to post sem, replay", tmq->consumerId);
}
int32_t code = taosReleaseRef(tmqMgmt.rsetId, refId);
if (code != 0){
tscError("failed to release ref:%"PRId64 ", code:%d", refId, code);
}
}
void tmqAssignDelayedCommitTask(void* param, void* tmrId) {
@ -835,17 +847,17 @@ void tmqAssignDelayedCommitTask(void* param, void* tmrId) {
int32_t tmqHbCb(void* param, SDataBuf* pMsg, int32_t code) {
if (code != 0) {
goto _return;
goto END;
}
if (pMsg == NULL || param == NULL) {
code = TSDB_CODE_INVALID_PARA;
goto _return;
goto END;
}
SMqHbRsp rsp = {0};
code = tDeserializeSMqHbRsp(pMsg->pData, pMsg->len, &rsp);
if (code != 0) {
goto _return;
goto END;
}
int64_t refId = (int64_t)param;
@ -866,13 +878,15 @@ int32_t tmqHbCb(void* param, SDataBuf* pMsg, int32_t code) {
}
}
taosWUnLockLatch(&tmq->lock);
(void)taosReleaseRef(tmqMgmt.rsetId, refId);
code = taosReleaseRef(tmqMgmt.rsetId, refId);
if (code != 0){
tscError("failed to release ref:%"PRId64 ", code:%d", refId, code);
}
}
tDestroySMqHbRsp(&rsp);
_return:
END:
taosMemoryFree(pMsg->pData);
taosMemoryFree(pMsg->pEpSet);
return code;
@ -974,9 +988,15 @@ void tmqSendHbReq(void* param, void* tmrId) {
OVER:
tDestroySMqHbReq(&req);
if (tmrId != NULL) {
(void)taosTmrReset(tmqSendHbReq, tmq->heartBeatIntervalMs, param, tmqMgmt.timer, &tmq->hbLiveTimer);
bool ret = taosTmrReset(tmqSendHbReq, tmq->heartBeatIntervalMs, param, tmqMgmt.timer, &tmq->hbLiveTimer);
if (!ret){
tscError("failed to reset timer fo tmq hb");
}
}
int32_t ret = taosReleaseRef(tmqMgmt.rsetId, refId);
if (ret != 0){
tscError("failed to release ref:%"PRId64 ", code:%d", refId, ret);
}
(void)taosReleaseRef(tmqMgmt.rsetId, refId);
}
static void defaultCommitCbFn(tmq_t* pTmq, int32_t code, void* param) {
@ -995,9 +1015,7 @@ void tmqHandleAllDelayedTask(tmq_t* pTmq) {
return;
}
(void)taosReadAllQitems(pTmq->delayedTask, qall);
int32_t numOfItems = taosQallItemSize(qall);
int32_t numOfItems = taosReadAllQitems(pTmq->delayedTask, qall);
if (numOfItems == 0) {
taosFreeQall(qall);
return;
@ -1005,9 +1023,7 @@ void tmqHandleAllDelayedTask(tmq_t* pTmq) {
tscDebug("consumer:0x%" PRIx64 " handle delayed %d tasks before poll data", pTmq->consumerId, numOfItems);
int8_t* pTaskType = NULL;
(void)taosGetQitem(qall, (void**)&pTaskType);
while (pTaskType != NULL) {
while (taosGetQitem(qall, (void**)&pTaskType) != 0) {
if (*pTaskType == TMQ_DELAYED_TASK__ASK_EP) {
code = askEp(pTmq, NULL, false, false);
if (code != 0) {
@ -1015,21 +1031,26 @@ void tmqHandleAllDelayedTask(tmq_t* pTmq) {
continue;
}
tscDebug("consumer:0x%" PRIx64 " retrieve ep from mnode in 1s", pTmq->consumerId);
(void)taosTmrReset(tmqAssignAskEpTask, DEFAULT_ASKEP_INTERVAL, (void*)(pTmq->refId), tmqMgmt.timer,
bool ret = taosTmrReset(tmqAssignAskEpTask, DEFAULT_ASKEP_INTERVAL, (void*)(pTmq->refId), tmqMgmt.timer,
&pTmq->epTimer);
if (!ret){
tscError("failed to reset timer fo tmq ask ep");
}
} else if (*pTaskType == TMQ_DELAYED_TASK__COMMIT) {
tmq_commit_cb* pCallbackFn = pTmq->commitCb ? pTmq->commitCb : defaultCommitCbFn;
asyncCommitAllOffsets(pTmq, pCallbackFn, pTmq->commitCbUserParam);
tscDebug("consumer:0x%" PRIx64 " next commit to vnode(s) in %.2fs", pTmq->consumerId,
pTmq->autoCommitInterval / 1000.0);
(void)taosTmrReset(tmqAssignDelayedCommitTask, pTmq->autoCommitInterval, (void*)(pTmq->refId), tmqMgmt.timer,
bool ret = taosTmrReset(tmqAssignDelayedCommitTask, pTmq->autoCommitInterval, (void*)(pTmq->refId), tmqMgmt.timer,
&pTmq->commitTimer);
if (!ret){
tscError("failed to reset timer fo commit");
}
} else {
tscError("consumer:0x%" PRIx64 " invalid task type:%d", pTmq->consumerId, *pTaskType);
}
taosFreeQitem(pTaskType);
(void)taosGetQitem(qall, (void**)&pTaskType);
}
taosFreeQall(qall);
@ -1061,26 +1082,18 @@ static void tmqFreeRspWrapper(SMqRspWrapper* rspWrapper) {
void tmqClearUnhandleMsg(tmq_t* tmq) {
SMqRspWrapper* rspWrapper = NULL;
while (1) {
(void)taosGetQitem(tmq->qall, (void**)&rspWrapper);
if (rspWrapper) {
while (taosGetQitem(tmq->qall, (void**)&rspWrapper) != 0) {
tmqFreeRspWrapper(rspWrapper);
taosFreeQitem(rspWrapper);
} else {
break;
}
}
rspWrapper = NULL;
(void)taosReadAllQitems(tmq->mqueue, tmq->qall);
while (1) {
(void)taosGetQitem(tmq->qall, (void**)&rspWrapper);
if (rspWrapper) {
tmqFreeRspWrapper(rspWrapper);
taosFreeQitem(rspWrapper);
} else {
break;
}
if (taosReadAllQitems(tmq->mqueue, tmq->qall) == 0){
return;
}
while (taosGetQitem(tmq->qall, (void**)&rspWrapper) != 0) {
tmqFreeRspWrapper(rspWrapper);
taosFreeQitem(rspWrapper);
}
}
@ -1095,7 +1108,9 @@ int32_t tmqSubscribeCb(void* param, SDataBuf* pMsg, int32_t code) {
if (pMsg) {
taosMemoryFree(pMsg->pEpSet);
}
(void)tsem2_post(&pParam->rspSem);
if (tsem2_post(&pParam->rspSem) != 0){
tscError("failed to post sem, subscribe cb");
}
return 0;
}
@ -1154,19 +1169,27 @@ void tmqFreeImpl(void* handle) {
}
taosFreeQall(tmq->qall);
(void)tsem2_destroy(&tmq->rspSem);
if(tsem2_destroy(&tmq->rspSem) != 0) {
tscError("failed to destroy sem in free tmq");
}
taosArrayDestroyEx(tmq->clientTopics, freeClientTopic);
taos_close_internal(tmq->pTscObj);
if (tmq->commitTimer) {
(void)taosTmrStopA(&tmq->commitTimer);
if (!taosTmrStopA(&tmq->commitTimer)) {
tscError("failed to stop commit timer");
}
}
if (tmq->epTimer) {
(void)taosTmrStopA(&tmq->epTimer);
if (!taosTmrStopA(&tmq->epTimer)) {
tscError("failed to stop ep timer");
}
}
if (tmq->hbLiveTimer) {
(void)taosTmrStopA(&tmq->hbLiveTimer);
if (!taosTmrStopA(&tmq->hbLiveTimer)) {
tscError("failed to stop hb timer");
}
}
taosMemoryFree(tmq);
@ -1312,7 +1335,6 @@ tmq_t* tmq_consumer_new(tmq_conf_t* conf, char* errstr, int32_t errstrLen) {
if (code) {
terrno = code;
tscError("consumer:0x%" PRIx64 " setup failed since %s, groupId:%s", pTmq->consumerId, terrstr(), pTmq->groupId);
(void)tsem2_destroy(&pTmq->rspSem);
SET_ERROR_MSG_TMQ("init tscObj failed")
goto _failed;
}
@ -1419,7 +1441,7 @@ int32_t tmq_subscribe(tmq_t* tmq, const tmq_list_t* topic_list) {
}
void* abuf = buf;
(void)tSerializeSCMSubscribeReq(&abuf, &req);
tlen = tSerializeSCMSubscribeReq(&abuf, &req);
sendInfo = taosMemoryCalloc(1, sizeof(SMsgSendInfo));
if (sendInfo == NULL) {
@ -1450,8 +1472,12 @@ int32_t tmq_subscribe(tmq_t* tmq, const tmq_list_t* topic_list) {
goto FAIL;
}
(void)tsem2_wait(&param.rspSem);
(void)tsem2_destroy(&param.rspSem);
if (tsem2_wait(&param.rspSem) != 0){
tscError("consumer:0x%" PRIx64 ", failed to wait semaphore in subscribe", tmq->consumerId);
}
if(tsem2_destroy(&param.rspSem) != 0) {
tscError("consumer:0x%" PRIx64 ", failed to destroy semaphore in subscribe", tmq->consumerId);
}
if (param.rspErr != 0) {
code = param.rspErr;
@ -1656,6 +1682,8 @@ END:
(void)strcpy(pRspWrapper->topicName, pParam->topicName);
code = taosWriteQitem(tmq->mqueue, pRspWrapper);
if (code != 0) {
tmqFreeRspWrapper((SMqRspWrapper*)pRspWrapper);
taosFreeQitem(pRspWrapper);
tscError("consumer:0x%" PRIx64 " put poll res into mqueue failed, code:%d", tmq->consumerId, code);
}
}
@ -1663,10 +1691,17 @@ END:
tscDebug("consumer:0x%" PRIx64 " put poll res into mqueue, type:%d, vgId:%d, total in queue:%d,QID:0x%" PRIx64,
tmq ? tmq->consumerId : 0, rspType, vgId, total, requestId);
if (tmq) (void)tsem2_post(&tmq->rspSem);
if (tmq) {
if (tsem2_post(&tmq->rspSem) != 0){
tscError("failed to post rsp sem, consumer:0x%" PRIx64, tmq->consumerId);
}
}
if (pMsg) taosMemoryFreeClear(pMsg->pData);
if (pMsg) taosMemoryFreeClear(pMsg->pEpSet);
(void)taosReleaseRef(tmqMgmt.rsetId, refId);
ret = taosReleaseRef(tmqMgmt.rsetId, refId);
if (ret != 0){
tscError("failed to release ref:%"PRId64 ", code:%d", refId, ret);
}
return code;
}
@ -1763,7 +1798,7 @@ static bool doUpdateLocalEp(tmq_t* tmq, int32_t epoch, const SMqAskEpRsp* pRsp)
SHashObj* pVgOffsetHashMap = taosHashInit(64, MurmurHash3_32, false, HASH_NO_LOCK);
if (pVgOffsetHashMap == NULL) {
(void)taosArrayDestroy(newTopics);
taosArrayDestroy(newTopics);
return false;
}
@ -1848,10 +1883,10 @@ void tmqBuildConsumeReqImpl(SMqPollReq* pReq, tmq_t* tmq, int64_t timeout, SMqCl
pReq->enableBatchMeta = tmq->enableBatchMeta;
}
int32_t tmqBuildMetaRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqMetaRspObj** ppRspObj) {
void tmqBuildMetaRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqMetaRspObj** ppRspObj) {
SMqMetaRspObj* pRspObj = taosMemoryCalloc(1, sizeof(SMqMetaRspObj));
if (pRspObj == NULL) {
return terrno;
return;
}
pRspObj->resType = RES_TYPE__TMQ_META;
tstrncpy(pRspObj->topic, pWrapper->topicHandle->topicName, TSDB_TOPIC_FNAME_LEN);
@ -1860,13 +1895,12 @@ int32_t tmqBuildMetaRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqMetaRspObj**
(void)memcpy(&pRspObj->metaRsp, &pWrapper->metaRsp, sizeof(SMqMetaRsp));
*ppRspObj = pRspObj;
return 0;
}
int32_t tmqBuildBatchMetaRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqBatchMetaRspObj** ppRspObj) {
void tmqBuildBatchMetaRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqBatchMetaRspObj** ppRspObj) {
SMqBatchMetaRspObj* pRspObj = taosMemoryCalloc(1, sizeof(SMqBatchMetaRspObj));
if (pRspObj == NULL) {
return terrno;
return;
}
pRspObj->common.resType = RES_TYPE__TMQ_BATCH_META;
tstrncpy(pRspObj->common.topic, pWrapper->topicHandle->topicName, TSDB_TOPIC_FNAME_LEN);
@ -1876,7 +1910,6 @@ int32_t tmqBuildBatchMetaRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqBatchMet
(void)memcpy(&pRspObj->rsp, &pWrapper->batchMetaRsp, sizeof(SMqBatchMetaRsp));
tscDebug("build batchmeta Rsp from wrapper");
*ppRspObj = pRspObj;
return 0;
}
void changeByteEndian(char* pData) {
@ -1973,31 +2006,29 @@ static void tmqBuildRspFromWrapperInner(SMqPollRspWrapper* pWrapper, SMqClientVg
}
}
int32_t tmqBuildRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqClientVg* pVg, int64_t* numOfRows,
void tmqBuildRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqClientVg* pVg, int64_t* numOfRows,
SMqRspObj** ppRspObj) {
SMqRspObj* pRspObj = taosMemoryCalloc(1, sizeof(SMqRspObj));
if (pRspObj == NULL) {
return terrno;
return;
}
pRspObj->common.resType = RES_TYPE__TMQ;
(void)memcpy(&pRspObj->rsp, &pWrapper->dataRsp, sizeof(SMqDataRsp));
tmqBuildRspFromWrapperInner(pWrapper, pVg, numOfRows, &pRspObj->common, &pRspObj->rsp.common);
*ppRspObj = pRspObj;
return 0;
}
int32_t tmqBuildTaosxRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqClientVg* pVg, int64_t* numOfRows,
void tmqBuildTaosxRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqClientVg* pVg, int64_t* numOfRows,
SMqTaosxRspObj** ppRspObj) {
SMqTaosxRspObj* pRspObj = taosMemoryCalloc(1, sizeof(SMqTaosxRspObj));
if (pRspObj == NULL) {
return terrno;
return;
}
pRspObj->common.resType = RES_TYPE__TMQ_METADATA;
(void)memcpy(&pRspObj->rsp, &pWrapper->taosxRsp, sizeof(STaosxRsp));
tmqBuildRspFromWrapperInner(pWrapper, pVg, numOfRows, &pRspObj->common, &pRspObj->rsp.common);
*ppRspObj = pRspObj;
return 0;
}
static int32_t doTmqPollImpl(tmq_t* pTmq, SMqClientTopic* pTopic, SMqClientVg* pVg, int64_t timeout) {
@ -2156,12 +2187,11 @@ static void* tmqHandleAllRsp(tmq_t* tmq, int64_t timeout) {
while (1) {
SMqRspWrapper* pRspWrapper = NULL;
(void)taosGetQitem(tmq->qall, (void**)&pRspWrapper);
if (pRspWrapper == NULL) {
(void)taosReadAllQitems(tmq->mqueue, tmq->qall);
(void)taosGetQitem(tmq->qall, (void**)&pRspWrapper);
if (pRspWrapper == NULL) {
if (taosGetQitem(tmq->qall, (void**)&pRspWrapper) == 0) {
if (taosReadAllQitems(tmq->mqueue, tmq->qall) == 0){
return NULL;
}
if (taosGetQitem(tmq->qall, (void**)&pRspWrapper) == 0) {
return NULL;
}
}
@ -2239,7 +2269,7 @@ static void* tmqHandleAllRsp(tmq_t* tmq, int64_t timeout) {
} else { // build rsp
int64_t numOfRows = 0;
SMqRspObj* pRsp = NULL;
(void)tmqBuildRspFromWrapper(pollRspWrapper, pVg, &numOfRows, &pRsp);
tmqBuildRspFromWrapper(pollRspWrapper, pVg, &numOfRows, &pRsp);
tmq->totalRows += numOfRows;
pVg->emptyBlockReceiveTs = 0;
if (pRsp && tmq->replayEnable) {
@ -2293,7 +2323,7 @@ static void* tmqHandleAllRsp(tmq_t* tmq, int64_t timeout) {
pollRspWrapper->metaRsp.head.walsver, pollRspWrapper->metaRsp.head.walever, tmq->consumerId, true);
// build rsp
SMqMetaRspObj* pRsp = NULL;
(void)tmqBuildMetaRspFromWrapper(pollRspWrapper, &pRsp);
tmqBuildMetaRspFromWrapper(pollRspWrapper, &pRsp);
taosMemoryFreeClear(pollRspWrapper->pEpset);
taosFreeQitem(pRspWrapper);
taosWUnLockLatch(&tmq->lock);
@ -2331,7 +2361,7 @@ static void* tmqHandleAllRsp(tmq_t* tmq, int64_t timeout) {
pollRspWrapper->batchMetaRsp.head.walsver, pollRspWrapper->batchMetaRsp.head.walever,
tmq->consumerId, true);
SMqBatchMetaRspObj* pRsp = NULL;
(void)tmqBuildBatchMetaRspFromWrapper(pollRspWrapper, &pRsp);
tmqBuildBatchMetaRspFromWrapper(pollRspWrapper, &pRsp);
taosMemoryFreeClear(pollRspWrapper->pEpset);
taosFreeQitem(pRspWrapper);
taosWUnLockLatch(&tmq->lock);
@ -2381,9 +2411,7 @@ static void* tmqHandleAllRsp(tmq_t* tmq, int64_t timeout) {
// build rsp
int64_t numOfRows = 0;
SMqTaosxRspObj* pRsp = NULL;
if (tmqBuildTaosxRspFromWrapper(pollRspWrapper, pVg, &numOfRows, &pRsp) != 0) {
tscError("consumer:0x%" PRIx64 " build taosx rsp failed, vgId:%d", tmq->consumerId, pVg->vgId);
}
tmqBuildTaosxRspFromWrapper(pollRspWrapper, pVg, &numOfRows, &pRsp);
tmq->totalRows += numOfRows;
char buf[TSDB_OFFSET_LEN] = {0};
@ -2551,7 +2579,10 @@ int32_t tmq_consumer_close(tmq_t* tmq) {
}
if (code == 0) {
(void)taosRemoveRef(tmqMgmt.rsetId, tmq->refId);
code = taosRemoveRef(tmqMgmt.rsetId, tmq->refId);
if (code != 0){
tscError("tmq close failed to remove ref:%" PRId64 ", code:%d", tmq->refId, code);
}
}
return code;
}
@ -2709,7 +2740,9 @@ void tmq_commit_async(tmq_t* tmq, const TAOS_RES* pRes, tmq_commit_cb* cb, void*
static void commitCallBackFn(tmq_t* UNUSED_PARAM(tmq), int32_t code, void* param) {
SSyncCommitInfo* pInfo = (SSyncCommitInfo*)param;
pInfo->code = code;
(void)tsem2_post(&pInfo->sem);
if (tsem2_post(&pInfo->sem) != 0){
tscError("failed to post rsp sem in commit cb");
}
}
int32_t tmq_commit_sync(tmq_t* tmq, const TAOS_RES* pRes) {
@ -2738,10 +2771,14 @@ int32_t tmq_commit_sync(tmq_t* tmq, const TAOS_RES* pRes) {
asyncCommitFromResult(tmq, pRes, commitCallBackFn, pInfo);
}
(void)tsem2_wait(&pInfo->sem);
if (tsem2_wait(&pInfo->sem) != 0){
tscError("failed to wait sem for sync commit");
}
code = pInfo->code;
(void)tsem2_destroy(&pInfo->sem);
if(tsem2_destroy(&pInfo->sem) != 0) {
tscError("failed to destroy sem for sync commit");
}
taosMemoryFree(pInfo);
tscInfo("consumer:0x%" PRIx64 " sync res commit done, code:%s", tmq->consumerId, tstrerror(code));
@ -2806,12 +2843,16 @@ int32_t tmq_commit_offset_sync(tmq_t* tmq, const char* pTopicName, int32_t vgId,
code = asyncCommitOffset(tmq, tname, vgId, &offsetVal, commitCallBackFn, pInfo);
if (code == 0) {
(void)tsem2_wait(&pInfo->sem);
if (tsem2_wait(&pInfo->sem) != 0){
tscError("failed to wait sem for sync commit offset");
}
code = pInfo->code;
}
if (code == TSDB_CODE_TMQ_SAME_COMMITTED_VALUE) code = TSDB_CODE_SUCCESS;
(void)tsem2_destroy(&pInfo->sem);
if(tsem2_destroy(&pInfo->sem) != 0) {
tscError("failed to destroy sem for sync commit offset");
}
taosMemoryFree(pInfo);
tscInfo("consumer:0x%" PRIx64 " sync send commit to vgId:%d, offset:%" PRId64 " code:%s", tmq->consumerId, vgId,
@ -2864,11 +2905,11 @@ end:
}
int32_t askEpCb(void* param, SDataBuf* pMsg, int32_t code) {
if (param == NULL) {
SMqAskEpCbParam* pParam = (SMqAskEpCbParam*)param;
if (pParam == NULL) {
goto FAIL;
}
SMqAskEpCbParam* pParam = (SMqAskEpCbParam*)param;
tmq_t* tmq = taosAcquireRef(tmqMgmt.rsetId, pParam->refId);
if (tmq == NULL) {
code = TSDB_CODE_TMQ_CONSUMER_CLOSED;
@ -2906,19 +2947,30 @@ int32_t askEpCb(void* param, SDataBuf* pMsg, int32_t code) {
tmqFreeRspWrapper((SMqRspWrapper*)pWrapper);
taosFreeQitem(pWrapper);
} else {
(void)taosWriteQitem(tmq->mqueue, pWrapper);
code = taosWriteQitem(tmq->mqueue, pWrapper);
if (code != 0) {
tmqFreeRspWrapper((SMqRspWrapper*)pWrapper);
taosFreeQitem(pWrapper);
tscError("consumer:0x%" PRIx64 " put ep res into mqueue failed, code:%d", tmq->consumerId, code);
}
}
}
END:
(void)taosReleaseRef(tmqMgmt.rsetId, pParam->refId);
{
int32_t ret = taosReleaseRef(tmqMgmt.rsetId, pParam->refId);
if (ret != 0){
tscError("failed to release ref:%"PRId64 ", code:%d", pParam->refId, ret);
}
}
FAIL:
if (pParam->sync) {
if (pParam && pParam->sync) {
SAskEpInfo* pInfo = pParam->pParam;
if (pInfo) {
pInfo->code = code;
(void)tsem2_post(&pInfo->sem);
if (tsem2_post(&pInfo->sem) != 0){
tscError("failed to post rsp sem askep cb");
}
}
}
@ -2940,11 +2992,15 @@ int32_t syncAskEp(tmq_t* pTmq) {
int32_t code = askEp(pTmq, pInfo, true, false);
if (code == 0) {
(void)tsem2_wait(&pInfo->sem);
if (tsem2_wait(&pInfo->sem) != 0){
tscError("consumer:0x%" PRIx64 ", failed to wait for sem", pTmq->consumerId);
}
code = pInfo->code;
}
(void)tsem2_destroy(&pInfo->sem);
if(tsem2_destroy(&pInfo->sem) != 0) {
tscError("failed to destroy sem sync ask ep");
}
taosMemoryFree(pInfo);
return code;
}
@ -3115,7 +3171,9 @@ static int32_t tmqGetWalInfoCb(void* param, SDataBuf* pMsg, int32_t code) {
END:
pCommon->code = code;
if (total == pParam->totalReq) {
(void)tsem2_post(&pCommon->rsp);
if (tsem2_post(&pCommon->rsp) != 0) {
tscError("failed to post semaphore in get wal cb");
}
}
if (pMsg) {
@ -3130,8 +3188,10 @@ static void destroyCommonInfo(SMqVgCommon* pCommon) {
if (pCommon == NULL) {
return;
}
(void)taosArrayDestroy(pCommon->pList);
(void)tsem2_destroy(&pCommon->rsp);
taosArrayDestroy(pCommon->pList);
if(tsem2_destroy(&pCommon->rsp) != 0) {
tscError("failed to destroy semaphore for topic:%s", pCommon->pTopicName);
}
(void)taosThreadMutexDestroy(&pCommon->mutex);
taosMemoryFree(pCommon->pTopicName);
taosMemoryFree(pCommon);
@ -3167,7 +3227,9 @@ end:
taosMemoryFree(pMsg->pEpSet);
}
pParam->code = code;
(void)tsem2_post(&pParam->sem);
if (tsem2_post(&pParam->sem) != 0){
tscError("failed to post semaphore in tmCommittedCb");
}
return code;
}
@ -3231,12 +3293,16 @@ int64_t getCommittedFromServer(tmq_t* tmq, char* tname, int32_t vgId, SEpSet* ep
code = asyncSendMsgToServer(tmq->pTscObj->pAppInfo->pTransporter, epSet, NULL, sendInfo);
if (code != 0) {
(void)tsem2_destroy(&pParam->sem);
if(tsem2_destroy(&pParam->sem) != 0) {
tscError("failed to destroy semaphore in get committed from server1");
}
taosMemoryFree(pParam);
return code;
}
(void)tsem2_wait(&pParam->sem);
if (tsem2_wait(&pParam->sem) != 0){
tscError("failed to wait semaphore in get committed from server");
}
code = pParam->code;
if (code == TSDB_CODE_SUCCESS) {
if (pParam->vgOffset.offset.val.type == TMQ_OFFSET__LOG) {
@ -3246,7 +3312,9 @@ int64_t getCommittedFromServer(tmq_t* tmq, char* tname, int32_t vgId, SEpSet* ep
code = TSDB_CODE_TMQ_SNAPSHOT_ERROR;
}
}
(void)tsem2_destroy(&pParam->sem);
if(tsem2_destroy(&pParam->sem) != 0) {
tscError("failed to destroy semaphore in get committed from server2");
}
taosMemoryFree(pParam);
return code;
@ -3517,7 +3585,9 @@ int32_t tmq_get_topic_assignment(tmq_t* tmq, const char* pTopicName, tmq_topic_a
}
}
(void)tsem2_wait(&pCommon->rsp);
if (tsem2_wait(&pCommon->rsp) != 0){
tscError("consumer:0x%" PRIx64 " failed to wait sem in get assignment", tmq->consumerId);
}
code = pCommon->code;
if (code != TSDB_CODE_SUCCESS) {
@ -3580,7 +3650,9 @@ static int32_t tmqSeekCb(void* param, SDataBuf* pMsg, int32_t code) {
}
SMqSeekParam* pParam = param;
pParam->code = code;
(void)tsem2_post(&pParam->sem);
if (tsem2_post(&pParam->sem) != 0){
tscError("failed to post sem in tmqSeekCb");
}
return 0;
}
@ -3677,14 +3749,20 @@ int32_t tmq_offset_seek(tmq_t* tmq, const char* pTopicName, int32_t vgId, int64_
code = asyncSendMsgToServer(tmq->pTscObj->pAppInfo->pTransporter, &epSet, NULL, sendInfo);
if (code != 0) {
(void)tsem2_destroy(&pParam->sem);
if(tsem2_destroy(&pParam->sem) != 0) {
tscError("consumer:0x%" PRIx64 "destroy rsp sem failed in seek offset", tmq->consumerId);
}
taosMemoryFree(pParam);
return code;
}
(void)tsem2_wait(&pParam->sem);
if (tsem2_wait(&pParam->sem) != 0){
tscError("consumer:0x%" PRIx64 "wait rsp sem failed in seek offset", tmq->consumerId);
}
code = pParam->code;
(void)tsem2_destroy(&pParam->sem);
if(tsem2_destroy(&pParam->sem) != 0) {
tscError("consumer:0x%" PRIx64 "destroy rsp sem failed in seek offset", tmq->consumerId);
}
taosMemoryFree(pParam);
tscInfo("consumer:0x%" PRIx64 "send seek to vgId:%d, return code:%s", tmq->consumerId, vgId, tstrerror(code));

2
source/common/src/cos_cp.c
View File

@ -170,7 +170,7 @@ int32_t cos_cp_load(char const* filepath, SCheckpoint* checkpoint) {
cp_body = taosMemoryMalloc(size + 1);
if (!cp_body) {
TAOS_CHECK_GOTO(TSDB_CODE_OUT_OF_MEMORY, &lino, _exit);
TAOS_CHECK_GOTO(terrno, &lino, _exit);
}
int64_t n = taosReadFile(fd, cp_body, size);

81
source/common/src/tdatablock.c
View File

@ -124,7 +124,7 @@ int32_t colDataSetVal(SColumnInfoData* pColumnInfoData, uint32_t rowIndex, const
char* buf = taosMemoryRealloc(pColumnInfoData->pData, newSize);
if (buf == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
pColumnInfoData->pData = buf;
@ -170,7 +170,7 @@ static int32_t colDataReserve(SColumnInfoData* pColumnInfoData, size_t newSize)
if (pColumnInfoData->varmeta.allocLen < newSize) {
char* buf = taosMemoryRealloc(pColumnInfoData->pData, newSize);
if (buf == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
pColumnInfoData->pData = buf;
@ -388,7 +388,7 @@ int32_t colDataMergeCol(SColumnInfoData* pColumnInfoData, int32_t numOfRow1, int
if (finalNumOfRows > (*capacity)) {
char* p = taosMemoryRealloc(pColumnInfoData->varmeta.offset, sizeof(int32_t) * (numOfRow1 + numOfRow2));
if (p == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
*capacity = finalNumOfRows;
@ -409,7 +409,7 @@ int32_t colDataMergeCol(SColumnInfoData* pColumnInfoData, int32_t numOfRow1, int
if (pColumnInfoData->varmeta.allocLen < len + oldLen) {
char* tmp = taosMemoryRealloc(pColumnInfoData->pData, len + oldLen);
if (tmp == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
pColumnInfoData->pData = tmp;
@ -425,14 +425,14 @@ int32_t colDataMergeCol(SColumnInfoData* pColumnInfoData, int32_t numOfRow1, int
// all data may be null, when the pColumnInfoData->info.type == 0, bytes == 0;
char* tmp = taosMemoryRealloc(pColumnInfoData->pData, finalNumOfRows * pColumnInfoData->info.bytes);
if (tmp == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
pColumnInfoData->pData = tmp;
if (BitmapLen(numOfRow1) < BitmapLen(finalNumOfRows)) {
char* btmp = taosMemoryRealloc(pColumnInfoData->nullbitmap, BitmapLen(finalNumOfRows));
if (btmp == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
uint32_t extend = BitmapLen(finalNumOfRows) - BitmapLen(numOfRow1);
memset(btmp + BitmapLen(numOfRow1), 0, extend);
@ -469,7 +469,7 @@ int32_t colDataAssign(SColumnInfoData* pColumnInfoData, const SColumnInfoData* p
if (pColumnInfoData->varmeta.allocLen < newLen) {
char* tmp = taosMemoryRealloc(pColumnInfoData->pData, newLen);
if (tmp == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
pColumnInfoData->pData = tmp;
@ -545,7 +545,7 @@ int32_t colDataAssignNRows(SColumnInfoData* pDst, int32_t dstIdx, const SColumnI
if (pDst->varmeta.allocLen < pDst->varmeta.length + allLen) {
char* tmp = taosMemoryRealloc(pDst->pData, pDst->varmeta.length + allLen);
if (tmp == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
pDst->pData = tmp;
@ -999,7 +999,7 @@ int32_t blockDataFromBuf(SSDataBlock* pBlock, const char* buf) {
size_t metaSize = pBlock->info.rows * sizeof(int32_t);
char* tmp = taosMemoryRealloc(pCol->varmeta.offset, metaSize); // preview calloc is too small
if (tmp == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
pCol->varmeta.offset = (int32_t*)tmp;
@ -1017,7 +1017,7 @@ int32_t blockDataFromBuf(SSDataBlock* pBlock, const char* buf) {
if (pCol->varmeta.allocLen < colLength) {
char* tmp = taosMemoryRealloc(pCol->pData, colLength);
if (tmp == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
pCol->pData = tmp;
@ -1098,7 +1098,7 @@ int32_t blockDataFromBuf1(SSDataBlock* pBlock, const char* buf, size_t capacity)
if (pCol->varmeta.allocLen < colLength) {
char* tmp = taosMemoryRealloc(pCol->pData, colLength);
if (tmp == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
pCol->pData = tmp;
@ -1421,7 +1421,6 @@ int32_t blockDataSort(SSDataBlock* pDataBlock, SArray* pOrderInfo) {
int32_t* index = createTupleIndex(rows);
if (index == NULL) {
terrno = TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
@ -1544,7 +1543,7 @@ int32_t doEnsureCapacity(SColumnInfoData* pColumn, const SDataBlockInfo* pBlockI
if (IS_VAR_DATA_TYPE(pColumn->info.type)) {
char* tmp = taosMemoryRealloc(pColumn->varmeta.offset, sizeof(int32_t) * numOfRows);
if (tmp == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
pColumn->varmeta.offset = (int32_t*)tmp;
@ -1553,7 +1552,7 @@ int32_t doEnsureCapacity(SColumnInfoData* pColumn, const SDataBlockInfo* pBlockI
// prepare for the null bitmap
char* tmp = taosMemoryRealloc(pColumn->nullbitmap, BitmapLen(numOfRows));
if (tmp == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
int32_t oldLen = BitmapLen(existedRows);
@ -1567,7 +1566,7 @@ int32_t doEnsureCapacity(SColumnInfoData* pColumn, const SDataBlockInfo* pBlockI
// to MALLOC_ALIGN_BYTES
tmp = taosMemoryMallocAlign(MALLOC_ALIGN_BYTES, numOfRows * pColumn->info.bytes);
if (tmp == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
// memset(tmp, 0, numOfRows * pColumn->info.bytes);
@ -2517,24 +2516,25 @@ int32_t dumpBlockData(SSDataBlock* pDataBlock, const char* flag, char** pDataBuf
pDataBlock->info.id.groupId, pDataBlock->info.id.uid, pDataBlock->info.rows, pDataBlock->info.version,
pDataBlock->info.calWin.skey, pDataBlock->info.calWin.ekey, pDataBlock->info.parTbName);
if (len >= size - 1) {
return code;
goto _exit;
}
for (int32_t j = 0; j < rows; j++) {
len += snprintf(dumpBuf + len, size - len, "%s|", flag);
if (len >= size - 1) {
return code;
goto _exit;
}
for (int32_t k = 0; k < colNum; k++) {
SColumnInfoData* pColInfoData = taosArrayGet(pDataBlock->pDataBlock, k);
if (pColInfoData == NULL) {
return terrno;
code = terrno;
goto _exit;
}
if (colDataIsNull(pColInfoData, rows, j, NULL) || !pColInfoData->pData) {
len += snprintf(dumpBuf + len, size - len, " %15s |", "NULL");
if (len >= size - 1) return 0;
if (len >= size - 1) goto _exit;
continue;
}
@ -2542,53 +2542,53 @@ int32_t dumpBlockData(SSDataBlock* pDataBlock, const char* flag, char** pDataBuf
switch (pColInfoData->info.type) {
case TSDB_DATA_TYPE_TIMESTAMP:
memset(pBuf, 0, sizeof(pBuf));
(void) formatTimestamp(pBuf, *(uint64_t*)var, pColInfoData->info.precision);
(void)formatTimestamp(pBuf, *(uint64_t*)var, pColInfoData->info.precision);
len += snprintf(dumpBuf + len, size - len, " %25s |", pBuf);
if (len >= size - 1) return 0;
if (len >= size - 1) goto _exit;
break;
case TSDB_DATA_TYPE_TINYINT:
len += snprintf(dumpBuf + len, size - len, " %15d |", *(int8_t*)var);
if (len >= size - 1) return 0;
if (len >= size - 1) goto _exit;
break;
case TSDB_DATA_TYPE_UTINYINT:
len += snprintf(dumpBuf + len, size - len, " %15d |", *(uint8_t*)var);
if (len >= size - 1) return 0;
if (len >= size - 1) goto _exit;
break;
case TSDB_DATA_TYPE_SMALLINT:
len += snprintf(dumpBuf + len, size - len, " %15d |", *(int16_t*)var);
if (len >= size - 1) return 0;
if (len >= size - 1) goto _exit;
break;
case TSDB_DATA_TYPE_USMALLINT:
len += snprintf(dumpBuf + len, size - len, " %15d |", *(uint16_t*)var);
if (len >= size - 1) return 0;
if (len >= size - 1) goto _exit;
break;
case TSDB_DATA_TYPE_INT:
len += snprintf(dumpBuf + len, size - len, " %15d |", *(int32_t*)var);
if (len >= size - 1) return 0;
if (len >= size - 1) goto _exit;
break;
case TSDB_DATA_TYPE_UINT:
len += snprintf(dumpBuf + len, size - len, " %15u |", *(uint32_t*)var);
if (len >= size - 1) return 0;
if (len >= size - 1) goto _exit;
break;
case TSDB_DATA_TYPE_BIGINT:
len += snprintf(dumpBuf + len, size - len, " %15" PRId64 " |", *(int64_t*)var);
if (len >= size - 1) return 0;
if (len >= size - 1) goto _exit;
break;
case TSDB_DATA_TYPE_UBIGINT:
len += snprintf(dumpBuf + len, size - len, " %15" PRIu64 " |", *(uint64_t*)var);
if (len >= size - 1) return 0;
if (len >= size - 1) goto _exit;
break;
case TSDB_DATA_TYPE_FLOAT:
len += snprintf(dumpBuf + len, size - len, " %15f |", *(float*)var);
if (len >= size - 1) return 0;
if (len >= size - 1) goto _exit;
break;
case TSDB_DATA_TYPE_DOUBLE:
len += snprintf(dumpBuf + len, size - len, " %15f |", *(double*)var);
if (len >= size - 1) return 0;
if (len >= size - 1) goto _exit;
break;
case TSDB_DATA_TYPE_BOOL:
len += snprintf(dumpBuf + len, size - len, " %15d |", *(bool*)var);
if (len >= size - 1) return 0;
if (len >= size - 1) goto _exit;
break;
case TSDB_DATA_TYPE_VARCHAR:
case TSDB_DATA_TYPE_VARBINARY:
@ -2599,24 +2599,33 @@ int32_t dumpBlockData(SSDataBlock* pDataBlock, const char* flag, char** pDataBuf
dataSize = TMIN(dataSize, 50);
memcpy(pBuf, varDataVal(pData), dataSize);
len += snprintf(dumpBuf + len, size - len, " %15s |", pBuf);
if (len >= size - 1) return 0;
if (len >= size - 1) goto _exit;
} break;
case TSDB_DATA_TYPE_NCHAR: {
char* pData = colDataGetVarData(pColInfoData, j);
int32_t dataSize = TMIN(sizeof(pBuf), varDataLen(pData));
memset(pBuf, 0, sizeof(pBuf));
(void)taosUcs4ToMbs((TdUcs4*)varDataVal(pData), dataSize, pBuf);
code = taosUcs4ToMbs((TdUcs4*)varDataVal(pData), dataSize, pBuf);
if (code < 0) {
uError("func %s failed to convert to ucs charset since %s", __func__, tstrerror(code));
goto _exit;
}
len += snprintf(dumpBuf + len, size - len, " %15s |", pBuf);
if (len >= size - 1) return 0;
if (len >= size - 1) goto _exit;
} break;
}
}
len += snprintf(dumpBuf + len, size - len, "%d\n", j);
if (len >= size - 1) return code;
if (len >= size - 1) goto _exit;
}
len += snprintf(dumpBuf + len, size - len, "%s |end\n", flag);
*pDataBuf = dumpBuf;
dumpBuf = NULL;
_exit:
if (dumpBuf) {
taosMemoryFree(dumpBuf);
}
return code;
}

18
source/common/src/tdataformat.c
View File

@ -489,7 +489,7 @@ int32_t tRowBuildFromBind(SBindInfo *infos, int32_t numOfInfos, bool infoSorted,
colVal = COL_VAL_VALUE(infos[iInfo].columnId, value);
}
if (taosArrayPush(colValArray, &colVal) == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
goto _exit;
}
}
@ -500,7 +500,7 @@ int32_t tRowBuildFromBind(SBindInfo *infos, int32_t numOfInfos, bool infoSorted,
}
if ((taosArrayPush(rowArray, &row)) == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
goto _exit;
}
}
@ -702,7 +702,7 @@ static int32_t tRowMergeImpl(SArray *aRowP, STSchema *pTSchema, int32_t iStart,
if (pColVal) {
if (taosArrayPush(aColVal, pColVal) == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
goto _exit;
}
}
@ -1756,7 +1756,7 @@ int32_t tTagToValArray(const STag *pTag, SArray **ppArray) {
}
(void)tGetTagVal(p + offset, &tv, pTag->flags & TD_TAG_JSON);
if (taosArrayPush(*ppArray, &tv) == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
goto _err;
}
}
@ -3251,13 +3251,13 @@ int32_t tRowBuildFromBind2(SBindInfo2 *infos, int32_t numOfInfos, bool infoSorte
}
if ((bufArray = taosArrayInit(numOfInfos, sizeof(uint8_t *))) == NULL) {
taosArrayDestroy(colValArray);
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
for (int i = 0; i < numOfInfos; ++i) {
if (!taosArrayPush(bufArray, &infos[i].bind->buffer)) {
taosArrayDestroy(colValArray);
taosArrayDestroy(bufArray);
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
}
@ -3294,7 +3294,7 @@ int32_t tRowBuildFromBind2(SBindInfo2 *infos, int32_t numOfInfos, bool infoSorte
colVal = COL_VAL_VALUE(infos[iInfo].columnId, value);
}
if (taosArrayPush(colValArray, &colVal) == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
goto _exit;
}
}
@ -3305,7 +3305,7 @@ int32_t tRowBuildFromBind2(SBindInfo2 *infos, int32_t numOfInfos, bool infoSorte
}
if ((taosArrayPush(rowArray, &row)) == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
goto _exit;
}
}
@ -3502,7 +3502,7 @@ static int32_t tColDataMerge(SArray **colArr) {
dst = taosArrayInit(taosArrayGetSize(src), sizeof(SColData));
if (dst == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
for (int32_t i = 0; i < taosArrayGetSize(src); i++) {

24
source/common/src/tglobal.c
View File

@ -81,6 +81,7 @@ int32_t tsElectInterval = 25 * 1000;
int32_t tsHeartbeatInterval = 1000;
int32_t tsHeartbeatTimeout = 20 * 1000;
int32_t tsSnapReplMaxWaitN = 128;
int64_t tsLogBufferMemoryAllowed = 0; // bytes
// mnode
int64_t tsMndSdbWriteDelta = 200;
@ -613,7 +614,7 @@ static int32_t taosAddClientCfg(SConfig *pCfg) {
cfgAddInt32(pCfg, "timeToGetAvailableConn", tsTimeToGetAvailableConn, 20, 1000000, CFG_SCOPE_BOTH, CFG_DYN_NONE));
tsKeepAliveIdle = TRANGE(tsKeepAliveIdle, 1, 72000);
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "keepAliveIdle", tsKeepAliveIdle, 1, 7200000, CFG_SCOPE_BOTH, CFG_DYN_ENT_BOTH));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "keepAliveIdle", tsKeepAliveIdle, 1, 7200000, CFG_SCOPE_BOTH, CFG_DYN_NONE));
tsNumOfTaskQueueThreads = tsNumOfCores * 2;
tsNumOfTaskQueueThreads = TMAX(tsNumOfTaskQueueThreads, 16);
@ -702,6 +703,9 @@ static int32_t taosAddServerCfg(SConfig *pCfg) {
tsQueueMemoryAllowed = tsTotalMemoryKB * 1024 * 0.1;
tsQueueMemoryAllowed = TRANGE(tsQueueMemoryAllowed, TSDB_MAX_MSG_SIZE * 10LL, TSDB_MAX_MSG_SIZE * 10000LL);
tsLogBufferMemoryAllowed = tsTotalMemoryKB * 1024 * 0.1;
tsLogBufferMemoryAllowed = TRANGE(tsLogBufferMemoryAllowed, TSDB_MAX_MSG_SIZE * 10LL, TSDB_MAX_MSG_SIZE * 10000LL);
// clang-format off
TAOS_CHECK_RETURN(cfgAddDir(pCfg, "dataDir", tsDataDir, CFG_SCOPE_SERVER, CFG_DYN_NONE));
TAOS_CHECK_RETURN(cfgAddFloat(pCfg, "minimalDataDirGB", 2.0f, 0.001f, 10000000, CFG_SCOPE_SERVER, CFG_DYN_NONE));
@ -736,6 +740,7 @@ static int32_t taosAddServerCfg(SConfig *pCfg) {
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "syncHeartbeatInterval", tsHeartbeatInterval, 10, 1000 * 60 * 24 * 2, CFG_SCOPE_SERVER, CFG_DYN_NONE));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "syncHeartbeatTimeout", tsHeartbeatTimeout, 10, 1000 * 60 * 24 * 2, CFG_SCOPE_SERVER, CFG_DYN_NONE));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "syncSnapReplMaxWaitN", tsSnapReplMaxWaitN, 16, (TSDB_SYNC_SNAP_BUFFER_SIZE >> 2), CFG_SCOPE_SERVER, CFG_DYN_NONE));
TAOS_CHECK_RETURN(cfgAddInt64(pCfg, "syncLogBufferMemoryAllowed", tsLogBufferMemoryAllowed, TSDB_MAX_MSG_SIZE * 10L, INT64_MAX, CFG_SCOPE_SERVER, CFG_DYN_ENT_SERVER));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "arbHeartBeatIntervalSec", tsArbHeartBeatIntervalSec, 1, 60 * 24 * 2, CFG_SCOPE_SERVER, CFG_DYN_NONE));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "arbCheckSyncIntervalSec", tsArbCheckSyncIntervalSec, 1, 60 * 24 * 2, CFG_SCOPE_SERVER, CFG_DYN_NONE));
@ -970,6 +975,14 @@ static int32_t taosUpdateServerCfg(SConfig *pCfg) {
pItem->stype = stype;
}
pItem = cfgGetItem(tsCfg, "syncLogBufferMemoryAllowed");
if (pItem != NULL && pItem->stype == CFG_STYPE_DEFAULT) {
tsLogBufferMemoryAllowed = totalMemoryKB * 1024 * 0.1;
tsLogBufferMemoryAllowed = TRANGE(tsLogBufferMemoryAllowed, TSDB_MAX_MSG_SIZE * 10LL, TSDB_MAX_MSG_SIZE * 10000LL);
pItem->i64 = tsLogBufferMemoryAllowed;
pItem->stype = stype;
}
TAOS_RETURN(TSDB_CODE_SUCCESS);
}
@ -1520,6 +1533,9 @@ static int32_t taosSetServerCfg(SConfig *pCfg) {
TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "syncSnapReplMaxWaitN");
tsSnapReplMaxWaitN = pItem->i32;
TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "syncLogBufferMemoryAllowed");
tsLogBufferMemoryAllowed = pItem->i64;
TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "arbHeartBeatIntervalSec");
tsArbHeartBeatIntervalSec = pItem->i32;
@ -1696,7 +1712,10 @@ int32_t taosReadDataFolder(const char *cfgDir, const char **envCmd, const char *
SArray *pArgs) {
int32_t code = TSDB_CODE_SUCCESS;
if (tsCfg == NULL) osDefaultInit();
if (tsCfg == NULL) code = osDefaultInit();
if (code != 0) {
(void)printf("failed to init os since %s\n", tstrerror(code));
}
SConfig *pCfg = NULL;
TAOS_CHECK_RETURN(cfgInit(&pCfg));
@ -1954,6 +1973,7 @@ static int32_t taosCfgDynamicOptionsForServer(SConfig *pCfg, const char *name) {
{"randErrorChance", &tsRandErrChance},
{"randErrorDivisor", &tsRandErrDivisor},
{"randErrorScope", &tsRandErrScope},
{"syncLogBufferMemoryAllowed", &tsLogBufferMemoryAllowed},
{"cacheLazyLoadThreshold", &tsCacheLazyLoadThreshold},
{"checkpointInterval", &tsStreamCheckpointInterval},

4
source/common/src/tmisce.c
View File

@ -179,7 +179,7 @@ int32_t epsetToStr(const SEpSet* pEpSet, char* pBuf, int32_t cap) {
int32_t taosGenCrashJsonMsg(int signum, char** pMsg, int64_t clusterId, int64_t startTime) {
int32_t code = 0;
SJson* pJson = tjsonCreateObject();
if (pJson == NULL) return TSDB_CODE_OUT_OF_MEMORY;
if (pJson == NULL) return terrno;
char tmp[4096] = {0};
@ -241,7 +241,7 @@ int32_t taosGenCrashJsonMsg(int signum, char** pMsg, int64_t clusterId, int64_t
char* pCont = tjsonToString(pJson);
if (pCont == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
TAOS_CHECK_GOTO(code, NULL, _exit);
goto _exit;
}

12
source/common/src/tmsg.c
View File

@ -10346,7 +10346,11 @@ int32_t tDecodeSMCreateStbRsp(SDecoder *pDecoder, SMCreateStbRsp *pRsp) {
}
tEndDecode(pDecoder);
return code;
_exit:
tFreeSTableMetaRsp(pRsp->pMeta);
taosMemoryFreeClear(pRsp->pMeta);
return code;
}
@ -12026,19 +12030,19 @@ int32_t tCloneTbTSMAInfo(STableTSMAInfo *pInfo, STableTSMAInfo **pRes) {
*pRet = *pInfo;
if (pInfo->pFuncs) {
pRet->pFuncs = taosArrayDup(pInfo->pFuncs, NULL);
if (!pRet->pFuncs) code = TSDB_CODE_OUT_OF_MEMORY;
if (!pRet->pFuncs) code = terrno;
}
if (pInfo->pTags && code == TSDB_CODE_SUCCESS) {
pRet->pTags = taosArrayDup(pInfo->pTags, NULL);
if (!pRet->pTags) code = TSDB_CODE_OUT_OF_MEMORY;
if (!pRet->pTags) code = terrno;
}
if (pInfo->pUsedCols && code == TSDB_CODE_SUCCESS) {
pRet->pUsedCols = taosArrayDup(pInfo->pUsedCols, NULL);
if (!pRet->pUsedCols) code = TSDB_CODE_OUT_OF_MEMORY;
if (!pRet->pUsedCols) code = terrno;
}
if (pInfo->ast && code == TSDB_CODE_SUCCESS) {
pRet->ast = taosStrdup(pInfo->ast);
if (!pRet->ast) code = TSDB_CODE_OUT_OF_MEMORY;
if (!pRet->ast) code = terrno;
}
if (code) {
tFreeAndClearTableTSMAInfo(pRet);

3
source/common/src/tname.c
View File

@ -88,6 +88,9 @@ int64_t taosGetIntervalStartTimestamp(int64_t startTime, int64_t slidingTime, in
#endif
void toName(int32_t acctId, const char* pDbName, const char* pTableName, SName* pName) {
if (pName == NULL){
return;
}
pName->type = TSDB_TABLE_NAME_T;
pName->acctId = acctId;
snprintf(pName->dbname, sizeof(pName->dbname), "%s", pDbName);

2
source/common/src/trow.c
View File

@ -482,7 +482,7 @@ int32_t tdSTSRowNew(SArray *pArray, STSchema *pTSchema, STSRow **ppRow, int8_t r
if (isAlloc) {
taosMemoryFreeClear(*ppRow);
}
TAOS_RETURN(TSDB_CODE_OUT_OF_MEMORY);
TAOS_RETURN(terrno);
}
}

12
source/common/src/ttime.c
View File

@ -1247,7 +1247,7 @@ static int32_t parseTsFormat(const char* formatStr, SArray* formats) {
const TSFormatKeyWord* key = keywordSearch(formatStr);
if (key) {
TSFormatNode format = {.key = key, .type = TS_FORMAT_NODE_TYPE_KEYWORD};
if (NULL == taosArrayPush(formats, &format)) TAOS_RETURN(TSDB_CODE_OUT_OF_MEMORY);
if (NULL == taosArrayPush(formats, &format)) TAOS_RETURN(terrno);
formatStr += key->len;
lastOtherFormat = NULL;
} else {
@ -1274,7 +1274,7 @@ static int32_t parseTsFormat(const char* formatStr, SArray* formats) {
TSFormatNode format = {.type = TS_FORMAT_NODE_TYPE_CHAR, .key = NULL};
format.c = formatStr;
format.len = 1;
if (NULL == taosArrayPush(formats, &format)) TAOS_RETURN(TSDB_CODE_OUT_OF_MEMORY);
if (NULL == taosArrayPush(formats, &format)) TAOS_RETURN(terrno);
formatStr++;
last = taosArrayGetLast(formats);
}
@ -1301,7 +1301,7 @@ static int32_t parseTsFormat(const char* formatStr, SArray* formats) {
.key = NULL};
format.c = formatStr;
format.len = 1;
if (NULL == taosArrayPush(formats, &format)) TAOS_RETURN(TSDB_CODE_OUT_OF_MEMORY);
if (NULL == taosArrayPush(formats, &format)) TAOS_RETURN(terrno);
formatStr++;
if (format.type == TS_FORMAT_NODE_TYPE_CHAR) lastOtherFormat = taosArrayGetLast(formats);
}
@ -1910,7 +1910,7 @@ int32_t taosTs2Char(const char* format, SArray** formats, int64_t ts, int32_t pr
if (!*formats) {
*formats = taosArrayInit(8, sizeof(TSFormatNode));
if (!*formats) {
TAOS_RETURN(TSDB_CODE_OUT_OF_MEMORY);
TAOS_RETURN(terrno);
}
TAOS_CHECK_RETURN(parseTsFormat(format, *formats));
}
@ -1926,7 +1926,7 @@ int32_t taosChar2Ts(const char* format, SArray** formats, const char* tsStr, int
if (!*formats) {
*formats = taosArrayInit(4, sizeof(TSFormatNode));
if (!*formats) {
TAOS_RETURN(TSDB_CODE_OUT_OF_MEMORY);
TAOS_RETURN(terrno);
}
TAOS_CHECK_RETURN(parseTsFormat(format, *formats));
}
@ -1951,7 +1951,7 @@ int32_t TEST_ts2char(const char* format, int64_t ts, int32_t precision, char* ou
SArray* formats = taosArrayInit(4, sizeof(TSFormatNode));
if (!formats) {
TAOS_RETURN(TSDB_CODE_OUT_OF_MEMORY);
TAOS_RETURN(terrno);
}
TAOS_CHECK_RETURN(parseTsFormat(format, formats));
struct STm tm;

8
source/dnode/mgmt/mgmt_dnode/src/dmHandle.c
View File

@ -345,7 +345,7 @@ int32_t dmProcessServerRunStatus(SDnodeMgmt *pMgmt, SRpcMsg *pMsg) {
void *pRsp = rpcMallocCont(rspLen);
if (pRsp == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
// rspMsg.code = TSDB_CODE_OUT_OF_MEMORY;
// return rspMsg.code;
}
@ -383,7 +383,7 @@ int32_t dmBuildVariablesBlock(SSDataBlock **ppBlock) {
pBlock->pDataBlock = taosArrayInit(pMeta[index].colNum, sizeof(SColumnInfoData));
if (pBlock->pDataBlock == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
goto _exit;
}
@ -393,7 +393,7 @@ int32_t dmBuildVariablesBlock(SSDataBlock **ppBlock) {
colInfoData.info.type = pMeta[index].schema[i].type;
colInfoData.info.bytes = pMeta[index].schema[i].bytes;
if (taosArrayPush(pBlock->pDataBlock, &colInfoData) == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
goto _exit;
}
}
@ -457,7 +457,7 @@ int32_t dmProcessRetrieve(SDnodeMgmt *pMgmt, SRpcMsg *pMsg) {
SRetrieveMetaTableRsp *pRsp = rpcMallocCont(size);
if (pRsp == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
dError("failed to retrieve data since %s", tstrerror(code));
blockDataDestroy(pBlock);
return code;

2
source/dnode/mgmt/mgmt_dnode/src/dmInt.c
View File

@ -54,7 +54,7 @@ static int32_t dmOpenMgmt(SMgmtInputOpt *pInput, SMgmtOutputOpt *pOutput) {
int32_t code = 0;
SDnodeMgmt *pMgmt = taosMemoryCalloc(1, sizeof(SDnodeMgmt));
if (pMgmt == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
pMgmt->pData = pInput->pData;

8
source/dnode/mgmt/mgmt_mnode/src/mmFile.c
View File

@ -91,7 +91,7 @@ int32_t mmReadFile(const char *path, SMnodeOpt *pOption) {
pData = taosMemoryMalloc(size + 1);
if (pData == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
goto _OVER;
}
@ -134,14 +134,14 @@ static int32_t mmEncodeOption(SJson *pJson, const SMnodeOpt *pOption) {
SJson *replicas = tjsonCreateArray();
if (replicas == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
if ((code = tjsonAddItemToObject(pJson, "replicas", replicas)) < 0) return code;
for (int32_t i = 0; i < pOption->numOfTotalReplicas; ++i) {
SJson *replica = tjsonCreateObject();
if (replica == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
const SReplica *pReplica = pOption->replicas + i;
@ -183,7 +183,7 @@ int32_t mmWriteFile(const char *path, const SMnodeOpt *pOption) {
// terrno = TSDB_CODE_OUT_OF_MEMORY;
pJson = tjsonCreateObject();
if (pJson == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
goto _OVER;
}

6
source/dnode/mgmt/mgmt_snode/src/smWorker.c
View File

@ -71,14 +71,14 @@ int32_t smStartWorker(SSnodeMgmt *pMgmt) {
int32_t code = 0;
pMgmt->writeWroker = taosArrayInit(0, sizeof(SMultiWorker *));
if (pMgmt->writeWroker == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
return code;
}
for (int32_t i = 0; i < tsNumOfSnodeWriteThreads; i++) {
SMultiWorker *pWriteWorker = taosMemoryMalloc(sizeof(SMultiWorker));
if (pWriteWorker == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
return code;
}
@ -93,7 +93,7 @@ int32_t smStartWorker(SSnodeMgmt *pMgmt) {
return code;
}
if (taosArrayPush(pMgmt->writeWroker, &pWriteWorker) == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
return code;
}
}

8
source/dnode/mgmt/mgmt_vnode/src/vmFile.c
View File

@ -128,7 +128,7 @@ int32_t vmGetVnodeListFromFile(SVnodeMgmt *pMgmt, SWrapperCfg **ppCfgs, int32_t
pData = taosMemoryMalloc(size + 1);
if (pData == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
goto _OVER;
}
@ -169,7 +169,7 @@ static int32_t vmEncodeVnodeList(SJson *pJson, SVnodeObj **ppVnodes, int32_t num
int32_t code = 0;
SJson *vnodes = tjsonCreateArray();
if (vnodes == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
if ((code = tjsonAddItemToObject(pJson, "vnodes", vnodes)) < 0) {
tjsonDelete(vnodes);
@ -181,7 +181,7 @@ static int32_t vmEncodeVnodeList(SJson *pJson, SVnodeObj **ppVnodes, int32_t num
if (pVnode == NULL) continue;
SJson *vnode = tjsonCreateObject();
if (vnode == NULL) return TSDB_CODE_OUT_OF_MEMORY;
if (vnode == NULL) return terrno;
if ((code = tjsonAddDoubleToObject(vnode, "vgId", pVnode->vgId)) < 0) return code;
if ((code = tjsonAddDoubleToObject(vnode, "dropped", pVnode->dropped)) < 0) return code;
if ((code = tjsonAddDoubleToObject(vnode, "vgVersion", pVnode->vgVersion)) < 0) return code;
@ -221,7 +221,7 @@ int32_t vmWriteVnodeListToFile(SVnodeMgmt *pMgmt) {
// terrno = TSDB_CODE_OUT_OF_MEMORY;
pJson = tjsonCreateObject();
if (pJson == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
goto _OVER;
}
if ((code = vmEncodeVnodeList(pJson, ppVnodes, numOfVnodes)) != 0) goto _OVER;

2
source/dnode/mgmt/mgmt_vnode/src/vmHandle.c
View File

@ -921,7 +921,7 @@ int32_t vmProcessArbHeartBeatReq(SVnodeMgmt *pMgmt, SRpcMsg *pMsg) {
void *pRsp = rpcMallocCont(rspLen);
if (pRsp == NULL) {
terrno = TSDB_CODE_OUT_OF_MEMORY;
terrno = terrno;
goto _OVER;
}

3
source/dnode/mgmt/node_mgmt/src/dmMgmt.c
View File

@ -161,8 +161,7 @@ int32_t dmInitVars(SDnode *pDnode) {
pData->dnodeHash = taosHashInit(4, taosGetDefaultHashFunction(TSDB_DATA_TYPE_INT), true, HASH_NO_LOCK);
if (pData->dnodeHash == NULL) {
dError("failed to init dnode hash");
code = TSDB_CODE_OUT_OF_MEMORY;
return terrno = code;
return terrno;
}
if ((code = dmReadEps(pData)) != 0) {

12
source/dnode/mgmt/node_util/src/dmEps.c
View File

@ -217,7 +217,7 @@ int32_t dmReadEps(SDnodeData *pData) {
content = taosMemoryMalloc(size + 1);
if (content == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
goto _OVER;
}
@ -321,7 +321,7 @@ int32_t dmWriteEps(SDnodeData *pData) {
TAOS_CHECK_GOTO(dmInitDndInfo(pData), NULL, _OVER);
pJson = tjsonCreateObject();
if (pJson == NULL) TAOS_CHECK_GOTO(TSDB_CODE_OUT_OF_MEMORY, NULL, _OVER);
if (pJson == NULL) TAOS_CHECK_GOTO(terrno, NULL, _OVER);
pData->engineVer = tsVersion;
@ -329,7 +329,7 @@ int32_t dmWriteEps(SDnodeData *pData) {
buffer = tjsonToString(pJson);
if (buffer == NULL) {
TAOS_CHECK_GOTO(TSDB_CODE_OUT_OF_MEMORY, NULL, _OVER);
TAOS_CHECK_GOTO(terrno, NULL, _OVER);
}
pFile = taosOpenFile(file, TD_FILE_CREATE | TD_FILE_WRITE | TD_FILE_TRUNC | TD_FILE_WRITE_THROUGH);
@ -574,7 +574,7 @@ static int32_t dmDecodeEpPairs(SJson *pJson, SDnodeData *pData) {
tjsonGetUInt16ValueFromDouble(dnode, "new_port", pair.newPort, code);
if (code < 0) return TSDB_CODE_INVALID_CFG_VALUE;
if (taosArrayPush(pData->oldDnodeEps, &pair) == NULL) return TSDB_CODE_OUT_OF_MEMORY;
if (taosArrayPush(pData->oldDnodeEps, &pair) == NULL) return terrno;
}
return code;
@ -620,7 +620,7 @@ static int32_t dmReadDnodePairs(SDnodeData *pData) {
content = taosMemoryMalloc(size + 1);
if (content == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
goto _OVER;
}
@ -640,7 +640,7 @@ static int32_t dmReadDnodePairs(SDnodeData *pData) {
pData->oldDnodeEps = taosArrayInit(1, sizeof(SDnodeEpPair));
if (pData->oldDnodeEps == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
dError("failed to calloc dnodeEp array since %s", strerror(errno));
goto _OVER;
}

8
source/dnode/mgmt/node_util/src/dmFile.c
View File

@ -70,7 +70,7 @@ int32_t dmReadFile(const char *path, const char *name, bool *pDeployed) {
content = taosMemoryMalloc(size + 1);
if (content == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
goto _OVER;
}
@ -134,7 +134,7 @@ int32_t dmWriteFile(const char *path, const char *name, bool deployed) {
pJson = tjsonCreateObject();
if (pJson == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
goto _OVER;
}
@ -226,7 +226,7 @@ static int32_t dmWriteCheckCodeFile(char *file, char *realfile, char *key, bool
int32_t len = ENCRYPTED_LEN(sizeof(DM_KEY_INDICATOR));
result = taosMemoryMalloc(len);
if (result == NULL) {
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
SCryptOpts opts;
@ -469,7 +469,7 @@ static int32_t dmReadEncryptCodeFile(char *file, char **output) {
content = taosMemoryMalloc(size + 1);
if (content == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
code = terrno;
goto _OVER;
}

13
source/dnode/mnode/impl/src/mndConsumer.c
View File

@ -199,7 +199,7 @@ static void storeOffsetRows(SMnode *pMnode, SMqHbReq *req, SMqConsumerObj *pCons
taosWLockLatch(&pSub->lock);
SMqConsumerEp *pConsumerEp = taosHashGet(pSub->consumerHash, &pConsumer->consumerId, sizeof(int64_t));
if (pConsumerEp) {
(void)taosArrayDestroy(pConsumerEp->offsetRows);
taosArrayDestroy(pConsumerEp->offsetRows);
pConsumerEp->offsetRows = data->offsetRows;
data->offsetRows = NULL;
}
@ -511,6 +511,11 @@ static int32_t getTopicAddDelete(SMqConsumerObj *pExistedConsumer, SMqConsumerOb
}
}
}
// no topics need to be rebalanced
if (taosArrayGetSize(pConsumerNew->rebNewTopics) == 0 && taosArrayGetSize(pConsumerNew->rebRemovedTopics) == 0) {
code = TSDB_CODE_TMQ_NO_NEED_REBALANCE;
}
END:
return code;
}
@ -581,6 +586,10 @@ int32_t mndProcessSubscribeReq(SRpcMsg *pMsg) {
if(ubSubscribe){
SMqConsumerObj *pConsumerTmp = NULL;
MND_TMQ_RETURN_CHECK(mndAcquireConsumer(pMnode, subscribe.consumerId, &pConsumerTmp));
if (taosArrayGetSize(pConsumerTmp->assignedTopics) == 0){
mndReleaseConsumer(pMnode, pConsumerTmp);
goto END;
}
mndReleaseConsumer(pMnode, pConsumerTmp);
}
MND_TMQ_RETURN_CHECK(checkAndSortTopic(pMnode, subscribe.topicNames));
@ -599,7 +608,7 @@ END:
mndTransDrop(pTrans);
tDeleteSMqConsumerObj(pConsumerNew);
taosArrayDestroyP(subscribe.topicNames, (FDelete)taosMemoryFree);
return code;
return (code == TSDB_CODE_TMQ_NO_NEED_REBALANCE || code == TSDB_CODE_MND_CONSUMER_NOT_EXIST) ? 0 : code;
}
SSdbRaw *mndConsumerActionEncode(SMqConsumerObj *pConsumer) {

2
source/dnode/mnode/impl/src/mndProfile.c
View File

@ -91,7 +91,7 @@ int32_t mndInitProfile(SMnode *pMnode) {
SProfileMgmt *pMgmt = &pMnode->profileMgmt;
// in ms
int32_t checkTime = tsShellActivityTimer * 2 * 1000;
int32_t checkTime = CACHE_OBJ_KEEP_TIME * 1000;
pMgmt->connCache = taosCacheInit(TSDB_DATA_TYPE_UINT, checkTime, false, (__cache_free_fn_t)mndFreeConn, "conn");
if (pMgmt->connCache == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;

12
source/dnode/mnode/impl/src/mndStream.c
View File

@ -2221,7 +2221,11 @@ static int32_t refreshNodeListFromExistedStreams(SMnode *pMnode, SArray *pNodeLi
}
char buf[256] = {0};
(void) epsetToStr(&pEntry->epset, buf, tListLen(buf)); // ignore this error since it is only for log file
int32_t ret = epsetToStr(&pEntry->epset, buf, tListLen(buf)); // ignore this error since it is only for log file
if (ret != 0) { // print error and continue
mError("failed to convert epset to str, code:%s", tstrerror(ret));
}
mDebug("extract nodeInfo from stream obj, nodeId:%d, %s", pEntry->nodeId, buf);
}
@ -2231,7 +2235,7 @@ static int32_t refreshNodeListFromExistedStreams(SMnode *pMnode, SArray *pNodeLi
return code;
}
static int32_t addAllDbsIntoHashmap(SHashObj *pDBMap, SSdb *pSdb) {
static void addAllDbsIntoHashmap(SHashObj *pDBMap, SSdb *pSdb) {
void *pIter = NULL;
int32_t code = 0;
while (1) {
@ -2249,8 +2253,6 @@ static int32_t addAllDbsIntoHashmap(SHashObj *pDBMap, SSdb *pSdb) {
mDebug("add Db:%s into Dbs list (total:%d) for kill checkpoint trans", pVgroup->dbName, size);
}
}
return code;
}
// this function runs by only one thread, so it is not multi-thread safe
@ -2311,7 +2313,7 @@ static int32_t mndProcessNodeCheckReq(SRpcMsg *pMsg) {
mInfo("rollback all stream due to mnode leader/follower switch by using nodeUpdate trans");
updateAllVgroups = true;
execInfo.switchFromFollower = false; // reset the flag
(void) addAllDbsIntoHashmap(changeInfo.pDBMap, pMnode->pSdb);
addAllDbsIntoHashmap(changeInfo.pDBMap, pMnode->pSdb);
}
}

11
source/dnode/mnode/impl/src/mndStreamHb.c
View File

@ -194,10 +194,13 @@ int32_t mndSendDropOrphanTasksMsg(SMnode *pMnode, SArray *pList) {
return terrno;
}
(void)tSerializeDropOrphanTaskMsg(pReq, contLen, &msg);
int32_t code = tSerializeDropOrphanTaskMsg(pReq, contLen, &msg);
if (code <= 0) {
mError("failed to serialize the drop orphan task msg, code:%s", tstrerror(code));
}
SRpcMsg rpcMsg = {.msgType = TDMT_MND_STREAM_DROP_ORPHANTASKS, .pCont = pReq, .contLen = contLen};
int32_t code = tmsgPutToQueue(&pMnode->msgCb, WRITE_QUEUE, &rpcMsg);
code = tmsgPutToQueue(&pMnode->msgCb, WRITE_QUEUE, &rpcMsg);
if (code) {
mError("failed to put drop-orphan task msg into write queue, code:%s", tstrerror(code));
} else {
@ -216,7 +219,7 @@ int32_t mndProcessResetStatusReq(SRpcMsg *pReq) {
mndKillTransImpl(pMnode, pMsg->transId, "");
streamMutexLock(&execInfo.lock);
(void) mndResetChkptReportInfo(execInfo.pChkptStreams, pMsg->streamId);
code = mndResetChkptReportInfo(execInfo.pChkptStreams, pMsg->streamId); // do thing if failed
streamMutexUnlock(&execInfo.lock);
code = mndGetStreamObj(pMnode, pMsg->streamId, &pStream);
@ -393,7 +396,7 @@ int32_t mndProcessStreamHb(SRpcMsg *pReq) {
int32_t numOfUpdated = taosArrayGetSize(req.pUpdateNodes);
if (numOfUpdated > 0) {
mDebug("%d stream node(s) need updated from hbMsg(vgId:%d)", numOfUpdated, req.vgId);
(void) setNodeEpsetExpiredFlag(req.pUpdateNodes);
int32_t unused = setNodeEpsetExpiredFlag(req.pUpdateNodes);
}
bool snodeChanged = false;

31
source/dnode/mnode/impl/src/mndStreamUtil.c
View File

@ -165,7 +165,10 @@ int32_t mndTakeVgroupSnapshot(SMnode *pMnode, bool *allReady, SArray **pList) {
}
char buf[256] = {0};
(void)epsetToStr(&entry.epset, buf, tListLen(buf));
code = epsetToStr(&entry.epset, buf, tListLen(buf));
if (code != 0) { // print error and continue
mError("failed to convert epset to str, code:%s", tstrerror(code));
}
void *p = taosArrayPush(pVgroupList, &entry);
if (p == NULL) {
@ -198,7 +201,10 @@ int32_t mndTakeVgroupSnapshot(SMnode *pMnode, bool *allReady, SArray **pList) {
}
char buf[256] = {0};
(void)epsetToStr(&entry.epset, buf, tListLen(buf));
code = epsetToStr(&entry.epset, buf, tListLen(buf));
if (code != 0) { // print error and continue
mError("failed to convert epset to str, code:%s", tstrerror(code));
}
void *p = taosArrayPush(pVgroupList, &entry);
if (p == NULL) {
@ -424,9 +430,12 @@ static int32_t doSetPauseAction(SMnode *pMnode, STrans *pTrans, SStreamTask *pTa
}
char buf[256] = {0};
(void) epsetToStr(&epset, buf, tListLen(buf));
mDebug("pause stream task in node:%d, epset:%s", pTask->info.nodeId, buf);
code = epsetToStr(&epset, buf, tListLen(buf));
if (code != 0) { // print error and continue
mError("failed to convert epset to str, code:%s", tstrerror(code));
}
mDebug("pause stream task in node:%d, epset:%s", pTask->info.nodeId, buf);
code = setTransAction(pTrans, pReq, sizeof(SVPauseStreamTaskReq), TDMT_STREAM_TASK_PAUSE, &epset, 0, TSDB_CODE_VND_INVALID_VGROUP_ID);
if (code != 0) {
taosMemoryFree(pReq);
@ -639,8 +648,7 @@ static int32_t doBuildStreamTaskUpdateMsg(void **pBuf, int32_t *pLen, SVgroupCha
static int32_t doSetUpdateTaskAction(SMnode *pMnode, STrans *pTrans, SStreamTask *pTask, SVgroupChangeInfo *pInfo) {
void *pBuf = NULL;
int32_t len = 0;
(void)streamTaskUpdateEpsetInfo(pTask, pInfo->pUpdateNodeList);
bool unusedRet = streamTaskUpdateEpsetInfo(pTask, pInfo->pUpdateNodeList);
int32_t code = doBuildStreamTaskUpdateMsg(&pBuf, &len, pInfo, pTask->info.nodeId, &pTask->id, pTrans->id);
if (code) {
return code;
@ -914,8 +922,15 @@ void removeStreamTasksInBuf(SStreamObj *pStream, SStreamExecInfo *pExecNode) {
}
// 2. remove stream entry in consensus hash table and checkpoint-report hash table
(void) mndClearConsensusCheckpointId(execInfo.pStreamConsensus, pStream->uid);
(void) mndClearChkptReportInfo(execInfo.pChkptStreams, pStream->uid);
code = mndClearConsensusCheckpointId(execInfo.pStreamConsensus, pStream->uid);
if (code) {
mError("failed to clear consensus checkpointId, code:%s", tstrerror(code));
}
code = mndClearChkptReportInfo(execInfo.pChkptStreams, pStream->uid);
if (code) {
mError("failed to clear the checkpoint report info, code:%s", tstrerror(code));
}
streamMutexUnlock(&pExecNode->lock);
destroyStreamTaskIter(pIter);

24
source/dnode/mnode/impl/src/mndSubscribe.c
View File

@ -248,7 +248,7 @@ static int32_t processRemovedConsumers(SMqRebOutputObj *pOutput, SHashObj *pHash
MND_TMQ_RETURN_CHECK(pushVgDataToHash(pConsumerEp->vgs, pHash, *consumerId, pOutput->pSub->key));
}
(void)taosArrayDestroy(pConsumerEp->vgs);
taosArrayDestroy(pConsumerEp->vgs);
MND_TMQ_RETURN_CHECK(taosHashRemove(pOutput->pSub->consumerHash, consumerId, sizeof(int64_t)));
MND_TMQ_NULL_CHECK(taosArrayPush(pOutput->removedConsumers, consumerId));
actualRemoved++;
@ -682,8 +682,8 @@ END:
static void freeRebalanceItem(void *param) {
SMqRebInfo *pInfo = param;
(void)taosArrayDestroy(pInfo->newConsumers);
(void)taosArrayDestroy(pInfo->removedConsumers);
taosArrayDestroy(pInfo->newConsumers);
taosArrayDestroy(pInfo->removedConsumers);
}
// type = 0 remove type = 1 add
@ -738,8 +738,12 @@ static void checkForVgroupSplit(SMnode *pMnode, SMqConsumerObj *pConsumer, SHash
}
SVgObj *pVgroup = mndAcquireVgroup(pMnode, pVgEp->vgId);
if (!pVgroup) {
(void)mndGetOrCreateRebSub(rebSubHash, key, NULL);
mInfo("vnode splitted, vgId:%d rebalance will be triggered", pVgEp->vgId);
code = mndGetOrCreateRebSub(rebSubHash, key, NULL);
if (code != 0){
mError("failed to mndGetOrCreateRebSub vgroup:%d, error:%s", pVgEp->vgId, tstrerror(code))
}else{
mInfo("vnode splitted, vgId:%d rebalance will be triggered", pVgEp->vgId);
}
}
mndReleaseVgroup(pMnode, pVgroup);
}
@ -813,10 +817,10 @@ void mndRebCntDec() {
}
static void clearRebOutput(SMqRebOutputObj *rebOutput) {
(void)taosArrayDestroy(rebOutput->newConsumers);
(void)taosArrayDestroy(rebOutput->modifyConsumers);
(void)taosArrayDestroy(rebOutput->removedConsumers);
(void)taosArrayDestroy(rebOutput->rebVgs);
taosArrayDestroy(rebOutput->newConsumers);
taosArrayDestroy(rebOutput->modifyConsumers);
taosArrayDestroy(rebOutput->removedConsumers);
taosArrayDestroy(rebOutput->rebVgs);
tDeleteSubscribeObj(rebOutput->pSub);
taosMemoryFree(rebOutput->pSub);
}
@ -858,7 +862,7 @@ static int32_t checkConsumer(SMnode *pMnode, SMqSubscribeObj *pSub) {
mError("consumer:0x%" PRIx64 " not exists in sdb for exception", pConsumerEp->consumerId);
MND_TMQ_NULL_CHECK(taosArrayAddAll(pSub->unassignedVgs, pConsumerEp->vgs));
(void)taosArrayDestroy(pConsumerEp->vgs);
taosArrayDestroy(pConsumerEp->vgs);
MND_TMQ_RETURN_CHECK(taosHashRemove(pSub->consumerHash, &pConsumerEp->consumerId, sizeof(int64_t)));
}
END:

16
source/dnode/mnode/impl/src/mndSync.c
View File

@ -543,14 +543,14 @@ void mndSyncCheckTimeout(SMnode *pMnode) {
if (delta > MNODE_TIMEOUT_SEC) {
mError("trans:%d, failed to propose since timeout, start:%d cur:%d delta:%d seq:%" PRId64, pMgmt->transId,
pMgmt->transSec, curSec, delta, pMgmt->transSeq);
pMgmt->transId = 0;
pMgmt->transSec = 0;
pMgmt->transSeq = 0;
terrno = TSDB_CODE_SYN_TIMEOUT;
pMgmt->errCode = TSDB_CODE_SYN_TIMEOUT;
if (tsem_post(&pMgmt->syncSem) < 0) {
mError("failed to post sem");
}
// pMgmt->transId = 0;
// pMgmt->transSec = 0;
// pMgmt->transSeq = 0;
// terrno = TSDB_CODE_SYN_TIMEOUT;
// pMgmt->errCode = TSDB_CODE_SYN_TIMEOUT;
//if (tsem_post(&pMgmt->syncSem) < 0) {
// mError("failed to post sem");
//}
} else {
mDebug("trans:%d, waiting for sync confirm, start:%d cur:%d delta:%d seq:%" PRId64, pMgmt->transId,
pMgmt->transSec, curSec, curSec - pMgmt->transSec, pMgmt->transSeq);

21
source/dnode/mnode/impl/src/mndTopic.c
View File

@ -302,7 +302,7 @@ static int32_t mndTopicActionDelete(SSdb *pSdb, SMqTopicObj *pTopic) {
taosMemoryFreeClear(pTopic->ast);
taosMemoryFreeClear(pTopic->physicalPlan);
if (pTopic->schema.nCols) taosMemoryFreeClear(pTopic->schema.pSchema);
(void)taosArrayDestroy(pTopic->ntbColIds);
taosArrayDestroy(pTopic->ntbColIds);
return 0;
}
@ -467,7 +467,7 @@ static int32_t mndCreateTopic(SMnode *pMnode, SRpcMsg *pReq, SCMCreateTopicReq *
MND_TMQ_NULL_CHECK(topicObj.ntbColIds);
MND_TMQ_RETURN_CHECK(extractTopicTbInfo(pAst, &topicObj));
if (topicObj.ntbUid == 0) {
(void)taosArrayDestroy(topicObj.ntbColIds);
taosArrayDestroy(topicObj.ntbColIds);
topicObj.ntbColIds = NULL;
}
@ -505,7 +505,7 @@ END:
taosMemoryFreeClear(topicObj.physicalPlan);
taosMemoryFreeClear(topicObj.sql);
taosMemoryFreeClear(topicObj.ast);
(void)taosArrayDestroy(topicObj.ntbColIds);
taosArrayDestroy(topicObj.ntbColIds);
if (topicObj.schema.nCols) {
taosMemoryFreeClear(topicObj.schema.pSchema);
}
@ -567,9 +567,15 @@ static int32_t mndProcessCreateTopicReq(SRpcMsg *pReq) {
{
SName dbname = {0};
(void)tNameFromString(&dbname, createTopicReq.subDbName, T_NAME_ACCT | T_NAME_DB); // ignore error
int32_t ret = tNameFromString(&dbname, createTopicReq.subDbName, T_NAME_ACCT | T_NAME_DB);
if (ret != 0){
mError("failed to parse db name:%s, ret:%d", createTopicReq.subDbName, ret);
}
SName topicName = {0};
(void)tNameFromString(&topicName, createTopicReq.name, T_NAME_ACCT | T_NAME_DB | T_NAME_TABLE); // ignore error
ret = tNameFromString(&topicName, createTopicReq.name, T_NAME_ACCT | T_NAME_DB | T_NAME_TABLE);
if (ret != 0){
mError("failed to parse topic name:%s, ret:%d", createTopicReq.name, ret);
}
auditRecord(pReq, pMnode->clusterId, "createTopic", dbname.dbname, topicName.dbname,
createTopicReq.sql, strlen(createTopicReq.sql));
}
@ -735,7 +741,10 @@ END:
}
SName name = {0};
(void)tNameFromString(&name, dropReq.name, T_NAME_ACCT | T_NAME_DB | T_NAME_TABLE); // ignore error
int32_t ret = tNameFromString(&name, dropReq.name, T_NAME_ACCT | T_NAME_DB | T_NAME_TABLE);
if (ret != 0) {
mError("topic:%s, failed to drop since %s", dropReq.name, tstrerror(ret));
}
auditRecord(pReq, pMnode->clusterId, "dropTopic", name.dbname, name.tname, dropReq.sql, dropReq.sqlLen);
tFreeSMDropTopicReq(&dropReq);

11
source/dnode/vnode/src/tq/tq.c
View File

@ -344,7 +344,9 @@ int32_t tqProcessPollPush(STQ* pTq, SRpcMsg* pMsg) {
.pCont = pHandle->msg->pCont,
.contLen = pHandle->msg->contLen,
.info = pHandle->msg->info};
(void)tmsgPutToQueue(&pTq->pVnode->msgCb, QUERY_QUEUE, &msg);
if (tmsgPutToQueue(&pTq->pVnode->msgCb, QUERY_QUEUE, &msg) != 0){
tqError("vgId:%d tmsgPutToQueue failed, consumer:0x%" PRIx64, vgId, pHandle->consumerId);
}
taosMemoryFree(pHandle->msg);
pHandle->msg = NULL;
}
@ -643,7 +645,6 @@ int32_t tqProcessSubscribeReq(STQ* pTq, int64_t sversion, char* msg, int32_t msg
tDecoderInit(&dc, (uint8_t*)msg, msgLen);
ret = tDecodeSMqRebVgReq(&dc, &req);
// decode req
if (ret < 0) {
goto end;
}
@ -653,7 +654,10 @@ int32_t tqProcessSubscribeReq(STQ* pTq, int64_t sversion, char* msg, int32_t msg
taosRLockLatch(&pTq->lock);
STqHandle* pHandle = NULL;
(void)tqMetaGetHandle(pTq, req.subKey, &pHandle); // ignore return code
int32_t code = tqMetaGetHandle(pTq, req.subKey, &pHandle);
if (code != 0){
tqInfo("vgId:%d, tq process sub req:%s, no such handle, create new one", pTq->pVnode->config.vgId, req.subKey);
}
taosRUnLockLatch(&pTq->lock);
if (pHandle == NULL) {
if (req.oldConsumerId != -1) {
@ -662,6 +666,7 @@ int32_t tqProcessSubscribeReq(STQ* pTq, int64_t sversion, char* msg, int32_t msg
}
if (req.newConsumerId == -1) {
tqError("vgId:%d, tq invalid rebalance request, new consumerId %" PRId64 "", req.vgId, req.newConsumerId);
ret = TSDB_CODE_INVALID_PARA;
goto end;
}
STqHandle handle = {0};

46
source/dnode/vnode/src/tq/tqMeta.c
View File

@ -390,8 +390,8 @@ END:
tdbFree(pKey);
tdbFree(pVal);
int32_t ret = tdbTbcClose(pCur);
if (code == 0 && ret != 0) {
code = ret;
if (ret != 0) {
tqError("failed to close tbc, ret:%d", ret);
}
return code;
}
@ -464,12 +464,10 @@ static int32_t tqMetaRestoreCheckInfo(STQ* pTq) {
END:
tdbFree(pKey);
tdbFree(pVal);
int32_t ret = tdbTbcClose(pCur);
if (code == 0) {
code = ret;
if (ret != 0) {
tqError("failed to close tbc, ret:%d", ret);
}
tDeleteSTqCheckInfo(&info);
return code;
}
@ -534,48 +532,44 @@ END:
taosMemoryFree(offset);
taosMemoryFree(offsetNew);
// return 0 always, so ignore
int32_t ret = tdbTbClose(pExecStore);
if (ret != 0) {
tqError("vgId:%d failed to close stream exec store, code:%s", pTq->pStreamMeta->vgId, tstrerror(ret));
tqError("failed to close tb, ret:%d", ret);
}
ret = tdbTbClose(pCheckStore);
if (ret != 0) {
tqError("vgId:%d failed to close stream check store, code:%s", pTq->pStreamMeta->vgId, tstrerror(ret));
tqError("failed to close tb, ret:%d", ret);
}
ret = tdbClose(pMetaDB);
if (ret != 0) {
tqError("vgId:%d failed to close stream meta db store, code:%s", pTq->pStreamMeta->vgId, tstrerror(ret));
tqError("failed to close tdb, ret:%d", ret);
}
return code;
}
void tqMetaClose(STQ* pTq) {
int32_t code = 0;
int32_t ret = 0;
if (pTq->pExecStore) {
code = tdbTbClose(pTq->pExecStore);
if (code) {
tqError("vgId:%d failed to close tq exec store, code:%s", pTq->pStreamMeta->vgId, tstrerror(code));
ret = tdbTbClose(pTq->pExecStore);
if (ret != 0) {
tqError("failed to close tb, ret:%d", ret);
}
}
if (pTq->pCheckStore) {
code = tdbTbClose(pTq->pCheckStore);
if (code) {
tqError("vgId:%d failed to close tq check store, code:%s", pTq->pStreamMeta->vgId, tstrerror(code));
ret = tdbTbClose(pTq->pCheckStore);
if (ret != 0) {
tqError("failed to close tb, ret:%d", ret);
}
}
if (pTq->pOffsetStore) {
code = tdbTbClose(pTq->pOffsetStore);
if (code) {
tqError("vgId:%d failed to close tq offset store, code:%s", pTq->pStreamMeta->vgId, tstrerror(code));
ret = tdbTbClose(pTq->pOffsetStore);
if (ret != 0) {
tqError("failed to close tb, ret:%d", ret);
}
}
code = tdbClose(pTq->pMetaDB);
if (code) {
tqError("vgId:%d failed to close tq meta db store, code:%s", pTq->pStreamMeta->vgId, tstrerror(code));
ret = tdbClose(pTq->pMetaDB);
if (ret != 0) {
tqError("failed to close tdb, ret:%d", ret);
}
}

5
source/dnode/vnode/src/tq/tqSnapshot.c
View File

@ -77,7 +77,10 @@ _err:
}
void tqSnapReaderClose(STqSnapReader** ppReader) {
(void)tdbTbcClose((*ppReader)->pCur);
int32_t ret = tdbTbcClose((*ppReader)->pCur);
if (ret != 0){
tqError("vgId:%d, vnode snapshot tq reader close failed since %s", TD_VID((*ppReader)->pTq->pVnode), tstrerror(ret));
}
taosMemoryFree(*ppReader);
*ppReader = NULL;
}

4
source/dnode/vnode/src/tq/tqStreamTask.c
View File

@ -112,8 +112,8 @@ int32_t tqScanWalInFuture(STQ* pTq, int32_t numOfTasks, int32_t idleDuration) {
if (pMeta->scanInfo.scanTimer == NULL) {
pMeta->scanInfo.scanTimer = taosTmrStart(doStartScanWal, idleDuration, pParam, pTimer);
} else {
code = taosTmrReset(doStartScanWal, idleDuration, pParam, pTimer, &pMeta->scanInfo.scanTimer);
if (code) {
bool ret = taosTmrReset(doStartScanWal, idleDuration, pParam, pTimer, &pMeta->scanInfo.scanTimer);
if (!ret) {
tqError("vgId:%d failed to start scan wal in:%dms", vgId, idleDuration);
}
}

2
source/dnode/vnode/src/tsdb/tsdbCacheRead.c
View File

@ -625,7 +625,7 @@ int32_t tsdbRetrieveCacheRows(void* pReader, SSDataBlock* pResBlock, const int32
}
double cost = (taosGetTimestampUs() - st) / 1000.0;
if (cost > tsCacheLazyLoadThreshold) {
pr->lastTs = totalLastTs;
// pr->lastTs = totalLastTs;
}
}
}

2
source/libs/catalog/test/catalogTests.cpp
View File

@ -2889,7 +2889,7 @@ TEST(apiTest, catalogChkAuth_test) {
SUserAuthInfo authInfo = {0};
SUserAuthRes authRes = {0};
TAOS_STRCPY(authInfo.user, ctgTestUsername);
(void)toName(1, ctgTestDbname, ctgTestSTablename, &authInfo.tbName);
toName(1, ctgTestDbname, ctgTestSTablename, &authInfo.tbName);
authInfo.type = AUTH_TYPE_READ;
bool exists = false;
code = catalogChkAuthFromCache(pCtg, &authInfo, &authRes, &exists);

2
source/libs/executor/src/mergeoperator.c
View File

@ -229,7 +229,7 @@ int32_t doSortMerge(SOperatorInfo* pOperator, SSDataBlock** pResBlock) {
resetLimitInfoForNextGroup(&pInfo->limitInfo);
}
(void)applyLimitOffset(&pInfo->limitInfo, p, pTaskInfo);
bool limitReached = applyLimitOffset(&pInfo->limitInfo, p, pTaskInfo);
if (p->info.rows > 0) {
break;

11
source/libs/executor/src/scanoperator.c
View File

@ -667,8 +667,8 @@ int32_t addTagPseudoColumnData(SReadHandle* pHandle, const SExprInfo* pExpr, int
val = *pVal;
} else {
pCache->cacheHit += 1;
STableCachedVal* pVal = taosLRUCacheValue(pCache->pTableMetaEntryCache, h);
val = *pVal;
STableCachedVal* pValTmp = taosLRUCacheValue(pCache->pTableMetaEntryCache, h);
val = *pValTmp;
bool bRes = taosLRUCacheRelease(pCache->pTableMetaEntryCache, h, false);
qTrace("release LRU cache, res %d", bRes);
@ -720,12 +720,7 @@ int32_t addTagPseudoColumnData(SReadHandle* pHandle, const SExprInfo* pExpr, int
if (IS_VAR_DATA_TYPE(((const STagVal*)p)->type)) {
taosMemoryFree(data);
}
if (code) {
if (freeReader) {
pHandle->api.metaReaderFn.clearReader(&mr);
}
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
} else { // todo opt for json tag
for (int32_t i = 0; i < pBlock->info.rows; ++i) {
code = colDataSetVal(pColInfoData, i, data, false);

41
source/libs/executor/src/streamtimewindowoperator.c
View File

@ -1991,10 +1991,12 @@ int32_t createStreamFinalIntervalOperatorInfo(SOperatorInfo* downstream, SPhysiN
pInfo->pUpdatedMap = NULL;
pInfo->stateStore = pTaskInfo->storageAPI.stateStore;
int32_t funResSize = getMaxFunResSize(&pOperator->exprSupp, numOfCols);
pInfo->pState->pFileState = pAPI->stateStore.streamFileStateInit(
tsStreamBufferSize, sizeof(SWinKey), pInfo->aggSup.resultRowSize, funResSize, compareTs, pInfo->pState,
pInfo->twAggSup.deleteMark, GET_TASKID(pTaskInfo), pHandle->checkpointId, STREAM_STATE_BUFF_HASH);
QUERY_CHECK_NULL(pInfo->pState->pFileState, code, lino, _error, terrno);
pInfo->pState->pFileState = NULL;
code =
pAPI->stateStore.streamFileStateInit(tsStreamBufferSize, sizeof(SWinKey), pInfo->aggSup.resultRowSize, funResSize,
compareTs, pInfo->pState, pInfo->twAggSup.deleteMark, GET_TASKID(pTaskInfo),
pHandle->checkpointId, STREAM_STATE_BUFF_HASH, &pInfo->pState->pFileState);
QUERY_CHECK_CODE(code, lino, _error);
pInfo->dataVersion = 0;
pInfo->recvGetAll = false;
@ -2176,39 +2178,33 @@ int32_t initStreamAggSupporter(SStreamAggSupporter* pSup, SExprSupp* pExpSup, in
pSup->resultRowSize = keySize + getResultRowSize(pExpSup->pCtx, numOfOutput);
int32_t lino = 0;
int32_t code = createSpecialDataBlock(STREAM_CLEAR, &pSup->pScanBlock);
if (code) {
return code;
}
QUERY_CHECK_CODE(code, lino, _end);
pSup->gap = gap;
pSup->stateKeySize = keySize;
pSup->stateKeyType = keyType;
pSup->pDummyCtx = (SqlFunctionCtx*)taosMemoryCalloc(numOfOutput, sizeof(SqlFunctionCtx));
if (pSup->pDummyCtx == NULL) {
return terrno;
}
QUERY_CHECK_NULL(pSup->pDummyCtx, code, lino, _end, terrno);
pSup->stateStore = *pStore;
pSup->pSessionAPI = pApi;
initDummyFunction(pSup->pDummyCtx, pExpSup->pCtx, numOfOutput);
pSup->pState = taosMemoryCalloc(1, sizeof(SStreamState));
if (!pSup->pState) {
return terrno;
}
QUERY_CHECK_NULL(pSup->pState, code, lino, _end, terrno);
*(pSup->pState) = *pState;
pSup->stateStore.streamStateSetNumber(pSup->pState, -1, tsIndex);
int32_t funResSize = getMaxFunResSize(pExpSup, numOfOutput);
pSup->pState->pFileState = pSup->stateStore.streamFileStateInit(
pSup->pState->pFileState = NULL;
code = pSup->stateStore.streamFileStateInit(
tsStreamBufferSize, sizeof(SSessionKey), pSup->resultRowSize, funResSize, sesionTs, pSup->pState,
pTwAggSup->deleteMark, taskIdStr, pHandle->checkpointId, STREAM_STATE_BUFF_SORT);
QUERY_CHECK_NULL(pSup->pState->pFileState, code, lino, _end, terrno);
pTwAggSup->deleteMark, taskIdStr, pHandle->checkpointId, STREAM_STATE_BUFF_SORT, &pSup->pState->pFileState);
QUERY_CHECK_CODE(code, lino, _end);
_hash_fn_t hashFn = taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY);
pSup->pResultRows = tSimpleHashInit(32, hashFn);
if (!pSup->pResultRows) {
return terrno;
}
QUERY_CHECK_NULL(pSup->pResultRows, code, lino, _end, terrno);
for (int32_t i = 0; i < numOfOutput; ++i) {
pExpSup->pCtx[i].saveHandle.pState = pSup->pState;
@ -5348,10 +5344,11 @@ int32_t createStreamIntervalOperatorInfo(SOperatorInfo* downstream, SPhysiNode*
int32_t funResSize = getMaxFunResSize(pSup, numOfCols);
pInfo->stateStore = pTaskInfo->storageAPI.stateStore;
pInfo->pState->pFileState = pTaskInfo->storageAPI.stateStore.streamFileStateInit(
pInfo->pState->pFileState = NULL;
code = pTaskInfo->storageAPI.stateStore.streamFileStateInit(
tsStreamBufferSize, sizeof(SWinKey), pInfo->aggSup.resultRowSize, funResSize, compareTs, pInfo->pState,
pInfo->twAggSup.deleteMark, GET_TASKID(pTaskInfo), pHandle->checkpointId, STREAM_STATE_BUFF_HASH);
QUERY_CHECK_NULL(pInfo->pState->pFileState, code, lino, _error, terrno);
pInfo->twAggSup.deleteMark, GET_TASKID(pTaskInfo), pHandle->checkpointId, STREAM_STATE_BUFF_HASH, &pInfo->pState->pFileState);
QUERY_CHECK_CODE(code, lino, _error);
setOperatorInfo(pOperator, "StreamIntervalOperator", QUERY_NODE_PHYSICAL_PLAN_STREAM_INTERVAL, true, OP_NOT_OPENED,
pInfo, pTaskInfo);

27
source/libs/executor/src/tlinearhash.c
View File

@ -150,10 +150,11 @@ static void doRemoveFromBucket(SFilePage* pPage, SLHashNode* pNode, SLHashBucket
pBucket->size -= 1;
}
static void doTrimBucketPages(SLHashObj* pHashObj, SLHashBucket* pBucket) {
static int32_t doTrimBucketPages(SLHashObj* pHashObj, SLHashBucket* pBucket) {
int32_t code = 0;
size_t numOfPages = taosArrayGetSize(pBucket->pPageIdList);
if (numOfPages <= 1) {
return;
return code;
}
int32_t* firstPage = taosArrayGet(pBucket->pPageIdList, 0);
@ -164,11 +165,14 @@ static void doTrimBucketPages(SLHashObj* pHashObj, SLHashBucket* pBucket) {
if (pLast->num <= sizeof(SFilePage)) {
// this is empty
// TODO check ret
(void)dBufSetBufPageRecycled(pHashObj->pBuf, pLast);
code = dBufSetBufPageRecycled(pHashObj->pBuf, pLast);
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed to recycle buf page since %s", __func__, tstrerror(code));
return code;
}
releaseBufPage(pHashObj->pBuf, pFirst);
taosArrayRemove(pBucket->pPageIdList, numOfPages - 1);
return;
return code;
}
char* pStart = pLast->data;
@ -191,8 +195,11 @@ static void doTrimBucketPages(SLHashObj* pHashObj, SLHashBucket* pBucket) {
pStart += nodeSize;
if (pLast->num <= sizeof(SFilePage)) {
// this is empty
// TODO check ret
(void)dBufSetBufPageRecycled(pHashObj->pBuf, pLast);
code = dBufSetBufPageRecycled(pHashObj->pBuf, pLast);
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed to recycle buf page since %s", __func__, tstrerror(code));
return code;
}
releaseBufPage(pHashObj->pBuf, pFirst);
taosArrayRemove(pBucket->pPageIdList, numOfPages - 1);
break;
@ -210,6 +217,7 @@ static void doTrimBucketPages(SLHashObj* pHashObj, SLHashBucket* pBucket) {
break;
}
}
return code;
}
static int32_t doAddNewBucket(SLHashObj* pHashObj) {
@ -403,7 +411,10 @@ int32_t tHashPut(SLHashObj* pHashObj, const void* key, size_t keyLen, void* data
releaseBufPage(pHashObj->pBuf, p);
}
doTrimBucketPages(pHashObj, pBucket);
code = doTrimBucketPages(pHashObj, pBucket);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
}
return code;

3
source/libs/executor/src/tsort.c
View File

@ -1308,8 +1308,7 @@ static int32_t getRowBufFromExtMemFile(SSortHandle* pHandle, int32_t regionId, i
return terrno;
}
// todo
(void)taosSeekCFile(pMemFile->pTdFile, pRegion->fileOffset, SEEK_SET);
TAOS_CHECK_RETURN(taosSeekCFile(pMemFile->pTdFile, pRegion->fileOffset, SEEK_SET));
int32_t readBytes = TMIN(pMemFile->blockSize, pRegion->regionSize);
int32_t ret = taosReadFromCFile(pRegion->buf, readBytes, 1, pMemFile->pTdFile);

5
source/libs/function/src/tudf.c
View File

@ -127,7 +127,10 @@ static int32_t udfSpawnUdfd(SUdfdData *pData) {
snprintf(dnodeIdEnvItem, 32, "%s=%d", "DNODE_ID", pData->dnodeId);
float numCpuCores = 4;
taosGetCpuCores(&numCpuCores, false);
int32_t code = taosGetCpuCores(&numCpuCores, false);
if(code != 0) {
fnError("failed to get cpu cores, code:%d", code);
}
numCpuCores = TMAX(numCpuCores, 2);
snprintf(thrdPoolSizeEnvItem, 32, "%s=%d", "UV_THREADPOOL_SIZE", (int)numCpuCores * 2);

3
source/libs/parser/src/parAuthenticator.c
View File

@ -169,7 +169,8 @@ static EDealRes authSelectImpl(SNode* pNode, void* pContext) {
SNode* pTagCond = NULL;
STableNode* pTable = (STableNode*)pNode;
#ifdef TD_ENTERPRISE
SName name;
SName name = {0};
toName(pAuthCxt->pParseCxt->acctId, pTable->dbName, pTable->tableName, &name);
STableMeta* pTableMeta = NULL;
toName(pAuthCxt->pParseCxt->acctId, pTable->dbName, pTable->tableName, &name);
int32_t code = getTargetMetaImpl(

101
source/libs/parser/src/parTranslater.c
View File

@ -525,7 +525,7 @@ static int32_t getTargetMeta(STranslateContext* pCxt, const SName* pName, STable
}
static int32_t getTableMeta(STranslateContext* pCxt, const char* pDbName, const char* pTableName, STableMeta** pMeta) {
SName name;
SName name = {0};
toName(pCxt->pParseCxt->acctId, pDbName, pTableName, &name);
return getTargetMeta(pCxt, &name, pMeta, false);
}
@ -557,7 +557,7 @@ static int32_t getTableCfg(STranslateContext* pCxt, const SName* pName, STableCf
static int32_t refreshGetTableMeta(STranslateContext* pCxt, const char* pDbName, const char* pTableName,
STableMeta** pMeta) {
SParseContext* pParCxt = pCxt->pParseCxt;
SName name;
SName name = {0};
toName(pCxt->pParseCxt->acctId, pDbName, pTableName, &name);
int32_t code = TSDB_CODE_SUCCESS;
if (pParCxt->async) {
@ -635,7 +635,7 @@ static int32_t getTableHashVgroupImpl(STranslateContext* pCxt, const SName* pNam
static int32_t getTableHashVgroup(STranslateContext* pCxt, const char* pDbName, const char* pTableName,
SVgroupInfo* pInfo) {
SName name;
SName name = {0};
toName(pCxt->pParseCxt->acctId, pDbName, pTableName, &name);
return getTableHashVgroupImpl(pCxt, &name, pInfo);
}
@ -4686,9 +4686,9 @@ int32_t translateTable(STranslateContext* pCxt, SNode** pTable, SNode* pJoinPare
pRealTable->ratio = (NULL != pCxt->pExplainOpt ? pCxt->pExplainOpt->ratio : 1.0);
// The SRealTableNode created through ROLLUP already has STableMeta.
if (NULL == pRealTable->pMeta) {
SName name;
SName name = {0};
toName(pCxt->pParseCxt->acctId, pRealTable->table.dbName, pRealTable->table.tableName, &name);
code = getTargetMeta( pCxt, &name, &(pRealTable->pMeta), true);
code = getTargetMeta(pCxt, &name, &(pRealTable->pMeta), true);
if (TSDB_CODE_SUCCESS != code) {
(void)generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_GET_META_ERROR, tstrerror(code));
return code;
@ -5994,7 +5994,7 @@ static int32_t isOperatorEqTbnameCond(STranslateContext* pCxt, SOperatorNode* pO
*pRet = false;
return TSDB_CODE_SUCCESS;
}
SFunctionNode* pTbnameFunc = NULL;
SValueNode* pValueNode = NULL;
if (nodeType(pOperator->pLeft) == QUERY_NODE_FUNCTION &&
@ -6238,7 +6238,7 @@ static void findVgroupsFromEqualTbname(STranslateContext* pCxt, SArray* aTbnames
}
for (int j = 0; j < nTbls; ++j) {
SName snameTb;
SName snameTb = {0};
char* tbName = taosArrayGetP(aTbnames, j);
toName(pCxt->pParseCxt->acctId, dbName, tbName, &snameTb);
SVgroupInfo vgInfo = {0};
@ -6265,7 +6265,7 @@ static void findVgroupsFromEqualTbname(STranslateContext* pCxt, SArray* aTbnames
}
static int32_t replaceToChildTableQuery(STranslateContext* pCxt, SEqCondTbNameTableInfo* pInfo) {
SName snameTb;
SName snameTb = {0};
int32_t code = 0;
SRealTableNode* pRealTable = pInfo->pRealTable;
char* tbName = taosArrayGetP(pInfo->aTbnames, 0);
@ -6276,7 +6276,7 @@ static int32_t replaceToChildTableQuery(STranslateContext* pCxt, SEqCondTbNameTa
if (NULL == pMeta || TSDB_CHILD_TABLE != pMeta->tableType || pMeta->suid != pRealTable->pMeta->suid) {
goto _return;
}
pRealTable->pMeta->uid = pMeta->uid;
pRealTable->pMeta->vgId = pMeta->vgId;
pRealTable->pMeta->tableType = pMeta->tableType;
@ -6387,11 +6387,11 @@ static int32_t setEqualTbnameTableVgroups(STranslateContext* pCxt, SSelectStmt*
}
qDebug("before ctbname optimize, code:%d, aTableNum:%d, nTbls:%d, stableQuery:%d", code, aTableNum, nTbls, stableQuery);
if (TSDB_CODE_SUCCESS == code && 1 == aTableNum && 1 == nTbls && stableQuery && NULL == pInfo->pRealTable->pTsmas) {
code = replaceToChildTableQuery(pCxt, pInfo);
}
return code;
}
@ -6798,7 +6798,7 @@ static int32_t translateSelect(STranslateContext* pCxt, SSelectStmt* pSelect) {
if (pCxt->pParseCxt && pCxt->pParseCxt->setQueryFp) {
(*pCxt->pParseCxt->setQueryFp)(pCxt->pParseCxt->requestRid);
}
if (NULL == pSelect->pFromTable) {
return translateSelectWithoutFrom(pCxt, pSelect);
} else {
@ -8798,7 +8798,7 @@ static int32_t buildRollupFuncs(SNodeList* pFuncs, SArray** pArray) {
static int32_t buildCreateStbReq(STranslateContext* pCxt, SCreateTableStmt* pStmt, SMCreateStbReq* pReq) {
int32_t code = TSDB_CODE_SUCCESS;
SName tableName;
SName tableName = {0};
pReq->igExists = pStmt->ignoreExists;
pReq->delay1 = pStmt->pOptions->maxDelay1;
pReq->delay2 = pStmt->pOptions->maxDelay2;
@ -8873,20 +8873,20 @@ static int32_t doTranslateDropSuperTable(STranslateContext* pCxt, const SName* p
static int32_t translateDropTable(STranslateContext* pCxt, SDropTableStmt* pStmt) {
SDropTableClause* pClause = (SDropTableClause*)nodesListGetNode(pStmt->pTables, 0);
SName tableName;
SName tableName = {0};
if (pStmt->withTsma) return TSDB_CODE_SUCCESS;
toName(pCxt->pParseCxt->acctId, pClause->dbName, pClause->tableName, &tableName);
return doTranslateDropSuperTable( pCxt, &tableName, pClause->ignoreNotExists);
return doTranslateDropSuperTable(pCxt, &tableName, pClause->ignoreNotExists);
}
static int32_t translateDropSuperTable(STranslateContext* pCxt, SDropSuperTableStmt* pStmt) {
SName tableName;
SName tableName = {0};
toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, &tableName);
return doTranslateDropSuperTable(pCxt, &tableName, pStmt->ignoreNotExists);
}
static int32_t buildAlterSuperTableReq(STranslateContext* pCxt, SAlterTableStmt* pStmt, SMAlterStbReq* pAlterReq) {
SName tableName;
SName tableName = {0};
toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, &tableName);
int32_t code = tNameExtractFullName(&tableName, pAlterReq->name);
if (TSDB_CODE_SUCCESS != code) return code;
@ -9403,7 +9403,7 @@ static int32_t getSmaIndexAst(STranslateContext* pCxt, SCreateIndexStmt* pStmt,
}
static int32_t buildCreateSmaReq(STranslateContext* pCxt, SCreateIndexStmt* pStmt, SMCreateSmaReq* pReq) {
SName name;
SName name = {0};
toName(pCxt->pParseCxt->acctId, pStmt->indexDbName, pStmt->indexName, &name);
int32_t code = tNameExtractFullName(&name, pReq->name);
if (TSDB_CODE_SUCCESS == code) {
@ -9556,7 +9556,7 @@ static int32_t buildCreateFullTextReq(STranslateContext* pCxt, SCreateIndexStmt*
}
static int32_t buildCreateTagIndexReq(STranslateContext* pCxt, SCreateIndexStmt* pStmt, SCreateTagIndexReq* pReq) {
SName name;
SName name = {0};
toName(pCxt->pParseCxt->acctId, pStmt->indexDbName, pStmt->indexName, &name);
int32_t code = tNameExtractFullName(&name, pReq->idxName);
if (TSDB_CODE_SUCCESS == code) {
@ -9596,7 +9596,7 @@ static int32_t translateCreateFullTextIndex(STranslateContext* pCxt, SCreateInde
static int32_t translateCreateNormalIndex(STranslateContext* pCxt, SCreateIndexStmt* pStmt) {
int32_t code = 0;
SName name;
SName name = {0};
STableMeta* pMeta = NULL;
toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, &name);
code = getTargetMeta(pCxt, &name, &pMeta, false);
@ -9640,7 +9640,7 @@ static int32_t translateCreateIndex(STranslateContext* pCxt, SCreateIndexStmt* p
static int32_t translateDropIndex(STranslateContext* pCxt, SDropIndexStmt* pStmt) {
SMDropSmaReq dropSmaReq = {0};
SName name;
SName name = {0};
toName(pCxt->pParseCxt->acctId, pStmt->indexDbName, pStmt->indexName, &name);
int32_t code = tNameExtractFullName(&name, dropSmaReq.name);
if (TSDB_CODE_SUCCESS != code) return code;
@ -9714,7 +9714,7 @@ static int32_t buildCreateTopicReq(STranslateContext* pCxt, SCreateTopicStmt* pS
}
int32_t code = TSDB_CODE_SUCCESS;
SName name;
SName name = {0};
if ('\0' != pStmt->subSTbName[0]) {
pReq->subType = TOPIC_SUB_TYPE__TABLE;
toName(pCxt->pParseCxt->acctId, pStmt->subDbName, pStmt->subSTbName, &name);
@ -9842,7 +9842,7 @@ static int32_t buildQueryForTableTopic(STranslateContext* pCxt, SCreateTopicStmt
.requestId = pParCxt->requestId,
.requestObjRefId = pParCxt->requestRid,
.mgmtEps = pParCxt->mgmtEpSet};
SName name;
SName name = {0};
STableMeta* pMeta = NULL;
toName(pParCxt->acctId, pStmt->subDbName, pStmt->subSTbName, &name);
int32_t code = getTargetMeta(pCxt, &name, &pMeta, false);
@ -9966,7 +9966,7 @@ static int32_t translateDescribe(STranslateContext* pCxt, SDescribeStmt* pStmt)
#ifdef TD_ENTERPRISE
if (TSDB_CODE_PAR_TABLE_NOT_EXIST == code) {
int32_t origCode = code;
SName name;
SName name = {0};
toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, &name);
SViewMeta* pMeta = NULL;
code = getViewMetaFromMetaCache(pCxt, &name, &pMeta);
@ -10084,7 +10084,7 @@ static int32_t checkCreateStream(STranslateContext* pCxt, SCreateStreamStmt* pSt
#ifdef TD_ENTERPRISE
SRealTableNode* pRealTable = (SRealTableNode*)((SSelectStmt*)pStmt->pQuery)->pFromTable;
SName name;
SName name = {0};
STableMeta* pMeta = NULL;
int8_t tableType = 0;
toName(pCxt->pParseCxt->acctId, pRealTable->table.dbName, pRealTable->table.tableName, &name);
@ -11325,7 +11325,7 @@ static int32_t translateCreateView(STranslateContext* pCxt, SCreateViewStmt* pSt
#endif
SParseSqlRes res = {.resType = PARSE_SQL_RES_SCHEMA};
SName name;
SName name = {0};
char dbFName[TSDB_DB_FNAME_LEN];
toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->viewName, &name);
(void)tNameGetFullDbName(&name, dbFName);
@ -11367,7 +11367,7 @@ static int32_t translateDropView(STranslateContext* pCxt, SDropViewStmt* pStmt)
#endif
SCMDropViewReq dropReq = {0};
SName name;
SName name = {0};
int32_t code = tNameSetDbName(&name, pCxt->pParseCxt->acctId, pStmt->dbName, strlen(pStmt->dbName));
if (TSDB_CODE_SUCCESS == code) {
(void)tNameGetFullDbName(&name, dropReq.dbFName);
@ -11488,7 +11488,7 @@ static int32_t translateGrantTagCond(STranslateContext* pCxt, SGrantStmt* pStmt,
int32_t code = createRealTableForGrantTable(pStmt, &pTable);
if (TSDB_CODE_SUCCESS == code) {
SName name;
SName name = {0};
toName(pCxt->pParseCxt->acctId, pTable->table.dbName, pTable->table.tableName, &name);
code = getTargetMeta(pCxt, &name, &(pTable->pMeta), false);
if (code) {
@ -11530,7 +11530,7 @@ static int32_t translateGrant(STranslateContext* pCxt, SGrantStmt* pStmt) {
req.privileges = pStmt->privileges;
#ifdef TD_ENTERPRISE
if (0 != pStmt->tabName[0]) {
SName name;
SName name = {0};
STableMeta* pTableMeta = NULL;
toName(pCxt->pParseCxt->acctId, pStmt->objName, pStmt->tabName, &name);
code = getTargetMeta(pCxt, &name, &pTableMeta, true);
@ -11566,7 +11566,7 @@ static int32_t translateRevoke(STranslateContext* pCxt, SRevokeStmt* pStmt) {
#ifdef TD_ENTERPRISE
if (0 != pStmt->tabName[0]) {
SName name;
SName name = {0};
STableMeta* pTableMeta = NULL;
toName(pCxt->pParseCxt->acctId, pStmt->objName, pStmt->tabName, &name);
code = getTargetMeta(pCxt, &name, &pTableMeta, true);
@ -11682,7 +11682,7 @@ static int32_t translateShowCreateTable(STranslateContext* pCxt, SShowCreateTabl
}
int32_t code = getDBCfg(pCxt, pStmt->dbName, (SDbCfgInfo*)pStmt->pDbCfg);
if (TSDB_CODE_SUCCESS == code) {
SName name;
SName name = {0};
toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, &name);
code = getTableCfg(pCxt, &name, (STableCfg**)&pStmt->pTableCfg);
}
@ -11693,7 +11693,7 @@ static int32_t translateShowCreateView(STranslateContext* pCxt, SShowCreateViewS
#ifndef TD_ENTERPRISE
return TSDB_CODE_OPS_NOT_SUPPORT;
#else
SName name;
SName name = {0};
toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->viewName, &name);
return getViewMetaFromMetaCache(pCxt, &name, (SViewMeta**)&pStmt->pViewMeta);
#endif
@ -11962,7 +11962,7 @@ static int32_t rewriteTSMAFuncs(STranslateContext* pCxt, SCreateTSMAStmt* pStmt,
static int32_t buildCreateTSMAReq(STranslateContext* pCxt, SCreateTSMAStmt* pStmt, SMCreateSmaReq* pReq,
SName* useTbName) {
SName name;
SName name = {0};
SDbCfgInfo pDbInfo = {0};
int32_t code = TSDB_CODE_SUCCESS;
toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tsmaName, &name);
@ -12181,7 +12181,7 @@ int32_t translatePostCreateTSMA(SParseContext* pParseCxt, SQuery* pQuery, SSData
static int32_t translateDropTSMA(STranslateContext* pCxt, SDropTSMAStmt* pStmt) {
int32_t code = TSDB_CODE_SUCCESS;
SMDropSmaReq dropReq = {0};
SName name;
SName name = {0};
STableTSMAInfo* pTsma = NULL;
toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tsmaName, &name);
code = tNameExtractFullName(&name, dropReq.name);
@ -13088,7 +13088,7 @@ static int32_t rewriteShowVgroups(STranslateContext* pCxt, SQuery* pQuery) {
static int32_t checkShowTags(STranslateContext* pCxt, const SShowStmt* pShow) {
int32_t code = 0;
SName name;
SName name = {0};
STableMeta* pTableMeta = NULL;
toName(pCxt->pParseCxt->acctId, ((SValueNode*)pShow->pDbName)->literal, ((SValueNode*)pShow->pTbName)->literal, &name);
code = getTargetMeta(pCxt, &name, &pTableMeta, true);
@ -13449,7 +13449,7 @@ static int32_t rewriteCreateTable(STranslateContext* pCxt, SQuery* pQuery) {
int32_t code = checkCreateTable(pCxt, pStmt, false);
SVgroupInfo info = {0};
SName name;
SName name = {0};
toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, &name);
if (TSDB_CODE_SUCCESS == code) {
code = getTableHashVgroupImpl(pCxt, &name, &info);
@ -13697,7 +13697,7 @@ static int32_t rewriteCreateSubTable(STranslateContext* pCxt, SCreateSubTableCla
code = getTableMeta(pCxt, pStmt->useDbName, pStmt->useTableName, &pSuperTableMeta);
}
if (TSDB_CODE_SUCCESS == code) {
SName name;
SName name = {0};
toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, &name);
code = collectUseTable(&name, pCxt->pTargetTables);
}
@ -13811,7 +13811,6 @@ _OUT:
}
typedef struct SParseFileContext {
SHashObj* pTbNameHash;
SArray* aTagNames;
bool tagNameFilled;
STableMeta* pStbMeta;
@ -13946,18 +13945,6 @@ static int32_t parseCsvFile(SMsgBuf* pMsgBuf, SParseContext* pParseCxt, SParseFi
code = parseOneStbRow(pMsgBuf, pParFileCxt);
if (TSDB_CODE_SUCCESS == code) {
if (taosHashGet(pParFileCxt->pTbNameHash, pParFileCxt->ctbName.tname, strlen(pParFileCxt->ctbName.tname) + 1) !=
NULL) {
taosMemoryFreeClear(pParFileCxt->pTag);
code = generateSyntaxErrMsg(pMsgBuf, TSDB_CODE_PAR_TBNAME_DUPLICATED, pParFileCxt->ctbName.tname);
break;
}
code = taosHashPut(pParFileCxt->pTbNameHash, pParFileCxt->ctbName.tname, strlen(pParFileCxt->ctbName.tname) + 1,
NULL, 0);
}
if (TSDB_CODE_SUCCESS == code) {
code = fillVgroupInfo(pParseCxt, &pParFileCxt->ctbName, &pParFileCxt->vg);
}
@ -13997,7 +13984,6 @@ static void destructParseFileContext(SParseFileContext** ppParFileCxt) {
SParseFileContext* pParFileCxt = *ppParFileCxt;
taosHashCleanup(pParFileCxt->pTbNameHash);
taosArrayDestroy(pParFileCxt->aTagNames);
taosMemoryFreeClear(pParFileCxt->pStbMeta);
taosArrayDestroy(pParFileCxt->aTagIndexs);
@ -14022,15 +14008,6 @@ static int32_t constructParseFileContext(SCreateSubTableFromFileClause* pStmt, S
pParFileCxt->ctbName.acctId = acctId;
strcpy(pParFileCxt->ctbName.dbname, pStmt->useDbName);
if (NULL == pParFileCxt->pTbNameHash) {
pParFileCxt->pTbNameHash =
taosHashInit(128, taosGetDefaultHashFunction(TSDB_DATA_TYPE_VARCHAR), false, HASH_NO_LOCK);
if (!pParFileCxt->pTbNameHash) {
code = TSDB_CODE_OUT_OF_MEMORY;
goto _ERR;
}
}
if (NULL == pParFileCxt->aTagNames) {
pParFileCxt->aTagNames = taosArrayInit(8, TSDB_COL_NAME_LEN);
if (NULL == pParFileCxt->aTagNames) {
@ -14463,7 +14440,7 @@ static int32_t rewriteDropTable(STranslateContext* pCxt, SQuery* pQuery) {
taosHashSetFreeFp(pVgroupHashmap, destroyDropTbReqBatch);
FOREACH(pNode, pStmt->pTables) {
SDropTableClause* pClause = (SDropTableClause*)pNode;
SName name;
SName name = {0};
toName(pCxt->pParseCxt->acctId, pClause->dbName, pClause->tableName, &name);
int32_t code = buildDropTableVgroupHashmap(pCxt, pClause, &name, &tableType, pVgroupHashmap);
if (TSDB_CODE_SUCCESS != code) {
@ -14722,7 +14699,7 @@ static int32_t buildRenameColReq(STranslateContext* pCxt, SAlterTableStmt* pStmt
}
if (TSDB_NORMAL_TABLE == pTableMeta->tableType) {
SArray* pTsmas = NULL;
SName tbName;
SName tbName = {0};
int32_t code = 0;
toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, &tbName);
if (pCxt->pMetaCache) code = getTableTsmasFromCache(pCxt->pMetaCache, &tbName, &pTsmas);

4
source/libs/planner/src/planLogicCreater.c
View File

@ -1312,6 +1312,10 @@ static int32_t createFillLogicNode(SLogicPlanContext* pCxt, SSelectStmt* pSelect
code = createColumnByRewriteExpr(pFill->pWStartTs, &pFill->node.pTargets);
}
if (TSDB_CODE_SUCCESS == code && NULL != pSelect->pHaving) {
code = nodesCloneNode(pSelect->pHaving, &pFill->node.pConditions);
}
if (TSDB_CODE_SUCCESS == code) {
*pLogicNode = (SLogicNode*)pFill;
} else {

39
source/libs/scalar/src/filter.c
View File

@ -2912,6 +2912,8 @@ int32_t filterRewrite(SFilterInfo *info, SFilterGroupCtx **gRes, int32_t gResNum
uint32_t uidx = 0;
uint32_t code = TSDB_CODE_SUCCESS;
SArray *group = taosArrayInit(FILTER_DEFAULT_GROUP_SIZE, sizeof(SFilterGroup));
SFilterGroup ng = {0};
if (group == NULL) {
FLT_ERR_JRET(terrno);
}
@ -2927,11 +2929,8 @@ int32_t filterRewrite(SFilterInfo *info, SFilterGroupCtx **gRes, int32_t gResNum
for (int32_t i = 0; i < gResNum; ++i) {
res = gRes[i];
optr = (res->colNum > 1) ? LOGIC_COND_TYPE_AND : LOGIC_COND_TYPE_OR;
SFilterGroup ng = {0};
for (uint32_t m = 0; m < res->colNum; ++m) {
colInfo = &res->colInfo[res->colIdx[m]];
if (FILTER_NO_MERGE_DATA_TYPE(colInfo->dataType)) {
@ -2944,28 +2943,16 @@ int32_t filterRewrite(SFilterInfo *info, SFilterGroupCtx **gRes, int32_t gResNum
for (int32_t n = 0; n < usize; ++n) {
SFilterUnit *u = (SFilterUnit *)taosArrayGetP((SArray *)colInfo->info, n);
if (NULL == u) {
code = TSDB_CODE_OUT_OF_RANGE;
break;
FLT_ERR_JRET(TSDB_CODE_OUT_OF_RANGE);
}
code = filterAddUnitFromUnit(info, &oinfo, u, &uidx);
if (TSDB_CODE_SUCCESS != code) {
break;
}
FLT_ERR_JRET(code);
code = filterAddUnitToGroup(&ng, uidx);
if (TSDB_CODE_SUCCESS != code) {
break;
}
}
if (TSDB_CODE_SUCCESS != code) {
break;
FLT_ERR_JRET(code);
}
continue;
}
if (TSDB_CODE_SUCCESS != code) {
filterFreeGroup((void*)&ng);
FLT_ERR_JRET(code);
}
if (colInfo->type != RANGE_TYPE_MR_CTX) {
fltError("filterRewrite get invalid col type : %d", colInfo->type);
@ -2973,24 +2960,28 @@ int32_t filterRewrite(SFilterInfo *info, SFilterGroupCtx **gRes, int32_t gResNum
}
code = filterAddGroupUnitFromCtx(info, &oinfo, colInfo->info, res->colIdx[m], &ng, optr, group);
if (TSDB_CODE_SUCCESS != code) {
filterFreeGroup((void*)&ng);
FLT_ERR_JRET(code);
}
FLT_ERR_JRET(code);
}
if (ng.unitNum > 0) {
if (NULL == taosArrayPush(group, &ng)) {
filterFreeGroup((void*)&ng);
FLT_ERR_JRET(TSDB_CODE_OUT_OF_MEMORY);
}
ng = (SFilterGroup){0};
}
}
FLT_ERR_JRET(filterConvertGroupFromArray(info, group));
_return:
taosArrayDestroy(group);
filterFreeInfo(&oinfo);
return 0;
_return:
filterFreeGroup((void*)&ng);
taosArrayDestroyEx(group, filterFreeGroup);
filterFreeInfo(&oinfo);
FLT_RET(code);

1
source/libs/scheduler/inc/schInt.h
View File

@ -572,6 +572,7 @@ int32_t schDelayLaunchTask(SSchJob *pJob, SSchTask *pTask);
int32_t schBuildAndSendMsg(SSchJob *job, SSchTask *task, SQueryNodeAddr *addr, int32_t msgType, void *param);
int32_t schAcquireJob(int64_t refId, SSchJob **ppJob);
int32_t schReleaseJob(int64_t refId);
int32_t schReleaseJobEx(int64_t refId, int32_t* released);
void schFreeFlowCtrl(SSchJob *pJob);
int32_t schChkJobNeedFlowCtrl(SSchJob *pJob, SSchLevel *pLevel);
int32_t schDecTaskFlowQuota(SSchJob *pJob, SSchTask *pTask);

13
source/libs/scheduler/src/schUtil.c
View File

@ -41,6 +41,15 @@ FORCE_INLINE int32_t schReleaseJob(int64_t refId) {
return taosReleaseRef(schMgmt.jobRef, refId);
}
FORCE_INLINE int32_t schReleaseJobEx(int64_t refId, int32_t* released) {
if (0 == refId) {
return TSDB_CODE_SUCCESS;
}
qDebug("sch release ex jobId:0x%" PRIx64, refId);
return taosReleaseRefEx(schMgmt.jobRef, refId, released);
}
int32_t schDumpEpSet(SEpSet *pEpSet, char** ppRes) {
*ppRes = NULL;
if (NULL == pEpSet) {
@ -196,7 +205,7 @@ void schDeregisterTaskHb(SSchJob *pJob, SSchTask *pTask) {
SCH_TASK_ELOG("fail to get the %dth condidateAddr in task, totalNum:%d", pTask->candidateIdx, (int32_t)taosArrayGetSize(pTask->candidateAddrs));
return;
}
SQueryNodeEpId epId = {0};
epId.nodeId = addr->nodeId;
@ -341,7 +350,7 @@ void schFreeRpcCtx(SRpcCtx *pCtx) {
void schGetTaskFromList(SHashObj *pTaskList, uint64_t taskId, SSchTask **pTask) {
*pTask = NULL;
int32_t s = taosHashGetSize(pTaskList);
if (s <= 0) {
return;

9
source/libs/scheduler/src/scheduler.c
View File

@ -119,7 +119,7 @@ int32_t schedulerGetTasksStatus(int64_t jobId, SArray *pSub) {
qError("failed to get task %d, total: %d", m, pLevel->taskNum);
SCH_ERR_JRET(TSDB_CODE_SCH_INTERNAL_ERROR);
}
SQuerySubDesc subDesc = {0};
subDesc.tid = pTask->taskId;
TAOS_STRCPY(subDesc.status, jobTaskStatusStr(pTask->status));
@ -179,8 +179,11 @@ void schedulerFreeJob(int64_t *jobId, int32_t errCode) {
SCH_JOB_DLOG("start to free job 0x%" PRIx64 ", code:%s", *jobId, tstrerror(errCode));
(void)schHandleJobDrop(pJob, errCode); // ignore any error
(void)schReleaseJob(*jobId); // ignore error
*jobId = 0;
int32_t released = false;
(void)schReleaseJobEx(*jobId, &released); // ignore error
if (released) {
*jobId = 0;
}
}
void schedulerDestroy(void) {

4
source/libs/stream/src/streamUpdate.c
View File

@ -345,7 +345,7 @@ bool updateInfoIsUpdated(SUpdateInfo* pInfo, uint64_t tableId, TSKEY ts, void* p
void** pMapMaxTs = taosHashGet(pInfo->pMap, &tableId, sizeof(uint64_t));
uint64_t index = ((uint64_t)tableId) % pInfo->numBuckets;
TSKEY maxTs = *(TSKEY*)taosArrayGet(pInfo->pTsBuckets, index);
if (ts < maxTs - pInfo->watermark && maxTs != INT64_MIN) {
if (maxTs != INT64_MIN && ts < maxTs - pInfo->watermark) {
// this window has been closed.
if (pInfo->pCloseWinSBF) {
code = tScalableBfPut(pInfo->pCloseWinSBF, pInfo->pKeyBuff, buffLen, &res);
@ -585,6 +585,8 @@ int32_t updateInfoDeserialize(void* buf, int32_t bufLen, SUpdateInfo* pInfo) {
int32_t sBfSize = 0;
if (tDecodeI32(&decoder, &sBfSize) < 0) return -1;
pInfo->pTsSBFs = taosArrayInit(sBfSize, sizeof(void*));
QUERY_CHECK_NULL(pInfo->pTsSBFs, code, lino, _error, terrno);
for (int32_t i = 0; i < sBfSize; i++) {
SScalableBf* pSBf = NULL;
code = tScalableBfDecode(&decoder, &pSBf);

48
source/libs/stream/src/tstreamFileState.c
View File

@ -127,9 +127,9 @@ static void streamFileStateEncode(TSKEY* pKey, void** pVal, int32_t* pLen) {
int32_t tmp = taosEncodeFixedI64(&buff, *pKey);
}
SStreamFileState* streamFileStateInit(int64_t memSize, uint32_t keySize, uint32_t rowSize, uint32_t selectRowSize,
GetTsFun fp, void* pFile, TSKEY delMark, const char* taskId, int64_t checkpointId,
int8_t type) {
int32_t streamFileStateInit(int64_t memSize, uint32_t keySize, uint32_t rowSize, uint32_t selectRowSize, GetTsFun fp,
void* pFile, TSKEY delMark, const char* taskId, int64_t checkpointId, int8_t type,
SStreamFileState** ppFileState) {
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
if (memSize <= 0) {
@ -194,10 +194,11 @@ SStreamFileState* streamFileStateInit(int64_t memSize, uint32_t keySize, uint32_
// todo(liuyao) optimize
if (type == STREAM_STATE_BUFF_HASH) {
recoverSnapshot(pFileState, checkpointId);
code = recoverSnapshot(pFileState, checkpointId);
} else {
recoverSesssion(pFileState, checkpointId);
code = recoverSesssion(pFileState, checkpointId);
}
QUERY_CHECK_CODE(code, lino, _error);
void* valBuf = NULL;
int32_t len = 0;
@ -208,14 +209,14 @@ SStreamFileState* streamFileStateInit(int64_t memSize, uint32_t keySize, uint32_
qDebug("===stream===flushMark read:%" PRId64, pFileState->flushMark);
}
taosMemoryFreeClear(valBuf);
return pFileState;
(*ppFileState) = pFileState;
_error:
if (code != TSDB_CODE_SUCCESS) {
streamFileStateDestroy(pFileState);
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
}
streamFileStateDestroy(pFileState);
return NULL;
return code;
}
void destroyRowBuffPos(SRowBuffPos* pPos) {
@ -806,8 +807,10 @@ int32_t deleteExpiredCheckPoint(SStreamFileState* pFileState, TSKEY mark) {
return code;
}
void recoverSesssion(SStreamFileState* pFileState, int64_t ckId) {
int32_t recoverSesssion(SStreamFileState* pFileState, int64_t ckId) {
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
int32_t winRes = TSDB_CODE_SUCCESS;
if (pFileState->maxTs != INT64_MIN) {
int64_t mark = (INT64_MIN + pFileState->deleteMark >= pFileState->maxTs)
? INT64_MIN
@ -818,7 +821,7 @@ void recoverSesssion(SStreamFileState* pFileState, int64_t ckId) {
SStreamStateCur* pCur = streamStateSessionSeekToLast_rocksdb(pFileState->pFileStore, INT64_MAX);
int32_t recoverNum = TMIN(MIN_NUM_OF_RECOVER_ROW_BUFF, pFileState->maxRowCount);
while (code == TSDB_CODE_SUCCESS) {
while (winRes == TSDB_CODE_SUCCESS) {
if (pFileState->curRowCount >= recoverNum) {
break;
}
@ -826,22 +829,34 @@ void recoverSesssion(SStreamFileState* pFileState, int64_t ckId) {
void* pVal = NULL;
int32_t vlen = 0;
SSessionKey key = {0};
code = streamStateSessionGetKVByCur_rocksdb(pCur, &key, &pVal, &vlen);
if (code != TSDB_CODE_SUCCESS) {
winRes = streamStateSessionGetKVByCur_rocksdb(pCur, &key, &pVal, &vlen);
if (winRes != TSDB_CODE_SUCCESS) {
break;
}
if (vlen != pFileState->rowSize) {
code = TSDB_CODE_QRY_EXECUTOR_INTERNAL_ERROR;
QUERY_CHECK_CODE(code, lino, _end);
}
SRowBuffPos* pPos = createSessionWinBuff(pFileState, &key, pVal, &vlen);
code = putSessionWinResultBuff(pFileState, pPos);
if (code != TSDB_CODE_SUCCESS) {
winRes = putSessionWinResultBuff(pFileState, pPos);
if (winRes != TSDB_CODE_SUCCESS) {
break;
}
code = streamStateSessionCurPrev_rocksdb(pCur);
winRes = streamStateSessionCurPrev_rocksdb(pCur);
}
_end:
if (code != TSDB_CODE_SUCCESS) {
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
}
streamStateFreeCur(pCur);
return code;
}
void recoverSnapshot(SStreamFileState* pFileState, int64_t ckId) {
int32_t recoverSnapshot(SStreamFileState* pFileState, int64_t ckId) {
int32_t code = TSDB_CODE_SUCCESS;
int32_t lino = 0;
int32_t winCode = TSDB_CODE_SUCCESS;
@ -896,6 +911,7 @@ _end:
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
}
streamStateFreeCur(pCur);
return code;
}
int32_t streamFileStateGetSelectRowSize(SStreamFileState* pFileState) { return pFileState->selectivityRowSize; }

2
source/libs/sync/inc/syncInt.h
View File

@ -21,7 +21,7 @@ extern "C" {
#endif
#include "sync.h"
#include "taosdef.h"
#include "tglobal.h"
#include "trpc.h"
#include "ttimer.h"

1
source/libs/sync/inc/syncPipeline.h
View File

@ -54,6 +54,7 @@ typedef struct SSyncLogBuffer {
int64_t matchIndex;
int64_t endIndex;
int64_t size;
int64_t bytes;
TdThreadMutex mutex;
TdThreadMutexAttr attr;
int64_t totalIndex;

54
source/libs/sync/src/syncPipeline.c
View File

@ -28,6 +28,8 @@
#include "syncUtil.h"
#include "syncVoteMgr.h"
static int64_t tsLogBufferMemoryUsed = 0; // total bytes of vnode log buffer
static bool syncIsMsgBlock(tmsg_t type) {
return (type == TDMT_VND_CREATE_TABLE) || (type == TDMT_VND_ALTER_TABLE) || (type == TDMT_VND_DROP_TABLE) ||
(type == TDMT_VND_UPDATE_TAG_VAL) || (type == TDMT_VND_ALTER_CONFIRM);
@ -101,6 +103,10 @@ int32_t syncLogBufferAppend(SSyncLogBuffer* pBuf, SSyncNode* pNode, SSyncRaftEnt
SSyncLogBufEntry tmp = {.pItem = pEntry, .prevLogIndex = pMatch->index, .prevLogTerm = pMatch->term};
pBuf->entries[index % pBuf->size] = tmp;
pBuf->endIndex = index + 1;
if (pNode->vgId > 1) {
pBuf->bytes += pEntry->bytes;
(void)atomic_add_fetch_64(&tsLogBufferMemoryUsed, (int64_t)pEntry->bytes);
}
(void)taosThreadMutexUnlock(&pBuf->mutex);
TAOS_CHECK_RETURN(syncLogBufferValidate(pBuf));
@ -260,6 +266,10 @@ int32_t syncLogBufferInitWithoutLock(SSyncLogBuffer* pBuf, SSyncNode* pNode) {
SSyncLogBufEntry tmp = {.pItem = pEntry, .prevLogIndex = -1, .prevLogTerm = -1};
pBuf->entries[index % pBuf->size] = tmp;
taken = true;
if (pNode->vgId > 1) {
pBuf->bytes += pEntry->bytes;
(void)atomic_add_fetch_64(&tsLogBufferMemoryUsed, (int64_t)pEntry->bytes);
}
}
if (index < toIndex) {
@ -286,6 +296,10 @@ int32_t syncLogBufferInitWithoutLock(SSyncLogBuffer* pBuf, SSyncNode* pNode) {
}
SSyncLogBufEntry tmp = {.pItem = pDummy, .prevLogIndex = commitIndex - 1, .prevLogTerm = commitTerm};
pBuf->entries[(commitIndex + pBuf->size) % pBuf->size] = tmp;
if (pNode->vgId > 1) {
pBuf->bytes += pDummy->bytes;
(void)atomic_add_fetch_64(&tsLogBufferMemoryUsed, (int64_t)pDummy->bytes);
}
if (index < toIndex) {
pBuf->entries[(index + 1) % pBuf->size].prevLogIndex = commitIndex;
@ -330,6 +344,7 @@ int32_t syncLogBufferReInit(SSyncLogBuffer* pBuf, SSyncNode* pNode) {
(void)memset(&pBuf->entries[(index + pBuf->size) % pBuf->size], 0, sizeof(pBuf->entries[0]));
}
pBuf->startIndex = pBuf->commitIndex = pBuf->matchIndex = pBuf->endIndex = 0;
pBuf->bytes = 0;
int32_t code = syncLogBufferInitWithoutLock(pBuf, pNode);
if (code < 0) {
sError("vgId:%d, failed to re-initialize sync log buffer since %s.", pNode->vgId, tstrerror(code));
@ -470,8 +485,12 @@ int32_t syncLogBufferAccept(SSyncLogBuffer* pBuf, SSyncNode* pNode, SSyncRaftEnt
goto _out;
}
SSyncLogBufEntry tmp = {.pItem = pEntry, .prevLogIndex = prevIndex, .prevLogTerm = prevTerm};
pEntry = NULL;
pBuf->entries[index % pBuf->size] = tmp;
if (pNode->vgId > 1) {
pBuf->bytes += pEntry->bytes;
(void)atomic_add_fetch_64(&tsLogBufferMemoryUsed, (int64_t)pEntry->bytes);
}
pEntry = NULL;
// update end index
pBuf->endIndex = TMAX(index + 1, pBuf->endIndex);
@ -846,14 +865,34 @@ int32_t syncLogBufferCommit(SSyncLogBuffer* pBuf, SSyncNode* pNode, int64_t comm
}
// recycle
bool isVnode = pNode->vgId > 1;
SyncIndex until = pBuf->commitIndex - TSDB_SYNC_LOG_BUFFER_RETENTION;
for (SyncIndex index = pBuf->startIndex; index < until; index++) {
SSyncRaftEntry* pEntry = pBuf->entries[(index + pBuf->size) % pBuf->size].pItem;
if (pEntry == NULL) return TSDB_CODE_SYN_INTERNAL_ERROR;
do {
if ((pBuf->startIndex >= pBuf->commitIndex) ||
!((pBuf->startIndex < until) || (isVnode && pBuf->bytes >= TSDB_SYNC_LOG_BUFFER_THRESHOLD &&
atomic_load_64(&tsLogBufferMemoryUsed) >= tsLogBufferMemoryAllowed))) {
break;
}
SSyncRaftEntry* pEntry = pBuf->entries[(pBuf->startIndex + pBuf->size) % pBuf->size].pItem;
if (pEntry == NULL) {
sError("vgId:%d, invalid log entry to recycle. index:%" PRId64 ", startIndex:%" PRId64 ", until:%" PRId64
", commitIndex:%" PRId64 ", endIndex:%" PRId64 ", term:%" PRId64,
pNode->vgId, pEntry->index, pBuf->startIndex, until, pBuf->commitIndex, pBuf->endIndex, pEntry->term);
return TSDB_CODE_SYN_INTERNAL_ERROR;
}
if (isVnode) {
pBuf->bytes -= pEntry->bytes;
(void)atomic_sub_fetch_64(&tsLogBufferMemoryUsed, (int64_t)pEntry->bytes);
}
sDebug("vgId:%d, recycle log entry. index:%" PRId64 ", startIndex:%" PRId64 ", until:%" PRId64
", commitIndex:%" PRId64 ", endIndex:%" PRId64 ", term:%" PRId64 ", entry bytes:%u, buf bytes:%" PRId64
", used:%" PRId64 ", allowed:%" PRId64,
pNode->vgId, pEntry->index, pBuf->startIndex, until, pBuf->commitIndex, pBuf->endIndex, pEntry->term,
pEntry->bytes, pBuf->bytes, atomic_load_64(&tsLogBufferMemoryUsed), tsLogBufferMemoryAllowed);
syncEntryDestroy(pEntry);
(void)memset(&pBuf->entries[(index + pBuf->size) % pBuf->size], 0, sizeof(pBuf->entries[0]));
pBuf->startIndex = index + 1;
}
(void)memset(&pBuf->entries[(pBuf->startIndex + pBuf->size) % pBuf->size], 0, sizeof(pBuf->entries[0]));
++pBuf->startIndex;
} while (true);
code = 0;
_out:
@ -1324,6 +1363,7 @@ void syncLogBufferClear(SSyncLogBuffer* pBuf) {
(void)memset(&pBuf->entries[(index + pBuf->size) % pBuf->size], 0, sizeof(pBuf->entries[0]));
}
pBuf->startIndex = pBuf->commitIndex = pBuf->matchIndex = pBuf->endIndex = 0;
pBuf->bytes = 0;
(void)taosThreadMutexUnlock(&pBuf->mutex);
}

6
source/os/src/osDir.c
View File

@ -109,8 +109,8 @@ void taosRemoveDir(const char *dirname) {
}
}
(void)taosCloseDir(&pDir);
(void)rmdir(dirname);
TAOS_UNUSED(taosCloseDir(&pDir));
TAOS_UNUSED(rmdir(dirname));
// printf("dir:%s is removed\n", dirname);
return;
@ -374,7 +374,7 @@ int32_t taosRealPath(char *dirname, char *realPath, int32_t maxlen) {
bool taosIsDir(const char *dirname) {
TdDirPtr pDir = taosOpenDir(dirname);
if (pDir != NULL) {
(void)taosCloseDir(&pDir);
TAOS_SKIP_ERROR(taosCloseDir(&pDir));
return true;
}
return false;

5
source/os/src/osEnv.c
View File

@ -50,7 +50,10 @@ int32_t osDefaultInit() {
taosSeedRand(taosSafeRand());
taosGetSystemLocale(tsLocale, tsCharset);
taosGetSystemTimezone(tsTimezoneStr, &tsTimezone);
code = taosGetSystemTimezone(tsTimezoneStr, &tsTimezone);
if(code != 0) {
return code;
}
if (strlen(tsTimezoneStr) > 0) { // ignore empty timezone
if ((code = taosSetSystemTimezone(tsTimezoneStr, tsTimezoneStr, &tsDaylight, &tsTimezone)) != TSDB_CODE_SUCCESS)
return code;

3
source/os/src/osFile.c
View File

@ -1535,8 +1535,9 @@ int taosSeekCFile(FILE *file, int64_t offset, int whence) {
int code = fseeko(file, offset, whence);
if (-1 == code) {
terrno = TAOS_SYSTEM_ERROR(errno);
code = terrno;
}
return terrno;
return code;
#endif
}

2
source/os/src/osRand.c
View File

@ -69,7 +69,7 @@ uint32_t taosSafeRand(void) {
if (len < 0) {
seed = (int)taosGetTimestampSec();
}
(void)taosCloseFile(&pFile);
TAOS_SKIP_ERROR(taosCloseFile(&pFile));
}
return (uint32_t)seed;

15
source/os/src/osSleep.c
View File

@ -37,7 +37,7 @@ void taosMsleep(int32_t ms) {
#ifdef WINDOWS
Sleep(ms);
#else
(void)usleep(ms * 1000);
TAOS_SKIP_ERROR(usleep(ms * 1000));
#endif
}
@ -49,10 +49,15 @@ void taosUsleep(int32_t us) {
interval.QuadPart = (10 * us);
timer = CreateWaitableTimer(NULL, TRUE, NULL);
SetWaitableTimer(timer, &interval, 0, NULL, NULL, 0);
WaitForSingleObject(timer, INFINITE);
CloseHandle(timer);
if (timer == NULL) {
return;
}
if (!SetWaitableTimer(timer, &interval, 0, NULL, NULL, 0)) {
return;
}
TAOS_SKIP_ERROR(WaitForSingleObject(timer, INFINITE));
TAOS_SKIP_ERROR(CloseHandle(timer));
#else
(void)usleep(us);
TAOS_SKIP_ERROR(usleep(us));
#endif
}

29
source/os/src/osSocket.c
View File

@ -120,7 +120,12 @@ int32_t taosReadFromSocket(TdSocketPtr pSocket, void *buf, int32_t len, int32_t
int32_t taosCloseSocketNoCheck1(SocketFd fd) {
#ifdef WINDOWS
return closesocket(fd);
int ret = closesocket(fd);
if (ret == SOCKET_ERROR) {
int errorCode = WSAGetLastError();
return terrno = TAOS_SYSTEM_WINSOCKET_ERROR(errorCode);
}
return 0;
#else
int32_t code = close(fd);
if (-1 == code) {
@ -770,10 +775,7 @@ bool taosValidIpAndPort(uint32_t ip, uint16_t port) {
TdSocketPtr pSocket = (TdSocketPtr)taosMemoryMalloc(sizeof(TdSocket));
if (pSocket == NULL) {
code = terrno;
(void)taosCloseSocketNoCheck1(fd);
terrno = code;
TAOS_SKIP_ERROR(taosCloseSocketNoCheck1(fd));
return false;
}
pSocket->refId = 0;
@ -782,22 +784,18 @@ bool taosValidIpAndPort(uint32_t ip, uint16_t port) {
/* set REUSEADDR option, so the portnumber can be re-used */
reuse = 1;
if (taosSetSockOpt(pSocket, SOL_SOCKET, SO_REUSEADDR, (void *)&reuse, sizeof(reuse)) < 0) {
code = terrno;
(void)taosCloseSocket(&pSocket);
terrno = code;
TAOS_SKIP_ERROR(taosCloseSocket(&pSocket));
return false;
}
/* bind socket to server address */
if (-1 == bind(pSocket->fd, (struct sockaddr *)&serverAdd, sizeof(serverAdd))) {
code = TAOS_SYSTEM_ERROR(errno);
(void)taosCloseSocket(&pSocket);
terrno = code;
terrno = TAOS_SYSTEM_ERROR(errno);
TAOS_SKIP_ERROR(taosCloseSocket(&pSocket));
return false;
}
(void)taosCloseSocket(&pSocket);
TAOS_SKIP_ERROR(taosCloseSocket(&pSocket));
return true;
}
@ -1160,9 +1158,8 @@ int32_t taosCreateSocketWithTimeout(uint32_t timeout) {
#else // Linux like systems
uint32_t conn_timeout_ms = timeout;
if (-1 == setsockopt(fd, IPPROTO_TCP, TCP_USER_TIMEOUT, (char *)&conn_timeout_ms, sizeof(conn_timeout_ms))) {
int32_t code = TAOS_SYSTEM_ERROR(errno);
(void)taosCloseSocketNoCheck1(fd);
terrno = code;
terrno = TAOS_SYSTEM_ERROR(errno);
TAOS_SKIP_ERROR(taosCloseSocketNoCheck1(fd));
return -1;
}
#endif

3
source/os/src/osString.c
View File

@ -351,7 +351,8 @@ int32_t taosUcs4ToMbs(TdUcs4 *ucs4, int32_t ucs4_max_len, char *mbs) {
int32_t code = 0;
iconv_t conv = taosAcquireConv(&idx, C2M);
if ((iconv_t)-1 == conv || (iconv_t)0 == conv) {
return TSDB_CODE_APP_ERROR;
code = TAOS_SYSTEM_ERROR(errno);;
return code;
}
size_t ucs4_input_len = ucs4_max_len;

27
source/os/src/osSysinfo.c
View File

@ -191,10 +191,11 @@ static int32_t taosGetSysCpuInfo(SysCpuInfo *cpuInfo) {
&cpuInfo->si, &cpuInfo->st, &cpuInfo->guest, &cpuInfo->guest_nice);
if (EOF == code) {
terrno = TAOS_SYSTEM_ERROR(errno);
TAOS_SKIP_ERROR(taosCloseFile(&pFile));
return terrno;
}
(void)taosCloseFile(&pFile);
TAOS_SKIP_ERROR(taosCloseFile(&pFile));
#endif
return 0;
@ -263,9 +264,9 @@ bool taosCheckSystemIsLittleEnd() {
void taosGetSystemInfo() {
#ifdef WINDOWS
taosGetCpuCores(&tsNumOfCores, false);
taosGetTotalMemory(&tsTotalMemoryKB);
taosGetCpuUsage(NULL, NULL);
TAOS_SKIP_ERROR(taosGetCpuCores(&tsNumOfCores, false));
TAOS_SKIP_ERROR(taosGetTotalMemory(&tsTotalMemoryKB));
TAOS_SKIP_ERROR(taosGetCpuUsage(NULL, NULL));
#elif defined(_TD_DARWIN_64)
long physical_pages = sysconf(_SC_PHYS_PAGES);
long page_size = sysconf(_SC_PAGESIZE);
@ -274,10 +275,10 @@ void taosGetSystemInfo() {
tsNumOfCores = sysconf(_SC_NPROCESSORS_ONLN);
#else
taosGetProcIOnfos();
(void)taosGetCpuCores(&tsNumOfCores, false);
(void)taosGetTotalMemory(&tsTotalMemoryKB);
(void)taosGetCpuUsage(NULL, NULL);
(void)taosGetCpuInstructions(&tsSSE42Supported, &tsAVXSupported, &tsAVX2Supported, &tsFMASupported, &tsAVX512Supported);
TAOS_SKIP_ERROR(taosGetCpuCores(&tsNumOfCores, false));
TAOS_SKIP_ERROR(taosGetTotalMemory(&tsTotalMemoryKB));
TAOS_SKIP_ERROR(taosGetCpuUsage(NULL, NULL));
TAOS_SKIP_ERROR(taosGetCpuInstructions(&tsSSE42Supported, &tsAVXSupported, &tsAVX2Supported, &tsFMASupported, &tsAVX512Supported));
#endif
}
@ -313,11 +314,11 @@ int32_t taosGetEmail(char *email, int32_t maxLen) {
if (taosReadFile(pFile, (void *)email, maxLen) < 0) {
int32_t code = terrno;
(void)taosCloseFile(&pFile);
TAOS_SKIP_ERROR(taosCloseFile(&pFile));
return code;
}
(void)taosCloseFile(&pFile);
TAOS_SKIP_ERROR(taosCloseFile(&pFile));
return 0;
#endif
@ -748,7 +749,7 @@ int32_t taosGetProcMemory(int64_t *usedKB) {
char tmp[10];
(void)sscanf(line, "%s %" PRId64, tmp, usedKB);
(void)taosCloseFile(&pFile);
TAOS_SKIP_ERROR(taosCloseFile(&pFile));
return 0;
#endif
@ -1045,7 +1046,7 @@ int32_t taosGetSystemUUID(char *uid, int32_t uidlen) {
return terrno;
} else {
len = taosReadFile(pFile, uid, uidlen);
(void)taosCloseFile(&pFile);
TAOS_SKIP_ERROR(taosCloseFile(&pFile));
if (len < 0) {
return len;
}
@ -1087,7 +1088,7 @@ char *taosGetCmdlineByPID(int pid) {
cmdline[n] = 0;
(void)taosCloseFile(&pFile);
TAOS_SKIP_ERROR(taosCloseFile(&pFile));
} else {
cmdline[0] = 0;
}

2
source/os/src/osTimer.c
View File

@ -80,7 +80,7 @@ void taos_block_sigalrm(void) {
static void taosDeleteTimer(void *tharg) {
timer_t *pTimer = tharg;
(void)timer_delete(*pTimer);
TAOS_SKIP_ERROR(timer_delete(*pTimer));
}
static TdThread timerThread;

102
source/os/src/osTimezone.c
View File

@ -811,7 +811,11 @@ int32_t taosSetSystemTimezone(const char *inTimezoneStr, char *outTimezoneStr, i
#elif defined(_TD_DARWIN_64)
setenv("TZ", buf, 1);
code = setenv("TZ", buf, 1);
if (-1 == code) {
terrno = TAOS_SYSTEM_ERROR(errno);
return terrno;
}
tzset();
int32_t tz = (int32_t)((-timezone * MILLISECOND_PER_SECOND) / MILLISECOND_PER_HOUR);
*tsTimezone = tz;
@ -839,13 +843,17 @@ int32_t taosSetSystemTimezone(const char *inTimezoneStr, char *outTimezoneStr, i
return code;
}
void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) {
int32_t taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) {
int32_t code = 0;
#ifdef WINDOWS
char value[100];
char keyPath[100];
DWORD bufferSize = sizeof(value);
RegGetValue(HKEY_LOCAL_MACHINE, "SYSTEM\\CurrentControlSet\\Control\\TimeZoneInformation", "TimeZoneKeyName",
LONG result = RegGetValue(HKEY_LOCAL_MACHINE, "SYSTEM\\CurrentControlSet\\Control\\TimeZoneInformation", "TimeZoneKeyName",
RRF_RT_ANY, NULL, (PVOID)&value, &bufferSize);
if (result != ERROR_SUCCESS) {
return TAOS_SYSTEM_WINAPI_ERROR(result);
}
strcpy(outTimezoneStr, "not configured");
*tsTimezone = 0;
if (bufferSize > 0) {
@ -854,7 +862,10 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) {
strcpy(outTimezoneStr, win_tz[i][1]);
bufferSize = sizeof(value);
sprintf(keyPath, "SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Time Zones\\%s", value);
RegGetValue(HKEY_LOCAL_MACHINE, keyPath, "Display", RRF_RT_ANY, NULL, (PVOID)&value, &bufferSize);
result = RegGetValue(HKEY_LOCAL_MACHINE, keyPath, "Display", RRF_RT_ANY, NULL, (PVOID)&value, &bufferSize);
if (result != ERROR_SUCCESS) {
return TAOS_SYSTEM_WINAPI_ERROR(result);
}
if (bufferSize > 0) {
// value[4] = (value[4] == '+' ? '-' : '+');
sprintf(outTimezoneStr, "%s (UTC, %c%c%c%c%c)", outTimezoneStr, value[4], value[5], value[6], value[8],
@ -865,39 +876,28 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) {
}
}
}
return 0;
#elif defined(_TD_DARWIN_64)
char buf[4096] = {0};
char *tz = NULL;
{
int n = readlink("/etc/localtime", buf, sizeof(buf));
if (n < 0) {
printf("read /etc/localtime error, reason:%s\n", strerror(errno));
return;
return TSDB_CODE_TIME_ERROR;
}
buf[n] = '\0';
char *zi = strstr(buf, "zoneinfo");
if (!zi) {
printf("parsing /etc/localtime failed\n");
return;
return TSDB_CODE_TIME_ERROR;
}
tz = zi + strlen("zoneinfo") + 1;
// for (int i = n - 1; i >= 0; --i) {
// if (buf[i] == '/') {
// if (tz) {
// tz = buf + i + 1;
// break;
// }
// tz = buf + i + 1;
// }
// }
// if (!tz || 0 == strchr(tz, '/')) {
// printf("parsing /etc/localtime failed\n");
// return;
// }
setenv("TZ", tz, 1);
code = setenv("TZ", tz, 1);
if (-1 == code) {
terrno = TAOS_SYSTEM_ERROR(errno);
return terrno;
}
tzset();
}
@ -908,7 +908,9 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) {
*/
time_t tx1 = taosGetTimestampSec();
struct tm tm1;
taosLocalTime(&tx1, &tm1, NULL);
if (taosLocalTime(&tx1, &tm1, NULL) == NULL) {
return TSDB_CODE_TIME_ERROR;
}
daylight = tm1.tm_isdst;
isdst_now = tm1.tm_isdst;
@ -920,6 +922,7 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) {
*/
snprintf(outTimezoneStr, TD_TIMEZONE_LEN, "%s (%s, %+03ld00)", tz, tm1.tm_isdst ? tzname[daylight] : tzname[0],
-timezone / 3600);
return 0;
#else
char buf[4096] = {0};
@ -927,8 +930,6 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) {
{
int n = readlink("/etc/localtime", buf, sizeof(buf)-1);
if (n < 0) {
(void)printf("read /etc/localtime error, reason:%s\n", strerror(errno));
if (taosCheckExistFile("/etc/timezone")) {
/*
* NOTE: do not remove it.
@ -937,7 +938,9 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) {
*/
time_t tx1 = taosGetTimestampSec();
struct tm tm1;
(void)taosLocalTime(&tx1, &tm1, NULL);
if(taosLocalTime(&tx1, &tm1, NULL) == NULL) {
return TSDB_CODE_TIME_ERROR;
}
/* load time zone string from /etc/timezone */
// FILE *f = fopen("/etc/timezone", "r");
errno = 0;
@ -946,12 +949,11 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) {
if (pFile != NULL) {
int len = taosReadFile(pFile, buf, 64);
if (len < 0) {
(void)taosCloseFile(&pFile);
(void)printf("read /etc/timezone error, reason:%s\n", strerror(errno));
return;
TAOS_UNUSED(taosCloseFile(&pFile));
return TSDB_CODE_TIME_ERROR;
}
(void)taosCloseFile(&pFile);
TAOS_UNUSED(taosCloseFile(&pFile));
buf[sizeof(buf) - 1] = 0;
char *lineEnd = strstr(buf, "\n");
@ -961,7 +963,11 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) {
// for CentOS system, /etc/timezone does not exist. Ignore the TZ environment variables
if (strlen(buf) > 0) {
(void)setenv("TZ", buf, 1);
code = setenv("TZ", buf, 1);
if (-1 == code) {
terrno = TAOS_SYSTEM_ERROR(errno);
return terrno;
}
}
}
// get and set default timezone
@ -986,34 +992,23 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) {
(void)snprintf(outTimezoneStr, TD_TIMEZONE_LEN, "%s (%s, %s%02d00)", buf, tzname[daylight], tz >= 0 ? "+" : "-",
abs(tz));
} else {
(void)printf("There is not /etc/timezone.\n");
return TSDB_CODE_TIME_ERROR;
}
return;
return 0;
}
buf[n] = '\0';
char *zi = strstr(buf, "zoneinfo");
if (!zi) {
(void)printf("parsing /etc/localtime failed\n");
return;
return TSDB_CODE_TIME_ERROR;
}
tz = zi + strlen("zoneinfo") + 1;
// for (int i = n - 1; i >= 0; --i) {
// if (buf[i] == '/') {
// if (tz) {
// tz = buf + i + 1;
// break;
// }
// tz = buf + i + 1;
// }
// }
// if (!tz || 0 == strchr(tz, '/')) {
// printf("parsing /etc/localtime failed");
// return;
// }
(void)setenv("TZ", tz, 1);
code = setenv("TZ", tz, 1);
if (-1 == code) {
terrno = TAOS_SYSTEM_ERROR(errno);
return terrno;
}
tzset();
}
@ -1024,7 +1019,9 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) {
*/
time_t tx1 = taosGetTimestampSec();
struct tm tm1;
(void)taosLocalTime(&tx1, &tm1, NULL);
if(taosLocalTime(&tx1, &tm1, NULL) == NULL) {
return TSDB_CODE_TIME_ERROR;
}
isdst_now = tm1.tm_isdst;
/*
@ -1035,5 +1032,6 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) {
*/
(void)snprintf(outTimezoneStr, TD_TIMEZONE_LEN, "%s (%s, %+03ld00)", tz, tm1.tm_isdst ? tzname[daylight] : tzname[0],
-timezone / 3600);
return 0;
#endif
}

2
source/util/src/terror.c
View File

@ -112,6 +112,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_FAILED_TO_CONNECT_S3, "Failed to connect to
TAOS_DEFINE_ERROR(TSDB_CODE_MSG_PREPROCESSED, "Message has been processed in preprocess")
TAOS_DEFINE_ERROR(TSDB_CODE_OUT_OF_BUFFER, "Out of buffer")
TAOS_DEFINE_ERROR(TSDB_CODE_INTERNAL_ERROR, "Internal error")
TAOS_DEFINE_ERROR(TSDB_CODE_TIME_ERROR, "Internal error in time")
//client
TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_OPERATION, "Invalid operation")
@ -806,6 +807,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_TMQ_SAME_COMMITTED_VALUE, "Same committed valu
TAOS_DEFINE_ERROR(TSDB_CODE_TMQ_REPLAY_NEED_ONE_VGROUP, "Replay need only one vgroup if subscribe super table")
TAOS_DEFINE_ERROR(TSDB_CODE_TMQ_REPLAY_NOT_SUPPORT, "Replay is disabled if subscribe db or stable")
TAOS_DEFINE_ERROR(TSDB_CODE_TMQ_NO_TABLE_QUALIFIED, "No table qualified for query")
TAOS_DEFINE_ERROR(TSDB_CODE_TMQ_NO_NEED_REBALANCE, "No need rebalance")
// stream
TAOS_DEFINE_ERROR(TSDB_CODE_STREAM_TASK_NOT_EXIST, "Stream task not exist")

5
source/util/src/tlog.c
View File

@ -215,7 +215,10 @@ int32_t taosInitSlowLog() {
int32_t taosInitLog(const char *logName, int32_t maxFiles, bool tsc) {
if (atomic_val_compare_exchange_8(&tsLogInited, 0, 1) != 0) return 0;
TAOS_CHECK_RETURN(osUpdate());
int32_t code = osUpdate();
if (code != 0) {
uError("failed to update os info, reason:%s", tstrerror(code));
}
TAOS_CHECK_RETURN(taosInitNormalLog(logName, maxFiles));
if (tsc){

11
source/util/src/tpagedbuf.c
View File

@ -316,7 +316,7 @@ static char* evictBufPage(SDiskbasedBuf* pBuf) {
}
terrno = 0;
(void)tdListPopNode(pBuf->lruList, pn);
pn = tdListPopNode(pBuf->lruList, pn);
SPageInfo* d = *(SPageInfo**)pn->data;
@ -337,7 +337,7 @@ static int32_t lruListPushFront(SList* pList, SPageInfo* pi) {
}
static void lruListMoveToFront(SList* pList, SPageInfo* pi) {
(void)tdListPopNode(pList, pi->pn);
pi->pn = tdListPopNode(pList, pi->pn);
tdListPrependNode(pList, pi->pn);
}
@ -474,8 +474,11 @@ void* getNewBufPage(SDiskbasedBuf* pBuf, int32_t* pageId) {
pBuf->totalBufSize += pBuf->pageSize;
} else {
taosMemoryFree(availablePage);
(void)taosArrayPop(pBuf->pIdList);
(void)tSimpleHashRemove(pBuf->all, pageId, sizeof(int32_t));
SPageInfo **pLast = taosArrayPop(pBuf->pIdList);
int32_t ret = tSimpleHashRemove(pBuf->all, pageId, sizeof(int32_t));
if (ret != TSDB_CODE_SUCCESS) {
uError("%s failed to clear pageId %d from buf hash-set since %s", __func__, *pageId, tstrerror(ret));
}
taosMemoryFree(pi);
terrno = code;
return NULL;

13
source/util/src/tref.c
View File

@ -56,7 +56,7 @@ static void taosLockList(int64_t *lockedBy);
static void taosUnlockList(int64_t *lockedBy);
static void taosIncRsetCount(SRefSet *pSet);
static void taosDecRsetCount(SRefSet *pSet);
static int32_t taosDecRefCount(int32_t rsetId, int64_t rid, int32_t remove);
static int32_t taosDecRefCount(int32_t rsetId, int64_t rid, int32_t remove, int32_t* isReleased);
int32_t taosOpenRef(int32_t max, RefFp fp) {
SRefNode **nodeList;
@ -182,7 +182,7 @@ int64_t taosAddRef(int32_t rsetId, void *p) {
return rid;
}
int32_t taosRemoveRef(int32_t rsetId, int64_t rid) { return taosDecRefCount(rsetId, rid, 1); }
int32_t taosRemoveRef(int32_t rsetId, int64_t rid) { return taosDecRefCount(rsetId, rid, 1, NULL); }
// if rid is 0, return the first p in hash list, otherwise, return the next after current rid
void *taosAcquireRef(int32_t rsetId, int64_t rid) {
@ -251,7 +251,8 @@ void *taosAcquireRef(int32_t rsetId, int64_t rid) {
return p;
}
int32_t taosReleaseRef(int32_t rsetId, int64_t rid) { return taosDecRefCount(rsetId, rid, 0); }
int32_t taosReleaseRef(int32_t rsetId, int64_t rid) { return taosDecRefCount(rsetId, rid, 0, NULL); }
int32_t taosReleaseRefEx(int32_t rsetId, int64_t rid, int32_t* isReleased) { return taosDecRefCount(rsetId, rid, 0, isReleased); }
// if rid is 0, return the first p in hash list, otherwise, return the next after current rid
void *taosIterateRef(int32_t rsetId, int64_t rid) {
@ -384,7 +385,7 @@ int32_t taosListRef() {
return num;
}
static int32_t taosDecRefCount(int32_t rsetId, int64_t rid, int32_t remove) {
static int32_t taosDecRefCount(int32_t rsetId, int64_t rid, int32_t remove, int32_t* isReleased) {
int32_t hash;
SRefSet *pSet;
SRefNode *pNode;
@ -458,6 +459,10 @@ static int32_t taosDecRefCount(int32_t rsetId, int64_t rid, int32_t remove) {
taosDecRsetCount(pSet);
}
if (isReleased) {
*isReleased = released;
}
return code;
}

28
source/util/src/tunit.c
View File

@ -23,14 +23,16 @@
#define UNIT_ONE_PEBIBYTE (UNIT_ONE_TEBIBYTE * UNIT_SIZE_CONVERT_FACTOR)
#define UNIT_ONE_EXBIBYTE (UNIT_ONE_PEBIBYTE * UNIT_SIZE_CONVERT_FACTOR)
static int32_t parseCfgIntWithUnit(const char* str, double* res) {
double val, temp = (double)INT64_MAX;
static int32_t parseCfgIntWithUnit(const char* str, int64_t* res) {
double val = 0, temp = (double)INT64_MAX;
char* endPtr;
bool useDouble = false;
errno = 0;
val = taosStr2Int64(str, &endPtr, 0);
int64_t int64Val = taosStr2Int64(str, &endPtr, 0);
if (*endPtr == '.' || errno == ERANGE) {
errno = 0;
val = taosStr2Double(str, &endPtr);
useDouble = true;
}
if (endPtr == str || errno == ERANGE || isnan(val)) {
return terrno = TSDB_CODE_INVALID_CFG_VALUE;
@ -67,23 +69,33 @@ static int32_t parseCfgIntWithUnit(const char* str, double* res) {
default:
return terrno = TSDB_CODE_INVALID_CFG_VALUE;
}
endPtr++;
if ((val > 0 && val > temp) || (val < 0 && val < -temp)) {
return terrno = TSDB_CODE_OUT_OF_RANGE;
}
endPtr++;
val *= factor;
int64Val *= factor;
}
while (isspace((unsigned char)*endPtr)) endPtr++;
if (*endPtr) {
return terrno = TSDB_CODE_INVALID_CFG_VALUE;
}
val = rint(val);
*res = val;
if (useDouble) {
val = rint(val);
if ((val > 0 && val >= (double)INT64_MAX) || (val < 0 && val <= (double)INT64_MIN)) {
return terrno = TSDB_CODE_OUT_OF_RANGE;
} else {
*res = (int64_t)val;
}
} else {
*res = int64Val;
}
return TSDB_CODE_SUCCESS;
}
int32_t taosStrHumanToInt64(const char* str, int64_t* out) {
double res;
int64_t res;
int32_t code = parseCfgIntWithUnit(str, &res);
if (code == TSDB_CODE_SUCCESS) *out = (int64_t)res;
return code;
@ -109,7 +121,7 @@ void taosInt64ToHumanStr(int64_t val, char* outStr) {
#endif
int32_t taosStrHumanToInt32(const char* str, int32_t* out) {
double res;
int64_t res;
int32_t code = parseCfgIntWithUnit(str, &res);
if (code == TSDB_CODE_SUCCESS) {
if (res < INT32_MIN || res > INT32_MAX) {

51
tests/army/query/accuracy/test_ts5400.py Normal file
View File

@ -0,0 +1,51 @@
import taos
import socket
from frame.log import *
from frame.cases import *
from frame.sql import *
from frame.caseBase import *
from frame import *
from frame.eos import *
from frame.server.dnodes import *
class TDTestCase(TBase):
"""Add test case to cover TS-5400
"""
updatecfgDict = {
"timezone": "UTC"
}
def init(self, conn, logSql, replicaVar=1):
host = socket.gethostname()
con = taos.connect(host=f"{host}", config=tdDnodes.getSimCfgPath(), timezone='UTC')
self.replicaVar = int(replicaVar)
tdLog.debug("start to execute %s" % __file__)
tdSql.init(con.cursor())
def prepare_data(self):
# prepare data for TS-5400
tdSql.execute("create database db_ts5400 BUFFER 512 CACHESIZE 1024 CACHEMODEL 'both' COMP 2 DURATION 10d WAL_FSYNC_PERIOD 3000 MAXROWS 4096 MINROWS 100 KEEP 365000d;")
tdSql.execute("use db_ts5400;")
#tdSql.execute("create stable st(ts TIMESTAMP ENCODE 'delta-i' COMPRESS 'lz4' LEVEL 'medium', `uk` VARCHAR(64) ENCODE 'disabled' COMPRESS 'lz4' LEVEL 'medium' PRIMARY KEY ) tags(ta int,tb int,tc int);")
tdSql.execute("create stable st(ts TIMESTAMP, `uk` VARCHAR(64)) tags(ta int,tb int,tc int);")
tdSql.execute("create table t1 using st tags(1,1,1);")
tdSql.execute("insert into t1 values ('1970-01-29 05:04:53.000','22:: ');")
def test_ts5400(self):
self.prepare_data()
tdSql.execute("use db_ts5400;")
tdSql.query("select _wstart, count(*) from st interval(1y);")
tdSql.checkRows(1)
tdSql.checkData(0, 0, '1970-01-01 00:00:00.000')
tdSql.checkData(0, 1, 1)
def run(self):
self.test_ts5400()
def stop(self):
tdSql.execute("drop database db_ts5400;")
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())

26
tests/army/query/sys/tb_perf_queries_exist_test.py Normal file
View File

@ -0,0 +1,26 @@
# -*- coding: utf-8 -*-
from frame.log import *
from frame.cases import *
from frame.sql import *
from frame.caseBase import *
from frame import *
from frame.autogen import *
'''
TS-5349: https://jira.taosdata.com:18080/browse/TS-5349
查询 performance_schema.perf_queries 后, 再查询 information_schema.perf_queries,
正常情况下在 information_schema 中不存在表 perf_queries
'''
class TDTestCase(TBase):
def run(self):
tdSql.query("select * from performance_schema.perf_queries;")
tdLog.info("Table [perf_queries] exist in schema [performance_schema]")
tdSql.error("select * from information_schema.perf_queries;")
tdLog.success(f"{__file__} successfully executed")
tdCases.addLinux(__file__, TDTestCase())
tdCases.addWindows(__file__, TDTestCase())

2
tests/parallel_test/cases.task
View File

@ -27,6 +27,7 @@
,,y,army,./pytest.sh python3 ./test.py -f cluster/clusterBasic.py -N 5
,,y,army,./pytest.sh python3 ./test.py -f query/query_basic.py -N 3
,,y,army,./pytest.sh python3 ./test.py -f query/accuracy/test_query_accuracy.py
,,y,army,./pytest.sh python3 ./test.py -f query/accuracy/test_ts5400.py
,,y,army,./pytest.sh python3 ./test.py -f insert/insert_basic.py -N 3
,,y,army,./pytest.sh python3 ./test.py -f cluster/splitVgroupByLearner.py -N 3
,,y,army,./pytest.sh python3 ./test.py -f authorith/authBasic.py -N 3
@ -42,6 +43,7 @@
,,y,army,./pytest.sh python3 ./test.py -f query/fill/fill_compare_asc_desc.py
,,y,army,./pytest.sh python3 ./test.py -f query/last/test_last.py
,,y,army,./pytest.sh python3 ./test.py -f query/window/base.py
,,y,army,./pytest.sh python3 ./test.py -f query/sys/tb_perf_queries_exist_test.py -N 3
#
# system test

14
tests/system-test/0-others/show.py
View File

@ -284,6 +284,20 @@ class TDTestCase:
for error_val in error_vals:
tdSql.error(f'ALTER DNODE 1 "{var}" "{error_val}"')
var = 'randErrorDivisor'
vals = ['9223372036854775807', '9223372036854775807.1', '9223372036854775806', '9223372036854775808', '9223372036854775808.1', '9223372036854775807.0', '9223372036854775806.1']
expected_vals = ['9223372036854775807', 'err', '9223372036854775806', 'err', 'err', 'err', 'err']
for val_str, expected_val in zip(vals, expected_vals):
sql = f'ALTER dnode 1 "{var}" "{val_str}"'
if expected_val == 'err':
tdSql.error(sql)
else:
tdSql.execute(sql, queryTimes=1)
actual_val = self.get_variable(var, False)
if expected_val != actual_val:
tdLog.exit(f"failed to set local {var} to {expected_val} actually {actual_val}")
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)